[ { "path": ".cirrus.yml", "content": "only_if: $CIRRUS_TAG == '' && ($CIRRUS_PR != '' || $CIRRUS_BRANCH == 'master' || $CIRRUS_BRANCH =~ 'tokio-.*')\nauto_cancellation: $CIRRUS_BRANCH != 'master' && $CIRRUS_BRANCH !=~ 'tokio-.*'\nfreebsd_instance:\n image_family: freebsd-14-3\nenv:\n RUST_STABLE: stable\n RUST_NIGHTLY: nightly-2025-10-12\n RUSTFLAGS: -D warnings\n # This excludes unstable features like io_uring, which require '--cfg tokio_unstable'.\n TOKIO_STABLE_FEATURES: full,test-util\n\n# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the\n# same VM. The binary will be built in 32-bit mode, but will execute on a\n# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of\n# the system's binaries, so the environment shouldn't matter.\ntask:\n name: FreeBSD 64-bit\n setup_script:\n - pkg install -y bash ca_root_nss\n - curl https://sh.rustup.rs -sSf --output rustup.sh\n - sh rustup.sh -y --profile minimal --default-toolchain $RUST_STABLE\n - . $HOME/.cargo/env\n - |\n echo \"~~~~ rustc --version ~~~~\"\n rustc --version\n test_script:\n - . $HOME/.cargo/env\n - cargo test --workspace --features $TOKIO_STABLE_FEATURES\n # Free the disk space before the next build,\n # otherwise cirrus-ci complains about \"No space left on device\".\n - cargo clean\n # Enable all unstable features except `taskdump`, which is Linux-only.\n - |\n RUSTFLAGS=\"$RUSTFLAGS --cfg tokio_unstable\" \\\n RUSTDOCFLAGS=\"$RUSTDOCFLAGS --cfg tokio_unstable\" \\\n cargo test \\\n --features $TOKIO_STABLE_FEATURES,io-uring,tracing\n\ntask:\n name: FreeBSD docs\n env:\n RUSTFLAGS: --cfg docsrs --cfg tokio_unstable\n RUSTDOCFLAGS: --cfg docsrs --cfg tokio_unstable -Dwarnings\n setup_script:\n - pkg install -y bash ca_root_nss\n - curl https://sh.rustup.rs -sSf --output rustup.sh\n - sh rustup.sh -y --profile minimal --default-toolchain $RUST_NIGHTLY\n - . $HOME/.cargo/env\n - |\n echo \"~~~~ rustc --version ~~~~\"\n rustc --version\n test_script:\n - . $HOME/.cargo/env\n # We use `--features $TOKIO_STABLE_FEATURES,io-uring,tracing` instead of\n # `--all-features` to exclude `taskdump`, which is Linux-only.\n - cargo doc --lib --no-deps --features $TOKIO_STABLE_FEATURES,io-uring,tracing --document-private-items\n\ntask:\n name: FreeBSD 32-bit\n setup_script:\n - pkg install -y bash ca_root_nss\n - curl https://sh.rustup.rs -sSf --output rustup.sh\n - sh rustup.sh -y --profile minimal --default-toolchain $RUST_STABLE\n - . $HOME/.cargo/env\n - rustup target add i686-unknown-freebsd\n - |\n echo \"~~~~ rustc --version ~~~~\"\n rustc --version\n test_script:\n - . $HOME/.cargo/env\n - cargo test --workspace --features $TOKIO_STABLE_FEATURES --target i686-unknown-freebsd\n" }, { "path": ".github/FUNDING.yml", "content": "# These are supported funding model platforms\n\ngithub: [tokio-rs]\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: A-tokio, C-bug\nassignees: ''\n\n---\n\n**Version**\nList the versions of all `tokio` crates you are using. The easiest way to get\nthis information is using `cargo tree` subcommand:\n\n`cargo tree | grep tokio`\n\n**Platform**\nThe output of `uname -a` (UNIX), or version and 32 or 64-bit (Windows)\n\n**Description**\nEnter your issue details here.\nOne way to structure the description:\n\n[short summary of the bug]\n\nI tried this code:\n\n[code sample that causes the bug]\n\nI expected to see this happen: [explanation]\n\nInstead, this happened: [explanation]\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "contact_links:\n - name: Question\n url: https://github.com/tokio-rs/tokio/discussions\n about: Questions about Tokio should be posted as a GitHub discussion.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: A-tokio, C-feature-request\nassignees: ''\n\n---\n\n**Is your feature request related to a problem? Please describe.**\nA clear and concise description of what the problem is. Ex. I'm always frustrated when [...]\n\n**Describe the solution you'd like**\nA clear and concise description of what you want to happen.\n\n**Describe alternatives you've considered**\nA clear and concise description of any alternative solutions or features you've considered.\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "\n\n## Motivation\n\n\n\n## Solution\n\n\n" }, { "path": ".github/buildomat/README.md", "content": "# Buildomat illumos CI\n\nThis directory contains CI configurations for the [illumos] operating system.\nTokio's illumos CI jobs are run using [Buildomat], a CI system developed by\nOxide Computer, which supports illumos. See [the Buildomat README] for more\ndetails.\n\n## illumos-Specific CI Failures\n\nIf your pull request's CI build fails on illumos, and you aren't able to easily\nreproduce the failure on other operating systems, don't worry! The\n[tokio-rs/illumos] team is responsible for maintaining Tokio's illumos support,\nand can be called on to assist contributors with illumos-specific issues. Please\nfeel free to tag @tokio-rs/illumos to ask for help resolving build failures on\nillumos.\n\n[illumos]: https://www.illumos.org/\n[Buildomat]: https://github.com/oxidecomputer/buildomat\n[the Buildomat README]: https://github.com/oxidecomputer/buildomat\n[tokio-rs/illumos]: https://github.com/orgs/tokio-rs/teams/illumos\n" }, { "path": ".github/buildomat/config.toml", "content": "# Repository-level Buildomat configuration.\n# See: https://github.com/oxidecomputer/buildomat#per-repository-configuration\n\n# Enable buildomat. This one should be self-explanatory.\nenable = true\n# Allow CI runs for PRs from users outside the `tokio-rs` organization. Our\n# buildomat jobs don't touch any secrets/keys, so this should be fine.\norg_only = false\n" }, { "path": ".github/labeler.yml", "content": "R-loom-blocking:\n- tokio/src/runtime/blocking/*\n- tokio/src/runtime/blocking/**/*\n\nR-loom-sync:\n- tokio/src/sync/*\n- tokio/src/sync/**/*\n\nR-loom-time-driver:\n- tokio/src/runtime/time/*\n- tokio/src/runtime/time/**/*\n\nR-loom-current-thread:\n- tokio/src/runtime/scheduler/*\n- tokio/src/runtime/scheduler/current_thread/*\n- tokio/src/runtime/task/*\n- tokio/src/runtime/task/**\n\nR-loom-multi-thread:\n- tokio/src/runtime/scheduler/*\n- tokio/src/runtime/scheduler/multi_thread/*\n- tokio/src/runtime/scheduler/multi_thread/**\n- tokio/src/runtime/task/*\n- tokio/src/runtime/task/**\n\nR-loom-util:\n- tokio-util/src/*\n- tokio-util/src/**/*\n" }, { "path": ".github/workflows/audit.yml", "content": "name: Security Audit\n\non:\n push:\n branches:\n - master\n paths:\n - '**/Cargo.toml'\n schedule:\n - cron: '0 2 * * *' # run at 2 AM UTC\n\npermissions:\n contents: read\n\njobs:\n cargo-deny:\n permissions:\n checks: write\n contents: read\n issues: write\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - uses: EmbarkStudios/cargo-deny-action@v2\n" }, { "path": ".github/workflows/ci.yml", "content": "on:\n push:\n branches: [\"master\", \"tokio-*.x\"]\n pull_request:\n branches: [\"master\", \"tokio-*.x\"]\n\nname: CI\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}\n cancel-in-progress: true\n\nenv:\n RUSTFLAGS: -Dwarnings\n RUST_BACKTRACE: 1\n RUSTUP_WINDOWS_PATH_ADD_BIN: 1\n # Change to specific Rust release to pin\n rust_stable: stable\n rust_nightly: nightly-2025-10-12\n # Pin a specific miri version\n rust_miri_nightly: nightly-2025-11-13\n rust_clippy: '1.88'\n # When updating this, also update:\n # - README.md\n # - tokio/README.md\n # - CONTRIBUTING.md\n # - tokio/Cargo.toml\n # - tokio-util/Cargo.toml\n # - tokio-test/Cargo.toml\n # - tokio-stream/Cargo.toml\n rust_min: '1.71'\n # This excludes unstable features like io_uring,\n # which require '--cfg tokio_unstable'.\n TOKIO_STABLE_FEATURES: \"full,test-util\"\n\ndefaults:\n run:\n shell: bash\n\npermissions:\n contents: read\n\njobs:\n # Basic actions that must pass before we kick off more expensive tests.\n basics:\n name: basic checks\n runs-on: ubuntu-latest\n needs:\n - clippy\n - fmt\n - docs\n - minrust\n steps:\n - run: exit 0\n\n test-tokio-full:\n needs: basics\n name: test tokio full\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: Swatinem/rust-cache@v2\n\n # Run `tokio` with stable features. This excludes testing utilities which\n # can alter the runtime behavior of Tokio.\n - name: test tokio full\n run: |\n set -euxo pipefail\n cargo nextest run --features full\n cargo test --doc --features full\n working-directory: tokio\n\n test-workspace-all-features:\n needs: basics\n name: test all crates in the workspace with all features\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: Swatinem/rust-cache@v2\n\n - name: test --features ${{ env.TOKIO_STABLE_FEATURES }}\n run: |\n set -euxo pipefail\n cargo nextest run --workspace --features $TOKIO_STABLE_FEATURES\n # Removing workspace patches to run tests without path dependencies\n # (if not specified differently in the crate)\n perl -0 -i -pe 's/\\[patch\\.crates-io\\].+\\n\\[/[/s' Cargo.toml\n cargo nextest run \\\n --workspace \\\n --exclude tokio \\\n --exclude examples \\\n --features $TOKIO_STABLE_FEATURES\n # Cargo nextest does not support doctest, so we run them separately\n # (see https://github.com/nextest-rs/nextest/issues/16)\n cargo test --doc --workspace --features $TOKIO_STABLE_FEATURES\n\n test-workspace-all-features-panic-abort:\n needs: basics\n name: test all crates in the workspace with all features and panic=abort\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: Swatinem/rust-cache@v2\n\n - name: test --features ${{ env.TOKIO_STABLE_FEATURES }} panic=abort\n run: |\n set -euxo pipefail\n RUSTFLAGS=\"$RUSTFLAGS -C panic=abort -Zpanic-abort-tests\" cargo nextest run \\\n --workspace \\\n --exclude tokio-macros \\\n --exclude tests-build \\\n --features $TOKIO_STABLE_FEATURES \\\n --tests\n\n test-integration-tests-per-feature:\n needs: basics\n name: Run integration tests for each feature\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install cargo-hack\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-hack\n\n - uses: Swatinem/rust-cache@v2\n\n # Run integration tests for each feature\n - name: test tests-integration --each-feature\n run: cargo hack test --each-feature\n working-directory: tests-integration\n\n # Run macro build tests\n - name: test tests-build --each-feature\n run: cargo hack test --each-feature\n working-directory: tests-build\n\n # Check benchmarks.\n - name: Check benches\n run: cargo check --benches\n working-directory: benches\n if: startsWith(matrix.os, 'ubuntu')\n\n test-parking_lot:\n # The parking_lot crate has a feature called send_guard which changes when\n # some of its types are Send. Tokio has some measures in place to prevent\n # this from affecting when Tokio types are Send, and this test exists to\n # ensure that those measures are working.\n #\n # This relies on the potentially affected Tokio type being listed in\n # `tokio/tokio/tests/async_send_sync.rs`.\n name: compile tests with parking lot send guards\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n\n - name: Enable parking_lot send_guard feature\n # Inserts the line \"plsend = [\"parking_lot/send_guard\"]\" right after [features]\n run: sed -i '/\\[features\\]/a plsend = [\"parking_lot/send_guard\"]' tokio/Cargo.toml\n\n - uses: Swatinem/rust-cache@v2\n\n - name: Check tests --features ${{ env.TOKIO_STABLE_FEATURES }}\n run: |\n set -euxo pipefail\n cargo check --workspace --tests --features $TOKIO_STABLE_FEATURES\n # Removing the tokio workspace patch to run tests without path dependencies\n # (if not specified differently in the crate)\n perl -0 -i -pe 's/\\[patch\\.crates-io\\].+\\n\\[/[/s' Cargo.toml\n cargo check --workspace --exclude tokio --tests --features $TOKIO_STABLE_FEATURES\n\n valgrind:\n name: valgrind\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: 1.82\n\n - name: Install Valgrind\n uses: taiki-e/install-action@valgrind\n\n - uses: Swatinem/rust-cache@v2\n # Compile tests\n - name: cargo build test-mem\n run: cargo build --features rt-net --bin test-mem\n working-directory: tests-integration\n\n # Run with valgrind\n - name: Run valgrind test-mem\n run: valgrind --error-exitcode=1 --leak-check=full --show-leak-kinds=all --fair-sched=yes ./target/debug/test-mem\n\n # Compile tests\n - name: cargo build test-process-signal\n run: cargo build --features rt-process-signal --bin test-process-signal\n working-directory: tests-integration\n\n # Run with valgrind\n - name: Run valgrind test-process-signal\n run: valgrind --error-exitcode=1 --leak-check=full --show-leak-kinds=all --fair-sched=yes ./target/debug/test-process-signal\n\n test-unstable:\n name: test tokio full --unstable\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - { os: windows-latest, extra_features: \"\" }\n - { os: ubuntu-latest, extra_features: \"\" }\n # only Linux supports io_uring\n - { os: ubuntu-latest, extra_features: io-uring }\n - { os: macos-latest, extra_features: \"\" }\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: Swatinem/rust-cache@v2\n # Run `tokio` with \"unstable\" cfg flag.\n - name: test tokio full --cfg unstable\n run: |\n set -euxo pipefail\n cargo nextest run --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }}\n cargo test --doc --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }}\n working-directory: tokio\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n # in order to run doctests for unstable features, we must also pass\n # the unstable cfg to RustDoc\n RUSTDOCFLAGS: --cfg tokio_unstable\n\n test-unstable-taskdump:\n name: test tokio full --unstable --taskdump\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - os: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: Swatinem/rust-cache@v2\n # Run `tokio` with \"unstable\" and \"taskdump\" cfg flags.\n - name: test tokio full --cfg unstable --cfg taskdump\n run: |\n set -euxo pipefail\n cargo nextest run --features $TOKIO_STABLE_FEATURES,taskdump\n cargo test --doc --features $TOKIO_STABLE_FEATURES\n working-directory: tokio\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n # in order to run doctests for unstable features, we must also pass\n # the unstable cfg to RustDoc\n RUSTDOCFLAGS: --cfg tokio_unstable\n\n check-unstable-mt-counters:\n name: check tokio full --internal-mt-counters\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - os: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n - uses: Swatinem/rust-cache@v2\n # Since the internal-mt-counters feature is only for debugging purposes,\n # we can enable all features including unstable.\n - name: check tokio full --cfg unstable --cfg internal-mt-counters\n run: |\n set -euxo pipefail\n cargo nextest run --all-features\n cargo test --doc --all-features\n working-directory: tokio\n env:\n RUSTFLAGS: --cfg tokio_unstable --cfg tokio_internal_mt_counters -Dwarnings\n # in order to run doctests for unstable features, we must also pass\n # the unstable cfg to RustDoc\n RUSTDOCFLAGS: --cfg tokio_unstable --cfg tokio_internal_mt_counters\n\n miri-lib:\n name: miri-lib\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_miri_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_miri_nightly }}\n components: miri\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n - uses: Swatinem/rust-cache@v2\n - name: miri\n run: |\n cargo miri nextest run --features full --lib --no-fail-fast\n working-directory: tokio\n env:\n MIRIFLAGS: -Zmiri-disable-isolation -Zmiri-strict-provenance\n\n miri-test:\n name: miri-test\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_miri_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_miri_nightly }}\n components: miri\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n - uses: Swatinem/rust-cache@v2\n - name: miri\n run: |\n cargo miri nextest run --features full --test '*' --no-fail-fast\n working-directory: tokio\n env:\n MIRIFLAGS: -Zmiri-disable-isolation -Zmiri-strict-provenance\n\n miri-doc:\n name: miri-doc\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_miri_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_miri_nightly }}\n components: miri\n - uses: Swatinem/rust-cache@v2\n - name: miri-doc-test\n run: |\n cargo miri test --doc --features $TOKIO_STABLE_FEATURES --no-fail-fast\n working-directory: tokio\n env:\n MIRIFLAGS: -Zmiri-disable-isolation -Zmiri-strict-provenance\n\n asan:\n name: asan\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install llvm\n # Required to resolve symbols in sanitizer output\n run: sudo apt-get install -y llvm\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n\n - uses: Swatinem/rust-cache@v2\n - name: asan\n run: cargo test --workspace --features $TOKIO_STABLE_FEATURES --target x86_64-unknown-linux-gnu --tests -- --test-threads 1 --nocapture\n env:\n RUSTFLAGS: -Z sanitizer=address --cfg tokio_no_tuning_tests\n # Ignore `trybuild` errors as they are irrelevant and flaky on nightly\n TRYBUILD: overwrite\n\n semver:\n name: semver\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Check `tokio` semver\n uses: obi1kenobi/cargo-semver-checks-action@v2\n with:\n rust-toolchain: ${{ env.rust_stable }}\n package: tokio\n release-type: minor\n feature-group: only-explicit-features\n # We don't care about the semver of unstable tokio features.\n features: ${{ env.TOKIO_STABLE_FEATURES }}\n - name: Check semver for rest of the workspace\n if: ${{ !startsWith(github.event.pull_request.base.ref, 'tokio-1.') }}\n uses: obi1kenobi/cargo-semver-checks-action@v2\n with:\n rust-toolchain: ${{ env.rust_stable }}\n exclude: tokio\n release-type: minor\n\n cross-check:\n name: cross-check\n needs: basics\n runs-on: ubuntu-latest\n strategy:\n matrix:\n target:\n - powerpc-unknown-linux-gnu\n - powerpc64-unknown-linux-gnu\n - arm-linux-androideabi\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n target: ${{ matrix.target }}\n\n - uses: Swatinem/rust-cache@v2\n # We don't use --all-features since io-uring will be enabled and is not supported on those targets.\n - run: cargo check --workspace --features $TOKIO_STABLE_FEATURES --target ${{ matrix.target }}\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n\n cross-check-tier3:\n name: cross-check-tier3\n needs: basics\n runs-on: ubuntu-latest\n strategy:\n matrix:\n target:\n - name: x86_64-unknown-haiku\n exclude_features: \"taskdump\" # taskdump is only available on Linux\n# - name: armv7-sony-vita-newlibeabihf\n# exclude_features: \"process,signal,rt-process-signal,full,taskdump\"\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@nightly\n with:\n toolchain: ${{ env.rust_nightly }}\n components: rust-src\n - name: Install cargo-hack\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-hack\n - uses: Swatinem/rust-cache@v2\n - run: cargo hack check -Zbuild-std --workspace --each-feature --target ${{ matrix.target.name }} --exclude-features \"${{ matrix.target.exclude_features }}\"\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n\n cross-test-with-parking_lot:\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - target: i686-unknown-linux-gnu\n os: ubuntu-latest\n extra_features: \"taskdump\"\n - target: armv5te-unknown-linux-gnueabi\n os: ubuntu-latest\n - target: armv7-unknown-linux-gnueabihf\n os: ubuntu-24.04-arm\n - target: aarch64-unknown-linux-gnu\n os: ubuntu-24.04-arm\n extra_features: \"io-uring,taskdump\"\n - target: aarch64-pc-windows-msvc\n os: windows-11-arm\n steps:\n - uses: actions/checkout@v4\n - name: Install Rust stable\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n target: ${{ matrix.target }}\n\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: taiki-e/setup-cross-toolchain-action@v1\n with:\n target: ${{ matrix.target }}\n qemu: '7.2'\n\n - uses: Swatinem/rust-cache@v2\n - name: Tests run with all features (including parking_lot)\n run: |\n set -euxo pipefail\n # We use `--features \"$TOKIO_STABLE_FEATURES\"` instead of `--all-features` since\n # `--all-features` includes `io_uring` and `taskdump`,\n # which is not available on all targets.\n cargo nextest run -p tokio --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }} --target ${{ matrix.target }}\n cargo test --doc -p tokio --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }} --target ${{ matrix.target }}\n env:\n RUST_TEST_THREADS: 1\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings --cfg tokio_no_tuning_tests\n RUSTDOCFLAGS: --cfg tokio_unstable -Dwarnings\n\n cross-test-without-parking_lot:\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - target: i686-unknown-linux-gnu\n os: ubuntu-latest\n extra_features: \"taskdump\"\n - target: armv5te-unknown-linux-gnueabi\n os: ubuntu-latest\n - target: armv7-unknown-linux-gnueabihf\n os: ubuntu-24.04-arm\n - target: aarch64-unknown-linux-gnu\n os: ubuntu-24.04-arm\n extra_features: \"io-uring,taskdump\"\n - target: aarch64-pc-windows-msvc\n os: windows-11-arm\n steps:\n - uses: actions/checkout@v4\n - name: Install Rust stable\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n target: ${{ matrix.target }}\n\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: taiki-e/setup-cross-toolchain-action@v1\n with:\n target: ${{ matrix.target }}\n qemu: '7.2'\n\n - name: Remove `parking_lot` from `full` feature\n run: sed -i '0,/parking_lot/{/parking_lot/d;}' tokio/Cargo.toml\n\n - uses: Swatinem/rust-cache@v2\n # The `tokio_no_parking_lot` cfg is here to ensure the `sed` above does not silently break.\n - name: Tests run with all features (without parking_lot)\n run: |\n set -euxo pipefail\n # We use `--features \"$TOKIO_STABLE_FEATURES\"` instead of `--all-features` since\n # `--all-features` includes `io_uring` and `taskdump`,\n # which is not available on all targets.\n cargo nextest run -p tokio --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }} --target ${{ matrix.target }}\n cargo test --doc -p tokio --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }} --target ${{ matrix.target }}\n env:\n RUST_TEST_THREADS: 1\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings --cfg tokio_no_parking_lot --cfg tokio_no_tuning_tests ${{ matrix.rustflags }}\n RUSTDOCFLAGS: --cfg tokio_unstable -Dwarnings\n\n # See https://github.com/tokio-rs/tokio/issues/5187\n no-atomic-u64-test:\n name: Test tokio --all-features on i686-unknown-linux-gnu without AtomicU64\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n components: rust-src\n\n - name: Install cargo-nextest\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-nextest\n\n - uses: taiki-e/setup-cross-toolchain-action@v1\n with:\n target: i686-unknown-linux-gnu\n qemu: '7.2'\n\n - uses: Swatinem/rust-cache@v2\n - name: test tokio --all-features\n run: |\n cargo nextest run -Zbuild-std --target target-specs/i686-unknown-linux-gnu.json -p tokio --features $TOKIO_STABLE_FEATURES,taskdump\n cargo test --doc -Zbuild-std --target target-specs/i686-unknown-linux-gnu.json -p tokio --features $TOKIO_STABLE_FEATURES,taskdump\n env:\n RUST_TEST_THREADS: 1\n RUSTDOCFLAGS: --cfg tokio_unstable\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings --cfg tokio_no_tuning_tests\n\n no-atomic-u64-check:\n name: Check tokio --feature-powerset --depth 2 on i686-unknown-linux-gnu without AtomicU64\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n components: rust-src\n - name: Install cargo-hack\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-hack\n\n - uses: Swatinem/rust-cache@v2\n\n # https://github.com/tokio-rs/tokio/pull/5356\n # https://github.com/tokio-rs/tokio/issues/5373\n - name: Check\n # We use `--skip io-uring` since io-uring crate doesn't provide a binding for the i686 target.\n run: cargo hack check -Zbuild-std --target target-specs/i686-unknown-linux-gnu.json -p tokio --feature-powerset --skip io-uring --depth 2 --keep-going\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n\n features:\n name: features exclude ${{ matrix.name }}\n needs: basics\n runs-on: ubuntu-latest\n strategy:\n matrix:\n include:\n - name: \"\"\n rustflags: \"\"\n exclude_features: \"io-uring,taskdump\"\n - name: \"--unstable\"\n rustflags: \"--cfg tokio_unstable -Dwarnings\"\n exclude_features: \"io-uring,taskdump\"\n - name: \"--unstable io-uring,taskdump\"\n rustflags: \"--cfg tokio_unstable -Dwarnings\"\n exclude_features: \"\"\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n target: ${{ matrix.target }}\n - name: Install cargo-hack\n uses: taiki-e/install-action@cargo-hack\n\n - uses: Swatinem/rust-cache@v2\n - name: check --feature-powerset ${{ matrix.name }}\n run: cargo hack check --all --feature-powerset --exclude-features \"${{ matrix.exclude_features }}\" --depth 2 --keep-going\n env:\n RUSTFLAGS: ${{ matrix.rustflags }}\n\n minrust:\n name: minrust\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_min }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_min }}\n - name: Install cargo-hack\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-hack\n - uses: Swatinem/rust-cache@v2\n - name: \"cargo check\"\n run: |\n if [[ \"${{ github.event.pull_request.base.ref }}\" =~ ^tokio-1\\..* ]]; then\n # Only check `tokio` crate as the PR is backporting to an earlier tokio release.\n\n cargo check -p tokio --features $TOKIO_STABLE_FEATURES\n else\n # Check all crates in the workspace\n\n cargo check -p tokio --features $TOKIO_STABLE_FEATURES\n # Other crates doesn't have unstable features, so we can use --all-features.\n cargo hack check -p tokio-macros -p tokio-stream -p tokio-util -p tokio-test --all-features\n fi\n\n minimal-versions:\n name: minimal-versions\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n - name: Install cargo-hack\n uses: taiki-e/install-action@cargo-hack\n\n - uses: Swatinem/rust-cache@v2\n - name: \"check -Z minimal-versions\"\n run: |\n # Remove dev-dependencies from Cargo.toml to prevent the next `cargo update`\n # from determining minimal versions based on dev-dependencies.\n cargo hack --remove-dev-deps --workspace\n # Update Cargo.lock to minimal version dependencies.\n cargo update -Z minimal-versions\n cargo hack check -p tokio --features $TOKIO_STABLE_FEATURES --ignore-private\n cargo hack check -p tokio-macros -p tokio-stream -p tokio-util -p tokio-test --all-features --ignore-private\n - name: \"check --all-features --unstable -Z minimal-versions\"\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n run: |\n # Remove dev-dependencies from Cargo.toml to prevent the next `cargo update`\n # from determining minimal versions based on dev-dependencies.\n cargo hack --remove-dev-deps --workspace\n # Update Cargo.lock to minimal version dependencies.\n cargo update -Z minimal-versions\n cargo hack check --all-features --ignore-private\n\n fmt:\n name: fmt\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n components: rustfmt\n - uses: Swatinem/rust-cache@v2\n # Check fmt\n - name: \"rustfmt --check\"\n # Workaround for rust-lang/cargo#7732\n run: |\n if ! rustfmt --check --edition 2021 $(git ls-files '*.rs'); then\n printf \"Please run \\`rustfmt --edition 2021 \\$(git ls-files '*.rs')\\` to fix rustfmt errors.\\nSee CONTRIBUTING.md for more details.\\n\" >&2\n exit 1\n fi\n\n clippy:\n name: clippy\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_clippy }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_clippy }}\n components: clippy\n - uses: Swatinem/rust-cache@v2\n # Run clippy\n - name: \"clippy --workspace --features ${{ env.TOKIO_STABLE_FEATURES }}\"\n run: |\n cargo clippy --workspace --tests --no-deps --features $TOKIO_STABLE_FEATURES\n # Removing the tokio workspace patch to check without path dependencies\n # (if not specified differently in the crate)\n perl -0 -i.bak -pe 's/\\[patch\\.crates-io\\].+\\n\\[/[/s' Cargo.toml\n cargo clippy --workspace --exclude tokio --tests --no-deps --features $TOKIO_STABLE_FEATURES\n - name: \"clippy --workspace --all-features --unstable\"\n run: |\n # Forcing the cargo lock regeneration to apply the tokio patch\n rm Cargo.lock\n mv Cargo.toml Cargo.toml.nopatch\n mv Cargo.toml.bak Cargo.toml\n cargo clippy --workspace --tests --no-deps --all-features\n # check without path dependencies\n mv Cargo.toml.nopatch Cargo.toml\n cargo clippy --workspace --exclude tokio --tests --no-deps --all-features\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n\n docs:\n name: docs\n runs-on: ${{ matrix.run.os }}\n strategy:\n matrix:\n run:\n - os: windows-latest\n extra_features: \"tracing\"\n - os: ubuntu-latest\n extra_features: \"tracing,io-uring,taskdump\"\n\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n - uses: Swatinem/rust-cache@v2\n - name: \"doc --lib --all-features\"\n run: cargo doc --lib --no-deps --document-private-items --features $TOKIO_STABLE_FEATURES,${{ matrix.run.extra_features }}\n env:\n RUSTFLAGS: --cfg docsrs --cfg tokio_unstable\n RUSTDOCFLAGS: --cfg docsrs --cfg tokio_unstable -Dwarnings\n\n loom-compile:\n name: build loom tests\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: build --cfg loom\n run: cargo test --no-run --lib --features full\n working-directory: tokio\n env:\n RUSTFLAGS: --cfg loom --cfg tokio_unstable -Dwarnings\n\n check-readme:\n name: Check README\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Verify that both READMEs are identical\n run: diff README.md tokio/README.md\n\n - name: Verify that Tokio version is up to date in README\n working-directory: tokio\n run: grep -q \"$(sed '/^version = /!d' Cargo.toml | head -n1)\" README.md\n\n test-hyper:\n name: Test hyper\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n\n - name: Clone hyper\n run: git clone https://github.com/hyperium/hyper.git\n\n - name: Checkout the latest release because HEAD maybe contains breakage.\n run: |\n set -x\n tag=$(git describe --abbrev=0 --tags)\n git checkout \"${tag}\"\n working-directory: hyper\n\n - name: Patch hyper to use tokio from this repository\n run: |\n set -x\n echo '[workspace]' >>Cargo.toml\n echo '[patch.crates-io]' >>Cargo.toml\n echo 'tokio = { path = \"../tokio\" }' >>Cargo.toml\n echo 'tokio-util = { path = \"../tokio-util\" }' >>Cargo.toml\n echo 'tokio-stream = { path = \"../tokio-stream\" }' >>Cargo.toml\n echo 'tokio-test = { path = \"../tokio-test\" }' >>Cargo.toml\n git diff\n working-directory: hyper\n\n - uses: Swatinem/rust-cache@v2\n with:\n # The cargo workspaces and target directory configuration.\n # These entries are separated by newlines and have the form\n # `$workspace -> $target`. The `$target` part is treated as a directory\n # relative to the `$workspace` and defaults to \"target\" if not explicitly given.\n # default: \". -> target\"\n workspaces: \"./hyper\"\n\n - name: Test hyper\n run: cargo test --features full\n working-directory: hyper\n\n test-quinn:\n name: Test Quinn\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n os:\n - windows-latest\n - ubuntu-latest\n - macos-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n\n - name: Clone Quinn\n run: git clone https://github.com/quinn-rs/quinn.git\n\n - name: Checkout the latest release because HEAD maybe contains breakage.\n run: |\n set -x\n tag=$(git describe --abbrev=0 --tags)\n git checkout \"${tag}\"\n working-directory: quinn\n\n - name: Patch Quinn to use tokio from this repository\n run: |\n set -x\n echo '[patch.crates-io]' >>Cargo.toml\n echo 'tokio = { path = \"../tokio\" }' >>Cargo.toml\n git diff\n working-directory: quinn\n\n - uses: Swatinem/rust-cache@v2\n with:\n # The cargo workspaces and target directory configuration.\n # These entries are separated by newlines and have the form\n # `$workspace -> $target`. The `$target` part is treated as a directory\n # relative to the `$workspace` and defaults to \"target\" if not explicitly given.\n # default: \". -> target\"\n workspaces: \"./quinn\"\n\n - name: Test Quinn\n working-directory: quinn\n env:\n RUSTFLAGS: \"\"\n run: cargo test\n\n x86_64-fortanix-unknown-sgx:\n name: build tokio for x86_64-fortanix-unknown-sgx\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n target: x86_64-fortanix-unknown-sgx\n - uses: Swatinem/rust-cache@v2\n # NOTE: Currently the only test we can run is to build tokio with rt and sync features.\n - name: build tokio\n run: cargo build --target x86_64-fortanix-unknown-sgx --features rt,sync\n working-directory: tokio\n\n check-redox:\n name: build tokio for redox-os\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_nightly }}\n target: x86_64-unknown-redox\n - name: check tokio on redox\n run: cargo check --target x86_64-unknown-redox --features $TOKIO_STABLE_FEATURES\n working-directory: tokio\n\n wasm32-unknown-unknown:\n name: test tokio for wasm32-unknown-unknown (${{ matrix.name }})\n needs: basics\n runs-on: ubuntu-latest\n strategy:\n matrix:\n include:\n - name: macros sync\n features: \"macros sync\"\n - name: macros sync rt\n features: \"macros sync rt\"\n - name: macros sync time rt\n features: \"macros sync time rt\"\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install wasm-pack\n uses: taiki-e/install-action@wasm-pack\n\n - uses: Swatinem/rust-cache@v2\n - name: test tokio (${{ matrix.name }})\n run: wasm-pack test --node -- --features \"${{ matrix.features }}\"\n working-directory: tokio\n\n wasm32-wasip1:\n name: ${{ matrix.target }}\n needs: basics\n runs-on: ubuntu-latest\n strategy:\n matrix:\n target:\n - wasm32-wasip1\n - wasm32-wasip1-threads\n env:\n rust_stable: '1.93.1'\n steps:\n - uses: actions/checkout@v4\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n targets: ${{ matrix.target }}\n\n # Install dependencies\n - name: Install cargo-hack, wasmtime\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-hack,wasmtime\n\n - uses: Swatinem/rust-cache@v2\n - name: WASI test tokio full\n run: cargo test -p tokio --target ${{ matrix.target }} --features \"sync,macros,io-util,rt,time\"\n env:\n CARGO_TARGET_WASM32_WASIP1_RUNNER: \"wasmtime run --\"\n CARGO_TARGET_WASM32_WASIP1_THREADS_RUNNER: \"wasmtime run -W bulk-memory=y -W threads=y -W shared-memory=y -S threads=y --\"\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings -C target-feature=+atomics,+bulk-memory -C link-args=--max-memory=67108864\n # in order to run doctests for unstable features, we must also pass\n # the unstable cfg to RustDoc\n RUSTDOCFLAGS: --cfg tokio_unstable\n\n - name: WASI test tokio-util full\n run: cargo test -p tokio-util --target ${{ matrix.target }} --features full\n env:\n CARGO_TARGET_WASM32_WASIP1_RUNNER: \"wasmtime run --\"\n CARGO_TARGET_WASM32_WASIP1_THREADS_RUNNER: \"wasmtime run -W bulk-memory=y -W threads=y -W shared-memory=y -S threads=y --\"\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings -C target-feature=+atomics,+bulk-memory -C link-args=--max-memory=67108864\n RUSTDOCFLAGS: -C link-args=--max-memory=67108864\n\n - name: WASI test tokio-stream\n run: cargo test --manifest-path=tokio-stream/Cargo.toml --target ${{ matrix.target }} --features time,net,io-util,sync\n env:\n CARGO_TARGET_WASM32_WASIP1_RUNNER: \"wasmtime run --\"\n CARGO_TARGET_WASM32_WASIP1_THREADS_RUNNER: \"wasmtime run -W bulk-memory=y -W threads=y -W shared-memory=y -S threads=y --\"\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings -C target-feature=+atomics,+bulk-memory -C link-args=--max-memory=67108864\n\n - name: test tests-integration --features wasi-rt\n # TODO: this should become: `cargo hack wasi test --each-feature`\n run: cargo test --target ${{ matrix.target }} --test rt_yield --features wasi-rt\n if: matrix.target == 'wasm32-wasip1'\n working-directory: tests-integration\n env:\n CARGO_TARGET_WASM32_WASIP1_RUNNER: \"wasmtime run --\"\n RUSTFLAGS: -Dwarnings -C target-feature=+atomics,+bulk-memory -C link-args=--max-memory=67108864\n\n - name: test tests-integration --features wasi-threads-rt\n run: cargo test --target ${{ matrix.target }} --features wasi-threads-rt\n if: matrix.target == 'wasm32-wasip1-threads'\n working-directory: tests-integration\n env:\n CARGO_TARGET_WASM32_WASIP1_THREADS_RUNNER: \"wasmtime run -W bulk-memory=y -W threads=y -W shared-memory=y -S threads=y --\"\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings -C target-feature=+atomics,+bulk-memory -C link-args=--max-memory=67108864\n\n check-external-types:\n name: check-external-types (${{ matrix.os }})\n needs: basics\n runs-on: ${{ matrix.os }}\n strategy:\n matrix:\n include:\n - os: windows-latest\n # Windows neither supports io-uring nor taskdump.\n extra_features: \"tracing\"\n - os: ubuntu-latest\n # includes all unstable features.\n extra_features: \"tracing,io-uring,taskdump\"\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ matrix.rust }}\n uses: dtolnay/rust-toolchain@stable\n with:\n # `check-external-types` requires a specific Rust nightly version. See\n # the README for details: https://github.com/awslabs/cargo-check-external-types\n toolchain: nightly-2025-08-06\n - uses: Swatinem/rust-cache@v2\n - name: Install cargo-check-external-types\n uses: taiki-e/cache-cargo-install-action@v1\n with:\n tool: cargo-check-external-types@0.3.0\n - name: check-external-types\n env:\n RUSTFLAGS: --cfg tokio_unstable -Dwarnings\n RUSTDOCFLAGS: --cfg tokio_unstable\n run: cargo check-external-types --features $TOKIO_STABLE_FEATURES,${{ matrix.extra_features }}\n working-directory: tokio\n\n check-fuzzing:\n name: check-fuzzing\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - name: Install Rust ${{ env.rust_nightly }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_nightly }}\n - uses: Swatinem/rust-cache@v2\n - name: Install cargo-fuzz\n run: cargo install cargo-fuzz\n - name: Check /tokio/\n run: cargo fuzz check --all-features\n working-directory: tokio\n - name: Check /tokio-stream/\n run: cargo fuzz check --all-features\n working-directory: tokio-stream\n\n check-spelling:\n name: check-spelling\n needs: basics\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@stable\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install cargo-spellcheck\n uses: taiki-e/install-action@v2\n with:\n tool: cargo-spellcheck\n - uses: actions/checkout@v4\n - name: Make sure dictionary words are sorted and unique\n run: |\n FILE=\"spellcheck.dic\"\n\n # Verify the first line is an integer.\n first_line=$(head -n 1 \"$FILE\")\n if ! [[ \"$first_line\" =~ ^[0-9]+$ ]]; then\n echo \"Error: The first line of $FILE must be an integer, but got: '$first_line'\"\n exit 1\n fi\n expected_count=\"$first_line\"\n\n # Check that the number of lines matches the integer.\n # xargs (with no arguments) will strip leading/trailing whitespacefrom wc's output.\n actual_count=$(sed '1d' \"$FILE\" | wc -l | xargs)\n if [ \"$expected_count\" -ne \"$actual_count\" ]; then\n echo \"Error: The number of lines ($actual_count) does not match $expected_count.\"\n exit 1\n fi\n\n # `sed` removes the first line (number of words).\n #\n # `sort` makes sure everything in between is sorted\n # and contains no duplicates.\n #\n # Since `sort` is sensitive to locale, we set it\n # using LC_ALL to en_US.UTF8 to be consistent in different\n # environments.\n\n (\n sed '1d' $FILE | LC_ALL=en_US.UTF8 sort -uc\n ) || {\n echo \"Dictionary is not in sorted order. Correct order is:\"\n LC_ALL=en_US.UTF8 sort -u <(sed '1d' $FILE)\n false\n }\n - name: Run cargo-spellcheck\n run: |\n if ! cargo spellcheck --code 1\n then\n echo ''\n echo ''\n echo 'If this is a Rust method/type/variable name, then you should'\n echo 'enclose it in backticks like this: `MyRustType`.'\n echo ''\n echo 'If this is a real word, then you can add it to spellcheck.dic'\n exit 1\n fi\n - name: Detect trailing whitespace\n run: |\n if grep --exclude-dir=.git --exclude-dir=target -rne '\\s$' .\n then\n echo ''\n echo 'Please remove trailing whitespace from these lines.'\n exit 1\n fi\n\n get-latest-kernel-version:\n runs-on: ubuntu-latest\n outputs:\n kernel_version: ${{ steps.fetch.outputs.kernel_version }}\n steps:\n - name: Fetch latest stable kernel\n id: fetch\n run: |\n KERNEL_VERSION=$(curl -s https://www.kernel.org/releases.json | jq -r '.latest_stable.version')\n echo \"kernel_version=$KERNEL_VERSION\" >> $GITHUB_OUTPUT\n\n test-io-uring-on-specific-kernel-versions:\n name: Test io_uring on Linux ${{ matrix.kernel_version }}\n needs: [get-latest-kernel-version, basics]\n strategy:\n matrix:\n kernel_version:\n # A latest stable kernel version\n - ${{ needs.get-latest-kernel-version.outputs.kernel_version }}\n # A kernel version that doesn't support io_uring\n - '4.19.325'\n uses: ./.github/workflows/uring-kernel-version-test.yml\n with:\n kernel_version: ${{ matrix.kernel_version }}\n" }, { "path": ".github/workflows/labeler.yml", "content": "name: \"Pull Request Labeler\"\non:\n- pull_request_target\n\n# See .github/labeler.yml file\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}\n cancel-in-progress: true\n\npermissions:\n contents: read\n\njobs:\n triage:\n permissions:\n contents: read # for actions/labeler to determine modified files\n pull-requests: write # for actions/labeler to add labels to PRs\n runs-on: ubuntu-latest\n if: github.repository_owner == 'tokio-rs'\n steps:\n - uses: actions/labeler@v3\n with:\n repo-token: \"${{ secrets.GITHUB_TOKEN }}\"\n sync-labels: true\n" }, { "path": ".github/workflows/loom.yml", "content": "on:\n push:\n branches: [\"master\", \"tokio-*.x\"]\n pull_request:\n types: [labeled, opened, synchronize, reopened]\n branches: [\"master\", \"tokio-*.x\"]\n\nname: Loom\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}\n cancel-in-progress: true\n\nenv:\n RUSTFLAGS: -Dwarnings --cfg loom --cfg tokio_unstable -C debug_assertions\n LOOM_MAX_PREEMPTIONS: 2\n LOOM_MAX_BRANCHES: 10000\n RUST_BACKTRACE: 1\n # Change to specific Rust release to pin\n rust_stable: stable\n\npermissions:\n contents: read\n\njobs:\n loom-blocking:\n name: loom tokio::runtime::spawn_blocking\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-blocking') || (github.base_ref == null))\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: run tests\n run: cargo test --lib --release --features full -- --nocapture loom_blocking\n working-directory: tokio\n\n loom-sync:\n name: loom tokio::sync\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-sync') || (github.base_ref == null))\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: run tests\n run: cargo test --lib --release --features full -- --nocapture sync::tests\n working-directory: tokio\n\n loom-time-driver:\n name: loom time driver\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-time-driver') || (github.base_ref == null))\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: run tests\n run: cargo test --lib --release --features full -- --nocapture runtime::time\n working-directory: tokio\n\n loom-current-thread:\n name: loom current-thread scheduler\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-current-thread') || (github.base_ref == null))\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: run tests\n run: cargo test --lib --release --features full -- --nocapture loom_current_thread\n working-directory: tokio\n\n loom-multi-thread:\n name: loom multi-thread scheduler\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-multi-thread') || (github.base_ref == null))\n runs-on: ubuntu-latest\n strategy:\n matrix:\n include:\n - scope: loom_multi_thread::group_a\n - scope: loom_multi_thread::group_b\n - scope: loom_multi_thread::group_c\n - scope: loom_multi_thread::group_d\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: loom ${{ matrix.scope }}\n run: cargo test --lib --release --features full -- --nocapture $SCOPE\n working-directory: tokio\n env:\n SCOPE: ${{ matrix.scope }}\n\n loom-util:\n name: loom tokio-util\n # base_ref is null when it's not a pull request\n if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-util') || (github.base_ref == null))\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - uses: Swatinem/rust-cache@v2\n - name: run tests\n run: cargo test --lib --release --features full -- --nocapture\n working-directory: tokio-util\n" }, { "path": ".github/workflows/pr-audit.yml", "content": "name: Pull Request Security Audit\n\non:\n push:\n paths:\n - '**/Cargo.toml'\n pull_request:\n paths:\n - '**/Cargo.toml'\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}\n cancel-in-progress: true\n\npermissions:\n contents: read\n\njobs:\n cargo-deny:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - uses: EmbarkStudios/cargo-deny-action@v2\n" }, { "path": ".github/workflows/stress-test.yml", "content": "name: Stress Test\non:\n pull_request:\n push:\n branches:\n - master\n\nconcurrency:\n group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}\n cancel-in-progress: true\n\nenv:\n RUSTFLAGS: -Dwarnings\n RUST_BACKTRACE: 1\n # Change to specific Rust release to pin\n rust_stable: 1.82\n\npermissions:\n contents: read\n\njobs:\n stress-test:\n name: Stress Test\n runs-on: ubuntu-latest\n strategy:\n matrix:\n stress-test:\n - simple_echo_tcp\n steps:\n - uses: actions/checkout@v5\n - name: Install Rust ${{ env.rust_stable }}\n uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{ env.rust_stable }}\n - name: Install Valgrind\n uses: taiki-e/install-action@valgrind\n\n - uses: Swatinem/rust-cache@v2\n # Compiles each of the stress test examples.\n - name: Compile stress test examples\n run: cargo build -p stress-test --release --example ${{ matrix.stress-test }}\n\n # Runs each of the examples using Valgrind. Detects leaks and displays them.\n - name: Run valgrind\n run: valgrind --error-exitcode=1 --leak-check=full --show-leak-kinds=all --fair-sched=yes ./target/release/examples/${{ matrix.stress-test }}\n" }, { "path": ".github/workflows/uring-kernel-version-test.yml", "content": "name: Uring Kernel Version Test\n\non:\n workflow_call:\n inputs:\n kernel_version:\n description: 'Version of the Linux kernel to build'\n required: true\n type: string\n\njobs:\n build:\n runs-on: ubuntu-latest\n env:\n KERNEL_VERSION: ${{ inputs.kernel_version }}\n steps:\n - uses: actions/checkout@v5\n\n - name: Install system dependencies\n run: |\n sudo apt-get update\n sudo apt-get install -y \\\n build-essential bison flex libssl-dev libelf-dev \\\n qemu-system-x86 busybox-static cpio xz-utils wget\n\n - name: Cache Linux source\n id: cache-kernel\n uses: actions/cache@v4\n with:\n path: linux-${{ env.KERNEL_VERSION }}\n key: kernel-${{ env.KERNEL_VERSION }}\n\n - name: Download & build Linux kernel\n if: steps.cache-kernel.outputs.cache-hit != 'true'\n run: |\n MAJOR=${KERNEL_VERSION%%.*}\n wget https://cdn.kernel.org/pub/linux/kernel/v${MAJOR}.x/linux-${KERNEL_VERSION}.tar.xz\n tar xf linux-${KERNEL_VERSION}.tar.xz\n cd linux-${KERNEL_VERSION}\n make defconfig\n make -j$(nproc)\n\n - name: Generate test binaries with io_uring enabled\n run: |\n # Build both integration (tokio/tests/) and unit (e.g., tokio/src/fs/file/tests.rs) tests with io_uring enabled\n rustup target add x86_64-unknown-linux-musl\n RUSTFLAGS=\"--cfg tokio_unstable\" \\\n cargo test -p tokio --features full,io-uring \\\n --target x86_64-unknown-linux-musl --test 'fs*' --lib --no-run\n\n - name: Prepare initramfs + tests binaries\n run: |\n set -e\n rm -rf initramfs\n mkdir -p initramfs/{bin,bin/tests,sbin,proc,sys,tmp}\n\n # Copy test binaries into initramfs\n for bin in target/x86_64-unknown-linux-musl/debug/deps/{fs_*,tokio-*}; do\n if [ -f \"$bin\" ] && [ -x \"$bin\" ]; then\n cp \"$bin\" initramfs/bin/tests\n fi\n done\n\n # Add BusyBox & symlinks\n cp /usr/bin/busybox initramfs/bin/\n for cmd in sh mount uname true sleep; do ln -sf busybox initramfs/bin/$cmd; done\n ln -sf ../bin/busybox initramfs/sbin/poweroff\n\n # Generate init script\n cat > initramfs/init << 'EOF'\n #!/bin/sh\n set -e\n mkdir -p /dev\n # create device nodes, as some tests require them\n mknod /dev/null c 1 3\n mknod /dev/zero c 1 5\n mknod /dev/tty c 5 0\n mount -t proc proc /proc\n mount -t sysfs sysfs /sys\n mkdir -p /tmp && mount -t tmpfs -o mode=1777 tmpfs /tmp\n for f in /bin/tests/*; do RUST_BACKTRACE=1 \"$f\" ; done\n EOF\n chmod +x initramfs/init\n\n # Pack into a CPIO archive\n (cd initramfs && find . -print0 \\\n | cpio --null -ov --format=newc | gzip -9 > ../initramfs.cpio.gz)\n\n - name: Run tests in QEMU\n run: |\n qemu-system-x86_64 \\\n -kernel linux-${{ env.KERNEL_VERSION }}/arch/x86/boot/bzImage \\\n -initrd initramfs.cpio.gz \\\n -append \"console=ttyS0 rootfstype=ramfs panic=1\" \\\n -nographic -no-reboot -m 1024 -action panic=exit-failure 2>&1 | tee qemu-output.log\n\n # qemu always exits with 0, so we check if the tests passed by using grep.\n if grep -q \"test result: FAILED\" qemu-output.log; then\n echo \"tests failed (QEMU exited abnormally)\"\n exit 1\n else\n echo \"all tests passed\"\n fi\n" }, { "path": ".gitignore", "content": "target\nCargo.lock\n\n.cargo/config.toml\n.cargo/config\n\n" }, { "path": "CODE_OF_CONDUCT.md", "content": "# Code of Conduct\n\nThe Tokio project adheres to the [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct). This describes the minimum behavior expected from all contributors.\n\n## Enforcement\n\nInstances of violations of the Code of Conduct can be reported by contacting the project team at [moderation@tokio.rs](mailto:moderation@tokio.rs).\n" }, { "path": "CONTRIBUTING.md", "content": "# Contributing to Tokio\n\nThanks for your help improving Tokio! We are so happy to have you!\n\nThere are opportunities to contribute to Tokio at any level. It doesn't matter if\nyou are just getting started with Rust or are the most weathered expert, we can\nuse your help.\n\n**No contribution is too small and all contributions are valued.**\n\nSee the [contributing guidelines] to get started.\n\n[contributing guidelines]: docs/contributing/README.md\n\n## Code of Conduct\n\nThe Tokio project adheres to the [Rust Code of Conduct][coc]. This describes\nthe _minimum_ behavior expected from all contributors. Instances of violations of the\nCode of Conduct can be reported by contacting the project team at\n[moderation@tokio.rs](mailto:moderation@tokio.rs).\n\n[coc]: https://github.com/rust-lang/rust/blob/master/CODE_OF_CONDUCT.md\n\n## Need Help?\n\nReach out to us on the [Discord server] for any concern not covered in this guide.\n\n[Discord server]: https://discord.gg/tokio\n\n## LTS guarantees\n\nIn Tokio ≥1.0.0, each LTS release comes with the guarantee of at least one year of\nbackported fixes.\n\nThe goal of these guarantees is to provide stability to the ecosystem.\n\n## Minimum Supported Rust Version (MSRV)\n\n* All Tokio ≥1.0.0 releases will support at least a 6-month old Rust\n compiler release.\n* The MSRV will only be increased on 1.x releases.\n\n## Versioning Policy\n\nWith Tokio ≥1.0.0:\n\n* Patch (1.\\_.x) releases _should only_ contain bug fixes or documentation\n changes. Besides this, these releases should not substantially change\n runtime behavior.\n* Minor (1.x) releases may contain new functionality, MSRV increases (see\n above), minor dependency updates, deprecations, and larger internal\n implementation changes.\n\nThis is as defined by [Semantic Versioning 2.0](https://semver.org/).\n" }, { "path": "Cargo.toml", "content": "[workspace]\nresolver = \"2\"\nmembers = [\n \"tokio\",\n \"tokio-macros\",\n \"tokio-test\",\n \"tokio-stream\",\n \"tokio-util\",\n\n # Internal\n \"benches\",\n \"examples\",\n \"stress-test\",\n \"tests-build\",\n \"tests-integration\",\n]\n\n[patch.crates-io]\ntokio = { path = \"tokio\" }\ntokio-macros = { path = \"tokio-macros\" }\ntokio-stream = { path = \"tokio-stream\" }\ntokio-test = { path = \"tokio-test\" }\ntokio-util = { path = \"tokio-util\" }\n\n[workspace.metadata.spellcheck]\nconfig = \"spellcheck.toml\"\n\n[workspace.lints.rust]\nunexpected_cfgs = { level = \"warn\", check-cfg = [\n 'cfg(fuzzing)',\n 'cfg(loom)',\n 'cfg(mio_unsupported_force_poll_poll)',\n 'cfg(tokio_allow_from_blocking_fd)',\n 'cfg(tokio_internal_mt_counters)',\n 'cfg(tokio_no_parking_lot)',\n 'cfg(tokio_no_tuning_tests)',\n 'cfg(tokio_unstable)',\n 'cfg(target_os, values(\"cygwin\"))',\n] }\n" }, { "path": "Cross.toml", "content": "[build.env]\npassthrough = [\n \"RUSTFLAGS\",\n \"RUST_BACKTRACE\",\n]\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "*[TokioConf 2026 program and tickets are now available!](https://tokioconf.com)*\n\n---\n\n# Tokio\n\nA runtime for writing reliable, asynchronous, and slim applications with\nthe Rust programming language. It is:\n\n* **Fast**: Tokio's zero-cost abstractions give you bare-metal\n performance.\n\n* **Reliable**: Tokio leverages Rust's ownership, type system, and\n concurrency model to reduce bugs and ensure thread safety.\n\n* **Scalable**: Tokio has a minimal footprint, and handles backpressure\n and cancellation naturally.\n\n[![Crates.io][crates-badge]][crates-url]\n[![MIT licensed][mit-badge]][mit-url]\n[![Build Status][actions-badge]][actions-url]\n[![Discord chat][discord-badge]][discord-url]\n\n[crates-badge]: https://img.shields.io/crates/v/tokio.svg\n[crates-url]: https://crates.io/crates/tokio\n[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg\n[mit-url]: https://github.com/tokio-rs/tokio/blob/master/LICENSE\n[actions-badge]: https://github.com/tokio-rs/tokio/workflows/CI/badge.svg\n[actions-url]: https://github.com/tokio-rs/tokio/actions?query=workflow%3ACI+branch%3Amaster\n[discord-badge]: https://img.shields.io/discord/500028886025895936.svg?logo=discord&style=flat-square\n[discord-url]: https://discord.gg/tokio\n\n[Website](https://tokio.rs) |\n[Guides](https://tokio.rs/tokio/tutorial) |\n[API Docs](https://docs.rs/tokio/latest/tokio) |\n[Chat](https://discord.gg/tokio)\n\n## Overview\n\nTokio is an event-driven, non-blocking I/O platform for writing\nasynchronous applications with the Rust programming language. At a high\nlevel, it provides a few major components:\n\n* A multithreaded, work-stealing based task [scheduler].\n* A reactor backed by the operating system's event queue (epoll, kqueue,\n IOCP, etc.).\n* Asynchronous [TCP and UDP][net] sockets.\n\nThese components provide the runtime components necessary for building\nan asynchronous application.\n\n[net]: https://docs.rs/tokio/latest/tokio/net/index.html\n[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html\n\n## Example\n\nA basic TCP echo server with Tokio.\n\nMake sure you enable the full features of the tokio crate on Cargo.toml:\n\n```toml\n[dependencies]\ntokio = { version = \"1.50.0\", features = [\"full\"] }\n```\nThen, on your main.rs:\n\n```rust,no_run\nuse tokio::net::TcpListener;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n\n loop {\n let (mut socket, _) = listener.accept().await?;\n\n tokio::spawn(async move {\n let mut buf = [0; 1024];\n\n // In a loop, read data from the socket and write the data back.\n loop {\n let n = match socket.read(&mut buf).await {\n // socket closed\n Ok(0) => return,\n Ok(n) => n,\n Err(e) => {\n eprintln!(\"failed to read from socket; err = {:?}\", e);\n return;\n }\n };\n\n // Write the data back\n if let Err(e) = socket.write_all(&buf[0..n]).await {\n eprintln!(\"failed to write to socket; err = {:?}\", e);\n return;\n }\n }\n });\n }\n}\n```\n\nMore examples can be found [here][examples]. For a larger \"real world\" example, see the\n[mini-redis] repository.\n\n[examples]: https://github.com/tokio-rs/tokio/tree/master/examples\n[mini-redis]: https://github.com/tokio-rs/mini-redis/\n\nTo see a list of the available feature flags that can be enabled, check our\n[docs][feature-flag-docs].\n\n## Getting Help\n\nFirst, see if the answer to your question can be found in the [Guides] or the\n[API documentation]. If the answer is not there, there is an active community in\nthe [Tokio Discord server][chat]. We would be happy to try to answer your\nquestion. You can also ask your question on [the discussions page][discussions].\n\n[Guides]: https://tokio.rs/tokio/tutorial\n[API documentation]: https://docs.rs/tokio/latest/tokio\n[chat]: https://discord.gg/tokio\n[discussions]: https://github.com/tokio-rs/tokio/discussions\n[feature-flag-docs]: https://docs.rs/tokio/#feature-flags\n\n## Contributing\n\n:balloon: Thanks for your help improving the project! We are so happy to have\nyou! We have a [contributing guide][guide] to help you get involved in the Tokio\nproject.\n\n[guide]: https://github.com/tokio-rs/tokio/blob/master/docs/contributing/README.md\n\n## Related Projects\n\nIn addition to the crates in this repository, the Tokio project also maintains\nseveral other libraries, including:\n\n* [`axum`]: A web application framework that focuses on ergonomics and modularity.\n\n* [`hyper`]: A fast and correct HTTP/1.1 and HTTP/2 implementation for Rust.\n\n* [`tonic`]: A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility.\n\n* [`warp`]: A super-easy, composable, web server framework for warp speeds.\n\n* [`tower`]: A library of modular and reusable components for building robust networking clients and servers.\n\n* [`tracing`] (formerly `tokio-trace`): A framework for application-level tracing and async-aware diagnostics.\n\n* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers `tokio`.\n\n* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.\n\n* [`loom`]: A testing tool for concurrent Rust code.\n\n[`axum`]: https://github.com/tokio-rs/axum\n[`warp`]: https://github.com/seanmonstar/warp\n[`hyper`]: https://github.com/hyperium/hyper\n[`tonic`]: https://github.com/hyperium/tonic\n[`tower`]: https://github.com/tower-rs/tower\n[`loom`]: https://github.com/tokio-rs/loom\n[`tracing`]: https://github.com/tokio-rs/tracing\n[`mio`]: https://github.com/tokio-rs/mio\n[`bytes`]: https://github.com/tokio-rs/bytes\n\n## Changelog\n\nThe Tokio repository contains multiple crates. Each crate has its own changelog.\n\n * `tokio` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio/CHANGELOG.md)\n * `tokio-util` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-util/CHANGELOG.md)\n * `tokio-stream` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-stream/CHANGELOG.md)\n * `tokio-macros` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-macros/CHANGELOG.md)\n * `tokio-test` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-test/CHANGELOG.md)\n\n## Supported Rust Versions\n\n\n\nTokio will keep a rolling MSRV (minimum supported rust version) policy of **at\nleast** 6 months. When increasing the MSRV, the new Rust version must have been\nreleased at least six months ago. The current MSRV is 1.71.\n\nNote that the MSRV is not increased automatically, and only as part of a minor\nrelease. The MSRV history for past minor releases can be found below:\n\n * 1.48 to now - Rust 1.71\n * 1.39 to 1.47 - Rust 1.70\n * 1.30 to 1.38 - Rust 1.63\n * 1.27 to 1.29 - Rust 1.56\n * 1.17 to 1.26 - Rust 1.49\n * 1.15 to 1.16 - Rust 1.46\n * 1.0 to 1.14 - Rust 1.45\n\nNote that although we try to avoid the situation where a dependency transitively\nincreases the MSRV of Tokio, we do not guarantee that this does not happen.\nHowever, every minor release will have some set of versions of dependencies that\nworks with the MSRV of that minor release.\n\n## Release schedule\n\nTokio doesn't follow a fixed release schedule, but we typically make one minor\nrelease each month. We make patch releases for bugfixes as necessary.\n\n## Bug patching policy\n\nFor the purposes of making patch releases with bugfixes, we have designated\ncertain minor releases as LTS (long term support) releases. Whenever a bug\nwarrants a patch release with a fix for the bug, it will be backported and\nreleased as a new patch release for each LTS minor version. Our current LTS\nreleases are:\n\n * `1.43.x` - LTS release until March 2026. (MSRV 1.70)\n * `1.47.x` - LTS release until September 2026. (MSRV 1.70)\n\nEach LTS release will continue to receive backported fixes for at least a year.\nIf you wish to use a fixed minor release in your project, we recommend that you\nuse an LTS release.\n\nTo use a fixed minor version, you can specify the version with a tilde. For\nexample, to specify that you wish to use the newest `1.43.x` patch release, you\ncan use the following dependency specification:\n```text\ntokio = { version = \"~1.43\", features = [...] }\n```\n\n### Previous LTS releases\n\n * `1.8.x` - LTS release until February 2022.\n * `1.14.x` - LTS release until June 2022.\n * `1.18.x` - LTS release until June 2023.\n * `1.20.x` - LTS release until September 2023.\n * `1.25.x` - LTS release until March 2024.\n * `1.32.x` - LTS release until September 2024.\n * `1.36.x` - LTS release until March 2025.\n * `1.38.x` - LTS release until July 2025.\n\n## License\n\nThis project is licensed under the [MIT license].\n\n[MIT license]: https://github.com/tokio-rs/tokio/blob/master/LICENSE\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "SECURITY.md", "content": "## Report a security issue\n\nThe Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [security@tokio.rs](mailto:security@tokio.rs). Security issues should not be reported via the public GitHub Issue tracker.\n\n## Vulnerability coordination\n\nRemediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [GitHub Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.\n\nDownstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, GitHub username(s) and any other salient information to [security@tokio.rs](mailto:security@tokio.rs). Participation in security issue coordination processes is at the discretion of the Tokio team.\n\n## Security advisories\n\nThe project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project GitHub Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).\n" }, { "path": "benches/Cargo.toml", "content": "[package]\nname = \"benches\"\nversion = \"0.0.0\"\npublish = false\nedition = \"2021\"\nlicense = \"MIT\"\n\n[features]\ntest-util = [\"tokio/test-util\"]\n\n[dependencies]\ntokio = { version = \"1.5.0\", path = \"../tokio\", features = [\"full\"] }\ncriterion = \"0.5.1\"\nrand = \"0.9\"\nrand_chacha = \"0.9\"\n\n[dev-dependencies]\ntokio-util = { version = \"0.7.0\", path = \"../tokio-util\", features = [\"full\"] }\ntokio-stream = { version = \"0.1\", path = \"../tokio-stream\" }\n\n[target.'cfg(unix)'.dependencies]\nlibc = \"0.2.42\"\n\n[[bench]]\nname = \"spawn\"\npath = \"spawn.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_broadcast\"\npath = \"sync_broadcast.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_mpsc\"\npath = \"sync_mpsc.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_mpsc_oneshot\"\npath = \"sync_mpsc_oneshot.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_watch\"\npath = \"sync_watch.rs\"\nharness = false\n\n[[bench]]\nname = \"rt_current_thread\"\npath = \"rt_current_thread.rs\"\nharness = false\n\n[[bench]]\nname = \"rt_multi_threaded\"\npath = \"rt_multi_threaded.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_notify\"\npath = \"sync_notify.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_rwlock\"\npath = \"sync_rwlock.rs\"\nharness = false\n\n[[bench]]\nname = \"sync_semaphore\"\npath = \"sync_semaphore.rs\"\nharness = false\n\n[[bench]]\nname = \"signal\"\npath = \"signal.rs\"\nharness = false\n\n[[bench]]\nname = \"fs\"\npath = \"fs.rs\"\nharness = false\n\n[[bench]]\nname = \"copy\"\npath = \"copy.rs\"\nharness = false\n\n[[bench]]\nname = \"time_now\"\npath = \"time_now.rs\"\nharness = false\n\n[[bench]]\nname = \"time_timeout\"\npath = \"time_timeout.rs\"\nharness = false\n\n[[bench]]\nname = \"spawn_blocking\"\npath = \"spawn_blocking.rs\"\nharness = false\n\n[[bench]]\nname = \"remote_spawn\"\npath = \"remote_spawn.rs\"\nharness = false\n\n[lints]\nworkspace = true\n" }, { "path": "benches/copy.rs", "content": "use criterion::{criterion_group, criterion_main, Criterion};\n\nuse rand::{Rng, SeedableRng};\nuse rand_chacha::ChaCha20Rng;\n\nuse tokio::io::{copy, repeat, AsyncRead, AsyncReadExt, AsyncWrite};\nuse tokio::time::{interval, Interval, MissedTickBehavior};\n\nuse std::task::Poll;\nuse std::time::Duration;\n\nconst KILO: usize = 1024;\n\n// Tunable parameters if you want to change this benchmark. If reader and writer\n// are matched in kilobytes per second, then this only exposes buffering to the\n// benchmark.\nconst RNG_SEED: u64 = 0;\n// How much data to copy in a single benchmark run\nconst SOURCE_SIZE: u64 = 256 * KILO as u64;\n// Read side provides CHUNK_SIZE every READ_SERVICE_PERIOD. If it's not called\n// frequently, it'll burst to catch up (representing OS buffers draining)\nconst CHUNK_SIZE: usize = 2 * KILO;\nconst READ_SERVICE_PERIOD: Duration = Duration::from_millis(1);\n// Write side buffers up to WRITE_BUFFER, and flushes to disk every\n// WRITE_SERVICE_PERIOD.\nconst WRITE_BUFFER: usize = 40 * KILO;\nconst WRITE_SERVICE_PERIOD: Duration = Duration::from_millis(20);\n// How likely you are to have to wait for previously written data to be flushed\n// because another writer claimed the buffer space\nconst PROBABILITY_FLUSH_WAIT: f64 = 0.1;\n\n/// A slow writer that aims to simulate HDD behavior under heavy load.\n///\n/// There is a limited buffer, which is fully drained on the next write after\n/// a time limit is reached. Flush waits for the time limit to be reached\n/// and then drains the buffer.\n///\n/// At random, the HDD will stall writers while it flushes out all buffers. If\n/// this happens to you, you will be unable to write until the next time the\n/// buffer is drained.\nstruct SlowHddWriter {\n service_intervals: Interval,\n blocking_rng: ChaCha20Rng,\n buffer_size: usize,\n buffer_used: usize,\n}\n\nimpl SlowHddWriter {\n fn new(service_interval: Duration, buffer_size: usize) -> Self {\n let blocking_rng = ChaCha20Rng::seed_from_u64(RNG_SEED);\n let mut service_intervals = interval(service_interval);\n service_intervals.set_missed_tick_behavior(MissedTickBehavior::Delay);\n Self {\n service_intervals,\n blocking_rng,\n buffer_size,\n buffer_used: 0,\n }\n }\n\n fn service_write(\n mut self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n // If we hit a service interval, the buffer can be cleared\n let res = self.service_intervals.poll_tick(cx).map(|_| Ok(()));\n if res.is_ready() {\n self.buffer_used = 0;\n }\n res\n }\n\n fn write_bytes(\n mut self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n writeable: usize,\n ) -> std::task::Poll> {\n let service_res = self.as_mut().service_write(cx);\n\n if service_res.is_pending() && self.blocking_rng.random_bool(PROBABILITY_FLUSH_WAIT) {\n return Poll::Pending;\n }\n let available = self.buffer_size - self.buffer_used;\n\n if available == 0 {\n assert!(service_res.is_pending());\n Poll::Pending\n } else {\n let written = available.min(writeable);\n self.buffer_used += written;\n Poll::Ready(Ok(written))\n }\n }\n}\n\nimpl Unpin for SlowHddWriter {}\n\nimpl AsyncWrite for SlowHddWriter {\n fn poll_write(\n self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n buf: &[u8],\n ) -> std::task::Poll> {\n self.write_bytes(cx, buf.len())\n }\n\n fn poll_flush(\n self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n self.service_write(cx)\n }\n\n fn poll_shutdown(\n self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n self.service_write(cx)\n }\n\n fn poll_write_vectored(\n self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> std::task::Poll> {\n let writeable = bufs.iter().fold(0, |acc, buf| acc + buf.len());\n self.write_bytes(cx, writeable)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n}\n\n/// A reader that limits the maximum chunk it'll give you back\n///\n/// Simulates something reading from a slow link - you get one chunk per call,\n/// and you are offered chunks on a schedule\nstruct ChunkReader {\n data: Vec,\n service_intervals: Interval,\n}\n\nimpl ChunkReader {\n fn new(chunk_size: usize, service_interval: Duration) -> Self {\n let mut service_intervals = interval(service_interval);\n service_intervals.set_missed_tick_behavior(MissedTickBehavior::Burst);\n let data: Vec = std::iter::repeat_n(0, chunk_size).collect();\n Self {\n data,\n service_intervals,\n }\n }\n}\n\nimpl AsyncRead for ChunkReader {\n fn poll_read(\n mut self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n buf: &mut tokio::io::ReadBuf<'_>,\n ) -> Poll> {\n if self.service_intervals.poll_tick(cx).is_pending() {\n return Poll::Pending;\n }\n buf.put_slice(&self.data[..buf.remaining().min(self.data.len())]);\n Poll::Ready(Ok(()))\n }\n}\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap()\n}\n\nfn copy_mem_to_mem(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"copy_mem_to_mem\", |b| {\n b.iter(|| {\n let task = || async {\n let mut source = repeat(0).take(SOURCE_SIZE);\n let mut dest = Vec::new();\n copy(&mut source, &mut dest).await.unwrap();\n };\n\n rt.block_on(task());\n })\n });\n}\n\nfn copy_mem_to_slow_hdd(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"copy_mem_to_slow_hdd\", |b| {\n b.iter(|| {\n let task = || async {\n let mut source = repeat(0).take(SOURCE_SIZE);\n let mut dest = SlowHddWriter::new(WRITE_SERVICE_PERIOD, WRITE_BUFFER);\n copy(&mut source, &mut dest).await.unwrap();\n };\n\n rt.block_on(task());\n })\n });\n}\n\nfn copy_chunk_to_mem(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"copy_chunk_to_mem\", |b| {\n b.iter(|| {\n let task = || async {\n let mut source =\n ChunkReader::new(CHUNK_SIZE, READ_SERVICE_PERIOD).take(SOURCE_SIZE);\n let mut dest = Vec::new();\n copy(&mut source, &mut dest).await.unwrap();\n };\n\n rt.block_on(task());\n })\n });\n}\n\nfn copy_chunk_to_slow_hdd(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"copy_chunk_to_slow_hdd\", |b| {\n b.iter(|| {\n let task = || async {\n let mut source =\n ChunkReader::new(CHUNK_SIZE, READ_SERVICE_PERIOD).take(SOURCE_SIZE);\n let mut dest = SlowHddWriter::new(WRITE_SERVICE_PERIOD, WRITE_BUFFER);\n copy(&mut source, &mut dest).await.unwrap();\n };\n\n rt.block_on(task());\n })\n });\n}\n\ncriterion_group!(\n copy_bench,\n copy_mem_to_mem,\n copy_mem_to_slow_hdd,\n copy_chunk_to_mem,\n copy_chunk_to_slow_hdd,\n);\ncriterion_main!(copy_bench);\n" }, { "path": "benches/fs.rs", "content": "#![cfg(unix)]\n\nuse tokio_stream::StreamExt;\n\nuse tokio::fs::File;\nuse tokio::io::AsyncReadExt;\nuse tokio_util::codec::{BytesCodec, FramedRead /*FramedWrite*/};\n\nuse criterion::{criterion_group, criterion_main, Criterion};\n\nuse std::fs::File as StdFile;\nuse std::io::Read as StdRead;\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .build()\n .unwrap()\n}\n\nconst BLOCK_COUNT: usize = 1_000;\n\nconst BUFFER_SIZE: usize = 4096;\nconst DEV_ZERO: &str = \"/dev/zero\";\n\nfn async_read_codec(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"async_read_codec\", |b| {\n b.iter(|| {\n let task = || async {\n let file = File::open(DEV_ZERO).await.unwrap();\n let mut input_stream =\n FramedRead::with_capacity(file, BytesCodec::new(), BUFFER_SIZE);\n\n for _i in 0..BLOCK_COUNT {\n let _bytes = input_stream.next().await.unwrap();\n }\n };\n\n rt.block_on(task());\n })\n });\n}\n\nfn async_read_buf(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"async_read_buf\", |b| {\n b.iter(|| {\n let task = || async {\n let mut file = File::open(DEV_ZERO).await.unwrap();\n let mut buffer = [0u8; BUFFER_SIZE];\n\n for _i in 0..BLOCK_COUNT {\n let count = file.read(&mut buffer).await.unwrap();\n if count == 0 {\n break;\n }\n }\n };\n\n rt.block_on(task());\n });\n });\n}\n\nfn async_read_std_file(c: &mut Criterion) {\n let rt = rt();\n\n c.bench_function(\"async_read_std_file\", |b| {\n b.iter(|| {\n let task = || async {\n let mut file =\n tokio::task::block_in_place(|| Box::pin(StdFile::open(DEV_ZERO).unwrap()));\n\n for _i in 0..BLOCK_COUNT {\n let mut buffer = [0u8; BUFFER_SIZE];\n let mut file_ref = file.as_mut();\n\n tokio::task::block_in_place(move || {\n file_ref.read_exact(&mut buffer).unwrap();\n });\n }\n };\n\n rt.block_on(task());\n });\n });\n}\n\nfn sync_read(c: &mut Criterion) {\n c.bench_function(\"sync_read\", |b| {\n b.iter(|| {\n let mut file = StdFile::open(DEV_ZERO).unwrap();\n let mut buffer = [0u8; BUFFER_SIZE];\n\n for _i in 0..BLOCK_COUNT {\n file.read_exact(&mut buffer).unwrap();\n }\n })\n });\n}\n\ncriterion_group!(\n file,\n async_read_std_file,\n async_read_buf,\n async_read_codec,\n sync_read\n);\ncriterion_main!(file);\n" }, { "path": "benches/remote_spawn.rs", "content": "//! Benchmark remote task spawning (push_remote_task) at different concurrency\n//! levels on the multi-threaded scheduler.\n//!\n//! This measures contention on the scheduler's inject queue mutex when multiple\n//! external (non-worker) threads spawn tasks into the tokio runtime simultaneously.\n//! Every rt.spawn() from an external thread unconditionally goes through\n//! push_remote_task, making this a direct measurement of inject queue contention.\n//!\n//! For each parallelism level N (1, 2, 4, 8, 16, 32, 64, capped at available parallelism):\n//! - Spawns N std::threads (external to the runtime)\n//! - Each thread spawns TOTAL_TASKS / N tasks into the runtime via rt.spawn()\n//! - All threads are synchronized with a barrier to maximize contention\n//! - Tasks are trivial no-ops to isolate the push overhead\n\nuse criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};\nuse std::sync::Barrier;\nuse tokio::runtime::{self, Runtime};\n\n/// Total number of tasks spawned across all threads per iteration.\n/// Must be divisible by the largest parallelism level (64).\nconst TOTAL_TASKS: usize = 12_800;\nconst _: () = assert!(TOTAL_TASKS % 64 == 0, \"TOTAL_TASKS must be divisible by 64\");\n\nfn remote_spawn_contention(c: &mut Criterion) {\n let parallelism_levels = parallelism_levels();\n let mut group = c.benchmark_group(\"remote_spawn\");\n\n for num_threads in ¶llelism_levels {\n let num_threads = *num_threads;\n group.bench_with_input(\n BenchmarkId::new(\"threads\", num_threads),\n &num_threads,\n |b, &num_threads| {\n let rt = rt();\n let tasks_per_thread = TOTAL_TASKS / num_threads;\n let barrier = Barrier::new(num_threads);\n\n b.iter_custom(|iters| {\n let mut total_duration = std::time::Duration::ZERO;\n for _ in 0..iters {\n let start = std::time::Instant::now();\n\n let all_handles = std::thread::scope(|s| {\n let handles: Vec<_> = (0..num_threads)\n .map(|_| {\n let barrier = &barrier;\n let rt = &rt;\n s.spawn(move || {\n let mut join_handles = Vec::with_capacity(tasks_per_thread);\n barrier.wait();\n\n for _ in 0..tasks_per_thread {\n join_handles.push(rt.spawn(async {}));\n }\n join_handles\n })\n })\n .collect();\n\n handles\n .into_iter()\n .flat_map(|h| h.join().unwrap())\n .collect::>()\n });\n\n total_duration += start.elapsed();\n\n rt.block_on(async {\n for h in all_handles {\n h.await.unwrap();\n }\n });\n }\n total_duration\n });\n },\n );\n }\n\n group.finish();\n}\n\nfn parallelism_levels() -> Vec {\n let max_parallelism = std::thread::available_parallelism()\n .map(|p| p.get())\n .unwrap_or(1);\n\n [1, 2, 4, 8, 16, 32, 64]\n .into_iter()\n .filter(|&n| n <= max_parallelism)\n .collect()\n}\n\nfn rt() -> Runtime {\n runtime::Builder::new_multi_thread().build().unwrap()\n}\n\ncriterion_group!(remote_spawn_benches, remote_spawn_contention);\n\ncriterion_main!(remote_spawn_benches);\n" }, { "path": "benches/rt_current_thread.rs", "content": "//! Benchmark implementation details of the threaded scheduler. These benches are\n//! intended to be used as a form of regression testing and not as a general\n//! purpose benchmark demonstrating real-world performance.\n\nuse tokio::runtime::{self, Runtime};\n\nuse criterion::{criterion_group, criterion_main, Criterion};\n\nconst NUM_SPAWN: usize = 1_000;\n\nfn rt_curr_spawn_many_local(c: &mut Criterion) {\n let rt = rt();\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n\n c.bench_function(\"spawn_many_local\", |b| {\n b.iter(|| {\n rt.block_on(async {\n for _ in 0..NUM_SPAWN {\n handles.push(tokio::spawn(async move {}));\n }\n\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n}\n\nfn rt_curr_spawn_many_remote_idle(c: &mut Criterion) {\n let rt = rt();\n let rt_handle = rt.handle();\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n\n c.bench_function(\"spawn_many_remote_idle\", |b| {\n b.iter(|| {\n for _ in 0..NUM_SPAWN {\n handles.push(rt_handle.spawn(async {}));\n }\n\n rt.block_on(async {\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n}\n\nfn rt_curr_spawn_many_remote_busy(c: &mut Criterion) {\n let rt = rt();\n let rt_handle = rt.handle();\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n\n rt.spawn(async {\n fn iter() {\n tokio::spawn(async { iter() });\n }\n\n iter()\n });\n\n c.bench_function(\"spawn_many_remote_busy\", |b| {\n b.iter(|| {\n for _ in 0..NUM_SPAWN {\n handles.push(rt_handle.spawn(async {}));\n }\n\n rt.block_on(async {\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n}\n\nfn rt() -> Runtime {\n runtime::Builder::new_current_thread().build().unwrap()\n}\n\ncriterion_group!(\n rt_curr_scheduler,\n rt_curr_spawn_many_local,\n rt_curr_spawn_many_remote_idle,\n rt_curr_spawn_many_remote_busy\n);\n\ncriterion_main!(rt_curr_scheduler);\n" }, { "path": "benches/rt_multi_threaded.rs", "content": "//! Benchmark implementation details of the threaded scheduler. These benches are\n//! intended to be used as a form of regression testing and not as a general\n//! purpose benchmark demonstrating real-world performance.\n\nuse tokio::runtime::{self, Runtime};\nuse tokio::sync::oneshot;\n\nuse std::sync::atomic::Ordering::Relaxed;\nuse std::sync::atomic::{AtomicBool, AtomicUsize};\nuse std::sync::{mpsc, Arc};\nuse std::time::{Duration, Instant};\n\nuse criterion::{criterion_group, criterion_main, Criterion};\n\nconst NUM_WORKERS: usize = 4;\nconst NUM_SPAWN: usize = 10_000;\nconst STALL_DUR: Duration = Duration::from_micros(10);\n\nfn rt_multi_spawn_many_local(c: &mut Criterion) {\n let rt = rt();\n\n let (tx, rx) = mpsc::sync_channel(1000);\n let rem = Arc::new(AtomicUsize::new(0));\n\n c.bench_function(\"spawn_many_local\", |b| {\n b.iter(|| {\n rem.store(NUM_SPAWN, Relaxed);\n\n rt.block_on(async {\n for _ in 0..NUM_SPAWN {\n let tx = tx.clone();\n let rem = rem.clone();\n\n tokio::spawn(async move {\n if 1 == rem.fetch_sub(1, Relaxed) {\n tx.send(()).unwrap();\n }\n });\n }\n\n rx.recv().unwrap();\n });\n })\n });\n}\n\nfn rt_multi_spawn_many_remote_idle(c: &mut Criterion) {\n let rt = rt();\n\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n\n c.bench_function(\"spawn_many_remote_idle\", |b| {\n b.iter(|| {\n for _ in 0..NUM_SPAWN {\n handles.push(rt.spawn(async {}));\n }\n\n rt.block_on(async {\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n}\n\n// The runtime is busy with tasks that consume CPU time and yield. Yielding is a\n// lower notification priority than spawning / regular notification.\nfn rt_multi_spawn_many_remote_busy1(c: &mut Criterion) {\n let rt = rt();\n let rt_handle = rt.handle();\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n let flag = Arc::new(AtomicBool::new(true));\n\n // Spawn some tasks to keep the runtimes busy\n for _ in 0..(2 * NUM_WORKERS) {\n let flag = flag.clone();\n rt.spawn(async move {\n while flag.load(Relaxed) {\n tokio::task::yield_now().await;\n stall();\n }\n });\n }\n\n c.bench_function(\"spawn_many_remote_busy1\", |b| {\n b.iter(|| {\n for _ in 0..NUM_SPAWN {\n handles.push(rt_handle.spawn(async {}));\n }\n\n rt.block_on(async {\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n\n flag.store(false, Relaxed);\n}\n\n// The runtime is busy with tasks that consume CPU time and spawn new high-CPU\n// tasks. Spawning goes via a higher notification priority than yielding.\nfn rt_multi_spawn_many_remote_busy2(c: &mut Criterion) {\n const NUM_SPAWN: usize = 1_000;\n\n let rt = rt();\n let rt_handle = rt.handle();\n let mut handles = Vec::with_capacity(NUM_SPAWN);\n let flag = Arc::new(AtomicBool::new(true));\n\n // Spawn some tasks to keep the runtimes busy\n for _ in 0..(NUM_WORKERS) {\n let flag = flag.clone();\n fn iter(flag: Arc) {\n tokio::spawn(async {\n if flag.load(Relaxed) {\n stall();\n iter(flag);\n }\n });\n }\n rt.spawn(async {\n iter(flag);\n });\n }\n\n c.bench_function(\"spawn_many_remote_busy2\", |b| {\n b.iter(|| {\n for _ in 0..NUM_SPAWN {\n handles.push(rt_handle.spawn(async {}));\n }\n\n rt.block_on(async {\n for handle in handles.drain(..) {\n handle.await.unwrap();\n }\n });\n })\n });\n\n flag.store(false, Relaxed);\n}\n\nfn rt_multi_yield_many(c: &mut Criterion) {\n const NUM_YIELD: usize = 1_000;\n const TASKS: usize = 200;\n\n c.bench_function(\"yield_many\", |b| {\n let rt = rt();\n let (tx, rx) = mpsc::sync_channel(TASKS);\n\n b.iter(move || {\n for _ in 0..TASKS {\n let tx = tx.clone();\n\n rt.spawn(async move {\n for _ in 0..NUM_YIELD {\n tokio::task::yield_now().await;\n }\n\n tx.send(()).unwrap();\n });\n }\n\n for _ in 0..TASKS {\n rx.recv().unwrap();\n }\n })\n });\n}\n\nfn rt_multi_ping_pong(c: &mut Criterion) {\n const NUM_PINGS: usize = 1_000;\n\n let rt = rt();\n\n let (done_tx, done_rx) = mpsc::sync_channel(1000);\n let rem = Arc::new(AtomicUsize::new(0));\n\n c.bench_function(\"ping_pong\", |b| {\n b.iter(|| {\n let done_tx = done_tx.clone();\n let rem = rem.clone();\n rem.store(NUM_PINGS, Relaxed);\n\n rt.block_on(async {\n tokio::spawn(async move {\n for _ in 0..NUM_PINGS {\n let rem = rem.clone();\n let done_tx = done_tx.clone();\n\n tokio::spawn(async move {\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n tokio::spawn(async move {\n rx1.await.unwrap();\n tx2.send(()).unwrap();\n });\n\n tx1.send(()).unwrap();\n rx2.await.unwrap();\n\n if 1 == rem.fetch_sub(1, Relaxed) {\n done_tx.send(()).unwrap();\n }\n });\n }\n });\n\n done_rx.recv().unwrap();\n });\n })\n });\n}\n\nfn rt_multi_chained_spawn(c: &mut Criterion) {\n const ITER: usize = 1_000;\n\n fn iter(done_tx: mpsc::SyncSender<()>, n: usize) {\n if n == 0 {\n done_tx.send(()).unwrap();\n } else {\n tokio::spawn(async move {\n iter(done_tx, n - 1);\n });\n }\n }\n\n c.bench_function(\"chained_spawn\", |b| {\n let rt = rt();\n let (done_tx, done_rx) = mpsc::sync_channel(1000);\n\n b.iter(move || {\n let done_tx = done_tx.clone();\n\n rt.block_on(async {\n tokio::spawn(async move {\n iter(done_tx, ITER);\n });\n\n done_rx.recv().unwrap();\n });\n })\n });\n}\n\nfn rt() -> Runtime {\n runtime::Builder::new_multi_thread()\n .worker_threads(NUM_WORKERS)\n .enable_all()\n .build()\n .unwrap()\n}\n\nfn stall() {\n let now = Instant::now();\n while now.elapsed() < STALL_DUR {\n std::thread::yield_now();\n }\n}\n\ncriterion_group!(\n rt_multi_scheduler,\n rt_multi_spawn_many_local,\n rt_multi_spawn_many_remote_idle,\n rt_multi_spawn_many_remote_busy1,\n rt_multi_spawn_many_remote_busy2,\n rt_multi_ping_pong,\n rt_multi_yield_many,\n rt_multi_chained_spawn,\n);\n\ncriterion_main!(rt_multi_scheduler);\n" }, { "path": "benches/signal.rs", "content": "//! Benchmark the delay in propagating OS signals to any listeners.\n#![cfg(unix)]\n\nuse criterion::{criterion_group, criterion_main, Criterion};\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::runtime;\nuse tokio::signal::unix::{signal, SignalKind};\nuse tokio::sync::mpsc;\n\nstruct Spinner {\n count: usize,\n}\n\nimpl Future for Spinner {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n if self.count > 3 {\n Poll::Ready(())\n } else {\n self.count += 1;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n}\n\nimpl Spinner {\n fn new() -> Self {\n Self { count: 0 }\n }\n}\n\npub fn send_signal(signal: libc::c_int) {\n use libc::{getpid, kill};\n\n unsafe {\n assert_eq!(kill(getpid(), signal), 0);\n }\n}\n\nfn many_signals(c: &mut Criterion) {\n let num_signals = 10;\n let (tx, mut rx) = mpsc::channel(num_signals);\n\n // Intentionally single threaded to measure delays in propagating wakes\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n let spawn_signal = |kind| {\n let tx = tx.clone();\n rt.spawn(async move {\n let mut signal = signal(kind).expect(\"failed to create signal\");\n\n while signal.recv().await.is_some() {\n if tx.send(()).await.is_err() {\n break;\n }\n }\n });\n };\n\n for _ in 0..num_signals {\n // Pick some random signals which don't terminate the test harness\n spawn_signal(SignalKind::child());\n spawn_signal(SignalKind::io());\n }\n drop(tx);\n\n // Turn the runtime for a while to ensure that all the spawned\n // tasks have been polled at least once\n rt.block_on(Spinner::new());\n\n c.bench_function(\"many_signals\", |b| {\n b.iter(|| {\n rt.block_on(async {\n send_signal(libc::SIGCHLD);\n for _ in 0..num_signals {\n rx.recv().await.expect(\"channel closed\");\n }\n\n send_signal(libc::SIGIO);\n for _ in 0..num_signals {\n rx.recv().await.expect(\"channel closed\");\n }\n });\n })\n });\n}\n\ncriterion_group!(signal_group, many_signals);\n\ncriterion_main!(signal_group);\n" }, { "path": "benches/spawn.rs", "content": "//! Benchmark spawning a task onto the basic and threaded Tokio executors.\n//! This essentially measure the time to enqueue a task in the local and remote\n//! case.\n\nuse criterion::{black_box, criterion_group, criterion_main, Criterion};\n\nasync fn work() -> usize {\n let val = 1 + 1;\n tokio::task::yield_now().await;\n black_box(val)\n}\n\nfn single_rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n}\n\nfn multi_rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap()\n}\n\nfn basic_scheduler_spawn(c: &mut Criterion) {\n let runtime = single_rt();\n\n c.bench_function(\"basic_scheduler_spawn\", |b| {\n b.iter(|| {\n runtime.block_on(async {\n let h = tokio::spawn(work());\n assert_eq!(h.await.unwrap(), 2);\n });\n })\n });\n}\n\nfn basic_scheduler_spawn10(c: &mut Criterion) {\n let runtime = single_rt();\n\n c.bench_function(\"basic_scheduler_spawn10\", |b| {\n b.iter(|| {\n runtime.block_on(async {\n let mut handles = Vec::with_capacity(10);\n for _ in 0..10 {\n handles.push(tokio::spawn(work()));\n }\n for handle in handles {\n assert_eq!(handle.await.unwrap(), 2);\n }\n });\n })\n });\n}\n\nfn threaded_scheduler_spawn(c: &mut Criterion) {\n let runtime = multi_rt();\n\n c.bench_function(\"threaded_scheduler_spawn\", |b| {\n b.iter(|| {\n runtime.block_on(async {\n let h = tokio::spawn(work());\n assert_eq!(h.await.unwrap(), 2);\n });\n })\n });\n}\n\nfn threaded_scheduler_spawn10(c: &mut Criterion) {\n let runtime = multi_rt();\n\n c.bench_function(\"threaded_scheduler_spawn10\", |b| {\n b.iter(|| {\n runtime.block_on(async {\n let mut handles = Vec::with_capacity(10);\n for _ in 0..10 {\n handles.push(tokio::spawn(work()));\n }\n for handle in handles {\n assert_eq!(handle.await.unwrap(), 2);\n }\n });\n })\n });\n}\n\ncriterion_group!(\n spawn,\n basic_scheduler_spawn,\n basic_scheduler_spawn10,\n threaded_scheduler_spawn,\n threaded_scheduler_spawn10,\n);\n\ncriterion_main!(spawn);\n" }, { "path": "benches/spawn_blocking.rs", "content": "//! Benchmark spawn_blocking at different concurrency levels on the multi-threaded scheduler.\n//!\n//! For each parallelism level N (1, 2, 4, 8, 16, 32, 64, capped at available parallelism):\n//! - Spawns N regular async tasks\n//! - Each task spawns M batches of B spawn_blocking tasks (no-ops)\n//! - Each batch is awaited to completion before starting the next\n\nuse criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion};\nuse tokio::runtime::{self, Runtime};\nuse tokio::task::JoinSet;\n\n/// Number of batches per task\nconst NUM_BATCHES: usize = 100;\n/// Number of spawn_blocking calls per batch\nconst BATCH_SIZE: usize = 16;\n\nfn spawn_blocking_concurrency(c: &mut Criterion) {\n let max_parallelism = std::thread::available_parallelism()\n .map(|p| p.get())\n .unwrap_or(1);\n\n let parallelism_levels: Vec = [1, 2, 4, 8, 16, 32, 64]\n .into_iter()\n .filter(|&n| n <= max_parallelism)\n .collect();\n\n let mut group = c.benchmark_group(\"spawn_blocking\");\n\n for num_tasks in parallelism_levels {\n group.bench_with_input(\n BenchmarkId::new(\"concurrency\", num_tasks),\n &num_tasks,\n |b, &num_tasks| {\n let rt = rt();\n\n b.iter(|| {\n rt.block_on(async {\n let mut tasks = JoinSet::new();\n\n for _ in 0..num_tasks {\n tasks.spawn(async {\n for _ in 0..NUM_BATCHES {\n let mut batch = JoinSet::new();\n\n for _ in 0..BATCH_SIZE {\n batch.spawn_blocking(|| black_box(0));\n }\n\n batch.join_all().await;\n }\n });\n }\n\n tasks.join_all().await;\n });\n });\n },\n );\n }\n\n group.finish();\n}\n\nfn rt() -> Runtime {\n runtime::Builder::new_multi_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\ncriterion_group!(spawn_blocking_benches, spawn_blocking_concurrency);\n\ncriterion_main!(spawn_blocking_benches);\n" }, { "path": "benches/sync_broadcast.rs", "content": "use rand::{Rng, RngCore, SeedableRng};\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::Arc;\nuse tokio::sync::{broadcast, Notify};\n\nuse criterion::measurement::WallTime;\nuse criterion::{black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap()\n}\n\nfn do_work(rng: &mut impl RngCore) -> u32 {\n use std::fmt::Write;\n let mut message = String::new();\n for i in 1..=10 {\n let _ = write!(&mut message, \" {i}={}\", rng.random::());\n }\n message\n .as_bytes()\n .iter()\n .map(|&c| c as u32)\n .fold(0, u32::wrapping_add)\n}\n\nfn contention_impl(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n let (tx, _rx) = broadcast::channel::(1000);\n let wg = Arc::new((AtomicUsize::new(0), Notify::new()));\n\n for n in 0..N_TASKS {\n let wg = wg.clone();\n let mut rx = tx.subscribe();\n let mut rng = rand::rngs::StdRng::seed_from_u64(n as u64);\n rt.spawn(async move {\n while (rx.recv().await).is_ok() {\n let r = do_work(&mut rng);\n let _ = black_box(r);\n if wg.0.fetch_sub(1, Ordering::Relaxed) == 1 {\n wg.1.notify_one();\n }\n }\n });\n }\n\n const N_ITERS: usize = 100;\n\n g.bench_function(N_TASKS.to_string(), |b| {\n b.iter(|| {\n rt.block_on({\n let wg = wg.clone();\n let tx = tx.clone();\n async move {\n for i in 0..N_ITERS {\n assert_eq!(wg.0.fetch_add(N_TASKS, Ordering::Relaxed), 0);\n tx.send(i).unwrap();\n while wg.0.load(Ordering::Relaxed) > 0 {\n wg.1.notified().await;\n }\n }\n }\n })\n })\n });\n}\n\nfn bench_contention(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"contention\");\n contention_impl::<10>(&mut group);\n contention_impl::<100>(&mut group);\n contention_impl::<500>(&mut group);\n contention_impl::<1000>(&mut group);\n group.finish();\n}\n\ncriterion_group!(contention, bench_contention);\n\ncriterion_main!(contention);\n" }, { "path": "benches/sync_mpsc.rs", "content": "use tokio::sync::mpsc;\n\nuse criterion::measurement::WallTime;\nuse criterion::{black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\n#[derive(Debug, Copy, Clone)]\nstruct Medium(#[allow(dead_code)] [usize; 64]);\nimpl Default for Medium {\n fn default() -> Self {\n Medium([0; 64])\n }\n}\n\n#[derive(Debug, Copy, Clone)]\nstruct Large(#[allow(dead_code)] [Medium; 64]);\nimpl Default for Large {\n fn default() -> Self {\n Large([Medium::default(); 64])\n }\n}\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap()\n}\n\nfn create_medium(g: &mut BenchmarkGroup) {\n g.bench_function(SIZE.to_string(), |b| {\n b.iter(|| {\n black_box(&mpsc::channel::(SIZE));\n })\n });\n}\n\nfn send_data(g: &mut BenchmarkGroup, prefix: &str) {\n let rt = rt();\n\n g.bench_function(format!(\"{prefix}_{SIZE}\"), |b| {\n b.iter(|| {\n let (tx, mut rx) = mpsc::channel::(SIZE);\n\n let _ = rt.block_on(tx.send(T::default()));\n\n rt.block_on(rx.recv()).unwrap();\n })\n });\n}\n\nfn contention_bounded(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(1_000_000);\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).await.unwrap();\n }\n });\n }\n\n for _ in 0..1_000 * 5 {\n let _ = rx.recv().await;\n }\n })\n })\n });\n}\n\nfn contention_bounded_recv_many(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded_recv_many\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(1_000_000);\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).await.unwrap();\n }\n });\n }\n\n let mut buffer = Vec::::with_capacity(5_000);\n let mut total = 0;\n while total < 1_000 * 5 {\n total += rx.recv_many(&mut buffer, 5_000).await;\n }\n })\n })\n });\n}\n\nfn contention_bounded_full(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded_full\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(100);\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).await.unwrap();\n }\n });\n }\n\n for _ in 0..1_000 * 5 {\n let _ = rx.recv().await;\n }\n })\n })\n });\n}\n\nfn contention_bounded_full_recv_many(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded_full_recv_many\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(100);\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).await.unwrap();\n }\n });\n }\n\n let mut buffer = Vec::::with_capacity(5_000);\n let mut total = 0;\n while total < 1_000 * 5 {\n total += rx.recv_many(&mut buffer, 5_000).await;\n }\n })\n })\n });\n}\n\nfn contention_unbounded(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"unbounded\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).unwrap();\n }\n });\n }\n\n for _ in 0..1_000 * 5 {\n let _ = rx.recv().await;\n }\n })\n })\n });\n}\n\nfn contention_unbounded_recv_many(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"unbounded_recv_many\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n for _ in 0..5 {\n let tx = tx.clone();\n tokio::spawn(async move {\n for i in 0..1000 {\n tx.send(i).unwrap();\n }\n });\n }\n\n let mut buffer = Vec::::with_capacity(5_000);\n let mut total = 0;\n while total < 1_000 * 5 {\n total += rx.recv_many(&mut buffer, 5_000).await;\n }\n })\n })\n });\n}\n\nfn uncontented_bounded(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(1_000_000);\n\n for i in 0..5000 {\n tx.send(i).await.unwrap();\n }\n\n for _ in 0..5_000 {\n let _ = rx.recv().await;\n }\n })\n })\n });\n}\n\nfn uncontented_bounded_recv_many(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"bounded_recv_many\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::(1_000_000);\n\n for i in 0..5000 {\n tx.send(i).await.unwrap();\n }\n\n let mut buffer = Vec::::with_capacity(5_000);\n let mut total = 0;\n while total < 1_000 * 5 {\n total += rx.recv_many(&mut buffer, 5_000).await;\n }\n })\n })\n });\n}\n\nfn uncontented_unbounded(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"unbounded\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n for i in 0..5000 {\n tx.send(i).unwrap();\n }\n\n for _ in 0..5_000 {\n let _ = rx.recv().await;\n }\n })\n })\n });\n}\n\nfn uncontented_unbounded_recv_many(g: &mut BenchmarkGroup) {\n let rt = rt();\n\n g.bench_function(\"unbounded_recv_many\", |b| {\n b.iter(|| {\n rt.block_on(async move {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n for i in 0..5000 {\n tx.send(i).unwrap();\n }\n\n let mut buffer = Vec::::with_capacity(5_000);\n let mut total = 0;\n while total < 1_000 * 5 {\n total += rx.recv_many(&mut buffer, 5_000).await;\n }\n })\n })\n });\n}\n\nfn bench_create_medium(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"create_medium\");\n create_medium::<1>(&mut group);\n create_medium::<100>(&mut group);\n create_medium::<100_000>(&mut group);\n group.finish();\n}\n\nfn bench_send(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"send\");\n send_data::(&mut group, \"medium\");\n send_data::(&mut group, \"large\");\n group.finish();\n}\n\nfn bench_contention(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"contention\");\n contention_bounded(&mut group);\n contention_bounded_recv_many(&mut group);\n contention_bounded_full(&mut group);\n contention_bounded_full_recv_many(&mut group);\n contention_unbounded(&mut group);\n contention_unbounded_recv_many(&mut group);\n group.finish();\n}\n\nfn bench_uncontented(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"uncontented\");\n uncontented_bounded(&mut group);\n uncontented_bounded_recv_many(&mut group);\n uncontented_unbounded(&mut group);\n uncontented_unbounded_recv_many(&mut group);\n group.finish();\n}\n\ncriterion_group!(create, bench_create_medium);\ncriterion_group!(send, bench_send);\ncriterion_group!(contention, bench_contention);\ncriterion_group!(uncontented, bench_uncontented);\n\ncriterion_main!(create, send, contention, uncontented);\n" }, { "path": "benches/sync_mpsc_oneshot.rs", "content": "use tokio::{\n runtime::Runtime,\n sync::{mpsc, oneshot},\n};\n\nuse criterion::{criterion_group, criterion_main, Criterion};\n\nfn request_reply_current_thread(c: &mut Criterion) {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n request_reply(c, rt);\n}\n\nfn request_reply_multi_threaded(c: &mut Criterion) {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n request_reply(c, rt);\n}\n\nfn request_reply(b: &mut Criterion, rt: Runtime) {\n let tx = rt.block_on(async move {\n let (tx, mut rx) = mpsc::channel::>(10);\n tokio::spawn(async move {\n while let Some(reply) = rx.recv().await {\n reply.send(()).unwrap();\n }\n });\n tx\n });\n\n b.bench_function(\"request_reply\", |b| {\n b.iter(|| {\n let task_tx = tx.clone();\n rt.block_on(async move {\n for _ in 0..1_000 {\n let (o_tx, o_rx) = oneshot::channel();\n task_tx.send(o_tx).await.unwrap();\n let _ = o_rx.await;\n }\n })\n })\n });\n}\n\ncriterion_group!(\n sync_mpsc_oneshot_group,\n request_reply_current_thread,\n request_reply_multi_threaded,\n);\n\ncriterion_main!(sync_mpsc_oneshot_group);\n" }, { "path": "benches/sync_notify.rs", "content": "use std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::Arc;\n\nuse tokio::sync::Notify;\n\nuse criterion::measurement::WallTime;\nuse criterion::{criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap()\n}\n\nfn notify_waiters(g: &mut BenchmarkGroup) {\n let rt = rt();\n let notify = Arc::new(Notify::new());\n let counter = Arc::new(AtomicUsize::new(0));\n for _ in 0..N_WAITERS {\n rt.spawn({\n let notify = notify.clone();\n let counter = counter.clone();\n async move {\n loop {\n notify.notified().await;\n counter.fetch_add(1, Ordering::Relaxed);\n }\n }\n });\n }\n\n const N_ITERS: usize = 500;\n g.bench_function(N_WAITERS.to_string(), |b| {\n b.iter(|| {\n counter.store(0, Ordering::Relaxed);\n loop {\n notify.notify_waiters();\n if counter.load(Ordering::Relaxed) >= N_ITERS {\n break;\n }\n }\n })\n });\n}\n\nfn notify_one(g: &mut BenchmarkGroup) {\n let rt = rt();\n let notify = Arc::new(Notify::new());\n let counter = Arc::new(AtomicUsize::new(0));\n for _ in 0..N_WAITERS {\n rt.spawn({\n let notify = notify.clone();\n let counter = counter.clone();\n async move {\n loop {\n notify.notified().await;\n counter.fetch_add(1, Ordering::Relaxed);\n }\n }\n });\n }\n\n const N_ITERS: usize = 500;\n g.bench_function(N_WAITERS.to_string(), |b| {\n b.iter(|| {\n counter.store(0, Ordering::Relaxed);\n loop {\n notify.notify_one();\n if counter.load(Ordering::Relaxed) >= N_ITERS {\n break;\n }\n }\n })\n });\n}\n\nfn bench_notify_one(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"notify_one\");\n notify_one::<10>(&mut group);\n notify_one::<50>(&mut group);\n notify_one::<100>(&mut group);\n notify_one::<200>(&mut group);\n notify_one::<500>(&mut group);\n group.finish();\n}\n\nfn bench_notify_waiters(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"notify_waiters\");\n notify_waiters::<10>(&mut group);\n notify_waiters::<50>(&mut group);\n notify_waiters::<100>(&mut group);\n notify_waiters::<200>(&mut group);\n notify_waiters::<500>(&mut group);\n group.finish();\n}\n\ncriterion_group!(\n notify_waiters_simple,\n bench_notify_one,\n bench_notify_waiters\n);\n\ncriterion_main!(notify_waiters_simple);\n" }, { "path": "benches/sync_rwlock.rs", "content": "use std::sync::Arc;\nuse tokio::{sync::RwLock, task};\n\nuse criterion::measurement::WallTime;\nuse criterion::{black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\nfn read_uncontended(g: &mut BenchmarkGroup) {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap();\n\n let lock = Arc::new(RwLock::new(()));\n g.bench_function(\"read\", |b| {\n b.iter(|| {\n let lock = lock.clone();\n rt.block_on(async move {\n for _ in 0..6 {\n let read = lock.read().await;\n let _read = black_box(read);\n }\n })\n })\n });\n}\n\nfn read_concurrent_uncontended_multi(g: &mut BenchmarkGroup) {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap();\n\n async fn task(lock: Arc>) {\n let read = lock.read().await;\n let _read = black_box(read);\n }\n\n let lock = Arc::new(RwLock::new(()));\n g.bench_function(\"read_concurrent_multi\", |b| {\n b.iter(|| {\n let lock = lock.clone();\n rt.block_on(async move {\n let j = tokio::try_join! {\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone()))\n };\n j.unwrap();\n })\n })\n });\n}\n\nfn read_concurrent_uncontended(g: &mut BenchmarkGroup) {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n async fn task(lock: Arc>) {\n let read = lock.read().await;\n let _read = black_box(read);\n }\n\n let lock = Arc::new(RwLock::new(()));\n g.bench_function(\"read_concurrent\", |b| {\n b.iter(|| {\n let lock = lock.clone();\n rt.block_on(async move {\n tokio::join! {\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone())\n };\n })\n })\n });\n}\n\nfn read_concurrent_contended_multi(g: &mut BenchmarkGroup) {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap();\n\n async fn task(lock: Arc>) {\n let read = lock.read().await;\n let _read = black_box(read);\n }\n\n let lock = Arc::new(RwLock::new(()));\n g.bench_function(\"read_concurrent_multi\", |b| {\n b.iter(|| {\n let lock = lock.clone();\n rt.block_on(async move {\n let write = lock.write().await;\n let j = tokio::try_join! {\n async move { drop(write); Ok(()) },\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n task::spawn(task(lock.clone())),\n };\n j.unwrap();\n })\n })\n });\n}\n\nfn read_concurrent_contended(g: &mut BenchmarkGroup) {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n async fn task(lock: Arc>) {\n let read = lock.read().await;\n let _read = black_box(read);\n }\n\n let lock = Arc::new(RwLock::new(()));\n g.bench_function(\"read_concurrent\", |b| {\n b.iter(|| {\n let lock = lock.clone();\n rt.block_on(async move {\n let write = lock.write().await;\n tokio::join! {\n async move { drop(write) },\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n task(lock.clone()),\n };\n })\n })\n });\n}\n\nfn bench_contention(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"contention\");\n read_concurrent_contended(&mut group);\n read_concurrent_contended_multi(&mut group);\n group.finish();\n}\n\nfn bench_uncontented(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"uncontented\");\n read_uncontended(&mut group);\n read_concurrent_uncontended(&mut group);\n read_concurrent_uncontended_multi(&mut group);\n group.finish();\n}\n\ncriterion_group!(contention, bench_contention);\ncriterion_group!(uncontented, bench_uncontented);\n\ncriterion_main!(contention, uncontented);\n" }, { "path": "benches/sync_semaphore.rs", "content": "use std::sync::Arc;\nuse tokio::runtime::Runtime;\nuse tokio::{sync::Semaphore, task};\n\nuse criterion::measurement::WallTime;\nuse criterion::{criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\nfn single_rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n}\n\nfn multi_rt() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap()\n}\n\nfn uncontended(g: &mut BenchmarkGroup) {\n let rt = multi_rt();\n\n let s = Arc::new(Semaphore::new(10));\n g.bench_function(\"multi\", |b| {\n b.iter(|| {\n let s = s.clone();\n rt.block_on(async move {\n for _ in 0..6 {\n let permit = s.acquire().await;\n drop(permit);\n }\n })\n })\n });\n}\n\nasync fn task(s: Arc) {\n let permit = s.acquire().await;\n drop(permit);\n}\n\nfn uncontended_concurrent_multi(g: &mut BenchmarkGroup) {\n let rt = multi_rt();\n\n let s = Arc::new(Semaphore::new(10));\n g.bench_function(\"concurrent_multi\", |b| {\n b.iter(|| {\n let s = s.clone();\n rt.block_on(async move {\n let j = tokio::try_join! {\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone()))\n };\n j.unwrap();\n })\n })\n });\n}\n\nfn uncontended_concurrent_single(g: &mut BenchmarkGroup) {\n let rt = single_rt();\n\n let s = Arc::new(Semaphore::new(10));\n g.bench_function(\"concurrent_single\", |b| {\n b.iter(|| {\n let s = s.clone();\n rt.block_on(async move {\n tokio::join! {\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone())\n };\n })\n })\n });\n}\n\nfn contended_concurrent_multi(g: &mut BenchmarkGroup) {\n let rt = multi_rt();\n\n let s = Arc::new(Semaphore::new(5));\n g.bench_function(\"concurrent_multi\", |b| {\n b.iter(|| {\n let s = s.clone();\n rt.block_on(async move {\n let j = tokio::try_join! {\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone())),\n task::spawn(task(s.clone()))\n };\n j.unwrap();\n })\n })\n });\n}\n\nfn contended_concurrent_single(g: &mut BenchmarkGroup) {\n let rt = single_rt();\n\n let s = Arc::new(Semaphore::new(5));\n g.bench_function(\"concurrent_single\", |b| {\n b.iter(|| {\n let s = s.clone();\n rt.block_on(async move {\n tokio::join! {\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone()),\n task(s.clone())\n };\n })\n })\n });\n}\n\nfn bench_contention(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"contention\");\n contended_concurrent_multi(&mut group);\n contended_concurrent_single(&mut group);\n group.finish();\n}\n\nfn bench_uncontented(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"uncontented\");\n uncontended(&mut group);\n uncontended_concurrent_multi(&mut group);\n uncontended_concurrent_single(&mut group);\n group.finish();\n}\n\ncriterion_group!(contention, bench_contention);\ncriterion_group!(uncontented, bench_uncontented);\n\ncriterion_main!(contention, uncontented);\n" }, { "path": "benches/sync_watch.rs", "content": "use rand::prelude::*;\nuse std::sync::atomic::{AtomicU64, Ordering};\nuse std::sync::Arc;\nuse tokio::sync::{watch, Notify};\n\nuse criterion::measurement::WallTime;\nuse criterion::{black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion};\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(6)\n .build()\n .unwrap()\n}\n\nfn do_work(rng: &mut impl RngCore) -> u32 {\n use std::fmt::Write;\n let mut message = String::new();\n for i in 1..=10 {\n let _ = write!(&mut message, \" {i}={}\", rng.random::());\n }\n message\n .as_bytes()\n .iter()\n .map(|&c| c as u32)\n .fold(0, u32::wrapping_add)\n}\n\nfn contention_resubscribe(g: &mut BenchmarkGroup) {\n let rt = rt();\n let (snd, _) = watch::channel(0i32);\n let snd = Arc::new(snd);\n let wg = Arc::new((AtomicU64::new(0), Notify::new()));\n for n in 0..N_TASKS {\n let mut rcv = snd.subscribe();\n let wg = wg.clone();\n let mut rng = rand::rngs::StdRng::seed_from_u64(n as u64);\n rt.spawn(async move {\n while rcv.changed().await.is_ok() {\n let _ = *rcv.borrow(); // contend on rwlock\n let r = do_work(&mut rng);\n let _ = black_box(r);\n if wg.0.fetch_sub(1, Ordering::Release) == 1 {\n wg.1.notify_one();\n }\n }\n });\n }\n\n const N_ITERS: usize = 100;\n g.bench_function(N_TASKS.to_string(), |b| {\n b.iter(|| {\n rt.block_on({\n let snd = snd.clone();\n let wg = wg.clone();\n async move {\n tokio::spawn(async move {\n for _ in 0..N_ITERS {\n assert_eq!(wg.0.fetch_add(N_TASKS as u64, Ordering::Relaxed), 0);\n let _ = snd.send(black_box(42));\n while wg.0.load(Ordering::Acquire) > 0 {\n wg.1.notified().await;\n }\n }\n })\n .await\n .unwrap();\n }\n });\n })\n });\n}\n\nfn bench_contention_resubscribe(c: &mut Criterion) {\n let mut group = c.benchmark_group(\"contention_resubscribe\");\n contention_resubscribe::<10>(&mut group);\n contention_resubscribe::<100>(&mut group);\n contention_resubscribe::<500>(&mut group);\n contention_resubscribe::<1000>(&mut group);\n group.finish();\n}\n\ncriterion_group!(contention, bench_contention_resubscribe);\n\ncriterion_main!(contention);\n" }, { "path": "benches/time_now.rs", "content": "//! Benchmark spawning a task onto the basic and threaded Tokio executors.\n//! This essentially measure the time to enqueue a task in the local and remote\n//! case.\n\nuse criterion::{black_box, criterion_group, criterion_main, Criterion};\n\nfn time_now_current_thread(c: &mut Criterion) {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap();\n\n c.bench_function(\"time_now_current_thread\", |b| {\n b.iter(|| {\n rt.block_on(async {\n black_box(tokio::time::Instant::now());\n })\n })\n });\n}\n\ncriterion_group!(time_now, time_now_current_thread);\n\ncriterion_main!(time_now);\n" }, { "path": "benches/time_timeout.rs", "content": "use std::time::{Duration, Instant};\n\nuse criterion::{black_box, criterion_group, criterion_main, Criterion};\nuse tokio::{\n runtime::Runtime,\n time::{sleep, timeout},\n};\n\n// a very quick async task, but might timeout\nasync fn quick_job() -> usize {\n 1\n}\n\nfn build_run_time(workers: usize) -> Runtime {\n if workers == 1 {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n } else {\n tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .worker_threads(workers)\n .build()\n .unwrap()\n }\n}\n\nfn single_thread_scheduler_timeout(c: &mut Criterion) {\n do_timeout_test(c, 1, \"single_thread_timeout\");\n}\n\nfn multi_thread_scheduler_timeout(c: &mut Criterion) {\n do_timeout_test(c, 8, \"multi_thread_timeout-8\");\n}\n\nfn do_timeout_test(c: &mut Criterion, workers: usize, name: &str) {\n let runtime = build_run_time(workers);\n c.bench_function(name, |b| {\n b.iter_custom(|iters| {\n let start = Instant::now();\n runtime.block_on(async {\n black_box(spawn_timeout_job(iters as usize, workers)).await;\n });\n start.elapsed()\n })\n });\n}\n\nasync fn spawn_timeout_job(iters: usize, procs: usize) {\n let mut handles = Vec::with_capacity(procs);\n for _ in 0..procs {\n handles.push(tokio::spawn(async move {\n for _ in 0..iters / procs {\n let h = timeout(Duration::from_secs(1), quick_job());\n assert_eq!(black_box(h.await.unwrap()), 1);\n }\n }));\n }\n for handle in handles {\n handle.await.unwrap();\n }\n}\n\nfn single_thread_scheduler_sleep(c: &mut Criterion) {\n do_sleep_test(c, 1, \"single_thread_sleep\");\n}\n\nfn multi_thread_scheduler_sleep(c: &mut Criterion) {\n do_sleep_test(c, 8, \"multi_thread_sleep-8\");\n}\n\nfn do_sleep_test(c: &mut Criterion, workers: usize, name: &str) {\n let runtime = build_run_time(workers);\n\n c.bench_function(name, |b| {\n b.iter_custom(|iters| {\n let start = Instant::now();\n runtime.block_on(async {\n black_box(spawn_sleep_job(iters as usize, workers)).await;\n });\n start.elapsed()\n })\n });\n}\n\nasync fn spawn_sleep_job(iters: usize, procs: usize) {\n let mut handles = Vec::with_capacity(procs);\n for _ in 0..procs {\n handles.push(tokio::spawn(async move {\n for _ in 0..iters / procs {\n let _h = black_box(sleep(Duration::from_secs(1)));\n }\n }));\n }\n for handle in handles {\n handle.await.unwrap();\n }\n}\n\ncriterion_group!(\n timeout_benchmark,\n single_thread_scheduler_timeout,\n multi_thread_scheduler_timeout,\n single_thread_scheduler_sleep,\n multi_thread_scheduler_sleep\n);\n\ncriterion_main!(timeout_benchmark);\n" }, { "path": "deny.toml", "content": "# https://embarkstudios.github.io/cargo-deny/cli/init.html\n\n[graph]\nall-features = true\n\n[licenses]\nallow = [\n \"MIT\",\n \"Apache-2.0\",\n]\nexceptions = [\n { allow = [\"Unicode-3.0\"], crate = \"unicode-ident\" },\n]\n\n[bans]\nmultiple-versions = \"allow\"\nwildcards = \"deny\"\n\n[sources]\nunknown-registry = \"deny\"\nunknown-git = \"deny\"\n" }, { "path": "docs/contributing/README.md", "content": "# Contributing\n\nThis guide will help you get started. **Do not let this guide intimidate you**.\nIt should be considered a map to help you navigate the process.\n\n## Quick start\n\nIf you are unsure where to begin, use the following guides:\n\n- Want to report or triage a bug? Start with [Contributing in Issues](contributing-in-issues.md).\n- Looking for something to work on? Filter issues by [`E-help-wanted`](https://github.com/tokio-rs/tokio/labels/E-help-wanted).\n- Planning to submit a PR? Read [Pull Requests](pull-requests.md) for the full workflow and required checks.\n- Want to understand what the labels on issues mean? See [Keeping track of issues and PRs](keeping-track-of-issues-and-prs.md).\n- Interested in code review? See [Reviewing Pull Requests](reviewing-pull-requests.md).\n\n## Table of Contents\n\n- [Contributing in Issues](contributing-in-issues.md)\n - [Asking for General Help](contributing-in-issues.md#asking-for-general-help)\n - [Submitting a Bug Report](contributing-in-issues.md#submitting-a-bug-report)\n - [Triaging a Bug Report](contributing-in-issues.md#triaging-a-bug-report)\n - [Resolving a Bug Report](contributing-in-issues.md#resolving-a-bug-report)\n- [Pull Requests](pull-requests.md)\n - [Cargo Commands](pull-requests.md#cargo-commands)\n - [Performing spellcheck on tokio codebase](pull-requests.md#performing-spellcheck-on-tokio-codebase)\n - [Tests](pull-requests.md#tests)\n - [Integration tests](pull-requests.md#integration-tests)\n - [Fuzz tests](pull-requests.md#fuzz-tests)\n - [Documentation tests](pull-requests.md#documentation-tests)\n - [Benchmarks](pull-requests.md#benchmarks)\n - [Commits](pull-requests.md#commits)\n - [Commit message guidelines](pull-requests.md#commit-message-guidelines)\n - [Opening the Pull Request](pull-requests.md#opening-the-pull-request)\n - [Discuss and update](pull-requests.md#discuss-and-update)\n - [Commit Squashing](pull-requests.md#commit-squashing)\n- [Reviewing Pull Requests](reviewing-pull-requests.md)\n - [Review a bit at a time](reviewing-pull-requests.md#review-a-bit-at-a-time)\n - [Be aware of the person behind the code](reviewing-pull-requests.md#be-aware-of-the-person-behind-the-code)\n - [Abandoned or Stalled Pull Requests](reviewing-pull-requests.md#abandoned-or-stalled-pull-requests)\n- [How to specify crates dependencies versions](how-to-specify-crates-dependencies-versions.md)\n- [Keeping track of issues and PRs](keeping-track-of-issues-and-prs.md)\n - [Area](keeping-track-of-issues-and-prs.md#area)\n - [Category](keeping-track-of-issues-and-prs.md#category)\n - [Calls for participation](keeping-track-of-issues-and-prs.md#calls-for-participation)\n - [Module](keeping-track-of-issues-and-prs.md#module)\n - [Topic](keeping-track-of-issues-and-prs.md#topic)\n" }, { "path": "docs/contributing/contributing-in-issues.md", "content": "## Contributing in Issues\n\nFor any issue, there are fundamentally three ways an individual can contribute:\n\n1. By opening the issue for discussion: For instance, if you believe that you\n have discovered a bug in Tokio, creating a new issue in [the tokio-rs/tokio\n issue tracker][issue] is the way to report it.\n\n2. By helping to triage the issue: This can be done by providing\n supporting details (a test case that demonstrates a bug), providing\n suggestions on how to address the issue, or ensuring that the issue is tagged\n correctly.\n\n3. By helping to resolve the issue: Typically this is done either in the form of\n demonstrating that the issue reported is not a problem after all, or more\n often, by opening a Pull Request that changes some bit of something in\n Tokio in a concrete and reviewable manner.\n\n[issue]: https://github.com/tokio-rs/tokio/issues\n\n**Anybody can participate in any stage of contribution**. We urge you to\nparticipate in the discussion around bugs and participate in reviewing PRs.\n\n### Asking for General Help\n\nIf you have reviewed existing documentation and still have questions or are\nhaving problems, you can [open a discussion] asking for help.\n\nIn exchange for receiving help, we ask that you contribute back a documentation\nPR that helps others avoid the problems that you encountered.\n\n[open a discussion]: https://github.com/tokio-rs/tokio/discussions/new/choose\n\n### Submitting a Bug Report\n\nWhen opening a new issue in the Tokio issue tracker, you will be presented\nwith a basic template that should be filled in. If you believe that you have\nuncovered a bug, please fill out this form, following the template to the best\nof your ability. Do not worry if you cannot answer every detail, just fill in\nwhat you can.\n\nThe two most important pieces of information we need in order to properly\nevaluate the report is a description of the behavior you are seeing and a simple\ntest case we can use to recreate the problem on our own. If we cannot recreate\nthe issue, it becomes impossible for us to fix.\n\nIn order to rule out the possibility of bugs introduced by userland code, test\ncases should be limited, as much as possible, to using only Tokio APIs.\n\nSee [How to create a Minimal, Complete, and Verifiable example][mcve].\n\n[mcve]: https://stackoverflow.com/help/mcve\n\n### Triaging a Bug Report\n\nOnce an issue has been opened, it is not uncommon for there to be discussion\naround it. Some contributors may have differing opinions about the issue,\nincluding whether the behavior being seen is a bug or a feature. This discussion\nis part of the process and should be kept focused, helpful, and professional.\n\nShort, clipped responses—that provide neither additional context nor supporting\ndetail—are not helpful or professional. To many, such responses are simply\nannoying and unfriendly.\n\nContributors are encouraged to help one another make forward progress as much as\npossible, empowering one another to solve issues collaboratively. If you choose\nto comment on an issue that you feel either is not a problem that needs to be\nfixed, or if you encounter information in an issue that you feel is incorrect,\nexplain why you feel that way with additional supporting context, and be willing\nto be convinced that you may be wrong. By doing so, we can often reach the\ncorrect outcome much faster.\n\n### Resolving a Bug Report\n\nIn the majority of cases, issues are resolved by opening a Pull Request. The\nprocess for opening and reviewing a Pull Request is similar to that of opening\nand triaging issues, but carries with it a necessary review and approval\nworkflow that ensures that the proposed changes meet the minimal quality and\nfunctional guidelines of the Tokio project.\n" }, { "path": "docs/contributing/how-to-specify-crates-dependencies-versions.md", "content": "# How to specify crates dependencies versions\n\nEach crate (e.g., `tokio-util`, `tokio-stream`, etc.) should specify dependencies\naccording to these rules:\n\n1. The listed version should be the oldest version that the crate works with\n(e.g., if `tokio-util` works with `tokio` version `1.44` but not `1.43`, then\n`tokio-util` should specify version `1.44` for its `tokio` dependency).\nWe don't require users to use the latest version unnecessarily.\n2. When a crate starts using a newer feature in a dependency, the version\nshould be bumped to the version that introduced it.\n3. If a crate depends on an unreleased feature in a dependency, it may use\n`path = ` dependency to specify this. Since path dependencies must be removed\nduring the release of the crate, this ensures that it can't be released until\nthe dependency has a new version.\n\nConsider the following example from `tokio-stream`:\n\n```toml\n[dependencies]\nfutures-core = { version = \"0.3.0\" }\npin-project-lite = \"0.2.11\"\ntokio = { path = \"../tokio\", features = [\"sync\"] }\n```\n\nIn this case, local development of `tokio-stream` uses the local version\nof `tokio` via the `path` dependency. This means that it's currently not\npossible to release `tokio-stream`. Once a new version of `tokio` is\nreleased, the path dependency will be removed from `tokio-stream`.\nAs mentioned before, this version should only be bumped when adding a new\nfeature in the crate that relies on a newer version.\n" }, { "path": "docs/contributing/keeping-track-of-issues-and-prs.md", "content": "## Keeping track of issues and PRs\n\nThe Tokio GitHub repository has a lot of issues and PRs to keep track of. This\nsection explains the meaning of various labels, as well as our [GitHub\nproject][project]. The section is primarily targeted at maintainers. Most\ncontributors aren't able to set these labels.\n\n### Area\n\nThe area label describes the crates relevant to this issue or PR.\n\n- **A-ci** This issue concerns our GitHub Actions setup.\n- **A-tokio** This issue concerns the main Tokio crate.\n- **A-readme** This issue is related to documentation such as README.md.\n- **A-benches** This issue concerns the benchmarks.\n- **A-examples** This issue concerns the examples.\n- **A-tokio-test** The issue concerns the `tokio-test` crate.\n- **A-tokio-util** This issue concerns the `tokio-util` crate.\n- **A-tokio-macros** This issue concerns the `tokio-macros` crate. Should only\n be used for the procedural macros, and not `join!` or `select!`.\n- **A-tokio-stream** This issue concerns the `tokio-stream` crate.\n\n### Category\n\n- **C-bug** This is a bug-report. Bug-fix PRs use `C-enhancement` instead.\n- **C-enhancement** This is a PR that adds a new features.\n- **C-maintenance** This is an issue or PR about stuff such as documentation,\n GitHub Actions or code quality.\n- **C-feature-request** This is a feature request. Implementations of feature\n requests use `C-enhancement` instead.\n- **C-feature-accepted** If you submit a PR for this feature request, we won't\n close it with the reason \"we don't want this\". Issues with this label should\n also have the `C-feature-request` label.\n- **C-musing** Stuff like tracking issues or roadmaps. \"musings about a better\n world\"\n- **C-proposal** A proposal of some kind, and a request for comments.\n- **C-question** A user question. Large overlap with GitHub discussions.\n- **C-request** A non-feature request, e.g. \"please add deprecation notices to\n `-alpha.*` versions of crates\"\n\n### Calls for participation\n\n- **E-help-wanted** Stuff where we want help. Often seen together with `C-bug`\n or `C-feature-accepted`.\n- **E-easy** This is easy, ranging from quick documentation fixes to stuff you\n can do after reading the tutorial on our website.\n- **E-medium** This is not `E-easy` or `E-hard`.\n- **E-hard** This either involves very tricky code, is something we don't know\n how to solve, or is challenging for some other reason.\n- **E-needs-mvce** This bug is missing a minimal complete and verifiable\n example.\n\nThe \"E-\" prefix is the same as used in the Rust compiler repository. Some\nissues are missing a difficulty rating, but feel free to ask on our Discord\nserver if you want to know how challenging an issue likely is.\n\n### Module\n\nThe module label provides a more fine grained categorization than **Area**.\n\n- **M-blocking** Things relevant to `spawn_blocking`, `block_in_place`.\n- **M-codec** The `tokio_util::codec` module.\n- **M-compat** The `tokio_util::compat` module.\n- **M-coop** Things relevant to coop.\n- **M-fs** The `tokio::fs` module.\n- **M-io** The `tokio::io` module.\n- **M-macros** Issues about any kind of macro.\n- **M-metrics** Things relevant to `tokio::runtime::metrics`.\n- **M-net** The `tokio::net` module.\n- **M-process** The `tokio::process` module.\n- **M-runtime** The `tokio::runtime` module.\n- **M-signal** The `tokio::signal` module.\n- **M-sync** The `tokio::sync` module.\n- **M-task** The `tokio::task` module.\n- **M-time** The `tokio::time` module.\n- **M-tracing** Tracing support in Tokio.\n- **M-taskdump** Things relevant to taskdump.\n\n### Topic\n\nSome extra information.\n\n- **T-docs** This is about documentation.\n- **T-performance** This is about performance.\n- **T-v0.1.x** This is about old Tokio.\n\nAny label not listed here is not in active use.\n\n[project]: https://github.com/orgs/tokio-rs/projects/1\n" }, { "path": "docs/contributing/pull-requests.md", "content": "## Pull Requests\n\nPull Requests are the way concrete changes are made to the code, documentation,\nand dependencies in the Tokio repository.\n\nEven tiny pull requests (e.g., one-character pull request fixing a typo in API\ndocumentation) are greatly appreciated. Before making a large change, it is\nusually a good idea to first open an issue describing the change to solicit\nfeedback and guidance. This will increase the likelihood of the PR getting\nmerged.\n\n### Cargo Commands\n\nDue to the extensive use of features in Tokio, you will often need to add extra\narguments to many common cargo commands. This section lists some commonly needed\ncommands.\n\nSome commands just need the `--all-features` argument:\n\n```\ncargo build --all-features\ncargo check --all-features\ncargo test --all-features\n```\n\n**NOTE**: there are some features that are not supported in every system, so you might\nneed to specify which features you want to pass to cargo (e.g., `cargo check --features=full,io-uring`)\n\nIdeally, you should use the same version of clippy as the one used in CI\n(defined by `env.rust_clippy` in [ci.yml][ci.yml]), because newer versions\nmight have new lints:\n\n[ci.yml]: ../../.github/workflows/ci.yml\n\n\n\n```\ncargo +1.88 clippy --all --tests --all-features\n```\n\nWhen building documentation, a simple `cargo doc` is not sufficient. To produce\ndocumentation equivalent to what will be produced in docs.rs's builds of Tokio's\ndocs, please use:\n\n```\nRUSTDOCFLAGS=\"--cfg docsrs --cfg tokio_unstable\" RUSTFLAGS=\"--cfg docsrs --cfg tokio_unstable\" cargo +nightly doc --all-features [--open]\n```\n\nThis turns on indicators to display the Cargo features required for\nconditionally compiled APIs in Tokio, and it enables documentation of unstable\nTokio features. Notice that it is necessary to pass cfg flags to both RustDoc\n*and* rustc.\n\nThere is a more concise way to build docs.rs-equivalent docs by using [`cargo\ndocs-rs`], which reads the above documentation flags out of Tokio's Cargo.toml\nas docs.rs itself does.\n\n[`cargo docs-rs`]: https://github.com/dtolnay/cargo-docs-rs\n\n```\ncargo install --locked cargo-docs-rs\ncargo +nightly docs-rs [--open]\n```\n\nThe `cargo fmt` command does not work on the Tokio codebase. You can use the\ncommand below instead:\n\n```\n# Mac or Linux\nrustfmt --check --edition 2021 $(git ls-files '*.rs')\n\n# Powershell\nGet-ChildItem . -Filter \"*.rs\" -Recurse | foreach { rustfmt --check --edition 2021 $_.FullName }\n```\nThe `--check` argument prints the things that need to be fixed. If you remove\nit, `rustfmt` will update your files locally instead.\n\nYou can run loom tests with\n```\ncd tokio # tokio crate in workspace\nLOOM_MAX_PREEMPTIONS=1 LOOM_MAX_BRANCHES=10000 RUSTFLAGS=\"--cfg loom -C debug_assertions\" \\\n cargo test --lib --release --features full -- --test-threads=1 --nocapture\n```\nAdditionally, you can also add `--cfg tokio_unstable` to the `RUSTFLAGS` environment variable to\nrun loom tests that test unstable features.\n\nYou can run miri tests with\n```\nMIRIFLAGS=\"-Zmiri-disable-isolation -Zmiri-strict-provenance\" \\\n cargo +nightly miri test --features full --lib --tests\n```\n\n### Performing spellcheck on tokio codebase\n\nYou can perform a spell-check on the Tokio codebase. For details of how to use the spellcheck tool, feel free to visit\nhttps://github.com/drahnr/cargo-spellcheck\n```\n# First install the spell-check plugin\ncargo install --locked cargo-spellcheck\n\n# Then run the cargo spell check command\ncargo spellcheck check\n```\n\nIf the command rejects a word, you should backtick the rejected word if it's code related. If not, the\nrejected word should be put into `spellcheck.dic` file.\n\nNote that when you add a word into the file, you should also update the first line which tells the spellcheck tool\nthe total number of words included in the file\n\n### Tests\n\nIf the change being proposed alters code (as opposed to only documentation for\nexample), it is either adding new functionality to Tokio or it is fixing\nexisting, broken functionality. In both of these cases, the pull request should\ninclude one or more tests to ensure that Tokio does not regress in the future.\nThere are two ways to write tests: [integration tests][integration-tests]\nand [documentation tests][documentation-tests].\n(Tokio avoids [unit tests][unit-tests] as much as possible).\n\nTokio uses [conditional compilation attributes][conditional-compilation]\nthroughout the codebase, to modify rustc's behavior. Code marked with such\nattributes can be enabled using `RUSTFLAGS` and `RUSTDOCFLAGS` environment\nvariables. One of the most prevalent flags passed in these variables is\nthe `--cfg` option. To run tests in a particular file, check first what\noptions #![cfg] declaration defines for that file.\n\nFor instance, to run a test marked with the 'tokio_unstable' cfg option,\nyou must pass this flag to the compiler when running the test.\n```\n$ RUSTFLAGS=\"--cfg tokio_unstable\" cargo test -p tokio --all-features --test rt_metrics\n```\n\n#### Integration tests\n\nIntegration tests go in the same crate as the code they are testing. Each sub\ncrate should have a `dev-dependency` on `tokio` itself. This makes all Tokio\nutilities available to use in tests, no matter the crate being tested.\n\nThe best strategy for writing a new integration test is to look at existing\nintegration tests in the crate and follow the style.\n\n#### Fuzz tests\n\nSome of our crates include a set of fuzz tests, this will be marked by a\ndirectory `fuzz`. It is a good idea to run fuzz tests after each change.\nTo get started with fuzz testing you'll need to install\n[cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz).\n\n`cargo install --locked cargo-fuzz`\n\nTo list the available fuzzing harnesses you can run;\n\n```bash\n$ cd tokio\n$ cargo fuzz list\nfuzz_linked_list\n```\n\nRunning a fuzz test is as simple as;\n\n`cargo fuzz run fuzz_linked_list`\n\n**NOTE**: Keep in mind that by default when running a fuzz test the fuzz\nharness will run forever and will only exit if you `ctrl-c` or it finds\na bug.\n\n#### Documentation tests\n\nIdeally, every API has at least one [documentation test] that demonstrates how to\nuse the API. Documentation tests are run with `cargo test --doc`. This ensures\nthat the example is correct and provides additional test coverage.\n\nThe trick to documentation tests is striking a balance between being succinct\nfor a reader to understand and actually testing the API.\n\nSame as with integration tests, when writing a documentation test, the full\n`tokio` crate is available. This is especially useful for getting access to the\nruntime to run the example.\n\nThe documentation tests will be visible from both the crate-specific\ndocumentation **and** the `tokio` facade documentation via the re-export. The\nexample should be written from the point of view of a user that is using the\n`tokio` crate. As such, the example should use the API via the facade and not by\ndirectly referencing the crate.\n\nThe type level example for `tokio::time::timeout` provides a good example of a\ndocumentation test:\n\n```\n/// Create a new `Timeout` set to expire in 10 milliseconds.\n///\n/// ```rust\n/// use tokio::time::timeout;\n/// use tokio::sync::oneshot;\n///\n/// use std::time::Duration;\n///\n/// # async fn dox() {\n/// let (tx, rx) = oneshot::channel();\n/// # tx.send(()).unwrap();\n///\n/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.\n/// if let Err(_) = timeout(Duration::from_millis(10), rx).await {\n/// println!(\"did not receive value within 10 ms\");\n/// }\n/// # }\n/// ```\n```\n\nLines that start with `/// #` are removed when the documentation is generated.\n\n### Benchmarks\n\nYou can run benchmarks locally for the changes you've made to the tokio codebase.\nTokio currently uses [Criterion](https://github.com/bheisler/criterion.rs) as its benchmarking tool. To run a benchmark\nagainst the changes you have made, for example, you can run;\n\n```bash\ncd benches\n\n# Run all benchmarks.\ncargo bench\n\n# Run all tests in the `benches/fs.rs` file\ncargo bench --bench fs\n\n# Run the `async_read_buf` benchmark in `benches/fs.rs` specifically.\ncargo bench async_read_buf\n\n# After running benches, you can check the statistics under `tokio/target/criterion/`\n```\n\nYou can also refer to [Criterion] docs for additional options and details.\n\n[Criterion]: https://docs.rs/criterion/latest/criterion/\n\n### Commits\n\nIt is a recommended best practice to keep your changes as logically grouped as\npossible within individual commits. There is no limit to the number of commits\nany single Pull Request may have, and many contributors find it easier to review\nchanges that are split across multiple commits.\n\nThat said, if you have a number of commits that are \"checkpoints\" and don't\nrepresent a single logical change, please squash those together.\n\nNote that multiple commits often get squashed when they are landed (see the\nnotes about [commit squashing](#commit-squashing)).\n\n#### Commit message guidelines\n\nA good commit message should describe what changed and why.\n\n1. The first line should:\n\n* contain a short description of the change (preferably 50 characters or less,\n and no more than 72 characters)\n* be entirely in lowercase with the exception of proper nouns, acronyms, and\n the words that refer to code, like function/variable names\n* start with an imperative verb\n* not have a period at the end\n* be prefixed with the name of the module being changed; usually this is the\n same as the M-* label on the PR\n\nExamples:\n\n* time: introduce `Timeout` and deprecate `Deadline`\n* codec: export `Encoder`, `Decoder`, `Framed*`\n* ci: fix the FreeBSD ci configuration\n\n2. Keep the second line blank.\n3. Wrap all other lines at 72 columns (except for long URLs).\n4. If your patch fixes an open issue, you can add a reference to it at the end\n of the log. Use the `Fixes: #` prefix and the issue number. For other\n references use `Refs: #`. `Refs` may include multiple issues, separated by a\n comma.\n\n Examples:\n\n - `Fixes: #1337`\n - `Refs: #1234`\n\nSample complete commit message:\n\n```txt\nmodule: explain the commit in one line\n\nBody of commit message is a few lines of text, explaining things\nin more detail, possibly giving some background about the issue\nbeing fixed, etc.\n\nThe body of the commit message can be several paragraphs, and\nplease do proper word-wrap and keep columns shorter than about\n72 characters or so. That way, `git log` will show things\nnicely even when it is indented.\n\nFixes: #1337\nRefs: #453, #154\n```\n\n### Opening the Pull Request\n\nFrom within GitHub, opening a new Pull Request will present you with a\n[template] that should be filled out. Please try to do your best at filling out\nthe details, but feel free to skip parts if you're not sure what to put.\n\n[template]: ../../.github/PULL_REQUEST_TEMPLATE.md\n\n### Discuss and update\n\nYou will probably get feedback or requests for changes to your Pull Request.\nThis is a big part of the submission process so don't be discouraged! Some\ncontributors may sign off on the Pull Request right away, others may have\nmore detailed comments or feedback. This is a necessary part of the process\nin order to evaluate whether the changes are correct and necessary.\n\n**Any community member can review a PR and you might get conflicting feedback**.\nKeep an eye out for comments from code owners to provide guidance on conflicting\nfeedback.\n\n**Once the PR is open, do not rebase the commits**. See [Commit Squashing](#commit-squashing) for\nmore details.\n\n### Commit Squashing\n\nIn most cases, **do not squash commits that you add to your Pull Request during\nthe review process**. When the commits in your Pull Request land, they may be\nsquashed into one commit per logical change. Metadata will be added to the\ncommit message (including links to the Pull Request, links to relevant issues,\nand the names of the reviewers). The commit history of your Pull Request,\nhowever, will stay intact on the Pull Request page.\n\n[integration-tests]: https://doc.rust-lang.org/rust-by-example/testing/integration_testing.html\n[unit-tests]: https://doc.rust-lang.org/rust-by-example/testing/unit_testing.html\n[documentation-tests]: https://doc.rust-lang.org/rust-by-example/testing/doc_testing.html\n[conditional-compilation]: https://doc.rust-lang.org/reference/conditional-compilation.html\n" }, { "path": "docs/contributing/reviewing-pull-requests.md", "content": "## Reviewing Pull Requests\n\n**Any Tokio community member is welcome to review any pull request**.\n\nAll Tokio contributors who choose to review and provide feedback on Pull\nRequests have a responsibility to both the project and the individual making the\ncontribution. Reviews and feedback must be helpful, insightful, and geared\ntowards improving the contribution as opposed to simply blocking it. If there\nare reasons why you feel the PR should not land, explain what those are. Do not\nexpect to be able to block a Pull Request from advancing simply because you say\n\"No\" without giving an explanation. Be open to having your mind changed. Be open\nto working with the contributor to make the Pull Request better.\n\nReviews that are dismissive or disrespectful of the contributor or any other\nreviewers are strictly counter to the Code of Conduct.\n\nWhen reviewing a Pull Request, the primary goals are for the codebase to improve\nand for the person submitting the request to succeed. **Even if a Pull Request\ndoes not land, the submitters should come away from the experience feeling like\ntheir effort was not wasted or unappreciated**. Every Pull Request from a new\ncontributor is an opportunity to grow the community.\n\n### Review a bit at a time\n\nDo not overwhelm new contributors.\n\nIt is tempting to micro-optimize and make everything about relative performance,\nperfect grammar, or exact style matches. Do not succumb to that temptation.\n\nFocus first on the most significant aspects of the change:\n\n1. Does this change make sense for Tokio?\n2. Does this change make Tokio better, even if only incrementally?\n3. Are there clear bugs or larger scale issues that need attending to?\n4. Is the commit message readable and correct? If it contains a breaking change\n is it clear enough?\n\nNote that only **incremental** improvement is needed to land a PR. This means\nthat the PR does not need to be perfect, only better than the status quo. Follow\nup PRs may be opened to continue iterating.\n\nWhen changes are necessary, *request* them, do not *demand* them, and **do not\nassume that the submitter already knows how to add a test or run a benchmark**.\n\nSpecific performance optimization techniques, coding styles and conventions\nchange over time. The first impression you give to a new contributor never does.\n\nNits (requests for small changes that are not essential) are fine, but try to\navoid stalling the Pull Request. Most nits can typically be fixed by the Tokio\nCollaborator landing the Pull Request but they can also be an opportunity for\nthe contributor to learn a bit more about the project.\n\nIt is always good to clearly indicate nits when you comment: e.g.\n`Nit: change foo() to bar(). But this is not blocking.`\n\nIf your comments were addressed but were not folded automatically after new\ncommits or if they proved to be mistaken, please, [hide them][hiding-a-comment]\nwith the appropriate reason to keep the conversation flow concise and relevant.\n\n### Be aware of the person behind the code\n\nBe aware that *how* you communicate requests and reviews in your feedback can\nhave a significant impact on the success of the Pull Request. Yes, we may land\na particular change that makes Tokio better, but the individual might just not\nwant to have anything to do with Tokio ever again. The goal is not just having\ngood code.\n\n### Abandoned or Stalled Pull Requests\n\nIf a Pull Request appears to be abandoned or stalled, it is polite to first\ncheck with the contributor to see if they intend to continue the work before\nchecking if they would mind if you took it over (especially if it just has nits\nleft). When doing so, it is courteous to give the original contributor credit\nfor the work they started (either by preserving their name and email address in\nthe commit log, or by using an `Author: ` meta-data tag in the commit.\n\n_Adapted from the [Node.js contributing guide][node]_.\n\n[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md\n[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment\n" }, { "path": "examples/Cargo.toml", "content": "[package]\nname = \"examples\"\nversion = \"0.0.0\"\npublish = false\nedition = \"2021\"\nlicense = \"MIT\"\n\n# If you copy one of the examples into a new project, you should be using\n# [dependencies] instead, and delete the **path**.\n[dev-dependencies]\ntokio = { version = \"1.0.0\", path = \"../tokio\", features = [\"full\", \"tracing\"] }\ntokio-util = { version = \"0.7.0\", path = \"../tokio-util\", features = [\"full\"] }\ntokio-stream = { version = \"0.1\", path = \"../tokio-stream\" }\n\ntracing = \"0.1\"\ntracing-subscriber = { version = \"0.3.1\", default-features = false, features = [\"fmt\", \"ansi\", \"env-filter\", \"tracing-log\"] }\nbytes = \"1.0.0\"\nfutures = { version = \"0.3.0\", features = [\"thread-pool\"]}\nhttp = \"1\"\nserde = \"1.0\"\nserde_derive = \"1.0\"\nserde_json = \"1.0\"\nhttparse = \"1.0\"\nhttpdate = \"1.0\"\nonce_cell = \"1.5.2\"\n\n[target.'cfg(all(tokio_unstable, target_os = \"linux\"))'.dev-dependencies]\ntokio = { version = \"1.0.0\", path = \"../tokio\", features = [\"full\", \"tracing\", \"taskdump\"] }\n\n[target.'cfg(windows)'.dev-dependencies.windows-sys]\nversion = \"0.61\"\n\n[[example]]\nname = \"chat\"\npath = \"chat.rs\"\n\n[[example]]\nname = \"connect-tcp\"\npath = \"connect-tcp.rs\"\n\n[[example]]\nname = \"connect-udp\"\npath = \"connect-udp.rs\"\n\n[[example]]\nname = \"echo-tcp\"\npath = \"echo-tcp.rs\"\n\n[[example]]\nname = \"echo-udp\"\npath = \"echo-udp.rs\"\n\n[[example]]\nname = \"hello_world\"\npath = \"hello_world.rs\"\n\n[[example]]\nname = \"print_each_packet\"\npath = \"print_each_packet.rs\"\n\n[[example]]\nname = \"proxy\"\npath = \"proxy.rs\"\n\n[[example]]\nname = \"tinydb\"\npath = \"tinydb.rs\"\n\n[[example]]\nname = \"udp-client\"\npath = \"udp-client.rs\"\n\n[[example]]\nname = \"udp-codec\"\npath = \"udp-codec.rs\"\n\n[[example]]\nname = \"tinyhttp\"\npath = \"tinyhttp.rs\"\n\n[[example]]\nname = \"custom-executor\"\npath = \"custom-executor.rs\"\n\n[[example]]\nname = \"custom-executor-tokio-context\"\npath = \"custom-executor-tokio-context.rs\"\n\n[[example]]\nname = \"named-pipe\"\npath = \"named-pipe.rs\"\n\n[[example]]\nname = \"named-pipe-ready\"\npath = \"named-pipe-ready.rs\"\n\n[[example]]\nname = \"named-pipe-multi-client\"\npath = \"named-pipe-multi-client.rs\"\n\n[[example]]\nname = \"dump\"\npath = \"dump.rs\"\n\n[lints]\nworkspace = true\n" }, { "path": "examples/README.md", "content": "## Examples of how to use Tokio\n\nThis directory contains a number of examples showcasing various capabilities of\nthe `tokio` crate.\n\nAll examples can be executed with:\n\n```\ncargo run --example $name\n```\n\nA good starting point for the examples would be [`hello_world`](hello_world.rs)\nand [`echo-tcp`](echo-tcp.rs). Additionally [the tokio website][tokioweb] contains\nadditional guides for some of the examples.\n\nFor a larger \"real world\" example, see the [`mini-redis`][redis] repository.\n\nIf you've got an example you'd like to see here, please feel free to open an\nissue. Otherwise if you've got an example you'd like to add, please feel free\nto make a PR!\n\n[tokioweb]: https://tokio.rs/tokio/tutorial\n[redis]: https://github.com/tokio-rs/mini-redis\n" }, { "path": "examples/chat.rs", "content": "//! A chat server that broadcasts a message to all connections.\n//!\n//! This example is explicitly more verbose than it has to be. This is to\n//! illustrate more concepts.\n//!\n//! A chat server for telnet clients. After a telnet client connects, the first\n//! line should contain the client's name. After that, all lines sent by a\n//! client are broadcasted to all other connected clients.\n//!\n//! Because the client is telnet, lines are delimited by \"\\r\\n\".\n//!\n//! You can test this out by running:\n//!\n//! cargo run --example chat\n//!\n//! And then in another terminal run:\n//!\n//! telnet localhost 6142\n//!\n//! You can run the `telnet` command in any number of additional windows.\n//!\n//! You can run the second command in multiple windows and then chat between the\n//! two, seeing the messages from the other client as they're received. For all\n//! connected clients they'll all join the same room and see everyone else's\n//! messages.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::sync::{mpsc, Mutex};\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::{Framed, LinesCodec};\n\nuse futures::SinkExt;\nuse std::collections::HashMap;\nuse std::env;\nuse std::error::Error;\nuse std::io;\nuse std::net::SocketAddr;\nuse std::sync::Arc;\n\nconst DEFAULT_ADDR: &str = \"127.0.0.1:6142\";\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};\n // Configure a `tracing` subscriber that logs traces emitted by the chat\n // server.\n tracing_subscriber::fmt()\n // Filter what traces are displayed based on the RUST_LOG environment\n // variable.\n //\n // Traces emitted by the example code will always be displayed. You\n // can set `RUST_LOG=tokio=trace` to enable additional traces emitted by\n // Tokio itself.\n .with_env_filter(EnvFilter::from_default_env().add_directive(\"chat=info\".parse()?))\n // Log events when `tracing` spans are created, entered, exited, or\n // closed. When Tokio's internal tracing support is enabled (as\n // described above), this can be used to track the lifecycle of spawned\n // tasks on the Tokio runtime.\n .with_span_events(FmtSpan::FULL)\n // Set this subscriber as the default, to collect all traces emitted by\n // the program.\n .init();\n\n // Create the shared state. This is how all the peers communicate.\n //\n // The server task will hold a handle to this. For every new client, the\n // `state` handle is cloned and passed into the task that processes the\n // client connection.\n let state = Arc::new(Mutex::new(Shared::new()));\n\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| DEFAULT_ADDR.to_string());\n\n // Bind a TCP listener to the socket address.\n //\n // Note that this is the Tokio TcpListener, which is fully async.\n let listener = TcpListener::bind(&addr).await?;\n\n tracing::info!(\"server running on {addr}\");\n\n loop {\n // Asynchronously wait for an inbound TcpStream.\n let (stream, addr) = listener.accept().await?;\n\n // Clone a handle to the `Shared` state for the new connection.\n let state = Arc::clone(&state);\n\n // Spawn our handler to be run asynchronously.\n tokio::spawn(async move {\n tracing::debug!(\"accepted connection from {addr}\");\n if let Err(e) = process(state, stream, addr).await {\n tracing::warn!(\"Connection from {addr} failed: {e:?}\");\n }\n });\n }\n}\n\n/// Shorthand for the transmit half of the message channel.\ntype Tx = mpsc::UnboundedSender;\n\n/// Shorthand for the receive half of the message channel.\ntype Rx = mpsc::UnboundedReceiver;\n\n/// Data that is shared between all peers in the chat server.\n///\n/// This is the set of `Tx` handles for all connected clients. Whenever a\n/// message is received from a client, it is broadcasted to all peers by\n/// iterating over the `peers` entries and sending a copy of the message on each\n/// `Tx`.\nstruct Shared {\n peers: HashMap,\n}\n\n/// The state for each connected client.\nstruct Peer {\n /// The TCP socket wrapped with the `Lines` codec, defined below.\n ///\n /// This handles sending and receiving data on the socket. When using\n /// `Lines`, we can work at the line level instead of having to manage the\n /// raw byte operations.\n lines: Framed,\n\n /// Receive half of the message channel.\n ///\n /// This is used to receive messages from peers. When a message is received\n /// off of this `Rx`, it will be written to the socket.\n rx: Rx,\n}\n\nimpl Shared {\n /// Create a new, empty, instance of `Shared`.\n fn new() -> Self {\n Shared {\n peers: HashMap::new(),\n }\n }\n\n /// Send a `LineCodec` encoded message to every peer, except\n /// for the sender.\n ///\n /// This function also cleans up disconnected peers automatically.\n async fn broadcast(&mut self, sender: SocketAddr, message: &str) {\n let mut failed_peers = Vec::new();\n let message = message.to_string(); // Clone once for all sends\n\n for (addr, tx) in self.peers.iter() {\n if *addr != sender && tx.send(message.clone()).is_err() {\n // Receiver has been dropped, mark for removal\n failed_peers.push(*addr);\n }\n }\n\n // Clean up disconnected peers\n for addr in failed_peers {\n self.peers.remove(&addr);\n tracing::debug!(\"Removed disconnected peer: {addr}\");\n }\n }\n}\n\nimpl Peer {\n /// Create a new instance of `Peer`.\n async fn new(\n state: Arc>,\n lines: Framed,\n ) -> io::Result {\n // Get the client socket address\n let addr = lines.get_ref().peer_addr()?;\n\n // Create a channel for this peer\n let (tx, rx) = mpsc::unbounded_channel();\n\n // Add an entry for this `Peer` in the shared state map.\n state.lock().await.peers.insert(addr, tx);\n\n Ok(Peer { lines, rx })\n }\n}\n\n/// Process an individual chat client\nasync fn process(\n state: Arc>,\n stream: TcpStream,\n addr: SocketAddr,\n) -> Result<(), Box> {\n let mut lines = Framed::new(stream, LinesCodec::new());\n\n // Send a prompt to the client to enter their username.\n lines.send(\"Please enter your username:\").await?;\n\n // Read the first line from the `LineCodec` stream to get the username.\n let Some(Ok(username)) = lines.next().await else {\n // We didn't get a line so we return early here.\n tracing::error!(\"Failed to get username from {addr}. Client disconnected.\");\n return Ok(());\n };\n\n // Register our peer with state which internally sets up some channels.\n let mut peer = Peer::new(state.clone(), lines).await?;\n\n // A client has connected, let's let everyone know.\n {\n let mut state = state.lock().await;\n let msg = format!(\"{username} has joined the chat\");\n tracing::info!(\"{msg}\");\n state.broadcast(addr, &msg).await;\n }\n\n // Process incoming messages until our stream is exhausted by a disconnect.\n loop {\n tokio::select! {\n // A message was received from a peer. Send it to the current user.\n Some(msg) = peer.rx.recv() => {\n if let Err(e) = peer.lines.send(&msg).await {\n tracing::error!(\"Failed to send message to {username}: {e:?}\");\n break;\n }\n }\n result = peer.lines.next() => match result {\n // A message was received from the current user, we should\n // broadcast this message to the other users.\n Some(Ok(msg)) => {\n let mut state = state.lock().await;\n let msg = format!(\"{username}: {msg}\");\n\n state.broadcast(addr, &msg).await;\n }\n // An error occurred.\n Some(Err(e)) => {\n tracing::error!(\n \"an error occurred while processing messages for {username}; error = {e:?}\"\n );\n break;\n }\n // The stream has been exhausted.\n None => break,\n },\n }\n }\n\n // If this section is reached it means that the client was disconnected!\n // Let's let everyone still connected know about it.\n {\n let mut state = state.lock().await;\n state.peers.remove(&addr);\n\n let msg = format!(\"{username} has left the chat\");\n tracing::info!(\"{msg}\");\n state.broadcast(addr, &msg).await;\n }\n\n Ok(())\n}\n" }, { "path": "examples/connect-tcp.rs", "content": "//! An example of hooking up stdin/stdout to a TCP stream.\n//!\n//! This example will connect to a socket address specified in the argument list\n//! and then forward all data read on stdin to the server, printing out all data\n//! received on stdout. Each line entered on stdin will be translated to a TCP\n//! packet which is then sent to the remote address.\n//!\n//! Note that this is not currently optimized for performance, especially\n//! around buffer management. Rather it's intended to show an example of\n//! working with a client.\n//!\n//! This example can be quite useful when interacting with the other examples in\n//! this repository! Many of them recommend running this as a simple \"hook up\n//! stdin/stdout to a server\" to get up and running.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::io::{stdin, stdout};\nuse tokio::net::TcpStream;\nuse tokio_util::codec::{BytesCodec, FramedRead, FramedWrite};\n\nuse bytes::Bytes;\nuse futures::{future, Sink, SinkExt, Stream, StreamExt};\nuse std::env;\nuse std::error::Error;\nuse std::net::SocketAddr;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Parse what address we're going to connect to\n let args = env::args().skip(1).collect::>();\n let addr = args\n .first()\n .ok_or(\"this program requires at least one argument\")?;\n let addr = addr.parse::()?;\n\n let stdin = FramedRead::new(stdin(), BytesCodec::new());\n let stdin = stdin.map(|i| i.map(|bytes| bytes.freeze()));\n let stdout = FramedWrite::new(stdout(), BytesCodec::new());\n\n connect(&addr, stdin, stdout).await?;\n\n Ok(())\n}\n\npub async fn connect(\n addr: &SocketAddr,\n mut stdin: impl Stream> + Unpin,\n mut stdout: impl Sink + Unpin,\n) -> Result<(), Box> {\n let mut stream = TcpStream::connect(addr).await?;\n let (r, w) = stream.split();\n let mut sink = FramedWrite::new(w, BytesCodec::new());\n // filter map Result stream into just a Bytes stream to match stdout Sink\n // on the event of an Error, log the error and end the stream\n let mut stream = FramedRead::new(r, BytesCodec::new())\n .filter_map(|i| match i {\n //BytesMut into Bytes\n Ok(i) => future::ready(Some(i.freeze())),\n Err(e) => {\n eprintln!(\"failed to read from socket; error={e}\");\n future::ready(None)\n }\n })\n .map(Ok);\n\n tokio::select! {\n r = sink.send_all(&mut stdin) => r?,\n r = stdout.send_all(&mut stream) => r?,\n }\n\n Ok(())\n}\n" }, { "path": "examples/connect-udp.rs", "content": "//! An example of hooking up stdin/stdout to a UDP stream.\n//!\n//! This example will connect to a socket address specified in the argument list\n//! and then forward all data read on stdin to the server, printing out all data\n//! received on stdout. Each line entered on stdin will be translated to a UDP\n//! packet which is then sent to the remote address.\n//!\n//! Note that this is not currently optimized for performance, especially\n//! around buffer management. Rather it's intended to show an example of\n//! working with a client.\n//!\n//! This example can be quite useful when interacting with the other examples in\n//! this repository! Many of them recommend running this as a simple \"hook up\n//! stdin/stdout to a server\" to get up and running.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::io::{stdin, stdout};\nuse tokio::net::UdpSocket;\nuse tokio_util::codec::{BytesCodec, FramedRead, FramedWrite};\n\nuse bytes::Bytes;\nuse futures::{Sink, SinkExt, Stream, StreamExt};\nuse std::env;\nuse std::error::Error;\nuse std::net::SocketAddr;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Parse what address we're going to connect to\n let args = env::args().skip(1).collect::>();\n let addr = args\n .first()\n .ok_or(\"this program requires at least one argument\")?;\n let addr = addr.parse::()?;\n\n let stdin = FramedRead::new(stdin(), BytesCodec::new());\n let stdin = stdin.map(|i| i.map(|bytes| bytes.freeze()));\n let stdout = FramedWrite::new(stdout(), BytesCodec::new());\n\n connect(&addr, stdin, stdout).await?;\n\n Ok(())\n}\n\npub async fn connect(\n addr: &SocketAddr,\n stdin: impl Stream> + Unpin,\n stdout: impl Sink + Unpin,\n) -> Result<(), Box> {\n // We'll bind our UDP socket to a local IP/port, but for now we\n // basically let the OS pick both of those.\n let bind_addr = if addr.ip().is_ipv4() {\n \"0.0.0.0:0\"\n } else {\n \"[::]:0\"\n };\n\n let socket = UdpSocket::bind(&bind_addr).await?;\n socket.connect(addr).await?;\n\n tokio::select! {\n r = send(stdin, &socket) => r?,\n r = recv(stdout, &socket) => r?,\n }\n\n Ok(())\n}\n\nasync fn send(\n mut stdin: impl Stream> + Unpin,\n writer: &UdpSocket,\n) -> Result<(), std::io::Error> {\n while let Some(item) = stdin.next().await {\n let buf = item?;\n writer.send(&buf[..]).await?;\n }\n\n Ok(())\n}\n\nasync fn recv(\n mut stdout: impl Sink + Unpin,\n reader: &UdpSocket,\n) -> Result<(), std::io::Error> {\n loop {\n let mut buf = vec![0; 1024];\n let n = reader.recv(&mut buf[..]).await?;\n\n if n > 0 {\n stdout.send(Bytes::copy_from_slice(&buf[..n])).await?;\n }\n }\n}\n" }, { "path": "examples/custom-executor-tokio-context.rs", "content": "// This example shows how to use the tokio runtime with any other executor\n//\n//It takes advantage from RuntimeExt which provides the extension to customize your\n//runtime.\n\nuse tokio::net::TcpListener;\nuse tokio::runtime::Builder;\nuse tokio::sync::oneshot;\nuse tokio_util::context::RuntimeExt;\n\nfn main() {\n let (tx, rx) = oneshot::channel();\n let rt1 = Builder::new_multi_thread()\n .worker_threads(1)\n // no timer!\n .build()\n .unwrap();\n let rt2 = Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n\n // Without the `HandleExt.wrap()` there would be a panic because there is\n // no timer running, since it would be referencing runtime r1.\n rt1.block_on(rt2.wrap(async move {\n let listener = TcpListener::bind(\"0.0.0.0:0\").await.unwrap();\n println!(\"addr: {:?}\", listener.local_addr());\n tx.send(()).unwrap();\n }));\n futures::executor::block_on(rx).unwrap();\n}\n" }, { "path": "examples/custom-executor.rs", "content": "// This example shows how to use the tokio runtime with any other executor\n//\n// The main components are a spawn fn that will wrap futures in a special future\n// that will always enter the tokio context on poll. This only spawns one extra thread\n// to manage and run the tokio drivers in the background.\n\nuse tokio::net::TcpListener;\nuse tokio::sync::oneshot;\n\nfn main() {\n let (tx, rx) = oneshot::channel();\n\n my_custom_runtime::spawn(async move {\n let listener = TcpListener::bind(\"0.0.0.0:0\").await.unwrap();\n\n println!(\"addr: {:?}\", listener.local_addr());\n\n tx.send(()).unwrap();\n });\n\n futures::executor::block_on(rx).unwrap();\n}\n\nmod my_custom_runtime {\n use once_cell::sync::Lazy;\n use std::future::Future;\n use tokio_util::context::TokioContext;\n\n pub fn spawn(f: impl Future + Send + 'static) {\n EXECUTOR.spawn(f);\n }\n\n struct ThreadPool {\n inner: futures::executor::ThreadPool,\n rt: tokio::runtime::Runtime,\n }\n\n static EXECUTOR: Lazy = Lazy::new(|| {\n // Spawn tokio runtime on a single background thread\n // enabling IO and timers.\n let rt = tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .build()\n .unwrap();\n let inner = futures::executor::ThreadPool::builder().create().unwrap();\n\n ThreadPool { inner, rt }\n });\n\n impl ThreadPool {\n fn spawn(&self, f: impl Future + Send + 'static) {\n let handle = self.rt.handle().clone();\n self.inner.spawn_ok(TokioContext::new(f, handle));\n }\n }\n}\n" }, { "path": "examples/dump.rs", "content": "//! This example demonstrates tokio's experimental task dumping functionality.\n//! This application deadlocks. Input CTRL+C to display traces of each task, or\n//! input CTRL+C twice within 1 second to quit.\n\n#[cfg(all(\n tokio_unstable,\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n))]\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n use std::sync::Arc;\n use tokio::sync::Barrier;\n\n #[inline(never)]\n async fn a(barrier: Arc) {\n b(barrier).await\n }\n\n #[inline(never)]\n async fn b(barrier: Arc) {\n c(barrier).await\n }\n\n #[inline(never)]\n async fn c(barrier: Arc) {\n barrier.wait().await;\n }\n\n // Prints a task dump upon receipt of CTRL+C, or returns if CTRL+C is\n // inputted twice within a second.\n async fn dump_or_quit() {\n use tokio::time::{timeout, Duration, Instant};\n let handle = tokio::runtime::Handle::current();\n let mut last_signal: Option = None;\n // wait for CTRL+C\n while let Ok(_) = tokio::signal::ctrl_c().await {\n // exit if a CTRL+C is inputted twice within 1 second\n if let Some(time_since_last_signal) = last_signal.map(|i| i.elapsed()) {\n if time_since_last_signal < Duration::from_secs(1) {\n return;\n }\n }\n last_signal = Some(Instant::now());\n\n // capture a dump, and print each trace\n println!(\"{:-<80}\", \"\");\n if let Ok(dump) = timeout(Duration::from_secs(2), handle.dump()).await {\n for task in dump.tasks().iter() {\n let id = task.id();\n let trace = task.trace();\n println!(\"TASK {id}:\");\n println!(\"{trace}\\n\");\n }\n } else {\n println!(\"Task dumping timed out. Use a native debugger (like gdb) to debug the deadlock.\");\n }\n println!(\"{:-<80}\", \"\");\n println!(\"Input CTRL+C twice within 1 second to exit.\");\n }\n }\n\n println!(\"This program has a deadlock.\");\n println!(\"Input CTRL+C to print a task dump.\");\n println!(\"Input CTRL+C twice within 1 second to exit.\");\n\n // oops! this barrier waits for one more task than will ever come.\n let barrier = Arc::new(Barrier::new(3));\n\n let task_1 = tokio::spawn(a(barrier.clone()));\n let task_2 = tokio::spawn(a(barrier));\n\n tokio::select!(\n _ = dump_or_quit() => {},\n _ = task_1 => {},\n _ = task_2 => {},\n );\n\n Ok(())\n}\n\n#[cfg(not(all(\n tokio_unstable,\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n)))]\nfn main() {\n println!(\"task dumps are not available\")\n}\n" }, { "path": "examples/echo-tcp.rs", "content": "//! A \"hello world\" echo server with Tokio\n//!\n//! This server will create a TCP listener, accept connections in a loop, and\n//! write back everything that's read off of each TCP connection.\n//!\n//! Because the Tokio runtime uses a thread pool, each TCP connection is\n//! processed concurrently with all other TCP connections across multiple\n//! threads.\n//!\n//! To see this server in action, you can run this in one terminal:\n//!\n//! cargo run --example echo-tcp\n//!\n//! and in another terminal you can run:\n//!\n//! cargo run --example connect-tcp 127.0.0.1:8080\n//!\n//! Each line you type in to the `connect-tcp` terminal should be echo'd back to\n//! you! If you open up multiple terminals running the `connect-tcp` example you\n//! should be able to see them all make progress simultaneously.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::TcpListener;\n\nuse std::env;\nuse std::error::Error;\n\nconst DEFAULT_ADDR: &str = \"127.0.0.1:8080\";\nconst BUFFER_SIZE: usize = 4096;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Allow passing an address to listen on as the first argument of this\n // program, but otherwise we'll just set up our TCP listener on\n // 127.0.0.1:8080 for connections.\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| DEFAULT_ADDR.to_string());\n\n // Next up we create a TCP listener which will listen for incoming\n // connections. This TCP listener is bound to the address we determined\n // above and must be associated with an event loop.\n let listener = TcpListener::bind(&addr).await?;\n println!(\"Listening on: {addr}\");\n\n loop {\n // Asynchronously wait for an inbound socket.\n let (mut socket, addr) = listener.accept().await?;\n\n // And this is where much of the magic of this server happens. We\n // crucially want all clients to make progress concurrently, rather than\n // blocking one on completion of another. To achieve this we use the\n // `tokio::spawn` function to execute the work in the background.\n //\n // Essentially here we're executing a new task to run concurrently,\n // which will allow all of our clients to be processed concurrently.\n\n tokio::spawn(async move {\n let mut buf = vec![0; BUFFER_SIZE];\n\n // In a loop, read data from the socket and write the data back.\n loop {\n match socket.read(&mut buf).await {\n Ok(0) => {\n // Connection closed by peer\n return;\n }\n Ok(n) => {\n // Write the data back. If writing fails, log the error and exit.\n if let Err(e) = socket.write_all(&buf[0..n]).await {\n eprintln!(\"Failed to write to socket {}: {}\", addr, e);\n return;\n }\n }\n Err(e) => {\n eprintln!(\"Failed to read from socket {}: {}\", addr, e);\n return;\n }\n }\n }\n });\n }\n}\n" }, { "path": "examples/echo-udp.rs", "content": "//! An UDP echo server that just sends back everything that it receives.\n//!\n//! If you're on Unix you can test this out by in one terminal executing:\n//!\n//! cargo run --example echo-udp\n//!\n//! and in another terminal you can run:\n//!\n//! cargo run --example connect-udp 127.0.0.1:8080\n//!\n//! Each line you type in to the `connect-udp` terminal should be echo'd back to you!\n\n#![warn(rust_2018_idioms)]\n\nuse std::error::Error;\nuse std::net::SocketAddr;\nuse std::{env, io};\nuse tokio::net::UdpSocket;\n\nstruct Server {\n socket: UdpSocket,\n buf: Vec,\n to_send: Option<(usize, SocketAddr)>,\n}\n\nimpl Server {\n async fn run(self) -> Result<(), io::Error> {\n let Server {\n socket,\n mut buf,\n mut to_send,\n } = self;\n\n loop {\n // First we check to see if there's a message we need to echo back.\n // If so then we try to send it back to the original source, waiting\n // until it's writable and we're able to do so.\n if let Some((size, peer)) = to_send {\n let amt = socket.send_to(&buf[..size], &peer).await?;\n\n println!(\"Echoed {amt}/{size} bytes to {peer}\");\n }\n\n // If we're here then `to_send` is `None`, so we take a look for the\n // next message we're going to echo back.\n to_send = Some(socket.recv_from(&mut buf).await?);\n }\n }\n}\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8080\".to_string());\n\n let socket = UdpSocket::bind(&addr).await?;\n println!(\"Listening on: {}\", socket.local_addr()?);\n\n let server = Server {\n socket,\n buf: vec![0; 1024],\n to_send: None,\n };\n\n // This starts the server task.\n server.run().await?;\n\n Ok(())\n}\n" }, { "path": "examples/hello_world.rs", "content": "//! A simple client that opens a TCP stream, writes \"hello world\\n\", and closes\n//! the connection.\n//!\n//! To start a server that this client can talk to on port 6142, you can use this command:\n//!\n//! ncat -l 6142\n//!\n//! And then in another terminal run:\n//!\n//! cargo run --example hello_world\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::io::AsyncWriteExt;\nuse tokio::net::TcpStream;\n\nuse std::error::Error;\n\n#[tokio::main]\npub async fn main() -> Result<(), Box> {\n // Open a TCP stream to the socket address.\n //\n // Note that this is the Tokio TcpStream, which is fully async.\n let mut stream = TcpStream::connect(\"127.0.0.1:6142\").await?;\n println!(\"created stream\");\n\n let result = stream.write_all(b\"hello world\\n\").await;\n println!(\"wrote to stream; success={:?}\", result.is_ok());\n\n Ok(())\n}\n" }, { "path": "examples/named-pipe-multi-client.rs", "content": "use std::io;\n\n#[cfg(windows)]\nasync fn windows_main() -> io::Result<()> {\n use std::time::Duration;\n use tokio::io::{AsyncReadExt, AsyncWriteExt};\n use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n use tokio::time;\n use windows_sys::Win32::Foundation::ERROR_PIPE_BUSY;\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\named-pipe-multi-client\";\n const N: usize = 10;\n\n // The first server needs to be constructed early so that clients can\n // be correctly connected. Otherwise a waiting client will error.\n //\n // Here we also make use of `first_pipe_instance`, which will ensure\n // that there are no other servers up and running already.\n let mut server = ServerOptions::new()\n .first_pipe_instance(true)\n .create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n // Artificial workload.\n time::sleep(Duration::from_secs(1)).await;\n\n for _ in 0..N {\n // Wait for client to connect.\n server.connect().await?;\n let mut inner = server;\n\n // Construct the next server to be connected before sending the one\n // we already have of onto a task. This ensures that the server\n // isn't closed (after it's done in the task) before a new one is\n // available. Otherwise the client might error with\n // `io::ErrorKind::NotFound`.\n server = ServerOptions::new().create(PIPE_NAME)?;\n\n let _ = tokio::spawn(async move {\n let mut buf = vec![0u8; 4];\n inner.read_exact(&mut buf).await?;\n inner.write_all(b\"pong\").await?;\n Ok::<_, io::Error>(())\n });\n }\n\n Ok::<_, io::Error>(())\n });\n\n let mut clients = Vec::new();\n\n for _ in 0..N {\n clients.push(tokio::spawn(async move {\n // This showcases a generic connect loop.\n //\n // We immediately try to create a client, if it's not found or\n // the pipe is busy we use the specialized wait function on the\n // client builder.\n let mut client = loop {\n match ClientOptions::new().open(PIPE_NAME) {\n Ok(client) => break client,\n Err(e) if e.raw_os_error() == Some(ERROR_PIPE_BUSY as i32) => (),\n Err(e) => return Err(e),\n }\n\n time::sleep(Duration::from_millis(5)).await;\n };\n\n let mut buf = [0u8; 4];\n client.write_all(b\"ping\").await?;\n client.read_exact(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n }));\n }\n\n for client in clients {\n let result = client.await?;\n assert_eq!(&result?[..], b\"pong\");\n }\n\n server.await??;\n Ok(())\n}\n\n#[tokio::main]\nasync fn main() -> io::Result<()> {\n #[cfg(windows)]\n {\n windows_main().await?;\n }\n\n #[cfg(not(windows))]\n {\n println!(\"Named pipes are only supported on Windows!\");\n }\n\n Ok(())\n}\n" }, { "path": "examples/named-pipe-ready.rs", "content": "use std::io;\n\n#[cfg(windows)]\nasync fn windows_main() -> io::Result<()> {\n use tokio::io::Interest;\n use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\named-pipe-single-client\";\n\n let server = ServerOptions::new().create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n // Note: we wait for a client to connect.\n server.connect().await?;\n\n let buf = {\n let mut read_buf = [0u8; 5];\n let mut read_buf_cursor = 0;\n\n loop {\n server.readable().await?;\n\n let buf = &mut read_buf[read_buf_cursor..];\n\n match server.try_read(buf) {\n Ok(n) => {\n read_buf_cursor += n;\n\n if read_buf_cursor == read_buf.len() {\n break;\n }\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n\n read_buf\n };\n\n {\n let write_buf = b\"pong\\n\";\n let mut write_buf_cursor = 0;\n\n loop {\n let buf = &write_buf[write_buf_cursor..];\n\n if buf.is_empty() {\n break;\n }\n\n server.writable().await?;\n\n match server.try_write(buf) {\n Ok(n) => {\n write_buf_cursor += n;\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n }\n\n Ok::<_, io::Error>(buf)\n });\n\n let client = tokio::spawn(async move {\n // There's no need to use a connect loop here, since we know that the\n // server is already up - `open` was called before spawning any of the\n // tasks.\n let client = ClientOptions::new().open(PIPE_NAME)?;\n\n let mut read_buf = [0u8; 5];\n let mut read_buf_cursor = 0;\n let write_buf = b\"ping\\n\";\n let mut write_buf_cursor = 0;\n\n loop {\n let mut interest = Interest::READABLE;\n if write_buf_cursor < write_buf.len() {\n interest |= Interest::WRITABLE;\n }\n\n let ready = client.ready(interest).await?;\n\n if ready.is_readable() {\n let buf = &mut read_buf[read_buf_cursor..];\n\n match client.try_read(buf) {\n Ok(n) => {\n read_buf_cursor += n;\n\n if read_buf_cursor == read_buf.len() {\n break;\n }\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n\n if ready.is_writable() {\n let buf = &write_buf[write_buf_cursor..];\n\n if buf.is_empty() {\n continue;\n }\n\n match client.try_write(buf) {\n Ok(n) => {\n write_buf_cursor += n;\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n }\n\n let buf = String::from_utf8_lossy(&read_buf).into_owned();\n\n Ok::<_, io::Error>(buf)\n });\n\n let (server, client) = tokio::try_join!(server, client)?;\n\n assert_eq!(server?, *b\"ping\\n\");\n assert_eq!(client?, \"pong\\n\");\n\n Ok(())\n}\n\n#[tokio::main]\nasync fn main() -> io::Result<()> {\n #[cfg(windows)]\n {\n windows_main().await?;\n }\n\n #[cfg(not(windows))]\n {\n println!(\"Named pipes are only supported on Windows!\");\n }\n\n Ok(())\n}\n" }, { "path": "examples/named-pipe.rs", "content": "use std::io;\n\n#[cfg(windows)]\nasync fn windows_main() -> io::Result<()> {\n use tokio::io::AsyncWriteExt;\n use tokio::io::{AsyncBufReadExt, BufReader};\n use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\named-pipe-single-client\";\n\n let server = ServerOptions::new().create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n // Note: we wait for a client to connect.\n server.connect().await?;\n\n let mut server = BufReader::new(server);\n\n let mut buf = String::new();\n server.read_line(&mut buf).await?;\n server.write_all(b\"pong\\n\").await?;\n Ok::<_, io::Error>(buf)\n });\n\n let client = tokio::spawn(async move {\n // There's no need to use a connect loop here, since we know that the\n // server is already up - `open` was called before spawning any of the\n // tasks.\n let client = ClientOptions::new().open(PIPE_NAME)?;\n\n let mut client = BufReader::new(client);\n\n let mut buf = String::new();\n client.write_all(b\"ping\\n\").await?;\n client.read_line(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n });\n\n let (server, client) = tokio::try_join!(server, client)?;\n\n assert_eq!(server?, \"ping\\n\");\n assert_eq!(client?, \"pong\\n\");\n\n Ok(())\n}\n\n#[tokio::main]\nasync fn main() -> io::Result<()> {\n #[cfg(windows)]\n {\n windows_main().await?;\n }\n\n #[cfg(not(windows))]\n {\n println!(\"Named pipes are only supported on Windows!\");\n }\n\n Ok(())\n}\n" }, { "path": "examples/print_each_packet.rs", "content": "//! A \"print-each-packet\" server with Tokio\n//!\n//! This server will create a TCP listener, accept connections in a loop, and\n//! put down in the stdout everything that's read off of each TCP connection.\n//!\n//! Because the Tokio runtime uses a thread pool, each TCP connection is\n//! processed concurrently with all other TCP connections across multiple\n//! threads.\n//!\n//! To see this server in action, you can run this in one terminal:\n//!\n//! cargo run --example print\\_each\\_packet\n//!\n//! and in another terminal you can run:\n//!\n//! cargo run --example connect-tcp 127.0.0.1:8080\n//!\n//! Each line you type in to the `connect-tcp` terminal should be written to terminal!\n//!\n//! Minimal js example:\n//!\n//! ```js\n//! var net = require(\"net\");\n//!\n//! var listenPort = 8080;\n//!\n//! var server = net.createServer(function (socket) {\n//! socket.on(\"data\", function (bytes) {\n//! console.log(\"bytes\", bytes);\n//! });\n//!\n//! socket.on(\"end\", function() {\n//! console.log(\"Socket received FIN packet and closed connection\");\n//! });\n//! socket.on(\"error\", function (error) {\n//! console.log(\"Socket closed with error\", error);\n//! });\n//!\n//! socket.on(\"close\", function (with_error) {\n//! if (with_error) {\n//! console.log(\"Socket closed with result: Err(SomeError)\");\n//! } else {\n//! console.log(\"Socket closed with result: Ok(())\");\n//! }\n//! });\n//!\n//! });\n//!\n//! server.listen(listenPort);\n//!\n//! console.log(\"Listening on:\", listenPort);\n//! ```\n//!\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::net::TcpListener;\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::{BytesCodec, Decoder};\n\nuse std::env;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Allow passing an address to listen on as the first argument of this\n // program, but otherwise we'll just set up our TCP listener on\n // 127.0.0.1:8080 for connections.\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8080\".to_string());\n\n // Next up we create a TCP listener which will listen for incoming\n // connections. This TCP listener is bound to the address we determined\n // above and must be associated with an event loop, so we pass in a handle\n // to our event loop. After the socket's created we inform that we're ready\n // to go and start accepting connections.\n let listener = TcpListener::bind(&addr).await?;\n println!(\"Listening on: {addr}\");\n\n loop {\n // Asynchronously wait for an inbound socket.\n let (socket, _) = listener.accept().await?;\n\n // And this is where much of the magic of this server happens. We\n // crucially want all clients to make progress concurrently, rather than\n // blocking one on completion of another. To achieve this we use the\n // `tokio::spawn` function to execute the work in the background.\n //\n // Essentially here we're executing a new task to run concurrently,\n // which will allow all of our clients to be processed concurrently.\n tokio::spawn(async move {\n // We're parsing each socket with the `BytesCodec` included in `tokio::codec`.\n let mut framed = BytesCodec::new().framed(socket);\n\n // We loop while there are messages coming from the Stream `framed`.\n // The stream will return None once the client disconnects.\n while let Some(message) = framed.next().await {\n match message {\n Ok(bytes) => println!(\"bytes: {bytes:?}\"),\n Err(err) => println!(\"Socket closed with error: {err:?}\"),\n }\n }\n println!(\"Socket received FIN packet and closed connection\");\n });\n }\n}\n" }, { "path": "examples/proxy.rs", "content": "//! A proxy that forwards data to another server and forwards that server's\n//! responses back to clients.\n//!\n//! Because the Tokio runtime uses a thread pool, each TCP connection is\n//! processed concurrently with all other TCP connections across multiple\n//! threads.\n//!\n//! You can showcase this by running this in one terminal:\n//!\n//! cargo run --example proxy\n//!\n//! This in another terminal\n//!\n//! cargo run --example echo-tcp\n//!\n//! And finally this in another terminal\n//!\n//! cargo run --example connect-tcp 127.0.0.1:8081\n//!\n//! This final terminal will connect to our proxy, which will in turn connect to\n//! the echo server, and you'll be able to see data flowing between them.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::io::copy_bidirectional;\nuse tokio::net::{TcpListener, TcpStream};\n\nuse futures::FutureExt;\nuse std::env;\nuse std::error::Error;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let listen_addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8081\".to_string());\n let server_addr = env::args()\n .nth(2)\n .unwrap_or_else(|| \"127.0.0.1:8080\".to_string());\n\n println!(\"Listening on: {listen_addr}\");\n println!(\"Proxying to: {server_addr}\");\n\n let listener = TcpListener::bind(listen_addr).await?;\n\n while let Ok((mut inbound, _)) = listener.accept().await {\n let mut outbound = TcpStream::connect(server_addr.clone()).await?;\n\n tokio::spawn(async move {\n copy_bidirectional(&mut inbound, &mut outbound)\n .map(|r| {\n if let Err(e) = r {\n println!(\"Failed to transfer; error={e}\");\n }\n })\n .await\n });\n }\n\n Ok(())\n}\n" }, { "path": "examples/tinydb.rs", "content": "//! A \"tiny database\" and accompanying protocol\n//!\n//! This example shows the usage of shared state amongst all connected clients,\n//! namely a database of key/value pairs. Each connected client can send a\n//! series of GET/SET commands to query the current value of a key or set the\n//! value of a key.\n//!\n//! This example has a simple protocol you can use to interact with the server.\n//! To run, first run this in one terminal window:\n//!\n//! cargo run --example tinydb\n//!\n//! and next in another windows run:\n//!\n//! cargo run --example connect-tcp 127.0.0.1:8080\n//!\n//! In the `connect-tcp` window you can type in commands where when you hit enter\n//! you'll get a response from the server for that command. An example session\n//! is:\n//!\n//!\n//! $ cargo run --example connect-tcp 127.0.0.1:8080\n//! GET foo\n//! foo = bar\n//! GET FOOBAR\n//! error: no key FOOBAR\n//! SET FOOBAR my awesome string\n//! set FOOBAR = `my awesome string`, previous: None\n//! SET foo tokio\n//! set foo = `tokio`, previous: Some(\"bar\")\n//! GET foo\n//! foo = tokio\n//!\n//! Namely you can issue two forms of commands:\n//!\n//! * `GET $key` - this will fetch the value of `$key` from the database and\n//! return it. The server's database is initially populated with the key `foo`\n//! set to the value `bar`\n//! * `SET $key $value` - this will set the value of `$key` to `$value`,\n//! returning the previous value, if any.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::net::TcpListener;\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::{Framed, LinesCodec};\n\nuse futures::SinkExt;\nuse std::collections::HashMap;\nuse std::env;\nuse std::error::Error;\nuse std::sync::{Arc, Mutex};\n\n/// The in-memory database shared amongst all clients.\n///\n/// This database will be shared via `Arc`, so to mutate the internal map we're\n/// going to use a `Mutex` for interior mutability.\nstruct Database {\n map: Mutex>,\n}\n\n/// Possible requests our clients can send us\nenum Request {\n Get { key: String },\n Set { key: String, value: String },\n}\n\n/// Responses to the `Request` commands above\nenum Response {\n Value {\n key: String,\n value: String,\n },\n Set {\n key: String,\n value: String,\n previous: Option,\n },\n Error {\n msg: String,\n },\n}\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Parse the address we're going to run this server on\n // and set up our TCP listener to accept connections.\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8080\".to_string());\n\n let listener = TcpListener::bind(&addr).await?;\n println!(\"Listening on: {addr}\");\n\n // Create the shared state of this server that will be shared amongst all\n // clients. We populate the initial database and then create the `Database`\n // structure. Note the usage of `Arc` here which will be used to ensure that\n // each independently spawned client will have a reference to the in-memory\n // database.\n let mut initial_db = HashMap::new();\n initial_db.insert(\"foo\".to_string(), \"bar\".to_string());\n let db = Arc::new(Database {\n map: Mutex::new(initial_db),\n });\n\n loop {\n match listener.accept().await {\n Ok((socket, _)) => {\n // After getting a new connection first we see a clone of the database\n // being created, which is creating a new reference for this connected\n // client to use.\n let db = db.clone();\n\n // Like with other small servers, we'll `spawn` this client to ensure it\n // runs concurrently with all other clients. The `move` keyword is used\n // here to move ownership of our db handle into the async closure.\n tokio::spawn(async move {\n // Since our protocol is line-based we use `tokio_codecs`'s `LineCodec`\n // to convert our stream of bytes, `socket`, into a `Stream` of lines\n // as well as convert our line based responses into a stream of bytes.\n let mut lines = Framed::new(socket, LinesCodec::new());\n\n // Here for every line we get back from the `Framed` decoder,\n // we parse the request, and if it's valid we generate a response\n // based on the values in the database.\n while let Some(result) = lines.next().await {\n match result {\n Ok(line) => {\n let response = handle_request(&line, &db);\n\n let response = response.serialize();\n\n if let Err(e) = lines.send(response.as_str()).await {\n println!(\"error on sending response; error = {e:?}\");\n }\n }\n Err(e) => {\n println!(\"error on decoding from socket; error = {e:?}\");\n }\n }\n }\n\n // The connection will be closed at this point as `lines.next()` has returned `None`.\n });\n }\n Err(e) => println!(\"error accepting socket; error = {e:?}\"),\n }\n }\n}\n\nfn handle_request(line: &str, db: &Arc) -> Response {\n let request = match Request::parse(line) {\n Ok(req) => req,\n Err(e) => return Response::Error { msg: e },\n };\n\n let mut db = db.map.lock().unwrap();\n match request {\n Request::Get { key } => match db.get(&key) {\n Some(value) => Response::Value {\n key,\n value: value.clone(),\n },\n None => Response::Error {\n msg: format!(\"no key {key}\"),\n },\n },\n Request::Set { key, value } => {\n let previous = db.insert(key.clone(), value.clone());\n Response::Set {\n key,\n value,\n previous,\n }\n }\n }\n}\n\nimpl Request {\n fn parse(input: &str) -> Result {\n let mut parts = input.splitn(3, ' ');\n match parts.next() {\n Some(\"GET\") => {\n let key = parts.next().ok_or(\"GET must be followed by a key\")?;\n if parts.next().is_some() {\n return Err(\"GET's key must not be followed by anything\".into());\n }\n Ok(Request::Get {\n key: key.to_string(),\n })\n }\n Some(\"SET\") => {\n let key = match parts.next() {\n Some(key) => key,\n None => return Err(\"SET must be followed by a key\".into()),\n };\n let value = match parts.next() {\n Some(value) => value,\n None => return Err(\"SET needs a value\".into()),\n };\n Ok(Request::Set {\n key: key.to_string(),\n value: value.to_string(),\n })\n }\n Some(cmd) => Err(format!(\"unknown command: {cmd}\")),\n None => Err(\"empty input\".into()),\n }\n }\n}\n\nimpl Response {\n fn serialize(&self) -> String {\n match *self {\n Response::Value { ref key, ref value } => format!(\"{key} = {value}\"),\n Response::Set {\n ref key,\n ref value,\n ref previous,\n } => format!(\"set {key} = `{value}`, previous: {previous:?}\"),\n Response::Error { ref msg } => format!(\"error: {msg}\"),\n }\n }\n}\n" }, { "path": "examples/tinyhttp.rs", "content": "//! A \"tiny\" example of HTTP request/response handling using transports.\n//!\n//! This example is intended for *learning purposes* to see how various pieces\n//! hook up together and how HTTP can get up and running. Note that this example\n//! is written with the restriction that it *can't* use any \"big\" library other\n//! than Tokio, if you'd like a \"real world\" HTTP library you likely want a\n//! crate like Hyper.\n//!\n//! Code here is based on the `echo-threads` example and implements two paths,\n//! the `/plaintext` and `/json` routes to respond with some text and json,\n//! respectively. By default this will run I/O on all the cores your system has\n//! available, and it doesn't support HTTP request bodies.\n\n#![warn(rust_2018_idioms)]\n\nuse bytes::BytesMut;\nuse futures::SinkExt;\nuse http::{header::HeaderValue, Request, Response, StatusCode};\n#[macro_use]\nextern crate serde_derive;\nuse std::{env, error::Error, fmt, io};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::{Decoder, Encoder, Framed};\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n // Parse the arguments, bind the TCP socket we'll be listening to, spin up\n // our worker threads, and start shipping sockets to those worker threads.\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8080\".to_string());\n let server = TcpListener::bind(&addr).await?;\n println!(\"Listening on: {addr}\");\n\n loop {\n let (stream, _) = server.accept().await?;\n tokio::spawn(async move {\n if let Err(e) = process(stream).await {\n println!(\"failed to process connection; error = {e}\");\n }\n });\n }\n}\n\nasync fn process(stream: TcpStream) -> Result<(), Box> {\n let mut transport = Framed::new(stream, Http);\n\n while let Some(request) = transport.next().await {\n match request {\n Ok(request) => {\n let response = respond(request).await?;\n transport.send(response).await?;\n }\n Err(e) => return Err(e.into()),\n }\n }\n\n Ok(())\n}\n\nasync fn respond(req: Request<()>) -> Result, Box> {\n let mut response = Response::builder();\n let body = match req.uri().path() {\n \"/plaintext\" => {\n response = response.header(\"Content-Type\", \"text/plain\");\n \"Hello, World!\".to_string()\n }\n \"/json\" => {\n response = response.header(\"Content-Type\", \"application/json\");\n\n #[derive(Serialize)]\n struct Message {\n message: &'static str,\n }\n serde_json::to_string(&Message {\n message: \"Hello, World!\",\n })?\n }\n _ => {\n response = response.status(StatusCode::NOT_FOUND);\n String::new()\n }\n };\n let response = response.body(body).map_err(io::Error::other)?;\n\n Ok(response)\n}\n\nstruct Http;\n\n/// Implementation of encoding an HTTP response into a `BytesMut`, basically\n/// just writing out an HTTP/1.1 response.\nimpl Encoder> for Http {\n type Error = io::Error;\n\n fn encode(&mut self, item: Response, dst: &mut BytesMut) -> io::Result<()> {\n use std::fmt::Write;\n\n write!(\n BytesWrite(dst),\n \"\\\n HTTP/1.1 {}\\r\\n\\\n Server: Example\\r\\n\\\n Content-Length: {}\\r\\n\\\n Date: {}\\r\\n\\\n \",\n item.status(),\n item.body().len(),\n date::now()\n )\n .unwrap();\n\n for (k, v) in item.headers() {\n dst.extend_from_slice(k.as_str().as_bytes());\n dst.extend_from_slice(b\": \");\n dst.extend_from_slice(v.as_bytes());\n dst.extend_from_slice(b\"\\r\\n\");\n }\n\n dst.extend_from_slice(b\"\\r\\n\");\n dst.extend_from_slice(item.body().as_bytes());\n\n return Ok(());\n\n // Right now `write!` on `Vec` goes through io::Write and is not\n // super speedy, so inline a less-crufty implementation here which\n // doesn't go through io::Error.\n struct BytesWrite<'a>(&'a mut BytesMut);\n\n impl fmt::Write for BytesWrite<'_> {\n fn write_str(&mut self, s: &str) -> fmt::Result {\n self.0.extend_from_slice(s.as_bytes());\n Ok(())\n }\n\n fn write_fmt(&mut self, args: fmt::Arguments<'_>) -> fmt::Result {\n fmt::write(self, args)\n }\n }\n }\n}\n\n/// Implementation of decoding an HTTP request from the bytes we've read so far.\n/// This leverages the `httparse` crate to do the actual parsing and then we use\n/// that information to construct an instance of a `http::Request` object,\n/// trying to avoid allocations where possible.\nimpl Decoder for Http {\n type Item = Request<()>;\n type Error = io::Error;\n\n fn decode(&mut self, src: &mut BytesMut) -> io::Result>> {\n // TODO: we should grow this headers array if parsing fails and asks\n // for more headers\n let mut headers = [None; 16];\n let (method, path, version, amt) = {\n let mut parsed_headers = [httparse::EMPTY_HEADER; 16];\n let mut r = httparse::Request::new(&mut parsed_headers);\n let status = r.parse(src).map_err(|e| {\n let msg = format!(\"failed to parse http request: {e:?}\");\n io::Error::other(msg)\n })?;\n\n let amt = match status {\n httparse::Status::Complete(amt) => amt,\n httparse::Status::Partial => return Ok(None),\n };\n\n let toslice = |a: &[u8]| {\n let start = a.as_ptr() as usize - src.as_ptr() as usize;\n assert!(start < src.len());\n (start, start + a.len())\n };\n\n for (i, header) in r.headers.iter().enumerate() {\n let k = toslice(header.name.as_bytes());\n let v = toslice(header.value);\n headers[i] = Some((k, v));\n }\n\n let method = http::Method::try_from(r.method.unwrap()).map_err(io::Error::other)?;\n\n (\n method,\n toslice(r.path.unwrap().as_bytes()),\n r.version.unwrap(),\n amt,\n )\n };\n if version != 1 {\n return Err(io::Error::other(\"only HTTP/1.1 accepted\"));\n }\n let data = src.split_to(amt).freeze();\n let mut ret = Request::builder();\n ret = ret.method(method);\n let s = data.slice(path.0..path.1);\n let s = unsafe { String::from_utf8_unchecked(Vec::from(s.as_ref())) };\n ret = ret.uri(s);\n ret = ret.version(http::Version::HTTP_11);\n for header in headers.iter() {\n let (k, v) = match *header {\n Some((ref k, ref v)) => (k, v),\n None => break,\n };\n let value = HeaderValue::from_bytes(data.slice(v.0..v.1).as_ref())\n .map_err(|_| io::Error::other(\"header decode error\"))?;\n ret = ret.header(&data[k.0..k.1], value);\n }\n\n let req = ret.body(()).map_err(io::Error::other)?;\n Ok(Some(req))\n }\n}\n\nmod date {\n use std::cell::RefCell;\n use std::fmt::{self, Write};\n use std::str;\n use std::time::SystemTime;\n\n use httpdate::HttpDate;\n\n pub struct Now(());\n\n /// Returns a struct, which when formatted, renders an appropriate `Date`\n /// header value.\n pub fn now() -> Now {\n Now(())\n }\n\n // Gee Alex, doesn't this seem like premature optimization. Well you see\n // there Billy, you're absolutely correct! If your server is *bottlenecked*\n // on rendering the `Date` header, well then boy do I have news for you, you\n // don't need this optimization.\n //\n // In all seriousness, though, a simple \"hello world\" benchmark which just\n // sends back literally \"hello world\" with standard headers actually is\n // bottlenecked on rendering a date into a byte buffer. Since it was at the\n // top of a profile, and this was done for some competitive benchmarks, this\n // module was written.\n //\n // Just to be clear, though, I was not intending on doing this because it\n // really does seem kinda absurd, but it was done by someone else [1], so I\n // blame them! :)\n //\n // [1]: https://github.com/rapidoid/rapidoid/blob/f1c55c0555007e986b5d069fe1086e6d09933f7b/rapidoid-commons/src/main/java/org/rapidoid/commons/Dates.java#L48-L66\n\n struct LastRenderedNow {\n bytes: [u8; 128],\n amt: usize,\n unix_date: u64,\n }\n\n thread_local!(static LAST: RefCell = const { RefCell::new(LastRenderedNow {\n bytes: [0; 128],\n amt: 0,\n unix_date: 0,\n }) });\n\n impl fmt::Display for Now {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n LAST.with(|cache| {\n let mut cache = cache.borrow_mut();\n let now = SystemTime::now();\n let now_unix = now\n .duration_since(SystemTime::UNIX_EPOCH)\n .map(|since_epoch| since_epoch.as_secs())\n .unwrap_or(0);\n if cache.unix_date != now_unix {\n cache.update(now, now_unix);\n }\n f.write_str(cache.buffer())\n })\n }\n }\n\n impl LastRenderedNow {\n fn buffer(&self) -> &str {\n str::from_utf8(&self.bytes[..self.amt]).unwrap()\n }\n\n fn update(&mut self, now: SystemTime, now_unix: u64) {\n self.amt = 0;\n self.unix_date = now_unix;\n write!(LocalBuffer(self), \"{}\", HttpDate::from(now)).unwrap();\n }\n }\n\n struct LocalBuffer<'a>(&'a mut LastRenderedNow);\n\n impl fmt::Write for LocalBuffer<'_> {\n fn write_str(&mut self, s: &str) -> fmt::Result {\n let start = self.0.amt;\n let end = start + s.len();\n self.0.bytes[start..end].copy_from_slice(s.as_bytes());\n self.0.amt += s.len();\n Ok(())\n }\n }\n}\n" }, { "path": "examples/udp-client.rs", "content": "//! A UDP client that just sends everything it gets via `stdio` in a single datagram, and then\n//! waits for a reply.\n//!\n//! For the reasons of simplicity data from `stdio` is read until `EOF` in a blocking manner.\n//!\n//! You can test this out by running an echo server:\n//!\n//! ```\n//! $ cargo run --example echo-udp -- 127.0.0.1:8080\n//! ```\n//!\n//! and running the client in another terminal:\n//!\n//! ```\n//! $ cargo run --example udp-client\n//! ```\n//!\n//! You can optionally provide any custom endpoint address for the client:\n//!\n//! ```\n//! $ cargo run --example udp-client -- 127.0.0.1:8080\n//! ```\n//!\n//! Don't forget to pass `EOF` to the standard input of the client!\n//!\n//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken\n//! connection the server needs to be run first, otherwise the client will block forever.\n\n#![warn(rust_2018_idioms)]\n\nuse std::env;\nuse std::error::Error;\nuse std::io::{stdin, Read};\nuse std::net::SocketAddr;\nuse tokio::net::UdpSocket;\n\nfn get_stdin_data() -> Result, Box> {\n let mut buf = Vec::new();\n stdin().read_to_end(&mut buf)?;\n Ok(buf)\n}\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let remote_addr: SocketAddr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:8080\".into())\n .parse()?;\n\n // We use port 0 to let the operating system allocate an available port for us.\n let local_addr: SocketAddr = if remote_addr.is_ipv4() {\n \"0.0.0.0:0\"\n } else {\n \"[::]:0\"\n }\n .parse()?;\n\n let socket = UdpSocket::bind(local_addr).await?;\n const MAX_DATAGRAM_SIZE: usize = 65_507;\n socket.connect(&remote_addr).await?;\n let data = get_stdin_data()?;\n socket.send(&data).await?;\n let mut data = vec![0u8; MAX_DATAGRAM_SIZE];\n let len = socket.recv(&mut data).await?;\n println!(\n \"Received {} bytes:\\n{}\",\n len,\n String::from_utf8_lossy(&data[..len])\n );\n\n Ok(())\n}\n" }, { "path": "examples/udp-codec.rs", "content": "//! This example leverages `BytesCodec` to create a UDP client and server which\n//! speak a custom protocol.\n//!\n//! Here we're using the codec from `tokio-codec` to convert a UDP socket to a stream of\n//! client messages. These messages are then processed and returned back as a\n//! new message with a new destination. Overall, we then use this to construct a\n//! \"ping pong\" pair where two sockets are sending messages back and forth.\n\n#![warn(rust_2018_idioms)]\n\nuse tokio::net::UdpSocket;\nuse tokio::{io, time};\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::BytesCodec;\nuse tokio_util::udp::UdpFramed;\n\nuse bytes::Bytes;\nuse futures::{FutureExt, SinkExt};\nuse std::env;\nuse std::error::Error;\nuse std::net::SocketAddr;\nuse std::time::Duration;\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let addr = env::args()\n .nth(1)\n .unwrap_or_else(|| \"127.0.0.1:0\".to_string());\n\n // Bind both our sockets and then figure out what ports we got.\n let a = UdpSocket::bind(&addr).await?;\n let b = UdpSocket::bind(&addr).await?;\n\n let b_addr = b.local_addr()?;\n\n let mut a = UdpFramed::new(a, BytesCodec::new());\n let mut b = UdpFramed::new(b, BytesCodec::new());\n\n // Start off by sending a ping from a to b, afterwards we just print out\n // what they send us and continually send pings\n let a = ping(&mut a, b_addr);\n\n // The second client we have will receive the pings from `a` and then send\n // back pongs.\n let b = pong(&mut b);\n\n // Run both futures simultaneously of `a` and `b` sending messages back and forth.\n match tokio::try_join!(a, b) {\n Err(e) => println!(\"an error occurred; error = {e:?}\"),\n _ => println!(\"done!\"),\n }\n\n Ok(())\n}\n\nasync fn ping(socket: &mut UdpFramed, b_addr: SocketAddr) -> Result<(), io::Error> {\n socket.send((Bytes::from(&b\"PING\"[..]), b_addr)).await?;\n\n for _ in 0..4usize {\n let (bytes, addr) = socket.next().map(|e| e.unwrap()).await?;\n\n println!(\"[a] recv: {}\", String::from_utf8_lossy(&bytes));\n\n socket.send((Bytes::from(&b\"PING\"[..]), addr)).await?;\n }\n\n Ok(())\n}\n\nasync fn pong(socket: &mut UdpFramed) -> Result<(), io::Error> {\n let timeout = Duration::from_millis(200);\n\n while let Ok(Some(Ok((bytes, addr)))) = time::timeout(timeout, socket.next()).await {\n println!(\"[b] recv: {}\", String::from_utf8_lossy(&bytes));\n\n socket.send((Bytes::from(&b\"PONG\"[..]), addr)).await?;\n }\n\n Ok(())\n}\n" }, { "path": "netlify.toml", "content": "[build]\n command = \"\"\"\n rustup install nightly --profile minimal && cargo doc --no-deps --all-features\n \"\"\"\n publish = \"target/doc\"\n\n[build.environment]\n RUSTDOCFLAGS=\"\"\"\n --cfg docsrs \\\n --cfg tokio_unstable \\\n \"\"\"\n RUSTFLAGS=\"\"\"\n --cfg docsrs \\\n --cfg tokio_unstable\n \"\"\"\n\n[[redirects]]\n from = \"/\"\n to = \"/tokio\"\n" }, { "path": "spellcheck.dic", "content": "312\n&\n+\n<\n=\n>\n\\\n~\n—\n0o777\n0s\n0xA\n0xD\n100ms\n100ns\n10ms\n10μs\n~12\n120s\n12.5%\n±1m\n±1ms\n1ms\n1s\n25%\n250ms\n2x\n~4\n443\n450ms\n50ms\n8MB\nABI\naccessors\nadaptor\nadaptors\nAdaptors\nAIO\nambient\namongst\napi\nAPIs\nasync\natomics\nawaitable\nbackend\nbackpressure\nbacktrace\nbacktraces\nbacktracing\nbinded\nbitfield\nbitfields\nBlockingly\nboolean\nbroadcasted\ncancelled\ncancelling\nCancelling\nCLI\ncloneable\ncodebase\ncodec\ncodecs\ncombinator\ncombinators\ncondvar\nconfig\nConfig\nconnectionless\ncoroutines\ncpu\ncpus\ncqe\nCQE\ncqe's\ncustomizable\nCustomizable\nCygwin\ndatagram\nDatagram\ndatagrams\ndeallocate\ndeallocated\nDeallocates\ndebuginfo\ndecrement\ndecrementing\ndemangled\ndequeued\ndereferenced\nderegister\nderegistered\nderegistering\nderegisters\nDeregisters\nderegistration\ndescriptor's\ndestructor\ndestructors\ndestructure\nDestructures\nDev\ndns\nDNS\nDoS\ndwOpenMode\nendian\nenqueue\nenqueued\nEntryInner\nenum\neof\nErlang's\nerrored\nEWMA\nexpirations\nfcntl\nfd\nfd's\nFIFOs\nfilename\nfilesystem\nfilesystems\nfn\nfns\nFreeBSD\nfrontend\nfs\nfunctionalities\ngetters\nGID\ngoroutines\nGrowable\ngzip\nH2\nhashmaps\nHashMaps\nhashsets\nHdrHistogram\nie\nIllumos\nimpl\nimplementers\nimplementor\nimplementors\nincrementing\ninlining\ninteroperate\ninvariants\nInvariants\nio\nIOCP\niOS\nIOs\nio_uring\nIP\nIPv4\nIPv6\niteratively\nKotlin's\nlatencies\nLauck\nlibc\nlifecycle\nlifo\nLLVM\nlookups\nmacOS\nMacOS\nMarsaglia's\nmetadata\nmio\nMio\nmio's\nmiri\nmisconfigured\nmock's\nmonomorphization\nmpmc\nmpsc\nmulti\nMulti\nmulticast\nMultithreaded\nmut\nmutex\nMutex\nNagle\nnamespace\nnonblocking\nnondecreasing\nnoop\nntasks\nNUMA\nok\noneshot\nopcode\nORed\nos\nparker\nparsers\npeekable\nPGID\nPID\nplaintext\npoller\nPOSIX\nproxied\nqos\nRAII\nreallocations\nrecv's\nrefactors\nrefcount\nrefcounting\nrepo\nrepr\nrepresentable\nreqwest\nreregistering\nresize\nresized\nRMW\nruntime\nruntime's\nruntimes\nrwlock\nrx\nscalability\nscheduler's\nsemver\nsetpgid\nsharded\nsignalled\nsignalling\nSmallCrush\nSolaris\nspawner\nSplitter\nspmc\nspsc\nsrc\nstabilised\nstartup\nstateful\nstderr\nstdin\nstdout\nstealable\nstealer\nstealers\nstruct\nstructs\nsubfield\nsuboptimal\nsubprocess\nsuperset\nsymlink\nsymlinks\nsys\nsyscall\nsyscalls\ntcp\nTCP\nTestU01\nthreadpool\ntimestamp\ntimestamps\nTLS\nTOCTOU\nTODO\ntokio\nTokio\ntokio's\nTokio's\ntuple\nTuple\ntx\nudp\nUDP\nUID\nunhandled\nunix\nunlink\nunpark\nUnpark\nunparked\nunparking\nunparks\nUnparks\nunreceived\nunsafety\nunsets\nUnsets\nunsynchronized\nuntrusted\nuring\nusecases\nValgrind\nVarghese\nvec\nversa\nversioned\nversioning\nvtable\nwaker\nwakers\nWakers\nwakeup\nwakeups\nworkstealing\nZST\n" }, { "path": "spellcheck.toml", "content": "dev_comments = false\nskip_readme = false\n\n[Hunspell]\nlang = \"en_US\"\nsearch_dirs = [\".\"]\nextra_dictionaries = [\"spellcheck.dic\"]\nskip_os_lookups = true\nuse_builtin = true\n\n[Hunspell.quirks]\nallow_concatenation = true\n\n" }, { "path": "stress-test/Cargo.toml", "content": "[package]\nname = \"stress-test\"\nversion = \"0.1.0\"\nauthors = [\"Tokio Contributors \"]\nedition = \"2021\"\nlicense = \"MIT\"\npublish = false\n\n# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html\n\n[dependencies]\ntokio = { version = \"1.0.0\", path = \"../tokio/\", features = [\"full\"] }\n\n[dev-dependencies]\nrand = \"0.9\"\n\n[lints]\nworkspace = true\n" }, { "path": "stress-test/examples/simple_echo_tcp.rs", "content": "//! Simple TCP echo server to check memory leaks using Valgrind.\nuse std::{thread::sleep, time::Duration};\n\nuse tokio::{\n io::{AsyncReadExt, AsyncWriteExt},\n net::{TcpListener, TcpSocket},\n runtime::Builder,\n sync::oneshot,\n};\n\nconst TCP_ENDPOINT: &str = \"127.0.0.1:8080\";\nconst NUM_MSGS: usize = 100;\nconst MSG_SIZE: usize = 1024;\n\nfn main() {\n let rt = Builder::new_multi_thread().enable_io().build().unwrap();\n let rt2 = Builder::new_multi_thread().enable_io().build().unwrap();\n\n rt.spawn(async {\n let listener = TcpListener::bind(TCP_ENDPOINT).await.unwrap();\n let (mut socket, _) = listener.accept().await.unwrap();\n let (mut rd, mut wr) = socket.split();\n while tokio::io::copy(&mut rd, &mut wr).await.is_ok() {}\n });\n\n // wait a bit so that the listener binds.\n sleep(Duration::from_millis(100));\n\n // create a channel to let the main thread know that all the messages were sent and received.\n let (tx, mut rx) = oneshot::channel();\n\n rt2.spawn(async {\n let addr = TCP_ENDPOINT.parse().unwrap();\n let socket = TcpSocket::new_v4().unwrap();\n let mut stream = socket.connect(addr).await.unwrap();\n\n let mut buff = [0; MSG_SIZE];\n for _ in 0..NUM_MSGS {\n let one_mega_random_bytes: Vec =\n (0..MSG_SIZE).map(|_| rand::random::()).collect();\n stream\n .write_all(one_mega_random_bytes.as_slice())\n .await\n .unwrap();\n let _ = stream.read(&mut buff).await.unwrap();\n }\n tx.send(()).unwrap();\n });\n\n loop {\n // check that we're done.\n match rx.try_recv() {\n Err(oneshot::error::TryRecvError::Empty) => (),\n Err(oneshot::error::TryRecvError::Closed) => panic!(\"channel got closed...\"),\n Ok(()) => break,\n }\n }\n}\n" }, { "path": "target-specs/README.md", "content": "This is used for the `no-atomic-u64-test` ci check that verifies that Tokio\nworks even if the `AtomicU64` type is missing.\n\nWhen increasing the nightly compiler version, you may need to regenerate this\ntarget using the following command:\n```\nrustc +nightly -Z unstable-options --print target-spec-json --target i686-unknown-linux-gnu | grep -v 'is-builtin' | sed 's/\"max-atomic-width\": 64/\"max-atomic-width\": 32/' > target-specs/i686-unknown-linux-gnu.json\n```\n\n" }, { "path": "target-specs/i686-unknown-linux-gnu.json", "content": "{\n \"arch\": \"x86\",\n \"cpu\": \"pentium4\",\n \"crt-objects-fallback\": \"false\",\n \"crt-static-respected\": true,\n \"data-layout\": \"e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-i128:128-f64:32:64-f80:32-n8:16:32-S128\",\n \"default-uwtable\": true,\n \"dynamic-linking\": true,\n \"env\": \"gnu\",\n \"has-rpath\": true,\n \"has-thread-local\": true,\n \"linker-flavor\": \"gnu-cc\",\n \"llvm-target\": \"i686-unknown-linux-gnu\",\n \"max-atomic-width\": 32,\n \"metadata\": {\n \"description\": \"32-bit Linux (kernel 3.2, glibc 2.17+)\",\n \"host_tools\": true,\n \"std\": true,\n \"tier\": 1\n },\n \"os\": \"linux\",\n \"position-independent-executables\": true,\n \"pre-link-args\": {\n \"gnu-cc\": [\n \"-m32\"\n ],\n \"gnu-lld-cc\": [\n \"-m32\"\n ]\n },\n \"relro-level\": \"full\",\n \"rustc-abi\": \"x86-sse2\",\n \"stack-probes\": {\n \"kind\": \"inline\"\n },\n \"supported-sanitizers\": [\n \"address\"\n ],\n \"supported-split-debuginfo\": [\n \"packed\",\n \"unpacked\",\n \"off\"\n ],\n \"target-family\": [\n \"unix\"\n ],\n \"target-pointer-width\": 32\n}\n" }, { "path": "tests-build/Cargo.toml", "content": "[package]\nname = \"tests-build\"\nversion = \"0.1.0\"\nauthors = [\"Tokio Contributors \"]\nedition = \"2021\"\nlicense = \"MIT\"\npublish = false\n\n[features]\nfull = [\"tokio/full\"]\nrt = [\"tokio/rt\", \"tokio/macros\"]\n\n[dependencies]\ntokio = { version = \"1.0.0\", path = \"../tokio\", optional = true }\n\n[dev-dependencies]\ntrybuild = \"1.0\"\n\n[lints]\nworkspace = true\n" }, { "path": "tests-build/README.md", "content": "Tests the various combination of feature flags. This is broken out to a separate\ncrate to work around limitations with cargo features.\n\nTo run all of the tests in this directory, run the following commands:\n```\ncargo test --features full\ncargo test --features rt\n```\nIf any of the tests fail, you can pass `TRYBUILD=overwrite` to the `cargo test`\ncommand that failed to have it regenerate the test output.\n" }, { "path": "tests-build/src/lib.rs", "content": "#[cfg(feature = \"tokio\")]\npub use tokio;\n" }, { "path": "tests-build/tests/fail/macros_core_no_default.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn my_fn() {}\n\nfn main() {}\n" }, { "path": "tests-build/tests/fail/macros_core_no_default.stderr", "content": "error: The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled.\n --> $DIR/macros_core_no_default.rs:3:1\n |\n3 | #[tokio::main]\n | ^^^^^^^^^^^^^^\n |\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n" }, { "path": "tests-build/tests/fail/macros_dead_code.rs", "content": "#![deny(dead_code)]\n\nuse tests_build::tokio;\n\n#[tokio::main]\nasync fn f() {}\n\nfn main() {}\n" }, { "path": "tests-build/tests/fail/macros_dead_code.stderr", "content": "error: function `f` is never used\n --> $DIR/macros_dead_code.rs:6:10\n |\n6 | async fn f() {}\n | ^\n |\nnote: the lint level is defined here\n --> $DIR/macros_dead_code.rs:1:9\n |\n1 | #![deny(dead_code)]\n | ^^^^^^^^^\n" }, { "path": "tests-build/tests/fail/macros_invalid_input.rs", "content": "#![deny(duplicate_macro_attributes)]\n\nuse tests_build::tokio;\n\n#[tokio::main]\nfn main_is_not_async() {}\n\n#[tokio::main(foo)]\nasync fn main_attr_has_unknown_args() {}\n\n#[tokio::main(threadpool::bar)]\nasync fn main_attr_has_path_args() {}\n\n#[tokio::test]\nfn test_is_not_async() {}\n\n#[tokio::test(foo)]\nasync fn test_attr_has_args() {}\n\n#[tokio::test(foo = 123)]\nasync fn test_unexpected_attr() {}\n\n#[tokio::test(flavor = 123)]\nasync fn test_flavor_not_string() {}\n\n#[tokio::test(flavor = \"foo\")]\nasync fn test_unknown_flavor() {}\n\n#[tokio::test(flavor = \"multi_thread\", start_paused = false)]\nasync fn test_multi_thread_with_start_paused() {}\n\n#[tokio::test(flavor = \"multi_thread\", worker_threads = \"foo\")]\nasync fn test_worker_threads_not_int() {}\n\n#[tokio::test(flavor = \"current_thread\", worker_threads = 4)]\nasync fn test_worker_threads_and_current_thread() {}\n\n#[tokio::test(crate = 456)]\nasync fn test_crate_not_path_int() {}\n\n#[tokio::test(crate = \"456\")]\nasync fn test_crate_not_path_invalid() {}\n\n#[tokio::test(flavor = \"multi_thread\", unhandled_panic = \"shutdown_runtime\")]\nasync fn test_multi_thread_with_unhandled_panic() {}\n\n#[tokio::test]\n#[test]\nasync fn test_has_second_test_attr() {}\n\n#[tokio::test]\n#[::core::prelude::v1::test]\nasync fn test_has_second_test_attr_v1() {}\n\n#[tokio::test]\n#[core::prelude::rust_2015::test]\nasync fn test_has_second_test_attr_rust_2015() {}\n\n#[tokio::test]\n#[::std::prelude::rust_2018::test]\nasync fn test_has_second_test_attr_rust_2018() {}\n\n#[tokio::test]\n#[std::prelude::rust_2021::test]\nasync fn test_has_second_test_attr_rust_2021() {}\n\n#[tokio::test]\n#[tokio::test]\nasync fn test_has_generated_second_test_attr() {}\n\nfn main() {}\n" }, { "path": "tests-build/tests/fail/macros_invalid_input.stderr", "content": "error: the `async` keyword is missing from the function declaration\n --> tests/fail/macros_invalid_input.rs:6:1\n |\n6 | fn main_is_not_async() {}\n | ^^\n\nerror: Unknown attribute foo is specified; expected one of: `flavor`, `worker_threads`, `start_paused`, `crate`, `unhandled_panic`.\n --> tests/fail/macros_invalid_input.rs:8:15\n |\n8 | #[tokio::main(foo)]\n | ^^^\n\nerror: Must have specified ident\n --> tests/fail/macros_invalid_input.rs:11:15\n |\n11 | #[tokio::main(threadpool::bar)]\n | ^^^^^^^^^^^^^^^\n\nerror: the `async` keyword is missing from the function declaration\n --> tests/fail/macros_invalid_input.rs:15:1\n |\n15 | fn test_is_not_async() {}\n | ^^\n\nerror: Unknown attribute foo is specified; expected one of: `flavor`, `worker_threads`, `start_paused`, `crate`, `unhandled_panic`.\n --> tests/fail/macros_invalid_input.rs:17:15\n |\n17 | #[tokio::test(foo)]\n | ^^^\n\nerror: Unknown attribute foo is specified; expected one of: `flavor`, `worker_threads`, `start_paused`, `crate`, `unhandled_panic`\n --> tests/fail/macros_invalid_input.rs:20:15\n |\n20 | #[tokio::test(foo = 123)]\n | ^^^^^^^^^\n\nerror: Failed to parse value of `flavor` as string.\n --> tests/fail/macros_invalid_input.rs:23:24\n |\n23 | #[tokio::test(flavor = 123)]\n | ^^^\n\nerror: No such runtime flavor `foo`. The runtime flavors are `current_thread`, `local`, and `multi_thread`.\n --> tests/fail/macros_invalid_input.rs:26:24\n |\n26 | #[tokio::test(flavor = \"foo\")]\n | ^^^^^\n\nerror: The `start_paused` option requires the `current_thread` runtime flavor. Use `#[tokio::test(flavor = \"current_thread\")]`\n --> tests/fail/macros_invalid_input.rs:29:55\n |\n29 | #[tokio::test(flavor = \"multi_thread\", start_paused = false)]\n | ^^^^^\n\nerror: Failed to parse value of `worker_threads` as integer.\n --> tests/fail/macros_invalid_input.rs:32:57\n |\n32 | #[tokio::test(flavor = \"multi_thread\", worker_threads = \"foo\")]\n | ^^^^^\n\nerror: The `worker_threads` option requires the `multi_thread` runtime flavor. Use `#[tokio::test(flavor = \"multi_thread\")]`\n --> tests/fail/macros_invalid_input.rs:35:59\n |\n35 | #[tokio::test(flavor = \"current_thread\", worker_threads = 4)]\n | ^\n\nerror: Failed to parse value of `crate` as path.\n --> tests/fail/macros_invalid_input.rs:38:23\n |\n38 | #[tokio::test(crate = 456)]\n | ^^^\n\nerror: Failed to parse value of `crate` as path: \"456\"\n --> tests/fail/macros_invalid_input.rs:41:23\n |\n41 | #[tokio::test(crate = \"456\")]\n | ^^^^^\n\nerror: The `unhandled_panic` option requires the `current_thread` runtime flavor. Use `#[tokio::test(flavor = \"current_thread\")]`\n --> tests/fail/macros_invalid_input.rs:44:58\n |\n44 | #[tokio::test(flavor = \"multi_thread\", unhandled_panic = \"shutdown_runtime\")]\n | ^^^^^^^^^^^^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:48:1\n |\n48 | #[test]\n | ^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:52:1\n |\n52 | #[::core::prelude::v1::test]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:56:1\n |\n56 | #[core::prelude::rust_2015::test]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:60:1\n |\n60 | #[::std::prelude::rust_2018::test]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:64:1\n |\n64 | #[std::prelude::rust_2021::test]\n | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror: second test attribute is supplied, consider removing or changing the order of your test attributes\n --> tests/fail/macros_invalid_input.rs:67:1\n |\n67 | #[tokio::test]\n | ^^^^^^^^^^^^^^\n |\n = note: this error originates in the attribute macro `tokio::test` (in Nightly builds, run with -Z macro-backtrace for more info)\n" }, { "path": "tests-build/tests/fail/macros_join.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn main() {\n // do not leak `RotatorSelect`\n let _ = tokio::join!(async {\n fn foo(_: impl RotatorSelect) {}\n });\n\n // do not leak `std::task::Poll::Pending`\n let _ = tokio::join!(async { Pending });\n\n // do not leak `std::task::Poll::Ready`\n let _ = tokio::join!(async { Ready(0) });\n\n // do not leak `std::future::Future`\n let _ = tokio::join!(async {\n struct MyFuture;\n\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(\n self: std::pin::Pin<&mut Self>,\n _cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll {\n todo!()\n }\n }\n });\n\n // do not leak `std::pin::Pin`\n let _ = tokio::join!(async {\n let mut x = 5;\n let _ = Pin::new(&mut x);\n });\n\n // do not leak `std::future::poll_fn`\n let _ = tokio::join!(async {\n let _ = poll_fn(|_cx| todo!());\n });\n}\n" }, { "path": "tests-build/tests/fail/macros_join.stderr", "content": "error[E0405]: cannot find trait `RotatorSelect` in this scope\n --> tests/fail/macros_join.rs:7:24\n |\n7 | fn foo(_: impl RotatorSelect) {}\n | ^^^^^^^^^^^^^ not found in this scope\n\nerror[E0425]: cannot find value `Pending` in this scope\n --> tests/fail/macros_join.rs:11:34\n |\n11 | let _ = tokio::join!(async { Pending });\n | ^^^^^^^ not found in this scope\n |\nhelp: consider importing this unit variant\n |\n 1 + use std::task::Poll::Pending;\n |\n\nerror[E0425]: cannot find function, tuple struct or tuple variant `Ready` in this scope\n --> tests/fail/macros_join.rs:14:34\n |\n14 | let _ = tokio::join!(async { Ready(0) });\n | ^^^^^ not found in this scope\n |\nhelp: consider importing this tuple variant\n |\n 1 + use std::task::Poll::Ready;\n |\n\nerror[E0405]: cannot find trait `Future` in this scope\n --> tests/fail/macros_join.rs:20:14\n |\n20 | impl Future for MyFuture {\n | ^^^^^^ not found in this scope\n |\nhelp: consider importing this trait\n |\n 1 + use std::future::Future;\n |\n\nerror[E0433]: failed to resolve: use of undeclared type `Pin`\n --> tests/fail/macros_join.rs:35:17\n |\n35 | let _ = Pin::new(&mut x);\n | ^^^ use of undeclared type `Pin`\n |\nhelp: consider importing this struct\n |\n 1 + use std::pin::Pin;\n |\n\nerror[E0425]: cannot find function `poll_fn` in this scope\n --> tests/fail/macros_join.rs:40:17\n |\n40 | let _ = poll_fn(|_cx| todo!());\n | ^^^^^^^ not found in this scope\n |\nhelp: consider importing this function\n |\n 1 + use std::future::poll_fn;\n |\n" }, { "path": "tests-build/tests/fail/macros_try_join.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn main() {\n // do not leak `RotatorSelect`\n let _ = tokio::try_join!(async {\n fn foo(_: impl RotatorSelect) {}\n });\n\n // do not leak `std::task::Poll::Pending`\n let _ = tokio::try_join!(async { Pending });\n\n // do not leak `std::task::Poll::Ready`\n let _ = tokio::try_join!(async { Ready(0) });\n\n // do not leak `std::future::Future`\n let _ = tokio::try_join!(async {\n struct MyFuture;\n\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(\n self: std::pin::Pin<&mut Self>,\n _cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll {\n todo!()\n }\n }\n });\n\n // do not leak `std::pin::Pin`\n let _ = tokio::try_join!(async {\n let mut x = 5;\n let _ = Pin::new(&mut x);\n });\n\n // do not leak `std::future::poll_fn`\n let _ = tokio::try_join!(async {\n let _ = poll_fn(|_cx| todo!());\n });\n}\n" }, { "path": "tests-build/tests/fail/macros_try_join.stderr", "content": "error[E0405]: cannot find trait `RotatorSelect` in this scope\n --> tests/fail/macros_try_join.rs:7:24\n |\n7 | fn foo(_: impl RotatorSelect) {}\n | ^^^^^^^^^^^^^ not found in this scope\n\nerror[E0425]: cannot find value `Pending` in this scope\n --> tests/fail/macros_try_join.rs:11:38\n |\n11 | let _ = tokio::try_join!(async { Pending });\n | ^^^^^^^ not found in this scope\n |\nhelp: consider importing this unit variant\n |\n 1 + use std::task::Poll::Pending;\n |\n\nerror[E0425]: cannot find function, tuple struct or tuple variant `Ready` in this scope\n --> tests/fail/macros_try_join.rs:14:38\n |\n14 | let _ = tokio::try_join!(async { Ready(0) });\n | ^^^^^ not found in this scope\n |\nhelp: consider importing this tuple variant\n |\n 1 + use std::task::Poll::Ready;\n |\n\nerror[E0405]: cannot find trait `Future` in this scope\n --> tests/fail/macros_try_join.rs:20:14\n |\n20 | impl Future for MyFuture {\n | ^^^^^^ not found in this scope\n |\nhelp: consider importing this trait\n |\n 1 + use std::future::Future;\n |\n\nerror[E0433]: failed to resolve: use of undeclared type `Pin`\n --> tests/fail/macros_try_join.rs:35:17\n |\n35 | let _ = Pin::new(&mut x);\n | ^^^ use of undeclared type `Pin`\n |\nhelp: consider importing this struct\n |\n 1 + use std::pin::Pin;\n |\n\nerror[E0425]: cannot find function `poll_fn` in this scope\n --> tests/fail/macros_try_join.rs:40:17\n |\n40 | let _ = poll_fn(|_cx| todo!());\n | ^^^^^^^ not found in this scope\n |\nhelp: consider importing this function\n |\n 1 + use std::future::poll_fn;\n |\n" }, { "path": "tests-build/tests/fail/macros_type_mismatch.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn missing_semicolon_or_return_type() {\n Ok(())\n}\n\n#[tokio::main]\nasync fn missing_return_type() {\n return Ok(());\n}\n\n#[tokio::main]\nasync fn extra_semicolon() -> Result<(), ()> {\n /* TODO(taiki-e): help message still wrong\n help: try using a variant of the expected enum\n |\n 23 | Ok(Ok(());)\n |\n 23 | Err(Ok(());)\n |\n */\n Ok(());\n}\n\n/// This test is a characterization test for the `?` operator.\n///\n/// See for more details.\n///\n/// It should fail with a single error message about the return type of the function, but instead\n/// if fails with an extra error message due to the `?` operator being used within the async block\n/// rather than the original function.\n///\n/// ```text\n/// 28 | None?;\n/// | ^ cannot use the `?` operator in an async block that returns `()`\n/// ```\n#[tokio::main]\nasync fn question_mark_operator_with_invalid_option() -> Option<()> {\n None?;\n}\n\n/// This test is a characterization test for the `?` operator.\n///\n/// See for more details.\n///\n/// It should fail with a single error message about the return type of the function, but instead\n/// if fails with an extra error message due to the `?` operator being used within the async block\n/// rather than the original function.\n///\n/// ```text\n/// 33 | Ok(())?;\n/// | ^ cannot use the `?` operator in an async block that returns `()`\n/// ```\n#[tokio::main]\nasync fn question_mark_operator_with_invalid_result() -> Result<(), ()> {\n Ok(())?;\n}\n\n// https://github.com/tokio-rs/tokio/issues/4635\n#[allow(redundant_semicolons)]\n#[rustfmt::skip]\n#[tokio::main]\nasync fn issue_4635() {\n return 1;\n ;\n}\n\nfn main() {}\n" }, { "path": "tests-build/tests/fail/macros_type_mismatch.stderr", "content": "error[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:3:1: 3:15}` to be a future that resolves to `()`, but it resolves to `Result<(), _>`\n --> tests/fail/macros_type_mismatch.rs:3:1\n |\n3 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `()`, found `Result<(), _>`\n |\n = note: expected unit type `()`\n found enum `Result<(), _>`\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:3:1: 3:15}` to `&dyn Future`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:5:5\n |\n5 | Ok(())\n | ^^^^^^ expected `()`, found `Result<(), _>`\n |\n = note: expected unit type `()`\n found enum `Result<(), _>`\nhelp: a return type might be missing here\n |\n4 | async fn missing_semicolon_or_return_type() -> _ {\n | ++++\nhelp: consider using `Result::expect` to unwrap the `Result<(), _>` value, panicking if the value is a `Result::Err`\n |\n5 | Ok(()).expect(\"REASON\")\n | +++++++++++++++++\n\nerror[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:8:1: 8:15}` to be a future that resolves to `()`, but it resolves to `Result<(), _>`\n --> tests/fail/macros_type_mismatch.rs:8:1\n |\n8 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `()`, found `Result<(), _>`\n |\n = note: expected unit type `()`\n found enum `Result<(), _>`\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:8:1: 8:15}` to `&dyn Future`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:10:5\n |\n10 | return Ok(());\n | ^^^^^^^^^^^^^^ expected `()`, found `Result<(), _>`\n |\n = note: expected unit type `()`\n found enum `Result<(), _>`\nhelp: a return type might be missing here\n |\n 9 | async fn missing_return_type() -> _ {\n | ++++\nhelp: consider using `Result::expect` to unwrap the `Result<(), _>` value, panicking if the value is a `Result::Err`\n |\n10 | return Ok(());.expect(\"REASON\")\n | +++++++++++++++++\n\nerror[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:13:1: 13:15}` to be a future that resolves to `Result<(), ()>`, but it resolves to `()`\n --> tests/fail/macros_type_mismatch.rs:13:1\n |\n13 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `Result<(), ()>`, found `()`\n |\n = note: expected enum `Result<(), ()>`\n found unit type `()`\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:13:1: 13:15}` to `&dyn Future>`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:23:5\n |\n14 | async fn extra_semicolon() -> Result<(), ()> {\n | -------------- expected `Result<(), ()>` because of return type\n...\n23 | Ok(());\n | ^^^^^^^ expected `Result<(), ()>`, found `()`\n |\n = note: expected enum `Result<(), ()>`\n found unit type `()`\nhelp: try adding an expression at the end of the block\n |\n23 ~ Ok(());;\n24 + Ok(())\n |\n\nerror[E0277]: the `?` operator can only be used in an async block that returns `Result` or `Option` (or another type that implements `FromResidual`)\n --> tests/fail/macros_type_mismatch.rs:40:9\n |\n38 | #[tokio::main]\n | -------------- this function should return `Result` or `Option` to accept `?`\n39 | async fn question_mark_operator_with_invalid_option() -> Option<()> {\n40 | None?;\n | ^ cannot use the `?` operator in an async block that returns `()`\n\nerror[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:38:1: 38:15}` to be a future that resolves to `Option<()>`, but it resolves to `()`\n --> tests/fail/macros_type_mismatch.rs:38:1\n |\n38 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `Option<()>`, found `()`\n |\n = note: expected enum `Option<()>`\n found unit type `()`\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:38:1: 38:15}` to `&dyn Future>`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:40:5\n |\n39 | async fn question_mark_operator_with_invalid_option() -> Option<()> {\n | ---------- expected `Option<()>` because of return type\n40 | None?;\n | ^^^^^^ expected `Option<()>`, found `()`\n |\n = note: expected enum `Option<()>`\n found unit type `()`\nhelp: try adding an expression at the end of the block\n |\n40 ~ None?;;\n41 + None\n |\n40 ~ None?;;\n41 + Some(())\n |\n\nerror[E0277]: the `?` operator can only be used in an async block that returns `Result` or `Option` (or another type that implements `FromResidual`)\n --> tests/fail/macros_type_mismatch.rs:57:11\n |\n55 | #[tokio::main]\n | -------------- this function should return `Result` or `Option` to accept `?`\n56 | async fn question_mark_operator_with_invalid_result() -> Result<(), ()> {\n57 | Ok(())?;\n | ^ cannot use the `?` operator in an async block that returns `()`\n\nerror[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:55:1: 55:15}` to be a future that resolves to `Result<(), ()>`, but it resolves to `()`\n --> tests/fail/macros_type_mismatch.rs:55:1\n |\n55 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `Result<(), ()>`, found `()`\n |\n = note: expected enum `Result<(), ()>`\n found unit type `()`\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:55:1: 55:15}` to `&dyn Future>`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:57:5\n |\n56 | async fn question_mark_operator_with_invalid_result() -> Result<(), ()> {\n | -------------- expected `Result<(), ()>` because of return type\n57 | Ok(())?;\n | ^^^^^^^^ expected `Result<(), ()>`, found `()`\n |\n = note: expected enum `Result<(), ()>`\n found unit type `()`\nhelp: try adding an expression at the end of the block\n |\n57 ~ Ok(())?;;\n58 + Ok(())\n |\n\nerror[E0271]: expected `{async block@$DIR/tests/fail/macros_type_mismatch.rs:63:1: 63:15}` to be a future that resolves to `()`, but it resolves to `{integer}`\n --> tests/fail/macros_type_mismatch.rs:63:1\n |\n63 | #[tokio::main]\n | ^^^^^^^^^^^^^^ expected `()`, found integer\n |\n = note: required for the cast from `&{async block@$DIR/tests/fail/macros_type_mismatch.rs:63:1: 63:15}` to `&dyn Future`\n = note: this error originates in the attribute macro `tokio::main` (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0308]: mismatched types\n --> tests/fail/macros_type_mismatch.rs:66:5\n |\n64 | async fn issue_4635() {\n | - help: try adding a return type: `-> i32`\n65 | return 1;\n66 | ;\n | ^ expected `()`, found integer\n" }, { "path": "tests-build/tests/macros.rs", "content": "#[test]\n#[cfg_attr(miri, ignore)]\nfn compile_fail_full() {\n let t = trybuild::TestCases::new();\n\n #[cfg(feature = \"full\")]\n t.pass(\"tests/pass/forward_args_and_output.rs\");\n\n #[cfg(feature = \"full\")]\n t.pass(\"tests/pass/macros_main_return.rs\");\n\n #[cfg(feature = \"full\")]\n t.pass(\"tests/pass/macros_main_loop.rs\");\n\n #[cfg(feature = \"full\")]\n t.pass(\"tests/pass/impl_trait.rs\");\n\n #[cfg(feature = \"full\")]\n t.pass(\"tests/pass/use_builder_outer.rs\");\n\n #[cfg(feature = \"full\")]\n t.compile_fail(\"tests/fail/macros_invalid_input.rs\");\n\n #[cfg(feature = \"full\")]\n t.compile_fail(\"tests/fail/macros_dead_code.rs\");\n\n #[cfg(feature = \"full\")]\n t.compile_fail(\"tests/fail/macros_join.rs\");\n\n #[cfg(feature = \"full\")]\n t.compile_fail(\"tests/fail/macros_try_join.rs\");\n\n #[cfg(feature = \"full\")]\n t.compile_fail(\"tests/fail/macros_type_mismatch.rs\");\n\n #[cfg(all(feature = \"rt\", not(feature = \"full\")))]\n t.compile_fail(\"tests/fail/macros_core_no_default.rs\");\n\n drop(t);\n}\n" }, { "path": "tests-build/tests/macros_clippy.rs", "content": "#[cfg(feature = \"full\")]\n#[tokio::test]\nasync fn test_with_semicolon_without_return_type() {\n #![deny(clippy::semicolon_if_nothing_returned)]\n\n dbg!(0);\n}\n" }, { "path": "tests-build/tests/pass/forward_args_and_output.rs", "content": "use tests_build::tokio;\n\nfn main() {}\n\n// arguments and output type is forwarded so other macros can access them\n\n#[tokio::test]\nasync fn test_fn_has_args(_x: u8) {}\n\n#[tokio::test]\nasync fn test_has_output() -> Result<(), Box> {\n Ok(())\n}\n" }, { "path": "tests-build/tests/pass/impl_trait.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn never() -> ! {\n loop {}\n}\n\n#[tokio::main]\nasync fn impl_trait() -> impl Iterator {\n [()].into_iter()\n}\n\n#[tokio::main]\nasync fn impl_trait2() -> Result<(), impl core::fmt::Debug> {\n Err(())\n}\n\nfn main() {\n if impl_trait().count() == 10 {\n never();\n }\n let _ = impl_trait2();\n}\n" }, { "path": "tests-build/tests/pass/macros_main_loop.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn main() -> Result<(), ()> {\n loop {\n if !never() {\n return Ok(());\n }\n }\n}\n\nfn never() -> bool {\n std::time::Instant::now() > std::time::Instant::now()\n}\n" }, { "path": "tests-build/tests/pass/macros_main_return.rs", "content": "use tests_build::tokio;\n\n#[tokio::main]\nasync fn main() -> Result<(), ()> {\n return Ok(());\n}\n" }, { "path": "tests-build/tests/pass/use_builder_outer.rs", "content": "#![deny(unused_qualifications)]\n\nuse tests_build::tokio;\npub use tokio::runtime;\n\n#[tokio::main]\nasync fn main() {\n if true {}\n}\n" }, { "path": "tests-integration/Cargo.toml", "content": "[package]\nname = \"tests-integration\"\nversion = \"0.1.0\"\nauthors = [\"Tokio Contributors \"]\nedition = \"2021\"\nlicense = \"MIT\"\npublish = false\n\n[[bin]]\nname = \"test-cat\"\n\n[[bin]]\nname = \"test-mem\"\nrequired-features = [\"rt-net\"]\n\n[[bin]]\nname = \"test-process-signal\"\nrequired-features = [\"rt-process-signal\"]\n\n[[test]]\nname = \"macros_main\"\n\n[[test]]\nname = \"macros_pin\"\n\n[[test]]\nname = \"macros_select\"\n\n[[test]]\nname = \"rt_yield\"\nrequired-features = [\"rt\", \"macros\", \"sync\"]\n\n[features]\n# For mem check\nrt-net = [\"tokio/rt\", \"tokio/rt-multi-thread\", \"tokio/net\"]\n# For test-process-signal\nrt-process-signal = [\"rt-net\", \"tokio/process\", \"tokio/signal\"]\n# For testing wasi + rt/macros/sync features\n#\n# This is an explicit feature so we can use `cargo hack` testing single features\n# instead of all possible permutations.\nwasi-rt = [\"rt\", \"macros\", \"sync\"]\nwasi-threads-rt = [\"wasi-rt\", \"rt-multi-thread\"]\n\nfull = [\n \"macros\",\n \"rt\",\n \"rt-multi-thread\",\n\n \"tokio/full\",\n \"tokio-test\"\n]\nmacros = [\"tokio/macros\"]\nsync = [\"tokio/sync\"]\nrt = [\"tokio/rt\"]\nrt-multi-thread = [\"rt\", \"tokio/rt-multi-thread\"]\n\n[dependencies]\ntokio = { version = \"1.0.0\", path = \"../tokio\" }\ntokio-test = { version = \"0.4\", path = \"../tokio-test\", optional = true }\ndoc-comment = \"0.3.1\"\nfutures = { version = \"0.3.0\", features = [\"async-await\"] }\nbytes = \"1.0.0\"\n\n[lints]\nworkspace = true\n" }, { "path": "tests-integration/README.md", "content": "Tests that require additional components than just the `tokio` crate.\n" }, { "path": "tests-integration/src/bin/test-cat.rs", "content": "//! A cat-like utility that can be used as a subprocess to test I/O\n//! stream communication.\n\nuse std::io;\nuse std::io::Write;\n\nfn main() {\n let stdin = io::stdin();\n let mut stdout = io::stdout();\n let mut line = String::new();\n loop {\n line.clear();\n stdin.read_line(&mut line).unwrap();\n if line.is_empty() {\n break;\n }\n stdout.write_all(line.as_bytes()).unwrap();\n }\n stdout.flush().unwrap();\n}\n" }, { "path": "tests-integration/src/bin/test-mem.rs", "content": "use std::future::poll_fn;\n\nfn main() {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_io()\n .build()\n .unwrap();\n\n rt.block_on(async {\n let listener = tokio::net::TcpListener::bind(\"0.0.0.0:0\").await.unwrap();\n tokio::spawn(async move {\n loop {\n poll_fn(|cx| listener.poll_accept(cx)).await.unwrap();\n }\n });\n });\n\n std::thread::sleep(std::time::Duration::from_millis(50));\n drop(rt);\n}\n" }, { "path": "tests-integration/src/bin/test-process-signal.rs", "content": "// https://github.com/tokio-rs/tokio/issues/3550\nfn main() {\n for _ in 0..1000 {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n drop(rt);\n }\n}\n" }, { "path": "tests-integration/src/lib.rs", "content": "#[cfg(feature = \"full\")]\ndoc_comment::doc_comment!(include_str!(\"../../README.md\"));\n" }, { "path": "tests-integration/tests/macros_main.rs", "content": "#![cfg(all(feature = \"macros\", feature = \"rt-multi-thread\"))]\n\n#[tokio::main]\nasync fn basic_main() -> usize {\n 1\n}\n\n#[tokio::main]\nasync fn generic_fun() -> T {\n T::default()\n}\n\n#[tokio::main]\nasync fn spawning() -> usize {\n let join = tokio::spawn(async { 1 });\n join.await.unwrap()\n}\n\n#[cfg(tokio_unstable)]\n#[tokio::main(flavor = \"local\")]\nasync fn local_main() -> usize {\n let join = tokio::task::spawn_local(async { 1 });\n join.await.unwrap()\n}\n\n#[test]\nfn main_with_spawn() {\n assert_eq!(1, spawning());\n}\n\n#[test]\nfn shell() {\n assert_eq!(1, basic_main());\n assert_eq!(bool::default(), generic_fun::());\n\n #[cfg(tokio_unstable)]\n assert_eq!(1, local_main());\n}\n" }, { "path": "tests-integration/tests/macros_pin.rs", "content": "use futures::executor::block_on;\n\nasync fn my_async_fn() {}\n\n#[test]\nfn pin() {\n block_on(async {\n let future = my_async_fn();\n tokio::pin!(future);\n (&mut future).await\n });\n}\n" }, { "path": "tests-integration/tests/macros_select.rs", "content": "#![cfg(feature = \"macros\")]\n\nuse futures::channel::oneshot;\nuse futures::executor::block_on;\nuse std::thread;\n\n#[cfg_attr(\n not(feature = \"rt-multi-thread\"),\n ignore = \"WASI: std::thread::spawn not supported\"\n)]\n#[test]\nfn join_with_select() {\n block_on(async {\n let (tx1, mut rx1) = oneshot::channel::();\n let (tx2, mut rx2) = oneshot::channel::();\n\n thread::spawn(move || {\n tx1.send(123).unwrap();\n tx2.send(456).unwrap();\n });\n\n let mut a = None;\n let mut b = None;\n\n while a.is_none() || b.is_none() {\n tokio::select! {\n v1 = (&mut rx1), if a.is_none() => a = Some(v1.unwrap()),\n v2 = (&mut rx2), if b.is_none() => b = Some(v2.unwrap()),\n }\n }\n\n let (a, b) = (a.unwrap(), b.unwrap());\n\n assert_eq!(a, 123);\n assert_eq!(b, 456);\n });\n}\n" }, { "path": "tests-integration/tests/process_stdio.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))]\n\nuse tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt, BufReader};\nuse tokio::join;\nuse tokio::process::{Child, Command};\nuse tokio_test::assert_ok;\n\nuse futures::future::{self, FutureExt};\nuse std::env;\nuse std::io;\nuse std::process::{ExitStatus, Stdio};\nuse std::task::ready;\n\nfn cat() -> Command {\n let mut cmd = Command::new(env!(\"CARGO_BIN_EXE_test-cat\"));\n cmd.stdin(Stdio::piped()).stdout(Stdio::piped());\n cmd\n}\n\nasync fn feed_cat(mut cat: Child, n: usize) -> io::Result {\n let mut stdin = cat.stdin.take().unwrap();\n let stdout = cat.stdout.take().unwrap();\n\n // Produce n lines on the child's stdout.\n let write = async {\n for i in 0..n {\n let bytes = format!(\"line {i}\\n\").into_bytes();\n stdin.write_all(&bytes).await.unwrap();\n }\n\n drop(stdin);\n };\n\n let read = async {\n let mut reader = BufReader::new(stdout).lines();\n let mut num_lines = 0;\n\n // Try to read `n + 1` lines, ensuring the last one is empty\n // (i.e. EOF is reached after `n` lines.\n loop {\n let data = reader\n .next_line()\n .await\n .unwrap_or_else(|_| Some(String::new()))\n .expect(\"failed to read line\");\n\n let num_read = data.len();\n let done = num_lines >= n;\n\n match (done, num_read) {\n (false, 0) => panic!(\"broken pipe\"),\n (true, n) if n != 0 => panic!(\"extraneous data\"),\n _ => {\n let expected = format!(\"line {num_lines}\");\n assert_eq!(expected, data);\n }\n };\n\n num_lines += 1;\n if num_lines >= n {\n break;\n }\n }\n };\n\n // Compose reading and writing concurrently.\n future::join3(write, read, cat.wait())\n .map(|(_, _, status)| status)\n .await\n}\n\n/// Check for the following properties when feeding stdin and\n/// consuming stdout of a cat-like process:\n///\n/// - A number of lines that amounts to a number of bytes exceeding a\n/// typical OS buffer size can be fed to the child without\n/// deadlock. This tests that we also consume the stdout\n/// concurrently; otherwise this would deadlock.\n///\n/// - We read the same lines from the child that we fed it.\n///\n/// - The child does produce EOF on stdout after the last line.\n#[tokio::test]\nasync fn feed_a_lot() {\n let child = cat().spawn().unwrap();\n let status = feed_cat(child, 10000).await.unwrap();\n assert_eq!(status.code(), Some(0));\n}\n\n#[tokio::test]\nasync fn wait_with_output_captures() {\n let mut child = cat().spawn().unwrap();\n let mut stdin = child.stdin.take().unwrap();\n\n let write_bytes = b\"1234\";\n\n let future = async {\n stdin.write_all(write_bytes).await?;\n drop(stdin);\n let out = child.wait_with_output();\n out.await\n };\n\n let output = future.await.unwrap();\n\n assert!(output.status.success());\n assert_eq!(output.stdout, write_bytes);\n assert_eq!(output.stderr.len(), 0);\n}\n\n#[tokio::test]\nasync fn status_closes_any_pipes() {\n // Cat will open a pipe between the parent and child.\n // If `status_async` doesn't ensure the handles are closed,\n // we would end up blocking forever (and time out).\n let child = cat().status();\n\n assert_ok!(child.await);\n}\n\n#[tokio::test]\nasync fn try_wait() {\n let mut child = cat().spawn().unwrap();\n\n let id = child.id().expect(\"missing id\");\n assert!(id > 0);\n\n assert_eq!(None, assert_ok!(child.try_wait()));\n\n // Drop the child's stdio handles so it can terminate\n drop(child.stdin.take());\n drop(child.stderr.take());\n drop(child.stdout.take());\n\n assert_ok!(child.wait().await);\n\n // test that the `.try_wait()` method is fused just like the stdlib\n assert!(assert_ok!(child.try_wait()).unwrap().success());\n\n // Can't get id after process has exited\n assert_eq!(child.id(), None);\n}\n\n#[tokio::test]\nasync fn pipe_from_one_command_to_another() {\n let mut first = cat().spawn().expect(\"first cmd\");\n let mut third = cat().spawn().expect(\"third cmd\");\n\n // Convert ChildStdout to Stdio\n let second_stdin: Stdio = first\n .stdout\n .take()\n .expect(\"first.stdout\")\n .try_into()\n .expect(\"first.stdout into Stdio\");\n\n // Convert ChildStdin to Stdio\n let second_stdout: Stdio = third\n .stdin\n .take()\n .expect(\"third.stdin\")\n .try_into()\n .expect(\"third.stdin into Stdio\");\n\n let mut second = cat()\n .stdin(second_stdin)\n .stdout(second_stdout)\n .spawn()\n .expect(\"first cmd\");\n\n let msg = \"hello world! please pipe this message through\";\n\n let mut stdin = first.stdin.take().expect(\"first.stdin\");\n let write = async move { stdin.write_all(msg.as_bytes()).await };\n\n let mut stdout = third.stdout.take().expect(\"third.stdout\");\n let read = async move {\n let mut data = String::new();\n stdout.read_to_string(&mut data).await.map(|_| data)\n };\n\n let (read, write, first_status, second_status, third_status) =\n join!(read, write, first.wait(), second.wait(), third.wait());\n\n assert_eq!(msg, read.expect(\"read result\"));\n write.expect(\"write result\");\n\n assert!(first_status.expect(\"first status\").success());\n assert!(second_status.expect(\"second status\").success());\n assert!(third_status.expect(\"third status\").success());\n}\n\n#[tokio::test]\nasync fn vectored_writes() {\n use bytes::{Buf, Bytes};\n use std::{io::IoSlice, pin::Pin};\n use tokio::io::AsyncWrite;\n\n let mut cat = cat().spawn().unwrap();\n let mut stdin = cat.stdin.take().unwrap();\n let are_writes_vectored = stdin.is_write_vectored();\n let mut stdout = cat.stdout.take().unwrap();\n\n let write = async {\n let mut input = Bytes::from_static(b\"hello\\n\").chain(Bytes::from_static(b\"world!\\n\"));\n let mut writes_completed = 0;\n\n std::future::poll_fn(|cx| loop {\n let mut slices = [IoSlice::new(&[]); 2];\n let vectored = input.chunks_vectored(&mut slices);\n if vectored == 0 {\n return std::task::Poll::Ready(std::io::Result::Ok(()));\n }\n let n = ready!(Pin::new(&mut stdin).poll_write_vectored(cx, &slices))?;\n writes_completed += 1;\n input.advance(n);\n })\n .await?;\n\n drop(stdin);\n\n std::io::Result::Ok(writes_completed)\n };\n\n let read = async {\n let mut buffer = Vec::with_capacity(6 + 7);\n stdout.read_to_end(&mut buffer).await?;\n std::io::Result::Ok(buffer)\n };\n\n let (write, read, status) = future::join3(write, read, cat.wait()).await;\n\n assert!(status.unwrap().success());\n\n let writes_completed = write.unwrap();\n // on unix our small payload should always fit in whatever default sized pipe with a single\n // syscall. if multiple are used, then the forwarding does not work, or we are on a platform\n // for which the `std` does not support vectored writes.\n assert_eq!(writes_completed == 1, are_writes_vectored);\n\n assert_eq!(&read.unwrap(), b\"hello\\nworld!\\n\");\n}\n" }, { "path": "tests-integration/tests/rt_yield.rs", "content": "use tokio::sync::oneshot;\nuse tokio::task;\n\nasync fn spawn_send() {\n let (tx, rx) = oneshot::channel();\n\n let task = tokio::spawn(async {\n for _ in 0..10 {\n task::yield_now().await;\n }\n\n tx.send(\"done\").unwrap();\n });\n\n assert_eq!(\"done\", rx.await.unwrap());\n task.await.unwrap();\n}\n\n#[tokio::main(flavor = \"current_thread\")]\nasync fn entry_point() {\n spawn_send().await;\n}\n\n#[tokio::test]\nasync fn test_macro() {\n spawn_send().await;\n}\n\n#[test]\nfn main_macro() {\n entry_point();\n}\n\n#[test]\nfn manual_rt() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n rt.block_on(async { spawn_send().await });\n}\n" }, { "path": "tokio/CHANGELOG.md", "content": "# 1.50.0 (Mar 3rd, 2026)\n\n### Added\n\n- net: add `TcpStream::set_zero_linger` ([#7837])\n- rt: add `is_rt_shutdown_err` ([#7771])\n\n### Changed\n\n- io: add optimizer hint that `memchr` returns in-bounds pointer ([#7792])\n- io: implement vectored writes for `write_buf` ([#7871])\n- runtime: panic when `event_interval` is set to 0 ([#7838])\n- runtime: shorten default thread name to fit in Linux limit ([#7880])\n- signal: remember the result of `SetConsoleCtrlHandler` ([#7833])\n- signal: specialize windows `Registry` ([#7885])\n\n### Fixed\n\n- io: always cleanup `AsyncFd` registration list on deregister ([#7773])\n- macros: remove (most) local `use` declarations in `tokio::select!` ([#7929])\n- net: fix `GET_BUF_SIZE` constant for `target_os = \"android\"` ([#7889])\n- runtime: avoid redundant unpark in current_thread scheduler ([#7834])\n- runtime: don't park in `current_thread` if `before_park` defers waker ([#7835])\n- io: fix write readiness on ESP32 on short writes ([#7872])\n- runtime: wake deferred tasks before entering `block_in_place` ([#7879])\n- sync: drop rx waker when oneshot receiver is dropped ([#7886])\n- runtime: fix double increment of `num_idle_threads` on shutdown ([#7910], [#7918], [#7922])\n\n### Unstable\n\n- fs: check for io-uring opcode support ([#7815])\n- runtime: avoid lock acquisition after uring init ([#7850])\n\n### Documented\n\n- docs: update outdated unstable features section ([#7839])\n- io: clarify the behavior of `AsyncWriteExt::shutdown()` ([#7908])\n- io: explain how to flush stdout/stderr ([#7904])\n- io: fix incorrect and confusing `AsyncWrite` documentation ([#7875])\n- rt: clarify the documentation of `Runtime::spawn` ([#7803])\n- rt: fix missing quotation in docs ([#7925])\n- runtime: correct the default thread name in docs ([#7896])\n- runtime: fix `event_interval` doc ([#7932])\n- sync: clarify RwLock fairness documentation ([#7919])\n- sync: clarify that `recv` returns `None` once closed and no more messages ([#7920])\n- task: clarify when to use `spawn_blocking` vs dedicated threads ([#7923])\n- task: doc that task drops before `JoinHandle` completion ([#7825])\n- signal: guarantee that listeners never return `None` ([#7869])\n- task: fix task module feature flags in docs ([#7891])\n- task: fix two typos ([#7913])\n- task: improve the docs of `Builder::spawn_local` ([#7828])\n- time: add docs about auto-advance and when to use sleep ([#7858])\n- util: fix typo in docs ([#7926])\n\n[#7771]: https://github.com/tokio-rs/tokio/pull/7771\n[#7773]: https://github.com/tokio-rs/tokio/pull/7773\n[#7792]: https://github.com/tokio-rs/tokio/pull/7792\n[#7803]: https://github.com/tokio-rs/tokio/pull/7803\n[#7815]: https://github.com/tokio-rs/tokio/pull/7815\n[#7825]: https://github.com/tokio-rs/tokio/pull/7825\n[#7828]: https://github.com/tokio-rs/tokio/pull/7828\n[#7833]: https://github.com/tokio-rs/tokio/pull/7833\n[#7834]: https://github.com/tokio-rs/tokio/pull/7834\n[#7835]: https://github.com/tokio-rs/tokio/pull/7835\n[#7837]: https://github.com/tokio-rs/tokio/pull/7837\n[#7838]: https://github.com/tokio-rs/tokio/pull/7838\n[#7839]: https://github.com/tokio-rs/tokio/pull/7839\n[#7850]: https://github.com/tokio-rs/tokio/pull/7850\n[#7858]: https://github.com/tokio-rs/tokio/pull/7858\n[#7869]: https://github.com/tokio-rs/tokio/pull/7869\n[#7871]: https://github.com/tokio-rs/tokio/pull/7871\n[#7872]: https://github.com/tokio-rs/tokio/pull/7872\n[#7875]: https://github.com/tokio-rs/tokio/pull/7875\n[#7879]: https://github.com/tokio-rs/tokio/pull/7879\n[#7880]: https://github.com/tokio-rs/tokio/pull/7880\n[#7885]: https://github.com/tokio-rs/tokio/pull/7885\n[#7886]: https://github.com/tokio-rs/tokio/pull/7886\n[#7889]: https://github.com/tokio-rs/tokio/pull/7889\n[#7891]: https://github.com/tokio-rs/tokio/pull/7891\n[#7896]: https://github.com/tokio-rs/tokio/pull/7896\n[#7904]: https://github.com/tokio-rs/tokio/pull/7904\n[#7908]: https://github.com/tokio-rs/tokio/pull/7908\n[#7910]: https://github.com/tokio-rs/tokio/pull/7910\n[#7913]: https://github.com/tokio-rs/tokio/pull/7913\n[#7918]: https://github.com/tokio-rs/tokio/pull/7918\n[#7919]: https://github.com/tokio-rs/tokio/pull/7919\n[#7920]: https://github.com/tokio-rs/tokio/pull/7920\n[#7922]: https://github.com/tokio-rs/tokio/pull/7922\n[#7923]: https://github.com/tokio-rs/tokio/pull/7923\n[#7925]: https://github.com/tokio-rs/tokio/pull/7925\n[#7926]: https://github.com/tokio-rs/tokio/pull/7926\n[#7929]: https://github.com/tokio-rs/tokio/pull/7929\n[#7932]: https://github.com/tokio-rs/tokio/pull/7932\n\n# 1.49.0 (January 3rd, 2026)\n\n### Added\n\n* net: add support for `TCLASS` option on IPv6 ([#7781])\n* runtime: stabilize `runtime::id::Id` ([#7125])\n* task: implement `Extend` for `JoinSet` ([#7195])\n* task: stabilize the `LocalSet::id()` ([#7776])\n\n### Changed\n\n* net: deprecate `{TcpStream,TcpSocket}::set_linger` ([#7752])\n\n### Fixed\n\n* macros: fix the hygiene issue of `join!` and `try_join!` ([#7766])\n* runtime: revert \"replace manual vtable definitions with Wake\" ([#7699])\n* sync: return `TryRecvError::Disconnected` from `Receiver::try_recv` after `Receiver::close` ([#7686])\n* task: remove unnecessary trait bounds on the `Debug` implementation ([#7720])\n\n### Unstable\n\n* fs: handle `EINTR` in `fs::write` for io-uring ([#7786])\n* fs: support io-uring with `tokio::fs::read` ([#7696])\n* runtime: disable io-uring on `EPERM` ([#7724])\n* time: add alternative timer for better multicore scalability ([#7467])\n\n### Documented\n\n* docs: fix a typos in `bounded.rs` and `park.rs` ([#7817])\n* io: add `SyncIoBridge` cross-references to `copy` and `copy_buf` ([#7798])\n* io: doc that `AsyncWrite` does not inherit from `std::io::Write` ([#7705])\n* metrics: clarify that `num_alive_tasks` is not strongly consistent ([#7614])\n* net: clarify the cancellation safety of the `TcpStream::peek` ([#7305])\n* net: clarify the drop behavior of `unix::OwnedWriteHalf` ([#7742])\n* net: clarify the platform-dependent backlog in `TcpSocket` docs ([#7738])\n* runtime: mention `LocalRuntime` in `new_current_thread` docs ([#7820])\n* sync: add missing period to `mpsc::Sender::try_send` docs ([#7721])\n* sync: clarify the cancellation safety of `oneshot::Receiver` ([#7780])\n* sync: improve the docs for the `errors` of mpsc ([#7722])\n* task: add example for `spawn_local` usage on local runtime ([#7689])\n\n[#7125]: https://github.com/tokio-rs/tokio/pull/7125\n[#7195]: https://github.com/tokio-rs/tokio/pull/7195\n[#7305]: https://github.com/tokio-rs/tokio/pull/7305\n[#7467]: https://github.com/tokio-rs/tokio/pull/7467\n[#7614]: https://github.com/tokio-rs/tokio/pull/7614\n[#7686]: https://github.com/tokio-rs/tokio/pull/7686\n[#7689]: https://github.com/tokio-rs/tokio/pull/7689\n[#7696]: https://github.com/tokio-rs/tokio/pull/7696\n[#7699]: https://github.com/tokio-rs/tokio/pull/7699\n[#7705]: https://github.com/tokio-rs/tokio/pull/7705\n[#7720]: https://github.com/tokio-rs/tokio/pull/7720\n[#7721]: https://github.com/tokio-rs/tokio/pull/7721\n[#7722]: https://github.com/tokio-rs/tokio/pull/7722\n[#7724]: https://github.com/tokio-rs/tokio/pull/7724\n[#7738]: https://github.com/tokio-rs/tokio/pull/7738\n[#7742]: https://github.com/tokio-rs/tokio/pull/7742\n[#7752]: https://github.com/tokio-rs/tokio/pull/7752\n[#7766]: https://github.com/tokio-rs/tokio/pull/7766\n[#7776]: https://github.com/tokio-rs/tokio/pull/7776\n[#7780]: https://github.com/tokio-rs/tokio/pull/7780\n[#7781]: https://github.com/tokio-rs/tokio/pull/7781\n[#7786]: https://github.com/tokio-rs/tokio/pull/7786\n[#7798]: https://github.com/tokio-rs/tokio/pull/7798\n[#7817]: https://github.com/tokio-rs/tokio/pull/7817\n[#7820]: https://github.com/tokio-rs/tokio/pull/7820\n\n# 1.48.0 (October 14th, 2025)\n\nThe MSRV is increased to 1.71.\n\n### Added\n\n- fs: add `File::max_buf_size` ([#7594])\n- io: export `Chain` of `AsyncReadExt::chain` ([#7599])\n- net: add `SocketAddr::as_abstract_name` ([#7491])\n- net: add `TcpStream::quickack` and `TcpStream::set_quickack` ([#7490])\n- net: implement `AsRef` for `TcpStream` and `UnixStream` ([#7573])\n- task: add `LocalKey::try_get` ([#7666])\n- task: implement `Ord` for `task::Id` ([#7530])\n\n### Changed\n\n- deps: bump windows-sys to version 0.61 ([#7645])\n- fs: preserve `max_buf_size` when cloning a `File` ([#7593])\n- macros: suppress `clippy::unwrap_in_result` in `#[tokio::main]` ([#7651])\n- net: remove `PollEvented` noise from Debug formats ([#7675])\n- process: upgrade `Command::spawn_with` to use `FnOnce` ([#7511])\n- sync: remove inner mutex in `SetOnce` ([#7554])\n- sync: use `UnsafeCell::get_mut` in `Mutex::get_mut` and `RwLock::get_mut` ([#7569])\n- time: reduce the generated code size of `Timeout::poll` ([#7535])\n\n### Fixed\n\n- macros: fix hygiene issue in `join!` and `try_join!` ([#7638])\n- net: fix copy/paste errors in udp peek methods ([#7604])\n- process: fix error when runtime is shut down on nightly-2025-10-12 ([#7672])\n- runtime: use release ordering in `wake_by_ref()` even if already woken ([#7622])\n- sync: close the `broadcast::Sender` in `broadcast::Sender::new()` ([#7629])\n- sync: fix implementation of unused `RwLock::try_*` methods ([#7587])\n\n### Unstable\n\n- tokio: use cargo features instead of `--cfg` flags for `taskdump` and `io_uring` ([#7655], [#7621])\n- fs: support `io_uring` in `fs::write` ([#7567])\n- fs: support `io_uring` with `File::open()` ([#7617])\n- fs: support `io_uring` with `OpenOptions` ([#7321])\n- macros: add `local` runtime flavor ([#7375], [#7597])\n\n### Documented\n\n- io: clarify the zero capacity case of `AsyncRead::poll_read` ([#7580])\n- io: fix typos in the docs of `AsyncFd` readiness guards ([#7583])\n- net: clarify socket gets closed on drop ([#7526])\n- net: clarify the behavior of `UCred::pid()` on Cygwin ([#7611])\n- net: clarify the supported platform of `set_reuseport()` and `reuseport()` ([#7628])\n- net: qualify that `SO_REUSEADDR` is only set on Unix ([#7533])\n- runtime: add guide for choosing between runtime types ([#7635])\n- runtime: clarify the behavior of `Handle::block_on` ([#7665])\n- runtime: clarify the edge case of `Builder::global_queue_interval()` ([#7605])\n- sync: clarify bounded channel panic behavior ([#7641])\n- sync: clarify the behavior of `tokio::sync::watch::Receiver` ([#7584])\n- sync: document cancel safety on `SetOnce::wait` ([#7506])\n- sync: fix the docs of `parking_lot` feature flag ([#7663])\n- sync: improve the docs of `UnboundedSender::send` ([#7661])\n- sync: improve the docs of `sync::watch` ([#7601])\n- sync: reword allocation failure paragraph in broadcast docs ([#7595])\n- task: clarify the behavior of several `spawn_local` methods ([#7669])\n- task: clarify the task ID reuse guarantees ([#7577])\n- task: improve the example of `poll_proceed` ([#7586])\n\n[#7321]: https://github.com/tokio-rs/tokio/pull/7321\n[#7375]: https://github.com/tokio-rs/tokio/pull/7375\n[#7490]: https://github.com/tokio-rs/tokio/pull/7490\n[#7491]: https://github.com/tokio-rs/tokio/pull/7491\n[#7494]: https://github.com/tokio-rs/tokio/pull/7494\n[#7506]: https://github.com/tokio-rs/tokio/pull/7506\n[#7511]: https://github.com/tokio-rs/tokio/pull/7511\n[#7526]: https://github.com/tokio-rs/tokio/pull/7526\n[#7530]: https://github.com/tokio-rs/tokio/pull/7530\n[#7533]: https://github.com/tokio-rs/tokio/pull/7533\n[#7535]: https://github.com/tokio-rs/tokio/pull/7535\n[#7554]: https://github.com/tokio-rs/tokio/pull/7554\n[#7567]: https://github.com/tokio-rs/tokio/pull/7567\n[#7569]: https://github.com/tokio-rs/tokio/pull/7569\n[#7573]: https://github.com/tokio-rs/tokio/pull/7573\n[#7577]: https://github.com/tokio-rs/tokio/pull/7577\n[#7580]: https://github.com/tokio-rs/tokio/pull/7580\n[#7583]: https://github.com/tokio-rs/tokio/pull/7583\n[#7584]: https://github.com/tokio-rs/tokio/pull/7584\n[#7586]: https://github.com/tokio-rs/tokio/pull/7586\n[#7587]: https://github.com/tokio-rs/tokio/pull/7587\n[#7593]: https://github.com/tokio-rs/tokio/pull/7593\n[#7594]: https://github.com/tokio-rs/tokio/pull/7594\n[#7595]: https://github.com/tokio-rs/tokio/pull/7595\n[#7597]: https://github.com/tokio-rs/tokio/pull/7597\n[#7599]: https://github.com/tokio-rs/tokio/pull/7599\n[#7601]: https://github.com/tokio-rs/tokio/pull/7601\n[#7604]: https://github.com/tokio-rs/tokio/pull/7604\n[#7605]: https://github.com/tokio-rs/tokio/pull/7605\n[#7611]: https://github.com/tokio-rs/tokio/pull/7611\n[#7617]: https://github.com/tokio-rs/tokio/pull/7617\n[#7621]: https://github.com/tokio-rs/tokio/pull/7621\n[#7622]: https://github.com/tokio-rs/tokio/pull/7622\n[#7628]: https://github.com/tokio-rs/tokio/pull/7628\n[#7629]: https://github.com/tokio-rs/tokio/pull/7629\n[#7635]: https://github.com/tokio-rs/tokio/pull/7635\n[#7638]: https://github.com/tokio-rs/tokio/pull/7638\n[#7641]: https://github.com/tokio-rs/tokio/pull/7641\n[#7645]: https://github.com/tokio-rs/tokio/pull/7645\n[#7651]: https://github.com/tokio-rs/tokio/pull/7651\n[#7655]: https://github.com/tokio-rs/tokio/pull/7655\n[#7661]: https://github.com/tokio-rs/tokio/pull/7661\n[#7663]: https://github.com/tokio-rs/tokio/pull/7663\n[#7665]: https://github.com/tokio-rs/tokio/pull/7665\n[#7666]: https://github.com/tokio-rs/tokio/pull/7666\n[#7669]: https://github.com/tokio-rs/tokio/pull/7669\n[#7672]: https://github.com/tokio-rs/tokio/pull/7672\n[#7675]: https://github.com/tokio-rs/tokio/pull/7675\n\n# 1.47.3 (Januar 3rd, 2026)\n\n### Fixed\n\n* sync: return `TryRecvError::Disconnected` from `Receiver::try_recv` after `Receiver::close` ([#7686])\n\n# 1.47.2 (October 14th, 2025)\n\n### Fixed\n\n- runtime: use release ordering in `wake_by_ref()` even if already woken ([#7622])\n- sync: close the `broadcast::Sender` in `broadcast::Sender::new()` ([#7629])\n- macros: fix hygiene issue in `join!` and `try_join!` ([#7638])\n- process: fix error when runtime is shut down on nightly-2025-10-12 ([#7672])\n\n[#7622]: https://github.com/tokio-rs/tokio/pull/7622\n[#7629]: https://github.com/tokio-rs/tokio/pull/7629\n[#7638]: https://github.com/tokio-rs/tokio/pull/7638\n[#7672]: https://github.com/tokio-rs/tokio/pull/7672\n\n# 1.47.1 (August 1st, 2025)\n\n### Fixed\n\n- process: fix panic from spurious pidfd wakeup ([#7494])\n- sync: fix broken link of Python `asyncio.Event` in `SetOnce` docs ([#7485])\n\n[#7485]: https://github.com/tokio-rs/tokio/pull/7485\n\n# 1.47.0 (July 25th, 2025)\n\nThis release adds `poll_proceed` and `cooperative` to the `coop` module for\ncooperative scheduling, adds `SetOnce` to the `sync` module which provides\nsimilar functionality to [`std::sync::OnceLock`], and adds a new method\n`sync::Notify::notified_owned()` which returns an `OwnedNotified` without\na lifetime parameter.\n\n## Added\n\n- coop: add `cooperative` and `poll_proceed` ([#7405])\n- sync: add `SetOnce` ([#7418])\n- sync: add `sync::Notify::notified_owned()` ([#7465])\n\n## Changed\n\n- deps: upgrade windows-sys 0.52 → 0.59 ([#7117])\n- deps: update to socket2 v0.6 ([#7443])\n- sync: improve `AtomicWaker::wake` performance ([#7450])\n\n## Documented\n\n- metrics: fix listed feature requirements for some metrics ([#7449])\n- runtime: improve safety comments of `Readiness<'_>` ([#7415])\n\n[#7117]: https://github.com/tokio-rs/tokio/pull/7117\n[#7405]: https://github.com/tokio-rs/tokio/pull/7405\n[#7415]: https://github.com/tokio-rs/tokio/pull/7415\n[#7418]: https://github.com/tokio-rs/tokio/pull/7418\n[#7443]: https://github.com/tokio-rs/tokio/pull/7443\n[#7449]: https://github.com/tokio-rs/tokio/pull/7449\n[#7450]: https://github.com/tokio-rs/tokio/pull/7450\n[#7465]: https://github.com/tokio-rs/tokio/pull/7465\n\n# 1.46.1 (July 4th, 2025)\n\nThis release fixes incorrect spawn locations in runtime task hooks for tasks\nspawned using `tokio::spawn` rather than `Runtime::spawn`. This issue only\naffected the spawn location in `TaskMeta::spawned_at`, and did not affect task\nlocations in Tracing events.\n\n## Unstable\n\n- runtime: add `TaskMeta::spawned_at` tracking where a task was spawned\n ([#7440])\n\n[#7440]: https://github.com/tokio-rs/tokio/pull/7440\n\n# 1.46.0 (July 2nd, 2025)\n\n## Fixed\n\n- net: fixed `TcpStream::shutdown` incorrectly returning an error on macOS ([#7290])\n\n## Added\n\n- sync: `mpsc::OwnedPermit::{same_channel, same_channel_as_sender}` methods ([#7389])\n- macros: `biased` option for `join!` and `try_join!`, similar to `select!` ([#7307])\n- net: support for cygwin ([#7393])\n- net: support `pipe::OpenOptions::read_write` on Android ([#7426])\n- net: add `Clone` implementation for `net::unix::SocketAddr` ([#7422])\n\n## Changed\n\n- runtime: eliminate unnecessary lfence while operating on `queue::Local` ([#7340])\n- task: disallow blocking in `LocalSet::{poll,drop}` ([#7372])\n\n## Unstable\n\n- runtime: add `TaskMeta::spawn_location` tracking where a task was spawned ([#7417])\n- runtime: removed borrow from `LocalOptions` parameter to `runtime::Builder::build_local` ([#7346])\n\n## Documented\n\n- io: clarify behavior of seeking when `start_seek` is not used ([#7366])\n- io: document cancellation safety of `AsyncWriteExt::flush` ([#7364])\n- net: fix docs for `recv_buffer_size` method ([#7336])\n- net: fix broken link of `RawFd` in `TcpSocket` docs ([#7416])\n- net: update `AsRawFd` doc link to current Rust stdlib location ([#7429])\n- readme: fix double period in reactor description ([#7363])\n- runtime: add doc note that `on_*_task_poll` is unstable ([#7311])\n- sync: update broadcast docs on allocation failure ([#7352])\n- time: add a missing panic scenario of `time::advance` ([#7394])\n\n[#7290]: https://github.com/tokio-rs/tokio/pull/7290\n[#7307]: https://github.com/tokio-rs/tokio/pull/7307\n[#7311]: https://github.com/tokio-rs/tokio/pull/7311\n[#7336]: https://github.com/tokio-rs/tokio/pull/7336\n[#7340]: https://github.com/tokio-rs/tokio/pull/7340\n[#7346]: https://github.com/tokio-rs/tokio/pull/7346\n[#7352]: https://github.com/tokio-rs/tokio/pull/7352\n[#7363]: https://github.com/tokio-rs/tokio/pull/7363\n[#7364]: https://github.com/tokio-rs/tokio/pull/7364\n[#7366]: https://github.com/tokio-rs/tokio/pull/7366\n[#7372]: https://github.com/tokio-rs/tokio/pull/7372\n[#7389]: https://github.com/tokio-rs/tokio/pull/7389\n[#7393]: https://github.com/tokio-rs/tokio/pull/7393\n[#7394]: https://github.com/tokio-rs/tokio/pull/7394\n[#7416]: https://github.com/tokio-rs/tokio/pull/7416\n[#7422]: https://github.com/tokio-rs/tokio/pull/7422\n[#7426]: https://github.com/tokio-rs/tokio/pull/7426\n[#7429]: https://github.com/tokio-rs/tokio/pull/7429\n[#7417]: https://github.com/tokio-rs/tokio/pull/7417\n\n# 1.45.1 (May 24th, 2025)\n\nThis fixes a regression on the wasm32-unknown-unknown target, where code that\npreviously did not panic due to calls to `Instant::now()` started failing. This\nis due to the stabilization of the first time-based metric.\n\n### Fixed\n\n- Disable time-based metrics on wasm32-unknown-unknown ([#7322])\n\n[#7322]: https://github.com/tokio-rs/tokio/pull/7322\n\n# 1.45.0 (May 5th, 2025)\n\n### Added\n\n- metrics: stabilize `worker_total_busy_duration`, `worker_park_count`, and\n `worker_unpark_count` ([#6899], [#7276])\n- process: add `Command::spawn_with` ([#7249])\n\n### Changed\n\n- io: do not require `Unpin` for some trait impls ([#7204])\n- rt: mark `runtime::Handle` as unwind safe ([#7230])\n- time: revert internal sharding implementation ([#7226])\n\n### Unstable\n\n- rt: remove alt multi-threaded runtime ([#7275])\n\n[#6899]: https://github.com/tokio-rs/tokio/pull/6899\n[#7276]: https://github.com/tokio-rs/tokio/pull/7276\n[#7249]: https://github.com/tokio-rs/tokio/pull/7249\n[#7204]: https://github.com/tokio-rs/tokio/pull/7204\n[#7230]: https://github.com/tokio-rs/tokio/pull/7230\n[#7226]: https://github.com/tokio-rs/tokio/pull/7226\n[#7275]: https://github.com/tokio-rs/tokio/pull/7275\n\n\n# 1.44.2 (April 5th, 2025)\n\nThis release fixes a soundness issue in the broadcast channel. The channel\naccepts values that are `Send` but `!Sync`. Previously, the channel called\n`clone()` on these values without synchronizing. This release fixes the channel\nby synchronizing calls to `.clone()` (Thanks Austin Bonander for finding and\nreporting the issue).\n\n### Fixed\n\n- sync: synchronize `clone()` call in broadcast channel ([#7232])\n\n[#7232]: https://github.com/tokio-rs/tokio/pull/7232\n\n# 1.44.1 (March 13th, 2025)\n\n### Fixed\n\n- rt: skip defer queue in `block_in_place` context ([#7216])\n\n[#7216]: https://github.com/tokio-rs/tokio/pull/7216\n\n# 1.44.0 (March 7th, 2025)\n\nThis release changes the `from_std` method on sockets to panic if a blocking\nsocket is provided. We determined this change is not a breaking change as Tokio is not\nintended to operate using blocking sockets. Doing so results in runtime hangs and\nshould be considered a bug. Accidentally passing a blocking socket to Tokio is one\nof the most common user mistakes. If this change causes an issue for you, please\ncomment on [#7172].\n\n### Added\n\n - coop: add `task::coop` module ([#7116])\n - process: add `Command::get_kill_on_drop()` ([#7086])\n - sync: add `broadcast::Sender::closed` ([#6685], [#7090])\n - sync: add `broadcast::WeakSender` ([#7100])\n - sync: add `oneshot::Receiver::is_empty()` ([#7153])\n - sync: add `oneshot::Receiver::is_terminated()` ([#7152])\n\n### Fixed\n\n - fs: empty reads on `File` should not start a background read ([#7139])\n - process: calling `start_kill` on exited child should not fail ([#7160])\n - signal: fix `CTRL_CLOSE`, `CTRL_LOGOFF`, `CTRL_SHUTDOWN` on windows ([#7122])\n - sync: properly handle panic during mpsc drop ([#7094])\n\n### Changes\n\n - runtime: clean up magic number in registration set ([#7112])\n - coop: make coop yield using waker defer strategy ([#7185])\n - macros: make `select!` budget-aware ([#7164])\n - net: panic when passing a blocking socket to `from_std` ([#7166])\n - io: clean up buffer casts ([#7142])\n\n### Changes to unstable APIs\n\n - rt: add before and after task poll callbacks ([#7120])\n - tracing: make the task tracing API unstable public ([#6972])\n\n### Documented\n\n - docs: fix nesting of sections in top-level docs ([#7159])\n - fs: rename symlink and hardlink parameter names ([#7143])\n - io: swap reader/writer in simplex doc test ([#7176])\n - macros: docs about `select!` alternatives ([#7110])\n - net: rename the argument for `send_to` ([#7146])\n - process: add example for reading `Child` stdout ([#7141])\n - process: clarify `Child::kill` behavior ([#7162])\n - process: fix grammar of the `ChildStdin` struct doc comment ([#7192])\n - runtime: consistently use `worker_threads` instead of `core_threads` ([#7186])\n\n[#6685]: https://github.com/tokio-rs/tokio/pull/6685\n[#6972]: https://github.com/tokio-rs/tokio/pull/6972\n[#7086]: https://github.com/tokio-rs/tokio/pull/7086\n[#7090]: https://github.com/tokio-rs/tokio/pull/7090\n[#7094]: https://github.com/tokio-rs/tokio/pull/7094\n[#7100]: https://github.com/tokio-rs/tokio/pull/7100\n[#7110]: https://github.com/tokio-rs/tokio/pull/7110\n[#7112]: https://github.com/tokio-rs/tokio/pull/7112\n[#7116]: https://github.com/tokio-rs/tokio/pull/7116\n[#7120]: https://github.com/tokio-rs/tokio/pull/7120\n[#7122]: https://github.com/tokio-rs/tokio/pull/7122\n[#7139]: https://github.com/tokio-rs/tokio/pull/7139\n[#7141]: https://github.com/tokio-rs/tokio/pull/7141\n[#7142]: https://github.com/tokio-rs/tokio/pull/7142\n[#7143]: https://github.com/tokio-rs/tokio/pull/7143\n[#7146]: https://github.com/tokio-rs/tokio/pull/7146\n[#7152]: https://github.com/tokio-rs/tokio/pull/7152\n[#7153]: https://github.com/tokio-rs/tokio/pull/7153\n[#7159]: https://github.com/tokio-rs/tokio/pull/7159\n[#7160]: https://github.com/tokio-rs/tokio/pull/7160\n[#7162]: https://github.com/tokio-rs/tokio/pull/7162\n[#7164]: https://github.com/tokio-rs/tokio/pull/7164\n[#7166]: https://github.com/tokio-rs/tokio/pull/7166\n[#7172]: https://github.com/tokio-rs/tokio/pull/7172\n[#7176]: https://github.com/tokio-rs/tokio/pull/7176\n[#7185]: https://github.com/tokio-rs/tokio/pull/7185\n[#7186]: https://github.com/tokio-rs/tokio/pull/7186\n[#7192]: https://github.com/tokio-rs/tokio/pull/7192\n\n# 1.43.4 (January 3rd, 2026)\n\n### Fixed\n\n* sync: return `TryRecvError::Disconnected` from `Receiver::try_recv` after `Receiver::close` ([#7686])\n\n[#7686]: https://github.com/tokio-rs/tokio/pull/7686\n\n# 1.43.3 (October 14th, 2025)\n\n### Fixed\n\n- runtime: use release ordering in `wake_by_ref()` even if already woken ([#7622])\n- sync: close the `broadcast::Sender` in `broadcast::Sender::new()` ([#7629])\n- process: fix error when runtime is shut down on nightly-2025-10-12 ([#7672])\n\n[#7622]: https://github.com/tokio-rs/tokio/pull/7622\n[#7629]: https://github.com/tokio-rs/tokio/pull/7629\n[#7672]: https://github.com/tokio-rs/tokio/pull/7672\n\n# 1.43.2 (August 1st, 2025)\n\n### Fixed\n\n- process: fix panic from spurious pidfd wakeup ([#7494])\n\n[#7494]: https://github.com/tokio-rs/tokio/pull/7494\n\n# 1.43.1 (April 5th, 2025)\n\nThis release fixes a soundness issue in the broadcast channel. The channel\naccepts values that are `Send` but `!Sync`. Previously, the channel called\n`clone()` on these values without synchronizing. This release fixes the channel\nby synchronizing calls to `.clone()` (Thanks Austin Bonander for finding and\nreporting the issue).\n\n### Fixed\n\n- sync: synchronize `clone()` call in broadcast channel ([#7232])\n\n[#7232]: https://github.com/tokio-rs/tokio/pull/7232\n\n# 1.43.0 (Jan 8th, 2025)\n\n### Added\n\n- net: add `UdpSocket::peek` methods ([#7068])\n- net: add support for Haiku OS ([#7042])\n- process: add `Command::into_std()` ([#7014])\n- signal: add `SignalKind::info` on illumos ([#6995])\n- signal: add support for realtime signals on illumos ([#7029])\n\n### Fixed\n\n- io: don't call `set_len` before initializing vector in `Blocking` ([#7054])\n- macros: suppress `clippy::needless_return` in `#[tokio::main]` ([#6874])\n- runtime: fix thread parking on WebAssembly ([#7041])\n\n### Changes\n\n- chore: use unsync loads for `unsync_load` ([#7073])\n- io: use `Buf::put_bytes` in `Repeat` read impl ([#7055])\n- task: drop the join waker of a task eagerly ([#6986])\n\n### Changes to unstable APIs\n\n- metrics: improve flexibility of H2Histogram Configuration ([#6963])\n- taskdump: add accessor methods for backtrace ([#6975])\n\n### Documented\n\n- io: clarify `ReadBuf::uninit` allows initialized buffers as well ([#7053])\n- net: fix ambiguity in `TcpStream::try_write_vectored` docs ([#7067])\n- runtime: fix `LocalRuntime` doc links ([#7074])\n- sync: extend documentation for `watch::Receiver::wait_for` ([#7038])\n- sync: fix typos in `OnceCell` docs ([#7047])\n\n[#6874]: https://github.com/tokio-rs/tokio/pull/6874\n[#6963]: https://github.com/tokio-rs/tokio/pull/6963\n[#6975]: https://github.com/tokio-rs/tokio/pull/6975\n[#6986]: https://github.com/tokio-rs/tokio/pull/6986\n[#6995]: https://github.com/tokio-rs/tokio/pull/6995\n[#7014]: https://github.com/tokio-rs/tokio/pull/7014\n[#7029]: https://github.com/tokio-rs/tokio/pull/7029\n[#7038]: https://github.com/tokio-rs/tokio/pull/7038\n[#7041]: https://github.com/tokio-rs/tokio/pull/7041\n[#7042]: https://github.com/tokio-rs/tokio/pull/7042\n[#7047]: https://github.com/tokio-rs/tokio/pull/7047\n[#7053]: https://github.com/tokio-rs/tokio/pull/7053\n[#7054]: https://github.com/tokio-rs/tokio/pull/7054\n[#7055]: https://github.com/tokio-rs/tokio/pull/7055\n[#7067]: https://github.com/tokio-rs/tokio/pull/7067\n[#7068]: https://github.com/tokio-rs/tokio/pull/7068\n[#7073]: https://github.com/tokio-rs/tokio/pull/7073\n[#7074]: https://github.com/tokio-rs/tokio/pull/7074\n\n# 1.42.1 (April 8th, 2025)\n\nThis release fixes a soundness issue in the broadcast channel. The channel\naccepts values that are `Send` but `!Sync`. Previously, the channel called\n`clone()` on these values without synchronizing. This release fixes the channel\nby synchronizing calls to `.clone()` (Thanks Austin Bonander for finding and\nreporting the issue).\n\n### Fixed\n\n- sync: synchronize `clone()` call in broadcast channel ([#7232])\n\n[#7232]: https://github.com/tokio-rs/tokio/pull/7232\n\n# 1.42.0 (Dec 3rd, 2024)\n\n### Added\n\n- io: add `AsyncFd::{try_io, try_io_mut}` ([#6967])\n\n### Fixed\n\n- io: avoid `ptr->ref->ptr` roundtrip in RegistrationSet ([#6929])\n- runtime: do not defer `yield_now` inside `block_in_place` ([#6999])\n\n### Changes\n\n- io: simplify io readiness logic ([#6966])\n\n### Documented\n\n- net: fix docs for `tokio::net::unix::{pid_t, gid_t, uid_t}` ([#6791])\n- time: fix a typo in `Instant` docs ([#6982])\n\n[#6791]: https://github.com/tokio-rs/tokio/pull/6791\n[#6929]: https://github.com/tokio-rs/tokio/pull/6929\n[#6966]: https://github.com/tokio-rs/tokio/pull/6966\n[#6967]: https://github.com/tokio-rs/tokio/pull/6967\n[#6982]: https://github.com/tokio-rs/tokio/pull/6982\n[#6999]: https://github.com/tokio-rs/tokio/pull/6999\n\n# 1.41.1 (Nov 7th, 2024)\n\n### Fixed\n\n- metrics: fix bug with wrong number of buckets for the histogram ([#6957])\n- net: display `net` requirement for `net::UdpSocket` in docs ([#6938])\n- net: fix typo in `TcpStream` internal comment ([#6944])\n\n[#6957]: https://github.com/tokio-rs/tokio/pull/6957\n[#6938]: https://github.com/tokio-rs/tokio/pull/6938\n[#6944]: https://github.com/tokio-rs/tokio/pull/6944\n\n# 1.41.0 (Oct 22nd, 2024)\n\n### Added\n\n- metrics: stabilize `global_queue_depth` ([#6854], [#6918])\n- net: add conversions for unix `SocketAddr` ([#6868])\n- sync: add `watch::Sender::sender_count` ([#6836])\n- sync: add `mpsc::Receiver::blocking_recv_many` ([#6867])\n- task: stabilize `Id` apis ([#6793], [#6891])\n\n### Added (unstable)\n\n- metrics: add H2 Histogram option to improve histogram granularity ([#6897])\n- metrics: rename some histogram apis ([#6924])\n- runtime: add `LocalRuntime` ([#6808])\n\n### Changed\n\n- runtime: box futures larger than 16k on release mode ([#6826])\n- sync: add `#[must_use]` to `Notified` ([#6828])\n- sync: make `watch` cooperative ([#6846])\n- sync: make `broadcast::Receiver` cooperative ([#6870])\n- task: add task size to tracing instrumentation ([#6881])\n- wasm: enable `cfg_fs` for `wasi` target ([#6822])\n\n### Fixed\n\n- net: fix regression of abstract socket path in unix socket ([#6838])\n\n### Documented\n\n- io: recommend `OwnedFd` with `AsyncFd` ([#6821])\n- io: document cancel safety of `AsyncFd` methods ([#6890])\n- macros: render more comprehensible documentation for `join` and `try_join` ([#6814], [#6841])\n- net: fix swapped examples for `TcpSocket::set_nodelay` and `TcpSocket::nodelay` ([#6840])\n- sync: document runtime compatibility ([#6833])\n\n[#6793]: https://github.com/tokio-rs/tokio/pull/6793\n[#6808]: https://github.com/tokio-rs/tokio/pull/6808\n[#6810]: https://github.com/tokio-rs/tokio/pull/6810\n[#6814]: https://github.com/tokio-rs/tokio/pull/6814\n[#6821]: https://github.com/tokio-rs/tokio/pull/6821\n[#6822]: https://github.com/tokio-rs/tokio/pull/6822\n[#6826]: https://github.com/tokio-rs/tokio/pull/6826\n[#6828]: https://github.com/tokio-rs/tokio/pull/6828\n[#6833]: https://github.com/tokio-rs/tokio/pull/6833\n[#6836]: https://github.com/tokio-rs/tokio/pull/6836\n[#6838]: https://github.com/tokio-rs/tokio/pull/6838\n[#6840]: https://github.com/tokio-rs/tokio/pull/6840\n[#6841]: https://github.com/tokio-rs/tokio/pull/6841\n[#6846]: https://github.com/tokio-rs/tokio/pull/6846\n[#6854]: https://github.com/tokio-rs/tokio/pull/6854\n[#6867]: https://github.com/tokio-rs/tokio/pull/6867\n[#6868]: https://github.com/tokio-rs/tokio/pull/6868\n[#6870]: https://github.com/tokio-rs/tokio/pull/6870\n[#6881]: https://github.com/tokio-rs/tokio/pull/6881\n[#6890]: https://github.com/tokio-rs/tokio/pull/6890\n[#6891]: https://github.com/tokio-rs/tokio/pull/6891\n[#6897]: https://github.com/tokio-rs/tokio/pull/6897\n[#6918]: https://github.com/tokio-rs/tokio/pull/6918\n[#6924]: https://github.com/tokio-rs/tokio/pull/6924\n\n# 1.40.0 (August 30th, 2024)\n\n### Added\n\n- io: add `util::SimplexStream` ([#6589])\n- process: stabilize `Command::process_group` ([#6731])\n- sync: add `{TrySendError,SendTimeoutError}::into_inner` ([#6755])\n- task: add `JoinSet::join_all` ([#6784])\n\n### Added (unstable)\n\n- runtime: add `Builder::{on_task_spawn, on_task_terminate}` ([#6742])\n\n### Changed\n\n- io: use vectored io for `write_all_buf` when possible ([#6724])\n- runtime: prevent niche-optimization to avoid triggering miri ([#6744])\n- sync: mark mpsc types as `UnwindSafe` ([#6783])\n- sync,time: make `Sleep` and `BatchSemaphore` instrumentation explicit roots ([#6727])\n- task: use `NonZeroU64` for `task::Id` ([#6733])\n- task: include panic message when printing `JoinError` ([#6753])\n- task: add `#[must_use]` to `JoinHandle::abort_handle` ([#6762])\n- time: eliminate timer wheel allocations ([#6779])\n\n### Documented\n\n- docs: clarify that `[build]` section doesn't go in Cargo.toml ([#6728])\n- io: clarify zero remaining capacity case ([#6790])\n- macros: improve documentation for `select!` ([#6774])\n- sync: document mpsc channel allocation behavior ([#6773])\n\n[#6589]: https://github.com/tokio-rs/tokio/pull/6589\n[#6724]: https://github.com/tokio-rs/tokio/pull/6724\n[#6727]: https://github.com/tokio-rs/tokio/pull/6727\n[#6728]: https://github.com/tokio-rs/tokio/pull/6728\n[#6731]: https://github.com/tokio-rs/tokio/pull/6731\n[#6733]: https://github.com/tokio-rs/tokio/pull/6733\n[#6742]: https://github.com/tokio-rs/tokio/pull/6742\n[#6744]: https://github.com/tokio-rs/tokio/pull/6744\n[#6753]: https://github.com/tokio-rs/tokio/pull/6753\n[#6755]: https://github.com/tokio-rs/tokio/pull/6755\n[#6762]: https://github.com/tokio-rs/tokio/pull/6762\n[#6773]: https://github.com/tokio-rs/tokio/pull/6773\n[#6774]: https://github.com/tokio-rs/tokio/pull/6774\n[#6779]: https://github.com/tokio-rs/tokio/pull/6779\n[#6783]: https://github.com/tokio-rs/tokio/pull/6783\n[#6784]: https://github.com/tokio-rs/tokio/pull/6784\n[#6790]: https://github.com/tokio-rs/tokio/pull/6790\n\n# 1.39.3 (August 17th, 2024)\n\nThis release fixes a regression where the unix socket api stopped accepting\nthe abstract socket namespace. ([#6772])\n\n[#6772]: https://github.com/tokio-rs/tokio/pull/6772\n\n# 1.39.2 (July 27th, 2024)\n\nThis release fixes a regression where the `select!` macro stopped accepting\nexpressions that make use of temporary lifetime extension. ([#6722])\n\n[#6722]: https://github.com/tokio-rs/tokio/pull/6722\n\n# 1.39.1 (July 23rd, 2024)\n\nThis release reverts \"time: avoid traversing entries in the time wheel twice\"\nbecause it contains a bug. ([#6715])\n\n[#6715]: https://github.com/tokio-rs/tokio/pull/6715\n\n# 1.39.0 (July 23rd, 2024)\n\nYanked. Please use 1.39.1 instead.\n\n- This release bumps the MSRV to 1.70. ([#6645])\n- This release upgrades to mio v1. ([#6635])\n- This release upgrades to windows-sys v0.52 ([#6154])\n\n### Added\n\n- io: implement `AsyncSeek` for `Empty` ([#6663])\n- metrics: stabilize `num_alive_tasks` ([#6619], [#6667])\n- process: add `Command::as_std_mut` ([#6608])\n- sync: add `watch::Sender::same_channel` ([#6637])\n- sync: add `{Receiver,UnboundedReceiver}::{sender_strong_count,sender_weak_count}` ([#6661])\n- sync: implement `Default` for `watch::Sender` ([#6626])\n- task: implement `Clone` for `AbortHandle` ([#6621])\n- task: stabilize `consume_budget` ([#6622])\n\n### Changed\n\n- io: improve panic message of `ReadBuf::put_slice()` ([#6629])\n- io: read during write in `copy_bidirectional` and `copy` ([#6532])\n- runtime: replace `num_cpus` with `available_parallelism` ([#6709])\n- task: avoid stack overflow when passing large future to `block_on` ([#6692])\n- time: avoid traversing entries in the time wheel twice ([#6584])\n- time: support `IntoFuture` with `timeout` ([#6666])\n- macros: support `IntoFuture` with `join!` and `select!` ([#6710])\n\n### Fixed\n\n- docs: fix docsrs builds with the fs feature enabled ([#6585])\n- io: only use short-read optimization on known-to-be-compatible platforms ([#6668])\n- time: fix overflow panic when using large durations with `Interval` ([#6612])\n\n### Added (unstable)\n\n- macros: allow `unhandled_panic` behavior for `#[tokio::main]` and `#[tokio::test]` ([#6593])\n- metrics: add `spawned_tasks_count` ([#6114])\n- metrics: add `worker_park_unpark_count` ([#6696])\n- metrics: add worker thread id ([#6695])\n\n### Documented\n\n- io: update `tokio::io::stdout` documentation ([#6674])\n- macros: typo fix in `join.rs` and `try_join.rs` ([#6641])\n- runtime: fix typo in `unhandled_panic` ([#6660])\n- task: document behavior of `JoinSet::try_join_next` when all tasks are running ([#6671])\n\n[#6114]: https://github.com/tokio-rs/tokio/pull/6114\n[#6154]: https://github.com/tokio-rs/tokio/pull/6154\n[#6532]: https://github.com/tokio-rs/tokio/pull/6532\n[#6584]: https://github.com/tokio-rs/tokio/pull/6584\n[#6585]: https://github.com/tokio-rs/tokio/pull/6585\n[#6593]: https://github.com/tokio-rs/tokio/pull/6593\n[#6608]: https://github.com/tokio-rs/tokio/pull/6608\n[#6612]: https://github.com/tokio-rs/tokio/pull/6612\n[#6619]: https://github.com/tokio-rs/tokio/pull/6619\n[#6621]: https://github.com/tokio-rs/tokio/pull/6621\n[#6622]: https://github.com/tokio-rs/tokio/pull/6622\n[#6626]: https://github.com/tokio-rs/tokio/pull/6626\n[#6629]: https://github.com/tokio-rs/tokio/pull/6629\n[#6635]: https://github.com/tokio-rs/tokio/pull/6635\n[#6637]: https://github.com/tokio-rs/tokio/pull/6637\n[#6641]: https://github.com/tokio-rs/tokio/pull/6641\n[#6645]: https://github.com/tokio-rs/tokio/pull/6645\n[#6660]: https://github.com/tokio-rs/tokio/pull/6660\n[#6661]: https://github.com/tokio-rs/tokio/pull/6661\n[#6663]: https://github.com/tokio-rs/tokio/pull/6663\n[#6666]: https://github.com/tokio-rs/tokio/pull/6666\n[#6667]: https://github.com/tokio-rs/tokio/pull/6667\n[#6668]: https://github.com/tokio-rs/tokio/pull/6668\n[#6671]: https://github.com/tokio-rs/tokio/pull/6671\n[#6674]: https://github.com/tokio-rs/tokio/pull/6674\n[#6692]: https://github.com/tokio-rs/tokio/pull/6692\n[#6695]: https://github.com/tokio-rs/tokio/pull/6695\n[#6696]: https://github.com/tokio-rs/tokio/pull/6696\n[#6709]: https://github.com/tokio-rs/tokio/pull/6709\n[#6710]: https://github.com/tokio-rs/tokio/pull/6710\n\n# 1.38.2 (April 2nd, 2025)\n\nThis release fixes a soundness issue in the broadcast channel. The channel\naccepts values that are `Send` but `!Sync`. Previously, the channel called\n`clone()` on these values without synchronizing. This release fixes the channel\nby synchronizing calls to `.clone()` (Thanks Austin Bonander for finding and\nreporting the issue).\n\n### Fixed\n\n- sync: synchronize `clone()` call in broadcast channel ([#7232])\n\n[#7232]: https://github.com/tokio-rs/tokio/pull/7232\n\n# 1.38.1 (July 16th, 2024)\n\nThis release fixes the bug identified as ([#6682]), which caused timers not\nto fire when they should.\n\n### Fixed\n\n- time: update `wake_up` while holding all the locks of sharded time wheels ([#6683])\n\n[#6682]: https://github.com/tokio-rs/tokio/pull/6682\n[#6683]: https://github.com/tokio-rs/tokio/pull/6683\n\n# 1.38.0 (May 30th, 2024)\n\nThis release marks the beginning of stabilization for runtime metrics. It\nstabilizes `RuntimeMetrics::worker_count`. Future releases will continue to\nstabilize more metrics.\n\n### Added\n\n- fs: add `File::create_new` ([#6573])\n- io: add `copy_bidirectional_with_sizes` ([#6500])\n- io: implement `AsyncBufRead` for `Join` ([#6449])\n- net: add Apple visionOS support ([#6465])\n- net: implement `Clone` for `NamedPipeInfo` ([#6586])\n- net: support QNX OS ([#6421])\n- sync: add `Notify::notify_last` ([#6520])\n- sync: add `mpsc::Receiver::{capacity,max_capacity}` ([#6511])\n- sync: add `split` method to the semaphore permit ([#6472], [#6478])\n- task: add `tokio::task::join_set::Builder::spawn_blocking` ([#6578])\n- wasm: support rt-multi-thread with wasm32-wasi-preview1-threads ([#6510])\n\n### Changed\n\n- macros: make `#[tokio::test]` append `#[test]` at the end of the attribute list ([#6497])\n- metrics: fix `blocking_threads` count ([#6551])\n- metrics: stabilize `RuntimeMetrics::worker_count` ([#6556])\n- runtime: move task out of the `lifo_slot` in `block_in_place` ([#6596])\n- runtime: panic if `global_queue_interval` is zero ([#6445])\n- sync: always drop message in destructor for oneshot receiver ([#6558])\n- sync: instrument `Semaphore` for task dumps ([#6499])\n- sync: use FIFO ordering when waking batches of wakers ([#6521])\n- task: make `LocalKey::get` work with Clone types ([#6433])\n- tests: update nix and mio-aio dev-dependencies ([#6552])\n- time: clean up implementation ([#6517])\n- time: lazily init timers on first poll ([#6512])\n- time: remove the `true_when` field in `TimerShared` ([#6563])\n- time: use sharding for timer implementation ([#6534])\n\n### Fixed\n\n- taskdump: allow building taskdump docs on non-unix machines ([#6564])\n- time: check for overflow in `Interval::poll_tick` ([#6487])\n- sync: fix incorrect `is_empty` on mpsc block boundaries ([#6603])\n\n### Documented\n\n- fs: rewrite file system docs ([#6467])\n- io: fix `stdin` documentation ([#6581])\n- io: fix obsolete reference in `ReadHalf::unsplit()` documentation ([#6498])\n- macros: render more comprehensible documentation for `select!` ([#6468])\n- net: add missing types to module docs ([#6482])\n- net: fix misleading `NamedPipeServer` example ([#6590])\n- sync: add examples for `SemaphorePermit`, `OwnedSemaphorePermit` ([#6477])\n- sync: document that `Barrier::wait` is not cancel safe ([#6494])\n- sync: explain relation between `watch::Sender::{subscribe,closed}` ([#6490])\n- task: clarify that you can't abort `spawn_blocking` tasks ([#6571])\n- task: fix a typo in doc of `LocalSet::run_until` ([#6599])\n- time: fix test-util requirement for pause and resume in docs ([#6503])\n\n[#6421]: https://github.com/tokio-rs/tokio/pull/6421\n[#6433]: https://github.com/tokio-rs/tokio/pull/6433\n[#6445]: https://github.com/tokio-rs/tokio/pull/6445\n[#6449]: https://github.com/tokio-rs/tokio/pull/6449\n[#6465]: https://github.com/tokio-rs/tokio/pull/6465\n[#6467]: https://github.com/tokio-rs/tokio/pull/6467\n[#6468]: https://github.com/tokio-rs/tokio/pull/6468\n[#6472]: https://github.com/tokio-rs/tokio/pull/6472\n[#6477]: https://github.com/tokio-rs/tokio/pull/6477\n[#6478]: https://github.com/tokio-rs/tokio/pull/6478\n[#6482]: https://github.com/tokio-rs/tokio/pull/6482\n[#6487]: https://github.com/tokio-rs/tokio/pull/6487\n[#6490]: https://github.com/tokio-rs/tokio/pull/6490\n[#6494]: https://github.com/tokio-rs/tokio/pull/6494\n[#6497]: https://github.com/tokio-rs/tokio/pull/6497\n[#6498]: https://github.com/tokio-rs/tokio/pull/6498\n[#6499]: https://github.com/tokio-rs/tokio/pull/6499\n[#6500]: https://github.com/tokio-rs/tokio/pull/6500\n[#6503]: https://github.com/tokio-rs/tokio/pull/6503\n[#6510]: https://github.com/tokio-rs/tokio/pull/6510\n[#6511]: https://github.com/tokio-rs/tokio/pull/6511\n[#6512]: https://github.com/tokio-rs/tokio/pull/6512\n[#6517]: https://github.com/tokio-rs/tokio/pull/6517\n[#6520]: https://github.com/tokio-rs/tokio/pull/6520\n[#6521]: https://github.com/tokio-rs/tokio/pull/6521\n[#6534]: https://github.com/tokio-rs/tokio/pull/6534\n[#6551]: https://github.com/tokio-rs/tokio/pull/6551\n[#6552]: https://github.com/tokio-rs/tokio/pull/6552\n[#6556]: https://github.com/tokio-rs/tokio/pull/6556\n[#6558]: https://github.com/tokio-rs/tokio/pull/6558\n[#6563]: https://github.com/tokio-rs/tokio/pull/6563\n[#6564]: https://github.com/tokio-rs/tokio/pull/6564\n[#6571]: https://github.com/tokio-rs/tokio/pull/6571\n[#6573]: https://github.com/tokio-rs/tokio/pull/6573\n[#6578]: https://github.com/tokio-rs/tokio/pull/6578\n[#6581]: https://github.com/tokio-rs/tokio/pull/6581\n[#6586]: https://github.com/tokio-rs/tokio/pull/6586\n[#6590]: https://github.com/tokio-rs/tokio/pull/6590\n[#6596]: https://github.com/tokio-rs/tokio/pull/6596\n[#6599]: https://github.com/tokio-rs/tokio/pull/6599\n[#6603]: https://github.com/tokio-rs/tokio/pull/6603\n\n# 1.37.0 (March 28th, 2024)\n\n### Added\n\n- fs: add `set_max_buf_size` to `tokio::fs::File` ([#6411])\n- io: add `try_new` and `try_with_interest` to `AsyncFd` ([#6345])\n- sync: add `forget_permits` method to semaphore ([#6331])\n- sync: add `is_closed`, `is_empty`, and `len` to mpsc receivers ([#6348])\n- sync: add a `rwlock()` method to owned `RwLock` guards ([#6418])\n- sync: expose strong and weak counts of mpsc sender handles ([#6405])\n- sync: implement `Clone` for `watch::Sender` ([#6388])\n- task: add `TaskLocalFuture::take_value` ([#6340])\n- task: implement `FromIterator` for `JoinSet` ([#6300])\n\n### Changed\n\n- io: make `io::split` use a mutex instead of a spinlock ([#6403])\n\n### Fixed\n\n- docs: fix docsrs build without net feature ([#6360])\n- macros: allow select with only else branch ([#6339])\n- runtime: fix leaking registration entries when os registration fails ([#6329])\n\n### Documented\n\n- io: document cancel safety of `AsyncBufReadExt::fill_buf` ([#6431])\n- io: document cancel safety of `AsyncReadExt`'s primitive read functions ([#6337])\n- runtime: add doc link from `Runtime` to `#[tokio::main]` ([#6366])\n- runtime: make the `enter` example deterministic ([#6351])\n- sync: add Semaphore example for limiting the number of outgoing requests ([#6419])\n- sync: fix missing period in broadcast docs ([#6377])\n- sync: mark `mpsc::Sender::downgrade` with `#[must_use]` ([#6326])\n- sync: reorder `const_new` before `new_with` ([#6392])\n- sync: update watch channel docs ([#6395])\n- task: fix documentation links ([#6336])\n\n### Changed (unstable)\n\n- runtime: include task `Id` in taskdumps ([#6328])\n- runtime: panic if `unhandled_panic` is enabled when not supported ([#6410])\n\n[#6300]: https://github.com/tokio-rs/tokio/pull/6300\n[#6326]: https://github.com/tokio-rs/tokio/pull/6326\n[#6328]: https://github.com/tokio-rs/tokio/pull/6328\n[#6329]: https://github.com/tokio-rs/tokio/pull/6329\n[#6331]: https://github.com/tokio-rs/tokio/pull/6331\n[#6336]: https://github.com/tokio-rs/tokio/pull/6336\n[#6337]: https://github.com/tokio-rs/tokio/pull/6337\n[#6339]: https://github.com/tokio-rs/tokio/pull/6339\n[#6340]: https://github.com/tokio-rs/tokio/pull/6340\n[#6345]: https://github.com/tokio-rs/tokio/pull/6345\n[#6348]: https://github.com/tokio-rs/tokio/pull/6348\n[#6351]: https://github.com/tokio-rs/tokio/pull/6351\n[#6360]: https://github.com/tokio-rs/tokio/pull/6360\n[#6366]: https://github.com/tokio-rs/tokio/pull/6366\n[#6377]: https://github.com/tokio-rs/tokio/pull/6377\n[#6388]: https://github.com/tokio-rs/tokio/pull/6388\n[#6392]: https://github.com/tokio-rs/tokio/pull/6392\n[#6395]: https://github.com/tokio-rs/tokio/pull/6395\n[#6403]: https://github.com/tokio-rs/tokio/pull/6403\n[#6405]: https://github.com/tokio-rs/tokio/pull/6405\n[#6410]: https://github.com/tokio-rs/tokio/pull/6410\n[#6411]: https://github.com/tokio-rs/tokio/pull/6411\n[#6418]: https://github.com/tokio-rs/tokio/pull/6418\n[#6419]: https://github.com/tokio-rs/tokio/pull/6419\n[#6431]: https://github.com/tokio-rs/tokio/pull/6431\n\n# 1.36.0 (February 2nd, 2024)\n\n### Added\n\n- io: add `tokio::io::Join` ([#6220])\n- io: implement `AsyncWrite` for `Empty` ([#6235])\n- net: add support for anonymous unix pipes ([#6127])\n- net: add `UnixSocket` ([#6290])\n- net: expose keepalive option on `TcpSocket` ([#6311])\n- sync: add `{Receiver,UnboundedReceiver}::poll_recv_many` ([#6236])\n- sync: add `Sender::{try_,}reserve_many` ([#6205])\n- sync: add `watch::Receiver::mark_unchanged` ([#6252])\n- task: add `JoinSet::try_join_next` ([#6280])\n\n### Changed\n\n- io: make `copy` cooperative ([#6265])\n- io: make `repeat` and `sink` cooperative ([#6254])\n- io: simplify check for empty slice ([#6293])\n- process: use pidfd on Linux when available ([#6152])\n- sync: use AtomicBool in broadcast channel future ([#6298])\n\n### Documented\n\n- io: clarify `clear_ready` docs ([#6304])\n- net: document that `*Fd` traits on `TcpSocket` are unix-only ([#6294])\n- sync: document FIFO behavior of `tokio::sync::Mutex` ([#6279])\n- chore: typographic improvements ([#6262])\n- runtime: remove obsolete comment ([#6303])\n- task: fix typo ([#6261])\n\n[#6220]: https://github.com/tokio-rs/tokio/pull/6220\n[#6235]: https://github.com/tokio-rs/tokio/pull/6235\n[#6127]: https://github.com/tokio-rs/tokio/pull/6127\n[#6290]: https://github.com/tokio-rs/tokio/pull/6290\n[#6311]: https://github.com/tokio-rs/tokio/pull/6311\n[#6236]: https://github.com/tokio-rs/tokio/pull/6236\n[#6205]: https://github.com/tokio-rs/tokio/pull/6205\n[#6252]: https://github.com/tokio-rs/tokio/pull/6252\n[#6280]: https://github.com/tokio-rs/tokio/pull/6280\n[#6265]: https://github.com/tokio-rs/tokio/pull/6265\n[#6254]: https://github.com/tokio-rs/tokio/pull/6254\n[#6293]: https://github.com/tokio-rs/tokio/pull/6293\n[#6238]: https://github.com/tokio-rs/tokio/pull/6238\n[#6152]: https://github.com/tokio-rs/tokio/pull/6152\n[#6298]: https://github.com/tokio-rs/tokio/pull/6298\n[#6262]: https://github.com/tokio-rs/tokio/pull/6262\n[#6303]: https://github.com/tokio-rs/tokio/pull/6303\n[#6261]: https://github.com/tokio-rs/tokio/pull/6261\n[#6304]: https://github.com/tokio-rs/tokio/pull/6304\n[#6294]: https://github.com/tokio-rs/tokio/pull/6294\n[#6279]: https://github.com/tokio-rs/tokio/pull/6279\n\n# 1.35.1 (December 19, 2023)\n\nThis is a forward part of a change that was backported to 1.25.3.\n\n### Fixed\n\n- io: add budgeting to `tokio::runtime::io::registration::async_io` ([#6221])\n\n[#6221]: https://github.com/tokio-rs/tokio/pull/6221\n\n# 1.35.0 (December 8th, 2023)\n\n### Added\n\n- net: add Apple watchOS support ([#6176])\n\n### Changed\n\n- io: drop the `Sized` requirements from `AsyncReadExt.read_buf` ([#6169])\n- runtime: make `Runtime` unwind safe ([#6189])\n- runtime: reduce the lock contention in task spawn ([#6001])\n- tokio: update nix dependency to 0.27.1 ([#6190])\n\n### Fixed\n\n- chore: make `--cfg docsrs` work without net feature ([#6166])\n- chore: use relaxed load for `unsync_load` on miri ([#6179])\n- runtime: handle missing context on wake ([#6148])\n- taskdump: fix taskdump cargo config example ([#6150])\n- taskdump: skip notified tasks during taskdumps ([#6194])\n- tracing: avoid creating resource spans with current parent, use a None parent instead ([#6107])\n- tracing: make task span explicit root ([#6158])\n\n### Documented\n\n- io: flush in `AsyncWriteExt` examples ([#6149])\n- runtime: document fairness guarantees and current behavior ([#6145])\n- task: document cancel safety of `LocalSet::run_until` ([#6147])\n\n[#6001]: https://github.com/tokio-rs/tokio/pull/6001\n[#6107]: https://github.com/tokio-rs/tokio/pull/6107\n[#6144]: https://github.com/tokio-rs/tokio/pull/6144\n[#6145]: https://github.com/tokio-rs/tokio/pull/6145\n[#6147]: https://github.com/tokio-rs/tokio/pull/6147\n[#6148]: https://github.com/tokio-rs/tokio/pull/6148\n[#6149]: https://github.com/tokio-rs/tokio/pull/6149\n[#6150]: https://github.com/tokio-rs/tokio/pull/6150\n[#6158]: https://github.com/tokio-rs/tokio/pull/6158\n[#6166]: https://github.com/tokio-rs/tokio/pull/6166\n[#6169]: https://github.com/tokio-rs/tokio/pull/6169\n[#6176]: https://github.com/tokio-rs/tokio/pull/6176\n[#6179]: https://github.com/tokio-rs/tokio/pull/6179\n[#6189]: https://github.com/tokio-rs/tokio/pull/6189\n[#6190]: https://github.com/tokio-rs/tokio/pull/6190\n[#6194]: https://github.com/tokio-rs/tokio/pull/6194\n\n# 1.34.0 (November 19th, 2023)\n\n### Fixed\n\n- io: allow `clear_readiness` after io driver shutdown ([#6067])\n- io: fix integer overflow in `take` ([#6080])\n- io: fix I/O resource hang ([#6134])\n- sync: fix `broadcast::channel` link ([#6100])\n\n### Changed\n\n- macros: use `::core` qualified imports instead of `::std` inside `tokio::test` macro ([#5973])\n\n### Added\n\n- fs: update cfg attr in `fs::read_dir` to include `aix` ([#6075])\n- sync: add `mpsc::Receiver::recv_many` ([#6010])\n- tokio: added vita target support ([#6094])\n\n[#5973]: https://github.com/tokio-rs/tokio/pull/5973\n[#6067]: https://github.com/tokio-rs/tokio/pull/6067\n[#6080]: https://github.com/tokio-rs/tokio/pull/6080\n[#6134]: https://github.com/tokio-rs/tokio/pull/6134\n[#6100]: https://github.com/tokio-rs/tokio/pull/6100\n[#6075]: https://github.com/tokio-rs/tokio/pull/6075\n[#6010]: https://github.com/tokio-rs/tokio/pull/6010\n[#6094]: https://github.com/tokio-rs/tokio/pull/6094\n\n# 1.33.0 (October 9, 2023)\n\n### Fixed\n\n- io: mark `Interest::add` with `#[must_use]` ([#6037])\n- runtime: fix cache line size for RISC-V ([#5994])\n- sync: prevent lock poisoning in `watch::Receiver::wait_for` ([#6021])\n- task: fix `spawn_local` source location ([#5984])\n\n### Changed\n\n- sync: use Acquire/Release orderings instead of SeqCst in `watch` ([#6018])\n\n### Added\n\n- fs: add vectored writes to `tokio::fs::File` ([#5958])\n- io: add `Interest::remove` method ([#5906])\n- io: add vectored writes to `DuplexStream` ([#5985])\n- net: add Apple tvOS support ([#6045])\n- sync: add `?Sized` bound to `{MutexGuard,OwnedMutexGuard}::map` ([#5997])\n- sync: add `watch::Receiver::mark_unseen` ([#5962], [#6014], [#6017])\n- sync: add `watch::Sender::new` ([#5998])\n- sync: add const fn `OnceCell::from_value` ([#5903])\n\n### Removed\n\n- remove unused `stats` feature ([#5952])\n\n### Documented\n\n- add missing backticks in code examples ([#5938], [#6056])\n- fix typos ([#5988], [#6030])\n- process: document that `Child::wait` is cancel safe ([#5977])\n- sync: add examples for `Semaphore` ([#5939], [#5956], [#5978], [#6031], [#6032], [#6050])\n- sync: document that `broadcast` capacity is a lower bound ([#6042])\n- sync: document that `const_new` is not instrumented ([#6002])\n- sync: improve cancel-safety documentation for `mpsc::Sender::send` ([#5947])\n- sync: improve docs for `watch` channel ([#5954])\n- taskdump: render taskdump documentation on docs.rs ([#5972])\n\n### Unstable\n\n- taskdump: fix potential deadlock ([#6036])\n\n[#5903]: https://github.com/tokio-rs/tokio/pull/5903\n[#5906]: https://github.com/tokio-rs/tokio/pull/5906\n[#5938]: https://github.com/tokio-rs/tokio/pull/5938\n[#5939]: https://github.com/tokio-rs/tokio/pull/5939\n[#5947]: https://github.com/tokio-rs/tokio/pull/5947\n[#5952]: https://github.com/tokio-rs/tokio/pull/5952\n[#5954]: https://github.com/tokio-rs/tokio/pull/5954\n[#5956]: https://github.com/tokio-rs/tokio/pull/5956\n[#5958]: https://github.com/tokio-rs/tokio/pull/5958\n[#5960]: https://github.com/tokio-rs/tokio/pull/5960\n[#5962]: https://github.com/tokio-rs/tokio/pull/5962\n[#5971]: https://github.com/tokio-rs/tokio/pull/5971\n[#5972]: https://github.com/tokio-rs/tokio/pull/5972\n[#5977]: https://github.com/tokio-rs/tokio/pull/5977\n[#5978]: https://github.com/tokio-rs/tokio/pull/5978\n[#5984]: https://github.com/tokio-rs/tokio/pull/5984\n[#5985]: https://github.com/tokio-rs/tokio/pull/5985\n[#5988]: https://github.com/tokio-rs/tokio/pull/5988\n[#5994]: https://github.com/tokio-rs/tokio/pull/5994\n[#5997]: https://github.com/tokio-rs/tokio/pull/5997\n[#5998]: https://github.com/tokio-rs/tokio/pull/5998\n[#6002]: https://github.com/tokio-rs/tokio/pull/6002\n[#6014]: https://github.com/tokio-rs/tokio/pull/6014\n[#6017]: https://github.com/tokio-rs/tokio/pull/6017\n[#6018]: https://github.com/tokio-rs/tokio/pull/6018\n[#6021]: https://github.com/tokio-rs/tokio/pull/6021\n[#6030]: https://github.com/tokio-rs/tokio/pull/6030\n[#6031]: https://github.com/tokio-rs/tokio/pull/6031\n[#6032]: https://github.com/tokio-rs/tokio/pull/6032\n[#6036]: https://github.com/tokio-rs/tokio/pull/6036\n[#6037]: https://github.com/tokio-rs/tokio/pull/6037\n[#6042]: https://github.com/tokio-rs/tokio/pull/6042\n[#6045]: https://github.com/tokio-rs/tokio/pull/6045\n[#6050]: https://github.com/tokio-rs/tokio/pull/6050\n[#6056]: https://github.com/tokio-rs/tokio/pull/6056\n[#6058]: https://github.com/tokio-rs/tokio/pull/6058\n\n# 1.32.1 (December 19, 2023)\n\nThis is a forward part of a change that was backported to 1.25.3.\n\n### Fixed\n\n- io: add budgeting to `tokio::runtime::io::registration::async_io` ([#6221])\n\n[#6221]: https://github.com/tokio-rs/tokio/pull/6221\n\n# 1.32.0 (August 16, 2023)\n\n### Fixed\n\n- sync: fix potential quadratic behavior in `broadcast::Receiver` ([#5925])\n\n### Added\n\n- process: stabilize `Command::raw_arg` ([#5930])\n- io: enable awaiting error readiness ([#5781])\n\n### Unstable\n\n- rt(alt): improve scalability of alt runtime as the number of cores grows ([#5935])\n\n[#5925]: https://github.com/tokio-rs/tokio/pull/5925\n[#5930]: https://github.com/tokio-rs/tokio/pull/5930\n[#5781]: https://github.com/tokio-rs/tokio/pull/5781\n[#5935]: https://github.com/tokio-rs/tokio/pull/5935\n\n# 1.31.0 (August 10, 2023)\n\n### Fixed\n\n* io: delegate `WriteHalf::poll_write_vectored` ([#5914])\n\n### Unstable\n\n* rt(alt): fix memory leak in unstable next-gen scheduler prototype ([#5911])\n* rt: expose mean task poll time metric ([#5927])\n\n[#5914]: https://github.com/tokio-rs/tokio/pull/5914\n[#5911]: https://github.com/tokio-rs/tokio/pull/5911\n[#5927]: https://github.com/tokio-rs/tokio/pull/5927\n\n# 1.30.0 (August 9, 2023)\n\nThis release bumps the MSRV of Tokio to 1.63. ([#5887])\n\n### Changed\n\n- tokio: reduce LLVM code generation ([#5859])\n- io: support `--cfg mio_unsupported_force_poll_poll` flag ([#5881])\n- sync: make `const_new` methods always available ([#5885])\n- sync: avoid false sharing in mpsc channel ([#5829])\n- rt: pop at least one task from inject queue ([#5908])\n\n### Added\n\n- sync: add `broadcast::Sender::new` ([#5824])\n- net: implement `UCred` for espidf ([#5868])\n- fs: add `File::options()` ([#5869])\n- time: implement extra reset variants for `Interval` ([#5878])\n- process: add `{ChildStd*}::into_owned_{fd, handle}` ([#5899])\n\n### Removed\n\n- tokio: removed unused `tokio_*` cfgs ([#5890])\n- remove build script to speed up compilation ([#5887])\n\n### Documented\n\n- sync: mention lagging in docs for `broadcast::send` ([#5820])\n- runtime: expand on sharing runtime docs ([#5858])\n- io: use vec in example for `AsyncReadExt::read_exact` ([#5863])\n- time: mark `Sleep` as `!Unpin` in docs ([#5916])\n- process: fix `raw_arg` not showing up in docs ([#5865])\n\n### Unstable\n\n- rt: add runtime ID ([#5864])\n- rt: initial implementation of new threaded runtime ([#5823])\n\n[#5820]: https://github.com/tokio-rs/tokio/pull/5820\n[#5823]: https://github.com/tokio-rs/tokio/pull/5823\n[#5824]: https://github.com/tokio-rs/tokio/pull/5824\n[#5829]: https://github.com/tokio-rs/tokio/pull/5829\n[#5858]: https://github.com/tokio-rs/tokio/pull/5858\n[#5859]: https://github.com/tokio-rs/tokio/pull/5859\n[#5863]: https://github.com/tokio-rs/tokio/pull/5863\n[#5864]: https://github.com/tokio-rs/tokio/pull/5864\n[#5865]: https://github.com/tokio-rs/tokio/pull/5865\n[#5868]: https://github.com/tokio-rs/tokio/pull/5868\n[#5869]: https://github.com/tokio-rs/tokio/pull/5869\n[#5878]: https://github.com/tokio-rs/tokio/pull/5878\n[#5881]: https://github.com/tokio-rs/tokio/pull/5881\n[#5885]: https://github.com/tokio-rs/tokio/pull/5885\n[#5887]: https://github.com/tokio-rs/tokio/pull/5887\n[#5890]: https://github.com/tokio-rs/tokio/pull/5890\n[#5899]: https://github.com/tokio-rs/tokio/pull/5899\n[#5908]: https://github.com/tokio-rs/tokio/pull/5908\n[#5916]: https://github.com/tokio-rs/tokio/pull/5916\n\n# 1.29.1 (June 29, 2023)\n\n### Fixed\n\n- rt: fix nesting two `block_in_place` with a `block_on` between ([#5837])\n\n[#5837]: https://github.com/tokio-rs/tokio/pull/5837\n\n# 1.29.0 (June 27, 2023)\n\nTechnically a breaking change, the `Send` implementation is removed from\n`runtime::EnterGuard`. This change fixes a bug and should not impact most users.\n\n### Breaking\n\n- rt: `EnterGuard` should not be `Send` ([#5766])\n\n### Fixed\n\n- fs: reduce blocking ops in `fs::read_dir` ([#5653])\n- rt: fix possible starvation ([#5686], [#5712])\n- rt: fix stacked borrows issue in `JoinSet` ([#5693])\n- rt: panic if `EnterGuard` dropped incorrect order ([#5772])\n- time: do not overflow to signal value ([#5710])\n- fs: wait for in-flight ops before cloning `File` ([#5803])\n\n### Changed\n\n- rt: reduce time to poll tasks scheduled from outside the runtime ([#5705], [#5720])\n\n### Added\n\n- net: add uds doc alias for unix sockets ([#5659])\n- rt: add metric for number of tasks ([#5628])\n- sync: implement more traits for channel errors ([#5666])\n- net: add nodelay methods on TcpSocket ([#5672])\n- sync: add `broadcast::Receiver::blocking_recv` ([#5690])\n- process: add `raw_arg` method to `Command` ([#5704])\n- io: support PRIORITY epoll events ([#5566])\n- task: add `JoinSet::poll_join_next` ([#5721])\n- net: add support for Redox OS ([#5790])\n\n\n### Unstable\n\n- rt: add the ability to dump task backtraces ([#5608], [#5676], [#5708], [#5717])\n- rt: instrument task poll times with a histogram ([#5685])\n\n[#5766]: https://github.com/tokio-rs/tokio/pull/5766\n[#5653]: https://github.com/tokio-rs/tokio/pull/5653\n[#5686]: https://github.com/tokio-rs/tokio/pull/5686\n[#5712]: https://github.com/tokio-rs/tokio/pull/5712\n[#5693]: https://github.com/tokio-rs/tokio/pull/5693\n[#5772]: https://github.com/tokio-rs/tokio/pull/5772\n[#5710]: https://github.com/tokio-rs/tokio/pull/5710\n[#5803]: https://github.com/tokio-rs/tokio/pull/5803\n[#5705]: https://github.com/tokio-rs/tokio/pull/5705\n[#5720]: https://github.com/tokio-rs/tokio/pull/5720\n[#5659]: https://github.com/tokio-rs/tokio/pull/5659\n[#5628]: https://github.com/tokio-rs/tokio/pull/5628\n[#5666]: https://github.com/tokio-rs/tokio/pull/5666\n[#5672]: https://github.com/tokio-rs/tokio/pull/5672\n[#5690]: https://github.com/tokio-rs/tokio/pull/5690\n[#5704]: https://github.com/tokio-rs/tokio/pull/5704\n[#5566]: https://github.com/tokio-rs/tokio/pull/5566\n[#5721]: https://github.com/tokio-rs/tokio/pull/5721\n[#5790]: https://github.com/tokio-rs/tokio/pull/5790\n[#5608]: https://github.com/tokio-rs/tokio/pull/5608\n[#5676]: https://github.com/tokio-rs/tokio/pull/5676\n[#5708]: https://github.com/tokio-rs/tokio/pull/5708\n[#5717]: https://github.com/tokio-rs/tokio/pull/5717\n[#5685]: https://github.com/tokio-rs/tokio/pull/5685\n\n# 1.28.2 (May 28, 2023)\n\nForward ports 1.18.6 changes.\n\n### Fixed\n\n- deps: disable default features for mio ([#5728])\n\n[#5728]: https://github.com/tokio-rs/tokio/pull/5728\n\n# 1.28.1 (May 10th, 2023)\n\nThis release fixes a mistake in the build script that makes `AsFd`\nimplementations unavailable on Rust 1.63. ([#5677])\n\n[#5677]: https://github.com/tokio-rs/tokio/pull/5677\n\n# 1.28.0 (April 25th, 2023)\n\n### Added\n\n- io: add `AsyncFd::async_io` ([#5542])\n- io: impl BufMut for ReadBuf ([#5590])\n- net: add `recv_buf` for `UdpSocket` and `UnixDatagram` ([#5583])\n- sync: add `OwnedSemaphorePermit::semaphore` ([#5618])\n- sync: add `same_channel` to broadcast channel ([#5607])\n- sync: add `watch::Receiver::wait_for` ([#5611])\n- task: add `JoinSet::spawn_blocking` and `JoinSet::spawn_blocking_on` ([#5612])\n\n### Changed\n\n- deps: update windows-sys to 0.48 ([#5591])\n- io: make `read_to_end` not grow unnecessarily ([#5610])\n- macros: make entrypoints more efficient ([#5621])\n- sync: improve Debug impl for `RwLock` ([#5647])\n- sync: reduce contention in `Notify` ([#5503])\n\n### Fixed\n\n- net: support `get_peer_cred` on AIX ([#5065])\n- sync: avoid deadlocks in `broadcast` with custom wakers ([#5578])\n\n### Documented\n\n- sync: fix typo in `Semaphore::MAX_PERMITS` ([#5645])\n- sync: fix typo in `tokio::sync::watch::Sender` docs ([#5587])\n\n[#5065]: https://github.com/tokio-rs/tokio/pull/5065\n[#5503]: https://github.com/tokio-rs/tokio/pull/5503\n[#5542]: https://github.com/tokio-rs/tokio/pull/5542\n[#5578]: https://github.com/tokio-rs/tokio/pull/5578\n[#5583]: https://github.com/tokio-rs/tokio/pull/5583\n[#5587]: https://github.com/tokio-rs/tokio/pull/5587\n[#5590]: https://github.com/tokio-rs/tokio/pull/5590\n[#5591]: https://github.com/tokio-rs/tokio/pull/5591\n[#5607]: https://github.com/tokio-rs/tokio/pull/5607\n[#5610]: https://github.com/tokio-rs/tokio/pull/5610\n[#5611]: https://github.com/tokio-rs/tokio/pull/5611\n[#5612]: https://github.com/tokio-rs/tokio/pull/5612\n[#5618]: https://github.com/tokio-rs/tokio/pull/5618\n[#5621]: https://github.com/tokio-rs/tokio/pull/5621\n[#5645]: https://github.com/tokio-rs/tokio/pull/5645\n[#5647]: https://github.com/tokio-rs/tokio/pull/5647\n\n# 1.27.0 (March 27th, 2023)\n\nThis release bumps the MSRV of Tokio to 1.56. ([#5559])\n\n### Added\n\n- io: add `async_io` helper method to sockets ([#5512])\n- io: add implementations of `AsFd`/`AsHandle`/`AsSocket` ([#5514], [#5540])\n- net: add `UdpSocket::peek_sender()` ([#5520])\n- sync: add `RwLockWriteGuard::{downgrade_map, try_downgrade_map}` ([#5527])\n- task: add `JoinHandle::abort_handle` ([#5543])\n\n### Changed\n\n- io: use `memchr` from `libc` ([#5558])\n- macros: accept path as crate rename in `#[tokio::main]` ([#5557])\n- macros: update to syn 2.0.0 ([#5572])\n- time: don't register for a wakeup when `Interval` returns `Ready` ([#5553])\n\n### Fixed\n\n- fs: fuse std iterator in `ReadDir` ([#5555])\n- tracing: fix `spawn_blocking` location fields ([#5573])\n- time: clean up redundant check in `Wheel::poll()` ([#5574])\n\n### Documented\n\n- macros: define cancellation safety ([#5525])\n- io: add details to docs of `tokio::io::copy[_buf]` ([#5575])\n- io: refer to `ReaderStream` and `StreamReader` in module docs ([#5576])\n\n[#5512]: https://github.com/tokio-rs/tokio/pull/5512\n[#5514]: https://github.com/tokio-rs/tokio/pull/5514\n[#5520]: https://github.com/tokio-rs/tokio/pull/5520\n[#5525]: https://github.com/tokio-rs/tokio/pull/5525\n[#5527]: https://github.com/tokio-rs/tokio/pull/5527\n[#5540]: https://github.com/tokio-rs/tokio/pull/5540\n[#5543]: https://github.com/tokio-rs/tokio/pull/5543\n[#5553]: https://github.com/tokio-rs/tokio/pull/5553\n[#5555]: https://github.com/tokio-rs/tokio/pull/5555\n[#5557]: https://github.com/tokio-rs/tokio/pull/5557\n[#5558]: https://github.com/tokio-rs/tokio/pull/5558\n[#5559]: https://github.com/tokio-rs/tokio/pull/5559\n[#5572]: https://github.com/tokio-rs/tokio/pull/5572\n[#5573]: https://github.com/tokio-rs/tokio/pull/5573\n[#5574]: https://github.com/tokio-rs/tokio/pull/5574\n[#5575]: https://github.com/tokio-rs/tokio/pull/5575\n[#5576]: https://github.com/tokio-rs/tokio/pull/5576\n\n# 1.26.0 (March 1st, 2023)\n\n### Fixed\n\n- macros: fix empty `join!` and `try_join!` ([#5504])\n- sync: don't leak tracing spans in mutex guards ([#5469])\n- sync: drop wakers after unlocking the mutex in Notify ([#5471])\n- sync: drop wakers outside lock in semaphore ([#5475])\n\n### Added\n\n- fs: add `fs::try_exists` ([#4299])\n- net: add types for named unix pipes ([#5351])\n- sync: add `MappedOwnedMutexGuard` ([#5474])\n\n### Changed\n\n- chore: update windows-sys to 0.45 ([#5386])\n- net: use Message Read Mode for named pipes ([#5350])\n- sync: mark lock guards with `#[clippy::has_significant_drop]` ([#5422])\n- sync: reduce contention in watch channel ([#5464])\n- time: remove cache padding in timer entries ([#5468])\n- time: Improve `Instant::now()` perf with test-util ([#5513])\n\n### Internal Changes\n\n- io: use `poll_fn` in `copy_bidirectional` ([#5486])\n- net: refactor named pipe builders to not use bitfields ([#5477])\n- rt: remove Arc from Clock ([#5434])\n- sync: make `notify_waiters` calls atomic ([#5458])\n- time: don't store deadline twice in sleep entries ([#5410])\n\n### Unstable\n\n- metrics: add a new metric for budget exhaustion yields ([#5517])\n\n### Documented\n\n- io: improve AsyncFd example ([#5481])\n- runtime: document the nature of the main future ([#5494])\n- runtime: remove extra period in docs ([#5511])\n- signal: updated Documentation for Signals ([#5459])\n- sync: add doc aliases for `blocking_*` methods ([#5448])\n- sync: fix docs for Send/Sync bounds in broadcast ([#5480])\n- sync: document drop behavior for channels ([#5497])\n- task: clarify what happens to spawned work during runtime shutdown ([#5394])\n- task: clarify `process::Command` docs ([#5413])\n- task: fix wording with 'unsend' ([#5452])\n- time: document immediate completion guarantee for timeouts ([#5509])\n- tokio: document supported platforms ([#5483])\n\n[#4299]: https://github.com/tokio-rs/tokio/pull/4299\n[#5350]: https://github.com/tokio-rs/tokio/pull/5350\n[#5351]: https://github.com/tokio-rs/tokio/pull/5351\n[#5386]: https://github.com/tokio-rs/tokio/pull/5386\n[#5394]: https://github.com/tokio-rs/tokio/pull/5394\n[#5410]: https://github.com/tokio-rs/tokio/pull/5410\n[#5413]: https://github.com/tokio-rs/tokio/pull/5413\n[#5422]: https://github.com/tokio-rs/tokio/pull/5422\n[#5434]: https://github.com/tokio-rs/tokio/pull/5434\n[#5448]: https://github.com/tokio-rs/tokio/pull/5448\n[#5452]: https://github.com/tokio-rs/tokio/pull/5452\n[#5458]: https://github.com/tokio-rs/tokio/pull/5458\n[#5459]: https://github.com/tokio-rs/tokio/pull/5459\n[#5464]: https://github.com/tokio-rs/tokio/pull/5464\n[#5468]: https://github.com/tokio-rs/tokio/pull/5468\n[#5469]: https://github.com/tokio-rs/tokio/pull/5469\n[#5471]: https://github.com/tokio-rs/tokio/pull/5471\n[#5474]: https://github.com/tokio-rs/tokio/pull/5474\n[#5475]: https://github.com/tokio-rs/tokio/pull/5475\n[#5477]: https://github.com/tokio-rs/tokio/pull/5477\n[#5480]: https://github.com/tokio-rs/tokio/pull/5480\n[#5481]: https://github.com/tokio-rs/tokio/pull/5481\n[#5483]: https://github.com/tokio-rs/tokio/pull/5483\n[#5486]: https://github.com/tokio-rs/tokio/pull/5486\n[#5494]: https://github.com/tokio-rs/tokio/pull/5494\n[#5497]: https://github.com/tokio-rs/tokio/pull/5497\n[#5504]: https://github.com/tokio-rs/tokio/pull/5504\n[#5509]: https://github.com/tokio-rs/tokio/pull/5509\n[#5511]: https://github.com/tokio-rs/tokio/pull/5511\n[#5513]: https://github.com/tokio-rs/tokio/pull/5513\n[#5517]: https://github.com/tokio-rs/tokio/pull/5517\n\n# 1.25.3 (December 17th, 2023)\n\n### Fixed\n- io: add budgeting to `tokio::runtime::io::registration::async_io` ([#6221])\n\n[#6221]: https://github.com/tokio-rs/tokio/pull/6221\n\n# 1.25.2 (September 22, 2023)\n\nForward ports 1.20.6 changes.\n\n### Changed\n\n- io: use `memchr` from `libc` ([#5960])\n\n[#5960]: https://github.com/tokio-rs/tokio/pull/5960\n\n# 1.25.1 (May 28, 2023)\n\nForward ports 1.18.6 changes.\n\n### Fixed\n\n- deps: disable default features for mio ([#5728])\n\n[#5728]: https://github.com/tokio-rs/tokio/pull/5728\n\n# 1.25.0 (January 28, 2023)\n\n### Fixed\n\n- rt: fix runtime metrics reporting ([#5330])\n\n### Added\n\n- sync: add `broadcast::Sender::len` ([#5343])\n\n### Changed\n\n- fs: increase maximum read buffer size to 2MiB ([#5397])\n\n[#5330]: https://github.com/tokio-rs/tokio/pull/5330\n[#5343]: https://github.com/tokio-rs/tokio/pull/5343\n[#5397]: https://github.com/tokio-rs/tokio/pull/5397\n\n# 1.24.2 (January 17, 2023)\n\nForward ports 1.18.5 changes.\n\n### Fixed\n\n- io: fix unsoundness in `ReadHalf::unsplit` ([#5375])\n\n[#5375]: https://github.com/tokio-rs/tokio/pull/5375\n\n# 1.24.1 (January 6, 2022)\n\nThis release fixes a compilation failure on targets without `AtomicU64` when using rustc older than 1.63. ([#5356])\n\n[#5356]: https://github.com/tokio-rs/tokio/pull/5356\n\n# 1.24.0 (January 5, 2022)\n\n### Fixed\n - rt: improve native `AtomicU64` support detection ([#5284])\n\n### Added\n - rt: add configuration option for max number of I/O events polled from the OS\n per tick ([#5186])\n - rt: add an environment variable for configuring the default number of worker\n threads per runtime instance ([#4250])\n\n### Changed\n - sync: reduce MPSC channel stack usage ([#5294])\n - io: reduce lock contention in I/O operations ([#5300])\n - fs: speed up `read_dir()` by chunking operations ([#5309])\n - rt: use internal `ThreadId` implementation ([#5329])\n - test: don't auto-advance time when a `spawn_blocking` task is running ([#5115])\n\n[#5186]: https://github.com/tokio-rs/tokio/pull/5186\n[#5294]: https://github.com/tokio-rs/tokio/pull/5294\n[#5284]: https://github.com/tokio-rs/tokio/pull/5284\n[#4250]: https://github.com/tokio-rs/tokio/pull/4250\n[#5300]: https://github.com/tokio-rs/tokio/pull/5300\n[#5329]: https://github.com/tokio-rs/tokio/pull/5329\n[#5115]: https://github.com/tokio-rs/tokio/pull/5115\n[#5309]: https://github.com/tokio-rs/tokio/pull/5309\n\n# 1.23.1 (January 4, 2022)\n\nThis release forward ports changes from 1.18.4.\n\n### Fixed\n\n- net: fix Windows named pipe server builder to maintain option when toggling\n pipe mode ([#5336]).\n\n[#5336]: https://github.com/tokio-rs/tokio/pull/5336\n\n# 1.23.0 (December 5, 2022)\n\n### Fixed\n\n - net: fix Windows named pipe connect ([#5208])\n - io: support vectored writes for `ChildStdin` ([#5216])\n - io: fix `async fn ready()` false positive for OS-specific events ([#5231])\n\n ### Changed\n - runtime: `yield_now` defers task until after driver poll ([#5223])\n - runtime: reduce amount of codegen needed per spawned task ([#5213])\n - windows: replace `winapi` dependency with `windows-sys` ([#5204])\n\n [#5208]: https://github.com/tokio-rs/tokio/pull/5208\n [#5216]: https://github.com/tokio-rs/tokio/pull/5216\n [#5213]: https://github.com/tokio-rs/tokio/pull/5213\n [#5204]: https://github.com/tokio-rs/tokio/pull/5204\n [#5223]: https://github.com/tokio-rs/tokio/pull/5223\n [#5231]: https://github.com/tokio-rs/tokio/pull/5231\n\n# 1.22.0 (November 17, 2022)\n\n### Added\n - runtime: add `Handle::runtime_flavor` ([#5138])\n - sync: add `Mutex::blocking_lock_owned` ([#5130])\n - sync: add `Semaphore::MAX_PERMITS` ([#5144])\n - sync: add `merge()` to semaphore permits ([#4948])\n - sync: add `mpsc::WeakUnboundedSender` ([#5189])\n\n### Added (unstable)\n\n - process: add `Command::process_group` ([#5114])\n - runtime: export metrics about the blocking thread pool ([#5161])\n - task: add `task::id()` and `task::try_id()` ([#5171])\n\n### Fixed\n - macros: don't take ownership of futures in macros ([#5087])\n - runtime: fix Stacked Borrows violation in `LocalOwnedTasks` ([#5099])\n - runtime: mitigate ABA with 32-bit queue indices when possible ([#5042])\n - task: wake local tasks to the local queue when woken by the same thread ([#5095])\n - time: panic in release mode when `mark_pending` called illegally ([#5093])\n - runtime: fix typo in expect message ([#5169])\n - runtime: fix `unsync_load` on atomic types ([#5175])\n - task: elaborate safety comments in task deallocation ([#5172])\n - runtime: fix `LocalSet` drop in thread local ([#5179])\n - net: remove libc type leakage in a public API ([#5191])\n - runtime: update the alignment of `CachePadded` ([#5106])\n\n### Changed\n - io: make `tokio::io::copy` continue filling the buffer when writer stalls ([#5066])\n - runtime: remove `coop::budget` from `LocalSet::run_until` ([#5155])\n - sync: make `Notify` panic safe ([#5154])\n\n### Documented\n - io: fix doc for `write_i8` to use signed integers ([#5040])\n - net: fix doc typos for TCP and UDP `set_tos` methods ([#5073])\n - net: fix function name in `UdpSocket::recv` documentation ([#5150])\n - sync: typo in `TryLockError` for `RwLock::try_write` ([#5160])\n - task: document that spawned tasks execute immediately ([#5117])\n - time: document return type of `timeout` ([#5118])\n - time: document that `timeout` checks only before poll ([#5126])\n - sync: specify return type of `oneshot::Receiver` in docs ([#5198])\n\n### Internal changes\n - runtime: use const `Mutex::new` for globals ([#5061])\n - runtime: remove `Option` around `mio::Events` in io driver ([#5078])\n - runtime: remove a conditional compilation clause ([#5104])\n - runtime: remove a reference to internal time handle ([#5107])\n - runtime: misc time driver cleanup ([#5120])\n - runtime: move signal driver to runtime module ([#5121])\n - runtime: signal driver now uses I/O driver directly ([#5125])\n - runtime: start decoupling I/O driver and I/O handle ([#5127])\n - runtime: switch `io::handle` refs with scheduler:Handle ([#5128])\n - runtime: remove Arc from I/O driver ([#5134])\n - runtime: use signal driver handle via `scheduler::Handle` ([#5135])\n - runtime: move internal clock fns out of context ([#5139])\n - runtime: remove `runtime::context` module ([#5140])\n - runtime: keep driver cfgs in `driver.rs` ([#5141])\n - runtime: add `runtime::context` to unify thread-locals ([#5143])\n - runtime: rename some confusing internal variables/fns ([#5151])\n - runtime: move `coop` mod into `runtime` ([#5152])\n - runtime: move budget state to context thread-local ([#5157])\n - runtime: move park logic into runtime module ([#5158])\n - runtime: move `Runtime` into its own file ([#5159])\n - runtime: unify entering a runtime with `Handle::enter` ([#5163])\n - runtime: remove handle reference from each scheduler ([#5166])\n - runtime: move `enter` into `context` ([#5167])\n - runtime: combine context and entered thread-locals ([#5168])\n - runtime: fix accidental unsetting of current handle ([#5178])\n - runtime: move `CoreStage` methods to `Core` ([#5182])\n - sync: name mpsc semaphore types ([#5146])\n\n[#4948]: https://github.com/tokio-rs/tokio/pull/4948\n[#5040]: https://github.com/tokio-rs/tokio/pull/5040\n[#5042]: https://github.com/tokio-rs/tokio/pull/5042\n[#5061]: https://github.com/tokio-rs/tokio/pull/5061\n[#5066]: https://github.com/tokio-rs/tokio/pull/5066\n[#5073]: https://github.com/tokio-rs/tokio/pull/5073\n[#5078]: https://github.com/tokio-rs/tokio/pull/5078\n[#5087]: https://github.com/tokio-rs/tokio/pull/5087\n[#5093]: https://github.com/tokio-rs/tokio/pull/5093\n[#5095]: https://github.com/tokio-rs/tokio/pull/5095\n[#5099]: https://github.com/tokio-rs/tokio/pull/5099\n[#5104]: https://github.com/tokio-rs/tokio/pull/5104\n[#5106]: https://github.com/tokio-rs/tokio/pull/5106\n[#5107]: https://github.com/tokio-rs/tokio/pull/5107\n[#5114]: https://github.com/tokio-rs/tokio/pull/5114\n[#5117]: https://github.com/tokio-rs/tokio/pull/5117\n[#5118]: https://github.com/tokio-rs/tokio/pull/5118\n[#5120]: https://github.com/tokio-rs/tokio/pull/5120\n[#5121]: https://github.com/tokio-rs/tokio/pull/5121\n[#5125]: https://github.com/tokio-rs/tokio/pull/5125\n[#5126]: https://github.com/tokio-rs/tokio/pull/5126\n[#5127]: https://github.com/tokio-rs/tokio/pull/5127\n[#5128]: https://github.com/tokio-rs/tokio/pull/5128\n[#5130]: https://github.com/tokio-rs/tokio/pull/5130\n[#5134]: https://github.com/tokio-rs/tokio/pull/5134\n[#5135]: https://github.com/tokio-rs/tokio/pull/5135\n[#5138]: https://github.com/tokio-rs/tokio/pull/5138\n[#5138]: https://github.com/tokio-rs/tokio/pull/5138\n[#5139]: https://github.com/tokio-rs/tokio/pull/5139\n[#5140]: https://github.com/tokio-rs/tokio/pull/5140\n[#5141]: https://github.com/tokio-rs/tokio/pull/5141\n[#5143]: https://github.com/tokio-rs/tokio/pull/5143\n[#5144]: https://github.com/tokio-rs/tokio/pull/5144\n[#5144]: https://github.com/tokio-rs/tokio/pull/5144\n[#5146]: https://github.com/tokio-rs/tokio/pull/5146\n[#5150]: https://github.com/tokio-rs/tokio/pull/5150\n[#5151]: https://github.com/tokio-rs/tokio/pull/5151\n[#5152]: https://github.com/tokio-rs/tokio/pull/5152\n[#5154]: https://github.com/tokio-rs/tokio/pull/5154\n[#5155]: https://github.com/tokio-rs/tokio/pull/5155\n[#5157]: https://github.com/tokio-rs/tokio/pull/5157\n[#5158]: https://github.com/tokio-rs/tokio/pull/5158\n[#5159]: https://github.com/tokio-rs/tokio/pull/5159\n[#5160]: https://github.com/tokio-rs/tokio/pull/5160\n[#5161]: https://github.com/tokio-rs/tokio/pull/5161\n[#5163]: https://github.com/tokio-rs/tokio/pull/5163\n[#5166]: https://github.com/tokio-rs/tokio/pull/5166\n[#5167]: https://github.com/tokio-rs/tokio/pull/5167\n[#5168]: https://github.com/tokio-rs/tokio/pull/5168\n[#5169]: https://github.com/tokio-rs/tokio/pull/5169\n[#5171]: https://github.com/tokio-rs/tokio/pull/5171\n[#5172]: https://github.com/tokio-rs/tokio/pull/5172\n[#5175]: https://github.com/tokio-rs/tokio/pull/5175\n[#5178]: https://github.com/tokio-rs/tokio/pull/5178\n[#5179]: https://github.com/tokio-rs/tokio/pull/5179\n[#5182]: https://github.com/tokio-rs/tokio/pull/5182\n[#5189]: https://github.com/tokio-rs/tokio/pull/5189\n[#5191]: https://github.com/tokio-rs/tokio/pull/5191\n[#5198]: https://github.com/tokio-rs/tokio/pull/5198\n\n# 1.21.2 (September 27, 2022)\n\nThis release removes the dependency on the `once_cell` crate to restore the MSRV\nof 1.21.x, which is the latest minor version at the time of release. ([#5048])\n\n[#5048]: https://github.com/tokio-rs/tokio/pull/5048\n\n# 1.21.1 (September 13, 2022)\n\n### Fixed\n\n- net: fix dependency resolution for socket2 ([#5000])\n- task: ignore failure to set TLS in `LocalSet` Drop ([#4976])\n\n[#4976]: https://github.com/tokio-rs/tokio/pull/4976\n[#5000]: https://github.com/tokio-rs/tokio/pull/5000\n\n# 1.21.0 (September 2, 2022)\n\nThis release is the first release of Tokio to intentionally support WASM. The\n`sync,macros,io-util,rt,time` features are stabilized on WASM. Additionally the\nwasm32-wasi target is given unstable support for the `net` feature.\n\n### Added\n\n- net: add `device` and `bind_device` methods to TCP/UDP sockets ([#4882])\n- net: add `tos` and `set_tos` methods to TCP and UDP sockets ([#4877])\n- net: add security flags to named pipe `ServerOptions` ([#4845])\n- signal: add more windows signal handlers ([#4924])\n- sync: add `mpsc::Sender::max_capacity` method ([#4904])\n- sync: implement Weak version of `mpsc::Sender` ([#4595])\n- task: add `LocalSet::enter` ([#4765])\n- task: stabilize `JoinSet` and `AbortHandle` ([#4920])\n- tokio: add `track_caller` to public APIs ([#4805], [#4848], [#4852])\n- wasm: initial support for `wasm32-wasi` target ([#4716])\n\n### Fixed\n\n- miri: improve miri compatibility by avoiding temporary references in `linked_list::Link` impls ([#4841])\n- signal: don't register write interest on signal pipe ([#4898])\n- sync: add `#[must_use]` to lock guards ([#4886])\n- sync: fix hang when calling `recv` on closed and reopened broadcast channel ([#4867])\n- task: propagate attributes on task-locals ([#4837])\n\n### Changed\n\n- fs: change panic to error in `File::start_seek` ([#4897])\n- io: reduce syscalls in `poll_read` ([#4840])\n- process: use blocking threadpool for child stdio I/O ([#4824])\n- signal: make `SignalKind` methods const ([#4956])\n\n### Internal changes\n\n- rt: extract `basic_scheduler::Config` ([#4935])\n- rt: move I/O driver into `runtime` module ([#4942])\n- rt: rename internal scheduler types ([#4945])\n\n### Documented\n\n- chore: fix typos and grammar ([#4858], [#4894], [#4928])\n- io: fix typo in `AsyncSeekExt::rewind` docs ([#4893])\n- net: add documentation to `try_read()` for zero-length buffers ([#4937])\n- runtime: remove incorrect panic section for `Builder::worker_threads` ([#4849])\n- sync: doc of `watch::Sender::send` improved ([#4959])\n- task: add cancel safety docs to `JoinHandle` ([#4901])\n- task: expand on cancellation of `spawn_blocking` ([#4811])\n- time: clarify that the first tick of `Interval::tick` happens immediately ([#4951])\n\n### Unstable\n\n- rt: add unstable option to disable the LIFO slot ([#4936])\n- task: fix incorrect signature in `Builder::spawn_on` ([#4953])\n- task: make `task::Builder::spawn*` methods fallible ([#4823])\n\n[#4595]: https://github.com/tokio-rs/tokio/pull/4595\n[#4716]: https://github.com/tokio-rs/tokio/pull/4716\n[#4765]: https://github.com/tokio-rs/tokio/pull/4765\n[#4805]: https://github.com/tokio-rs/tokio/pull/4805\n[#4811]: https://github.com/tokio-rs/tokio/pull/4811\n[#4823]: https://github.com/tokio-rs/tokio/pull/4823\n[#4824]: https://github.com/tokio-rs/tokio/pull/4824\n[#4837]: https://github.com/tokio-rs/tokio/pull/4837\n[#4840]: https://github.com/tokio-rs/tokio/pull/4840\n[#4841]: https://github.com/tokio-rs/tokio/pull/4841\n[#4845]: https://github.com/tokio-rs/tokio/pull/4845\n[#4848]: https://github.com/tokio-rs/tokio/pull/4848\n[#4849]: https://github.com/tokio-rs/tokio/pull/4849\n[#4852]: https://github.com/tokio-rs/tokio/pull/4852\n[#4858]: https://github.com/tokio-rs/tokio/pull/4858\n[#4867]: https://github.com/tokio-rs/tokio/pull/4867\n[#4877]: https://github.com/tokio-rs/tokio/pull/4877\n[#4882]: https://github.com/tokio-rs/tokio/pull/4882\n[#4886]: https://github.com/tokio-rs/tokio/pull/4886\n[#4893]: https://github.com/tokio-rs/tokio/pull/4893\n[#4894]: https://github.com/tokio-rs/tokio/pull/4894\n[#4897]: https://github.com/tokio-rs/tokio/pull/4897\n[#4898]: https://github.com/tokio-rs/tokio/pull/4898\n[#4901]: https://github.com/tokio-rs/tokio/pull/4901\n[#4904]: https://github.com/tokio-rs/tokio/pull/4904\n[#4920]: https://github.com/tokio-rs/tokio/pull/4920\n[#4924]: https://github.com/tokio-rs/tokio/pull/4924\n[#4928]: https://github.com/tokio-rs/tokio/pull/4928\n[#4935]: https://github.com/tokio-rs/tokio/pull/4935\n[#4936]: https://github.com/tokio-rs/tokio/pull/4936\n[#4937]: https://github.com/tokio-rs/tokio/pull/4937\n[#4942]: https://github.com/tokio-rs/tokio/pull/4942\n[#4945]: https://github.com/tokio-rs/tokio/pull/4945\n[#4951]: https://github.com/tokio-rs/tokio/pull/4951\n[#4953]: https://github.com/tokio-rs/tokio/pull/4953\n[#4956]: https://github.com/tokio-rs/tokio/pull/4956\n[#4959]: https://github.com/tokio-rs/tokio/pull/4959\n\n# 1.20.6 (September 22, 2023)\n\nThis is a backport of a change from 1.27.0.\n\n### Changed\n\n- io: use `memchr` from `libc` ([#5960])\n\n[#5960]: https://github.com/tokio-rs/tokio/pull/5960\n\n# 1.20.5 (May 28, 2023)\n\nForward ports 1.18.6 changes.\n\n### Fixed\n\n- deps: disable default features for mio ([#5728])\n\n[#5728]: https://github.com/tokio-rs/tokio/pull/5728\n\n# 1.20.4 (January 17, 2023)\n\nForward ports 1.18.5 changes.\n\n### Fixed\n\n- io: fix unsoundness in `ReadHalf::unsplit` ([#5375])\n\n[#5375]: https://github.com/tokio-rs/tokio/pull/5375\n\n# 1.20.3 (January 3, 2022)\n\nThis release forward ports changes from 1.18.4.\n\n### Fixed\n\n- net: fix Windows named pipe server builder to maintain option when toggling\n pipe mode ([#5336]).\n\n[#5336]: https://github.com/tokio-rs/tokio/pull/5336\n\n# 1.20.2 (September 27, 2022)\n\nThis release removes the dependency on the `once_cell` crate to restore the MSRV\nof the 1.20.x LTS release. ([#5048])\n\n[#5048]: https://github.com/tokio-rs/tokio/pull/5048\n\n# 1.20.1 (July 25, 2022)\n\n### Fixed\n\n- chore: fix version detection in build script ([#4860])\n\n[#4860]: https://github.com/tokio-rs/tokio/pull/4860\n\n# 1.20.0 (July 12, 2022)\n\n### Added\n- tokio: add `track_caller` to public APIs ([#4772], [#4791], [#4793], [#4806], [#4808])\n- sync: Add `has_changed` method to `watch::Ref` ([#4758])\n\n### Changed\n\n- time: remove `src/time/driver/wheel/stack.rs` ([#4766])\n- rt: clean up arguments passed to basic scheduler ([#4767])\n- net: be more specific about winapi features ([#4764])\n- tokio: use const initialized thread locals where possible ([#4677])\n- task: various small improvements to LocalKey ([#4795])\n\n### Documented\n\n- fs: warn about performance pitfall ([#4762])\n- chore: fix spelling ([#4769])\n- sync: document spurious failures in oneshot ([#4777])\n- sync: add warning for watch in non-Send futures ([#4741])\n- chore: fix typo ([#4798])\n\n### Unstable\n\n- joinset: rename `join_one` to `join_next` ([#4755])\n- rt: unhandled panic config for current thread rt ([#4770])\n\n[#4677]: https://github.com/tokio-rs/tokio/pull/4677\n[#4741]: https://github.com/tokio-rs/tokio/pull/4741\n[#4755]: https://github.com/tokio-rs/tokio/pull/4755\n[#4758]: https://github.com/tokio-rs/tokio/pull/4758\n[#4762]: https://github.com/tokio-rs/tokio/pull/4762\n[#4764]: https://github.com/tokio-rs/tokio/pull/4764\n[#4766]: https://github.com/tokio-rs/tokio/pull/4766\n[#4767]: https://github.com/tokio-rs/tokio/pull/4767\n[#4769]: https://github.com/tokio-rs/tokio/pull/4769\n[#4770]: https://github.com/tokio-rs/tokio/pull/4770\n[#4772]: https://github.com/tokio-rs/tokio/pull/4772\n[#4777]: https://github.com/tokio-rs/tokio/pull/4777\n[#4791]: https://github.com/tokio-rs/tokio/pull/4791\n[#4793]: https://github.com/tokio-rs/tokio/pull/4793\n[#4795]: https://github.com/tokio-rs/tokio/pull/4795\n[#4798]: https://github.com/tokio-rs/tokio/pull/4798\n[#4806]: https://github.com/tokio-rs/tokio/pull/4806\n[#4808]: https://github.com/tokio-rs/tokio/pull/4808\n\n# 1.19.2 (June 6, 2022)\n\nThis release fixes another bug in `Notified::enable`. ([#4751])\n\n[#4751]: https://github.com/tokio-rs/tokio/pull/4751\n\n# 1.19.1 (June 5, 2022)\n\nThis release fixes a bug in `Notified::enable`. ([#4747])\n\n[#4747]: https://github.com/tokio-rs/tokio/pull/4747\n\n# 1.19.0 (June 3, 2022)\n\n### Added\n\n- runtime: add `is_finished` method for `JoinHandle` and `AbortHandle` ([#4709])\n- runtime: make global queue and event polling intervals configurable ([#4671])\n- sync: add `Notified::enable` ([#4705])\n- sync: add `watch::Sender::send_if_modified` ([#4591])\n- sync: add resubscribe method to broadcast::Receiver ([#4607])\n- net: add `take_error` to `TcpSocket` and `TcpStream` ([#4739])\n\n### Changed\n\n- io: refactor out usage of Weak in the io handle ([#4656])\n\n### Fixed\n\n- macros: avoid starvation in `join!` and `try_join!` ([#4624])\n\n### Documented\n\n- runtime: clarify semantics of tasks outliving `block_on` ([#4729])\n- time: fix example for `MissedTickBehavior::Burst` ([#4713])\n\n### Unstable\n\n- metrics: correctly update atomics in `IoDriverMetrics` ([#4725])\n- metrics: fix compilation with unstable, process, and rt, but without net ([#4682])\n- task: add `#[track_caller]` to `JoinSet`/`JoinMap` ([#4697])\n- task: add `Builder::{spawn_on, spawn_local_on, spawn_blocking_on}` ([#4683])\n- task: add `consume_budget` for cooperative scheduling ([#4498])\n- task: add `join_set::Builder` for configuring `JoinSet` tasks ([#4687])\n- task: update return value of `JoinSet::join_one` ([#4726])\n\n[#4498]: https://github.com/tokio-rs/tokio/pull/4498\n[#4591]: https://github.com/tokio-rs/tokio/pull/4591\n[#4607]: https://github.com/tokio-rs/tokio/pull/4607\n[#4624]: https://github.com/tokio-rs/tokio/pull/4624\n[#4656]: https://github.com/tokio-rs/tokio/pull/4656\n[#4671]: https://github.com/tokio-rs/tokio/pull/4671\n[#4682]: https://github.com/tokio-rs/tokio/pull/4682\n[#4683]: https://github.com/tokio-rs/tokio/pull/4683\n[#4687]: https://github.com/tokio-rs/tokio/pull/4687\n[#4697]: https://github.com/tokio-rs/tokio/pull/4697\n[#4705]: https://github.com/tokio-rs/tokio/pull/4705\n[#4709]: https://github.com/tokio-rs/tokio/pull/4709\n[#4713]: https://github.com/tokio-rs/tokio/pull/4713\n[#4725]: https://github.com/tokio-rs/tokio/pull/4725\n[#4726]: https://github.com/tokio-rs/tokio/pull/4726\n[#4729]: https://github.com/tokio-rs/tokio/pull/4729\n[#4739]: https://github.com/tokio-rs/tokio/pull/4739\n\n# 1.18.6 (May 28, 2023)\n\n### Fixed\n\n- deps: disable default features for mio ([#5728])\n\n[#5728]: https://github.com/tokio-rs/tokio/pull/5728\n\n# 1.18.5 (January 17, 2023)\n\n### Fixed\n\n- io: fix unsoundness in `ReadHalf::unsplit` ([#5375])\n\n[#5375]: https://github.com/tokio-rs/tokio/pull/5375\n\n# 1.18.4 (January 3, 2022)\n\n### Fixed\n\n- net: fix Windows named pipe server builder to maintain option when toggling\n pipe mode ([#5336]).\n\n[#5336]: https://github.com/tokio-rs/tokio/pull/5336\n\n# 1.18.3 (September 27, 2022)\n\nThis release removes the dependency on the `once_cell` crate to restore the MSRV\nof the 1.18.x LTS release. ([#5048])\n\n[#5048]: https://github.com/tokio-rs/tokio/pull/5048\n\n# 1.18.2 (May 5, 2022)\n\nAdd missing features for the `winapi` dependency. ([#4663])\n\n[#4663]: https://github.com/tokio-rs/tokio/pull/4663\n\n# 1.18.1 (May 2, 2022)\n\nThe 1.18.0 release broke the build for targets without 64-bit atomics when\nbuilding with `tokio_unstable`. This release fixes that. ([#4649])\n\n[#4649]: https://github.com/tokio-rs/tokio/pull/4649\n\n# 1.18.0 (April 27, 2022)\n\nThis release adds a number of new APIs in `tokio::net`, `tokio::signal`, and\n`tokio::sync`. In addition, it adds new unstable APIs to `tokio::task` (`Id`s\nfor uniquely identifying a task, and `AbortHandle` for remotely cancelling a\ntask), as well as a number of bugfixes.\n\n### Fixed\n\n- blocking: add missing `#[track_caller]` for `spawn_blocking` ([#4616])\n- macros: fix `select` macro to process 64 branches ([#4519])\n- net: fix `try_io` methods not calling Mio's `try_io` internally ([#4582])\n- runtime: recover when OS fails to spawn a new thread ([#4485])\n\n### Added\n\n- net: add `UdpSocket::peer_addr` ([#4611])\n- net: add `try_read_buf` method for named pipes ([#4626])\n- signal: add `SignalKind` `Hash`/`Eq` impls and `c_int` conversion ([#4540])\n- signal: add support for signals up to `SIGRTMAX` ([#4555])\n- sync: add `watch::Sender::send_modify` method ([#4310])\n- sync: add `broadcast::Receiver::len` method ([#4542])\n- sync: add `watch::Receiver::same_channel` method ([#4581])\n- sync: implement `Clone` for `RecvError` types ([#4560])\n\n### Changed\n\n- update `mio` to 0.8.1 ([#4582])\n- macros: rename `tokio::select!`'s internal `util` module ([#4543])\n- runtime: use `Vec::with_capacity` when building runtime ([#4553])\n\n### Documented\n\n- improve docs for `tokio_unstable` ([#4524])\n- runtime: include more documentation for thread_pool/worker ([#4511])\n- runtime: update `Handle::current`'s docs to mention `EnterGuard` ([#4567])\n- time: clarify platform specific timer resolution ([#4474])\n- signal: document that `Signal::recv` is cancel-safe ([#4634])\n- sync: `UnboundedReceiver` close docs ([#4548])\n\n### Unstable\n\nThe following changes only apply when building with `--cfg tokio_unstable`:\n\n- task: add `task::Id` type ([#4630])\n- task: add `AbortHandle` type for cancelling tasks in a `JoinSet` ([#4530],\n [#4640])\n- task: fix missing `doc(cfg(...))` attributes for `JoinSet` ([#4531])\n- task: fix broken link in `AbortHandle` RustDoc ([#4545])\n- metrics: add initial IO driver metrics ([#4507])\n\n\n[#4616]: https://github.com/tokio-rs/tokio/pull/4616\n[#4519]: https://github.com/tokio-rs/tokio/pull/4519\n[#4582]: https://github.com/tokio-rs/tokio/pull/4582\n[#4485]: https://github.com/tokio-rs/tokio/pull/4485\n[#4613]: https://github.com/tokio-rs/tokio/pull/4613\n[#4611]: https://github.com/tokio-rs/tokio/pull/4611\n[#4626]: https://github.com/tokio-rs/tokio/pull/4626\n[#4540]: https://github.com/tokio-rs/tokio/pull/4540\n[#4555]: https://github.com/tokio-rs/tokio/pull/4555\n[#4310]: https://github.com/tokio-rs/tokio/pull/4310\n[#4542]: https://github.com/tokio-rs/tokio/pull/4542\n[#4581]: https://github.com/tokio-rs/tokio/pull/4581\n[#4560]: https://github.com/tokio-rs/tokio/pull/4560\n[#4631]: https://github.com/tokio-rs/tokio/pull/4631\n[#4582]: https://github.com/tokio-rs/tokio/pull/4582\n[#4543]: https://github.com/tokio-rs/tokio/pull/4543\n[#4553]: https://github.com/tokio-rs/tokio/pull/4553\n[#4524]: https://github.com/tokio-rs/tokio/pull/4524\n[#4511]: https://github.com/tokio-rs/tokio/pull/4511\n[#4567]: https://github.com/tokio-rs/tokio/pull/4567\n[#4474]: https://github.com/tokio-rs/tokio/pull/4474\n[#4634]: https://github.com/tokio-rs/tokio/pull/4634\n[#4548]: https://github.com/tokio-rs/tokio/pull/4548\n[#4630]: https://github.com/tokio-rs/tokio/pull/4630\n[#4530]: https://github.com/tokio-rs/tokio/pull/4530\n[#4640]: https://github.com/tokio-rs/tokio/pull/4640\n[#4531]: https://github.com/tokio-rs/tokio/pull/4531\n[#4545]: https://github.com/tokio-rs/tokio/pull/4545\n[#4507]: https://github.com/tokio-rs/tokio/pull/4507\n\n# 1.17.0 (February 16, 2022)\n\nThis release updates the minimum supported Rust version (MSRV) to 1.49, the\n`mio` dependency to v0.8, and the (optional) `parking_lot` dependency to v0.12.\nAdditionally, it contains several bug fixes, as well as internal refactoring and\nperformance improvements.\n\n### Fixed\n\n- time: prevent panicking in `sleep` with large durations ([#4495])\n- time: eliminate potential panics in `Instant` arithmetic on platforms where\n `Instant::now` is not monotonic ([#4461])\n- io: fix `DuplexStream` not participating in cooperative yielding ([#4478])\n- rt: fix potential double panic when dropping a `JoinHandle` ([#4430])\n\n### Changed\n\n- update minimum supported Rust version to 1.49 ([#4457])\n- update `parking_lot` dependency to v0.12.0 ([#4459])\n- update `mio` dependency to v0.8 ([#4449])\n- rt: remove an unnecessary lock in the blocking pool ([#4436])\n- rt: remove an unnecessary enum in the basic scheduler ([#4462])\n- time: use bit manipulation instead of modulo to improve performance ([#4480])\n- net: use `std::future::Ready` instead of our own `Ready` future ([#4271])\n- replace deprecated `atomic::spin_loop_hint` with `hint::spin_loop` ([#4491])\n- fix miri failures in intrusive linked lists ([#4397])\n\n### Documented\n\n- io: add an example for `tokio::process::ChildStdin` ([#4479])\n\n### Unstable\n\nThe following changes only apply when building with `--cfg tokio_unstable`:\n\n- task: fix missing location information in `tracing` spans generated by\n `spawn_local` ([#4483])\n- task: add `JoinSet` for managing sets of tasks ([#4335])\n- metrics: fix compilation error on MIPS ([#4475])\n- metrics: fix compilation error on arm32v7 ([#4453])\n\n[#4495]: https://github.com/tokio-rs/tokio/pull/4495\n[#4461]: https://github.com/tokio-rs/tokio/pull/4461\n[#4478]: https://github.com/tokio-rs/tokio/pull/4478\n[#4430]: https://github.com/tokio-rs/tokio/pull/4430\n[#4457]: https://github.com/tokio-rs/tokio/pull/4457\n[#4459]: https://github.com/tokio-rs/tokio/pull/4459\n[#4449]: https://github.com/tokio-rs/tokio/pull/4449\n[#4462]: https://github.com/tokio-rs/tokio/pull/4462\n[#4436]: https://github.com/tokio-rs/tokio/pull/4436\n[#4480]: https://github.com/tokio-rs/tokio/pull/4480\n[#4271]: https://github.com/tokio-rs/tokio/pull/4271\n[#4491]: https://github.com/tokio-rs/tokio/pull/4491\n[#4397]: https://github.com/tokio-rs/tokio/pull/4397\n[#4479]: https://github.com/tokio-rs/tokio/pull/4479\n[#4483]: https://github.com/tokio-rs/tokio/pull/4483\n[#4335]: https://github.com/tokio-rs/tokio/pull/4335\n[#4475]: https://github.com/tokio-rs/tokio/pull/4475\n[#4453]: https://github.com/tokio-rs/tokio/pull/4453\n\n# 1.16.1 (January 28, 2022)\n\nThis release fixes a bug in [#4428] with the change [#4437].\n\n[#4428]: https://github.com/tokio-rs/tokio/pull/4428\n[#4437]: https://github.com/tokio-rs/tokio/pull/4437\n\n# 1.16.0 (January 27, 2022)\n\nFixes a soundness bug in `io::Take` ([#4428]). The unsoundness is exposed when\nleaking memory in the given `AsyncRead` implementation and then overwriting the\nsupplied buffer:\n\n```rust\nimpl AsyncRead for Buggy {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>\n ) -> Poll> {\n let new_buf = vec![0; 5].leak();\n *buf = ReadBuf::new(new_buf);\n buf.put_slice(b\"hello\");\n Poll::Ready(Ok(()))\n }\n}\n```\n\nAlso, this release includes improvements to the multi-threaded scheduler that\ncan increase throughput by up to 20% in some cases ([#4383]).\n\n### Fixed\n\n- io: **soundness** don't expose uninitialized memory when using `io::Take` in edge case ([#4428])\n- fs: ensure `File::write` results in a `write` syscall when the runtime shuts down ([#4316])\n- process: drop pipe after child exits in `wait_with_output` ([#4315])\n- rt: improve error message when spawning a thread fails ([#4398])\n- rt: reduce false-positive thread wakups in the multi-threaded scheduler ([#4383])\n- sync: don't inherit `Send` from `parking_lot::*Guard` ([#4359])\n\n### Added\n\n- net: `TcpSocket::linger()` and `set_linger()` ([#4324])\n- net: impl `UnwindSafe` for socket types ([#4384])\n- rt: impl `UnwindSafe` for `JoinHandle` ([#4418])\n- sync: `watch::Receiver::has_changed()` ([#4342])\n- sync: `oneshot::Receiver::blocking_recv()` ([#4334])\n- sync: `RwLock` blocking operations ([#4425])\n\n### Unstable\n\nThe following changes only apply when building with `--cfg tokio_unstable`\n\n- rt: **breaking change** overhaul runtime metrics API ([#4373])\n\n[#4428]: https://github.com/tokio-rs/tokio/pull/4428\n[#4316]: https://github.com/tokio-rs/tokio/pull/4316\n[#4315]: https://github.com/tokio-rs/tokio/pull/4315\n[#4398]: https://github.com/tokio-rs/tokio/pull/4398\n[#4383]: https://github.com/tokio-rs/tokio/pull/4383\n[#4359]: https://github.com/tokio-rs/tokio/pull/4359\n[#4324]: https://github.com/tokio-rs/tokio/pull/4324\n[#4384]: https://github.com/tokio-rs/tokio/pull/4384\n[#4418]: https://github.com/tokio-rs/tokio/pull/4418\n[#4342]: https://github.com/tokio-rs/tokio/pull/4342\n[#4334]: https://github.com/tokio-rs/tokio/pull/4334\n[#4425]: https://github.com/tokio-rs/tokio/pull/4425\n[#4373]: https://github.com/tokio-rs/tokio/pull/4373\n\n# 1.15.0 (December 15, 2021)\n\n### Fixed\n\n- io: add cooperative yielding support to `io::empty()` ([#4300])\n- time: make timeout robust against budget-depleting tasks ([#4314])\n\n### Changed\n\n- update minimum supported Rust version to 1.46.\n\n### Added\n\n- time: add `Interval::reset()` ([#4248])\n- io: add explicit lifetimes to `AsyncFdReadyGuard` ([#4267])\n- process: add `Command::as_std()` ([#4295])\n\n### Added (unstable)\n\n- tracing: instrument `tokio::sync` types ([#4302])\n\n[#4302]: https://github.com/tokio-rs/tokio/pull/4302\n[#4300]: https://github.com/tokio-rs/tokio/pull/4300\n[#4295]: https://github.com/tokio-rs/tokio/pull/4295\n[#4267]: https://github.com/tokio-rs/tokio/pull/4267\n[#4248]: https://github.com/tokio-rs/tokio/pull/4248\n[#4314]: https://github.com/tokio-rs/tokio/pull/4314\n\n# 1.14.0 (November 15, 2021)\n\n### Fixed\n\n- macros: fix compiler errors when using `mut` patterns in `select!` ([#4211])\n- sync: fix a data race between `oneshot::Sender::send` and awaiting a\n `oneshot::Receiver` when the oneshot has been closed ([#4226])\n- sync: make `AtomicWaker` panic safe ([#3689])\n- runtime: fix basic scheduler dropping tasks outside a runtime context\n ([#4213])\n\n### Added\n\n- stats: add `RuntimeStats::busy_duration_total` ([#4179], [#4223])\n\n### Changed\n\n- io: updated `copy` buffer size to match `std::io::copy` ([#4209])\n\n### Documented\n\n- io: rename buffer to file in doc-test ([#4230])\n- sync: fix Notify example ([#4212])\n\n[#4211]: https://github.com/tokio-rs/tokio/pull/4211\n[#4226]: https://github.com/tokio-rs/tokio/pull/4226\n[#3689]: https://github.com/tokio-rs/tokio/pull/3689\n[#4213]: https://github.com/tokio-rs/tokio/pull/4213\n[#4179]: https://github.com/tokio-rs/tokio/pull/4179\n[#4223]: https://github.com/tokio-rs/tokio/pull/4223\n[#4209]: https://github.com/tokio-rs/tokio/pull/4209\n[#4230]: https://github.com/tokio-rs/tokio/pull/4230\n[#4212]: https://github.com/tokio-rs/tokio/pull/4212\n\n# 1.13.1 (November 15, 2021)\n\n### Fixed\n\n- sync: fix a data race between `oneshot::Sender::send` and awaiting a\n `oneshot::Receiver` when the oneshot has been closed ([#4226])\n\n[#4226]: https://github.com/tokio-rs/tokio/pull/4226\n\n# 1.13.0 (October 29, 2021)\n\n### Fixed\n\n- sync: fix `Notify` to clone the waker before locking its waiter list ([#4129])\n- tokio: add riscv32 to non atomic64 architectures ([#4185])\n\n### Added\n\n- net: add `poll_{recv,send}_ready` methods to `udp` and `uds_datagram` ([#4131])\n- net: add `try_*`, `readable`, `writable`, `ready`, and `peer_addr` methods to split halves ([#4120])\n- sync: add `blocking_lock` to `Mutex` ([#4130])\n- sync: add `watch::Sender::send_replace` ([#3962], [#4195])\n- sync: expand `Debug` for `Mutex` impl to unsized `T` ([#4134])\n- tracing: instrument time::Sleep ([#4072])\n- tracing: use structured location fields for spawned tasks ([#4128])\n\n### Changed\n\n- io: add assert in `copy_bidirectional` that `poll_write` is sensible ([#4125])\n- macros: use qualified syntax when polling in `select!` ([#4192])\n- runtime: handle `block_on` wakeups better ([#4157])\n- task: allocate callback on heap immediately in debug mode ([#4203])\n- tokio: assert platform-minimum requirements at build time ([#3797])\n\n### Documented\n\n- docs: conversion of doc comments to indicative mood ([#4174])\n- docs: add returning on the first error example for `try_join!` ([#4133])\n- docs: fixing broken links in `tokio/src/lib.rs` ([#4132])\n- signal: add example with background listener ([#4171])\n- sync: add more oneshot examples ([#4153])\n- time: document `Interval::tick` cancel safety ([#4152])\n\n[#3797]: https://github.com/tokio-rs/tokio/pull/3797\n[#3962]: https://github.com/tokio-rs/tokio/pull/3962\n[#4072]: https://github.com/tokio-rs/tokio/pull/4072\n[#4120]: https://github.com/tokio-rs/tokio/pull/4120\n[#4125]: https://github.com/tokio-rs/tokio/pull/4125\n[#4128]: https://github.com/tokio-rs/tokio/pull/4128\n[#4129]: https://github.com/tokio-rs/tokio/pull/4129\n[#4130]: https://github.com/tokio-rs/tokio/pull/4130\n[#4131]: https://github.com/tokio-rs/tokio/pull/4131\n[#4132]: https://github.com/tokio-rs/tokio/pull/4132\n[#4133]: https://github.com/tokio-rs/tokio/pull/4133\n[#4134]: https://github.com/tokio-rs/tokio/pull/4134\n[#4152]: https://github.com/tokio-rs/tokio/pull/4152\n[#4153]: https://github.com/tokio-rs/tokio/pull/4153\n[#4157]: https://github.com/tokio-rs/tokio/pull/4157\n[#4171]: https://github.com/tokio-rs/tokio/pull/4171\n[#4174]: https://github.com/tokio-rs/tokio/pull/4174\n[#4185]: https://github.com/tokio-rs/tokio/pull/4185\n[#4192]: https://github.com/tokio-rs/tokio/pull/4192\n[#4195]: https://github.com/tokio-rs/tokio/pull/4195\n[#4203]: https://github.com/tokio-rs/tokio/pull/4203\n\n# 1.12.0 (September 21, 2021)\n\n### Fixed\n\n- mpsc: ensure `try_reserve` error is consistent with `try_send` ([#4119])\n- mpsc: use `spin_loop_hint` instead of `yield_now` ([#4115])\n- sync: make `SendError` field public ([#4097])\n\n### Added\n\n- io: add POSIX AIO on FreeBSD ([#4054])\n- io: add convenience method `AsyncSeekExt::rewind` ([#4107])\n- runtime: add tracing span for `block_on` futures ([#4094])\n- runtime: callback when a worker parks and unparks ([#4070])\n- sync: implement `try_recv` for mpsc channels ([#4113])\n\n### Documented\n\n- docs: clarify CPU-bound tasks on Tokio ([#4105])\n- mpsc: document spurious failures on `poll_recv` ([#4117])\n- mpsc: document that `PollSender` impls `Sink` ([#4110])\n- task: document non-guarantees of `yield_now` ([#4091])\n- time: document paused time details better ([#4061], [#4103])\n\n[#4027]: https://github.com/tokio-rs/tokio/pull/4027\n[#4054]: https://github.com/tokio-rs/tokio/pull/4054\n[#4061]: https://github.com/tokio-rs/tokio/pull/4061\n[#4070]: https://github.com/tokio-rs/tokio/pull/4070\n[#4091]: https://github.com/tokio-rs/tokio/pull/4091\n[#4094]: https://github.com/tokio-rs/tokio/pull/4094\n[#4097]: https://github.com/tokio-rs/tokio/pull/4097\n[#4103]: https://github.com/tokio-rs/tokio/pull/4103\n[#4105]: https://github.com/tokio-rs/tokio/pull/4105\n[#4107]: https://github.com/tokio-rs/tokio/pull/4107\n[#4110]: https://github.com/tokio-rs/tokio/pull/4110\n[#4113]: https://github.com/tokio-rs/tokio/pull/4113\n[#4115]: https://github.com/tokio-rs/tokio/pull/4115\n[#4117]: https://github.com/tokio-rs/tokio/pull/4117\n[#4119]: https://github.com/tokio-rs/tokio/pull/4119\n\n# 1.11.0 (August 31, 2021)\n\n### Fixed\n\n - time: don't panic when Instant is not monotonic ([#4044])\n - io: fix panic in `fill_buf` by not calling `poll_fill_buf` twice ([#4084])\n\n### Added\n\n - watch: add `watch::Sender::subscribe` ([#3800])\n - process: add `from_std` to `ChildStd*` ([#4045])\n - stats: initial work on runtime stats ([#4043])\n\n### Changed\n\n - tracing: change span naming to new console convention ([#4042])\n - io: speed-up waking by using uninitialized array ([#4055], [#4071], [#4075])\n\n### Documented\n\n - time: make Sleep examples easier to find ([#4040])\n\n[#3800]: https://github.com/tokio-rs/tokio/pull/3800\n[#4040]: https://github.com/tokio-rs/tokio/pull/4040\n[#4042]: https://github.com/tokio-rs/tokio/pull/4042\n[#4043]: https://github.com/tokio-rs/tokio/pull/4043\n[#4044]: https://github.com/tokio-rs/tokio/pull/4044\n[#4045]: https://github.com/tokio-rs/tokio/pull/4045\n[#4055]: https://github.com/tokio-rs/tokio/pull/4055\n[#4071]: https://github.com/tokio-rs/tokio/pull/4071\n[#4075]: https://github.com/tokio-rs/tokio/pull/4075\n[#4084]: https://github.com/tokio-rs/tokio/pull/4084\n\n# 1.10.1 (August 24, 2021)\n\n### Fixed\n\n - runtime: fix leak in UnownedTask ([#4063])\n\n[#4063]: https://github.com/tokio-rs/tokio/pull/4063\n\n# 1.10.0 (August 12, 2021)\n\n### Added\n\n - io: add `(read|write)_f(32|64)[_le]` methods ([#4022])\n - io: add `fill_buf` and `consume` to `AsyncBufReadExt` ([#3991])\n - process: add `Child::raw_handle()` on windows ([#3998])\n\n### Fixed\n\n - doc: fix non-doc builds with `--cfg docsrs` ([#4020])\n - io: flush eagerly in `io::copy` ([#4001])\n - runtime: a debug assert was sometimes triggered during shutdown ([#4005])\n - sync: use `spin_loop_hint` instead of `yield_now` in mpsc ([#4037])\n - tokio: the test-util feature depends on rt, sync, and time ([#4036])\n\n### Changes\n\n - runtime: reorganize parts of the runtime ([#3979], [#4005])\n - signal: make windows docs for signal module show up on unix builds ([#3770])\n - task: quickly send task to heap on debug mode ([#4009])\n\n### Documented\n\n - io: document cancellation safety of `AsyncBufReadExt` ([#3997])\n - sync: document when `watch::send` fails ([#4021])\n\n[#3770]: https://github.com/tokio-rs/tokio/pull/3770\n[#3979]: https://github.com/tokio-rs/tokio/pull/3979\n[#3991]: https://github.com/tokio-rs/tokio/pull/3991\n[#3997]: https://github.com/tokio-rs/tokio/pull/3997\n[#3998]: https://github.com/tokio-rs/tokio/pull/3998\n[#4001]: https://github.com/tokio-rs/tokio/pull/4001\n[#4005]: https://github.com/tokio-rs/tokio/pull/4005\n[#4009]: https://github.com/tokio-rs/tokio/pull/4009\n[#4020]: https://github.com/tokio-rs/tokio/pull/4020\n[#4021]: https://github.com/tokio-rs/tokio/pull/4021\n[#4022]: https://github.com/tokio-rs/tokio/pull/4022\n[#4036]: https://github.com/tokio-rs/tokio/pull/4036\n[#4037]: https://github.com/tokio-rs/tokio/pull/4037\n\n# 1.9.0 (July 22, 2021)\n\n### Added\n\n - net: allow customized I/O operations for `TcpStream` ([#3888])\n - sync: add getter for the mutex from a guard ([#3928])\n - task: expose nameable future for `TaskLocal::scope` ([#3273])\n\n### Fixed\n\n - Fix leak if output of future panics on drop ([#3967])\n - Fix leak in `LocalSet` ([#3978])\n\n### Changes\n\n - runtime: reorganize parts of the runtime ([#3909], [#3939], [#3950], [#3955], [#3980])\n - sync: clean up `OnceCell` ([#3945])\n - task: remove mutex in `JoinError` ([#3959])\n\n[#3273]: https://github.com/tokio-rs/tokio/pull/3273\n[#3888]: https://github.com/tokio-rs/tokio/pull/3888\n[#3909]: https://github.com/tokio-rs/tokio/pull/3909\n[#3928]: https://github.com/tokio-rs/tokio/pull/3928\n[#3934]: https://github.com/tokio-rs/tokio/pull/3934\n[#3939]: https://github.com/tokio-rs/tokio/pull/3939\n[#3945]: https://github.com/tokio-rs/tokio/pull/3945\n[#3950]: https://github.com/tokio-rs/tokio/pull/3950\n[#3955]: https://github.com/tokio-rs/tokio/pull/3955\n[#3959]: https://github.com/tokio-rs/tokio/pull/3959\n[#3967]: https://github.com/tokio-rs/tokio/pull/3967\n[#3978]: https://github.com/tokio-rs/tokio/pull/3978\n[#3980]: https://github.com/tokio-rs/tokio/pull/3980\n\n# 1.8.3 (July 26, 2021)\n\nThis release backports two fixes from 1.9.0\n\n### Fixed\n\n - Fix leak if output of future panics on drop ([#3967])\n - Fix leak in `LocalSet` ([#3978])\n\n[#3967]: https://github.com/tokio-rs/tokio/pull/3967\n[#3978]: https://github.com/tokio-rs/tokio/pull/3978\n\n# 1.8.2 (July 19, 2021)\n\nFixes a missed edge case from 1.8.1.\n\n### Fixed\n\n- runtime: drop canceled future on next poll ([#3965])\n\n[#3965]: https://github.com/tokio-rs/tokio/pull/3965\n\n# 1.8.1 (July 6, 2021)\n\nForward ports 1.5.1 fixes.\n\n### Fixed\n\n- runtime: remotely abort tasks on `JoinHandle::abort` ([#3934])\n\n[#3934]: https://github.com/tokio-rs/tokio/pull/3934\n\n# 1.8.0 (July 2, 2021)\n\n### Added\n\n- io: add `get_{ref,mut}` methods to `AsyncFdReadyGuard` and `AsyncFdReadyMutGuard` ([#3807])\n- io: efficient implementation of vectored writes for `BufWriter` ([#3163])\n- net: add ready/try methods to `NamedPipe{Client,Server}` ([#3866], [#3899])\n- sync: add `watch::Receiver::borrow_and_update` ([#3813])\n- sync: implement `From` for `OnceCell` ([#3877])\n- time: allow users to specify Interval behavior when delayed ([#3721])\n\n### Added (unstable)\n\n- rt: add `tokio::task::Builder` ([#3881])\n\n### Fixed\n\n- net: handle HUP event with `UnixStream` ([#3898])\n\n### Documented\n\n- doc: document cancellation safety ([#3900])\n- time: add wait alias to sleep ([#3897])\n- time: document auto-advancing behavior of runtime ([#3763])\n\n[#3163]: https://github.com/tokio-rs/tokio/pull/3163\n[#3721]: https://github.com/tokio-rs/tokio/pull/3721\n[#3763]: https://github.com/tokio-rs/tokio/pull/3763\n[#3807]: https://github.com/tokio-rs/tokio/pull/3807\n[#3813]: https://github.com/tokio-rs/tokio/pull/3813\n[#3866]: https://github.com/tokio-rs/tokio/pull/3866\n[#3877]: https://github.com/tokio-rs/tokio/pull/3877\n[#3881]: https://github.com/tokio-rs/tokio/pull/3881\n[#3897]: https://github.com/tokio-rs/tokio/pull/3897\n[#3898]: https://github.com/tokio-rs/tokio/pull/3898\n[#3899]: https://github.com/tokio-rs/tokio/pull/3899\n[#3900]: https://github.com/tokio-rs/tokio/pull/3900\n\n# 1.7.2 (July 6, 2021)\n\nForward ports 1.5.1 fixes.\n\n### Fixed\n\n- runtime: remotely abort tasks on `JoinHandle::abort` ([#3934])\n\n[#3934]: https://github.com/tokio-rs/tokio/pull/3934\n\n# 1.7.1 (June 18, 2021)\n\n### Fixed\n\n- runtime: fix early task shutdown during runtime shutdown ([#3870])\n\n[#3870]: https://github.com/tokio-rs/tokio/pull/3870\n\n# 1.7.0 (June 15, 2021)\n\n### Added\n\n- net: add named pipes on windows ([#3760])\n- net: add `TcpSocket` from `std::net::TcpStream` conversion ([#3838])\n- sync: add `receiver_count` to `watch::Sender` ([#3729])\n- sync: export `sync::notify::Notified` future publicly ([#3840])\n- tracing: instrument task wakers ([#3836])\n\n### Fixed\n\n- macros: suppress `clippy::default_numeric_fallback` lint in generated code ([#3831])\n- runtime: immediately drop new tasks when runtime is shut down ([#3752])\n- sync: deprecate unused `mpsc::RecvError` type ([#3833])\n\n### Documented\n\n- io: clarify EOF condition for `AsyncReadExt::read_buf` ([#3850])\n- io: clarify limits on return values of `AsyncWrite::poll_write` ([#3820])\n- sync: add examples to Semaphore ([#3808])\n\n[#3729]: https://github.com/tokio-rs/tokio/pull/3729\n[#3752]: https://github.com/tokio-rs/tokio/pull/3752\n[#3760]: https://github.com/tokio-rs/tokio/pull/3760\n[#3808]: https://github.com/tokio-rs/tokio/pull/3808\n[#3820]: https://github.com/tokio-rs/tokio/pull/3820\n[#3831]: https://github.com/tokio-rs/tokio/pull/3831\n[#3833]: https://github.com/tokio-rs/tokio/pull/3833\n[#3836]: https://github.com/tokio-rs/tokio/pull/3836\n[#3838]: https://github.com/tokio-rs/tokio/pull/3838\n[#3840]: https://github.com/tokio-rs/tokio/pull/3840\n[#3850]: https://github.com/tokio-rs/tokio/pull/3850\n\n# 1.6.3 (July 6, 2021)\n\nForward ports 1.5.1 fixes.\n\n### Fixed\n\n- runtime: remotely abort tasks on `JoinHandle::abort` ([#3934])\n\n[#3934]: https://github.com/tokio-rs/tokio/pull/3934\n\n# 1.6.2 (June 14, 2021)\n\n### Fixes\n\n- test: sub-ms `time:advance` regression introduced in 1.6 ([#3852])\n\n[#3852]: https://github.com/tokio-rs/tokio/pull/3852\n\n# 1.6.1 (May 28, 2021)\n\nThis release reverts [#3518] because it doesn't work on some kernels due to\na kernel bug. ([#3803])\n\n[#3518]: https://github.com/tokio-rs/tokio/issues/3518\n[#3803]: https://github.com/tokio-rs/tokio/issues/3803\n\n# 1.6.0 (May 14, 2021)\n\n### Added\n\n- fs: try doing a non-blocking read before punting to the threadpool ([#3518])\n- io: add `write_all_buf` to `AsyncWriteExt` ([#3737])\n- io: implement `AsyncSeek` for `BufReader`, `BufWriter`, and `BufStream` ([#3491])\n- net: support non-blocking vectored I/O ([#3761])\n- sync: add `mpsc::Sender::{reserve_owned, try_reserve_owned}` ([#3704])\n- sync: add a `MutexGuard::map` method that returns a `MappedMutexGuard` ([#2472])\n- time: add getter for Interval's period ([#3705])\n\n### Fixed\n\n- io: wake pending writers on `DuplexStream` close ([#3756])\n- process: avoid redundant effort to reap orphan processes ([#3743])\n- signal: use `std::os::raw::c_int` instead of `libc::c_int` on public API ([#3774])\n- sync: preserve permit state in `notify_waiters` ([#3660])\n- task: update `JoinHandle` panic message ([#3727])\n- time: prevent `time::advance` from going too far ([#3712])\n\n### Documented\n\n- net: hide `net::unix::datagram` module from docs ([#3775])\n- process: updated example ([#3748])\n- sync: `Barrier` doc should use task, not thread ([#3780])\n- task: update documentation on `block_in_place` ([#3753])\n\n[#2472]: https://github.com/tokio-rs/tokio/pull/2472\n[#3491]: https://github.com/tokio-rs/tokio/pull/3491\n[#3518]: https://github.com/tokio-rs/tokio/pull/3518\n[#3660]: https://github.com/tokio-rs/tokio/pull/3660\n[#3704]: https://github.com/tokio-rs/tokio/pull/3704\n[#3705]: https://github.com/tokio-rs/tokio/pull/3705\n[#3712]: https://github.com/tokio-rs/tokio/pull/3712\n[#3727]: https://github.com/tokio-rs/tokio/pull/3727\n[#3737]: https://github.com/tokio-rs/tokio/pull/3737\n[#3743]: https://github.com/tokio-rs/tokio/pull/3743\n[#3748]: https://github.com/tokio-rs/tokio/pull/3748\n[#3753]: https://github.com/tokio-rs/tokio/pull/3753\n[#3756]: https://github.com/tokio-rs/tokio/pull/3756\n[#3761]: https://github.com/tokio-rs/tokio/pull/3761\n[#3774]: https://github.com/tokio-rs/tokio/pull/3774\n[#3775]: https://github.com/tokio-rs/tokio/pull/3775\n[#3780]: https://github.com/tokio-rs/tokio/pull/3780\n\n# 1.5.1 (July 6, 2021)\n\n### Fixed\n\n- runtime: remotely abort tasks on `JoinHandle::abort` ([#3934])\n\n[#3934]: https://github.com/tokio-rs/tokio/pull/3934\n\n# 1.5.0 (April 12, 2021)\n\n### Added\n\n- io: add `AsyncSeekExt::stream_position` ([#3650])\n- io: add `AsyncWriteExt::write_vectored` ([#3678])\n- io: add a `copy_bidirectional` utility ([#3572])\n- net: implement `IntoRawFd` for `TcpSocket` ([#3684])\n- sync: add `OnceCell` ([#3591])\n- sync: add `OwnedRwLockReadGuard` and `OwnedRwLockWriteGuard` ([#3340])\n- sync: add `Semaphore::is_closed` ([#3673])\n- sync: add `mpsc::Sender::capacity` ([#3690])\n- sync: allow configuring `RwLock` max reads ([#3644])\n- task: add `sync_scope` for `LocalKey` ([#3612])\n\n### Fixed\n\n- chore: try to avoid `noalias` attributes on intrusive linked list ([#3654])\n- rt: fix panic in `JoinHandle::abort()` when called from other threads ([#3672])\n- sync: don't panic in `oneshot::try_recv` ([#3674])\n- sync: fix notifications getting dropped on receiver drop ([#3652])\n- sync: fix `Semaphore` permit overflow calculation ([#3644])\n\n### Documented\n\n- io: clarify requirements of `AsyncFd` ([#3635])\n- runtime: fix unclear docs for `{Handle,Runtime}::block_on` ([#3628])\n- sync: document that `Semaphore` is fair ([#3693])\n- sync: improve doc on blocking mutex ([#3645])\n\n[#3340]: https://github.com/tokio-rs/tokio/pull/3340\n[#3572]: https://github.com/tokio-rs/tokio/pull/3572\n[#3591]: https://github.com/tokio-rs/tokio/pull/3591\n[#3612]: https://github.com/tokio-rs/tokio/pull/3612\n[#3628]: https://github.com/tokio-rs/tokio/pull/3628\n[#3635]: https://github.com/tokio-rs/tokio/pull/3635\n[#3644]: https://github.com/tokio-rs/tokio/pull/3644\n[#3645]: https://github.com/tokio-rs/tokio/pull/3645\n[#3650]: https://github.com/tokio-rs/tokio/pull/3650\n[#3652]: https://github.com/tokio-rs/tokio/pull/3652\n[#3654]: https://github.com/tokio-rs/tokio/pull/3654\n[#3672]: https://github.com/tokio-rs/tokio/pull/3672\n[#3673]: https://github.com/tokio-rs/tokio/pull/3673\n[#3674]: https://github.com/tokio-rs/tokio/pull/3674\n[#3678]: https://github.com/tokio-rs/tokio/pull/3678\n[#3684]: https://github.com/tokio-rs/tokio/pull/3684\n[#3690]: https://github.com/tokio-rs/tokio/pull/3690\n[#3693]: https://github.com/tokio-rs/tokio/pull/3693\n\n# 1.4.0 (March 20, 2021)\n\n### Added\n\n- macros: introduce biased argument for `select!` ([#3603])\n- runtime: add `Handle::block_on` ([#3569])\n\n### Fixed\n\n- runtime: avoid unnecessary polling of `block_on` future ([#3582])\n- runtime: fix memory leak/growth when creating many runtimes ([#3564])\n- runtime: mark `EnterGuard` with `must_use` ([#3609])\n\n### Documented\n\n- chore: mention fix for building docs in contributing guide ([#3618])\n- doc: add link to `PollSender` ([#3613])\n- doc: alias sleep to delay ([#3604])\n- sync: improve `Mutex` FIFO explanation ([#3615])\n- timer: fix double newline in module docs ([#3617])\n\n[#3564]: https://github.com/tokio-rs/tokio/pull/3564\n[#3613]: https://github.com/tokio-rs/tokio/pull/3613\n[#3618]: https://github.com/tokio-rs/tokio/pull/3618\n[#3617]: https://github.com/tokio-rs/tokio/pull/3617\n[#3582]: https://github.com/tokio-rs/tokio/pull/3582\n[#3615]: https://github.com/tokio-rs/tokio/pull/3615\n[#3603]: https://github.com/tokio-rs/tokio/pull/3603\n[#3609]: https://github.com/tokio-rs/tokio/pull/3609\n[#3604]: https://github.com/tokio-rs/tokio/pull/3604\n[#3569]: https://github.com/tokio-rs/tokio/pull/3569\n\n# 1.3.0 (March 9, 2021)\n\n### Added\n\n- coop: expose an `unconstrained()` opt-out ([#3547])\n- net: add `into_std` for net types without it ([#3509])\n- sync: add `same_channel` method to `mpsc::Sender` ([#3532])\n- sync: add `{try_,}acquire_many_owned` to `Semaphore` ([#3535])\n- sync: add back `RwLockWriteGuard::map` and `RwLockWriteGuard::try_map` ([#3348])\n\n### Fixed\n\n- sync: allow `oneshot::Receiver::close` after successful `try_recv` ([#3552])\n- time: do not panic on `timeout(Duration::MAX)` ([#3551])\n\n### Documented\n\n- doc: doc aliases for pre-1.0 function names ([#3523])\n- io: fix typos ([#3541])\n- io: note the EOF behavior of `read_until` ([#3536])\n- io: update `AsyncRead::poll_read` doc ([#3557])\n- net: update `UdpSocket` splitting doc ([#3517])\n- runtime: add link to `LocalSet` on `new_current_thread` ([#3508])\n- runtime: update documentation of thread limits ([#3527])\n- sync: do not recommend `join_all` for `Barrier` ([#3514])\n- sync: documentation for `oneshot` ([#3592])\n- sync: rename `notify` to `notify_one` ([#3526])\n- time: fix typo in `Sleep` doc ([#3515])\n- time: sync `interval.rs` and `time/mod.rs` docs ([#3533])\n\n[#3348]: https://github.com/tokio-rs/tokio/pull/3348\n[#3508]: https://github.com/tokio-rs/tokio/pull/3508\n[#3509]: https://github.com/tokio-rs/tokio/pull/3509\n[#3514]: https://github.com/tokio-rs/tokio/pull/3514\n[#3515]: https://github.com/tokio-rs/tokio/pull/3515\n[#3517]: https://github.com/tokio-rs/tokio/pull/3517\n[#3523]: https://github.com/tokio-rs/tokio/pull/3523\n[#3526]: https://github.com/tokio-rs/tokio/pull/3526\n[#3527]: https://github.com/tokio-rs/tokio/pull/3527\n[#3532]: https://github.com/tokio-rs/tokio/pull/3532\n[#3533]: https://github.com/tokio-rs/tokio/pull/3533\n[#3535]: https://github.com/tokio-rs/tokio/pull/3535\n[#3536]: https://github.com/tokio-rs/tokio/pull/3536\n[#3541]: https://github.com/tokio-rs/tokio/pull/3541\n[#3547]: https://github.com/tokio-rs/tokio/pull/3547\n[#3551]: https://github.com/tokio-rs/tokio/pull/3551\n[#3552]: https://github.com/tokio-rs/tokio/pull/3552\n[#3557]: https://github.com/tokio-rs/tokio/pull/3557\n[#3592]: https://github.com/tokio-rs/tokio/pull/3592\n\n# 1.2.0 (February 5, 2021)\n\n### Added\n\n- signal: make `Signal::poll_recv` method public ([#3383])\n\n### Fixed\n\n- time: make `test-util` paused time fully deterministic ([#3492])\n\n### Documented\n\n- sync: link to new broadcast and watch wrappers ([#3504])\n\n[#3383]: https://github.com/tokio-rs/tokio/pull/3383\n[#3492]: https://github.com/tokio-rs/tokio/pull/3492\n[#3504]: https://github.com/tokio-rs/tokio/pull/3504\n\n# 1.1.1 (January 29, 2021)\n\nForward ports 1.0.3 fix.\n\n### Fixed\n- io: memory leak during shutdown ([#3477]).\n\n# 1.1.0 (January 22, 2021)\n\n### Added\n\n- net: add `try_read_buf` and `try_recv_buf` ([#3351])\n- mpsc: Add `Sender::try_reserve` function ([#3418])\n- sync: add `RwLock` `try_read` and `try_write` methods ([#3400])\n- io: add `ReadBuf::inner_mut` ([#3443])\n\n### Changed\n\n- macros: improve `select!` error message ([#3352])\n- io: keep track of initialized bytes in `read_to_end` ([#3426])\n- runtime: consolidate errors for context missing ([#3441])\n\n### Fixed\n\n- task: wake `LocalSet` on `spawn_local` ([#3369])\n- sync: fix panic in broadcast::Receiver drop ([#3434])\n\n### Documented\n- stream: link to new `Stream` wrappers in `tokio-stream` ([#3343])\n- docs: mention that `test-util` feature is not enabled with full ([#3397])\n- process: add documentation to process::Child fields ([#3437])\n- io: clarify `AsyncFd` docs about changes of the inner fd ([#3430])\n- net: update datagram docs on splitting ([#3448])\n- time: document that `Sleep` is not `Unpin` ([#3457])\n- sync: add link to `PollSemaphore` ([#3456])\n- task: add `LocalSet` example ([#3438])\n- sync: improve bounded `mpsc` documentation ([#3458])\n\n[#3343]: https://github.com/tokio-rs/tokio/pull/3343\n[#3351]: https://github.com/tokio-rs/tokio/pull/3351\n[#3352]: https://github.com/tokio-rs/tokio/pull/3352\n[#3369]: https://github.com/tokio-rs/tokio/pull/3369\n[#3397]: https://github.com/tokio-rs/tokio/pull/3397\n[#3400]: https://github.com/tokio-rs/tokio/pull/3400\n[#3418]: https://github.com/tokio-rs/tokio/pull/3418\n[#3426]: https://github.com/tokio-rs/tokio/pull/3426\n[#3430]: https://github.com/tokio-rs/tokio/pull/3430\n[#3434]: https://github.com/tokio-rs/tokio/pull/3434\n[#3437]: https://github.com/tokio-rs/tokio/pull/3437\n[#3438]: https://github.com/tokio-rs/tokio/pull/3438\n[#3441]: https://github.com/tokio-rs/tokio/pull/3441\n[#3443]: https://github.com/tokio-rs/tokio/pull/3443\n[#3448]: https://github.com/tokio-rs/tokio/pull/3448\n[#3456]: https://github.com/tokio-rs/tokio/pull/3456\n[#3457]: https://github.com/tokio-rs/tokio/pull/3457\n[#3458]: https://github.com/tokio-rs/tokio/pull/3458\n\n# 1.0.3 (January 28, 2021)\n\n### Fixed\n- io: memory leak during shutdown ([#3477]).\n\n[#3477]: https://github.com/tokio-rs/tokio/pull/3477\n\n# 1.0.2 (January 14, 2021)\n\n### Fixed\n- io: soundness in `read_to_end` ([#3428]).\n\n[#3428]: https://github.com/tokio-rs/tokio/pull/3428\n\n# 1.0.1 (December 25, 2020)\n\nThis release fixes a soundness hole caused by the combination of `RwLockWriteGuard::map`\nand `RwLockWriteGuard::downgrade` by removing the `map` function. This is a breaking\nchange, but breaking changes are allowed under our semver policy when they are required\nto fix a soundness hole. (See [this RFC][semver] for more.)\n\nNote that we have chosen not to do a deprecation cycle or similar because Tokio 1.0.0 was\nreleased two days ago, and therefore the impact should be minimal.\n\nDue to the soundness hole, we have also yanked Tokio version 1.0.0.\n\n### Removed\n\n- sync: remove `RwLockWriteGuard::map` and `RwLockWriteGuard::try_map` ([#3345])\n\n### Fixed\n\n- docs: remove stream feature from docs ([#3335])\n\n[semver]: https://github.com/rust-lang/rfcs/blob/master/text/1122-language-semver.md#soundness-changes\n[#3335]: https://github.com/tokio-rs/tokio/pull/3335\n[#3345]: https://github.com/tokio-rs/tokio/pull/3345\n\n# 1.0.0 (December 23, 2020)\n\nCommit to the API and long-term support.\n\n### Fixed\n\n- sync: spurious wakeup in `watch` ([#3234]).\n\n### Changed\n\n- io: rename `AsyncFd::with_io()` to `try_io()` ([#3306])\n- fs: avoid OS specific `*Ext` traits in favor of conditionally defining the fn ([#3264]).\n- fs: `Sleep` is `!Unpin` ([#3278]).\n- net: pass `SocketAddr` by value ([#3125]).\n- net: `TcpStream::poll_peek` takes `ReadBuf` ([#3259]).\n- rt: rename `runtime::Builder::max_threads()` to `max_blocking_threads()` ([#3287]).\n- time: require `current_thread` runtime when calling `time::pause()` ([#3289]).\n\n### Removed\n\n- remove `tokio::prelude` ([#3299]).\n- io: remove `AsyncFd::with_poll()` ([#3306]).\n- net: remove `{Tcp,Unix}Stream::shutdown()` in favor of `AsyncWrite::shutdown()` ([#3298]).\n- stream: move all stream utilities to `tokio-stream` until `Stream` is added to\n `std` ([#3277]).\n- sync: mpsc `try_recv()` due to unexpected behavior ([#3263]).\n- tracing: make unstable as `tracing-core` is not 1.0 yet ([#3266]).\n\n### Added\n\n- fs: `poll_*` fns to `DirEntry` ([#3308]).\n- io: `poll_*` fns to `io::Lines`, `io::Split` ([#3308]).\n- io: `_mut` method variants to `AsyncFd` ([#3304]).\n- net: `poll_*` fns to `UnixDatagram` ([#3223]).\n- net: `UnixStream` readiness and non-blocking ops ([#3246]).\n- sync: `UnboundedReceiver::blocking_recv()` ([#3262]).\n- sync: `watch::Sender::borrow()` ([#3269]).\n- sync: `Semaphore::close()` ([#3065]).\n- sync: `poll_recv` fns to `mpsc::Receiver`, `mpsc::UnboundedReceiver` ([#3308]).\n- time: `poll_tick` fn to `time::Interval` ([#3316]).\n\n[#3065]: https://github.com/tokio-rs/tokio/pull/3065\n[#3125]: https://github.com/tokio-rs/tokio/pull/3125\n[#3223]: https://github.com/tokio-rs/tokio/pull/3223\n[#3234]: https://github.com/tokio-rs/tokio/pull/3234\n[#3246]: https://github.com/tokio-rs/tokio/pull/3246\n[#3259]: https://github.com/tokio-rs/tokio/pull/3259\n[#3262]: https://github.com/tokio-rs/tokio/pull/3262\n[#3263]: https://github.com/tokio-rs/tokio/pull/3263\n[#3264]: https://github.com/tokio-rs/tokio/pull/3264\n[#3266]: https://github.com/tokio-rs/tokio/pull/3266\n[#3269]: https://github.com/tokio-rs/tokio/pull/3269\n[#3277]: https://github.com/tokio-rs/tokio/pull/3277\n[#3278]: https://github.com/tokio-rs/tokio/pull/3278\n[#3287]: https://github.com/tokio-rs/tokio/pull/3287\n[#3289]: https://github.com/tokio-rs/tokio/pull/3289\n[#3298]: https://github.com/tokio-rs/tokio/pull/3298\n[#3299]: https://github.com/tokio-rs/tokio/pull/3299\n[#3304]: https://github.com/tokio-rs/tokio/pull/3304\n[#3306]: https://github.com/tokio-rs/tokio/pull/3306\n[#3308]: https://github.com/tokio-rs/tokio/pull/3308\n[#3316]: https://github.com/tokio-rs/tokio/pull/3316\n\n# 0.3.6 (December 14, 2020)\n\n### Fixed\n\n- rt: fix deadlock in shutdown ([#3228])\n- rt: fix panic in task abort when off rt ([#3159])\n- sync: make `add_permits` panic with usize::MAX >> 3 permits ([#3188])\n- time: Fix race condition in timer drop ([#3229])\n- watch: fix spurious wakeup ([#3244])\n\n### Added\n\n- example: add back udp-codec example ([#3205])\n- net: add `TcpStream::into_std` ([#3189])\n\n[#3159]: https://github.com/tokio-rs/tokio/pull/3159\n[#3188]: https://github.com/tokio-rs/tokio/pull/3188\n[#3189]: https://github.com/tokio-rs/tokio/pull/3189\n[#3205]: https://github.com/tokio-rs/tokio/pull/3205\n[#3228]: https://github.com/tokio-rs/tokio/pull/3228\n[#3229]: https://github.com/tokio-rs/tokio/pull/3229\n[#3244]: https://github.com/tokio-rs/tokio/pull/3244\n\n# 0.3.5 (November 30, 2020)\n\n### Fixed\n\n- rt: fix `shutdown_timeout(0)` ([#3196]).\n- time: fixed race condition with small sleeps ([#3069]).\n\n### Added\n\n- io: `AsyncFd::with_interest()` ([#3167]).\n- signal: `CtrlC` stream on windows ([#3186]).\n\n[#3069]: https://github.com/tokio-rs/tokio/pull/3069\n[#3167]: https://github.com/tokio-rs/tokio/pull/3167\n[#3186]: https://github.com/tokio-rs/tokio/pull/3186\n[#3196]: https://github.com/tokio-rs/tokio/pull/3196\n\n# 0.3.4 (November 18, 2020)\n\n### Fixed\n\n- stream: `StreamMap` `Default` impl bound ([#3093]).\n- io: `AsyncFd::into_inner()` should deregister the FD ([#3104]).\n\n### Changed\n\n- meta: `parking_lot` feature enabled with `full` ([#3119]).\n\n### Added\n\n- io: `AsyncWrite` vectored writes ([#3149]).\n- net: TCP/UDP readiness and non-blocking ops ([#3130], [#2743], [#3138]).\n- net: TCP socket option (linger, send/recv buf size) ([#3145], [#3143]).\n- net: PID field in `UCred` with solaris/illumos ([#3085]).\n- rt: `runtime::Handle` allows spawning onto a runtime ([#3079]).\n- sync: `Notify::notify_waiters()` ([#3098]).\n- sync: `acquire_many()`, `try_acquire_many()` to `Semaphore` ([#3067]).\n\n[#2743]: https://github.com/tokio-rs/tokio/pull/2743\n[#3067]: https://github.com/tokio-rs/tokio/pull/3067\n[#3079]: https://github.com/tokio-rs/tokio/pull/3079\n[#3085]: https://github.com/tokio-rs/tokio/pull/3085\n[#3093]: https://github.com/tokio-rs/tokio/pull/3093\n[#3098]: https://github.com/tokio-rs/tokio/pull/3098\n[#3104]: https://github.com/tokio-rs/tokio/pull/3104\n[#3119]: https://github.com/tokio-rs/tokio/pull/3119\n[#3130]: https://github.com/tokio-rs/tokio/pull/3130\n[#3138]: https://github.com/tokio-rs/tokio/pull/3138\n[#3143]: https://github.com/tokio-rs/tokio/pull/3143\n[#3145]: https://github.com/tokio-rs/tokio/pull/3145\n[#3149]: https://github.com/tokio-rs/tokio/pull/3149\n\n# 0.3.3 (November 2, 2020)\n\nFixes a soundness hole by adding a missing `Send` bound to\n`Runtime::spawn_blocking()`.\n\n### Fixed\n\n- rt: include missing `Send`, fixing soundness hole ([#3089]).\n- tracing: avoid huge trace span names ([#3074]).\n\n### Added\n\n- net: `TcpSocket::reuseport()`, `TcpSocket::set_reuseport()` ([#3083]).\n- net: `TcpSocket::reuseaddr()` ([#3093]).\n- net: `TcpSocket::local_addr()` ([#3093]).\n- net: add pid to `UCred` ([#2633]).\n\n[#2633]: https://github.com/tokio-rs/tokio/pull/2633\n[#3074]: https://github.com/tokio-rs/tokio/pull/3074\n[#3083]: https://github.com/tokio-rs/tokio/pull/3083\n[#3089]: https://github.com/tokio-rs/tokio/pull/3089\n[#3093]: https://github.com/tokio-rs/tokio/pull/3093\n\n# 0.3.2 (October 27, 2020)\n\nAdds `AsyncFd` as a replacement for v0.2's `PollEvented`.\n\n### Fixed\n\n- io: fix a potential deadlock when shutting down the I/O driver ([#2903]).\n- sync: `RwLockWriteGuard::downgrade()` bug ([#2957]).\n\n### Added\n\n- io: `AsyncFd` for receiving readiness events on raw FDs ([#2903]).\n- net: `poll_*` function on `UdpSocket` ([#2981]).\n- net: `UdpSocket::take_error()` ([#3051]).\n- sync: `oneshot::Sender::poll_closed()` ([#3032]).\n\n[#2903]: https://github.com/tokio-rs/tokio/pull/2903\n[#2957]: https://github.com/tokio-rs/tokio/pull/2957\n[#2981]: https://github.com/tokio-rs/tokio/pull/2981\n[#3032]: https://github.com/tokio-rs/tokio/pull/3032\n[#3051]: https://github.com/tokio-rs/tokio/pull/3051\n\n# 0.3.1 (October 21, 2020)\n\nThis release fixes an use-after-free in the IO driver. Additionally, the `read_buf`\nand `write_buf` methods have been added back to the IO traits, as the bytes crate\nis now on track to reach version 1.0 together with Tokio.\n\n### Fixed\n\n- net: fix use-after-free ([#3019]).\n- fs: ensure buffered data is written on shutdown ([#3009]).\n\n### Added\n\n- io: `copy_buf()` ([#2884]).\n- io: `AsyncReadExt::read_buf()`, `AsyncReadExt::write_buf()` for working with\n `Buf`/`BufMut` ([#3003]).\n- rt: `Runtime::spawn_blocking()` ([#2980]).\n- sync: `watch::Sender::is_closed()` ([#2991]).\n\n[#2884]: https://github.com/tokio-rs/tokio/pull/2884\n[#2980]: https://github.com/tokio-rs/tokio/pull/2980\n[#2991]: https://github.com/tokio-rs/tokio/pull/2991\n[#3003]: https://github.com/tokio-rs/tokio/pull/3003\n[#3009]: https://github.com/tokio-rs/tokio/pull/3009\n[#3019]: https://github.com/tokio-rs/tokio/pull/3019\n\n# 0.3.0 (October 15, 2020)\n\nThis represents a 1.0 beta release. APIs are polished and future-proofed. APIs\nnot included for 1.0 stabilization have been removed.\n\nBiggest changes are:\n\n- I/O driver internal rewrite. The windows implementation includes significant\n changes.\n- Runtime API is polished, especially with how it interacts with feature flag\n combinations.\n- Feature flags are simplified\n - `rt-core` and `rt-util` are combined to `rt`\n - `rt-threaded` is renamed to `rt-multi-thread` to match builder API\n - `tcp`, `udp`, `uds`, `dns` are combined to `net`.\n - `parking_lot` is included with `full`\n\n### Changes\n\n- meta: Minimum supported Rust version is now 1.45.\n- io: `AsyncRead` trait now takes `ReadBuf` in order to safely handle reading\n into uninitialized memory ([#2758]).\n- io: Internal I/O driver storage is now able to compact ([#2757]).\n- rt: `Runtime::block_on` now takes `&self` ([#2782]).\n- sync: `watch` reworked to decouple receiving a change notification from\n receiving the value ([#2814], [#2806]).\n- sync: `Notify::notify` is renamed to `notify_one` ([#2822]).\n- process: `Child::kill` is now an `async fn` that cleans zombies ([#2823]).\n- sync: use `const fn` constructors as possible ([#2833], [#2790])\n- signal: reduce cross-thread notification ([#2835]).\n- net: tcp,udp,uds types support operations with `&self` ([#2828], [#2919], [#2934]).\n- sync: blocking `mpsc` channel supports `send` with `&self` ([#2861]).\n- time: rename `delay_for` and `delay_until` to `sleep` and `sleep_until` ([#2826]).\n- io: upgrade to `mio` 0.7 ([#2893]).\n- io: `AsyncSeek` trait is tweaked ([#2885]).\n- fs: `File` operations take `&self` ([#2930]).\n- rt: runtime API, and `#[tokio::main]` macro polish ([#2876])\n- rt: `Runtime::enter` uses an RAII guard instead of a closure ([#2954]).\n- net: the `from_std` function on all sockets no longer sets socket into non-blocking mode ([#2893])\n\n### Added\n\n- sync: `map` function to lock guards ([#2445]).\n- sync: `blocking_recv` and `blocking_send` fns to `mpsc` for use outside of Tokio ([#2685]).\n- rt: `Builder::thread_name_fn` for configuring thread names ([#1921]).\n- fs: impl `FromRawFd` and `FromRawHandle` for `File` ([#2792]).\n- process: `Child::wait` and `Child::try_wait` ([#2796]).\n- rt: support configuring thread keep-alive duration ([#2809]).\n- rt: `task::JoinHandle::abort` forcibly cancels a spawned task ([#2474]).\n- sync: `RwLock` write guard to read guard downgrading ([#2733]).\n- net: add `poll_*` functions that take `&self` to all net types ([#2845])\n- sync: `get_mut()` for `Mutex`, `RwLock` ([#2856]).\n- sync: `mpsc::Sender::closed()` waits for `Receiver` half to close ([#2840]).\n- sync: `mpsc::Sender::is_closed()` returns true if `Receiver` half is closed ([#2726]).\n- stream: `iter` and `iter_mut` to `StreamMap` ([#2890]).\n- net: implement `AsRawSocket` on windows ([#2911]).\n- net: `TcpSocket` creates a socket without binding or listening ([#2920]).\n\n### Removed\n\n- io: vectored ops are removed from `AsyncRead`, `AsyncWrite` traits ([#2882]).\n- io: `mio` is removed from the public API. `PollEvented` and` Registration` are\n removed ([#2893]).\n- io: remove `bytes` from public API. `Buf` and `BufMut` implementation are\n removed ([#2908]).\n- time: `DelayQueue` is moved to `tokio-util` ([#2897]).\n\n### Fixed\n\n- io: `stdout` and `stderr` buffering on windows ([#2734]).\n\n[#1921]: https://github.com/tokio-rs/tokio/pull/1921\n[#2445]: https://github.com/tokio-rs/tokio/pull/2445\n[#2474]: https://github.com/tokio-rs/tokio/pull/2474\n[#2685]: https://github.com/tokio-rs/tokio/pull/2685\n[#2726]: https://github.com/tokio-rs/tokio/pull/2726\n[#2733]: https://github.com/tokio-rs/tokio/pull/2733\n[#2734]: https://github.com/tokio-rs/tokio/pull/2734\n[#2757]: https://github.com/tokio-rs/tokio/pull/2757\n[#2758]: https://github.com/tokio-rs/tokio/pull/2758\n[#2782]: https://github.com/tokio-rs/tokio/pull/2782\n[#2790]: https://github.com/tokio-rs/tokio/pull/2790\n[#2792]: https://github.com/tokio-rs/tokio/pull/2792\n[#2796]: https://github.com/tokio-rs/tokio/pull/2796\n[#2806]: https://github.com/tokio-rs/tokio/pull/2806\n[#2809]: https://github.com/tokio-rs/tokio/pull/2809\n[#2814]: https://github.com/tokio-rs/tokio/pull/2814\n[#2822]: https://github.com/tokio-rs/tokio/pull/2822\n[#2823]: https://github.com/tokio-rs/tokio/pull/2823\n[#2826]: https://github.com/tokio-rs/tokio/pull/2826\n[#2828]: https://github.com/tokio-rs/tokio/pull/2828\n[#2833]: https://github.com/tokio-rs/tokio/pull/2833\n[#2835]: https://github.com/tokio-rs/tokio/pull/2835\n[#2840]: https://github.com/tokio-rs/tokio/pull/2840\n[#2845]: https://github.com/tokio-rs/tokio/pull/2845\n[#2856]: https://github.com/tokio-rs/tokio/pull/2856\n[#2861]: https://github.com/tokio-rs/tokio/pull/2861\n[#2876]: https://github.com/tokio-rs/tokio/pull/2876\n[#2882]: https://github.com/tokio-rs/tokio/pull/2882\n[#2885]: https://github.com/tokio-rs/tokio/pull/2885\n[#2890]: https://github.com/tokio-rs/tokio/pull/2890\n[#2893]: https://github.com/tokio-rs/tokio/pull/2893\n[#2897]: https://github.com/tokio-rs/tokio/pull/2897\n[#2908]: https://github.com/tokio-rs/tokio/pull/2908\n[#2911]: https://github.com/tokio-rs/tokio/pull/2911\n[#2919]: https://github.com/tokio-rs/tokio/pull/2919\n[#2920]: https://github.com/tokio-rs/tokio/pull/2920\n[#2930]: https://github.com/tokio-rs/tokio/pull/2930\n[#2934]: https://github.com/tokio-rs/tokio/pull/2934\n[#2954]: https://github.com/tokio-rs/tokio/pull/2954\n\n# 0.2.22 (July 21, 2020)\n\n### Fixes\n\n- docs: misc improvements ([#2572], [#2658], [#2663], [#2656], [#2647], [#2630], [#2487], [#2621],\n [#2624], [#2600], [#2623], [#2622], [#2577], [#2569], [#2589], [#2575], [#2540], [#2564], [#2567],\n [#2520], [#2521], [#2493])\n- rt: allow calls to `block_on` inside calls to `block_in_place` that are\n themselves inside `block_on` ([#2645])\n- net: fix non-portable behavior when dropping `TcpStream` `OwnedWriteHalf` ([#2597])\n- io: improve stack usage by allocating large buffers on directly on the heap\n ([#2634])\n- io: fix unsound pin projection in `AsyncReadExt::read_buf` and\n `AsyncWriteExt::write_buf` ([#2612])\n- io: fix unnecessary zeroing for `AsyncRead` implementors ([#2525])\n- io: Fix `BufReader` not correctly forwarding `poll_write_buf` ([#2654])\n- io: fix panic in `AsyncReadExt::read_line` ([#2541])\n\n### Changes\n\n- coop: returning `Poll::Pending` no longer decrements the task budget ([#2549])\n\n### Added\n\n- io: little-endian variants of `AsyncReadExt` and `AsyncWriteExt` methods\n ([#1915])\n- task: add [`tracing`] instrumentation to spawned tasks ([#2655])\n- sync: allow unsized types in `Mutex` and `RwLock` (via `default` constructors)\n ([#2615])\n- net: add `ToSocketAddrs` implementation for `&[SocketAddr]` ([#2604])\n- fs: add `OpenOptionsExt` for `OpenOptions` ([#2515])\n- fs: add `DirBuilder` ([#2524])\n\n[`tracing`]: https://crates.io/crates/tracing\n[#1915]: https://github.com/tokio-rs/tokio/pull/1915\n[#2487]: https://github.com/tokio-rs/tokio/pull/2487\n[#2493]: https://github.com/tokio-rs/tokio/pull/2493\n[#2515]: https://github.com/tokio-rs/tokio/pull/2515\n[#2520]: https://github.com/tokio-rs/tokio/pull/2520\n[#2521]: https://github.com/tokio-rs/tokio/pull/2521\n[#2524]: https://github.com/tokio-rs/tokio/pull/2524\n[#2525]: https://github.com/tokio-rs/tokio/pull/2525\n[#2540]: https://github.com/tokio-rs/tokio/pull/2540\n[#2541]: https://github.com/tokio-rs/tokio/pull/2541\n[#2549]: https://github.com/tokio-rs/tokio/pull/2549\n[#2564]: https://github.com/tokio-rs/tokio/pull/2564\n[#2567]: https://github.com/tokio-rs/tokio/pull/2567\n[#2569]: https://github.com/tokio-rs/tokio/pull/2569\n[#2572]: https://github.com/tokio-rs/tokio/pull/2572\n[#2575]: https://github.com/tokio-rs/tokio/pull/2575\n[#2577]: https://github.com/tokio-rs/tokio/pull/2577\n[#2589]: https://github.com/tokio-rs/tokio/pull/2589\n[#2597]: https://github.com/tokio-rs/tokio/pull/2597\n[#2600]: https://github.com/tokio-rs/tokio/pull/2600\n[#2604]: https://github.com/tokio-rs/tokio/pull/2604\n[#2612]: https://github.com/tokio-rs/tokio/pull/2612\n[#2615]: https://github.com/tokio-rs/tokio/pull/2615\n[#2621]: https://github.com/tokio-rs/tokio/pull/2621\n[#2622]: https://github.com/tokio-rs/tokio/pull/2622\n[#2623]: https://github.com/tokio-rs/tokio/pull/2623\n[#2624]: https://github.com/tokio-rs/tokio/pull/2624\n[#2630]: https://github.com/tokio-rs/tokio/pull/2630\n[#2634]: https://github.com/tokio-rs/tokio/pull/2634\n[#2645]: https://github.com/tokio-rs/tokio/pull/2645\n[#2647]: https://github.com/tokio-rs/tokio/pull/2647\n[#2654]: https://github.com/tokio-rs/tokio/pull/2654\n[#2655]: https://github.com/tokio-rs/tokio/pull/2655\n[#2656]: https://github.com/tokio-rs/tokio/pull/2656\n[#2658]: https://github.com/tokio-rs/tokio/pull/2658\n[#2663]: https://github.com/tokio-rs/tokio/pull/2663\n\n# 0.2.21 (May 13, 2020)\n\n### Fixes\n\n- macros: disambiguate built-in `#[test]` attribute in macro expansion ([#2503])\n- rt: `LocalSet` and task budgeting ([#2462]).\n- rt: task budgeting with `block_in_place` ([#2502]).\n- sync: release `broadcast` channel memory without sending a value ([#2509]).\n- time: notify when resetting a `Delay` to a time in the past ([#2290])\n\n### Added\n\n- io: `get_mut`, `get_ref`, and `into_inner` to `Lines` ([#2450]).\n- io: `mio::Ready` argument to `PollEvented` ([#2419]).\n- os: illumos support ([#2486]).\n- rt: `Handle::spawn_blocking` ([#2501]).\n- sync: `OwnedMutexGuard` for `Arc>` ([#2455]).\n\n[#2290]: https://github.com/tokio-rs/tokio/pull/2290\n[#2419]: https://github.com/tokio-rs/tokio/pull/2419\n[#2450]: https://github.com/tokio-rs/tokio/pull/2450\n[#2455]: https://github.com/tokio-rs/tokio/pull/2455\n[#2462]: https://github.com/tokio-rs/tokio/pull/2462\n[#2486]: https://github.com/tokio-rs/tokio/pull/2486\n[#2501]: https://github.com/tokio-rs/tokio/pull/2501\n[#2502]: https://github.com/tokio-rs/tokio/pull/2502\n[#2503]: https://github.com/tokio-rs/tokio/pull/2503\n[#2509]: https://github.com/tokio-rs/tokio/pull/2509\n\n# 0.2.20 (April 28, 2020)\n\n### Fixes\n\n- sync: `broadcast` closing the channel no longer requires capacity ([#2448]).\n- rt: regression when configuring runtime with `max_threads` less than number of CPUs ([#2457]).\n\n[#2448]: https://github.com/tokio-rs/tokio/pull/2448\n[#2457]: https://github.com/tokio-rs/tokio/pull/2457\n\n# 0.2.19 (April 24, 2020)\n\n### Fixes\n\n- docs: misc improvements ([#2400], [#2405], [#2414], [#2420], [#2423], [#2426], [#2427], [#2434], [#2436], [#2440]).\n- rt: support `block_in_place` in more contexts ([#2409], [#2410]).\n- stream: no panic in `merge()` and `chain()` when using `size_hint()` ([#2430]).\n- task: include visibility modifier when defining a task-local ([#2416]).\n\n### Added\n\n- rt: `runtime::Handle::block_on` ([#2437]).\n- sync: owned `Semaphore` permit ([#2421]).\n- tcp: owned split ([#2270]).\n\n[#2270]: https://github.com/tokio-rs/tokio/pull/2270\n[#2400]: https://github.com/tokio-rs/tokio/pull/2400\n[#2405]: https://github.com/tokio-rs/tokio/pull/2405\n[#2409]: https://github.com/tokio-rs/tokio/pull/2409\n[#2410]: https://github.com/tokio-rs/tokio/pull/2410\n[#2414]: https://github.com/tokio-rs/tokio/pull/2414\n[#2416]: https://github.com/tokio-rs/tokio/pull/2416\n[#2420]: https://github.com/tokio-rs/tokio/pull/2420\n[#2421]: https://github.com/tokio-rs/tokio/pull/2421\n[#2423]: https://github.com/tokio-rs/tokio/pull/2423\n[#2426]: https://github.com/tokio-rs/tokio/pull/2426\n[#2427]: https://github.com/tokio-rs/tokio/pull/2427\n[#2430]: https://github.com/tokio-rs/tokio/pull/2430\n[#2434]: https://github.com/tokio-rs/tokio/pull/2434\n[#2436]: https://github.com/tokio-rs/tokio/pull/2436\n[#2437]: https://github.com/tokio-rs/tokio/pull/2437\n[#2440]: https://github.com/tokio-rs/tokio/pull/2440\n\n# 0.2.18 (April 12, 2020)\n\n### Fixes\n\n- task: `LocalSet` was incorrectly marked as `Send` ([#2398])\n- io: correctly report `WriteZero` failure in `write_int` ([#2334])\n\n[#2334]: https://github.com/tokio-rs/tokio/pull/2334\n[#2398]: https://github.com/tokio-rs/tokio/pull/2398\n\n# 0.2.17 (April 9, 2020)\n\n### Fixes\n\n- rt: bug in work-stealing queue ([#2387])\n\n### Changes\n\n- rt: threadpool uses logical CPU count instead of physical by default ([#2391])\n\n[#2387]: https://github.com/tokio-rs/tokio/pull/2387\n[#2391]: https://github.com/tokio-rs/tokio/pull/2391\n\n# 0.2.16 (April 3, 2020)\n\n### Fixes\n\n- sync: fix a regression where `Mutex`, `Semaphore`, and `RwLock` futures no\n longer implement `Sync` ([#2375])\n- fs: fix `fs::copy` not copying file permissions ([#2354])\n\n### Added\n\n- time: added `deadline` method to `delay_queue::Expired` ([#2300])\n- io: added `StreamReader` ([#2052])\n\n[#2052]: https://github.com/tokio-rs/tokio/pull/2052\n[#2300]: https://github.com/tokio-rs/tokio/pull/2300\n[#2354]: https://github.com/tokio-rs/tokio/pull/2354\n[#2375]: https://github.com/tokio-rs/tokio/pull/2375\n\n# 0.2.15 (April 2, 2020)\n\n### Fixes\n\n- rt: fix queue regression ([#2362]).\n\n### Added\n\n- sync: Add disarm to `mpsc::Sender` ([#2358]).\n\n[#2358]: https://github.com/tokio-rs/tokio/pull/2358\n[#2362]: https://github.com/tokio-rs/tokio/pull/2362\n\n# 0.2.14 (April 1, 2020)\n\n### Fixes\n\n- rt: concurrency bug in scheduler ([#2273]).\n- rt: concurrency bug with shell runtime ([#2333]).\n- test-util: correct pause/resume of time ([#2253]).\n- time: `DelayQueue` correct wakeup after `insert` ([#2285]).\n\n### Added\n\n- io: impl `RawFd`, `AsRawHandle` for std io types ([#2335]).\n- rt: automatic cooperative task yielding ([#2160], [#2343], [#2349]).\n- sync: `RwLock::into_inner` ([#2321]).\n\n### Changed\n\n- sync: semaphore, mutex internals rewritten to avoid allocations ([#2325]).\n\n[#2160]: https://github.com/tokio-rs/tokio/pull/2160\n[#2253]: https://github.com/tokio-rs/tokio/pull/2253\n[#2273]: https://github.com/tokio-rs/tokio/pull/2273\n[#2285]: https://github.com/tokio-rs/tokio/pull/2285\n[#2321]: https://github.com/tokio-rs/tokio/pull/2321\n[#2325]: https://github.com/tokio-rs/tokio/pull/2325\n[#2333]: https://github.com/tokio-rs/tokio/pull/2333\n[#2335]: https://github.com/tokio-rs/tokio/pull/2335\n[#2343]: https://github.com/tokio-rs/tokio/pull/2343\n[#2349]: https://github.com/tokio-rs/tokio/pull/2349\n\n# 0.2.13 (February 28, 2020)\n\n### Fixes\n\n- macros: unresolved import in `pin!` ([#2281]).\n\n[#2281]: https://github.com/tokio-rs/tokio/pull/2281\n\n# 0.2.12 (February 27, 2020)\n\n### Fixes\n\n- net: `UnixStream::poll_shutdown` should call `shutdown(Write)` ([#2245]).\n- process: Wake up read and write on `EPOLLERR` ([#2218]).\n- rt: potential deadlock when using `block_in_place` and shutting down the\n runtime ([#2119]).\n- rt: only detect number of CPUs if `core_threads` not specified ([#2238]).\n- sync: reduce `watch::Receiver` struct size ([#2191]).\n- time: succeed when setting delay of `$MAX-1` ([#2184]).\n- time: avoid having to poll `DelayQueue` after inserting new delay ([#2217]).\n\n### Added\n\n- macros: `pin!` variant that assigns to identifier and pins ([#2274]).\n- net: impl `Stream` for `Listener` types ([#2275]).\n- rt: `Runtime::shutdown_timeout` waits for runtime to shutdown for specified\n duration ([#2186]).\n- stream: `StreamMap` merges streams and can insert / remove streams at\n runtime ([#2185]).\n- stream: `StreamExt::skip()` skips a fixed number of items ([#2204]).\n- stream: `StreamExt::skip_while()` skips items based on a predicate ([#2205]).\n- sync: `Notify` provides basic `async` / `await` task notification ([#2210]).\n- sync: `Mutex::into_inner` retrieves guarded data ([#2250]).\n- sync: `mpsc::Sender::send_timeout` sends, waiting for up to specified duration\n for channel capacity ([#2227]).\n- time: impl `Ord` and `Hash` for `Instant` ([#2239]).\n\n[#2119]: https://github.com/tokio-rs/tokio/pull/2119\n[#2184]: https://github.com/tokio-rs/tokio/pull/2184\n[#2185]: https://github.com/tokio-rs/tokio/pull/2185\n[#2186]: https://github.com/tokio-rs/tokio/pull/2186\n[#2191]: https://github.com/tokio-rs/tokio/pull/2191\n[#2204]: https://github.com/tokio-rs/tokio/pull/2204\n[#2205]: https://github.com/tokio-rs/tokio/pull/2205\n[#2210]: https://github.com/tokio-rs/tokio/pull/2210\n[#2217]: https://github.com/tokio-rs/tokio/pull/2217\n[#2218]: https://github.com/tokio-rs/tokio/pull/2218\n[#2227]: https://github.com/tokio-rs/tokio/pull/2227\n[#2238]: https://github.com/tokio-rs/tokio/pull/2238\n[#2239]: https://github.com/tokio-rs/tokio/pull/2239\n[#2245]: https://github.com/tokio-rs/tokio/pull/2245\n[#2250]: https://github.com/tokio-rs/tokio/pull/2250\n[#2274]: https://github.com/tokio-rs/tokio/pull/2274\n[#2275]: https://github.com/tokio-rs/tokio/pull/2275\n\n# 0.2.11 (January 27, 2020)\n\n### Fixes\n\n- docs: misc fixes and tweaks ([#2155], [#2103], [#2027], [#2167], [#2175]).\n- macros: handle generics in `#[tokio::main]` method ([#2177]).\n- sync: `broadcast` potential lost notifications ([#2135]).\n- rt: improve \"no runtime\" panic messages ([#2145]).\n\n### Added\n\n- optional support for using `parking_lot` internally ([#2164]).\n- fs: `fs::copy`, an async version of `std::fs::copy` ([#2079]).\n- macros: `select!` waits for the first branch to complete ([#2152]).\n- macros: `join!` waits for all branches to complete ([#2158]).\n- macros: `try_join!` waits for all branches to complete or the first error ([#2169]).\n- macros: `pin!` pins a value to the stack ([#2163]).\n- net: `ReadHalf::poll()` and `ReadHalf::poll_peak` ([#2151])\n- stream: `StreamExt::timeout()` sets a per-item max duration ([#2149]).\n- stream: `StreamExt::fold()` applies a function, producing a single value. ([#2122]).\n- sync: impl `Eq`, `PartialEq` for `oneshot::RecvError` ([#2168]).\n- task: methods for inspecting the `JoinError` cause ([#2051]).\n\n[#2027]: https://github.com/tokio-rs/tokio/pull/2027\n[#2051]: https://github.com/tokio-rs/tokio/pull/2051\n[#2079]: https://github.com/tokio-rs/tokio/pull/2079\n[#2103]: https://github.com/tokio-rs/tokio/pull/2103\n[#2122]: https://github.com/tokio-rs/tokio/pull/2122\n[#2135]: https://github.com/tokio-rs/tokio/pull/2135\n[#2145]: https://github.com/tokio-rs/tokio/pull/2145\n[#2149]: https://github.com/tokio-rs/tokio/pull/2149\n[#2151]: https://github.com/tokio-rs/tokio/pull/2151\n[#2152]: https://github.com/tokio-rs/tokio/pull/2152\n[#2155]: https://github.com/tokio-rs/tokio/pull/2155\n[#2158]: https://github.com/tokio-rs/tokio/pull/2158\n[#2163]: https://github.com/tokio-rs/tokio/pull/2163\n[#2164]: https://github.com/tokio-rs/tokio/pull/2164\n[#2167]: https://github.com/tokio-rs/tokio/pull/2167\n[#2168]: https://github.com/tokio-rs/tokio/pull/2168\n[#2169]: https://github.com/tokio-rs/tokio/pull/2169\n[#2175]: https://github.com/tokio-rs/tokio/pull/2175\n[#2177]: https://github.com/tokio-rs/tokio/pull/2177\n\n# 0.2.10 (January 21, 2020)\n\n### Fixes\n\n- `#[tokio::main]` when `rt-core` feature flag is not enabled ([#2139]).\n- remove `AsyncBufRead` from `BufStream` impl block ([#2108]).\n- potential undefined behavior when implementing `AsyncRead` incorrectly ([#2030]).\n\n### Added\n\n- `BufStream::with_capacity` ([#2125]).\n- impl `From` and `Default` for `RwLock` ([#2089]).\n- `io::ReadHalf::is_pair_of` checks if provided `WriteHalf` is for the same\n underlying object ([#1762], [#2144]).\n- `runtime::Handle::try_current()` returns a handle to the current runtime ([#2118]).\n- `stream::empty()` returns an immediately ready empty stream ([#2092]).\n- `stream::once(val)` returns a stream that yields a single value: `val` ([#2094]).\n- `stream::pending()` returns a stream that never becomes ready ([#2092]).\n- `StreamExt::chain()` sequences a second stream after the first completes ([#2093]).\n- `StreamExt::collect()` transform a stream into a collection ([#2109]).\n- `StreamExt::fuse` ends the stream after the first `None` ([#2085]).\n- `StreamExt::merge` combines two streams, yielding values as they become ready ([#2091]).\n- Task-local storage ([#2126]).\n\n[#1762]: https://github.com/tokio-rs/tokio/pull/1762\n[#2030]: https://github.com/tokio-rs/tokio/pull/2030\n[#2085]: https://github.com/tokio-rs/tokio/pull/2085\n[#2089]: https://github.com/tokio-rs/tokio/pull/2089\n[#2091]: https://github.com/tokio-rs/tokio/pull/2091\n[#2092]: https://github.com/tokio-rs/tokio/pull/2092\n[#2093]: https://github.com/tokio-rs/tokio/pull/2093\n[#2094]: https://github.com/tokio-rs/tokio/pull/2094\n[#2108]: https://github.com/tokio-rs/tokio/pull/2108\n[#2109]: https://github.com/tokio-rs/tokio/pull/2109\n[#2118]: https://github.com/tokio-rs/tokio/pull/2118\n[#2125]: https://github.com/tokio-rs/tokio/pull/2125\n[#2126]: https://github.com/tokio-rs/tokio/pull/2126\n[#2139]: https://github.com/tokio-rs/tokio/pull/2139\n[#2144]: https://github.com/tokio-rs/tokio/pull/2144\n\n# 0.2.9 (January 9, 2020)\n\n### Fixes\n\n- `AsyncSeek` impl for `File` ([#1986]).\n- rt: shutdown deadlock in `threaded_scheduler` ([#2074], [#2082]).\n- rt: memory ordering when dropping `JoinHandle` ([#2044]).\n- docs: misc API documentation fixes and improvements.\n\n[#1986]: https://github.com/tokio-rs/tokio/pull/1986\n[#2044]: https://github.com/tokio-rs/tokio/pull/2044\n[#2074]: https://github.com/tokio-rs/tokio/pull/2074\n[#2082]: https://github.com/tokio-rs/tokio/pull/2082\n\n# 0.2.8 (January 7, 2020)\n\n### Fixes\n\n- depend on new version of `tokio-macros`.\n\n# 0.2.7 (January 7, 2020)\n\n### Fixes\n\n- potential deadlock when dropping `basic_scheduler` Runtime.\n- calling `spawn_blocking` from within a `spawn_blocking` ([#2006]).\n- storing a `Runtime` instance in a thread-local ([#2011]).\n- miscellaneous documentation fixes.\n- rt: fix `Waker::will_wake` to return true when tasks match ([#2045]).\n- test-util: `time::advance` runs pending tasks before changing the time ([#2059]).\n\n### Added\n\n- `net::lookup_host` maps a `T: ToSocketAddrs` to a stream of `SocketAddrs` ([#1870]).\n- `process::Child` fields are made public to match `std` ([#2014]).\n- impl `Stream` for `sync::broadcast::Receiver` ([#2012]).\n- `sync::RwLock` provides an asynchronous read-write lock ([#1699]).\n- `runtime::Handle::current` returns the handle for the current runtime ([#2040]).\n- `StreamExt::filter` filters stream values according to a predicate ([#2001]).\n- `StreamExt::filter_map` simultaneously filter and map stream values ([#2001]).\n- `StreamExt::try_next` convenience for streams of `Result` ([#2005]).\n- `StreamExt::take` limits a stream to a specified number of values ([#2025]).\n- `StreamExt::take_while` limits a stream based on a predicate ([#2029]).\n- `StreamExt::all` tests if every element of the stream matches a predicate ([#2035]).\n- `StreamExt::any` tests if any element of the stream matches a predicate ([#2034]).\n- `task::LocalSet.await` runs spawned tasks until the set is idle ([#1971]).\n- `time::DelayQueue::len` returns the number entries in the queue ([#1755]).\n- expose runtime options from the `#[tokio::main]` and `#[tokio::test]` ([#2022]).\n\n[#1699]: https://github.com/tokio-rs/tokio/pull/1699\n[#1755]: https://github.com/tokio-rs/tokio/pull/1755\n[#1870]: https://github.com/tokio-rs/tokio/pull/1870\n[#1971]: https://github.com/tokio-rs/tokio/pull/1971\n[#2001]: https://github.com/tokio-rs/tokio/pull/2001\n[#2005]: https://github.com/tokio-rs/tokio/pull/2005\n[#2006]: https://github.com/tokio-rs/tokio/pull/2006\n[#2011]: https://github.com/tokio-rs/tokio/pull/2011\n[#2012]: https://github.com/tokio-rs/tokio/pull/2012\n[#2014]: https://github.com/tokio-rs/tokio/pull/2014\n[#2022]: https://github.com/tokio-rs/tokio/pull/2022\n[#2025]: https://github.com/tokio-rs/tokio/pull/2025\n[#2029]: https://github.com/tokio-rs/tokio/pull/2029\n[#2034]: https://github.com/tokio-rs/tokio/pull/2034\n[#2035]: https://github.com/tokio-rs/tokio/pull/2035\n[#2040]: https://github.com/tokio-rs/tokio/pull/2040\n[#2045]: https://github.com/tokio-rs/tokio/pull/2045\n[#2059]: https://github.com/tokio-rs/tokio/pull/2059\n\n# 0.2.6 (December 19, 2019)\n\n### Fixes\n\n- `fs::File::seek` API regression ([#1991]).\n\n[#1991]: https://github.com/tokio-rs/tokio/pull/1991\n\n# 0.2.5 (December 18, 2019)\n\n### Added\n\n- `io::AsyncSeek` trait ([#1924]).\n- `Mutex::try_lock` ([#1939])\n- `mpsc::Receiver::try_recv` and `mpsc::UnboundedReceiver::try_recv` ([#1939]).\n- `writev` support for `TcpStream` ([#1956]).\n- `time::throttle` for throttling streams ([#1949]).\n- implement `Stream` for `time::DelayQueue` ([#1975]).\n- `sync::broadcast` provides a fan-out channel ([#1943]).\n- `sync::Semaphore` provides an async semaphore ([#1973]).\n- `stream::StreamExt` provides stream utilities ([#1962]).\n\n### Fixes\n\n- deadlock risk while shutting down the runtime ([#1972]).\n- panic while shutting down the runtime ([#1978]).\n- `sync::MutexGuard` debug output ([#1961]).\n- misc doc improvements ([#1933], [#1934], [#1940], [#1942]).\n\n### Changes\n\n- runtime threads are configured with `runtime::Builder::core_threads` and\n `runtime::Builder::max_threads`. `runtime::Builder::num_threads` is\n deprecated ([#1977]).\n\n[#1924]: https://github.com/tokio-rs/tokio/pull/1924\n[#1933]: https://github.com/tokio-rs/tokio/pull/1933\n[#1934]: https://github.com/tokio-rs/tokio/pull/1934\n[#1939]: https://github.com/tokio-rs/tokio/pull/1939\n[#1940]: https://github.com/tokio-rs/tokio/pull/1940\n[#1942]: https://github.com/tokio-rs/tokio/pull/1942\n[#1943]: https://github.com/tokio-rs/tokio/pull/1943\n[#1949]: https://github.com/tokio-rs/tokio/pull/1949\n[#1956]: https://github.com/tokio-rs/tokio/pull/1956\n[#1961]: https://github.com/tokio-rs/tokio/pull/1961\n[#1962]: https://github.com/tokio-rs/tokio/pull/1962\n[#1972]: https://github.com/tokio-rs/tokio/pull/1972\n[#1973]: https://github.com/tokio-rs/tokio/pull/1973\n[#1975]: https://github.com/tokio-rs/tokio/pull/1975\n[#1977]: https://github.com/tokio-rs/tokio/pull/1977\n[#1978]: https://github.com/tokio-rs/tokio/pull/1978\n\n# 0.2.4 (December 6, 2019)\n\n### Fixes\n\n- `sync::Mutex` deadlock when `lock()` future is dropped early ([#1898]).\n\n[#1898]: https://github.com/tokio-rs/tokio/pull/1898\n\n# 0.2.3 (December 6, 2019)\n\n### Added\n\n- read / write integers using `AsyncReadExt` and `AsyncWriteExt` ([#1863]).\n- `read_buf` / `write_buf` for reading / writing `Buf` / `BufMut` ([#1881]).\n- `TcpStream::poll_peek` - pollable API for performing TCP peek ([#1864]).\n- `sync::oneshot::error::TryRecvError` provides variants to detect the error\n kind ([#1874]).\n- `LocalSet::block_on` accepts `!'static` task ([#1882]).\n- `task::JoinError` is now `Sync` ([#1888]).\n- impl conversions between `tokio::time::Instant` and\n `std::time::Instant` ([#1904]).\n\n### Fixes\n\n- calling `spawn_blocking` after runtime shutdown ([#1875]).\n- `LocalSet` drop infinite loop ([#1892]).\n- `LocalSet` hang under load ([#1905]).\n- improved documentation ([#1865], [#1866], [#1868], [#1874], [#1876], [#1911]).\n\n[#1863]: https://github.com/tokio-rs/tokio/pull/1863\n[#1864]: https://github.com/tokio-rs/tokio/pull/1864\n[#1865]: https://github.com/tokio-rs/tokio/pull/1865\n[#1866]: https://github.com/tokio-rs/tokio/pull/1866\n[#1868]: https://github.com/tokio-rs/tokio/pull/1868\n[#1874]: https://github.com/tokio-rs/tokio/pull/1874\n[#1875]: https://github.com/tokio-rs/tokio/pull/1875\n[#1876]: https://github.com/tokio-rs/tokio/pull/1876\n[#1881]: https://github.com/tokio-rs/tokio/pull/1881\n[#1882]: https://github.com/tokio-rs/tokio/pull/1882\n[#1888]: https://github.com/tokio-rs/tokio/pull/1888\n[#1892]: https://github.com/tokio-rs/tokio/pull/1892\n[#1904]: https://github.com/tokio-rs/tokio/pull/1904\n[#1905]: https://github.com/tokio-rs/tokio/pull/1905\n[#1911]: https://github.com/tokio-rs/tokio/pull/1911\n\n# 0.2.2 (November 29, 2019)\n\n### Fixes\n\n- scheduling with `basic_scheduler` ([#1861]).\n- update `spawn` panic message to specify that a task scheduler is required ([#1839]).\n- API docs example for `runtime::Builder` to include a task scheduler ([#1841]).\n- general documentation ([#1834]).\n- building on illumos/solaris ([#1772]).\n- panic when dropping `LocalSet` ([#1843]).\n- API docs mention the required Cargo features for `Builder::{basic, threaded}_scheduler` ([#1858]).\n\n### Added\n\n- impl `Stream` for `signal::unix::Signal` ([#1849]).\n- API docs for platform specific behavior of `signal::ctrl_c` and `signal::unix::Signal` ([#1854]).\n- API docs for `signal::unix::Signal::{recv, poll_recv}` and `signal::windows::CtrlBreak::{recv, poll_recv}` ([#1854]).\n- `File::into_std` and `File::try_into_std` methods ([#1856]).\n\n[#1772]: https://github.com/tokio-rs/tokio/pull/1772\n[#1834]: https://github.com/tokio-rs/tokio/pull/1834\n[#1839]: https://github.com/tokio-rs/tokio/pull/1839\n[#1841]: https://github.com/tokio-rs/tokio/pull/1841\n[#1843]: https://github.com/tokio-rs/tokio/pull/1843\n[#1849]: https://github.com/tokio-rs/tokio/pull/1849\n[#1854]: https://github.com/tokio-rs/tokio/pull/1854\n[#1856]: https://github.com/tokio-rs/tokio/pull/1856\n[#1858]: https://github.com/tokio-rs/tokio/pull/1858\n[#1861]: https://github.com/tokio-rs/tokio/pull/1861\n\n# 0.2.1 (November 26, 2019)\n\n### Fixes\n\n- API docs for `TcpListener::incoming`, `UnixListener::incoming` ([#1831]).\n\n### Added\n\n- `tokio::task::LocalSet` provides a strategy for spawning `!Send` tasks ([#1733]).\n- export `tokio::time::Elapsed` ([#1826]).\n- impl `AsRawFd`, `AsRawHandle` for `tokio::fs::File` ([#1827]).\n\n[#1733]: https://github.com/tokio-rs/tokio/pull/1733\n[#1826]: https://github.com/tokio-rs/tokio/pull/1826\n[#1827]: https://github.com/tokio-rs/tokio/pull/1827\n[#1831]: https://github.com/tokio-rs/tokio/pull/1831\n\n# 0.2.0 (November 26, 2019)\n\nA major breaking change. Most implementation and APIs have changed one way or\nanother. This changelog entry contains a highlight\n\n### Changed\n\n- APIs are updated to use `async / await`.\n- most `tokio-*` crates are collapsed into this crate.\n- Scheduler is rewritten.\n- `tokio::spawn` returns a `JoinHandle`.\n- A single I/O / timer is used per runtime.\n- I/O driver uses a concurrent slab for allocating state.\n- components are made available via feature flag.\n- Use `bytes` 0.5\n- `tokio::codec` is moved to `tokio-util`.\n\n### Removed\n\n- Standalone `timer` and `net` drivers are removed, use `Runtime` instead\n- `current_thread` runtime is removed, use `tokio::runtime::Runtime` with\n `basic_scheduler` instead.\n\n# 0.1.21 (May 30, 2019)\n\n### Changed\n\n- Bump `tokio-trace-core` version to 0.2 ([#1111]).\n\n[#1111]: https://github.com/tokio-rs/tokio/pull/1111\n\n# 0.1.20 (May 14, 2019)\n\n### Added\n\n- `tokio::runtime::Builder::panic_handler` allows configuring handling\n panics on the runtime ([#1055]).\n\n[#1055]: https://github.com/tokio-rs/tokio/pull/1055\n\n# 0.1.19 (April 22, 2019)\n\n### Added\n\n- Re-export `tokio::sync::Mutex` primitive ([#964]).\n\n[#964]: https://github.com/tokio-rs/tokio/pull/964\n\n# 0.1.18 (March 22, 2019)\n\n### Added\n\n- `TypedExecutor` re-export and implementations ([#993]).\n\n[#993]: https://github.com/tokio-rs/tokio/pull/993\n\n# 0.1.17 (March 13, 2019)\n\n### Added\n\n- Propagate trace subscriber in the runtime ([#966]).\n\n[#966]: https://github.com/tokio-rs/tokio/pull/966\n\n# 0.1.16 (March 1, 2019)\n\n### Fixed\n\n- async-await: track latest nightly changes ([#940]).\n\n### Added\n\n- `sync::Watch`, a single value broadcast channel ([#922]).\n- Async equivalent of read / write file helpers being added to `std` ([#896]).\n\n[#896]: https://github.com/tokio-rs/tokio/pull/896\n[#922]: https://github.com/tokio-rs/tokio/pull/922\n[#940]: https://github.com/tokio-rs/tokio/pull/940\n\n# 0.1.15 (January 24, 2019)\n\n### Added\n\n- Re-export tokio-sync APIs ([#839]).\n- Stream enumerate combinator ([#832]).\n\n[#832]: https://github.com/tokio-rs/tokio/pull/832\n[#839]: https://github.com/tokio-rs/tokio/pull/839\n\n# 0.1.14 (January 6, 2019)\n\n- Use feature flags to break up the crate, allowing users to pick & choose\n components ([#808]).\n- Export `UnixDatagram` and `UnixDatagramFramed` ([#772]).\n\n[#772]: https://github.com/tokio-rs/tokio/pull/772\n[#808]: https://github.com/tokio-rs/tokio/pull/808\n\n# 0.1.13 (November 21, 2018)\n\n- Fix `Runtime::reactor()` when no tasks are spawned ([#721]).\n- `runtime::Builder` no longer uses deprecated methods ([#749]).\n- Provide `after_start` and `before_stop` configuration settings for\n `Runtime` ([#756]).\n- Implement throttle stream combinator ([#736]).\n\n[#721]: https://github.com/tokio-rs/tokio/pull/721\n[#736]: https://github.com/tokio-rs/tokio/pull/736\n[#749]: https://github.com/tokio-rs/tokio/pull/749\n[#756]: https://github.com/tokio-rs/tokio/pull/756\n\n# 0.1.12 (October 23, 2018)\n\n- runtime: expose `keep_alive` on runtime builder ([#676]).\n- runtime: create a reactor per worker thread ([#660]).\n- codec: fix panic in `LengthDelimitedCodec` ([#682]).\n- io: re-export `tokio_io::io::read` function ([#689]).\n- runtime: check for executor re-entry in more places ([#708]).\n\n[#660]: https://github.com/tokio-rs/tokio/pull/660\n[#676]: https://github.com/tokio-rs/tokio/pull/676\n[#682]: https://github.com/tokio-rs/tokio/pull/682\n[#689]: https://github.com/tokio-rs/tokio/pull/689\n[#708]: https://github.com/tokio-rs/tokio/pull/708\n\n# 0.1.11 (September 28, 2018)\n\n- Fix `tokio-async-await` dependency ([#675]).\n\n[#675]: https://github.com/tokio-rs/tokio/pull/675\n\n# 0.1.10 (September 27, 2018)\n\n- Fix minimal versions\n\n# 0.1.9 (September 27, 2018)\n\n- Experimental async/await improvements ([#661]).\n- Re-export `TaskExecutor` from `tokio-current-thread` ([#652]).\n- Improve `Runtime` builder API ([#645]).\n- `tokio::run` panics when called from the context of an executor\n ([#646]).\n- Introduce `StreamExt` with a `timeout` helper ([#573]).\n- Move `length_delimited` into `tokio` ([#575]).\n- Re-organize `tokio::net` module ([#548]).\n- Re-export `tokio-current-thread::spawn` in current_thread runtime\n ([#579]).\n\n[#548]: https://github.com/tokio-rs/tokio/pull/548\n[#573]: https://github.com/tokio-rs/tokio/pull/573\n[#575]: https://github.com/tokio-rs/tokio/pull/575\n[#579]: https://github.com/tokio-rs/tokio/pull/579\n[#645]: https://github.com/tokio-rs/tokio/pull/645\n[#646]: https://github.com/tokio-rs/tokio/pull/646\n[#652]: https://github.com/tokio-rs/tokio/pull/652\n[#661]: https://github.com/tokio-rs/tokio/pull/661\n\n# 0.1.8 (August 23, 2018)\n\n- Extract tokio::executor::current_thread to a sub crate ([#370])\n- Add `Runtime::block_on` ([#398])\n- Add `runtime::current_thread::block_on_all` ([#477])\n- Misc documentation improvements ([#450])\n- Implement `std::error::Error` for error types ([#501])\n\n[#370]: https://github.com/tokio-rs/tokio/pull/370\n[#398]: https://github.com/tokio-rs/tokio/pull/398\n[#450]: https://github.com/tokio-rs/tokio/pull/450\n[#477]: https://github.com/tokio-rs/tokio/pull/477\n[#501]: https://github.com/tokio-rs/tokio/pull/501\n\n# 0.1.7 (June 6, 2018)\n\n- Add `Runtime::block_on` for concurrent runtime ([#391]).\n- Provide handle to `current_thread::Runtime` that allows spawning tasks from\n other threads ([#340]).\n- Provide `clock::now()`, a configurable source of time ([#381]).\n\n[#340]: https://github.com/tokio-rs/tokio/pull/340\n[#381]: https://github.com/tokio-rs/tokio/pull/381\n[#391]: https://github.com/tokio-rs/tokio/pull/391\n\n# 0.1.6 (May 2, 2018)\n\n- Add asynchronous filesystem APIs ([#323]).\n- Add \"current thread\" runtime variant ([#308]).\n- `CurrentThread`: Expose inner `Park` instance.\n- Improve fairness of `CurrentThread` executor ([#313]).\n\n[#308]: https://github.com/tokio-rs/tokio/pull/308\n[#313]: https://github.com/tokio-rs/tokio/pull/313\n[#323]: https://github.com/tokio-rs/tokio/pull/323\n\n# 0.1.5 (March 30, 2018)\n\n- Provide timer API ([#266])\n\n[#266]: https://github.com/tokio-rs/tokio/pull/266\n\n# 0.1.4 (March 22, 2018)\n\n- Fix build on FreeBSD ([#218])\n- Shutdown the Runtime when the handle is dropped ([#214])\n- Set Runtime thread name prefix for worker threads ([#232])\n- Add builder for Runtime ([#234])\n- Extract TCP and UDP types into separate crates ([#224])\n- Optionally support futures 0.2.\n\n[#214]: https://github.com/tokio-rs/tokio/pull/214\n[#218]: https://github.com/tokio-rs/tokio/pull/218\n[#224]: https://github.com/tokio-rs/tokio/pull/224\n[#232]: https://github.com/tokio-rs/tokio/pull/232\n[#234]: https://github.com/tokio-rs/tokio/pull/234\n\n# 0.1.3 (March 09, 2018)\n\n- Fix `CurrentThread::turn` to block on idle ([#212]).\n\n[#212]: https://github.com/tokio-rs/tokio/pull/212\n\n# 0.1.2 (March 09, 2018)\n\n- Introduce Tokio Runtime ([#141])\n- Provide `CurrentThread` for more flexible usage of current thread executor ([#141]).\n- Add Lio for platforms that support it ([#142]).\n- I/O resources now lazily bind to the reactor ([#160]).\n- Extract Reactor to dedicated crate ([#169])\n- Add facade to sub crates and add prelude ([#166]).\n- Switch TCP/UDP fns to poll\\_ -> Poll<...> style ([#175])\n\n[#141]: https://github.com/tokio-rs/tokio/pull/141\n[#142]: https://github.com/tokio-rs/tokio/pull/142\n[#160]: https://github.com/tokio-rs/tokio/pull/160\n[#166]: https://github.com/tokio-rs/tokio/pull/166\n[#169]: https://github.com/tokio-rs/tokio/pull/169\n[#175]: https://github.com/tokio-rs/tokio/pull/175\n\n# 0.1.1 (February 09, 2018)\n\n- Doc fixes\n\n# 0.1.0 (February 07, 2018)\n\n- Initial crate released based on [RFC](https://github.com/tokio-rs/tokio-rfcs/pull/3).\n" }, { "path": "tokio/Cargo.toml", "content": "[package]\nname = \"tokio\"\n# When releasing to crates.io:\n# - Remove path dependencies (if any)\n# - Update doc url\n# - README.md\n# - Update CHANGELOG.md.\n# - Create \"v1.x.y\" git tag.\nversion = \"1.50.0\"\nedition = \"2021\"\nrust-version = \"1.71\"\nauthors = [\"Tokio Contributors \"]\nlicense = \"MIT\"\nreadme = \"README.md\"\nrepository = \"https://github.com/tokio-rs/tokio\"\nhomepage = \"https://tokio.rs\"\ndescription = \"\"\"\nAn event-driven, non-blocking I/O platform for writing asynchronous I/O\nbacked applications.\n\"\"\"\ncategories = [\"asynchronous\", \"network-programming\"]\nkeywords = [\"io\", \"async\", \"non-blocking\", \"futures\"]\n\n[features]\n# Include nothing by default\ndefault = []\n\n# enable everything\nfull = [\n \"fs\",\n \"io-util\",\n \"io-std\",\n \"macros\",\n \"net\",\n \"parking_lot\",\n \"process\",\n \"rt\",\n \"rt-multi-thread\",\n \"signal\",\n \"sync\",\n \"time\",\n]\n\nfs = []\nio-util = [\"bytes\"]\n# stdin, stdout, stderr\nio-std = []\nmacros = [\"tokio-macros\"]\nnet = [\n \"libc\",\n \"mio/os-poll\",\n \"mio/os-ext\",\n \"mio/net\",\n \"socket2\",\n \"windows-sys/Win32_Foundation\",\n \"windows-sys/Win32_Security\",\n \"windows-sys/Win32_Storage_FileSystem\",\n \"windows-sys/Win32_System_Pipes\",\n \"windows-sys/Win32_System_SystemServices\",\n]\nprocess = [\n \"bytes\",\n \"libc\",\n \"mio/os-poll\",\n \"mio/os-ext\",\n \"mio/net\",\n \"signal-hook-registry\",\n \"windows-sys/Win32_Foundation\",\n \"windows-sys/Win32_System_Threading\",\n \"windows-sys/Win32_System_WindowsProgramming\",\n]\n# Includes basic task execution capabilities\nrt = []\nrt-multi-thread = [\"rt\"]\nsignal = [\n \"libc\",\n \"mio/os-poll\",\n \"mio/net\",\n \"mio/os-ext\",\n \"signal-hook-registry\",\n \"windows-sys/Win32_Foundation\",\n \"windows-sys/Win32_System_Console\",\n]\nsync = []\ntest-util = [\"rt\", \"sync\", \"time\"]\ntime = []\n# Unstable feature. Requires `--cfg tokio_unstable` to enable.\nio-uring = [\"dep:io-uring\", \"libc\", \"mio/os-poll\", \"mio/os-ext\", \"dep:slab\"]\n# Unstable feature. Requires `--cfg tokio_unstable` to enable.\ntaskdump = [\"dep:backtrace\"]\n\n[dependencies]\ntokio-macros = { version = \"~2.6.0\", optional = true }\n\npin-project-lite = \"0.2.11\"\n\n# Everything else is optional...\nbytes = { version = \"1.2.1\", optional = true }\nmio = { version = \"1.0.1\", optional = true, default-features = false }\nparking_lot = { version = \"0.12.0\", optional = true }\n\n[target.'cfg(not(target_family = \"wasm\"))'.dependencies]\nsocket2 = { version = \"0.6.0\", optional = true, features = [\"all\"] }\n\n# Currently unstable. The API exposed by these features may be broken at any time.\n# Requires `--cfg tokio_unstable` to enable.\n[target.'cfg(tokio_unstable)'.dependencies]\ntracing = { version = \"0.1.29\", default-features = false, features = [\"std\"], optional = true } # Not in full\n\n# Currently unstable. The API exposed by these features may be broken at any time.\n# Requires `--cfg tokio_unstable` to enable.\n[target.'cfg(all(tokio_unstable, target_os = \"linux\"))'.dependencies]\nio-uring = { version = \"0.7.11\", default-features = false, optional = true }\nlibc = { version = \"0.2.168\", optional = true }\nmio = { version = \"1.0.1\", default-features = false, features = [\"os-poll\", \"os-ext\"], optional = true }\nslab = { version = \"0.4.9\", optional = true }\nbacktrace = { version = \"0.3.58\", optional = true }\n\n[target.'cfg(unix)'.dependencies]\nlibc = { version = \"0.2.168\", optional = true }\nsignal-hook-registry = { version = \"1.1.1\", optional = true }\n\n[target.'cfg(unix)'.dev-dependencies]\nlibc = { version = \"0.2.168\" }\nnix = { version = \"0.29.0\", default-features = false, features = [\"aio\", \"fs\", \"socket\"] }\n\n[target.'cfg(windows)'.dependencies.windows-sys]\nversion = \"0.61\"\noptional = true\n\n[target.'cfg(windows)'.dev-dependencies.windows-sys]\nversion = \"0.61\"\nfeatures = [\n \"Win32_Foundation\",\n \"Win32_Security_Authorization\",\n]\n\n[dev-dependencies]\ntokio-test = \"0.4.0\"\ntokio-stream = \"0.1\"\ntokio-util = { version = \"0.7\", features = [\"rt\"] }\nfutures = { version = \"0.3.0\", features = [\"async-await\"] }\nfutures-test = \"0.3.31\"\nmockall = \"0.13.0\"\nasync-stream = \"0.3\"\nfutures-concurrency = \"7.6.3\"\n\n[target.'cfg(not(target_family = \"wasm\"))'.dev-dependencies]\nsocket2 = \"0.6.0\"\ntempfile = \"3.1.0\"\nproptest = \"1\"\n\n[target.'cfg(not(all(target_family = \"wasm\", target_os = \"unknown\")))'.dev-dependencies]\nrand = \"0.9\"\n\n[target.'cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))'.dev-dependencies]\nwasm-bindgen-test = \"0.3.0\"\n\n[target.'cfg(target_os = \"freebsd\")'.dev-dependencies]\nmio-aio = { version = \"1\", features = [\"tokio\"] }\n\n[target.'cfg(loom)'.dev-dependencies]\nloom = { version = \"0.7\", features = [\"futures\", \"checkpoint\"] }\n\n[target.'cfg(all(tokio_unstable, target_has_atomic = \"64\"))'.dev-dependencies]\ntracing-mock = \"= 0.1.0-beta.1\"\n\n[package.metadata.docs.rs]\nall-features = true\n# enable unstable features in the documentation\nrustdoc-args = [\"--cfg\", \"docsrs\", \"--cfg\", \"tokio_unstable\"]\n# it's necessary to _also_ pass `--cfg tokio_unstable`\n# to rustc, or else dependencies will not be enabled, and the docs build will fail.\nrustc-args = [\"--cfg\", \"tokio_unstable\"]\n\n[package.metadata.playground]\nfeatures = [\"full\", \"test-util\"]\n\n[package.metadata.cargo_check_external_types]\n# The following are types that are allowed to be exposed in Tokio's public API.\n# The standard library is allowed by default.\nallowed_external_types = [\n \"bytes::buf::buf_impl::Buf\",\n \"bytes::buf::buf_mut::BufMut\",\n \"tokio_macros::*\",\n]\n\n[lints]\nworkspace = true\n" }, { "path": "tokio/LICENSE", "content": "MIT License\n\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "tokio/README.md", "content": "*[TokioConf 2026 program and tickets are now available!](https://tokioconf.com)*\n\n---\n\n# Tokio\n\nA runtime for writing reliable, asynchronous, and slim applications with\nthe Rust programming language. It is:\n\n* **Fast**: Tokio's zero-cost abstractions give you bare-metal\n performance.\n\n* **Reliable**: Tokio leverages Rust's ownership, type system, and\n concurrency model to reduce bugs and ensure thread safety.\n\n* **Scalable**: Tokio has a minimal footprint, and handles backpressure\n and cancellation naturally.\n\n[![Crates.io][crates-badge]][crates-url]\n[![MIT licensed][mit-badge]][mit-url]\n[![Build Status][actions-badge]][actions-url]\n[![Discord chat][discord-badge]][discord-url]\n\n[crates-badge]: https://img.shields.io/crates/v/tokio.svg\n[crates-url]: https://crates.io/crates/tokio\n[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg\n[mit-url]: https://github.com/tokio-rs/tokio/blob/master/LICENSE\n[actions-badge]: https://github.com/tokio-rs/tokio/workflows/CI/badge.svg\n[actions-url]: https://github.com/tokio-rs/tokio/actions?query=workflow%3ACI+branch%3Amaster\n[discord-badge]: https://img.shields.io/discord/500028886025895936.svg?logo=discord&style=flat-square\n[discord-url]: https://discord.gg/tokio\n\n[Website](https://tokio.rs) |\n[Guides](https://tokio.rs/tokio/tutorial) |\n[API Docs](https://docs.rs/tokio/latest/tokio) |\n[Chat](https://discord.gg/tokio)\n\n## Overview\n\nTokio is an event-driven, non-blocking I/O platform for writing\nasynchronous applications with the Rust programming language. At a high\nlevel, it provides a few major components:\n\n* A multithreaded, work-stealing based task [scheduler].\n* A reactor backed by the operating system's event queue (epoll, kqueue,\n IOCP, etc.).\n* Asynchronous [TCP and UDP][net] sockets.\n\nThese components provide the runtime components necessary for building\nan asynchronous application.\n\n[net]: https://docs.rs/tokio/latest/tokio/net/index.html\n[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html\n\n## Example\n\nA basic TCP echo server with Tokio.\n\nMake sure you enable the full features of the tokio crate on Cargo.toml:\n\n```toml\n[dependencies]\ntokio = { version = \"1.50.0\", features = [\"full\"] }\n```\nThen, on your main.rs:\n\n```rust,no_run\nuse tokio::net::TcpListener;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\n\n#[tokio::main]\nasync fn main() -> Result<(), Box> {\n let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n\n loop {\n let (mut socket, _) = listener.accept().await?;\n\n tokio::spawn(async move {\n let mut buf = [0; 1024];\n\n // In a loop, read data from the socket and write the data back.\n loop {\n let n = match socket.read(&mut buf).await {\n // socket closed\n Ok(0) => return,\n Ok(n) => n,\n Err(e) => {\n eprintln!(\"failed to read from socket; err = {:?}\", e);\n return;\n }\n };\n\n // Write the data back\n if let Err(e) = socket.write_all(&buf[0..n]).await {\n eprintln!(\"failed to write to socket; err = {:?}\", e);\n return;\n }\n }\n });\n }\n}\n```\n\nMore examples can be found [here][examples]. For a larger \"real world\" example, see the\n[mini-redis] repository.\n\n[examples]: https://github.com/tokio-rs/tokio/tree/master/examples\n[mini-redis]: https://github.com/tokio-rs/mini-redis/\n\nTo see a list of the available feature flags that can be enabled, check our\n[docs][feature-flag-docs].\n\n## Getting Help\n\nFirst, see if the answer to your question can be found in the [Guides] or the\n[API documentation]. If the answer is not there, there is an active community in\nthe [Tokio Discord server][chat]. We would be happy to try to answer your\nquestion. You can also ask your question on [the discussions page][discussions].\n\n[Guides]: https://tokio.rs/tokio/tutorial\n[API documentation]: https://docs.rs/tokio/latest/tokio\n[chat]: https://discord.gg/tokio\n[discussions]: https://github.com/tokio-rs/tokio/discussions\n[feature-flag-docs]: https://docs.rs/tokio/#feature-flags\n\n## Contributing\n\n:balloon: Thanks for your help improving the project! We are so happy to have\nyou! We have a [contributing guide][guide] to help you get involved in the Tokio\nproject.\n\n[guide]: https://github.com/tokio-rs/tokio/blob/master/docs/contributing/README.md\n\n## Related Projects\n\nIn addition to the crates in this repository, the Tokio project also maintains\nseveral other libraries, including:\n\n* [`axum`]: A web application framework that focuses on ergonomics and modularity.\n\n* [`hyper`]: A fast and correct HTTP/1.1 and HTTP/2 implementation for Rust.\n\n* [`tonic`]: A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility.\n\n* [`warp`]: A super-easy, composable, web server framework for warp speeds.\n\n* [`tower`]: A library of modular and reusable components for building robust networking clients and servers.\n\n* [`tracing`] (formerly `tokio-trace`): A framework for application-level tracing and async-aware diagnostics.\n\n* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers `tokio`.\n\n* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.\n\n* [`loom`]: A testing tool for concurrent Rust code.\n\n[`axum`]: https://github.com/tokio-rs/axum\n[`warp`]: https://github.com/seanmonstar/warp\n[`hyper`]: https://github.com/hyperium/hyper\n[`tonic`]: https://github.com/hyperium/tonic\n[`tower`]: https://github.com/tower-rs/tower\n[`loom`]: https://github.com/tokio-rs/loom\n[`tracing`]: https://github.com/tokio-rs/tracing\n[`mio`]: https://github.com/tokio-rs/mio\n[`bytes`]: https://github.com/tokio-rs/bytes\n\n## Changelog\n\nThe Tokio repository contains multiple crates. Each crate has its own changelog.\n\n * `tokio` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio/CHANGELOG.md)\n * `tokio-util` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-util/CHANGELOG.md)\n * `tokio-stream` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-stream/CHANGELOG.md)\n * `tokio-macros` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-macros/CHANGELOG.md)\n * `tokio-test` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-test/CHANGELOG.md)\n\n## Supported Rust Versions\n\n\n\nTokio will keep a rolling MSRV (minimum supported rust version) policy of **at\nleast** 6 months. When increasing the MSRV, the new Rust version must have been\nreleased at least six months ago. The current MSRV is 1.71.\n\nNote that the MSRV is not increased automatically, and only as part of a minor\nrelease. The MSRV history for past minor releases can be found below:\n\n * 1.48 to now - Rust 1.71\n * 1.39 to 1.47 - Rust 1.70\n * 1.30 to 1.38 - Rust 1.63\n * 1.27 to 1.29 - Rust 1.56\n * 1.17 to 1.26 - Rust 1.49\n * 1.15 to 1.16 - Rust 1.46\n * 1.0 to 1.14 - Rust 1.45\n\nNote that although we try to avoid the situation where a dependency transitively\nincreases the MSRV of Tokio, we do not guarantee that this does not happen.\nHowever, every minor release will have some set of versions of dependencies that\nworks with the MSRV of that minor release.\n\n## Release schedule\n\nTokio doesn't follow a fixed release schedule, but we typically make one minor\nrelease each month. We make patch releases for bugfixes as necessary.\n\n## Bug patching policy\n\nFor the purposes of making patch releases with bugfixes, we have designated\ncertain minor releases as LTS (long term support) releases. Whenever a bug\nwarrants a patch release with a fix for the bug, it will be backported and\nreleased as a new patch release for each LTS minor version. Our current LTS\nreleases are:\n\n * `1.43.x` - LTS release until March 2026. (MSRV 1.70)\n * `1.47.x` - LTS release until September 2026. (MSRV 1.70)\n\nEach LTS release will continue to receive backported fixes for at least a year.\nIf you wish to use a fixed minor release in your project, we recommend that you\nuse an LTS release.\n\nTo use a fixed minor version, you can specify the version with a tilde. For\nexample, to specify that you wish to use the newest `1.43.x` patch release, you\ncan use the following dependency specification:\n```text\ntokio = { version = \"~1.43\", features = [...] }\n```\n\n### Previous LTS releases\n\n * `1.8.x` - LTS release until February 2022.\n * `1.14.x` - LTS release until June 2022.\n * `1.18.x` - LTS release until June 2023.\n * `1.20.x` - LTS release until September 2023.\n * `1.25.x` - LTS release until March 2024.\n * `1.32.x` - LTS release until September 2024.\n * `1.36.x` - LTS release until March 2025.\n * `1.38.x` - LTS release until July 2025.\n\n## License\n\nThis project is licensed under the [MIT license].\n\n[MIT license]: https://github.com/tokio-rs/tokio/blob/master/LICENSE\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "tokio/docs/reactor-refactor.md", "content": "# Refactor I/O driver\n\nDescribes changes to the I/O driver for the Tokio 0.3 release.\n\n## Goals\n\n* Support `async fn` on I/O types with `&self`.\n* Refine the `Registration` API.\n\n### Non-goals\n\n* Implement `AsyncRead` / `AsyncWrite` for `&TcpStream` or other reference type.\n\n## Overview\n\nCurrently, I/O types require `&mut self` for `async` functions. The reason for\nthis is the task's waker is stored in the I/O resource's internal state\n(`ScheduledIo`) instead of in the future returned by the `async` function.\nBecause of this limitation, I/O types limit the number of wakers to one per\ndirection (a direction is either read-related events or write-related events).\n\nMoving the waker from the internal I/O resource's state to the operation's\nfuture enables multiple wakers to be registered per operation. The \"intrusive\nwake list\" strategy used by `Notify` applies to this case, though there are some\nconcerns unique to the I/O driver.\n\n## Reworking the `Registration` type\n\nWhile `Registration` is made private (per #2728), it remains in Tokio as an\nimplementation detail backing I/O resources such as `TcpStream`. The API of\n`Registration` is updated to support waiting for an arbitrary interest set with\n`&self`. This supports concurrent waiters with a different readiness interest.\n\n```rust\nstruct Registration { ... }\n\n// TODO: naming\nstruct ReadyEvent {\n tick: u32,\n ready: mio::Ready,\n}\n\nimpl Registration {\n /// `interest` must be a super set of **all** interest sets specified in\n /// the other methods. This is the interest set passed to `mio`.\n pub fn new(io: &T, interest: mio::Ready) -> io::Result\n where T: mio::Evented;\n\n /// Awaits for any readiness event included in `interest`. Returns a\n /// `ReadyEvent` representing the received readiness event.\n async fn readiness(&self, interest: mio::Ready) -> io::Result;\n\n /// Clears resource level readiness represented by the specified `ReadyEvent`\n async fn clear_readiness(&self, ready_event: ReadyEvent);\n```\n\nA new registration is created for a `T: mio::Evented` and a `interest`. This\ncreates a `ScheduledIo` entry with the I/O driver and registers the resource\nwith `mio`.\n\nBecause Tokio uses **edge-triggered** notifications, the I/O driver only\nreceives readiness from the OS once the ready state **changes**. The I/O driver\nmust track each resource's known readiness state. This helps prevent syscalls\nwhen the process knows the syscall should return with `EWOULDBLOCK`.\n\nA call to `readiness()` checks if the currently known resource readiness\noverlaps with `interest`. If it does, then the `readiness()` immediately\nreturns. If it does not, then the task waits until the I/O driver receives a\nreadiness event.\n\nThe pseudocode to perform a TCP read is as follows.\n\n```rust\nasync fn read(&self, buf: &mut [u8]) -> io::Result {\n loop {\n // Await readiness\n let event = self.readiness(interest).await?;\n\n match self.mio_socket.read(buf) {\n Ok(v) => return Ok(v),\n Err(ref e) if e.kind() == WouldBlock => {\n self.clear_readiness(event);\n }\n Err(e) => return Err(e),\n }\n }\n}\n```\n\n## Reworking the `ScheduledIo` type\n\nThe `ScheduledIo` type is switched to use an intrusive waker linked list. Each\nentry in the linked list includes the `interest` set passed to `readiness()`.\n\n```rust\n#[derive(Debug)]\npub(crate) struct ScheduledIo {\n /// Resource's known state packed with other state that must be\n /// atomically updated.\n readiness: AtomicUsize,\n\n /// Tracks tasks waiting on the resource\n waiters: Mutex,\n}\n\n#[derive(Debug)]\nstruct Waiters {\n // List of intrusive waiters.\n list: LinkedList,\n\n /// Waiter used by `AsyncRead` implementations.\n reader: Option,\n\n /// Waiter used by `AsyncWrite` implementations.\n writer: Option,\n}\n\n// This struct is contained by the **future** returned by `readiness()`.\n#[derive(Debug)]\nstruct Waiter {\n /// Intrusive linked-list pointers\n pointers: linked_list::Pointers,\n\n /// Waker for task waiting on I/O resource\n waiter: Option,\n\n /// Readiness events being waited on. This is\n /// the value passed to `readiness()`\n interest: mio::Ready,\n\n /// Should not be `Unpin`.\n _p: PhantomPinned,\n}\n```\n\nWhen an I/O event is received from `mio`, the associated resources' readiness is\nupdated and the waiter list is iterated. All waiters with `interest` that\noverlap the received readiness event are notified. Any waiter with an `interest`\nthat does not overlap the readiness event remains in the list.\n\n## Cancel interest on drop\n\nThe future returned by `readiness()` uses an intrusive linked list to store the\nwaker with `ScheduledIo`. Because `readiness()` can be called concurrently, many\nwakers may be stored simultaneously in the list. If the `readiness()` future is\ndropped early, it is essential that the waker is removed from the list. This\nprevents leaking memory.\n\n## Race condition\n\nConsider how many tasks may concurrently attempt I/O operations. This, combined\nwith how Tokio uses edge-triggered events, can result in a race condition. Let's\nrevisit the TCP read function:\n\n```rust\nasync fn read(&self, buf: &mut [u8]) -> io::Result {\n loop {\n // Await readiness\n let event = self.readiness(interest).await?;\n\n match self.mio_socket.read(buf) {\n Ok(v) => return Ok(v),\n Err(ref e) if e.kind() == WouldBlock => {\n self.clear_readiness(event);\n }\n Err(e) => return Err(e),\n }\n }\n}\n```\n\nIf care is not taken, if between `mio_socket.read(buf)` returning and\n`clear_readiness(event)` is called, a readiness event arrives, the `read()`\nfunction could deadlock. This happens because the readiness event is received,\n`clear_readiness()` unsets the readiness event, and on the next iteration,\n`readiness().await` will block forever as a new readiness event is not received.\n\nThe current I/O driver handles this condition by always registering the task's\nwaker before performing the operation. This is not ideal as it will result in\nunnecessary task notification.\n\nInstead, we will use a strategy to prevent clearing readiness if an \"unseen\"\nreadiness event has been received. The I/O driver will maintain a \"tick\" value.\nEvery time the `mio` `poll()` function is called, the tick is incremented. Each\nreadiness event has an associated tick. When the I/O driver sets the resource's\nreadiness, the driver's tick is packed into the atomic `usize`.\n\nThe `ScheduledIo` readiness `AtomicUsize` is structured as:\n\n```\n| shutdown | generation | driver tick | readiness |\n|----------+------------+--------------+-----------|\n| 1 bit | 7 bits + 8 bits + 16 bits |\n```\n\nThe `shutdown` and `generation` components exist today.\n\nThe `readiness()` function returns a `ReadyEvent` value. This value includes the\n`tick` component read with the resource's readiness value. When\n`clear_readiness()` is called, the `ReadyEvent` is provided. Readiness is only\ncleared if the current `tick` matches the `tick` included in the `ReadyEvent`.\nIf the tick values do not match, the call to `readiness()` on the next iteration\nwill not block and the new `tick` is included in the new `ReadyToken.`\n\nTODO\n\n## Implementing `AsyncRead` / `AsyncWrite`\n\nThe `AsyncRead` and `AsyncWrite` traits use a \"poll\" based API. This means that\nit is not possible to use an intrusive linked list to track the waker.\nAdditionally, there is no future associated with the operation which means it is\nnot possible to cancel interest in the readiness events.\n\nTo implement `AsyncRead` and `AsyncWrite`, `ScheduledIo` includes dedicated\nwaker values for the read direction and the write direction. These values are\nused to store the waker. Specific `interest` is not tracked for `AsyncRead` and\n`AsyncWrite` implementations. It is assumed that only events of interest are:\n\n* Read ready\n* Read closed\n* Write ready\n* Write closed\n\nNote that \"read closed\" and \"write closed\" are only available with Mio 0.7. With\nMio 0.6, things were a bit messy.\n\nIt is only possible to implement `AsyncRead` and `AsyncWrite` for resource types\nthemselves and not for `&Resource`. Implementing the traits for `&Resource`\nwould permit concurrent operations to the resource. Because only a single waker\nis stored per direction, any concurrent usage would result in deadlocks. An\nalternate implementation would call for a `Vec` but this would result in\nmemory leaks.\n\n## Enabling reads and writes for `&TcpStream`\n\nInstead of implementing `AsyncRead` and `AsyncWrite` for `&TcpStream`, a new\nfunction is added to `TcpStream`.\n\n```rust\nimpl TcpStream {\n /// Naming TBD\n fn by_ref(&self) -> TcpStreamRef<'_>;\n}\n\nstruct TcpStreamRef<'a> {\n stream: &'a TcpStream,\n\n // `Waiter` is the node in the intrusive waiter linked-list\n read_waiter: Waiter,\n write_waiter: Waiter,\n}\n```\n\nNow, `AsyncRead` and `AsyncWrite` can be implemented on `TcpStreamRef<'a>`. When\nthe `TcpStreamRef` is dropped, all associated waker resources are cleaned up.\n\n### Removing all the `split()` functions\n\nWith `TcpStream::by_ref()`, `TcpStream::split()` is no longer needed. Instead,\nit is possible to do something as follows.\n\n```rust\nlet rd = my_stream.by_ref();\nlet wr = my_stream.by_ref();\n\nselect! {\n // use `rd` and `wr` in separate branches.\n}\n```\n\nIt is also possible to store a `TcpStream` in an `Arc`.\n\n```rust\nlet arc_stream = Arc::new(my_tcp_stream);\nlet n = arc_stream.by_ref().read(buf).await?;\n```\n" }, { "path": "tokio/fuzz/.gitignore", "content": "target\ncorpus\nartifacts\ncoverage\n" }, { "path": "tokio/fuzz/Cargo.toml", "content": "[package]\nname = \"tokio-fuzz\"\nversion = \"0.0.0\"\npublish = false\nedition = \"2021\"\n\n[package.metadata]\ncargo-fuzz = true\n\n[dependencies]\nlibfuzzer-sys = \"0.4\"\n\n[dependencies.tokio]\npath = \"..\"\nfeatures = [\"fs\",\"net\",\"process\",\"rt\",\"sync\",\"signal\",\"time\"]\n\n\n# Prevent this from interfering with workspaces\n[workspace]\nmembers = [\".\"]\n\n[profile.release]\ndebug = 1\n\n[[bin]]\nname = \"fuzz_linked_list\"\npath = \"fuzz_targets/fuzz_linked_list.rs\"\ntest = false\ndoc = false\n" }, { "path": "tokio/fuzz/fuzz_targets/fuzz_linked_list.rs", "content": "#![no_main]\n\nuse libfuzzer_sys::fuzz_target;\n\nfuzz_target!(|data: &[u8]| {\n tokio::fuzz::fuzz_linked_list(data);\n});\n" }, { "path": "tokio/src/blocking.rs", "content": "cfg_rt! {\n pub(crate) use crate::runtime::spawn_blocking;\n\n cfg_fs! {\n #[allow(unused_imports)]\n pub(crate) use crate::runtime::spawn_mandatory_blocking;\n }\n\n pub(crate) use crate::task::JoinHandle;\n}\n\ncfg_not_rt! {\n use std::fmt;\n use std::future::Future;\n use std::pin::Pin;\n use std::task::{Context, Poll};\n\n pub(crate) fn spawn_blocking(_f: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n assert_send_sync::>>();\n panic!(\"requires the `rt` Tokio feature flag\")\n }\n\n cfg_fs! {\n pub(crate) fn spawn_mandatory_blocking(_f: F) -> Option>\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n panic!(\"requires the `rt` Tokio feature flag\")\n }\n }\n\n pub(crate) struct JoinHandle {\n _p: std::marker::PhantomData,\n }\n\n unsafe impl Send for JoinHandle {}\n unsafe impl Sync for JoinHandle {}\n\n impl Future for JoinHandle {\n type Output = Result;\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n unreachable!()\n }\n }\n\n impl fmt::Debug for JoinHandle\n where\n T: fmt::Debug,\n {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"JoinHandle\").finish()\n }\n }\n\n fn assert_send_sync() {\n }\n}\n" }, { "path": "tokio/src/doc/mod.rs", "content": "//! Types which are documented locally in the Tokio crate, but does not actually\n//! live here.\n//!\n//! **Note** this module is only visible on docs.rs, you cannot use it directly\n//! in your own code.\n\n/// The name of a type which is not defined here.\n///\n/// This is typically used as an alias for another type, like so:\n///\n/// ```rust,ignore\n/// /// See [some::other::location](https://example.com).\n/// type DEFINED_ELSEWHERE = crate::doc::NotDefinedHere;\n/// ```\n///\n/// This type is uninhabitable like the [`never` type] to ensure that no one\n/// will ever accidentally use it.\n///\n/// [`never` type]: https://doc.rust-lang.org/std/primitive.never.html\n#[derive(Debug)]\npub enum NotDefinedHere {}\n\n#[cfg(feature = \"net\")]\nimpl mio::event::Source for NotDefinedHere {\n fn register(\n &mut self,\n _registry: &mio::Registry,\n _token: mio::Token,\n _interests: mio::Interest,\n ) -> std::io::Result<()> {\n Ok(())\n }\n fn reregister(\n &mut self,\n _registry: &mio::Registry,\n _token: mio::Token,\n _interests: mio::Interest,\n ) -> std::io::Result<()> {\n Ok(())\n }\n fn deregister(&mut self, _registry: &mio::Registry) -> std::io::Result<()> {\n Ok(())\n }\n}\n\n#[cfg(any(feature = \"net\", feature = \"fs\"))]\npub mod os;\n" }, { "path": "tokio/src/doc/os.rs", "content": "//! See [`std::os`](https://doc.rust-lang.org/std/os/index.html).\n\n/// Platform-specific extensions to `std` for Windows.\n///\n/// See [`std::os::windows`](https://doc.rust-lang.org/std/os/windows/index.html).\npub mod windows {\n /// Windows-specific extensions to general I/O primitives.\n ///\n /// See [`std::os::windows::io`](https://doc.rust-lang.org/std/os/windows/io/index.html).\n pub mod io {\n /// See [`std::os::windows::io::RawHandle`](https://doc.rust-lang.org/std/os/windows/io/type.RawHandle.html)\n pub type RawHandle = crate::doc::NotDefinedHere;\n\n /// See [`std::os::windows::io::OwnedHandle`](https://doc.rust-lang.org/std/os/windows/io/struct.OwnedHandle.html)\n pub type OwnedHandle = crate::doc::NotDefinedHere;\n\n /// See [`std::os::windows::io::AsRawHandle`](https://doc.rust-lang.org/std/os/windows/io/trait.AsRawHandle.html)\n pub trait AsRawHandle {\n /// See [`std::os::windows::io::AsRawHandle::as_raw_handle`](https://doc.rust-lang.org/std/os/windows/io/trait.AsRawHandle.html#tymethod.as_raw_handle)\n fn as_raw_handle(&self) -> RawHandle;\n }\n\n /// See [`std::os::windows::io::FromRawHandle`](https://doc.rust-lang.org/std/os/windows/io/trait.FromRawHandle.html)\n pub trait FromRawHandle {\n /// See [`std::os::windows::io::FromRawHandle::from_raw_handle`](https://doc.rust-lang.org/std/os/windows/io/trait.FromRawHandle.html#tymethod.from_raw_handle)\n unsafe fn from_raw_handle(handle: RawHandle) -> Self;\n }\n\n /// See [`std::os::windows::io::RawSocket`](https://doc.rust-lang.org/std/os/windows/io/type.RawSocket.html)\n pub type RawSocket = crate::doc::NotDefinedHere;\n\n /// See [`std::os::windows::io::AsRawSocket`](https://doc.rust-lang.org/std/os/windows/io/trait.AsRawSocket.html)\n pub trait AsRawSocket {\n /// See [`std::os::windows::io::AsRawSocket::as_raw_socket`](https://doc.rust-lang.org/std/os/windows/io/trait.AsRawSocket.html#tymethod.as_raw_socket)\n fn as_raw_socket(&self) -> RawSocket;\n }\n\n /// See [`std::os::windows::io::FromRawSocket`](https://doc.rust-lang.org/std/os/windows/io/trait.FromRawSocket.html)\n pub trait FromRawSocket {\n /// See [`std::os::windows::io::FromRawSocket::from_raw_socket`](https://doc.rust-lang.org/std/os/windows/io/trait.FromRawSocket.html#tymethod.from_raw_socket)\n unsafe fn from_raw_socket(sock: RawSocket) -> Self;\n }\n\n /// See [`std::os::windows::io::IntoRawSocket`](https://doc.rust-lang.org/std/os/windows/io/trait.IntoRawSocket.html)\n pub trait IntoRawSocket {\n /// See [`std::os::windows::io::IntoRawSocket::into_raw_socket`](https://doc.rust-lang.org/std/os/windows/io/trait.IntoRawSocket.html#tymethod.into_raw_socket)\n fn into_raw_socket(self) -> RawSocket;\n }\n\n /// See [`std::os::windows::io::BorrowedHandle`](https://doc.rust-lang.org/std/os/windows/io/struct.BorrowedHandle.html)\n pub type BorrowedHandle<'handle> = crate::doc::NotDefinedHere;\n\n /// See [`std::os::windows::io::AsHandle`](https://doc.rust-lang.org/std/os/windows/io/trait.AsHandle.html)\n pub trait AsHandle {\n /// See [`std::os::windows::io::AsHandle::as_handle`](https://doc.rust-lang.org/std/os/windows/io/trait.AsHandle.html#tymethod.as_handle)\n fn as_handle(&self) -> BorrowedHandle<'_>;\n }\n\n /// See [`std::os::windows::io::BorrowedSocket`](https://doc.rust-lang.org/std/os/windows/io/struct.BorrowedSocket.html)\n pub type BorrowedSocket<'socket> = crate::doc::NotDefinedHere;\n\n /// See [`std::os::windows::io::AsSocket`](https://doc.rust-lang.org/std/os/windows/io/trait.AsSocket.html)\n pub trait AsSocket {\n /// See [`std::os::windows::io::AsSocket::as_socket`](https://doc.rust-lang.org/std/os/windows/io/trait.AsSocket.html#tymethod.as_socket)\n fn as_socket(&self) -> BorrowedSocket<'_>;\n }\n }\n}\n" }, { "path": "tokio/src/fs/canonicalize.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::{Path, PathBuf};\n\n/// Returns the canonical, absolute form of a path with all intermediate\n/// components normalized and symbolic links resolved.\n///\n/// This is an async version of [`std::fs::canonicalize`].\n///\n/// # Platform-specific behavior\n///\n/// This function currently corresponds to the `realpath` function on Unix\n/// and the `CreateFile` and `GetFinalPathNameByHandle` functions on Windows.\n/// Note that, this [may change in the future][changes].\n///\n/// On Windows, this converts the path to use [extended length path][path]\n/// syntax, which allows your program to use longer path names, but means you\n/// can only join backslash-delimited paths to it, and it may be incompatible\n/// with other applications (if passed to the application on the command-line,\n/// or written to a file another application may read).\n///\n/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n/// [path]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath\n///\n/// # Errors\n///\n/// This function will return an error in the following situations, but is not\n/// limited to just these cases:\n///\n/// * `path` does not exist.\n/// * A non-final component in path is not a directory.\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n/// use std::io;\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// let path = fs::canonicalize(\"../a/../foo.txt\").await?;\n/// Ok(())\n/// }\n/// ```\npub async fn canonicalize(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::canonicalize(path)).await\n}\n" }, { "path": "tokio/src/fs/copy.rs", "content": "use crate::fs::asyncify;\nuse std::path::Path;\n\n/// Copies the contents of one file to another. This function will also copy the permission bits\n/// of the original file to the destination file.\n/// This function will overwrite the contents of to.\n///\n/// This is the async equivalent of [`std::fs::copy`].\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// fs::copy(\"foo.txt\", \"bar.txt\").await?;\n/// # Ok(())\n/// # }\n/// ```\npub async fn copy(from: impl AsRef, to: impl AsRef) -> Result {\n let from = from.as_ref().to_owned();\n let to = to.as_ref().to_owned();\n asyncify(|| std::fs::copy(from, to)).await\n}\n" }, { "path": "tokio/src/fs/create_dir.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Creates a new, empty directory at the provided path.\n///\n/// This is an async version of [`std::fs::create_dir`].\n///\n/// # Platform-specific behavior\n///\n/// This function currently corresponds to the `mkdir` function on Unix\n/// and the `CreateDirectory` function on Windows.\n/// Note that, this [may change in the future][changes].\n///\n/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n///\n/// **NOTE**: If a parent of the given path doesn't exist, this function will\n/// return an error. To create a directory and all its missing parents at the\n/// same time, use the [`create_dir_all`] function.\n///\n/// # Errors\n///\n/// This function will return an error in the following situations, but is not\n/// limited to just these cases:\n///\n/// * User lacks permissions to create directory at `path`.\n/// * A parent of the given path doesn't exist. (To create a directory and all\n/// its missing parents at the same time, use the [`create_dir_all`]\n/// function.)\n/// * `path` already exists.\n///\n/// [`create_dir_all`]: super::create_dir_all()\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n/// use std::io;\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// fs::create_dir(\"/some/dir\").await?;\n/// Ok(())\n/// }\n/// ```\npub async fn create_dir(path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::create_dir(path)).await\n}\n" }, { "path": "tokio/src/fs/create_dir_all.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Recursively creates a directory and all of its parent components if they\n/// are missing.\n///\n/// This is an async version of [`std::fs::create_dir_all`].\n///\n/// # Platform-specific behavior\n///\n/// This function currently corresponds to the `mkdir` function on Unix\n/// and the `CreateDirectory` function on Windows.\n/// Note that, this [may change in the future][changes].\n///\n/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n///\n/// # Errors\n///\n/// This function will return an error in the following situations, but is not\n/// limited to just these cases:\n///\n/// * If any directory in the path specified by `path` does not already exist\n/// and it could not be created otherwise. The specific error conditions for\n/// when a directory is being created (after it is determined to not exist) are\n/// outlined by [`fs::create_dir`].\n///\n/// Notable exception is made for situations where any of the directories\n/// specified in the `path` could not be created as it was being created concurrently.\n/// Such cases are considered to be successful. That is, calling `create_dir_all`\n/// concurrently from multiple threads or processes is guaranteed not to fail\n/// due to a race condition with itself.\n///\n/// [`fs::create_dir`]: std::fs::create_dir\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// #[tokio::main]\n/// async fn main() -> std::io::Result<()> {\n/// fs::create_dir_all(\"/some/dir\").await?;\n/// Ok(())\n/// }\n/// ```\npub async fn create_dir_all(path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::create_dir_all(path)).await\n}\n" }, { "path": "tokio/src/fs/dir_builder.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// A builder for creating directories in various manners.\n///\n/// This is a specialized version of [`std::fs::DirBuilder`] for usage on\n/// the Tokio runtime.\n#[derive(Debug, Default)]\npub struct DirBuilder {\n /// Indicates whether to create parent directories if they are missing.\n recursive: bool,\n\n /// Sets the Unix mode for newly created directories.\n #[cfg(unix)]\n pub(super) mode: Option,\n}\n\nimpl DirBuilder {\n /// Creates a new set of options with default mode/security settings for all\n /// platforms and also non-recursive.\n ///\n /// This is an async version of [`std::fs::DirBuilder::new`].\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::DirBuilder;\n ///\n /// let builder = DirBuilder::new();\n /// ```\n pub fn new() -> Self {\n DirBuilder::default()\n }\n\n /// Indicates whether to create directories recursively (including all parent directories).\n /// Parents that do not exist are created with the same security and permissions settings.\n ///\n /// This option defaults to `false`.\n ///\n /// This is an async version of [`std::fs::DirBuilder::recursive`].\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::DirBuilder;\n ///\n /// let mut builder = DirBuilder::new();\n /// builder.recursive(true);\n /// ```\n pub fn recursive(&mut self, recursive: bool) -> &mut Self {\n self.recursive = recursive;\n self\n }\n\n /// Creates the specified directory with the configured options.\n ///\n /// It is considered an error if the directory already exists unless\n /// recursive mode is enabled.\n ///\n /// This is an async version of [`std::fs::DirBuilder::create`].\n ///\n /// # Errors\n ///\n /// An error will be returned under the following circumstances:\n ///\n /// * Path already points to an existing file.\n /// * Path already points to an existing directory and the mode is\n /// non-recursive.\n /// * The calling process doesn't have permissions to create the directory\n /// or its missing parents.\n /// * Other I/O error occurred.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::DirBuilder;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// DirBuilder::new()\n /// .recursive(true)\n /// .create(\"/tmp/foo/bar/baz\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn create(&self, path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n let mut builder = std::fs::DirBuilder::new();\n builder.recursive(self.recursive);\n\n #[cfg(unix)]\n {\n if let Some(mode) = self.mode {\n std::os::unix::fs::DirBuilderExt::mode(&mut builder, mode);\n }\n }\n\n asyncify(move || builder.create(path)).await\n }\n}\n\nfeature! {\n #![unix]\n\n impl DirBuilder {\n /// Sets the mode to create new directories with.\n ///\n /// This option defaults to 0o777.\n ///\n /// # Examples\n ///\n ///\n /// ```no_run\n /// use tokio::fs::DirBuilder;\n ///\n /// let mut builder = DirBuilder::new();\n /// builder.mode(0o775);\n /// ```\n pub fn mode(&mut self, mode: u32) -> &mut Self {\n self.mode = Some(mode);\n self\n }\n }\n}\n" }, { "path": "tokio/src/fs/file/tests.rs", "content": "use super::*;\nuse crate::{\n fs::mocks::*,\n io::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt},\n};\nuse mockall::{predicate::eq, Sequence};\nuse tokio_test::{assert_pending, assert_ready_err, assert_ready_ok, task};\n\nconst HELLO: &[u8] = b\"hello world...\";\nconst FOO: &[u8] = b\"foo bar baz...\";\n\n#[test]\nfn open_read() {\n let mut file = MockFile::default();\n file.expect_inner_read().once().returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n let mut file = File::from_std(file);\n\n let mut buf = [0; 1024];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_eq!(0, pool::len());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n\n pool::run_one();\n\n assert!(t.is_woken());\n\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[test]\nfn read_twice_before_dispatch() {\n let mut file = MockFile::default();\n file.expect_inner_read().once().returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n let mut file = File::from_std(file);\n\n let mut buf = [0; 1024];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_pending!(t.poll());\n assert_pending!(t.poll());\n\n assert_eq!(pool::len(), 1);\n pool::run_one();\n\n assert!(t.is_woken());\n\n let n = assert_ready_ok!(t.poll());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[test]\nfn read_with_smaller_buf() {\n let mut file = MockFile::default();\n file.expect_inner_read().once().returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n\n let mut file = File::from_std(file);\n\n {\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n }\n\n pool::run_one();\n\n {\n let mut buf = [0; 4];\n let mut t = task::spawn(file.read(&mut buf));\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, 4);\n assert_eq!(&buf[..], &HELLO[..n]);\n }\n\n // Calling again immediately succeeds with the rest of the buffer\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, 10);\n assert_eq!(&buf[..n], &HELLO[4..]);\n\n assert_eq!(0, pool::len());\n}\n\n#[test]\nfn read_with_bigger_buf() {\n let mut seq = Sequence::new();\n let mut file = MockFile::default();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..4].copy_from_slice(&HELLO[..4]);\n Ok(4)\n });\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len() - 4].copy_from_slice(&HELLO[4..]);\n Ok(HELLO.len() - 4)\n });\n\n let mut file = File::from_std(file);\n\n {\n let mut buf = [0; 4];\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n }\n\n pool::run_one();\n\n {\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, 4);\n assert_eq!(&buf[..n], &HELLO[..n]);\n }\n\n // Calling again immediately succeeds with the rest of the buffer\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, 10);\n assert_eq!(&buf[..n], &HELLO[4..]);\n\n assert_eq!(0, pool::len());\n}\n\n#[test]\nfn read_err_then_read_success() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n\n let mut file = File::from_std(file);\n\n {\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n\n pool::run_one();\n\n assert_ready_err!(t.poll());\n }\n\n {\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let n = assert_ready_ok!(t.poll());\n\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n }\n}\n\n#[test]\nfn open_write() {\n let mut file = MockFile::default();\n file.expect_inner_write()\n .once()\n .with(eq(HELLO))\n .returning(|buf| Ok(buf.len()));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n\n assert_eq!(0, pool::len());\n assert_ready_ok!(t.poll());\n\n assert_eq!(1, pool::len());\n\n pool::run_one();\n\n assert!(!t.is_woken());\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn flush_while_idle() {\n let file = MockFile::default();\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // takes a really long time with miri\nfn read_with_buffer_larger_than_max() {\n // Chunks\n let chunk_a = crate::io::blocking::DEFAULT_MAX_BUF_SIZE;\n let chunk_b = chunk_a * 2;\n let chunk_c = chunk_a * 3;\n let chunk_d = chunk_a * 4;\n\n assert_eq!(chunk_d / 1024 / 1024, 8);\n\n let mut data = vec![];\n for i in 0..(chunk_d - 1) {\n data.push((i % 151) as u8);\n }\n let data = Arc::new(data);\n let d0 = data.clone();\n let d1 = data.clone();\n let d2 = data.clone();\n let d3 = data.clone();\n\n let mut seq = Sequence::new();\n let mut file = MockFile::default();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(move |buf| {\n buf[0..chunk_a].copy_from_slice(&d0[0..chunk_a]);\n Ok(chunk_a)\n });\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(move |buf| {\n buf[..chunk_a].copy_from_slice(&d1[chunk_a..chunk_b]);\n Ok(chunk_b - chunk_a)\n });\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(move |buf| {\n buf[..chunk_a].copy_from_slice(&d2[chunk_b..chunk_c]);\n Ok(chunk_c - chunk_b)\n });\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(move |buf| {\n buf[..chunk_a - 1].copy_from_slice(&d3[chunk_c..]);\n Ok(chunk_a - 1)\n });\n let mut file = File::from_std(file);\n\n let mut actual = vec![0; chunk_d];\n let mut pos = 0;\n\n while pos < data.len() {\n let mut t = task::spawn(file.read(&mut actual[pos..]));\n\n assert_pending!(t.poll());\n pool::run_one();\n assert!(t.is_woken());\n\n let n = assert_ready_ok!(t.poll());\n assert!(n <= chunk_a);\n\n pos += n;\n }\n\n assert_eq!(&data[..], &actual[..data.len()]);\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // takes a really long time with miri\nfn write_with_buffer_larger_than_max() {\n // Chunks\n let chunk_a = crate::io::blocking::DEFAULT_MAX_BUF_SIZE;\n let chunk_b = chunk_a * 2;\n let chunk_c = chunk_a * 3;\n let chunk_d = chunk_a * 4;\n\n assert_eq!(chunk_d / 1024 / 1024, 8);\n\n let mut data = vec![];\n for i in 0..(chunk_d - 1) {\n data.push((i % 151) as u8);\n }\n let data = Arc::new(data);\n let d0 = data.clone();\n let d1 = data.clone();\n let d2 = data.clone();\n let d3 = data.clone();\n\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .withf(move |buf| buf == &d0[0..chunk_a])\n .returning(|buf| Ok(buf.len()));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .withf(move |buf| buf == &d1[chunk_a..chunk_b])\n .returning(|buf| Ok(buf.len()));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .withf(move |buf| buf == &d2[chunk_b..chunk_c])\n .returning(|buf| Ok(buf.len()));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .withf(move |buf| buf == &d3[chunk_c..chunk_d - 1])\n .returning(|buf| Ok(buf.len()));\n\n let mut file = File::from_std(file);\n\n let mut rem = &data[..];\n\n let mut first = true;\n\n while !rem.is_empty() {\n let mut task = task::spawn(file.write(rem));\n\n if !first {\n assert_pending!(task.poll());\n pool::run_one();\n assert!(task.is_woken());\n }\n\n first = false;\n\n let n = assert_ready_ok!(task.poll());\n\n rem = &rem[n..];\n }\n\n pool::run_one();\n}\n\n#[test]\nfn write_twice_before_dispatch() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|buf| Ok(buf.len()));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(FOO))\n .returning(|buf| Ok(buf.len()));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.write(FOO));\n assert_pending!(t.poll());\n\n assert_eq!(pool::len(), 1);\n pool::run_one();\n\n assert!(t.is_woken());\n\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.flush());\n assert_pending!(t.poll());\n\n assert_eq!(pool::len(), 1);\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn incomplete_read_followed_by_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n file.expect_inner_seek()\n .once()\n .with(eq(SeekFrom::Current(-(HELLO.len() as i64))))\n .in_sequence(&mut seq)\n .returning(|_| Ok(0));\n file.expect_inner_write()\n .once()\n .with(eq(FOO))\n .returning(|_| Ok(FOO.len()));\n\n let mut file = File::from_std(file);\n\n let mut buf = [0; 32];\n\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_ok!(t.poll());\n\n assert_eq!(pool::len(), 1);\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn incomplete_partial_read_followed_by_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n file.expect_inner_seek()\n .once()\n .in_sequence(&mut seq)\n .with(eq(SeekFrom::Current(-10)))\n .returning(|_| Ok(0));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(FOO))\n .returning(|_| Ok(FOO.len()));\n\n let mut file = File::from_std(file);\n\n let mut buf = [0; 32];\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let mut buf = [0; 4];\n let mut t = task::spawn(file.read(&mut buf));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_ok!(t.poll());\n\n assert_eq!(pool::len(), 1);\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn incomplete_read_followed_by_flush() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n file.expect_inner_seek()\n .once()\n .in_sequence(&mut seq)\n .with(eq(SeekFrom::Current(-(HELLO.len() as i64))))\n .returning(|_| Ok(0));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(FOO))\n .returning(|_| Ok(FOO.len()));\n\n let mut file = File::from_std(file);\n\n let mut buf = [0; 32];\n\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n}\n\n#[test]\nfn incomplete_flush_followed_by_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|_| Ok(HELLO.len()));\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(FOO))\n .returning(|_| Ok(FOO.len()));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n let n = assert_ready_ok!(t.poll());\n assert_eq!(n, HELLO.len());\n\n let mut t = task::spawn(file.flush());\n assert_pending!(t.poll());\n\n // TODO: Move under write\n pool::run_one();\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn read_err() {\n let mut file = MockFile::default();\n file.expect_inner_read()\n .once()\n .returning(|_| Err(io::ErrorKind::Other.into()));\n\n let mut file = File::from_std(file);\n\n let mut buf = [0; 1024];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_pending!(t.poll());\n\n pool::run_one();\n assert!(t.is_woken());\n\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn write_write_err() {\n let mut file = MockFile::default();\n file.expect_inner_write()\n .once()\n .returning(|_| Err(io::ErrorKind::Other.into()));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn write_read_write_err() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n let mut buf = [0; 1024];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn write_read_flush_err() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n let mut buf = [0; 1024];\n let mut t = task::spawn(file.read(&mut buf));\n\n assert_pending!(t.poll());\n\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn write_seek_write_err() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n file.expect_inner_seek()\n .once()\n .with(eq(SeekFrom::Start(0)))\n .in_sequence(&mut seq)\n .returning(|_| Ok(0));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n {\n let mut t = task::spawn(file.seek(SeekFrom::Start(0)));\n assert_pending!(t.poll());\n }\n\n pool::run_one();\n\n let mut t = task::spawn(file.write(FOO));\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn write_seek_flush_err() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n file.expect_inner_seek()\n .once()\n .with(eq(SeekFrom::Start(0)))\n .in_sequence(&mut seq)\n .returning(|_| Ok(0));\n\n let mut file = File::from_std(file);\n\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n pool::run_one();\n\n {\n let mut t = task::spawn(file.seek(SeekFrom::Start(0)));\n assert_pending!(t.poll());\n }\n\n pool::run_one();\n\n let mut t = task::spawn(file.flush());\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn sync_all_ordered_after_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|_| Ok(HELLO.len()));\n file.expect_sync_all().once().returning(|| Ok(()));\n\n let mut file = File::from_std(file);\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.sync_all());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn sync_all_err_ordered_after_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|_| Ok(HELLO.len()));\n file.expect_sync_all()\n .once()\n .returning(|| Err(io::ErrorKind::Other.into()));\n\n let mut file = File::from_std(file);\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.sync_all());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn sync_data_ordered_after_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|_| Ok(HELLO.len()));\n file.expect_sync_data().once().returning(|| Ok(()));\n\n let mut file = File::from_std(file);\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.sync_data());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn sync_data_err_ordered_after_write() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .with(eq(HELLO))\n .returning(|_| Ok(HELLO.len()));\n file.expect_sync_data()\n .once()\n .returning(|| Err(io::ErrorKind::Other.into()));\n\n let mut file = File::from_std(file);\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n\n let mut t = task::spawn(file.sync_data());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_pending!(t.poll());\n\n assert_eq!(1, pool::len());\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn open_set_len_ok() {\n let mut file = MockFile::default();\n file.expect_set_len().with(eq(123)).returning(|_| Ok(()));\n\n let file = File::from_std(file);\n let mut t = task::spawn(file.set_len(123));\n\n assert_pending!(t.poll());\n\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_ok!(t.poll());\n}\n\n#[test]\nfn open_set_len_err() {\n let mut file = MockFile::default();\n file.expect_set_len()\n .with(eq(123))\n .returning(|_| Err(io::ErrorKind::Other.into()));\n\n let file = File::from_std(file);\n let mut t = task::spawn(file.set_len(123));\n\n assert_pending!(t.poll());\n\n pool::run_one();\n\n assert!(t.is_woken());\n assert_ready_err!(t.poll());\n}\n\n#[test]\nfn partial_read_set_len_ok() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..HELLO.len()].copy_from_slice(HELLO);\n Ok(HELLO.len())\n });\n file.expect_inner_seek()\n .once()\n .with(eq(SeekFrom::Current(-(HELLO.len() as i64))))\n .in_sequence(&mut seq)\n .returning(|_| Ok(0));\n file.expect_set_len()\n .once()\n .in_sequence(&mut seq)\n .with(eq(123))\n .returning(|_| Ok(()));\n file.expect_inner_read()\n .once()\n .in_sequence(&mut seq)\n .returning(|buf| {\n buf[0..FOO.len()].copy_from_slice(FOO);\n Ok(FOO.len())\n });\n\n let mut buf = [0; 32];\n let mut file = File::from_std(file);\n\n {\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n }\n\n pool::run_one();\n\n {\n let mut t = task::spawn(file.set_len(123));\n\n assert_pending!(t.poll());\n pool::run_one();\n assert_ready_ok!(t.poll());\n }\n\n let mut t = task::spawn(file.read(&mut buf));\n assert_pending!(t.poll());\n pool::run_one();\n let n = assert_ready_ok!(t.poll());\n\n assert_eq!(n, FOO.len());\n assert_eq!(&buf[..n], FOO);\n}\n\n#[test]\nfn busy_file_seek_error() {\n let mut file = MockFile::default();\n let mut seq = Sequence::new();\n file.expect_inner_write()\n .once()\n .in_sequence(&mut seq)\n .returning(|_| Err(io::ErrorKind::Other.into()));\n\n let mut file = crate::io::BufReader::new(File::from_std(file));\n {\n let mut t = task::spawn(file.write(HELLO));\n assert_ready_ok!(t.poll());\n }\n\n pool::run_one();\n\n let mut t = task::spawn(file.seek(SeekFrom::Start(0)));\n assert_ready_err!(t.poll());\n}\n" }, { "path": "tokio/src/fs/file.rs", "content": "//! Types for working with [`File`].\n//!\n//! [`File`]: File\n\nuse crate::fs::{asyncify, OpenOptions};\nuse crate::io::blocking::{Buf, DEFAULT_MAX_BUF_SIZE};\nuse crate::io::{AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};\nuse crate::sync::Mutex;\n\nuse std::cmp;\nuse std::fmt;\nuse std::fs::{Metadata, Permissions};\nuse std::future::Future;\nuse std::io::{self, Seek, SeekFrom};\nuse std::path::Path;\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{ready, Context, Poll};\n\n#[cfg(test)]\nuse super::mocks::JoinHandle;\n#[cfg(test)]\nuse super::mocks::MockFile as StdFile;\n#[cfg(test)]\nuse super::mocks::{spawn_blocking, spawn_mandatory_blocking};\n#[cfg(not(test))]\nuse crate::blocking::JoinHandle;\n#[cfg(not(test))]\nuse crate::blocking::{spawn_blocking, spawn_mandatory_blocking};\n#[cfg(not(test))]\nuse std::fs::File as StdFile;\n\n/// A reference to an open file on the filesystem.\n///\n/// This is a specialized version of [`std::fs::File`] for usage from the\n/// Tokio runtime.\n///\n/// An instance of a `File` can be read and/or written depending on what options\n/// it was opened with. Files also implement [`AsyncSeek`] to alter the logical\n/// cursor that the file contains internally.\n///\n/// A file will not be closed immediately when it goes out of scope if there\n/// are any IO operations that have not yet completed. To ensure that a file is\n/// closed immediately when it is dropped, you should call [`flush`] before\n/// dropping it. Note that this does not ensure that the file has been fully\n/// written to disk; the operating system might keep the changes around in an\n/// in-memory buffer. See the [`sync_all`] method for telling the OS to write\n/// the data to disk.\n///\n/// Reading and writing to a `File` is usually done using the convenience\n/// methods found on the [`AsyncReadExt`] and [`AsyncWriteExt`] traits.\n///\n/// [`AsyncSeek`]: trait@crate::io::AsyncSeek\n/// [`flush`]: fn@crate::io::AsyncWriteExt::flush\n/// [`sync_all`]: fn@crate::fs::File::sync_all\n/// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n///\n/// # Examples\n///\n/// Create a new file and asynchronously write bytes to it:\n///\n/// ```no_run\n/// use tokio::fs::File;\n/// use tokio::io::AsyncWriteExt; // for write_all()\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// let mut file = File::create(\"foo.txt\").await?;\n/// file.write_all(b\"hello, world!\").await?;\n/// # Ok(())\n/// # }\n/// ```\n///\n/// Read the contents of a file into a buffer:\n///\n/// ```no_run\n/// use tokio::fs::File;\n/// use tokio::io::AsyncReadExt; // for read_to_end()\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// let mut file = File::open(\"foo.txt\").await?;\n///\n/// let mut contents = vec![];\n/// file.read_to_end(&mut contents).await?;\n///\n/// println!(\"len = {}\", contents.len());\n/// # Ok(())\n/// # }\n/// ```\npub struct File {\n std: Arc,\n inner: Mutex,\n max_buf_size: usize,\n}\n\nstruct Inner {\n state: State,\n\n /// Errors from writes/flushes are returned in write/flush calls. If a write\n /// error is observed while performing a read, it is saved until the next\n /// write / flush call.\n last_write_err: Option,\n\n pos: u64,\n}\n\n#[derive(Debug)]\nenum State {\n Idle(Option),\n Busy(JoinHandle<(Operation, Buf)>),\n}\n\n#[derive(Debug)]\nenum Operation {\n Read(io::Result),\n Write(io::Result<()>),\n Seek(io::Result),\n}\n\nimpl File {\n /// Attempts to open a file in read-only mode.\n ///\n /// See [`OpenOptions`] for more details.\n ///\n /// # Errors\n ///\n /// This function will return an error if called from outside of the Tokio\n /// runtime or if path does not already exist. Other errors may also be\n /// returned according to `OpenOptions::open`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncReadExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::open(\"foo.txt\").await?;\n ///\n /// let mut contents = vec![];\n /// file.read_to_end(&mut contents).await?;\n ///\n /// println!(\"len = {}\", contents.len());\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`read_to_end`] method is defined on the [`AsyncReadExt`] trait.\n ///\n /// [`read_to_end`]: fn@crate::io::AsyncReadExt::read_to_end\n /// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n pub async fn open(path: impl AsRef) -> io::Result {\n Self::options().read(true).open(path).await\n }\n\n /// Opens a file in write-only mode.\n ///\n /// This function will create a file if it does not exist, and will truncate\n /// it if it does.\n ///\n /// See [`OpenOptions`] for more details.\n ///\n /// # Errors\n ///\n /// Results in an error if called from outside of the Tokio runtime or if\n /// the underlying [`create`] call results in an error.\n ///\n /// [`create`]: std::fs::File::create\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// file.write_all(b\"hello, world!\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn create(path: impl AsRef) -> io::Result {\n Self::options()\n .write(true)\n .create(true)\n .truncate(true)\n .open(path)\n .await\n }\n\n /// Opens a file in read-write mode.\n ///\n /// This function will create a file if it does not exist, or return an error\n /// if it does. This way, if the call succeeds, the file returned is guaranteed\n /// to be new.\n ///\n /// This option is useful because it is atomic. Otherwise between checking\n /// whether a file exists and creating a new one, the file may have been\n /// created by another process (a TOCTOU race condition / attack).\n ///\n /// This can also be written using `File::options().read(true).write(true).create_new(true).open(...)`.\n ///\n /// See [`OpenOptions`] for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::create_new(\"foo.txt\").await?;\n /// file.write_all(b\"hello, world!\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn create_new>(path: P) -> std::io::Result {\n Self::options()\n .read(true)\n .write(true)\n .create_new(true)\n .open(path)\n .await\n }\n\n /// Returns a new [`OpenOptions`] object.\n ///\n /// This function returns a new `OpenOptions` object that you can use to\n /// open or create a file with specific options if `open()` or `create()`\n /// are not appropriate.\n ///\n /// It is equivalent to `OpenOptions::new()`, but allows you to write more\n /// readable code. Instead of\n /// `OpenOptions::new().append(true).open(\"example.log\")`,\n /// you can write `File::options().append(true).open(\"example.log\")`. This\n /// also avoids the need to import `OpenOptions`.\n ///\n /// See the [`OpenOptions::new`] function for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut f = File::options().append(true).open(\"example.log\").await?;\n /// f.write_all(b\"new line\\n\").await?;\n /// # Ok(())\n /// # }\n /// ```\n #[must_use]\n pub fn options() -> OpenOptions {\n OpenOptions::new()\n }\n\n /// Converts a [`std::fs::File`] to a [`tokio::fs::File`](File).\n ///\n /// # Examples\n ///\n /// ```no_run\n /// // This line could block. It is not recommended to do this on the Tokio\n /// // runtime.\n /// let std_file = std::fs::File::open(\"foo.txt\").unwrap();\n /// let file = tokio::fs::File::from_std(std_file);\n /// ```\n pub fn from_std(std: StdFile) -> File {\n File {\n std: Arc::new(std),\n inner: Mutex::new(Inner {\n state: State::Idle(Some(Buf::with_capacity(0))),\n last_write_err: None,\n pos: 0,\n }),\n max_buf_size: DEFAULT_MAX_BUF_SIZE,\n }\n }\n\n /// Attempts to sync all OS-internal metadata to disk.\n ///\n /// This function will attempt to ensure that all in-core data reaches the\n /// filesystem before returning.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// file.write_all(b\"hello, world!\").await?;\n /// file.sync_all().await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn sync_all(&self) -> io::Result<()> {\n let mut inner = self.inner.lock().await;\n inner.complete_inflight().await;\n\n let std = self.std.clone();\n asyncify(move || std.sync_all()).await\n }\n\n /// This function is similar to `sync_all`, except that it may not\n /// synchronize file metadata to the filesystem.\n ///\n /// This is intended for use cases that must synchronize content, but don't\n /// need the metadata on disk. The goal of this method is to reduce disk\n /// operations.\n ///\n /// Note that some platforms may simply implement this in terms of `sync_all`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// file.write_all(b\"hello, world!\").await?;\n /// file.sync_data().await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn sync_data(&self) -> io::Result<()> {\n let mut inner = self.inner.lock().await;\n inner.complete_inflight().await;\n\n let std = self.std.clone();\n asyncify(move || std.sync_data()).await\n }\n\n /// Truncates or extends the underlying file, updating the size of this file to become size.\n ///\n /// If the size is less than the current file's size, then the file will be\n /// shrunk. If it is greater than the current file's size, then the file\n /// will be extended to size and have all of the intermediate data filled in\n /// with 0s.\n ///\n /// # Errors\n ///\n /// This function will return an error if the file is not opened for\n /// writing.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// file.write_all(b\"hello, world!\").await?;\n /// file.set_len(10).await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn set_len(&self, size: u64) -> io::Result<()> {\n let mut inner = self.inner.lock().await;\n inner.complete_inflight().await;\n\n let mut buf = match inner.state {\n State::Idle(ref mut buf_cell) => buf_cell.take().unwrap(),\n _ => unreachable!(),\n };\n\n let seek = if !buf.is_empty() {\n Some(SeekFrom::Current(buf.discard_read()))\n } else {\n None\n };\n\n let std = self.std.clone();\n\n inner.state = State::Busy(spawn_blocking(move || {\n let res = if let Some(seek) = seek {\n (&*std).seek(seek).and_then(|_| std.set_len(size))\n } else {\n std.set_len(size)\n }\n .map(|()| 0); // the value is discarded later\n\n // Return the result as a seek\n (Operation::Seek(res), buf)\n }));\n\n let (op, buf) = match inner.state {\n State::Idle(_) => unreachable!(),\n State::Busy(ref mut rx) => rx.await?,\n };\n\n inner.state = State::Idle(Some(buf));\n\n match op {\n Operation::Seek(res) => res.map(|pos| {\n inner.pos = pos;\n }),\n _ => unreachable!(),\n }\n }\n\n /// Queries metadata about the underlying file.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let file = File::open(\"foo.txt\").await?;\n /// let metadata = file.metadata().await?;\n ///\n /// println!(\"{:?}\", metadata);\n /// # Ok(())\n /// # }\n /// ```\n pub async fn metadata(&self) -> io::Result {\n let std = self.std.clone();\n asyncify(move || std.metadata()).await\n }\n\n /// Creates a new `File` instance that shares the same underlying file handle\n /// as the existing `File` instance. Reads, writes, and seeks will affect both\n /// File instances simultaneously.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let file = File::open(\"foo.txt\").await?;\n /// let file_clone = file.try_clone().await?;\n /// # Ok(())\n /// # }\n /// ```\n pub async fn try_clone(&self) -> io::Result {\n self.inner.lock().await.complete_inflight().await;\n let std = self.std.clone();\n let std_file = asyncify(move || std.try_clone()).await?;\n let mut file = File::from_std(std_file);\n file.set_max_buf_size(self.max_buf_size);\n Ok(file)\n }\n\n /// Destructures `File` into a [`std::fs::File`]. This function is\n /// async to allow any in-flight operations to complete.\n ///\n /// Use `File::try_into_std` to attempt conversion immediately.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let tokio_file = File::open(\"foo.txt\").await?;\n /// let std_file = tokio_file.into_std().await;\n /// # Ok(())\n /// # }\n /// ```\n pub async fn into_std(mut self) -> StdFile {\n self.inner.get_mut().complete_inflight().await;\n Arc::try_unwrap(self.std).expect(\"Arc::try_unwrap failed\")\n }\n\n /// Tries to immediately destructure `File` into a [`std::fs::File`].\n ///\n /// # Errors\n ///\n /// This function will return an error containing the file if some\n /// operation is in-flight.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let tokio_file = File::open(\"foo.txt\").await?;\n /// let std_file = tokio_file.try_into_std().unwrap();\n /// # Ok(())\n /// # }\n /// ```\n #[allow(clippy::result_large_err)]\n pub fn try_into_std(mut self) -> Result {\n match Arc::try_unwrap(self.std) {\n Ok(file) => Ok(file),\n Err(std_file_arc) => {\n self.std = std_file_arc;\n Err(self)\n }\n }\n }\n\n /// Changes the permissions on the underlying file.\n ///\n /// # Platform-specific behavior\n ///\n /// This function currently corresponds to the `fchmod` function on Unix and\n /// the `SetFileInformationByHandle` function on Windows. Note that, this\n /// [may change in the future][changes].\n ///\n /// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n ///\n /// # Errors\n ///\n /// This function will return an error if the user lacks permission change\n /// attributes on the underlying file. It may also return an error in other\n /// os-specific unspecified cases.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let file = File::open(\"foo.txt\").await?;\n /// let mut perms = file.metadata().await?.permissions();\n /// perms.set_readonly(true);\n /// file.set_permissions(perms).await?;\n /// # Ok(())\n /// # }\n /// ```\n pub async fn set_permissions(&self, perm: Permissions) -> io::Result<()> {\n let std = self.std.clone();\n asyncify(move || std.set_permissions(perm)).await\n }\n\n /// Set the maximum buffer size for the underlying [`AsyncRead`] / [`AsyncWrite`] operation.\n ///\n /// Although Tokio uses a sensible default value for this buffer size, this function would be\n /// useful for changing that default depending on the situation.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::File;\n /// use tokio::io::AsyncWriteExt;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::open(\"foo.txt\").await?;\n ///\n /// // Set maximum buffer size to 8 MiB\n /// file.set_max_buf_size(8 * 1024 * 1024);\n ///\n /// let mut buf = vec![1; 1024 * 1024 * 1024];\n ///\n /// // Write the 1 GiB buffer in chunks up to 8 MiB each.\n /// file.write_all(&mut buf).await?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_max_buf_size(&mut self, max_buf_size: usize) {\n self.max_buf_size = max_buf_size;\n }\n\n /// Get the maximum buffer size for the underlying [`AsyncRead`] / [`AsyncWrite`] operation.\n pub fn max_buf_size(&self) -> usize {\n self.max_buf_size\n }\n}\n\nimpl AsyncRead for File {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n dst: &mut ReadBuf<'_>,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n\n let me = self.get_mut();\n let inner = me.inner.get_mut();\n\n loop {\n match inner.state {\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n if !buf.is_empty() || dst.remaining() == 0 {\n buf.copy_to(dst);\n *buf_cell = Some(buf);\n return Poll::Ready(Ok(()));\n }\n\n let std = me.std.clone();\n\n let max_buf_size = cmp::min(dst.remaining(), me.max_buf_size);\n inner.state = State::Busy(spawn_blocking(move || {\n // SAFETY: the `Read` implementation of `std` does not\n // read from the buffer it is borrowing and correctly\n // reports the length of the data written into the buffer.\n let res = unsafe { buf.read_from(&mut &*std, max_buf_size) };\n (Operation::Read(res), buf)\n }));\n }\n State::Busy(ref mut rx) => {\n let (op, mut buf) = ready!(Pin::new(rx).poll(cx))?;\n\n match op {\n Operation::Read(Ok(_)) => {\n buf.copy_to(dst);\n inner.state = State::Idle(Some(buf));\n return Poll::Ready(Ok(()));\n }\n Operation::Read(Err(e)) => {\n assert!(buf.is_empty());\n\n inner.state = State::Idle(Some(buf));\n return Poll::Ready(Err(e));\n }\n Operation::Write(Ok(())) => {\n assert!(buf.is_empty());\n inner.state = State::Idle(Some(buf));\n continue;\n }\n Operation::Write(Err(e)) => {\n assert!(inner.last_write_err.is_none());\n inner.last_write_err = Some(e.kind());\n inner.state = State::Idle(Some(buf));\n }\n Operation::Seek(result) => {\n assert!(buf.is_empty());\n inner.state = State::Idle(Some(buf));\n if let Ok(pos) = result {\n inner.pos = pos;\n }\n continue;\n }\n }\n }\n }\n }\n }\n}\n\nimpl AsyncSeek for File {\n fn start_seek(self: Pin<&mut Self>, mut pos: SeekFrom) -> io::Result<()> {\n let me = self.get_mut();\n let inner = me.inner.get_mut();\n\n match inner.state {\n State::Busy(_) => Err(io::Error::new(\n io::ErrorKind::Other,\n \"other file operation is pending, call poll_complete before start_seek\",\n )),\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n // Factor in any unread data from the buf\n if !buf.is_empty() {\n let n = buf.discard_read();\n\n if let SeekFrom::Current(ref mut offset) = pos {\n *offset += n;\n }\n }\n\n let std = me.std.clone();\n\n inner.state = State::Busy(spawn_blocking(move || {\n let res = (&*std).seek(pos);\n (Operation::Seek(res), buf)\n }));\n Ok(())\n }\n }\n }\n\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let inner = self.inner.get_mut();\n\n loop {\n match inner.state {\n State::Idle(_) => return Poll::Ready(Ok(inner.pos)),\n State::Busy(ref mut rx) => {\n let (op, buf) = ready!(Pin::new(rx).poll(cx))?;\n inner.state = State::Idle(Some(buf));\n\n match op {\n Operation::Read(_) => {}\n Operation::Write(Err(e)) => {\n assert!(inner.last_write_err.is_none());\n inner.last_write_err = Some(e.kind());\n }\n Operation::Write(_) => {}\n Operation::Seek(res) => {\n if let Ok(pos) = res {\n inner.pos = pos;\n }\n return Poll::Ready(res);\n }\n }\n }\n }\n }\n }\n}\n\nimpl AsyncWrite for File {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n src: &[u8],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let me = self.get_mut();\n let inner = me.inner.get_mut();\n\n if let Some(e) = inner.last_write_err.take() {\n return Poll::Ready(Err(e.into()));\n }\n\n loop {\n match inner.state {\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n let seek = if !buf.is_empty() {\n Some(SeekFrom::Current(buf.discard_read()))\n } else {\n None\n };\n\n let n = buf.copy_from(src, me.max_buf_size);\n let std = me.std.clone();\n\n let blocking_task_join_handle = spawn_mandatory_blocking(move || {\n let res = if let Some(seek) = seek {\n (&*std).seek(seek).and_then(|_| buf.write_to(&mut &*std))\n } else {\n buf.write_to(&mut &*std)\n };\n\n (Operation::Write(res), buf)\n })\n .ok_or_else(|| {\n io::Error::new(io::ErrorKind::Other, \"background task failed\")\n })?;\n\n inner.state = State::Busy(blocking_task_join_handle);\n\n return Poll::Ready(Ok(n));\n }\n State::Busy(ref mut rx) => {\n let (op, buf) = ready!(Pin::new(rx).poll(cx))?;\n inner.state = State::Idle(Some(buf));\n\n match op {\n Operation::Read(_) => {\n // We don't care about the result here. The fact\n // that the cursor has advanced will be reflected in\n // the next iteration of the loop\n continue;\n }\n Operation::Write(res) => {\n // If the previous write was successful, continue.\n // Otherwise, error.\n res?;\n continue;\n }\n Operation::Seek(_) => {\n // Ignore the seek\n continue;\n }\n }\n }\n }\n }\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let me = self.get_mut();\n let inner = me.inner.get_mut();\n\n if let Some(e) = inner.last_write_err.take() {\n return Poll::Ready(Err(e.into()));\n }\n\n loop {\n match inner.state {\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n let seek = if !buf.is_empty() {\n Some(SeekFrom::Current(buf.discard_read()))\n } else {\n None\n };\n\n let n = buf.copy_from_bufs(bufs, me.max_buf_size);\n let std = me.std.clone();\n\n let blocking_task_join_handle = spawn_mandatory_blocking(move || {\n let res = if let Some(seek) = seek {\n (&*std).seek(seek).and_then(|_| buf.write_to(&mut &*std))\n } else {\n buf.write_to(&mut &*std)\n };\n\n (Operation::Write(res), buf)\n })\n .ok_or_else(|| {\n io::Error::new(io::ErrorKind::Other, \"background task failed\")\n })?;\n\n inner.state = State::Busy(blocking_task_join_handle);\n\n return Poll::Ready(Ok(n));\n }\n State::Busy(ref mut rx) => {\n let (op, buf) = ready!(Pin::new(rx).poll(cx))?;\n inner.state = State::Idle(Some(buf));\n\n match op {\n Operation::Read(_) => {\n // We don't care about the result here. The fact\n // that the cursor has advanced will be reflected in\n // the next iteration of the loop\n continue;\n }\n Operation::Write(res) => {\n // If the previous write was successful, continue.\n // Otherwise, error.\n res?;\n continue;\n }\n Operation::Seek(_) => {\n // Ignore the seek\n continue;\n }\n }\n }\n }\n }\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let inner = self.inner.get_mut();\n inner.poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n self.poll_flush(cx)\n }\n}\n\nimpl From for File {\n fn from(std: StdFile) -> Self {\n Self::from_std(std)\n }\n}\n\nimpl fmt::Debug for File {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"tokio::fs::File\")\n .field(\"std\", &self.std)\n .finish()\n }\n}\n\n#[cfg(unix)]\nimpl std::os::unix::io::AsRawFd for File {\n fn as_raw_fd(&self) -> std::os::unix::io::RawFd {\n self.std.as_raw_fd()\n }\n}\n\n#[cfg(unix)]\nimpl std::os::unix::io::AsFd for File {\n fn as_fd(&self) -> std::os::unix::io::BorrowedFd<'_> {\n unsafe {\n std::os::unix::io::BorrowedFd::borrow_raw(std::os::unix::io::AsRawFd::as_raw_fd(self))\n }\n }\n}\n\n#[cfg(unix)]\nimpl std::os::unix::io::FromRawFd for File {\n unsafe fn from_raw_fd(fd: std::os::unix::io::RawFd) -> Self {\n // Safety: exactly the same safety contract as\n // `std::os::unix::io::FromRawFd::from_raw_fd`.\n unsafe { StdFile::from_raw_fd(fd).into() }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawHandle, FromRawHandle, RawHandle, AsHandle, BorrowedHandle};\n\n impl AsRawHandle for File {\n fn as_raw_handle(&self) -> RawHandle {\n self.std.as_raw_handle()\n }\n }\n\n impl AsHandle for File {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe {\n BorrowedHandle::borrow_raw(\n AsRawHandle::as_raw_handle(self),\n )\n }\n }\n }\n\n impl FromRawHandle for File {\n unsafe fn from_raw_handle(handle: RawHandle) -> Self {\n // Safety: exactly the same safety contract as\n // `FromRawHandle::from_raw_handle`.\n unsafe { StdFile::from_raw_handle(handle).into() }\n }\n }\n}\n\nimpl Inner {\n async fn complete_inflight(&mut self) {\n use std::future::poll_fn;\n\n poll_fn(|cx| self.poll_complete_inflight(cx)).await;\n }\n\n fn poll_complete_inflight(&mut self, cx: &mut Context<'_>) -> Poll<()> {\n ready!(crate::trace::trace_leaf(cx));\n match self.poll_flush(cx) {\n Poll::Ready(Err(e)) => {\n self.last_write_err = Some(e.kind());\n Poll::Ready(())\n }\n Poll::Ready(Ok(())) => Poll::Ready(()),\n Poll::Pending => Poll::Pending,\n }\n }\n\n fn poll_flush(&mut self, cx: &mut Context<'_>) -> Poll> {\n if let Some(e) = self.last_write_err.take() {\n return Poll::Ready(Err(e.into()));\n }\n\n let (op, buf) = match self.state {\n State::Idle(_) => return Poll::Ready(Ok(())),\n State::Busy(ref mut rx) => ready!(Pin::new(rx).poll(cx))?,\n };\n\n // The buffer is not used here\n self.state = State::Idle(Some(buf));\n\n match op {\n Operation::Read(_) => Poll::Ready(Ok(())),\n Operation::Write(res) => Poll::Ready(res),\n Operation::Seek(_) => Poll::Ready(Ok(())),\n }\n }\n}\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/fs/hard_link.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Creates a new hard link on the filesystem.\n///\n/// This is an async version of [`std::fs::hard_link`].\n///\n/// The `link` path will be a link pointing to the `original` path. Note that systems\n/// often require these two paths to both be located on the same filesystem.\n///\n/// # Platform-specific behavior\n///\n/// This function currently corresponds to the `link` function on Unix\n/// and the `CreateHardLink` function on Windows.\n/// Note that, this [may change in the future][changes].\n///\n/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n///\n/// # Errors\n///\n/// This function will return an error in the following situations, but is not\n/// limited to just these cases:\n///\n/// * The `original` path is not a file or doesn't exist.\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// #[tokio::main]\n/// async fn main() -> std::io::Result<()> {\n/// fs::hard_link(\"a.txt\", \"b.txt\").await?; // Hard link a.txt to b.txt\n/// Ok(())\n/// }\n/// ```\npub async fn hard_link(original: impl AsRef, link: impl AsRef) -> io::Result<()> {\n let original = original.as_ref().to_owned();\n let link = link.as_ref().to_owned();\n\n asyncify(move || std::fs::hard_link(original, link)).await\n}\n" }, { "path": "tokio/src/fs/metadata.rs", "content": "use crate::fs::asyncify;\n\nuse std::fs::Metadata;\nuse std::io;\nuse std::path::Path;\n\n/// Given a path, queries the file system to get information about a file,\n/// directory, etc.\n///\n/// This is an async version of [`std::fs::metadata`].\n///\n/// This function will traverse symbolic links to query information about the\n/// destination file.\n///\n/// # Platform-specific behavior\n///\n/// This function currently corresponds to the `stat` function on Unix and the\n/// `GetFileAttributesEx` function on Windows. Note that, this [may change in\n/// the future][changes].\n///\n/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior\n///\n/// # Errors\n///\n/// This function will return an error in the following situations, but is not\n/// limited to just these cases:\n///\n/// * The user lacks permissions to perform `metadata` call on `path`.\n/// * `path` does not exist.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::fs;\n///\n/// #[tokio::main]\n/// async fn main() -> std::io::Result<()> {\n/// let attr = fs::metadata(\"/some/file/path.txt\").await?;\n/// // inspect attr ...\n/// Ok(())\n/// }\n/// ```\npub async fn metadata(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(|| std::fs::metadata(path)).await\n}\n" }, { "path": "tokio/src/fs/mocks.rs", "content": "//! Mock version of std::fs::File;\nuse mockall::mock;\n\nuse crate::sync::oneshot;\n#[cfg(all(test, unix))]\nuse std::os::fd::{AsRawFd, FromRawFd, OwnedFd};\nuse std::{\n cell::RefCell,\n collections::VecDeque,\n fs::{Metadata, Permissions},\n future::Future,\n io::{self, Read, Seek, SeekFrom, Write},\n path::PathBuf,\n pin::Pin,\n task::{Context, Poll},\n};\n\nmock! {\n #[derive(Debug)]\n pub File {\n pub fn create(pb: PathBuf) -> io::Result;\n // These inner_ methods exist because std::fs::File has two\n // implementations for each of these methods: one on \"&mut self\" and\n // one on \"&&self\". Defining both of those in terms of an inner_ method\n // allows us to specify the expectation the same way, regardless of\n // which method is used.\n pub fn inner_flush(&self) -> io::Result<()>;\n pub fn inner_read(&self, dst: &mut [u8]) -> io::Result;\n pub fn inner_seek(&self, pos: SeekFrom) -> io::Result;\n pub fn inner_write(&self, src: &[u8]) -> io::Result;\n pub fn metadata(&self) -> io::Result;\n pub fn open(pb: PathBuf) -> io::Result;\n pub fn set_len(&self, size: u64) -> io::Result<()>;\n pub fn set_permissions(&self, _perm: Permissions) -> io::Result<()>;\n pub fn set_max_buf_size(&self, max_buf_size: usize);\n pub fn sync_all(&self) -> io::Result<()>;\n pub fn sync_data(&self) -> io::Result<()>;\n pub fn try_clone(&self) -> io::Result;\n }\n #[cfg(windows)]\n impl std::os::windows::io::AsRawHandle for File {\n fn as_raw_handle(&self) -> std::os::windows::io::RawHandle;\n }\n #[cfg(windows)]\n impl std::os::windows::io::FromRawHandle for File {\n unsafe fn from_raw_handle(h: std::os::windows::io::RawHandle) -> Self;\n }\n #[cfg(unix)]\n impl std::os::unix::io::AsRawFd for File {\n fn as_raw_fd(&self) -> std::os::unix::io::RawFd;\n }\n\n #[cfg(unix)]\n impl std::os::unix::io::FromRawFd for File {\n unsafe fn from_raw_fd(h: std::os::unix::io::RawFd) -> Self;\n }\n}\n\nimpl Read for MockFile {\n fn read(&mut self, dst: &mut [u8]) -> io::Result {\n // Placate Miri. Tokio will call this method with an uninitialized\n // buffer, which is ok because std::io::Read::read implementations don't usually read\n // from their input buffers. But Mockall 0.12-0.13 will try to Debug::fmt the\n // buffer, even if there is no failure, triggering an uninitialized data access alert from\n // Miri. Initialize the data here just to prevent those Miri alerts.\n // This can be removed after upgrading to Mockall 0.14.\n dst.fill(0);\n self.inner_read(dst)\n }\n}\n\nimpl Read for &'_ MockFile {\n fn read(&mut self, dst: &mut [u8]) -> io::Result {\n // Placate Miri. Tokio will call this method with an uninitialized\n // buffer, which is ok because std::io::Read::read implementations don't usually read\n // from their input buffers. But Mockall 0.12-0.13 will try to Debug::fmt the\n // buffer, even if there is no failure, triggering an uninitialized data access alert from\n // Miri. Initialize the data here just to prevent those Miri alerts.\n // This can be removed after upgrading to Mockall 0.14.\n dst.fill(0);\n self.inner_read(dst)\n }\n}\n\nimpl Seek for &'_ MockFile {\n fn seek(&mut self, pos: SeekFrom) -> io::Result {\n self.inner_seek(pos)\n }\n}\n\nimpl Write for &'_ MockFile {\n fn write(&mut self, src: &[u8]) -> io::Result {\n self.inner_write(src)\n }\n\n fn flush(&mut self) -> io::Result<()> {\n self.inner_flush()\n }\n}\n\n#[cfg(all(test, unix))]\nimpl From for OwnedFd {\n #[inline]\n fn from(file: MockFile) -> OwnedFd {\n unsafe { OwnedFd::from_raw_fd(file.as_raw_fd()) }\n }\n}\n\ntokio_thread_local! {\n static QUEUE: RefCell>> = RefCell::new(VecDeque::new())\n}\n\n#[derive(Debug)]\npub(super) struct JoinHandle {\n rx: oneshot::Receiver,\n}\n\npub(super) fn spawn_blocking(f: F) -> JoinHandle\nwhere\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n{\n let (tx, rx) = oneshot::channel();\n let task = Box::new(move || {\n let _ = tx.send(f());\n });\n\n QUEUE.with(|cell| cell.borrow_mut().push_back(task));\n\n JoinHandle { rx }\n}\n\npub(super) fn spawn_mandatory_blocking(f: F) -> Option>\nwhere\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n{\n let (tx, rx) = oneshot::channel();\n let task = Box::new(move || {\n let _ = tx.send(f());\n });\n\n QUEUE.with(|cell| cell.borrow_mut().push_back(task));\n\n Some(JoinHandle { rx })\n}\n\nimpl Future for JoinHandle {\n type Output = Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n use std::task::Poll;\n\n match Pin::new(&mut self.rx).poll(cx) {\n Poll::Ready(Ok(v)) => Poll::Ready(Ok(v)),\n Poll::Ready(Err(e)) => panic!(\"error = {e:?}\"),\n Poll::Pending => Poll::Pending,\n }\n }\n}\n\npub(super) mod pool {\n use super::*;\n\n pub(in super::super) fn len() -> usize {\n QUEUE.with(|cell| cell.borrow().len())\n }\n\n pub(in super::super) fn run_one() {\n let task = QUEUE\n .with(|cell| cell.borrow_mut().pop_front())\n .expect(\"expected task to run, but none ready\");\n\n task();\n }\n}\n" }, { "path": "tokio/src/fs/mod.rs", "content": "#![cfg(not(loom))]\n\n//! Asynchronous file utilities.\n//!\n//! This module contains utility methods for working with the file system\n//! asynchronously. This includes reading/writing to files, and working with\n//! directories.\n//!\n//! Be aware that most operating systems do not provide asynchronous file system\n//! APIs. Because of that, Tokio will use ordinary blocking file operations\n//! behind the scenes. This is done using the [`spawn_blocking`] threadpool to\n//! run them in the background.\n//!\n//! The `tokio::fs` module should only be used for ordinary files. Trying to use\n//! it with e.g., a named pipe on Linux can result in surprising behavior,\n//! such as hangs during runtime shutdown. For special files, you should use a\n//! dedicated type such as [`tokio::net::unix::pipe`] or [`AsyncFd`] instead.\n//!\n//! Currently, Tokio will always use [`spawn_blocking`] on all platforms, but it\n//! may be changed to use asynchronous file system APIs such as io_uring in the\n//! future.\n//!\n//! # Usage\n//!\n//! The easiest way to use this module is to use the utility functions that\n//! operate on entire files:\n//!\n//! * [`tokio::fs::read`](fn@crate::fs::read)\n//! * [`tokio::fs::read_to_string`](fn@crate::fs::read_to_string)\n//! * [`tokio::fs::write`](fn@crate::fs::write)\n//!\n//! The two `read` functions reads the entire file and returns its contents.\n//! The `write` function takes the contents of the file and writes those\n//! contents to the file. It overwrites the existing file, if any.\n//!\n//! For example, to read the file:\n//!\n//! ```\n//! # async fn dox() -> std::io::Result<()> {\n//! let contents = tokio::fs::read_to_string(\"my_file.txt\").await?;\n//!\n//! println!(\"File has {} lines.\", contents.lines().count());\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! To overwrite the file:\n//!\n//! ```\n//! # async fn dox() -> std::io::Result<()> {\n//! let contents = \"First line.\\nSecond line.\\nThird line.\\n\";\n//!\n//! tokio::fs::write(\"my_file.txt\", contents.as_bytes()).await?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! ## Using `File`\n//!\n//! The main type for interacting with files is [`File`]. It can be used to read\n//! from and write to a given file. This is done using the [`AsyncRead`] and\n//! [`AsyncWrite`] traits. This type is generally used when you want to do\n//! something more complex than just reading or writing the entire contents in\n//! one go.\n//!\n//! **Note:** It is important to use [`flush`] when writing to a Tokio\n//! [`File`]. This is because calls to `write` will return before the write has\n//! finished, and [`flush`] will wait for the write to finish. (The write will\n//! happen even if you don't flush; it will just happen later.) This is\n//! different from [`std::fs::File`], and is due to the fact that `File` uses\n//! `spawn_blocking` behind the scenes.\n//!\n//! For example, to count the number of lines in a file without loading the\n//! entire file into memory:\n//!\n//! ```no_run\n//! use tokio::fs::File;\n//! use tokio::io::AsyncReadExt;\n//!\n//! # async fn dox() -> std::io::Result<()> {\n//! let mut file = File::open(\"my_file.txt\").await?;\n//!\n//! let mut chunk = vec![0; 4096];\n//! let mut number_of_lines = 0;\n//! loop {\n//! let len = file.read(&mut chunk).await?;\n//! if len == 0 {\n//! // Length of zero means end of file.\n//! break;\n//! }\n//! for &b in &chunk[..len] {\n//! if b == b'\\n' {\n//! number_of_lines += 1;\n//! }\n//! }\n//! }\n//!\n//! println!(\"File has {} lines.\", number_of_lines);\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! For example, to write a file line-by-line:\n//!\n//! ```no_run\n//! use tokio::fs::File;\n//! use tokio::io::AsyncWriteExt;\n//!\n//! # async fn dox() -> std::io::Result<()> {\n//! let mut file = File::create(\"my_file.txt\").await?;\n//!\n//! file.write_all(b\"First line.\\n\").await?;\n//! file.write_all(b\"Second line.\\n\").await?;\n//! file.write_all(b\"Third line.\\n\").await?;\n//!\n//! // Remember to call `flush` after writing!\n//! file.flush().await?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! ## Tuning your file IO\n//!\n//! Tokio's file uses [`spawn_blocking`] behind the scenes, and this has serious\n//! performance consequences. To get good performance with file IO on Tokio, it\n//! is recommended to batch your operations into as few `spawn_blocking` calls\n//! as possible.\n//!\n//! One example of this difference can be seen by comparing the two reading\n//! examples above. The first example uses [`tokio::fs::read`], which reads the\n//! entire file in a single `spawn_blocking` call, and then returns it. The\n//! second example will read the file in chunks using many `spawn_blocking`\n//! calls. This means that the second example will most likely be more expensive\n//! for large files. (Of course, using chunks may be necessary for very large\n//! files that don't fit in memory.)\n//!\n//! The following examples will show some strategies for this:\n//!\n//! When creating a file, write the data to a `String` or `Vec` and then\n//! write the entire file in a single `spawn_blocking` call with\n//! `tokio::fs::write`.\n//!\n//! ```no_run\n//! # async fn dox() -> std::io::Result<()> {\n//! let mut contents = String::new();\n//!\n//! contents.push_str(\"First line.\\n\");\n//! contents.push_str(\"Second line.\\n\");\n//! contents.push_str(\"Third line.\\n\");\n//!\n//! tokio::fs::write(\"my_file.txt\", contents.as_bytes()).await?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! Use [`BufReader`] and [`BufWriter`] to buffer many small reads or writes\n//! into a few large ones. This example will most likely only perform one\n//! `spawn_blocking` call.\n//!\n//! ```no_run\n//! use tokio::fs::File;\n//! use tokio::io::{AsyncWriteExt, BufWriter};\n//!\n//! # async fn dox() -> std::io::Result<()> {\n//! let mut file = BufWriter::new(File::create(\"my_file.txt\").await?);\n//!\n//! file.write_all(b\"First line.\\n\").await?;\n//! file.write_all(b\"Second line.\\n\").await?;\n//! file.write_all(b\"Third line.\\n\").await?;\n//!\n//! // Due to the BufWriter, the actual write and spawn_blocking\n//! // call happens when you flush.\n//! file.flush().await?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! Manually use [`std::fs`] inside [`spawn_blocking`].\n//!\n//! ```no_run\n//! use std::fs::File;\n//! use std::io::{self, Write};\n//! use tokio::task::spawn_blocking;\n//!\n//! # async fn dox() -> std::io::Result<()> {\n//! spawn_blocking(move || {\n//! let mut file = File::create(\"my_file.txt\")?;\n//!\n//! file.write_all(b\"First line.\\n\")?;\n//! file.write_all(b\"Second line.\\n\")?;\n//! file.write_all(b\"Third line.\\n\")?;\n//!\n//! // Unlike Tokio's file, the std::fs file does\n//! // not need flush.\n//!\n//! io::Result::Ok(())\n//! }).await.unwrap()?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! It's also good to be aware of [`File::set_max_buf_size`], which controls the\n//! maximum amount of bytes that Tokio's [`File`] will read or write in a single\n//! [`spawn_blocking`] call. The default is two megabytes, but this is subject\n//! to change.\n//!\n//! [`spawn_blocking`]: fn@crate::task::spawn_blocking\n//! [`AsyncRead`]: trait@crate::io::AsyncRead\n//! [`AsyncWrite`]: trait@crate::io::AsyncWrite\n//! [`BufReader`]: struct@crate::io::BufReader\n//! [`BufWriter`]: struct@crate::io::BufWriter\n//! [`tokio::net::unix::pipe`]: crate::net::unix::pipe\n//! [`AsyncFd`]: crate::io::unix::AsyncFd\n//! [`flush`]: crate::io::AsyncWriteExt::flush\n//! [`tokio::fs::read`]: fn@crate::fs::read\n\nmod canonicalize;\npub use self::canonicalize::canonicalize;\n\nmod create_dir;\npub use self::create_dir::create_dir;\n\nmod create_dir_all;\npub use self::create_dir_all::create_dir_all;\n\nmod dir_builder;\npub use self::dir_builder::DirBuilder;\n\nmod file;\npub use self::file::File;\n\nmod hard_link;\npub use self::hard_link::hard_link;\n\nmod metadata;\npub use self::metadata::metadata;\n\nmod open_options;\npub use self::open_options::OpenOptions;\n\nmod read;\npub use self::read::read;\n\nmod read_dir;\npub use self::read_dir::{read_dir, DirEntry, ReadDir};\n\nmod read_link;\npub use self::read_link::read_link;\n\nmod read_to_string;\npub use self::read_to_string::read_to_string;\n\nmod remove_dir;\npub use self::remove_dir::remove_dir;\n\nmod remove_dir_all;\npub use self::remove_dir_all::remove_dir_all;\n\nmod remove_file;\npub use self::remove_file::remove_file;\n\nmod rename;\npub use self::rename::rename;\n\nmod set_permissions;\npub use self::set_permissions::set_permissions;\n\nmod symlink_metadata;\npub use self::symlink_metadata::symlink_metadata;\n\nmod write;\npub use self::write::write;\n\nmod copy;\npub use self::copy::copy;\n\nmod try_exists;\npub use self::try_exists::try_exists;\n\n#[cfg(test)]\nmod mocks;\n\nfeature! {\n #![unix]\n\n mod symlink;\n pub use self::symlink::symlink;\n}\n\ncfg_windows! {\n mod symlink_dir;\n pub use self::symlink_dir::symlink_dir;\n\n mod symlink_file;\n pub use self::symlink_file::symlink_file;\n}\n\ncfg_io_uring! {\n pub(crate) mod read_uring;\n pub(crate) use self::read_uring::read_uring;\n\n pub(crate) use self::open_options::UringOpenOptions;\n}\n\nuse std::io;\n\n#[cfg(not(test))]\nuse crate::blocking::spawn_blocking;\n#[cfg(test)]\nuse mocks::spawn_blocking;\n\npub(crate) async fn asyncify(f: F) -> io::Result\nwhere\n F: FnOnce() -> io::Result + Send + 'static,\n T: Send + 'static,\n{\n match spawn_blocking(f).await {\n Ok(res) => res,\n Err(_) => Err(io::Error::new(\n io::ErrorKind::Other,\n \"background task failed\",\n )),\n }\n}\n" }, { "path": "tokio/src/fs/open_options/mock_open_options.rs", "content": "#![allow(unreachable_pub)]\n//! Mock version of `std::fs::OpenOptions`;\nuse mockall::mock;\n\nuse crate::fs::mocks::MockFile;\nuse std::{io, path::Path};\n\nmock! {\n #[derive(Debug)]\n pub OpenOptions {\n pub fn append(&mut self, append: bool) -> &mut Self;\n pub fn create(&mut self, create: bool) -> &mut Self;\n pub fn create_new(&mut self, create_new: bool) -> &mut Self;\n pub fn open + 'static>(&self, path: P) -> io::Result;\n pub fn read(&mut self, read: bool) -> &mut Self;\n pub fn truncate(&mut self, truncate: bool) -> &mut Self;\n pub fn write(&mut self, write: bool) -> &mut Self;\n\n // Not mocking OpenOptionsExt trait due to:\n // https://github.com/rust-lang/rust/issues/153486\n #[cfg(unix)]\n pub fn custom_flags(&mut self, flags: i32) -> &mut Self;\n #[cfg(unix)]\n pub fn mode(&mut self, mode: u32) -> &mut Self;\n #[cfg(windows)]\n pub fn access_mode(&mut self, access: u32) -> &mut Self;\n #[cfg(windows)]\n pub fn share_mode(&mut self, val: u32) -> &mut Self;\n #[cfg(windows)]\n pub fn custom_flags(&mut self, flags: u32) -> &mut Self;\n #[cfg(windows)]\n pub fn attributes(&mut self, val: u32) -> &mut Self;\n #[cfg(windows)]\n pub fn security_qos_flags(&mut self, flags: u32) -> &mut Self;\n }\n impl Clone for OpenOptions {\n fn clone(&self) -> Self;\n }\n}\n" }, { "path": "tokio/src/fs/open_options/uring_open_options.rs", "content": "use std::io;\n\n#[cfg(not(test))]\nuse std::os::unix::fs::OpenOptionsExt;\n\n#[cfg(test)]\nuse super::mock_open_options::MockOpenOptions as StdOpenOptions;\n#[cfg(not(test))]\nuse std::fs::OpenOptions as StdOpenOptions;\n\n#[derive(Debug, Clone)]\npub(crate) struct UringOpenOptions {\n pub(crate) read: bool,\n pub(crate) write: bool,\n pub(crate) append: bool,\n pub(crate) truncate: bool,\n pub(crate) create: bool,\n pub(crate) create_new: bool,\n pub(crate) mode: libc::mode_t,\n pub(crate) custom_flags: libc::c_int,\n}\n\nimpl UringOpenOptions {\n pub(crate) fn new() -> Self {\n Self {\n read: false,\n write: false,\n append: false,\n truncate: false,\n create: false,\n create_new: false,\n mode: 0o666,\n custom_flags: 0,\n }\n }\n\n pub(crate) fn append(&mut self, append: bool) -> &mut Self {\n self.append = append;\n self\n }\n\n pub(crate) fn create(&mut self, create: bool) -> &mut Self {\n self.create = create;\n self\n }\n\n pub(crate) fn create_new(&mut self, create_new: bool) -> &mut Self {\n self.create_new = create_new;\n self\n }\n\n pub(crate) fn read(&mut self, read: bool) -> &mut Self {\n self.read = read;\n self\n }\n\n pub(crate) fn write(&mut self, write: bool) -> &mut Self {\n self.write = write;\n self\n }\n\n pub(crate) fn truncate(&mut self, truncate: bool) -> &mut Self {\n self.truncate = truncate;\n self\n }\n\n pub(crate) fn mode(&mut self, mode: u32) -> &mut Self {\n self.mode = mode as libc::mode_t;\n self\n }\n\n pub(crate) fn custom_flags(&mut self, flags: i32) -> &mut Self {\n self.custom_flags = flags;\n self\n }\n\n // Equivalent to https://github.com/rust-lang/rust/blob/64c81fd10509924ca4da5d93d6052a65b75418a5/library/std/src/sys/fs/unix.rs#L1118-L1127\n pub(crate) fn access_mode(&self) -> io::Result {\n match (self.read, self.write, self.append) {\n (true, false, false) => Ok(libc::O_RDONLY),\n (false, true, false) => Ok(libc::O_WRONLY),\n (true, true, false) => Ok(libc::O_RDWR),\n (false, _, true) => Ok(libc::O_WRONLY | libc::O_APPEND),\n (true, _, true) => Ok(libc::O_RDWR | libc::O_APPEND),\n (false, false, false) => Err(io::Error::from_raw_os_error(libc::EINVAL)),\n }\n }\n\n // Equivalent to https://github.com/rust-lang/rust/blob/64c81fd10509924ca4da5d93d6052a65b75418a5/library/std/src/sys/fs/unix.rs#L1129-L1151\n pub(crate) fn creation_mode(&self) -> io::Result {\n match (self.write, self.append) {\n (true, false) => {}\n (false, false) => {\n if self.truncate || self.create || self.create_new {\n return Err(io::Error::from_raw_os_error(libc::EINVAL));\n }\n }\n (_, true) => {\n if self.truncate && !self.create_new {\n return Err(io::Error::from_raw_os_error(libc::EINVAL));\n }\n }\n }\n\n Ok(match (self.create, self.truncate, self.create_new) {\n (false, false, false) => 0,\n (true, false, false) => libc::O_CREAT,\n (false, true, false) => libc::O_TRUNC,\n (true, true, false) => libc::O_CREAT | libc::O_TRUNC,\n (_, _, true) => libc::O_CREAT | libc::O_EXCL,\n })\n }\n}\n\nimpl From for StdOpenOptions {\n fn from(value: UringOpenOptions) -> Self {\n let mut std = StdOpenOptions::new();\n\n std.append(value.append);\n std.create(value.create);\n std.create_new(value.create_new);\n std.read(value.read);\n std.truncate(value.truncate);\n std.write(value.write);\n\n std.mode(value.mode);\n std.custom_flags(value.custom_flags);\n\n std\n }\n}\n" }, { "path": "tokio/src/fs/open_options.rs", "content": "use crate::fs::{asyncify, File};\n\nuse std::io;\nuse std::path::Path;\n\ncfg_io_uring! {\n mod uring_open_options;\n pub(crate) use uring_open_options::UringOpenOptions;\n use crate::runtime::driver::op::Op;\n}\n\n#[cfg(test)]\nmod mock_open_options;\n#[cfg(test)]\nuse mock_open_options::MockOpenOptions as StdOpenOptions;\n#[cfg(not(test))]\nuse std::fs::OpenOptions as StdOpenOptions;\n\n#[cfg(all(unix, not(test)))]\nuse std::os::unix::fs::OpenOptionsExt;\n#[cfg(all(windows, not(test)))]\nuse std::os::windows::fs::OpenOptionsExt;\n\n/// Options and flags which can be used to configure how a file is opened.\n///\n/// This builder exposes the ability to configure how a [`File`] is opened and\n/// what operations are permitted on the open file. The [`File::open`] and\n/// [`File::create`] methods are aliases for commonly used options using this\n/// builder.\n///\n/// Generally speaking, when using `OpenOptions`, you'll first call [`new`],\n/// then chain calls to methods to set each option, then call [`open`], passing\n/// the path of the file you're trying to open. This will give you a\n/// [`io::Result`] with a [`File`] inside that you can further operate\n/// on.\n///\n/// This is a specialized version of [`std::fs::OpenOptions`] for usage from\n/// the Tokio runtime.\n///\n/// `From` is implemented for more advanced configuration\n/// than the methods provided here.\n///\n/// [`new`]: OpenOptions::new\n/// [`open`]: OpenOptions::open\n/// [`File`]: File\n/// [`File::open`]: File::open\n/// [`File::create`]: File::create\n///\n/// # Examples\n///\n/// Opening a file to read:\n///\n/// ```no_run\n/// use tokio::fs::OpenOptions;\n/// use std::io;\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// let file = OpenOptions::new()\n/// .read(true)\n/// .open(\"foo.txt\")\n/// .await?;\n///\n/// Ok(())\n/// }\n/// ```\n///\n/// Opening a file for both reading and writing, as well as creating it if it\n/// doesn't exist:\n///\n/// ```no_run\n/// use tokio::fs::OpenOptions;\n/// use std::io;\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// let file = OpenOptions::new()\n/// .read(true)\n/// .write(true)\n/// .create(true)\n/// .open(\"foo.txt\")\n/// .await?;\n///\n/// Ok(())\n/// }\n/// ```\n#[derive(Clone, Debug)]\npub struct OpenOptions {\n inner: Kind,\n}\n\n#[derive(Debug, Clone)]\nenum Kind {\n Std(StdOpenOptions),\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Uring(UringOpenOptions),\n}\n\nimpl OpenOptions {\n /// Creates a blank new set of options ready for configuration.\n ///\n /// All options are initially set to `false`.\n ///\n /// This is an async version of [`std::fs::OpenOptions::new`][std]\n ///\n /// [std]: std::fs::OpenOptions::new\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n ///\n /// let mut options = OpenOptions::new();\n /// let future = options.read(true).open(\"foo.txt\");\n /// ```\n pub fn new() -> OpenOptions {\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n let inner = Kind::Uring(UringOpenOptions::new());\n #[cfg(not(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n )))]\n let inner = Kind::Std(StdOpenOptions::new());\n\n OpenOptions { inner }\n }\n\n /// Sets the option for read access.\n ///\n /// This option, when true, will indicate that the file should be\n /// `read`-able if opened.\n ///\n /// This is an async version of [`std::fs::OpenOptions::read`][std]\n ///\n /// [std]: std::fs::OpenOptions::read\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .read(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn read(&mut self, read: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.read(read);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.read(read);\n }\n }\n self\n }\n\n /// Sets the option for write access.\n ///\n /// This option, when true, will indicate that the file should be\n /// `write`-able if opened.\n ///\n /// This is an async version of [`std::fs::OpenOptions::write`][std]\n ///\n /// [std]: std::fs::OpenOptions::write\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn write(&mut self, write: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.write(write);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.write(write);\n }\n }\n self\n }\n\n /// Sets the option for the append mode.\n ///\n /// This option, when true, means that writes will append to a file instead\n /// of overwriting previous contents. Note that setting\n /// `.write(true).append(true)` has the same effect as setting only\n /// `.append(true)`.\n ///\n /// For most filesystems, the operating system guarantees that all writes are\n /// atomic: no writes get mangled because another process writes at the same\n /// time.\n ///\n /// One maybe obvious note when using append-mode: make sure that all data\n /// that belongs together is written to the file in one operation. This\n /// can be done by concatenating strings before passing them to [`write()`],\n /// or using a buffered writer (with a buffer of adequate size),\n /// and calling [`flush()`] when the message is complete.\n ///\n /// If a file is opened with both read and append access, beware that after\n /// opening, and after every write, the position for reading may be set at the\n /// end of the file. So, before writing, save the current position (using\n /// [`seek`]`(`[`SeekFrom`]`::`[`Current`]`(0))`), and restore it before the next read.\n ///\n /// This is an async version of [`std::fs::OpenOptions::append`][std]\n ///\n /// [std]: std::fs::OpenOptions::append\n ///\n /// ## Note\n ///\n /// This function doesn't create the file if it doesn't exist. Use the [`create`]\n /// method to do so.\n ///\n /// [`write()`]: crate::io::AsyncWriteExt::write\n /// [`flush()`]: crate::io::AsyncWriteExt::flush\n /// [`seek`]: crate::io::AsyncSeekExt::seek\n /// [`SeekFrom`]: std::io::SeekFrom\n /// [`Current`]: std::io::SeekFrom::Current\n /// [`create`]: OpenOptions::create\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .append(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn append(&mut self, append: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.append(append);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.append(append);\n }\n }\n self\n }\n\n /// Sets the option for truncating a previous file.\n ///\n /// If a file is successfully opened with this option set it will truncate\n /// the file to 0 length if it already exists.\n ///\n /// The file must be opened with write access for truncate to work.\n ///\n /// This is an async version of [`std::fs::OpenOptions::truncate`][std]\n ///\n /// [std]: std::fs::OpenOptions::truncate\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .truncate(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn truncate(&mut self, truncate: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.truncate(truncate);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.truncate(truncate);\n }\n }\n self\n }\n\n /// Sets the option for creating a new file.\n ///\n /// This option indicates whether a new file will be created if the file\n /// does not yet already exist.\n ///\n /// In order for the file to be created, [`write`] or [`append`] access must\n /// be used.\n ///\n /// This is an async version of [`std::fs::OpenOptions::create`][std]\n ///\n /// [std]: std::fs::OpenOptions::create\n /// [`write`]: OpenOptions::write\n /// [`append`]: OpenOptions::append\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .create(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn create(&mut self, create: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.create(create);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.create(create);\n }\n }\n self\n }\n\n /// Sets the option to always create a new file.\n ///\n /// This option indicates whether a new file will be created. No file is\n /// allowed to exist at the target location, also no (dangling) symlink.\n ///\n /// This option is useful because it is atomic. Otherwise between checking\n /// whether a file exists and creating a new one, the file may have been\n /// created by another process (a TOCTOU race condition / attack).\n ///\n /// If `.create_new(true)` is set, [`.create()`] and [`.truncate()`] are\n /// ignored.\n ///\n /// The file must be opened with write or append access in order to create a\n /// new file.\n ///\n /// This is an async version of [`std::fs::OpenOptions::create_new`][std]\n ///\n /// [std]: std::fs::OpenOptions::create_new\n /// [`.create()`]: OpenOptions::create\n /// [`.truncate()`]: OpenOptions::truncate\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .create_new(true)\n /// .open(\"foo.txt\")\n /// .await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn create_new(&mut self, create_new: bool) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.create_new(create_new);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.create_new(create_new);\n }\n }\n self\n }\n\n /// Opens a file at `path` with the options specified by `self`.\n ///\n /// This is an async version of [`std::fs::OpenOptions::open`][std]\n ///\n /// [std]: std::fs::OpenOptions::open\n ///\n /// # Errors\n ///\n /// This function will return an error under a number of different\n /// circumstances. Some of these error conditions are listed here, together\n /// with their [`ErrorKind`]. The mapping to [`ErrorKind`]s is not part of\n /// the compatibility contract of the function, especially the `Other` kind\n /// might change to more specific kinds in the future.\n ///\n /// * [`NotFound`]: The specified file does not exist and neither `create`\n /// or `create_new` is set.\n /// * [`NotFound`]: One of the directory components of the file path does\n /// not exist.\n /// * [`PermissionDenied`]: The user lacks permission to get the specified\n /// access rights for the file.\n /// * [`PermissionDenied`]: The user lacks permission to open one of the\n /// directory components of the specified path.\n /// * [`AlreadyExists`]: `create_new` was specified and the file already\n /// exists.\n /// * [`InvalidInput`]: Invalid combinations of open options (truncate\n /// without write access, no access mode set, etc.).\n /// * [`Other`]: One of the directory components of the specified file path\n /// was not, in fact, a directory.\n /// * [`Other`]: Filesystem-level errors: full disk, write permission\n /// requested on a read-only file system, exceeded disk quota, too many\n /// open files, too long filename, too many symbolic links in the\n /// specified path (Unix-like systems only), etc.\n ///\n /// # io_uring support\n ///\n /// On Linux, you can also use `io_uring` for executing system calls.\n /// To enable `io_uring`, you need to specify the `--cfg tokio_unstable`\n /// flag at compile time, enable the `io-uring` cargo feature, and set the\n /// `Builder::enable_io_uring` runtime option.\n ///\n /// Support for `io_uring` is currently experimental, so its behavior may\n /// change or it may be removed in future versions.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let file = OpenOptions::new().open(\"foo.txt\").await?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`ErrorKind`]: std::io::ErrorKind\n /// [`AlreadyExists`]: std::io::ErrorKind::AlreadyExists\n /// [`InvalidInput`]: std::io::ErrorKind::InvalidInput\n /// [`NotFound`]: std::io::ErrorKind::NotFound\n /// [`Other`]: std::io::ErrorKind::Other\n /// [`PermissionDenied`]: std::io::ErrorKind::PermissionDenied\n pub async fn open(&self, path: impl AsRef) -> io::Result {\n match &self.inner {\n Kind::Std(opts) => Self::std_open(opts, path).await,\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n let handle = crate::runtime::Handle::current();\n let driver_handle = handle.inner.driver().io();\n\n if driver_handle\n .check_and_init(io_uring::opcode::OpenAt::CODE)\n .await?\n {\n Op::open(path.as_ref(), opts)?.await\n } else {\n let opts = opts.clone().into();\n Self::std_open(&opts, path).await\n }\n }\n }\n }\n\n async fn std_open(opts: &StdOpenOptions, path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n let opts = opts.clone();\n\n let std = asyncify(move || opts.open(path)).await?;\n Ok(File::from_std(std))\n }\n\n #[cfg(windows)]\n pub(super) fn as_inner_mut(&mut self) -> &mut StdOpenOptions {\n match &mut self.inner {\n Kind::Std(ref mut opts) => opts,\n }\n }\n}\n\nfeature! {\n #![unix]\n\n impl OpenOptions {\n /// Sets the mode bits that a new file will be created with.\n ///\n /// If a new file is created as part of an `OpenOptions::open` call then this\n /// specified `mode` will be used as the permission bits for the new file.\n /// If no `mode` is set, the default of `0o666` will be used.\n /// The operating system masks out bits with the system's `umask`, to produce\n /// the final permissions.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut options = OpenOptions::new();\n /// options.mode(0o644); // Give read/write for owner and read for others.\n /// let file = options.open(\"foo.txt\").await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn mode(&mut self, mode: u32) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.mode(mode);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.mode(mode);\n }\n }\n self\n }\n\n /// Passes custom flags to the `flags` argument of `open`.\n ///\n /// The bits that define the access mode are masked out with `O_ACCMODE`, to\n /// ensure they do not interfere with the access mode set by Rusts options.\n ///\n /// Custom flags can only set flags, not remove flags set by Rusts options.\n /// This options overwrites any previously set custom flags.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut options = OpenOptions::new();\n /// options.write(true);\n /// if cfg!(unix) {\n /// options.custom_flags(libc::O_NOFOLLOW);\n /// }\n /// let file = options.open(\"foo.txt\").await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn custom_flags(&mut self, flags: i32) -> &mut OpenOptions {\n match &mut self.inner {\n Kind::Std(opts) => {\n opts.custom_flags(flags);\n }\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n Kind::Uring(opts) => {\n opts.custom_flags(flags);\n }\n }\n self\n }\n }\n}\n\ncfg_windows! {\n impl OpenOptions {\n /// Overrides the `dwDesiredAccess` argument to the call to [`CreateFile`]\n /// with the specified value.\n ///\n /// This will override the `read`, `write`, and `append` flags on the\n /// `OpenOptions` structure. This method provides fine-grained control over\n /// the permissions to read, write and append data, attributes (like hidden\n /// and system), and extended attributes.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// // Open without read and write permission, for example if you only need\n /// // to call `stat` on the file\n /// let file = OpenOptions::new().access_mode(0).open(\"foo.txt\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n pub fn access_mode(&mut self, access: u32) -> &mut OpenOptions {\n self.as_inner_mut().access_mode(access);\n self\n }\n\n /// Overrides the `dwShareMode` argument to the call to [`CreateFile`] with\n /// the specified value.\n ///\n /// By default `share_mode` is set to\n /// `FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE`. This allows\n /// other processes to read, write, and delete/rename the same file\n /// while it is open. Removing any of the flags will prevent other\n /// processes from performing the corresponding operation until the file\n /// handle is closed.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs::OpenOptions;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// // Do not allow others to read or modify this file while we have it open\n /// // for writing.\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .share_mode(0)\n /// .open(\"foo.txt\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n pub fn share_mode(&mut self, share: u32) -> &mut OpenOptions {\n self.as_inner_mut().share_mode(share);\n self\n }\n\n /// Sets extra flags for the `dwFileFlags` argument to the call to\n /// [`CreateFile2`] to the specified value (or combines it with\n /// `attributes` and `security_qos_flags` to set the `dwFlagsAndAttributes`\n /// for [`CreateFile`]).\n ///\n /// Custom flags can only set flags, not remove flags set by Rust's options.\n /// This option overwrites any previously set custom flags.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use windows_sys::Win32::Storage::FileSystem::FILE_FLAG_DELETE_ON_CLOSE;\n /// use tokio::fs::OpenOptions;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// let file = OpenOptions::new()\n /// .create(true)\n /// .write(true)\n /// .custom_flags(FILE_FLAG_DELETE_ON_CLOSE)\n /// .open(\"foo.txt\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n /// [`CreateFile2`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfile2\n pub fn custom_flags(&mut self, flags: u32) -> &mut OpenOptions {\n self.as_inner_mut().custom_flags(flags);\n self\n }\n\n /// Sets the `dwFileAttributes` argument to the call to [`CreateFile2`] to\n /// the specified value (or combines it with `custom_flags` and\n /// `security_qos_flags` to set the `dwFlagsAndAttributes` for\n /// [`CreateFile`]).\n ///\n /// If a _new_ file is created because it does not yet exist and\n /// `.create(true)` or `.create_new(true)` are specified, the new file is\n /// given the attributes declared with `.attributes()`.\n ///\n /// If an _existing_ file is opened with `.create(true).truncate(true)`, its\n /// existing attributes are preserved and combined with the ones declared\n /// with `.attributes()`.\n ///\n /// In all other cases the attributes get ignored.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use windows_sys::Win32::Storage::FileSystem::FILE_ATTRIBUTE_HIDDEN;\n /// use tokio::fs::OpenOptions;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .create(true)\n /// .attributes(FILE_ATTRIBUTE_HIDDEN)\n /// .open(\"foo.txt\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n /// [`CreateFile2`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfile2\n pub fn attributes(&mut self, attributes: u32) -> &mut OpenOptions {\n self.as_inner_mut().attributes(attributes);\n self\n }\n\n /// Sets the `dwSecurityQosFlags` argument to the call to [`CreateFile2`] to\n /// the specified value (or combines it with `custom_flags` and `attributes`\n /// to set the `dwFlagsAndAttributes` for [`CreateFile`]).\n ///\n /// By default `security_qos_flags` is not set. It should be specified when\n /// opening a named pipe, to control to which degree a server process can\n /// act on behalf of a client process (security impersonation level).\n ///\n /// When `security_qos_flags` is not set, a malicious program can gain the\n /// elevated privileges of a privileged Rust process when it allows opening\n /// user-specified paths, by tricking it into opening a named pipe. So\n /// arguably `security_qos_flags` should also be set when opening arbitrary\n /// paths. However the bits can then conflict with other flags, specifically\n /// `FILE_FLAG_OPEN_NO_RECALL`.\n ///\n /// For information about possible values, see [Impersonation Levels] on the\n /// Windows Dev Center site. The `SECURITY_SQOS_PRESENT` flag is set\n /// automatically when using this method.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use windows_sys::Win32::Storage::FileSystem::SECURITY_IDENTIFICATION;\n /// use tokio::fs::OpenOptions;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .create(true)\n ///\n /// // Sets the flag value to `SecurityIdentification`.\n /// .security_qos_flags(SECURITY_IDENTIFICATION)\n ///\n /// .open(r\"\\\\.\\pipe\\MyPipe\").await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n /// [`CreateFile2`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfile2\n /// [Impersonation Levels]:\n /// https://docs.microsoft.com/en-us/windows/win32/api/winnt/ne-winnt-security_impersonation_level\n pub fn security_qos_flags(&mut self, flags: u32) -> &mut OpenOptions {\n self.as_inner_mut().security_qos_flags(flags);\n self\n }\n }\n}\n\nimpl From for OpenOptions {\n fn from(options: StdOpenOptions) -> OpenOptions {\n OpenOptions {\n inner: Kind::Std(options),\n // TODO: Add support for converting `StdOpenOptions` to `UringOpenOptions`\n // if user enables `io-uring` cargo feature. It is blocked by:\n // * https://github.com/rust-lang/rust/issues/74943\n // * https://github.com/rust-lang/rust/issues/76801\n }\n }\n}\n\nimpl Default for OpenOptions {\n fn default() -> Self {\n Self::new()\n }\n}\n" }, { "path": "tokio/src/fs/read.rs", "content": "use crate::fs::asyncify;\n\nuse std::{io, path::Path};\n\n/// Reads the entire contents of a file into a bytes vector.\n///\n/// This is an async version of [`std::fs::read`].\n///\n/// This is a convenience function for using [`File::open`] and [`read_to_end`]\n/// with fewer imports and without an intermediate variable. It pre-allocates a\n/// buffer based on the file size when available, so it is generally faster than\n/// reading into a vector created with `Vec::new()`.\n///\n/// This operation is implemented by running the equivalent blocking operation\n/// on a separate thread pool using [`spawn_blocking`].\n///\n/// [`File::open`]: super::File::open\n/// [`read_to_end`]: crate::io::AsyncReadExt::read_to_end\n/// [`spawn_blocking`]: crate::task::spawn_blocking\n///\n/// # Errors\n///\n/// This function will return an error if `path` does not already exist.\n/// Other errors may also be returned according to [`OpenOptions::open`].\n///\n/// [`OpenOptions::open`]: super::OpenOptions::open\n///\n/// It will also return an error if it encounters while reading an error\n/// of a kind other than [`ErrorKind::Interrupted`].\n///\n/// [`ErrorKind::Interrupted`]: std::io::ErrorKind::Interrupted\n///\n/// # io_uring support\n///\n/// On Linux, you can also use io_uring for executing system calls. To enable\n/// io_uring, you need to specify the `--cfg tokio_unstable` flag at compile time,\n/// enable the io-uring cargo feature, and set the `Builder::enable_io_uring`\n/// runtime option.\n///\n/// Support for io_uring is currently experimental, so its behavior may change\n/// or it may be removed in future versions.\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n/// use std::net::SocketAddr;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// let contents = fs::read(\"address.txt\").await?;\n/// let foo: SocketAddr = String::from_utf8_lossy(&contents).parse()?;\n/// Ok(())\n/// }\n/// ```\npub async fn read(path: impl AsRef) -> io::Result> {\n let path = path.as_ref().to_owned();\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n {\n use crate::fs::read_uring;\n\n let handle = crate::runtime::Handle::current();\n let driver_handle = handle.inner.driver().io();\n if driver_handle\n .check_and_init(io_uring::opcode::Read::CODE)\n .await?\n {\n return read_uring(&path).await;\n }\n }\n\n asyncify(move || std::fs::read(path)).await\n}\n" }, { "path": "tokio/src/fs/read_dir.rs", "content": "use crate::fs::asyncify;\n\nuse std::collections::VecDeque;\nuse std::ffi::OsString;\nuse std::fs::{FileType, Metadata};\nuse std::future::Future;\nuse std::io;\nuse std::path::{Path, PathBuf};\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{ready, Context, Poll};\n\n#[cfg(test)]\nuse super::mocks::spawn_blocking;\n#[cfg(test)]\nuse super::mocks::JoinHandle;\n#[cfg(not(test))]\nuse crate::blocking::spawn_blocking;\n#[cfg(not(test))]\nuse crate::blocking::JoinHandle;\n\nconst CHUNK_SIZE: usize = 32;\n\n/// Returns a stream over the entries within a directory.\n///\n/// This is an async version of [`std::fs::read_dir`].\n///\n/// This operation is implemented by running the equivalent blocking\n/// operation on a separate thread pool using [`spawn_blocking`].\n///\n/// [`spawn_blocking`]: crate::task::spawn_blocking\npub async fn read_dir(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(|| -> io::Result {\n let mut std = std::fs::read_dir(path)?;\n let mut buf = VecDeque::with_capacity(CHUNK_SIZE);\n let remain = ReadDir::next_chunk(&mut buf, &mut std);\n\n Ok(ReadDir(State::Idle(Some((buf, std, remain)))))\n })\n .await\n}\n\n/// Reads the entries in a directory.\n///\n/// This struct is returned from the [`read_dir`] function of this module and\n/// will yield instances of [`DirEntry`]. Through a [`DirEntry`] information\n/// like the entry's path and possibly other metadata can be learned.\n///\n/// A `ReadDir` can be turned into a `Stream` with [`ReadDirStream`].\n///\n/// [`ReadDirStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.ReadDirStream.html\n///\n/// # Errors\n///\n/// This stream will return an [`Err`] if there's some sort of intermittent\n/// IO error during iteration.\n///\n/// [`read_dir`]: read_dir\n/// [`DirEntry`]: DirEntry\n/// [`Err`]: std::result::Result::Err\n#[derive(Debug)]\n#[must_use = \"streams do nothing unless polled\"]\npub struct ReadDir(State);\n\n#[derive(Debug)]\nenum State {\n Idle(Option<(VecDeque>, std::fs::ReadDir, bool)>),\n Pending(JoinHandle<(VecDeque>, std::fs::ReadDir, bool)>),\n}\n\nimpl ReadDir {\n /// Returns the next entry in the directory stream.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe.\n pub async fn next_entry(&mut self) -> io::Result> {\n use std::future::poll_fn;\n poll_fn(|cx| self.poll_next_entry(cx)).await\n }\n\n /// Polls for the next directory entry in the stream.\n ///\n /// This method returns:\n ///\n /// * `Poll::Pending` if the next directory entry is not yet available.\n /// * `Poll::Ready(Ok(Some(entry)))` if the next directory entry is available.\n /// * `Poll::Ready(Ok(None))` if there are no more directory entries in this\n /// stream.\n /// * `Poll::Ready(Err(err))` if an IO error occurred while reading the next\n /// directory entry.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when the next directory entry\n /// becomes available on the underlying IO resource.\n ///\n /// Note that on multiple calls to `poll_next_entry`, only the `Waker` from\n /// the `Context` passed to the most recent call is scheduled to receive a\n /// wakeup.\n pub fn poll_next_entry(&mut self, cx: &mut Context<'_>) -> Poll>> {\n loop {\n match self.0 {\n State::Idle(ref mut data) => {\n let (buf, _, ref remain) = data.as_mut().unwrap();\n\n if let Some(ent) = buf.pop_front() {\n return Poll::Ready(ent.map(Some));\n } else if !remain {\n return Poll::Ready(Ok(None));\n }\n\n let (mut buf, mut std, _) = data.take().unwrap();\n\n self.0 = State::Pending(spawn_blocking(move || {\n let remain = ReadDir::next_chunk(&mut buf, &mut std);\n (buf, std, remain)\n }));\n }\n State::Pending(ref mut rx) => {\n self.0 = State::Idle(Some(ready!(Pin::new(rx).poll(cx))?));\n }\n }\n }\n }\n\n fn next_chunk(buf: &mut VecDeque>, std: &mut std::fs::ReadDir) -> bool {\n for _ in 0..CHUNK_SIZE {\n let ret = match std.next() {\n Some(ret) => ret,\n None => return false,\n };\n\n let success = ret.is_ok();\n\n buf.push_back(ret.map(|std| DirEntry {\n #[cfg(not(any(\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\",\n target_os = \"vxworks\",\n target_os = \"aix\",\n target_os = \"nto\",\n target_os = \"vita\",\n )))]\n file_type: std.file_type().ok(),\n std: Arc::new(std),\n }));\n\n if !success {\n break;\n }\n }\n\n true\n }\n}\n\nfeature! {\n #![unix]\n\n use std::os::unix::fs::DirEntryExt;\n\n impl DirEntry {\n /// Returns the underlying `d_ino` field in the contained `dirent`\n /// structure.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::fs;\n ///\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// let mut entries = fs::read_dir(\".\").await?;\n /// while let Some(entry) = entries.next_entry().await? {\n /// // Here, `entry` is a `DirEntry`.\n /// println!(\"{:?}: {}\", entry.file_name(), entry.ino());\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub fn ino(&self) -> u64 {\n self.as_inner().ino()\n }\n }\n}\n\n/// Entries returned by the [`ReadDir`] stream.\n///\n/// [`ReadDir`]: struct@ReadDir\n///\n/// This is a specialized version of [`std::fs::DirEntry`] for usage from the\n/// Tokio runtime.\n///\n/// An instance of `DirEntry` represents an entry inside of a directory on the\n/// filesystem. Each entry can be inspected via methods to learn about the full\n/// path or possibly other metadata through per-platform extension traits.\n#[derive(Debug)]\npub struct DirEntry {\n #[cfg(not(any(\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\",\n target_os = \"vxworks\",\n target_os = \"aix\",\n target_os = \"nto\",\n target_os = \"vita\",\n )))]\n file_type: Option,\n std: Arc,\n}\n\nimpl DirEntry {\n /// Returns the full path to the file that this entry represents.\n ///\n /// The full path is created by joining the original path to `read_dir`\n /// with the filename of this entry.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::fs;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut entries = fs::read_dir(\".\").await?;\n ///\n /// while let Some(entry) = entries.next_entry().await? {\n /// println!(\"{:?}\", entry.path());\n /// }\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// This prints output like:\n ///\n /// ```text\n /// \"./whatever.txt\"\n /// \"./foo.html\"\n /// \"./hello_world.rs\"\n /// ```\n ///\n /// The exact text, of course, depends on what files you have in `.`.\n pub fn path(&self) -> PathBuf {\n self.std.path()\n }\n\n /// Returns the bare file name of this directory entry without any other\n /// leading path component.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::fs;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut entries = fs::read_dir(\".\").await?;\n ///\n /// while let Some(entry) = entries.next_entry().await? {\n /// println!(\"{:?}\", entry.file_name());\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub fn file_name(&self) -> OsString {\n self.std.file_name()\n }\n\n /// Returns the metadata for the file that this entry points at.\n ///\n /// This function will not traverse symlinks if this entry points at a\n /// symlink.\n ///\n /// # Platform-specific behavior\n ///\n /// On Windows this function is cheap to call (no extra system calls\n /// needed), but on Unix platforms this function is the equivalent of\n /// calling `symlink_metadata` on the path.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::fs;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut entries = fs::read_dir(\".\").await?;\n ///\n /// while let Some(entry) = entries.next_entry().await? {\n /// if let Ok(metadata) = entry.metadata().await {\n /// // Now let's show our entry's permissions!\n /// println!(\"{:?}: {:?}\", entry.path(), metadata.permissions());\n /// } else {\n /// println!(\"Couldn't get file type for {:?}\", entry.path());\n /// }\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub async fn metadata(&self) -> io::Result {\n let std = self.std.clone();\n asyncify(move || std.metadata()).await\n }\n\n /// Returns the file type for the file that this entry points at.\n ///\n /// This function will not traverse symlinks if this entry points at a\n /// symlink.\n ///\n /// # Platform-specific behavior\n ///\n /// On Windows and most Unix platforms this function is free (no extra\n /// system calls needed), but some Unix platforms may require the equivalent\n /// call to `symlink_metadata` to learn about the target file type.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::fs;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut entries = fs::read_dir(\".\").await?;\n ///\n /// while let Some(entry) = entries.next_entry().await? {\n /// if let Ok(file_type) = entry.file_type().await {\n /// // Now let's show our entry's file type!\n /// println!(\"{:?}: {:?}\", entry.path(), file_type);\n /// } else {\n /// println!(\"Couldn't get file type for {:?}\", entry.path());\n /// }\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub async fn file_type(&self) -> io::Result {\n #[cfg(not(any(\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\",\n target_os = \"vxworks\",\n target_os = \"aix\",\n target_os = \"nto\",\n target_os = \"vita\",\n )))]\n if let Some(file_type) = self.file_type {\n return Ok(file_type);\n }\n\n let std = self.std.clone();\n asyncify(move || std.file_type()).await\n }\n\n /// Returns a reference to the underlying `std::fs::DirEntry`.\n #[cfg(unix)]\n pub(super) fn as_inner(&self) -> &std::fs::DirEntry {\n &self.std\n }\n}\n" }, { "path": "tokio/src/fs/read_link.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::{Path, PathBuf};\n\n/// Reads a symbolic link, returning the file that the link points to.\n///\n/// This is an async version of [`std::fs::read_link`].\npub async fn read_link(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::read_link(path)).await\n}\n" }, { "path": "tokio/src/fs/read_to_string.rs", "content": "use crate::fs::asyncify;\n\nuse std::{io, path::Path};\n\n/// Creates a future which will open a file for reading and read the entire\n/// contents into a string and return said string.\n///\n/// This is the async equivalent of [`std::fs::read_to_string`][std].\n///\n/// This operation is implemented by running the equivalent blocking operation\n/// on a separate thread pool using [`spawn_blocking`].\n///\n/// [`spawn_blocking`]: crate::task::spawn_blocking\n/// [std]: fn@std::fs::read_to_string\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// let contents = fs::read_to_string(\"foo.txt\").await?;\n/// println!(\"foo.txt contains {} bytes\", contents.len());\n/// # Ok(())\n/// # }\n/// ```\npub async fn read_to_string(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::read_to_string(path)).await\n}\n" }, { "path": "tokio/src/fs/read_uring.rs", "content": "use crate::fs::OpenOptions;\nuse crate::runtime::driver::op::Op;\n\nuse std::io;\nuse std::io::ErrorKind;\nuse std::os::fd::OwnedFd;\nuse std::path::Path;\n\n// this algorithm is inspired from rust std lib version 1.90.0\n// https://doc.rust-lang.org/1.90.0/src/std/io/mod.rs.html#409\nconst PROBE_SIZE: usize = 32;\nconst PROBE_SIZE_U32: u32 = PROBE_SIZE as u32;\n\n// Max bytes we can read using io uring submission at a time\n// SAFETY: cannot be higher than u32::MAX for safe cast\n// Set to read max 64 MiB at time\nconst MAX_READ_SIZE: usize = 64 * 1024 * 1024;\n\npub(crate) async fn read_uring(path: &Path) -> io::Result> {\n let file = OpenOptions::new().read(true).open(path).await?;\n\n // TODO: use io uring in the future to obtain metadata\n let size_hint: Option = file.metadata().await.map(|m| m.len() as usize).ok();\n\n let fd: OwnedFd = file\n .try_into_std()\n .expect(\"unexpected in-flight operation detected\")\n .into();\n\n let mut buf = Vec::new();\n\n if let Some(size_hint) = size_hint {\n buf.try_reserve(size_hint)?;\n }\n\n read_to_end_uring(fd, buf).await\n}\n\nasync fn read_to_end_uring(mut fd: OwnedFd, mut buf: Vec) -> io::Result> {\n let mut offset = 0;\n let start_cap = buf.capacity();\n\n loop {\n if buf.len() == buf.capacity() && buf.capacity() == start_cap && buf.len() >= PROBE_SIZE {\n // The buffer might be an exact fit. Let's read into a probe buffer\n // and see if it returns `Ok(0)`. If so, we've avoided an\n // unnecessary increasing of the capacity. But if not, append the\n // probe buffer to the primary buffer and let its capacity grow.\n let (r_fd, r_buf, is_eof) = small_probe_read(fd, buf, &mut offset).await?;\n\n if is_eof {\n return Ok(r_buf);\n }\n\n buf = r_buf;\n fd = r_fd;\n }\n\n // buf is full, need more capacity\n if buf.len() == buf.capacity() {\n buf.try_reserve(PROBE_SIZE)?;\n }\n\n // prepare the spare capacity to be read into\n let buf_len = usize::min(buf.spare_capacity_mut().len(), MAX_READ_SIZE);\n\n // buf_len cannot be greater than u32::MAX because MAX_READ_SIZE\n // is less than u32::MAX\n let read_len = u32::try_from(buf_len).expect(\"buf_len must always fit in u32\");\n\n // read into spare capacity\n let (r_fd, r_buf, is_eof) = op_read(fd, buf, &mut offset, read_len).await?;\n\n if is_eof {\n return Ok(r_buf);\n }\n\n fd = r_fd;\n buf = r_buf;\n }\n}\n\nasync fn small_probe_read(\n fd: OwnedFd,\n mut buf: Vec,\n offset: &mut u64,\n) -> io::Result<(OwnedFd, Vec, bool)> {\n let read_len = PROBE_SIZE_U32;\n\n let mut temp_arr = [0; PROBE_SIZE];\n // we don't call this function if the buffer's length < PROBE_SIZE\n let back_bytes_len = buf.len() - PROBE_SIZE;\n\n temp_arr.copy_from_slice(&buf[back_bytes_len..]);\n\n // We're decreasing the length of the buffer and len is greater\n // than PROBE_SIZE. So we can read into the discarded length\n buf.truncate(back_bytes_len);\n\n let (r_fd, mut r_buf, is_eof) = op_read(fd, buf, offset, read_len).await?;\n // If `size_read` returns zero due to reasons such as the buffer's exact fit,\n // then this `try_reserve` does not perform allocation.\n r_buf.try_reserve(PROBE_SIZE)?;\n r_buf.splice(back_bytes_len..back_bytes_len, temp_arr);\n\n Ok((r_fd, r_buf, is_eof))\n}\n\n// Takes a length to read and returns a single read in the buffer\n//\n// Returns the file descriptor, buffer and EOF reached or not\nasync fn op_read(\n mut fd: OwnedFd,\n mut buf: Vec,\n offset: &mut u64,\n read_len: u32,\n) -> io::Result<(OwnedFd, Vec, bool)> {\n loop {\n let (res, r_fd, r_buf) = Op::read(fd, buf, read_len, *offset).await;\n\n match res {\n Err(e) if e.kind() == ErrorKind::Interrupted => {\n buf = r_buf;\n fd = r_fd;\n }\n Err(e) => return Err(e),\n Ok(size_read) => {\n *offset += size_read as u64;\n\n return Ok((r_fd, r_buf, size_read == 0));\n }\n }\n }\n}\n" }, { "path": "tokio/src/fs/remove_dir.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Removes an existing, empty directory.\n///\n/// This is an async version of [`std::fs::remove_dir`].\npub async fn remove_dir(path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::remove_dir(path)).await\n}\n" }, { "path": "tokio/src/fs/remove_dir_all.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Removes a directory at this path, after removing all its contents. Use carefully!\n///\n/// This is an async version of [`std::fs::remove_dir_all`][std]\n///\n/// [std]: fn@std::fs::remove_dir_all\npub async fn remove_dir_all(path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::remove_dir_all(path)).await\n}\n" }, { "path": "tokio/src/fs/remove_file.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Removes a file from the filesystem.\n///\n/// Note that there is no guarantee that the file is immediately deleted (e.g.\n/// depending on platform, other open file descriptors may prevent immediate\n/// removal).\n///\n/// This is an async version of [`std::fs::remove_file`].\npub async fn remove_file(path: impl AsRef) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(move || std::fs::remove_file(path)).await\n}\n" }, { "path": "tokio/src/fs/rename.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Renames a file or directory to a new name, replacing the original file if\n/// `to` already exists.\n///\n/// This will not work if the new name is on a different mount point.\n///\n/// This is an async version of [`std::fs::rename`].\npub async fn rename(from: impl AsRef, to: impl AsRef) -> io::Result<()> {\n let from = from.as_ref().to_owned();\n let to = to.as_ref().to_owned();\n\n asyncify(move || std::fs::rename(from, to)).await\n}\n" }, { "path": "tokio/src/fs/set_permissions.rs", "content": "use crate::fs::asyncify;\n\nuse std::fs::Permissions;\nuse std::io;\nuse std::path::Path;\n\n/// Changes the permissions found on a file or a directory.\n///\n/// This is an async version of [`std::fs::set_permissions`][std]\n///\n/// [std]: fn@std::fs::set_permissions\npub async fn set_permissions(path: impl AsRef, perm: Permissions) -> io::Result<()> {\n let path = path.as_ref().to_owned();\n asyncify(|| std::fs::set_permissions(path, perm)).await\n}\n" }, { "path": "tokio/src/fs/symlink.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Creates a new symbolic link on the filesystem.\n///\n/// The `link` path will be a symbolic link pointing to the `original` path.\n///\n/// This is an async version of [`std::os::unix::fs::symlink`].\npub async fn symlink(original: impl AsRef, link: impl AsRef) -> io::Result<()> {\n let original = original.as_ref().to_owned();\n let link = link.as_ref().to_owned();\n\n asyncify(move || std::os::unix::fs::symlink(original, link)).await\n}\n" }, { "path": "tokio/src/fs/symlink_dir.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Creates a new directory symlink on the filesystem.\n///\n/// The `link` path will be a directory symbolic link pointing to the `original`\n/// path.\n///\n/// This is an async version of [`std::os::windows::fs::symlink_dir`][std]\n///\n/// [std]: https://doc.rust-lang.org/std/os/windows/fs/fn.symlink_dir.html\npub async fn symlink_dir(original: impl AsRef, link: impl AsRef) -> io::Result<()> {\n let original = original.as_ref().to_owned();\n let link = link.as_ref().to_owned();\n\n asyncify(move || std::os::windows::fs::symlink_dir(original, link)).await\n}\n" }, { "path": "tokio/src/fs/symlink_file.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Creates a new file symbolic link on the filesystem.\n///\n/// The `link` path will be a file symbolic link pointing to the `original`\n/// path.\n///\n/// This is an async version of [`std::os::windows::fs::symlink_file`][std]\n///\n/// [std]: https://doc.rust-lang.org/std/os/windows/fs/fn.symlink_file.html\npub async fn symlink_file(original: impl AsRef, link: impl AsRef) -> io::Result<()> {\n let original = original.as_ref().to_owned();\n let link = link.as_ref().to_owned();\n\n asyncify(move || std::os::windows::fs::symlink_file(original, link)).await\n}\n" }, { "path": "tokio/src/fs/symlink_metadata.rs", "content": "use crate::fs::asyncify;\n\nuse std::fs::Metadata;\nuse std::io;\nuse std::path::Path;\n\n/// Queries the file system metadata for a path.\n///\n/// This is an async version of [`std::fs::symlink_metadata`][std]\n///\n/// [std]: fn@std::fs::symlink_metadata\npub async fn symlink_metadata(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(|| std::fs::symlink_metadata(path)).await\n}\n" }, { "path": "tokio/src/fs/try_exists.rs", "content": "use crate::fs::asyncify;\n\nuse std::io;\nuse std::path::Path;\n\n/// Returns `Ok(true)` if the path points at an existing entity.\n///\n/// This function will traverse symbolic links to query information about the\n/// destination file. In case of broken symbolic links this will return `Ok(false)`.\n///\n/// This is the async equivalent of [`std::path::Path::try_exists`][std].\n///\n/// [std]: fn@std::path::Path::try_exists\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// fs::try_exists(\"foo.txt\").await?;\n/// # Ok(())\n/// # }\n/// ```\npub async fn try_exists(path: impl AsRef) -> io::Result {\n let path = path.as_ref().to_owned();\n asyncify(move || path.try_exists()).await\n}\n" }, { "path": "tokio/src/fs/write.rs", "content": "use crate::{fs::asyncify, util::as_ref::OwnedBuf};\n\nuse std::{io, path::Path};\n\n/// Creates a future that will open a file for writing and write the entire\n/// contents of `contents` to it.\n///\n/// This is the async equivalent of [`std::fs::write`][std].\n///\n/// This operation is implemented by running the equivalent blocking operation\n/// on a separate thread pool using [`spawn_blocking`].\n///\n/// [`spawn_blocking`]: crate::task::spawn_blocking\n/// [std]: fn@std::fs::write\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::fs;\n///\n/// # async fn dox() -> std::io::Result<()> {\n/// fs::write(\"foo.txt\", b\"Hello world!\").await?;\n/// # Ok(())\n/// # }\n/// ```\npub async fn write(path: impl AsRef, contents: impl AsRef<[u8]>) -> io::Result<()> {\n let path = path.as_ref();\n let contents = crate::util::as_ref::upgrade(contents);\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n ))]\n {\n let handle = crate::runtime::Handle::current();\n let driver_handle = handle.inner.driver().io();\n if driver_handle\n .check_and_init(io_uring::opcode::Write::CODE)\n .await?\n {\n return write_uring(path, contents).await;\n }\n }\n\n write_spawn_blocking(path, contents).await\n}\n\n#[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n))]\nasync fn write_uring(path: &Path, mut buf: OwnedBuf) -> io::Result<()> {\n use crate::{fs::OpenOptions, runtime::driver::op::Op};\n use std::os::fd::OwnedFd;\n\n let file = OpenOptions::new()\n .write(true)\n .create(true)\n .truncate(true)\n .open(path)\n .await?;\n\n let mut fd: OwnedFd = file\n .try_into_std()\n .expect(\"unexpected in-flight operation detected\")\n .into();\n\n let total: usize = buf.as_ref().len();\n let mut buf_offset: usize = 0;\n let mut file_offset: u64 = 0;\n while buf_offset < total {\n let (res, _buf, _fd) = Op::write_at(fd, buf, buf_offset, file_offset)?.await;\n\n let n = match res {\n Ok(0) => return Err(io::ErrorKind::WriteZero.into()),\n Ok(n) => n,\n Err(e) if e.kind() == io::ErrorKind::Interrupted => 0,\n Err(e) => return Err(e),\n };\n\n buf = _buf;\n fd = _fd;\n buf_offset += n as usize;\n file_offset += n as u64;\n }\n\n Ok(())\n}\n\nasync fn write_spawn_blocking(path: &Path, contents: OwnedBuf) -> io::Result<()> {\n let path = path.to_owned();\n asyncify(move || std::fs::write(path, contents)).await\n}\n" }, { "path": "tokio/src/future/block_on.rs", "content": "use std::future::Future;\n\ncfg_rt! {\n #[track_caller]\n pub(crate) fn block_on(f: F) -> F::Output {\n let mut e = crate::runtime::context::try_enter_blocking_region().expect(\n \"Cannot block the current thread from within a runtime. This \\\n happens because a function attempted to block the current \\\n thread while the thread is being used to drive asynchronous \\\n tasks.\"\n );\n e.block_on(f).unwrap()\n }\n}\n\ncfg_not_rt! {\n #[track_caller]\n pub(crate) fn block_on(f: F) -> F::Output {\n let mut park = crate::runtime::park::CachedParkThread::new();\n park.block_on(f).unwrap()\n }\n}\n" }, { "path": "tokio/src/future/maybe_done.rs", "content": "//! Definition of the [`MaybeDone`] combinator.\n\nuse pin_project_lite::pin_project;\nuse std::future::{Future, IntoFuture};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// A future that may have completed.\n #[derive(Debug)]\n #[project = MaybeDoneProj]\n #[project_replace = MaybeDoneProjReplace]\n #[repr(C)] // https://github.com/rust-lang/miri/issues/3780\n pub enum MaybeDone {\n /// A not-yet-completed future.\n Future { #[pin] future: Fut },\n /// The output of the completed future.\n Done { output: Fut::Output },\n /// The empty variant after the result of a [`MaybeDone`] has been\n /// taken using the [`take_output`](MaybeDone::take_output) method.\n Gone,\n }\n}\n\n/// Wraps a future into a `MaybeDone`.\npub fn maybe_done(future: F) -> MaybeDone {\n MaybeDone::Future {\n future: future.into_future(),\n }\n}\n\nimpl MaybeDone {\n /// Returns an [`Option`] containing a mutable reference to the output of the future.\n /// The output of this method will be [`Some`] if and only if the inner\n /// future has been completed and [`take_output`](MaybeDone::take_output)\n /// has not yet been called.\n pub fn output_mut(self: Pin<&mut Self>) -> Option<&mut Fut::Output> {\n match self.project() {\n MaybeDoneProj::Done { output } => Some(output),\n _ => None,\n }\n }\n\n /// Attempts to take the output of a `MaybeDone` without driving it\n /// towards completion.\n #[inline]\n pub fn take_output(self: Pin<&mut Self>) -> Option {\n match *self {\n MaybeDone::Done { .. } => {}\n MaybeDone::Future { .. } | MaybeDone::Gone => return None,\n };\n if let MaybeDoneProjReplace::Done { output } = self.project_replace(MaybeDone::Gone) {\n Some(output)\n } else {\n unreachable!()\n }\n }\n}\n\nimpl Future for MaybeDone {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let output = match self.as_mut().project() {\n MaybeDoneProj::Future { future } => ready!(future.poll(cx)),\n MaybeDoneProj::Done { .. } => return Poll::Ready(()),\n MaybeDoneProj::Gone => panic!(\"MaybeDone polled after value taken\"),\n };\n self.set(MaybeDone::Done { output });\n Poll::Ready(())\n }\n}\n\n// Test for https://github.com/tokio-rs/tokio/issues/6729\n#[cfg(test)]\nmod miri_tests {\n use super::maybe_done;\n\n use std::{\n future::Future,\n pin::Pin,\n sync::Arc,\n task::{Context, Poll, Wake},\n };\n\n struct ThingAdder<'a> {\n thing: &'a mut String,\n }\n\n impl Future for ThingAdder<'_> {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n unsafe {\n *self.get_unchecked_mut().thing += \", world\";\n }\n Poll::Pending\n }\n }\n\n #[test]\n fn maybe_done_miri() {\n let mut thing = \"hello\".to_owned();\n\n // The async block is necessary to trigger the miri failure.\n #[allow(clippy::redundant_async_block)]\n let fut = async move { ThingAdder { thing: &mut thing }.await };\n\n let mut fut = maybe_done(fut);\n let mut fut = unsafe { Pin::new_unchecked(&mut fut) };\n\n let waker = Arc::new(DummyWaker).into();\n let mut ctx = Context::from_waker(&waker);\n assert_eq!(fut.as_mut().poll(&mut ctx), Poll::Pending);\n assert_eq!(fut.as_mut().poll(&mut ctx), Poll::Pending);\n }\n\n struct DummyWaker;\n\n impl Wake for DummyWaker {\n fn wake(self: Arc) {}\n }\n}\n" }, { "path": "tokio/src/future/mod.rs", "content": "#![cfg_attr(not(feature = \"macros\"), allow(unreachable_pub))]\n\n//! Asynchronous values.\n\n#[cfg(any(feature = \"macros\", feature = \"process\"))]\npub(crate) mod maybe_done;\n\ncfg_process! {\n mod try_join;\n pub(crate) use try_join::try_join3;\n}\n\ncfg_sync! {\n mod block_on;\n pub(crate) use block_on::block_on;\n}\n\ncfg_trace! {\n mod trace;\n #[allow(unused_imports)]\n pub(crate) use trace::InstrumentedFuture as Future;\n}\n\ncfg_not_trace! {\n cfg_rt! {\n pub(crate) use std::future::Future;\n }\n}\n" }, { "path": "tokio/src/future/trace.rs", "content": "use std::future::Future;\n\npub(crate) trait InstrumentedFuture: Future {\n fn id(&self) -> Option;\n}\n\nimpl InstrumentedFuture for tracing::instrument::Instrumented {\n fn id(&self) -> Option {\n self.span().id()\n }\n}\n" }, { "path": "tokio/src/future/try_join.rs", "content": "use crate::future::maybe_done::{maybe_done, MaybeDone};\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npub(crate) fn try_join3(\n future1: F1,\n future2: F2,\n future3: F3,\n) -> TryJoin3\nwhere\n F1: Future>,\n F2: Future>,\n F3: Future>,\n{\n TryJoin3 {\n future1: maybe_done(future1),\n future2: maybe_done(future2),\n future3: maybe_done(future3),\n }\n}\n\npin_project! {\n pub(crate) struct TryJoin3\n where\n F1: Future,\n F2: Future,\n F3: Future,\n {\n #[pin]\n future1: MaybeDone,\n #[pin]\n future2: MaybeDone,\n #[pin]\n future3: MaybeDone,\n }\n}\n\nimpl Future for TryJoin3\nwhere\n F1: Future>,\n F2: Future>,\n F3: Future>,\n{\n type Output = Result<(T1, T2, T3), E>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let mut all_done = true;\n\n let mut me = self.project();\n\n if me.future1.as_mut().poll(cx).is_pending() {\n all_done = false;\n } else if me.future1.as_mut().output_mut().unwrap().is_err() {\n return Poll::Ready(Err(me.future1.take_output().unwrap().err().unwrap()));\n }\n\n if me.future2.as_mut().poll(cx).is_pending() {\n all_done = false;\n } else if me.future2.as_mut().output_mut().unwrap().is_err() {\n return Poll::Ready(Err(me.future2.take_output().unwrap().err().unwrap()));\n }\n\n if me.future3.as_mut().poll(cx).is_pending() {\n all_done = false;\n } else if me.future3.as_mut().output_mut().unwrap().is_err() {\n return Poll::Ready(Err(me.future3.take_output().unwrap().err().unwrap()));\n }\n\n if all_done {\n Poll::Ready(Ok((\n me.future1.take_output().unwrap().ok().unwrap(),\n me.future2.take_output().unwrap().ok().unwrap(),\n me.future3.take_output().unwrap().ok().unwrap(),\n )))\n } else {\n Poll::Pending\n }\n }\n}\n" }, { "path": "tokio/src/fuzz.rs", "content": "pub use crate::util::linked_list::tests::fuzz_linked_list;\n" }, { "path": "tokio/src/io/async_buf_read.rs", "content": "use crate::io::AsyncRead;\n\nuse std::io;\nuse std::ops::DerefMut;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Reads bytes asynchronously.\n///\n/// This trait is analogous to [`std::io::BufRead`], but integrates with\n/// the asynchronous task system. In particular, the [`poll_fill_buf`] method,\n/// unlike [`BufRead::fill_buf`], will automatically queue the current task for wakeup\n/// and return if data is not yet available, rather than blocking the calling\n/// thread.\n///\n/// Utilities for working with `AsyncBufRead` values are provided by\n/// [`AsyncBufReadExt`].\n///\n/// [`std::io::BufRead`]: std::io::BufRead\n/// [`poll_fill_buf`]: AsyncBufRead::poll_fill_buf\n/// [`BufRead::fill_buf`]: std::io::BufRead::fill_buf\n/// [`AsyncBufReadExt`]: crate::io::AsyncBufReadExt\npub trait AsyncBufRead: AsyncRead {\n /// Attempts to return the contents of the internal buffer, filling it with more data\n /// from the inner reader if it is empty.\n ///\n /// On success, returns `Poll::Ready(Ok(buf))`.\n ///\n /// If no data is available for reading, the method returns\n /// `Poll::Pending` and arranges for the current task (via\n /// `cx.waker().wake_by_ref()`) to receive a notification when the object becomes\n /// readable or is closed.\n ///\n /// This function is a lower-level call. It needs to be paired with the\n /// [`consume`] method to function properly. When calling this\n /// method, none of the contents will be \"read\" in the sense that later\n /// calling [`poll_read`] may return the same contents. As such, [`consume`] must\n /// be called with the number of bytes that are consumed from this buffer to\n /// ensure that the bytes are never returned twice.\n ///\n /// An empty buffer returned indicates that the stream has reached EOF.\n ///\n /// [`poll_read`]: AsyncRead::poll_read\n /// [`consume`]: AsyncBufRead::consume\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll>;\n\n /// Tells this buffer that `amt` bytes have been consumed from the buffer,\n /// so they should no longer be returned in calls to [`poll_read`].\n ///\n /// This function is a lower-level call. It needs to be paired with the\n /// [`poll_fill_buf`] method to function properly. This function does\n /// not perform any I/O, it simply informs this object that some amount of\n /// its buffer, returned from [`poll_fill_buf`], has been consumed and should\n /// no longer be returned. As such, this function may do odd things if\n /// [`poll_fill_buf`] isn't called before calling it.\n ///\n /// The `amt` must be `<=` the number of bytes in the buffer returned by\n /// [`poll_fill_buf`].\n ///\n /// [`poll_read`]: AsyncRead::poll_read\n /// [`poll_fill_buf`]: AsyncBufRead::poll_fill_buf\n fn consume(self: Pin<&mut Self>, amt: usize);\n}\n\nmacro_rules! deref_async_buf_read {\n () => {\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut **self.get_mut()).poll_fill_buf(cx)\n }\n\n fn consume(mut self: Pin<&mut Self>, amt: usize) {\n Pin::new(&mut **self).consume(amt)\n }\n };\n}\n\nimpl AsyncBufRead for Box {\n deref_async_buf_read!();\n}\n\nimpl AsyncBufRead for &mut T {\n deref_async_buf_read!();\n}\n\nimpl

AsyncBufRead for Pin

\nwhere\n P: DerefMut,\n P::Target: AsyncBufRead,\n{\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_fill_buf(cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n crate::util::pin_as_deref_mut(self).consume(amt);\n }\n}\n\nimpl AsyncBufRead for &[u8] {\n fn poll_fill_buf(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(*self))\n }\n\n fn consume(mut self: Pin<&mut Self>, amt: usize) {\n *self = &self[amt..];\n }\n}\n\nimpl + Unpin> AsyncBufRead for io::Cursor {\n fn poll_fill_buf(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(io::BufRead::fill_buf(self.get_mut()))\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n io::BufRead::consume(self.get_mut(), amt);\n }\n}\n" }, { "path": "tokio/src/io/async_fd.rs", "content": "use crate::io::{Interest, Ready};\nuse crate::runtime::io::{ReadyEvent, Registration};\nuse crate::runtime::scheduler;\n\nuse mio::unix::SourceFd;\nuse std::error::Error;\nuse std::fmt;\nuse std::io;\nuse std::os::unix::io::{AsRawFd, RawFd};\nuse std::task::{ready, Context, Poll};\n\n/// Associates an IO object backed by a Unix file descriptor with the tokio\n/// reactor, allowing for readiness to be polled. The file descriptor must be of\n/// a type that can be used with the OS polling facilities (ie, `poll`, `epoll`,\n/// `kqueue`, etc), such as a network socket or pipe, and the file descriptor\n/// must have the nonblocking mode set to true.\n///\n/// Creating an [`AsyncFd`] registers the file descriptor with the current tokio\n/// Reactor, allowing you to directly await the file descriptor being readable\n/// or writable. Once registered, the file descriptor remains registered until\n/// the [`AsyncFd`] is dropped.\n///\n/// The [`AsyncFd`] takes ownership of an arbitrary object to represent the IO\n/// object. It is intended that the inner object will handle closing the file\n/// descriptor when it is dropped, avoiding resource leaks and ensuring that the\n/// [`AsyncFd`] can clean up the registration before closing the file descriptor.\n/// The [`AsyncFd::into_inner`] function can be used to extract the inner object\n/// to retake control from the tokio IO reactor. The [`OwnedFd`] type is often\n/// used as the inner object, as it is the simplest type that closes the fd on\n/// drop.\n///\n/// The inner object is required to implement [`AsRawFd`]. This file descriptor\n/// must not change while [`AsyncFd`] owns the inner object, i.e. the\n/// [`AsRawFd::as_raw_fd`] method on the inner type must always return the same\n/// file descriptor when called multiple times. Failure to uphold this results\n/// in unspecified behavior in the IO driver, which may include breaking\n/// notifications for other sockets/etc.\n///\n/// Polling for readiness is done by calling the async functions [`readable`]\n/// and [`writable`]. These functions complete when the associated readiness\n/// condition is observed. Any number of tasks can query the same `AsyncFd` in\n/// parallel, on the same or different conditions.\n///\n/// On some platforms, the readiness detecting mechanism relies on\n/// edge-triggered notifications. This means that the OS will only notify Tokio\n/// when the file descriptor transitions from not-ready to ready. For this to\n/// work you should first try to read or write and only poll for readiness\n/// if that fails with an error of [`std::io::ErrorKind::WouldBlock`].\n///\n/// Tokio internally tracks when it has received a ready notification, and when\n/// readiness checking functions like [`readable`] and [`writable`] are called,\n/// if the readiness flag is set, these async functions will complete\n/// immediately. This however does mean that it is critical to ensure that this\n/// ready flag is cleared when (and only when) the file descriptor ceases to be\n/// ready. The [`AsyncFdReadyGuard`] returned from readiness checking functions\n/// serves this function; after calling a readiness-checking async function,\n/// you must use this [`AsyncFdReadyGuard`] to signal to tokio whether the file\n/// descriptor is no longer in a ready state.\n///\n/// ## Use with to a poll-based API\n///\n/// In some cases it may be desirable to use `AsyncFd` from APIs similar to\n/// [`TcpStream::poll_read_ready`]. The [`AsyncFd::poll_read_ready`] and\n/// [`AsyncFd::poll_write_ready`] functions are provided for this purpose.\n/// Because these functions don't create a future to hold their state, they have\n/// the limitation that only one task can wait on each direction (read or write)\n/// at a time.\n///\n/// # Examples\n///\n/// This example shows how to turn [`std::net::TcpStream`] asynchronous using\n/// `AsyncFd`. It implements the read/write operations both as an `async fn`\n/// and using the IO traits [`AsyncRead`] and [`AsyncWrite`].\n///\n/// ```no_run\n/// use std::io::{self, Read, Write};\n/// use std::net::TcpStream;\n/// use std::pin::Pin;\n/// use std::task::{ready, Context, Poll};\n/// use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\n/// use tokio::io::unix::AsyncFd;\n///\n/// pub struct AsyncTcpStream {\n/// inner: AsyncFd,\n/// }\n///\n/// impl AsyncTcpStream {\n/// pub fn new(tcp: TcpStream) -> io::Result {\n/// tcp.set_nonblocking(true)?;\n/// Ok(Self {\n/// inner: AsyncFd::new(tcp)?,\n/// })\n/// }\n///\n/// pub async fn read(&self, out: &mut [u8]) -> io::Result {\n/// loop {\n/// let mut guard = self.inner.readable().await?;\n///\n/// match guard.try_io(|inner| inner.get_ref().read(out)) {\n/// Ok(result) => return result,\n/// Err(_would_block) => continue,\n/// }\n/// }\n/// }\n///\n/// pub async fn write(&self, buf: &[u8]) -> io::Result {\n/// loop {\n/// let mut guard = self.inner.writable().await?;\n///\n/// match guard.try_io(|inner| inner.get_ref().write(buf)) {\n/// Ok(result) => return result,\n/// Err(_would_block) => continue,\n/// }\n/// }\n/// }\n/// }\n///\n/// impl AsyncRead for AsyncTcpStream {\n/// fn poll_read(\n/// self: Pin<&mut Self>,\n/// cx: &mut Context<'_>,\n/// buf: &mut ReadBuf<'_>\n/// ) -> Poll> {\n/// loop {\n/// let mut guard = ready!(self.inner.poll_read_ready(cx))?;\n///\n/// let unfilled = buf.initialize_unfilled();\n/// match guard.try_io(|inner| inner.get_ref().read(unfilled)) {\n/// Ok(Ok(len)) => {\n/// buf.advance(len);\n/// return Poll::Ready(Ok(()));\n/// },\n/// Ok(Err(err)) => return Poll::Ready(Err(err)),\n/// Err(_would_block) => continue,\n/// }\n/// }\n/// }\n/// }\n///\n/// impl AsyncWrite for AsyncTcpStream {\n/// fn poll_write(\n/// self: Pin<&mut Self>,\n/// cx: &mut Context<'_>,\n/// buf: &[u8]\n/// ) -> Poll> {\n/// loop {\n/// let mut guard = ready!(self.inner.poll_write_ready(cx))?;\n///\n/// match guard.try_io(|inner| inner.get_ref().write(buf)) {\n/// Ok(result) => return Poll::Ready(result),\n/// Err(_would_block) => continue,\n/// }\n/// }\n/// }\n///\n/// fn poll_flush(\n/// self: Pin<&mut Self>,\n/// cx: &mut Context<'_>,\n/// ) -> Poll> {\n/// // tcp flush is a no-op\n/// Poll::Ready(Ok(()))\n/// }\n///\n/// fn poll_shutdown(\n/// self: Pin<&mut Self>,\n/// cx: &mut Context<'_>,\n/// ) -> Poll> {\n/// self.inner.get_ref().shutdown(std::net::Shutdown::Write)?;\n/// Poll::Ready(Ok(()))\n/// }\n/// }\n/// ```\n///\n/// [`readable`]: method@Self::readable\n/// [`writable`]: method@Self::writable\n/// [`AsyncFdReadyGuard`]: struct@self::AsyncFdReadyGuard\n/// [`TcpStream::poll_read_ready`]: struct@crate::net::TcpStream\n/// [`AsyncRead`]: trait@crate::io::AsyncRead\n/// [`AsyncWrite`]: trait@crate::io::AsyncWrite\n/// [`OwnedFd`]: struct@std::os::fd::OwnedFd\npub struct AsyncFd {\n registration: Registration,\n // The inner value is always present. the Option is required for `drop` and `into_inner`.\n // In all other methods `unwrap` is valid, and will never panic.\n inner: Option,\n}\n\n/// Represents an IO-ready event detected on a particular file descriptor that\n/// has not yet been acknowledged. This is a `must_use` structure to help ensure\n/// that you do not forget to explicitly clear (or not clear) the event.\n///\n/// This type exposes an immutable reference to the underlying IO object.\n#[must_use = \"You must explicitly choose whether to clear the readiness state by calling a method on ReadyGuard\"]\npub struct AsyncFdReadyGuard<'a, T: AsRawFd> {\n async_fd: &'a AsyncFd,\n event: Option,\n}\n\n/// Represents an IO-ready event detected on a particular file descriptor that\n/// has not yet been acknowledged. This is a `must_use` structure to help ensure\n/// that you do not forget to explicitly clear (or not clear) the event.\n///\n/// This type exposes a mutable reference to the underlying IO object.\n#[must_use = \"You must explicitly choose whether to clear the readiness state by calling a method on ReadyGuard\"]\npub struct AsyncFdReadyMutGuard<'a, T: AsRawFd> {\n async_fd: &'a mut AsyncFd,\n event: Option,\n}\n\nimpl AsyncFd {\n /// Creates an [`AsyncFd`] backed by (and taking ownership of) an object\n /// implementing [`AsRawFd`]. The backing file descriptor is cached at the\n /// time of creation.\n ///\n /// Only configures the [`Interest::READABLE`] and [`Interest::WRITABLE`] interests. For more\n /// control, use [`AsyncFd::with_interest`].\n ///\n /// This method must be called in the context of a tokio runtime.\n ///\n /// # Panics\n ///\n /// This function panics if there is no current reactor set, or if the `rt`\n /// feature flag is not enabled.\n #[inline]\n #[track_caller]\n pub fn new(inner: T) -> io::Result\n where\n T: AsRawFd,\n {\n Self::with_interest(inner, Interest::READABLE | Interest::WRITABLE)\n }\n\n /// Creates an [`AsyncFd`] backed by (and taking ownership of) an object\n /// implementing [`AsRawFd`], with a specific [`Interest`]. The backing\n /// file descriptor is cached at the time of creation.\n ///\n /// # Panics\n ///\n /// This function panics if there is no current reactor set, or if the `rt`\n /// feature flag is not enabled.\n #[inline]\n #[track_caller]\n pub fn with_interest(inner: T, interest: Interest) -> io::Result\n where\n T: AsRawFd,\n {\n Self::new_with_handle_and_interest(inner, scheduler::Handle::current(), interest)\n }\n\n #[track_caller]\n pub(crate) fn new_with_handle_and_interest(\n inner: T,\n handle: scheduler::Handle,\n interest: Interest,\n ) -> io::Result {\n Self::try_new_with_handle_and_interest(inner, handle, interest).map_err(Into::into)\n }\n\n /// Creates an [`AsyncFd`] backed by (and taking ownership of) an object\n /// implementing [`AsRawFd`]. The backing file descriptor is cached at the\n /// time of creation.\n ///\n /// Only configures the [`Interest::READABLE`] and [`Interest::WRITABLE`] interests. For more\n /// control, use [`AsyncFd::try_with_interest`].\n ///\n /// This method must be called in the context of a tokio runtime.\n ///\n /// In the case of failure, it returns [`AsyncFdTryNewError`] that contains the original object\n /// passed to this function.\n ///\n /// # Panics\n ///\n /// This function panics if there is no current reactor set, or if the `rt`\n /// feature flag is not enabled.\n #[inline]\n #[track_caller]\n pub fn try_new(inner: T) -> Result>\n where\n T: AsRawFd,\n {\n Self::try_with_interest(inner, Interest::READABLE | Interest::WRITABLE)\n }\n\n /// Creates an [`AsyncFd`] backed by (and taking ownership of) an object\n /// implementing [`AsRawFd`], with a specific [`Interest`]. The backing\n /// file descriptor is cached at the time of creation.\n ///\n /// In the case of failure, it returns [`AsyncFdTryNewError`] that contains the original object\n /// passed to this function.\n ///\n /// # Panics\n ///\n /// This function panics if there is no current reactor set, or if the `rt`\n /// feature flag is not enabled.\n #[inline]\n #[track_caller]\n pub fn try_with_interest(inner: T, interest: Interest) -> Result>\n where\n T: AsRawFd,\n {\n Self::try_new_with_handle_and_interest(inner, scheduler::Handle::current(), interest)\n }\n\n #[track_caller]\n pub(crate) fn try_new_with_handle_and_interest(\n inner: T,\n handle: scheduler::Handle,\n interest: Interest,\n ) -> Result> {\n let fd = inner.as_raw_fd();\n\n match Registration::new_with_interest_and_handle(&mut SourceFd(&fd), interest, handle) {\n Ok(registration) => Ok(AsyncFd {\n registration,\n inner: Some(inner),\n }),\n Err(cause) => Err(AsyncFdTryNewError { inner, cause }),\n }\n }\n\n /// Returns a shared reference to the backing object of this [`AsyncFd`].\n #[inline]\n pub fn get_ref(&self) -> &T {\n self.inner.as_ref().unwrap()\n }\n\n /// Returns a mutable reference to the backing object of this [`AsyncFd`].\n #[inline]\n pub fn get_mut(&mut self) -> &mut T {\n self.inner.as_mut().unwrap()\n }\n\n fn take_inner(&mut self) -> Option {\n let inner = self.inner.take()?;\n let fd = inner.as_raw_fd();\n\n let _ = self.registration.deregister(&mut SourceFd(&fd));\n\n Some(inner)\n }\n\n /// Deregisters this file descriptor and returns ownership of the backing\n /// object.\n pub fn into_inner(mut self) -> T {\n self.take_inner().unwrap()\n }\n\n /// Polls for read readiness.\n ///\n /// If the file descriptor is not currently ready for reading, this method\n /// will store a clone of the [`Waker`] from the provided [`Context`]. When the\n /// file descriptor becomes ready for reading, [`Waker::wake`] will be called.\n ///\n /// Note that on multiple calls to [`poll_read_ready`] or\n /// [`poll_read_ready_mut`], only the `Waker` from the `Context` passed to the\n /// most recent call is scheduled to receive a wakeup. (However,\n /// [`poll_write_ready`] retains a second, independent waker).\n ///\n /// This method is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// This method takes `&self`, so it is possible to call this method\n /// concurrently with other methods on this struct. This method only\n /// provides shared access to the inner IO resource when handling the\n /// [`AsyncFdReadyGuard`].\n ///\n /// [`poll_read_ready`]: method@Self::poll_read_ready\n /// [`poll_read_ready_mut`]: method@Self::poll_read_ready_mut\n /// [`poll_write_ready`]: method@Self::poll_write_ready\n /// [`readable`]: method@Self::readable\n /// [`Context`]: struct@std::task::Context\n /// [`Waker`]: struct@std::task::Waker\n /// [`Waker::wake`]: method@std::task::Waker::wake\n pub fn poll_read_ready<'a>(\n &'a self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let event = ready!(self.registration.poll_read_ready(cx))?;\n\n Poll::Ready(Ok(AsyncFdReadyGuard {\n async_fd: self,\n event: Some(event),\n }))\n }\n\n /// Polls for read readiness.\n ///\n /// If the file descriptor is not currently ready for reading, this method\n /// will store a clone of the [`Waker`] from the provided [`Context`]. When the\n /// file descriptor becomes ready for reading, [`Waker::wake`] will be called.\n ///\n /// Note that on multiple calls to [`poll_read_ready`] or\n /// [`poll_read_ready_mut`], only the `Waker` from the `Context` passed to the\n /// most recent call is scheduled to receive a wakeup. (However,\n /// [`poll_write_ready`] retains a second, independent waker).\n ///\n /// This method is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// This method takes `&mut self`, so it is possible to access the inner IO\n /// resource mutably when handling the [`AsyncFdReadyMutGuard`].\n ///\n /// [`poll_read_ready`]: method@Self::poll_read_ready\n /// [`poll_read_ready_mut`]: method@Self::poll_read_ready_mut\n /// [`poll_write_ready`]: method@Self::poll_write_ready\n /// [`readable`]: method@Self::readable\n /// [`Context`]: struct@std::task::Context\n /// [`Waker`]: struct@std::task::Waker\n /// [`Waker::wake`]: method@std::task::Waker::wake\n pub fn poll_read_ready_mut<'a>(\n &'a mut self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let event = ready!(self.registration.poll_read_ready(cx))?;\n\n Poll::Ready(Ok(AsyncFdReadyMutGuard {\n async_fd: self,\n event: Some(event),\n }))\n }\n\n /// Polls for write readiness.\n ///\n /// If the file descriptor is not currently ready for writing, this method\n /// will store a clone of the [`Waker`] from the provided [`Context`]. When the\n /// file descriptor becomes ready for writing, [`Waker::wake`] will be called.\n ///\n /// Note that on multiple calls to [`poll_write_ready`] or\n /// [`poll_write_ready_mut`], only the `Waker` from the `Context` passed to the\n /// most recent call is scheduled to receive a wakeup. (However,\n /// [`poll_read_ready`] retains a second, independent waker).\n ///\n /// This method is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// This method takes `&self`, so it is possible to call this method\n /// concurrently with other methods on this struct. This method only\n /// provides shared access to the inner IO resource when handling the\n /// [`AsyncFdReadyGuard`].\n ///\n /// [`poll_read_ready`]: method@Self::poll_read_ready\n /// [`poll_write_ready`]: method@Self::poll_write_ready\n /// [`poll_write_ready_mut`]: method@Self::poll_write_ready_mut\n /// [`writable`]: method@Self::readable\n /// [`Context`]: struct@std::task::Context\n /// [`Waker`]: struct@std::task::Waker\n /// [`Waker::wake`]: method@std::task::Waker::wake\n pub fn poll_write_ready<'a>(\n &'a self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let event = ready!(self.registration.poll_write_ready(cx))?;\n\n Poll::Ready(Ok(AsyncFdReadyGuard {\n async_fd: self,\n event: Some(event),\n }))\n }\n\n /// Polls for write readiness.\n ///\n /// If the file descriptor is not currently ready for writing, this method\n /// will store a clone of the [`Waker`] from the provided [`Context`]. When the\n /// file descriptor becomes ready for writing, [`Waker::wake`] will be called.\n ///\n /// Note that on multiple calls to [`poll_write_ready`] or\n /// [`poll_write_ready_mut`], only the `Waker` from the `Context` passed to the\n /// most recent call is scheduled to receive a wakeup. (However,\n /// [`poll_read_ready`] retains a second, independent waker).\n ///\n /// This method is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// This method takes `&mut self`, so it is possible to access the inner IO\n /// resource mutably when handling the [`AsyncFdReadyMutGuard`].\n ///\n /// [`poll_read_ready`]: method@Self::poll_read_ready\n /// [`poll_write_ready`]: method@Self::poll_write_ready\n /// [`poll_write_ready_mut`]: method@Self::poll_write_ready_mut\n /// [`writable`]: method@Self::readable\n /// [`Context`]: struct@std::task::Context\n /// [`Waker`]: struct@std::task::Waker\n /// [`Waker::wake`]: method@std::task::Waker::wake\n pub fn poll_write_ready_mut<'a>(\n &'a mut self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let event = ready!(self.registration.poll_write_ready(cx))?;\n\n Poll::Ready(Ok(AsyncFdReadyMutGuard {\n async_fd: self,\n event: Some(event),\n }))\n }\n\n /// Waits for any of the requested ready states, returning a\n /// [`AsyncFdReadyGuard`] that must be dropped to resume\n /// polling for the requested ready states.\n ///\n /// The function may complete without the file descriptor being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// When an IO operation does return `io::ErrorKind::WouldBlock`, the readiness must be cleared.\n /// When a combined interest is used, it is important to clear only the readiness\n /// that is actually observed to block. For instance when the combined\n /// interest `Interest::READABLE | Interest::WRITABLE` is used, and a read blocks, only\n /// read readiness should be cleared using the [`AsyncFdReadyGuard::clear_ready_matching`] method:\n /// `guard.clear_ready_matching(Ready::READABLE)`.\n /// Also clearing the write readiness in this case would be incorrect. The [`AsyncFdReadyGuard::clear_ready`]\n /// method clears all readiness flags.\n ///\n /// This method takes `&self`, so it is possible to call this method\n /// concurrently with other methods on this struct. This method only\n /// provides shared access to the inner IO resource when handling the\n /// [`AsyncFdReadyGuard`].\n ///\n /// # Examples\n ///\n /// Concurrently read and write to a [`std::net::TcpStream`] on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use std::error::Error;\n /// use std::io;\n /// use std::io::{Read, Write};\n /// use std::net::TcpStream;\n /// use tokio::io::unix::AsyncFd;\n /// use tokio::io::{Interest, Ready};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\")?;\n /// stream.set_nonblocking(true)?;\n /// let stream = AsyncFd::new(stream)?;\n ///\n /// loop {\n /// let mut guard = stream\n /// .ready(Interest::READABLE | Interest::WRITABLE)\n /// .await?;\n ///\n /// if guard.ready().is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.get_ref().read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a read has blocked, but a write might still succeed.\n /// // clear only the read readiness.\n /// guard.clear_ready_matching(Ready::READABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if guard.ready().is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.get_ref().write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a write has blocked, but a read might still succeed.\n /// // clear only the write readiness.\n /// guard.clear_ready_matching(Ready::WRITABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result> {\n let event = self.registration.readiness(interest).await?;\n\n Ok(AsyncFdReadyGuard {\n async_fd: self,\n event: Some(event),\n })\n }\n\n /// Waits for any of the requested ready states, returning a\n /// [`AsyncFdReadyMutGuard`] that must be dropped to resume\n /// polling for the requested ready states.\n ///\n /// The function may complete without the file descriptor being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// When an IO operation does return `io::ErrorKind::WouldBlock`, the readiness must be cleared.\n /// When a combined interest is used, it is important to clear only the readiness\n /// that is actually observed to block. For instance when the combined\n /// interest `Interest::READABLE | Interest::WRITABLE` is used, and a read blocks, only\n /// read readiness should be cleared using the [`AsyncFdReadyMutGuard::clear_ready_matching`] method:\n /// `guard.clear_ready_matching(Ready::READABLE)`.\n /// Also clearing the write readiness in this case would be incorrect.\n /// The [`AsyncFdReadyMutGuard::clear_ready`] method clears all readiness flags.\n ///\n /// This method takes `&mut self`, so it is possible to access the inner IO\n /// resource mutably when handling the [`AsyncFdReadyMutGuard`].\n ///\n /// # Examples\n ///\n /// Concurrently read and write to a [`std::net::TcpStream`] on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use std::error::Error;\n /// use std::io;\n /// use std::io::{Read, Write};\n /// use std::net::TcpStream;\n /// use tokio::io::unix::AsyncFd;\n /// use tokio::io::{Interest, Ready};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\")?;\n /// stream.set_nonblocking(true)?;\n /// let mut stream = AsyncFd::new(stream)?;\n ///\n /// loop {\n /// let mut guard = stream\n /// .ready_mut(Interest::READABLE | Interest::WRITABLE)\n /// .await?;\n ///\n /// if guard.ready().is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match guard.get_inner_mut().read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a read has blocked, but a write might still succeed.\n /// // clear only the read readiness.\n /// guard.clear_ready_matching(Ready::READABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if guard.ready().is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match guard.get_inner_mut().write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a write has blocked, but a read might still succeed.\n /// // clear only the write readiness.\n /// guard.clear_ready_matching(Ready::WRITABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready_mut(\n &mut self,\n interest: Interest,\n ) -> io::Result> {\n let event = self.registration.readiness(interest).await?;\n\n Ok(AsyncFdReadyMutGuard {\n async_fd: self,\n event: Some(event),\n })\n }\n\n /// Waits for the file descriptor to become readable, returning a\n /// [`AsyncFdReadyGuard`] that must be dropped to resume read-readiness\n /// polling.\n ///\n /// This method takes `&self`, so it is possible to call this method\n /// concurrently with other methods on this struct. This method only\n /// provides shared access to the inner IO resource when handling the\n /// [`AsyncFdReadyGuard`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n #[allow(clippy::needless_lifetimes)] // The lifetime improves rustdoc rendering.\n pub async fn readable<'a>(&'a self) -> io::Result> {\n self.ready(Interest::READABLE).await\n }\n\n /// Waits for the file descriptor to become readable, returning a\n /// [`AsyncFdReadyMutGuard`] that must be dropped to resume read-readiness\n /// polling.\n ///\n /// This method takes `&mut self`, so it is possible to access the inner IO\n /// resource mutably when handling the [`AsyncFdReadyMutGuard`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n #[allow(clippy::needless_lifetimes)] // The lifetime improves rustdoc rendering.\n pub async fn readable_mut<'a>(&'a mut self) -> io::Result> {\n self.ready_mut(Interest::READABLE).await\n }\n\n /// Waits for the file descriptor to become writable, returning a\n /// [`AsyncFdReadyGuard`] that must be dropped to resume write-readiness\n /// polling.\n ///\n /// This method takes `&self`, so it is possible to call this method\n /// concurrently with other methods on this struct. This method only\n /// provides shared access to the inner IO resource when handling the\n /// [`AsyncFdReadyGuard`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n #[allow(clippy::needless_lifetimes)] // The lifetime improves rustdoc rendering.\n pub async fn writable<'a>(&'a self) -> io::Result> {\n self.ready(Interest::WRITABLE).await\n }\n\n /// Waits for the file descriptor to become writable, returning a\n /// [`AsyncFdReadyMutGuard`] that must be dropped to resume write-readiness\n /// polling.\n ///\n /// This method takes `&mut self`, so it is possible to access the inner IO\n /// resource mutably when handling the [`AsyncFdReadyMutGuard`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n #[allow(clippy::needless_lifetimes)] // The lifetime improves rustdoc rendering.\n pub async fn writable_mut<'a>(&'a mut self) -> io::Result> {\n self.ready_mut(Interest::WRITABLE).await\n }\n\n /// Reads or writes from the file descriptor using a user-provided IO operation.\n ///\n /// The `async_io` method is a convenience utility that waits for the file\n /// descriptor to become ready, and then executes the provided IO operation.\n /// Since file descriptors may be marked ready spuriously, the closure will\n /// be called repeatedly until it returns something other than a\n /// [`WouldBlock`] error. This is done using the following loop:\n ///\n /// ```no_run\n /// # use std::io::{self, Result};\n /// # struct Dox { inner: T }\n /// # impl Dox {\n /// # async fn writable(&self) -> Result<&Self> {\n /// # Ok(self)\n /// # }\n /// # fn try_io(&self, _: impl FnMut(&T) -> Result) -> Result> {\n /// # panic!()\n /// # }\n /// async fn async_io(&self, mut f: impl FnMut(&T) -> io::Result) -> io::Result {\n /// loop {\n /// // or `readable` if called with the read interest.\n /// let guard = self.writable().await?;\n ///\n /// match guard.try_io(&mut f) {\n /// Ok(result) => return result,\n /// Err(_would_block) => continue,\n /// }\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// The closure should only return a [`WouldBlock`] error if it has performed\n /// an IO operation on the file descriptor that failed due to the file descriptor not being\n /// ready. Returning a [`WouldBlock`] error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the file descriptor to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio [`AsyncFd`] type, as this will mess with the\n /// readiness flag and can cause the file descriptor to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// # Examples\n ///\n /// This example sends some bytes on the inner [`std::net::UdpSocket`]. The `async_io`\n /// method waits for readiness, and retries if the send operation does block. This example\n /// is equivalent to the one given for [`try_io`].\n ///\n /// ```no_run\n /// use tokio::io::{Interest, unix::AsyncFd};\n ///\n /// use std::io;\n /// use std::net::UdpSocket;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"0.0.0.0:8080\")?;\n /// socket.set_nonblocking(true)?;\n /// let async_fd = AsyncFd::new(socket)?;\n ///\n /// let written = async_fd\n /// .async_io(Interest::WRITABLE, |inner| inner.send(&[1, 2]))\n /// .await?;\n ///\n /// println!(\"wrote {written} bytes\");\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`try_io`]: AsyncFdReadyGuard::try_io\n /// [`WouldBlock`]: std::io::ErrorKind::WouldBlock\n pub async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut(&T) -> io::Result,\n ) -> io::Result {\n self.registration\n .async_io(interest, || f(self.get_ref()))\n .await\n }\n\n /// Reads or writes from the file descriptor using a user-provided IO operation.\n ///\n /// The behavior is the same as [`async_io`], except that the closure can mutate the inner\n /// value of the [`AsyncFd`].\n ///\n /// [`async_io`]: AsyncFd::async_io\n pub async fn async_io_mut(\n &mut self,\n interest: Interest,\n mut f: impl FnMut(&mut T) -> io::Result,\n ) -> io::Result {\n self.registration\n .async_io(interest, || f(self.inner.as_mut().unwrap()))\n .await\n }\n\n /// Tries to read or write from the file descriptor using a user-provided IO operation.\n ///\n /// If the file descriptor is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation on the file descriptor by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// file descriptor is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the file descriptor is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the file descriptor that failed due to the file descriptor not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the file descriptor to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `AsyncFd` type, as this will mess with the\n /// readiness flag and can cause the file descriptor to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce(&T) -> io::Result,\n ) -> io::Result {\n self.registration\n .try_io(interest, || f(self.inner.as_ref().unwrap()))\n }\n\n /// Tries to read or write from the file descriptor using a user-provided IO operation.\n ///\n /// The behavior is the same as [`try_io`], except that the closure can mutate the inner\n /// value of the [`AsyncFd`].\n ///\n /// [`try_io`]: AsyncFd::try_io\n pub fn try_io_mut(\n &mut self,\n interest: Interest,\n f: impl FnOnce(&mut T) -> io::Result,\n ) -> io::Result {\n self.registration\n .try_io(interest, || f(self.inner.as_mut().unwrap()))\n }\n}\n\nimpl AsRawFd for AsyncFd {\n fn as_raw_fd(&self) -> RawFd {\n self.inner.as_ref().unwrap().as_raw_fd()\n }\n}\n\nimpl std::os::unix::io::AsFd for AsyncFd {\n fn as_fd(&self) -> std::os::unix::io::BorrowedFd<'_> {\n unsafe { std::os::unix::io::BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\nimpl std::fmt::Debug for AsyncFd {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"AsyncFd\")\n .field(\"inner\", &self.inner)\n .finish()\n }\n}\n\nimpl Drop for AsyncFd {\n fn drop(&mut self) {\n let _ = self.take_inner();\n }\n}\n\nimpl<'a, Inner: AsRawFd> AsyncFdReadyGuard<'a, Inner> {\n /// Indicates to tokio that the file descriptor is no longer ready. All\n /// internal readiness flags will be cleared, and tokio will wait for the\n /// next edge-triggered readiness notification from the OS.\n ///\n /// This function is commonly used with guards returned by [`AsyncFd::readable`] and\n /// [`AsyncFd::writable`].\n ///\n /// It is critical that this function not be called unless your code\n /// _actually observes_ that the file descriptor is _not_ ready. Do not call\n /// it simply because, for example, a read succeeded; it should be called\n /// when a read is observed to block.\n ///\n /// This method only clears readiness events that happened before the creation of this guard.\n /// In other words, if the IO resource becomes ready between the creation of the guard and\n /// this call to `clear_ready`, then the readiness is not actually cleared.\n pub fn clear_ready(&mut self) {\n if let Some(event) = self.event.take() {\n self.async_fd.registration.clear_readiness(event);\n }\n }\n\n /// Indicates to tokio that the file descriptor no longer has a specific readiness.\n /// The internal readiness flag will be cleared, and tokio will wait for the\n /// next edge-triggered readiness notification from the OS.\n ///\n /// This function is useful in combination with the [`AsyncFd::ready`] method when a\n /// combined interest like `Interest::READABLE | Interest::WRITABLE` is used.\n ///\n /// It is critical that this function not be called unless your code\n /// _actually observes_ that the file descriptor is _not_ ready for the provided `Ready`.\n /// Do not call it simply because, for example, a read succeeded; it should be called\n /// when a read is observed to block. Only clear the specific readiness that is observed to\n /// block. For example when a read blocks when using a combined interest,\n /// only clear `Ready::READABLE`.\n ///\n /// This method only clears readiness events that happened before the creation of this guard.\n /// In other words, if the IO resource becomes ready between the creation of the guard and\n /// this call to `clear_ready_matching`, then the readiness is not actually cleared.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to a [`std::net::TcpStream`] on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use std::error::Error;\n /// use std::io;\n /// use std::io::{Read, Write};\n /// use std::net::TcpStream;\n /// use tokio::io::unix::AsyncFd;\n /// use tokio::io::{Interest, Ready};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\")?;\n /// stream.set_nonblocking(true)?;\n /// let stream = AsyncFd::new(stream)?;\n ///\n /// loop {\n /// let mut guard = stream\n /// .ready(Interest::READABLE | Interest::WRITABLE)\n /// .await?;\n ///\n /// if guard.ready().is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.get_ref().read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a read has blocked, but a write might still succeed.\n /// // clear only the read readiness.\n /// guard.clear_ready_matching(Ready::READABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if guard.ready().is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.get_ref().write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a write has blocked, but a read might still succeed.\n /// // clear only the write readiness.\n /// guard.clear_ready_matching(Ready::WRITABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub fn clear_ready_matching(&mut self, ready: Ready) {\n if let Some(mut event) = self.event.take() {\n self.async_fd\n .registration\n .clear_readiness(event.with_ready(ready));\n\n // the event is no longer ready for the readiness that was just cleared\n event.ready = event.ready - ready;\n\n if !event.ready.is_empty() {\n self.event = Some(event);\n }\n }\n }\n\n /// This method should be invoked when you intentionally want to keep the\n /// ready flag asserted.\n ///\n /// While this function is itself a no-op, it satisfies the `#[must_use]`\n /// constraint on the [`AsyncFdReadyGuard`] type.\n pub fn retain_ready(&mut self) {\n // no-op\n }\n\n /// Get the [`Ready`] value associated with this guard.\n ///\n /// This method will return the empty readiness state if\n /// [`AsyncFdReadyGuard::clear_ready`] has been called on\n /// the guard.\n ///\n /// [`Ready`]: crate::io::Ready\n pub fn ready(&self) -> Ready {\n match &self.event {\n Some(event) => event.ready,\n None => Ready::EMPTY,\n }\n }\n\n /// Performs the provided IO operation.\n ///\n /// If `f` returns a [`WouldBlock`] error, the readiness state associated\n /// with this file descriptor is cleared, and the method returns\n /// `Err(TryIoError::WouldBlock)`. You will typically need to poll the\n /// `AsyncFd` again when this happens.\n ///\n /// This method helps ensure that the readiness state of the underlying file\n /// descriptor remains in sync with the tokio-side readiness state, by\n /// clearing the tokio-side state only when a [`WouldBlock`] condition\n /// occurs. It is the responsibility of the caller to ensure that `f`\n /// returns [`WouldBlock`] only if the file descriptor that originated this\n /// `AsyncFdReadyGuard` no longer expresses the readiness state that was queried to\n /// create this `AsyncFdReadyGuard`.\n ///\n /// # Examples\n ///\n /// This example sends some bytes to the inner [`std::net::UdpSocket`]. Waiting\n /// for write-readiness and retrying when the send operation does block are explicit.\n /// This example can be written more succinctly using [`AsyncFd::async_io`].\n ///\n /// ```no_run\n /// use tokio::io::unix::AsyncFd;\n ///\n /// use std::io;\n /// use std::net::UdpSocket;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"0.0.0.0:8080\")?;\n /// socket.set_nonblocking(true)?;\n /// let async_fd = AsyncFd::new(socket)?;\n ///\n /// let written = loop {\n /// let mut guard = async_fd.writable().await?;\n /// match guard.try_io(|inner| inner.get_ref().send(&[1, 2])) {\n /// Ok(result) => {\n /// break result?;\n /// }\n /// Err(_would_block) => {\n /// // try_io already cleared the file descriptor's readiness state\n /// continue;\n /// }\n /// }\n /// };\n ///\n /// println!(\"wrote {written} bytes\");\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`WouldBlock`]: std::io::ErrorKind::WouldBlock\n // Alias for old name in 0.x\n #[cfg_attr(docsrs, doc(alias = \"with_io\"))]\n pub fn try_io(\n &mut self,\n f: impl FnOnce(&'a AsyncFd) -> io::Result,\n ) -> Result, TryIoError> {\n let result = f(self.async_fd);\n\n match result {\n Err(err) if err.kind() == io::ErrorKind::WouldBlock => {\n self.clear_ready();\n Err(TryIoError(()))\n }\n result => Ok(result),\n }\n }\n\n /// Returns a shared reference to the inner [`AsyncFd`].\n pub fn get_ref(&self) -> &'a AsyncFd {\n self.async_fd\n }\n\n /// Returns a shared reference to the backing object of the inner [`AsyncFd`].\n pub fn get_inner(&self) -> &'a Inner {\n self.get_ref().get_ref()\n }\n}\n\nimpl<'a, Inner: AsRawFd> AsyncFdReadyMutGuard<'a, Inner> {\n /// Indicates to tokio that the file descriptor is no longer ready. All\n /// internal readiness flags will be cleared, and tokio will wait for the\n /// next edge-triggered readiness notification from the OS.\n ///\n /// This function is commonly used with guards returned by [`AsyncFd::readable_mut`] and\n /// [`AsyncFd::writable_mut`].\n ///\n /// It is critical that this function not be called unless your code\n /// _actually observes_ that the file descriptor is _not_ ready. Do not call\n /// it simply because, for example, a read succeeded; it should be called\n /// when a read is observed to block.\n ///\n /// This method only clears readiness events that happened before the creation of this guard.\n /// In other words, if the IO resource becomes ready between the creation of the guard and\n /// this call to `clear_ready`, then the readiness is not actually cleared.\n pub fn clear_ready(&mut self) {\n if let Some(event) = self.event.take() {\n self.async_fd.registration.clear_readiness(event);\n }\n }\n\n /// Indicates to tokio that the file descriptor no longer has a specific readiness.\n /// The internal readiness flag will be cleared, and tokio will wait for the\n /// next edge-triggered readiness notification from the OS.\n ///\n /// This function is useful in combination with the [`AsyncFd::ready_mut`] method when a\n /// combined interest like `Interest::READABLE | Interest::WRITABLE` is used.\n ///\n /// It is critical that this function not be called unless your code\n /// _actually observes_ that the file descriptor is _not_ ready for the provided `Ready`.\n /// Do not call it simply because, for example, a read succeeded; it should be called\n /// when a read is observed to block. Only clear the specific readiness that is observed to\n /// block. For example when a read blocks when using a combined interest,\n /// only clear `Ready::READABLE`.\n ///\n /// This method only clears readiness events that happened before the creation of this guard.\n /// In other words, if the IO resource becomes ready between the creation of the guard and\n /// this call to `clear_ready_matching`, then the readiness is not actually cleared.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to a [`std::net::TcpStream`] on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use std::error::Error;\n /// use std::io;\n /// use std::io::{Read, Write};\n /// use std::net::TcpStream;\n /// use tokio::io::unix::AsyncFd;\n /// use tokio::io::{Interest, Ready};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\")?;\n /// stream.set_nonblocking(true)?;\n /// let mut stream = AsyncFd::new(stream)?;\n ///\n /// loop {\n /// let mut guard = stream\n /// .ready_mut(Interest::READABLE | Interest::WRITABLE)\n /// .await?;\n ///\n /// if guard.ready().is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match guard.get_inner_mut().read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a read has blocked, but a write might still succeed.\n /// // clear only the read readiness.\n /// guard.clear_ready_matching(Ready::READABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if guard.ready().is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match guard.get_inner_mut().write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // a write has blocked, but a read might still succeed.\n /// // clear only the write readiness.\n /// guard.clear_ready_matching(Ready::WRITABLE);\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub fn clear_ready_matching(&mut self, ready: Ready) {\n if let Some(mut event) = self.event.take() {\n self.async_fd\n .registration\n .clear_readiness(event.with_ready(ready));\n\n // the event is no longer ready for the readiness that was just cleared\n event.ready = event.ready - ready;\n\n if !event.ready.is_empty() {\n self.event = Some(event);\n }\n }\n }\n\n /// This method should be invoked when you intentionally want to keep the\n /// ready flag asserted.\n ///\n /// While this function is itself a no-op, it satisfies the `#[must_use]`\n /// constraint on the [`AsyncFdReadyGuard`] type.\n pub fn retain_ready(&mut self) {\n // no-op\n }\n\n /// Get the [`Ready`] value associated with this guard.\n ///\n /// This method will return the empty readiness state if\n /// [`AsyncFdReadyGuard::clear_ready`] has been called on\n /// the guard.\n ///\n /// [`Ready`]: super::Ready\n pub fn ready(&self) -> Ready {\n match &self.event {\n Some(event) => event.ready,\n None => Ready::EMPTY,\n }\n }\n\n /// Performs the provided IO operation.\n ///\n /// If `f` returns a [`WouldBlock`] error, the readiness state associated\n /// with this file descriptor is cleared, and the method returns\n /// `Err(TryIoError::WouldBlock)`. You will typically need to poll the\n /// `AsyncFd` again when this happens.\n ///\n /// This method helps ensure that the readiness state of the underlying file\n /// descriptor remains in sync with the tokio-side readiness state, by\n /// clearing the tokio-side state only when a [`WouldBlock`] condition\n /// occurs. It is the responsibility of the caller to ensure that `f`\n /// returns [`WouldBlock`] only if the file descriptor that originated this\n /// `AsyncFdReadyGuard` no longer expresses the readiness state that was queried to\n /// create this `AsyncFdReadyGuard`.\n ///\n /// [`WouldBlock`]: std::io::ErrorKind::WouldBlock\n pub fn try_io(\n &mut self,\n f: impl FnOnce(&mut AsyncFd) -> io::Result,\n ) -> Result, TryIoError> {\n let result = f(self.async_fd);\n\n match result {\n Err(err) if err.kind() == io::ErrorKind::WouldBlock => {\n self.clear_ready();\n Err(TryIoError(()))\n }\n result => Ok(result),\n }\n }\n\n /// Returns a shared reference to the inner [`AsyncFd`].\n pub fn get_ref(&self) -> &AsyncFd {\n self.async_fd\n }\n\n /// Returns a mutable reference to the inner [`AsyncFd`].\n pub fn get_mut(&mut self) -> &mut AsyncFd {\n self.async_fd\n }\n\n /// Returns a shared reference to the backing object of the inner [`AsyncFd`].\n pub fn get_inner(&self) -> &Inner {\n self.get_ref().get_ref()\n }\n\n /// Returns a mutable reference to the backing object of the inner [`AsyncFd`].\n pub fn get_inner_mut(&mut self) -> &mut Inner {\n self.get_mut().get_mut()\n }\n}\n\nimpl<'a, T: std::fmt::Debug + AsRawFd> std::fmt::Debug for AsyncFdReadyGuard<'a, T> {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"ReadyGuard\")\n .field(\"async_fd\", &self.async_fd)\n .finish()\n }\n}\n\nimpl<'a, T: std::fmt::Debug + AsRawFd> std::fmt::Debug for AsyncFdReadyMutGuard<'a, T> {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"MutReadyGuard\")\n .field(\"async_fd\", &self.async_fd)\n .finish()\n }\n}\n\n/// The error type returned by [`try_io`].\n///\n/// This error indicates that the IO resource returned a [`WouldBlock`] error.\n///\n/// [`WouldBlock`]: std::io::ErrorKind::WouldBlock\n/// [`try_io`]: method@AsyncFdReadyGuard::try_io\n#[derive(Debug)]\npub struct TryIoError(());\n\n/// Error returned by [`try_new`] or [`try_with_interest`].\n///\n/// [`try_new`]: AsyncFd::try_new\n/// [`try_with_interest`]: AsyncFd::try_with_interest\npub struct AsyncFdTryNewError {\n inner: T,\n cause: io::Error,\n}\n\nimpl AsyncFdTryNewError {\n /// Returns the original object passed to [`try_new`] or [`try_with_interest`]\n /// alongside the error that caused these functions to fail.\n ///\n /// [`try_new`]: AsyncFd::try_new\n /// [`try_with_interest`]: AsyncFd::try_with_interest\n pub fn into_parts(self) -> (T, io::Error) {\n (self.inner, self.cause)\n }\n}\n\nimpl fmt::Display for AsyncFdTryNewError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&self.cause, f)\n }\n}\n\nimpl fmt::Debug for AsyncFdTryNewError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&self.cause, f)\n }\n}\n\nimpl Error for AsyncFdTryNewError {\n fn source(&self) -> Option<&(dyn Error + 'static)> {\n Some(&self.cause)\n }\n}\n\nimpl From> for io::Error {\n fn from(value: AsyncFdTryNewError) -> Self {\n value.cause\n }\n}\n" }, { "path": "tokio/src/io/async_read.rs", "content": "use super::ReadBuf;\nuse std::io;\nuse std::ops::DerefMut;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Reads bytes from a source.\n///\n/// This trait is analogous to the [`std::io::Read`] trait, but integrates with\n/// the asynchronous task system. In particular, the [`poll_read`] method,\n/// unlike [`Read::read`], will automatically queue the current task for wakeup\n/// and return if data is not yet available, rather than blocking the calling\n/// thread.\n///\n/// Specifically, this means that the `poll_read` function will return one of\n/// the following:\n///\n/// * `Poll::Ready(Ok(()))` means that data was immediately read and placed into\n/// the output buffer. The amount of data read can be determined by the\n/// increase in the length of the slice returned by `ReadBuf::filled`. If the\n/// difference is 0, either EOF has been reached, or the output buffer had zero\n/// capacity (i.e. `buf.remaining()` == 0).\n///\n/// * `Poll::Pending` means that no data was read into the buffer\n/// provided. The I/O object is not currently readable but may become readable\n/// in the future. Most importantly, **the current future's task is scheduled\n/// to get unparked when the object is readable**. This means that like\n/// `Future::poll` you'll receive a notification when the I/O object is\n/// readable again.\n///\n/// * `Poll::Ready(Err(e))` for other errors are standard I/O errors coming from the\n/// underlying object.\n///\n/// This trait importantly means that the `read` method only works in the\n/// context of a future's task. The object may panic if used outside of a task.\n///\n/// Utilities for working with `AsyncRead` values are provided by\n/// [`AsyncReadExt`].\n///\n/// [`poll_read`]: AsyncRead::poll_read\n/// [`std::io::Read`]: std::io::Read\n/// [`Read::read`]: std::io::Read::read\n/// [`AsyncReadExt`]: crate::io::AsyncReadExt\npub trait AsyncRead {\n /// Attempts to read from the `AsyncRead` into `buf`.\n ///\n /// On success, returns `Poll::Ready(Ok(()))` and places data in the\n /// unfilled portion of `buf`. If no data was read (`buf.filled().len()` is\n /// unchanged), it implies that EOF has been reached, or the output buffer\n /// had zero capacity (i.e. `buf.remaining()` == 0).\n ///\n /// If no data is available for reading, the method returns `Poll::Pending`\n /// and arranges for the current task (via `cx.waker()`) to receive a\n /// notification when the object becomes readable or is closed.\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll>;\n}\n\nmacro_rules! deref_async_read {\n () => {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut **self).poll_read(cx, buf)\n }\n };\n}\n\nimpl AsyncRead for Box {\n deref_async_read!();\n}\n\nimpl AsyncRead for &mut T {\n deref_async_read!();\n}\n\nimpl

AsyncRead for Pin

\nwhere\n P: DerefMut,\n P::Target: AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_read(cx, buf)\n }\n}\n\nimpl AsyncRead for &[u8] {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let amt = std::cmp::min(self.len(), buf.remaining());\n let (a, b) = self.split_at(amt);\n buf.put_slice(a);\n *self = b;\n Poll::Ready(Ok(()))\n }\n}\n\nimpl + Unpin> AsyncRead for io::Cursor {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let pos = self.position();\n let slice: &[u8] = (*self).get_ref().as_ref();\n\n // The position could technically be out of bounds, so don't panic...\n if pos > slice.len() as u64 {\n return Poll::Ready(Ok(()));\n }\n\n let start = pos as usize;\n let amt = std::cmp::min(slice.len() - start, buf.remaining());\n // Add won't overflow because of pos check above.\n let end = start + amt;\n buf.put_slice(&slice[start..end]);\n self.set_position(end as u64);\n\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio/src/io/async_seek.rs", "content": "use std::io::{self, SeekFrom};\nuse std::ops::DerefMut;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Seek bytes asynchronously.\n///\n/// This trait is analogous to the [`std::io::Seek`] trait, but integrates\n/// with the asynchronous task system. In particular, the `start_seek`\n/// method, unlike [`Seek::seek`], will not block the calling thread.\n///\n/// Utilities for working with `AsyncSeek` values are provided by\n/// [`AsyncSeekExt`].\n///\n/// [`std::io::Seek`]: std::io::Seek\n/// [`Seek::seek`]: std::io::Seek::seek()\n/// [`AsyncSeekExt`]: crate::io::AsyncSeekExt\npub trait AsyncSeek {\n /// Attempts to seek to an offset, in bytes, in a stream.\n ///\n /// A seek beyond the end of a stream is allowed, but behavior is defined\n /// by the implementation.\n ///\n /// If this function returns successfully, then the job has been submitted.\n /// To find out when it completes, call `poll_complete`.\n ///\n /// # Errors\n ///\n /// This function can return [`io::ErrorKind::Other`] in case there is\n /// another seek in progress. To avoid this, it is advisable that any call\n /// to `start_seek` is preceded by a call to `poll_complete` to ensure all\n /// pending seeks have completed.\n fn start_seek(self: Pin<&mut Self>, position: SeekFrom) -> io::Result<()>;\n\n /// Waits for a seek operation to complete.\n ///\n /// If the seek operation completed successfully, this method returns the\n /// new position from the start of the stream. That position can be used\n /// later with [`SeekFrom::Start`].\n ///\n /// The position returned by calling this method can only be relied on right\n /// after `start_seek`. If you have changed the position by e.g. reading or\n /// writing since calling `start_seek`, then it is unspecified whether the\n /// returned position takes that position change into account. Similarly, if\n /// `start_seek` has never been called, then it is unspecified whether\n /// `poll_complete` returns the actual position or some other placeholder\n /// value (such as 0).\n ///\n /// # Errors\n ///\n /// Seeking to a negative offset is considered an error.\n fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll>;\n}\n\nmacro_rules! deref_async_seek {\n () => {\n fn start_seek(mut self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n Pin::new(&mut **self).start_seek(pos)\n }\n\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut **self).poll_complete(cx)\n }\n };\n}\n\nimpl AsyncSeek for Box {\n deref_async_seek!();\n}\n\nimpl AsyncSeek for &mut T {\n deref_async_seek!();\n}\n\nimpl

AsyncSeek for Pin

\nwhere\n P: DerefMut,\n P::Target: AsyncSeek,\n{\n fn start_seek(self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n crate::util::pin_as_deref_mut(self).start_seek(pos)\n }\n\n fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_complete(cx)\n }\n}\n\nimpl + Unpin> AsyncSeek for io::Cursor {\n fn start_seek(mut self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n io::Seek::seek(&mut *self, pos).map(drop)\n }\n fn poll_complete(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(self.get_mut().position()))\n }\n}\n" }, { "path": "tokio/src/io/async_write.rs", "content": "use std::io::{self, IoSlice};\nuse std::ops::DerefMut;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Writes bytes asynchronously.\n///\n/// This trait is analogous to the [`std::io::Write`] trait, but integrates with\n/// the asynchronous task system. In particular, the [`poll_write`] method,\n/// unlike [`Write::write`], will automatically queue the current task for wakeup\n/// and return if data is not yet available, rather than blocking the calling\n/// thread.\n///\n/// Specifically, this means that the [`poll_write`] function will return one of\n/// the following:\n///\n/// * `Poll::Ready(Ok(n))` means that `n` bytes of data was immediately\n/// written.\n///\n/// * `Poll::Pending` means that no data was written from the buffer\n/// provided. The I/O object is not currently writable but may become writable\n/// in the future. Most importantly, **the current future's task is scheduled\n/// to get unparked when the object is writable**. This means that like\n/// `Future::poll` you'll receive a notification when the I/O object is\n/// writable again.\n///\n/// * `Poll::Ready(Err(e))` for other errors are standard I/O errors coming from the\n/// underlying object.\n///\n/// Utilities for working with `AsyncWrite` values are provided by\n/// [`AsyncWriteExt`]. Most users will interact with `AsyncWrite` types through\n/// these extension methods, which provide ergonomic async functions such as\n/// `write_all` and `flush`.\n///\n/// [`std::io::Write`]: std::io::Write\n/// [`Write::write`]: std::io::Write::write()\n/// [`poll_write`]: AsyncWrite::poll_write()\n/// [`AsyncWriteExt`]: crate::io::AsyncWriteExt\npub trait AsyncWrite {\n /// Attempt to write bytes from `buf` into the object.\n ///\n /// On success, returns `Poll::Ready(Ok(num_bytes_written))`. If successful,\n /// then it must be guaranteed that `n <= buf.len()`. A return value of `0`\n /// typically means that the underlying object is no longer able to accept\n /// bytes and will likely not be able to in the future as well, or that the\n /// buffer provided is empty.\n ///\n /// If the object is not ready for writing, the method returns\n /// `Poll::Pending` and arranges for the current task (via\n /// `cx.waker()`) to receive a notification when the object becomes\n /// writable or is closed.\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll>;\n\n /// Attempts to flush the object, ensuring that any buffered data reach\n /// their destination.\n ///\n /// On success, returns `Poll::Ready(Ok(()))`.\n ///\n /// If flushing cannot immediately complete, this method returns\n /// `Poll::Pending` and arranges for the current task (via\n /// `cx.waker()`) to receive a notification when the object can make\n /// progress towards flushing.\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll>;\n\n /// Initiates or attempts to shut down this writer, returning success when\n /// the I/O connection has completely shut down.\n ///\n /// This method is intended to be used for asynchronous shutdown of I/O\n /// connections. For example this is suitable for implementing shutdown of a\n /// TLS connection or calling `TcpStream::shutdown` on a proxied connection.\n /// Protocols sometimes need to flush out final pieces of data or otherwise\n /// perform a graceful shutdown handshake, reading/writing more data as\n /// appropriate. This method is the hook for such protocols to implement the\n /// graceful shutdown logic.\n ///\n /// This `shutdown` method is required by implementers of the\n /// `AsyncWrite` trait. Wrappers typically just want to proxy this call\n /// through to the wrapped type, and base types will typically implement\n /// shutdown logic here or just return `Ok(().into())`. Note that if you're\n /// wrapping an underlying `AsyncWrite` a call to `shutdown` implies that\n /// transitively the entire stream has been shut down. After your wrapper's\n /// shutdown logic has been executed you should shut down the underlying\n /// stream.\n ///\n /// Invocation of a `shutdown` implies an invocation of `flush`. Once this\n /// method returns `Ready` it implies that a flush successfully happened\n /// before the shutdown happened. That is, callers don't need to call\n /// `flush` before calling `shutdown`. They can rely that by calling\n /// `shutdown` any pending buffered data will be written out.\n ///\n /// # Return value\n ///\n /// This function returns a `Poll>` classified as such:\n ///\n /// * `Poll::Ready(Ok(()))` - indicates that the connection was\n /// successfully shut down and is now safe to deallocate/drop/close\n /// resources associated with it. This method means that the current task\n /// will no longer receive any notifications due to this method and the\n /// I/O object itself is likely no longer usable.\n ///\n /// * `Poll::Pending` - indicates that shutdown is initiated but could\n /// not complete just yet. This may mean that more I/O needs to happen to\n /// continue this shutdown operation. The current task is scheduled to\n /// receive a notification when it's otherwise ready to continue the\n /// shutdown operation. When woken up this method should be called again.\n ///\n /// * `Poll::Ready(Err(e))` - indicates a fatal error has happened with shutdown,\n /// indicating that the shutdown operation did not complete successfully.\n /// This typically means that the I/O object is no longer usable.\n ///\n /// # Errors\n ///\n /// This function can return normal I/O errors through `Err`, described\n /// above. Additionally this method may also render the underlying\n /// `Write::write` method no longer usable (e.g. will return errors in the\n /// future). It's recommended that once `shutdown` is called the\n /// `write` method is no longer called.\n ///\n /// # Panics\n ///\n /// This function will panic if not called within the context of a future's\n /// task.\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll>;\n\n /// Like [`poll_write`], except that it writes from a slice of buffers.\n ///\n /// Data is copied from each buffer in order, with the final buffer\n /// read from possibly being only partially consumed. This method must\n /// behave as a call to [`write`] with the buffers concatenated would.\n ///\n /// The default implementation calls [`poll_write`] with either the first nonempty\n /// buffer provided, or an empty one if none exists.\n ///\n /// On success, returns `Poll::Ready(Ok(num_bytes_written))`.\n ///\n /// If the object is not ready for writing, the method returns\n /// `Poll::Pending` and arranges for the current task (via\n /// `cx.waker()`) to receive a notification when the object becomes\n /// writable or is closed.\n ///\n /// # Note\n ///\n /// This should be implemented as a single \"atomic\" write action. If any\n /// data has been partially written, it is wrong to return an error or\n /// pending.\n ///\n /// [`poll_write`]: AsyncWrite::poll_write\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n let buf = bufs\n .iter()\n .find(|b| !b.is_empty())\n .map_or(&[][..], |b| &**b);\n self.poll_write(cx, buf)\n }\n\n /// Determines if this writer has an efficient [`poll_write_vectored`]\n /// implementation.\n ///\n /// If a writer does not override the default [`poll_write_vectored`]\n /// implementation, code using it may want to avoid the method all together\n /// and coalesce writes into a single buffer for higher performance.\n ///\n /// The default implementation returns `false`.\n ///\n /// [`poll_write_vectored`]: AsyncWrite::poll_write_vectored\n fn is_write_vectored(&self) -> bool {\n false\n }\n}\n\nmacro_rules! deref_async_write {\n () => {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut **self).poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Pin::new(&mut **self).poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n (**self).is_write_vectored()\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut **self).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut **self).poll_shutdown(cx)\n }\n };\n}\n\nimpl AsyncWrite for Box {\n deref_async_write!();\n}\n\nimpl AsyncWrite for &mut T {\n deref_async_write!();\n}\n\nimpl

AsyncWrite for Pin

\nwhere\n P: DerefMut,\n P::Target: AsyncWrite,\n{\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n (**self).is_write_vectored()\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n crate::util::pin_as_deref_mut(self).poll_shutdown(cx)\n }\n}\n\nimpl AsyncWrite for Vec {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.get_mut().extend_from_slice(buf);\n Poll::Ready(Ok(buf.len()))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(io::Write::write_vectored(&mut *self, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncWrite for io::Cursor<&mut [u8]> {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Poll::Ready(io::Write::write(&mut *self, buf))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(io::Write::write_vectored(&mut *self, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(io::Write::flush(&mut *self))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n\nimpl AsyncWrite for io::Cursor<&mut Vec> {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Poll::Ready(io::Write::write(&mut *self, buf))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(io::Write::write_vectored(&mut *self, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(io::Write::flush(&mut *self))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n\nimpl AsyncWrite for io::Cursor> {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Poll::Ready(io::Write::write(&mut *self, buf))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(io::Write::write_vectored(&mut *self, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(io::Write::flush(&mut *self))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n\nimpl AsyncWrite for io::Cursor> {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Poll::Ready(io::Write::write(&mut *self, buf))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(io::Write::write_vectored(&mut *self, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(io::Write::flush(&mut *self))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n" }, { "path": "tokio/src/io/blocking.rs", "content": "use crate::io::sys;\nuse crate::io::{AsyncRead, AsyncWrite, ReadBuf};\n\nuse std::cmp;\nuse std::future::Future;\nuse std::io;\nuse std::io::prelude::*;\nuse std::mem::MaybeUninit;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// `T` should not implement _both_ Read and Write.\n#[derive(Debug)]\npub(crate) struct Blocking {\n inner: Option,\n state: State,\n /// `true` if the lower IO layer needs flushing.\n need_flush: bool,\n}\n\n#[derive(Debug)]\npub(crate) struct Buf {\n buf: Vec,\n pos: usize,\n}\n\npub(crate) const DEFAULT_MAX_BUF_SIZE: usize = 2 * 1024 * 1024;\n\n#[derive(Debug)]\nenum State {\n Idle(Option),\n Busy(sys::Blocking<(io::Result, Buf, T)>),\n}\n\ncfg_io_blocking! {\n impl Blocking {\n /// # Safety\n ///\n /// The `Read` implementation of `inner` must never read from the buffer\n /// it is borrowing and must correctly report the length of the data\n /// written into the buffer.\n #[cfg_attr(feature = \"fs\", allow(dead_code))]\n pub(crate) unsafe fn new(inner: T) -> Blocking {\n Blocking {\n inner: Some(inner),\n state: State::Idle(Some(Buf::with_capacity(0))),\n need_flush: false,\n }\n }\n }\n}\n\nimpl AsyncRead for Blocking\nwhere\n T: Read + Unpin + Send + 'static,\n{\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n dst: &mut ReadBuf<'_>,\n ) -> Poll> {\n loop {\n match self.state {\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n if !buf.is_empty() {\n buf.copy_to(dst);\n *buf_cell = Some(buf);\n return Poll::Ready(Ok(()));\n }\n\n let mut inner = self.inner.take().unwrap();\n\n let max_buf_size = cmp::min(dst.remaining(), DEFAULT_MAX_BUF_SIZE);\n self.state = State::Busy(sys::run(move || {\n // SAFETY: the requirements are satisfied by `Blocking::new`.\n let res = unsafe { buf.read_from(&mut inner, max_buf_size) };\n (res, buf, inner)\n }));\n }\n State::Busy(ref mut rx) => {\n let (res, mut buf, inner) = ready!(Pin::new(rx).poll(cx))?;\n self.inner = Some(inner);\n\n match res {\n Ok(_) => {\n buf.copy_to(dst);\n self.state = State::Idle(Some(buf));\n return Poll::Ready(Ok(()));\n }\n Err(e) => {\n assert!(buf.is_empty());\n\n self.state = State::Idle(Some(buf));\n return Poll::Ready(Err(e));\n }\n }\n }\n }\n }\n }\n}\n\nimpl AsyncWrite for Blocking\nwhere\n T: Write + Unpin + Send + 'static,\n{\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n src: &[u8],\n ) -> Poll> {\n loop {\n match self.state {\n State::Idle(ref mut buf_cell) => {\n let mut buf = buf_cell.take().unwrap();\n\n assert!(buf.is_empty());\n\n let n = buf.copy_from(src, DEFAULT_MAX_BUF_SIZE);\n let mut inner = self.inner.take().unwrap();\n\n self.state = State::Busy(sys::run(move || {\n let n = buf.len();\n let res = buf.write_to(&mut inner).map(|()| n);\n\n (res, buf, inner)\n }));\n self.need_flush = true;\n\n return Poll::Ready(Ok(n));\n }\n State::Busy(ref mut rx) => {\n let (res, buf, inner) = ready!(Pin::new(rx).poll(cx))?;\n self.state = State::Idle(Some(buf));\n self.inner = Some(inner);\n\n // If error, return\n res?;\n }\n }\n }\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n loop {\n let need_flush = self.need_flush;\n match self.state {\n // The buffer is not used here\n State::Idle(ref mut buf_cell) => {\n if need_flush {\n let buf = buf_cell.take().unwrap();\n let mut inner = self.inner.take().unwrap();\n\n self.state = State::Busy(sys::run(move || {\n let res = inner.flush().map(|()| 0);\n (res, buf, inner)\n }));\n\n self.need_flush = false;\n } else {\n return Poll::Ready(Ok(()));\n }\n }\n State::Busy(ref mut rx) => {\n let (res, buf, inner) = ready!(Pin::new(rx).poll(cx))?;\n self.state = State::Idle(Some(buf));\n self.inner = Some(inner);\n\n // If error, return\n res?;\n }\n }\n }\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\n/// Repeats operations that are interrupted.\nmacro_rules! uninterruptibly {\n ($e:expr) => {{\n loop {\n match $e {\n Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {}\n res => break res,\n }\n }\n }};\n}\n\nimpl Buf {\n pub(crate) fn with_capacity(n: usize) -> Buf {\n Buf {\n buf: Vec::with_capacity(n),\n pos: 0,\n }\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n pub(crate) fn len(&self) -> usize {\n self.buf.len() - self.pos\n }\n\n pub(crate) fn copy_to(&mut self, dst: &mut ReadBuf<'_>) -> usize {\n let n = cmp::min(self.len(), dst.remaining());\n dst.put_slice(&self.bytes()[..n]);\n self.pos += n;\n\n if self.pos == self.buf.len() {\n self.buf.truncate(0);\n self.pos = 0;\n }\n\n n\n }\n\n pub(crate) fn copy_from(&mut self, src: &[u8], max_buf_size: usize) -> usize {\n assert!(self.is_empty());\n\n let n = cmp::min(src.len(), max_buf_size);\n\n self.buf.extend_from_slice(&src[..n]);\n n\n }\n\n pub(crate) fn bytes(&self) -> &[u8] {\n &self.buf[self.pos..]\n }\n\n /// # Safety\n ///\n /// `rd` must not read from the buffer `read` is borrowing and must correctly\n /// report the length of the data written into the buffer.\n pub(crate) unsafe fn read_from(\n &mut self,\n rd: &mut T,\n max_buf_size: usize,\n ) -> io::Result {\n assert!(self.is_empty());\n self.buf.reserve(max_buf_size);\n\n let buf = &mut self.buf.spare_capacity_mut()[..max_buf_size];\n // SAFETY: The memory may be uninitialized, but `rd.read` will only write to the buffer.\n let buf = unsafe { &mut *(buf as *mut [MaybeUninit] as *mut [u8]) };\n let res = uninterruptibly!(rd.read(buf));\n\n if let Ok(n) = res {\n // SAFETY: the caller promises that `rd.read` initializes\n // a section of `buf` and correctly reports that length.\n // The `self.is_empty()` assertion verifies that `n`\n // equals the length of the `buf` capacity that was written\n // to (and that `buf` isn't being shrunk).\n unsafe { self.buf.set_len(n) }\n } else {\n self.buf.clear();\n }\n\n assert_eq!(self.pos, 0);\n\n res\n }\n\n pub(crate) fn write_to(&mut self, wr: &mut T) -> io::Result<()> {\n assert_eq!(self.pos, 0);\n\n // `write_all` already ignores interrupts\n let res = wr.write_all(&self.buf);\n self.buf.clear();\n res\n }\n}\n\ncfg_fs! {\n impl Buf {\n pub(crate) fn discard_read(&mut self) -> i64 {\n let ret = -(self.bytes().len() as i64);\n self.pos = 0;\n self.buf.truncate(0);\n ret\n }\n\n pub(crate) fn copy_from_bufs(&mut self, bufs: &[io::IoSlice<'_>], max_buf_size: usize) -> usize {\n assert!(self.is_empty());\n\n let mut rem = max_buf_size;\n for buf in bufs {\n if rem == 0 {\n break\n }\n\n let len = buf.len().min(rem);\n self.buf.extend_from_slice(&buf[..len]);\n rem -= len;\n }\n\n max_buf_size - rem\n }\n }\n}\n" }, { "path": "tokio/src/io/bsd/poll_aio.rs", "content": "//! Use POSIX AIO futures with Tokio.\n\nuse crate::io::interest::Interest;\nuse crate::runtime::io::{ReadyEvent, Registration};\nuse crate::runtime::scheduler;\nuse mio::event::Source;\nuse mio::Registry;\nuse mio::Token;\nuse std::fmt;\nuse std::io;\nuse std::ops::{Deref, DerefMut};\nuse std::os::unix::io::AsRawFd;\nuse std::os::unix::prelude::RawFd;\nuse std::task::{ready, Context, Poll};\n\n/// Like [`mio::event::Source`], but for POSIX AIO only.\n///\n/// Tokio's consumer must pass an implementor of this trait to create a\n/// [`Aio`] object.\npub trait AioSource {\n /// Registers this AIO event source with Tokio's reactor.\n fn register(&mut self, kq: RawFd, token: usize);\n\n /// Deregisters this AIO event source with Tokio's reactor.\n fn deregister(&mut self);\n}\n\n/// Wraps the user's AioSource in order to implement mio::event::Source, which\n/// is what the rest of the crate wants.\nstruct MioSource(T);\n\nimpl Source for MioSource {\n fn register(\n &mut self,\n registry: &Registry,\n token: Token,\n interests: mio::Interest,\n ) -> io::Result<()> {\n assert!(interests.is_aio() || interests.is_lio());\n self.0.register(registry.as_raw_fd(), usize::from(token));\n Ok(())\n }\n\n fn deregister(&mut self, _registry: &Registry) -> io::Result<()> {\n self.0.deregister();\n Ok(())\n }\n\n fn reregister(\n &mut self,\n registry: &Registry,\n token: Token,\n interests: mio::Interest,\n ) -> io::Result<()> {\n assert!(interests.is_aio() || interests.is_lio());\n self.0.register(registry.as_raw_fd(), usize::from(token));\n Ok(())\n }\n}\n\n/// Associates a POSIX AIO control block with the reactor that drives it.\n///\n/// `Aio`'s wrapped type must implement [`AioSource`] to be driven\n/// by the reactor.\n///\n/// The wrapped source may be accessed through the `Aio` via the `Deref` and\n/// `DerefMut` traits.\n///\n/// ## Clearing readiness\n///\n/// If [`Aio::poll_ready`] returns ready, but the consumer determines that the\n/// Source is not completely ready and must return to the Pending state,\n/// [`Aio::clear_ready`] may be used. This can be useful with\n/// [`lio_listio`], which may generate a kevent when only a portion of the\n/// operations have completed.\n///\n/// ## Platforms\n///\n/// Only FreeBSD implements POSIX AIO with kqueue notification, so\n/// `Aio` is only available for that operating system.\n///\n/// [`lio_listio`]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/lio_listio.html\n// Note: Unlike every other kqueue event source, POSIX AIO registers events not\n// via kevent(2) but when the aiocb is submitted to the kernel via aio_read,\n// aio_write, etc. It needs the kqueue's file descriptor to do that. So\n// AsyncFd can't be used for POSIX AIO.\n//\n// Note that Aio doesn't implement Drop. There's no need. Unlike other\n// kqueue sources, simply dropping the object effectively deregisters it.\npub struct Aio {\n io: MioSource,\n registration: Registration,\n}\n\n// ===== impl Aio =====\n\nimpl Aio {\n /// Creates a new `Aio` suitable for use with POSIX AIO functions.\n ///\n /// It will be associated with the default reactor. The runtime is usually\n /// set implicitly when this function is called from a future driven by a\n /// Tokio runtime, otherwise runtime can be set explicitly with\n /// [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn new_for_aio(io: E) -> io::Result {\n Self::new_with_interest(io, Interest::AIO)\n }\n\n /// Creates a new `Aio` suitable for use with [`lio_listio`].\n ///\n /// It will be associated with the default reactor. The runtime is usually\n /// set implicitly when this function is called from a future driven by a\n /// Tokio runtime, otherwise runtime can be set explicitly with\n /// [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n ///\n /// [`lio_listio`]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/lio_listio.html\n pub fn new_for_lio(io: E) -> io::Result {\n Self::new_with_interest(io, Interest::LIO)\n }\n\n fn new_with_interest(io: E, interest: Interest) -> io::Result {\n let mut io = MioSource(io);\n let handle = scheduler::Handle::current();\n let registration = Registration::new_with_interest_and_handle(&mut io, interest, handle)?;\n Ok(Self { io, registration })\n }\n\n /// Indicates to Tokio that the source is no longer ready. The internal\n /// readiness flag will be cleared, and tokio will wait for the next\n /// edge-triggered readiness notification from the OS.\n ///\n /// It is critical that this method not be called unless your code\n /// _actually observes_ that the source is _not_ ready. The OS must\n /// deliver a subsequent notification, or this source will block\n /// forever. It is equally critical that you `do` call this method if you\n /// resubmit the same structure to the kernel and poll it again.\n ///\n /// This method is not very useful with AIO readiness, since each `aiocb`\n /// structure is typically only used once. It's main use with\n /// [`lio_listio`], which will sometimes send notification when only a\n /// portion of its elements are complete. In that case, the caller must\n /// call `clear_ready` before resubmitting it.\n ///\n /// [`lio_listio`]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/lio_listio.html\n pub fn clear_ready(&self, ev: AioEvent) {\n self.registration.clear_readiness(ev.0)\n }\n\n /// Destroy the [`Aio`] and return its inner source.\n pub fn into_inner(self) -> E {\n self.io.0\n }\n\n /// Polls for readiness. Either AIO or LIO counts.\n ///\n /// This method returns:\n /// * `Poll::Pending` if the underlying operation is not complete, whether\n /// or not it completed successfully. This will be true if the OS is\n /// still processing it, or if it has not yet been submitted to the OS.\n /// * `Poll::Ready(Ok(_))` if the underlying operation is complete.\n /// * `Poll::Ready(Err(_))` if the reactor has been shutdown. This does\n /// _not_ indicate that the underlying operation encountered an error.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided `Context`\n /// is scheduled to receive a wakeup when the underlying operation\n /// completes. Note that on multiple calls to `poll_ready`, only the `Waker` from the\n /// `Context` passed to the most recent call is scheduled to receive a wakeup.\n pub fn poll_ready(&self, cx: &mut Context<'_>) -> Poll> {\n let ev = ready!(self.registration.poll_read_ready(cx))?;\n Poll::Ready(Ok(AioEvent(ev)))\n }\n}\n\nimpl Deref for Aio {\n type Target = E;\n\n fn deref(&self) -> &E {\n &self.io.0\n }\n}\n\nimpl DerefMut for Aio {\n fn deref_mut(&mut self) -> &mut E {\n &mut self.io.0\n }\n}\n\nimpl fmt::Debug for Aio {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Aio\").field(\"io\", &self.io.0).finish()\n }\n}\n\n/// Opaque data returned by [`Aio::poll_ready`].\n///\n/// It can be fed back to [`Aio::clear_ready`].\n#[derive(Debug)]\npub struct AioEvent(ReadyEvent);\n" }, { "path": "tokio/src/io/interest.rs", "content": "#![cfg_attr(not(feature = \"net\"), allow(dead_code, unreachable_pub))]\n\nuse crate::io::ready::Ready;\n\nuse std::fmt;\nuse std::ops;\n\n// These must be unique.\n// same as mio\nconst READABLE: usize = 0b0001;\nconst WRITABLE: usize = 0b0010;\n// The following are not available on all platforms.\n#[cfg(target_os = \"freebsd\")]\nconst AIO: usize = 0b0100;\n#[cfg(target_os = \"freebsd\")]\nconst LIO: usize = 0b1000;\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nconst PRIORITY: usize = 0b0001_0000;\n// error is available on all platforms, but behavior is platform-specific\n// mio does not have this interest\nconst ERROR: usize = 0b0010_0000;\n\n/// Readiness event interest.\n///\n/// Specifies the readiness events the caller is interested in when awaiting on\n/// I/O resource readiness states.\n#[cfg_attr(docsrs, doc(cfg(feature = \"net\")))]\n#[derive(Clone, Copy, Eq, PartialEq)]\npub struct Interest(usize);\n\nimpl Interest {\n // The non-FreeBSD definitions in this block are active only when\n // building documentation.\n cfg_aio! {\n /// Interest for POSIX AIO.\n #[cfg(target_os = \"freebsd\")]\n pub const AIO: Interest = Interest(AIO);\n\n /// Interest for POSIX AIO.\n #[cfg(not(target_os = \"freebsd\"))]\n pub const AIO: Interest = Interest(READABLE);\n\n /// Interest for POSIX AIO `lio_listio` events.\n #[cfg(target_os = \"freebsd\")]\n pub const LIO: Interest = Interest(LIO);\n\n /// Interest for POSIX AIO `lio_listio` events.\n #[cfg(not(target_os = \"freebsd\"))]\n pub const LIO: Interest = Interest(READABLE);\n }\n\n /// Interest in all readable events.\n ///\n /// Readable interest includes read-closed events.\n pub const READABLE: Interest = Interest(READABLE);\n\n /// Interest in all writable events.\n ///\n /// Writable interest includes write-closed events.\n pub const WRITABLE: Interest = Interest(WRITABLE);\n\n /// Interest in error events.\n ///\n /// Passes error interest to the underlying OS selector.\n /// Behavior is platform-specific, read your platform's documentation.\n pub const ERROR: Interest = Interest(ERROR);\n\n /// Returns a `Interest` set representing priority completion interests.\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub const PRIORITY: Interest = Interest(PRIORITY);\n\n /// Returns true if the value includes readable interest.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// assert!(Interest::READABLE.is_readable());\n /// assert!(!Interest::WRITABLE.is_readable());\n ///\n /// let both = Interest::READABLE | Interest::WRITABLE;\n /// assert!(both.is_readable());\n /// ```\n pub const fn is_readable(self) -> bool {\n self.0 & READABLE != 0\n }\n\n /// Returns true if the value includes writable interest.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// assert!(!Interest::READABLE.is_writable());\n /// assert!(Interest::WRITABLE.is_writable());\n ///\n /// let both = Interest::READABLE | Interest::WRITABLE;\n /// assert!(both.is_writable());\n /// ```\n pub const fn is_writable(self) -> bool {\n self.0 & WRITABLE != 0\n }\n\n /// Returns true if the value includes error interest.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// assert!(Interest::ERROR.is_error());\n /// assert!(!Interest::WRITABLE.is_error());\n ///\n /// let combined = Interest::READABLE | Interest::ERROR;\n /// assert!(combined.is_error());\n /// ```\n pub const fn is_error(self) -> bool {\n self.0 & ERROR != 0\n }\n\n #[cfg(target_os = \"freebsd\")]\n const fn is_aio(self) -> bool {\n self.0 & AIO != 0\n }\n\n #[cfg(target_os = \"freebsd\")]\n const fn is_lio(self) -> bool {\n self.0 & LIO != 0\n }\n\n /// Returns true if the value includes priority interest.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// assert!(!Interest::READABLE.is_priority());\n /// assert!(Interest::PRIORITY.is_priority());\n ///\n /// let both = Interest::READABLE | Interest::PRIORITY;\n /// assert!(both.is_priority());\n /// ```\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub const fn is_priority(self) -> bool {\n self.0 & PRIORITY != 0\n }\n\n /// Add together two `Interest` values.\n ///\n /// This function works from a `const` context.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// const BOTH: Interest = Interest::READABLE.add(Interest::WRITABLE);\n ///\n /// assert!(BOTH.is_readable());\n /// assert!(BOTH.is_writable());\n #[must_use = \"this returns the result of the operation, without modifying the original\"]\n pub const fn add(self, other: Interest) -> Interest {\n Self(self.0 | other.0)\n }\n\n /// Remove `Interest` from `self`.\n ///\n /// Interests present in `other` but *not* in `self` are ignored.\n ///\n /// Returns `None` if the set would be empty after removing `Interest`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Interest;\n ///\n /// const RW_INTEREST: Interest = Interest::READABLE.add(Interest::WRITABLE);\n ///\n /// let w_interest = RW_INTEREST.remove(Interest::READABLE).unwrap();\n /// assert!(!w_interest.is_readable());\n /// assert!(w_interest.is_writable());\n ///\n /// // Removing all interests from the set returns `None`.\n /// assert_eq!(w_interest.remove(Interest::WRITABLE), None);\n ///\n /// // Remove all interests at once.\n /// assert_eq!(RW_INTEREST.remove(RW_INTEREST), None);\n /// ```\n #[must_use = \"this returns the result of the operation, without modifying the original\"]\n pub fn remove(self, other: Interest) -> Option {\n let value = self.0 & !other.0;\n\n if value != 0 {\n Some(Self(value))\n } else {\n None\n }\n }\n\n // This function must be crate-private to avoid exposing a `mio` dependency.\n pub(crate) fn to_mio(self) -> mio::Interest {\n fn mio_add(wrapped: &mut Option, add: mio::Interest) {\n match wrapped {\n Some(inner) => *inner |= add,\n None => *wrapped = Some(add),\n }\n }\n\n // mio does not allow and empty interest, so use None for empty\n let mut mio = None;\n\n if self.is_readable() {\n mio_add(&mut mio, mio::Interest::READABLE);\n }\n\n if self.is_writable() {\n mio_add(&mut mio, mio::Interest::WRITABLE);\n }\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n if self.is_priority() {\n mio_add(&mut mio, mio::Interest::PRIORITY);\n }\n\n #[cfg(target_os = \"freebsd\")]\n if self.is_aio() {\n mio_add(&mut mio, mio::Interest::AIO);\n }\n\n #[cfg(target_os = \"freebsd\")]\n if self.is_lio() {\n mio_add(&mut mio, mio::Interest::LIO);\n }\n\n if self.is_error() {\n // There is no error interest in mio, because error events are always reported.\n // But mio interests cannot be empty and an interest is needed just for the registration.\n //\n // read readiness is filtered out in `Interest::mask` or `Ready::from_interest` if\n // the read interest was not specified by the user.\n mio_add(&mut mio, mio::Interest::READABLE);\n }\n\n // the default `mio::Interest::READABLE` should never be used in practice. Either\n //\n // - at least one tokio interest with a mio counterpart was used\n // - only the error tokio interest was specified\n //\n // in both cases, `mio` is Some already\n mio.unwrap_or(mio::Interest::READABLE)\n }\n\n pub(crate) fn mask(self) -> Ready {\n match self {\n Interest::READABLE => Ready::READABLE | Ready::READ_CLOSED,\n Interest::WRITABLE => Ready::WRITABLE | Ready::WRITE_CLOSED,\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n Interest::PRIORITY => Ready::PRIORITY | Ready::READ_CLOSED,\n Interest::ERROR => Ready::ERROR,\n _ => Ready::EMPTY,\n }\n }\n}\n\nimpl ops::BitOr for Interest {\n type Output = Self;\n\n #[inline]\n fn bitor(self, other: Self) -> Self {\n self.add(other)\n }\n}\n\nimpl ops::BitOrAssign for Interest {\n #[inline]\n fn bitor_assign(&mut self, other: Self) {\n *self = *self | other;\n }\n}\n\nimpl fmt::Debug for Interest {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n let mut separator = false;\n\n if self.is_readable() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"READABLE\")?;\n separator = true;\n }\n\n if self.is_writable() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"WRITABLE\")?;\n separator = true;\n }\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n if self.is_priority() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"PRIORITY\")?;\n separator = true;\n }\n\n #[cfg(target_os = \"freebsd\")]\n if self.is_aio() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"AIO\")?;\n separator = true;\n }\n\n #[cfg(target_os = \"freebsd\")]\n if self.is_lio() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"LIO\")?;\n separator = true;\n }\n\n if self.is_error() {\n if separator {\n write!(fmt, \" | \")?;\n }\n write!(fmt, \"ERROR\")?;\n separator = true;\n }\n\n let _ = separator;\n\n Ok(())\n }\n}\n" }, { "path": "tokio/src/io/join.rs", "content": "//! Join two values implementing `AsyncRead` and `AsyncWrite` into a single one.\n\nuse crate::io::{AsyncBufRead, AsyncRead, AsyncWrite, ReadBuf};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Join two values implementing `AsyncRead` and `AsyncWrite` into a\n/// single handle.\npub fn join(reader: R, writer: W) -> Join\nwhere\n R: AsyncRead,\n W: AsyncWrite,\n{\n Join { reader, writer }\n}\n\npin_project_lite::pin_project! {\n /// Joins two values implementing `AsyncRead` and `AsyncWrite` into a\n /// single handle.\n #[derive(Debug)]\n pub struct Join {\n #[pin]\n reader: R,\n #[pin]\n writer: W,\n }\n}\n\nimpl Join\nwhere\n R: AsyncRead,\n W: AsyncWrite,\n{\n /// Splits this `Join` back into its `AsyncRead` and `AsyncWrite`\n /// components.\n pub fn into_inner(self) -> (R, W) {\n (self.reader, self.writer)\n }\n\n /// Returns a reference to the inner reader.\n pub fn reader(&self) -> &R {\n &self.reader\n }\n\n /// Returns a reference to the inner writer.\n pub fn writer(&self) -> &W {\n &self.writer\n }\n\n /// Returns a mutable reference to the inner reader.\n pub fn reader_mut(&mut self) -> &mut R {\n &mut self.reader\n }\n\n /// Returns a mutable reference to the inner writer.\n pub fn writer_mut(&mut self) -> &mut W {\n &mut self.writer\n }\n\n /// Returns a pinned mutable reference to the inner reader.\n pub fn reader_pin_mut(self: Pin<&mut Self>) -> Pin<&mut R> {\n self.project().reader\n }\n\n /// Returns a pinned mutable reference to the inner writer.\n pub fn writer_pin_mut(self: Pin<&mut Self>) -> Pin<&mut W> {\n self.project().writer\n }\n}\n\nimpl AsyncRead for Join\nwhere\n R: AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.project().reader.poll_read(cx, buf)\n }\n}\n\nimpl AsyncWrite for Join\nwhere\n W: AsyncWrite,\n{\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.project().writer.poll_write(cx, buf)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().writer.poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().writer.poll_shutdown(cx)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.project().writer.poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.writer.is_write_vectored()\n }\n}\n\nimpl AsyncBufRead for Join\nwhere\n R: AsyncBufRead,\n{\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().reader.poll_fill_buf(cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n self.project().reader.consume(amt)\n }\n}\n" }, { "path": "tokio/src/io/mod.rs", "content": "//! Traits, helpers, and type definitions for asynchronous I/O functionality.\n//!\n//! This module is the asynchronous version of `std::io`. Primarily, it\n//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which are asynchronous\n//! versions of the [`Read`] and [`Write`] traits in the standard library.\n//!\n//! # `AsyncRead` and `AsyncWrite`\n//!\n//! Like the standard library's [`Read`] and [`Write`] traits, [`AsyncRead`] and\n//! [`AsyncWrite`] provide the most general interface for reading and writing\n//! input and output. Unlike the standard library's traits, however, they are\n//! _asynchronous_ — meaning that reading from or writing to a `tokio::io`\n//! type will _yield_ to the Tokio scheduler when IO is not ready, rather than\n//! blocking. This allows other tasks to run while waiting on IO.\n//!\n//! Another difference is that `AsyncRead` and `AsyncWrite` only contain\n//! core methods needed to provide asynchronous reading and writing\n//! functionality. Instead, utility methods are defined in the [`AsyncReadExt`]\n//! and [`AsyncWriteExt`] extension traits. These traits are automatically\n//! implemented for all values that implement `AsyncRead` and `AsyncWrite`\n//! respectively.\n//!\n//! End users will rarely interact directly with `AsyncRead` and\n//! `AsyncWrite`. Instead, they will use the async functions defined in the\n//! extension traits. Library authors are expected to implement `AsyncRead`\n//! and `AsyncWrite` in order to provide types that behave like byte streams.\n//!\n//! Even with these differences, Tokio's `AsyncRead` and `AsyncWrite` traits\n//! can be used in almost exactly the same manner as the standard library's\n//! `Read` and `Write`. Most types in the standard library that implement `Read`\n//! and `Write` have asynchronous equivalents in `tokio` that implement\n//! `AsyncRead` and `AsyncWrite`, such as [`File`] and [`TcpStream`].\n//!\n//! For example, the standard library documentation introduces `Read` by\n//! [demonstrating][std_example] reading some bytes from a [`std::fs::File`]. We\n//! can do the same with [`tokio::fs::File`][`File`]:\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::io::{self, AsyncReadExt};\n//! use tokio::fs::File;\n//!\n//! #[tokio::main]\n//! async fn main() -> io::Result<()> {\n//! let mut f = File::open(\"foo.txt\").await?;\n//! let mut buffer = [0; 10];\n//!\n//! // read up to 10 bytes\n//! let n = f.read(&mut buffer).await?;\n//!\n//! println!(\"The bytes: {:?}\", &buffer[..n]);\n//! Ok(())\n//! }\n//! # }\n//! ```\n//!\n//! [`File`]: crate::fs::File\n//! [`TcpStream`]: crate::net::TcpStream\n//! [`std::fs::File`]: std::fs::File\n//! [std_example]: std::io#read-and-write\n//!\n//! ## Buffered Readers and Writers\n//!\n//! Byte-based interfaces are unwieldy and can be inefficient, as we'd need to be\n//! making near-constant calls to the operating system. To help with this,\n//! `std::io` comes with [support for _buffered_ readers and writers][stdbuf],\n//! and therefore, `tokio::io` does as well.\n//!\n//! Tokio provides an async version of the [`std::io::BufRead`] trait,\n//! [`AsyncBufRead`]; and async [`BufReader`] and [`BufWriter`] structs, which\n//! wrap readers and writers. These wrappers use a buffer, reducing the number\n//! of calls and providing nicer methods for accessing exactly what you want.\n//!\n//! For example, [`BufReader`] works with the [`AsyncBufRead`] trait to add\n//! extra methods to any async reader:\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::io::{self, BufReader, AsyncBufReadExt};\n//! use tokio::fs::File;\n//!\n//! #[tokio::main]\n//! async fn main() -> io::Result<()> {\n//! let f = File::open(\"foo.txt\").await?;\n//! let mut reader = BufReader::new(f);\n//! let mut buffer = String::new();\n//!\n//! // read a line into buffer\n//! reader.read_line(&mut buffer).await?;\n//!\n//! println!(\"{}\", buffer);\n//! Ok(())\n//! }\n//! # }\n//! ```\n//!\n//! [`BufWriter`] doesn't add any new ways of writing; it just buffers every call\n//! to [`write`](crate::io::AsyncWriteExt::write). However, you **must** flush\n//! [`BufWriter`] to ensure that any buffered data is written.\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::io::{self, BufWriter, AsyncWriteExt};\n//! use tokio::fs::File;\n//!\n//! #[tokio::main]\n//! async fn main() -> io::Result<()> {\n//! let f = File::create(\"foo.txt\").await?;\n//! {\n//! let mut writer = BufWriter::new(f);\n//!\n//! // Write a byte to the buffer.\n//! writer.write(&[42u8]).await?;\n//!\n//! // Flush the buffer before it goes out of scope.\n//! writer.flush().await?;\n//!\n//! } // Unless flushed or shut down, the contents of the buffer is discarded on drop.\n//!\n//! Ok(())\n//! }\n//! # }\n//! ```\n//!\n//! [stdbuf]: std::io#bufreader-and-bufwriter\n//! [`std::io::BufRead`]: std::io::BufRead\n//! [`AsyncBufRead`]: crate::io::AsyncBufRead\n//! [`BufReader`]: crate::io::BufReader\n//! [`BufWriter`]: crate::io::BufWriter\n//!\n//! ## Implementing `AsyncRead` and `AsyncWrite`\n//!\n//! Because they are traits, we can implement [`AsyncRead`] and [`AsyncWrite`] for\n//! our own types, as well. Note that these traits must only be implemented for\n//! non-blocking I/O types that integrate with the futures type system. In\n//! other words, these types must never block the thread, and instead the\n//! current task is notified when the I/O resource is ready.\n//!\n//! ## Conversion to and from Stream/Sink\n//!\n//! It is often convenient to encapsulate the reading and writing of bytes in a\n//! [`Stream`] or [`Sink`] of data.\n//!\n//! Tokio provides simple wrappers for converting [`AsyncRead`] to [`Stream`]\n//! and vice-versa in the [tokio-util] crate, see [`ReaderStream`] and\n//! [`StreamReader`].\n//!\n//! There are also utility traits that abstract the asynchronous buffering\n//! necessary to write your own adaptors for encoding and decoding bytes to/from\n//! your structured data, allowing to transform something that implements\n//! [`AsyncRead`]/[`AsyncWrite`] into a [`Stream`]/[`Sink`], see [`Decoder`] and\n//! [`Encoder`] in the [tokio-util::codec] module.\n//!\n//! [tokio-util]: https://docs.rs/tokio-util\n//! [tokio-util::codec]: https://docs.rs/tokio-util/latest/tokio_util/codec/index.html\n//!\n//! # Standard input and output\n//!\n//! Tokio provides asynchronous APIs to standard [input], [output], and [error].\n//! These APIs are very similar to the ones provided by `std`, but they also\n//! implement [`AsyncRead`] and [`AsyncWrite`].\n//!\n//! Note that the standard input / output APIs **must** be used from the\n//! context of the Tokio runtime, as they require Tokio-specific features to\n//! function. Calling these functions outside of a Tokio runtime will panic.\n//!\n//! [input]: fn@stdin\n//! [output]: fn@stdout\n//! [error]: fn@stderr\n//!\n//! # `std` re-exports\n//!\n//! Additionally, [`Error`], [`ErrorKind`], [`Result`], and [`SeekFrom`] are\n//! re-exported from `std::io` for ease of use.\n//!\n//! [`AsyncRead`]: trait@AsyncRead\n//! [`AsyncWrite`]: trait@AsyncWrite\n//! [`AsyncReadExt`]: trait@AsyncReadExt\n//! [`AsyncWriteExt`]: trait@AsyncWriteExt\n//! [\"codec\"]: https://docs.rs/tokio-util/latest/tokio_util/codec/index.html\n//! [`Encoder`]: https://docs.rs/tokio-util/latest/tokio_util/codec/trait.Encoder.html\n//! [`Decoder`]: https://docs.rs/tokio-util/latest/tokio_util/codec/trait.Decoder.html\n//! [`ReaderStream`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.ReaderStream.html\n//! [`StreamReader`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.StreamReader.html\n//! [`Error`]: struct@Error\n//! [`ErrorKind`]: enum@ErrorKind\n//! [`Result`]: type@Result\n//! [`Read`]: std::io::Read\n//! [`SeekFrom`]: enum@SeekFrom\n//! [`Sink`]: https://docs.rs/futures/0.3/futures/sink/trait.Sink.html\n//! [`Stream`]: https://docs.rs/futures/0.3/futures/stream/trait.Stream.html\n//! [`Write`]: std::io::Write\n\n#![cfg_attr(\n not(all(feature = \"rt\", feature = \"net\")),\n allow(dead_code, unused_imports)\n)]\n\ncfg_io_blocking! {\n pub(crate) mod blocking;\n}\n\nmod async_buf_read;\npub use self::async_buf_read::AsyncBufRead;\n\nmod async_read;\npub use self::async_read::AsyncRead;\n\nmod async_seek;\npub use self::async_seek::AsyncSeek;\n\nmod async_write;\npub use self::async_write::AsyncWrite;\n\nmod read_buf;\npub use self::read_buf::ReadBuf;\n\n// Re-export some types from `std::io` so that users don't have to deal\n// with conflicts when `use`ing `tokio::io` and `std::io`.\n#[doc(no_inline)]\npub use std::io::{Error, ErrorKind, Result, SeekFrom};\n\ncfg_io_driver_impl! {\n pub(crate) mod interest;\n pub(crate) mod ready;\n\n cfg_net_or_uring! {\n pub use interest::Interest;\n pub use ready::Ready;\n }\n\n #[cfg_attr(target_os = \"wasi\", allow(unused_imports))]\n mod poll_evented;\n\n #[cfg(not(loom))]\n #[cfg_attr(target_os = \"wasi\", allow(unused_imports))]\n pub(crate) use poll_evented::PollEvented;\n}\n\n// The bsd module can't be build on Windows, so we completely ignore it, even\n// when building documentation.\n#[cfg(unix)]\ncfg_aio! {\n /// BSD-specific I/O types.\n pub mod bsd {\n mod poll_aio;\n\n pub use poll_aio::{Aio, AioEvent, AioSource};\n }\n}\n\ncfg_net_unix! {\n mod async_fd;\n\n pub mod unix {\n //! Asynchronous IO structures specific to Unix-like operating systems.\n pub use super::async_fd::{AsyncFd, AsyncFdTryNewError, AsyncFdReadyGuard, AsyncFdReadyMutGuard, TryIoError};\n }\n}\n\ncfg_io_std! {\n mod stdio_common;\n\n mod stderr;\n pub use stderr::{stderr, Stderr};\n\n mod stdin;\n pub use stdin::{stdin, Stdin};\n\n mod stdout;\n pub use stdout::{stdout, Stdout};\n}\n\ncfg_io_util! {\n mod split;\n pub use split::{split, ReadHalf, WriteHalf};\n mod join;\n pub use join::{join, Join};\n\n pub(crate) mod seek;\n pub(crate) mod util;\n pub use util::{\n copy, copy_bidirectional, copy_bidirectional_with_sizes, copy_buf, duplex, empty, repeat, sink, simplex, AsyncBufReadExt, AsyncReadExt, AsyncSeekExt, AsyncWriteExt,\n BufReader, BufStream, BufWriter, Chain, DuplexStream, Empty, Lines, Repeat, Sink, Split, Take, SimplexStream,\n };\n}\n\ncfg_not_io_util! {\n cfg_process! {\n pub(crate) mod util;\n }\n}\n\ncfg_io_blocking! {\n /// Types in this module can be mocked out in tests.\n mod sys {\n // TODO: don't rename\n pub(crate) use crate::blocking::spawn_blocking as run;\n pub(crate) use crate::blocking::JoinHandle as Blocking;\n }\n}\n\ncfg_io_uring! {\n pub(crate) mod uring;\n}\n" }, { "path": "tokio/src/io/poll_evented.rs", "content": "use crate::io::interest::Interest;\nuse crate::runtime::io::Registration;\nuse crate::runtime::scheduler;\n\nuse mio::event::Source;\nuse std::fmt;\nuse std::io;\nuse std::ops::Deref;\nuse std::panic::{RefUnwindSafe, UnwindSafe};\nuse std::task::ready;\n\ncfg_io_driver! {\n /// Associates an I/O resource that implements the [`std::io::Read`] and/or\n /// [`std::io::Write`] traits with the reactor that drives it.\n ///\n /// `PollEvented` uses [`Registration`] internally to take a type that\n /// implements [`mio::event::Source`] as well as [`std::io::Read`] and/or\n /// [`std::io::Write`] and associate it with a reactor that will drive it.\n ///\n /// Once the [`mio::event::Source`] type is wrapped by `PollEvented`, it can be\n /// used from within the future's execution model. As such, the\n /// `PollEvented` type provides [`AsyncRead`] and [`AsyncWrite`]\n /// implementations using the underlying I/O resource as well as readiness\n /// events provided by the reactor.\n ///\n /// **Note**: While `PollEvented` is `Sync` (if the underlying I/O type is\n /// `Sync`), the caller must ensure that there are at most two tasks that\n /// use a `PollEvented` instance concurrently. One for reading and one for\n /// writing. While violating this requirement is \"safe\" from a Rust memory\n /// model point of view, it will result in unexpected behavior in the form\n /// of lost notifications and tasks hanging.\n ///\n /// ## Readiness events\n ///\n /// Besides just providing [`AsyncRead`] and [`AsyncWrite`] implementations,\n /// this type also supports access to the underlying readiness event stream.\n /// While similar in function to what [`Registration`] provides, the\n /// semantics are a bit different.\n ///\n /// Two functions are provided to access the readiness events:\n /// [`poll_read_ready`] and [`poll_write_ready`]. These functions return the\n /// current readiness state of the `PollEvented` instance. If\n /// [`poll_read_ready`] indicates read readiness, immediately calling\n /// [`poll_read_ready`] again will also indicate read readiness.\n ///\n /// When the operation is attempted and is unable to succeed due to the I/O\n /// resource not being ready, the caller must call [`clear_readiness`].\n /// This clears the readiness state until a new readiness event is received.\n ///\n /// This allows the caller to implement additional functions. For example,\n /// [`TcpListener`] implements `poll_accept` by using [`poll_read_ready`] and\n /// [`clear_readiness`].\n ///\n /// ## Platform-specific events\n ///\n /// `PollEvented` also allows receiving platform-specific `mio::Ready` events.\n /// These events are included as part of the read readiness event stream. The\n /// write readiness event stream is only for `Ready::writable()` events.\n ///\n /// [`AsyncRead`]: crate::io::AsyncRead\n /// [`AsyncWrite`]: crate::io::AsyncWrite\n /// [`TcpListener`]: crate::net::TcpListener\n /// [`clear_readiness`]: Registration::clear_readiness\n /// [`poll_read_ready`]: Registration::poll_read_ready\n /// [`poll_write_ready`]: Registration::poll_write_ready\n pub(crate) struct PollEvented {\n io: Option,\n registration: Registration,\n }\n}\n\n// ===== impl PollEvented =====\n\nimpl PollEvented {\n /// Creates a new `PollEvented` associated with the default reactor.\n ///\n /// The returned `PollEvented` has readable and writable interests. For more control, use\n /// [`Self::new_with_interest`].\n ///\n /// # Panics\n ///\n /// This function panics if thread-local runtime is not set.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n #[cfg_attr(feature = \"signal\", allow(unused))]\n pub(crate) fn new(io: E) -> io::Result {\n PollEvented::new_with_interest(io, Interest::READABLE | Interest::WRITABLE)\n }\n\n /// Creates a new `PollEvented` associated with the default reactor, for\n /// specific `Interest` state. `new_with_interest` should be used over `new`\n /// when you need control over the readiness state, such as when a file\n /// descriptor only allows reads. This does not add `hup` or `error` so if\n /// you are interested in those states, you will need to add them to the\n /// readiness state passed to this function.\n ///\n /// # Panics\n ///\n /// This function panics if thread-local runtime is not set.\n ///\n /// The runtime is usually set implicitly when this function is called from\n /// a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter)\n /// function.\n #[track_caller]\n #[cfg_attr(feature = \"signal\", allow(unused))]\n pub(crate) fn new_with_interest(io: E, interest: Interest) -> io::Result {\n Self::new_with_interest_and_handle(io, interest, scheduler::Handle::current())\n }\n\n #[track_caller]\n pub(crate) fn new_with_interest_and_handle(\n mut io: E,\n interest: Interest,\n handle: scheduler::Handle,\n ) -> io::Result {\n let registration = Registration::new_with_interest_and_handle(&mut io, interest, handle)?;\n Ok(Self {\n io: Some(io),\n registration,\n })\n }\n\n /// Returns a reference to the registration.\n #[cfg(any(feature = \"net\", all(feature = \"process\", target_os = \"linux\")))]\n pub(crate) fn registration(&self) -> &Registration {\n &self.registration\n }\n\n /// Deregisters the inner io from the registration and returns a Result containing the inner io.\n #[cfg(any(feature = \"net\", feature = \"process\"))]\n pub(crate) fn into_inner(mut self) -> io::Result {\n let mut inner = self.io.take().unwrap(); // As io shouldn't ever be None, just unwrap here.\n self.registration.deregister(&mut inner)?;\n Ok(inner)\n }\n\n /// Re-register under new runtime with `interest`.\n #[cfg(all(feature = \"process\", target_os = \"linux\"))]\n pub(crate) fn reregister(&mut self, interest: Interest) -> io::Result<()> {\n let io = self.io.as_mut().unwrap(); // As io shouldn't ever be None, just unwrap here.\n let _ = self.registration.deregister(io);\n self.registration =\n Registration::new_with_interest_and_handle(io, interest, scheduler::Handle::current())?;\n\n Ok(())\n }\n}\n\nfeature! {\n #![any(feature = \"net\", all(unix, feature = \"process\"))]\n\n use crate::io::ReadBuf;\n use std::task::{Context, Poll};\n\n impl PollEvented {\n // Safety: The caller must ensure that `E` can read into uninitialized memory\n pub(crate) unsafe fn poll_read<'a>(\n &'a self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll>\n where\n &'a E: io::Read + 'a,\n {\n use std::io::Read;\n\n loop {\n let evt = ready!(self.registration.poll_read_ready(cx))?;\n\n let b = unsafe { &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // used only when the cfgs below apply\n #[allow(unused_variables)]\n let len = b.len();\n\n match self.io.as_ref().unwrap().read(b) {\n Ok(n) => {\n // When mio is using the epoll or kqueue selector, reading a partially full\n // buffer is sufficient to show that the socket buffer has been drained.\n //\n // This optimization does not work for level-triggered selectors such as\n // windows or when poll is used.\n //\n // Read more:\n // https://github.com/tokio-rs/tokio/issues/5866\n #[cfg(all(\n // keep in sync with poll_write\n not(mio_unsupported_force_poll_poll),\n any(\n // epoll\n target_os = \"android\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"redox\",\n // kqueue\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"ios\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"tvos\",\n target_os = \"visionos\",\n target_os = \"watchos\",\n )\n ))]\n if 0 < n && n < len {\n self.registration.clear_readiness(evt);\n }\n\n // Safety: We trust `TcpStream::read` to have filled up `n` bytes in the\n // buffer.\n unsafe { buf.assume_init(n) };\n buf.advance(n);\n return Poll::Ready(Ok(()));\n },\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.registration.clear_readiness(evt);\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n\n pub(crate) fn poll_write<'a>(&'a self, cx: &mut Context<'_>, buf: &[u8]) -> Poll>\n where\n &'a E: io::Write + 'a,\n {\n use std::io::Write;\n\n loop {\n let evt = ready!(self.registration.poll_write_ready(cx))?;\n\n match self.io.as_ref().unwrap().write(buf) {\n Ok(n) => {\n // if we write only part of our buffer, this is sufficient on unix to show\n // that the socket buffer is full. Unfortunately this assumption\n // fails for level-triggered selectors (like on Windows or poll even for\n // UNIX): https://github.com/tokio-rs/tokio/issues/5866\n #[cfg(all(\n // keep in sync with poll_read\n not(mio_unsupported_force_poll_poll),\n any(\n // epoll\n target_os = \"android\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"redox\",\n // kqueue\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"ios\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"tvos\",\n target_os = \"visionos\",\n target_os = \"watchos\",\n )\n ))]\n if 0 < n && n < buf.len() {\n self.registration.clear_readiness(evt);\n }\n\n return Poll::Ready(Ok(n));\n },\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.registration.clear_readiness(evt);\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n\n #[cfg(any(feature = \"net\", feature = \"process\"))]\n pub(crate) fn poll_write_vectored<'a>(\n &'a self,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll>\n where\n &'a E: io::Write + 'a,\n {\n use std::io::Write;\n self.registration.poll_write_io(cx, || self.io.as_ref().unwrap().write_vectored(bufs))\n }\n }\n}\n\nimpl UnwindSafe for PollEvented {}\n\nimpl RefUnwindSafe for PollEvented {}\n\nimpl Deref for PollEvented {\n type Target = E;\n\n fn deref(&self) -> &E {\n self.io.as_ref().unwrap()\n }\n}\n\nimpl fmt::Debug for PollEvented {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"PollEvented\").field(\"io\", &self.io).finish()\n }\n}\n\nimpl Drop for PollEvented {\n fn drop(&mut self) {\n if let Some(mut io) = self.io.take() {\n // Ignore errors\n let _ = self.registration.deregister(&mut io);\n }\n }\n}\n" }, { "path": "tokio/src/io/read_buf.rs", "content": "use std::fmt;\nuse std::mem::MaybeUninit;\n\n/// A wrapper around a byte buffer that is incrementally filled and initialized.\n///\n/// This type is a sort of \"double cursor\". It tracks three regions in the\n/// buffer: a region at the beginning of the buffer that has been logically\n/// filled with data, a region that has been initialized at some point but not\n/// yet logically filled, and a region at the end that may be uninitialized.\n/// The filled region is guaranteed to be a subset of the initialized region.\n///\n/// In summary, the contents of the buffer can be visualized as:\n///\n/// ```not_rust\n/// [ capacity ]\n/// [ filled | unfilled ]\n/// [ initialized | uninitialized ]\n/// ```\n///\n/// It is undefined behavior to de-initialize any bytes from the uninitialized\n/// region, since it is merely unknown whether this region is uninitialized or\n/// not, and if part of it turns out to be initialized, it must stay initialized.\npub struct ReadBuf<'a> {\n buf: &'a mut [MaybeUninit],\n filled: usize,\n initialized: usize,\n}\n\nimpl<'a> ReadBuf<'a> {\n /// Creates a new `ReadBuf` from a fully initialized buffer.\n #[inline]\n pub fn new(buf: &'a mut [u8]) -> ReadBuf<'a> {\n let initialized = buf.len();\n let buf = unsafe { slice_to_uninit_mut(buf) };\n ReadBuf {\n buf,\n filled: 0,\n initialized,\n }\n }\n\n /// Creates a new `ReadBuf` from a buffer that may be uninitialized.\n ///\n /// The internal cursor will mark the entire buffer as uninitialized. If\n /// the buffer is known to be partially initialized, then use `assume_init`\n /// to move the internal cursor.\n #[inline]\n pub fn uninit(buf: &'a mut [MaybeUninit]) -> ReadBuf<'a> {\n ReadBuf {\n buf,\n filled: 0,\n initialized: 0,\n }\n }\n\n /// Returns the total capacity of the buffer.\n #[inline]\n pub fn capacity(&self) -> usize {\n self.buf.len()\n }\n\n /// Returns a shared reference to the filled portion of the buffer.\n #[inline]\n pub fn filled(&self) -> &[u8] {\n let slice = &self.buf[..self.filled];\n // safety: filled describes how far into the buffer that the\n // user has filled with bytes, so it's been initialized.\n unsafe { slice_assume_init(slice) }\n }\n\n /// Returns a mutable reference to the filled portion of the buffer.\n #[inline]\n pub fn filled_mut(&mut self) -> &mut [u8] {\n let slice = &mut self.buf[..self.filled];\n // safety: filled describes how far into the buffer that the\n // user has filled with bytes, so it's been initialized.\n unsafe { slice_assume_init_mut(slice) }\n }\n\n /// Returns a new `ReadBuf` comprised of the unfilled section up to `n`.\n #[inline]\n pub fn take(&mut self, n: usize) -> ReadBuf<'_> {\n let max = std::cmp::min(self.remaining(), n);\n // Safety: We don't set any of the `unfilled_mut` with `MaybeUninit::uninit`.\n unsafe { ReadBuf::uninit(&mut self.unfilled_mut()[..max]) }\n }\n\n /// Returns a shared reference to the initialized portion of the buffer.\n ///\n /// This includes the filled portion.\n #[inline]\n pub fn initialized(&self) -> &[u8] {\n let slice = &self.buf[..self.initialized];\n // safety: initialized describes how far into the buffer that the\n // user has at some point initialized with bytes.\n unsafe { slice_assume_init(slice) }\n }\n\n /// Returns a mutable reference to the initialized portion of the buffer.\n ///\n /// This includes the filled portion.\n #[inline]\n pub fn initialized_mut(&mut self) -> &mut [u8] {\n let slice = &mut self.buf[..self.initialized];\n // safety: initialized describes how far into the buffer that the\n // user has at some point initialized with bytes.\n unsafe { slice_assume_init_mut(slice) }\n }\n\n /// Returns a mutable reference to the entire buffer, without ensuring that it has been fully\n /// initialized.\n ///\n /// The elements between 0 and `self.filled().len()` are filled, and those between 0 and\n /// `self.initialized().len()` are initialized (and so can be converted to a `&mut [u8]`).\n ///\n /// The caller of this method must ensure that these invariants are upheld. For example, if the\n /// caller initializes some of the uninitialized section of the buffer, it must call\n /// [`assume_init`](Self::assume_init) with the number of bytes initialized.\n ///\n /// # Safety\n ///\n /// The caller must not de-initialize portions of the buffer that have already been initialized.\n /// This includes any bytes in the region marked as uninitialized by `ReadBuf`.\n #[inline]\n pub unsafe fn inner_mut(&mut self) -> &mut [MaybeUninit] {\n self.buf\n }\n\n /// Returns a mutable reference to the unfilled part of the buffer without ensuring that it has been fully\n /// initialized.\n ///\n /// # Safety\n ///\n /// The caller must not de-initialize portions of the buffer that have already been initialized.\n /// This includes any bytes in the region marked as uninitialized by `ReadBuf`.\n #[inline]\n pub unsafe fn unfilled_mut(&mut self) -> &mut [MaybeUninit] {\n &mut self.buf[self.filled..]\n }\n\n /// Returns a mutable reference to the unfilled part of the buffer, ensuring it is fully initialized.\n ///\n /// Since `ReadBuf` tracks the region of the buffer that has been initialized, this is effectively \"free\" after\n /// the first use.\n #[inline]\n pub fn initialize_unfilled(&mut self) -> &mut [u8] {\n self.initialize_unfilled_to(self.remaining())\n }\n\n /// Returns a mutable reference to the first `n` bytes of the unfilled part of the buffer, ensuring it is\n /// fully initialized.\n ///\n /// # Panics\n ///\n /// Panics if `self.remaining()` is less than `n`.\n #[inline]\n #[track_caller]\n pub fn initialize_unfilled_to(&mut self, n: usize) -> &mut [u8] {\n assert!(self.remaining() >= n, \"n overflows remaining\");\n\n // This can't overflow, otherwise the assert above would have failed.\n let end = self.filled + n;\n\n if self.initialized < end {\n unsafe {\n self.buf[self.initialized..end]\n .as_mut_ptr()\n .write_bytes(0, end - self.initialized);\n }\n self.initialized = end;\n }\n\n let slice = &mut self.buf[self.filled..end];\n // safety: just above, we checked that the end of the buf has\n // been initialized to some value.\n unsafe { slice_assume_init_mut(slice) }\n }\n\n /// Returns the number of bytes at the end of the slice that have not yet been filled.\n #[inline]\n pub fn remaining(&self) -> usize {\n self.capacity() - self.filled\n }\n\n /// Clears the buffer, resetting the filled region to empty.\n ///\n /// The number of initialized bytes is not changed, and the contents of the buffer are not modified.\n #[inline]\n pub fn clear(&mut self) {\n self.filled = 0;\n }\n\n /// Advances the size of the filled region of the buffer.\n ///\n /// The number of initialized bytes is not changed.\n ///\n /// # Panics\n ///\n /// Panics if the filled region of the buffer would become larger than the initialized region.\n #[inline]\n #[track_caller]\n pub fn advance(&mut self, n: usize) {\n let new = self.filled.checked_add(n).expect(\"filled overflow\");\n self.set_filled(new);\n }\n\n /// Sets the size of the filled region of the buffer.\n ///\n /// The number of initialized bytes is not changed.\n ///\n /// Note that this can be used to *shrink* the filled region of the buffer in addition to growing it (for\n /// example, by a `AsyncRead` implementation that compresses data in-place).\n ///\n /// # Panics\n ///\n /// Panics if the filled region of the buffer would become larger than the initialized region.\n #[inline]\n #[track_caller]\n pub fn set_filled(&mut self, n: usize) {\n assert!(\n n <= self.initialized,\n \"filled must not become larger than initialized\"\n );\n self.filled = n;\n }\n\n /// Asserts that the first `n` unfilled bytes of the buffer are initialized.\n ///\n /// `ReadBuf` assumes that bytes are never de-initialized, so this method does nothing when called with fewer\n /// bytes than are already known to be initialized.\n ///\n /// # Safety\n ///\n /// The caller must ensure that `n` unfilled bytes of the buffer have already been initialized.\n #[inline]\n pub unsafe fn assume_init(&mut self, n: usize) {\n let new = self.filled + n;\n if new > self.initialized {\n self.initialized = new;\n }\n }\n\n /// Appends data to the buffer, advancing the written position and possibly also the initialized position.\n ///\n /// # Panics\n ///\n /// Panics if `self.remaining()` is less than `buf.len()`.\n #[inline]\n #[track_caller]\n pub fn put_slice(&mut self, buf: &[u8]) {\n assert!(\n self.remaining() >= buf.len(),\n \"buf.len() must fit in remaining(); buf.len() = {}, remaining() = {}\",\n buf.len(),\n self.remaining()\n );\n\n let amt = buf.len();\n // Cannot overflow, asserted above\n let end = self.filled + amt;\n\n // Safety: the length is asserted above\n unsafe {\n self.buf[self.filled..end]\n .as_mut_ptr()\n .cast::()\n .copy_from_nonoverlapping(buf.as_ptr(), amt);\n }\n\n if self.initialized < end {\n self.initialized = end;\n }\n self.filled = end;\n }\n}\n\n#[cfg(feature = \"io-util\")]\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\nunsafe impl<'a> bytes::BufMut for ReadBuf<'a> {\n fn remaining_mut(&self) -> usize {\n self.remaining()\n }\n\n // SAFETY: The caller guarantees that at least `cnt` unfilled bytes have been initialized.\n unsafe fn advance_mut(&mut self, cnt: usize) {\n unsafe {\n self.assume_init(cnt);\n }\n self.advance(cnt);\n }\n\n fn chunk_mut(&mut self) -> &mut bytes::buf::UninitSlice {\n // SAFETY: No region of `unfilled` will be deinitialized because it is\n // exposed as an `UninitSlice`, whose API guarantees that the memory is\n // never deinitialized.\n let unfilled = unsafe { self.unfilled_mut() };\n let len = unfilled.len();\n let ptr = unfilled.as_mut_ptr() as *mut u8;\n\n // SAFETY: The pointer is valid for `len` bytes because it comes from a\n // slice of that length.\n unsafe { bytes::buf::UninitSlice::from_raw_parts_mut(ptr, len) }\n }\n}\n\nimpl fmt::Debug for ReadBuf<'_> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"ReadBuf\")\n .field(\"filled\", &self.filled)\n .field(\"initialized\", &self.initialized)\n .field(\"capacity\", &self.capacity())\n .finish()\n }\n}\n\n/// # Safety\n///\n/// The caller must ensure that `slice` is fully initialized\n/// and never writes uninitialized bytes to the returned slice.\nunsafe fn slice_to_uninit_mut(slice: &mut [u8]) -> &mut [MaybeUninit] {\n // SAFETY: `MaybeUninit` has the same memory layout as u8, and the caller\n // promises to not write uninitialized bytes to the returned slice.\n unsafe { &mut *(slice as *mut [u8] as *mut [MaybeUninit]) }\n}\n\n/// # Safety\n///\n/// The caller must ensure that `slice` is fully initialized.\n// TODO: This could use `MaybeUninit::slice_assume_init` when it is stable.\nunsafe fn slice_assume_init(slice: &[MaybeUninit]) -> &[u8] {\n // SAFETY: `MaybeUninit` has the same memory layout as u8, and the caller\n // promises that `slice` is fully initialized.\n unsafe { &*(slice as *const [MaybeUninit] as *const [u8]) }\n}\n\n/// # Safety\n///\n/// The caller must ensure that `slice` is fully initialized.\n// TODO: This could use `MaybeUninit::slice_assume_init_mut` when it is stable.\nunsafe fn slice_assume_init_mut(slice: &mut [MaybeUninit]) -> &mut [u8] {\n // SAFETY: `MaybeUninit` has the same memory layout as `u8`, and the caller\n // promises that `slice` is fully initialized.\n unsafe { &mut *(slice as *mut [MaybeUninit] as *mut [u8]) }\n}\n" }, { "path": "tokio/src/io/ready.rs", "content": "#![cfg_attr(not(feature = \"net\"), allow(unreachable_pub))]\n\nuse crate::io::interest::Interest;\n\nuse std::fmt;\nuse std::ops;\n\nconst READABLE: usize = 0b0_01;\nconst WRITABLE: usize = 0b0_10;\nconst READ_CLOSED: usize = 0b0_0100;\nconst WRITE_CLOSED: usize = 0b0_1000;\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nconst PRIORITY: usize = 0b1_0000;\nconst ERROR: usize = 0b10_0000;\n\n/// Describes the readiness state of an I/O resources.\n///\n/// `Ready` tracks which operation an I/O resource is ready to perform.\n#[cfg_attr(docsrs, doc(cfg(feature = \"net\")))]\n#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]\npub struct Ready(usize);\n\nimpl Ready {\n /// Returns the empty `Ready` set.\n pub const EMPTY: Ready = Ready(0);\n\n /// Returns a `Ready` representing readable readiness.\n pub const READABLE: Ready = Ready(READABLE);\n\n /// Returns a `Ready` representing writable readiness.\n pub const WRITABLE: Ready = Ready(WRITABLE);\n\n /// Returns a `Ready` representing read closed readiness.\n pub const READ_CLOSED: Ready = Ready(READ_CLOSED);\n\n /// Returns a `Ready` representing write closed readiness.\n pub const WRITE_CLOSED: Ready = Ready(WRITE_CLOSED);\n\n /// Returns a `Ready` representing priority readiness.\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub const PRIORITY: Ready = Ready(PRIORITY);\n\n /// Returns a `Ready` representing error readiness.\n pub const ERROR: Ready = Ready(ERROR);\n\n /// Returns a `Ready` representing readiness for all operations.\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n pub const ALL: Ready =\n Ready(READABLE | WRITABLE | READ_CLOSED | WRITE_CLOSED | ERROR | PRIORITY);\n\n /// Returns a `Ready` representing readiness for all operations.\n #[cfg(not(any(target_os = \"linux\", target_os = \"android\")))]\n pub const ALL: Ready = Ready(READABLE | WRITABLE | READ_CLOSED | WRITE_CLOSED | ERROR);\n\n // Must remain crate-private to avoid adding a public dependency on Mio.\n pub(crate) fn from_mio(event: &mio::event::Event) -> Ready {\n let mut ready = Ready::EMPTY;\n\n #[cfg(all(target_os = \"freebsd\", feature = \"net\"))]\n {\n if event.is_aio() {\n ready |= Ready::READABLE;\n }\n\n if event.is_lio() {\n ready |= Ready::READABLE;\n }\n }\n\n if event.is_readable() {\n ready |= Ready::READABLE;\n }\n\n if event.is_writable() {\n ready |= Ready::WRITABLE;\n }\n\n if event.is_read_closed() {\n ready |= Ready::READ_CLOSED;\n }\n\n if event.is_write_closed() {\n ready |= Ready::WRITE_CLOSED;\n }\n\n if event.is_error() {\n ready |= Ready::ERROR;\n }\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n {\n if event.is_priority() {\n ready |= Ready::PRIORITY;\n }\n }\n\n ready\n }\n\n /// Returns true if `Ready` is the empty set.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(Ready::EMPTY.is_empty());\n /// assert!(!Ready::READABLE.is_empty());\n /// ```\n pub fn is_empty(self) -> bool {\n self == Ready::EMPTY\n }\n\n /// Returns `true` if the value includes `readable`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_readable());\n /// assert!(Ready::READABLE.is_readable());\n /// assert!(Ready::READ_CLOSED.is_readable());\n /// assert!(!Ready::WRITABLE.is_readable());\n /// ```\n pub fn is_readable(self) -> bool {\n self.contains(Ready::READABLE) || self.is_read_closed()\n }\n\n /// Returns `true` if the value includes writable `readiness`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_writable());\n /// assert!(!Ready::READABLE.is_writable());\n /// assert!(Ready::WRITABLE.is_writable());\n /// assert!(Ready::WRITE_CLOSED.is_writable());\n /// ```\n pub fn is_writable(self) -> bool {\n self.contains(Ready::WRITABLE) || self.is_write_closed()\n }\n\n /// Returns `true` if the value includes read-closed `readiness`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_read_closed());\n /// assert!(!Ready::READABLE.is_read_closed());\n /// assert!(Ready::READ_CLOSED.is_read_closed());\n /// ```\n pub fn is_read_closed(self) -> bool {\n self.contains(Ready::READ_CLOSED)\n }\n\n /// Returns `true` if the value includes write-closed `readiness`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_write_closed());\n /// assert!(!Ready::WRITABLE.is_write_closed());\n /// assert!(Ready::WRITE_CLOSED.is_write_closed());\n /// ```\n pub fn is_write_closed(self) -> bool {\n self.contains(Ready::WRITE_CLOSED)\n }\n\n /// Returns `true` if the value includes priority `readiness`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_priority());\n /// assert!(!Ready::WRITABLE.is_priority());\n /// assert!(Ready::PRIORITY.is_priority());\n /// ```\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub fn is_priority(self) -> bool {\n self.contains(Ready::PRIORITY)\n }\n\n /// Returns `true` if the value includes error `readiness`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::Ready;\n ///\n /// assert!(!Ready::EMPTY.is_error());\n /// assert!(!Ready::WRITABLE.is_error());\n /// assert!(Ready::ERROR.is_error());\n /// ```\n pub fn is_error(self) -> bool {\n self.contains(Ready::ERROR)\n }\n\n /// Returns true if `self` is a superset of `other`.\n ///\n /// `other` may represent more than one readiness operations, in which case\n /// the function only returns true if `self` contains all readiness\n /// specified in `other`.\n pub(crate) fn contains>(self, other: T) -> bool {\n let other = other.into();\n (self & other) == other\n }\n\n /// Creates a `Ready` instance using the given `usize` representation.\n ///\n /// The `usize` representation must have been obtained from a call to\n /// `Readiness::as_usize`.\n ///\n /// This function is mainly provided to allow the caller to get a\n /// readiness value from an `AtomicUsize`.\n pub(crate) fn from_usize(val: usize) -> Ready {\n Ready(val & Ready::ALL.as_usize())\n }\n\n /// Returns a `usize` representation of the `Ready` value.\n ///\n /// This function is mainly provided to allow the caller to store a\n /// readiness value in an `AtomicUsize`.\n pub(crate) fn as_usize(self) -> usize {\n self.0\n }\n\n pub(crate) fn from_interest(interest: Interest) -> Ready {\n let mut ready = Ready::EMPTY;\n\n if interest.is_readable() {\n ready |= Ready::READABLE;\n ready |= Ready::READ_CLOSED;\n }\n\n if interest.is_writable() {\n ready |= Ready::WRITABLE;\n ready |= Ready::WRITE_CLOSED;\n }\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n if interest.is_priority() {\n ready |= Ready::PRIORITY;\n ready |= Ready::READ_CLOSED;\n }\n\n if interest.is_error() {\n ready |= Ready::ERROR;\n }\n\n ready\n }\n\n pub(crate) fn intersection(self, interest: Interest) -> Ready {\n Ready(self.0 & Ready::from_interest(interest).0)\n }\n\n pub(crate) fn satisfies(self, interest: Interest) -> bool {\n self.0 & Ready::from_interest(interest).0 != 0\n }\n}\n\nimpl ops::BitOr for Ready {\n type Output = Ready;\n\n #[inline]\n fn bitor(self, other: Ready) -> Ready {\n Ready(self.0 | other.0)\n }\n}\n\nimpl ops::BitOrAssign for Ready {\n #[inline]\n fn bitor_assign(&mut self, other: Ready) {\n self.0 |= other.0;\n }\n}\n\nimpl ops::BitAnd for Ready {\n type Output = Ready;\n\n #[inline]\n fn bitand(self, other: Ready) -> Ready {\n Ready(self.0 & other.0)\n }\n}\n\nimpl ops::Sub for Ready {\n type Output = Ready;\n\n #[inline]\n fn sub(self, other: Ready) -> Ready {\n Ready(self.0 & !other.0)\n }\n}\n\nimpl fmt::Debug for Ready {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n let mut fmt = fmt.debug_struct(\"Ready\");\n\n fmt.field(\"is_readable\", &self.is_readable())\n .field(\"is_writable\", &self.is_writable())\n .field(\"is_read_closed\", &self.is_read_closed())\n .field(\"is_write_closed\", &self.is_write_closed())\n .field(\"is_error\", &self.is_error());\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n fmt.field(\"is_priority\", &self.is_priority());\n\n fmt.finish()\n }\n}\n" }, { "path": "tokio/src/io/seek.rs", "content": "use crate::io::AsyncSeek;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io::{self, SeekFrom};\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Future for the [`seek`](crate::io::AsyncSeekExt::seek) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Seek<'a, S: ?Sized> {\n seek: &'a mut S,\n pos: Option,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn seek(seek: &mut S, pos: SeekFrom) -> Seek<'_, S>\nwhere\n S: AsyncSeek + ?Sized + Unpin,\n{\n Seek {\n seek,\n pos: Some(pos),\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for Seek<'_, S>\nwhere\n S: AsyncSeek + ?Sized + Unpin,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n match me.pos {\n Some(pos) => {\n // ensure no seek in progress\n ready!(Pin::new(&mut *me.seek).poll_complete(cx))?;\n match Pin::new(&mut *me.seek).start_seek(*pos) {\n Ok(()) => {\n *me.pos = None;\n Pin::new(&mut *me.seek).poll_complete(cx)\n }\n Err(e) => Poll::Ready(Err(e)),\n }\n }\n None => Pin::new(&mut *me.seek).poll_complete(cx),\n }\n }\n}\n" }, { "path": "tokio/src/io/split.rs", "content": "//! Split a single value implementing `AsyncRead + AsyncWrite` into separate\n//! `AsyncRead` and `AsyncWrite` handles.\n//!\n//! To restore this read/write object from its `split::ReadHalf` and\n//! `split::WriteHalf` use `unsplit`.\n\nuse crate::io::{AsyncRead, AsyncWrite, ReadBuf};\n\nuse std::fmt;\nuse std::io;\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::sync::Mutex;\nuse std::task::{Context, Poll};\n\ncfg_io_util! {\n /// The readable half of a value returned from [`split`](split()).\n pub struct ReadHalf {\n inner: Arc>,\n }\n\n /// The writable half of a value returned from [`split`](split()).\n pub struct WriteHalf {\n inner: Arc>,\n }\n\n /// Splits a single value implementing `AsyncRead + AsyncWrite` into separate\n /// `AsyncRead` and `AsyncWrite` handles.\n ///\n /// To restore this read/write object from its `ReadHalf` and\n /// `WriteHalf` use [`unsplit`](ReadHalf::unsplit()).\n pub fn split(stream: T) -> (ReadHalf, WriteHalf)\n where\n T: AsyncRead + AsyncWrite,\n {\n let is_write_vectored = stream.is_write_vectored();\n\n let inner = Arc::new(Inner {\n stream: Mutex::new(stream),\n is_write_vectored,\n });\n\n let rd = ReadHalf {\n inner: inner.clone(),\n };\n\n let wr = WriteHalf { inner };\n\n (rd, wr)\n }\n}\n\nstruct Inner {\n stream: Mutex,\n is_write_vectored: bool,\n}\n\nimpl Inner {\n fn with_lock(&self, f: impl FnOnce(Pin<&mut T>) -> R) -> R {\n let mut guard = self.stream.lock().unwrap();\n\n // safety: we do not move the stream.\n let stream = unsafe { Pin::new_unchecked(&mut *guard) };\n\n f(stream)\n }\n}\n\nimpl ReadHalf {\n /// Checks if this `ReadHalf` and some `WriteHalf` were split from the same\n /// stream.\n pub fn is_pair_of(&self, other: &WriteHalf) -> bool {\n other.is_pair_of(self)\n }\n\n /// Reunites with a previously split `WriteHalf`.\n ///\n /// # Panics\n ///\n /// If this `ReadHalf` and the given `WriteHalf` do not originate from the\n /// same `split` operation this method will panic.\n /// This can be checked ahead of time by calling [`is_pair_of()`](Self::is_pair_of).\n #[track_caller]\n pub fn unsplit(self, wr: WriteHalf) -> T\n where\n T: Unpin,\n {\n if self.is_pair_of(&wr) {\n drop(wr);\n\n let inner = Arc::try_unwrap(self.inner)\n .ok()\n .expect(\"`Arc::try_unwrap` failed\");\n\n inner.stream.into_inner().unwrap()\n } else {\n panic!(\"Unrelated `split::Write` passed to `split::Read::unsplit`.\")\n }\n }\n}\n\nimpl WriteHalf {\n /// Checks if this `WriteHalf` and some `ReadHalf` were split from the same\n /// stream.\n pub fn is_pair_of(&self, other: &ReadHalf) -> bool {\n Arc::ptr_eq(&self.inner, &other.inner)\n }\n}\n\nimpl AsyncRead for ReadHalf {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.inner.with_lock(|stream| stream.poll_read(cx, buf))\n }\n}\n\nimpl AsyncWrite for WriteHalf {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.inner.with_lock(|stream| stream.poll_write(cx, buf))\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.with_lock(|stream| stream.poll_flush(cx))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.with_lock(|stream| stream.poll_shutdown(cx))\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.inner\n .with_lock(|stream| stream.poll_write_vectored(cx, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n self.inner.is_write_vectored\n }\n}\n\nunsafe impl Send for ReadHalf {}\nunsafe impl Send for WriteHalf {}\nunsafe impl Sync for ReadHalf {}\nunsafe impl Sync for WriteHalf {}\n\nimpl fmt::Debug for ReadHalf {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"split::ReadHalf\").finish()\n }\n}\n\nimpl fmt::Debug for WriteHalf {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"split::WriteHalf\").finish()\n }\n}\n" }, { "path": "tokio/src/io/stderr.rs", "content": "use crate::io::blocking::Blocking;\nuse crate::io::stdio_common::SplitByUtf8BoundaryIfWindows;\nuse crate::io::AsyncWrite;\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::Context;\nuse std::task::Poll;\n\ncfg_io_std! {\n /// A handle to the standard error stream of a process.\n ///\n /// Concurrent writes to stderr must be executed with care: Only individual\n /// writes to this [`AsyncWrite`] are guaranteed to be intact. In particular\n /// you should be aware that writes using [`write_all`] are not guaranteed\n /// to occur as a single write, so multiple threads writing data with\n /// [`write_all`] may result in interleaved output.\n ///\n /// Created by the [`stderr`] function.\n ///\n /// [`stderr`]: stderr()\n /// [`AsyncWrite`]: AsyncWrite\n /// [`write_all`]: crate::io::AsyncWriteExt::write_all()\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut stderr = io::stderr();\n /// stderr.write_all(b\"Print some error here.\").await?;\n /// Ok(())\n /// }\n /// ```\n #[derive(Debug)]\n pub struct Stderr {\n std: SplitByUtf8BoundaryIfWindows>,\n }\n\n /// Constructs a new handle to the standard error of the current process.\n ///\n /// The returned handle allows writing to standard error from the within the\n /// Tokio runtime.\n ///\n /// Concurrent writes to stderr must be executed with care: Only individual\n /// writes to this [`AsyncWrite`] are guaranteed to be intact. In particular\n /// you should be aware that writes using [`write_all`] are not guaranteed\n /// to occur as a single write, so multiple threads writing data with\n /// [`write_all`] may result in interleaved output.\n ///\n /// Note that unlike [`std::io::stderr`], each call to this `stderr()`\n /// produces a new writer, so for example, this program does **not** flush stderr:\n ///\n /// ```no_run\n /// # use tokio::io::AsyncWriteExt;\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// tokio::io::stderr().write_all(b\"aa\").await?;\n /// tokio::io::stderr().flush().await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`std::io::stderr`]: std::io::stderr\n /// [`AsyncWrite`]: AsyncWrite\n /// [`write_all`]: crate::io::AsyncWriteExt::write_all()\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut stderr = io::stderr();\n /// stderr.write_all(b\"Print some error here.\").await?;\n /// Ok(())\n /// }\n /// ```\n pub fn stderr() -> Stderr {\n let std = io::stderr();\n // SAFETY: The `Read` implementation of `std` does not read from the\n // buffer it is borrowing and correctly reports the length of the data\n // written into the buffer.\n let blocking = unsafe { Blocking::new(std) };\n Stderr {\n std: SplitByUtf8BoundaryIfWindows::new(blocking),\n }\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, RawFd};\n\n use super::Stderr;\n\n impl AsRawFd for Stderr {\n fn as_raw_fd(&self) -> RawFd {\n std::io::stderr().as_raw_fd()\n }\n }\n\n impl AsFd for Stderr {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsHandle, BorrowedHandle, AsRawHandle, RawHandle};\n\n impl AsRawHandle for Stderr {\n fn as_raw_handle(&self) -> RawHandle {\n std::io::stderr().as_raw_handle()\n }\n }\n\n impl AsHandle for Stderr {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n }\n}\n\nimpl AsyncWrite for Stderr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut self.std).poll_write(cx, buf)\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.std).poll_flush(cx)\n }\n\n fn poll_shutdown(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n Pin::new(&mut self.std).poll_shutdown(cx)\n }\n}\n" }, { "path": "tokio/src/io/stdin.rs", "content": "use crate::io::blocking::Blocking;\nuse crate::io::{AsyncRead, ReadBuf};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::Context;\nuse std::task::Poll;\n\ncfg_io_std! {\n /// A handle to the standard input stream of a process.\n ///\n /// The handle implements the [`AsyncRead`] trait, but beware that concurrent\n /// reads of `Stdin` must be executed with care.\n ///\n /// This handle is best used for non-interactive uses, such as when a file\n /// is piped into the application. For technical reasons, `stdin` is\n /// implemented by using an ordinary blocking read on a separate thread, and\n /// it is impossible to cancel that read. This can make shutdown of the\n /// runtime hang until the user presses enter.\n ///\n /// For interactive uses, it is recommended to spawn a thread dedicated to\n /// user input and use blocking IO directly in that thread.\n ///\n /// Created by the [`stdin`] function.\n ///\n /// [`stdin`]: fn@stdin\n /// [`AsyncRead`]: trait@AsyncRead\n #[derive(Debug)]\n pub struct Stdin {\n std: Blocking,\n }\n\n /// Constructs a new handle to the standard input of the current process.\n ///\n /// This handle is best used for non-interactive uses, such as when a file\n /// is piped into the application. For technical reasons, `stdin` is\n /// implemented by using an ordinary blocking read on a separate thread, and\n /// it is impossible to cancel that read. This can make shutdown of the\n /// runtime hang until the user presses enter.\n ///\n /// For interactive uses, it is recommended to spawn a thread dedicated to\n /// user input and use blocking IO directly in that thread.\n pub fn stdin() -> Stdin {\n let std = io::stdin();\n // SAFETY: The `Read` implementation of `std` does not read from the\n // buffer it is borrowing and correctly reports the length of the data\n // written into the buffer.\n let std = unsafe { Blocking::new(std) };\n Stdin {\n std,\n }\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, RawFd};\n\n use super::Stdin;\n\n impl AsRawFd for Stdin {\n fn as_raw_fd(&self) -> RawFd {\n std::io::stdin().as_raw_fd()\n }\n }\n\n impl AsFd for Stdin {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsHandle, BorrowedHandle, AsRawHandle, RawHandle};\n\n impl AsRawHandle for Stdin {\n fn as_raw_handle(&self) -> RawHandle {\n std::io::stdin().as_raw_handle()\n }\n }\n\n impl AsHandle for Stdin {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n }\n}\n\nimpl AsyncRead for Stdin {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut self.std).poll_read(cx, buf)\n }\n}\n" }, { "path": "tokio/src/io/stdio_common.rs", "content": "//! Contains utilities for stdout and stderr.\nuse crate::io::AsyncWrite;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n/// # Windows\n/// [`AsyncWrite`] adapter that finds last char boundary in given buffer and does not write the rest,\n/// if buffer contents seems to be `utf8`. Otherwise it only trims buffer down to `DEFAULT_MAX_BUF_SIZE`.\n/// That's why, wrapped writer will always receive well-formed utf-8 bytes.\n/// # Other platforms\n/// Passes data to `inner` as is.\n#[derive(Debug)]\npub(crate) struct SplitByUtf8BoundaryIfWindows {\n inner: W,\n}\n\nimpl SplitByUtf8BoundaryIfWindows {\n pub(crate) fn new(inner: W) -> Self {\n Self { inner }\n }\n}\n\n// this constant is defined by Unicode standard.\nconst MAX_BYTES_PER_CHAR: usize = 4;\n\n// Subject for tweaking here\nconst MAGIC_CONST: usize = 8;\n\nimpl crate::io::AsyncWrite for SplitByUtf8BoundaryIfWindows\nwhere\n W: AsyncWrite + Unpin,\n{\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n mut buf: &[u8],\n ) -> Poll> {\n // just a closure to avoid repetitive code\n let mut call_inner = move |buf| Pin::new(&mut self.inner).poll_write(cx, buf);\n\n // 1. Only windows stdio can suffer from non-utf8.\n // We also check for `test` so that we can write some tests\n // for further code. Since `AsyncWrite` can always shrink\n // buffer at its discretion, excessive (i.e. in tests) shrinking\n // does not break correctness.\n // 2. If buffer is small, it will not be shrunk.\n // That's why, it's \"textness\" will not change, so we don't have\n // to fixup it.\n if cfg!(not(any(target_os = \"windows\", test)))\n || buf.len() <= crate::io::blocking::DEFAULT_MAX_BUF_SIZE\n {\n return call_inner(buf);\n }\n\n buf = &buf[..crate::io::blocking::DEFAULT_MAX_BUF_SIZE];\n\n // Now there are two possibilities.\n // If caller gave is binary buffer, we **should not** shrink it\n // anymore, because excessive shrinking hits performance.\n // If caller gave as binary buffer, we **must** additionally\n // shrink it to strip incomplete char at the end of buffer.\n // that's why check we will perform now is allowed to have\n // false-positive.\n\n // Now let's look at the first MAX_BYTES_PER_CHAR * MAGIC_CONST bytes.\n // if they are (possibly incomplete) utf8, then we can be quite sure\n // that input buffer was utf8.\n\n let have_to_fix_up = match std::str::from_utf8(&buf[..MAX_BYTES_PER_CHAR * MAGIC_CONST]) {\n Ok(_) => true,\n Err(err) => {\n let incomplete_bytes = MAX_BYTES_PER_CHAR * MAGIC_CONST - err.valid_up_to();\n incomplete_bytes < MAX_BYTES_PER_CHAR\n }\n };\n\n if have_to_fix_up {\n // We must pop several bytes at the end which form incomplete\n // character. To achieve it, we exploit UTF8 encoding:\n // for any code point, all bytes except first start with 0b10 prefix.\n // see https://en.wikipedia.org/wiki/UTF-8#Encoding for details\n let trailing_incomplete_char_size = buf\n .iter()\n .rev()\n .take(MAX_BYTES_PER_CHAR)\n .position(|byte| *byte < 0b1000_0000 || *byte >= 0b1100_0000)\n .unwrap_or(0)\n + 1;\n buf = &buf[..buf.len() - trailing_incomplete_char_size];\n }\n\n call_inner(buf)\n }\n\n fn poll_flush(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_flush(cx)\n }\n\n fn poll_shutdown(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_shutdown(cx)\n }\n}\n\n#[cfg(test)]\n#[cfg(not(loom))]\nmod tests {\n use crate::io::blocking::DEFAULT_MAX_BUF_SIZE;\n use crate::io::AsyncWriteExt;\n use std::io;\n use std::pin::Pin;\n use std::task::Context;\n use std::task::Poll;\n\n struct TextMockWriter;\n\n impl crate::io::AsyncWrite for TextMockWriter {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n assert!(buf.len() <= DEFAULT_MAX_BUF_SIZE);\n assert!(std::str::from_utf8(buf).is_ok());\n Poll::Ready(Ok(buf.len()))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n ) -> Poll> {\n Poll::Ready(Ok(()))\n }\n }\n\n struct LoggingMockWriter {\n write_history: Vec,\n }\n\n impl LoggingMockWriter {\n fn new() -> Self {\n LoggingMockWriter {\n write_history: Vec::new(),\n }\n }\n }\n\n impl crate::io::AsyncWrite for LoggingMockWriter {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n assert!(buf.len() <= DEFAULT_MAX_BUF_SIZE);\n self.write_history.push(buf.len());\n Poll::Ready(Ok(buf.len()))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n ) -> Poll> {\n Poll::Ready(Ok(()))\n }\n }\n\n #[test]\n #[cfg_attr(miri, ignore)] // takes a really long time with miri\n fn test_splitter() {\n let data = str::repeat(\"█\", DEFAULT_MAX_BUF_SIZE);\n let mut wr = super::SplitByUtf8BoundaryIfWindows::new(TextMockWriter);\n let fut = async move {\n wr.write_all(data.as_bytes()).await.unwrap();\n };\n crate::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n .block_on(fut);\n }\n\n #[test]\n #[cfg_attr(miri, ignore)] // takes a really long time with miri\n fn test_pseudo_text() {\n // In this test we write a piece of binary data, whose beginning is\n // text though. We then validate that even in this corner case buffer\n // was not shrunk too much.\n let checked_count = super::MAGIC_CONST * super::MAX_BYTES_PER_CHAR;\n let mut data: Vec = str::repeat(\"a\", checked_count).into();\n data.extend(std::iter::repeat(0b1010_1010).take(DEFAULT_MAX_BUF_SIZE - checked_count + 1));\n let mut writer = LoggingMockWriter::new();\n let mut splitter = super::SplitByUtf8BoundaryIfWindows::new(&mut writer);\n crate::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n .block_on(async {\n splitter.write_all(&data).await.unwrap();\n });\n // Check that at most two writes were performed\n assert!(writer.write_history.len() <= 2);\n // Check that all has been written\n assert_eq!(\n writer.write_history.iter().copied().sum::(),\n data.len()\n );\n // Check that at most MAX_BYTES_PER_CHAR + 1 (i.e. 5) bytes were shrunk\n // from the buffer: one because it was outside of DEFAULT_MAX_BUF_SIZE boundary, and\n // up to one \"utf8 code point\".\n assert!(data.len() - writer.write_history[0] <= super::MAX_BYTES_PER_CHAR + 1);\n }\n}\n" }, { "path": "tokio/src/io/stdout.rs", "content": "use crate::io::blocking::Blocking;\nuse crate::io::stdio_common::SplitByUtf8BoundaryIfWindows;\nuse crate::io::AsyncWrite;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::Context;\nuse std::task::Poll;\n\ncfg_io_std! {\n /// A handle to the standard output stream of a process.\n ///\n /// Concurrent writes to stdout must be executed with care: Only individual\n /// writes to this [`AsyncWrite`] are guaranteed to be intact. In particular\n /// you should be aware that writes using [`write_all`] are not guaranteed\n /// to occur as a single write, so multiple threads writing data with\n /// [`write_all`] may result in interleaved output.\n ///\n /// Created by the [`stdout`] function.\n ///\n /// [`stdout`]: stdout()\n /// [`AsyncWrite`]: AsyncWrite\n /// [`write_all`]: crate::io::AsyncWriteExt::write_all()\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut stdout = io::stdout();\n /// stdout.write_all(b\"Hello world!\").await?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// The following is an example of using `stdio` with loop.\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let messages = vec![\"hello\", \" world\\n\"];\n ///\n /// // When you use `stdio` in a loop, it is recommended to create\n /// // a single `stdio` instance outside the loop and call a write\n /// // operation against that instance on each loop.\n /// //\n /// // Repeatedly creating `stdout` instances inside the loop and\n /// // writing to that handle could result in mangled output since\n /// // each write operation is handled by a different blocking thread.\n /// let mut stdout = io::stdout();\n ///\n /// for message in &messages {\n /// stdout.write_all(message.as_bytes()).await.unwrap();\n /// stdout.flush().await.unwrap();\n /// }\n /// }\n /// ```\n #[derive(Debug)]\n pub struct Stdout {\n std: SplitByUtf8BoundaryIfWindows>,\n }\n\n /// Constructs a new handle to the standard output of the current process.\n ///\n /// The returned handle allows writing to standard out from the within the\n /// Tokio runtime.\n ///\n /// Concurrent writes to stdout must be executed with care: Only individual\n /// writes to this [`AsyncWrite`] are guaranteed to be intact. In particular\n /// you should be aware that writes using [`write_all`] are not guaranteed\n /// to occur as a single write, so multiple threads writing data with\n /// [`write_all`] may result in interleaved output.\n ///\n /// Note that unlike [`std::io::stdout`], each call to this `stdout()`\n /// produces a new writer, so for example, this program does **not** flush stdout:\n ///\n /// ```no_run\n /// # use tokio::io::AsyncWriteExt;\n /// # #[tokio::main]\n /// # async fn main() -> std::io::Result<()> {\n /// tokio::io::stdout().write_all(b\"aa\").await?;\n /// tokio::io::stdout().flush().await?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`std::io::stdout`]: std::io::stdout\n /// [`AsyncWrite`]: AsyncWrite\n /// [`write_all`]: crate::io::AsyncWriteExt::write_all()\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut stdout = io::stdout();\n /// stdout.write_all(b\"Hello world!\").await?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// The following is an example of using `stdio` with loop.\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let messages = vec![\"hello\", \" world\\n\"];\n ///\n /// // When you use `stdio` in a loop, it is recommended to create\n /// // a single `stdio` instance outside the loop and call a write\n /// // operation against that instance on each loop.\n /// //\n /// // Repeatedly creating `stdout` instances inside the loop and\n /// // writing to that handle could result in mangled output since\n /// // each write operation is handled by a different blocking thread.\n /// let mut stdout = io::stdout();\n ///\n /// for message in &messages {\n /// stdout.write_all(message.as_bytes()).await.unwrap();\n /// stdout.flush().await.unwrap();\n /// }\n /// }\n /// ```\n pub fn stdout() -> Stdout {\n let std = io::stdout();\n // SAFETY: The `Read` implementation of `std` does not read from the\n // buffer it is borrowing and correctly reports the length of the data\n // written into the buffer.\n let blocking = unsafe { Blocking::new(std) };\n Stdout {\n std: SplitByUtf8BoundaryIfWindows::new(blocking),\n }\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, RawFd};\n\n use super::Stdout;\n\n impl AsRawFd for Stdout {\n fn as_raw_fd(&self) -> RawFd {\n std::io::stdout().as_raw_fd()\n }\n }\n\n impl AsFd for Stdout {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsHandle, BorrowedHandle, AsRawHandle, RawHandle};\n\n impl AsRawHandle for Stdout {\n fn as_raw_handle(&self) -> RawHandle {\n std::io::stdout().as_raw_handle()\n }\n }\n\n impl AsHandle for Stdout {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n }\n}\n\nimpl AsyncWrite for Stdout {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut self.std).poll_write(cx, buf)\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.std).poll_flush(cx)\n }\n\n fn poll_shutdown(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n Pin::new(&mut self.std).poll_shutdown(cx)\n }\n}\n" }, { "path": "tokio/src/io/uring/mod.rs", "content": "pub(crate) mod open;\npub(crate) mod read;\npub(crate) mod utils;\npub(crate) mod write;\n" }, { "path": "tokio/src/io/uring/open.rs", "content": "use super::utils::cstr;\n\nuse crate::fs::UringOpenOptions;\nuse crate::runtime::driver::op::{CancelData, Cancellable, Completable, CqeResult, Op};\n\nuse io_uring::{opcode, types};\nuse std::ffi::CString;\nuse std::io::{self, Error};\nuse std::os::fd::FromRawFd;\nuse std::path::Path;\n\n#[derive(Debug)]\npub(crate) struct Open {\n /// This field will be read by the kernel during the operation, so we\n /// need to ensure it is valid for the entire duration of the operation.\n #[allow(dead_code)]\n path: CString,\n}\n\nimpl Completable for Open {\n type Output = io::Result;\n fn complete(self, cqe: CqeResult) -> Self::Output {\n cqe.result\n .map(|fd| unsafe { crate::fs::File::from_raw_fd(fd as i32) })\n }\n\n fn complete_with_error(self, err: Error) -> Self::Output {\n Err(err)\n }\n}\n\nimpl Cancellable for Open {\n fn cancel(self) -> CancelData {\n CancelData::Open(self)\n }\n}\n\nimpl Op {\n /// Submit a request to open a file.\n pub(crate) fn open(path: &Path, options: &UringOpenOptions) -> io::Result> {\n let inner_opt = options;\n let path = cstr(path)?;\n\n let custom_flags = inner_opt.custom_flags;\n let flags = libc::O_CLOEXEC\n | options.access_mode()?\n | options.creation_mode()?\n | (custom_flags & !libc::O_ACCMODE);\n\n let open_op = opcode::OpenAt::new(types::Fd(libc::AT_FDCWD), path.as_ptr())\n .flags(flags)\n .mode(inner_opt.mode)\n .build();\n\n // SAFETY: Parameters are valid for the entire duration of the operation\n let op = unsafe { Op::new(open_op, Open { path }) };\n Ok(op)\n }\n}\n" }, { "path": "tokio/src/io/uring/read.rs", "content": "use crate::runtime::driver::op::{CancelData, Cancellable, Completable, CqeResult, Op};\n\nuse io_uring::{opcode, types};\nuse std::io::{self, Error};\nuse std::os::fd::{AsRawFd, OwnedFd};\n\n#[derive(Debug)]\npub(crate) struct Read {\n fd: OwnedFd,\n buf: Vec,\n}\n\nimpl Completable for Read {\n type Output = (io::Result, OwnedFd, Vec);\n\n fn complete(self, cqe: CqeResult) -> Self::Output {\n let mut buf = self.buf;\n\n if let Ok(len) = cqe.result {\n let new_len = buf.len() + len as usize;\n // SAFETY: Kernel read len bytes\n unsafe { buf.set_len(new_len) };\n }\n\n (cqe.result, self.fd, buf)\n }\n\n fn complete_with_error(self, err: Error) -> Self::Output {\n (Err(err), self.fd, self.buf)\n }\n}\n\nimpl Cancellable for Read {\n fn cancel(self) -> CancelData {\n CancelData::Read(self)\n }\n}\n\nimpl Op {\n // Submit a request to read a FD at given length and offset into a\n // dynamic buffer with uninitialized memory. The read happens on uninitialized\n // buffer and no overwriting happens.\n\n // SAFETY: The `len` of the amount to be read and the buffer that is passed\n // should have capacity > len.\n //\n // If `len` read is higher than vector capacity then setting its length by\n // the caller in terms of size_read can be unsound.\n pub(crate) fn read(fd: OwnedFd, mut buf: Vec, len: u32, offset: u64) -> Self {\n // don't overwrite on already written part\n assert!(buf.spare_capacity_mut().len() >= len as usize);\n let buf_mut_ptr = buf.spare_capacity_mut().as_mut_ptr().cast();\n\n let read_op = opcode::Read::new(types::Fd(fd.as_raw_fd()), buf_mut_ptr, len)\n .offset(offset)\n .build();\n\n // SAFETY: Parameters are valid for the entire duration of the operation\n unsafe { Op::new(read_op, Read { fd, buf }) }\n }\n}\n" }, { "path": "tokio/src/io/uring/utils.rs", "content": "use std::os::unix::ffi::OsStrExt;\nuse std::{ffi::CString, io, path::Path};\n\npub(crate) fn cstr(p: &Path) -> io::Result {\n Ok(CString::new(p.as_os_str().as_bytes())?)\n}\n" }, { "path": "tokio/src/io/uring/write.rs", "content": "use crate::runtime::driver::op::{CancelData, Cancellable, Completable, CqeResult, Op};\nuse crate::util::as_ref::OwnedBuf;\n\nuse io_uring::{opcode, types};\nuse std::io::{self, Error};\nuse std::os::fd::{AsRawFd, OwnedFd};\n\n#[derive(Debug)]\npub(crate) struct Write {\n buf: OwnedBuf,\n fd: OwnedFd,\n}\n\nimpl Completable for Write {\n type Output = (io::Result, OwnedBuf, OwnedFd);\n fn complete(self, cqe: CqeResult) -> Self::Output {\n (cqe.result, self.buf, self.fd)\n }\n\n fn complete_with_error(self, err: Error) -> Self::Output {\n (Err(err), self.buf, self.fd)\n }\n}\n\nimpl Cancellable for Write {\n fn cancel(self) -> CancelData {\n CancelData::Write(self)\n }\n}\n\nimpl Op {\n /// Issue a write that starts at `buf_offset` within `buf` and writes some bytes\n /// into `file` at `file_offset`.\n pub(crate) fn write_at(\n fd: OwnedFd,\n buf: OwnedBuf,\n buf_offset: usize,\n file_offset: u64,\n ) -> io::Result {\n // There is a cap on how many bytes we can write in a single uring write operation.\n // ref: https://github.com/axboe/liburing/discussions/497\n let len = u32::try_from(buf.as_ref().len() - buf_offset).unwrap_or(u32::MAX);\n\n let ptr = buf.as_ref()[buf_offset..buf_offset + len as usize].as_ptr();\n\n let sqe = opcode::Write::new(types::Fd(fd.as_raw_fd()), ptr, len)\n .offset(file_offset)\n .build();\n\n // SAFETY: parameters of the entry, such as `fd` and `buf`, are valid\n // until this operation completes.\n let op = unsafe { Op::new(sqe, Write { buf, fd }) };\n Ok(op)\n }\n}\n" }, { "path": "tokio/src/io/util/async_buf_read_ext.rs", "content": "use crate::io::util::fill_buf::{fill_buf, FillBuf};\nuse crate::io::util::lines::{lines, Lines};\nuse crate::io::util::read_line::{read_line, ReadLine};\nuse crate::io::util::read_until::{read_until, ReadUntil};\nuse crate::io::util::split::{split, Split};\nuse crate::io::AsyncBufRead;\n\ncfg_io_util! {\n /// An extension trait which adds utility methods to [`AsyncBufRead`] types.\n ///\n /// [`AsyncBufRead`]: crate::io::AsyncBufRead\n pub trait AsyncBufReadExt: AsyncBufRead {\n /// Reads all bytes into `buf` until the delimiter `byte` or EOF is reached.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_until(&mut self, byte: u8, buf: &mut Vec) -> io::Result;\n /// ```\n ///\n /// This function will read bytes from the underlying stream until the\n /// delimiter or EOF is found. Once found, all bytes up to, and including,\n /// the delimiter (if found) will be appended to `buf`.\n ///\n /// If successful, this function will return the total number of bytes read.\n ///\n /// If this function returns `Ok(0)`, the stream has reached EOF.\n ///\n /// # Errors\n ///\n /// This function will ignore all instances of [`ErrorKind::Interrupted`] and\n /// will otherwise return any errors returned by [`fill_buf`].\n ///\n /// If an I/O error is encountered then all bytes read so far will be\n /// present in `buf` and its length will have been adjusted appropriately.\n ///\n /// [`fill_buf`]: AsyncBufRead::poll_fill_buf\n /// [`ErrorKind::Interrupted`]: std::io::ErrorKind::Interrupted\n ///\n /// # Cancel safety\n ///\n /// If the method is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then some data may have been partially read. Any\n /// partially read bytes are appended to `buf`, and the method can be\n /// called again to continue reading until `byte`.\n ///\n /// This method returns the total number of bytes read. If you cancel\n /// the call to `read_until` and then call it again to continue reading,\n /// the counter is reset.\n ///\n /// # Examples\n ///\n /// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In\n /// this example, we use [`Cursor`] to read all the bytes in a byte slice\n /// in hyphen delimited segments:\n ///\n /// [`Cursor`]: std::io::Cursor\n ///\n /// ```\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut cursor = Cursor::new(b\"lorem-ipsum\");\n /// let mut buf = vec![];\n ///\n /// // cursor is at 'l'\n /// let num_bytes = cursor.read_until(b'-', &mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n ///\n /// assert_eq!(num_bytes, 6);\n /// assert_eq!(buf, b\"lorem-\");\n /// buf.clear();\n ///\n /// // cursor is at 'i'\n /// let num_bytes = cursor.read_until(b'-', &mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n ///\n /// assert_eq!(num_bytes, 5);\n /// assert_eq!(buf, b\"ipsum\");\n /// buf.clear();\n ///\n /// // cursor is at EOF\n /// let num_bytes = cursor.read_until(b'-', &mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n /// assert_eq!(num_bytes, 0);\n /// assert_eq!(buf, b\"\");\n /// # }\n /// ```\n fn read_until<'a>(&'a mut self, byte: u8, buf: &'a mut Vec) -> ReadUntil<'a, Self>\n where\n Self: Unpin,\n {\n read_until(self, byte, buf)\n }\n\n /// Reads all bytes until a newline (the 0xA byte) is reached, and append\n /// them to the provided buffer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_line(&mut self, buf: &mut String) -> io::Result;\n /// ```\n ///\n /// This function will read bytes from the underlying stream until the\n /// newline delimiter (the 0xA byte) or EOF is found. Once found, all bytes\n /// up to, and including, the delimiter (if found) will be appended to\n /// `buf`.\n ///\n /// If successful, this function will return the total number of bytes read.\n ///\n /// If this function returns `Ok(0)`, the stream has reached EOF.\n ///\n /// # Errors\n ///\n /// This function has the same error semantics as [`read_until`] and will\n /// also return an error if the read bytes are not valid UTF-8. If an I/O\n /// error is encountered then `buf` may contain some bytes already read in\n /// the event that all data read so far was valid UTF-8.\n ///\n /// [`read_until`]: AsyncBufReadExt::read_until\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may have been partially\n /// read, and this data is lost. There are no guarantees regarding the\n /// contents of `buf` when the call is cancelled. The current\n /// implementation replaces `buf` with the empty string, but this may\n /// change in the future.\n ///\n /// This function does not behave like [`read_until`] because of the\n /// requirement that a string contains only valid utf-8. If you need a\n /// cancellation safe `read_line`, there are three options:\n ///\n /// * Call [`read_until`] with a newline character and manually perform the utf-8 check.\n /// * The stream returned by [`lines`] has a cancellation safe\n /// [`next_line`] method.\n /// * Use [`tokio_util::codec::LinesCodec`][LinesCodec].\n ///\n /// [LinesCodec]: https://docs.rs/tokio-util/latest/tokio_util/codec/struct.LinesCodec.html\n /// [`read_until`]: Self::read_until\n /// [`lines`]: Self::lines\n /// [`next_line`]: crate::io::Lines::next_line\n ///\n /// # Examples\n ///\n /// [`std::io::Cursor`][`Cursor`] is a type that implements\n /// `AsyncBufRead`. In this example, we use [`Cursor`] to read all the\n /// lines in a byte slice:\n ///\n /// [`Cursor`]: std::io::Cursor\n ///\n /// ```\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut cursor = Cursor::new(b\"foo\\nbar\");\n /// let mut buf = String::new();\n ///\n /// // cursor is at 'f'\n /// let num_bytes = cursor.read_line(&mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n ///\n /// assert_eq!(num_bytes, 4);\n /// assert_eq!(buf, \"foo\\n\");\n /// buf.clear();\n ///\n /// // cursor is at 'b'\n /// let num_bytes = cursor.read_line(&mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n ///\n /// assert_eq!(num_bytes, 3);\n /// assert_eq!(buf, \"bar\");\n /// buf.clear();\n ///\n /// // cursor is at EOF\n /// let num_bytes = cursor.read_line(&mut buf)\n /// .await\n /// .expect(\"reading from cursor won't fail\");\n ///\n /// assert_eq!(num_bytes, 0);\n /// assert_eq!(buf, \"\");\n /// # }\n /// ```\n fn read_line<'a>(&'a mut self, buf: &'a mut String) -> ReadLine<'a, Self>\n where\n Self: Unpin,\n {\n read_line(self, buf)\n }\n\n /// Returns a stream of the contents of this reader split on the byte\n /// `byte`.\n ///\n /// This method is the asynchronous equivalent to\n /// [`BufRead::split`](std::io::BufRead::split).\n ///\n /// The stream returned from this function will yield instances of\n /// [`io::Result`]`<`[`Option`]`<`[`Vec`]`>>`. Each vector returned will *not* have\n /// the delimiter byte at the end.\n ///\n /// [`io::Result`]: std::io::Result\n /// [`Option`]: core::option::Option\n /// [`Vec`]: std::vec::Vec\n ///\n /// # Errors\n ///\n /// Each item of the stream has the same error semantics as\n /// [`AsyncBufReadExt::read_until`](AsyncBufReadExt::read_until).\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncBufRead;\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// # async fn dox(my_buf_read: impl AsyncBufRead + Unpin) -> std::io::Result<()> {\n /// let mut segments = my_buf_read.split(b'f');\n ///\n /// while let Some(segment) = segments.next_segment().await? {\n /// println!(\"length = {}\", segment.len())\n /// }\n /// # Ok(())\n /// # }\n /// ```\n fn split(self, byte: u8) -> Split\n where\n Self: Sized + Unpin,\n {\n split(self, byte)\n }\n\n /// Returns the contents of the internal buffer, filling it with more\n /// data from the inner reader if it is empty.\n ///\n /// This function is a lower-level call. It needs to be paired with the\n /// [`consume`] method to function properly. When calling this method,\n /// none of the contents will be \"read\" in the sense that later calling\n /// `read` may return the same contents. As such, [`consume`] must be\n /// called with the number of bytes that are consumed from this buffer\n /// to ensure that the bytes are never returned twice.\n ///\n /// An empty buffer returned indicates that the stream has reached EOF.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn fill_buf(&mut self) -> io::Result<&[u8]>;\n /// ```\n ///\n /// # Errors\n ///\n /// This function will return an I/O error if the underlying reader was\n /// read, but returned an error.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no data was read.\n ///\n /// [`consume`]: crate::io::AsyncBufReadExt::consume\n fn fill_buf(&mut self) -> FillBuf<'_, Self>\n where\n Self: Unpin,\n {\n fill_buf(self)\n }\n\n /// Tells this buffer that `amt` bytes have been consumed from the\n /// buffer, so they should no longer be returned in calls to [`read`].\n ///\n /// This function is a lower-level call. It needs to be paired with the\n /// [`fill_buf`] method to function properly. This function does not\n /// perform any I/O, it simply informs this object that some amount of\n /// its buffer, returned from [`fill_buf`], has been consumed and should\n /// no longer be returned. As such, this function may do odd things if\n /// [`fill_buf`] isn't called before calling it.\n ///\n /// The `amt` must be less than the number of bytes in the buffer\n /// returned by [`fill_buf`].\n ///\n /// [`read`]: crate::io::AsyncReadExt::read\n /// [`fill_buf`]: crate::io::AsyncBufReadExt::fill_buf\n fn consume(&mut self, amt: usize)\n where\n Self: Unpin,\n {\n std::pin::Pin::new(self).consume(amt);\n }\n\n /// Returns a stream over the lines of this reader.\n /// This method is the async equivalent to [`BufRead::lines`](std::io::BufRead::lines).\n ///\n /// The stream returned from this function will yield instances of\n /// [`io::Result`]`<`[`Option`]`<`[`String`]`>>`. Each string returned will *not* have a newline\n /// byte (the 0xA byte) or `CRLF` (0xD, 0xA bytes) at the end.\n ///\n /// [`io::Result`]: std::io::Result\n /// [`Option`]: core::option::Option\n /// [`String`]: String\n ///\n /// # Errors\n ///\n /// Each line of the stream has the same error semantics as [`AsyncBufReadExt::read_line`].\n ///\n /// # Examples\n ///\n /// [`std::io::Cursor`][`Cursor`] is a type that implements `BufRead`. In\n /// this example, we use [`Cursor`] to iterate over all the lines in a byte\n /// slice.\n ///\n /// [`Cursor`]: std::io::Cursor\n ///\n /// ```\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let cursor = Cursor::new(b\"lorem\\nipsum\\r\\ndolor\");\n ///\n /// let mut lines = cursor.lines();\n ///\n /// assert_eq!(lines.next_line().await.unwrap(), Some(String::from(\"lorem\")));\n /// assert_eq!(lines.next_line().await.unwrap(), Some(String::from(\"ipsum\")));\n /// assert_eq!(lines.next_line().await.unwrap(), Some(String::from(\"dolor\")));\n /// assert_eq!(lines.next_line().await.unwrap(), None);\n /// # }\n /// ```\n ///\n /// [`AsyncBufReadExt::read_line`]: AsyncBufReadExt::read_line\n fn lines(self) -> Lines\n where\n Self: Sized,\n {\n lines(self)\n }\n }\n}\n\nimpl AsyncBufReadExt for R {}\n" }, { "path": "tokio/src/io/util/async_read_ext.rs", "content": "use crate::io::util::chain::{chain, Chain};\nuse crate::io::util::read::{read, Read};\nuse crate::io::util::read_buf::{read_buf, ReadBuf};\nuse crate::io::util::read_exact::{read_exact, ReadExact};\nuse crate::io::util::read_int::{ReadF32, ReadF32Le, ReadF64, ReadF64Le};\nuse crate::io::util::read_int::{\n ReadI128, ReadI128Le, ReadI16, ReadI16Le, ReadI32, ReadI32Le, ReadI64, ReadI64Le, ReadI8,\n};\nuse crate::io::util::read_int::{\n ReadU128, ReadU128Le, ReadU16, ReadU16Le, ReadU32, ReadU32Le, ReadU64, ReadU64Le, ReadU8,\n};\nuse crate::io::util::read_to_end::{read_to_end, ReadToEnd};\nuse crate::io::util::read_to_string::{read_to_string, ReadToString};\nuse crate::io::util::take::{take, Take};\nuse crate::io::AsyncRead;\n\nuse bytes::BufMut;\n\ncfg_io_util! {\n /// Defines numeric reader\n macro_rules! read_impl {\n (\n $(\n $(#[$outer:meta])*\n fn $name:ident(&mut self) -> $($fut:ident)*;\n )*\n ) => {\n $(\n $(#[$outer])*\n fn $name(&mut self) -> $($fut)*<&mut Self> where Self: Unpin {\n $($fut)*::new(self)\n }\n )*\n }\n }\n\n /// Reads bytes from a source.\n ///\n /// Implemented as an extension trait, adding utility methods to all\n /// [`AsyncRead`] types. Callers will tend to import this trait instead of\n /// [`AsyncRead`].\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let mut buffer = [0; 10];\n ///\n /// // The `read` method is defined by this trait.\n /// let n = f.read(&mut buffer[..]).await?;\n ///\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// See [module][crate::io] documentation for more details.\n ///\n /// [`AsyncRead`]: AsyncRead\n pub trait AsyncReadExt: AsyncRead {\n /// Creates a new `AsyncRead` instance that chains this stream with\n /// `next`.\n ///\n /// The returned `AsyncRead` instance will first read all bytes from this object\n /// until EOF is encountered. Afterwards the output is equivalent to the\n /// output of `next`.\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `AsyncRead`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let f1 = File::open(\"foo.txt\").await?;\n /// let f2 = File::open(\"bar.txt\").await?;\n ///\n /// let mut handle = f1.chain(f2);\n /// let mut buffer = String::new();\n ///\n /// // read the value into a String. We could use any AsyncRead\n /// // method here, this is just one example.\n /// handle.read_to_string(&mut buffer).await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn chain(self, next: R) -> Chain\n where\n Self: Sized,\n R: AsyncRead,\n {\n chain(self, next)\n }\n\n /// Pulls some bytes from this source into the specified buffer,\n /// returning how many bytes were read.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read(&mut self, buf: &mut [u8]) -> io::Result;\n /// ```\n ///\n /// This method does not provide any guarantees about whether it\n /// completes immediately or asynchronously.\n ///\n /// # Return\n ///\n /// If the return value of this method is `Ok(n)`, then it must be\n /// guaranteed that `0 <= n <= buf.len()`. A nonzero `n` value indicates\n /// that the buffer `buf` has been filled in with `n` bytes of data from\n /// this source. If `n` is `0`, then it can indicate one of two\n /// scenarios:\n ///\n /// 1. This reader has reached its \"end of file\" and will likely no longer\n /// be able to produce bytes. Note that this does not mean that the\n /// reader will *always* no longer be able to produce bytes.\n /// 2. The buffer specified was 0 bytes in length.\n ///\n /// No guarantees are provided about the contents of `buf` when this\n /// function is called, implementations cannot rely on any property of the\n /// contents of `buf` being true. It is recommended that *implementations*\n /// only write data to `buf` instead of reading its contents.\n ///\n /// Correspondingly, however, *callers* of this method may not assume\n /// any guarantees about how the implementation uses `buf`. It is\n /// possible that the code that's supposed to write to the buffer might\n /// also read from it. It is your responsibility to make sure that `buf`\n /// is initialized before calling `read`.\n ///\n /// # Errors\n ///\n /// If this function encounters any form of I/O or other error, an error\n /// variant will be returned. If an error is returned then it must be\n /// guaranteed that no bytes were read.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no data was read.\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `Read`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let mut buffer = [0; 10];\n ///\n /// // read up to 10 bytes\n /// let n = f.read(&mut buffer[..]).await?;\n ///\n /// println!(\"The bytes: {:?}\", &buffer[..n]);\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn read<'a>(&'a mut self, buf: &'a mut [u8]) -> Read<'a, Self>\n where\n Self: Unpin,\n {\n read(self, buf)\n }\n\n /// Pulls some bytes from this source into the specified buffer,\n /// advancing the buffer's internal cursor.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_buf(&mut self, buf: &mut B) -> io::Result;\n /// ```\n ///\n /// Usually, only a single `read` syscall is issued, even if there is\n /// more space in the supplied buffer.\n ///\n /// This method does not provide any guarantees about whether it\n /// completes immediately or asynchronously.\n ///\n /// # Return\n ///\n /// A nonzero `n` value indicates that the buffer `buf` has been filled\n /// in with `n` bytes of data from this source. If `n` is `0`, then it\n /// can indicate one of two scenarios:\n ///\n /// 1. This reader has reached its \"end of file\" and will likely no longer\n /// be able to produce bytes. Note that this does not mean that the\n /// reader will *always* no longer be able to produce bytes.\n /// 2. The buffer specified had a remaining capacity of zero.\n ///\n /// # Errors\n ///\n /// If this function encounters any form of I/O or other error, an error\n /// variant will be returned. If an error is returned then it must be\n /// guaranteed that no bytes were read.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no data was read.\n ///\n /// # Examples\n ///\n /// [`File`] implements `Read` and [`BytesMut`] implements [`BufMut`]:\n ///\n /// [`File`]: crate::fs::File\n /// [`BytesMut`]: bytes::BytesMut\n /// [`BufMut`]: bytes::BufMut\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use bytes::BytesMut;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let mut buffer = BytesMut::with_capacity(10);\n ///\n /// assert!(buffer.is_empty());\n /// assert!(buffer.capacity() >= 10);\n ///\n /// // note that the return value is not needed to access the data\n /// // that was read as `buffer`'s internal cursor is updated.\n /// //\n /// // this might read more than 10 bytes if the capacity of `buffer`\n /// // is larger than 10.\n /// f.read_buf(&mut buffer).await?;\n ///\n /// println!(\"The bytes: {:?}\", &buffer[..]);\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn read_buf<'a, B>(&'a mut self, buf: &'a mut B) -> ReadBuf<'a, Self, B>\n where\n Self: Unpin,\n B: BufMut + ?Sized,\n {\n read_buf(self, buf)\n }\n\n /// Reads the exact number of bytes required to fill `buf`.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_exact(&mut self, buf: &mut [u8]) -> io::Result;\n /// ```\n ///\n /// This function reads as many bytes as necessary to completely fill\n /// the specified buffer `buf`.\n ///\n /// # Errors\n ///\n /// If the operation encounters an \"end of file\" before completely\n /// filling the buffer, it returns an error of the kind\n /// [`ErrorKind::UnexpectedEof`]. The contents of `buf` are unspecified\n /// in this case.\n ///\n /// If any other read error is encountered then the operation\n /// immediately returns. The contents of `buf` are unspecified in this\n /// case.\n ///\n /// If this operation returns an error, it is unspecified how many bytes\n /// it has read, but it will never read more than would be necessary to\n /// completely fill the buffer.\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may already have been\n /// read into `buf`.\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `Read`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let len = 10;\n /// let mut buffer = vec![0; len];\n ///\n /// // read exactly 10 bytes\n /// f.read_exact(&mut buffer).await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// [`ErrorKind::UnexpectedEof`]: std::io::ErrorKind::UnexpectedEof\n fn read_exact<'a>(&'a mut self, buf: &'a mut [u8]) -> ReadExact<'a, Self>\n where\n Self: Unpin,\n {\n read_exact(self, buf)\n }\n\n read_impl! {\n /// Reads an unsigned 8 bit integer from the underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u8(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If this method is used as an event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no data were read.\n ///\n /// # Examples\n ///\n /// Read unsigned 8 bit integers from an `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![2, 5]);\n ///\n /// assert_eq!(2, reader.read_u8().await?);\n /// assert_eq!(5, reader.read_u8().await?);\n ///\n /// Ok(())\n /// # }\n /// ```\n fn read_u8(&mut self) -> ReadU8;\n\n /// Reads a signed 8 bit integer from the underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i8(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If this method is used as an event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no data were read.\n ///\n /// # Examples\n ///\n /// Read unsigned 8 bit integers from an `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x02, 0xfb]);\n ///\n /// assert_eq!(2, reader.read_i8().await?);\n /// assert_eq!(-5, reader.read_i8().await?);\n ///\n /// Ok(())\n /// # }\n /// ```\n fn read_i8(&mut self) -> ReadI8;\n\n /// Reads an unsigned 16-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u16(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 16 bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![2, 5, 3, 0]);\n ///\n /// assert_eq!(517, reader.read_u16().await?);\n /// assert_eq!(768, reader.read_u16().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u16(&mut self) -> ReadU16;\n\n /// Reads a signed 16-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i16(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 16 bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x00, 0xc1, 0xff, 0x7c]);\n ///\n /// assert_eq!(193, reader.read_i16().await?);\n /// assert_eq!(-132, reader.read_i16().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i16(&mut self) -> ReadI16;\n\n /// Reads an unsigned 32-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u32(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 32-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x00, 0x00, 0x01, 0x0b]);\n ///\n /// assert_eq!(267, reader.read_u32().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u32(&mut self) -> ReadU32;\n\n /// Reads a signed 32-bit integer in big-endian order from the\n /// underlying reader.\n ///\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i32(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 32-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0xff, 0xff, 0x7a, 0x33]);\n ///\n /// assert_eq!(-34253, reader.read_i32().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i32(&mut self) -> ReadI32;\n\n /// Reads an unsigned 64-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u64(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 64-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83\n /// ]);\n ///\n /// assert_eq!(918733457491587, reader.read_u64().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u64(&mut self) -> ReadU64;\n\n /// Reads an signed 64-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i64(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 64-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x80, 0, 0, 0, 0, 0, 0, 0]);\n ///\n /// assert_eq!(i64::MIN, reader.read_i64().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i64(&mut self) -> ReadI64;\n\n /// Reads an unsigned 128-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u128(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 128-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83,\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83\n /// ]);\n ///\n /// assert_eq!(16947640962301618749969007319746179, reader.read_u128().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u128(&mut self) -> ReadU128;\n\n /// Reads an signed 128-bit integer in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i128(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 128-bit big-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x80, 0, 0, 0, 0, 0, 0, 0,\n /// 0, 0, 0, 0, 0, 0, 0, 0\n /// ]);\n ///\n /// assert_eq!(i128::MIN, reader.read_i128().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i128(&mut self) -> ReadI128;\n\n /// Reads an 32-bit floating point type in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_f32(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read 32-bit floating point type from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0xff, 0x7f, 0xff, 0xff]);\n ///\n /// assert_eq!(f32::MIN, reader.read_f32().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_f32(&mut self) -> ReadF32;\n\n /// Reads an 64-bit floating point type in big-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_f64(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read 64-bit floating point type from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0xff, 0xef, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff\n /// ]);\n ///\n /// assert_eq!(f64::MIN, reader.read_f64().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_f64(&mut self) -> ReadF64;\n\n /// Reads an unsigned 16-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u16_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 16 bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![2, 5, 3, 0]);\n ///\n /// assert_eq!(1282, reader.read_u16_le().await?);\n /// assert_eq!(3, reader.read_u16_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u16_le(&mut self) -> ReadU16Le;\n\n /// Reads a signed 16-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i16_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 16 bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x00, 0xc1, 0xff, 0x7c]);\n ///\n /// assert_eq!(-16128, reader.read_i16_le().await?);\n /// assert_eq!(31999, reader.read_i16_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i16_le(&mut self) -> ReadI16Le;\n\n /// Reads an unsigned 32-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u32_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 32-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x00, 0x00, 0x01, 0x0b]);\n ///\n /// assert_eq!(184614912, reader.read_u32_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u32_le(&mut self) -> ReadU32Le;\n\n /// Reads a signed 32-bit integer in little-endian order from the\n /// underlying reader.\n ///\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i32_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 32-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0xff, 0xff, 0x7a, 0x33]);\n ///\n /// assert_eq!(863698943, reader.read_i32_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i32_le(&mut self) -> ReadI32Le;\n\n /// Reads an unsigned 64-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u64_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 64-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83\n /// ]);\n ///\n /// assert_eq!(9477368352180732672, reader.read_u64_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u64_le(&mut self) -> ReadU64Le;\n\n /// Reads an signed 64-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i64_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 64-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0x80, 0, 0, 0, 0, 0, 0, 0]);\n ///\n /// assert_eq!(128, reader.read_i64_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i64_le(&mut self) -> ReadI64Le;\n\n /// Reads an unsigned 128-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_u128_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read unsigned 128-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83,\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83\n /// ]);\n ///\n /// assert_eq!(174826588484952389081207917399662330624, reader.read_u128_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_u128_le(&mut self) -> ReadU128Le;\n\n /// Reads an signed 128-bit integer in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_i128_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read signed 128-bit little-endian integers from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0x80, 0, 0, 0, 0, 0, 0, 0,\n /// 0, 0, 0, 0, 0, 0, 0, 0\n /// ]);\n ///\n /// assert_eq!(128, reader.read_i128_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_i128_le(&mut self) -> ReadI128Le;\n\n /// Reads an 32-bit floating point type in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_f32_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read 32-bit floating point type from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![0xff, 0xff, 0x7f, 0xff]);\n ///\n /// assert_eq!(f32::MIN, reader.read_f32_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_f32_le(&mut self) -> ReadF32Le;\n\n /// Reads an 64-bit floating point type in little-endian order from the\n /// underlying reader.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_f64_le(&mut self) -> io::Result;\n /// ```\n ///\n /// It is recommended to use a buffered reader to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncReadExt::read_exact`].\n ///\n /// [`AsyncReadExt::read_exact`]: AsyncReadExt::read_exact\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If the method is used as the\n /// event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then some data may be lost.\n ///\n /// # Examples\n ///\n /// Read 64-bit floating point type from a `AsyncRead`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// use std::io::Cursor;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut reader = Cursor::new(vec![\n /// 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xef, 0xff\n /// ]);\n ///\n /// assert_eq!(f64::MIN, reader.read_f64_le().await?);\n /// Ok(())\n /// # }\n /// ```\n fn read_f64_le(&mut self) -> ReadF64Le;\n }\n\n /// Reads all bytes until EOF in this source, placing them into `buf`.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_to_end(&mut self, buf: &mut Vec) -> io::Result;\n /// ```\n ///\n /// All bytes read from this source will be appended to the specified\n /// buffer `buf`. This function will continuously call [`read()`] to\n /// append more data to `buf` until [`read()`] returns `Ok(0)`.\n ///\n /// If successful, the total number of bytes read is returned.\n ///\n /// [`read()`]: AsyncReadExt::read\n ///\n /// # Errors\n ///\n /// If a read error is encountered then the `read_to_end` operation\n /// immediately completes. Any bytes which have already been read will\n /// be appended to `buf`.\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `Read`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncReadExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let mut buffer = Vec::new();\n ///\n /// // read the whole file\n /// f.read_to_end(&mut buffer).await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// (See also the [`tokio::fs::read`] convenience function for reading from a\n /// file.)\n ///\n /// [`tokio::fs::read`]: fn@crate::fs::read\n fn read_to_end<'a>(&'a mut self, buf: &'a mut Vec) -> ReadToEnd<'a, Self>\n where\n Self: Unpin,\n {\n read_to_end(self, buf)\n }\n\n /// Reads all bytes until EOF in this source, appending them to `buf`.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn read_to_string(&mut self, buf: &mut String) -> io::Result;\n /// ```\n ///\n /// If successful, the number of bytes which were read and appended to\n /// `buf` is returned.\n ///\n /// # Errors\n ///\n /// If the data in this stream is *not* valid UTF-8 then an error is\n /// returned and `buf` is unchanged.\n ///\n /// See [`read_to_end`][AsyncReadExt::read_to_end] for other error semantics.\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `Read`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncReadExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut f = File::open(\"foo.txt\").await?;\n /// let mut buffer = String::new();\n ///\n /// f.read_to_string(&mut buffer).await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// (See also the [`crate::fs::read_to_string`] convenience function for\n /// reading from a file.)\n ///\n /// [`crate::fs::read_to_string`]: fn@crate::fs::read_to_string\n fn read_to_string<'a>(&'a mut self, dst: &'a mut String) -> ReadToString<'a, Self>\n where\n Self: Unpin,\n {\n read_to_string(self, dst)\n }\n\n /// Creates an adaptor which reads at most `limit` bytes from it.\n ///\n /// This function returns a new instance of `AsyncRead` which will read\n /// at most `limit` bytes, after which it will always return EOF\n /// (`Ok(0)`). Any read errors will not count towards the number of\n /// bytes read and future calls to [`read()`] may succeed.\n ///\n /// [`read()`]: fn@crate::io::AsyncReadExt::read\n ///\n /// [read]: AsyncReadExt::read\n ///\n /// # Examples\n ///\n /// [`File`][crate::fs::File]s implement `Read`:\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncReadExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let f = File::open(\"foo.txt\").await?;\n /// let mut buffer = [0; 5];\n ///\n /// // read at most five bytes\n /// let mut handle = f.take(5);\n ///\n /// handle.read(&mut buffer).await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn take(self, limit: u64) -> Take\n where\n Self: Sized,\n {\n take(self, limit)\n }\n }\n}\n\nimpl AsyncReadExt for R {}\n" }, { "path": "tokio/src/io/util/async_seek_ext.rs", "content": "use crate::io::seek::{seek, Seek};\nuse crate::io::AsyncSeek;\nuse std::io::SeekFrom;\n\ncfg_io_util! {\n /// An extension trait that adds utility methods to [`AsyncSeek`] types.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::io::{self, Cursor, SeekFrom};\n /// use tokio::io::{AsyncSeekExt, AsyncReadExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut cursor = Cursor::new(b\"abcdefg\");\n ///\n /// // the `seek` method is defined by this trait\n /// cursor.seek(SeekFrom::Start(3)).await?;\n ///\n /// let mut buf = [0; 1];\n /// let n = cursor.read(&mut buf).await?;\n /// assert_eq!(n, 1);\n /// assert_eq!(buf, [b'd']);\n ///\n /// Ok(())\n /// # }\n /// ```\n ///\n /// See [module][crate::io] documentation for more details.\n ///\n /// [`AsyncSeek`]: AsyncSeek\n pub trait AsyncSeekExt: AsyncSeek {\n /// Creates a future which will seek an IO object, and then yield the\n /// new position in the object and the object itself.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn seek(&mut self, pos: SeekFrom) -> io::Result;\n /// ```\n ///\n /// In the case of an error the buffer and the object will be discarded, with\n /// the error yielded.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::fs::File;\n /// use tokio::io::{AsyncSeekExt, AsyncReadExt};\n ///\n /// use std::io::SeekFrom;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut file = File::open(\"foo.txt\").await?;\n /// file.seek(SeekFrom::Start(6)).await?;\n ///\n /// let mut contents = vec![0u8; 10];\n /// file.read_exact(&mut contents).await?;\n /// # Ok(())\n /// # }\n /// # }\n /// ```\n fn seek(&mut self, pos: SeekFrom) -> Seek<'_, Self>\n where\n Self: Unpin,\n {\n seek(self, pos)\n }\n\n /// Creates a future which will rewind to the beginning of the stream.\n ///\n /// This is convenience method, equivalent to `self.seek(SeekFrom::Start(0))`.\n fn rewind(&mut self) -> Seek<'_, Self>\n where\n Self: Unpin,\n {\n self.seek(SeekFrom::Start(0))\n }\n\n /// Creates a future which will return the current seek position from the\n /// start of the stream.\n ///\n /// This is equivalent to `self.seek(SeekFrom::Current(0))`.\n fn stream_position(&mut self) -> Seek<'_, Self>\n where\n Self: Unpin,\n {\n self.seek(SeekFrom::Current(0))\n }\n }\n}\n\nimpl AsyncSeekExt for S {}\n" }, { "path": "tokio/src/io/util/async_write_ext.rs", "content": "use crate::io::util::flush::{flush, Flush};\nuse crate::io::util::shutdown::{shutdown, Shutdown};\nuse crate::io::util::write::{write, Write};\nuse crate::io::util::write_all::{write_all, WriteAll};\nuse crate::io::util::write_all_buf::{write_all_buf, WriteAllBuf};\nuse crate::io::util::write_buf::{write_buf, WriteBuf};\nuse crate::io::util::write_int::{WriteF32, WriteF32Le, WriteF64, WriteF64Le};\nuse crate::io::util::write_int::{\n WriteI128, WriteI128Le, WriteI16, WriteI16Le, WriteI32, WriteI32Le, WriteI64, WriteI64Le,\n WriteI8,\n};\nuse crate::io::util::write_int::{\n WriteU128, WriteU128Le, WriteU16, WriteU16Le, WriteU32, WriteU32Le, WriteU64, WriteU64Le,\n WriteU8,\n};\nuse crate::io::util::write_vectored::{write_vectored, WriteVectored};\nuse crate::io::AsyncWrite;\nuse std::io::IoSlice;\n\nuse bytes::Buf;\n\ncfg_io_util! {\n /// Defines numeric writer.\n macro_rules! write_impl {\n (\n $(\n $(#[$outer:meta])*\n fn $name:ident(&mut self, n: $ty:ty) -> $($fut:ident)*;\n )*\n ) => {\n $(\n $(#[$outer])*\n fn $name(&mut self, n: $ty) -> $($fut)*<&mut Self> where Self: Unpin {\n $($fut)*::new(self, n)\n }\n )*\n }\n }\n\n /// Writes bytes to a sink.\n ///\n /// Implemented as an extension trait, adding utility methods to all\n /// [`AsyncWrite`] types. Callers will tend to import this trait instead of\n /// [`AsyncWrite`].\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let data = b\"some bytes\";\n ///\n /// let mut pos = 0;\n /// let mut buffer = File::create(\"foo.txt\").await?;\n ///\n /// while pos < data.len() {\n /// let bytes_written = buffer.write(&data[pos..]).await?;\n /// pos += bytes_written;\n /// }\n ///\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// See [module][crate::io] documentation for more details.\n ///\n /// [`AsyncWrite`]: AsyncWrite\n pub trait AsyncWriteExt: AsyncWrite {\n /// Writes a buffer into this writer, returning how many bytes were\n /// written.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write(&mut self, buf: &[u8]) -> io::Result;\n /// ```\n ///\n /// This function will attempt to write the entire contents of `buf`, but\n /// the entire write may not succeed, or the write may also generate an\n /// error. A call to `write` represents *at most one* attempt to write to\n /// any wrapped object.\n ///\n /// # Return\n ///\n /// If the return value is `Ok(n)` then it must be guaranteed that `n <=\n /// buf.len()`. A return value of `0` typically means that the\n /// underlying object is no longer able to accept bytes and will likely\n /// not be able to in the future as well, or that the buffer provided is\n /// empty.\n ///\n /// # Errors\n ///\n /// Each call to `write` may generate an I/O error indicating that the\n /// operation could not be completed. If an error is returned then no bytes\n /// in the buffer were written to this writer.\n ///\n /// It is **not** considered an error if the entire buffer could not be\n /// written to this writer.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe in the sense that if it is used as\n /// the event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then it is guaranteed that no data was\n /// written to this `AsyncWrite`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n ///\n /// // Writes some prefix of the byte string, not necessarily all of it.\n /// file.write(b\"some bytes\").await?;\n /// file.flush().await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn write<'a>(&'a mut self, src: &'a [u8]) -> Write<'a, Self>\n where\n Self: Unpin,\n {\n write(self, src)\n }\n\n /// Like [`write`], except that it writes from a slice of buffers.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_vectored(&mut self, bufs: &[IoSlice<'_>]) -> io::Result;\n /// ```\n ///\n /// See [`AsyncWrite::poll_write_vectored`] for more details.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe in the sense that if it is used as\n /// the event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then it is guaranteed that no data was\n /// written to this `AsyncWrite`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n /// use std::io::IoSlice;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n ///\n /// let bufs: &[_] = &[\n /// IoSlice::new(b\"hello\"),\n /// IoSlice::new(b\" \"),\n /// IoSlice::new(b\"world\"),\n /// ];\n ///\n /// file.write_vectored(&bufs).await?;\n /// file.flush().await?;\n ///\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// [`write`]: AsyncWriteExt::write\n fn write_vectored<'a, 'b>(&'a mut self, bufs: &'a [IoSlice<'b>]) -> WriteVectored<'a, 'b, Self>\n where\n Self: Unpin,\n {\n write_vectored(self, bufs)\n }\n\n /// Writes a buffer into this writer, advancing the buffer's internal\n /// cursor.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_buf(&mut self, buf: &mut B) -> io::Result;\n /// ```\n ///\n /// This function will attempt to write the entire contents of `buf`, but\n /// the entire write may not succeed, or the write may also generate an\n /// error. After the operation completes, the buffer's\n /// internal cursor is advanced by the number of bytes written. A\n /// subsequent call to `write_buf` using the **same** `buf` value will\n /// resume from the point that the first call to `write_buf` completed.\n /// A call to `write_buf` represents *at most one* attempt to write to any\n /// wrapped object.\n ///\n /// # Return\n ///\n /// If the return value is `Ok(n)` then it must be guaranteed that `n <=\n /// buf.len()`. A return value of `0` typically means that the\n /// underlying object is no longer able to accept bytes and will likely\n /// not be able to in the future as well, or that the buffer provided is\n /// empty.\n ///\n /// # Errors\n ///\n /// Each call to `write` may generate an I/O error indicating that the\n /// operation could not be completed. If an error is returned then no bytes\n /// in the buffer were written to this writer.\n ///\n /// It is **not** considered an error if the entire buffer could not be\n /// written to this writer.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe in the sense that if it is used as\n /// the event in a [`tokio::select!`](crate::select) statement and some\n /// other branch completes first, then it is guaranteed that no data was\n /// written to this `AsyncWrite`.\n ///\n /// # Examples\n ///\n /// [`File`] implements [`AsyncWrite`] and [`Cursor`]`<&[u8]>` implements [`Buf`]:\n ///\n /// [`File`]: crate::fs::File\n /// [`Buf`]: bytes::Buf\n /// [`Cursor`]: std::io::Cursor\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// use bytes::Buf;\n /// use std::io::Cursor;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// let mut buffer = Cursor::new(b\"data to write\");\n ///\n /// // Loop until the entire contents of the buffer are written to\n /// // the file.\n /// while buffer.has_remaining() {\n /// // Writes some prefix of the byte string, not necessarily\n /// // all of it.\n /// file.write_buf(&mut buffer).await?;\n /// }\n /// file.flush().await?;\n ///\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn write_buf<'a, B>(&'a mut self, src: &'a mut B) -> WriteBuf<'a, Self, B>\n where\n Self: Sized + Unpin,\n B: Buf,\n {\n write_buf(self, src)\n }\n\n /// Attempts to write an entire buffer into this writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_all_buf(&mut self, buf: impl Buf) -> Result<(), io::Error> {\n /// while buf.has_remaining() {\n /// self.write_buf(&mut buf).await?;\n /// }\n /// Ok(())\n /// }\n /// ```\n ///\n /// This method will continuously call [`write`] until\n /// [`buf.has_remaining()`](bytes::Buf::has_remaining) returns false. This method will not\n /// return until the entire buffer has been successfully written or an error occurs. The\n /// first error generated will be returned.\n ///\n /// The buffer is advanced after each chunk is successfully written. After failure,\n /// `src.chunk()` will return the chunk that failed to write.\n ///\n /// # Cancel safety\n ///\n /// If `write_all_buf` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then the data in the provided buffer may have been\n /// partially written. However, it is guaranteed that the provided\n /// buffer has been [advanced] by the amount of bytes that have been\n /// partially written.\n ///\n /// # Examples\n ///\n /// [`File`] implements [`AsyncWrite`] and [`Cursor`]`<&[u8]>` implements [`Buf`]:\n ///\n /// [`File`]: crate::fs::File\n /// [`Buf`]: bytes::Buf\n /// [`Cursor`]: std::io::Cursor\n /// [advanced]: bytes::Buf::advance\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// use std::io::Cursor;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n /// let mut buffer = Cursor::new(b\"data to write\");\n ///\n /// file.write_all_buf(&mut buffer).await?;\n /// file.flush().await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// [`write`]: AsyncWriteExt::write\n fn write_all_buf<'a, B>(&'a mut self, src: &'a mut B) -> WriteAllBuf<'a, Self, B>\n where\n Self: Sized + Unpin,\n B: Buf,\n {\n write_all_buf(self, src)\n }\n\n /// Attempts to write an entire buffer into this writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_all(&mut self, buf: &[u8]) -> io::Result<()>;\n /// ```\n ///\n /// This method will continuously call [`write`] until there is no more data\n /// to be written. This method will not return until the entire buffer\n /// has been successfully written or such an error occurs. The first\n /// error generated from this method will be returned.\n ///\n /// # Cancel safety\n ///\n /// This method is not cancellation safe. If it is used as the event\n /// in a [`tokio::select!`](crate::select) statement and some other\n /// branch completes first, then the provided buffer may have been\n /// partially written, but future calls to `write_all` will start over\n /// from the beginning of the buffer.\n ///\n /// # Errors\n ///\n /// This function will return the first error that [`write`] returns.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut file = File::create(\"foo.txt\").await?;\n ///\n /// file.write_all(b\"some bytes\").await?;\n /// file.flush().await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n ///\n /// [`write`]: AsyncWriteExt::write\n fn write_all<'a>(&'a mut self, src: &'a [u8]) -> WriteAll<'a, Self>\n where\n Self: Unpin,\n {\n write_all(self, src)\n }\n\n write_impl! {\n /// Writes an unsigned 8-bit integer to the underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u8(&mut self, n: u8) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 8 bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u8(2).await?;\n /// writer.write_u8(5).await?;\n ///\n /// assert_eq!(writer, b\"\\x02\\x05\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u8(&mut self, n: u8) -> WriteU8;\n\n /// Writes a signed 8-bit integer to the underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i8(&mut self, n: i8) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 8 bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i8(-2).await?;\n /// writer.write_i8(126).await?;\n ///\n /// assert_eq!(writer, b\"\\xFE\\x7E\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i8(&mut self, n: i8) -> WriteI8;\n\n /// Writes an unsigned 16-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u16(&mut self, n: u16) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 16-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u16(517).await?;\n /// writer.write_u16(768).await?;\n ///\n /// assert_eq!(writer, b\"\\x02\\x05\\x03\\x00\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u16(&mut self, n: u16) -> WriteU16;\n\n /// Writes a signed 16-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i16(&mut self, n: i16) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 16-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i16(193).await?;\n /// writer.write_i16(-132).await?;\n ///\n /// assert_eq!(writer, b\"\\x00\\xc1\\xff\\x7c\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i16(&mut self, n: i16) -> WriteI16;\n\n /// Writes an unsigned 32-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u32(&mut self, n: u32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 32-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u32(267).await?;\n /// writer.write_u32(1205419366).await?;\n ///\n /// assert_eq!(writer, b\"\\x00\\x00\\x01\\x0b\\x47\\xd9\\x3d\\x66\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u32(&mut self, n: u32) -> WriteU32;\n\n /// Writes a signed 32-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i32(&mut self, n: i32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 32-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i32(267).await?;\n /// writer.write_i32(1205419366).await?;\n ///\n /// assert_eq!(writer, b\"\\x00\\x00\\x01\\x0b\\x47\\xd9\\x3d\\x66\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i32(&mut self, n: i32) -> WriteI32;\n\n /// Writes an unsigned 64-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u64(&mut self, n: u64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 64-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u64(918733457491587).await?;\n /// writer.write_u64(143).await?;\n ///\n /// assert_eq!(writer, b\"\\x00\\x03\\x43\\x95\\x4d\\x60\\x86\\x83\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x8f\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u64(&mut self, n: u64) -> WriteU64;\n\n /// Writes an signed 64-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i64(&mut self, n: i64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 64-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i64(i64::MIN).await?;\n /// writer.write_i64(i64::MAX).await?;\n ///\n /// assert_eq!(writer, b\"\\x80\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x7f\\xff\\xff\\xff\\xff\\xff\\xff\\xff\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i64(&mut self, n: i64) -> WriteI64;\n\n /// Writes an unsigned 128-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u128(&mut self, n: u128) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 128-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u128(16947640962301618749969007319746179).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83,\n /// 0x00, 0x03, 0x43, 0x95, 0x4d, 0x60, 0x86, 0x83\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_u128(&mut self, n: u128) -> WriteU128;\n\n /// Writes an signed 128-bit integer in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i128(&mut self, n: i128) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 128-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i128(i128::MIN).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0x80, 0, 0, 0, 0, 0, 0, 0,\n /// 0, 0, 0, 0, 0, 0, 0, 0\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_i128(&mut self, n: i128) -> WriteI128;\n\n /// Writes an 32-bit floating point type in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_f32(&mut self, n: f32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write 32-bit floating point type to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_f32(f32::MIN).await?;\n ///\n /// assert_eq!(writer, vec![0xff, 0x7f, 0xff, 0xff]);\n /// Ok(())\n /// # }\n /// ```\n fn write_f32(&mut self, n: f32) -> WriteF32;\n\n /// Writes an 64-bit floating point type in big-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_f64(&mut self, n: f64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write 64-bit floating point type to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_f64(f64::MIN).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0xff, 0xef, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_f64(&mut self, n: f64) -> WriteF64;\n\n /// Writes an unsigned 16-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u16_le(&mut self, n: u16) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 16-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u16_le(517).await?;\n /// writer.write_u16_le(768).await?;\n ///\n /// assert_eq!(writer, b\"\\x05\\x02\\x00\\x03\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u16_le(&mut self, n: u16) -> WriteU16Le;\n\n /// Writes a signed 16-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i16_le(&mut self, n: i16) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 16-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i16_le(193).await?;\n /// writer.write_i16_le(-132).await?;\n ///\n /// assert_eq!(writer, b\"\\xc1\\x00\\x7c\\xff\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i16_le(&mut self, n: i16) -> WriteI16Le;\n\n /// Writes an unsigned 32-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u32_le(&mut self, n: u32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 32-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u32_le(267).await?;\n /// writer.write_u32_le(1205419366).await?;\n ///\n /// assert_eq!(writer, b\"\\x0b\\x01\\x00\\x00\\x66\\x3d\\xd9\\x47\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u32_le(&mut self, n: u32) -> WriteU32Le;\n\n /// Writes a signed 32-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i32_le(&mut self, n: i32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 32-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i32_le(267).await?;\n /// writer.write_i32_le(1205419366).await?;\n ///\n /// assert_eq!(writer, b\"\\x0b\\x01\\x00\\x00\\x66\\x3d\\xd9\\x47\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i32_le(&mut self, n: i32) -> WriteI32Le;\n\n /// Writes an unsigned 64-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u64_le(&mut self, n: u64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 64-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u64_le(918733457491587).await?;\n /// writer.write_u64_le(143).await?;\n ///\n /// assert_eq!(writer, b\"\\x83\\x86\\x60\\x4d\\x95\\x43\\x03\\x00\\x8f\\x00\\x00\\x00\\x00\\x00\\x00\\x00\");\n /// Ok(())\n /// # }\n /// ```\n fn write_u64_le(&mut self, n: u64) -> WriteU64Le;\n\n /// Writes an signed 64-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i64_le(&mut self, n: i64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 64-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i64_le(i64::MIN).await?;\n /// writer.write_i64_le(i64::MAX).await?;\n ///\n /// assert_eq!(writer, b\"\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x80\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x7f\");\n /// Ok(())\n /// # }\n /// ```\n fn write_i64_le(&mut self, n: i64) -> WriteI64Le;\n\n /// Writes an unsigned 128-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_u128_le(&mut self, n: u128) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write unsigned 128-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_u128_le(16947640962301618749969007319746179).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0x83, 0x86, 0x60, 0x4d, 0x95, 0x43, 0x03, 0x00,\n /// 0x83, 0x86, 0x60, 0x4d, 0x95, 0x43, 0x03, 0x00,\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_u128_le(&mut self, n: u128) -> WriteU128Le;\n\n /// Writes an signed 128-bit integer in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_i128_le(&mut self, n: i128) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write signed 128-bit integers to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_i128_le(i128::MIN).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0, 0, 0, 0, 0, 0, 0,\n /// 0, 0, 0, 0, 0, 0, 0, 0, 0x80\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_i128_le(&mut self, n: i128) -> WriteI128Le;\n\n /// Writes an 32-bit floating point type in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_f32_le(&mut self, n: f32) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write 32-bit floating point type to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_f32_le(f32::MIN).await?;\n ///\n /// assert_eq!(writer, vec![0xff, 0xff, 0x7f, 0xff]);\n /// Ok(())\n /// # }\n /// ```\n fn write_f32_le(&mut self, n: f32) -> WriteF32Le;\n\n /// Writes an 64-bit floating point type in little-endian order to the\n /// underlying writer.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn write_f64_le(&mut self, n: f64) -> io::Result<()>;\n /// ```\n ///\n /// It is recommended to use a buffered writer to avoid excessive\n /// syscalls.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::write_all`].\n ///\n /// [`AsyncWriteExt::write_all`]: AsyncWriteExt::write_all\n ///\n /// # Examples\n ///\n /// Write 64-bit floating point type to a `AsyncWrite`:\n ///\n /// ```rust\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let mut writer = Vec::new();\n ///\n /// writer.write_f64_le(f64::MIN).await?;\n ///\n /// assert_eq!(writer, vec![\n /// 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xef, 0xff\n /// ]);\n /// Ok(())\n /// # }\n /// ```\n fn write_f64_le(&mut self, n: f64) -> WriteF64Le;\n }\n\n /// Flushes this output stream, ensuring that all intermediately buffered\n /// contents reach their destination.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn flush(&mut self) -> io::Result<()>;\n /// ```\n ///\n /// # Errors\n ///\n /// It is considered an error if not all bytes could be written due to\n /// I/O errors or EOF being reached.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe.\n ///\n /// If `flush` is used as the event in a [`tokio::select!`](crate::select)\n /// statement and some other branch completes first, then the data in the\n /// buffered data in this `AsyncWrite` may have been partially flushed.\n /// However, it is guaranteed that the buffer is advanced by the amount of\n /// bytes that have been partially flushed.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, BufWriter, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let f = File::create(\"foo.txt\").await?;\n /// let mut buffer = BufWriter::new(f);\n ///\n /// buffer.write_all(b\"some bytes\").await?;\n /// buffer.flush().await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn flush(&mut self) -> Flush<'_, Self>\n where\n Self: Unpin,\n {\n flush(self)\n }\n\n /// Shuts down the output stream, ensuring that the value can be dropped\n /// cleanly.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn shutdown(&mut self) -> io::Result<()>;\n /// ```\n ///\n /// Similar to [`flush`], all intermediately buffered content is written to\n /// the underlying stream. Once the operation completes, the caller should\n /// no longer attempt to write to the stream. For example, the\n /// `TcpStream` implementation will issue a `shutdown(Write)` sys call.\n ///\n /// [`flush`]: fn@crate::io::AsyncWriteExt::flush\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::io::{self, BufWriter, AsyncWriteExt};\n /// use tokio::fs::File;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let f = File::create(\"foo.txt\").await?;\n /// let mut buffer = BufWriter::new(f);\n ///\n /// buffer.write_all(b\"some bytes\").await?;\n /// buffer.shutdown().await?;\n /// Ok(())\n /// }\n /// # }\n /// ```\n fn shutdown(&mut self) -> Shutdown<'_, Self>\n where\n Self: Unpin,\n {\n shutdown(self)\n }\n }\n}\n\nimpl AsyncWriteExt for W {}\n" }, { "path": "tokio/src/io/util/buf_reader.rs", "content": "use crate::io::util::DEFAULT_BUF_SIZE;\nuse crate::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::io::{self, IoSlice, SeekFrom};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse std::{cmp, fmt, mem};\n\npin_project! {\n /// The `BufReader` struct adds buffering to any reader.\n ///\n /// It can be excessively inefficient to work directly with a [`AsyncRead`]\n /// instance. A `BufReader` performs large, infrequent reads on the underlying\n /// [`AsyncRead`] and maintains an in-memory buffer of the results.\n ///\n /// `BufReader` can improve the speed of programs that make *small* and\n /// *repeated* read calls to the same file or network socket. It does not\n /// help when reading very large amounts at once, or reading just one or a few\n /// times. It also provides no advantage when reading from a source that is\n /// already in memory, like a `Vec`.\n ///\n /// When the `BufReader` is dropped, the contents of its buffer will be\n /// discarded. Creating multiple instances of a `BufReader` on the same\n /// stream can cause data loss.\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct BufReader {\n #[pin]\n pub(super) inner: R,\n pub(super) buf: Box<[u8]>,\n pub(super) pos: usize,\n pub(super) cap: usize,\n pub(super) seek_state: SeekState,\n }\n}\n\nimpl BufReader {\n /// Creates a new `BufReader` with a default buffer capacity. The default is currently 8 KB,\n /// but may change in the future.\n pub fn new(inner: R) -> Self {\n Self::with_capacity(DEFAULT_BUF_SIZE, inner)\n }\n\n /// Creates a new `BufReader` with the specified buffer capacity.\n pub fn with_capacity(capacity: usize, inner: R) -> Self {\n let buffer = vec![0; capacity];\n Self {\n inner,\n buf: buffer.into_boxed_slice(),\n pos: 0,\n cap: 0,\n seek_state: SeekState::Init,\n }\n }\n\n /// Gets a reference to the underlying reader.\n ///\n /// It is inadvisable to directly read from the underlying reader.\n pub fn get_ref(&self) -> &R {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying reader.\n ///\n /// It is inadvisable to directly read from the underlying reader.\n pub fn get_mut(&mut self) -> &mut R {\n &mut self.inner\n }\n\n /// Gets a pinned mutable reference to the underlying reader.\n ///\n /// It is inadvisable to directly read from the underlying reader.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut R> {\n self.project().inner\n }\n\n /// Consumes this `BufReader`, returning the underlying reader.\n ///\n /// Note that any leftover data in the internal buffer is lost.\n pub fn into_inner(self) -> R {\n self.inner\n }\n\n /// Returns a reference to the internally buffered data.\n ///\n /// Unlike `fill_buf`, this will not attempt to fill the buffer if it is empty.\n pub fn buffer(&self) -> &[u8] {\n &self.buf[self.pos..self.cap]\n }\n\n /// Invalidates all data in the internal buffer.\n #[inline]\n fn discard_buffer(self: Pin<&mut Self>) {\n let me = self.project();\n *me.pos = 0;\n *me.cap = 0;\n }\n}\n\nimpl AsyncRead for BufReader {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n // If we don't have any buffered data and we're doing a massive read\n // (larger than our internal buffer), bypass our internal buffer\n // entirely.\n if self.pos == self.cap && buf.remaining() >= self.buf.len() {\n let res = ready!(self.as_mut().get_pin_mut().poll_read(cx, buf));\n self.discard_buffer();\n return Poll::Ready(res);\n }\n let rem = ready!(self.as_mut().poll_fill_buf(cx))?;\n let amt = std::cmp::min(rem.len(), buf.remaining());\n buf.put_slice(&rem[..amt]);\n self.consume(amt);\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncBufRead for BufReader {\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n\n // If we've reached the end of our internal buffer then we need to fetch\n // some more data from the underlying reader.\n // Branch using `>=` instead of the more correct `==`\n // to tell the compiler that the pos..cap slice is always valid.\n if *me.pos >= *me.cap {\n debug_assert!(*me.pos == *me.cap);\n let mut buf = ReadBuf::new(me.buf);\n ready!(me.inner.poll_read(cx, &mut buf))?;\n *me.cap = buf.filled().len();\n *me.pos = 0;\n }\n Poll::Ready(Ok(&me.buf[*me.pos..*me.cap]))\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n let me = self.project();\n *me.pos = cmp::min(*me.pos + amt, *me.cap);\n }\n}\n\n#[derive(Debug, Clone, Copy)]\npub(super) enum SeekState {\n /// `start_seek` has not been called.\n Init,\n /// `start_seek` has been called, but `poll_complete` has not yet been called.\n Start(SeekFrom),\n /// Waiting for completion of the first `poll_complete` in the `n.checked_sub(remainder).is_none()` branch.\n PendingOverflowed(i64),\n /// Waiting for completion of `poll_complete`.\n Pending,\n}\n\n/// Seeks to an offset, in bytes, in the underlying reader.\n///\n/// The position used for seeking with `SeekFrom::Current(_)` is the\n/// position the underlying reader would be at if the `BufReader` had no\n/// internal buffer.\n///\n/// Seeking always discards the internal buffer, even if the seek position\n/// would otherwise fall within it. This guarantees that calling\n/// `.into_inner()` immediately after a seek yields the underlying reader\n/// at the same position.\n///\n/// See [`AsyncSeek`] for more details.\n///\n/// Note: In the edge case where you're seeking with `SeekFrom::Current(n)`\n/// where `n` minus the internal buffer length overflows an `i64`, two\n/// seeks will be performed instead of one. If the second seek returns\n/// `Err`, the underlying reader will be left at the same position it would\n/// have if you called `seek` with `SeekFrom::Current(0)`.\nimpl AsyncSeek for BufReader {\n fn start_seek(self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n // We needs to call seek operation multiple times.\n // And we should always call both start_seek and poll_complete,\n // as start_seek alone cannot guarantee that the operation will be completed.\n // poll_complete receives a Context and returns a Poll, so it cannot be called\n // inside start_seek.\n *self.project().seek_state = SeekState::Start(pos);\n Ok(())\n }\n\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let res = match mem::replace(self.as_mut().project().seek_state, SeekState::Init) {\n SeekState::Init => {\n // 1.x AsyncSeek recommends calling poll_complete before start_seek.\n // We don't have to guarantee that the value returned by\n // poll_complete called without start_seek is correct,\n // so we'll return 0.\n return Poll::Ready(Ok(0));\n }\n SeekState::Start(SeekFrom::Current(n)) => {\n let remainder = (self.cap - self.pos) as i64;\n // it should be safe to assume that remainder fits within an i64 as the alternative\n // means we managed to allocate 8 exbibytes and that's absurd.\n // But it's not out of the realm of possibility for some weird underlying reader to\n // support seeking by i64::MIN so we need to handle underflow when subtracting\n // remainder.\n if let Some(offset) = n.checked_sub(remainder) {\n self.as_mut()\n .get_pin_mut()\n .start_seek(SeekFrom::Current(offset))?;\n } else {\n // seek backwards by our remainder, and then by the offset\n self.as_mut()\n .get_pin_mut()\n .start_seek(SeekFrom::Current(-remainder))?;\n if self.as_mut().get_pin_mut().poll_complete(cx)?.is_pending() {\n *self.as_mut().project().seek_state = SeekState::PendingOverflowed(n);\n return Poll::Pending;\n }\n\n // https://github.com/rust-lang/rust/pull/61157#issuecomment-495932676\n self.as_mut().discard_buffer();\n\n self.as_mut()\n .get_pin_mut()\n .start_seek(SeekFrom::Current(n))?;\n }\n self.as_mut().get_pin_mut().poll_complete(cx)?\n }\n SeekState::PendingOverflowed(n) => {\n if self.as_mut().get_pin_mut().poll_complete(cx)?.is_pending() {\n *self.as_mut().project().seek_state = SeekState::PendingOverflowed(n);\n return Poll::Pending;\n }\n\n // https://github.com/rust-lang/rust/pull/61157#issuecomment-495932676\n self.as_mut().discard_buffer();\n\n self.as_mut()\n .get_pin_mut()\n .start_seek(SeekFrom::Current(n))?;\n self.as_mut().get_pin_mut().poll_complete(cx)?\n }\n SeekState::Start(pos) => {\n // Seeking with Start/End doesn't care about our buffer length.\n self.as_mut().get_pin_mut().start_seek(pos)?;\n self.as_mut().get_pin_mut().poll_complete(cx)?\n }\n SeekState::Pending => self.as_mut().get_pin_mut().poll_complete(cx)?,\n };\n\n match res {\n Poll::Ready(res) => {\n self.discard_buffer();\n Poll::Ready(Ok(res))\n }\n Poll::Pending => {\n *self.as_mut().project().seek_state = SeekState::Pending;\n Poll::Pending\n }\n }\n }\n}\n\nimpl AsyncWrite for BufReader {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.get_pin_mut().poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n self.get_pin_mut().poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.get_ref().is_write_vectored()\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.get_pin_mut().poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.get_pin_mut().poll_shutdown(cx)\n }\n}\n\nimpl fmt::Debug for BufReader {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"BufReader\")\n .field(\"reader\", &self.inner)\n .field(\n \"buffer\",\n &format_args!(\"{}/{}\", self.cap - self.pos, self.buf.len()),\n )\n .finish()\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/buf_stream.rs", "content": "use crate::io::util::{BufReader, BufWriter};\nuse crate::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::io::{self, IoSlice, SeekFrom};\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// Wraps a type that is [`AsyncWrite`] and [`AsyncRead`], and buffers its input and output.\n ///\n /// It can be excessively inefficient to work directly with something that implements [`AsyncWrite`]\n /// and [`AsyncRead`]. For example, every `write`, however small, has to traverse the syscall\n /// interface, and similarly, every read has to do the same. The [`BufWriter`] and [`BufReader`]\n /// types aid with these problems respectively, but do so in only one direction. `BufStream` wraps\n /// one in the other so that both directions are buffered. See their documentation for details.\n #[derive(Debug)]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct BufStream {\n #[pin]\n inner: BufReader>,\n }\n}\n\nimpl BufStream {\n /// Wraps a type in both [`BufWriter`] and [`BufReader`].\n ///\n /// See the documentation for those types and [`BufStream`] for details.\n pub fn new(stream: RW) -> BufStream {\n BufStream {\n inner: BufReader::new(BufWriter::new(stream)),\n }\n }\n\n /// Creates a `BufStream` with the specified [`BufReader`] capacity and [`BufWriter`]\n /// capacity.\n ///\n /// See the documentation for those types and [`BufStream`] for details.\n pub fn with_capacity(\n reader_capacity: usize,\n writer_capacity: usize,\n stream: RW,\n ) -> BufStream {\n BufStream {\n inner: BufReader::with_capacity(\n reader_capacity,\n BufWriter::with_capacity(writer_capacity, stream),\n ),\n }\n }\n\n /// Gets a reference to the underlying I/O object.\n ///\n /// It is inadvisable to directly read from the underlying I/O object.\n pub fn get_ref(&self) -> &RW {\n self.inner.get_ref().get_ref()\n }\n\n /// Gets a mutable reference to the underlying I/O object.\n ///\n /// It is inadvisable to directly read from the underlying I/O object.\n pub fn get_mut(&mut self) -> &mut RW {\n self.inner.get_mut().get_mut()\n }\n\n /// Gets a pinned mutable reference to the underlying I/O object.\n ///\n /// It is inadvisable to directly read from the underlying I/O object.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut RW> {\n self.project().inner.get_pin_mut().get_pin_mut()\n }\n\n /// Consumes this `BufStream`, returning the underlying I/O object.\n ///\n /// Note that any leftover data in the internal buffer is lost.\n pub fn into_inner(self) -> RW {\n self.inner.into_inner().into_inner()\n }\n}\n\nimpl From>> for BufStream {\n fn from(b: BufReader>) -> Self {\n BufStream { inner: b }\n }\n}\n\nimpl From>> for BufStream {\n fn from(b: BufWriter>) -> Self {\n // we need to \"invert\" the reader and writer\n let BufWriter {\n inner:\n BufReader {\n inner,\n buf: rbuf,\n pos,\n cap,\n seek_state: rseek_state,\n },\n buf: wbuf,\n written,\n seek_state: wseek_state,\n } = b;\n\n BufStream {\n inner: BufReader {\n inner: BufWriter {\n inner,\n buf: wbuf,\n written,\n seek_state: wseek_state,\n },\n buf: rbuf,\n pos,\n cap,\n seek_state: rseek_state,\n },\n }\n }\n}\n\nimpl AsyncWrite for BufStream {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.project().inner.poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n self.project().inner.poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.inner.is_write_vectored()\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_shutdown(cx)\n }\n}\n\nimpl AsyncRead for BufStream {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.project().inner.poll_read(cx, buf)\n }\n}\n\n/// Seek to an offset, in bytes, in the underlying stream.\n///\n/// The position used for seeking with `SeekFrom::Current(_)` is the\n/// position the underlying stream would be at if the `BufStream` had no\n/// internal buffer.\n///\n/// Seeking always discards the internal buffer, even if the seek position\n/// would otherwise fall within it. This guarantees that calling\n/// `.into_inner()` immediately after a seek yields the underlying reader\n/// at the same position.\n///\n/// See [`AsyncSeek`] for more details.\n///\n/// Note: In the edge case where you're seeking with `SeekFrom::Current(n)`\n/// where `n` minus the internal buffer length overflows an `i64`, two\n/// seeks will be performed instead of one. If the second seek returns\n/// `Err`, the underlying reader will be left at the same position it would\n/// have if you called `seek` with `SeekFrom::Current(0)`.\nimpl AsyncSeek for BufStream {\n fn start_seek(self: Pin<&mut Self>, position: SeekFrom) -> io::Result<()> {\n self.project().inner.start_seek(position)\n }\n\n fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_complete(cx)\n }\n}\n\nimpl AsyncBufRead for BufStream {\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_fill_buf(cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n self.project().inner.consume(amt);\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/buf_writer.rs", "content": "use crate::io::util::DEFAULT_BUF_SIZE;\nuse crate::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::io::{self, IoSlice, SeekFrom, Write};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Wraps a writer and buffers its output.\n ///\n /// It can be excessively inefficient to work directly with something that\n /// implements [`AsyncWrite`]. A `BufWriter` keeps an in-memory buffer of data and\n /// writes it to an underlying writer in large, infrequent batches.\n ///\n /// `BufWriter` can improve the speed of programs that make *small* and\n /// *repeated* write calls to the same file or network socket. It does not\n /// help when writing very large amounts at once, or writing just one or a few\n /// times. It also provides no advantage when writing to a destination that is\n /// in memory, like a `Vec`.\n ///\n /// When the `BufWriter` is dropped, the contents of its buffer will be\n /// discarded. Creating multiple instances of a `BufWriter` on the same\n /// stream can cause data loss. If you need to write out the contents of its\n /// buffer, you must manually call flush before the writer is dropped.\n ///\n /// [`AsyncWrite`]: AsyncWrite\n /// [`flush`]: super::AsyncWriteExt::flush\n ///\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct BufWriter {\n #[pin]\n pub(super) inner: W,\n pub(super) buf: Vec,\n pub(super) written: usize,\n pub(super) seek_state: SeekState,\n }\n}\n\nimpl BufWriter {\n /// Creates a new `BufWriter` with a default buffer capacity. The default is currently 8 KB,\n /// but may change in the future.\n pub fn new(inner: W) -> Self {\n Self::with_capacity(DEFAULT_BUF_SIZE, inner)\n }\n\n /// Creates a new `BufWriter` with the specified buffer capacity.\n pub fn with_capacity(cap: usize, inner: W) -> Self {\n Self {\n inner,\n buf: Vec::with_capacity(cap),\n written: 0,\n seek_state: SeekState::Init,\n }\n }\n\n fn flush_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let mut me = self.project();\n\n let len = me.buf.len();\n let mut ret = Ok(());\n while *me.written < len {\n match ready!(me.inner.as_mut().poll_write(cx, &me.buf[*me.written..])) {\n Ok(0) => {\n ret = Err(io::Error::new(\n io::ErrorKind::WriteZero,\n \"failed to write the buffered data\",\n ));\n break;\n }\n Ok(n) => *me.written += n,\n Err(e) => {\n ret = Err(e);\n break;\n }\n }\n }\n if *me.written > 0 {\n me.buf.drain(..*me.written);\n }\n *me.written = 0;\n Poll::Ready(ret)\n }\n\n /// Gets a reference to the underlying writer.\n pub fn get_ref(&self) -> &W {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying writer.\n ///\n /// It is inadvisable to directly write to the underlying writer.\n pub fn get_mut(&mut self) -> &mut W {\n &mut self.inner\n }\n\n /// Gets a pinned mutable reference to the underlying writer.\n ///\n /// It is inadvisable to directly write to the underlying writer.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut W> {\n self.project().inner\n }\n\n /// Consumes this `BufWriter`, returning the underlying writer.\n ///\n /// Note that any leftover data in the internal buffer is lost.\n pub fn into_inner(self) -> W {\n self.inner\n }\n\n /// Returns a reference to the internally buffered data.\n pub fn buffer(&self) -> &[u8] {\n &self.buf\n }\n}\n\nimpl AsyncWrite for BufWriter {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n if self.buf.len() + buf.len() > self.buf.capacity() {\n ready!(self.as_mut().flush_buf(cx))?;\n }\n\n let me = self.project();\n if buf.len() >= me.buf.capacity() {\n me.inner.poll_write(cx, buf)\n } else {\n Poll::Ready(me.buf.write(buf))\n }\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n mut bufs: &[IoSlice<'_>],\n ) -> Poll> {\n if self.inner.is_write_vectored() {\n let total_len = bufs\n .iter()\n .fold(0usize, |acc, b| acc.saturating_add(b.len()));\n if total_len > self.buf.capacity() - self.buf.len() {\n ready!(self.as_mut().flush_buf(cx))?;\n }\n let me = self.as_mut().project();\n if total_len >= me.buf.capacity() {\n // It's more efficient to pass the slices directly to the\n // underlying writer than to buffer them.\n // The case when the total_len calculation saturates at\n // usize::MAX is also handled here.\n me.inner.poll_write_vectored(cx, bufs)\n } else {\n bufs.iter().for_each(|b| me.buf.extend_from_slice(b));\n Poll::Ready(Ok(total_len))\n }\n } else {\n // Remove empty buffers at the beginning of bufs.\n while bufs.first().map(|buf| buf.len()) == Some(0) {\n bufs = &bufs[1..];\n }\n if bufs.is_empty() {\n return Poll::Ready(Ok(0));\n }\n // Flush if the first buffer doesn't fit.\n let first_len = bufs[0].len();\n if first_len > self.buf.capacity() - self.buf.len() {\n ready!(self.as_mut().flush_buf(cx))?;\n debug_assert!(self.buf.is_empty());\n }\n let me = self.as_mut().project();\n if first_len >= me.buf.capacity() {\n // The slice is at least as large as the buffering capacity,\n // so it's better to write it directly, bypassing the buffer.\n debug_assert!(me.buf.is_empty());\n return me.inner.poll_write(cx, &bufs[0]);\n } else {\n me.buf.extend_from_slice(&bufs[0]);\n bufs = &bufs[1..];\n }\n let mut total_written = first_len;\n debug_assert!(total_written != 0);\n // Append the buffers that fit in the internal buffer.\n for buf in bufs {\n if buf.len() > me.buf.capacity() - me.buf.len() {\n break;\n } else {\n me.buf.extend_from_slice(buf);\n total_written += buf.len();\n }\n }\n Poll::Ready(Ok(total_written))\n }\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(self.as_mut().flush_buf(cx))?;\n self.get_pin_mut().poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(self.as_mut().flush_buf(cx))?;\n self.get_pin_mut().poll_shutdown(cx)\n }\n}\n\n#[derive(Debug, Clone, Copy)]\npub(super) enum SeekState {\n /// `start_seek` has not been called.\n Init,\n /// `start_seek` has been called, but `poll_complete` has not yet been called.\n Start(SeekFrom),\n /// Waiting for completion of `poll_complete`.\n Pending,\n}\n\n/// Seek to the offset, in bytes, in the underlying writer.\n///\n/// Seeking always writes out the internal buffer before seeking.\nimpl AsyncSeek for BufWriter {\n fn start_seek(self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n // We need to flush the internal buffer before seeking.\n // It receives a `Context` and returns a `Poll`, so it cannot be called\n // inside `start_seek`.\n *self.project().seek_state = SeekState::Start(pos);\n Ok(())\n }\n\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let pos = match self.seek_state {\n SeekState::Init => {\n return self.project().inner.poll_complete(cx);\n }\n SeekState::Start(pos) => Some(pos),\n SeekState::Pending => None,\n };\n\n // Flush the internal buffer before seeking.\n ready!(self.as_mut().flush_buf(cx))?;\n\n let mut me = self.project();\n if let Some(pos) = pos {\n // Ensure previous seeks have finished before starting a new one\n ready!(me.inner.as_mut().poll_complete(cx))?;\n if let Err(e) = me.inner.as_mut().start_seek(pos) {\n *me.seek_state = SeekState::Init;\n return Poll::Ready(Err(e));\n }\n }\n match me.inner.poll_complete(cx) {\n Poll::Ready(res) => {\n *me.seek_state = SeekState::Init;\n Poll::Ready(res)\n }\n Poll::Pending => {\n *me.seek_state = SeekState::Pending;\n Poll::Pending\n }\n }\n }\n}\n\nimpl AsyncRead for BufWriter {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.get_pin_mut().poll_read(cx, buf)\n }\n}\n\nimpl AsyncBufRead for BufWriter {\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.get_pin_mut().poll_fill_buf(cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n self.get_pin_mut().consume(amt);\n }\n}\n\nimpl fmt::Debug for BufWriter {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"BufWriter\")\n .field(\"writer\", &self.inner)\n .field(\n \"buffer\",\n &format_args!(\"{}/{}\", self.buf.len(), self.buf.capacity()),\n )\n .field(\"written\", &self.written)\n .finish()\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/chain.rs", "content": "use crate::io::{AsyncBufRead, AsyncRead, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Stream for the [`chain`](super::AsyncReadExt::chain) method.\n #[must_use = \"streams do nothing unless polled\"]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct Chain {\n #[pin]\n first: T,\n #[pin]\n second: U,\n done_first: bool,\n }\n}\n\npub(super) fn chain(first: T, second: U) -> Chain\nwhere\n T: AsyncRead,\n U: AsyncRead,\n{\n Chain {\n first,\n second,\n done_first: false,\n }\n}\n\nimpl Chain\nwhere\n T: AsyncRead,\n U: AsyncRead,\n{\n /// Gets references to the underlying readers in this `Chain`.\n pub fn get_ref(&self) -> (&T, &U) {\n (&self.first, &self.second)\n }\n\n /// Gets mutable references to the underlying readers in this `Chain`.\n ///\n /// Care should be taken to avoid modifying the internal I/O state of the\n /// underlying readers as doing so may corrupt the internal state of this\n /// `Chain`.\n pub fn get_mut(&mut self) -> (&mut T, &mut U) {\n (&mut self.first, &mut self.second)\n }\n\n /// Gets pinned mutable references to the underlying readers in this `Chain`.\n ///\n /// Care should be taken to avoid modifying the internal I/O state of the\n /// underlying readers as doing so may corrupt the internal state of this\n /// `Chain`.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> (Pin<&mut T>, Pin<&mut U>) {\n let me = self.project();\n (me.first, me.second)\n }\n\n /// Consumes the `Chain`, returning the wrapped readers.\n pub fn into_inner(self) -> (T, U) {\n (self.first, self.second)\n }\n}\n\nimpl fmt::Debug for Chain\nwhere\n T: fmt::Debug,\n U: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Chain\")\n .field(\"t\", &self.first)\n .field(\"u\", &self.second)\n .finish()\n }\n}\n\nimpl AsyncRead for Chain\nwhere\n T: AsyncRead,\n U: AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let me = self.project();\n\n if !*me.done_first {\n let rem = buf.remaining();\n ready!(me.first.poll_read(cx, buf))?;\n if buf.remaining() == rem {\n *me.done_first = true;\n } else {\n return Poll::Ready(Ok(()));\n }\n }\n me.second.poll_read(cx, buf)\n }\n}\n\nimpl AsyncBufRead for Chain\nwhere\n T: AsyncBufRead,\n U: AsyncBufRead,\n{\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n\n if !*me.done_first {\n match ready!(me.first.poll_fill_buf(cx)?) {\n [] => {\n *me.done_first = true;\n }\n buf => return Poll::Ready(Ok(buf)),\n }\n }\n me.second.poll_fill_buf(cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n let me = self.project();\n if !*me.done_first {\n me.first.consume(amt)\n } else {\n me.second.consume(amt)\n }\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/copy.rs", "content": "use crate::io::{AsyncRead, AsyncWrite, ReadBuf};\n\nuse std::future::Future;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n#[derive(Debug)]\npub(super) struct CopyBuffer {\n read_done: bool,\n need_flush: bool,\n pos: usize,\n cap: usize,\n amt: u64,\n buf: Box<[u8]>,\n}\n\nimpl CopyBuffer {\n pub(super) fn new(buf_size: usize) -> Self {\n Self {\n read_done: false,\n need_flush: false,\n pos: 0,\n cap: 0,\n amt: 0,\n buf: vec![0; buf_size].into_boxed_slice(),\n }\n }\n\n fn poll_fill_buf(\n &mut self,\n cx: &mut Context<'_>,\n reader: Pin<&mut R>,\n ) -> Poll>\n where\n R: AsyncRead + ?Sized,\n {\n let me = &mut *self;\n let mut buf = ReadBuf::new(&mut me.buf);\n buf.set_filled(me.cap);\n\n let res = reader.poll_read(cx, &mut buf);\n if let Poll::Ready(Ok(())) = res {\n let filled_len = buf.filled().len();\n me.read_done = me.cap == filled_len;\n me.cap = filled_len;\n }\n res\n }\n\n fn poll_write_buf(\n &mut self,\n cx: &mut Context<'_>,\n mut reader: Pin<&mut R>,\n mut writer: Pin<&mut W>,\n ) -> Poll>\n where\n R: AsyncRead + ?Sized,\n W: AsyncWrite + ?Sized,\n {\n let me = &mut *self;\n match writer.as_mut().poll_write(cx, &me.buf[me.pos..me.cap]) {\n Poll::Pending => {\n // Top up the buffer towards full if we can read a bit more\n // data - this should improve the chances of a large write\n if !me.read_done && me.cap < me.buf.len() {\n ready!(me.poll_fill_buf(cx, reader.as_mut()))?;\n }\n Poll::Pending\n }\n res => res,\n }\n }\n\n pub(super) fn poll_copy(\n &mut self,\n cx: &mut Context<'_>,\n mut reader: Pin<&mut R>,\n mut writer: Pin<&mut W>,\n ) -> Poll>\n where\n R: AsyncRead + ?Sized,\n W: AsyncWrite + ?Sized,\n {\n ready!(crate::trace::trace_leaf(cx));\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n loop {\n // If there is some space left in our buffer, then we try to read some\n // data to continue, thus maximizing the chances of a large write.\n if self.cap < self.buf.len() && !self.read_done {\n match self.poll_fill_buf(cx, reader.as_mut()) {\n Poll::Ready(Ok(())) => {\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n coop.made_progress();\n }\n Poll::Ready(Err(err)) => {\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n coop.made_progress();\n return Poll::Ready(Err(err));\n }\n Poll::Pending => {\n // Ignore pending reads when our buffer is not empty, because\n // we can try to write data immediately.\n if self.pos == self.cap {\n // Try flushing when the reader has no progress to avoid deadlock\n // when the reader depends on buffered writer.\n if self.need_flush {\n ready!(writer.as_mut().poll_flush(cx))?;\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n coop.made_progress();\n self.need_flush = false;\n }\n\n return Poll::Pending;\n }\n }\n }\n }\n\n // If our buffer has some data, let's write it out!\n while self.pos < self.cap {\n let i = ready!(self.poll_write_buf(cx, reader.as_mut(), writer.as_mut()))?;\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n coop.made_progress();\n if i == 0 {\n return Poll::Ready(Err(io::Error::new(\n io::ErrorKind::WriteZero,\n \"write zero byte into writer\",\n )));\n } else {\n self.pos += i;\n self.amt += i as u64;\n self.need_flush = true;\n }\n }\n\n // If pos larger than cap, this loop will never stop.\n // In particular, user's wrong poll_write implementation returning\n // incorrect written length may lead to thread blocking.\n debug_assert!(\n self.pos <= self.cap,\n \"writer returned length larger than input slice\"\n );\n\n // All data has been written, the buffer can be considered empty again\n self.pos = 0;\n self.cap = 0;\n\n // If we've written all the data and we've seen EOF, flush out the\n // data and finish the transfer.\n if self.read_done {\n ready!(writer.as_mut().poll_flush(cx))?;\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n coop.made_progress();\n return Poll::Ready(Ok(self.amt));\n }\n }\n }\n}\n\n/// A future that asynchronously copies the entire contents of a reader into a\n/// writer.\n#[derive(Debug)]\n#[must_use = \"futures do nothing unless you `.await` or poll them\"]\nstruct Copy<'a, R: ?Sized, W: ?Sized> {\n reader: &'a mut R,\n writer: &'a mut W,\n buf: CopyBuffer,\n}\n\ncfg_io_util! {\n /// Asynchronously copies the entire contents of a reader into a writer.\n ///\n /// This function returns a future that will continuously read data from\n /// `reader` and then write it into `writer` in a streaming fashion until\n /// `reader` returns EOF or fails.\n ///\n /// On success, the total number of bytes that were copied from `reader` to\n /// `writer` is returned.\n ///\n /// This is an asynchronous version of [`std::io::copy`][std].\n ///\n /// A heap-allocated copy buffer with 8 KB is created to take data from the\n /// reader to the writer, check [`copy_buf`] if you want an alternative for\n /// [`AsyncBufRead`]. You can use `copy_buf` with [`BufReader`] to change the\n /// buffer capacity.\n ///\n /// # When to use async alternatives instead of `SyncIoBridge`\n ///\n /// If you are looking to use [`std::io::copy`] with a synchronous consumer\n /// (like a `hasher` or compressor), consider using async alternatives instead of\n /// wrapping the reader with [`SyncIoBridge`].\n /// See the [`SyncIoBridge`] documentation for detailed examples and guidance.\n ///\n /// [std]: std::io::copy\n /// [`copy_buf`]: crate::io::copy_buf\n /// [`AsyncBufRead`]: crate::io::AsyncBufRead\n /// [`BufReader`]: crate::io::BufReader\n /// [`SyncIoBridge`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.SyncIoBridge.html\n ///\n /// # Errors\n ///\n /// The returned future will return an error immediately if any call to\n /// `poll_read` or `poll_write` returns an error.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut reader: &[u8] = b\"hello\";\n /// let mut writer: Vec = vec![];\n ///\n /// io::copy(&mut reader, &mut writer).await?;\n ///\n /// assert_eq!(&b\"hello\"[..], &writer[..]);\n /// # Ok(())\n /// # }\n /// ```\n pub async fn copy<'a, R, W>(reader: &'a mut R, writer: &'a mut W) -> io::Result\n where\n R: AsyncRead + Unpin + ?Sized,\n W: AsyncWrite + Unpin + ?Sized,\n {\n Copy {\n reader,\n writer,\n buf: CopyBuffer::new(super::DEFAULT_BUF_SIZE)\n }.await\n }\n}\n\nimpl Future for Copy<'_, R, W>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = &mut *self;\n\n me.buf\n .poll_copy(cx, Pin::new(&mut *me.reader), Pin::new(&mut *me.writer))\n }\n}\n" }, { "path": "tokio/src/io/util/copy_bidirectional.rs", "content": "use super::copy::CopyBuffer;\n\nuse crate::io::{AsyncRead, AsyncWrite};\n\nuse std::future::poll_fn;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\nenum TransferState {\n Running(CopyBuffer),\n ShuttingDown(u64),\n Done(u64),\n}\n\nfn transfer_one_direction(\n cx: &mut Context<'_>,\n state: &mut TransferState,\n r: &mut A,\n w: &mut B,\n) -> Poll>\nwhere\n A: AsyncRead + AsyncWrite + Unpin + ?Sized,\n B: AsyncRead + AsyncWrite + Unpin + ?Sized,\n{\n let mut r = Pin::new(r);\n let mut w = Pin::new(w);\n\n loop {\n match state {\n TransferState::Running(buf) => {\n let count = ready!(buf.poll_copy(cx, r.as_mut(), w.as_mut()))?;\n *state = TransferState::ShuttingDown(count);\n }\n TransferState::ShuttingDown(count) => {\n ready!(w.as_mut().poll_shutdown(cx))?;\n\n *state = TransferState::Done(*count);\n }\n TransferState::Done(count) => return Poll::Ready(Ok(*count)),\n }\n }\n}\n/// Copies data in both directions between `a` and `b`.\n///\n/// This function returns a future that will read from both streams,\n/// writing any data read to the opposing stream.\n/// This happens in both directions concurrently.\n///\n/// If an EOF is observed on one stream, [`shutdown()`] will be invoked on\n/// the other, and reading from that stream will stop. Copying of data in\n/// the other direction will continue.\n///\n/// The future will complete successfully once both directions of communication has been shut down.\n/// A direction is shut down when the reader reports EOF,\n/// at which point [`shutdown()`] is called on the corresponding writer. When finished,\n/// it will return a tuple of the number of bytes copied from a to b\n/// and the number of bytes copied from b to a, in that order.\n///\n/// It uses two 8 KB buffers for transferring bytes between `a` and `b` by default.\n/// To set your own buffers sizes use [`copy_bidirectional_with_sizes()`].\n///\n/// [`shutdown()`]: crate::io::AsyncWriteExt::shutdown\n///\n/// # Errors\n///\n/// The future will immediately return an error if any IO operation on `a`\n/// or `b` returns an error. Some data read from either stream may be lost (not\n/// written to the other stream) in this case.\n///\n/// # Return value\n///\n/// Returns a tuple of bytes copied `a` to `b` and bytes copied `b` to `a`.\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub async fn copy_bidirectional(a: &mut A, b: &mut B) -> io::Result<(u64, u64)>\nwhere\n A: AsyncRead + AsyncWrite + Unpin + ?Sized,\n B: AsyncRead + AsyncWrite + Unpin + ?Sized,\n{\n copy_bidirectional_impl(\n a,\n b,\n CopyBuffer::new(super::DEFAULT_BUF_SIZE),\n CopyBuffer::new(super::DEFAULT_BUF_SIZE),\n )\n .await\n}\n\n/// Copies data in both directions between `a` and `b` using buffers of the specified size.\n///\n/// This method is the same as the [`copy_bidirectional()`], except that it allows you to set the\n/// size of the internal buffers used when copying data.\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub async fn copy_bidirectional_with_sizes(\n a: &mut A,\n b: &mut B,\n a_to_b_buf_size: usize,\n b_to_a_buf_size: usize,\n) -> io::Result<(u64, u64)>\nwhere\n A: AsyncRead + AsyncWrite + Unpin + ?Sized,\n B: AsyncRead + AsyncWrite + Unpin + ?Sized,\n{\n copy_bidirectional_impl(\n a,\n b,\n CopyBuffer::new(a_to_b_buf_size),\n CopyBuffer::new(b_to_a_buf_size),\n )\n .await\n}\n\nasync fn copy_bidirectional_impl(\n a: &mut A,\n b: &mut B,\n a_to_b_buffer: CopyBuffer,\n b_to_a_buffer: CopyBuffer,\n) -> io::Result<(u64, u64)>\nwhere\n A: AsyncRead + AsyncWrite + Unpin + ?Sized,\n B: AsyncRead + AsyncWrite + Unpin + ?Sized,\n{\n let mut a_to_b = TransferState::Running(a_to_b_buffer);\n let mut b_to_a = TransferState::Running(b_to_a_buffer);\n poll_fn(|cx| {\n let a_to_b = transfer_one_direction(cx, &mut a_to_b, a, b)?;\n let b_to_a = transfer_one_direction(cx, &mut b_to_a, b, a)?;\n\n // It is not a problem if ready! returns early because transfer_one_direction for the\n // other direction will keep returning TransferState::Done(count) in future calls to poll\n let a_to_b = ready!(a_to_b);\n let b_to_a = ready!(b_to_a);\n\n Poll::Ready(Ok((a_to_b, b_to_a)))\n })\n .await\n}\n" }, { "path": "tokio/src/io/util/copy_buf.rs", "content": "use crate::io::{AsyncBufRead, AsyncWrite};\nuse std::future::Future;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n /// A future that asynchronously copies the entire contents of a reader into a\n /// writer.\n ///\n /// This struct is generally created by calling [`copy_buf`][copy_buf]. Please\n /// see the documentation of `copy_buf()` for more details.\n ///\n /// [copy_buf]: copy_buf()\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n struct CopyBuf<'a, R: ?Sized, W: ?Sized> {\n reader: &'a mut R,\n writer: &'a mut W,\n amt: u64,\n }\n\n /// Asynchronously copies the entire contents of a reader into a writer.\n ///\n /// This function returns a future that will continuously read data from\n /// `reader` and then write it into `writer` in a streaming fashion until\n /// `reader` returns EOF or fails.\n ///\n /// On success, the total number of bytes that were copied from `reader` to\n /// `writer` is returned.\n ///\n /// This is a [`tokio::io::copy`] alternative for [`AsyncBufRead`] readers\n /// with no extra buffer allocation, since [`AsyncBufRead`] allow access\n /// to the reader's inner buffer.\n ///\n /// # When to use async alternatives instead of `SyncIoBridge`\n ///\n /// If you are looking to use [`std::io::copy`] with a synchronous consumer\n /// (like a `hasher` or compressor), consider using async alternatives instead of\n /// wrapping the reader with [`SyncIoBridge`]. See the [`SyncIoBridge`]\n /// documentation for detailed examples and guidance on hashing, compression,\n /// and data parsing.\n ///\n /// [`tokio::io::copy`]: crate::io::copy\n /// [`AsyncBufRead`]: crate::io::AsyncBufRead\n /// [`SyncIoBridge`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.SyncIoBridge.html\n ///\n /// # Errors\n ///\n /// The returned future will finish with an error will return an error\n /// immediately if any call to `poll_fill_buf` or `poll_write` returns an\n /// error.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io;\n ///\n /// # async fn dox() -> std::io::Result<()> {\n /// let mut reader: &[u8] = b\"hello\";\n /// let mut writer: Vec = vec![];\n ///\n /// io::copy_buf(&mut reader, &mut writer).await?;\n ///\n /// assert_eq!(b\"hello\", &writer[..]);\n /// # Ok(())\n /// # }\n /// ```\n pub async fn copy_buf<'a, R, W>(reader: &'a mut R, writer: &'a mut W) -> io::Result\n where\n R: AsyncBufRead + Unpin + ?Sized,\n W: AsyncWrite + Unpin + ?Sized,\n {\n CopyBuf {\n reader,\n writer,\n amt: 0,\n }.await\n }\n}\n\nimpl Future for CopyBuf<'_, R, W>\nwhere\n R: AsyncBufRead + Unpin + ?Sized,\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n loop {\n let me = &mut *self;\n let buffer = ready!(Pin::new(&mut *me.reader).poll_fill_buf(cx))?;\n if buffer.is_empty() {\n ready!(Pin::new(&mut self.writer).poll_flush(cx))?;\n return Poll::Ready(Ok(self.amt));\n }\n\n let i = ready!(Pin::new(&mut *me.writer).poll_write(cx, buffer))?;\n if i == 0 {\n return Poll::Ready(Err(std::io::ErrorKind::WriteZero.into()));\n }\n self.amt += i as u64;\n Pin::new(&mut *self.reader).consume(i);\n }\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n use std::marker::PhantomPinned;\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/empty.rs", "content": "use crate::io::util::poll_proceed_and_make_progress;\nuse crate::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};\n\nuse std::fmt;\nuse std::io::{self, SeekFrom};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n /// `Empty` ignores any data written via [`AsyncWrite`], and will always be empty\n /// (returning zero bytes) when read via [`AsyncRead`].\n ///\n /// This struct is generally created by calling [`empty`]. Please see\n /// the documentation of [`empty()`][`empty`] for more details.\n ///\n /// This is an asynchronous version of [`std::io::empty`][std].\n ///\n /// [`empty`]: fn@empty\n /// [std]: std::io::empty\n pub struct Empty {\n _p: (),\n }\n\n /// Creates a value that is always at EOF for reads, and ignores all data written.\n ///\n /// All writes on the returned instance will return `Poll::Ready(Ok(buf.len()))`\n /// and the contents of the buffer will not be inspected.\n ///\n /// All reads from the returned instance will return `Poll::Ready(Ok(0))`.\n ///\n /// This is an asynchronous version of [`std::io::empty`][std].\n ///\n /// [std]: std::io::empty\n ///\n /// # Examples\n ///\n /// A slightly sad example of not reading anything into a buffer:\n ///\n /// ```\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut buffer = String::new();\n /// io::empty().read_to_string(&mut buffer).await.unwrap();\n /// assert!(buffer.is_empty());\n /// # }\n /// ```\n ///\n /// A convoluted way of getting the length of a buffer:\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let buffer = vec![1, 2, 3, 5, 8];\n /// let num_bytes = io::empty().write(&buffer).await.unwrap();\n /// assert_eq!(num_bytes, 5);\n /// # }\n /// ```\n pub fn empty() -> Empty {\n Empty { _p: () }\n }\n}\n\nimpl AsyncRead for Empty {\n #[inline]\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n _: &mut ReadBuf<'_>,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncBufRead for Empty {\n #[inline]\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(&[]))\n }\n\n #[inline]\n fn consume(self: Pin<&mut Self>, _: usize) {}\n}\n\nimpl AsyncWrite for Empty {\n #[inline]\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(buf.len()))\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(()))\n }\n\n #[inline]\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(()))\n }\n\n #[inline]\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n #[inline]\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n let num_bytes = bufs.iter().map(|b| b.len()).sum();\n Poll::Ready(Ok(num_bytes))\n }\n}\n\nimpl AsyncSeek for Empty {\n #[inline]\n fn start_seek(self: Pin<&mut Self>, _position: SeekFrom) -> io::Result<()> {\n Ok(())\n }\n\n #[inline]\n fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(0))\n }\n}\n\nimpl fmt::Debug for Empty {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.pad(\"Empty { .. }\")\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::();\n }\n}\n" }, { "path": "tokio/src/io/util/fill_buf.rs", "content": "use crate::io::AsyncBufRead;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// Future for the [`fill_buf`](crate::io::AsyncBufReadExt::fill_buf) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct FillBuf<'a, R: ?Sized> {\n reader: Option<&'a mut R>,\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn fill_buf(reader: &mut R) -> FillBuf<'_, R>\nwhere\n R: AsyncBufRead + ?Sized + Unpin,\n{\n FillBuf {\n reader: Some(reader),\n _pin: PhantomPinned,\n }\n}\n\nimpl<'a, R: AsyncBufRead + ?Sized + Unpin> Future for FillBuf<'a, R> {\n type Output = io::Result<&'a [u8]>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n let reader = me.reader.take().expect(\"Polled after completion.\");\n match Pin::new(&mut *reader).poll_fill_buf(cx) {\n Poll::Ready(Ok(slice)) => unsafe {\n // Safety: This is necessary only due to a limitation in the\n // borrow checker. Once Rust starts using the polonius borrow\n // checker, this can be simplified.\n //\n // The safety of this transmute relies on the fact that the\n // value of `reader` is `None` when we return in this branch.\n // Otherwise the caller could poll us again after\n // completion, and access the mutable reference while the\n // returned immutable reference still exists.\n let slice = std::mem::transmute::<&[u8], &'a [u8]>(slice);\n Poll::Ready(Ok(slice))\n },\n Poll::Ready(Err(err)) => Poll::Ready(Err(err)),\n Poll::Pending => {\n *me.reader = Some(reader);\n Poll::Pending\n }\n }\n }\n}\n" }, { "path": "tokio/src/io/util/flush.rs", "content": "use crate::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A future used to fully flush an I/O object.\n ///\n /// Created by the [`AsyncWriteExt::flush`][flush] function.\n ///\n /// [flush]: crate::io::AsyncWriteExt::flush\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Flush<'a, A: ?Sized> {\n a: &'a mut A,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Creates a future which will entirely flush an I/O object.\npub(super) fn flush(a: &mut A) -> Flush<'_, A>\nwhere\n A: AsyncWrite + Unpin + ?Sized,\n{\n Flush {\n a,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for Flush<'_, A>\nwhere\n A: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n Pin::new(&mut *me.a).poll_flush(cx)\n }\n}\n" }, { "path": "tokio/src/io/util/lines.rs", "content": "use crate::io::util::read_line::read_line_internal;\nuse crate::io::AsyncBufRead;\n\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::mem;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Reads lines from an [`AsyncBufRead`].\n ///\n /// A `Lines` can be turned into a `Stream` with [`LinesStream`].\n ///\n /// This type is usually created using the [`lines`] method.\n ///\n /// [`AsyncBufRead`]: crate::io::AsyncBufRead\n /// [`LinesStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.LinesStream.html\n /// [`lines`]: crate::io::AsyncBufReadExt::lines\n #[derive(Debug)]\n #[must_use = \"streams do nothing unless polled\"]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct Lines {\n #[pin]\n reader: R,\n buf: String,\n bytes: Vec,\n read: usize,\n }\n}\n\npub(crate) fn lines(reader: R) -> Lines\nwhere\n R: AsyncBufRead,\n{\n Lines {\n reader,\n buf: String::new(),\n bytes: Vec::new(),\n read: 0,\n }\n}\n\nimpl Lines\nwhere\n R: AsyncBufRead + Unpin,\n{\n /// Returns the next line in the stream.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncBufRead;\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// # async fn dox(my_buf_read: impl AsyncBufRead + Unpin) -> std::io::Result<()> {\n /// let mut lines = my_buf_read.lines();\n ///\n /// while let Some(line) = lines.next_line().await? {\n /// println!(\"length = {}\", line.len())\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub async fn next_line(&mut self) -> io::Result> {\n use std::future::poll_fn;\n\n poll_fn(|cx| Pin::new(&mut *self).poll_next_line(cx)).await\n }\n\n /// Obtains a mutable reference to the underlying reader.\n pub fn get_mut(&mut self) -> &mut R {\n &mut self.reader\n }\n\n /// Obtains a reference to the underlying reader.\n pub fn get_ref(&mut self) -> &R {\n &self.reader\n }\n\n /// Unwraps this `Lines`, returning the underlying reader.\n ///\n /// Note that any leftover data in the internal buffer is lost.\n /// Therefore, a following read from the underlying reader may lead to data loss.\n pub fn into_inner(self) -> R {\n self.reader\n }\n}\n\nimpl Lines\nwhere\n R: AsyncBufRead,\n{\n /// Polls for the next line in the stream.\n ///\n /// This method returns:\n ///\n /// * `Poll::Pending` if the next line is not yet available.\n /// * `Poll::Ready(Ok(Some(line)))` if the next line is available.\n /// * `Poll::Ready(Ok(None))` if there are no more lines in this stream.\n /// * `Poll::Ready(Err(err))` if an IO error occurred while reading the next line.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when more bytes become\n /// available on the underlying IO resource. Note that on multiple calls to\n /// `poll_next_line`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup.\n pub fn poll_next_line(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let me = self.project();\n\n let n = ready!(read_line_internal(me.reader, cx, me.buf, me.bytes, me.read))?;\n debug_assert_eq!(*me.read, 0);\n\n if n == 0 && me.buf.is_empty() {\n return Poll::Ready(Ok(None));\n }\n\n if me.buf.ends_with('\\n') {\n me.buf.pop();\n\n if me.buf.ends_with('\\r') {\n me.buf.pop();\n }\n }\n\n Poll::Ready(Ok(Some(mem::take(me.buf))))\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/mem.rs", "content": "//! In-process memory IO types.\n\nuse crate::io::{split, AsyncRead, AsyncWrite, ReadBuf, ReadHalf, WriteHalf};\nuse crate::loom::sync::Mutex;\n\nuse bytes::{Buf, BytesMut};\nuse std::{\n pin::Pin,\n sync::Arc,\n task::{self, ready, Poll, Waker},\n};\n\n/// A bidirectional pipe to read and write bytes in memory.\n///\n/// A pair of `DuplexStream`s are created together, and they act as a \"channel\"\n/// that can be used as in-memory IO types. Writing to one of the pairs will\n/// allow that data to be read from the other, and vice versa.\n///\n/// # Closing a `DuplexStream`\n///\n/// If one end of the `DuplexStream` channel is dropped, any pending reads on\n/// the other side will continue to read data until the buffer is drained, then\n/// they will signal EOF by returning 0 bytes. Any writes to the other side,\n/// including pending ones (that are waiting for free space in the buffer) will\n/// return `Err(BrokenPipe)` immediately.\n///\n/// # Example\n///\n/// ```\n/// # async fn ex() -> std::io::Result<()> {\n/// # use tokio::io::{AsyncReadExt, AsyncWriteExt};\n/// let (mut client, mut server) = tokio::io::duplex(64);\n///\n/// client.write_all(b\"ping\").await?;\n///\n/// let mut buf = [0u8; 4];\n/// server.read_exact(&mut buf).await?;\n/// assert_eq!(&buf, b\"ping\");\n///\n/// server.write_all(b\"pong\").await?;\n///\n/// client.read_exact(&mut buf).await?;\n/// assert_eq!(&buf, b\"pong\");\n/// # Ok(())\n/// # }\n/// ```\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub struct DuplexStream {\n read: Arc>,\n write: Arc>,\n}\n\n/// A unidirectional pipe to read and write bytes in memory.\n///\n/// It can be constructed by [`simplex`] function which will create a pair of\n/// reader and writer or by calling [`SimplexStream::new_unsplit`] that will\n/// create a handle for both reading and writing.\n///\n/// # Example\n///\n/// ```\n/// # async fn ex() -> std::io::Result<()> {\n/// # use tokio::io::{AsyncReadExt, AsyncWriteExt};\n/// let (mut receiver, mut sender) = tokio::io::simplex(64);\n///\n/// sender.write_all(b\"ping\").await?;\n///\n/// let mut buf = [0u8; 4];\n/// receiver.read_exact(&mut buf).await?;\n/// assert_eq!(&buf, b\"ping\");\n/// # Ok(())\n/// # }\n/// ```\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub struct SimplexStream {\n /// The buffer storing the bytes written, also read from.\n ///\n /// Using a `BytesMut` because it has efficient `Buf` and `BufMut`\n /// functionality already. Additionally, it can try to copy data in the\n /// same buffer if there read index has advanced far enough.\n buffer: BytesMut,\n /// Determines if the write side has been closed.\n is_closed: bool,\n /// The maximum amount of bytes that can be written before returning\n /// `Poll::Pending`.\n max_buf_size: usize,\n /// If the `read` side has been polled and is pending, this is the waker\n /// for that parked task.\n read_waker: Option,\n /// If the `write` side has filled the `max_buf_size` and returned\n /// `Poll::Pending`, this is the waker for that parked task.\n write_waker: Option,\n}\n\n// ===== impl DuplexStream =====\n\n/// Create a new pair of `DuplexStream`s that act like a pair of connected sockets.\n///\n/// The `max_buf_size` argument is the maximum amount of bytes that can be\n/// written to a side before the write returns `Poll::Pending`.\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub fn duplex(max_buf_size: usize) -> (DuplexStream, DuplexStream) {\n let one = Arc::new(Mutex::new(SimplexStream::new_unsplit(max_buf_size)));\n let two = Arc::new(Mutex::new(SimplexStream::new_unsplit(max_buf_size)));\n\n (\n DuplexStream {\n read: one.clone(),\n write: two.clone(),\n },\n DuplexStream {\n read: two,\n write: one,\n },\n )\n}\n\nimpl AsyncRead for DuplexStream {\n // Previous rustc required this `self` to be `mut`, even though newer\n // versions recognize it isn't needed to call `lock()`. So for\n // compatibility, we include the `mut` and `allow` the lint.\n //\n // See https://github.com/rust-lang/rust/issues/73592\n #[allow(unused_mut)]\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut *self.read.lock()).poll_read(cx, buf)\n }\n}\n\nimpl AsyncWrite for DuplexStream {\n #[allow(unused_mut)]\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut *self.write.lock()).poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> Poll> {\n Pin::new(&mut *self.write.lock()).poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n #[allow(unused_mut)]\n fn poll_flush(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n ) -> Poll> {\n Pin::new(&mut *self.write.lock()).poll_flush(cx)\n }\n\n #[allow(unused_mut)]\n fn poll_shutdown(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n ) -> Poll> {\n Pin::new(&mut *self.write.lock()).poll_shutdown(cx)\n }\n}\n\nimpl Drop for DuplexStream {\n fn drop(&mut self) {\n // notify the other side of the closure\n self.write.lock().close_write();\n self.read.lock().close_read();\n }\n}\n\n// ===== impl SimplexStream =====\n\n/// Creates unidirectional buffer that acts like in memory pipe.\n///\n/// The `max_buf_size` argument is the maximum amount of bytes that can be\n/// written to a buffer before the it returns `Poll::Pending`.\n///\n/// # Unify reader and writer\n///\n/// The reader and writer half can be unified into a single structure\n/// of `SimplexStream` that supports both reading and writing or\n/// the `SimplexStream` can be already created as unified structure\n/// using [`SimplexStream::new_unsplit()`].\n///\n/// ```\n/// # async fn ex() -> std::io::Result<()> {\n/// # use tokio::io::{AsyncReadExt, AsyncWriteExt};\n/// let (reader, writer) = tokio::io::simplex(64);\n/// let mut simplex_stream = reader.unsplit(writer);\n/// simplex_stream.write_all(b\"hello\").await?;\n///\n/// let mut buf = [0u8; 5];\n/// simplex_stream.read_exact(&mut buf).await?;\n/// assert_eq!(&buf, b\"hello\");\n/// # Ok(())\n/// # }\n/// ```\n#[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\npub fn simplex(max_buf_size: usize) -> (ReadHalf, WriteHalf) {\n split(SimplexStream::new_unsplit(max_buf_size))\n}\n\nimpl SimplexStream {\n /// Creates unidirectional buffer that acts like in memory pipe. To create split\n /// version with separate reader and writer you can use [`simplex`] function.\n ///\n /// The `max_buf_size` argument is the maximum amount of bytes that can be\n /// written to a buffer before the it returns `Poll::Pending`.\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub fn new_unsplit(max_buf_size: usize) -> SimplexStream {\n SimplexStream {\n buffer: BytesMut::new(),\n is_closed: false,\n max_buf_size,\n read_waker: None,\n write_waker: None,\n }\n }\n\n fn close_write(&mut self) {\n self.is_closed = true;\n // needs to notify any readers that no more data will come\n if let Some(waker) = self.read_waker.take() {\n waker.wake();\n }\n }\n\n fn close_read(&mut self) {\n self.is_closed = true;\n // needs to notify any writers that they have to abort\n if let Some(waker) = self.write_waker.take() {\n waker.wake();\n }\n }\n\n fn poll_read_internal(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if self.buffer.has_remaining() {\n let max = self.buffer.remaining().min(buf.remaining());\n buf.put_slice(&self.buffer[..max]);\n self.buffer.advance(max);\n if max > 0 {\n // The passed `buf` might have been empty, don't wake up if\n // no bytes have been moved.\n if let Some(waker) = self.write_waker.take() {\n waker.wake();\n }\n }\n Poll::Ready(Ok(()))\n } else if self.is_closed {\n Poll::Ready(Ok(()))\n } else {\n self.read_waker = Some(cx.waker().clone());\n Poll::Pending\n }\n }\n\n fn poll_write_internal(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n if self.is_closed {\n return Poll::Ready(Err(std::io::ErrorKind::BrokenPipe.into()));\n }\n let avail = self.max_buf_size - self.buffer.len();\n if avail == 0 {\n self.write_waker = Some(cx.waker().clone());\n return Poll::Pending;\n }\n\n let len = buf.len().min(avail);\n self.buffer.extend_from_slice(&buf[..len]);\n if let Some(waker) = self.read_waker.take() {\n waker.wake();\n }\n Poll::Ready(Ok(len))\n }\n\n fn poll_write_vectored_internal(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> Poll> {\n if self.is_closed {\n return Poll::Ready(Err(std::io::ErrorKind::BrokenPipe.into()));\n }\n let avail = self.max_buf_size - self.buffer.len();\n if avail == 0 {\n self.write_waker = Some(cx.waker().clone());\n return Poll::Pending;\n }\n\n let mut rem = avail;\n for buf in bufs {\n if rem == 0 {\n break;\n }\n\n let len = buf.len().min(rem);\n self.buffer.extend_from_slice(&buf[..len]);\n rem -= len;\n }\n\n if let Some(waker) = self.read_waker.take() {\n waker.wake();\n }\n Poll::Ready(Ok(avail - rem))\n }\n}\n\nimpl AsyncRead for SimplexStream {\n cfg_coop! {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let ret = self.poll_read_internal(cx, buf);\n if ret.is_ready() {\n coop.made_progress();\n }\n ret\n }\n }\n\n cfg_not_coop! {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n self.poll_read_internal(cx, buf)\n }\n }\n}\n\nimpl AsyncWrite for SimplexStream {\n cfg_coop! {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let ret = self.poll_write_internal(cx, buf);\n if ret.is_ready() {\n coop.made_progress();\n }\n ret\n }\n }\n\n cfg_not_coop! {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n self.poll_write_internal(cx, buf)\n }\n }\n\n cfg_coop! {\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let ret = self.poll_write_vectored_internal(cx, bufs);\n if ret.is_ready() {\n coop.made_progress();\n }\n ret\n }\n }\n\n cfg_not_coop! {\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n self.poll_write_vectored_internal(cx, bufs)\n }\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(self: Pin<&mut Self>, _: &mut task::Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(\n mut self: Pin<&mut Self>,\n _: &mut task::Context<'_>,\n ) -> Poll> {\n self.close_write();\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio/src/io/util/mod.rs", "content": "#![allow(unreachable_pub)] // https://github.com/rust-lang/rust/issues/57411\n\ncfg_io_util! {\n mod async_buf_read_ext;\n pub use async_buf_read_ext::AsyncBufReadExt;\n\n mod async_read_ext;\n pub use async_read_ext::AsyncReadExt;\n\n mod async_seek_ext;\n pub use async_seek_ext::AsyncSeekExt;\n\n mod async_write_ext;\n pub use async_write_ext::AsyncWriteExt;\n\n mod buf_reader;\n pub use buf_reader::BufReader;\n\n mod buf_stream;\n pub use buf_stream::BufStream;\n\n mod buf_writer;\n pub use buf_writer::BufWriter;\n\n mod chain;\n pub use chain::Chain;\n\n mod copy;\n pub use copy::copy;\n\n mod copy_bidirectional;\n pub use copy_bidirectional::{copy_bidirectional, copy_bidirectional_with_sizes};\n\n mod copy_buf;\n pub use copy_buf::copy_buf;\n\n mod empty;\n pub use empty::{empty, Empty};\n\n mod flush;\n\n mod lines;\n pub use lines::Lines;\n\n mod mem;\n pub use mem::{duplex, simplex, DuplexStream, SimplexStream};\n\n mod read;\n mod read_buf;\n mod read_exact;\n mod read_int;\n mod read_line;\n mod fill_buf;\n\n mod read_to_end;\n mod vec_with_initialized;\n cfg_process! {\n pub(crate) use read_to_end::read_to_end;\n }\n\n mod read_to_string;\n mod read_until;\n\n mod repeat;\n pub use repeat::{repeat, Repeat};\n\n mod shutdown;\n\n mod sink;\n pub use sink::{sink, Sink};\n\n mod split;\n pub use split::Split;\n\n mod take;\n pub use take::Take;\n\n mod write;\n mod write_vectored;\n mod write_all;\n mod write_buf;\n mod write_all_buf;\n mod write_int;\n\n\n // used by `BufReader` and `BufWriter`\n // https://github.com/rust-lang/rust/blob/master/library/std/src/sys_common/io.rs#L1\n const DEFAULT_BUF_SIZE: usize = 8 * 1024;\n\n cfg_coop! {\n fn poll_proceed_and_make_progress(cx: &mut std::task::Context<'_>) -> std::task::Poll<()> {\n let coop = std::task::ready!(crate::task::coop::poll_proceed(cx));\n coop.made_progress();\n std::task::Poll::Ready(())\n }\n }\n\n cfg_not_coop! {\n fn poll_proceed_and_make_progress(_: &mut std::task::Context<'_>) -> std::task::Poll<()> {\n std::task::Poll::Ready(())\n }\n }\n}\n\ncfg_not_io_util! {\n cfg_process! {\n mod vec_with_initialized;\n mod read_to_end;\n // Used by process\n pub(crate) use read_to_end::read_to_end;\n }\n}\n" }, { "path": "tokio/src/io/util/read.rs", "content": "use crate::io::{AsyncRead, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::marker::Unpin;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// Tries to read some bytes directly into the given `buf` in asynchronous\n/// manner, returning a future type.\n///\n/// The returned future will resolve to both the I/O stream and the buffer\n/// as well as the number of bytes read once the read operation is completed.\npub(crate) fn read<'a, R>(reader: &'a mut R, buf: &'a mut [u8]) -> Read<'a, R>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n{\n Read {\n reader,\n buf,\n _pin: PhantomPinned,\n }\n}\n\npin_project! {\n /// A future which can be used to easily read available number of bytes to fill\n /// a buffer.\n ///\n /// Created by the [`read`] function.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Read<'a, R: ?Sized> {\n reader: &'a mut R,\n buf: &'a mut [u8],\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for Read<'_, R>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n let mut buf = ReadBuf::new(me.buf);\n ready!(Pin::new(me.reader).poll_read(cx, &mut buf))?;\n Poll::Ready(Ok(buf.filled().len()))\n }\n}\n" }, { "path": "tokio/src/io/util/read_buf.rs", "content": "use crate::io::AsyncRead;\n\nuse bytes::BufMut;\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npub(crate) fn read_buf<'a, R, B>(reader: &'a mut R, buf: &'a mut B) -> ReadBuf<'a, R, B>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n B: BufMut + ?Sized,\n{\n ReadBuf {\n reader,\n buf,\n _pin: PhantomPinned,\n }\n}\n\npin_project! {\n /// Future returned by [`read_buf`](crate::io::AsyncReadExt::read_buf).\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadBuf<'a, R: ?Sized, B: ?Sized> {\n reader: &'a mut R,\n buf: &'a mut B,\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for ReadBuf<'_, R, B>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n B: BufMut + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n use crate::io::ReadBuf;\n\n let me = self.project();\n\n if !me.buf.has_remaining_mut() {\n return Poll::Ready(Ok(0));\n }\n\n let n = {\n let dst = me.buf.chunk_mut();\n let dst = unsafe { dst.as_uninit_slice_mut() };\n let mut buf = ReadBuf::uninit(dst);\n let ptr = buf.filled().as_ptr();\n ready!(Pin::new(me.reader).poll_read(cx, &mut buf)?);\n\n // Ensure the pointer does not change from under us\n assert_eq!(ptr, buf.filled().as_ptr());\n buf.filled().len()\n };\n\n // Safety: This is guaranteed to be the number of initialized (and read)\n // bytes due to the invariants provided by `ReadBuf::filled`.\n unsafe {\n me.buf.advance_mut(n);\n }\n\n Poll::Ready(Ok(n))\n }\n}\n" }, { "path": "tokio/src/io/util/read_exact.rs", "content": "use crate::io::{AsyncRead, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::marker::Unpin;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// A future which can be used to easily read exactly enough bytes to fill\n/// a buffer.\n///\n/// Created by the [`AsyncReadExt::read_exact`][read_exact].\n/// [`read_exact`]: [`crate::io::AsyncReadExt::read_exact`]\npub(crate) fn read_exact<'a, A>(reader: &'a mut A, buf: &'a mut [u8]) -> ReadExact<'a, A>\nwhere\n A: AsyncRead + Unpin + ?Sized,\n{\n ReadExact {\n reader,\n buf: ReadBuf::new(buf),\n _pin: PhantomPinned,\n }\n}\n\npin_project! {\n /// Creates a future which will read exactly enough bytes to fill `buf`,\n /// returning an error if EOF is hit sooner.\n ///\n /// On success the number of bytes is returned\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadExact<'a, A: ?Sized> {\n reader: &'a mut A,\n buf: ReadBuf<'a>,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nfn eof() -> io::Error {\n io::Error::new(io::ErrorKind::UnexpectedEof, \"early eof\")\n}\n\nimpl Future for ReadExact<'_, A>\nwhere\n A: AsyncRead + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n\n loop {\n // if our buffer is empty, then we need to read some data to continue.\n let rem = me.buf.remaining();\n if rem != 0 {\n ready!(Pin::new(&mut *me.reader).poll_read(cx, me.buf))?;\n if me.buf.remaining() == rem {\n return Err(eof()).into();\n }\n } else {\n return Poll::Ready(Ok(me.buf.capacity()));\n }\n }\n }\n}\n" }, { "path": "tokio/src/io/util/read_int.rs", "content": "use crate::io::{AsyncRead, ReadBuf};\n\nuse bytes::Buf;\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::io::ErrorKind::UnexpectedEof;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nmacro_rules! reader {\n ($name:ident, $ty:ty, $reader:ident) => {\n reader!($name, $ty, $reader, std::mem::size_of::<$ty>());\n };\n ($name:ident, $ty:ty, $reader:ident, $bytes:expr) => {\n pin_project! {\n #[doc(hidden)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct $name {\n #[pin]\n src: R,\n buf: [u8; $bytes],\n read: u8,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n }\n\n impl $name {\n pub(crate) fn new(src: R) -> Self {\n $name {\n src,\n buf: [0; $bytes],\n read: 0,\n _pin: PhantomPinned,\n }\n }\n }\n\n impl Future for $name\n where\n R: AsyncRead,\n {\n type Output = io::Result<$ty>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let mut me = self.project();\n\n if *me.read == $bytes as u8 {\n return Poll::Ready(Ok(Buf::$reader(&mut &me.buf[..])));\n }\n\n while *me.read < $bytes as u8 {\n let mut buf = ReadBuf::new(&mut me.buf[*me.read as usize..]);\n\n *me.read += match me.src.as_mut().poll_read(cx, &mut buf) {\n Poll::Pending => return Poll::Pending,\n Poll::Ready(Err(e)) => return Poll::Ready(Err(e.into())),\n Poll::Ready(Ok(())) => {\n let n = buf.filled().len();\n if n == 0 {\n return Poll::Ready(Err(UnexpectedEof.into()));\n }\n\n n as u8\n }\n };\n }\n\n let num = Buf::$reader(&mut &me.buf[..]);\n\n Poll::Ready(Ok(num))\n }\n }\n };\n}\n\nmacro_rules! reader8 {\n ($name:ident, $ty:ty) => {\n pin_project! {\n /// Future returned from `read_u8`\n #[doc(hidden)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct $name {\n #[pin]\n reader: R,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n }\n\n impl $name {\n pub(crate) fn new(reader: R) -> $name {\n $name {\n reader,\n _pin: PhantomPinned,\n }\n }\n }\n\n impl Future for $name\n where\n R: AsyncRead,\n {\n type Output = io::Result<$ty>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n let mut buf = [0; 1];\n let mut buf = ReadBuf::new(&mut buf);\n match me.reader.poll_read(cx, &mut buf) {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(e)) => Poll::Ready(Err(e.into())),\n Poll::Ready(Ok(())) => {\n if buf.filled().len() == 0 {\n return Poll::Ready(Err(UnexpectedEof.into()));\n }\n\n Poll::Ready(Ok(buf.filled()[0] as $ty))\n }\n }\n }\n }\n };\n}\n\nreader8!(ReadU8, u8);\nreader8!(ReadI8, i8);\n\nreader!(ReadU16, u16, get_u16);\nreader!(ReadU32, u32, get_u32);\nreader!(ReadU64, u64, get_u64);\nreader!(ReadU128, u128, get_u128);\n\nreader!(ReadI16, i16, get_i16);\nreader!(ReadI32, i32, get_i32);\nreader!(ReadI64, i64, get_i64);\nreader!(ReadI128, i128, get_i128);\n\nreader!(ReadF32, f32, get_f32);\nreader!(ReadF64, f64, get_f64);\n\nreader!(ReadU16Le, u16, get_u16_le);\nreader!(ReadU32Le, u32, get_u32_le);\nreader!(ReadU64Le, u64, get_u64_le);\nreader!(ReadU128Le, u128, get_u128_le);\n\nreader!(ReadI16Le, i16, get_i16_le);\nreader!(ReadI32Le, i32, get_i32_le);\nreader!(ReadI64Le, i64, get_i64_le);\nreader!(ReadI128Le, i128, get_i128_le);\n\nreader!(ReadF32Le, f32, get_f32_le);\nreader!(ReadF64Le, f64, get_f64_le);\n" }, { "path": "tokio/src/io/util/read_line.rs", "content": "use crate::io::util::read_until::read_until_internal;\nuse crate::io::AsyncBufRead;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::mem;\nuse std::pin::Pin;\nuse std::string::FromUtf8Error;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Future for the [`read_line`](crate::io::AsyncBufReadExt::read_line) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadLine<'a, R: ?Sized> {\n reader: &'a mut R,\n // This is the buffer we were provided. It will be replaced with an empty string\n // while reading to postpone utf-8 handling until after reading.\n output: &'a mut String,\n // The actual allocation of the string is moved into this vector instead.\n buf: Vec,\n // The number of bytes appended to buf. This can be less than buf.len() if\n // the buffer was not empty when the operation was started.\n read: usize,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn read_line<'a, R>(reader: &'a mut R, string: &'a mut String) -> ReadLine<'a, R>\nwhere\n R: AsyncBufRead + ?Sized + Unpin,\n{\n ReadLine {\n reader,\n buf: mem::take(string).into_bytes(),\n output: string,\n read: 0,\n _pin: PhantomPinned,\n }\n}\n\nfn put_back_original_data(output: &mut String, mut vector: Vec, num_bytes_read: usize) {\n let original_len = vector.len() - num_bytes_read;\n vector.truncate(original_len);\n *output = String::from_utf8(vector).expect(\"The original data must be valid utf-8.\");\n}\n\n/// This handles the various failure cases and puts the string back into `output`.\n///\n/// The `truncate_on_io_error` `bool` is necessary because `read_to_string` and `read_line`\n/// disagree on what should happen when an IO error occurs.\npub(super) fn finish_string_read(\n io_res: io::Result,\n utf8_res: Result,\n read: usize,\n output: &mut String,\n truncate_on_io_error: bool,\n) -> Poll> {\n match (io_res, utf8_res) {\n (Ok(num_bytes), Ok(string)) => {\n debug_assert_eq!(read, 0);\n *output = string;\n Poll::Ready(Ok(num_bytes))\n }\n (Err(io_err), Ok(string)) => {\n *output = string;\n if truncate_on_io_error {\n let original_len = output.len() - read;\n output.truncate(original_len);\n }\n Poll::Ready(Err(io_err))\n }\n (Ok(num_bytes), Err(utf8_err)) => {\n debug_assert_eq!(read, 0);\n put_back_original_data(output, utf8_err.into_bytes(), num_bytes);\n\n Poll::Ready(Err(io::Error::new(\n io::ErrorKind::InvalidData,\n \"stream did not contain valid UTF-8\",\n )))\n }\n (Err(io_err), Err(utf8_err)) => {\n put_back_original_data(output, utf8_err.into_bytes(), read);\n\n Poll::Ready(Err(io_err))\n }\n }\n}\n\npub(super) fn read_line_internal(\n reader: Pin<&mut R>,\n cx: &mut Context<'_>,\n output: &mut String,\n buf: &mut Vec,\n read: &mut usize,\n) -> Poll> {\n let io_res = ready!(read_until_internal(reader, cx, b'\\n', buf, read));\n let utf8_res = String::from_utf8(mem::take(buf));\n\n // At this point both buf and output are empty. The allocation is in utf8_res.\n\n debug_assert!(buf.is_empty());\n debug_assert!(output.is_empty());\n finish_string_read(io_res, utf8_res, *read, output, false)\n}\n\nimpl Future for ReadLine<'_, R> {\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n read_line_internal(Pin::new(*me.reader), cx, me.output, me.buf, me.read)\n }\n}\n" }, { "path": "tokio/src/io/util/read_to_end.rs", "content": "use crate::io::util::vec_with_initialized::{into_read_buf_parts, VecU8, VecWithInitialized};\nuse crate::io::{AsyncRead, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::mem::{self, MaybeUninit};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadToEnd<'a, R: ?Sized> {\n reader: &'a mut R,\n buf: VecWithInitialized<&'a mut Vec>,\n // The number of bytes appended to buf. This can be less than buf.len() if\n // the buffer was not empty when the operation was started.\n read: usize,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn read_to_end<'a, R>(reader: &'a mut R, buffer: &'a mut Vec) -> ReadToEnd<'a, R>\nwhere\n R: AsyncRead + Unpin + ?Sized,\n{\n ReadToEnd {\n reader,\n buf: VecWithInitialized::new(buffer),\n read: 0,\n _pin: PhantomPinned,\n }\n}\n\npub(super) fn read_to_end_internal(\n buf: &mut VecWithInitialized,\n mut reader: Pin<&mut R>,\n num_read: &mut usize,\n cx: &mut Context<'_>,\n) -> Poll> {\n loop {\n let ret = ready!(poll_read_to_end(buf, reader.as_mut(), cx));\n match ret {\n Err(err) => return Poll::Ready(Err(err)),\n Ok(0) => return Poll::Ready(Ok(mem::replace(num_read, 0))),\n Ok(num) => {\n *num_read += num;\n }\n }\n }\n}\n\n/// Tries to read from the provided [`AsyncRead`].\n///\n/// The length of the buffer is increased by the number of bytes read.\nfn poll_read_to_end(\n buf: &mut VecWithInitialized,\n read: Pin<&mut R>,\n cx: &mut Context<'_>,\n) -> Poll> {\n // This uses an adaptive system to extend the vector when it fills. We want to\n // avoid paying to allocate and zero a huge chunk of memory if the reader only\n // has 4 bytes while still making large reads if the reader does have a ton\n // of data to return. Simply tacking on an extra DEFAULT_BUF_SIZE space every\n // time is 4,500 times (!) slower than this if the reader has a very small\n // amount of data to return. When the vector is full with its starting\n // capacity, we first try to read into a small buffer to see if we reached\n // an EOF. This only happens when the starting capacity is >= NUM_BYTES, since\n // we allocate at least NUM_BYTES each time. This avoids the unnecessary\n // allocation that we attempt before reading into the vector.\n\n const NUM_BYTES: usize = 32;\n let try_small_read = buf.try_small_read_first(NUM_BYTES);\n\n // Get a ReadBuf into the vector.\n let mut read_buf;\n let poll_result;\n\n let n = if try_small_read {\n // Read some bytes using a small read.\n let mut small_buf: [MaybeUninit; NUM_BYTES] = [MaybeUninit::uninit(); NUM_BYTES];\n let mut small_read_buf = ReadBuf::uninit(&mut small_buf);\n poll_result = read.poll_read(cx, &mut small_read_buf);\n let to_write = small_read_buf.filled();\n\n // Ensure we have enough space to fill our vector with what we read.\n read_buf = buf.get_read_buf();\n if to_write.len() > read_buf.remaining() {\n buf.reserve(NUM_BYTES);\n read_buf = buf.get_read_buf();\n }\n read_buf.put_slice(to_write);\n\n to_write.len()\n } else {\n // Ensure we have enough space for reading.\n buf.reserve(NUM_BYTES);\n read_buf = buf.get_read_buf();\n\n // Read data directly into vector.\n let filled_before = read_buf.filled().len();\n poll_result = read.poll_read(cx, &mut read_buf);\n\n // Compute the number of bytes read.\n read_buf.filled().len() - filled_before\n };\n\n // Update the length of the vector using the result of poll_read.\n let read_buf_parts = into_read_buf_parts(read_buf);\n buf.apply_read_buf(read_buf_parts);\n\n match poll_result {\n Poll::Pending => {\n // In this case, nothing should have been read. However we still\n // update the vector in case the poll_read call initialized parts of\n // the vector's unused capacity.\n debug_assert_eq!(n, 0);\n Poll::Pending\n }\n Poll::Ready(Err(err)) => {\n debug_assert_eq!(n, 0);\n Poll::Ready(Err(err))\n }\n Poll::Ready(Ok(())) => Poll::Ready(Ok(n)),\n }\n}\n\nimpl Future for ReadToEnd<'_, A>\nwhere\n A: AsyncRead + ?Sized + Unpin,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n read_to_end_internal(me.buf, Pin::new(*me.reader), me.read, cx)\n }\n}\n" }, { "path": "tokio/src/io/util/read_to_string.rs", "content": "use crate::io::util::read_line::finish_string_read;\nuse crate::io::util::read_to_end::read_to_end_internal;\nuse crate::io::util::vec_with_initialized::VecWithInitialized;\nuse crate::io::AsyncRead;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse std::{io, mem};\n\npin_project! {\n /// Future for the [`read_to_string`](super::AsyncReadExt::read_to_string) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadToString<'a, R: ?Sized> {\n reader: &'a mut R,\n // This is the buffer we were provided. It will be replaced with an empty string\n // while reading to postpone utf-8 handling until after reading.\n output: &'a mut String,\n // The actual allocation of the string is moved into this vector instead.\n buf: VecWithInitialized>,\n // The number of bytes appended to buf. This can be less than buf.len() if\n // the buffer was not empty when the operation was started.\n read: usize,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn read_to_string<'a, R>(\n reader: &'a mut R,\n string: &'a mut String,\n) -> ReadToString<'a, R>\nwhere\n R: AsyncRead + ?Sized + Unpin,\n{\n let buf = mem::take(string).into_bytes();\n ReadToString {\n reader,\n buf: VecWithInitialized::new(buf),\n output: string,\n read: 0,\n _pin: PhantomPinned,\n }\n}\n\nfn read_to_string_internal(\n reader: Pin<&mut R>,\n output: &mut String,\n buf: &mut VecWithInitialized>,\n read: &mut usize,\n cx: &mut Context<'_>,\n) -> Poll> {\n let io_res = ready!(read_to_end_internal(buf, reader, read, cx));\n let utf8_res = String::from_utf8(buf.take());\n\n // At this point both buf and output are empty. The allocation is in utf8_res.\n\n debug_assert!(buf.is_empty());\n debug_assert!(output.is_empty());\n finish_string_read(io_res, utf8_res, *read, output, true)\n}\n\nimpl Future for ReadToString<'_, A>\nwhere\n A: AsyncRead + ?Sized + Unpin,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n read_to_string_internal(Pin::new(*me.reader), me.output, me.buf, me.read, cx)\n }\n}\n" }, { "path": "tokio/src/io/util/read_until.rs", "content": "use crate::io::AsyncBufRead;\nuse crate::util::memchr;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::mem;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Future for the [`read_until`](crate::io::AsyncBufReadExt::read_until) method.\n /// The delimiter is included in the resulting vector.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct ReadUntil<'a, R: ?Sized> {\n reader: &'a mut R,\n delimiter: u8,\n buf: &'a mut Vec,\n // The number of bytes appended to buf. This can be less than buf.len() if\n // the buffer was not empty when the operation was started.\n read: usize,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn read_until<'a, R>(\n reader: &'a mut R,\n delimiter: u8,\n buf: &'a mut Vec,\n) -> ReadUntil<'a, R>\nwhere\n R: AsyncBufRead + ?Sized + Unpin,\n{\n ReadUntil {\n reader,\n delimiter,\n buf,\n read: 0,\n _pin: PhantomPinned,\n }\n}\n\npub(super) fn read_until_internal(\n mut reader: Pin<&mut R>,\n cx: &mut Context<'_>,\n delimiter: u8,\n buf: &mut Vec,\n read: &mut usize,\n) -> Poll> {\n loop {\n let (done, used) = {\n let available = ready!(reader.as_mut().poll_fill_buf(cx))?;\n if let Some(i) = memchr::memchr(delimiter, available) {\n buf.extend_from_slice(&available[..=i]);\n (true, i + 1)\n } else {\n buf.extend_from_slice(available);\n (false, available.len())\n }\n };\n reader.as_mut().consume(used);\n *read += used;\n if done || used == 0 {\n return Poll::Ready(Ok(mem::replace(read, 0)));\n }\n }\n}\n\nimpl Future for ReadUntil<'_, R> {\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n read_until_internal(Pin::new(*me.reader), cx, *me.delimiter, me.buf, me.read)\n }\n}\n" }, { "path": "tokio/src/io/util/repeat.rs", "content": "use bytes::BufMut;\n\nuse crate::io::util::poll_proceed_and_make_progress;\nuse crate::io::{AsyncRead, ReadBuf};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n /// An async reader which yields one byte over and over and over and over and\n /// over and...\n ///\n /// This struct is generally created by calling [`repeat`][repeat]. Please\n /// see the documentation of `repeat()` for more details.\n ///\n /// This is an asynchronous version of [`std::io::Repeat`][std].\n ///\n /// [repeat]: fn@repeat\n /// [std]: std::io::Repeat\n #[derive(Debug)]\n pub struct Repeat {\n byte: u8,\n }\n\n /// Creates an instance of an async reader that infinitely repeats one byte.\n ///\n /// All reads from this reader will succeed by filling the specified buffer with\n /// the given byte.\n ///\n /// This is an asynchronous version of [`std::io::repeat`][std].\n ///\n /// [std]: std::io::repeat\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncReadExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut buffer = [0; 3];\n /// io::repeat(0b101).read_exact(&mut buffer).await.unwrap();\n /// assert_eq!(buffer, [0b101, 0b101, 0b101]);\n /// # }\n /// ```\n pub fn repeat(byte: u8) -> Repeat {\n Repeat { byte }\n }\n}\n\nimpl AsyncRead for Repeat {\n #[inline]\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n buf.put_bytes(self.byte, buf.remaining());\n Poll::Ready(Ok(()))\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::();\n }\n}\n" }, { "path": "tokio/src/io/util/shutdown.rs", "content": "use crate::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A future used to shutdown an I/O object.\n ///\n /// Created by the [`AsyncWriteExt::shutdown`][shutdown] function.\n /// [shutdown]: [`crate::io::AsyncWriteExt::shutdown`]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n #[derive(Debug)]\n pub struct Shutdown<'a, A: ?Sized> {\n a: &'a mut A,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Creates a future which will shutdown an I/O object.\npub(super) fn shutdown(a: &mut A) -> Shutdown<'_, A>\nwhere\n A: AsyncWrite + Unpin + ?Sized,\n{\n Shutdown {\n a,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for Shutdown<'_, A>\nwhere\n A: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n Pin::new(me.a).poll_shutdown(cx)\n }\n}\n" }, { "path": "tokio/src/io/util/sink.rs", "content": "use crate::io::util::poll_proceed_and_make_progress;\nuse crate::io::AsyncWrite;\n\nuse std::fmt;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n /// An async writer which will move data into the void.\n ///\n /// This struct is generally created by calling [`sink`][sink]. Please\n /// see the documentation of `sink()` for more details.\n ///\n /// This is an asynchronous version of [`std::io::Sink`][std].\n ///\n /// [sink]: sink()\n /// [std]: std::io::Sink\n pub struct Sink {\n _p: (),\n }\n\n /// Creates an instance of an async writer which will successfully consume all\n /// data.\n ///\n /// All calls to [`poll_write`] on the returned instance will return\n /// `Poll::Ready(Ok(buf.len()))` and the contents of the buffer will not be\n /// inspected.\n ///\n /// This is an asynchronous version of [`std::io::sink`][std].\n ///\n /// [`poll_write`]: crate::io::AsyncWrite::poll_write()\n /// [std]: std::io::sink\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::io::{self, AsyncWriteExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let buffer = vec![1, 2, 3, 5, 8];\n /// let num_bytes = io::sink().write(&buffer).await?;\n /// assert_eq!(num_bytes, 5);\n /// Ok(())\n /// # }\n /// ```\n pub fn sink() -> Sink {\n Sink { _p: () }\n }\n}\n\nimpl AsyncWrite for Sink {\n #[inline]\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(buf.len()))\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(()))\n }\n\n #[inline]\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n ready!(poll_proceed_and_make_progress(cx));\n Poll::Ready(Ok(()))\n }\n}\n\nimpl fmt::Debug for Sink {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.pad(\"Sink { .. }\")\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::();\n }\n}\n" }, { "path": "tokio/src/io/util/split.rs", "content": "use crate::io::util::read_until::read_until_internal;\nuse crate::io::AsyncBufRead;\n\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::mem;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Splitter for the [`split`](crate::io::AsyncBufReadExt::split) method.\n ///\n /// A `Split` can be turned into a `Stream` with [`SplitStream`].\n ///\n /// [`SplitStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.SplitStream.html\n #[derive(Debug)]\n #[must_use = \"streams do nothing unless polled\"]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct Split {\n #[pin]\n reader: R,\n buf: Vec,\n delim: u8,\n read: usize,\n }\n}\n\npub(crate) fn split(reader: R, delim: u8) -> Split\nwhere\n R: AsyncBufRead,\n{\n Split {\n reader,\n buf: Vec::new(),\n delim,\n read: 0,\n }\n}\n\nimpl Split\nwhere\n R: AsyncBufRead + Unpin,\n{\n /// Returns the next segment in the stream.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncBufRead;\n /// use tokio::io::AsyncBufReadExt;\n ///\n /// # async fn dox(my_buf_read: impl AsyncBufRead + Unpin) -> std::io::Result<()> {\n /// let mut segments = my_buf_read.split(b'f');\n ///\n /// while let Some(segment) = segments.next_segment().await? {\n /// println!(\"length = {}\", segment.len())\n /// }\n /// # Ok(())\n /// # }\n /// ```\n pub async fn next_segment(&mut self) -> io::Result>> {\n use std::future::poll_fn;\n\n poll_fn(|cx| Pin::new(&mut *self).poll_next_segment(cx)).await\n }\n}\n\nimpl Split\nwhere\n R: AsyncBufRead,\n{\n /// Polls for the next segment in the stream.\n ///\n /// This method returns:\n ///\n /// * `Poll::Pending` if the next segment is not yet available.\n /// * `Poll::Ready(Ok(Some(segment)))` if the next segment is available.\n /// * `Poll::Ready(Ok(None))` if there are no more segments in this stream.\n /// * `Poll::Ready(Err(err))` if an IO error occurred while reading the\n /// next segment.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when more bytes become\n /// available on the underlying IO resource.\n ///\n /// Note that on multiple calls to `poll_next_segment`, only the `Waker`\n /// from the `Context` passed to the most recent call is scheduled to\n /// receive a wakeup.\n pub fn poll_next_segment(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll>>> {\n let me = self.project();\n\n let n = ready!(read_until_internal(\n me.reader, cx, *me.delim, me.buf, me.read,\n ))?;\n // read_until_internal resets me.read to zero once it finds the delimiter\n debug_assert_eq!(*me.read, 0);\n\n if n == 0 && me.buf.is_empty() {\n return Poll::Ready(Ok(None));\n }\n\n if me.buf.last() == Some(me.delim) {\n me.buf.pop();\n }\n\n Poll::Ready(Ok(Some(mem::take(me.buf))))\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/take.rs", "content": "use crate::io::{AsyncBufRead, AsyncRead, ReadBuf};\n\nuse pin_project_lite::pin_project;\nuse std::convert::TryFrom;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse std::{cmp, io};\n\npin_project! {\n /// Stream for the [`take`](super::AsyncReadExt::take) method.\n #[derive(Debug)]\n #[must_use = \"streams do nothing unless you `.await` or poll them\"]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct Take {\n #[pin]\n inner: R,\n // Add '_' to avoid conflicts with `limit` method.\n limit_: u64,\n }\n}\n\npub(super) fn take(inner: R, limit: u64) -> Take {\n Take {\n inner,\n limit_: limit,\n }\n}\n\nimpl Take {\n /// Returns the remaining number of bytes that can be\n /// read before this instance will return EOF.\n ///\n /// # Note\n ///\n /// This instance may reach `EOF` after reading fewer bytes than indicated by\n /// this method if the underlying [`AsyncRead`] instance reaches EOF.\n pub fn limit(&self) -> u64 {\n self.limit_\n }\n\n /// Sets the number of bytes that can be read before this instance will\n /// return EOF. This is the same as constructing a new `Take` instance, so\n /// the amount of bytes read and the previous limit value don't matter when\n /// calling this method.\n pub fn set_limit(&mut self, limit: u64) {\n self.limit_ = limit;\n }\n\n /// Gets a reference to the underlying reader.\n pub fn get_ref(&self) -> &R {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying reader.\n ///\n /// Care should be taken to avoid modifying the internal I/O state of the\n /// underlying reader as doing so may corrupt the internal limit of this\n /// `Take`.\n pub fn get_mut(&mut self) -> &mut R {\n &mut self.inner\n }\n\n /// Gets a pinned mutable reference to the underlying reader.\n ///\n /// Care should be taken to avoid modifying the internal I/O state of the\n /// underlying reader as doing so may corrupt the internal limit of this\n /// `Take`.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut R> {\n self.project().inner\n }\n\n /// Consumes the `Take`, returning the wrapped reader.\n pub fn into_inner(self) -> R {\n self.inner\n }\n}\n\nimpl AsyncRead for Take {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if self.limit_ == 0 {\n return Poll::Ready(Ok(()));\n }\n\n let me = self.project();\n let mut b = buf.take(usize::try_from(*me.limit_).unwrap_or(usize::MAX));\n\n let buf_ptr = b.filled().as_ptr();\n ready!(me.inner.poll_read(cx, &mut b))?;\n assert_eq!(b.filled().as_ptr(), buf_ptr);\n\n let n = b.filled().len();\n\n // We need to update the original ReadBuf\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n *me.limit_ -= n as u64;\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncBufRead for Take {\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n\n // Don't call into inner reader at all at EOF because it may still block\n if *me.limit_ == 0 {\n return Poll::Ready(Ok(&[]));\n }\n\n let buf = ready!(me.inner.poll_fill_buf(cx)?);\n let cap = cmp::min(buf.len() as u64, *me.limit_) as usize;\n Poll::Ready(Ok(&buf[..cap]))\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n let me = self.project();\n // Don't let callers reset the limit by passing an overlarge value\n let amt = cmp::min(amt as u64, *me.limit_) as usize;\n *me.limit_ -= amt as u64;\n me.inner.consume(amt);\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn assert_unpin() {\n crate::is_unpin::>();\n }\n}\n" }, { "path": "tokio/src/io/util/vec_with_initialized.rs", "content": "use crate::io::ReadBuf;\nuse std::mem::MaybeUninit;\n\n/// Something that looks like a `Vec`.\n///\n/// # Safety\n///\n/// The implementor must guarantee that the vector returned by the\n/// `as_mut` and `as_mut` methods do not change from one call to\n/// another.\npub(crate) unsafe trait VecU8: AsRef> + AsMut> {}\n\nunsafe impl VecU8 for Vec {}\nunsafe impl VecU8 for &mut Vec {}\n\n/// This struct wraps a `Vec` or `&mut Vec`, combining it with a\n/// `num_initialized`, which keeps track of the number of initialized bytes\n/// in the unused capacity.\n///\n/// The purpose of this struct is to remember how many bytes were initialized\n/// through a `ReadBuf` from call to call.\n///\n/// This struct has the safety invariant that the first `num_initialized` of the\n/// vector's allocation must be initialized at any time.\n#[derive(Debug)]\npub(crate) struct VecWithInitialized {\n vec: V,\n // The number of initialized bytes in the vector.\n // Always between `vec.len()` and `vec.capacity()`.\n num_initialized: usize,\n starting_capacity: usize,\n}\n\nimpl VecWithInitialized> {\n #[cfg(feature = \"io-util\")]\n pub(crate) fn take(&mut self) -> Vec {\n self.num_initialized = 0;\n std::mem::take(&mut self.vec)\n }\n}\n\nimpl VecWithInitialized\nwhere\n V: VecU8,\n{\n pub(crate) fn new(mut vec: V) -> Self {\n // SAFETY: The safety invariants of vector guarantee that the bytes up\n // to its length are initialized.\n Self {\n num_initialized: vec.as_mut().len(),\n starting_capacity: vec.as_ref().capacity(),\n vec,\n }\n }\n\n pub(crate) fn reserve(&mut self, num_bytes: usize) {\n let vec = self.vec.as_mut();\n if vec.capacity() - vec.len() >= num_bytes {\n return;\n }\n // SAFETY: Setting num_initialized to `vec.len()` is correct as\n // `reserve` does not change the length of the vector.\n self.num_initialized = vec.len();\n vec.reserve(num_bytes);\n }\n\n #[cfg(feature = \"io-util\")]\n pub(crate) fn is_empty(&self) -> bool {\n self.vec.as_ref().is_empty()\n }\n\n pub(crate) fn get_read_buf<'a>(&'a mut self) -> ReadBuf<'a> {\n let num_initialized = self.num_initialized;\n\n // SAFETY: Creating the slice is safe because of the safety invariants\n // on Vec. The safety invariants of `ReadBuf` will further guarantee\n // that no bytes in the slice are de-initialized.\n let vec = self.vec.as_mut();\n let len = vec.len();\n let cap = vec.capacity();\n let ptr = vec.as_mut_ptr().cast::>();\n let slice = unsafe { std::slice::from_raw_parts_mut::<'a, MaybeUninit>(ptr, cap) };\n\n // SAFETY: This is safe because the safety invariants of\n // VecWithInitialized say that the first num_initialized bytes must be\n // initialized.\n let mut read_buf = ReadBuf::uninit(slice);\n unsafe {\n read_buf.assume_init(num_initialized);\n }\n read_buf.set_filled(len);\n\n read_buf\n }\n\n pub(crate) fn apply_read_buf(&mut self, parts: ReadBufParts) {\n let vec = self.vec.as_mut();\n assert_eq!(vec.as_ptr(), parts.ptr);\n\n // SAFETY:\n // The ReadBufParts really does point inside `self.vec` due to the above\n // check, and the safety invariants of `ReadBuf` guarantee that the\n // first `parts.initialized` bytes of `self.vec` really have been\n // initialized. Additionally, `ReadBuf` guarantees that `parts.len` is\n // at most `parts.initialized`, so the first `parts.len` bytes are also\n // initialized.\n //\n // Note that this relies on the fact that `V` is either `Vec` or\n // `&mut Vec`, so the vector returned by `self.vec.as_mut()` cannot\n // change from call to call.\n unsafe {\n self.num_initialized = parts.initialized;\n vec.set_len(parts.len);\n }\n }\n\n // Returns a boolean telling the caller to try reading into a small local buffer first if true.\n // Doing so would avoid overallocating when vec is filled to capacity and we reached EOF.\n pub(crate) fn try_small_read_first(&self, num_bytes: usize) -> bool {\n let vec = self.vec.as_ref();\n vec.capacity() - vec.len() < num_bytes\n && self.starting_capacity == vec.capacity()\n && self.starting_capacity >= num_bytes\n }\n}\n\npub(crate) struct ReadBufParts {\n // Pointer is only used to check that the ReadBuf actually came from the\n // right VecWithInitialized.\n ptr: *const u8,\n len: usize,\n initialized: usize,\n}\n\n// This is needed to release the borrow on `VecWithInitialized`.\npub(crate) fn into_read_buf_parts(rb: ReadBuf<'_>) -> ReadBufParts {\n ReadBufParts {\n ptr: rb.filled().as_ptr(),\n len: rb.filled().len(),\n initialized: rb.initialized().len(),\n }\n}\n" }, { "path": "tokio/src/io/util/write.rs", "content": "use crate::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A future to write some of the buffer to an `AsyncWrite`.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Write<'a, W: ?Sized> {\n writer: &'a mut W,\n buf: &'a [u8],\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Tries to write some bytes from the given `buf` to the writer in an\n/// asynchronous manner, returning a future.\npub(crate) fn write<'a, W>(writer: &'a mut W, buf: &'a [u8]) -> Write<'a, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n Write {\n writer,\n buf,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for Write<'_, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n Pin::new(&mut *me.writer).poll_write(cx, me.buf)\n }\n}\n" }, { "path": "tokio/src/io/util/write_all.rs", "content": "use crate::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::mem;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct WriteAll<'a, W: ?Sized> {\n writer: &'a mut W,\n buf: &'a [u8],\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn write_all<'a, W>(writer: &'a mut W, buf: &'a [u8]) -> WriteAll<'a, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n WriteAll {\n writer,\n buf,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for WriteAll<'_, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n while !me.buf.is_empty() {\n let n = ready!(Pin::new(&mut *me.writer).poll_write(cx, me.buf))?;\n {\n let (_, rest) = mem::take(&mut *me.buf).split_at(n);\n *me.buf = rest;\n }\n if n == 0 {\n return Poll::Ready(Err(io::ErrorKind::WriteZero.into()));\n }\n }\n\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio/src/io/util/write_all_buf.rs", "content": "use crate::io::AsyncWrite;\n\nuse bytes::Buf;\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io::{self, IoSlice};\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// A future to write some of the buffer to an `AsyncWrite`.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct WriteAllBuf<'a, W, B> {\n writer: &'a mut W,\n buf: &'a mut B,\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Tries to write some bytes from the given `buf` to the writer in an\n/// asynchronous manner, returning a future.\npub(crate) fn write_all_buf<'a, W, B>(writer: &'a mut W, buf: &'a mut B) -> WriteAllBuf<'a, W, B>\nwhere\n W: AsyncWrite + Unpin,\n B: Buf,\n{\n WriteAllBuf {\n writer,\n buf,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for WriteAllBuf<'_, W, B>\nwhere\n W: AsyncWrite + Unpin,\n B: Buf,\n{\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n const MAX_VECTOR_ELEMENTS: usize = 64;\n\n let me = self.project();\n while me.buf.has_remaining() {\n let n = if me.writer.is_write_vectored() {\n let mut slices = [IoSlice::new(&[]); MAX_VECTOR_ELEMENTS];\n let cnt = me.buf.chunks_vectored(&mut slices);\n ready!(Pin::new(&mut *me.writer).poll_write_vectored(cx, &slices[..cnt]))?\n } else {\n ready!(Pin::new(&mut *me.writer).poll_write(cx, me.buf.chunk())?)\n };\n me.buf.advance(n);\n if n == 0 {\n return Poll::Ready(Err(io::ErrorKind::WriteZero.into()));\n }\n }\n\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio/src/io/util/write_buf.rs", "content": "use crate::io::AsyncWrite;\n\nuse bytes::Buf;\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io::{self, IoSlice};\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// A future to write some of the buffer to an `AsyncWrite`.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct WriteBuf<'a, W, B> {\n writer: &'a mut W,\n buf: &'a mut B,\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Tries to write some bytes from the given `buf` to the writer in an\n/// asynchronous manner, returning a future.\npub(crate) fn write_buf<'a, W, B>(writer: &'a mut W, buf: &'a mut B) -> WriteBuf<'a, W, B>\nwhere\n W: AsyncWrite + Unpin,\n B: Buf,\n{\n WriteBuf {\n writer,\n buf,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for WriteBuf<'_, W, B>\nwhere\n W: AsyncWrite + Unpin,\n B: Buf,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n const MAX_VECTOR_ELEMENTS: usize = 64;\n\n let me = self.project();\n\n if !me.buf.has_remaining() {\n return Poll::Ready(Ok(0));\n }\n\n let n = if me.writer.is_write_vectored() {\n let mut slices = [IoSlice::new(&[]); MAX_VECTOR_ELEMENTS];\n let cnt = me.buf.chunks_vectored(&mut slices);\n ready!(Pin::new(&mut *me.writer).poll_write_vectored(cx, &slices[..cnt]))?\n } else {\n ready!(Pin::new(&mut *me.writer).poll_write(cx, me.buf.chunk()))?\n };\n\n me.buf.advance(n);\n Poll::Ready(Ok(n))\n }\n}\n" }, { "path": "tokio/src/io/util/write_int.rs", "content": "use crate::io::AsyncWrite;\n\nuse bytes::BufMut;\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nmacro_rules! writer {\n ($name:ident, $ty:ty, $writer:ident) => {\n writer!($name, $ty, $writer, std::mem::size_of::<$ty>());\n };\n ($name:ident, $ty:ty, $writer:ident, $bytes:expr) => {\n pin_project! {\n #[doc(hidden)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct $name {\n #[pin]\n dst: W,\n buf: [u8; $bytes],\n written: u8,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n }\n\n impl $name {\n pub(crate) fn new(w: W, value: $ty) -> Self {\n let mut writer = Self {\n buf: [0; $bytes],\n written: 0,\n dst: w,\n _pin: PhantomPinned,\n };\n BufMut::$writer(&mut &mut writer.buf[..], value);\n writer\n }\n }\n\n impl Future for $name\n where\n W: AsyncWrite,\n {\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let mut me = self.project();\n\n if *me.written == $bytes as u8 {\n return Poll::Ready(Ok(()));\n }\n\n while *me.written < $bytes as u8 {\n *me.written += match me\n .dst\n .as_mut()\n .poll_write(cx, &me.buf[*me.written as usize..])\n {\n Poll::Pending => return Poll::Pending,\n Poll::Ready(Err(e)) => return Poll::Ready(Err(e.into())),\n Poll::Ready(Ok(0)) => {\n return Poll::Ready(Err(io::ErrorKind::WriteZero.into()));\n }\n Poll::Ready(Ok(n)) => n as u8,\n };\n }\n Poll::Ready(Ok(()))\n }\n }\n };\n}\n\nmacro_rules! writer8 {\n ($name:ident, $ty:ty) => {\n pin_project! {\n #[doc(hidden)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct $name {\n #[pin]\n dst: W,\n byte: $ty,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n }\n\n impl $name {\n pub(crate) fn new(dst: W, byte: $ty) -> Self {\n Self {\n dst,\n byte,\n _pin: PhantomPinned,\n }\n }\n }\n\n impl Future for $name\n where\n W: AsyncWrite,\n {\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n let buf = [*me.byte as u8];\n\n match me.dst.poll_write(cx, &buf[..]) {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(e)) => Poll::Ready(Err(e.into())),\n Poll::Ready(Ok(0)) => Poll::Ready(Err(io::ErrorKind::WriteZero.into())),\n Poll::Ready(Ok(1)) => Poll::Ready(Ok(())),\n Poll::Ready(Ok(_)) => unreachable!(),\n }\n }\n }\n };\n}\n\nwriter8!(WriteU8, u8);\nwriter8!(WriteI8, i8);\n\nwriter!(WriteU16, u16, put_u16);\nwriter!(WriteU32, u32, put_u32);\nwriter!(WriteU64, u64, put_u64);\nwriter!(WriteU128, u128, put_u128);\n\nwriter!(WriteI16, i16, put_i16);\nwriter!(WriteI32, i32, put_i32);\nwriter!(WriteI64, i64, put_i64);\nwriter!(WriteI128, i128, put_i128);\n\nwriter!(WriteF32, f32, put_f32);\nwriter!(WriteF64, f64, put_f64);\n\nwriter!(WriteU16Le, u16, put_u16_le);\nwriter!(WriteU32Le, u32, put_u32_le);\nwriter!(WriteU64Le, u64, put_u64_le);\nwriter!(WriteU128Le, u128, put_u128_le);\n\nwriter!(WriteI16Le, i16, put_i16_le);\nwriter!(WriteI32Le, i32, put_i32_le);\nwriter!(WriteI64Le, i64, put_i64_le);\nwriter!(WriteI128Le, i128, put_i128_le);\n\nwriter!(WriteF32Le, f32, put_f32_le);\nwriter!(WriteF64Le, f64, put_f64_le);\n" }, { "path": "tokio/src/io/util/write_vectored.rs", "content": "use crate::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse std::{future::Future, io::IoSlice};\n\npin_project! {\n /// A future to write a slice of buffers to an `AsyncWrite`.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct WriteVectored<'a, 'b, W: ?Sized> {\n writer: &'a mut W,\n bufs: &'a [IoSlice<'b>],\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\npub(crate) fn write_vectored<'a, 'b, W>(\n writer: &'a mut W,\n bufs: &'a [IoSlice<'b>],\n) -> WriteVectored<'a, 'b, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n WriteVectored {\n writer,\n bufs,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for WriteVectored<'_, '_, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n Pin::new(&mut *me.writer).poll_write_vectored(cx, me.bufs)\n }\n}\n" }, { "path": "tokio/src/lib.rs", "content": "#![allow(\n clippy::cognitive_complexity,\n clippy::large_enum_variant,\n clippy::module_inception,\n clippy::needless_doctest_main\n)]\n#![warn(\n missing_debug_implementations,\n missing_docs,\n rust_2018_idioms,\n unreachable_pub\n)]\n#![deny(unused_must_use, unsafe_op_in_unsafe_fn)]\n#![doc(test(\n no_crate_inject,\n attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))\n))]\n// loom is an internal implementation detail.\n// Do not show \"Available on non-loom only\" label\n#![cfg_attr(docsrs, doc(auto_cfg(hide(loom))))]\n#![cfg_attr(docsrs, feature(doc_cfg))]\n#![cfg_attr(docsrs, allow(unused_attributes))]\n#![cfg_attr(loom, allow(dead_code, unreachable_pub))]\n#![cfg_attr(windows, allow(rustdoc::broken_intra_doc_links))]\n\n//! A runtime for writing reliable network applications without compromising speed.\n//!\n//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous\n//! applications with the Rust programming language. At a high level, it\n//! provides a few major components:\n//!\n//! * Tools for [working with asynchronous tasks][tasks], including\n//! [synchronization primitives and channels][sync] and [timeouts, sleeps, and\n//! intervals][time].\n//! * APIs for [performing asynchronous I/O][io], including [TCP and UDP][net] sockets,\n//! [filesystem][fs] operations, and [process] and [signal] management.\n//! * A [runtime] for executing asynchronous code, including a task scheduler,\n//! an I/O driver backed by the operating system's event queue (`epoll`, `kqueue`,\n//! `IOCP`, etc...), and a high performance timer.\n//!\n//! Guide level documentation is found on the [website].\n//!\n//! [tasks]: #working-with-tasks\n//! [sync]: crate::sync\n//! [time]: crate::time\n//! [io]: #asynchronous-io\n//! [net]: crate::net\n//! [fs]: crate::fs\n//! [process]: crate::process\n//! [signal]: crate::signal\n//! [fs]: crate::fs\n//! [runtime]: crate::runtime\n//! [website]: https://tokio.rs/tokio/tutorial\n//!\n//! # A Tour of Tokio\n//!\n//! Tokio consists of a number of modules that provide a range of functionality\n//! essential for implementing asynchronous applications in Rust. In this\n//! section, we will take a brief tour of Tokio, summarizing the major APIs and\n//! their uses.\n//!\n//! The easiest way to get started is to enable all features. Do this by\n//! enabling the `full` feature flag:\n//!\n//! ```toml\n//! tokio = { version = \"1\", features = [\"full\"] }\n//! ```\n//!\n//! ### Authoring applications\n//!\n//! Tokio is great for writing applications and most users in this case shouldn't\n//! worry too much about what features they should pick. If you're unsure, we suggest\n//! going with `full` to ensure that you don't run into any road blocks while you're\n//! building your application.\n//!\n//! #### Example\n//!\n//! This example shows the quickest way to get started with Tokio.\n//!\n//! ```toml\n//! tokio = { version = \"1\", features = [\"full\"] }\n//! ```\n//!\n//! ### Authoring libraries\n//!\n//! As a library author your goal should be to provide the lightest weight crate\n//! that is based on Tokio. To achieve this you should ensure that you only enable\n//! the features you need. This allows users to pick up your crate without having\n//! to enable unnecessary features.\n//!\n//! #### Example\n//!\n//! This example shows how you may want to import features for a library that just\n//! needs to `tokio::spawn` and use a `TcpStream`.\n//!\n//! ```toml\n//! tokio = { version = \"1\", features = [\"rt\", \"net\"] }\n//! ```\n//!\n//! ## Working With Tasks\n//!\n//! Asynchronous programs in Rust are based around lightweight, non-blocking\n//! units of execution called [_tasks_][tasks]. The [`tokio::task`] module provides\n//! important tools for working with tasks:\n//!\n//! * The [`spawn`] function and [`JoinHandle`] type, for scheduling a new task\n//! on the Tokio runtime and awaiting the output of a spawned task, respectively,\n//! * Functions for [running blocking operations][blocking] in an asynchronous\n//! task context.\n//!\n//! The [`tokio::task`] module is present only when the \"rt\" feature flag\n//! is enabled.\n//!\n//! [tasks]: task/index.html#what-are-tasks\n//! [`tokio::task`]: crate::task\n//! [`spawn`]: crate::task::spawn()\n//! [`JoinHandle`]: crate::task::JoinHandle\n//! [blocking]: task/index.html#blocking-and-yielding\n//!\n//! The [`tokio::sync`] module contains synchronization primitives to use when\n//! needing to communicate or share data. These include:\n//!\n//! * channels ([`oneshot`], [`mpsc`], [`watch`], and [`broadcast`]), for sending values\n//! between tasks,\n//! * a non-blocking [`Mutex`], for controlling access to a shared, mutable\n//! value,\n//! * an asynchronous [`Barrier`] type, for multiple tasks to synchronize before\n//! beginning a computation.\n//!\n//! The `tokio::sync` module is present only when the \"sync\" feature flag is\n//! enabled.\n//!\n//! [`tokio::sync`]: crate::sync\n//! [`Mutex`]: crate::sync::Mutex\n//! [`Barrier`]: crate::sync::Barrier\n//! [`oneshot`]: crate::sync::oneshot\n//! [`mpsc`]: crate::sync::mpsc\n//! [`watch`]: crate::sync::watch\n//! [`broadcast`]: crate::sync::broadcast\n//!\n//! The [`tokio::time`] module provides utilities for tracking time and\n//! scheduling work. This includes functions for setting [timeouts][timeout] for\n//! tasks, [sleeping][sleep] work to run in the future, or [repeating an operation at an\n//! interval][interval].\n//!\n//! In order to use `tokio::time`, the \"time\" feature flag must be enabled.\n//!\n//! [`tokio::time`]: crate::time\n//! [sleep]: crate::time::sleep()\n//! [interval]: crate::time::interval()\n//! [timeout]: crate::time::timeout()\n//!\n//! Finally, Tokio provides a _runtime_ for executing asynchronous tasks. Most\n//! applications can use the [`#[tokio::main]`][main] macro to run their code on the\n//! Tokio runtime. However, this macro provides only basic configuration options. As\n//! an alternative, the [`tokio::runtime`] module provides more powerful APIs for configuring\n//! and managing runtimes. You should use that module if the `#[tokio::main]` macro doesn't\n//! provide the functionality you need.\n//!\n//! Using the runtime requires the \"rt\" or \"rt-multi-thread\" feature flags, to\n//! enable the current-thread [single-threaded scheduler][rt] and the [multi-thread\n//! scheduler][rt-multi-thread], respectively. See the [`runtime` module\n//! documentation][rt-features] for details. In addition, the \"macros\" feature\n//! flag enables the `#[tokio::main]` and `#[tokio::test]` attributes.\n//!\n//! [main]: attr.main.html\n//! [`tokio::runtime`]: crate::runtime\n//! [`Builder`]: crate::runtime::Builder\n//! [`Runtime`]: crate::runtime::Runtime\n//! [rt]: runtime/index.html#current-thread-scheduler\n//! [rt-multi-thread]: runtime/index.html#multi-thread-scheduler\n//! [rt-features]: runtime/index.html#runtime-scheduler\n//!\n//! ## CPU-bound tasks and blocking code\n//!\n//! Tokio is able to concurrently run many tasks on a few threads by repeatedly\n//! swapping the currently running task on each thread. However, this kind of\n//! swapping can only happen at `.await` points, so code that spends a long time\n//! without reaching an `.await` will prevent other tasks from running. To\n//! combat this, Tokio provides two kinds of threads: Core threads and blocking threads.\n//!\n//! The core threads are where all asynchronous code runs, and Tokio will by default\n//! spawn one for each CPU core. You can use the environment variable `TOKIO_WORKER_THREADS`\n//! to override the default value.\n//!\n//! The blocking threads are spawned on demand, can be used to run blocking code\n//! that would otherwise block other tasks from running and are kept alive when\n//! not used for a certain amount of time which can be configured with [`thread_keep_alive`].\n//! Since it is not possible for Tokio to swap out blocking tasks, like it\n//! can do with asynchronous code, the upper limit on the number of blocking\n//! threads is very large. These limits can be configured on the [`Builder`].\n//!\n//! To spawn a blocking task, you should use the [`spawn_blocking`] function.\n//!\n//! [`Builder`]: crate::runtime::Builder\n//! [`spawn_blocking`]: crate::task::spawn_blocking()\n//! [`thread_keep_alive`]: crate::runtime::Builder::thread_keep_alive()\n//!\n//! ```\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! #[tokio::main]\n//! async fn main() {\n//! // This is running on a core thread.\n//!\n//! let blocking_task = tokio::task::spawn_blocking(|| {\n//! // This is running on a blocking thread.\n//! // Blocking here is ok.\n//! });\n//!\n//! // We can wait for the blocking task like this:\n//! // If the blocking task panics, the unwrap below will propagate the\n//! // panic.\n//! blocking_task.await.unwrap();\n//! }\n//! # }\n//! ```\n//!\n//! If your code is CPU-bound and you wish to limit the number of threads used\n//! to run it, you should use a separate thread pool dedicated to CPU bound tasks.\n//! For example, you could consider using the [rayon] library for CPU-bound\n//! tasks. It is also possible to create an extra Tokio runtime dedicated to\n//! CPU-bound tasks, but if you do this, you should be careful that the extra\n//! runtime runs _only_ CPU-bound tasks, as IO-bound tasks on that runtime\n//! will behave poorly.\n//!\n//! Hint: If using rayon, you can use a [`oneshot`] channel to send the result back\n//! to Tokio when the rayon task finishes.\n//!\n//! [rayon]: https://docs.rs/rayon\n//! [`oneshot`]: crate::sync::oneshot\n//!\n//! ## Asynchronous IO\n//!\n//! As well as scheduling and running tasks, Tokio provides everything you need\n//! to perform input and output asynchronously.\n//!\n//! The [`tokio::io`] module provides Tokio's asynchronous core I/O primitives,\n//! the [`AsyncRead`], [`AsyncWrite`], and [`AsyncBufRead`] traits. In addition,\n//! when the \"io-util\" feature flag is enabled, it also provides combinators and\n//! functions for working with these traits, forming as an asynchronous\n//! counterpart to [`std::io`].\n//!\n//! Tokio also includes APIs for performing various kinds of I/O and interacting\n//! with the operating system asynchronously. These include:\n//!\n//! * [`tokio::net`], which contains non-blocking versions of [TCP], [UDP], and\n//! [Unix Domain Sockets][UDS] (enabled by the \"net\" feature flag),\n//! * [`tokio::fs`], similar to [`std::fs`] but for performing filesystem I/O\n//! asynchronously (enabled by the \"fs\" feature flag),\n//! * [`tokio::signal`], for asynchronously handling Unix and Windows OS signals\n//! (enabled by the \"signal\" feature flag),\n//! * [`tokio::process`], for spawning and managing child processes (enabled by\n//! the \"process\" feature flag).\n//!\n//! [`tokio::io`]: crate::io\n//! [`AsyncRead`]: crate::io::AsyncRead\n//! [`AsyncWrite`]: crate::io::AsyncWrite\n//! [`AsyncBufRead`]: crate::io::AsyncBufRead\n//! [`std::io`]: std::io\n//! [`tokio::net`]: crate::net\n//! [TCP]: crate::net::tcp\n//! [UDP]: crate::net::UdpSocket\n//! [UDS]: crate::net::unix\n//! [`tokio::fs`]: crate::fs\n//! [`std::fs`]: std::fs\n//! [`tokio::signal`]: crate::signal\n//! [`tokio::process`]: crate::process\n//!\n//! # Examples\n//!\n//! A simple TCP echo server:\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::net::TcpListener;\n//! use tokio::io::{AsyncReadExt, AsyncWriteExt};\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n//!\n//! loop {\n//! let (mut socket, _) = listener.accept().await?;\n//!\n//! tokio::spawn(async move {\n//! let mut buf = [0; 1024];\n//!\n//! // In a loop, read data from the socket and write the data back.\n//! loop {\n//! let n = match socket.read(&mut buf).await {\n//! // socket closed\n//! Ok(0) => return,\n//! Ok(n) => n,\n//! Err(e) => {\n//! eprintln!(\"failed to read from socket; err = {:?}\", e);\n//! return;\n//! }\n//! };\n//!\n//! // Write the data back\n//! if let Err(e) = socket.write_all(&buf[0..n]).await {\n//! eprintln!(\"failed to write to socket; err = {:?}\", e);\n//! return;\n//! }\n//! }\n//! });\n//! }\n//! }\n//! # }\n//! ```\n//!\n//! # Feature flags\n//!\n//! Tokio uses a set of [feature flags] to reduce the amount of compiled code. It\n//! is possible to just enable certain features over others. By default, Tokio\n//! does not enable any features but allows one to enable a subset for their use\n//! case. Below is a list of the available feature flags. You may also notice\n//! above each function, struct and trait there is listed one or more feature flags\n//! that are required for that item to be used. If you are new to Tokio it is\n//! recommended that you use the `full` feature flag which will enable all public APIs.\n//! Beware though that this will pull in many extra dependencies that you may not\n//! need.\n//!\n//! - `full`: Enables all features listed below except `test-util` and unstable features.\n//! - `rt`: Enables `tokio::spawn`, the current-thread scheduler,\n//! and non-scheduler utilities.\n//! - `rt-multi-thread`: Enables the heavier, multi-threaded, work-stealing scheduler.\n//! - `io-util`: Enables the IO based `Ext` traits.\n//! - `io-std`: Enable `Stdout`, `Stdin` and `Stderr` types.\n//! - `net`: Enables `tokio::net` types such as `TcpStream`, `UnixStream` and\n//! `UdpSocket`, as well as (on Unix-like systems) `AsyncFd` and (on\n//! FreeBSD) `PollAio`.\n//! - `time`: Enables `tokio::time` types and allows the schedulers to enable\n//! the built-in timer.\n//! - `process`: Enables `tokio::process` types.\n//! - `macros`: Enables `#[tokio::main]` and `#[tokio::test]` macros.\n//! - `sync`: Enables all `tokio::sync` types.\n//! - `signal`: Enables all `tokio::signal` types.\n//! - `fs`: Enables `tokio::fs` types.\n//! - `test-util`: Enables testing based infrastructure for the Tokio runtime.\n//! - `parking_lot`: As a potential optimization, use the [`parking_lot`] crate's\n//! synchronization primitives internally. Also, this\n//! dependency is necessary to construct some of our primitives\n//! in a `const` context. `MSRV` may increase according to the\n//! [`parking_lot`] release in use.\n//!\n//! _Note: `AsyncRead` and `AsyncWrite` traits do not require any features and are\n//! always available._\n//!\n//! ## Unstable features\n//!\n//! Some feature flags are only available when specifying the `tokio_unstable` flag:\n//!\n//! - `tracing`: Enables tracing events.\n//! - `io-uring`: Enables `io-uring` (Linux only).\n//! - `taskdump`: Enables `taskdump` (Linux only).\n//!\n//! Likewise, this flag enables access to unstable APIs.\n//!\n//! This flag enables **unstable** features. The public API of these features\n//! may break in 1.x releases. To enable these features, the `--cfg\n//! tokio_unstable` argument must be passed to `rustc` when compiling. This\n//! serves to explicitly opt-in to features which may break semver conventions,\n//! since Cargo [does not yet directly support such opt-ins][unstable features].\n//!\n//! You can specify it in your project's `.cargo/config.toml` file:\n//!\n//! ```toml\n//! [build]\n//! rustflags = [\"--cfg\", \"tokio_unstable\"]\n//! ```\n//!\n//!

\n//! The [build] section does not go in a\n//! Cargo.toml file. Instead it must be placed in the Cargo config\n//! file .cargo/config.toml.\n//!
\n//!\n//! Alternatively, you can specify it with an environment variable:\n//!\n//! ```sh\n//! ## Many *nix shells:\n//! export RUSTFLAGS=\"--cfg tokio_unstable\"\n//! cargo build\n//! ```\n//!\n//! ```powershell\n//! ## Windows PowerShell:\n//! $Env:RUSTFLAGS=\"--cfg tokio_unstable\"\n//! cargo build\n//! ```\n//!\n//! [unstable features]: https://internals.rust-lang.org/t/feature-request-unstable-opt-in-non-transitive-crate-features/16193#why-not-a-crate-feature-2\n//! [feature flags]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-features-section\n//!\n//! # Supported platforms\n//!\n//! Tokio currently guarantees support for the following platforms:\n//!\n//! * Linux\n//! * Windows\n//! * Android (API level 21)\n//! * macOS\n//! * iOS\n//! * FreeBSD\n//!\n//! Tokio will continue to support these platforms in the future. However,\n//! future releases may change requirements such as the minimum required libc\n//! version on Linux, the API level on Android, or the supported FreeBSD\n//! release.\n//!\n//! Beyond the above platforms, Tokio is intended to work on all platforms\n//! supported by the mio crate. You can find a longer list [in mio's\n//! documentation][mio-supported]. However, these additional platforms may\n//! become unsupported in the future.\n//!\n//! Note that Wine is considered to be a different platform from Windows. See\n//! mio's documentation for more information on Wine support.\n//!\n//! [mio-supported]: https://crates.io/crates/mio#platforms\n//!\n//! ## `WASM` support\n//!\n//! Tokio has some limited support for the `WASM` platform. Without the\n//! `tokio_unstable` flag, the following features are supported:\n//!\n//! * `sync`\n//! * `macros`\n//! * `io-util`\n//! * `rt`\n//! * `time`\n//!\n//! Enabling any other feature (including `full`) will cause a compilation\n//! failure.\n//!\n//! The `time` module will only work on `WASM` platforms that have support for\n//! timers (e.g. wasm32-wasi). The timing functions will panic if used on a `WASM`\n//! platform that does not support timers.\n//!\n//! Note also that if the runtime becomes indefinitely idle, it will panic\n//! immediately instead of blocking forever. On platforms that don't support\n//! time, this means that the runtime can never be idle in any way.\n//!\n//! ## Unstable `WASM` support\n//!\n//! Tokio also has unstable support for some additional `WASM` features. This\n//! requires the use of the `tokio_unstable` flag.\n//!\n//! Using this flag enables the use of `tokio::net` on the wasm32-wasi target.\n//! However, not all methods are available on the networking types as `WASI`\n//! currently does not support the creation of new sockets from within `WASM`.\n//! Because of this, sockets must currently be created via the `FromRawFd`\n//! trait.\n\n// Test that pointer width is compatible. This asserts that e.g. usize is at\n// least 32 bits, which a lot of components in Tokio currently assumes.\n//\n// TODO: improve once we have MSRV access to const eval to make more flexible.\n#[cfg(not(any(target_pointer_width = \"32\", target_pointer_width = \"64\")))]\ncompile_error! {\n \"Tokio requires the platform pointer width to be at least 32 bits\"\n}\n\n#[cfg(all(\n not(tokio_unstable),\n target_family = \"wasm\",\n any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt-multi-thread\",\n feature = \"signal\"\n )\n))]\ncompile_error!(\"Only features sync,macros,io-util,rt,time are supported on wasm.\");\n\n#[cfg(all(not(tokio_unstable), feature = \"io-uring\"))]\ncompile_error!(\"The `io-uring` feature requires `--cfg tokio_unstable`.\");\n\n#[cfg(all(not(tokio_unstable), feature = \"taskdump\"))]\ncompile_error!(\"The `taskdump` feature requires `--cfg tokio_unstable`.\");\n\n#[cfg(all(\n feature = \"taskdump\",\n not(doc),\n not(all(\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))\n))]\ncompile_error!(\n \"The `taskdump` feature is only currently supported on \\\nlinux, on `aarch64`, `x86` and `x86_64`.\"\n);\n\n// Includes re-exports used by macros.\n//\n// This module is not intended to be part of the public API. In general, any\n// `doc(hidden)` code is not part of Tokio's public and stable API.\n#[macro_use]\n#[doc(hidden)]\npub mod macros;\n\ncfg_fs! {\n pub mod fs;\n}\n\nmod future;\n\npub mod io;\npub mod net;\n\nmod loom;\n\ncfg_process! {\n pub mod process;\n}\n\n#[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n all(windows, feature = \"process\"),\n))]\nmod blocking;\n\ncfg_rt! {\n pub mod runtime;\n}\ncfg_not_rt! {\n pub(crate) mod runtime;\n}\n\ncfg_signal! {\n pub mod signal;\n}\n\ncfg_signal_internal! {\n #[cfg(not(feature = \"signal\"))]\n #[allow(dead_code)]\n #[allow(unreachable_pub)]\n pub(crate) mod signal;\n}\n\ncfg_sync! {\n pub mod sync;\n}\ncfg_not_sync! {\n mod sync;\n}\n\n// Currently, task module does not expose any public API outside `rt`\n// feature, so we mark it in the docs. This happens only to docs to\n// avoid introducing breaking changes by restricting the visibility\n// of the task module.\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub mod task;\ncfg_rt! {\n pub use task::spawn;\n}\n\ncfg_time! {\n pub mod time;\n}\n\nmod trace {\n cfg_taskdump! {\n pub(crate) use crate::runtime::task::trace::trace_leaf;\n }\n\n cfg_not_taskdump! {\n #[inline(always)]\n #[allow(dead_code)]\n pub(crate) fn trace_leaf(_: &mut std::task::Context<'_>) -> std::task::Poll<()> {\n std::task::Poll::Ready(())\n }\n }\n\n #[cfg_attr(not(feature = \"sync\"), allow(dead_code))]\n pub(crate) async fn async_trace_leaf() {\n std::future::poll_fn(trace_leaf).await\n }\n}\n\nmod util;\n\n/// Due to the `Stream` trait's inclusion in `std` landing later than Tokio's 1.0\n/// release, most of the Tokio stream utilities have been moved into the [`tokio-stream`]\n/// crate.\n///\n/// # Why was `Stream` not included in Tokio 1.0?\n///\n/// Originally, we had planned to ship Tokio 1.0 with a stable `Stream` type\n/// but unfortunately the [RFC] had not been merged in time for `Stream` to\n/// reach `std` on a stable compiler in time for the 1.0 release of Tokio. For\n/// this reason, the team has decided to move all `Stream` based utilities to\n/// the [`tokio-stream`] crate. While this is not ideal, once `Stream` has made\n/// it into the standard library and the `MSRV` period has passed, we will implement\n/// stream for our different types.\n///\n/// While this may seem unfortunate, not all is lost as you can get much of the\n/// `Stream` support with `async/await` and `while let` loops. It is also possible\n/// to create a `impl Stream` from `async fn` using the [`async-stream`] crate.\n///\n/// [`tokio-stream`]: https://docs.rs/tokio-stream\n/// [`async-stream`]: https://docs.rs/async-stream\n/// [RFC]: https://github.com/rust-lang/rfcs/pull/2996\n///\n/// # Example\n///\n/// Convert a [`sync::mpsc::Receiver`] to an `impl Stream`.\n///\n/// ```rust,no_run\n/// use tokio::sync::mpsc;\n///\n/// let (tx, mut rx) = mpsc::channel::(16);\n///\n/// let stream = async_stream::stream! {\n/// while let Some(item) = rx.recv().await {\n/// yield item;\n/// }\n/// };\n/// ```\npub mod stream {}\n\n// local re-exports of platform specific things, allowing for decent\n// documentation to be shimmed in on docs.rs\n\n#[cfg(all(docsrs, unix))]\npub mod doc;\n\n#[cfg(any(feature = \"net\", feature = \"fs\"))]\n#[cfg(all(docsrs, unix))]\n#[allow(unused)]\npub(crate) use self::doc::os;\n\n#[cfg(not(all(docsrs, unix)))]\n#[allow(unused)]\npub(crate) use std::os;\n\ncfg_macros! {\n /// Implementation detail of the `select!` macro. This macro is **not**\n /// intended to be used as part of the public API and is permitted to\n /// change.\n #[doc(hidden)]\n pub use tokio_macros::select_priv_declare_output_enum;\n\n /// Implementation detail of the `select!` macro. This macro is **not**\n /// intended to be used as part of the public API and is permitted to\n /// change.\n #[doc(hidden)]\n pub use tokio_macros::select_priv_clean_pattern;\n\n cfg_rt! {\n #[cfg(feature = \"rt-multi-thread\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n #[doc(inline)]\n pub use tokio_macros::main;\n\n #[cfg(feature = \"rt-multi-thread\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n #[doc(inline)]\n pub use tokio_macros::test;\n\n cfg_not_rt_multi_thread! {\n #[doc(inline)]\n pub use tokio_macros::main_rt as main;\n\n #[doc(inline)]\n pub use tokio_macros::test_rt as test;\n }\n }\n\n // Always fail if rt is not enabled.\n cfg_not_rt! {\n #[doc(inline)]\n pub use tokio_macros::main_fail as main;\n\n #[doc(inline)]\n pub use tokio_macros::test_fail as test;\n }\n}\n\n// TODO: rm\n#[cfg(feature = \"io-util\")]\n#[cfg(test)]\nfn is_unpin() {}\n\n/// fuzz test (`fuzz_linked_list`)\n#[cfg(fuzzing)]\npub mod fuzz;\n" }, { "path": "tokio/src/loom/mocked.rs", "content": "pub(crate) use loom::*;\n\npub(crate) mod sync {\n\n pub(crate) use loom::sync::{MutexGuard, RwLockReadGuard, RwLockWriteGuard};\n\n #[derive(Debug)]\n pub(crate) struct Mutex(loom::sync::Mutex);\n\n #[allow(dead_code)]\n impl Mutex {\n #[inline]\n pub(crate) fn new(t: T) -> Mutex {\n Mutex(loom::sync::Mutex::new(t))\n }\n\n #[inline]\n #[track_caller]\n pub(crate) fn lock(&self) -> MutexGuard<'_, T> {\n self.0.lock().unwrap()\n }\n\n #[inline]\n pub(crate) fn try_lock(&self) -> Option> {\n self.0.try_lock().ok()\n }\n }\n\n #[derive(Debug)]\n pub(crate) struct RwLock(loom::sync::RwLock);\n\n #[allow(dead_code)]\n impl RwLock {\n #[inline]\n pub(crate) fn new(t: T) -> Self {\n Self(loom::sync::RwLock::new(t))\n }\n\n #[inline]\n pub(crate) fn read(&self) -> RwLockReadGuard<'_, T> {\n self.0.read().unwrap()\n }\n\n #[inline]\n pub(crate) fn try_read(&self) -> Option> {\n self.0.try_read().ok()\n }\n\n #[inline]\n pub(crate) fn write(&self) -> RwLockWriteGuard<'_, T> {\n self.0.write().unwrap()\n }\n\n #[inline]\n pub(crate) fn try_write(&self) -> Option> {\n self.0.try_write().ok()\n }\n }\n\n pub(crate) use loom::sync::*;\n}\n\npub(crate) mod rand {\n pub(crate) fn seed() -> u64 {\n 1\n }\n}\n\npub(crate) mod sys {\n pub(crate) fn num_cpus() -> usize {\n 2\n }\n}\n\npub(crate) mod thread {\n pub use loom::lazy_static::AccessError;\n pub use loom::thread::*;\n}\n" }, { "path": "tokio/src/loom/mod.rs", "content": "//! This module abstracts over `loom` and `std::sync` depending on whether we\n//! are running tests or not.\n\n#![allow(unused)]\n\n#[cfg(not(all(test, loom)))]\nmod std;\n#[cfg(not(all(test, loom)))]\npub(crate) use self::std::*;\n\n#[cfg(all(test, loom))]\nmod mocked;\n#[cfg(all(test, loom))]\npub(crate) use self::mocked::*;\n" }, { "path": "tokio/src/loom/std/atomic_u16.rs", "content": "use std::cell::UnsafeCell;\nuse std::fmt;\nuse std::ops::Deref;\nuse std::panic;\n\n/// `AtomicU16` providing an additional `unsync_load` function.\npub(crate) struct AtomicU16 {\n inner: UnsafeCell,\n}\n\nunsafe impl Send for AtomicU16 {}\nunsafe impl Sync for AtomicU16 {}\nimpl panic::RefUnwindSafe for AtomicU16 {}\nimpl panic::UnwindSafe for AtomicU16 {}\n\nimpl AtomicU16 {\n pub(crate) const fn new(val: u16) -> AtomicU16 {\n let inner = UnsafeCell::new(std::sync::atomic::AtomicU16::new(val));\n AtomicU16 { inner }\n }\n\n /// Performs an unsynchronized load.\n ///\n /// # Safety\n ///\n /// All mutations must have happened before the unsynchronized load.\n /// Additionally, there must be no concurrent mutations.\n pub(crate) unsafe fn unsync_load(&self) -> u16 {\n unsafe { core::ptr::read(self.inner.get() as *const u16) }\n }\n}\n\nimpl Deref for AtomicU16 {\n type Target = std::sync::atomic::AtomicU16;\n\n fn deref(&self) -> &Self::Target {\n // safety: it is always safe to access `&self` fns on the inner value as\n // we never perform unsafe mutations.\n unsafe { &*self.inner.get() }\n }\n}\n\nimpl fmt::Debug for AtomicU16 {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.deref().fmt(fmt)\n }\n}\n" }, { "path": "tokio/src/loom/std/atomic_u32.rs", "content": "use std::cell::UnsafeCell;\nuse std::fmt;\nuse std::ops::Deref;\nuse std::panic;\n\n/// `AtomicU32` providing an additional `unsync_load` function.\npub(crate) struct AtomicU32 {\n inner: UnsafeCell,\n}\n\nunsafe impl Send for AtomicU32 {}\nunsafe impl Sync for AtomicU32 {}\nimpl panic::RefUnwindSafe for AtomicU32 {}\nimpl panic::UnwindSafe for AtomicU32 {}\n\nimpl AtomicU32 {\n pub(crate) const fn new(val: u32) -> AtomicU32 {\n let inner = UnsafeCell::new(std::sync::atomic::AtomicU32::new(val));\n AtomicU32 { inner }\n }\n\n /// Performs an unsynchronized load.\n ///\n /// # Safety\n ///\n /// All mutations must have happened before the unsynchronized load.\n /// Additionally, there must be no concurrent mutations.\n pub(crate) unsafe fn unsync_load(&self) -> u32 {\n unsafe { core::ptr::read(self.inner.get() as *const u32) }\n }\n}\n\nimpl Deref for AtomicU32 {\n type Target = std::sync::atomic::AtomicU32;\n\n fn deref(&self) -> &Self::Target {\n // safety: it is always safe to access `&self` fns on the inner value as\n // we never perform unsafe mutations.\n unsafe { &*self.inner.get() }\n }\n}\n\nimpl fmt::Debug for AtomicU32 {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.deref().fmt(fmt)\n }\n}\n" }, { "path": "tokio/src/loom/std/atomic_u64.rs", "content": "//! Implementation of an atomic `u64` cell. On 64 bit platforms, this is a\n//! re-export of `AtomicU64`. On 32 bit platforms, this is implemented using a\n//! `Mutex`.\n\n// `AtomicU64` can only be used on targets with `target_has_atomic` is 64 or greater.\n// Once `cfg_target_has_atomic` feature is stable, we can replace it with\n// `#[cfg(target_has_atomic = \"64\")]`.\n// Refs: https://github.com/rust-lang/rust/tree/master/src/librustc_target\ncfg_has_atomic_u64! {\n #[path = \"atomic_u64_native.rs\"]\n mod imp;\n}\n\ncfg_not_has_atomic_u64! {\n #[path = \"atomic_u64_as_mutex.rs\"]\n mod imp;\n}\n\npub(crate) use imp::AtomicU64;\n" }, { "path": "tokio/src/loom/std/atomic_u64_as_mutex.rs", "content": "use crate::loom::sync::Mutex;\nuse std::sync::atomic::Ordering;\n\ncfg_has_const_mutex_new! {\n #[path = \"atomic_u64_static_const_new.rs\"]\n mod static_macro;\n}\n\ncfg_not_has_const_mutex_new! {\n #[path = \"atomic_u64_static_once_cell.rs\"]\n mod static_macro;\n}\n\npub(crate) use static_macro::StaticAtomicU64;\n\n#[derive(Debug)]\npub(crate) struct AtomicU64 {\n inner: Mutex,\n}\n\nimpl AtomicU64 {\n pub(crate) fn load(&self, _: Ordering) -> u64 {\n *self.inner.lock()\n }\n\n pub(crate) fn store(&self, val: u64, _: Ordering) {\n *self.inner.lock() = val;\n }\n\n pub(crate) fn fetch_add(&self, val: u64, _: Ordering) -> u64 {\n let mut lock = self.inner.lock();\n let prev = *lock;\n *lock = prev + val;\n prev\n }\n\n pub(crate) fn fetch_or(&self, val: u64, _: Ordering) -> u64 {\n let mut lock = self.inner.lock();\n let prev = *lock;\n *lock = prev | val;\n prev\n }\n\n pub(crate) fn compare_exchange(\n &self,\n current: u64,\n new: u64,\n _success: Ordering,\n _failure: Ordering,\n ) -> Result {\n let mut lock = self.inner.lock();\n\n if *lock == current {\n *lock = new;\n Ok(current)\n } else {\n Err(*lock)\n }\n }\n\n pub(crate) fn compare_exchange_weak(\n &self,\n current: u64,\n new: u64,\n success: Ordering,\n failure: Ordering,\n ) -> Result {\n self.compare_exchange(current, new, success, failure)\n }\n}\n\nimpl Default for AtomicU64 {\n fn default() -> AtomicU64 {\n AtomicU64::new(u64::default())\n }\n}\n" }, { "path": "tokio/src/loom/std/atomic_u64_native.rs", "content": "pub(crate) use std::sync::atomic::{AtomicU64, Ordering};\n\n/// Alias `AtomicU64` to `StaticAtomicU64`\npub(crate) type StaticAtomicU64 = AtomicU64;\n" }, { "path": "tokio/src/loom/std/atomic_u64_static_const_new.rs", "content": "use super::AtomicU64;\nuse crate::loom::sync::Mutex;\n\npub(crate) type StaticAtomicU64 = AtomicU64;\n\nimpl AtomicU64 {\n pub(crate) const fn new(val: u64) -> Self {\n Self {\n inner: Mutex::const_new(val),\n }\n }\n}\n" }, { "path": "tokio/src/loom/std/atomic_u64_static_once_cell.rs", "content": "use super::AtomicU64;\nuse crate::loom::sync::{atomic::Ordering, Mutex};\nuse std::sync::OnceLock;\n\npub(crate) struct StaticAtomicU64 {\n init: u64,\n cell: OnceLock>,\n}\n\nimpl AtomicU64 {\n pub(crate) fn new(val: u64) -> Self {\n Self {\n inner: Mutex::new(val),\n }\n }\n}\n\nimpl StaticAtomicU64 {\n pub(crate) const fn new(val: u64) -> StaticAtomicU64 {\n StaticAtomicU64 {\n init: val,\n cell: OnceLock::new(),\n }\n }\n\n pub(crate) fn load(&self, order: Ordering) -> u64 {\n *self.inner().lock()\n }\n\n pub(crate) fn fetch_add(&self, val: u64, order: Ordering) -> u64 {\n let mut lock = self.inner().lock();\n let prev = *lock;\n *lock = prev + val;\n prev\n }\n\n pub(crate) fn compare_exchange_weak(\n &self,\n current: u64,\n new: u64,\n _success: Ordering,\n _failure: Ordering,\n ) -> Result {\n let mut lock = self.inner().lock();\n\n if *lock == current {\n *lock = new;\n Ok(current)\n } else {\n Err(*lock)\n }\n }\n\n fn inner(&self) -> &Mutex {\n self.cell.get_or_init(|| Mutex::new(self.init))\n }\n}\n" }, { "path": "tokio/src/loom/std/atomic_usize.rs", "content": "use std::cell::UnsafeCell;\nuse std::fmt;\nuse std::ops;\nuse std::panic;\n\n/// `AtomicUsize` providing an additional `unsync_load` function.\npub(crate) struct AtomicUsize {\n inner: UnsafeCell,\n}\n\nunsafe impl Send for AtomicUsize {}\nunsafe impl Sync for AtomicUsize {}\nimpl panic::RefUnwindSafe for AtomicUsize {}\nimpl panic::UnwindSafe for AtomicUsize {}\n\nimpl AtomicUsize {\n pub(crate) const fn new(val: usize) -> AtomicUsize {\n let inner = UnsafeCell::new(std::sync::atomic::AtomicUsize::new(val));\n AtomicUsize { inner }\n }\n\n /// Performs an unsynchronized load.\n ///\n /// # Safety\n ///\n /// All mutations must have happened before the unsynchronized load.\n /// Additionally, there must be no concurrent mutations.\n pub(crate) unsafe fn unsync_load(&self) -> usize {\n unsafe { core::ptr::read(self.inner.get() as *const usize) }\n }\n\n pub(crate) fn with_mut(&mut self, f: impl FnOnce(&mut usize) -> R) -> R {\n // safety: we have mutable access\n f(unsafe { (*self.inner.get()).get_mut() })\n }\n}\n\nimpl ops::Deref for AtomicUsize {\n type Target = std::sync::atomic::AtomicUsize;\n\n fn deref(&self) -> &Self::Target {\n // safety: it is always safe to access `&self` fns on the inner value as\n // we never perform unsafe mutations.\n unsafe { &*self.inner.get() }\n }\n}\n\nimpl ops::DerefMut for AtomicUsize {\n fn deref_mut(&mut self) -> &mut Self::Target {\n // safety: we hold `&mut self`\n unsafe { &mut *self.inner.get() }\n }\n}\n\nimpl fmt::Debug for AtomicUsize {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n (**self).fmt(fmt)\n }\n}\n" }, { "path": "tokio/src/loom/std/barrier.rs", "content": "//! A `Barrier` that provides `wait_timeout`.\n//!\n//! This implementation mirrors that of the Rust standard library.\n\nuse crate::loom::sync::{Condvar, Mutex};\nuse std::fmt;\nuse std::time::{Duration, Instant};\n\n/// A barrier enables multiple threads to synchronize the beginning\n/// of some computation.\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use std::sync::{Arc, Barrier};\n/// use std::thread;\n///\n/// let mut handles = Vec::with_capacity(10);\n/// let barrier = Arc::new(Barrier::new(10));\n/// for _ in 0..10 {\n/// let c = Arc::clone(&barrier);\n/// // The same messages will be printed together.\n/// // You will NOT see any interleaving.\n/// handles.push(thread::spawn(move|| {\n/// println!(\"before wait\");\n/// c.wait();\n/// println!(\"after wait\");\n/// }));\n/// }\n/// // Wait for other threads to finish.\n/// for handle in handles {\n/// handle.join().unwrap();\n/// }\n/// # }\n/// ```\npub(crate) struct Barrier {\n lock: Mutex,\n cvar: Condvar,\n num_threads: usize,\n}\n\n// The inner state of a double barrier\nstruct BarrierState {\n count: usize,\n generation_id: usize,\n}\n\n/// A `BarrierWaitResult` is returned by [`Barrier::wait()`] when all threads\n/// in the [`Barrier`] have rendezvoused.\n///\n/// # Examples\n///\n/// ```\n/// use std::sync::Barrier;\n///\n/// let barrier = Barrier::new(1);\n/// let barrier_wait_result = barrier.wait();\n/// ```\npub(crate) struct BarrierWaitResult(bool);\n\nimpl fmt::Debug for Barrier {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Barrier\").finish_non_exhaustive()\n }\n}\n\nimpl Barrier {\n /// Creates a new barrier that can block a given number of threads.\n ///\n /// A barrier will block `n`-1 threads which call [`wait()`] and then wake\n /// up all threads at once when the `n`th thread calls [`wait()`].\n ///\n /// [`wait()`]: Barrier::wait\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Barrier;\n ///\n /// let barrier = Barrier::new(10);\n /// ```\n #[must_use]\n pub(crate) fn new(n: usize) -> Barrier {\n Barrier {\n lock: Mutex::new(BarrierState {\n count: 0,\n generation_id: 0,\n }),\n cvar: Condvar::new(),\n num_threads: n,\n }\n }\n\n /// Blocks the current thread until all threads have rendezvoused here.\n ///\n /// Barriers are re-usable after all threads have rendezvoused once, and can\n /// be used continuously.\n ///\n /// A single (arbitrary) thread will receive a [`BarrierWaitResult`] that\n /// returns `true` from [`BarrierWaitResult::is_leader()`] when returning\n /// from this function, and all other threads will receive a result that\n /// will return `false` from [`BarrierWaitResult::is_leader()`].\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::sync::{Arc, Barrier};\n /// use std::thread;\n ///\n /// let mut handles = Vec::with_capacity(10);\n /// let barrier = Arc::new(Barrier::new(10));\n /// for _ in 0..10 {\n /// let c = Arc::clone(&barrier);\n /// // The same messages will be printed together.\n /// // You will NOT see any interleaving.\n /// handles.push(thread::spawn(move|| {\n /// println!(\"before wait\");\n /// c.wait();\n /// println!(\"after wait\");\n /// }));\n /// }\n /// // Wait for other threads to finish.\n /// for handle in handles {\n /// handle.join().unwrap();\n /// }\n /// # }\n /// ```\n pub(crate) fn wait(&self) -> BarrierWaitResult {\n let mut lock = self.lock.lock();\n let local_gen = lock.generation_id;\n lock.count += 1;\n if lock.count < self.num_threads {\n // We need a while loop to guard against spurious wakeups.\n // https://en.wikipedia.org/wiki/Spurious_wakeup\n while local_gen == lock.generation_id {\n lock = self.cvar.wait(lock).unwrap();\n }\n BarrierWaitResult(false)\n } else {\n lock.count = 0;\n lock.generation_id = lock.generation_id.wrapping_add(1);\n self.cvar.notify_all();\n BarrierWaitResult(true)\n }\n }\n\n /// Blocks the current thread until all threads have rendezvoused here for\n /// at most `timeout` duration.\n pub(crate) fn wait_timeout(&self, timeout: Duration) -> Option {\n // This implementation mirrors `wait`, but with each blocking operation\n // replaced by a timeout-amenable alternative.\n\n let deadline = Instant::now() + timeout;\n\n // Acquire `self.lock` with at most `timeout` duration.\n let mut lock = loop {\n if let Some(guard) = self.lock.try_lock() {\n break guard;\n } else if Instant::now() > deadline {\n return None;\n } else {\n std::thread::yield_now();\n }\n };\n\n // Shrink the `timeout` to account for the time taken to acquire `lock`.\n let timeout = deadline.saturating_duration_since(Instant::now());\n\n let local_gen = lock.generation_id;\n lock.count += 1;\n if lock.count < self.num_threads {\n // We need a while loop to guard against spurious wakeups.\n // https://en.wikipedia.org/wiki/Spurious_wakeup\n while local_gen == lock.generation_id {\n let (guard, timeout_result) = self.cvar.wait_timeout(lock, timeout).unwrap();\n lock = guard;\n if timeout_result.timed_out() {\n return None;\n }\n }\n Some(BarrierWaitResult(false))\n } else {\n lock.count = 0;\n lock.generation_id = lock.generation_id.wrapping_add(1);\n self.cvar.notify_all();\n Some(BarrierWaitResult(true))\n }\n }\n}\n\nimpl fmt::Debug for BarrierWaitResult {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"BarrierWaitResult\")\n .field(\"is_leader\", &self.is_leader())\n .finish()\n }\n}\n\nimpl BarrierWaitResult {\n /// Returns `true` if this thread is the \"leader thread\" for the call to\n /// [`Barrier::wait()`].\n ///\n /// Only one thread will have `true` returned from their result, all other\n /// threads will have `false` returned.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Barrier;\n ///\n /// let barrier = Barrier::new(1);\n /// let barrier_wait_result = barrier.wait();\n /// println!(\"{:?}\", barrier_wait_result.is_leader());\n /// ```\n #[must_use]\n pub(crate) fn is_leader(&self) -> bool {\n self.0\n }\n}\n" }, { "path": "tokio/src/loom/std/mod.rs", "content": "#![cfg_attr(any(not(feature = \"full\"), loom), allow(unused_imports, dead_code))]\n\nmod atomic_u16;\nmod atomic_u32;\nmod atomic_u64;\nmod atomic_usize;\nmod barrier;\nmod mutex;\n#[cfg(all(feature = \"parking_lot\", not(miri)))]\nmod parking_lot;\nmod rwlock;\nmod unsafe_cell;\n\npub(crate) mod cell {\n pub(crate) use super::unsafe_cell::UnsafeCell;\n}\n\n#[cfg(any(\n feature = \"net\",\n feature = \"process\",\n feature = \"signal\",\n feature = \"sync\",\n))]\npub(crate) mod future {\n pub(crate) use crate::sync::AtomicWaker;\n}\n\npub(crate) mod hint {\n pub(crate) use std::hint::spin_loop;\n}\n\npub(crate) mod rand {\n use std::collections::hash_map::RandomState;\n use std::hash::{BuildHasher, Hash, Hasher};\n use std::sync::atomic::AtomicU32;\n use std::sync::atomic::Ordering::Relaxed;\n\n static COUNTER: AtomicU32 = AtomicU32::new(1);\n\n pub(crate) fn seed() -> u64 {\n let rand_state = RandomState::new();\n // Hash some unique-ish data to generate some new state\n rand_state.hash_one(COUNTER.fetch_add(1, Relaxed))\n }\n}\n\npub(crate) mod sync {\n pub(crate) use std::sync::{Arc, Weak};\n\n // Below, make sure all the feature-influenced types are exported for\n // internal use. Note however that some are not _currently_ named by\n // consuming code.\n\n // Not using parking_lot in Miri due to .\n #[cfg(all(feature = \"parking_lot\", not(miri)))]\n #[allow(unused_imports)]\n pub(crate) use crate::loom::std::parking_lot::{\n Condvar, Mutex, MutexGuard, RwLock, RwLockReadGuard, WaitTimeoutResult,\n };\n\n #[cfg(not(all(feature = \"parking_lot\", not(miri))))]\n #[allow(unused_imports)]\n pub(crate) use std::sync::{Condvar, MutexGuard, RwLockReadGuard, WaitTimeoutResult};\n\n #[cfg(not(all(feature = \"parking_lot\", not(miri))))]\n pub(crate) use crate::loom::std::mutex::Mutex;\n\n #[cfg(not(all(feature = \"parking_lot\", not(miri))))]\n pub(crate) use crate::loom::std::rwlock::RwLock;\n\n pub(crate) mod atomic {\n pub(crate) use crate::loom::std::atomic_u16::AtomicU16;\n pub(crate) use crate::loom::std::atomic_u32::AtomicU32;\n pub(crate) use crate::loom::std::atomic_u64::AtomicU64;\n pub(crate) use crate::loom::std::atomic_usize::AtomicUsize;\n\n pub(crate) use std::sync::atomic::{fence, AtomicBool, AtomicPtr, AtomicU8, Ordering};\n }\n\n pub(crate) use super::barrier::Barrier;\n}\n\npub(crate) mod sys {\n #[cfg(feature = \"rt-multi-thread\")]\n pub(crate) fn num_cpus() -> usize {\n use std::num::NonZeroUsize;\n\n const ENV_WORKER_THREADS: &str = \"TOKIO_WORKER_THREADS\";\n\n match std::env::var(ENV_WORKER_THREADS) {\n Ok(s) => {\n let n = s.parse().unwrap_or_else(|e| {\n panic!(\"\\\"{ENV_WORKER_THREADS}\\\" must be usize, error: {e}, value: {s}\")\n });\n assert!(n > 0, \"\\\"{ENV_WORKER_THREADS}\\\" cannot be set to 0\");\n n\n }\n Err(std::env::VarError::NotPresent) => {\n std::thread::available_parallelism().map_or(1, NonZeroUsize::get)\n }\n Err(std::env::VarError::NotUnicode(e)) => {\n panic!(\"\\\"{ENV_WORKER_THREADS}\\\" must be valid unicode, error: {e:?}\")\n }\n }\n }\n\n #[cfg(not(feature = \"rt-multi-thread\"))]\n pub(crate) fn num_cpus() -> usize {\n 1\n }\n}\n\npub(crate) mod thread {\n #[inline]\n pub(crate) fn yield_now() {\n std::hint::spin_loop();\n }\n\n #[allow(unused_imports)]\n pub(crate) use std::thread::{\n current, panicking, park, park_timeout, sleep, spawn, AccessError, Builder, JoinHandle,\n LocalKey, Result, Thread, ThreadId,\n };\n}\n" }, { "path": "tokio/src/loom/std/mutex.rs", "content": "use std::sync::{self, MutexGuard, TryLockError};\n\n/// Adapter for `std::Mutex` that removes the poisoning aspects\n/// from its API.\n#[derive(Debug)]\npub(crate) struct Mutex(sync::Mutex);\n\n#[allow(dead_code)]\nimpl Mutex {\n #[inline]\n pub(crate) fn new(t: T) -> Mutex {\n Mutex(sync::Mutex::new(t))\n }\n\n #[inline]\n pub(crate) const fn const_new(t: T) -> Mutex {\n Mutex(sync::Mutex::new(t))\n }\n\n #[inline]\n pub(crate) fn lock(&self) -> MutexGuard<'_, T> {\n match self.0.lock() {\n Ok(guard) => guard,\n Err(p_err) => p_err.into_inner(),\n }\n }\n\n #[inline]\n pub(crate) fn try_lock(&self) -> Option> {\n match self.0.try_lock() {\n Ok(guard) => Some(guard),\n Err(TryLockError::Poisoned(p_err)) => Some(p_err.into_inner()),\n Err(TryLockError::WouldBlock) => None,\n }\n }\n}\n" }, { "path": "tokio/src/loom/std/parking_lot.rs", "content": "//! A minimal adaption of the `parking_lot` synchronization primitives to the\n//! equivalent `std::sync` types.\n//!\n//! This can be extended to additional types/methods as required.\n\nuse std::fmt;\nuse std::marker::PhantomData;\nuse std::ops::{Deref, DerefMut};\nuse std::sync::{LockResult, TryLockError};\nuse std::time::Duration;\n\n// All types in this file are marked with PhantomData to ensure that\n// parking_lot's send_guard feature does not leak through and affect when Tokio\n// types are Send.\n//\n// See for more info.\n\n// Types that do not need wrapping\npub(crate) use parking_lot::WaitTimeoutResult;\n\n#[derive(Debug)]\npub(crate) struct Mutex(PhantomData>, parking_lot::Mutex);\n\n#[derive(Debug)]\npub(crate) struct RwLock(PhantomData>, parking_lot::RwLock);\n\n#[derive(Debug)]\npub(crate) struct Condvar(PhantomData, parking_lot::Condvar);\n\n#[derive(Debug)]\npub(crate) struct MutexGuard<'a, T: ?Sized>(\n PhantomData>,\n parking_lot::MutexGuard<'a, T>,\n);\n\n#[derive(Debug)]\npub(crate) struct RwLockReadGuard<'a, T: ?Sized>(\n PhantomData>,\n parking_lot::RwLockReadGuard<'a, T>,\n);\n\n#[derive(Debug)]\npub(crate) struct RwLockWriteGuard<'a, T: ?Sized>(\n PhantomData>,\n parking_lot::RwLockWriteGuard<'a, T>,\n);\n\nimpl Mutex {\n #[inline]\n pub(crate) fn new(t: T) -> Mutex {\n Mutex(PhantomData, parking_lot::Mutex::new(t))\n }\n\n #[inline]\n #[cfg(not(all(loom, test)))]\n pub(crate) const fn const_new(t: T) -> Mutex {\n Mutex(PhantomData, parking_lot::const_mutex(t))\n }\n\n #[inline]\n pub(crate) fn lock(&self) -> MutexGuard<'_, T> {\n MutexGuard(PhantomData, self.1.lock())\n }\n\n #[inline]\n pub(crate) fn try_lock(&self) -> Option> {\n self.1\n .try_lock()\n .map(|guard| MutexGuard(PhantomData, guard))\n }\n\n #[inline]\n pub(crate) fn get_mut(&mut self) -> &mut T {\n self.1.get_mut()\n }\n\n // Note: Additional methods `is_poisoned` and `into_inner`, can be\n // provided here as needed.\n}\n\nimpl<'a, T: ?Sized> Deref for MutexGuard<'a, T> {\n type Target = T;\n fn deref(&self) -> &T {\n self.1.deref()\n }\n}\n\nimpl<'a, T: ?Sized> DerefMut for MutexGuard<'a, T> {\n fn deref_mut(&mut self) -> &mut T {\n self.1.deref_mut()\n }\n}\n\nimpl RwLock {\n pub(crate) fn new(t: T) -> RwLock {\n RwLock(PhantomData, parking_lot::RwLock::new(t))\n }\n\n pub(crate) fn read(&self) -> RwLockReadGuard<'_, T> {\n RwLockReadGuard(PhantomData, self.1.read())\n }\n\n pub(crate) fn try_read(&self) -> Option> {\n self.1\n .try_read()\n .map(|guard| RwLockReadGuard(PhantomData, guard))\n }\n\n pub(crate) fn write(&self) -> RwLockWriteGuard<'_, T> {\n RwLockWriteGuard(PhantomData, self.1.write())\n }\n\n pub(crate) fn try_write(&self) -> Option> {\n self.1\n .try_write()\n .map(|guard| RwLockWriteGuard(PhantomData, guard))\n }\n}\n\nimpl<'a, T: ?Sized> Deref for RwLockReadGuard<'a, T> {\n type Target = T;\n fn deref(&self) -> &T {\n self.1.deref()\n }\n}\n\nimpl<'a, T: ?Sized> Deref for RwLockWriteGuard<'a, T> {\n type Target = T;\n fn deref(&self) -> &T {\n self.1.deref()\n }\n}\n\nimpl<'a, T: ?Sized> DerefMut for RwLockWriteGuard<'a, T> {\n fn deref_mut(&mut self) -> &mut T {\n self.1.deref_mut()\n }\n}\n\nimpl Condvar {\n #[inline]\n pub(crate) fn new() -> Condvar {\n Condvar(PhantomData, parking_lot::Condvar::new())\n }\n\n #[inline]\n pub(crate) fn notify_one(&self) {\n self.1.notify_one();\n }\n\n #[inline]\n pub(crate) fn notify_all(&self) {\n self.1.notify_all();\n }\n\n #[inline]\n pub(crate) fn wait<'a, T>(\n &self,\n mut guard: MutexGuard<'a, T>,\n ) -> LockResult> {\n self.1.wait(&mut guard.1);\n Ok(guard)\n }\n\n #[inline]\n pub(crate) fn wait_timeout<'a, T>(\n &self,\n mut guard: MutexGuard<'a, T>,\n timeout: Duration,\n ) -> LockResult<(MutexGuard<'a, T>, WaitTimeoutResult)> {\n let wtr = self.1.wait_for(&mut guard.1, timeout);\n Ok((guard, wtr))\n }\n\n // Note: Additional methods `wait_timeout_ms`, `wait_timeout_until`,\n // `wait_until` can be provided here as needed.\n}\n\nimpl<'a, T: ?Sized + fmt::Display> fmt::Display for MutexGuard<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&self.1, f)\n }\n}\n\nimpl<'a, T: ?Sized + fmt::Display> fmt::Display for RwLockReadGuard<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&self.1, f)\n }\n}\n\nimpl<'a, T: ?Sized + fmt::Display> fmt::Display for RwLockWriteGuard<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&self.1, f)\n }\n}\n" }, { "path": "tokio/src/loom/std/rwlock.rs", "content": "use std::sync::{self, RwLockReadGuard, RwLockWriteGuard, TryLockError};\n\n/// Adapter for `std::sync::RwLock` that removes the poisoning aspects\n/// from its api.\n#[derive(Debug)]\npub(crate) struct RwLock(sync::RwLock);\n\n#[allow(dead_code)]\nimpl RwLock {\n #[inline]\n pub(crate) fn new(t: T) -> Self {\n Self(sync::RwLock::new(t))\n }\n\n #[inline]\n pub(crate) fn read(&self) -> RwLockReadGuard<'_, T> {\n match self.0.read() {\n Ok(guard) => guard,\n Err(p_err) => p_err.into_inner(),\n }\n }\n\n #[inline]\n pub(crate) fn try_read(&self) -> Option> {\n match self.0.try_read() {\n Ok(guard) => Some(guard),\n Err(TryLockError::Poisoned(p_err)) => Some(p_err.into_inner()),\n Err(TryLockError::WouldBlock) => None,\n }\n }\n\n #[inline]\n pub(crate) fn write(&self) -> RwLockWriteGuard<'_, T> {\n match self.0.write() {\n Ok(guard) => guard,\n Err(p_err) => p_err.into_inner(),\n }\n }\n\n #[inline]\n pub(crate) fn try_write(&self) -> Option> {\n match self.0.try_write() {\n Ok(guard) => Some(guard),\n Err(TryLockError::Poisoned(p_err)) => Some(p_err.into_inner()),\n Err(TryLockError::WouldBlock) => None,\n }\n }\n}\n" }, { "path": "tokio/src/loom/std/unsafe_cell.rs", "content": "#[derive(Debug)]\npub(crate) struct UnsafeCell(std::cell::UnsafeCell);\n\nimpl UnsafeCell {\n pub(crate) const fn new(data: T) -> UnsafeCell {\n UnsafeCell(std::cell::UnsafeCell::new(data))\n }\n\n #[inline(always)]\n pub(crate) fn with(&self, f: impl FnOnce(*const T) -> R) -> R {\n f(self.0.get())\n }\n\n #[inline(always)]\n pub(crate) fn with_mut(&self, f: impl FnOnce(*mut T) -> R) -> R {\n f(self.0.get())\n }\n}\n" }, { "path": "tokio/src/macros/addr_of.rs", "content": "//! This module defines a macro that lets you go from a raw pointer to a struct\n//! to a raw pointer to a field of the struct.\n\nmacro_rules! generate_addr_of_methods {\n (\n impl<$($gen:ident)*> $struct_name:ty {$(\n $(#[$attrs:meta])*\n $vis:vis unsafe fn $fn_name:ident(self: NonNull) -> NonNull<$field_type:ty> {\n &self$(.$field_name:tt)+\n }\n )*}\n ) => {\n impl<$($gen)*> $struct_name {$(\n #[doc = \"# Safety\"]\n #[doc = \"\"]\n #[doc = \"The `me` pointer must be valid.\"]\n $(#[$attrs])*\n $vis unsafe fn $fn_name(me: ::core::ptr::NonNull) -> ::core::ptr::NonNull<$field_type> {\n let me = me.as_ptr();\n // safety: the caller guarantees that `me` is valid\n let field = unsafe { ::std::ptr::addr_of_mut!((*me) $(.$field_name)+ ) };\n // safety: the field pointer is never null\n unsafe { ::core::ptr::NonNull::new_unchecked(field) }\n }\n )*}\n };\n}\n" }, { "path": "tokio/src/macros/cfg.rs", "content": "#![allow(unused_macros)]\n\n/// Allows specifying arbitrary combinations of features and config flags,\n/// which are also propagated to `docsrs` config.\n///\n/// Each contained item will have the annotations applied\n///\n/// ## Example usage:\n/// ```no-compile\n/// feature! {\n/// #![any(\n/// feature = \"process\",\n/// feature = \"sync\",\n/// feature = \"rt\",\n/// tokio_unstable\n/// )]\n/// /// docs\n/// pub struct MyStruct {};\n/// /// docs\n/// pub struct AnotherStruct {};\n/// }\n/// ```\n///\nmacro_rules! feature {\n (\n #![$meta:meta]\n $($item:item)*\n ) => {\n $(\n #[cfg($meta)]\n #[cfg_attr(docsrs, doc(cfg($meta)))]\n $item\n )*\n }\n}\n\n/// Enables Windows-specific code.\n/// Use this macro instead of `cfg(windows)` to generate docs properly.\nmacro_rules! cfg_windows {\n ($($item:item)*) => {\n $(\n #[cfg(any(all(doc, docsrs), windows))]\n #[cfg_attr(docsrs, doc(cfg(windows)))]\n $item\n )*\n }\n}\n\n/// Enables Unix-specific code.\n/// Use this macro instead of `cfg(unix)` to generate docs properly.\nmacro_rules! cfg_unix {\n ($($item:item)*) => {\n $(\n #[cfg(any(all(doc, docsrs), unix))]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n $item\n )*\n }\n}\n\n/// Enables unstable Windows-specific code.\n/// Use this macro instead of `cfg(windows)` to generate docs properly.\nmacro_rules! cfg_unstable_windows {\n ($($item:item)*) => {\n $(\n #[cfg(all(any(all(doc, docsrs), windows), tokio_unstable))]\n #[cfg_attr(docsrs, doc(cfg(all(windows, tokio_unstable))))]\n $item\n )*\n }\n}\n\n/// Enables `enter::block_on`.\nmacro_rules! cfg_block_on {\n ($($item:item)*) => {\n $(\n #[cfg(any(\n feature = \"fs\",\n feature = \"net\",\n feature = \"io-std\",\n feature = \"rt\",\n ))]\n $item\n )*\n }\n}\n\n/// Enables internal `AtomicWaker` impl.\nmacro_rules! cfg_atomic_waker_impl {\n ($($item:item)*) => {\n $(\n #[cfg(any(\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"time\",\n ))]\n #[cfg(not(loom))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_aio {\n ($($item:item)*) => {\n $(\n #[cfg(all(any(docsrs, target_os = \"freebsd\"), feature = \"net\"))]\n #[cfg_attr(docsrs,\n doc(cfg(all(target_os = \"freebsd\", feature = \"net\")))\n )]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_fs {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"fs\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"fs\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_blocking {\n ($($item:item)*) => {\n $( #[cfg(any(\n feature = \"io-std\",\n feature = \"fs\",\n all(windows, feature = \"process\"),\n ))] $item )*\n }\n}\n\nmacro_rules! cfg_io_driver {\n ($($item:item)*) => {\n $(\n #[cfg(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\"),\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n )\n ))]\n #[cfg_attr(docsrs, doc(cfg(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\"),\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n )\n ))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_driver_impl {\n ( $( $item:item )* ) => {\n $(\n #[cfg(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\"),\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n )\n ))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_io_driver {\n ($($item:item)*) => {\n $(\n #[cfg(not(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\"),\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n )\n )))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_readiness {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"net\")]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_std {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"io-std\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-std\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_util {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"io-util\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_io_util {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"io-util\"))] $item )*\n }\n}\n\nmacro_rules! cfg_loom {\n ($($item:item)*) => {\n $( #[cfg(loom)] $item )*\n }\n}\n\nmacro_rules! cfg_not_loom {\n ($($item:item)*) => {\n $( #[cfg(not(loom))] $item )*\n }\n}\n\nmacro_rules! cfg_macros {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"macros\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_unstable_metrics {\n ($($item:item)*) => {\n $(\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n $item\n )*\n }\n}\n\n/// Some metrics require 64-bit atomics.\nmacro_rules! cfg_64bit_metrics {\n ($($item:item)*) => {\n $(\n #[cfg(target_has_atomic = \"64\")]\n #[cfg_attr(docsrs, doc(cfg(target_has_atomic = \"64\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_no_64bit_metrics {\n ($($item:item)*) => {\n $(\n #[cfg(not(target_has_atomic = \"64\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_unstable_metrics {\n ($($item:item)*) => {\n $(\n #[cfg(not(tokio_unstable))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_rt_and_metrics_and_net {\n ($($item:item)*) => {\n $( #[cfg(not(all(feature = \"net\", feature = \"rt\", tokio_unstable)))]$item )*\n }\n}\n\nmacro_rules! cfg_net_or_process {\n ($($item:item)*) => {\n $(\n #[cfg(any(feature = \"net\", feature = \"process\"))]\n #[cfg_attr(docsrs, doc(cfg(any(feature = \"net\", feature = \"process\"))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"net\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"net\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net_or_uring {\n ($($item:item)*) => {\n $(\n #[cfg(any(\n feature = \"net\",\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n )\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n feature = \"net\",\n all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n )\n )))\n )]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net_unix {\n ($($item:item)*) => {\n $(\n #[cfg(all(unix, feature = \"net\"))]\n #[cfg_attr(docsrs, doc(cfg(all(unix, feature = \"net\"))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net_windows {\n ($($item:item)*) => {\n $(\n #[cfg(all(any(all(doc, docsrs), windows), feature = \"net\"))]\n #[cfg_attr(docsrs, doc(cfg(all(windows, feature = \"net\"))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_process {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"process\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"process\")))]\n #[cfg(not(loom))]\n #[cfg(not(target_os = \"wasi\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_process_driver {\n ($($item:item)*) => {\n #[cfg(unix)]\n #[cfg(not(loom))]\n cfg_process! { $($item)* }\n }\n}\n\nmacro_rules! cfg_not_process_driver {\n ($($item:item)*) => {\n $(\n #[cfg(not(all(unix, not(loom), feature = \"process\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_signal {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"signal\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"signal\")))]\n #[cfg(not(loom))]\n #[cfg(not(target_os = \"wasi\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_signal_internal {\n ($($item:item)*) => {\n $(\n #[cfg(any(feature = \"signal\", all(unix, feature = \"process\")))]\n #[cfg(not(loom))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_signal_internal_and_unix {\n ($($item:item)*) => {\n #[cfg(unix)]\n cfg_signal_internal! { $($item)* }\n }\n}\n\nmacro_rules! cfg_not_signal_internal {\n ($($item:item)*) => {\n $(\n #[cfg(any(loom, not(unix), not(any(feature = \"signal\", all(unix, feature = \"process\")))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_sync {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"sync\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_sync {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"sync\"))] $item )*\n }\n}\n\nmacro_rules! cfg_rt {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_rt {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"rt\"))] $item )*\n }\n}\n\nmacro_rules! cfg_rt_multi_thread {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"rt-multi-thread\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt-multi-thread\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_rt_multi_thread {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"rt-multi-thread\"))] $item )*\n }\n}\n\nmacro_rules! cfg_taskdump {\n ($($item:item)*) => {\n $(\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(\n target_arch = \"aarch64\",\n target_arch = \"x86\",\n target_arch = \"x86_64\"\n )\n ))]\n $item\n )*\n };\n}\n\nmacro_rules! cfg_not_taskdump {\n ($($item:item)*) => {\n $(\n #[cfg(not(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(\n target_arch = \"aarch64\",\n target_arch = \"x86\",\n target_arch = \"x86_64\"\n )\n )))]\n $item\n )*\n };\n}\n\nmacro_rules! cfg_test_util {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"test-util\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"test-util\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_test_util {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"test-util\"))] $item )*\n }\n}\n\nmacro_rules! cfg_time {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_time {\n ($($item:item)*) => {\n $( #[cfg(not(feature = \"time\"))] $item )*\n }\n}\n\nmacro_rules! cfg_trace {\n ($($item:item)*) => {\n $(\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n #[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\n $item\n )*\n };\n}\n\nmacro_rules! cfg_unstable {\n ($($item:item)*) => {\n $(\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n $item\n )*\n };\n}\n\nmacro_rules! cfg_not_trace {\n ($($item:item)*) => {\n $(\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_coop {\n ($($item:item)*) => {\n $(\n #[cfg(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n ))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_coop {\n ($($item:item)*) => {\n $(\n #[cfg(not(any(\n feature = \"fs\",\n feature = \"io-std\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"sync\",\n feature = \"time\",\n )))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_has_atomic_u64 {\n ($($item:item)*) => {\n $(\n #[cfg(target_has_atomic = \"64\")]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_has_atomic_u64 {\n ($($item:item)*) => {\n $(\n #[cfg(not(target_has_atomic = \"64\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_has_const_mutex_new {\n ($($item:item)*) => {\n $(\n #[cfg(not(all(loom, test)))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_has_const_mutex_new {\n ($($item:item)*) => {\n $(\n #[cfg(all(loom, test))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_wasi {\n ($($item:item)*) => {\n $(\n #[cfg(not(target_os = \"wasi\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_is_wasm_not_wasi {\n ($($item:item)*) => {\n $(\n #[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\n $item\n )*\n }\n}\n\n/// Use this macro to provide two different implementations of the same API — one for stable\n/// builds and one for unstable builds.\nmacro_rules! cfg_metrics_variant {\n (stable: {$($stable_code:tt)*}, unstable: {$($unstable_code:tt)*}) => {\n cfg_not_unstable_metrics! {\n $($stable_code)*\n }\n\n cfg_unstable_metrics! {\n $($unstable_code)*\n }\n }\n}\n\nmacro_rules! cfg_io_uring {\n ($($item:item)*) => {\n $(\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n $item\n )*\n };\n}\n" }, { "path": "tokio/src/macros/join.rs", "content": "macro_rules! doc {\n ($join:item) => {\n /// Waits on multiple concurrent branches, returning when **all** branches\n /// complete.\n ///\n /// The `join!` macro must be used inside of async functions, closures, and\n /// blocks.\n ///\n /// The `join!` macro takes a list of async expressions and evaluates them\n /// concurrently on the same task. Each async expression evaluates to a future\n /// and the futures from each expression are multiplexed on the current task.\n ///\n /// When working with async expressions returning `Result`, `join!` will wait\n /// for **all** branches complete regardless if any complete with `Err`. Use\n /// [`try_join!`] to return early when `Err` is encountered.\n ///\n /// [`try_join!`]: crate::try_join\n ///\n /// # Notes\n ///\n /// The supplied futures are stored inline and do not require allocating a\n /// `Vec`.\n ///\n /// ## Runtime characteristics\n ///\n /// By running all async expressions on the current task, the expressions are\n /// able to run **concurrently** but not in **parallel**. This means all\n /// expressions are run on the same thread and if one branch blocks the thread,\n /// all other expressions will be unable to continue. If parallelism is\n /// required, spawn each async expression using [`tokio::spawn`] and pass the\n /// join handle to `join!`.\n ///\n /// [`tokio::spawn`]: crate::spawn\n ///\n /// ## Fairness\n ///\n /// By default, `join!`'s generated future rotates which contained\n /// future is polled first whenever it is woken.\n ///\n /// This behavior can be overridden by adding `biased;` to the beginning of the\n /// macro usage. See the examples for details. This will cause `join` to poll\n /// the futures in the order they appear from top to bottom.\n ///\n /// You may want this if your futures may interact in a way where known polling order is significant.\n ///\n /// But there is an important caveat to this mode. It becomes your responsibility\n /// to ensure that the polling order of your futures is fair. If for example you\n /// are joining a stream and a shutdown future, and the stream has a\n /// huge volume of messages that takes a long time to finish processing per poll, you should\n /// place the shutdown future earlier in the `join!` list to ensure that it is\n /// always polled, and will not be delayed due to the stream future taking a long time to return\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Basic join with two branches\n ///\n /// ```\n /// async fn do_stuff_async() {\n /// // async work\n /// }\n ///\n /// async fn more_async_work() {\n /// // more here\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (first, second) = tokio::join!(\n /// do_stuff_async(),\n /// more_async_work());\n ///\n /// // do something with the values\n /// # }\n /// ```\n ///\n /// Using the `biased;` mode to control polling order.\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// async fn do_stuff_async() {\n /// // async work\n /// }\n ///\n /// async fn more_async_work() {\n /// // more here\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (first, second) = tokio::join!(\n /// biased;\n /// do_stuff_async(),\n /// more_async_work()\n /// );\n ///\n /// // do something with the values\n /// # }\n /// # }\n /// ```\n\n #[macro_export]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n $join\n };\n}\n\n#[cfg(doc)]\ndoc! {macro_rules! join {\n ($(biased;)? $($future:expr),*) => { unimplemented!() }\n}}\n\n#[cfg(not(doc))]\ndoc! {macro_rules! join {\n (@ {\n // Type of rotator that controls which inner future to start with\n // when polling our output future.\n rotator_select=$rotator_select:ty;\n\n // One `_` for each branch in the `join!` macro. This is not used once\n // normalization is complete.\n ( $($count:tt)* )\n\n // The expression `0+1+1+ ... +1` equal to the number of branches.\n ( $($total:tt)* )\n\n // Normalized join! branches\n $( ( $($skip:tt)* ) $e:expr, )*\n\n }) => {{\n // Safety: nothing must be moved out of `futures`. This is to satisfy\n // the requirement of `Pin::new_unchecked` called below.\n //\n // We can't use the `pin!` macro for this because `futures` is a tuple\n // and the standard library provides no way to pin-project to the fields\n // of a tuple.\n let mut futures = ( $( $crate::macros::support::maybe_done($e), )* );\n\n // This assignment makes sure that the `poll_fn` closure only has a\n // reference to the futures, instead of taking ownership of them. This\n // mitigates the issue described in\n // \n let mut futures = &mut futures;\n\n // Each time the future created by poll_fn is polled, if not using biased mode,\n // a different future is polled first to ensure every future passed to join!\n // can make progress even if one of the futures consumes the whole budget.\n let mut rotator = <$rotator_select as $crate::macros::support::RotatorSelect>::Rotator::<{$($total)*}>::default();\n\n $crate::macros::support::poll_fn(move |cx| {\n const COUNT: u32 = $($total)*;\n\n let mut is_pending = false;\n let mut to_run = COUNT;\n\n // The number of futures that will be skipped in the first loop iteration.\n let mut skip = rotator.num_skip();\n\n // This loop runs twice and the first `skip` futures\n // are not polled in the first iteration.\n loop {\n $(\n if skip == 0 {\n if to_run == 0 {\n // Every future has been polled\n break;\n }\n to_run -= 1;\n\n // Extract the future for this branch from the tuple.\n let ( $($skip,)* fut, .. ) = &mut *futures;\n\n // Safety: future is stored on the stack above\n // and never moved.\n let mut fut = unsafe { $crate::macros::support::Pin::new_unchecked(fut) };\n\n // Try polling\n if $crate::macros::support::Future::poll(fut.as_mut(), cx).is_pending() {\n is_pending = true;\n }\n } else {\n // Future skipped, one less future to skip in the next iteration\n skip -= 1;\n }\n )*\n }\n\n if is_pending {\n $crate::macros::support::Poll::Pending\n } else {\n $crate::macros::support::Poll::Ready(($({\n // Extract the future for this branch from the tuple.\n let ( $($skip,)* fut, .. ) = &mut futures;\n\n // Safety: future is stored on the stack above\n // and never moved.\n let mut fut = unsafe { $crate::macros::support::Pin::new_unchecked(fut) };\n\n fut.take_output().expect(\"expected completed future\")\n },)*))\n }\n }).await\n }};\n\n // ===== Normalize =====\n\n (@ { rotator_select=$rotator_select:ty; ( $($s:tt)* ) ( $($n:tt)* ) $($t:tt)* } $e:expr, $($r:tt)* ) => {\n $crate::join!(@{ rotator_select=$rotator_select; ($($s)* _) ($($n)* + 1) $($t)* ($($s)*) $e, } $($r)*)\n };\n\n // ===== Entry point =====\n ( biased; $($e:expr),+ $(,)?) => {\n $crate::join!(@{ rotator_select=$crate::macros::support::SelectBiased; () (0) } $($e,)*)\n };\n\n ( $($e:expr),+ $(,)?) => {\n $crate::join!(@{ rotator_select=$crate::macros::support::SelectNormal; () (0) } $($e,)*)\n };\n\n (biased;) => { async {}.await };\n\n () => { async {}.await }\n}}\n\n/// Helper trait to select which type of `Rotator` to use.\n// We need this to allow specifying a const generic without\n// colliding with caller const names due to macro hygiene.\npub trait RotatorSelect {\n type Rotator: Default;\n}\n\n/// Marker type indicating that the starting branch should\n/// rotate each poll.\n#[derive(Debug)]\npub struct SelectNormal;\n/// Marker type indicating that the starting branch should\n/// be the first declared branch each poll.\n#[derive(Debug)]\npub struct SelectBiased;\n\nimpl RotatorSelect for SelectNormal {\n type Rotator = Rotator;\n}\n\nimpl RotatorSelect for SelectBiased {\n type Rotator = BiasedRotator;\n}\n\n/// Rotates by one each [`Self::num_skip`] call up to COUNT - 1.\n#[derive(Default, Debug)]\npub struct Rotator {\n next: u32,\n}\n\nimpl Rotator {\n /// Rotates by one each [`Self::num_skip`] call up to COUNT - 1\n #[inline]\n pub fn num_skip(&mut self) -> u32 {\n let num_skip = self.next;\n self.next += 1;\n if self.next == COUNT {\n self.next = 0;\n }\n num_skip\n }\n}\n\n/// [`Self::num_skip`] always returns 0.\n#[derive(Default, Debug)]\npub struct BiasedRotator {}\n\nimpl BiasedRotator {\n /// Always returns 0.\n #[inline]\n pub fn num_skip(&mut self) -> u32 {\n 0\n }\n}\n" }, { "path": "tokio/src/macros/loom.rs", "content": "macro_rules! if_loom {\n ($($t:tt)*) => {{\n #[cfg(loom)]\n {\n $($t)*\n }\n }}\n}\n" }, { "path": "tokio/src/macros/mod.rs", "content": "#![cfg_attr(not(feature = \"full\"), allow(unused_macros))]\n\n#[macro_use]\nmod cfg;\n\n#[macro_use]\nmod loom;\n\n#[macro_use]\nmod pin;\n\n#[macro_use]\nmod thread_local;\n\n#[macro_use]\nmod addr_of;\n\ncfg_trace! {\n #[macro_use]\n mod trace;\n}\n\ncfg_macros! {\n #[macro_use]\n mod select;\n\n #[macro_use]\n mod join;\n\n #[macro_use]\n mod try_join;\n}\n\n// Includes re-exports needed to implement macros\n#[doc(hidden)]\npub mod support;\n" }, { "path": "tokio/src/macros/pin.rs", "content": "/// Pins a value on the stack.\n///\n/// Calls to `async fn` return anonymous [`Future`] values that are `!Unpin`.\n/// These values must be pinned before they can be polled. Calling `.await` will\n/// handle this, but consumes the future. If it is required to call `.await` on\n/// a `&mut _` reference, the caller is responsible for pinning the future.\n///\n/// Pinning may be done by allocating with [`Box::pin`] or by using the stack\n/// with the `pin!` macro.\n///\n/// The following will **fail to compile**:\n///\n/// ```compile_fail\n/// async fn my_async_fn() {\n/// // async logic here\n/// }\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let mut future = my_async_fn();\n/// (&mut future).await;\n/// }\n/// ```\n///\n/// To make this work requires pinning:\n///\n/// ```\n/// use tokio::pin;\n///\n/// async fn my_async_fn() {\n/// // async logic here\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let future = my_async_fn();\n/// pin!(future);\n///\n/// (&mut future).await;\n/// # }\n/// ```\n///\n/// Pinning is useful when using `select!` and stream operators that require `T:\n/// Stream + Unpin`.\n///\n/// [`Future`]: trait@std::future::Future\n/// [`Box::pin`]: std::boxed::Box::pin\n///\n/// # Usage\n///\n/// The `pin!` macro takes **identifiers** as arguments. It does **not** work\n/// with expressions.\n///\n/// The following does not compile as an expression is passed to `pin!`.\n///\n/// ```compile_fail\n/// async fn my_async_fn() {\n/// // async logic here\n/// }\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let mut future = pin!(my_async_fn());\n/// (&mut future).await;\n/// }\n/// ```\n///\n/// # Examples\n///\n/// Using with select:\n///\n/// ```\n/// use tokio::{pin, select};\n/// use tokio_stream::{self as stream, StreamExt};\n///\n/// async fn my_async_fn() {\n/// // async logic here\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut stream = stream::iter(vec![1, 2, 3, 4]);\n///\n/// let future = my_async_fn();\n/// pin!(future);\n///\n/// loop {\n/// select! {\n/// _ = &mut future => {\n/// // Stop looping `future` will be polled after completion\n/// break;\n/// }\n/// Some(val) = stream.next() => {\n/// println!(\"got value = {}\", val);\n/// }\n/// }\n/// }\n/// # }\n/// ```\n///\n/// Because assigning to a variable followed by pinning is common, there is also\n/// a variant of the macro that supports doing both in one go.\n///\n/// ```\n/// use tokio::{pin, select};\n///\n/// async fn my_async_fn() {\n/// // async logic here\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// pin! {\n/// let future1 = my_async_fn();\n/// let future2 = my_async_fn();\n/// }\n///\n/// select! {\n/// _ = &mut future1 => {}\n/// _ = &mut future2 => {}\n/// }\n/// # }\n/// ```\n#[macro_export]\nmacro_rules! pin {\n ($($x:ident),*) => { $(\n // Move the value to ensure that it is owned\n let mut $x = $x;\n // Shadow the original binding so that it can't be directly accessed\n // ever again.\n #[allow(unused_mut)]\n let mut $x = unsafe {\n $crate::macros::support::Pin::new_unchecked(&mut $x)\n };\n )* };\n ($(\n let $x:ident = $init:expr;\n )*) => {\n $(\n let $x = $init;\n $crate::pin!($x);\n )*\n };\n}\n" }, { "path": "tokio/src/macros/select.rs", "content": "macro_rules! doc {\n ($select:item) => {\n /// Waits on multiple concurrent branches, returning when the **first** branch\n /// completes, cancelling the remaining branches.\n ///\n /// The `select!` macro must be used inside of async functions, closures, and\n /// blocks.\n ///\n /// The `select!` macro accepts one or more branches with the following pattern:\n ///\n /// ```text\n /// = (, if )? => ,\n /// ```\n ///\n /// Additionally, the `select!` macro may include a single, optional `else`\n /// branch, which evaluates if none of the other branches match their patterns:\n ///\n /// ```text\n /// else => \n /// ```\n ///\n /// The macro aggregates all `` expressions and runs them\n /// concurrently on the **current** task. Once the **first** expression\n /// completes with a value that matches its ``, the `select!` macro\n /// returns the result of evaluating the completed branch's ``\n /// expression.\n ///\n /// Additionally, each branch may include an optional `if` precondition. If the\n /// precondition returns `false`, then the branch is disabled. The provided\n /// `` is still evaluated but the resulting future is never\n /// polled. This capability is useful when using `select!` within a loop.\n ///\n /// The complete lifecycle of a `select!` expression is as follows:\n ///\n /// 1. Evaluate all provided `` expressions. If the precondition\n /// returns `false`, disable the branch for the remainder of the current call\n /// to `select!`. Re-entering `select!` due to a loop clears the \"disabled\"\n /// state.\n /// 2. Aggregate the ``s from each branch, including the\n /// disabled ones. If the branch is disabled, `` is still\n /// evaluated, but the resulting future is not polled.\n /// 3. If **all** branches are disabled: go to step 6.\n /// 4. Concurrently await on the results for all remaining ``s.\n /// 5. Once an `` returns a value, attempt to apply the value to the\n /// provided ``. If the pattern matches, evaluate the `` and return.\n /// If the pattern **does not** match, disable the current branch for the remainder of\n /// the current call to `select!`. Continue from step 3.\n /// 6. Evaluate the `else` expression. If no else expression is provided, panic.\n ///\n /// # Runtime characteristics\n ///\n /// By running all async expressions on the current task, the expressions are\n /// able to run **concurrently** but not in **parallel**. This means all\n /// expressions are run on the same thread and if one branch blocks the thread,\n /// all other expressions will be unable to continue. If parallelism is\n /// required, spawn each async expression using [`tokio::spawn`] and pass the\n /// join handle to `select!`.\n ///\n /// [`tokio::spawn`]: crate::spawn\n ///\n /// # Fairness\n ///\n /// By default, `select!` randomly picks a branch to check first. This provides\n /// some level of fairness when calling `select!` in a loop with branches that\n /// are always ready.\n ///\n /// This behavior can be overridden by adding `biased;` to the beginning of the\n /// macro usage. See the examples for details. This will cause `select` to poll\n /// the futures in the order they appear from top to bottom. There are a few\n /// reasons you may want this:\n ///\n /// - The random number generation of `tokio::select!` has a non-zero CPU cost\n /// - Your futures may interact in a way where known polling order is significant\n ///\n /// But there is an important caveat to this mode. It becomes your responsibility\n /// to ensure that the polling order of your futures is fair. If for example you\n /// are selecting between a stream and a shutdown future, and the stream has a\n /// huge volume of messages and zero or nearly zero time between them, you should\n /// place the shutdown future earlier in the `select!` list to ensure that it is\n /// always polled, and will not be ignored due to the stream being constantly\n /// ready.\n ///\n /// # Panics\n ///\n /// The `select!` macro panics if all branches are disabled **and** there is no\n /// provided `else` branch. A branch is disabled when the provided `if`\n /// precondition returns `false` **or** when the pattern does not match the\n /// result of ``.\n ///\n /// # Cancellation safety\n ///\n /// When using `select!` in a loop to receive messages from multiple sources,\n /// you should make sure that the receive call is cancellation safe to avoid\n /// losing messages. This section goes through various common methods and\n /// describes whether they are cancel safe. The lists in this section are not\n /// exhaustive.\n ///\n /// The following methods are cancellation safe:\n ///\n /// * [`tokio::sync::mpsc::Receiver::recv`](crate::sync::mpsc::Receiver::recv)\n /// * [`tokio::sync::mpsc::UnboundedReceiver::recv`](crate::sync::mpsc::UnboundedReceiver::recv)\n /// * [`tokio::sync::broadcast::Receiver::recv`](crate::sync::broadcast::Receiver::recv)\n /// * [`tokio::sync::watch::Receiver::changed`](crate::sync::watch::Receiver::changed)\n /// * [`tokio::net::TcpListener::accept`](crate::net::TcpListener::accept)\n /// * [`tokio::net::UnixListener::accept`](crate::net::UnixListener::accept)\n /// * [`tokio::signal::unix::Signal::recv`](crate::signal::unix::Signal::recv)\n /// * [`tokio::io::AsyncReadExt::read`](crate::io::AsyncReadExt::read) on any `AsyncRead`\n /// * [`tokio::io::AsyncReadExt::read_buf`](crate::io::AsyncReadExt::read_buf) on any `AsyncRead`\n /// * [`tokio::io::AsyncWriteExt::write`](crate::io::AsyncWriteExt::write) on any `AsyncWrite`\n /// * [`tokio::io::AsyncWriteExt::write_buf`](crate::io::AsyncWriteExt::write_buf) on any `AsyncWrite`\n /// * [`tokio_stream::StreamExt::next`](https://docs.rs/tokio-stream/0.1/tokio_stream/trait.StreamExt.html#method.next) on any `Stream`\n /// * [`futures::stream::StreamExt::next`](https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.next) on any `Stream`\n ///\n /// The following methods are not cancellation safe and can lead to loss of data:\n ///\n /// * [`tokio::io::AsyncReadExt::read_exact`](crate::io::AsyncReadExt::read_exact)\n /// * [`tokio::io::AsyncReadExt::read_to_end`](crate::io::AsyncReadExt::read_to_end)\n /// * [`tokio::io::AsyncReadExt::read_to_string`](crate::io::AsyncReadExt::read_to_string)\n /// * [`tokio::io::AsyncWriteExt::write_all`](crate::io::AsyncWriteExt::write_all)\n ///\n /// The following methods are not cancellation safe because they use a queue for\n /// fairness and cancellation makes you lose your place in the queue:\n ///\n /// * [`tokio::sync::Mutex::lock`](crate::sync::Mutex::lock)\n /// * [`tokio::sync::RwLock::read`](crate::sync::RwLock::read)\n /// * [`tokio::sync::RwLock::write`](crate::sync::RwLock::write)\n /// * [`tokio::sync::Semaphore::acquire`](crate::sync::Semaphore::acquire)\n /// * [`tokio::sync::Notify::notified`](crate::sync::Notify::notified)\n ///\n /// To determine whether your own methods are cancellation safe, look for the\n /// location of uses of `.await`. This is because when an asynchronous method is\n /// cancelled, that always happens at an `.await`. If your function behaves\n /// correctly even if it is restarted while waiting at an `.await`, then it is\n /// cancellation safe.\n ///\n /// Cancellation safety can be defined in the following way: If you have a\n /// future that has not yet completed, then it must be a no-op to drop that\n /// future and recreate it. This definition is motivated by the situation where\n /// a `select!` is used in a loop. Without this guarantee, you would lose your\n /// progress when another branch completes and you restart the `select!` by\n /// going around the loop.\n ///\n /// Be aware that cancelling something that is not cancellation safe is not\n /// necessarily wrong. For example, if you are cancelling a task because the\n /// application is shutting down, then you probably don't care that partially\n /// read data is lost.\n ///\n /// # Examples\n ///\n /// Basic select with two branches.\n ///\n /// ```\n /// async fn do_stuff_async() {\n /// // async work\n /// }\n ///\n /// async fn more_async_work() {\n /// // more here\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// tokio::select! {\n /// _ = do_stuff_async() => {\n /// println!(\"do_stuff_async() completed first\")\n /// }\n /// _ = more_async_work() => {\n /// println!(\"more_async_work() completed first\")\n /// }\n /// };\n /// # }\n /// ```\n ///\n /// Basic stream selecting.\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut stream1 = stream::iter(vec![1, 2, 3]);\n /// let mut stream2 = stream::iter(vec![4, 5, 6]);\n ///\n /// let next = tokio::select! {\n /// v = stream1.next() => v.unwrap(),\n /// v = stream2.next() => v.unwrap(),\n /// };\n ///\n /// assert!(next == 1 || next == 4);\n /// # }\n /// ```\n ///\n /// Collect the contents of two streams. In this example, we rely on pattern\n /// matching and the fact that `stream::iter` is \"fused\", i.e. once the stream\n /// is complete, all calls to `next()` return `None`.\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut stream1 = stream::iter(vec![1, 2, 3]);\n /// let mut stream2 = stream::iter(vec![4, 5, 6]);\n ///\n /// let mut values = vec![];\n ///\n /// loop {\n /// tokio::select! {\n /// Some(v) = stream1.next() => values.push(v),\n /// Some(v) = stream2.next() => values.push(v),\n /// else => break,\n /// }\n /// }\n ///\n /// values.sort();\n /// assert_eq!(&[1, 2, 3, 4, 5, 6], &values[..]);\n /// # }\n /// ```\n ///\n /// Using the same future in multiple `select!` expressions can be done by passing\n /// a reference to the future. Doing so requires the future to be [`Unpin`]. A\n /// future can be made [`Unpin`] by either using [`Box::pin`] or stack pinning.\n ///\n /// [`Unpin`]: std::marker::Unpin\n /// [`Box::pin`]: std::boxed::Box::pin\n ///\n /// Here, a stream is consumed for at most 1 second.\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n /// use tokio::time::{self, Duration};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut stream = stream::iter(vec![1, 2, 3]);\n /// let sleep = time::sleep(Duration::from_secs(1));\n /// tokio::pin!(sleep);\n ///\n /// loop {\n /// tokio::select! {\n /// maybe_v = stream.next() => {\n /// if let Some(v) = maybe_v {\n /// println!(\"got = {}\", v);\n /// } else {\n /// break;\n /// }\n /// }\n /// _ = &mut sleep => {\n /// println!(\"timeout\");\n /// break;\n /// }\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// Joining two values using `select!`.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx1, mut rx1) = oneshot::channel();\n /// let (tx2, mut rx2) = oneshot::channel();\n ///\n /// tokio::spawn(async move {\n /// tx1.send(\"first\").unwrap();\n /// });\n ///\n /// tokio::spawn(async move {\n /// tx2.send(\"second\").unwrap();\n /// });\n ///\n /// let mut a = None;\n /// let mut b = None;\n ///\n /// while a.is_none() || b.is_none() {\n /// tokio::select! {\n /// v1 = (&mut rx1), if a.is_none() => a = Some(v1.unwrap()),\n /// v2 = (&mut rx2), if b.is_none() => b = Some(v2.unwrap()),\n /// }\n /// }\n ///\n /// let res = (a.unwrap(), b.unwrap());\n ///\n /// assert_eq!(res.0, \"first\");\n /// assert_eq!(res.1, \"second\");\n /// # }\n /// ```\n ///\n /// Using the `biased;` mode to control polling order.\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut count = 0u8;\n ///\n /// loop {\n /// tokio::select! {\n /// // If you run this example without `biased;`, the polling order is\n /// // pseudo-random, and the assertions on the value of count will\n /// // (probably) fail.\n /// biased;\n ///\n /// _ = async {}, if count < 1 => {\n /// count += 1;\n /// assert_eq!(count, 1);\n /// }\n /// _ = async {}, if count < 2 => {\n /// count += 1;\n /// assert_eq!(count, 2);\n /// }\n /// _ = async {}, if count < 3 => {\n /// count += 1;\n /// assert_eq!(count, 3);\n /// }\n /// _ = async {}, if count < 4 => {\n /// count += 1;\n /// assert_eq!(count, 4);\n /// }\n ///\n /// else => {\n /// break;\n /// }\n /// };\n /// }\n /// # }\n /// ```\n ///\n /// ## Avoid racy `if` preconditions\n ///\n /// Given that `if` preconditions are used to disable `select!` branches, some\n /// caution must be used to avoid missing values.\n ///\n /// For example, here is **incorrect** usage of `sleep` with `if`. The objective\n /// is to repeatedly run an asynchronous task for up to 50 milliseconds.\n /// However, there is a potential for the `sleep` completion to be missed.\n ///\n /// ```no_run,should_panic\n /// use tokio::time::{self, Duration};\n ///\n /// async fn some_async_work() {\n /// // do work\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let sleep = time::sleep(Duration::from_millis(50));\n /// tokio::pin!(sleep);\n ///\n /// while !sleep.is_elapsed() {\n /// tokio::select! {\n /// _ = &mut sleep, if !sleep.is_elapsed() => {\n /// println!(\"operation timed out\");\n /// }\n /// _ = some_async_work() => {\n /// println!(\"operation completed\");\n /// }\n /// }\n /// }\n ///\n /// panic!(\"This example shows how not to do it!\");\n /// # }\n /// ```\n ///\n /// In the above example, `sleep.is_elapsed()` may return `true` even if\n /// `sleep.poll()` never returned `Ready`. This opens up a potential race\n /// condition where `sleep` expires between the `while !sleep.is_elapsed()`\n /// check and the call to `select!` resulting in the `some_async_work()` call to\n /// run uninterrupted despite the sleep having elapsed.\n ///\n /// One way to write the above example without the race would be:\n ///\n /// ```\n /// use tokio::time::{self, Duration};\n ///\n /// async fn some_async_work() {\n /// # time::sleep(Duration::from_millis(10)).await;\n /// // do work\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let sleep = time::sleep(Duration::from_millis(50));\n /// tokio::pin!(sleep);\n ///\n /// loop {\n /// tokio::select! {\n /// _ = &mut sleep => {\n /// println!(\"operation timed out\");\n /// break;\n /// }\n /// _ = some_async_work() => {\n /// println!(\"operation completed\");\n /// }\n /// }\n /// }\n /// # }\n /// ```\n /// # Alternatives from the Ecosystem\n ///\n /// The `select!` macro is a powerful tool for managing multiple asynchronous\n /// branches, enabling tasks to run concurrently within the same thread. However,\n /// its use can introduce challenges, particularly around cancellation safety, which\n /// can lead to subtle and hard-to-debug errors. For many use cases, ecosystem\n /// alternatives may be preferable as they mitigate these concerns by offering\n /// clearer syntax, more predictable control flow, and reducing the need to manually\n /// handle issues like fuse semantics or cancellation safety.\n ///\n /// ## Merging Streams\n ///\n /// For cases where `loop { select! { ... } }` is used to poll multiple tasks,\n /// stream merging offers a concise alternative, inherently handle cancellation-safe\n /// processing, removing the risk of data loss. Libraries such as [`tokio_stream`],\n /// [`futures::stream`] and [`futures_concurrency`] provide tools for merging\n /// streams and handling their outputs sequentially.\n ///\n /// [`tokio_stream`]: https://docs.rs/tokio-stream/latest/tokio_stream/\n /// [`futures::stream`]: https://docs.rs/futures/latest/futures/stream/\n /// [`futures_concurrency`]: https://docs.rs/futures-concurrency/latest/futures_concurrency/\n ///\n /// ### Example with `select!`\n ///\n /// ```\n /// struct File;\n /// struct Channel;\n /// struct Socket;\n ///\n /// impl Socket {\n /// async fn read_packet(&mut self) -> Vec {\n /// vec![]\n /// }\n /// }\n ///\n /// async fn read_send(_file: &mut File, _channel: &mut Channel) {\n /// // do work that is not cancel safe\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// // open our IO types\n /// let mut file = File;\n /// let mut channel = Channel;\n /// let mut socket = Socket;\n ///\n /// loop {\n /// tokio::select! {\n /// _ = read_send(&mut file, &mut channel) => { /* ... */ },\n /// _data = socket.read_packet() => { /* ... */ }\n /// _ = futures::future::ready(()) => break\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// ### Moving to `merge`\n ///\n /// By using merge, you can unify multiple asynchronous tasks into a single stream,\n /// eliminating the need to manage tasks manually and reducing the risk of\n /// unintended behavior like data loss.\n ///\n /// ```\n /// use std::pin::pin;\n ///\n /// use futures::stream::unfold;\n /// use tokio_stream::StreamExt;\n ///\n /// struct File;\n /// struct Channel;\n /// struct Socket;\n ///\n /// impl Socket {\n /// async fn read_packet(&mut self) -> Vec {\n /// vec![]\n /// }\n /// }\n ///\n /// async fn read_send(_file: &mut File, _channel: &mut Channel) {\n /// // do work that is not cancel safe\n /// }\n ///\n /// enum Message {\n /// Stop,\n /// Sent,\n /// Data(Vec),\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// // open our IO types\n /// let file = File;\n /// let channel = Channel;\n /// let socket = Socket;\n ///\n /// let a = unfold((file, channel), |(mut file, mut channel)| async {\n /// read_send(&mut file, &mut channel).await;\n /// Some((Message::Sent, (file, channel)))\n /// });\n /// let b = unfold(socket, |mut socket| async {\n /// let data = socket.read_packet().await;\n /// Some((Message::Data(data), socket))\n /// });\n /// let c = tokio_stream::iter([Message::Stop]);\n ///\n /// let mut s = pin!(a.merge(b).merge(c));\n /// while let Some(msg) = s.next().await {\n /// match msg {\n /// Message::Data(_data) => { /* ... */ }\n /// Message::Sent => continue,\n /// Message::Stop => break,\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// ## Racing Futures\n ///\n /// If you need to wait for the first completion among several asynchronous tasks,\n /// ecosystem utilities such as\n /// [`futures`](https://docs.rs/futures/latest/futures/),\n /// [`futures-lite`](https://docs.rs/futures-lite/latest/futures_lite/) or\n /// [`futures-concurrency`](https://docs.rs/futures-concurrency/latest/futures_concurrency/)\n /// provide streamlined syntax for racing futures:\n ///\n /// - [`futures_concurrency::future::Race`](https://docs.rs/futures-concurrency/latest/futures_concurrency/future/trait.Race.html)\n /// - [`futures::select`](https://docs.rs/futures/latest/futures/macro.select.html)\n /// - [`futures::stream::select_all`](https://docs.rs/futures/latest/futures/stream/select_all/index.html) (for streams)\n /// - [`futures_lite::future::or`](https://docs.rs/futures-lite/latest/futures_lite/future/fn.or.html)\n /// - [`futures_lite::future::race`](https://docs.rs/futures-lite/latest/futures_lite/future/fn.race.html)\n ///\n /// ```\n /// use futures_concurrency::future::Race;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let task_a = async { Ok(\"ok\") };\n /// let task_b = async { Err(\"error\") };\n /// let result = (task_a, task_b).race().await;\n ///\n /// match result {\n /// Ok(output) => println!(\"First task completed with: {output}\"),\n /// Err(err) => eprintln!(\"Error occurred: {err}\"),\n /// }\n /// # }\n /// ```\n #[macro_export]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n $select\n };\n}\n\n#[cfg(doc)]\ndoc! {macro_rules! select {\n {\n $(\n biased;\n )?\n $(\n $bind:pat = $fut:expr $(, if $cond:expr)? => $handler:expr,\n )*\n $(\n else => $els:expr $(,)?\n )?\n } => {\n unimplemented!()\n };\n}}\n\n#[cfg(not(doc))]\ndoc! {macro_rules! select {\n // Uses a declarative macro to do **most** of the work. While it is possible\n // to implement fully with a declarative macro, a procedural macro is used\n // to enable improved error messages.\n //\n // The macro is structured as a tt-muncher. All branches are processed and\n // normalized. Once the input is normalized, it is passed to the top-most\n // rule. When entering the macro, `@{ }` is inserted at the front. This is\n // used to collect the normalized input.\n //\n // The macro only recurses once per branch. This allows using `select!`\n // without requiring the user to increase the recursion limit.\n\n // All input is normalized, now transform.\n (@ {\n // The index of the future to poll first (in bias mode), or the RNG\n // expression to use to pick a future to poll first.\n start=$start:expr;\n\n // One `_` for each branch in the `select!` macro. Passing this to\n // `count!` converts $skip to an integer.\n ( $($count:tt)* )\n\n // Normalized select branches. `( $skip )` is a set of `_` characters.\n // There is one `_` for each select branch **before** this one. Given\n // that all input futures are stored in a tuple, $skip is useful for\n // generating a pattern to reference the future for the current branch.\n // $skip is also used as an argument to `count!`, returning the index of\n // the current select branch.\n $( ( $($skip:tt)* ) $bind:pat = $fut:expr, if $c:expr => $handle:expr, )+\n\n // Fallback expression used when all select branches have been disabled.\n ; $else:expr\n\n }) => {{\n // Enter a context where stable \"function-like\" proc macros can be used.\n //\n // This module is defined within a scope and should not leak out of this\n // macro.\n #[doc(hidden)]\n mod __tokio_select_util {\n // Generate an enum with one variant per select branch\n $crate::select_priv_declare_output_enum!( ( $($count)* ) );\n }\n\n // `tokio::macros::support` is a public, but doc(hidden) module\n // including a re-export of all types needed by this macro.\n use $crate::macros::support::Pin;\n\n const BRANCHES: u32 = $crate::count!( $($count)* );\n\n let mut disabled: __tokio_select_util::Mask = Default::default();\n\n // First, invoke all the pre-conditions. For any that return true,\n // set the appropriate bit in `disabled`.\n $(\n if !$c {\n let mask: __tokio_select_util::Mask = 1 << $crate::count!( $($skip)* );\n disabled |= mask;\n }\n )*\n\n // Create a scope to separate polling from handling the output. This\n // adds borrow checker flexibility when using the macro.\n let mut output = {\n // Store each future directly first (that is, without wrapping the future in a call to\n // `IntoFuture::into_future`). This allows the `$fut` expression to make use of\n // temporary lifetime extension.\n //\n // https://doc.rust-lang.org/1.58.1/reference/destructors.html#temporary-lifetime-extension\n let futures_init = ($( $fut, )+);\n\n // Safety: Nothing must be moved out of `futures`. This is to\n // satisfy the requirement of `Pin::new_unchecked` called below.\n //\n // We can't use the `pin!` macro for this because `futures` is a\n // tuple and the standard library provides no way to pin-project to\n // the fields of a tuple.\n let mut futures = ($( $crate::macros::support::IntoFuture::into_future(\n $crate::count_field!( futures_init.$($skip)* )\n ),)+);\n\n // This assignment makes sure that the `poll_fn` closure only has a\n // reference to the futures, instead of taking ownership of them.\n // This mitigates the issue described in\n // \n let mut futures = &mut futures;\n\n $crate::macros::support::poll_fn(|cx| {\n // Return `Pending` when the task budget is depleted since budget-aware futures\n // are going to yield anyway and other futures will not cooperate.\n ::std::task::ready!($crate::macros::support::poll_budget_available(cx));\n\n // Track if any branch returns pending. If no branch completes\n // **or** returns pending, this implies that all branches are\n // disabled.\n let mut is_pending = false;\n\n // Choose a starting index to begin polling the futures at. In\n // practice, this will either be a pseudo-randomly generated\n // number by default, or the constant 0 if `biased;` is\n // supplied.\n let start = $start;\n\n for i in 0..BRANCHES {\n let branch;\n #[allow(clippy::modulo_one)]\n {\n branch = (start + i) % BRANCHES;\n }\n match branch {\n $(\n #[allow(unreachable_code)]\n $crate::count!( $($skip)* ) => {\n // First, if the future has previously been\n // disabled, do not poll it again. This is done\n // by checking the associated bit in the\n // `disabled` bit field.\n let mask = 1 << branch;\n\n if disabled & mask == mask {\n // The future has been disabled.\n continue;\n }\n\n // Extract the future for this branch from the\n // tuple\n let ( $($skip,)* fut, .. ) = &mut *futures;\n\n // Safety: future is stored on the stack above\n // and never moved.\n let mut fut = unsafe { Pin::new_unchecked(fut) };\n\n // Try polling it\n let out = match $crate::macros::support::Future::poll(fut, cx) {\n $crate::macros::support::Poll::Ready(out) => out,\n $crate::macros::support::Poll::Pending => {\n // Track that at least one future is\n // still pending and continue polling.\n is_pending = true;\n continue;\n }\n };\n\n // Disable the future from future polling.\n disabled |= mask;\n\n // The future returned a value, check if matches\n // the specified pattern.\n #[allow(unused_variables)]\n #[allow(unused_mut)]\n match &out {\n $crate::select_priv_clean_pattern!($bind) => {}\n _ => continue,\n }\n\n // The select is complete, return the value\n return $crate::macros::support::Poll::Ready($crate::select_variant!(__tokio_select_util::Out, ($($skip)*))(out));\n }\n )*\n _ => unreachable!(\"reaching this means there probably is an off by one bug\"),\n }\n }\n\n if is_pending {\n $crate::macros::support::Poll::Pending\n } else {\n // All branches have been disabled.\n $crate::macros::support::Poll::Ready(__tokio_select_util::Out::Disabled)\n }\n }).await\n };\n\n match output {\n $(\n $crate::select_variant!(__tokio_select_util::Out, ($($skip)*) ($bind)) => $handle,\n )*\n __tokio_select_util::Out::Disabled => $else,\n _ => unreachable!(\"failed to match bind\"),\n }\n }};\n\n // ==== Normalize =====\n\n // These rules match a single `select!` branch and normalize it for\n // processing by the first rule.\n\n (@ { start=$start:expr; $($t:tt)* } ) => {\n // No `else` branch\n $crate::select!(@{ start=$start; $($t)*; panic!(\"all branches are disabled and there is no else branch\") })\n };\n (@ { start=$start:expr; $($t:tt)* } else => $else:expr $(,)?) => {\n $crate::select!(@{ start=$start; $($t)*; $else })\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr, if $c:expr => $h:block, $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if $c => $h, } $($r)*)\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr => $h:block, $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if true => $h, } $($r)*)\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr, if $c:expr => $h:block $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if $c => $h, } $($r)*)\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr => $h:block $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if true => $h, } $($r)*)\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr, if $c:expr => $h:expr ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if $c => $h, })\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr => $h:expr ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if true => $h, })\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr, if $c:expr => $h:expr, $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if $c => $h, } $($r)*)\n };\n (@ { start=$start:expr; ( $($s:tt)* ) $($t:tt)* } $p:pat = $f:expr => $h:expr, $($r:tt)* ) => {\n $crate::select!(@{ start=$start; ($($s)* _) $($t)* ($($s)*) $p = $f, if true => $h, } $($r)*)\n };\n\n // ===== Entry point =====\n\n ($(biased;)? else => $else:expr $(,)? ) => {{\n $else\n }};\n\n (biased; $p:pat = $($t:tt)* ) => {\n $crate::select!(@{ start=0; () } $p = $($t)*)\n };\n\n ( $p:pat = $($t:tt)* ) => {\n // Randomly generate a starting point. This makes `select!` a bit more\n // fair and avoids always polling the first future.\n $crate::select!(@{ start={ $crate::macros::support::thread_rng_n(BRANCHES) }; () } $p = $($t)*)\n };\n\n () => {\n compile_error!(\"select! requires at least one branch.\")\n };\n}}\n\n// And here... we manually list out matches for up to 64 branches... I'm not\n// happy about it either, but this is how we manage to use a declarative macro!\n\n#[macro_export]\n#[doc(hidden)]\nmacro_rules! count {\n () => {\n 0\n };\n (_) => {\n 1\n };\n (_ _) => {\n 2\n };\n (_ _ _) => {\n 3\n };\n (_ _ _ _) => {\n 4\n };\n (_ _ _ _ _) => {\n 5\n };\n (_ _ _ _ _ _) => {\n 6\n };\n (_ _ _ _ _ _ _) => {\n 7\n };\n (_ _ _ _ _ _ _ _) => {\n 8\n };\n (_ _ _ _ _ _ _ _ _) => {\n 9\n };\n (_ _ _ _ _ _ _ _ _ _) => {\n 10\n };\n (_ _ _ _ _ _ _ _ _ _ _) => {\n 11\n };\n (_ _ _ _ _ _ _ _ _ _ _ _) => {\n 12\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 13\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 14\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 15\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 16\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 17\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 18\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 19\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 20\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 21\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 22\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 23\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 24\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 25\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 26\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 27\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 28\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 29\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 30\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 31\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 32\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 33\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 34\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 35\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 36\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 37\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 38\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 39\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 40\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 41\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 42\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 43\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 44\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 45\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 46\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 47\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 48\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 49\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 50\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 51\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 52\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 53\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 54\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 55\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 56\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 57\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 58\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 59\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 60\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 61\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 62\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 63\n };\n (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n 64\n };\n}\n\n#[macro_export]\n#[doc(hidden)]\nmacro_rules! count_field {\n ($var:ident. ) => {\n $var.0\n };\n ($var:ident. _) => {\n $var.1\n };\n ($var:ident. _ _) => {\n $var.2\n };\n ($var:ident. _ _ _) => {\n $var.3\n };\n ($var:ident. _ _ _ _) => {\n $var.4\n };\n ($var:ident. _ _ _ _ _) => {\n $var.5\n };\n ($var:ident. _ _ _ _ _ _) => {\n $var.6\n };\n ($var:ident. _ _ _ _ _ _ _) => {\n $var.7\n };\n ($var:ident. _ _ _ _ _ _ _ _) => {\n $var.8\n };\n ($var:ident. _ _ _ _ _ _ _ _ _) => {\n $var.9\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _) => {\n $var.10\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _) => {\n $var.11\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.12\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.13\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.14\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.15\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.16\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.17\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.18\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.19\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.20\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.21\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.22\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.23\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.24\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.25\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.26\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.27\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.28\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.29\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.30\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.31\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.32\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.33\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.34\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.35\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.36\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.37\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.38\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.39\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.40\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.41\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.42\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.43\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.44\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.45\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.46\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.47\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.48\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.49\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.50\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.51\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.52\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.53\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.54\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.55\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.56\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.57\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.58\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.59\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.60\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.61\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.62\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.63\n };\n ($var:ident. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) => {\n $var.64\n };\n}\n\n#[macro_export]\n#[doc(hidden)]\nmacro_rules! select_variant {\n ($($p:ident)::*, () $($t:tt)*) => {\n $($p)::*::_0 $($t)*\n };\n ($($p:ident)::*, (_) $($t:tt)*) => {\n $($p)::*::_1 $($t)*\n };\n ($($p:ident)::*, (_ _) $($t:tt)*) => {\n $($p)::*::_2 $($t)*\n };\n ($($p:ident)::*, (_ _ _) $($t:tt)*) => {\n $($p)::*::_3 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _) $($t:tt)*) => {\n $($p)::*::_4 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_5 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_6 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_7 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_8 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_9 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_10 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_11 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_12 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_13 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_14 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_15 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_16 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_17 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_18 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_19 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_20 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_21 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_22 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_23 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_24 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_25 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_26 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_27 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_28 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_29 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_30 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_31 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_32 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_33 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_34 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_35 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_36 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_37 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_38 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_39 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_40 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_41 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_42 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_43 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_44 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_45 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_46 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_47 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_48 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_49 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_50 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_51 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_52 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_53 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_54 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_55 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_56 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_57 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_58 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_59 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_60 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_61 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_62 $($t)*\n };\n ($($p:ident)::*, (_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _) $($t:tt)*) => {\n $($p)::*::_63 $($t)*\n };\n}\n" }, { "path": "tokio/src/macros/support.rs", "content": "cfg_macros! {\n pub use crate::future::maybe_done::maybe_done;\n\n pub use std::future::poll_fn;\n\n pub use crate::macros::join::{BiasedRotator, Rotator, RotatorSelect, SelectNormal, SelectBiased};\n\n #[doc(hidden)]\n pub fn thread_rng_n(n: u32) -> u32 {\n crate::runtime::context::thread_rng_n(n)\n }\n\n cfg_coop! {\n #[doc(hidden)]\n #[inline]\n pub fn poll_budget_available(cx: &mut Context<'_>) -> Poll<()> {\n crate::task::coop::poll_budget_available(cx)\n }\n }\n\n cfg_not_coop! {\n #[doc(hidden)]\n #[inline]\n pub fn poll_budget_available(_: &mut Context<'_>) -> Poll<()> {\n Poll::Ready(())\n }\n }\n}\n\npub use std::future::{Future, IntoFuture};\npub use std::pin::Pin;\npub use std::task::{Context, Poll};\n" }, { "path": "tokio/src/macros/thread_local.rs", "content": "#[cfg(all(loom, test))]\nmacro_rules! tokio_thread_local {\n ($(#[$attrs:meta])* $vis:vis static $name:ident: $ty:ty = const { $expr:expr } $(;)?) => {\n loom::thread_local! {\n $(#[$attrs])*\n $vis static $name: $ty = $expr;\n }\n };\n\n ($($tts:tt)+) => { loom::thread_local!{ $($tts)+ } }\n}\n\n#[cfg(not(all(loom, test)))]\nmacro_rules! tokio_thread_local {\n ($($tts:tt)+) => {\n ::std::thread_local!{ $($tts)+ }\n }\n}\n" }, { "path": "tokio/src/macros/trace.rs", "content": "cfg_trace! {\n macro_rules! trace_op {\n ($name:expr, $readiness:literal) => {\n tracing::trace!(\n target: \"runtime::resource::poll_op\",\n op_name = $name,\n is_ready = $readiness\n );\n }\n }\n\n macro_rules! trace_poll_op {\n ($name:expr, $poll:expr $(,)*) => {\n match $poll {\n std::task::Poll::Ready(t) => {\n trace_op!($name, true);\n std::task::Poll::Ready(t)\n }\n std::task::Poll::Pending => {\n trace_op!($name, false);\n return std::task::Poll::Pending;\n }\n }\n };\n }\n}\n" }, { "path": "tokio/src/macros/try_join.rs", "content": "macro_rules! doc {\n ($try_join:item) => {\n /// Waits on multiple concurrent branches, returning when **all** branches\n /// complete with `Ok(_)` or on the first `Err(_)`.\n ///\n /// The `try_join!` macro must be used inside of async functions, closures, and\n /// blocks.\n ///\n /// Similar to [`join!`], the `try_join!` macro takes a list of async\n /// expressions and evaluates them concurrently on the same task. Each async\n /// expression evaluates to a future and the futures from each expression are\n /// multiplexed on the current task. The `try_join!` macro returns when **all**\n /// branches return with `Ok` or when the **first** branch returns with `Err`.\n ///\n /// [`join!`]: macro@join\n ///\n /// # Notes\n ///\n /// The supplied futures are stored inline and do not require allocating a\n /// `Vec`.\n ///\n /// ## Runtime characteristics\n ///\n /// By running all async expressions on the current task, the expressions are\n /// able to run **concurrently** but not in **parallel**. This means all\n /// expressions are run on the same thread and if one branch blocks the thread,\n /// all other expressions will be unable to continue. If parallelism is\n /// required, spawn each async expression using [`tokio::spawn`] and pass the\n /// join handle to `try_join!`.\n ///\n /// [`tokio::spawn`]: crate::spawn\n ///\n /// ## Fairness\n ///\n /// By default, `try_join!`'s generated future rotates which\n /// contained future is polled first whenever it is woken.\n ///\n /// This behavior can be overridden by adding `biased;` to the beginning of the\n /// macro usage. See the examples for details. This will cause `try_join` to poll\n /// the futures in the order they appear from top to bottom.\n ///\n /// You may want this if your futures may interact in a way where known polling order is significant.\n ///\n /// But there is an important caveat to this mode. It becomes your responsibility\n /// to ensure that the polling order of your futures is fair. If for example you\n /// are joining a stream and a shutdown future, and the stream has a\n /// huge volume of messages that takes a long time to finish processing per poll, you should\n /// place the shutdown future earlier in the `try_join!` list to ensure that it is\n /// always polled, and will not be delayed due to the stream future taking a long time to return\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Basic `try_join` with two branches.\n ///\n /// ```\n /// async fn do_stuff_async() -> Result<(), &'static str> {\n /// // async work\n /// # Ok(())\n /// }\n ///\n /// async fn more_async_work() -> Result<(), &'static str> {\n /// // more here\n /// # Ok(())\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let res = tokio::try_join!(\n /// do_stuff_async(),\n /// more_async_work());\n ///\n /// match res {\n /// Ok((first, second)) => {\n /// // do something with the values\n /// }\n /// Err(err) => {\n /// println!(\"processing failed; error = {}\", err);\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// Using `try_join!` with spawned tasks.\n ///\n /// ```\n /// use tokio::task::JoinHandle;\n ///\n /// async fn do_stuff_async() -> Result<(), &'static str> {\n /// // async work\n /// # Err(\"failed\")\n /// }\n ///\n /// async fn more_async_work() -> Result<(), &'static str> {\n /// // more here\n /// # Ok(())\n /// }\n ///\n /// async fn flatten(handle: JoinHandle>) -> Result {\n /// match handle.await {\n /// Ok(Ok(result)) => Ok(result),\n /// Ok(Err(err)) => Err(err),\n /// Err(err) => Err(\"handling failed\"),\n /// }\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let handle1 = tokio::spawn(do_stuff_async());\n /// let handle2 = tokio::spawn(more_async_work());\n /// match tokio::try_join!(flatten(handle1), flatten(handle2)) {\n /// Ok(val) => {\n /// // do something with the values\n /// }\n /// Err(err) => {\n /// println!(\"Failed with {}.\", err);\n /// # assert_eq!(err, \"failed\");\n /// }\n /// }\n /// # }\n /// ```\n /// Using the `biased;` mode to control polling order.\n ///\n /// ```\n /// async fn do_stuff_async() -> Result<(), &'static str> {\n /// // async work\n /// # Ok(())\n /// }\n ///\n /// async fn more_async_work() -> Result<(), &'static str> {\n /// // more here\n /// # Ok(())\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let res = tokio::try_join!(\n /// biased;\n /// do_stuff_async(),\n /// more_async_work()\n /// );\n ///\n /// match res {\n /// Ok((first, second)) => {\n /// // do something with the values\n /// }\n /// Err(err) => {\n /// println!(\"processing failed; error = {}\", err);\n /// }\n /// }\n /// # }\n /// ```\n #[macro_export]\n #[cfg_attr(docsrs, doc(cfg(feature = \"macros\")))]\n $try_join\n };\n}\n\n#[cfg(doc)]\ndoc! {macro_rules! try_join {\n ($(biased;)? $($future:expr),*) => { unimplemented!() }\n}}\n\n#[cfg(not(doc))]\ndoc! {macro_rules! try_join {\n (@ {\n // Type of rotator that controls which inner future to start with\n // when polling our output future.\n rotator_select=$rotator_select:ty;\n\n // One `_` for each branch in the `try_join!` macro. This is not used once\n // normalization is complete.\n ( $($count:tt)* )\n\n // The expression `0+1+1+ ... +1` equal to the number of branches.\n ( $($total:tt)* )\n\n // Normalized try_join! branches\n $( ( $($skip:tt)* ) $e:expr, )*\n\n }) => {{\n // Safety: nothing must be moved out of `futures`. This is to satisfy\n // the requirement of `Pin::new_unchecked` called below.\n //\n // We can't use the `pin!` macro for this because `futures` is a tuple\n // and the standard library provides no way to pin-project to the fields\n // of a tuple.\n let mut futures = ( $( $crate::macros::support::maybe_done($e), )* );\n\n // This assignment makes sure that the `poll_fn` closure only has a\n // reference to the futures, instead of taking ownership of them. This\n // mitigates the issue described in\n // \n let mut futures = &mut futures;\n\n // Each time the future created by poll_fn is polled, if not using biased mode,\n // a different future is polled first to ensure every future passed to try_join!\n // can make progress even if one of the futures consumes the whole budget.\n let mut rotator = <$rotator_select as $crate::macros::support::RotatorSelect>::Rotator::<{$($total)*}>::default();\n\n $crate::macros::support::poll_fn(move |cx| {\n const COUNT: u32 = $($total)*;\n\n let mut is_pending = false;\n let mut to_run = COUNT;\n\n // The number of futures that will be skipped in the first loop iteration.\n let mut skip = rotator.num_skip();\n\n // This loop runs twice and the first `skip` futures\n // are not polled in the first iteration.\n loop {\n $(\n if skip == 0 {\n if to_run == 0 {\n // Every future has been polled\n break;\n }\n to_run -= 1;\n\n // Extract the future for this branch from the tuple.\n let ( $($skip,)* fut, .. ) = &mut *futures;\n\n // Safety: future is stored on the stack above\n // and never moved.\n let mut fut = unsafe { $crate::macros::support::Pin::new_unchecked(fut) };\n\n // Try polling\n if $crate::macros::support::Future::poll(fut.as_mut(), cx).is_pending() {\n is_pending = true;\n } else if fut.as_mut().output_mut().expect(\"expected completed future\").is_err() {\n return $crate::macros::support::Poll::Ready(Err(fut.take_output().expect(\"expected completed future\").err().unwrap()))\n }\n } else {\n // Future skipped, one less future to skip in the next iteration\n skip -= 1;\n }\n )*\n }\n\n if is_pending {\n $crate::macros::support::Poll::Pending\n } else {\n $crate::macros::support::Poll::Ready(Ok(($({\n // Extract the future for this branch from the tuple.\n let ( $($skip,)* fut, .. ) = &mut futures;\n\n // Safety: future is stored on the stack above\n // and never moved.\n let mut fut = unsafe { $crate::macros::support::Pin::new_unchecked(fut) };\n\n fut\n .take_output()\n .expect(\"expected completed future\")\n .ok()\n .expect(\"expected Ok(_)\")\n },)*)))\n }\n }).await\n }};\n\n // ===== Normalize =====\n\n (@ { rotator_select=$rotator_select:ty; ( $($s:tt)* ) ( $($n:tt)* ) $($t:tt)* } $e:expr, $($r:tt)* ) => {\n $crate::try_join!(@{ rotator_select=$rotator_select; ($($s)* _) ($($n)* + 1) $($t)* ($($s)*) $e, } $($r)*)\n };\n\n // ===== Entry point =====\n ( biased; $($e:expr),+ $(,)?) => {\n $crate::try_join!(@{ rotator_select=$crate::macros::support::SelectBiased; () (0) } $($e,)*)\n };\n\n ( $($e:expr),+ $(,)?) => {\n $crate::try_join!(@{ rotator_select=$crate::macros::support::SelectNormal; () (0) } $($e,)*)\n };\n\n (biased;) => { async { Ok(()) }.await };\n\n () => { async { Ok(()) }.await }\n}}\n" }, { "path": "tokio/src/net/addr.rs", "content": "use std::future;\nuse std::io;\nuse std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr, SocketAddrV4, SocketAddrV6};\n\n/// Converts or resolves without blocking to one or more `SocketAddr` values.\n///\n/// # DNS\n///\n/// Implementations of `ToSocketAddrs` for string types require a DNS lookup.\n///\n/// # Calling\n///\n/// Currently, this trait is only used as an argument to Tokio functions that\n/// need to reference a target socket address. To perform a `SocketAddr`\n/// conversion directly, use [`lookup_host()`](super::lookup_host()).\n///\n/// This trait is sealed and is intended to be opaque. The details of the trait\n/// will change. Stabilization is pending enhancements to the Rust language.\npub trait ToSocketAddrs: sealed::ToSocketAddrsPriv {}\n\ntype ReadyFuture = future::Ready>;\n\ncfg_net! {\n pub(crate) fn to_socket_addrs(arg: T) -> T::Future\n where\n T: ToSocketAddrs,\n {\n arg.to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl &impl ToSocketAddrs =====\n\nimpl ToSocketAddrs for &T {}\n\nimpl sealed::ToSocketAddrsPriv for &T\nwhere\n T: sealed::ToSocketAddrsPriv + ?Sized,\n{\n type Iter = T::Iter;\n type Future = T::Future;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n (**self).to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl SocketAddr =====\n\nimpl ToSocketAddrs for SocketAddr {}\n\nimpl sealed::ToSocketAddrsPriv for SocketAddr {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n let iter = Some(*self).into_iter();\n future::ready(Ok(iter))\n }\n}\n\n// ===== impl SocketAddrV4 =====\n\nimpl ToSocketAddrs for SocketAddrV4 {}\n\nimpl sealed::ToSocketAddrsPriv for SocketAddrV4 {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n SocketAddr::V4(*self).to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl SocketAddrV6 =====\n\nimpl ToSocketAddrs for SocketAddrV6 {}\n\nimpl sealed::ToSocketAddrsPriv for SocketAddrV6 {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n SocketAddr::V6(*self).to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl (IpAddr, u16) =====\n\nimpl ToSocketAddrs for (IpAddr, u16) {}\n\nimpl sealed::ToSocketAddrsPriv for (IpAddr, u16) {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n let iter = Some(SocketAddr::from(*self)).into_iter();\n future::ready(Ok(iter))\n }\n}\n\n// ===== impl (Ipv4Addr, u16) =====\n\nimpl ToSocketAddrs for (Ipv4Addr, u16) {}\n\nimpl sealed::ToSocketAddrsPriv for (Ipv4Addr, u16) {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n let (ip, port) = *self;\n SocketAddrV4::new(ip, port).to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl (Ipv6Addr, u16) =====\n\nimpl ToSocketAddrs for (Ipv6Addr, u16) {}\n\nimpl sealed::ToSocketAddrsPriv for (Ipv6Addr, u16) {\n type Iter = std::option::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n let (ip, port) = *self;\n SocketAddrV6::new(ip, port, 0, 0).to_socket_addrs(sealed::Internal)\n }\n}\n\n// ===== impl &[SocketAddr] =====\n\nimpl ToSocketAddrs for &[SocketAddr] {}\n\nimpl sealed::ToSocketAddrsPriv for &[SocketAddr] {\n type Iter = std::vec::IntoIter;\n type Future = ReadyFuture;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n #[inline]\n fn slice_to_vec(addrs: &[SocketAddr]) -> Vec {\n addrs.to_vec()\n }\n\n // This uses a helper method because clippy doesn't like the `to_vec()`\n // call here (it will allocate, whereas `self.iter().copied()` would\n // not), but it's actually necessary in order to ensure that the\n // returned iterator is valid for the `'static` lifetime, which the\n // borrowed `slice::Iter` iterator would not be.\n //\n // Note that we can't actually add an `allow` attribute for\n // `clippy::unnecessary_to_owned` here, as Tokio's CI runs clippy lints\n // on Rust 1.52 to avoid breaking LTS releases of Tokio. Users of newer\n // Rust versions who see this lint should just ignore it.\n let iter = slice_to_vec(self).into_iter();\n future::ready(Ok(iter))\n }\n}\n\ncfg_net! {\n // ===== impl str =====\n\n impl ToSocketAddrs for str {}\n\n impl sealed::ToSocketAddrsPriv for str {\n type Iter = sealed::OneOrMore;\n type Future = sealed::MaybeReady;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n use crate::blocking::spawn_blocking;\n use sealed::MaybeReady;\n\n // First check if the input parses as a socket address\n let res: Result = self.parse();\n\n if let Ok(addr) = res {\n return MaybeReady(sealed::State::Ready(Some(addr)));\n }\n\n // Run DNS lookup on the blocking pool\n let s = self.to_owned();\n\n MaybeReady(sealed::State::Blocking(spawn_blocking(move || {\n std::net::ToSocketAddrs::to_socket_addrs(&s)\n })))\n }\n }\n\n // ===== impl (&str, u16) =====\n\n impl ToSocketAddrs for (&str, u16) {}\n\n impl sealed::ToSocketAddrsPriv for (&str, u16) {\n type Iter = sealed::OneOrMore;\n type Future = sealed::MaybeReady;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n use crate::blocking::spawn_blocking;\n use sealed::MaybeReady;\n\n let (host, port) = *self;\n\n // try to parse the host as a regular IP address first\n if let Ok(addr) = host.parse::() {\n let addr = SocketAddrV4::new(addr, port);\n let addr = SocketAddr::V4(addr);\n\n return MaybeReady(sealed::State::Ready(Some(addr)));\n }\n\n if let Ok(addr) = host.parse::() {\n let addr = SocketAddrV6::new(addr, port, 0, 0);\n let addr = SocketAddr::V6(addr);\n\n return MaybeReady(sealed::State::Ready(Some(addr)));\n }\n\n let host = host.to_owned();\n\n MaybeReady(sealed::State::Blocking(spawn_blocking(move || {\n std::net::ToSocketAddrs::to_socket_addrs(&(&host[..], port))\n })))\n }\n }\n\n // ===== impl (String, u16) =====\n\n impl ToSocketAddrs for (String, u16) {}\n\n impl sealed::ToSocketAddrsPriv for (String, u16) {\n type Iter = sealed::OneOrMore;\n type Future = sealed::MaybeReady;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n (self.0.as_str(), self.1).to_socket_addrs(sealed::Internal)\n }\n }\n\n // ===== impl String =====\n\n impl ToSocketAddrs for String {}\n\n impl sealed::ToSocketAddrsPriv for String {\n type Iter = ::Iter;\n type Future = ::Future;\n\n fn to_socket_addrs(&self, _: sealed::Internal) -> Self::Future {\n self[..].to_socket_addrs(sealed::Internal)\n }\n }\n}\n\npub(crate) mod sealed {\n //! The contents of this trait are intended to remain private and __not__\n //! part of the `ToSocketAddrs` public API. The details will change over\n //! time.\n\n use std::future::Future;\n use std::io;\n use std::net::SocketAddr;\n\n #[doc(hidden)]\n pub trait ToSocketAddrsPriv {\n type Iter: Iterator + Send + 'static;\n type Future: Future> + Send + 'static;\n\n fn to_socket_addrs(&self, internal: Internal) -> Self::Future;\n }\n\n #[allow(missing_debug_implementations)]\n pub struct Internal;\n\n cfg_net! {\n use crate::blocking::JoinHandle;\n\n use std::option;\n use std::pin::Pin;\n use std::task::{ready,Context, Poll};\n use std::vec;\n\n #[doc(hidden)]\n #[derive(Debug)]\n pub struct MaybeReady(pub(super) State);\n\n #[derive(Debug)]\n pub(super) enum State {\n Ready(Option),\n Blocking(JoinHandle>>),\n }\n\n #[doc(hidden)]\n #[derive(Debug)]\n pub enum OneOrMore {\n One(option::IntoIter),\n More(vec::IntoIter),\n }\n\n impl Future for MaybeReady {\n type Output = io::Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n match self.0 {\n State::Ready(ref mut i) => {\n let iter = OneOrMore::One(i.take().into_iter());\n Poll::Ready(Ok(iter))\n }\n State::Blocking(ref mut rx) => {\n let res = ready!(Pin::new(rx).poll(cx))?.map(OneOrMore::More);\n\n Poll::Ready(res)\n }\n }\n }\n }\n\n impl Iterator for OneOrMore {\n type Item = SocketAddr;\n\n fn next(&mut self) -> Option {\n match self {\n OneOrMore::One(i) => i.next(),\n OneOrMore::More(i) => i.next(),\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n match self {\n OneOrMore::One(i) => i.size_hint(),\n OneOrMore::More(i) => i.size_hint(),\n }\n }\n }\n }\n}\n" }, { "path": "tokio/src/net/lookup_host.rs", "content": "cfg_net! {\n use crate::net::addr::{self, ToSocketAddrs};\n\n use std::io;\n use std::net::SocketAddr;\n\n /// Performs a DNS resolution.\n ///\n /// The returned iterator may not actually yield any values depending on the\n /// outcome of any resolution performed.\n ///\n /// This API is not intended to cover all DNS use cases. Anything beyond the\n /// basic use case should be done with a specialized library.\n ///\n /// # Examples\n ///\n /// To resolve a DNS entry:\n ///\n /// ```no_run\n /// use tokio::net;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// for addr in net::lookup_host(\"localhost:3000\").await? {\n /// println!(\"socket address is {}\", addr);\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn lookup_host(host: T) -> io::Result>\n where\n T: ToSocketAddrs\n {\n addr::to_socket_addrs(host).await\n }\n}\n" }, { "path": "tokio/src/net/mod.rs", "content": "#![cfg(not(loom))]\n\n//! TCP/UDP/Unix bindings for `tokio`.\n//!\n//! This module contains the TCP/UDP/Unix networking types, similar to the standard\n//! library, which can be used to implement networking protocols.\n//!\n//! # Organization\n//!\n//! * [`TcpListener`] and [`TcpStream`] provide functionality for communication over TCP\n//! * [`UdpSocket`] provides functionality for communication over UDP\n//! * [`UnixListener`] and [`UnixStream`] provide functionality for communication over a\n//! Unix Domain Stream Socket **(available on Unix only)**\n//! * [`UnixDatagram`] provides functionality for communication\n//! over Unix Domain Datagram Socket **(available on Unix only)**\n//! * [`tokio::net::unix::pipe`] for FIFO pipes **(available on Unix only)**\n//! * [`tokio::net::windows::named_pipe`] for Named Pipes **(available on Windows only)**\n//!\n//! For IO resources not available in `tokio::net`, you can use [`AsyncFd`].\n//!\n//! [`TcpListener`]: TcpListener\n//! [`TcpStream`]: TcpStream\n//! [`UdpSocket`]: UdpSocket\n//! [`UnixListener`]: UnixListener\n//! [`UnixStream`]: UnixStream\n//! [`UnixDatagram`]: UnixDatagram\n//! [`tokio::net::unix::pipe`]: unix::pipe\n//! [`tokio::net::windows::named_pipe`]: windows::named_pipe\n//! [`AsyncFd`]: crate::io::unix::AsyncFd\n\nmod addr;\ncfg_not_wasi! {\n #[cfg(feature = \"net\")]\n pub(crate) use addr::to_socket_addrs;\n}\npub use addr::ToSocketAddrs;\n\ncfg_net! {\n mod lookup_host;\n pub use lookup_host::lookup_host;\n\n pub mod tcp;\n pub use tcp::listener::TcpListener;\n pub use tcp::stream::TcpStream;\n cfg_not_wasi! {\n pub use tcp::socket::TcpSocket;\n\n mod udp;\n #[doc(inline)]\n pub use udp::UdpSocket;\n }\n}\n\ncfg_net_unix! {\n pub mod unix;\n pub use unix::datagram::socket::UnixDatagram;\n pub use unix::listener::UnixListener;\n pub use unix::stream::UnixStream;\n pub use unix::socket::UnixSocket;\n}\n\ncfg_net_windows! {\n pub mod windows;\n}\n" }, { "path": "tokio/src/net/tcp/listener.rs", "content": "use crate::io::{Interest, PollEvented};\nuse crate::net::tcp::TcpStream;\nuse crate::util::check_socket_for_blocking;\n\ncfg_not_wasi! {\n use crate::net::{to_socket_addrs, ToSocketAddrs};\n}\n\nuse std::fmt;\nuse std::io;\nuse std::net::{self, SocketAddr};\nuse std::task::{ready, Context, Poll};\n\ncfg_net! {\n /// A TCP socket server, listening for connections.\n ///\n /// You can accept a new connection by using the [`accept`](`TcpListener::accept`)\n /// method.\n ///\n /// A `TcpListener` can be turned into a `Stream` with [`TcpListenerStream`].\n ///\n /// The socket will be closed when the value is dropped.\n ///\n /// [`TcpListenerStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.TcpListenerStream.html\n ///\n /// # Errors\n ///\n /// Note that accepting a connection can lead to various errors and not all\n /// of them are necessarily fatal ‒ for example having too many open file\n /// descriptors or the other side closing the connection while it waits in\n /// an accept queue. These would terminate the stream if not handled in any\n /// way.\n ///\n /// # Examples\n ///\n /// Using `accept`:\n /// ```no_run\n /// use tokio::net::TcpListener;\n ///\n /// use std::io;\n ///\n /// async fn process_socket(socket: T) {\n /// # drop(socket);\n /// // do work with socket here\n /// }\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// let (socket, _) = listener.accept().await?;\n /// process_socket(socket).await;\n /// }\n /// }\n /// ```\n pub struct TcpListener {\n io: PollEvented,\n }\n}\n\nimpl TcpListener {\n cfg_not_wasi! {\n /// Creates a new `TcpListener`, which will be bound to the specified address.\n ///\n /// The returned listener is ready for accepting connections.\n ///\n /// Binding with a port number of 0 will request that the OS assigns a port\n /// to this listener. The port allocated can be queried via the `local_addr`\n /// method.\n ///\n /// The address type can be any implementor of the [`ToSocketAddrs`] trait.\n /// If `addr` yields multiple addresses, bind will be attempted with each of\n /// the addresses until one succeeds and returns the listener. If none of\n /// the addresses succeed in creating a listener, the error returned from\n /// the last attempt (the last address) is returned.\n ///\n /// This function sets the `SO_REUSEADDR` option on the socket on Unix.\n ///\n /// To configure the socket before binding, you can use the [`TcpSocket`]\n /// type.\n ///\n /// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs\n /// [`TcpSocket`]: struct@crate::net::TcpSocket\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpListener;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let listener = TcpListener::bind(\"127.0.0.1:2345\").await?;\n ///\n /// // use the listener\n ///\n /// # let _ = listener;\n /// Ok(())\n /// }\n /// ```\n pub async fn bind(addr: A) -> io::Result {\n let addrs = to_socket_addrs(addr).await?;\n\n let mut last_err = None;\n\n for addr in addrs {\n match TcpListener::bind_addr(addr) {\n Ok(listener) => return Ok(listener),\n Err(e) => last_err = Some(e),\n }\n }\n\n Err(last_err.unwrap_or_else(|| {\n io::Error::new(\n io::ErrorKind::InvalidInput,\n \"could not resolve to any address\",\n )\n }))\n }\n\n fn bind_addr(addr: SocketAddr) -> io::Result {\n let listener = mio::net::TcpListener::bind(addr)?;\n TcpListener::new(listener)\n }\n }\n\n /// Accepts a new incoming connection from this listener.\n ///\n /// This function will yield once a new TCP connection is established. When\n /// established, the corresponding [`TcpStream`] and the remote peer's\n /// address will be returned.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If the method is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no new connections were\n /// accepted by this method.\n ///\n /// [`TcpStream`]: struct@crate::net::TcpStream\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpListener;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n ///\n /// match listener.accept().await {\n /// Ok((_socket, addr)) => println!(\"new client: {:?}\", addr),\n /// Err(e) => println!(\"couldn't get client: {:?}\", e),\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn accept(&self) -> io::Result<(TcpStream, SocketAddr)> {\n let (mio, addr) = self\n .io\n .registration()\n .async_io(Interest::READABLE, || self.io.accept())\n .await?;\n\n let stream = TcpStream::new(mio)?;\n Ok((stream, addr))\n }\n\n /// Polls to accept a new incoming connection to this listener.\n ///\n /// If there is no connection to accept, `Poll::Pending` is returned and the\n /// current task will be notified by a waker. Note that on multiple calls\n /// to `poll_accept`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup.\n pub fn poll_accept(&self, cx: &mut Context<'_>) -> Poll> {\n loop {\n let ev = ready!(self.io.registration().poll_read_ready(cx))?;\n\n match self.io.accept() {\n Ok((io, addr)) => {\n let io = TcpStream::new(io)?;\n return Poll::Ready(Ok((io, addr)));\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.io.registration().clear_readiness(ev);\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n\n /// Creates new `TcpListener` from a `std::net::TcpListener`.\n ///\n /// This function is intended to be used to wrap a TCP listener from the\n /// standard library in the Tokio equivalent.\n ///\n /// This API is typically paired with the `socket2` crate and the `Socket`\n /// type to build up and customize a listener before it's shipped off to the\n /// backing event loop. This allows configuration of options like\n /// `SO_REUSEPORT`, binding to multiple addresses, etc.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the listener is in\n /// non-blocking mode. Otherwise all I/O operations on the listener\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::net::TcpListener::set_nonblocking\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use std::error::Error;\n /// use tokio::net::TcpListener;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let std_listener = std::net::TcpListener::bind(\"127.0.0.1:0\")?;\n /// std_listener.set_nonblocking(true)?;\n /// let listener = TcpListener::from_std(std_listener)?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n pub fn from_std(listener: net::TcpListener) -> io::Result {\n check_socket_for_blocking(&listener)?;\n\n let io = mio::net::TcpListener::from_std(listener);\n let io = PollEvented::new(io)?;\n Ok(TcpListener { io })\n }\n\n /// Turns a [`tokio::net::TcpListener`] into a [`std::net::TcpListener`].\n ///\n /// The returned [`std::net::TcpListener`] will have nonblocking mode set as\n /// `true`. Use [`set_nonblocking`] to change the blocking mode if needed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let tokio_listener = tokio::net::TcpListener::bind(\"127.0.0.1:0\").await?;\n /// let std_listener = tokio_listener.into_std()?;\n /// std_listener.set_nonblocking(false)?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`tokio::net::TcpListener`]: TcpListener\n /// [`std::net::TcpListener`]: std::net::TcpListener\n /// [`set_nonblocking`]: fn@std::net::TcpListener::set_nonblocking\n pub fn into_std(self) -> io::Result {\n #[cfg(unix)]\n {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { std::net::TcpListener::from_raw_fd(raw_fd) })\n }\n\n #[cfg(windows)]\n {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n self.io\n .into_inner()\n .map(|io| io.into_raw_socket())\n .map(|raw_socket| unsafe { std::net::TcpListener::from_raw_socket(raw_socket) })\n }\n\n #[cfg(target_os = \"wasi\")]\n {\n use std::os::wasi::io::{FromRawFd, IntoRawFd};\n self.io\n .into_inner()\n .map(|io| io.into_raw_fd())\n .map(|raw_fd| unsafe { std::net::TcpListener::from_raw_fd(raw_fd) })\n }\n }\n\n cfg_not_wasi! {\n pub(crate) fn new(listener: mio::net::TcpListener) -> io::Result {\n let io = PollEvented::new(listener)?;\n Ok(TcpListener { io })\n }\n }\n\n /// Returns the local address that this listener is bound to.\n ///\n /// This can be useful, for example, when binding to port 0 to figure out\n /// which port was actually bound.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::net::TcpListener;\n ///\n /// use std::io;\n /// use std::net::{Ipv4Addr, SocketAddr, SocketAddrV4};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n ///\n /// assert_eq!(listener.local_addr()?,\n /// SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 8080)));\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr()\n }\n\n /// Gets the value of the `IP_TTL` option for this socket.\n ///\n /// For more information about this option, see [`set_ttl`].\n ///\n /// [`set_ttl`]: method@Self::set_ttl\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpListener;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:0\").await?;\n ///\n /// listener.set_ttl(100).expect(\"could not set TTL\");\n /// assert_eq!(listener.ttl()?, 100);\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn ttl(&self) -> io::Result {\n self.io.ttl()\n }\n\n /// Sets the value for the `IP_TTL` option on this socket.\n ///\n /// This value sets the time-to-live field that is used in every packet sent\n /// from this socket.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpListener;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:0\").await?;\n ///\n /// listener.set_ttl(100).expect(\"could not set TTL\");\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn set_ttl(&self, ttl: u32) -> io::Result<()> {\n self.io.set_ttl(ttl)\n }\n}\n\nimpl TryFrom for TcpListener {\n type Error = io::Error;\n\n /// Consumes stream, returning the tokio I/O object.\n ///\n /// This is equivalent to\n /// [`TcpListener::from_std(stream)`](TcpListener::from_std).\n fn try_from(stream: net::TcpListener) -> Result {\n Self::from_std(stream)\n }\n}\n\nimpl fmt::Debug for TcpListener {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n (*self.io).fmt(f)\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use super::TcpListener;\n use std::os::unix::prelude::*;\n\n impl AsRawFd for TcpListener {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n }\n\n impl AsFd for TcpListener {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_unstable! {\n #[cfg(target_os = \"wasi\")]\n mod sys {\n use super::TcpListener;\n use std::os::wasi::prelude::*;\n\n impl AsRawFd for TcpListener {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n }\n\n impl AsFd for TcpListener {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawSocket, RawSocket, AsSocket, BorrowedSocket};\n\n impl AsRawSocket for TcpListener {\n fn as_raw_socket(&self) -> RawSocket {\n self.io.as_raw_socket()\n }\n }\n\n impl AsSocket for TcpListener {\n fn as_socket(&self) -> BorrowedSocket<'_> {\n unsafe { BorrowedSocket::borrow_raw(self.as_raw_socket()) }\n }\n }\n}\n" }, { "path": "tokio/src/net/tcp/mod.rs", "content": "//! TCP utility types.\n\npub(crate) mod listener;\n\ncfg_not_wasi! {\n pub(crate) mod socket;\n}\n\nmod split;\npub use split::{ReadHalf, WriteHalf};\n\nmod split_owned;\npub use split_owned::{OwnedReadHalf, OwnedWriteHalf, ReuniteError};\n\npub(crate) mod stream;\npub(crate) use stream::TcpStream;\n" }, { "path": "tokio/src/net/tcp/socket.rs", "content": "use crate::net::{TcpListener, TcpStream};\n\nuse std::fmt;\nuse std::io;\nuse std::net::SocketAddr;\n\n#[cfg(unix)]\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd};\nuse std::time::Duration;\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawSocket, FromRawSocket, IntoRawSocket, RawSocket, AsSocket, BorrowedSocket};\n}\n\ncfg_net! {\n /// A TCP socket that has not yet been converted to a `TcpStream` or\n /// `TcpListener`.\n ///\n /// `TcpSocket` wraps an operating system socket and enables the caller to\n /// configure the socket before establishing a TCP connection or accepting\n /// inbound connections. The caller is able to set socket option and explicitly\n /// bind the socket with a socket address.\n ///\n /// The underlying socket is closed when the `TcpSocket` value is dropped.\n ///\n /// `TcpSocket` should only be used directly if the default configuration used\n /// by `TcpStream::connect` and `TcpListener::bind` does not meet the required\n /// use case.\n ///\n /// Calling `TcpStream::connect(\"127.0.0.1:8080\")` is equivalent to:\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// let stream = socket.connect(addr).await?;\n /// # drop(stream);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// Calling `TcpListener::bind(\"127.0.0.1:8080\")` is equivalent to:\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// // On platforms with Berkeley-derived sockets, this allows to quickly\n /// // rebind a socket, without needing to wait for the OS to clean up the\n /// // previous one.\n /// //\n /// // On Windows, this allows rebinding sockets which are actively in use,\n /// // which allows \"socket hijacking\", so we explicitly don't set it here.\n /// // https://docs.microsoft.com/en-us/windows/win32/winsock/using-so-reuseaddr-and-so-exclusiveaddruse\n /// socket.set_reuseaddr(true)?;\n /// socket.bind(addr)?;\n ///\n /// // Note: the actual backlog used by `TcpListener::bind` is platform-dependent,\n /// // as Tokio relies on Mio's default backlog value configuration. The `1024` here is only\n /// // illustrative and does not reflect the real value used.\n /// let listener = socket.listen(1024)?;\n /// # drop(listener);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// Setting socket options not explicitly provided by `TcpSocket` may be done by\n /// accessing the `RawFd`/`RawSocket` using [`AsRawFd`]/[`AsRawSocket`] and\n /// setting the option with a crate like [`socket2`].\n ///\n /// [`RawFd`]: https://doc.rust-lang.org/std/os/fd/type.RawFd.html\n /// [`RawSocket`]: https://doc.rust-lang.org/std/os/windows/io/type.RawSocket.html\n /// [`AsRawFd`]: https://doc.rust-lang.org/std/os/fd/trait.AsRawFd.html\n /// [`AsRawSocket`]: https://doc.rust-lang.org/std/os/windows/io/trait.AsRawSocket.html\n /// [`socket2`]: https://docs.rs/socket2/\n #[cfg_attr(docsrs, doc(alias = \"connect_std\"))]\n pub struct TcpSocket {\n inner: socket2::Socket,\n }\n}\n\nimpl TcpSocket {\n /// Creates a new socket configured for IPv4.\n ///\n /// Calls `socket(2)` with `AF_INET` and `SOCK_STREAM`.\n ///\n /// # Returns\n ///\n /// On success, the newly created `TcpSocket` is returned. If an error is\n /// encountered, it is returned instead.\n ///\n /// # Examples\n ///\n /// Create a new IPv4 socket and start listening.\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n /// let socket = TcpSocket::new_v4()?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(128)?;\n /// # drop(listener);\n /// Ok(())\n /// }\n /// ```\n pub fn new_v4() -> io::Result {\n TcpSocket::new(socket2::Domain::IPV4)\n }\n\n /// Creates a new socket configured for IPv6.\n ///\n /// Calls `socket(2)` with `AF_INET6` and `SOCK_STREAM`.\n ///\n /// # Returns\n ///\n /// On success, the newly created `TcpSocket` is returned. If an error is\n /// encountered, it is returned instead.\n ///\n /// # Examples\n ///\n /// Create a new IPv6 socket and start listening.\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"[::1]:8080\".parse().unwrap();\n /// let socket = TcpSocket::new_v6()?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(128)?;\n /// # drop(listener);\n /// Ok(())\n /// }\n /// ```\n pub fn new_v6() -> io::Result {\n TcpSocket::new(socket2::Domain::IPV6)\n }\n\n fn new(domain: socket2::Domain) -> io::Result {\n let ty = socket2::Type::STREAM;\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"netbsd\",\n target_os = \"openbsd\"\n ))]\n let ty = ty.nonblocking();\n let inner = socket2::Socket::new(domain, ty, Some(socket2::Protocol::TCP))?;\n #[cfg(not(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"netbsd\",\n target_os = \"openbsd\"\n )))]\n inner.set_nonblocking(true)?;\n Ok(TcpSocket { inner })\n }\n\n /// Sets value for the `SO_KEEPALIVE` option on this socket.\n pub fn set_keepalive(&self, keepalive: bool) -> io::Result<()> {\n self.inner.set_keepalive(keepalive)\n }\n\n /// Gets the value of the `SO_KEEPALIVE` option on this socket.\n pub fn keepalive(&self) -> io::Result {\n self.inner.keepalive()\n }\n\n /// Allows the socket to bind to an in-use address.\n ///\n /// Behavior is platform specific. Refer to the target platform's\n /// documentation for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.set_reuseaddr(true)?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// # drop(listener);\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn set_reuseaddr(&self, reuseaddr: bool) -> io::Result<()> {\n self.inner.set_reuse_address(reuseaddr)\n }\n\n /// Retrieves the value set for `SO_REUSEADDR` on this socket.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.set_reuseaddr(true)?;\n /// assert!(socket.reuseaddr().unwrap());\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// Ok(())\n /// }\n /// ```\n pub fn reuseaddr(&self) -> io::Result {\n self.inner.reuse_address()\n }\n\n /// Allows the socket to bind to an in-use port. Only available for unix systems\n /// (excluding Solaris, Illumos, and Cygwin).\n ///\n /// Behavior is platform specific. Refer to the target platform's\n /// documentation for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.set_reuseport(true)?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// Ok(())\n /// }\n /// ```\n #[cfg(all(\n unix,\n not(target_os = \"solaris\"),\n not(target_os = \"illumos\"),\n not(target_os = \"cygwin\"),\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(all(\n unix,\n not(target_os = \"solaris\"),\n not(target_os = \"illumos\"),\n not(target_os = \"cygwin\"),\n )))\n )]\n pub fn set_reuseport(&self, reuseport: bool) -> io::Result<()> {\n self.inner.set_reuse_port(reuseport)\n }\n\n /// Allows the socket to bind to an in-use port. Only available for unix systems\n /// (excluding Solaris, Illumos, and Cygwin).\n ///\n /// Behavior is platform specific. Refer to the target platform's\n /// documentation for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.set_reuseport(true)?;\n /// assert!(socket.reuseport().unwrap());\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// Ok(())\n /// }\n /// ```\n #[cfg(all(\n unix,\n not(target_os = \"solaris\"),\n not(target_os = \"illumos\"),\n not(target_os = \"cygwin\"),\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(all(\n unix,\n not(target_os = \"solaris\"),\n not(target_os = \"illumos\"),\n not(target_os = \"cygwin\"),\n )))\n )]\n pub fn reuseport(&self) -> io::Result {\n self.inner.reuse_port()\n }\n\n /// Sets the size of the TCP send buffer on this socket.\n ///\n /// On most operating systems, this sets the `SO_SNDBUF` socket option.\n pub fn set_send_buffer_size(&self, size: u32) -> io::Result<()> {\n self.inner.set_send_buffer_size(size as usize)\n }\n\n /// Returns the size of the TCP send buffer for this socket.\n ///\n /// On most operating systems, this is the value of the `SO_SNDBUF` socket\n /// option.\n ///\n /// Note that if [`set_send_buffer_size`] has been called on this socket\n /// previously, the value returned by this function may not be the same as\n /// the argument provided to `set_send_buffer_size`. This is for the\n /// following reasons:\n ///\n /// * Most operating systems have minimum and maximum allowed sizes for the\n /// send buffer, and will clamp the provided value if it is below the\n /// minimum or above the maximum. The minimum and maximum buffer sizes are\n /// OS-dependent.\n /// * Linux will double the buffer size to account for internal bookkeeping\n /// data, and returns the doubled value from `getsockopt(2)`. As per `man\n /// 7 socket`:\n /// > Sets or gets the maximum socket send buffer in bytes. The\n /// > kernel doubles this value (to allow space for bookkeeping\n /// > overhead) when it is set using `setsockopt(2)`, and this doubled\n /// > value is returned by `getsockopt(2)`.\n ///\n /// [`set_send_buffer_size`]: #method.set_send_buffer_size\n pub fn send_buffer_size(&self) -> io::Result {\n self.inner.send_buffer_size().map(|n| n as u32)\n }\n\n /// Sets the size of the TCP receive buffer on this socket.\n ///\n /// On most operating systems, this sets the `SO_RCVBUF` socket option.\n pub fn set_recv_buffer_size(&self, size: u32) -> io::Result<()> {\n self.inner.set_recv_buffer_size(size as usize)\n }\n\n /// Returns the size of the TCP receive buffer for this socket.\n ///\n /// On most operating systems, this is the value of the `SO_RCVBUF` socket\n /// option.\n ///\n /// Note that if [`set_recv_buffer_size`] has been called on this socket\n /// previously, the value returned by this function may not be the same as\n /// the argument provided to `set_recv_buffer_size`. This is for the\n /// following reasons:\n ///\n /// * Most operating systems have minimum and maximum allowed sizes for the\n /// receive buffer, and will clamp the provided value if it is below the\n /// minimum or above the maximum. The minimum and maximum buffer sizes are\n /// OS-dependent.\n /// * Linux will double the buffer size to account for internal bookkeeping\n /// data, and returns the doubled value from `getsockopt(2)`. As per `man\n /// 7 socket`:\n /// > Sets or gets the maximum socket send buffer in bytes. The\n /// > kernel doubles this value (to allow space for bookkeeping\n /// > overhead) when it is set using `setsockopt(2)`, and this doubled\n /// > value is returned by `getsockopt(2)`.\n ///\n /// [`set_recv_buffer_size`]: #method.set_recv_buffer_size\n pub fn recv_buffer_size(&self) -> io::Result {\n self.inner.recv_buffer_size().map(|n| n as u32)\n }\n\n /// Sets the linger duration of this socket by setting the `SO_LINGER` option.\n ///\n /// This option controls the action taken when a stream has unsent messages and the stream is\n /// closed. If `SO_LINGER` is set, the system shall block the process until it can transmit the\n /// data or until the time expires.\n ///\n /// If `SO_LINGER` is not specified, and the socket is closed, the system handles the call in a\n /// way that allows the process to continue as quickly as possible.\n ///\n /// This option is deprecated because setting `SO_LINGER` on a socket used with Tokio is always\n /// incorrect as it leads to blocking the thread when the socket is closed. For more details,\n /// please see:\n ///\n /// > Volumes of communications have been devoted to the intricacies of `SO_LINGER` versus\n /// > non-blocking (`O_NONBLOCK`) sockets. From what I can tell, the final word is: don't do\n /// > it. Rely on the `shutdown()`-followed-by-`read()`-eof technique instead.\n /// >\n /// > From [The ultimate `SO_LINGER` page, or: why is my tcp not reliable](https://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-so_linger-page-or-why-is-my-tcp-not-reliable)\n ///\n /// Although this method is deprecated, it will not be removed from Tokio.\n ///\n /// Note that the special case of setting `SO_LINGER` to zero does not lead to blocking. Tokio\n /// provides [`set_zero_linger`](Self::set_zero_linger) for this purpose.\n #[deprecated = \"`SO_LINGER` causes the socket to block the thread on drop\"]\n pub fn set_linger(&self, dur: Option) -> io::Result<()> {\n self.inner.set_linger(dur)\n }\n\n /// Sets a linger duration of zero on this socket by setting the `SO_LINGER` option.\n ///\n /// This causes the connection to be forcefully aborted (\"abortive close\") when the socket is\n /// dropped or closed. Instead of the normal TCP shutdown handshake (`FIN`/`ACK`), a TCP `RST`\n /// (reset) segment is sent to the peer, and the socket immediately discards any unsent data\n /// residing in the socket send buffer. This prevents the socket from entering the `TIME_WAIT`\n /// state after closing it.\n ///\n /// This is a destructive action. Any data currently buffered by the OS but not yet transmitted\n /// will be lost. The peer will likely receive a \"Connection Reset\" error rather than a clean\n /// end-of-stream.\n ///\n /// See the documentation for [`set_linger`](Self::set_linger) for additional details on how\n /// `SO_LINGER` works.\n pub fn set_zero_linger(&self) -> io::Result<()> {\n self.inner.set_linger(Some(Duration::ZERO))\n }\n\n /// Reads the linger duration for this socket by getting the `SO_LINGER`\n /// option.\n ///\n /// For more information about this option, see [`set_zero_linger`] and [`set_linger`].\n ///\n /// [`set_linger`]: TcpSocket::set_linger\n /// [`set_zero_linger`]: TcpSocket::set_zero_linger\n pub fn linger(&self) -> io::Result> {\n self.inner.linger()\n }\n\n /// Sets the value of the `TCP_NODELAY` option on this socket.\n ///\n /// If set, this option disables the Nagle algorithm. This means that segments are always\n /// sent as soon as possible, even if there is only a small amount of data. When not set,\n /// data is buffered until there is a sufficient amount to send out, thereby avoiding\n /// the frequent sending of small packets.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let socket = TcpSocket::new_v4()?;\n ///\n /// socket.set_nodelay(true)?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_nodelay(&self, nodelay: bool) -> io::Result<()> {\n self.inner.set_tcp_nodelay(nodelay)\n }\n\n /// Gets the value of the `TCP_NODELAY` option on this socket.\n ///\n /// For more information about this option, see [`set_nodelay`].\n ///\n /// [`set_nodelay`]: TcpSocket::set_nodelay\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let socket = TcpSocket::new_v4()?;\n ///\n /// println!(\"{:?}\", socket.nodelay()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn nodelay(&self) -> io::Result {\n self.inner.tcp_nodelay()\n }\n\n /// Gets the value of the `IPV6_TCLASS` option for this socket.\n ///\n /// For more information about this option, see [`set_tclass_v6`].\n ///\n /// [`set_tclass_v6`]: Self::set_tclass_v6\n // https://docs.rs/socket2/0.6.1/src/socket2/sys/unix.rs.html#2541\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n )))\n )]\n pub fn tclass_v6(&self) -> io::Result {\n self.inner.tclass_v6()\n }\n\n /// Sets the value for the `IPV6_TCLASS` option on this socket.\n ///\n /// Specifies the traffic class field that is used in every packet\n /// sent from this socket.\n ///\n /// # Note\n ///\n /// This may not have any effect on IPv4 sockets.\n // https://docs.rs/socket2/0.6.1/src/socket2/sys/unix.rs.html#2566\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n )))\n )]\n pub fn set_tclass_v6(&self, tclass: u32) -> io::Result<()> {\n self.inner.set_tclass_v6(tclass)\n }\n\n /// Gets the value of the `IP_TOS` option for this socket.\n ///\n /// For more information about this option, see [`set_tos_v4`].\n ///\n /// [`set_tos_v4`]: Self::set_tos_v4\n // https://docs.rs/socket2/0.6.1/src/socket2/socket.rs.html#1585\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn tos_v4(&self) -> io::Result {\n self.inner.tos_v4()\n }\n\n /// Deprecated. Use [`tos_v4()`] instead.\n ///\n /// [`tos_v4()`]: Self::tos_v4\n #[deprecated(\n note = \"`tos` related methods have been renamed `tos_v4` since they are IPv4-specific.\"\n )]\n #[doc(hidden)]\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn tos(&self) -> io::Result {\n self.tos_v4()\n }\n\n /// Sets the value for the `IP_TOS` option on this socket.\n ///\n /// This value sets the type-of-service field that is used in every packet\n /// sent from this socket.\n ///\n /// # Note\n ///\n /// - This may not have any effect on IPv6 sockets.\n /// - On Windows, `IP_TOS` is only supported on [Windows 8+ or\n /// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)\n // https://docs.rs/socket2/0.6.1/src/socket2/socket.rs.html#1566\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn set_tos_v4(&self, tos: u32) -> io::Result<()> {\n self.inner.set_tos_v4(tos)\n }\n\n /// Deprecated. Use [`set_tos_v4()`] instead.\n ///\n /// [`set_tos_v4()`]: Self::set_tos_v4\n #[deprecated(\n note = \"`tos` related methods have been renamed `tos_v4` since they are IPv4-specific.\"\n )]\n #[doc(hidden)]\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn set_tos(&self, tos: u32) -> io::Result<()> {\n self.set_tos_v4(tos)\n }\n\n /// Gets the value for the `SO_BINDTODEVICE` option on this socket\n ///\n /// This value gets the socket binded device's interface name.\n #[cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\",))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\",)))\n )]\n pub fn device(&self) -> io::Result>> {\n self.inner.device()\n }\n\n /// Sets the value for the `SO_BINDTODEVICE` option on this socket\n ///\n /// If a socket is bound to an interface, only packets received from that\n /// particular interface are processed by the socket. Note that this only\n /// works for some socket types, particularly `AF_INET` sockets.\n ///\n /// If `interface` is `None` or an empty string it removes the binding.\n #[cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\"))]\n #[cfg_attr(\n docsrs,\n doc(cfg(all(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\"))))\n )]\n pub fn bind_device(&self, interface: Option<&[u8]>) -> io::Result<()> {\n self.inner.bind_device(interface)\n }\n\n /// Gets the local address of this socket.\n ///\n /// Will fail on windows if called before `bind`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.bind(addr)?;\n /// assert_eq!(socket.local_addr().unwrap().to_string(), \"127.0.0.1:8080\");\n /// let listener = socket.listen(1024)?;\n /// Ok(())\n /// }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.inner.local_addr().and_then(convert_address)\n }\n\n /// Returns the value of the `SO_ERROR` option.\n pub fn take_error(&self) -> io::Result> {\n self.inner.take_error()\n }\n\n /// Binds the socket to the given address.\n ///\n /// This calls the `bind(2)` operating-system function. Behavior is\n /// platform specific. Refer to the target platform's documentation for more\n /// details.\n ///\n /// # Examples\n ///\n /// Bind a socket before listening.\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// # drop(listener);\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn bind(&self, addr: SocketAddr) -> io::Result<()> {\n self.inner.bind(&addr.into())\n }\n\n /// Establishes a TCP connection with a peer at the specified socket address.\n ///\n /// The `TcpSocket` is consumed. Once the connection is established, a\n /// connected [`TcpStream`] is returned. If the connection fails, the\n /// encountered error is returned.\n ///\n /// [`TcpStream`]: TcpStream\n ///\n /// This calls the `connect(2)` operating-system function. Behavior is\n /// platform specific. Refer to the target platform's documentation for more\n /// details.\n ///\n /// # Examples\n ///\n /// Connecting to a peer.\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// let stream = socket.connect(addr).await?;\n /// # drop(stream);\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn connect(self, addr: SocketAddr) -> io::Result {\n if let Err(err) = self.inner.connect(&addr.into()) {\n #[cfg(unix)]\n if err.raw_os_error() != Some(libc::EINPROGRESS) {\n return Err(err);\n }\n #[cfg(windows)]\n if err.kind() != io::ErrorKind::WouldBlock {\n return Err(err);\n }\n }\n #[cfg(unix)]\n let mio = {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = self.inner.into_raw_fd();\n unsafe { mio::net::TcpStream::from_raw_fd(raw_fd) }\n };\n\n #[cfg(windows)]\n let mio = {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n\n let raw_socket = self.inner.into_raw_socket();\n unsafe { mio::net::TcpStream::from_raw_socket(raw_socket) }\n };\n\n TcpStream::connect_mio(mio).await\n }\n\n /// Converts the socket into a `TcpListener`.\n ///\n /// `backlog` defines the maximum number of pending connections are queued\n /// by the operating system at any given time. Connection are removed from\n /// the queue with [`TcpListener::accept`]. When the queue is full, the\n /// operating-system will start rejecting connections.\n ///\n /// [`TcpListener::accept`]: TcpListener::accept\n ///\n /// This calls the `listen(2)` operating-system function, marking the socket\n /// as a passive socket. Behavior is platform specific. Refer to the target\n /// platform's documentation for more details.\n ///\n /// # Examples\n ///\n /// Create a `TcpListener`.\n ///\n /// ```no_run\n /// use tokio::net::TcpSocket;\n ///\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let addr = \"127.0.0.1:8080\".parse().unwrap();\n ///\n /// let socket = TcpSocket::new_v4()?;\n /// socket.bind(addr)?;\n ///\n /// let listener = socket.listen(1024)?;\n /// # drop(listener);\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn listen(self, backlog: u32) -> io::Result {\n self.inner.listen(backlog as i32)?;\n #[cfg(unix)]\n let mio = {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = self.inner.into_raw_fd();\n unsafe { mio::net::TcpListener::from_raw_fd(raw_fd) }\n };\n\n #[cfg(windows)]\n let mio = {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n\n let raw_socket = self.inner.into_raw_socket();\n unsafe { mio::net::TcpListener::from_raw_socket(raw_socket) }\n };\n\n TcpListener::new(mio)\n }\n\n /// Converts a [`std::net::TcpStream`] into a `TcpSocket`. The provided\n /// socket must not have been connected prior to calling this function. This\n /// function is typically used together with crates such as [`socket2`] to\n /// configure socket options that are not available on `TcpSocket`.\n ///\n /// [`std::net::TcpStream`]: struct@std::net::TcpStream\n /// [`socket2`]: https://docs.rs/socket2/\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the socket is in\n /// non-blocking mode. Otherwise all I/O operations on the socket\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// [`set_nonblocking`]: std::net::TcpStream::set_nonblocking\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::net::TcpSocket;\n /// use socket2::{Domain, Socket, Type};\n ///\n /// #[tokio::main]\n /// async fn main() -> std::io::Result<()> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let socket2_socket = Socket::new(Domain::IPV4, Type::STREAM, None)?;\n /// socket2_socket.set_nonblocking(true)?;\n ///\n /// let socket = TcpSocket::from_std_stream(socket2_socket.into());\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn from_std_stream(std_stream: std::net::TcpStream) -> TcpSocket {\n #[cfg(unix)]\n {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = std_stream.into_raw_fd();\n unsafe { TcpSocket::from_raw_fd(raw_fd) }\n }\n\n #[cfg(windows)]\n {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n\n let raw_socket = std_stream.into_raw_socket();\n unsafe { TcpSocket::from_raw_socket(raw_socket) }\n }\n }\n}\n\nfn convert_address(address: socket2::SockAddr) -> io::Result {\n match address.as_socket() {\n Some(address) => Ok(address),\n None => Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n \"invalid address family (not IPv4 or IPv6)\",\n )),\n }\n}\n\nimpl fmt::Debug for TcpSocket {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.inner.fmt(fmt)\n }\n}\n\n// These trait implementations can't be build on Windows, so we completely\n// ignore them, even when building documentation.\n#[cfg(unix)]\ncfg_unix! {\n impl AsRawFd for TcpSocket {\n fn as_raw_fd(&self) -> RawFd {\n self.inner.as_raw_fd()\n }\n }\n\n impl AsFd for TcpSocket {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n\n impl FromRawFd for TcpSocket {\n /// Converts a `RawFd` to a `TcpSocket`.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the socket is in\n /// non-blocking mode.\n unsafe fn from_raw_fd(fd: RawFd) -> TcpSocket {\n // Safety: exactly the same safety requirements as the\n // `FromRawFd::from_raw_fd` trait method.\n let inner = unsafe { socket2::Socket::from_raw_fd(fd) };\n TcpSocket { inner }\n }\n }\n\n impl IntoRawFd for TcpSocket {\n fn into_raw_fd(self) -> RawFd {\n self.inner.into_raw_fd()\n }\n }\n}\n\ncfg_windows! {\n impl IntoRawSocket for TcpSocket {\n fn into_raw_socket(self) -> RawSocket {\n self.inner.into_raw_socket()\n }\n }\n\n impl AsRawSocket for TcpSocket {\n fn as_raw_socket(&self) -> RawSocket {\n self.inner.as_raw_socket()\n }\n }\n\n impl AsSocket for TcpSocket {\n fn as_socket(&self) -> BorrowedSocket<'_> {\n unsafe { BorrowedSocket::borrow_raw(self.as_raw_socket()) }\n }\n }\n\n impl FromRawSocket for TcpSocket {\n /// Converts a `RawSocket` to a `TcpStream`.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the socket is in\n /// non-blocking mode.\n unsafe fn from_raw_socket(socket: RawSocket) -> TcpSocket {\n let inner = unsafe { socket2::Socket::from_raw_socket(socket) };\n TcpSocket { inner }\n }\n }\n}\n" }, { "path": "tokio/src/net/tcp/split.rs", "content": "//! `TcpStream` split support.\n//!\n//! A `TcpStream` can be split into a `ReadHalf` and a\n//! `WriteHalf` with the `TcpStream::split` method. `ReadHalf`\n//! implements `AsyncRead` while `WriteHalf` implements `AsyncWrite`.\n//!\n//! Compared to the generic split of `AsyncRead + AsyncWrite`, this specialized\n//! split has no associated overhead and enforces all invariants at the type\n//! level.\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, ReadBuf, Ready};\nuse crate::net::TcpStream;\n\nuse std::future::poll_fn;\nuse std::io;\nuse std::net::{Shutdown, SocketAddr};\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n/// Borrowed read half of a [`TcpStream`], created by [`split`].\n///\n/// Reading from a `ReadHalf` is usually done using the convenience methods found on the\n/// [`AsyncReadExt`] trait.\n///\n/// [`TcpStream`]: TcpStream\n/// [`split`]: TcpStream::split()\n/// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n#[derive(Debug)]\npub struct ReadHalf<'a>(&'a TcpStream);\n\n/// Borrowed write half of a [`TcpStream`], created by [`split`].\n///\n/// Note that in the [`AsyncWrite`] implementation of this type, [`poll_shutdown`] will\n/// shut down the TCP stream in the write direction.\n///\n/// Writing to an `WriteHalf` is usually done using the convenience methods found\n/// on the [`AsyncWriteExt`] trait.\n///\n/// [`TcpStream`]: TcpStream\n/// [`split`]: TcpStream::split()\n/// [`AsyncWrite`]: trait@crate::io::AsyncWrite\n/// [`poll_shutdown`]: fn@crate::io::AsyncWrite::poll_shutdown\n/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n#[derive(Debug)]\npub struct WriteHalf<'a>(&'a TcpStream);\n\npub(crate) fn split(stream: &mut TcpStream) -> (ReadHalf<'_>, WriteHalf<'_>) {\n (ReadHalf(&*stream), WriteHalf(&*stream))\n}\n\nimpl ReadHalf<'_> {\n /// Attempts to receive data on the socket, without removing that data from\n /// the queue, registering the current task for wakeup if data is not yet\n /// available.\n ///\n /// Note that on multiple calls to `poll_peek` or `poll_read`, only the\n /// `Waker` from the `Context` passed to the most recent call is scheduled\n /// to receive a wakeup.\n ///\n /// See the [`TcpStream::poll_peek`] level documentation for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::io::{self, ReadBuf};\n /// use tokio::net::TcpStream;\n ///\n /// use std::future::poll_fn;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let mut stream = TcpStream::connect(\"127.0.0.1:8000\").await?;\n /// let (mut read_half, _) = stream.split();\n /// let mut buf = [0; 10];\n /// let mut buf = ReadBuf::new(&mut buf);\n ///\n /// poll_fn(|cx| {\n /// read_half.poll_peek(cx, &mut buf)\n /// }).await?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`TcpStream::poll_peek`]: TcpStream::poll_peek\n pub fn poll_peek(\n &mut self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.0.poll_peek(cx, buf)\n }\n\n /// Receives data on the socket from the remote address to which it is\n /// connected, without removing that data from the queue. On success,\n /// returns the number of bytes peeked.\n ///\n /// See the [`TcpStream::peek`] level documentation for more details.\n ///\n /// [`TcpStream::peek`]: TcpStream::peek\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use tokio::io::AsyncReadExt;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let mut stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n /// let (mut read_half, _) = stream.split();\n ///\n /// let mut b1 = [0; 10];\n /// let mut b2 = [0; 10];\n ///\n /// // Peek at the data\n /// let n = read_half.peek(&mut b1).await?;\n ///\n /// // Read the data\n /// assert_eq!(n, read_half.read(&mut b2[..n]).await?);\n /// assert_eq!(&b1[..n], &b2[..n]);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// The [`read`] method is defined on the [`AsyncReadExt`] trait.\n ///\n /// [`read`]: fn@crate::io::AsyncReadExt::read\n /// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n pub async fn peek(&mut self, buf: &mut [u8]) -> io::Result {\n let mut buf = ReadBuf::new(buf);\n poll_fn(|cx| self.poll_peek(cx, &mut buf)).await\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_read()`]. It can be used instead\n /// of [`readable()`] to check the returned ready set for [`Ready::READABLE`]\n /// and [`Ready::READ_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`TcpStream::ready`].\n ///\n /// [`try_read()`]: Self::try_read\n /// [`readable()`]: Self::readable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.0.ready(interest).await\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// This function is also equivalent to [`TcpStream::ready`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn readable(&self) -> io::Result<()> {\n self.0.readable().await\n }\n\n /// Tries to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.0.try_read(buf)\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: Self::try_read()\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.0.try_read_vectored(bufs)\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.0.try_read_buf(buf)\n }\n }\n\n /// Returns the remote address that this stream is connected to.\n pub fn peer_addr(&self) -> io::Result {\n self.0.peer_addr()\n }\n\n /// Returns the local address that this stream is bound to.\n pub fn local_addr(&self) -> io::Result {\n self.0.local_addr()\n }\n}\n\nimpl WriteHalf<'_> {\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_write()`]. It can be used instead\n /// of [`writable()`] to check the returned ready set for [`Ready::WRITABLE`]\n /// and [`Ready::WRITE_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`TcpStream::ready`].\n ///\n /// [`try_write()`]: Self::try_write\n /// [`writable()`]: Self::writable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.0.ready(interest).await\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn writable(&self) -> io::Result<()> {\n self.0.writable().await\n }\n\n /// Tries to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.0.try_write(buf)\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: Self::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write_vectored(&self, bufs: &[io::IoSlice<'_>]) -> io::Result {\n self.0.try_write_vectored(bufs)\n }\n\n /// Returns the remote address that this stream is connected to.\n pub fn peer_addr(&self) -> io::Result {\n self.0.peer_addr()\n }\n\n /// Returns the local address that this stream is bound to.\n pub fn local_addr(&self) -> io::Result {\n self.0.local_addr()\n }\n}\n\nimpl AsyncRead for ReadHalf<'_> {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.0.poll_read_priv(cx, buf)\n }\n}\n\nimpl AsyncWrite for WriteHalf<'_> {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.0.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.0.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.0.is_write_vectored()\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n // tcp flush is a no-op\n Poll::Ready(Ok(()))\n }\n\n // `poll_shutdown` on a write half shutdowns the stream in the \"write\" direction.\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n self.0.shutdown_std(Shutdown::Write).into()\n }\n}\n\nimpl AsRef for ReadHalf<'_> {\n fn as_ref(&self) -> &TcpStream {\n self.0\n }\n}\n\nimpl AsRef for WriteHalf<'_> {\n fn as_ref(&self) -> &TcpStream {\n self.0\n }\n}\n" }, { "path": "tokio/src/net/tcp/split_owned.rs", "content": "//! `TcpStream` owned split support.\n//!\n//! A `TcpStream` can be split into an `OwnedReadHalf` and a `OwnedWriteHalf`\n//! with the `TcpStream::into_split` method. `OwnedReadHalf` implements\n//! `AsyncRead` while `OwnedWriteHalf` implements `AsyncWrite`.\n//!\n//! Compared to the generic split of `AsyncRead + AsyncWrite`, this specialized\n//! split has no associated overhead and enforces all invariants at the type\n//! level.\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, ReadBuf, Ready};\nuse crate::net::TcpStream;\n\nuse std::error::Error;\nuse std::future::poll_fn;\nuse std::net::{Shutdown, SocketAddr};\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{Context, Poll};\nuse std::{fmt, io};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n/// Owned read half of a [`TcpStream`], created by [`into_split`].\n///\n/// Reading from an `OwnedReadHalf` is usually done using the convenience methods found\n/// on the [`AsyncReadExt`] trait.\n///\n/// [`TcpStream`]: TcpStream\n/// [`into_split`]: TcpStream::into_split()\n/// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n#[derive(Debug)]\npub struct OwnedReadHalf {\n inner: Arc,\n}\n\n/// Owned write half of a [`TcpStream`], created by [`into_split`].\n///\n/// Note that in the [`AsyncWrite`] implementation of this type, [`poll_shutdown`] will\n/// shut down the TCP stream in the write direction. Dropping the write half\n/// will also shut down the write half of the TCP stream.\n///\n/// Writing to an `OwnedWriteHalf` is usually done using the convenience methods found\n/// on the [`AsyncWriteExt`] trait.\n///\n/// [`TcpStream`]: TcpStream\n/// [`into_split`]: TcpStream::into_split()\n/// [`AsyncWrite`]: trait@crate::io::AsyncWrite\n/// [`poll_shutdown`]: fn@crate::io::AsyncWrite::poll_shutdown\n/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n#[derive(Debug)]\npub struct OwnedWriteHalf {\n inner: Arc,\n shutdown_on_drop: bool,\n}\n\npub(crate) fn split_owned(stream: TcpStream) -> (OwnedReadHalf, OwnedWriteHalf) {\n let arc = Arc::new(stream);\n let read = OwnedReadHalf {\n inner: Arc::clone(&arc),\n };\n let write = OwnedWriteHalf {\n inner: arc,\n shutdown_on_drop: true,\n };\n (read, write)\n}\n\npub(crate) fn reunite(\n read: OwnedReadHalf,\n write: OwnedWriteHalf,\n) -> Result {\n if Arc::ptr_eq(&read.inner, &write.inner) {\n write.forget();\n // This unwrap cannot fail as the api does not allow creating more than two Arcs,\n // and we just dropped the other half.\n Ok(Arc::try_unwrap(read.inner).expect(\"TcpStream: try_unwrap failed in reunite\"))\n } else {\n Err(ReuniteError(read, write))\n }\n}\n\n/// Error indicating that two halves were not from the same socket, and thus could\n/// not be reunited.\n#[derive(Debug)]\npub struct ReuniteError(pub OwnedReadHalf, pub OwnedWriteHalf);\n\nimpl fmt::Display for ReuniteError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(\n f,\n \"tried to reunite halves that are not from the same socket\"\n )\n }\n}\n\nimpl Error for ReuniteError {}\n\nimpl OwnedReadHalf {\n /// Attempts to put the two halves of a `TcpStream` back together and\n /// recover the original socket. Succeeds only if the two halves\n /// originated from the same call to [`into_split`].\n ///\n /// [`into_split`]: TcpStream::into_split()\n pub fn reunite(self, other: OwnedWriteHalf) -> Result {\n reunite(self, other)\n }\n\n /// Attempt to receive data on the socket, without removing that data from\n /// the queue, registering the current task for wakeup if data is not yet\n /// available.\n ///\n /// Note that on multiple calls to `poll_peek` or `poll_read`, only the\n /// `Waker` from the `Context` passed to the most recent call is scheduled\n /// to receive a wakeup.\n ///\n /// See the [`TcpStream::poll_peek`] level documentation for more details.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::io::{self, ReadBuf};\n /// use tokio::net::TcpStream;\n ///\n /// use std::future::poll_fn;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8000\").await?;\n /// let (mut read_half, _) = stream.into_split();\n /// let mut buf = [0; 10];\n /// let mut buf = ReadBuf::new(&mut buf);\n ///\n /// poll_fn(|cx| {\n /// read_half.poll_peek(cx, &mut buf)\n /// }).await?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`TcpStream::poll_peek`]: TcpStream::poll_peek\n pub fn poll_peek(\n &mut self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.inner.poll_peek(cx, buf)\n }\n\n /// Receives data on the socket from the remote address to which it is\n /// connected, without removing that data from the queue. On success,\n /// returns the number of bytes peeked.\n ///\n /// See the [`TcpStream::peek`] level documentation for more details.\n ///\n /// [`TcpStream::peek`]: TcpStream::peek\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use tokio::io::AsyncReadExt;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n /// let (mut read_half, _) = stream.into_split();\n ///\n /// let mut b1 = [0; 10];\n /// let mut b2 = [0; 10];\n ///\n /// // Peek at the data\n /// let n = read_half.peek(&mut b1).await?;\n ///\n /// // Read the data\n /// assert_eq!(n, read_half.read(&mut b2[..n]).await?);\n /// assert_eq!(&b1[..n], &b2[..n]);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// The [`read`] method is defined on the [`AsyncReadExt`] trait.\n ///\n /// [`read`]: fn@crate::io::AsyncReadExt::read\n /// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n pub async fn peek(&mut self, buf: &mut [u8]) -> io::Result {\n let mut buf = ReadBuf::new(buf);\n poll_fn(|cx| self.poll_peek(cx, &mut buf)).await\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_read()`]. It can be used instead\n /// of [`readable()`] to check the returned ready set for [`Ready::READABLE`]\n /// and [`Ready::READ_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`TcpStream::ready`].\n ///\n /// [`try_read()`]: Self::try_read\n /// [`readable()`]: Self::readable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.inner.ready(interest).await\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// This function is also equivalent to [`TcpStream::ready`].\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn readable(&self) -> io::Result<()> {\n self.inner.readable().await\n }\n\n /// Tries to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.inner.try_read(buf)\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: Self::try_read()\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.inner.try_read_vectored(bufs)\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.inner.try_read_buf(buf)\n }\n }\n\n /// Returns the remote address that this stream is connected to.\n pub fn peer_addr(&self) -> io::Result {\n self.inner.peer_addr()\n }\n\n /// Returns the local address that this stream is bound to.\n pub fn local_addr(&self) -> io::Result {\n self.inner.local_addr()\n }\n}\n\nimpl AsyncRead for OwnedReadHalf {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.inner.poll_read_priv(cx, buf)\n }\n}\n\nimpl OwnedWriteHalf {\n /// Attempts to put the two halves of a `TcpStream` back together and\n /// recover the original socket. Succeeds only if the two halves\n /// originated from the same call to [`into_split`].\n ///\n /// [`into_split`]: TcpStream::into_split()\n pub fn reunite(self, other: OwnedReadHalf) -> Result {\n reunite(other, self)\n }\n\n /// Destroys the write half, but don't close the write half of the stream\n /// until the read half is dropped. If the read half has already been\n /// dropped, this closes the stream.\n pub fn forget(mut self) {\n self.shutdown_on_drop = false;\n drop(self);\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_write()`]. It can be used instead\n /// of [`writable()`] to check the returned ready set for [`Ready::WRITABLE`]\n /// and [`Ready::WRITE_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`TcpStream::ready`].\n ///\n /// [`try_write()`]: Self::try_write\n /// [`writable()`]: Self::writable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.inner.ready(interest).await\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn writable(&self) -> io::Result<()> {\n self.inner.writable().await\n }\n\n /// Tries to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.inner.try_write(buf)\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: Self::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write_vectored(&self, bufs: &[io::IoSlice<'_>]) -> io::Result {\n self.inner.try_write_vectored(bufs)\n }\n\n /// Returns the remote address that this stream is connected to.\n pub fn peer_addr(&self) -> io::Result {\n self.inner.peer_addr()\n }\n\n /// Returns the local address that this stream is bound to.\n pub fn local_addr(&self) -> io::Result {\n self.inner.local_addr()\n }\n}\n\nimpl Drop for OwnedWriteHalf {\n fn drop(&mut self) {\n if self.shutdown_on_drop {\n let _ = self.inner.shutdown_std(Shutdown::Write);\n }\n }\n}\n\nimpl AsyncWrite for OwnedWriteHalf {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.inner.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.inner.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.inner.is_write_vectored()\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n // tcp flush is a no-op\n Poll::Ready(Ok(()))\n }\n\n // `poll_shutdown` on a write half shutdowns the stream in the \"write\" direction.\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n let res = self.inner.shutdown_std(Shutdown::Write);\n if res.is_ok() {\n Pin::into_inner(self).shutdown_on_drop = false;\n }\n res.into()\n }\n}\n\nimpl AsRef for OwnedReadHalf {\n fn as_ref(&self) -> &TcpStream {\n &self.inner\n }\n}\n\nimpl AsRef for OwnedWriteHalf {\n fn as_ref(&self) -> &TcpStream {\n &self.inner\n }\n}\n" }, { "path": "tokio/src/net/tcp/stream.rs", "content": "cfg_not_wasi! {\n use crate::net::{to_socket_addrs, ToSocketAddrs};\n use std::future::poll_fn;\n use std::time::Duration;\n}\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, PollEvented, ReadBuf, Ready};\nuse crate::net::tcp::split::{split, ReadHalf, WriteHalf};\nuse crate::net::tcp::split_owned::{split_owned, OwnedReadHalf, OwnedWriteHalf};\nuse crate::util::check_socket_for_blocking;\n\nuse std::fmt;\nuse std::io;\nuse std::net::{Shutdown, SocketAddr};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\ncfg_net! {\n /// A TCP stream between a local and a remote socket.\n ///\n /// A TCP stream can either be created by connecting to an endpoint, via the\n /// [`connect`] method, or by [accepting] a connection from a [listener]. A\n /// TCP stream can also be created via the [`TcpSocket`] type.\n ///\n /// Reading and writing to a `TcpStream` is usually done using the\n /// convenience methods found on the [`AsyncReadExt`] and [`AsyncWriteExt`]\n /// traits.\n ///\n /// [`connect`]: method@TcpStream::connect\n /// [accepting]: method@crate::net::TcpListener::accept\n /// [listener]: struct@crate::net::TcpListener\n /// [`TcpSocket`]: struct@crate::net::TcpSocket\n /// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use tokio::io::AsyncWriteExt;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let mut stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// // Write some data.\n /// stream.write_all(b\"hello world!\").await?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n ///\n /// To shut down the stream in the write direction, you can call the\n /// [`shutdown()`] method. This will cause the other peer to receive a read of\n /// length 0, indicating that no more data will be sent. This only closes\n /// the stream in one direction.\n ///\n /// [`shutdown()`]: fn@crate::io::AsyncWriteExt::shutdown\n pub struct TcpStream {\n io: PollEvented,\n }\n}\n\nimpl TcpStream {\n cfg_not_wasi! {\n /// Opens a TCP connection to a remote host.\n ///\n /// `addr` is an address of the remote host. Anything which implements the\n /// [`ToSocketAddrs`] trait can be supplied as the address. If `addr`\n /// yields multiple addresses, connect will be attempted with each of the\n /// addresses until a connection is successful. If none of the addresses\n /// result in a successful connection, the error returned from the last\n /// connection attempt (the last address) is returned.\n ///\n /// To configure the socket before connecting, you can use the [`TcpSocket`]\n /// type.\n ///\n /// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs\n /// [`TcpSocket`]: struct@crate::net::TcpSocket\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use tokio::io::AsyncWriteExt;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let mut stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// // Write some data.\n /// stream.write_all(b\"hello world!\").await?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.\n ///\n /// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all\n /// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n pub async fn connect(addr: A) -> io::Result {\n let addrs = to_socket_addrs(addr).await?;\n\n let mut last_err = None;\n\n for addr in addrs {\n match TcpStream::connect_addr(addr).await {\n Ok(stream) => return Ok(stream),\n Err(e) => last_err = Some(e),\n }\n }\n\n Err(last_err.unwrap_or_else(|| {\n io::Error::new(\n io::ErrorKind::InvalidInput,\n \"could not resolve to any address\",\n )\n }))\n }\n\n /// Establishes a connection to the specified `addr`.\n async fn connect_addr(addr: SocketAddr) -> io::Result {\n let sys = mio::net::TcpStream::connect(addr)?;\n TcpStream::connect_mio(sys).await\n }\n\n pub(crate) async fn connect_mio(sys: mio::net::TcpStream) -> io::Result {\n let stream = TcpStream::new(sys)?;\n\n // Once we've connected, wait for the stream to be writable as\n // that's when the actual connection has been initiated. Once we're\n // writable we check for `take_socket_error` to see if the connect\n // actually hit an error or not.\n //\n // If all that succeeded then we ship everything on up.\n poll_fn(|cx| stream.io.registration().poll_write_ready(cx)).await?;\n\n if let Some(e) = stream.io.take_error()? {\n return Err(e);\n }\n\n Ok(stream)\n }\n }\n\n pub(crate) fn new(connected: mio::net::TcpStream) -> io::Result {\n let io = PollEvented::new(connected)?;\n Ok(TcpStream { io })\n }\n\n /// Creates new `TcpStream` from a `std::net::TcpStream`.\n ///\n /// This function is intended to be used to wrap a TCP stream from the\n /// standard library in the Tokio equivalent.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the stream is in\n /// non-blocking mode. Otherwise all I/O operations on the stream\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::net::TcpStream::set_nonblocking\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use std::error::Error;\n /// use tokio::net::TcpStream;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let std_stream = std::net::TcpStream::connect(\"127.0.0.1:34254\")?;\n /// std_stream.set_nonblocking(true)?;\n /// let stream = TcpStream::from_std(std_stream)?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n pub fn from_std(stream: std::net::TcpStream) -> io::Result {\n check_socket_for_blocking(&stream)?;\n\n let io = mio::net::TcpStream::from_std(stream);\n let io = PollEvented::new(io)?;\n Ok(TcpStream { io })\n }\n\n /// Turns a [`tokio::net::TcpStream`] into a [`std::net::TcpStream`].\n ///\n /// The returned [`std::net::TcpStream`] will have nonblocking mode set as `true`.\n /// Use [`set_nonblocking`] to change the blocking mode if needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::error::Error;\n /// use std::io::Read;\n /// use tokio::net::TcpListener;\n /// # use tokio::net::TcpStream;\n /// # use tokio::io::AsyncWriteExt;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let mut data = [0u8; 12];\n /// # if false {\n /// let listener = TcpListener::bind(\"127.0.0.1:34254\").await?;\n /// # }\n /// # let listener = TcpListener::bind(\"127.0.0.1:0\").await?;\n /// # let addr = listener.local_addr().unwrap();\n /// # let handle = tokio::spawn(async move {\n /// # let mut stream: TcpStream = TcpStream::connect(addr).await.unwrap();\n /// # stream.write_all(b\"Hello world!\").await.unwrap();\n /// # });\n /// let (tokio_tcp_stream, _) = listener.accept().await?;\n /// let mut std_tcp_stream = tokio_tcp_stream.into_std()?;\n /// # handle.await.expect(\"The task being joined has panicked\");\n /// std_tcp_stream.set_nonblocking(false)?;\n /// std_tcp_stream.read_exact(&mut data)?;\n /// # assert_eq!(b\"Hello world!\", &data);\n /// Ok(())\n /// }\n /// ```\n /// [`tokio::net::TcpStream`]: TcpStream\n /// [`std::net::TcpStream`]: std::net::TcpStream\n /// [`set_nonblocking`]: fn@std::net::TcpStream::set_nonblocking\n pub fn into_std(self) -> io::Result {\n #[cfg(unix)]\n {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { std::net::TcpStream::from_raw_fd(raw_fd) })\n }\n\n #[cfg(windows)]\n {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n self.io\n .into_inner()\n .map(|io| io.into_raw_socket())\n .map(|raw_socket| unsafe { std::net::TcpStream::from_raw_socket(raw_socket) })\n }\n\n #[cfg(target_os = \"wasi\")]\n {\n use std::os::wasi::io::{FromRawFd, IntoRawFd};\n self.io\n .into_inner()\n .map(|io| io.into_raw_fd())\n .map(|raw_fd| unsafe { std::net::TcpStream::from_raw_fd(raw_fd) })\n }\n }\n\n /// Returns the local address that this stream is bound to.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", stream.local_addr()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr()\n }\n\n /// Returns the value of the `SO_ERROR` option.\n pub fn take_error(&self) -> io::Result> {\n self.io.take_error()\n }\n\n /// Returns the remote address that this stream is connected to.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", stream.peer_addr()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn peer_addr(&self) -> io::Result {\n self.io.peer_addr()\n }\n\n /// Attempts to receive data on the socket, without removing that data from\n /// the queue, registering the current task for wakeup if data is not yet\n /// available.\n ///\n /// Note that on multiple calls to `poll_peek`, `poll_read` or\n /// `poll_read_ready`, only the `Waker` from the `Context` passed to the\n /// most recent call is scheduled to receive a wakeup. (However,\n /// `poll_write` retains a second, independent waker.)\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if data is not yet available.\n /// * `Poll::Ready(Ok(n))` if data is available. `n` is the number of bytes peeked.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::io::{self, ReadBuf};\n /// use tokio::net::TcpStream;\n ///\n /// use std::future::poll_fn;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8000\").await?;\n /// let mut buf = [0; 10];\n /// let mut buf = ReadBuf::new(&mut buf);\n ///\n /// poll_fn(|cx| {\n /// stream.poll_peek(cx, &mut buf)\n /// }).await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn poll_peek(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n loop {\n let ev = ready!(self.io.registration().poll_read_ready(cx))?;\n\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n match self.io.peek(b) {\n Ok(ret) => {\n unsafe { buf.assume_init(ret) };\n buf.advance(ret);\n return Poll::Ready(Ok(ret));\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.io.registration().clear_readiness(ev);\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_read()` or `try_write()`. It\n /// can be used to concurrently read / write to the same socket on a single\n /// task without splitting the socket.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to the stream on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use tokio::io::Interest;\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// let ready = stream.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n ///\n /// }\n ///\n /// if ready.is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read readiness.\n ///\n /// If the tcp stream is not currently ready for reading, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the tcp\n /// stream becomes ready for reading, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_read_ready`, `poll_read` or\n /// `poll_peek`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup. (However,\n /// `poll_write_ready` retains a second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the tcp stream is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the tcp stream is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: TcpStream::readable()\n /// [`ready()`]: TcpStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf = [0; 4096];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n use std::io::Read;\n\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read(buf))\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: TcpStream::try_read()\n /// [`readable()`]: TcpStream::readable()\n /// [`ready()`]: TcpStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io::{self, IoSliceMut};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf_a = [0; 512];\n /// let mut buf_b = [0; 1024];\n /// let mut bufs = [\n /// IoSliceMut::new(&mut buf_a),\n /// IoSliceMut::new(&mut buf_b),\n /// ];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read_vectored(&mut bufs) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n use std::io::Read;\n\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read_vectored(bufs))\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: TcpStream::readable()\n /// [`ready()`]: TcpStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(4096);\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read_buf(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n use std::io::Read;\n\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `TcpStream::read` to have filled up `n` bytes in the\n // buffer.\n let n = (&*self.io).read(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write readiness.\n ///\n /// If the tcp stream is not currently ready for writing, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the tcp\n /// stream becomes ready for writing, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_write_ready` or `poll_write`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_read_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the tcp stream is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the tcp stream is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Try to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n use std::io::Write;\n\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write(buf))\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possibly being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: TcpStream::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// let bufs = [io::IoSlice::new(b\"hello \"), io::IoSlice::new(b\"world\")];\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write_vectored(&bufs) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write_vectored(&self, bufs: &[io::IoSlice<'_>]) -> io::Result {\n use std::io::Write;\n\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write_vectored(bufs))\n }\n\n /// Tries to read or write from the socket using a user-provided IO operation.\n ///\n /// If the socket is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation on the socket by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// socket is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the socket is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `TcpStream` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: TcpStream::readable()\n /// [`writable()`]: TcpStream::writable()\n /// [`ready()`]: TcpStream::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .try_io(interest, || self.io.try_io(f))\n }\n\n /// Reads or writes from the socket using a user-provided IO operation.\n ///\n /// The readiness of the socket is awaited and when the socket is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the socket by manually calling the appropriate syscall.\n /// If the operation fails because the socket is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the socket readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `TcpStream` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .async_io(interest, || self.io.try_io(&mut f))\n .await\n }\n\n /// Receives data on the socket from the remote address to which it is\n /// connected, without removing that data from the queue. On success,\n /// returns the number of bytes peeked.\n ///\n /// Successive calls return the same data. This is accomplished by passing\n /// `MSG_PEEK` as a flag to the underlying `recv` system call.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If the method is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no peek was performed, and\n /// that `buf` has not been modified.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n /// use tokio::io::AsyncReadExt;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let mut stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// let mut b1 = [0; 10];\n /// let mut b2 = [0; 10];\n ///\n /// // Peek at the data\n /// let n = stream.peek(&mut b1).await?;\n ///\n /// // Read the data\n /// assert_eq!(n, stream.read(&mut b2[..n]).await?);\n /// assert_eq!(&b1[..n], &b2[..n]);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// The [`read`] method is defined on the [`AsyncReadExt`] trait.\n ///\n /// [`read`]: fn@crate::io::AsyncReadExt::read\n /// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n pub async fn peek(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.peek(buf))\n .await\n }\n\n /// Shuts down the read, write, or both halves of this connection.\n ///\n /// This function will cause all pending and future I/O on the specified\n /// portions to return immediately with an appropriate value (see the\n /// documentation of `Shutdown`).\n ///\n /// Remark: this function transforms `Err(std::io::ErrorKind::NotConnected)` to `Ok(())`.\n /// It does this to abstract away OS specific logic and to prevent a race condition between\n /// this function call and the OS closing this socket because of external events (e.g. TCP reset).\n /// See for more information.\n pub(super) fn shutdown_std(&self, how: Shutdown) -> io::Result<()> {\n match self.io.shutdown(how) {\n Err(err) if err.kind() == std::io::ErrorKind::NotConnected => Ok(()),\n result => result,\n }\n }\n\n /// Gets the value of the `TCP_NODELAY` option on this socket.\n ///\n /// For more information about this option, see [`set_nodelay`].\n ///\n /// [`set_nodelay`]: TcpStream::set_nodelay\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", stream.nodelay()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn nodelay(&self) -> io::Result {\n self.io.nodelay()\n }\n\n /// Sets the value of the `TCP_NODELAY` option on this socket.\n ///\n /// If set, this option disables the Nagle algorithm. This means that\n /// segments are always sent as soon as possible, even if there is only a\n /// small amount of data. When not set, data is buffered until there is a\n /// sufficient amount to send out, thereby avoiding the frequent sending of\n /// small packets.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.set_nodelay(true)?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_nodelay(&self, nodelay: bool) -> io::Result<()> {\n self.io.set_nodelay(nodelay)\n }\n\n /// Gets the value of the `TCP_QUICKACK` option on this socket.\n ///\n /// For more information about this option, see [`TcpStream::set_quickack`].\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.quickack()?;\n /// # Ok(())\n /// # }\n /// ```\n #[cfg(any(\n target_os = \"linux\",\n target_os = \"android\",\n target_os = \"fuchsia\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"linux\",\n target_os = \"android\",\n target_os = \"fuchsia\",\n target_os = \"cygwin\"\n )))\n )]\n pub fn quickack(&self) -> io::Result {\n socket2::SockRef::from(self).tcp_quickack()\n }\n\n /// Enable or disable `TCP_QUICKACK`.\n ///\n /// This flag causes Linux to eagerly send `ACK`s rather than delaying them.\n /// Linux may reset this flag after further operations on the socket.\n ///\n /// See [`man 7 tcp`](https://man7.org/linux/man-pages/man7/tcp.7.html) and\n /// [TCP delayed acknowledgment](https://en.wikipedia.org/wiki/TCP_delayed_acknowledgment)\n /// for more information.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.set_quickack(true)?;\n /// # Ok(())\n /// # }\n /// ```\n #[cfg(any(\n target_os = \"linux\",\n target_os = \"android\",\n target_os = \"fuchsia\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"linux\",\n target_os = \"android\",\n target_os = \"fuchsia\",\n target_os = \"cygwin\"\n )))\n )]\n pub fn set_quickack(&self, quickack: bool) -> io::Result<()> {\n socket2::SockRef::from(self).set_tcp_quickack(quickack)\n }\n\n cfg_not_wasi! {\n /// Reads the linger duration for this socket by getting the `SO_LINGER`\n /// option.\n ///\n /// For more information about this option, see [`set_zero_linger`] and [`set_linger`].\n ///\n /// [`set_linger`]: TcpStream::set_linger\n /// [`set_zero_linger`]: TcpStream::set_zero_linger\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", stream.linger()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn linger(&self) -> io::Result> {\n socket2::SockRef::from(self).linger()\n }\n\n /// Sets the linger duration of this socket by setting the `SO_LINGER` option.\n ///\n /// This option controls the action taken when a stream has unsent messages and the stream is\n /// closed. If `SO_LINGER` is set, the system shall block the process until it can transmit the\n /// data or until the time expires.\n ///\n /// If `SO_LINGER` is not specified, and the stream is closed, the system handles the call in a\n /// way that allows the process to continue as quickly as possible.\n ///\n /// This option is deprecated because setting `SO_LINGER` on a socket used with Tokio is\n /// always incorrect as it leads to blocking the thread when the socket is closed. For more\n /// details, please see:\n ///\n /// > Volumes of communications have been devoted to the intricacies of `SO_LINGER` versus\n /// > non-blocking (`O_NONBLOCK`) sockets. From what I can tell, the final word is: don't\n /// > do it. Rely on the `shutdown()`-followed-by-`read()`-eof technique instead.\n /// >\n /// > From [The ultimate `SO_LINGER` page, or: why is my tcp not reliable](https://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-so_linger-page-or-why-is-my-tcp-not-reliable)\n ///\n /// Although this method is deprecated, it will not be removed from Tokio.\n ///\n /// Note that the special case of setting `SO_LINGER` to zero does not lead to blocking.\n /// Tokio provides [`set_zero_linger`](Self::set_zero_linger) for this purpose.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #![allow(deprecated)]\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.set_linger(None)?;\n /// # Ok(())\n /// # }\n /// ```\n #[deprecated = \"`SO_LINGER` causes the socket to block the thread on drop\"]\n pub fn set_linger(&self, dur: Option) -> io::Result<()> {\n socket2::SockRef::from(self).set_linger(dur)\n }\n\n /// Sets a linger duration of zero on this socket by setting the `SO_LINGER` option.\n ///\n /// This causes the connection to be forcefully aborted (\"abortive close\") when the socket\n /// is dropped or closed. Instead of the normal TCP shutdown handshake (`FIN`/`ACK`), a TCP\n /// `RST` (reset) segment is sent to the peer, and the socket immediately discards any\n /// unsent data residing in the socket send buffer. This prevents the socket from entering\n /// the `TIME_WAIT` state after closing it.\n ///\n /// This is a destructive action. Any data currently buffered by the OS but not yet\n /// transmitted will be lost. The peer will likely receive a \"Connection Reset\" error\n /// rather than a clean end-of-stream.\n ///\n /// See the documentation for [`set_linger`](Self::set_linger) for additional details on\n /// how `SO_LINGER` works.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use std::time::Duration;\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.set_zero_linger()?;\n /// assert_eq!(stream.linger()?, Some(Duration::ZERO));\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_zero_linger(&self) -> io::Result<()> {\n socket2::SockRef::from(self).set_linger(Some(Duration::ZERO))\n }\n }\n\n /// Gets the value of the `IP_TTL` option for this socket.\n ///\n /// For more information about this option, see [`set_ttl`].\n ///\n /// [`set_ttl`]: TcpStream::set_ttl\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", stream.ttl()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn ttl(&self) -> io::Result {\n self.io.ttl()\n }\n\n /// Sets the value for the `IP_TTL` option on this socket.\n ///\n /// This value sets the time-to-live field that is used in every packet sent\n /// from this socket.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::TcpStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let stream = TcpStream::connect(\"127.0.0.1:8080\").await?;\n ///\n /// stream.set_ttl(123)?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_ttl(&self, ttl: u32) -> io::Result<()> {\n self.io.set_ttl(ttl)\n }\n\n // These lifetime markers also appear in the generated documentation, and make\n // it more clear that this is a *borrowed* split.\n #[allow(clippy::needless_lifetimes)]\n /// Splits a `TcpStream` into a read half and a write half, which can be used\n /// to read and write the stream concurrently.\n ///\n /// This method is more efficient than [`into_split`], but the halves cannot be\n /// moved into independently spawned tasks.\n ///\n /// [`into_split`]: TcpStream::into_split()\n pub fn split<'a>(&'a mut self) -> (ReadHalf<'a>, WriteHalf<'a>) {\n split(self)\n }\n\n /// Splits a `TcpStream` into a read half and a write half, which can be used\n /// to read and write the stream concurrently.\n ///\n /// Unlike [`split`], the owned halves can be moved to separate tasks, however\n /// this comes at the cost of a heap allocation.\n ///\n /// **Note:** Dropping the write half will shut down the write half of the TCP\n /// stream. This is equivalent to calling [`shutdown()`] on the `TcpStream`.\n ///\n /// [`split`]: TcpStream::split()\n /// [`shutdown()`]: fn@crate::io::AsyncWriteExt::shutdown\n pub fn into_split(self) -> (OwnedReadHalf, OwnedWriteHalf) {\n split_owned(self)\n }\n\n // == Poll IO functions that takes `&self` ==\n //\n // To read or write without mutable access to the `TcpStream`, combine the\n // `poll_read_ready` or `poll_write_ready` methods with the `try_read` or\n // `try_write` methods.\n\n pub(crate) fn poll_read_priv(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n // Safety: `TcpStream::read` correctly handles reads into uninitialized memory\n unsafe { self.io.poll_read(cx, buf) }\n }\n\n pub(super) fn poll_write_priv(\n &self,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.io.poll_write(cx, buf)\n }\n\n pub(super) fn poll_write_vectored_priv(\n &self,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.io.poll_write_vectored(cx, bufs)\n }\n}\n\nimpl TryFrom for TcpStream {\n type Error = io::Error;\n\n /// Consumes stream, returning the tokio I/O object.\n ///\n /// This is equivalent to\n /// [`TcpStream::from_std(stream)`](TcpStream::from_std).\n fn try_from(stream: std::net::TcpStream) -> Result {\n Self::from_std(stream)\n }\n}\n\n// ===== impl Read / Write =====\n\nimpl AsyncRead for TcpStream {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.poll_read_priv(cx, buf)\n }\n}\n\nimpl AsyncWrite for TcpStream {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n // tcp flush is a no-op\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n self.shutdown_std(std::net::Shutdown::Write)?;\n Poll::Ready(Ok(()))\n }\n}\n\nimpl fmt::Debug for TcpStream {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n // skip PollEvented noise\n (*self.io).fmt(f)\n }\n}\n\nimpl AsRef for TcpStream {\n fn as_ref(&self) -> &Self {\n self\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use super::TcpStream;\n use std::os::unix::prelude::*;\n\n impl AsRawFd for TcpStream {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n }\n\n impl AsFd for TcpStream {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawSocket, RawSocket, AsSocket, BorrowedSocket};\n\n impl AsRawSocket for TcpStream {\n fn as_raw_socket(&self) -> RawSocket {\n self.io.as_raw_socket()\n }\n }\n\n impl AsSocket for TcpStream {\n fn as_socket(&self) -> BorrowedSocket<'_> {\n unsafe { BorrowedSocket::borrow_raw(self.as_raw_socket()) }\n }\n }\n}\n\n#[cfg(all(tokio_unstable, target_os = \"wasi\"))]\nmod sys {\n use super::TcpStream;\n use std::os::wasi::prelude::*;\n\n impl AsRawFd for TcpStream {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n }\n\n impl AsFd for TcpStream {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n" }, { "path": "tokio/src/net/udp.rs", "content": "use crate::io::{Interest, PollEvented, ReadBuf, Ready};\nuse crate::net::{to_socket_addrs, ToSocketAddrs};\nuse crate::util::check_socket_for_blocking;\n\nuse std::fmt;\nuse std::io;\nuse std::net::{self, Ipv4Addr, Ipv6Addr, SocketAddr};\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\ncfg_net! {\n /// A UDP socket.\n ///\n /// UDP is \"connectionless\", unlike TCP. Meaning, regardless of what address you've bound to, a `UdpSocket`\n /// is free to communicate with many different remotes. In tokio there are basically two main ways to use `UdpSocket`:\n ///\n /// * one to many: [`bind`](`UdpSocket::bind`) and use [`send_to`](`UdpSocket::send_to`)\n /// and [`recv_from`](`UdpSocket::recv_from`) to communicate with many different addresses\n /// * one to one: [`connect`](`UdpSocket::connect`) and associate with a single address, using [`send`](`UdpSocket::send`)\n /// and [`recv`](`UdpSocket::recv`) to communicate only with that remote address\n ///\n /// This type does not provide a `split` method, because this functionality\n /// can be achieved by instead wrapping the socket in an [`Arc`]. Note that\n /// you do not need a `Mutex` to share the `UdpSocket` — an `Arc`\n /// is enough. This is because all of the methods take `&self` instead of\n /// `&mut self`. Once you have wrapped it in an `Arc`, you can call\n /// `.clone()` on the `Arc` to get multiple shared handles to the\n /// same socket. An example of such usage can be found further down.\n ///\n /// [`Arc`]: std::sync::Arc\n ///\n /// # Streams\n ///\n /// If you need to listen over UDP and produce a [`Stream`], you can look\n /// at [`UdpFramed`].\n ///\n /// [`UdpFramed`]: https://docs.rs/tokio-util/latest/tokio_util/udp/struct.UdpFramed.html\n /// [`Stream`]: https://docs.rs/futures/0.3/futures/stream/trait.Stream.html\n ///\n /// # Example: one to many (bind)\n ///\n /// Using `bind` we can create a simple echo server that sends and recv's with many different clients:\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"0.0.0.0:8080\").await?;\n /// let mut buf = [0; 1024];\n /// loop {\n /// let (len, addr) = sock.recv_from(&mut buf).await?;\n /// println!(\"{:?} bytes received from {:?}\", len, addr);\n ///\n /// let len = sock.send_to(&buf[..len], addr).await?;\n /// println!(\"{:?} bytes sent\", len);\n /// }\n /// }\n /// ```\n ///\n /// # Example: one to one (connect)\n ///\n /// Or using `connect` we can echo with a single remote address using `send` and `recv`:\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"0.0.0.0:8080\").await?;\n ///\n /// let remote_addr = \"127.0.0.1:59611\";\n /// sock.connect(remote_addr).await?;\n /// let mut buf = [0; 1024];\n /// loop {\n /// let len = sock.recv(&mut buf).await?;\n /// println!(\"{:?} bytes received from {:?}\", len, remote_addr);\n ///\n /// let len = sock.send(&buf[..len]).await?;\n /// println!(\"{:?} bytes sent\", len);\n /// }\n /// }\n /// ```\n ///\n /// # Example: Splitting with `Arc`\n ///\n /// Because `send_to` and `recv_from` take `&self`. It's perfectly alright\n /// to use an `Arc` and share the references to multiple tasks.\n /// Here is a similar \"echo\" example that supports concurrent\n /// sending/receiving:\n ///\n /// ```no_run\n /// use tokio::{net::UdpSocket, sync::mpsc};\n /// use std::{io, net::SocketAddr, sync::Arc};\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"0.0.0.0:8080\".parse::().unwrap()).await?;\n /// let r = Arc::new(sock);\n /// let s = r.clone();\n /// let (tx, mut rx) = mpsc::channel::<(Vec, SocketAddr)>(1_000);\n ///\n /// tokio::spawn(async move {\n /// while let Some((bytes, addr)) = rx.recv().await {\n /// let len = s.send_to(&bytes, &addr).await.unwrap();\n /// println!(\"{:?} bytes sent\", len);\n /// }\n /// });\n ///\n /// let mut buf = [0; 1024];\n /// loop {\n /// let (len, addr) = r.recv_from(&mut buf).await?;\n /// println!(\"{:?} bytes received from {:?}\", len, addr);\n /// tx.send((buf[..len].to_vec(), addr)).await.unwrap();\n /// }\n /// }\n /// ```\n ///\n pub struct UdpSocket {\n io: PollEvented,\n }\n}\n\nimpl UdpSocket {\n /// This function will create a new UDP socket and attempt to bind it to\n /// the `addr` provided.\n ///\n /// Binding with a port number of 0 will request that the OS assigns a port\n /// to this listener. The port allocated can be queried via the `local_addr`\n /// method.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let sock = UdpSocket::bind(\"0.0.0.0:8080\").await?;\n /// // use `sock`\n /// # let _ = sock;\n /// Ok(())\n /// }\n /// ```\n pub async fn bind(addr: A) -> io::Result {\n let addrs = to_socket_addrs(addr).await?;\n let mut last_err = None;\n\n for addr in addrs {\n match UdpSocket::bind_addr(addr) {\n Ok(socket) => return Ok(socket),\n Err(e) => last_err = Some(e),\n }\n }\n\n Err(last_err.unwrap_or_else(|| {\n io::Error::new(\n io::ErrorKind::InvalidInput,\n \"could not resolve to any address\",\n )\n }))\n }\n\n fn bind_addr(addr: SocketAddr) -> io::Result {\n let sys = mio::net::UdpSocket::bind(addr)?;\n UdpSocket::new(sys)\n }\n\n #[track_caller]\n fn new(socket: mio::net::UdpSocket) -> io::Result {\n let io = PollEvented::new(socket)?;\n Ok(UdpSocket { io })\n }\n\n /// Creates new `UdpSocket` from a previously bound `std::net::UdpSocket`.\n ///\n /// This function is intended to be used to wrap a UDP socket from the\n /// standard library in the Tokio equivalent.\n ///\n /// This can be used in conjunction with `socket2`'s `Socket` interface to\n /// configure a socket before it's handed off, such as setting options like\n /// `reuse_address` or binding to multiple addresses.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the socket is in\n /// non-blocking mode. Otherwise all I/O operations on the socket\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::net::UdpSocket::set_nonblocking\n ///\n /// # Panics\n ///\n /// This function panics if thread-local runtime is not set.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// # use std::{io, net::SocketAddr};\n ///\n /// # #[tokio::main]\n /// # async fn main() -> io::Result<()> {\n /// let addr = \"0.0.0.0:8080\".parse::().unwrap();\n /// let std_sock = std::net::UdpSocket::bind(addr)?;\n /// std_sock.set_nonblocking(true)?;\n /// let sock = UdpSocket::from_std(std_sock)?;\n /// // use `sock`\n /// # Ok(())\n /// # }\n /// ```\n #[track_caller]\n pub fn from_std(socket: net::UdpSocket) -> io::Result {\n check_socket_for_blocking(&socket)?;\n\n let io = mio::net::UdpSocket::from_std(socket);\n UdpSocket::new(io)\n }\n\n /// Turns a [`tokio::net::UdpSocket`] into a [`std::net::UdpSocket`].\n ///\n /// The returned [`std::net::UdpSocket`] will have nonblocking mode set as\n /// `true`. Use [`set_nonblocking`] to change the blocking mode if needed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let tokio_socket = tokio::net::UdpSocket::bind(\"127.0.0.1:0\").await?;\n /// let std_socket = tokio_socket.into_std()?;\n /// std_socket.set_nonblocking(false)?;\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`tokio::net::UdpSocket`]: UdpSocket\n /// [`std::net::UdpSocket`]: std::net::UdpSocket\n /// [`set_nonblocking`]: fn@std::net::UdpSocket::set_nonblocking\n pub fn into_std(self) -> io::Result {\n #[cfg(unix)]\n {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { std::net::UdpSocket::from_raw_fd(raw_fd) })\n }\n\n #[cfg(windows)]\n {\n use std::os::windows::io::{FromRawSocket, IntoRawSocket};\n self.io\n .into_inner()\n .map(|io| io.into_raw_socket())\n .map(|raw_socket| unsafe { std::net::UdpSocket::from_raw_socket(raw_socket) })\n }\n }\n\n fn as_socket(&self) -> socket2::SockRef<'_> {\n socket2::SockRef::from(self)\n }\n\n /// Returns the local address that this socket is bound to.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// # use std::{io, net::SocketAddr};\n ///\n /// # #[tokio::main]\n /// # async fn main() -> io::Result<()> {\n /// let addr = \"0.0.0.0:8080\".parse::().unwrap();\n /// let sock = UdpSocket::bind(addr).await?;\n /// // the address the socket is bound to\n /// let local_addr = sock.local_addr()?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr()\n }\n\n /// Returns the socket address of the remote peer this socket was connected to.\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::net::UdpSocket;\n ///\n /// # use std::{io, net::SocketAddr};\n /// # #[tokio::main]\n /// # async fn main() -> io::Result<()> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let addr = \"0.0.0.0:8080\".parse::().unwrap();\n /// let peer = \"127.0.0.1:11100\".parse::().unwrap();\n /// let sock = UdpSocket::bind(addr).await?;\n /// sock.connect(peer).await?;\n /// assert_eq!(peer, sock.peer_addr()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn peer_addr(&self) -> io::Result {\n self.io.peer_addr()\n }\n\n /// Connects the UDP socket setting the default destination for send() and\n /// limiting packets that are read via `recv` from the address specified in\n /// `addr`.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// # use std::{io, net::SocketAddr};\n ///\n /// # #[tokio::main]\n /// # async fn main() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"0.0.0.0:8080\".parse::().unwrap()).await?;\n ///\n /// let remote_addr = \"127.0.0.1:59600\".parse::().unwrap();\n /// sock.connect(remote_addr).await?;\n /// let mut buf = [0u8; 32];\n /// // recv from remote_addr\n /// let len = sock.recv(&mut buf).await?;\n /// // send to remote_addr\n /// let _len = sock.send(&buf[..len]).await?;\n /// # Ok(())\n /// # }\n /// ```\n pub async fn connect(&self, addr: A) -> io::Result<()> {\n let addrs = to_socket_addrs(addr).await?;\n let mut last_err = None;\n\n for addr in addrs {\n match self.io.connect(addr) {\n Ok(()) => return Ok(()),\n Err(e) => last_err = Some(e),\n }\n }\n\n Err(last_err.unwrap_or_else(|| {\n io::Error::new(\n io::ErrorKind::InvalidInput,\n \"could not resolve to any address\",\n )\n }))\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_recv()` or `try_send()`. It\n /// can be used to concurrently `recv` / `send` to the same socket on a single\n /// task without splitting the socket.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Concurrently receive from and send to the socket on the same task\n /// without splitting.\n ///\n /// ```no_run\n /// use tokio::io::{self, Interest};\n /// use tokio::net::UdpSocket;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// let ready = socket.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// // The buffer is **not** included in the async task and will only exist\n /// // on the stack.\n /// let mut data = [0; 1024];\n /// match socket.try_recv(&mut data[..]) {\n /// Ok(n) => {\n /// println!(\"received {:?}\", &data[..n]);\n /// }\n /// // False-positive, continue\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {}\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// if ready.is_writable() {\n /// // Write some data\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"sent {} bytes\", n);\n /// }\n /// // False-positive, continue\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {}\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is\n /// usually paired with `try_send()` or `try_send_to()`.\n ///\n /// The function may complete without the socket being writable. This is a\n /// false-positive and attempting a `try_send()` will return with\n /// `io::ErrorKind::WouldBlock`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Bind socket\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// socket.writable().await?;\n ///\n /// // Try to send data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write/send readiness.\n ///\n /// If the udp stream is not currently ready for sending, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the udp\n /// stream becomes ready for sending, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_send_ready` or `poll_send`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_recv_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the udp stream is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the udp stream is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_send_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Sends data on the socket to the remote address that the socket is\n /// connected to.\n ///\n /// The [`connect`] method will connect this socket to a remote address.\n /// This method will fail if the socket is not connected.\n ///\n /// [`connect`]: method@Self::connect\n ///\n /// # Return\n ///\n /// On success, the number of bytes sent is returned, otherwise, the\n /// encountered error is returned.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `send` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that the message was not sent.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::io;\n /// use tokio::net::UdpSocket;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Bind socket\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// // Send a message\n /// socket.send(b\"hello world\").await?;\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn send(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::WRITABLE, || self.io.send(buf))\n .await\n }\n\n /// Attempts to send data on the socket to the remote address to which it\n /// was previously `connect`ed.\n ///\n /// The [`connect`] method will connect this socket to a remote address.\n /// This method will fail if the socket is not connected.\n ///\n /// Note that on multiple calls to a `poll_*` method in the send direction,\n /// only the `Waker` from the `Context` passed to the most recent call will\n /// be scheduled to receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not available to write\n /// * `Poll::Ready(Ok(n))` `n` is the number of bytes sent\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`connect`]: method@Self::connect\n pub fn poll_send(&self, cx: &mut Context<'_>, buf: &[u8]) -> Poll> {\n self.io\n .registration()\n .poll_write_io(cx, || self.io.send(buf))\n }\n\n /// Tries to send data on the socket to the remote address to which it is\n /// connected.\n ///\n /// When the socket buffer is full, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `writable()`.\n ///\n /// # Returns\n ///\n /// If successful, `Ok(n)` is returned, where `n` is the number of bytes\n /// sent. If the socket is not ready to send data,\n /// `Err(ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Bind a UDP socket\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// // Connect to a peer\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// socket.writable().await?;\n ///\n /// // Try to send data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_send(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || self.io.send(buf))\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_recv()`.\n ///\n /// The function may complete without the socket being readable. This is a\n /// false-positive and attempting a `try_recv()` will return with\n /// `io::ErrorKind::WouldBlock`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read/receive readiness.\n ///\n /// If the udp stream is not currently ready for receiving, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the udp\n /// socket becomes ready for reading, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_recv_ready`, `poll_recv` or\n /// `poll_peek`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup. (However,\n /// `poll_send_ready` retains a second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the udp stream is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the udp stream is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_recv_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Receives a single datagram message on the socket from the remote address\n /// to which it is connected. On success, returns the number of bytes read.\n ///\n /// The function must be called with valid byte array `buf` of sufficient\n /// size to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// The [`connect`] method will connect this socket to a remote address.\n /// This method will fail if the socket is not connected.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// socket.\n ///\n /// [`connect`]: method@Self::connect\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Bind socket\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// let mut buf = vec![0; 10];\n /// let n = socket.recv(&mut buf).await?;\n ///\n /// println!(\"received {} bytes {:?}\", n, &buf[..n]);\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.recv(buf))\n .await\n }\n\n /// Attempts to receive a single datagram message on the socket from the remote\n /// address to which it is `connect`ed.\n ///\n /// The [`connect`] method will connect this socket to a remote address. This method\n /// resolves to an error if the socket is not connected.\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(()))` reads data `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`connect`]: method@Self::connect\n pub fn poll_recv(&self, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let n = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.recv(b)\n }))?;\n\n // Safety: We trust `recv` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(()))\n }\n\n /// Tries to receive a single datagram message on the socket from the remote\n /// address to which it is connected. On success, returns the number of\n /// bytes read.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.io.recv(buf))\n }\n\n cfg_io_util! {\n /// Tries to receive data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(1024);\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_buf(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n let n = (*self.io).recv(dst)?;\n\n // Safety: We trust `UdpSocket::recv` to have filled up `n` bytes in the\n // buffer.\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n\n /// Receives a single datagram message on the socket from the remote address\n /// to which it is connected, advancing the buffer's internal cursor,\n /// returning how many bytes were read.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// let mut buf = Vec::with_capacity(512);\n /// let len = socket.recv_buf(&mut buf).await?;\n ///\n /// println!(\"received {} bytes {:?}\", len, &buf[..len]);\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().async_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n let n = (*self.io).recv(dst)?;\n\n // Safety: We trust `UdpSocket::recv` to have filled up `n` bytes in the\n // buffer.\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n }).await\n }\n\n /// Tries to receive a single datagram message on the socket. On success,\n /// returns the number of bytes read and the origin.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Notes\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(1024);\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_buf_from(&mut buf) {\n /// Ok((n, _addr)) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_buf_from(&self, buf: &mut B) -> io::Result<(usize, SocketAddr)> {\n self.io.registration().try_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n let (n, addr) = (*self.io).recv_from(dst)?;\n\n // Safety: We trust `UdpSocket::recv_from` to have filled up `n` bytes in the\n // buffer.\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok((n, addr))\n })\n }\n\n /// Receives a single datagram message on the socket, advancing the\n /// buffer's internal cursor, returning how many bytes were read and the origin.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Notes\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// socket.connect(\"127.0.0.1:8081\").await?;\n ///\n /// let mut buf = Vec::with_capacity(512);\n /// let (len, addr) = socket.recv_buf_from(&mut buf).await?;\n ///\n /// println!(\"received {:?} bytes from {:?}\", len, addr);\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv_buf_from(&self, buf: &mut B) -> io::Result<(usize, SocketAddr)> {\n self.io.registration().async_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n let (n, addr) = (*self.io).recv_from(dst)?;\n\n // Safety: We trust `UdpSocket::recv_from` to have filled up `n` bytes in the\n // buffer.\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok((n,addr))\n }).await\n }\n }\n\n /// Sends data on the socket to the given address. On success, returns the\n /// number of bytes written.\n ///\n /// Address type can be any implementor of [`ToSocketAddrs`] trait. See its\n /// documentation for concrete examples.\n ///\n /// It is possible for `addr` to yield multiple addresses, but `send_to`\n /// will only send data to the first address yielded by `addr`.\n ///\n /// This will return an error when the IP version of the local socket does\n /// not match that returned from [`ToSocketAddrs`].\n ///\n /// [`ToSocketAddrs`]: crate::net::ToSocketAddrs\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `send_to` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that the message was not sent.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// let len = socket.send_to(b\"hello world\", \"127.0.0.1:8081\").await?;\n ///\n /// println!(\"Sent {} bytes\", len);\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn send_to(&self, buf: &[u8], addr: A) -> io::Result {\n let mut addrs = to_socket_addrs(addr).await?;\n\n match addrs.next() {\n Some(target) => self.send_to_addr(buf, target).await,\n None => Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n \"no addresses to send data to\",\n )),\n }\n }\n\n /// Attempts to send data on the socket to a given address.\n ///\n /// Note that on multiple calls to a `poll_*` method in the send direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to write\n /// * `Poll::Ready(Ok(n))` `n` is the number of bytes sent.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n pub fn poll_send_to(\n &self,\n cx: &mut Context<'_>,\n buf: &[u8],\n target: SocketAddr,\n ) -> Poll> {\n self.io\n .registration()\n .poll_write_io(cx, || self.io.send_to(buf, target))\n }\n\n /// Tries to send data on the socket to the given address, but if the send is\n /// blocked this will return right away.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Returns\n ///\n /// If successful, returns the number of bytes sent\n ///\n /// Users should ensure that when the remote cannot receive, the\n /// [`ErrorKind::WouldBlock`] is properly handled. An error can also occur\n /// if the IP version of the socket does not match that of `target`.\n ///\n /// [`ErrorKind::WouldBlock`]: std::io::ErrorKind::WouldBlock\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// let dst = \"127.0.0.1:8081\".parse()?;\n ///\n /// loop {\n /// socket.writable().await?;\n ///\n /// match socket.try_send_to(&b\"hello world\"[..], dst) {\n /// Ok(sent) => {\n /// println!(\"sent {} bytes\", sent);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// // Writable false positive.\n /// continue;\n /// }\n /// Err(e) => return Err(e.into()),\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_send_to(&self, buf: &[u8], target: SocketAddr) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || self.io.send_to(buf, target))\n }\n\n async fn send_to_addr(&self, buf: &[u8], target: SocketAddr) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::WRITABLE, || self.io.send_to(buf, target))\n .await\n }\n\n /// Receives a single datagram message on the socket. On success, returns\n /// the number of bytes read and the origin.\n ///\n /// The function must be called with valid byte array `buf` of sufficient\n /// size to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv_from` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// socket.\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// let mut buf = vec![0u8; 32];\n /// let (len, addr) = socket.recv_from(&mut buf).await?;\n ///\n /// println!(\"received {:?} bytes from {:?}\", len, addr);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// # Notes\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub async fn recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.recv_from(buf))\n .await\n }\n\n /// Attempts to receive a single datagram on the socket.\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(addr))` reads data from `addr` into `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// # Notes\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn poll_recv_from(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let (n, addr) = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.recv_from(b)\n }))?;\n\n // Safety: We trust `recv` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(addr))\n }\n\n /// Tries to receive a single datagram message on the socket. On success,\n /// returns the number of bytes read and the origin.\n ///\n /// This method must be called with valid byte array `buf` of sufficient size\n /// to hold the message bytes. If a message is too long to fit in the\n /// supplied buffer, excess bytes may be discarded.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Notes\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_from(&mut buf) {\n /// Ok((n, _addr)) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.io.recv_from(buf))\n }\n\n /// Tries to read or write from the socket using a user-provided IO operation.\n ///\n /// If the socket is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation on the socket by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// socket is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the socket is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UdpSocket` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: UdpSocket::readable()\n /// [`writable()`]: UdpSocket::writable()\n /// [`ready()`]: UdpSocket::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .try_io(interest, || self.io.try_io(f))\n }\n\n /// Reads or writes from the socket using a user-provided IO operation.\n ///\n /// The readiness of the socket is awaited and when the socket is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the socket by manually calling the appropriate syscall.\n /// If the operation fails because the socket is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the socket readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UdpSocket` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .async_io(interest, || self.io.try_io(&mut f))\n .await\n }\n\n /// Receives a single datagram from the connected address without removing it from the queue.\n /// On success, returns the number of bytes read from whence the data came.\n ///\n /// # Notes\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and peek returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// let mut buf = vec![0u8; 32];\n /// let len = socket.peek(&mut buf).await?;\n ///\n /// println!(\"peeked {:?} bytes\", len);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`peek_sender`]: method@Self::peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub async fn peek(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.peek(buf))\n .await\n }\n\n /// Receives data from the connected address, without removing it from the input queue.\n ///\n /// # Notes\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and peek returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`poll_peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(()))` reads data into `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`poll_peek_sender`]: method@Self::poll_peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn poll_peek(&self, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let n = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.peek(b)\n }))?;\n\n // Safety: We trust `peek` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(()))\n }\n\n /// Tries to receive data on the connected address without removing it from the input queue.\n /// On success, returns the number of bytes read.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Notes\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and peek returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`try_peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [`try_peek_sender`]: method@Self::try_peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn try_peek(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.io.peek(buf))\n }\n\n /// Receives data from the socket, without removing it from the input queue.\n /// On success, returns the number of bytes read and the address from whence\n /// the data came.\n ///\n /// # Notes\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and `peek_from` returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let socket = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// let mut buf = vec![0u8; 32];\n /// let (len, addr) = socket.peek_from(&mut buf).await?;\n ///\n /// println!(\"peeked {:?} bytes from {:?}\", len, addr);\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// [`peek_sender`]: method@Self::peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub async fn peek_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.peek_from(buf))\n .await\n }\n\n /// Receives data from the socket, without removing it from the input queue.\n /// On success, returns the sending address of the datagram.\n ///\n /// # Notes\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and peek returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`poll_peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(addr))` reads data from `addr` into `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`poll_peek_sender`]: method@Self::poll_peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn poll_peek_from(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let (n, addr) = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.peek_from(b)\n }))?;\n\n // Safety: We trust `recv` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(addr))\n }\n\n /// Tries to receive data on the socket without removing it from the input queue.\n /// On success, returns the number of bytes read and the sending address of the\n /// datagram.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// # Notes\n ///\n /// On Windows, if the data is larger than the buffer specified, the buffer\n /// is filled with the first part of the data, and peek returns the error\n /// `WSAEMSGSIZE(10040)`. The excess data is lost.\n /// Make sure to always use a sufficiently large buffer to hold the\n /// maximum UDP packet size, which can be up to 65536 bytes in size.\n ///\n /// MacOS will return an error if you pass a zero-sized buffer.\n ///\n /// If you're merely interested in learning the sender of the data at the head of the queue,\n /// try [`try_peek_sender`].\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [`try_peek_sender`]: method@Self::try_peek_sender\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn try_peek_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.io.peek_from(buf))\n }\n\n /// Retrieve the sender of the data at the head of the input queue, waiting if empty.\n ///\n /// This is equivalent to calling [`peek_from`] with a zero-sized buffer,\n /// but suppresses the `WSAEMSGSIZE` error on Windows and the \"invalid argument\" error on macOS.\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [`peek_from`]: method@Self::peek_from\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub async fn peek_sender(&self) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.peek_sender_inner())\n .await\n }\n\n /// Retrieve the sender of the data at the head of the input queue,\n /// scheduling a wakeup if empty.\n ///\n /// This is equivalent to calling [`poll_peek_from`] with a zero-sized buffer,\n /// but suppresses the `WSAEMSGSIZE` error on Windows and the \"invalid argument\" error on macOS.\n ///\n /// # Notes\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [`poll_peek_from`]: method@Self::poll_peek_from\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn poll_peek_sender(&self, cx: &mut Context<'_>) -> Poll> {\n self.io\n .registration()\n .poll_read_io(cx, || self.peek_sender_inner())\n }\n\n /// Try to retrieve the sender of the data at the head of the input queue.\n ///\n /// When there is no pending data, `Err(io::ErrorKind::WouldBlock)` is\n /// returned. This function is usually paired with `readable()`.\n ///\n /// Note that the socket address **cannot** be implicitly trusted, because it is relatively\n /// trivial to send a UDP datagram with a spoofed origin in a [packet injection attack].\n /// Because UDP is stateless and does not validate the origin of a packet,\n /// the attacker does not need to be able to intercept traffic in order to interfere.\n /// It is important to be aware of this when designing your application-level protocol.\n ///\n /// [packet injection attack]: https://en.wikipedia.org/wiki/Packet_injection\n pub fn try_peek_sender(&self) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.peek_sender_inner())\n }\n\n #[inline]\n fn peek_sender_inner(&self) -> io::Result {\n self.io.try_io(|| {\n self.as_socket()\n .peek_sender()?\n // May be `None` if the platform doesn't populate the sender for some reason.\n // In testing, that only occurred on macOS if you pass a zero-sized buffer,\n // but the implementation of `Socket::peek_sender()` covers that.\n .as_socket()\n .ok_or_else(|| io::Error::new(io::ErrorKind::Other, \"sender not available\"))\n })\n }\n\n /// Gets the value of the `SO_BROADCAST` option for this socket.\n ///\n /// For more information about this option, see [`set_broadcast`].\n ///\n /// [`set_broadcast`]: method@Self::set_broadcast\n pub fn broadcast(&self) -> io::Result {\n self.io.broadcast()\n }\n\n /// Sets the value of the `SO_BROADCAST` option for this socket.\n ///\n /// When enabled, this socket is allowed to send packets to a broadcast\n /// address.\n pub fn set_broadcast(&self, on: bool) -> io::Result<()> {\n self.io.set_broadcast(on)\n }\n\n /// Gets the value of the `IP_MULTICAST_LOOP` option for this socket.\n ///\n /// For more information about this option, see [`set_multicast_loop_v4`].\n ///\n /// [`set_multicast_loop_v4`]: method@Self::set_multicast_loop_v4\n pub fn multicast_loop_v4(&self) -> io::Result {\n self.io.multicast_loop_v4()\n }\n\n /// Sets the value of the `IP_MULTICAST_LOOP` option for this socket.\n ///\n /// If enabled, multicast packets will be looped back to the local socket.\n ///\n /// # Note\n ///\n /// This may not have any effect on IPv6 sockets.\n pub fn set_multicast_loop_v4(&self, on: bool) -> io::Result<()> {\n self.io.set_multicast_loop_v4(on)\n }\n\n /// Gets the value of the `IP_MULTICAST_TTL` option for this socket.\n ///\n /// For more information about this option, see [`set_multicast_ttl_v4`].\n ///\n /// [`set_multicast_ttl_v4`]: method@Self::set_multicast_ttl_v4\n pub fn multicast_ttl_v4(&self) -> io::Result {\n self.io.multicast_ttl_v4()\n }\n\n /// Sets the value of the `IP_MULTICAST_TTL` option for this socket.\n ///\n /// Indicates the time-to-live value of outgoing multicast packets for\n /// this socket. The default value is 1 which means that multicast packets\n /// don't leave the local network unless explicitly requested.\n ///\n /// # Note\n ///\n /// This may not have any effect on IPv6 sockets.\n pub fn set_multicast_ttl_v4(&self, ttl: u32) -> io::Result<()> {\n self.io.set_multicast_ttl_v4(ttl)\n }\n\n /// Gets the value of the `IPV6_MULTICAST_LOOP` option for this socket.\n ///\n /// For more information about this option, see [`set_multicast_loop_v6`].\n ///\n /// [`set_multicast_loop_v6`]: method@Self::set_multicast_loop_v6\n pub fn multicast_loop_v6(&self) -> io::Result {\n self.io.multicast_loop_v6()\n }\n\n /// Sets the value of the `IPV6_MULTICAST_LOOP` option for this socket.\n ///\n /// Controls whether this socket sees the multicast packets it sends itself.\n ///\n /// # Note\n ///\n /// This may not have any effect on IPv4 sockets.\n pub fn set_multicast_loop_v6(&self, on: bool) -> io::Result<()> {\n self.io.set_multicast_loop_v6(on)\n }\n\n /// Gets the value of the `IPV6_TCLASS` option for this socket.\n ///\n /// For more information about this option, see [`set_tclass_v6`].\n ///\n /// [`set_tclass_v6`]: Self::set_tclass_v6\n // https://docs.rs/socket2/0.6.1/src/socket2/sys/unix.rs.html#2541\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n )))\n )]\n pub fn tclass_v6(&self) -> io::Result {\n self.as_socket().tclass_v6()\n }\n\n /// Sets the value for the `IPV6_TCLASS` option on this socket.\n ///\n /// Specifies the traffic class field that is used in every packet\n /// sent from this socket.\n ///\n /// # Note\n ///\n /// This may not have any effect on IPv4 sockets.\n // https://docs.rs/socket2/0.6.1/src/socket2/sys/unix.rs.html#2566\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n ))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n )))\n )]\n pub fn set_tclass_v6(&self, tclass: u32) -> io::Result<()> {\n self.as_socket().set_tclass_v6(tclass)\n }\n\n /// Gets the value of the `IP_TTL` option for this socket.\n ///\n /// For more information about this option, see [`set_ttl`].\n ///\n /// [`set_ttl`]: method@Self::set_ttl\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// # use std::io;\n ///\n /// # async fn dox() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n ///\n /// println!(\"{:?}\", sock.ttl()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn ttl(&self) -> io::Result {\n self.io.ttl()\n }\n\n /// Sets the value for the `IP_TTL` option on this socket.\n ///\n /// This value sets the time-to-live field that is used in every packet sent\n /// from this socket.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UdpSocket;\n /// # use std::io;\n ///\n /// # async fn dox() -> io::Result<()> {\n /// let sock = UdpSocket::bind(\"127.0.0.1:8080\").await?;\n /// sock.set_ttl(60)?;\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn set_ttl(&self, ttl: u32) -> io::Result<()> {\n self.io.set_ttl(ttl)\n }\n\n /// Gets the value of the `IP_TOS` option for this socket.\n ///\n /// For more information about this option, see [`set_tos_v4`].\n ///\n /// [`set_tos_v4`]: Self::set_tos_v4\n // https://docs.rs/socket2/0.6.1/src/socket2/socket.rs.html#1585\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn tos_v4(&self) -> io::Result {\n self.as_socket().tos_v4()\n }\n\n /// Deprecated. Use [`tos_v4()`] instead.\n ///\n /// [`tos_v4()`]: Self::tos_v4\n #[deprecated(\n note = \"`tos` related methods have been renamed `tos_v4` since they are IPv4-specific.\"\n )]\n #[doc(hidden)]\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn tos(&self) -> io::Result {\n self.tos_v4()\n }\n\n /// Sets the value for the `IP_TOS` option on this socket.\n ///\n /// This value sets the type-of-service field that is used in every packet\n /// sent from this socket.\n ///\n /// # Note\n ///\n /// - This may not have any effect on IPv6 sockets.\n /// - On Windows, `IP_TOS` is only supported on [Windows 8+ or\n /// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)\n // https://docs.rs/socket2/0.6.1/src/socket2/socket.rs.html#1566\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn set_tos_v4(&self, tos: u32) -> io::Result<()> {\n self.as_socket().set_tos_v4(tos)\n }\n\n /// Deprecated. Use [`set_tos_v4()`] instead.\n ///\n /// [`set_tos_v4()`]: Self::set_tos_v4\n #[deprecated(\n note = \"`tos` related methods have been renamed `tos_v4` since they are IPv4-specific.\"\n )]\n #[doc(hidden)]\n #[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n )))]\n #[cfg_attr(\n docsrs,\n doc(cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n ))))\n )]\n pub fn set_tos(&self, tos: u32) -> io::Result<()> {\n self.set_tos_v4(tos)\n }\n\n /// Gets the value for the `SO_BINDTODEVICE` option on this socket\n ///\n /// This value gets the socket-bound device's interface name.\n #[cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\",))]\n #[cfg_attr(\n docsrs,\n doc(cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\",)))\n )]\n pub fn device(&self) -> io::Result>> {\n self.as_socket().device()\n }\n\n /// Sets the value for the `SO_BINDTODEVICE` option on this socket\n ///\n /// If a socket is bound to an interface, only packets received from that\n /// particular interface are processed by the socket. Note that this only\n /// works for some socket types, particularly `AF_INET` sockets.\n ///\n /// If `interface` is `None` or an empty string it removes the binding.\n #[cfg(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\"))]\n #[cfg_attr(\n docsrs,\n doc(cfg(all(any(target_os = \"android\", target_os = \"fuchsia\", target_os = \"linux\"))))\n )]\n pub fn bind_device(&self, interface: Option<&[u8]>) -> io::Result<()> {\n self.as_socket().bind_device(interface)\n }\n\n /// Executes an operation of the `IP_ADD_MEMBERSHIP` type.\n ///\n /// This function specifies a new multicast group for this socket to join.\n /// The address must be a valid multicast address, and `interface` is the\n /// address of the local interface with which the system should join the\n /// multicast group. If it's equal to `INADDR_ANY` then an appropriate\n /// interface is chosen by the system.\n pub fn join_multicast_v4(&self, multiaddr: Ipv4Addr, interface: Ipv4Addr) -> io::Result<()> {\n self.io.join_multicast_v4(&multiaddr, &interface)\n }\n\n /// Executes an operation of the `IPV6_ADD_MEMBERSHIP` type.\n ///\n /// This function specifies a new multicast group for this socket to join.\n /// The address must be a valid multicast address, and `interface` is the\n /// index of the interface to join/leave (or 0 to indicate any interface).\n pub fn join_multicast_v6(&self, multiaddr: &Ipv6Addr, interface: u32) -> io::Result<()> {\n self.io.join_multicast_v6(multiaddr, interface)\n }\n\n /// Executes an operation of the `IP_DROP_MEMBERSHIP` type.\n ///\n /// For more information about this option, see [`join_multicast_v4`].\n ///\n /// [`join_multicast_v4`]: method@Self::join_multicast_v4\n pub fn leave_multicast_v4(&self, multiaddr: Ipv4Addr, interface: Ipv4Addr) -> io::Result<()> {\n self.io.leave_multicast_v4(&multiaddr, &interface)\n }\n\n /// Executes an operation of the `IPV6_DROP_MEMBERSHIP` type.\n ///\n /// For more information about this option, see [`join_multicast_v6`].\n ///\n /// [`join_multicast_v6`]: method@Self::join_multicast_v6\n pub fn leave_multicast_v6(&self, multiaddr: &Ipv6Addr, interface: u32) -> io::Result<()> {\n self.io.leave_multicast_v6(multiaddr, interface)\n }\n\n /// Returns the value of the `SO_ERROR` option.\n ///\n /// # Examples\n /// ```\n /// use tokio::net::UdpSocket;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// // Create a socket\n /// let socket = UdpSocket::bind(\"0.0.0.0:8080\").await?;\n ///\n /// if let Ok(Some(err)) = socket.take_error() {\n /// println!(\"Got error: {:?}\", err);\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn take_error(&self) -> io::Result> {\n self.io.take_error()\n }\n}\n\nimpl TryFrom for UdpSocket {\n type Error = io::Error;\n\n /// Consumes stream, returning the tokio I/O object.\n ///\n /// This is equivalent to\n /// [`UdpSocket::from_std(stream)`](UdpSocket::from_std).\n fn try_from(stream: std::net::UdpSocket) -> Result {\n Self::from_std(stream)\n }\n}\n\nimpl fmt::Debug for UdpSocket {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.io.fmt(f)\n }\n}\n\n#[cfg(unix)]\nmod sys {\n use super::UdpSocket;\n use std::os::unix::prelude::*;\n\n impl AsRawFd for UdpSocket {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n }\n\n impl AsFd for UdpSocket {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n}\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawSocket, RawSocket};\n use crate::os::windows::io::{AsSocket, BorrowedSocket};\n\n impl AsRawSocket for UdpSocket {\n fn as_raw_socket(&self) -> RawSocket {\n self.io.as_raw_socket()\n }\n }\n\n impl AsSocket for UdpSocket {\n fn as_socket(&self) -> BorrowedSocket<'_> {\n unsafe { BorrowedSocket::borrow_raw(self.as_raw_socket()) }\n }\n }\n}\n" }, { "path": "tokio/src/net/unix/datagram/mod.rs", "content": "//! Unix datagram types.\n\npub(crate) mod socket;\n" }, { "path": "tokio/src/net/unix/datagram/socket.rs", "content": "use crate::io::{Interest, PollEvented, ReadBuf, Ready};\nuse crate::net::unix::SocketAddr;\nuse crate::util::check_socket_for_blocking;\n\nuse std::fmt;\nuse std::io;\nuse std::net::Shutdown;\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd};\nuse std::os::unix::net;\nuse std::path::Path;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\ncfg_net_unix! {\n /// An I/O object representing a Unix datagram socket.\n ///\n /// A socket can be either named (associated with a filesystem path) or\n /// unnamed.\n ///\n /// This type does not provide a `split` method, because this functionality\n /// can be achieved by wrapping the socket in an [`Arc`]. Note that you do\n /// not need a `Mutex` to share the `UnixDatagram` — an `Arc`\n /// is enough. This is because all of the methods take `&self` instead of\n /// `&mut self`.\n ///\n /// **Note:** named sockets are persisted even after the object is dropped\n /// and the program has exited, and cannot be reconnected. It is advised\n /// that you either check for and unlink the existing socket if it exists,\n /// or use a temporary file that is guaranteed to not already exist.\n ///\n /// [`Arc`]: std::sync::Arc\n ///\n /// # Examples\n /// Using named sockets, associated with a filesystem path:\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind each socket to a filesystem path\n /// let tx_path = tmp.path().join(\"tx\");\n /// let tx = UnixDatagram::bind(&tx_path)?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// let bytes = b\"hello world\";\n /// tx.send_to(bytes, &rx_path).await?;\n ///\n /// let mut buf = vec![0u8; 24];\n /// let (size, addr) = rx.recv_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n /// assert_eq!(addr.as_pathname().unwrap(), &tx_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// Using unnamed sockets, created as a pair\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// // Since the sockets are paired, the paired send/recv\n /// // functions can be used\n /// let bytes = b\"hello world\";\n /// sock1.send(bytes).await?;\n ///\n /// let mut buff = vec![0u8; 24];\n /// let size = sock2.recv(&mut buff).await?;\n ///\n /// let dgram = &buff[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n #[cfg_attr(docsrs, doc(alias = \"uds\"))]\n pub struct UnixDatagram {\n io: PollEvented,\n }\n}\n\nimpl UnixDatagram {\n pub(crate) fn from_mio(sys: mio::net::UnixDatagram) -> io::Result {\n let datagram = UnixDatagram::new(sys)?;\n\n if let Some(e) = datagram.io.take_error()? {\n return Err(e);\n }\n\n Ok(datagram)\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_recv()` or `try_send()`. It\n /// can be used to concurrently `recv` / `send` to the same socket on a single\n /// task without splitting the socket.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Concurrently receive from and send to the socket on the same task\n /// without splitting.\n ///\n /// ```no_run\n /// use tokio::io::Interest;\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// let ready = socket.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// let mut data = [0; 1024];\n /// match socket.try_recv(&mut data[..]) {\n /// Ok(n) => {\n /// println!(\"received {:?}\", &data[..n]);\n /// }\n /// // False-positive, continue\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {}\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// if ready.is_writable() {\n /// // Write some data\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"sent {} bytes\", n);\n /// }\n /// // False-positive, continue\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {}\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is\n /// usually paired with `try_send()` or `try_send_to()`.\n ///\n /// The function may complete without the socket being writable. This is a\n /// false-positive and attempting a `try_send()` will return with\n /// `io::ErrorKind::WouldBlock`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// socket.writable().await?;\n ///\n /// // Try to send data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write/send readiness.\n ///\n /// If the socket is not currently ready for sending, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the socket\n /// becomes ready for sending, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_send_ready` or `poll_send`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_recv_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the socket is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_send_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_recv()`.\n ///\n /// The function may complete without the socket being readable. This is a\n /// false-positive and attempting a `try_recv()` will return with\n /// `io::ErrorKind::WouldBlock`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read/receive readiness.\n ///\n /// If the socket is not currently ready for receiving, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the\n /// socket becomes ready for reading, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_recv_ready`, `poll_recv` or\n /// `poll_peek`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup. (However,\n /// `poll_send_ready` retains a second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the socket is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_recv_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Creates a new `UnixDatagram` bound to the specified path.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind the socket to a filesystem path\n /// let socket_path = tmp.path().join(\"socket\");\n /// let socket = UnixDatagram::bind(&socket_path)?;\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn bind

(path: P) -> io::Result\n where\n P: AsRef,\n {\n let socket = mio::net::UnixDatagram::bind(path)?;\n UnixDatagram::new(socket)\n }\n\n /// Creates an unnamed pair of connected sockets.\n ///\n /// This function will create a pair of interconnected Unix sockets for\n /// communicating back and forth between one another.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// // Since the sockets are paired, the paired send/recv\n /// // functions can be used\n /// let bytes = b\"hail eris\";\n /// sock1.send(bytes).await?;\n ///\n /// let mut buff = vec![0u8; 24];\n /// let size = sock2.recv(&mut buff).await?;\n ///\n /// let dgram = &buff[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn pair() -> io::Result<(UnixDatagram, UnixDatagram)> {\n let (a, b) = mio::net::UnixDatagram::pair()?;\n let a = UnixDatagram::new(a)?;\n let b = UnixDatagram::new(b)?;\n\n Ok((a, b))\n }\n\n /// Creates new [`UnixDatagram`] from a [`std::os::unix::net::UnixDatagram`].\n ///\n /// This function is intended to be used to wrap a `UnixDatagram` from the\n /// standard library in the Tokio equivalent.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the socket is in\n /// non-blocking mode. Otherwise all I/O operations on the socket\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::os::unix::net::UnixDatagram::set_nonblocking\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a Tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use std::os::unix::net::UnixDatagram as StdUDS;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind the socket to a filesystem path\n /// let socket_path = tmp.path().join(\"socket\");\n /// let std_socket = StdUDS::bind(&socket_path)?;\n /// std_socket.set_nonblocking(true)?;\n /// let tokio_socket = UnixDatagram::from_std(std_socket)?;\n ///\n /// # Ok(())\n /// # }\n /// ```\n #[track_caller]\n pub fn from_std(datagram: net::UnixDatagram) -> io::Result {\n check_socket_for_blocking(&datagram)?;\n\n let socket = mio::net::UnixDatagram::from_std(datagram);\n let io = PollEvented::new(socket)?;\n Ok(UnixDatagram { io })\n }\n\n /// Turns a [`tokio::net::UnixDatagram`] into a [`std::os::unix::net::UnixDatagram`].\n ///\n /// The returned [`std::os::unix::net::UnixDatagram`] will have nonblocking\n /// mode set as `true`. Use [`set_nonblocking`] to change the blocking mode\n /// if needed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// # use std::error::Error;\n /// # async fn dox() -> Result<(), Box> {\n /// let tokio_socket = tokio::net::UnixDatagram::bind(\"/path/to/the/socket\")?;\n /// let std_socket = tokio_socket.into_std()?;\n /// std_socket.set_nonblocking(false)?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`tokio::net::UnixDatagram`]: UnixDatagram\n /// [`std::os::unix::net::UnixDatagram`]: std::os::unix::net::UnixDatagram\n /// [`set_nonblocking`]: fn@std::os::unix::net::UnixDatagram::set_nonblocking\n pub fn into_std(self) -> io::Result {\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { std::os::unix::net::UnixDatagram::from_raw_fd(raw_fd) })\n }\n\n fn new(socket: mio::net::UnixDatagram) -> io::Result {\n let io = PollEvented::new(socket)?;\n Ok(UnixDatagram { io })\n }\n\n /// Creates a new `UnixDatagram` which is not bound to any address.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // Create an unbound socket\n /// let tx = UnixDatagram::unbound()?;\n ///\n /// // Create another, bound socket\n /// let tmp = tempdir()?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// // Send to the bound socket\n /// let bytes = b\"hello world\";\n /// tx.send_to(bytes, &rx_path).await?;\n ///\n /// let mut buf = vec![0u8; 24];\n /// let (size, addr) = rx.recv_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn unbound() -> io::Result {\n let socket = mio::net::UnixDatagram::unbound()?;\n UnixDatagram::new(socket)\n }\n\n /// Connects the socket to the specified address.\n ///\n /// The `send` method may be used to send data to the specified address.\n /// `recv` and `recv_from` will only receive data from that address.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // Create an unbound socket\n /// let tx = UnixDatagram::unbound()?;\n ///\n /// // Create another, bound socket\n /// let tmp = tempdir()?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// // Connect to the bound socket\n /// tx.connect(&rx_path)?;\n ///\n /// // Send to the bound socket\n /// let bytes = b\"hello world\";\n /// tx.send(bytes).await?;\n ///\n /// let mut buf = vec![0u8; 24];\n /// let (size, addr) = rx.recv_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn connect>(&self, path: P) -> io::Result<()> {\n self.io.connect(path)\n }\n\n /// Sends data on the socket to the socket's peer.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `send` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that the message was not sent.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// // Since the sockets are paired, the paired send/recv\n /// // functions can be used\n /// let bytes = b\"hello world\";\n /// sock1.send(bytes).await?;\n ///\n /// let mut buff = vec![0u8; 24];\n /// let size = sock2.recv(&mut buff).await?;\n ///\n /// let dgram = &buff[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn send(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::WRITABLE, || self.io.send(buf))\n .await\n }\n\n /// Tries to send a datagram to the peer without waiting.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// socket.writable().await?;\n ///\n /// // Try to send data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_send(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_send(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || self.io.send(buf))\n }\n\n /// Tries to send a datagram to the peer without waiting.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// socket.writable().await?;\n ///\n /// // Try to send data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_send_to(b\"hello world\", &server_path) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_send_to

(&self, buf: &[u8], target: P) -> io::Result\n where\n P: AsRef,\n {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || self.io.send_to(buf, target))\n }\n\n /// Receives data from the socket.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// socket.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// // Since the sockets are paired, the paired send/recv\n /// // functions can be used\n /// let bytes = b\"hello world\";\n /// sock1.send(bytes).await?;\n ///\n /// let mut buff = vec![0u8; 24];\n /// let size = sock2.recv(&mut buff).await?;\n ///\n /// let dgram = &buff[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn recv(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .async_io(Interest::READABLE, || self.io.recv(buf))\n .await\n }\n\n /// Tries to receive a datagram from the peer without waiting.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || self.io.recv(buf))\n }\n\n cfg_io_util! {\n /// Tries to receive data from the socket without waiting.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(1024);\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_buf_from(&mut buf) {\n /// Ok((n, _addr)) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_buf_from(&self, buf: &mut B) -> io::Result<(usize, SocketAddr)> {\n let (n, addr) = self.io.registration().try_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `UnixDatagram::recv_from` to have filled up `n` bytes in the\n // buffer.\n let (n, addr) = (*self.io).recv_from(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok((n, addr))\n })?;\n\n Ok((n, SocketAddr(addr)))\n }\n\n /// Receives from the socket, advances the\n /// buffer's internal cursor and returns how many bytes were read and the origin.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind each socket to a filesystem path\n /// let tx_path = tmp.path().join(\"tx\");\n /// let tx = UnixDatagram::bind(&tx_path)?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// let bytes = b\"hello world\";\n /// tx.send_to(bytes, &rx_path).await?;\n ///\n /// let mut buf = Vec::with_capacity(24);\n /// let (size, addr) = rx.recv_buf_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n /// assert_eq!(addr.as_pathname().unwrap(), &tx_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn recv_buf_from(&self, buf: &mut B) -> io::Result<(usize, SocketAddr)> {\n self.io.registration().async_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `UnixDatagram::recv_from` to have filled up `n` bytes in the\n // buffer.\n let (n, addr) = (*self.io).recv_from(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n Ok((n,SocketAddr(addr)))\n }).await\n }\n\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n /// socket.connect(&server_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(1024);\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_buf(&mut buf) {\n /// Ok(n) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `UnixDatagram::recv` to have filled up `n` bytes in the\n // buffer.\n let n = (*self.io).recv(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n\n /// Receives data from the socket from the address to which it is connected,\n /// advancing the buffer's internal cursor, returning how many bytes were read.\n ///\n /// This method can be used even if `buf` is uninitialized.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// // Since the sockets are paired, the paired send/recv\n /// // functions can be used\n /// let bytes = b\"hello world\";\n /// sock1.send(bytes).await?;\n ///\n /// let mut buff = Vec::with_capacity(24);\n /// let size = sock2.recv_buf(&mut buff).await?;\n ///\n /// let dgram = &buff[..size];\n /// assert_eq!(dgram, bytes);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn recv_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().async_io(Interest::READABLE, || {\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `UnixDatagram::recv_from` to have filled up `n` bytes in the\n // buffer.\n let n = (*self.io).recv(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n Ok(n)\n }).await\n }\n }\n\n /// Sends data on the socket to the specified address.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `send_to` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that the message was not sent.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind each socket to a filesystem path\n /// let tx_path = tmp.path().join(\"tx\");\n /// let tx = UnixDatagram::bind(&tx_path)?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// let bytes = b\"hello world\";\n /// tx.send_to(bytes, &rx_path).await?;\n ///\n /// let mut buf = vec![0u8; 24];\n /// let (size, addr) = rx.recv_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n /// assert_eq!(addr.as_pathname().unwrap(), &tx_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn send_to

(&self, buf: &[u8], target: P) -> io::Result\n where\n P: AsRef,\n {\n self.io\n .registration()\n .async_io(Interest::WRITABLE, || self.io.send_to(buf, target.as_ref()))\n .await\n }\n\n /// Receives data from the socket.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv_from` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// socket.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind each socket to a filesystem path\n /// let tx_path = tmp.path().join(\"tx\");\n /// let tx = UnixDatagram::bind(&tx_path)?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// let bytes = b\"hello world\";\n /// tx.send_to(bytes, &rx_path).await?;\n ///\n /// let mut buf = vec![0u8; 24];\n /// let (size, addr) = rx.recv_from(&mut buf).await?;\n ///\n /// let dgram = &buf[..size];\n /// assert_eq!(dgram, bytes);\n /// assert_eq!(addr.as_pathname().unwrap(), &tx_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub async fn recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n let (n, addr) = self\n .io\n .registration()\n .async_io(Interest::READABLE, || self.io.recv_from(buf))\n .await?;\n\n Ok((n, SocketAddr(addr)))\n }\n\n /// Attempts to receive a single datagram on the specified address.\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(addr))` reads data from `addr` into `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n pub fn poll_recv_from(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let (n, addr) = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.recv_from(b)\n }))?;\n\n // Safety: We trust `recv` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(SocketAddr(addr)))\n }\n\n /// Attempts to send data to the specified address.\n ///\n /// Note that on multiple calls to a `poll_*` method in the send direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to write\n /// * `Poll::Ready(Ok(n))` `n` is the number of bytes sent.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n pub fn poll_send_to

(\n &self,\n cx: &mut Context<'_>,\n buf: &[u8],\n target: P,\n ) -> Poll>\n where\n P: AsRef,\n {\n self.io\n .registration()\n .poll_write_io(cx, || self.io.send_to(buf, target.as_ref()))\n }\n\n /// Attempts to send data on the socket to the remote address to which it\n /// was previously `connect`ed.\n ///\n /// The [`connect`] method will connect this socket to a remote address.\n /// This method will fail if the socket is not connected.\n ///\n /// Note that on multiple calls to a `poll_*` method in the send direction,\n /// only the `Waker` from the `Context` passed to the most recent call will\n /// be scheduled to receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not available to write\n /// * `Poll::Ready(Ok(n))` `n` is the number of bytes sent\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`connect`]: method@Self::connect\n pub fn poll_send(&self, cx: &mut Context<'_>, buf: &[u8]) -> Poll> {\n self.io\n .registration()\n .poll_write_io(cx, || self.io.send(buf))\n }\n\n /// Attempts to receive a single datagram message on the socket from the remote\n /// address to which it is `connect`ed.\n ///\n /// The [`connect`] method will connect this socket to a remote address. This method\n /// resolves to an error if the socket is not connected.\n ///\n /// Note that on multiple calls to a `poll_*` method in the `recv` direction, only the\n /// `Waker` from the `Context` passed to the most recent call will be scheduled to\n /// receive a wakeup.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the socket is not ready to read\n /// * `Poll::Ready(Ok(()))` reads data `ReadBuf` if the socket is ready\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`connect`]: method@Self::connect\n pub fn poll_recv(&self, cx: &mut Context<'_>, buf: &mut ReadBuf<'_>) -> Poll> {\n #[allow(clippy::blocks_in_conditions)]\n let n = ready!(self.io.registration().poll_read_io(cx, || {\n // Safety: will not read the maybe uninitialized bytes.\n let b = unsafe {\n &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8])\n };\n\n self.io.recv(b)\n }))?;\n\n // Safety: We trust `recv` to have filled up `n` bytes in the buffer.\n unsafe {\n buf.assume_init(n);\n }\n buf.advance(n);\n Poll::Ready(Ok(()))\n }\n\n /// Tries to receive data from the socket without waiting.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixDatagram;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let client_path = dir.path().join(\"client.sock\");\n /// let server_path = dir.path().join(\"server.sock\");\n /// let socket = UnixDatagram::bind(&client_path)?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// socket.readable().await?;\n ///\n /// // The buffer is **not** included in the async task and will\n /// // only exist on the stack.\n /// let mut buf = [0; 1024];\n ///\n /// // Try to recv data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match socket.try_recv_from(&mut buf) {\n /// Ok((n, _addr)) => {\n /// println!(\"GOT {:?}\", &buf[..n]);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e);\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {\n let (n, addr) = self\n .io\n .registration()\n .try_io(Interest::READABLE, || self.io.recv_from(buf))?;\n\n Ok((n, SocketAddr(addr)))\n }\n\n /// Tries to read or write from the socket using a user-provided IO operation.\n ///\n /// If the socket is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation on the socket by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// socket is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the socket is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UnixDatagram` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: UnixDatagram::readable()\n /// [`writable()`]: UnixDatagram::writable()\n /// [`ready()`]: UnixDatagram::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .try_io(interest, || self.io.try_io(f))\n }\n\n /// Reads or writes from the socket using a user-provided IO operation.\n ///\n /// The readiness of the socket is awaited and when the socket is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the socket by manually calling the appropriate syscall.\n /// If the operation fails because the socket is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the socket readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UnixDatagram` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .async_io(interest, || self.io.try_io(&mut f))\n .await\n }\n\n /// Returns the local address that this socket is bound to.\n ///\n /// # Examples\n /// For a socket bound to a local path\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // We use a temporary directory so that the socket\n /// // files left by the bound sockets will get cleaned up.\n /// let tmp = tempdir()?;\n ///\n /// // Bind socket to a filesystem path\n /// let socket_path = tmp.path().join(\"socket\");\n /// let socket = UnixDatagram::bind(&socket_path)?;\n ///\n /// assert_eq!(socket.local_addr()?.as_pathname().unwrap(), &socket_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// For an unbound socket\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create an unbound socket\n /// let socket = UnixDatagram::unbound()?;\n ///\n /// assert!(socket.local_addr()?.is_unnamed());\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr().map(SocketAddr)\n }\n\n /// Returns the address of this socket's peer.\n ///\n /// The `connect` method will connect the socket to a peer.\n ///\n /// # Examples\n /// For a peer with a local path\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n /// use tempfile::tempdir;\n ///\n /// // Create an unbound socket\n /// let tx = UnixDatagram::unbound()?;\n ///\n /// // Create another, bound socket\n /// let tmp = tempdir()?;\n /// let rx_path = tmp.path().join(\"rx\");\n /// let rx = UnixDatagram::bind(&rx_path)?;\n ///\n /// // Connect to the bound socket\n /// tx.connect(&rx_path)?;\n ///\n /// assert_eq!(tx.peer_addr()?.as_pathname().unwrap(), &rx_path);\n ///\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// For an unbound peer\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create the pair of sockets\n /// let (sock1, sock2) = UnixDatagram::pair()?;\n ///\n /// assert!(sock1.peer_addr()?.is_unnamed());\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn peer_addr(&self) -> io::Result {\n self.io.peer_addr().map(SocketAddr)\n }\n\n /// Returns the value of the `SO_ERROR` option.\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// use tokio::net::UnixDatagram;\n ///\n /// // Create an unbound socket\n /// let socket = UnixDatagram::unbound()?;\n ///\n /// if let Ok(Some(err)) = socket.take_error() {\n /// println!(\"Got error: {:?}\", err);\n /// }\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn take_error(&self) -> io::Result> {\n self.io.take_error()\n }\n\n /// Shuts down the read, write, or both halves of this connection.\n ///\n /// This function will cause all pending and future I/O calls on the\n /// specified portions to immediately return with an appropriate value\n /// (see the documentation of `Shutdown`).\n ///\n /// # Examples\n /// ```\n /// # use std::error::Error;\n /// # #[tokio::main]\n /// # async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No SOCK_DGRAM for `socketpair` in miri.\n /// use tokio::net::UnixDatagram;\n /// use std::net::Shutdown;\n ///\n /// // Create an unbound socket\n /// let (socket, other) = UnixDatagram::pair()?;\n ///\n /// socket.shutdown(Shutdown::Both)?;\n ///\n /// // NOTE: the following commented out code does NOT work as expected.\n /// // Due to an underlying issue, the recv call will block indefinitely.\n /// // See: https://github.com/tokio-rs/tokio/issues/1679\n /// //let mut buff = vec![0u8; 24];\n /// //let size = socket.recv(&mut buff).await?;\n /// //assert_eq!(size, 0);\n ///\n /// let send_result = socket.send(b\"hello world\").await;\n /// assert!(send_result.is_err());\n ///\n /// # Ok(())\n /// # }\n /// ```\n pub fn shutdown(&self, how: Shutdown) -> io::Result<()> {\n self.io.shutdown(how)\n }\n}\n\nimpl TryFrom for UnixDatagram {\n type Error = io::Error;\n\n /// Consumes stream, returning the Tokio I/O object.\n ///\n /// This is equivalent to\n /// [`UnixDatagram::from_std(stream)`](UnixDatagram::from_std).\n fn try_from(stream: std::os::unix::net::UnixDatagram) -> Result {\n Self::from_std(stream)\n }\n}\n\nimpl fmt::Debug for UnixDatagram {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n (*self.io).fmt(f)\n }\n}\n\nimpl AsRawFd for UnixDatagram {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n}\n\nimpl AsFd for UnixDatagram {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n" }, { "path": "tokio/src/net/unix/listener.rs", "content": "use crate::io::{Interest, PollEvented};\nuse crate::net::unix::{SocketAddr, UnixStream};\nuse crate::util::check_socket_for_blocking;\n\nuse std::fmt;\nuse std::io;\n#[cfg(target_os = \"android\")]\nuse std::os::android::net::SocketAddrExt;\n#[cfg(target_os = \"linux\")]\nuse std::os::linux::net::SocketAddrExt;\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nuse std::os::unix::ffi::OsStrExt;\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd};\nuse std::os::unix::net::{self, SocketAddr as StdSocketAddr};\nuse std::path::Path;\nuse std::task::{ready, Context, Poll};\n\ncfg_net_unix! {\n /// A Unix socket which can accept connections from other Unix sockets.\n ///\n /// You can accept a new connection by using the [`accept`](`UnixListener::accept`) method.\n ///\n /// A `UnixListener` can be turned into a `Stream` with [`UnixListenerStream`].\n ///\n /// [`UnixListenerStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.UnixListenerStream.html\n ///\n /// # Errors\n ///\n /// Note that accepting a connection can lead to various errors and not all\n /// of them are necessarily fatal ‒ for example having too many open file\n /// descriptors or the other side closing the connection while it waits in\n /// an accept queue. These would terminate the stream if not handled in any\n /// way.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixListener;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let listener = UnixListener::bind(\"/path/to/the/socket\").unwrap();\n /// loop {\n /// match listener.accept().await {\n /// Ok((stream, _addr)) => {\n /// println!(\"new client!\");\n /// }\n /// Err(e) => { /* connection failed */ }\n /// }\n /// }\n /// }\n /// ```\n #[cfg_attr(docsrs, doc(alias = \"uds\"))]\n pub struct UnixListener {\n io: PollEvented,\n }\n}\n\nimpl UnixListener {\n pub(crate) fn new(listener: mio::net::UnixListener) -> io::Result {\n let io = PollEvented::new(listener)?;\n Ok(UnixListener { io })\n }\n\n /// Creates a new `UnixListener` bound to the specified path.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n pub fn bind

(path: P) -> io::Result\n where\n P: AsRef,\n {\n // For now, we handle abstract socket paths on linux here.\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n let addr = {\n let os_str_bytes = path.as_ref().as_os_str().as_bytes();\n if os_str_bytes.starts_with(b\"\\0\") {\n StdSocketAddr::from_abstract_name(&os_str_bytes[1..])?\n } else {\n StdSocketAddr::from_pathname(path)?\n }\n };\n #[cfg(not(any(target_os = \"linux\", target_os = \"android\")))]\n let addr = StdSocketAddr::from_pathname(path)?;\n\n let listener = mio::net::UnixListener::bind_addr(&addr)?;\n let io = PollEvented::new(listener)?;\n Ok(UnixListener { io })\n }\n\n /// Creates new [`UnixListener`] from a [`std::os::unix::net::UnixListener`].\n ///\n /// This function is intended to be used to wrap a `UnixListener` from the\n /// standard library in the Tokio equivalent.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the listener is in\n /// non-blocking mode. Otherwise all I/O operations on the listener\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::os::unix::net::UnixListener::set_nonblocking\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixListener;\n /// use std::os::unix::net::UnixListener as StdUnixListener;\n /// # use std::error::Error;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let std_listener = StdUnixListener::bind(\"/path/to/the/socket\")?;\n /// std_listener.set_nonblocking(true)?;\n /// let listener = UnixListener::from_std(std_listener)?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n pub fn from_std(listener: net::UnixListener) -> io::Result {\n check_socket_for_blocking(&listener)?;\n\n let listener = mio::net::UnixListener::from_std(listener);\n let io = PollEvented::new(listener)?;\n Ok(UnixListener { io })\n }\n\n /// Turns a [`tokio::net::UnixListener`] into a [`std::os::unix::net::UnixListener`].\n ///\n /// The returned [`std::os::unix::net::UnixListener`] will have nonblocking mode\n /// set as `true`. Use [`set_nonblocking`] to change the blocking mode if needed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// # use std::error::Error;\n /// # async fn dox() -> Result<(), Box> {\n /// let tokio_listener = tokio::net::UnixListener::bind(\"/path/to/the/socket\")?;\n /// let std_listener = tokio_listener.into_std()?;\n /// std_listener.set_nonblocking(false)?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`tokio::net::UnixListener`]: UnixListener\n /// [`std::os::unix::net::UnixListener`]: std::os::unix::net::UnixListener\n /// [`set_nonblocking`]: fn@std::os::unix::net::UnixListener::set_nonblocking\n pub fn into_std(self) -> io::Result {\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { net::UnixListener::from_raw_fd(raw_fd) })\n }\n\n /// Returns the local socket address of this listener.\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr().map(SocketAddr)\n }\n\n /// Returns the value of the `SO_ERROR` option.\n pub fn take_error(&self) -> io::Result> {\n self.io.take_error()\n }\n\n /// Accepts a new incoming connection to this listener.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If the method is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no new connections were\n /// accepted by this method.\n pub async fn accept(&self) -> io::Result<(UnixStream, SocketAddr)> {\n let (mio, addr) = self\n .io\n .registration()\n .async_io(Interest::READABLE, || self.io.accept())\n .await?;\n\n let addr = SocketAddr(addr);\n let stream = UnixStream::new(mio)?;\n Ok((stream, addr))\n }\n\n /// Polls to accept a new incoming connection to this listener.\n ///\n /// If there is no connection to accept, `Poll::Pending` is returned and the\n /// current task will be notified by a waker. Note that on multiple calls\n /// to `poll_accept`, only the `Waker` from the `Context` passed to the most\n /// recent call is scheduled to receive a wakeup.\n pub fn poll_accept(&self, cx: &mut Context<'_>) -> Poll> {\n let (sock, addr) = ready!(self.io.registration().poll_read_io(cx, || self.io.accept()))?;\n let addr = SocketAddr(addr);\n let sock = UnixStream::new(sock)?;\n Poll::Ready(Ok((sock, addr)))\n }\n}\n\nimpl TryFrom for UnixListener {\n type Error = io::Error;\n\n /// Consumes stream, returning the tokio I/O object.\n ///\n /// This is equivalent to\n /// [`UnixListener::from_std(stream)`](UnixListener::from_std).\n fn try_from(stream: std::os::unix::net::UnixListener) -> io::Result {\n Self::from_std(stream)\n }\n}\n\nimpl fmt::Debug for UnixListener {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n (*self.io).fmt(f)\n }\n}\n\nimpl AsRawFd for UnixListener {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n}\n\nimpl AsFd for UnixListener {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n" }, { "path": "tokio/src/net/unix/mod.rs", "content": "//! Unix specific network types.\n// This module does not currently provide any public API, but it was\n// unintentionally defined as a public module. Hide it from the documentation\n// instead of changing it to a private module to avoid breakage.\n#[doc(hidden)]\npub mod datagram;\n\npub(crate) mod listener;\n\npub(crate) mod socket;\n\nmod split;\npub use split::{ReadHalf, WriteHalf};\n\nmod split_owned;\npub use split_owned::{OwnedReadHalf, OwnedWriteHalf, ReuniteError};\n\nmod socketaddr;\npub use socketaddr::SocketAddr;\n\npub(crate) mod stream;\npub(crate) use stream::UnixStream;\n\nmod ucred;\npub use ucred::UCred;\n\npub mod pipe;\n\n/// A type representing user ID.\n#[allow(non_camel_case_types)]\npub type uid_t = u32;\n\n/// A type representing group ID.\n#[allow(non_camel_case_types)]\npub type gid_t = u32;\n\n/// A type representing process and process group IDs.\n#[allow(non_camel_case_types)]\npub type pid_t = i32;\n" }, { "path": "tokio/src/net/unix/pipe.rs", "content": "//! Unix pipe types.\n\nuse crate::io::interest::Interest;\nuse crate::io::{AsyncRead, AsyncWrite, PollEvented, ReadBuf, Ready};\n\nuse mio::unix::pipe as mio_pipe;\nuse std::fs::File;\nuse std::io::{self, Read, Write};\nuse std::os::unix::fs::OpenOptionsExt;\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, OwnedFd, RawFd};\nuse std::path::Path;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n/// Creates a new anonymous Unix pipe.\n///\n/// This function will open a new pipe and associate both pipe ends with the default\n/// event loop.\n///\n/// If you need to create a pipe for communication with a spawned process, you can\n/// use [`Stdio::piped()`] instead.\n///\n/// [`Stdio::piped()`]: std::process::Stdio::piped\n///\n/// # Errors\n///\n/// If creating a pipe fails, this function will return with the related OS error.\n///\n/// # Examples\n///\n/// Create a pipe and pass the writing end to a spawned process.\n///\n/// ```no_run\n/// use tokio::net::unix::pipe;\n/// use tokio::process::Command;\n/// # use tokio::io::AsyncReadExt;\n/// # use std::error::Error;\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let (tx, mut rx) = pipe::pipe()?;\n/// let mut buffer = String::new();\n///\n/// let status = Command::new(\"echo\")\n/// .arg(\"Hello, world!\")\n/// .stdout(tx.into_blocking_fd()?)\n/// .status();\n/// rx.read_to_string(&mut buffer).await?;\n///\n/// assert!(status.await?.success());\n/// assert_eq!(buffer, \"Hello, world!\\n\");\n/// # Ok(())\n/// # }\n/// ```\n///\n/// # Panics\n///\n/// This function panics if it is not called from within a runtime with\n/// IO enabled.\n///\n/// The runtime is usually set implicitly when this function is called\n/// from a future driven by a tokio runtime, otherwise runtime can be set\n/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\npub fn pipe() -> io::Result<(Sender, Receiver)> {\n let (tx, rx) = mio_pipe::new()?;\n Ok((Sender::from_mio(tx)?, Receiver::from_mio(rx)?))\n}\n\n/// Options and flags which can be used to configure how a FIFO file is opened.\n///\n/// This builder allows configuring how to create a pipe end from a FIFO file.\n/// Generally speaking, when using `OpenOptions`, you'll first call [`new`],\n/// then chain calls to methods to set each option, then call either\n/// [`open_receiver`] or [`open_sender`], passing the path of the FIFO file you\n/// are trying to open. This will give you a [`io::Result`] with a pipe end\n/// inside that you can further operate on.\n///\n/// [`new`]: OpenOptions::new\n/// [`open_receiver`]: OpenOptions::open_receiver\n/// [`open_sender`]: OpenOptions::open_sender\n///\n/// # Examples\n///\n/// Opening a pair of pipe ends from a FIFO file:\n///\n/// ```no_run\n/// use tokio::net::unix::pipe;\n/// # use std::error::Error;\n///\n/// const FIFO_NAME: &str = \"path/to/a/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let rx = pipe::OpenOptions::new().open_receiver(FIFO_NAME)?;\n/// let tx = pipe::OpenOptions::new().open_sender(FIFO_NAME)?;\n/// # Ok(())\n/// # }\n/// ```\n///\n/// Opening a [`Sender`] on Linux when you are sure the file is a FIFO:\n///\n/// ```ignore\n/// use tokio::net::unix::pipe;\n/// use nix::{unistd::mkfifo, sys::stat::Mode};\n/// # use std::error::Error;\n///\n/// // Our program has exclusive access to this path.\n/// const FIFO_NAME: &str = \"path/to/a/new/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// mkfifo(FIFO_NAME, Mode::S_IRWXU)?;\n/// let tx = pipe::OpenOptions::new()\n/// .read_write(true)\n/// .unchecked(true)\n/// .open_sender(FIFO_NAME)?;\n/// # Ok(())\n/// # }\n/// ```\n#[derive(Clone, Debug)]\npub struct OpenOptions {\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n read_write: bool,\n unchecked: bool,\n}\n\nimpl OpenOptions {\n /// Creates a blank new set of options ready for configuration.\n ///\n /// All options are initially set to `false`.\n pub fn new() -> OpenOptions {\n OpenOptions {\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n read_write: false,\n unchecked: false,\n }\n }\n\n /// Sets the option for read-write access.\n ///\n /// This option, when true, will indicate that a FIFO file will be opened\n /// in read-write access mode. This operation is not defined by the POSIX\n /// standard and is only guaranteed to work on Linux.\n ///\n /// # Examples\n ///\n /// Opening a [`Sender`] even if there are no open reading ends:\n ///\n /// ```ignore\n /// use tokio::net::unix::pipe;\n ///\n /// let tx = pipe::OpenOptions::new()\n /// .read_write(true)\n /// .open_sender(\"path/to/a/fifo\");\n /// ```\n ///\n /// Opening a resilient [`Receiver`] i.e. a reading pipe end which will not\n /// fail with [`UnexpectedEof`] during reading if all writing ends of the\n /// pipe close the FIFO file.\n ///\n /// [`UnexpectedEof`]: std::io::ErrorKind::UnexpectedEof\n ///\n /// ```ignore\n /// use tokio::net::unix::pipe;\n ///\n /// let tx = pipe::OpenOptions::new()\n /// .read_write(true)\n /// .open_receiver(\"path/to/a/fifo\");\n /// ```\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub fn read_write(&mut self, value: bool) -> &mut Self {\n self.read_write = value;\n self\n }\n\n /// Sets the option to skip the check for FIFO file type.\n ///\n /// By default, [`open_receiver`] and [`open_sender`] functions will check\n /// if the opened file is a FIFO file. Set this option to `true` if you are\n /// sure the file is a FIFO file.\n ///\n /// [`open_receiver`]: OpenOptions::open_receiver\n /// [`open_sender`]: OpenOptions::open_sender\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use nix::{unistd::mkfifo, sys::stat::Mode};\n /// # use std::error::Error;\n ///\n /// // Our program has exclusive access to this path.\n /// const FIFO_NAME: &str = \"path/to/a/new/fifo\";\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// mkfifo(FIFO_NAME, Mode::S_IRWXU)?;\n /// let rx = pipe::OpenOptions::new()\n /// .unchecked(true)\n /// .open_receiver(FIFO_NAME)?;\n /// # Ok(())\n /// # }\n /// ```\n pub fn unchecked(&mut self, value: bool) -> &mut Self {\n self.unchecked = value;\n self\n }\n\n /// Creates a [`Receiver`] from a FIFO file with the options specified by `self`.\n ///\n /// This function will open the FIFO file at the specified path, possibly\n /// check if it is a pipe, and associate the pipe with the default event\n /// loop for reading.\n ///\n /// # Errors\n ///\n /// If the file type check fails, this function will fail with `io::ErrorKind::InvalidInput`.\n /// This function may also fail with other standard OS errors.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn open_receiver>(&self, path: P) -> io::Result {\n let file = self.open(path.as_ref(), PipeEnd::Receiver)?;\n Receiver::from_file_unchecked(file)\n }\n\n /// Creates a [`Sender`] from a FIFO file with the options specified by `self`.\n ///\n /// This function will open the FIFO file at the specified path, possibly\n /// check if it is a pipe, and associate the pipe with the default event\n /// loop for writing.\n ///\n /// # Errors\n ///\n /// If the file type check fails, this function will fail with `io::ErrorKind::InvalidInput`.\n /// If the file is not opened in read-write access mode and the file is not\n /// currently open for reading, this function will fail with `ENXIO`.\n /// This function may also fail with other standard OS errors.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn open_sender>(&self, path: P) -> io::Result {\n let file = self.open(path.as_ref(), PipeEnd::Sender)?;\n Sender::from_file_unchecked(file)\n }\n\n fn open(&self, path: &Path, pipe_end: PipeEnd) -> io::Result {\n let mut options = std::fs::OpenOptions::new();\n options\n .read(pipe_end == PipeEnd::Receiver)\n .write(pipe_end == PipeEnd::Sender)\n .custom_flags(libc::O_NONBLOCK);\n\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n if self.read_write {\n options.read(true).write(true);\n }\n\n let file = options.open(path)?;\n\n if !self.unchecked && !is_pipe(file.as_fd())? {\n return Err(io::Error::new(io::ErrorKind::InvalidInput, \"not a pipe\"));\n }\n\n Ok(file)\n }\n}\n\nimpl Default for OpenOptions {\n fn default() -> OpenOptions {\n OpenOptions::new()\n }\n}\n\n#[derive(Clone, Copy, PartialEq, Eq, Debug)]\nenum PipeEnd {\n Sender,\n Receiver,\n}\n\n/// Writing end of a Unix pipe.\n///\n/// It can be constructed from a FIFO file with [`OpenOptions::open_sender`].\n///\n/// Opening a named pipe for writing involves a few steps.\n/// Call to [`OpenOptions::open_sender`] might fail with an error indicating\n/// different things:\n///\n/// * [`io::ErrorKind::NotFound`] - There is no file at the specified path.\n/// * [`io::ErrorKind::InvalidInput`] - The file exists, but it is not a FIFO.\n/// * [`ENXIO`] - The file is a FIFO, but no process has it open for reading.\n/// Sleep for a while and try again.\n/// * Other OS errors not specific to opening FIFO files.\n///\n/// Opening a `Sender` from a FIFO file should look like this:\n///\n/// ```no_run\n/// use tokio::net::unix::pipe;\n/// use tokio::time::{self, Duration};\n///\n/// const FIFO_NAME: &str = \"path/to/a/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// // Wait for a reader to open the file.\n/// let tx = loop {\n/// match pipe::OpenOptions::new().open_sender(FIFO_NAME) {\n/// Ok(tx) => break tx,\n/// Err(e) if e.raw_os_error() == Some(libc::ENXIO) => {},\n/// Err(e) => return Err(e.into()),\n/// }\n///\n/// time::sleep(Duration::from_millis(50)).await;\n/// };\n/// # Ok(())\n/// # }\n/// ```\n///\n/// On Linux, it is possible to create a `Sender` without waiting in a sleeping\n/// loop. This is done by opening a named pipe in read-write access mode with\n/// `OpenOptions::read_write`. This way, a `Sender` can at the same time hold\n/// both a writing end and a reading end, and the latter allows to open a FIFO\n/// without [`ENXIO`] error since the pipe is open for reading as well.\n///\n/// `Sender` cannot be used to read from a pipe, so in practice the read access\n/// is only used when a FIFO is opened. However, using a `Sender` in read-write\n/// mode **may lead to lost data**, because written data will be dropped by the\n/// system as soon as all pipe ends are closed. To avoid lost data you have to\n/// make sure that a reading end has been opened before dropping a `Sender`.\n///\n/// Note that using read-write access mode with FIFO files is not defined by\n/// the POSIX standard and it is only guaranteed to work on Linux.\n///\n/// ```ignore\n/// use tokio::io::AsyncWriteExt;\n/// use tokio::net::unix::pipe;\n///\n/// const FIFO_NAME: &str = \"path/to/a/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let mut tx = pipe::OpenOptions::new()\n/// .read_write(true)\n/// .open_sender(FIFO_NAME)?;\n///\n/// // Asynchronously write to the pipe before a reader.\n/// tx.write_all(b\"hello world\").await?;\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`ENXIO`]: https://docs.rs/libc/latest/libc/constant.ENXIO.html\n#[derive(Debug)]\npub struct Sender {\n io: PollEvented,\n}\n\nimpl Sender {\n fn from_mio(mio_tx: mio_pipe::Sender) -> io::Result {\n let io = PollEvented::new_with_interest(mio_tx, Interest::WRITABLE)?;\n Ok(Sender { io })\n }\n\n /// Creates a new `Sender` from a [`File`].\n ///\n /// This function is intended to construct a pipe from a [`File`] representing\n /// a special FIFO file. It will check if the file is a pipe and has write access,\n /// set it in non-blocking mode and perform the conversion.\n ///\n /// # Errors\n ///\n /// Fails with `io::ErrorKind::InvalidInput` if the file is not a pipe or it\n /// does not have write access. Also fails with any standard OS error if it occurs.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_file(file: File) -> io::Result {\n Sender::from_owned_fd(file.into())\n }\n\n /// Creates a new `Sender` from an [`OwnedFd`].\n ///\n /// This function is intended to construct a pipe from an [`OwnedFd`] representing\n /// an anonymous pipe or a special FIFO file. It will check if the file descriptor\n /// is a pipe and has write access, set it in non-blocking mode and perform the\n /// conversion.\n ///\n /// # Errors\n ///\n /// Fails with `io::ErrorKind::InvalidInput` if the file descriptor is not a pipe\n /// or it does not have write access. Also fails with any standard OS error if it\n /// occurs.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_owned_fd(owned_fd: OwnedFd) -> io::Result {\n if !is_pipe(owned_fd.as_fd())? {\n return Err(io::Error::new(io::ErrorKind::InvalidInput, \"not a pipe\"));\n }\n\n let flags = get_file_flags(owned_fd.as_fd())?;\n if has_write_access(flags) {\n set_nonblocking(owned_fd.as_fd(), flags)?;\n Sender::from_owned_fd_unchecked(owned_fd)\n } else {\n Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n \"not in O_WRONLY or O_RDWR access mode\",\n ))\n }\n }\n\n /// Creates a new `Sender` from a [`File`] without checking pipe properties.\n ///\n /// This function is intended to construct a pipe from a File representing\n /// a special FIFO file. The conversion assumes nothing about the underlying\n /// file; it is left up to the user to make sure it is opened with write access,\n /// represents a pipe and is set in non-blocking mode.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::fs::OpenOptions;\n /// use std::os::unix::fs::{FileTypeExt, OpenOptionsExt};\n /// # use std::error::Error;\n ///\n /// const FIFO_NAME: &str = \"path/to/a/fifo\";\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let file = OpenOptions::new()\n /// .write(true)\n /// .custom_flags(libc::O_NONBLOCK)\n /// .open(FIFO_NAME)?;\n /// if file.metadata()?.file_type().is_fifo() {\n /// let tx = pipe::Sender::from_file_unchecked(file)?;\n /// /* use the Sender */\n /// }\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_file_unchecked(file: File) -> io::Result {\n Sender::from_owned_fd_unchecked(file.into())\n }\n\n /// Creates a new `Sender` from an [`OwnedFd`] without checking pipe properties.\n ///\n /// This function is intended to construct a pipe from an [`OwnedFd`] representing\n /// an anonymous pipe or a special FIFO file. The conversion assumes nothing about\n /// the underlying pipe; it is left up to the user to make sure that the file\n /// descriptor represents the writing end of a pipe and the pipe is set in\n /// non-blocking mode.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_owned_fd_unchecked(owned_fd: OwnedFd) -> io::Result {\n // Safety: OwnedFd represents a valid, open file descriptor.\n let mio_tx = unsafe { mio_pipe::Sender::from_raw_fd(owned_fd.into_raw_fd()) };\n Sender::from_mio(mio_tx)\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function can be used instead of [`writable()`] to check the returned\n /// ready set for [`Ready::WRITABLE`] and [`Ready::WRITE_CLOSED`] events.\n ///\n /// The function may complete without the pipe being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// [`writable()`]: Self::writable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the pipe to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with [`try_write()`].\n ///\n /// [`try_write()`]: Self::try_write\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a writing end of a fifo\n /// let tx = pipe::OpenOptions::new().open_sender(\"path/to/a/fifo\")?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// tx.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match tx.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write readiness.\n ///\n /// If the pipe is not currently ready for writing, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for writing, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_write_ready` or `poll_write`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// [`writable`]: Self::writable\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n pub fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to write a buffer to the pipe, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written. If the length of `buf` is not\n /// greater than `PIPE_BUF` (an OS constant, 4096 under Linux), then the\n /// write is guaranteed to be atomic, i.e. either the entire content of\n /// `buf` will be written or this method will fail with `WouldBlock`. There\n /// is no such guarantee if `buf` is larger than `PIPE_BUF`.\n ///\n /// This function is usually paired with [`writable`].\n ///\n /// [`writable`]: Self::writable\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a writing end of a fifo\n /// let tx = pipe::OpenOptions::new().open_sender(\"path/to/a/fifo\")?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// tx.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match tx.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write(buf))\n }\n\n /// Tries to write several buffers to the pipe, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// If the total length of buffers is not greater than `PIPE_BUF` (an OS\n /// constant, 4096 under Linux), then the write is guaranteed to be atomic,\n /// i.e. either the entire contents of buffers will be written or this\n /// method will fail with `WouldBlock`. There is no such guarantee if the\n /// total length of buffers is greater than `PIPE_BUF`.\n ///\n /// This function is usually paired with [`writable`].\n ///\n /// [`try_write()`]: Self::try_write()\n /// [`writable`]: Self::writable\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a writing end of a fifo\n /// let tx = pipe::OpenOptions::new().open_sender(\"path/to/a/fifo\")?;\n ///\n /// let bufs = [io::IoSlice::new(b\"hello \"), io::IoSlice::new(b\"world\")];\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// tx.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match tx.try_write_vectored(&bufs) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write_vectored(buf))\n }\n\n /// Converts the pipe into an [`OwnedFd`] in blocking mode.\n ///\n /// This function will deregister this pipe end from the event loop, set\n /// it in blocking mode and perform the conversion.\n pub fn into_blocking_fd(self) -> io::Result {\n let fd = self.into_nonblocking_fd()?;\n set_blocking(&fd)?;\n Ok(fd)\n }\n\n /// Converts the pipe into an [`OwnedFd`] in nonblocking mode.\n ///\n /// This function will deregister this pipe end from the event loop and\n /// perform the conversion. The returned file descriptor will be in nonblocking\n /// mode.\n pub fn into_nonblocking_fd(self) -> io::Result {\n let mio_pipe = self.io.into_inner()?;\n\n // Safety: the pipe is now deregistered from the event loop\n // and we are the only owner of this pipe end.\n let owned_fd = unsafe { OwnedFd::from_raw_fd(mio_pipe.into_raw_fd()) };\n\n Ok(owned_fd)\n }\n}\n\nimpl AsyncWrite for Sender {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.io.poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.io.poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsRawFd for Sender {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n}\n\nimpl AsFd for Sender {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\n/// Reading end of a Unix pipe.\n///\n/// It can be constructed from a FIFO file with [`OpenOptions::open_receiver`].\n///\n/// # Examples\n///\n/// Receiving messages from a named pipe in a loop:\n///\n/// ```no_run\n/// use tokio::net::unix::pipe;\n/// use tokio::io::{self, AsyncReadExt};\n///\n/// const FIFO_NAME: &str = \"path/to/a/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let mut rx = pipe::OpenOptions::new().open_receiver(FIFO_NAME)?;\n/// loop {\n/// let mut msg = vec![0; 256];\n/// match rx.read_exact(&mut msg).await {\n/// Ok(_) => {\n/// /* handle the message */\n/// }\n/// Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => {\n/// // Writing end has been closed, we should reopen the pipe.\n/// rx = pipe::OpenOptions::new().open_receiver(FIFO_NAME)?;\n/// }\n/// Err(e) => return Err(e.into()),\n/// }\n/// }\n/// # }\n/// ```\n///\n/// On Linux, you can use a `Receiver` in read-write access mode to implement\n/// resilient reading from a named pipe. Unlike `Receiver` opened in read-only\n/// mode, read from a pipe in read-write mode will not fail with `UnexpectedEof`\n/// when the writing end is closed. This way, a `Receiver` can asynchronously\n/// wait for the next writer to open the pipe.\n///\n/// You should not use functions waiting for EOF such as [`read_to_end`] with\n/// a `Receiver` in read-write access mode, since it **may wait forever**.\n/// `Receiver` in this mode also holds an open writing end, which prevents\n/// receiving EOF.\n///\n/// To set the read-write access mode you can use `OpenOptions::read_write`.\n/// Note that using read-write access mode with FIFO files is not defined by\n/// the POSIX standard and it is only guaranteed to work on Linux.\n///\n/// ```ignore\n/// use tokio::net::unix::pipe;\n/// use tokio::io::AsyncReadExt;\n/// # use std::error::Error;\n///\n/// const FIFO_NAME: &str = \"path/to/a/fifo\";\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let mut rx = pipe::OpenOptions::new()\n/// .read_write(true)\n/// .open_receiver(FIFO_NAME)?;\n/// loop {\n/// let mut msg = vec![0; 256];\n/// rx.read_exact(&mut msg).await?;\n/// /* handle the message */\n/// }\n/// # }\n/// ```\n///\n/// [`read_to_end`]: crate::io::AsyncReadExt::read_to_end\n#[derive(Debug)]\npub struct Receiver {\n io: PollEvented,\n}\n\nimpl Receiver {\n fn from_mio(mio_rx: mio_pipe::Receiver) -> io::Result {\n let io = PollEvented::new_with_interest(mio_rx, Interest::READABLE)?;\n Ok(Receiver { io })\n }\n\n /// Creates a new `Receiver` from a [`File`].\n ///\n /// This function is intended to construct a pipe from a [`File`] representing\n /// a special FIFO file. It will check if the file is a pipe and has read access,\n /// set it in non-blocking mode and perform the conversion.\n ///\n /// # Errors\n ///\n /// Fails with `io::ErrorKind::InvalidInput` if the file is not a pipe or it\n /// does not have read access. Also fails with any standard OS error if it occurs.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_file(file: File) -> io::Result {\n Receiver::from_owned_fd(file.into())\n }\n\n /// Creates a new `Receiver` from an [`OwnedFd`].\n ///\n /// This function is intended to construct a pipe from an [`OwnedFd`] representing\n /// an anonymous pipe or a special FIFO file. It will check if the file descriptor\n /// is a pipe and has read access, set it in non-blocking mode and perform the\n /// conversion.\n ///\n /// # Errors\n ///\n /// Fails with `io::ErrorKind::InvalidInput` if the file descriptor is not a pipe\n /// or it does not have read access. Also fails with any standard OS error if it\n /// occurs.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_owned_fd(owned_fd: OwnedFd) -> io::Result {\n if !is_pipe(owned_fd.as_fd())? {\n return Err(io::Error::new(io::ErrorKind::InvalidInput, \"not a pipe\"));\n }\n\n let flags = get_file_flags(owned_fd.as_fd())?;\n if has_read_access(flags) {\n set_nonblocking(owned_fd.as_fd(), flags)?;\n Receiver::from_owned_fd_unchecked(owned_fd)\n } else {\n Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n \"not in O_RDONLY or O_RDWR access mode\",\n ))\n }\n }\n\n /// Creates a new `Receiver` from a [`File`] without checking pipe properties.\n ///\n /// This function is intended to construct a pipe from a File representing\n /// a special FIFO file. The conversion assumes nothing about the underlying\n /// file; it is left up to the user to make sure it is opened with read access,\n /// represents a pipe and is set in non-blocking mode.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::fs::OpenOptions;\n /// use std::os::unix::fs::{FileTypeExt, OpenOptionsExt};\n /// # use std::error::Error;\n ///\n /// const FIFO_NAME: &str = \"path/to/a/fifo\";\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let file = OpenOptions::new()\n /// .read(true)\n /// .custom_flags(libc::O_NONBLOCK)\n /// .open(FIFO_NAME)?;\n /// if file.metadata()?.file_type().is_fifo() {\n /// let rx = pipe::Receiver::from_file_unchecked(file)?;\n /// /* use the Receiver */\n /// }\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_file_unchecked(file: File) -> io::Result {\n Receiver::from_owned_fd_unchecked(file.into())\n }\n\n /// Creates a new `Receiver` from an [`OwnedFd`] without checking pipe properties.\n ///\n /// This function is intended to construct a pipe from an [`OwnedFd`] representing\n /// an anonymous pipe or a special FIFO file. The conversion assumes nothing about\n /// the underlying pipe; it is left up to the user to make sure that the file\n /// descriptor represents the reading end of a pipe and the pipe is set in\n /// non-blocking mode.\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n pub fn from_owned_fd_unchecked(owned_fd: OwnedFd) -> io::Result {\n // Safety: OwnedFd represents a valid, open file descriptor.\n let mio_rx = unsafe { mio_pipe::Receiver::from_raw_fd(owned_fd.into_raw_fd()) };\n Receiver::from_mio(mio_rx)\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function can be used instead of [`readable()`] to check the returned\n /// ready set for [`Ready::READABLE`] and [`Ready::READ_CLOSED`] events.\n ///\n /// The function may complete without the pipe being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// [`readable()`]: Self::readable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the pipe to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with [`try_read()`].\n ///\n /// [`try_read()`]: Self::try_read()\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a reading end of a fifo\n /// let rx = pipe::OpenOptions::new().open_receiver(\"path/to/a/fifo\")?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// rx.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match rx.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read readiness.\n ///\n /// If the pipe is not currently ready for reading, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for reading, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_read_ready` or `poll_read`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// [`readable`]: Self::readable\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n pub fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to read data from the pipe into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Reads any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually [`readable()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The pipe's writing end is closed and will no longer write data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the pipe is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a reading end of a fifo\n /// let rx = pipe::OpenOptions::new().open_receiver(\"path/to/a/fifo\")?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// rx.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match rx.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read(buf))\n }\n\n /// Tries to read data from the pipe into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Reads any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] is used with this function.\n ///\n /// [`try_read()`]: Self::try_read()\n /// [`readable()`]: Self::readable()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the pipe's writing end is\n /// closed and will no longer write data. If the pipe is not ready to read\n /// data `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a reading end of a fifo\n /// let rx = pipe::OpenOptions::new().open_receiver(\"path/to/a/fifo\")?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// rx.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf_a = [0; 512];\n /// let mut buf_b = [0; 1024];\n /// let mut bufs = [\n /// io::IoSliceMut::new(&mut buf_a),\n /// io::IoSliceMut::new(&mut buf_b),\n /// ];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match rx.try_read_vectored(&mut bufs) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read_vectored(bufs))\n }\n\n cfg_io_util! {\n /// Tries to read data from the pipe into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Reads any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable\n /// [`ready()`]: Self::ready\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the pipe's writing end is\n /// closed and will no longer write data. If the pipe is not ready to read\n /// data `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::unix::pipe;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// // Open a reading end of a fifo\n /// let rx = pipe::OpenOptions::new().open_receiver(\"path/to/a/fifo\")?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// rx.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(4096);\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match rx.try_read_buf(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n use std::io::Read;\n\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: `mio_pipe::Receiver` uses a `std::fs::File` underneath,\n // which correctly handles reads into uninitialized memory.\n let n = (&*self.io).read(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n }\n\n /// Converts the pipe into an [`OwnedFd`] in blocking mode.\n ///\n /// This function will deregister this pipe end from the event loop, set\n /// it in blocking mode and perform the conversion.\n pub fn into_blocking_fd(self) -> io::Result {\n let fd = self.into_nonblocking_fd()?;\n set_blocking(&fd)?;\n Ok(fd)\n }\n\n /// Converts the pipe into an [`OwnedFd`] in nonblocking mode.\n ///\n /// This function will deregister this pipe end from the event loop and\n /// perform the conversion. Returned file descriptor will be in nonblocking\n /// mode.\n pub fn into_nonblocking_fd(self) -> io::Result {\n let mio_pipe = self.io.into_inner()?;\n\n // Safety: the pipe is now deregistered from the event loop\n // and we are the only owner of this pipe end.\n let owned_fd = unsafe { OwnedFd::from_raw_fd(mio_pipe.into_raw_fd()) };\n\n Ok(owned_fd)\n }\n}\n\nimpl AsyncRead for Receiver {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n // Safety: `mio_pipe::Receiver` uses a `std::fs::File` underneath,\n // which correctly handles reads into uninitialized memory.\n unsafe { self.io.poll_read(cx, buf) }\n }\n}\n\nimpl AsRawFd for Receiver {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n}\n\nimpl AsFd for Receiver {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\n/// Checks if the file descriptor is a pipe or a FIFO.\nfn is_pipe(fd: BorrowedFd<'_>) -> io::Result {\n // Safety: `libc::stat` is C-like struct used for syscalls and all-zero\n // byte pattern forms a valid value.\n let mut stat: libc::stat = unsafe { std::mem::zeroed() };\n\n // Safety: it's safe to call `fstat` with a valid, open file descriptor\n // and a valid pointer to a `stat` struct.\n let r = unsafe { libc::fstat(fd.as_raw_fd(), &mut stat) };\n\n if r == -1 {\n Err(io::Error::last_os_error())\n } else {\n Ok((stat.st_mode as libc::mode_t & libc::S_IFMT) == libc::S_IFIFO)\n }\n}\n\n/// Gets file descriptor's flags by fcntl.\nfn get_file_flags(fd: BorrowedFd<'_>) -> io::Result {\n // Safety: it's safe to use `fcntl` to read flags of a valid, open file descriptor.\n let flags = unsafe { libc::fcntl(fd.as_raw_fd(), libc::F_GETFL) };\n if flags < 0 {\n Err(io::Error::last_os_error())\n } else {\n Ok(flags)\n }\n}\n\n/// Checks for `O_RDONLY` or `O_RDWR` access mode.\nfn has_read_access(flags: libc::c_int) -> bool {\n let mode = flags & libc::O_ACCMODE;\n mode == libc::O_RDONLY || mode == libc::O_RDWR\n}\n\n/// Checks for `O_WRONLY` or `O_RDWR` access mode.\nfn has_write_access(flags: libc::c_int) -> bool {\n let mode = flags & libc::O_ACCMODE;\n mode == libc::O_WRONLY || mode == libc::O_RDWR\n}\n\n/// Sets file descriptor's flags with `O_NONBLOCK` by fcntl.\nfn set_nonblocking(fd: BorrowedFd<'_>, current_flags: libc::c_int) -> io::Result<()> {\n let flags = current_flags | libc::O_NONBLOCK;\n\n if flags != current_flags {\n // Safety: it's safe to use `fcntl` to set the `O_NONBLOCK` flag of a valid,\n // open file descriptor.\n let ret = unsafe { libc::fcntl(fd.as_raw_fd(), libc::F_SETFL, flags) };\n if ret < 0 {\n return Err(io::Error::last_os_error());\n }\n }\n\n Ok(())\n}\n\n/// Removes `O_NONBLOCK` from fd's flags.\nfn set_blocking(fd: &T) -> io::Result<()> {\n // Safety: it's safe to use `fcntl` to read flags of a valid, open file descriptor.\n let previous = unsafe { libc::fcntl(fd.as_raw_fd(), libc::F_GETFL) };\n if previous == -1 {\n return Err(io::Error::last_os_error());\n }\n\n let new = previous & !libc::O_NONBLOCK;\n\n // Safety: it's safe to use `fcntl` to unset the `O_NONBLOCK` flag of a valid,\n // open file descriptor.\n let r = unsafe { libc::fcntl(fd.as_raw_fd(), libc::F_SETFL, new) };\n if r == -1 {\n Err(io::Error::last_os_error())\n } else {\n Ok(())\n }\n}\n" }, { "path": "tokio/src/net/unix/socket.rs", "content": "use std::io;\nuse std::path::Path;\n\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd};\n\nuse crate::net::{UnixDatagram, UnixListener, UnixStream};\n\ncfg_net_unix! {\n /// A Unix socket that has not yet been converted to a [`UnixStream`], [`UnixDatagram`], or\n /// [`UnixListener`].\n ///\n /// `UnixSocket` wraps an operating system socket and enables the caller to\n /// configure the socket before establishing a connection or accepting\n /// inbound connections. The caller is able to set socket option and explicitly\n /// bind the socket with a socket address.\n ///\n /// The underlying socket is closed when the `UnixSocket` value is dropped.\n ///\n /// `UnixSocket` should only be used directly if the default configuration used\n /// by [`UnixStream::connect`], [`UnixDatagram::bind`], and [`UnixListener::bind`]\n /// does not meet the required use case.\n ///\n /// Calling `UnixStream::connect(path)` effectively performs the same function as:\n ///\n /// ```no_run\n /// use tokio::net::UnixSocket;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let path = dir.path().join(\"bind_path\");\n /// let socket = UnixSocket::new_stream()?;\n ///\n /// let stream = socket.connect(path).await?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// Calling `UnixDatagram::bind(path)` effectively performs the same function as:\n ///\n /// ```no_run\n /// use tokio::net::UnixSocket;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let path = dir.path().join(\"bind_path\");\n /// let socket = UnixSocket::new_datagram()?;\n /// socket.bind(path)?;\n ///\n /// let datagram = socket.datagram()?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// Calling `UnixListener::bind(path)` effectively performs the same function as:\n ///\n /// ```no_run\n /// use tokio::net::UnixSocket;\n /// use std::error::Error;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let path = dir.path().join(\"bind_path\");\n /// let socket = UnixSocket::new_stream()?;\n /// socket.bind(path)?;\n ///\n /// let listener = socket.listen(1024)?;\n ///\n /// Ok(())\n /// }\n /// ```\n ///\n /// Setting socket options not explicitly provided by `UnixSocket` may be done by\n /// accessing the [`RawFd`]/[`RawSocket`] using [`AsRawFd`]/[`AsRawSocket`] and\n /// setting the option with a crate like [`socket2`].\n ///\n /// [`RawFd`]: std::os::fd::RawFd\n /// [`RawSocket`]: https://doc.rust-lang.org/std/os/windows/io/type.RawSocket.html\n /// [`AsRawFd`]: std::os::fd::AsRawFd\n /// [`AsRawSocket`]: https://doc.rust-lang.org/std/os/windows/io/trait.AsRawSocket.html\n /// [`socket2`]: https://docs.rs/socket2/\n #[derive(Debug)]\n pub struct UnixSocket {\n inner: socket2::Socket,\n }\n}\n\nimpl UnixSocket {\n fn ty(&self) -> socket2::Type {\n self.inner.r#type().unwrap()\n }\n\n /// Creates a new Unix datagram socket.\n ///\n /// Calls `socket(2)` with `AF_UNIX` and `SOCK_DGRAM`.\n ///\n /// # Returns\n ///\n /// On success, the newly created [`UnixSocket`] is returned. If an error is\n /// encountered, it is returned instead.\n pub fn new_datagram() -> io::Result {\n UnixSocket::new(socket2::Type::DGRAM)\n }\n\n /// Creates a new Unix stream socket.\n ///\n /// Calls `socket(2)` with `AF_UNIX` and `SOCK_STREAM`.\n ///\n /// # Returns\n ///\n /// On success, the newly created [`UnixSocket`] is returned. If an error is\n /// encountered, it is returned instead.\n pub fn new_stream() -> io::Result {\n UnixSocket::new(socket2::Type::STREAM)\n }\n\n fn new(ty: socket2::Type) -> io::Result {\n #[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"netbsd\",\n target_os = \"openbsd\"\n ))]\n let ty = ty.nonblocking();\n let inner = socket2::Socket::new(socket2::Domain::UNIX, ty, None)?;\n #[cfg(not(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"illumos\",\n target_os = \"linux\",\n target_os = \"netbsd\",\n target_os = \"openbsd\"\n )))]\n inner.set_nonblocking(true)?;\n Ok(UnixSocket { inner })\n }\n\n /// Binds the socket to the given address.\n ///\n /// This calls the `bind(2)` operating-system function.\n pub fn bind(&self, path: impl AsRef) -> io::Result<()> {\n let addr = socket2::SockAddr::unix(path)?;\n self.inner.bind(&addr)\n }\n\n /// Converts the socket into a `UnixListener`.\n ///\n /// `backlog` defines the maximum number of pending connections are queued\n /// by the operating system at any given time. Connection are removed from\n /// the queue with [`UnixListener::accept`]. When the queue is full, the\n /// operating-system will start rejecting connections.\n ///\n /// Calling this function on a socket created by [`new_datagram`] will return an error.\n ///\n /// This calls the `listen(2)` operating-system function, marking the socket\n /// as a passive socket.\n ///\n /// [`new_datagram`]: `UnixSocket::new_datagram`\n pub fn listen(self, backlog: u32) -> io::Result {\n if self.ty() == socket2::Type::DGRAM {\n return Err(io::Error::new(\n io::ErrorKind::Other,\n \"listen cannot be called on a datagram socket\",\n ));\n }\n\n self.inner.listen(backlog as i32)?;\n let mio = {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = self.inner.into_raw_fd();\n unsafe { mio::net::UnixListener::from_raw_fd(raw_fd) }\n };\n\n UnixListener::new(mio)\n }\n\n /// Establishes a Unix connection with a peer at the specified socket address.\n ///\n /// The `UnixSocket` is consumed. Once the connection is established, a\n /// connected [`UnixStream`] is returned. If the connection fails, the\n /// encountered error is returned.\n ///\n /// Calling this function on a socket created by [`new_datagram`] will return an error.\n ///\n /// This calls the `connect(2)` operating-system function.\n ///\n /// [`new_datagram`]: `UnixSocket::new_datagram`\n pub async fn connect(self, path: impl AsRef) -> io::Result {\n if self.ty() == socket2::Type::DGRAM {\n return Err(io::Error::new(\n io::ErrorKind::Other,\n \"connect cannot be called on a datagram socket\",\n ));\n }\n\n let addr = socket2::SockAddr::unix(path)?;\n if let Err(err) = self.inner.connect(&addr) {\n if err.raw_os_error() != Some(libc::EINPROGRESS) {\n return Err(err);\n }\n }\n let mio = {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = self.inner.into_raw_fd();\n unsafe { mio::net::UnixStream::from_raw_fd(raw_fd) }\n };\n\n UnixStream::connect_mio(mio).await\n }\n\n /// Converts the socket into a [`UnixDatagram`].\n ///\n /// Calling this function on a socket created by [`new_stream`] will return an error.\n ///\n /// [`new_stream`]: `UnixSocket::new_stream`\n pub fn datagram(self) -> io::Result {\n if self.ty() == socket2::Type::STREAM {\n return Err(io::Error::new(\n io::ErrorKind::Other,\n \"datagram cannot be called on a stream socket\",\n ));\n }\n let mio = {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let raw_fd = self.inner.into_raw_fd();\n unsafe { mio::net::UnixDatagram::from_raw_fd(raw_fd) }\n };\n\n UnixDatagram::from_mio(mio)\n }\n}\n\nimpl AsRawFd for UnixSocket {\n fn as_raw_fd(&self) -> RawFd {\n self.inner.as_raw_fd()\n }\n}\n\nimpl AsFd for UnixSocket {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\nimpl FromRawFd for UnixSocket {\n unsafe fn from_raw_fd(fd: RawFd) -> UnixSocket {\n // Safety: exactly the same safety requirements as the\n // `FromRawFd::from_raw_fd` trait method.\n let inner = unsafe { socket2::Socket::from_raw_fd(fd) };\n UnixSocket { inner }\n }\n}\n\nimpl IntoRawFd for UnixSocket {\n fn into_raw_fd(self) -> RawFd {\n self.inner.into_raw_fd()\n }\n}\n" }, { "path": "tokio/src/net/unix/socketaddr.rs", "content": "use std::fmt;\nuse std::path::Path;\n\n/// An address associated with a Tokio Unix socket.\n///\n/// This type is a thin wrapper around [`std::os::unix::net::SocketAddr`]. You\n/// can convert to and from the standard library `SocketAddr` type using the\n/// [`From`] trait.\n#[derive(Clone)]\npub struct SocketAddr(pub(super) std::os::unix::net::SocketAddr);\n\nimpl SocketAddr {\n /// Returns `true` if the address is unnamed.\n ///\n /// Documentation reflected in [`SocketAddr`].\n ///\n /// [`SocketAddr`]: std::os::unix::net::SocketAddr\n pub fn is_unnamed(&self) -> bool {\n self.0.is_unnamed()\n }\n\n /// Returns the contents of this address if it is a `pathname` address.\n ///\n /// Documentation reflected in [`SocketAddr`].\n ///\n /// [`SocketAddr`]: std::os::unix::net::SocketAddr\n pub fn as_pathname(&self) -> Option<&Path> {\n self.0.as_pathname()\n }\n\n /// Returns the contents of this address if it is in the abstract namespace.\n ///\n /// Documentation reflected in [`SocketAddrExt`].\n /// The abstract namespace is a Linux-specific feature.\n ///\n ///\n /// [`SocketAddrExt`]: std::os::linux::net::SocketAddrExt\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n #[cfg_attr(docsrs, doc(cfg(any(target_os = \"linux\", target_os = \"android\"))))]\n pub fn as_abstract_name(&self) -> Option<&[u8]> {\n #[cfg(target_os = \"android\")]\n use std::os::android::net::SocketAddrExt;\n #[cfg(target_os = \"linux\")]\n use std::os::linux::net::SocketAddrExt;\n\n self.0.as_abstract_name()\n }\n}\n\nimpl fmt::Debug for SocketAddr {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.0.fmt(fmt)\n }\n}\n\nimpl From for SocketAddr {\n fn from(value: std::os::unix::net::SocketAddr) -> Self {\n SocketAddr(value)\n }\n}\n\nimpl From for std::os::unix::net::SocketAddr {\n fn from(value: SocketAddr) -> Self {\n value.0\n }\n}\n" }, { "path": "tokio/src/net/unix/split.rs", "content": "//! `UnixStream` split support.\n//!\n//! A `UnixStream` can be split into a read half and a write half with\n//! `UnixStream::split`. The read half implements `AsyncRead` while the write\n//! half implements `AsyncWrite`.\n//!\n//! Compared to the generic split of `AsyncRead + AsyncWrite`, this specialized\n//! split has no associated overhead and enforces all invariants at the type\n//! level.\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, ReadBuf, Ready};\nuse crate::net::UnixStream;\n\nuse crate::net::unix::SocketAddr;\nuse std::io;\nuse std::net::Shutdown;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n/// Borrowed read half of a [`UnixStream`], created by [`split`].\n///\n/// Reading from a `ReadHalf` is usually done using the convenience methods found on the\n/// [`AsyncReadExt`] trait.\n///\n/// [`UnixStream`]: UnixStream\n/// [`split`]: UnixStream::split()\n/// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n#[derive(Debug)]\npub struct ReadHalf<'a>(&'a UnixStream);\n\n/// Borrowed write half of a [`UnixStream`], created by [`split`].\n///\n/// Note that in the [`AsyncWrite`] implementation of this type, [`poll_shutdown`] will\n/// shut down the [`UnixStream`] stream in the write direction.\n///\n/// Writing to an `WriteHalf` is usually done using the convenience methods found\n/// on the [`AsyncWriteExt`] trait.\n///\n/// [`UnixStream`]: UnixStream\n/// [`split`]: UnixStream::split()\n/// [`AsyncWrite`]: trait@crate::io::AsyncWrite\n/// [`poll_shutdown`]: fn@crate::io::AsyncWrite::poll_shutdown\n/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n#[derive(Debug)]\npub struct WriteHalf<'a>(&'a UnixStream);\n\npub(crate) fn split(stream: &mut UnixStream) -> (ReadHalf<'_>, WriteHalf<'_>) {\n (ReadHalf(stream), WriteHalf(stream))\n}\n\nimpl ReadHalf<'_> {\n /// Wait for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_read()`]. It can be used instead\n /// of [`readable()`] to check the returned ready set for [`Ready::READABLE`]\n /// and [`Ready::READ_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`UnixStream::ready`].\n ///\n /// [`try_read()`]: Self::try_read\n /// [`readable()`]: Self::readable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.0.ready(interest).await\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn readable(&self) -> io::Result<()> {\n self.0.readable().await\n }\n\n /// Tries to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.0.try_read(buf)\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.0.try_read_buf(buf)\n }\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: Self::try_read()\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.0.try_read_vectored(bufs)\n }\n\n /// Returns the socket address of the remote half of this connection.\n pub fn peer_addr(&self) -> io::Result {\n self.0.peer_addr()\n }\n\n /// Returns the socket address of the local half of this connection.\n pub fn local_addr(&self) -> io::Result {\n self.0.local_addr()\n }\n}\n\nimpl WriteHalf<'_> {\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_write()`]. It can be used instead\n /// of [`writable()`] to check the returned ready set for [`Ready::WRITABLE`]\n /// and [`Ready::WRITE_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`UnixStream::ready`].\n ///\n /// [`try_write()`]: Self::try_write\n /// [`writable()`]: Self::writable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.0.ready(interest).await\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn writable(&self) -> io::Result<()> {\n self.0.writable().await\n }\n\n /// Tries to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.0.try_write(buf)\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: Self::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.0.try_write_vectored(buf)\n }\n\n /// Returns the socket address of the remote half of this connection.\n pub fn peer_addr(&self) -> io::Result {\n self.0.peer_addr()\n }\n\n /// Returns the socket address of the local half of this connection.\n pub fn local_addr(&self) -> io::Result {\n self.0.local_addr()\n }\n}\n\nimpl AsyncRead for ReadHalf<'_> {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.0.poll_read_priv(cx, buf)\n }\n}\n\nimpl AsyncWrite for WriteHalf<'_> {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.0.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.0.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.0.is_write_vectored()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n self.0.shutdown_std(Shutdown::Write).into()\n }\n}\n\nimpl AsRef for ReadHalf<'_> {\n fn as_ref(&self) -> &UnixStream {\n self.0\n }\n}\n\nimpl AsRef for WriteHalf<'_> {\n fn as_ref(&self) -> &UnixStream {\n self.0\n }\n}\n" }, { "path": "tokio/src/net/unix/split_owned.rs", "content": "//! `UnixStream` owned split support.\n//!\n//! A `UnixStream` can be split into an `OwnedReadHalf` and a `OwnedWriteHalf`\n//! with the `UnixStream::into_split` method. `OwnedReadHalf` implements\n//! `AsyncRead` while `OwnedWriteHalf` implements `AsyncWrite`.\n//!\n//! Compared to the generic split of `AsyncRead + AsyncWrite`, this specialized\n//! split has no associated overhead and enforces all invariants at the type\n//! level.\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, ReadBuf, Ready};\nuse crate::net::UnixStream;\n\nuse crate::net::unix::SocketAddr;\nuse std::error::Error;\nuse std::net::Shutdown;\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{Context, Poll};\nuse std::{fmt, io};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n/// Owned read half of a [`UnixStream`], created by [`into_split`].\n///\n/// Reading from an `OwnedReadHalf` is usually done using the convenience methods found\n/// on the [`AsyncReadExt`] trait.\n///\n/// [`UnixStream`]: crate::net::UnixStream\n/// [`into_split`]: crate::net::UnixStream::into_split()\n/// [`AsyncReadExt`]: trait@crate::io::AsyncReadExt\n#[derive(Debug)]\npub struct OwnedReadHalf {\n inner: Arc,\n}\n\n/// Owned write half of a [`UnixStream`], created by [`into_split`].\n///\n/// Note that in the [`AsyncWrite`] implementation of this type,\n/// [`poll_shutdown`] will shut down the stream in the write direction.\n/// Dropping the write half will also shut down the write half of the stream.\n///\n/// Writing to an `OwnedWriteHalf` is usually done using the convenience methods\n/// found on the [`AsyncWriteExt`] trait.\n///\n/// [`UnixStream`]: crate::net::UnixStream\n/// [`into_split`]: crate::net::UnixStream::into_split()\n/// [`AsyncWrite`]: trait@crate::io::AsyncWrite\n/// [`poll_shutdown`]: fn@crate::io::AsyncWrite::poll_shutdown\n/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt\n#[derive(Debug)]\npub struct OwnedWriteHalf {\n inner: Arc,\n shutdown_on_drop: bool,\n}\n\npub(crate) fn split_owned(stream: UnixStream) -> (OwnedReadHalf, OwnedWriteHalf) {\n let arc = Arc::new(stream);\n let read = OwnedReadHalf {\n inner: Arc::clone(&arc),\n };\n let write = OwnedWriteHalf {\n inner: arc,\n shutdown_on_drop: true,\n };\n (read, write)\n}\n\npub(crate) fn reunite(\n read: OwnedReadHalf,\n write: OwnedWriteHalf,\n) -> Result {\n if Arc::ptr_eq(&read.inner, &write.inner) {\n write.forget();\n // This unwrap cannot fail as the api does not allow creating more than two Arcs,\n // and we just dropped the other half.\n Ok(Arc::try_unwrap(read.inner).expect(\"UnixStream: try_unwrap failed in reunite\"))\n } else {\n Err(ReuniteError(read, write))\n }\n}\n\n/// Error indicating that two halves were not from the same socket, and thus could\n/// not be reunited.\n#[derive(Debug)]\npub struct ReuniteError(pub OwnedReadHalf, pub OwnedWriteHalf);\n\nimpl fmt::Display for ReuniteError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(\n f,\n \"tried to reunite halves that are not from the same socket\"\n )\n }\n}\n\nimpl Error for ReuniteError {}\n\nimpl OwnedReadHalf {\n /// Attempts to put the two halves of a `UnixStream` back together and\n /// recover the original socket. Succeeds only if the two halves\n /// originated from the same call to [`into_split`].\n ///\n /// [`into_split`]: crate::net::UnixStream::into_split()\n pub fn reunite(self, other: OwnedWriteHalf) -> Result {\n reunite(self, other)\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_read()`]. It can be used instead\n /// of [`readable()`] to check the returned ready set for [`Ready::READABLE`]\n /// and [`Ready::READ_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`UnixStream::ready`].\n ///\n /// [`try_read()`]: Self::try_read\n /// [`readable()`]: Self::readable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.inner.ready(interest).await\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn readable(&self) -> io::Result<()> {\n self.inner.readable().await\n }\n\n /// Tries to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.inner.try_read(buf)\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.inner.try_read_buf(buf)\n }\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: Self::try_read()\n /// [`readable()`]: Self::readable()\n /// [`ready()`]: Self::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.inner.try_read_vectored(bufs)\n }\n\n /// Returns the socket address of the remote half of this connection.\n pub fn peer_addr(&self) -> io::Result {\n self.inner.peer_addr()\n }\n\n /// Returns the socket address of the local half of this connection.\n pub fn local_addr(&self) -> io::Result {\n self.inner.local_addr()\n }\n}\n\nimpl AsyncRead for OwnedReadHalf {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.inner.poll_read_priv(cx, buf)\n }\n}\n\nimpl OwnedWriteHalf {\n /// Attempts to put the two halves of a `UnixStream` back together and\n /// recover the original socket. Succeeds only if the two halves\n /// originated from the same call to [`into_split`].\n ///\n /// [`into_split`]: crate::net::UnixStream::into_split()\n pub fn reunite(self, other: OwnedReadHalf) -> Result {\n reunite(other, self)\n }\n\n /// Destroys the write half, but don't close the write half of the stream\n /// until the read half is dropped. If the read half has already been\n /// dropped, this closes the stream.\n pub fn forget(mut self) {\n self.shutdown_on_drop = false;\n drop(self);\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with [`try_write()`]. It can be used instead\n /// of [`writable()`] to check the returned ready set for [`Ready::WRITABLE`]\n /// and [`Ready::WRITE_CLOSED`] events.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// This function is equivalent to [`UnixStream::ready`].\n ///\n /// [`try_write()`]: Self::try_write\n /// [`writable()`]: Self::writable\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn ready(&self, interest: Interest) -> io::Result {\n self.inner.ready(interest).await\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n pub async fn writable(&self) -> io::Result<()> {\n self.inner.writable().await\n }\n\n /// Tries to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.inner.try_write(buf)\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: Self::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.inner.try_write_vectored(buf)\n }\n\n /// Returns the socket address of the remote half of this connection.\n pub fn peer_addr(&self) -> io::Result {\n self.inner.peer_addr()\n }\n\n /// Returns the socket address of the local half of this connection.\n pub fn local_addr(&self) -> io::Result {\n self.inner.local_addr()\n }\n}\n\nimpl Drop for OwnedWriteHalf {\n fn drop(&mut self) {\n if self.shutdown_on_drop {\n let _ = self.inner.shutdown_std(Shutdown::Write);\n }\n }\n}\n\nimpl AsyncWrite for OwnedWriteHalf {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.inner.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.inner.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.inner.is_write_vectored()\n }\n\n #[inline]\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n // flush is a no-op\n Poll::Ready(Ok(()))\n }\n\n // `poll_shutdown` on a write half shutdowns the stream in the \"write\" direction.\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n let res = self.inner.shutdown_std(Shutdown::Write);\n if res.is_ok() {\n Pin::into_inner(self).shutdown_on_drop = false;\n }\n res.into()\n }\n}\n\nimpl AsRef for OwnedReadHalf {\n fn as_ref(&self) -> &UnixStream {\n &self.inner\n }\n}\n\nimpl AsRef for OwnedWriteHalf {\n fn as_ref(&self) -> &UnixStream {\n &self.inner\n }\n}\n" }, { "path": "tokio/src/net/unix/stream.rs", "content": "use crate::io::{AsyncRead, AsyncWrite, Interest, PollEvented, ReadBuf, Ready};\nuse crate::net::unix::split::{split, ReadHalf, WriteHalf};\nuse crate::net::unix::split_owned::{split_owned, OwnedReadHalf, OwnedWriteHalf};\nuse crate::net::unix::ucred::{self, UCred};\nuse crate::net::unix::SocketAddr;\nuse crate::util::check_socket_for_blocking;\n\nuse std::fmt;\nuse std::future::poll_fn;\nuse std::io::{self, Read, Write};\nuse std::net::Shutdown;\n#[cfg(target_os = \"android\")]\nuse std::os::android::net::SocketAddrExt;\n#[cfg(target_os = \"linux\")]\nuse std::os::linux::net::SocketAddrExt;\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nuse std::os::unix::ffi::OsStrExt;\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, RawFd};\nuse std::os::unix::net::{self, SocketAddr as StdSocketAddr};\nuse std::path::Path;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\ncfg_net_unix! {\n /// A structure representing a connected Unix socket.\n ///\n /// This socket can be connected directly with [`UnixStream::connect`] or accepted\n /// from a listener with [`UnixListener::accept`]. Additionally, a pair of\n /// anonymous Unix sockets can be created with `UnixStream::pair`.\n ///\n /// To shut down the stream in the write direction, you can call the\n /// [`shutdown()`] method. This will cause the other peer to receive a read of\n /// length 0, indicating that no more data will be sent. This only closes\n /// the stream in one direction.\n ///\n /// [`shutdown()`]: fn@crate::io::AsyncWriteExt::shutdown\n /// [`UnixListener::accept`]: crate::net::UnixListener::accept\n #[cfg_attr(docsrs, doc(alias = \"uds\"))]\n pub struct UnixStream {\n io: PollEvented,\n }\n}\n\nimpl UnixStream {\n pub(crate) async fn connect_mio(sys: mio::net::UnixStream) -> io::Result {\n let stream = UnixStream::new(sys)?;\n\n // Once we've connected, wait for the stream to be writable as\n // that's when the actual connection has been initiated. Once we're\n // writable we check for `take_socket_error` to see if the connect\n // actually hit an error or not.\n //\n // If all that succeeded then we ship everything on up.\n poll_fn(|cx| stream.io.registration().poll_write_ready(cx)).await?;\n\n if let Some(e) = stream.io.take_error()? {\n return Err(e);\n }\n\n Ok(stream)\n }\n\n /// Connects to the socket named by `path`.\n ///\n /// This function will create a new Unix socket and connect to the path\n /// specified, associating the returned stream with the default event loop's\n /// handle.\n pub async fn connect

(path: P) -> io::Result\n where\n P: AsRef,\n {\n // On linux, abstract socket paths need to be considered.\n #[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n let addr = {\n let os_str_bytes = path.as_ref().as_os_str().as_bytes();\n if os_str_bytes.starts_with(b\"\\0\") {\n StdSocketAddr::from_abstract_name(&os_str_bytes[1..])?\n } else {\n StdSocketAddr::from_pathname(path)?\n }\n };\n #[cfg(not(any(target_os = \"linux\", target_os = \"android\")))]\n let addr = StdSocketAddr::from_pathname(path)?;\n\n let stream = mio::net::UnixStream::connect_addr(&addr)?;\n let stream = UnixStream::new(stream)?;\n\n poll_fn(|cx| stream.io.registration().poll_write_ready(cx)).await?;\n\n if let Some(e) = stream.io.take_error()? {\n return Err(e);\n }\n\n Ok(stream)\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_read()` or `try_write()`. It\n /// can be used to concurrently read / write to the same socket on a single\n /// task without splitting the socket.\n ///\n /// The function may complete without the socket being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read or write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to the stream on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use tokio::io::Interest;\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// let ready = stream.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n ///\n /// }\n ///\n /// if ready.is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the socket to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to read that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read readiness.\n ///\n /// If the unix stream is not currently ready for reading, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the unix\n /// stream becomes ready for reading, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_read_ready` or `poll_read`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_write_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the unix stream is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the unix stream is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Try to read data from the stream into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: UnixStream::readable()\n /// [`ready()`]: UnixStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The stream's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the stream is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf = [0; 4096];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read(buf))\n }\n\n /// Tries to read data from the stream into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: UnixStream::try_read()\n /// [`readable()`]: UnixStream::readable()\n /// [`ready()`]: UnixStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io::{self, IoSliceMut};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf_a = [0; 512];\n /// let mut buf_b = [0; 1024];\n /// let mut bufs = [\n /// IoSliceMut::new(&mut buf_a),\n /// IoSliceMut::new(&mut buf_b),\n /// ];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read_vectored(&mut bufs) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read_vectored(bufs))\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the socket but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: UnixStream::readable()\n /// [`ready()`]: UnixStream::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// // Wait for the socket to be readable\n /// stream.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(4096);\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_read_buf(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n use std::io::Read;\n\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `UnixStream::read` to have filled up `n` bytes in the\n // buffer.\n let n = (&*self.io).read(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n }\n\n /// Waits for the socket to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once a readiness event occurs, the method\n /// will continue to return immediately until the readiness event is\n /// consumed by an attempt to write that fails with `WouldBlock` or\n /// `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write readiness.\n ///\n /// If the unix stream is not currently ready for writing, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the unix\n /// stream becomes ready for writing, `Waker::wake` will be called on the\n /// waker.\n ///\n /// Note that on multiple calls to `poll_write_ready` or `poll_write`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_read_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the unix stream is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the unix stream is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to write a buffer to the stream, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write(buf))\n }\n\n /// Tries to write several buffers to the stream, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: UnixStream::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the stream is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // Connect to a peer\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// let bufs = [io::IoSlice::new(b\"hello \"), io::IoSlice::new(b\"world\")];\n ///\n /// loop {\n /// // Wait for the socket to be writable\n /// stream.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match stream.try_write_vectored(&bufs) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write_vectored(buf))\n }\n\n /// Tries to read or write from the socket using a user-provided IO operation.\n ///\n /// If the socket is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation on the socket by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// socket is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the socket is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UnixStream` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: UnixStream::readable()\n /// [`writable()`]: UnixStream::writable()\n /// [`ready()`]: UnixStream::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .try_io(interest, || self.io.try_io(f))\n }\n\n /// Reads or writes from the socket using a user-provided IO operation.\n ///\n /// The readiness of the socket is awaited and when the socket is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the socket by manually calling the appropriate syscall.\n /// If the operation fails because the socket is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the socket readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the socket that failed due to the socket not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the socket to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `UnixStream` type, as this will mess with the\n /// readiness flag and can cause the socket to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io\n .registration()\n .async_io(interest, || self.io.try_io(&mut f))\n .await\n }\n\n /// Creates new [`UnixStream`] from a [`std::os::unix::net::UnixStream`].\n ///\n /// This function is intended to be used to wrap a `UnixStream` from the\n /// standard library in the Tokio equivalent.\n ///\n /// # Notes\n ///\n /// The caller is responsible for ensuring that the stream is in\n /// non-blocking mode. Otherwise all I/O operations on the stream\n /// will block the thread, which will cause unexpected behavior.\n /// Non-blocking mode can be set using [`set_nonblocking`].\n ///\n /// Passing a listener in blocking mode is always erroneous,\n /// and the behavior in that case may change in the future.\n /// For example, it could panic.\n ///\n /// [`set_nonblocking`]: std::os::unix::net::UnixStream::set_nonblocking\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n /// use std::os::unix::net::UnixStream as StdUnixStream;\n /// # use std::error::Error;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let std_stream = StdUnixStream::connect(\"/path/to/the/socket\")?;\n /// std_stream.set_nonblocking(true)?;\n /// let stream = UnixStream::from_std(std_stream)?;\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if it is not called from within a runtime with\n /// IO enabled.\n ///\n /// The runtime is usually set implicitly when this function is called\n /// from a future driven by a tokio runtime, otherwise runtime can be set\n /// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.\n #[track_caller]\n pub fn from_std(stream: net::UnixStream) -> io::Result {\n check_socket_for_blocking(&stream)?;\n\n let stream = mio::net::UnixStream::from_std(stream);\n let io = PollEvented::new(stream)?;\n\n Ok(UnixStream { io })\n }\n\n /// Turns a [`tokio::net::UnixStream`] into a [`std::os::unix::net::UnixStream`].\n ///\n /// The returned [`std::os::unix::net::UnixStream`] will have nonblocking\n /// mode set as `true`. Use [`set_nonblocking`] to change the blocking\n /// mode if needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::error::Error;\n /// use std::io::Read;\n /// use tokio::net::UnixListener;\n /// # use tokio::net::UnixStream;\n /// # use tokio::io::AsyncWriteExt;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// # if cfg!(miri) { return Ok(()); } // No `socket` in miri.\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n ///\n /// let mut data = [0u8; 12];\n /// let listener = UnixListener::bind(&bind_path)?;\n /// # let handle = tokio::spawn(async {\n /// # let mut stream = UnixStream::connect(bind_path).await.unwrap();\n /// # stream.write(b\"Hello world!\").await.unwrap();\n /// # });\n /// let (tokio_unix_stream, _) = listener.accept().await?;\n /// let mut std_unix_stream = tokio_unix_stream.into_std()?;\n /// # handle.await.expect(\"The task being joined has panicked\");\n /// std_unix_stream.set_nonblocking(false)?;\n /// std_unix_stream.read_exact(&mut data)?;\n /// # assert_eq!(b\"Hello world!\", &data);\n /// Ok(())\n /// }\n /// ```\n /// [`tokio::net::UnixStream`]: UnixStream\n /// [`std::os::unix::net::UnixStream`]: std::os::unix::net::UnixStream\n /// [`set_nonblocking`]: fn@std::os::unix::net::UnixStream::set_nonblocking\n pub fn into_std(self) -> io::Result {\n self.io\n .into_inner()\n .map(IntoRawFd::into_raw_fd)\n .map(|raw_fd| unsafe { std::os::unix::net::UnixStream::from_raw_fd(raw_fd) })\n }\n\n /// Creates an unnamed pair of connected sockets.\n ///\n /// This function will create a pair of interconnected Unix sockets for\n /// communicating back and forth between one another. Each socket will\n /// be associated with the default event loop's handle.\n pub fn pair() -> io::Result<(UnixStream, UnixStream)> {\n let (a, b) = mio::net::UnixStream::pair()?;\n let a = UnixStream::new(a)?;\n let b = UnixStream::new(b)?;\n\n Ok((a, b))\n }\n\n pub(crate) fn new(stream: mio::net::UnixStream) -> io::Result {\n let io = PollEvented::new(stream)?;\n Ok(UnixStream { io })\n }\n\n /// Returns the socket address of the local half of this connection.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// println!(\"{:?}\", stream.local_addr()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn local_addr(&self) -> io::Result {\n self.io.local_addr().map(SocketAddr)\n }\n\n /// Returns the socket address of the remote half of this connection.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::UnixStream;\n ///\n /// # async fn dox() -> Result<(), Box> {\n /// let dir = tempfile::tempdir().unwrap();\n /// let bind_path = dir.path().join(\"bind_path\");\n /// let stream = UnixStream::connect(bind_path).await?;\n ///\n /// println!(\"{:?}\", stream.peer_addr()?);\n /// # Ok(())\n /// # }\n /// ```\n pub fn peer_addr(&self) -> io::Result {\n self.io.peer_addr().map(SocketAddr)\n }\n\n /// Returns effective credentials of the process which called `connect` or `pair`.\n pub fn peer_cred(&self) -> io::Result {\n ucred::get_peer_cred(self)\n }\n\n /// Returns the value of the `SO_ERROR` option.\n pub fn take_error(&self) -> io::Result> {\n self.io.take_error()\n }\n\n /// Shuts down the read, write, or both halves of this connection.\n ///\n /// This function will cause all pending and future I/O calls on the\n /// specified portions to immediately return with an appropriate value\n /// (see the documentation of `Shutdown`).\n pub(super) fn shutdown_std(&self, how: Shutdown) -> io::Result<()> {\n self.io.shutdown(how)\n }\n\n // These lifetime markers also appear in the generated documentation, and make\n // it more clear that this is a *borrowed* split.\n #[allow(clippy::needless_lifetimes)]\n /// Splits a `UnixStream` into a read half and a write half, which can be used\n /// to read and write the stream concurrently.\n ///\n /// This method is more efficient than [`into_split`], but the halves cannot be\n /// moved into independently spawned tasks.\n ///\n /// [`into_split`]: Self::into_split()\n pub fn split<'a>(&'a mut self) -> (ReadHalf<'a>, WriteHalf<'a>) {\n split(self)\n }\n\n /// Splits a `UnixStream` into a read half and a write half, which can be used\n /// to read and write the stream concurrently.\n ///\n /// Unlike [`split`], the owned halves can be moved to separate tasks, however\n /// this comes at the cost of a heap allocation.\n ///\n /// **Note:** Dropping the write half will only shut down the write half of the\n /// stream. This is equivalent to calling [`shutdown()`] on the `UnixStream`.\n ///\n /// [`split`]: Self::split()\n /// [`shutdown()`]: fn@crate::io::AsyncWriteExt::shutdown\n pub fn into_split(self) -> (OwnedReadHalf, OwnedWriteHalf) {\n split_owned(self)\n }\n}\n\nimpl TryFrom for UnixStream {\n type Error = io::Error;\n\n /// Consumes stream, returning the tokio I/O object.\n ///\n /// This is equivalent to\n /// [`UnixStream::from_std(stream)`](UnixStream::from_std).\n fn try_from(stream: net::UnixStream) -> io::Result {\n Self::from_std(stream)\n }\n}\n\nimpl AsyncRead for UnixStream {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.poll_read_priv(cx, buf)\n }\n}\n\nimpl AsyncWrite for UnixStream {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.poll_write_priv(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.poll_write_vectored_priv(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n self.shutdown_std(std::net::Shutdown::Write)?;\n Poll::Ready(Ok(()))\n }\n}\n\nimpl UnixStream {\n // == Poll IO functions that takes `&self` ==\n //\n // To read or write without mutable access to the `UnixStream`, combine the\n // `poll_read_ready` or `poll_write_ready` methods with the `try_read` or\n // `try_write` methods.\n\n pub(crate) fn poll_read_priv(\n &self,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n // Safety: `UnixStream::read` correctly handles reads into uninitialized memory\n unsafe { self.io.poll_read(cx, buf) }\n }\n\n pub(crate) fn poll_write_priv(\n &self,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.io.poll_write(cx, buf)\n }\n\n pub(super) fn poll_write_vectored_priv(\n &self,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.io.poll_write_vectored(cx, bufs)\n }\n}\n\nimpl fmt::Debug for UnixStream {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n (*self.io).fmt(f)\n }\n}\n\nimpl AsRef for UnixStream {\n fn as_ref(&self) -> &Self {\n self\n }\n}\n\nimpl AsRawFd for UnixStream {\n fn as_raw_fd(&self) -> RawFd {\n self.io.as_raw_fd()\n }\n}\n\nimpl AsFd for UnixStream {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n" }, { "path": "tokio/src/net/unix/ucred.rs", "content": "use crate::net::unix;\n\n/// Credentials of a process.\n#[derive(Copy, Clone, Eq, PartialEq, Hash, Debug)]\npub struct UCred {\n /// PID (process ID) of the process.\n pid: Option,\n /// UID (user ID) of the process.\n uid: unix::uid_t,\n /// GID (group ID) of the process.\n gid: unix::gid_t,\n}\n\nimpl UCred {\n /// Gets UID (user ID) of the process.\n pub fn uid(&self) -> unix::uid_t {\n self.uid\n }\n\n /// Gets GID (group ID) of the process.\n pub fn gid(&self) -> unix::gid_t {\n self.gid\n }\n\n /// Gets PID (process ID) of the process.\n ///\n /// This is only implemented under Linux, Android, iOS, macOS, Solaris,\n /// Illumos and Cygwin. On other platforms this will always return `None`.\n pub fn pid(&self) -> Option {\n self.pid\n }\n}\n\n#[cfg(any(\n target_os = \"linux\",\n target_os = \"redox\",\n target_os = \"android\",\n target_os = \"openbsd\",\n target_os = \"haiku\",\n target_os = \"cygwin\"\n))]\npub(crate) use self::impl_linux::get_peer_cred;\n\n#[cfg(any(target_os = \"netbsd\", target_os = \"nto\"))]\npub(crate) use self::impl_netbsd::get_peer_cred;\n\n#[cfg(any(target_os = \"dragonfly\", target_os = \"freebsd\"))]\npub(crate) use self::impl_bsd::get_peer_cred;\n\n#[cfg(any(\n target_os = \"macos\",\n target_os = \"ios\",\n target_os = \"tvos\",\n target_os = \"watchos\",\n target_os = \"visionos\"\n))]\npub(crate) use self::impl_macos::get_peer_cred;\n\n#[cfg(any(target_os = \"solaris\", target_os = \"illumos\"))]\npub(crate) use self::impl_solaris::get_peer_cred;\n\n#[cfg(target_os = \"aix\")]\npub(crate) use self::impl_aix::get_peer_cred;\n\n#[cfg(any(target_os = \"espidf\", target_os = \"vita\"))]\npub(crate) use self::impl_noproc::get_peer_cred;\n\n#[cfg(any(\n target_os = \"linux\",\n target_os = \"redox\",\n target_os = \"android\",\n target_os = \"openbsd\",\n target_os = \"haiku\",\n target_os = \"cygwin\"\n))]\npub(crate) mod impl_linux {\n use crate::net::unix::{self, UnixStream};\n\n use libc::{c_void, getsockopt, socklen_t, SOL_SOCKET, SO_PEERCRED};\n use std::{io, mem};\n\n #[cfg(target_os = \"openbsd\")]\n use libc::sockpeercred as ucred;\n #[cfg(any(\n target_os = \"linux\",\n target_os = \"redox\",\n target_os = \"android\",\n target_os = \"haiku\",\n target_os = \"cygwin\"\n ))]\n use libc::ucred;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n use std::os::unix::io::AsRawFd;\n\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut ucred = ucred {\n pid: 0,\n uid: 0,\n gid: 0,\n };\n\n let ucred_size = mem::size_of::();\n\n // These paranoid checks should be optimized-out\n assert!(mem::size_of::() <= mem::size_of::());\n assert!(ucred_size <= u32::MAX as usize);\n\n let mut ucred_size = ucred_size as socklen_t;\n\n let ret = getsockopt(\n raw_fd,\n SOL_SOCKET,\n SO_PEERCRED,\n &mut ucred as *mut ucred as *mut c_void,\n &mut ucred_size,\n );\n if ret == 0 && ucred_size as usize == mem::size_of::() {\n Ok(super::UCred {\n uid: ucred.uid as unix::uid_t,\n gid: ucred.gid as unix::gid_t,\n pid: Some(ucred.pid as unix::pid_t),\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(any(target_os = \"netbsd\", target_os = \"nto\"))]\npub(crate) mod impl_netbsd {\n use crate::net::unix::{self, UnixStream};\n\n use libc::{c_void, getsockopt, socklen_t, unpcbid, LOCAL_PEEREID, SOL_SOCKET};\n use std::io;\n use std::mem::size_of;\n use std::os::unix::io::AsRawFd;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut unpcbid = unpcbid {\n unp_pid: 0,\n unp_euid: 0,\n unp_egid: 0,\n };\n\n let unpcbid_size = size_of::();\n let mut unpcbid_size = unpcbid_size as socklen_t;\n\n let ret = getsockopt(\n raw_fd,\n SOL_SOCKET,\n LOCAL_PEEREID,\n &mut unpcbid as *mut unpcbid as *mut c_void,\n &mut unpcbid_size,\n );\n if ret == 0 && unpcbid_size as usize == size_of::() {\n Ok(super::UCred {\n uid: unpcbid.unp_euid as unix::uid_t,\n gid: unpcbid.unp_egid as unix::gid_t,\n pid: Some(unpcbid.unp_pid as unix::pid_t),\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(any(target_os = \"dragonfly\", target_os = \"freebsd\"))]\npub(crate) mod impl_bsd {\n use crate::net::unix::{self, UnixStream};\n\n use libc::getpeereid;\n use std::io;\n use std::mem::MaybeUninit;\n use std::os::unix::io::AsRawFd;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut uid = MaybeUninit::uninit();\n let mut gid = MaybeUninit::uninit();\n\n let ret = getpeereid(raw_fd, uid.as_mut_ptr(), gid.as_mut_ptr());\n\n if ret == 0 {\n Ok(super::UCred {\n uid: uid.assume_init() as unix::uid_t,\n gid: gid.assume_init() as unix::gid_t,\n pid: None,\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(any(\n target_os = \"macos\",\n target_os = \"ios\",\n target_os = \"tvos\",\n target_os = \"watchos\",\n target_os = \"visionos\"\n))]\npub(crate) mod impl_macos {\n use crate::net::unix::{self, UnixStream};\n\n use libc::{c_void, getpeereid, getsockopt, pid_t, LOCAL_PEEREPID, SOL_LOCAL};\n use std::io;\n use std::mem::size_of;\n use std::mem::MaybeUninit;\n use std::os::unix::io::AsRawFd;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut uid = MaybeUninit::uninit();\n let mut gid = MaybeUninit::uninit();\n let mut pid: MaybeUninit = MaybeUninit::uninit();\n let mut pid_size: MaybeUninit = MaybeUninit::new(size_of::() as u32);\n\n if getsockopt(\n raw_fd,\n SOL_LOCAL,\n LOCAL_PEEREPID,\n pid.as_mut_ptr() as *mut c_void,\n pid_size.as_mut_ptr(),\n ) != 0\n {\n return Err(io::Error::last_os_error());\n }\n\n assert!(pid_size.assume_init() == (size_of::() as u32));\n\n let ret = getpeereid(raw_fd, uid.as_mut_ptr(), gid.as_mut_ptr());\n\n if ret == 0 {\n Ok(super::UCred {\n uid: uid.assume_init() as unix::uid_t,\n gid: gid.assume_init() as unix::gid_t,\n pid: Some(pid.assume_init() as unix::pid_t),\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(any(target_os = \"solaris\", target_os = \"illumos\"))]\npub(crate) mod impl_solaris {\n use crate::net::unix::{self, UnixStream};\n use std::io;\n use std::os::unix::io::AsRawFd;\n use std::ptr;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut cred = ptr::null_mut();\n let ret = libc::getpeerucred(raw_fd, &mut cred);\n\n if ret == 0 {\n let uid = libc::ucred_geteuid(cred);\n let gid = libc::ucred_getegid(cred);\n let pid = libc::ucred_getpid(cred);\n\n libc::ucred_free(cred);\n\n Ok(super::UCred {\n uid: uid as unix::uid_t,\n gid: gid as unix::gid_t,\n pid: Some(pid as unix::pid_t),\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(target_os = \"aix\")]\npub(crate) mod impl_aix {\n use crate::net::unix::UnixStream;\n use std::io;\n use std::os::unix::io::AsRawFd;\n\n pub(crate) fn get_peer_cred(sock: &UnixStream) -> io::Result {\n unsafe {\n let raw_fd = sock.as_raw_fd();\n\n let mut uid = std::mem::MaybeUninit::uninit();\n let mut gid = std::mem::MaybeUninit::uninit();\n\n let ret = libc::getpeereid(raw_fd, uid.as_mut_ptr(), gid.as_mut_ptr());\n\n if ret == 0 {\n Ok(super::UCred {\n uid: uid.assume_init(),\n gid: gid.assume_init(),\n pid: None,\n })\n } else {\n Err(io::Error::last_os_error())\n }\n }\n }\n}\n\n#[cfg(any(target_os = \"espidf\", target_os = \"vita\"))]\npub(crate) mod impl_noproc {\n use crate::net::unix::UnixStream;\n use std::io;\n\n pub(crate) fn get_peer_cred(_sock: &UnixStream) -> io::Result {\n Ok(super::UCred {\n uid: 0,\n gid: 0,\n pid: None,\n })\n }\n}\n" }, { "path": "tokio/src/net/windows/mod.rs", "content": "//! Windows specific network types.\n\npub mod named_pipe;\n" }, { "path": "tokio/src/net/windows/named_pipe.rs", "content": "//! Tokio support for [Windows named pipes].\n//!\n//! [Windows named pipes]: https://docs.microsoft.com/en-us/windows/win32/ipc/named-pipes\n\nuse std::ffi::c_void;\nuse std::ffi::OsStr;\nuse std::io::{self, Read, Write};\nuse std::pin::Pin;\nuse std::ptr;\nuse std::ptr::null_mut;\nuse std::task::{Context, Poll};\n\nuse crate::io::{AsyncRead, AsyncWrite, Interest, PollEvented, ReadBuf, Ready};\nuse crate::os::windows::io::{AsHandle, AsRawHandle, BorrowedHandle, FromRawHandle, RawHandle};\n\ncfg_io_util! {\n use bytes::BufMut;\n}\n\n// Hide imports which are not used when generating documentation.\n#[cfg(windows)]\nmod doc {\n pub(super) use crate::os::windows::ffi::OsStrExt;\n pub(super) mod windows_sys {\n pub(crate) use windows_sys::{\n Win32::Foundation::*, Win32::Storage::FileSystem::*, Win32::System::Pipes::*,\n Win32::System::SystemServices::*,\n };\n }\n pub(super) use mio::windows as mio_windows;\n}\n\n// NB: none of these shows up in public API, so don't document them.\n#[cfg(not(windows))]\nmod doc {\n pub(super) mod mio_windows {\n pub type NamedPipe = crate::doc::NotDefinedHere;\n }\n}\n\nuse self::doc::*;\n\n/// A [Windows named pipe] server.\n///\n/// Accepting client connections involves creating a server with\n/// [`ServerOptions::create`] and waiting for clients to connect using\n/// [`NamedPipeServer::connect`].\n///\n/// To avoid having clients sporadically fail with\n/// [`std::io::ErrorKind::NotFound`] when they connect to a server, we must\n/// ensure that at least one server instance is available at all times. This\n/// means that the typical listen loop for a server is a bit involved, because\n/// we have to ensure that we never drop a server accidentally while a client\n/// might connect.\n///\n/// So a correctly implemented server looks like this:\n///\n/// ```no_run\n/// use std::io;\n/// use tokio::net::windows::named_pipe::ServerOptions;\n///\n/// const PIPE_NAME: &str = r\"\\\\.\\pipe\\named-pipe-idiomatic-server\";\n///\n/// # #[tokio::main] async fn main() -> std::io::Result<()> {\n/// // The first server needs to be constructed early so that clients can\n/// // be correctly connected. Otherwise calling .wait will cause the client to\n/// // error.\n/// //\n/// // Here we also make use of `first_pipe_instance`, which will ensure that\n/// // there are no other servers up and running already.\n/// let mut server = ServerOptions::new()\n/// .first_pipe_instance(true)\n/// .create(PIPE_NAME)?;\n///\n/// // Spawn the server loop.\n/// let server = tokio::spawn(async move {\n/// loop {\n/// // Wait for a client to connect.\n/// server.connect().await?;\n/// let connected_client = server;\n///\n/// // Construct the next server to be connected before sending the one\n/// // we already have of onto a task. This ensures that the server\n/// // isn't closed (after it's done in the task) before a new one is\n/// // available. Otherwise the client might error with\n/// // `io::ErrorKind::NotFound`.\n/// server = ServerOptions::new().create(PIPE_NAME)?;\n///\n/// let client = tokio::spawn(async move {\n/// /* use the connected client */\n/// # Ok::<_, std::io::Error>(())\n/// });\n/// # if true { break } // needed for type inference to work\n/// }\n///\n/// Ok::<_, io::Error>(())\n/// });\n///\n/// /* do something else not server related here */\n/// # Ok(()) }\n/// ```\n///\n/// [Windows named pipe]: https://docs.microsoft.com/en-us/windows/win32/ipc/named-pipes\n#[derive(Debug)]\npub struct NamedPipeServer {\n io: PollEvented,\n}\n\nimpl NamedPipeServer {\n /// Constructs a new named pipe server from the specified raw handle.\n ///\n /// This function will consume ownership of the handle given, passing\n /// responsibility for closing the handle to the returned object.\n ///\n /// This function is also unsafe as the primitives currently returned have\n /// the contract that they are the sole owner of the file descriptor they\n /// are wrapping. Usage of this function could accidentally allow violating\n /// this contract which can cause memory unsafety in code that relies on it\n /// being true.\n ///\n /// # Errors\n ///\n /// This errors if called outside of a [Tokio Runtime], or in a runtime that\n /// has not [enabled I/O], or if any OS-specific I/O errors occur.\n ///\n /// [Tokio Runtime]: crate::runtime::Runtime\n /// [enabled I/O]: crate::runtime::Builder::enable_io\n pub unsafe fn from_raw_handle(handle: RawHandle) -> io::Result {\n let named_pipe = unsafe { mio_windows::NamedPipe::from_raw_handle(handle) };\n\n Ok(Self {\n io: PollEvented::new(named_pipe)?,\n })\n }\n\n /// Retrieves information about the named pipe the server is associated\n /// with.\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe::{PipeEnd, PipeMode, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-info\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let server = ServerOptions::new()\n /// .pipe_mode(PipeMode::Message)\n /// .max_instances(5)\n /// .create(PIPE_NAME)?;\n ///\n /// let server_info = server.info()?;\n ///\n /// assert_eq!(server_info.end, PipeEnd::Server);\n /// assert_eq!(server_info.mode, PipeMode::Message);\n /// assert_eq!(server_info.max_instances, 5);\n /// # Ok(()) }\n /// ```\n pub fn info(&self) -> io::Result {\n // Safety: we're ensuring the lifetime of the named pipe.\n unsafe { named_pipe_info(self.io.as_raw_handle()) }\n }\n\n /// Enables a named pipe server process to wait for a client process to\n /// connect to an instance of a named pipe. A client process connects by\n /// creating a named pipe with the same name.\n ///\n /// This corresponds to the [`ConnectNamedPipe`] system call.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe in the sense that if it is used as the\n /// event in a [`select!`](crate::select) statement and some other branch\n /// completes first, then no connection events have been lost.\n ///\n /// [`ConnectNamedPipe`]: https://docs.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-connectnamedpipe\n ///\n /// # Example\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\mynamedpipe\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let pipe = ServerOptions::new().create(PIPE_NAME)?;\n ///\n /// // Wait for a client to connect.\n /// pipe.connect().await?;\n ///\n /// // Use the connected client...\n /// # Ok(()) }\n /// ```\n pub async fn connect(&self) -> io::Result<()> {\n match self.io.connect() {\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.io\n .registration()\n .async_io(Interest::WRITABLE, || self.io.connect())\n .await\n }\n x => x,\n }\n }\n\n /// Disconnects the server end of a named pipe instance from a client\n /// process.\n ///\n /// ```\n /// use tokio::io::AsyncWriteExt;\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n /// use windows_sys::Win32::Foundation::ERROR_PIPE_NOT_CONNECTED;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-disconnect\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let server = ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// let mut client = ClientOptions::new()\n /// .open(PIPE_NAME)?;\n ///\n /// // Wait for a client to become connected.\n /// server.connect().await?;\n ///\n /// // Forcibly disconnect the client.\n /// server.disconnect()?;\n ///\n /// // Write fails with an OS-specific error after client has been\n /// // disconnected.\n /// let e = client.write(b\"ping\").await.unwrap_err();\n /// assert_eq!(e.raw_os_error(), Some(ERROR_PIPE_NOT_CONNECTED as i32));\n /// # Ok(()) }\n /// ```\n pub fn disconnect(&self) -> io::Result<()> {\n self.io.disconnect()\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_read()` or `try_write()`. It\n /// can be used to concurrently read / write to the same pipe on a single\n /// task without splitting the pipe.\n ///\n /// The function may complete without the pipe being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to the pipe on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use tokio::io::Interest;\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-ready\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// loop {\n /// let ready = server.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if ready.is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the pipe to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-readable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// server.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read readiness.\n ///\n /// If the pipe is not currently ready for reading, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for reading, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_read_ready` or `poll_read`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_write_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to read data from the pipe into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeServer::readable()\n /// [`ready()`]: NamedPipeServer::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The pipe's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the pipe is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-try-read\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// server.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf = [0; 4096];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_read(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read(buf))\n }\n\n /// Tries to read data from the pipe into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: NamedPipeServer::try_read()\n /// [`readable()`]: NamedPipeServer::readable()\n /// [`ready()`]: NamedPipeServer::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the pipe's read half is closed\n /// and will no longer yield data. If the pipe is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io::{self, IoSliceMut};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-try-read-vectored\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// server.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf_a = [0; 512];\n /// let mut buf_b = [0; 1024];\n /// let mut bufs = [\n /// IoSliceMut::new(&mut buf_a),\n /// IoSliceMut::new(&mut buf_b),\n /// ];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_read_vectored(&mut bufs) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read_vectored(bufs))\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeServer::readable()\n /// [`ready()`]: NamedPipeServer::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-readable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new().create(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// server.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(4096);\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_read_buf(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n use std::io::Read;\n\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `NamedPipeServer::read` to have filled up `n` bytes in the\n // buffer.\n let n = (&*self.io).read(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n }\n\n /// Waits for the pipe to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-writable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// server.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write readiness.\n ///\n /// If the pipe is not currently ready for writing, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for writing, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_write_ready` or `poll_write`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_read_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to write a buffer to the pipe, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-try-write\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// server.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write(buf))\n }\n\n /// Tries to write several buffers to the pipe, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: NamedPipeServer::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-server-try-write-vectored\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let server = named_pipe::ServerOptions::new()\n /// .create(PIPE_NAME)?;\n ///\n /// let bufs = [io::IoSlice::new(b\"hello \"), io::IoSlice::new(b\"world\")];\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// server.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match server.try_write_vectored(&bufs) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write_vectored(buf))\n }\n\n /// Tries to read or write from the pipe using a user-provided IO operation.\n ///\n /// If the pipe is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation from the pipe by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// pipe is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the pipe is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the pipe that failed due to the pipe not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the pipe to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the\n /// methods defined on the Tokio `NamedPipeServer` type, as this will mess with\n /// the readiness flag and can cause the pipe to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeServer::readable()\n /// [`writable()`]: NamedPipeServer::writable()\n /// [`ready()`]: NamedPipeServer::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io.registration().try_io(interest, f)\n }\n\n /// Reads or writes from the pipe using a user-provided IO operation.\n ///\n /// The readiness of the pipe is awaited and when the pipe is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the pipe by manually calling the appropriate syscall.\n /// If the operation fails because the pipe is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the pipe readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the pipe that failed due to the pipe not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the pipe to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `NamedPipeServer` type, as this will mess with the\n /// readiness flag and can cause the pipe to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io.registration().async_io(interest, f).await\n }\n}\n\nimpl AsyncRead for NamedPipeServer {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n unsafe { self.io.poll_read(cx, buf) }\n }\n}\n\nimpl AsyncWrite for NamedPipeServer {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.io.poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.io.poll_write_vectored(cx, bufs)\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n\nimpl AsRawHandle for NamedPipeServer {\n fn as_raw_handle(&self) -> RawHandle {\n self.io.as_raw_handle()\n }\n}\n\nimpl AsHandle for NamedPipeServer {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n}\n\n/// A [Windows named pipe] client.\n///\n/// Constructed using [`ClientOptions::open`].\n///\n/// Connecting a client correctly involves a few steps. When connecting through\n/// [`ClientOptions::open`], it might error indicating one of two things:\n///\n/// * [`std::io::ErrorKind::NotFound`] - There is no server available.\n/// * [`ERROR_PIPE_BUSY`] - There is a server available, but it is busy. Sleep\n/// for a while and try again.\n///\n/// So a correctly implemented client looks like this:\n///\n/// ```no_run\n/// use std::time::Duration;\n/// use tokio::net::windows::named_pipe::ClientOptions;\n/// use tokio::time;\n/// use windows_sys::Win32::Foundation::ERROR_PIPE_BUSY;\n///\n/// const PIPE_NAME: &str = r\"\\\\.\\pipe\\named-pipe-idiomatic-client\";\n///\n/// # #[tokio::main] async fn main() -> std::io::Result<()> {\n/// let client = loop {\n/// match ClientOptions::new().open(PIPE_NAME) {\n/// Ok(client) => break client,\n/// Err(e) if e.raw_os_error() == Some(ERROR_PIPE_BUSY as i32) => (),\n/// Err(e) => return Err(e),\n/// }\n///\n/// time::sleep(Duration::from_millis(50)).await;\n/// };\n///\n/// /* use the connected client */\n/// # Ok(()) }\n/// ```\n///\n/// [`ERROR_PIPE_BUSY`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Foundation/constant.ERROR_PIPE_BUSY.html\n/// [Windows named pipe]: https://docs.microsoft.com/en-us/windows/win32/ipc/named-pipes\n#[derive(Debug)]\npub struct NamedPipeClient {\n io: PollEvented,\n}\n\nimpl NamedPipeClient {\n /// Constructs a new named pipe client from the specified raw handle.\n ///\n /// This function will consume ownership of the handle given, passing\n /// responsibility for closing the handle to the returned object.\n ///\n /// This function is also unsafe as the primitives currently returned have\n /// the contract that they are the sole owner of the file descriptor they\n /// are wrapping. Usage of this function could accidentally allow violating\n /// this contract which can cause memory unsafety in code that relies on it\n /// being true.\n ///\n /// # Errors\n ///\n /// This errors if called outside of a [Tokio Runtime], or in a runtime that\n /// has not [enabled I/O], or if any OS-specific I/O errors occur.\n ///\n /// [Tokio Runtime]: crate::runtime::Runtime\n /// [enabled I/O]: crate::runtime::Builder::enable_io\n pub unsafe fn from_raw_handle(handle: RawHandle) -> io::Result {\n let named_pipe = unsafe { mio_windows::NamedPipe::from_raw_handle(handle) };\n\n Ok(Self {\n io: PollEvented::new(named_pipe)?,\n })\n }\n\n /// Retrieves information about the named pipe the client is associated\n /// with.\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe::{ClientOptions, PipeEnd, PipeMode};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-info\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let client = ClientOptions::new()\n /// .open(PIPE_NAME)?;\n ///\n /// let client_info = client.info()?;\n ///\n /// assert_eq!(client_info.end, PipeEnd::Client);\n /// assert_eq!(client_info.mode, PipeMode::Message);\n /// assert_eq!(client_info.max_instances, 5);\n /// # Ok(()) }\n /// ```\n pub fn info(&self) -> io::Result {\n // Safety: we're ensuring the lifetime of the named pipe.\n unsafe { named_pipe_info(self.io.as_raw_handle()) }\n }\n\n /// Waits for any of the requested ready states.\n ///\n /// This function is usually paired with `try_read()` or `try_write()`. It\n /// can be used to concurrently read / write to the same pipe on a single\n /// task without splitting the pipe.\n ///\n /// The function may complete without the pipe being ready. This is a\n /// false-positive and attempting an operation will return with\n /// `io::ErrorKind::WouldBlock`. The function can also return with an empty\n /// [`Ready`] set, so you should always check the returned value and possibly\n /// wait again if the requested states are not set.\n ///\n /// # Examples\n ///\n /// Concurrently read and write to the pipe on the same task without\n /// splitting.\n ///\n /// ```no_run\n /// use tokio::io::Interest;\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-ready\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// let ready = client.ready(Interest::READABLE | Interest::WRITABLE).await?;\n ///\n /// if ready.is_readable() {\n /// let mut data = vec![0; 1024];\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_read(&mut data) {\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// if ready.is_writable() {\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// println!(\"write {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n /// }\n /// }\n /// ```\n pub async fn ready(&self, interest: Interest) -> io::Result {\n let event = self.io.registration().readiness(interest).await?;\n Ok(event.ready)\n }\n\n /// Waits for the pipe to become readable.\n ///\n /// This function is equivalent to `ready(Interest::READABLE)` and is usually\n /// paired with `try_read()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-readable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// let mut msg = vec![0; 1024];\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// client.readable().await?;\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_read(&mut msg) {\n /// Ok(n) => {\n /// msg.truncate(n);\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// println!(\"GOT = {:?}\", msg);\n /// Ok(())\n /// }\n /// ```\n pub async fn readable(&self) -> io::Result<()> {\n self.ready(Interest::READABLE).await?;\n Ok(())\n }\n\n /// Polls for read readiness.\n ///\n /// If the pipe is not currently ready for reading, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for reading, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_read_ready` or `poll_read`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_write_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`readable`] is not feasible. Where possible, using [`readable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for reading.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for reading.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`readable`]: method@Self::readable\n pub fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_read_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to read data from the pipe into the provided buffer, returning how\n /// many bytes were read.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeClient::readable()\n /// [`ready()`]: NamedPipeClient::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:\n ///\n /// 1. The pipe's read half is closed and will no longer yield data.\n /// 2. The specified buffer was 0 bytes in length.\n ///\n /// If the pipe is not ready to read data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-try-read\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// client.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf = [0; 4096];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_read(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read(&self, buf: &mut [u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read(buf))\n }\n\n /// Tries to read data from the pipe into the provided buffers, returning\n /// how many bytes were read.\n ///\n /// Data is copied to fill each buffer in order, with the final buffer\n /// written to possibly being only partially filled. This method behaves\n /// equivalently to a single call to [`try_read()`] with concatenated\n /// buffers.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_vectored()` is non-blocking, the buffer does not have to be\n /// stored by the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`try_read()`]: NamedPipeClient::try_read()\n /// [`readable()`]: NamedPipeClient::readable()\n /// [`ready()`]: NamedPipeClient::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the pipe's read half is closed\n /// and will no longer yield data. If the pipe is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io::{self, IoSliceMut};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-try-read-vectored\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// client.readable().await?;\n ///\n /// // Creating the buffer **after** the `await` prevents it from\n /// // being stored in the async task.\n /// let mut buf_a = [0; 512];\n /// let mut buf_b = [0; 1024];\n /// let mut bufs = [\n /// IoSliceMut::new(&mut buf_a),\n /// IoSliceMut::new(&mut buf_b),\n /// ];\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_read_vectored(&mut bufs) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_vectored(&self, bufs: &mut [io::IoSliceMut<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::READABLE, || (&*self.io).read_vectored(bufs))\n }\n\n cfg_io_util! {\n /// Tries to read data from the stream into the provided buffer, advancing the\n /// buffer's internal cursor, returning how many bytes were read.\n ///\n /// Receives any pending data from the pipe but does not wait for new data\n /// to arrive. On success, returns the number of bytes read. Because\n /// `try_read_buf()` is non-blocking, the buffer does not have to be stored by\n /// the async task and can exist entirely on the stack.\n ///\n /// Usually, [`readable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeClient::readable()\n /// [`ready()`]: NamedPipeClient::ready()\n ///\n /// # Return\n ///\n /// If data is successfully read, `Ok(n)` is returned, where `n` is the\n /// number of bytes read. `Ok(0)` indicates the stream's read half is closed\n /// and will no longer yield data. If the stream is not ready to read data\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-readable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be readable\n /// client.readable().await?;\n ///\n /// let mut buf = Vec::with_capacity(4096);\n ///\n /// // Try to read data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_read_buf(&mut buf) {\n /// Ok(0) => break,\n /// Ok(n) => {\n /// println!(\"read {} bytes\", n);\n /// }\n /// Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_read_buf(&self, buf: &mut B) -> io::Result {\n self.io.registration().try_io(Interest::READABLE, || {\n use std::io::Read;\n\n let dst = buf.chunk_mut();\n let dst =\n unsafe { &mut *(dst as *mut _ as *mut [std::mem::MaybeUninit] as *mut [u8]) };\n\n // Safety: We trust `NamedPipeClient::read` to have filled up `n` bytes in the\n // buffer.\n let n = (&*self.io).read(dst)?;\n\n unsafe {\n buf.advance_mut(n);\n }\n\n Ok(n)\n })\n }\n }\n\n /// Waits for the pipe to become writable.\n ///\n /// This function is equivalent to `ready(Interest::WRITABLE)` and is usually\n /// paired with `try_write()`.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-writable\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// client.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn writable(&self) -> io::Result<()> {\n self.ready(Interest::WRITABLE).await?;\n Ok(())\n }\n\n /// Polls for write readiness.\n ///\n /// If the pipe is not currently ready for writing, this method will\n /// store a clone of the `Waker` from the provided `Context`. When the pipe\n /// becomes ready for writing, `Waker::wake` will be called on the waker.\n ///\n /// Note that on multiple calls to `poll_write_ready` or `poll_write`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup. (However, `poll_read_ready` retains a\n /// second, independent waker.)\n ///\n /// This function is intended for cases where creating and pinning a future\n /// via [`writable`] is not feasible. Where possible, using [`writable`] is\n /// preferred, as this supports polling from multiple tasks at once.\n ///\n /// # Return value\n ///\n /// The function returns:\n ///\n /// * `Poll::Pending` if the pipe is not ready for writing.\n /// * `Poll::Ready(Ok(()))` if the pipe is ready for writing.\n /// * `Poll::Ready(Err(e))` if an error is encountered.\n ///\n /// # Errors\n ///\n /// This function may encounter any standard I/O error except `WouldBlock`.\n ///\n /// [`writable`]: method@Self::writable\n pub fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.io.registration().poll_write_ready(cx).map_ok(|_| ())\n }\n\n /// Tries to write a buffer to the pipe, returning how many bytes were\n /// written.\n ///\n /// The function will attempt to write the entire contents of `buf`, but\n /// only part of the buffer may be written.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-try-write\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// client.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_write(b\"hello world\") {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write(&self, buf: &[u8]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write(buf))\n }\n\n /// Tries to write several buffers to the pipe, returning how many bytes\n /// were written.\n ///\n /// Data is written from each buffer in order, with the final buffer read\n /// from possible being only partially consumed. This method behaves\n /// equivalently to a single call to [`try_write()`] with concatenated\n /// buffers.\n ///\n /// This function is usually paired with `writable()`.\n ///\n /// [`try_write()`]: NamedPipeClient::try_write()\n ///\n /// # Return\n ///\n /// If data is successfully written, `Ok(n)` is returned, where `n` is the\n /// number of bytes written. If the pipe is not ready to write data,\n /// `Err(io::ErrorKind::WouldBlock)` is returned.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::net::windows::named_pipe;\n /// use std::error::Error;\n /// use std::io;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-try-write-vectored\";\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let client = named_pipe::ClientOptions::new().open(PIPE_NAME)?;\n ///\n /// let bufs = [io::IoSlice::new(b\"hello \"), io::IoSlice::new(b\"world\")];\n ///\n /// loop {\n /// // Wait for the pipe to be writable\n /// client.writable().await?;\n ///\n /// // Try to write data, this may still fail with `WouldBlock`\n /// // if the readiness event is a false positive.\n /// match client.try_write_vectored(&bufs) {\n /// Ok(n) => {\n /// break;\n /// }\n /// Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n /// continue;\n /// }\n /// Err(e) => {\n /// return Err(e.into());\n /// }\n /// }\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub fn try_write_vectored(&self, buf: &[io::IoSlice<'_>]) -> io::Result {\n self.io\n .registration()\n .try_io(Interest::WRITABLE, || (&*self.io).write_vectored(buf))\n }\n\n /// Tries to read or write from the pipe using a user-provided IO operation.\n ///\n /// If the pipe is ready, the provided closure is called. The closure\n /// should attempt to perform IO operation from the pipe by manually\n /// calling the appropriate syscall. If the operation fails because the\n /// pipe is not actually ready, then the closure should return a\n /// `WouldBlock` error and the readiness flag is cleared. The return value\n /// of the closure is then returned by `try_io`.\n ///\n /// If the pipe is not ready, then the closure is not called\n /// and a `WouldBlock` error is returned.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the pipe that failed due to the pipe not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the pipe to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `NamedPipeClient` type, as this will mess with the\n /// readiness flag and can cause the pipe to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n ///\n /// Usually, [`readable()`], [`writable()`] or [`ready()`] is used with this function.\n ///\n /// [`readable()`]: NamedPipeClient::readable()\n /// [`writable()`]: NamedPipeClient::writable()\n /// [`ready()`]: NamedPipeClient::ready()\n pub fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n self.io.registration().try_io(interest, f)\n }\n\n /// Reads or writes from the pipe using a user-provided IO operation.\n ///\n /// The readiness of the pipe is awaited and when the pipe is ready,\n /// the provided closure is called. The closure should attempt to perform\n /// IO operation on the pipe by manually calling the appropriate syscall.\n /// If the operation fails because the pipe is not actually ready,\n /// then the closure should return a `WouldBlock` error. In such case the\n /// readiness flag is cleared and the pipe readiness is awaited again.\n /// This loop is repeated until the closure returns an `Ok` or an error\n /// other than `WouldBlock`.\n ///\n /// The closure should only return a `WouldBlock` error if it has performed\n /// an IO operation on the pipe that failed due to the pipe not being\n /// ready. Returning a `WouldBlock` error in any other situation will\n /// incorrectly clear the readiness flag, which can cause the pipe to\n /// behave incorrectly.\n ///\n /// The closure should not perform the IO operation using any of the methods\n /// defined on the Tokio `NamedPipeClient` type, as this will mess with the\n /// readiness flag and can cause the pipe to behave incorrectly.\n ///\n /// This method is not intended to be used with combined interests.\n /// The closure should perform only one type of IO operation, so it should not\n /// require more than one ready state. This method may panic or sleep forever\n /// if it is called with a combined interest.\n pub async fn async_io(\n &self,\n interest: Interest,\n f: impl FnMut() -> io::Result,\n ) -> io::Result {\n self.io.registration().async_io(interest, f).await\n }\n}\n\nimpl AsyncRead for NamedPipeClient {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n unsafe { self.io.poll_read(cx, buf) }\n }\n}\n\nimpl AsyncWrite for NamedPipeClient {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.io.poll_write(cx, buf)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.io.poll_write_vectored(cx, bufs)\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.poll_flush(cx)\n }\n}\n\nimpl AsRawHandle for NamedPipeClient {\n fn as_raw_handle(&self) -> RawHandle {\n self.io.as_raw_handle()\n }\n}\n\nimpl AsHandle for NamedPipeClient {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n}\n\n/// A builder structure for construct a named pipe with named pipe-specific\n/// options. This is required to use for named pipe servers who wants to modify\n/// pipe-related options.\n///\n/// See [`ServerOptions::create`].\n#[derive(Debug, Clone)]\npub struct ServerOptions {\n // dwOpenMode\n access_inbound: bool,\n access_outbound: bool,\n first_pipe_instance: bool,\n write_dac: bool,\n write_owner: bool,\n access_system_security: bool,\n // dwPipeMode\n pipe_mode: PipeMode,\n reject_remote_clients: bool,\n // other options\n max_instances: u32,\n out_buffer_size: u32,\n in_buffer_size: u32,\n default_timeout: u32,\n}\n\nimpl ServerOptions {\n /// Creates a new named pipe builder with the default settings.\n ///\n /// ```\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-new\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let server = ServerOptions::new().create(PIPE_NAME)?;\n /// # Ok(()) }\n /// ```\n pub fn new() -> ServerOptions {\n ServerOptions {\n access_inbound: true,\n access_outbound: true,\n first_pipe_instance: false,\n write_dac: false,\n write_owner: false,\n access_system_security: false,\n pipe_mode: PipeMode::Byte,\n reject_remote_clients: true,\n max_instances: windows_sys::PIPE_UNLIMITED_INSTANCES,\n out_buffer_size: 65536,\n in_buffer_size: 65536,\n default_timeout: 0,\n }\n }\n\n /// The pipe mode.\n ///\n /// The default pipe mode is [`PipeMode::Byte`]. See [`PipeMode`] for\n /// documentation of what each mode means.\n ///\n /// This corresponds to specifying `PIPE_TYPE_` and `PIPE_READMODE_` in [`dwPipeMode`].\n ///\n /// [`dwPipeMode`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn pipe_mode(&mut self, pipe_mode: PipeMode) -> &mut Self {\n self.pipe_mode = pipe_mode;\n self\n }\n\n /// The flow of data in the pipe goes from client to server only.\n ///\n /// This corresponds to setting [`PIPE_ACCESS_INBOUND`].\n ///\n /// [`PIPE_ACCESS_INBOUND`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea#pipe_access_inbound\n ///\n /// # Errors\n ///\n /// Server side prevents connecting by denying inbound access, client errors\n /// with [`std::io::ErrorKind::PermissionDenied`] when attempting to create\n /// the connection.\n ///\n /// ```\n /// use std::io;\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-inbound-err1\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let _server = ServerOptions::new()\n /// .access_inbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let e = ClientOptions::new()\n /// .open(PIPE_NAME)\n /// .unwrap_err();\n ///\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// # Ok(()) }\n /// ```\n ///\n /// Disabling writing allows a client to connect, but errors with\n /// [`std::io::ErrorKind::PermissionDenied`] if a write is attempted.\n ///\n /// ```\n /// use std::io;\n /// use tokio::io::AsyncWriteExt;\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-inbound-err2\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let server = ServerOptions::new()\n /// .access_inbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let mut client = ClientOptions::new()\n /// .write(false)\n /// .open(PIPE_NAME)?;\n ///\n /// server.connect().await?;\n ///\n /// let e = client.write(b\"ping\").await.unwrap_err();\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// # Ok(()) }\n /// ```\n ///\n /// # Examples\n ///\n /// A unidirectional named pipe that only supports server-to-client\n /// communication.\n ///\n /// ```\n /// use std::io;\n /// use tokio::io::{AsyncReadExt, AsyncWriteExt};\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-inbound\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let mut server = ServerOptions::new()\n /// .access_inbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let mut client = ClientOptions::new()\n /// .write(false)\n /// .open(PIPE_NAME)?;\n ///\n /// server.connect().await?;\n ///\n /// let write = server.write_all(b\"ping\");\n ///\n /// let mut buf = [0u8; 4];\n /// let read = client.read_exact(&mut buf);\n ///\n /// let ((), read) = tokio::try_join!(write, read)?;\n ///\n /// assert_eq!(read, 4);\n /// assert_eq!(&buf[..], b\"ping\");\n /// # Ok(()) }\n /// ```\n pub fn access_inbound(&mut self, allowed: bool) -> &mut Self {\n self.access_inbound = allowed;\n self\n }\n\n /// The flow of data in the pipe goes from server to client only.\n ///\n /// This corresponds to setting [`PIPE_ACCESS_OUTBOUND`].\n ///\n /// [`PIPE_ACCESS_OUTBOUND`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea#pipe_access_outbound\n ///\n /// # Errors\n ///\n /// Server side prevents connecting by denying outbound access, client\n /// errors with [`std::io::ErrorKind::PermissionDenied`] when attempting to\n /// create the connection.\n ///\n /// ```\n /// use std::io;\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-outbound-err1\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let server = ServerOptions::new()\n /// .access_outbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let e = ClientOptions::new()\n /// .open(PIPE_NAME)\n /// .unwrap_err();\n ///\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// # Ok(()) }\n /// ```\n ///\n /// Disabling reading allows a client to connect, but attempting to read\n /// will error with [`std::io::ErrorKind::PermissionDenied`].\n ///\n /// ```\n /// use std::io;\n /// use tokio::io::AsyncReadExt;\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-outbound-err2\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let server = ServerOptions::new()\n /// .access_outbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let mut client = ClientOptions::new()\n /// .read(false)\n /// .open(PIPE_NAME)?;\n ///\n /// server.connect().await?;\n ///\n /// let mut buf = [0u8; 4];\n /// let e = client.read(&mut buf).await.unwrap_err();\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// # Ok(()) }\n /// ```\n ///\n /// # Examples\n ///\n /// A unidirectional named pipe that only supports client-to-server\n /// communication.\n ///\n /// ```\n /// use tokio::io::{AsyncReadExt, AsyncWriteExt};\n /// use tokio::net::windows::named_pipe::{ClientOptions, ServerOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-access-outbound\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let mut server = ServerOptions::new()\n /// .access_outbound(false)\n /// .create(PIPE_NAME)?;\n ///\n /// let mut client = ClientOptions::new()\n /// .read(false)\n /// .open(PIPE_NAME)?;\n ///\n /// server.connect().await?;\n ///\n /// let write = client.write_all(b\"ping\");\n ///\n /// let mut buf = [0u8; 4];\n /// let read = server.read_exact(&mut buf);\n ///\n /// let ((), read) = tokio::try_join!(write, read)?;\n ///\n /// println!(\"done reading and writing\");\n ///\n /// assert_eq!(read, 4);\n /// assert_eq!(&buf[..], b\"ping\");\n /// # Ok(()) }\n /// ```\n pub fn access_outbound(&mut self, allowed: bool) -> &mut Self {\n self.access_outbound = allowed;\n self\n }\n\n /// If you attempt to create multiple instances of a pipe with this flag\n /// set, creation of the first server instance succeeds, but creation of any\n /// subsequent instances will fail with\n /// [`std::io::ErrorKind::PermissionDenied`].\n ///\n /// This option is intended to be used with servers that want to ensure that\n /// they are the only process listening for clients on a given named pipe.\n /// This is accomplished by enabling it for the first server instance\n /// created in a process.\n ///\n /// This corresponds to setting [`FILE_FLAG_FIRST_PIPE_INSTANCE`].\n ///\n /// # Errors\n ///\n /// If this option is set and more than one instance of the server for a\n /// given named pipe exists, calling [`create`] will fail with\n /// [`std::io::ErrorKind::PermissionDenied`].\n ///\n /// ```\n /// use std::io;\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-first-instance-error\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let server1 = ServerOptions::new()\n /// .first_pipe_instance(true)\n /// .create(PIPE_NAME)?;\n ///\n /// // Second server errs, since it's not the first instance.\n /// let e = ServerOptions::new()\n /// .first_pipe_instance(true)\n /// .create(PIPE_NAME)\n /// .unwrap_err();\n ///\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// # Ok(()) }\n /// ```\n ///\n /// # Examples\n ///\n /// ```\n /// use std::io;\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-first-instance\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let mut builder = ServerOptions::new();\n /// builder.first_pipe_instance(true);\n ///\n /// let server = builder.create(PIPE_NAME)?;\n /// let e = builder.create(PIPE_NAME).unwrap_err();\n /// assert_eq!(e.kind(), io::ErrorKind::PermissionDenied);\n /// drop(server);\n ///\n /// // OK: since, we've closed the other instance.\n /// let _server2 = builder.create(PIPE_NAME)?;\n /// # Ok(()) }\n /// ```\n ///\n /// [`create`]: ServerOptions::create\n /// [`FILE_FLAG_FIRST_PIPE_INSTANCE`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea#pipe_first_pipe_instance\n pub fn first_pipe_instance(&mut self, first: bool) -> &mut Self {\n self.first_pipe_instance = first;\n self\n }\n\n /// Requests permission to modify the pipe's discretionary access control list.\n ///\n /// This corresponds to setting [`WRITE_DAC`] in dwOpenMode.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::{io, os::windows::prelude::AsRawHandle, ptr};\n ///\n /// use tokio::net::windows::named_pipe::ServerOptions;\n /// use windows_sys::{\n /// Win32::Foundation::ERROR_SUCCESS,\n /// Win32::Security::DACL_SECURITY_INFORMATION,\n /// Win32::Security::Authorization::{SetSecurityInfo, SE_KERNEL_OBJECT},\n /// };\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\write_dac_pipe\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let mut pipe_template = ServerOptions::new();\n /// pipe_template.write_dac(true);\n /// let pipe = pipe_template.create(PIPE_NAME)?;\n ///\n /// unsafe {\n /// assert_eq!(\n /// ERROR_SUCCESS,\n /// SetSecurityInfo(\n /// pipe.as_raw_handle() as _,\n /// SE_KERNEL_OBJECT,\n /// DACL_SECURITY_INFORMATION,\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// )\n /// );\n /// }\n ///\n /// # Ok(()) }\n /// ```\n ///\n /// ```\n /// use std::{io, os::windows::prelude::AsRawHandle, ptr};\n ///\n /// use tokio::net::windows::named_pipe::ServerOptions;\n /// use windows_sys::{\n /// Win32::Foundation::ERROR_ACCESS_DENIED,\n /// Win32::Security::DACL_SECURITY_INFORMATION,\n /// Win32::Security::Authorization::{SetSecurityInfo, SE_KERNEL_OBJECT},\n /// };\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\write_dac_pipe_fail\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let mut pipe_template = ServerOptions::new();\n /// pipe_template.write_dac(false);\n /// let pipe = pipe_template.create(PIPE_NAME)?;\n ///\n /// unsafe {\n /// assert_eq!(\n /// ERROR_ACCESS_DENIED,\n /// SetSecurityInfo(\n /// pipe.as_raw_handle() as _,\n /// SE_KERNEL_OBJECT,\n /// DACL_SECURITY_INFORMATION,\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// ptr::null_mut(),\n /// )\n /// );\n /// }\n ///\n /// # Ok(()) }\n /// ```\n ///\n /// [`WRITE_DAC`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn write_dac(&mut self, requested: bool) -> &mut Self {\n self.write_dac = requested;\n self\n }\n\n /// Requests permission to modify the pipe's owner.\n ///\n /// This corresponds to setting [`WRITE_OWNER`] in dwOpenMode.\n ///\n /// [`WRITE_OWNER`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn write_owner(&mut self, requested: bool) -> &mut Self {\n self.write_owner = requested;\n self\n }\n\n /// Requests permission to modify the pipe's system access control list.\n ///\n /// This corresponds to setting [`ACCESS_SYSTEM_SECURITY`] in dwOpenMode.\n ///\n /// [`ACCESS_SYSTEM_SECURITY`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn access_system_security(&mut self, requested: bool) -> &mut Self {\n self.access_system_security = requested;\n self\n }\n\n /// Indicates whether this server can accept remote clients or not. Remote\n /// clients are disabled by default.\n ///\n /// This corresponds to setting [`PIPE_REJECT_REMOTE_CLIENTS`].\n ///\n /// [`PIPE_REJECT_REMOTE_CLIENTS`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea#pipe_reject_remote_clients\n pub fn reject_remote_clients(&mut self, reject: bool) -> &mut Self {\n self.reject_remote_clients = reject;\n self\n }\n\n /// The maximum number of instances that can be created for this pipe. The\n /// first instance of the pipe can specify this value; the same number must\n /// be specified for other instances of the pipe. Acceptable values are in\n /// the range 1 through 254. The default value is unlimited.\n ///\n /// This corresponds to specifying [`nMaxInstances`].\n ///\n /// [`nMaxInstances`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n ///\n /// # Errors\n ///\n /// The same numbers of `max_instances` have to be used by all servers. Any\n /// additional servers trying to be built which uses a mismatching value\n /// might error.\n ///\n /// ```\n /// use std::io;\n /// use tokio::net::windows::named_pipe::{ServerOptions, ClientOptions};\n /// use windows_sys::Win32::Foundation::ERROR_PIPE_BUSY;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-max-instances\";\n ///\n /// # #[tokio::main] async fn main() -> io::Result<()> {\n /// let mut server = ServerOptions::new();\n /// server.max_instances(2);\n ///\n /// let s1 = server.create(PIPE_NAME)?;\n /// let c1 = ClientOptions::new().open(PIPE_NAME);\n ///\n /// let s2 = server.create(PIPE_NAME)?;\n /// let c2 = ClientOptions::new().open(PIPE_NAME);\n ///\n /// // Too many servers!\n /// let e = server.create(PIPE_NAME).unwrap_err();\n /// assert_eq!(e.raw_os_error(), Some(ERROR_PIPE_BUSY as i32));\n ///\n /// // Still too many servers even if we specify a higher value!\n /// let e = server.max_instances(100).create(PIPE_NAME).unwrap_err();\n /// assert_eq!(e.raw_os_error(), Some(ERROR_PIPE_BUSY as i32));\n /// # Ok(()) }\n /// ```\n ///\n /// # Panics\n ///\n /// This function will panic if more than 254 instances are specified. If\n /// you do not wish to set an instance limit, leave it unspecified.\n ///\n /// ```should_panic\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let builder = ServerOptions::new().max_instances(255);\n /// # Ok(()) }\n /// ```\n #[track_caller]\n pub fn max_instances(&mut self, instances: usize) -> &mut Self {\n assert!(instances < 255, \"cannot specify more than 254 instances\");\n self.max_instances = instances as u32;\n self\n }\n\n /// The number of bytes to reserve for the output buffer.\n ///\n /// This corresponds to specifying [`nOutBufferSize`].\n ///\n /// [`nOutBufferSize`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn out_buffer_size(&mut self, buffer: u32) -> &mut Self {\n self.out_buffer_size = buffer;\n self\n }\n\n /// The number of bytes to reserve for the input buffer.\n ///\n /// This corresponds to specifying [`nInBufferSize`].\n ///\n /// [`nInBufferSize`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n pub fn in_buffer_size(&mut self, buffer: u32) -> &mut Self {\n self.in_buffer_size = buffer;\n self\n }\n\n /// Creates the named pipe identified by `addr` for use as a server.\n ///\n /// This uses the [`CreateNamedPipe`] function.\n ///\n /// [`CreateNamedPipe`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea\n ///\n /// # Errors\n ///\n /// This errors if called outside of a [Tokio Runtime], or in a runtime that\n /// has not [enabled I/O], or if any OS-specific I/O errors occur.\n ///\n /// [Tokio Runtime]: crate::runtime::Runtime\n /// [enabled I/O]: crate::runtime::Builder::enable_io\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::net::windows::named_pipe::ServerOptions;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-create\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let server = ServerOptions::new().create(PIPE_NAME)?;\n /// # Ok(()) }\n /// ```\n pub fn create(&self, addr: impl AsRef) -> io::Result {\n // Safety: We're calling create_with_security_attributes_raw w/ a null\n // pointer which disables it.\n unsafe { self.create_with_security_attributes_raw(addr, ptr::null_mut()) }\n }\n\n /// Creates the named pipe identified by `addr` for use as a server.\n ///\n /// This is the same as [`create`] except that it supports providing the raw\n /// pointer to a structure of [`SECURITY_ATTRIBUTES`] which will be passed\n /// as the `lpSecurityAttributes` argument to [`CreateFile`].\n ///\n /// # Errors\n ///\n /// This errors if called outside of a [Tokio Runtime], or in a runtime that\n /// has not [enabled I/O], or if any OS-specific I/O errors occur.\n ///\n /// [Tokio Runtime]: crate::runtime::Runtime\n /// [enabled I/O]: crate::runtime::Builder::enable_io\n ///\n /// # Safety\n ///\n /// The `attrs` argument must either be null or point at a valid instance of\n /// the [`SECURITY_ATTRIBUTES`] structure. If the argument is null, the\n /// behavior is identical to calling the [`create`] method.\n ///\n /// [`create`]: ServerOptions::create\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew\n /// [`SECURITY_ATTRIBUTES`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Security/struct.SECURITY_ATTRIBUTES.html\n pub unsafe fn create_with_security_attributes_raw(\n &self,\n addr: impl AsRef,\n attrs: *mut c_void,\n ) -> io::Result {\n let addr = encode_addr(addr);\n\n let pipe_mode = {\n let mut mode = if matches!(self.pipe_mode, PipeMode::Message) {\n windows_sys::PIPE_TYPE_MESSAGE | windows_sys::PIPE_READMODE_MESSAGE\n } else {\n windows_sys::PIPE_TYPE_BYTE | windows_sys::PIPE_READMODE_BYTE\n };\n if self.reject_remote_clients {\n mode |= windows_sys::PIPE_REJECT_REMOTE_CLIENTS;\n } else {\n mode |= windows_sys::PIPE_ACCEPT_REMOTE_CLIENTS;\n }\n mode\n };\n let open_mode = {\n let mut mode = windows_sys::FILE_FLAG_OVERLAPPED;\n if self.access_inbound {\n mode |= windows_sys::PIPE_ACCESS_INBOUND;\n }\n if self.access_outbound {\n mode |= windows_sys::PIPE_ACCESS_OUTBOUND;\n }\n if self.first_pipe_instance {\n mode |= windows_sys::FILE_FLAG_FIRST_PIPE_INSTANCE;\n }\n if self.write_dac {\n mode |= windows_sys::WRITE_DAC;\n }\n if self.write_owner {\n mode |= windows_sys::WRITE_OWNER;\n }\n if self.access_system_security {\n mode |= windows_sys::ACCESS_SYSTEM_SECURITY;\n }\n mode\n };\n\n let h = unsafe {\n windows_sys::CreateNamedPipeW(\n addr.as_ptr(),\n open_mode,\n pipe_mode,\n self.max_instances,\n self.out_buffer_size,\n self.in_buffer_size,\n self.default_timeout,\n attrs as *mut _,\n )\n };\n\n if h == windows_sys::INVALID_HANDLE_VALUE {\n return Err(io::Error::last_os_error());\n }\n\n unsafe { NamedPipeServer::from_raw_handle(h as _) }\n }\n}\n\n/// A builder suitable for building and interacting with named pipes from the\n/// client side.\n///\n/// See [`ClientOptions::open`].\n#[derive(Debug, Clone)]\npub struct ClientOptions {\n generic_read: bool,\n generic_write: bool,\n security_qos_flags: u32,\n pipe_mode: PipeMode,\n}\n\nimpl ClientOptions {\n /// Creates a new named pipe builder with the default settings.\n ///\n /// ```\n /// use tokio::net::windows::named_pipe::{ServerOptions, ClientOptions};\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\tokio-named-pipe-client-new\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// // Server must be created in order for the client creation to succeed.\n /// let server = ServerOptions::new().create(PIPE_NAME)?;\n /// let client = ClientOptions::new().open(PIPE_NAME)?;\n /// # Ok(()) }\n /// ```\n pub fn new() -> Self {\n Self {\n generic_read: true,\n generic_write: true,\n security_qos_flags: windows_sys::SECURITY_IDENTIFICATION\n | windows_sys::SECURITY_SQOS_PRESENT,\n pipe_mode: PipeMode::Byte,\n }\n }\n\n /// If the client supports reading data. This is enabled by default.\n ///\n /// This corresponds to setting [`GENERIC_READ`] in the call to [`CreateFile`].\n ///\n /// [`GENERIC_READ`]: https://docs.microsoft.com/en-us/windows/win32/secauthz/generic-access-rights\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew\n pub fn read(&mut self, allowed: bool) -> &mut Self {\n self.generic_read = allowed;\n self\n }\n\n /// If the created pipe supports writing data. This is enabled by default.\n ///\n /// This corresponds to setting [`GENERIC_WRITE`] in the call to [`CreateFile`].\n ///\n /// [`GENERIC_WRITE`]: https://docs.microsoft.com/en-us/windows/win32/secauthz/generic-access-rights\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew\n pub fn write(&mut self, allowed: bool) -> &mut Self {\n self.generic_write = allowed;\n self\n }\n\n /// Sets qos flags which are combined with other flags and attributes in the\n /// call to [`CreateFile`].\n ///\n /// By default `security_qos_flags` is set to [`SECURITY_IDENTIFICATION`],\n /// calling this function would override that value completely with the\n /// argument specified.\n ///\n /// When `security_qos_flags` is not set, a malicious program can gain the\n /// elevated privileges of a privileged Rust process when it allows opening\n /// user-specified paths, by tricking it into opening a named pipe. So\n /// arguably `security_qos_flags` should also be set when opening arbitrary\n /// paths. However the bits can then conflict with other flags, specifically\n /// `FILE_FLAG_OPEN_NO_RECALL`.\n ///\n /// For information about possible values, see [Impersonation Levels] on the\n /// Windows Dev Center site. The `SECURITY_SQOS_PRESENT` flag is set\n /// automatically when using this method.\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n /// [`SECURITY_IDENTIFICATION`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Storage/FileSystem/constant.SECURITY_IDENTIFICATION.html\n /// [Impersonation Levels]: https://docs.microsoft.com/en-us/windows/win32/api/winnt/ne-winnt-security_impersonation_level\n pub fn security_qos_flags(&mut self, flags: u32) -> &mut Self {\n // See: https://github.com/rust-lang/rust/pull/58216\n self.security_qos_flags = flags | windows_sys::SECURITY_SQOS_PRESENT;\n self\n }\n\n /// The pipe mode.\n ///\n /// The default pipe mode is [`PipeMode::Byte`]. See [`PipeMode`] for\n /// documentation of what each mode means.\n pub fn pipe_mode(&mut self, pipe_mode: PipeMode) -> &mut Self {\n self.pipe_mode = pipe_mode;\n self\n }\n\n /// Opens the named pipe identified by `addr`.\n ///\n /// This opens the client using [`CreateFile`] with the\n /// `dwCreationDisposition` option set to `OPEN_EXISTING`.\n ///\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea\n ///\n /// # Errors\n ///\n /// This errors if called outside of a [Tokio Runtime], or in a runtime that\n /// has not [enabled I/O], or if any OS-specific I/O errors occur.\n ///\n /// There are a few errors you need to take into account when creating a\n /// named pipe on the client side:\n ///\n /// * [`std::io::ErrorKind::NotFound`] - This indicates that the named pipe\n /// does not exist. Presumably the server is not up.\n /// * [`ERROR_PIPE_BUSY`] - This error is raised when the named pipe exists,\n /// but the server is not currently waiting for a connection. Please see the\n /// examples for how to check for this error.\n ///\n /// [`ERROR_PIPE_BUSY`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Foundation/constant.ERROR_PIPE_BUSY.html\n /// [enabled I/O]: crate::runtime::Builder::enable_io\n /// [Tokio Runtime]: crate::runtime::Runtime\n ///\n /// A connect loop that waits until a pipe becomes available looks like\n /// this:\n ///\n /// ```no_run\n /// use std::time::Duration;\n /// use tokio::net::windows::named_pipe::ClientOptions;\n /// use tokio::time;\n /// use windows_sys::Win32::Foundation::ERROR_PIPE_BUSY;\n ///\n /// const PIPE_NAME: &str = r\"\\\\.\\pipe\\mynamedpipe\";\n ///\n /// # #[tokio::main] async fn main() -> std::io::Result<()> {\n /// let client = loop {\n /// match ClientOptions::new().open(PIPE_NAME) {\n /// Ok(client) => break client,\n /// Err(e) if e.raw_os_error() == Some(ERROR_PIPE_BUSY as i32) => (),\n /// Err(e) => return Err(e),\n /// }\n ///\n /// time::sleep(Duration::from_millis(50)).await;\n /// };\n ///\n /// // use the connected client.\n /// # Ok(()) }\n /// ```\n pub fn open(&self, addr: impl AsRef) -> io::Result {\n // Safety: We're calling open_with_security_attributes_raw w/ a null\n // pointer which disables it.\n unsafe { self.open_with_security_attributes_raw(addr, ptr::null_mut()) }\n }\n\n /// Opens the named pipe identified by `addr`.\n ///\n /// This is the same as [`open`] except that it supports providing the raw\n /// pointer to a structure of [`SECURITY_ATTRIBUTES`] which will be passed\n /// as the `lpSecurityAttributes` argument to [`CreateFile`].\n ///\n /// # Safety\n ///\n /// The `attrs` argument must either be null or point at a valid instance of\n /// the [`SECURITY_ATTRIBUTES`] structure. If the argument is null, the\n /// behavior is identical to calling the [`open`] method.\n ///\n /// [`open`]: ClientOptions::open\n /// [`CreateFile`]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew\n /// [`SECURITY_ATTRIBUTES`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Security/struct.SECURITY_ATTRIBUTES.html\n pub unsafe fn open_with_security_attributes_raw(\n &self,\n addr: impl AsRef,\n attrs: *mut c_void,\n ) -> io::Result {\n let addr = encode_addr(addr);\n\n let desired_access = {\n let mut access = 0;\n if self.generic_read {\n access |= windows_sys::GENERIC_READ;\n }\n if self.generic_write {\n access |= windows_sys::GENERIC_WRITE;\n }\n access\n };\n\n // NB: We could use a platform specialized `OpenOptions` here, but since\n // we have access to windows_sys it ultimately doesn't hurt to use\n // `CreateFile` explicitly since it allows the use of our already\n // well-structured wide `addr` to pass into CreateFileW.\n let h = unsafe {\n windows_sys::CreateFileW(\n addr.as_ptr(),\n desired_access,\n 0,\n attrs as *mut _,\n windows_sys::OPEN_EXISTING,\n self.get_flags(),\n null_mut(),\n )\n };\n\n if h == windows_sys::INVALID_HANDLE_VALUE {\n return Err(io::Error::last_os_error());\n }\n\n if matches!(self.pipe_mode, PipeMode::Message) {\n let mode = windows_sys::PIPE_READMODE_MESSAGE;\n let result = unsafe {\n windows_sys::SetNamedPipeHandleState(h, &mode, ptr::null_mut(), ptr::null_mut())\n };\n\n if result == 0 {\n return Err(io::Error::last_os_error());\n }\n }\n\n unsafe { NamedPipeClient::from_raw_handle(h as _) }\n }\n\n fn get_flags(&self) -> u32 {\n self.security_qos_flags | windows_sys::FILE_FLAG_OVERLAPPED\n }\n}\n\n/// The pipe mode of a named pipe.\n///\n/// Set through [`ServerOptions::pipe_mode`].\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\n#[non_exhaustive]\npub enum PipeMode {\n /// Data is written to the pipe as a stream of bytes. The pipe does not\n /// distinguish bytes written during different write operations.\n ///\n /// Corresponds to [`PIPE_TYPE_BYTE`].\n ///\n /// [`PIPE_TYPE_BYTE`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/System/Pipes/constant.PIPE_TYPE_BYTE.html\n Byte,\n /// Data is written to the pipe as a stream of messages. The pipe treats the\n /// bytes written during each write operation as a message unit. Any reading\n /// on a named pipe returns [`ERROR_MORE_DATA`] when a message is not read\n /// completely.\n ///\n /// Corresponds to [`PIPE_TYPE_MESSAGE`].\n ///\n /// [`ERROR_MORE_DATA`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/Foundation/constant.ERROR_MORE_DATA.html\n /// [`PIPE_TYPE_MESSAGE`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/System/Pipes/constant.PIPE_TYPE_MESSAGE.html\n Message,\n}\n\n/// Indicates the end of a named pipe.\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\n#[non_exhaustive]\npub enum PipeEnd {\n /// The named pipe refers to the client end of a named pipe instance.\n ///\n /// Corresponds to [`PIPE_CLIENT_END`].\n ///\n /// [`PIPE_CLIENT_END`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/System/Pipes/constant.PIPE_CLIENT_END.html\n Client,\n /// The named pipe refers to the server end of a named pipe instance.\n ///\n /// Corresponds to [`PIPE_SERVER_END`].\n ///\n /// [`PIPE_SERVER_END`]: https://docs.rs/windows-sys/latest/windows_sys/Win32/System/Pipes/constant.PIPE_SERVER_END.html\n Server,\n}\n\n/// Information about a named pipe.\n///\n/// Constructed through [`NamedPipeServer::info`] or [`NamedPipeClient::info`].\n#[derive(Debug, Clone)]\n#[non_exhaustive]\npub struct PipeInfo {\n /// Indicates the mode of a named pipe.\n pub mode: PipeMode,\n /// Indicates the end of a named pipe.\n pub end: PipeEnd,\n /// The maximum number of instances that can be created for this pipe.\n pub max_instances: u32,\n /// The number of bytes to reserve for the output buffer.\n pub out_buffer_size: u32,\n /// The number of bytes to reserve for the input buffer.\n pub in_buffer_size: u32,\n}\n\n/// Encodes an address so that it is a null-terminated wide string.\nfn encode_addr(addr: impl AsRef) -> Box<[u16]> {\n let len = addr.as_ref().encode_wide().count();\n let mut vec = Vec::with_capacity(len + 1);\n vec.extend(addr.as_ref().encode_wide());\n vec.push(0);\n vec.into_boxed_slice()\n}\n\n/// Internal function to get the info out of a raw named pipe.\nunsafe fn named_pipe_info(handle: RawHandle) -> io::Result {\n let mut flags = 0;\n let mut out_buffer_size = 0;\n let mut in_buffer_size = 0;\n let mut max_instances = 0;\n\n let result = unsafe {\n windows_sys::GetNamedPipeInfo(\n handle as _,\n &mut flags,\n &mut out_buffer_size,\n &mut in_buffer_size,\n &mut max_instances,\n )\n };\n\n if result == 0 {\n return Err(io::Error::last_os_error());\n }\n\n let mut end = PipeEnd::Client;\n let mut mode = PipeMode::Byte;\n\n if flags & windows_sys::PIPE_SERVER_END != 0 {\n end = PipeEnd::Server;\n }\n\n if flags & windows_sys::PIPE_TYPE_MESSAGE != 0 {\n mode = PipeMode::Message;\n }\n\n Ok(PipeInfo {\n end,\n mode,\n out_buffer_size,\n in_buffer_size,\n max_instances,\n })\n}\n" }, { "path": "tokio/src/process/kill.rs", "content": "use std::io;\n\n/// An interface for killing a running process.\npub(crate) trait Kill {\n /// Forcefully kills the process.\n fn kill(&mut self) -> io::Result<()>;\n}\n\nimpl Kill for &mut T {\n fn kill(&mut self) -> io::Result<()> {\n (**self).kill()\n }\n}\n" }, { "path": "tokio/src/process/mod.rs", "content": "//! An implementation of asynchronous process management for Tokio.\n//!\n//! This module provides a [`Command`] struct that imitates the interface of the\n//! [`std::process::Command`] type in the standard library, but provides asynchronous versions of\n//! functions that create processes. These functions (`spawn`, `status`, `output` and their\n//! variants) return \"future aware\" types that interoperate with Tokio. The asynchronous process\n//! support is provided through signal handling on Unix and system APIs on Windows.\n//!\n//! [`std::process::Command`]: std::process::Command\n//!\n//! # Examples\n//!\n//! Here's an example program which will spawn `echo hello world` and then wait\n//! for it complete.\n//!\n//! ```no_run\n//! use tokio::process::Command;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! // The usage is similar as with the standard library's `Command` type\n//! let mut child = Command::new(\"echo\")\n//! .arg(\"hello\")\n//! .arg(\"world\")\n//! .spawn()\n//! .expect(\"failed to spawn\");\n//!\n//! // Await until the command completes\n//! let status = child.wait().await?;\n//! println!(\"the command exited with: {}\", status);\n//! Ok(())\n//! }\n//! ```\n//!\n//! Next, let's take a look at an example where we not only spawn `echo hello\n//! world` but we also capture its output.\n//!\n//! ```no_run\n//! use tokio::process::Command;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! // Like above, but use `output` which returns a future instead of\n//! // immediately returning the `Child`.\n//! let output = Command::new(\"echo\").arg(\"hello\").arg(\"world\")\n//! .output();\n//!\n//! let output = output.await?;\n//!\n//! assert!(output.status.success());\n//! assert_eq!(output.stdout, b\"hello world\\n\");\n//! Ok(())\n//! }\n//! ```\n//!\n//! We can also read input line by line.\n//!\n//! ```no_run\n//! use tokio::io::{BufReader, AsyncBufReadExt};\n//! use tokio::process::Command;\n//!\n//! use std::process::Stdio;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! let mut cmd = Command::new(\"cat\");\n//!\n//! // Specify that we want the command's standard output piped back to us.\n//! // By default, standard input/output/error will be inherited from the\n//! // current process (for example, this means that standard input will\n//! // come from the keyboard and standard output/error will go directly to\n//! // the terminal if this process is invoked from the command line).\n//! cmd.stdout(Stdio::piped());\n//!\n//! let mut child = cmd.spawn()\n//! .expect(\"failed to spawn command\");\n//!\n//! let stdout = child.stdout.take()\n//! .expect(\"child did not have a handle to stdout\");\n//!\n//! let mut reader = BufReader::new(stdout).lines();\n//!\n//! // Ensure the child process is spawned in the runtime so it can\n//! // make progress on its own while we await for any output.\n//! tokio::spawn(async move {\n//! let status = child.wait().await\n//! .expect(\"child process encountered an error\");\n//!\n//! println!(\"child status was: {}\", status);\n//! });\n//!\n//! while let Some(line) = reader.next_line().await? {\n//! println!(\"Line: {}\", line);\n//! }\n//!\n//! Ok(())\n//! }\n//! ```\n//!\n//! Here is another example using `sort` writing into the child process\n//! standard input, capturing the output of the sorted text.\n//!\n//! ```no_run\n//! use tokio::io::AsyncWriteExt;\n//! use tokio::process::Command;\n//!\n//! use std::process::Stdio;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! let mut cmd = Command::new(\"sort\");\n//!\n//! // Specifying that we want pipe both the output and the input.\n//! // Similarly to capturing the output, by configuring the pipe\n//! // to stdin it can now be used as an asynchronous writer.\n//! cmd.stdout(Stdio::piped());\n//! cmd.stdin(Stdio::piped());\n//!\n//! let mut child = cmd.spawn().expect(\"failed to spawn command\");\n//!\n//! // These are the animals we want to sort\n//! let animals: &[&str] = &[\"dog\", \"bird\", \"frog\", \"cat\", \"fish\"];\n//!\n//! let mut stdin = child\n//! .stdin\n//! .take()\n//! .expect(\"child did not have a handle to stdin\");\n//!\n//! // Write our animals to the child process\n//! // Note that the behavior of `sort` is to buffer _all input_ before writing any output.\n//! // In the general sense, it is recommended to write to the child in a separate task as\n//! // awaiting its exit (or output) to avoid deadlocks (for example, the child tries to write\n//! // some output but gets stuck waiting on the parent to read from it, meanwhile the parent\n//! // is stuck waiting to write its input completely before reading the output).\n//! stdin\n//! .write(animals.join(\"\\n\").as_bytes())\n//! .await\n//! .expect(\"could not write to stdin\");\n//!\n//! // We drop the handle here which signals EOF to the child process.\n//! // This tells the child process that it there is no more data on the pipe.\n//! drop(stdin);\n//!\n//! let op = child.wait_with_output().await?;\n//!\n//! // Results should come back in sorted order\n//! assert_eq!(op.stdout, \"bird\\ncat\\ndog\\nfish\\nfrog\\n\".as_bytes());\n//!\n//! Ok(())\n//! }\n//! ```\n//!\n//! With some coordination, we can also pipe the output of one command into\n//! another.\n//!\n//! ```no_run\n//! use tokio::join;\n//! use tokio::process::Command;\n//! use std::process::Stdio;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! let mut echo = Command::new(\"echo\")\n//! .arg(\"hello world!\")\n//! .stdout(Stdio::piped())\n//! .spawn()\n//! .expect(\"failed to spawn echo\");\n//!\n//! let tr_stdin: Stdio = echo\n//! .stdout\n//! .take()\n//! .unwrap()\n//! .try_into()\n//! .expect(\"failed to convert to Stdio\");\n//!\n//! let tr = Command::new(\"tr\")\n//! .arg(\"a-z\")\n//! .arg(\"A-Z\")\n//! .stdin(tr_stdin)\n//! .stdout(Stdio::piped())\n//! .spawn()\n//! .expect(\"failed to spawn tr\");\n//!\n//! let (echo_result, tr_output) = join!(echo.wait(), tr.wait_with_output());\n//!\n//! assert!(echo_result.unwrap().success());\n//!\n//! let tr_output = tr_output.expect(\"failed to await tr\");\n//! assert!(tr_output.status.success());\n//!\n//! assert_eq!(tr_output.stdout, b\"HELLO WORLD!\\n\");\n//!\n//! Ok(())\n//! }\n//! ```\n//!\n//! # Caveats\n//!\n//! ## Dropping/Cancellation\n//!\n//! Similar to the behavior to the standard library, and unlike the futures\n//! paradigm of dropping-implies-cancellation, a spawned process will, by\n//! default, continue to execute even after the `Child` handle has been dropped.\n//!\n//! The [`Command::kill_on_drop`] method can be used to modify this behavior\n//! and kill the child process if the `Child` wrapper is dropped before it\n//! has exited.\n//!\n//! ## Unix Processes\n//!\n//! On Unix platforms processes must be \"reaped\" by their parent process after\n//! they have exited in order to release all OS resources. A child process which\n//! has exited, but has not yet been reaped by its parent is considered a \"zombie\"\n//! process. Such processes continue to count against limits imposed by the system,\n//! and having too many zombie processes present can prevent additional processes\n//! from being spawned.\n//!\n//! The tokio runtime will, on a best-effort basis, attempt to reap and clean up\n//! any process which it has spawned. No additional guarantees are made with regard to\n//! how quickly or how often this procedure will take place.\n//!\n//! It is recommended to avoid dropping a [`Child`] process handle before it has been\n//! fully `await`ed if stricter cleanup guarantees are required.\n//!\n//! [`Command`]: crate::process::Command\n//! [`Command::kill_on_drop`]: crate::process::Command::kill_on_drop\n//! [`Child`]: crate::process::Child\n\n#[path = \"unix/mod.rs\"]\n#[cfg(unix)]\nmod imp;\n\n#[cfg(unix)]\npub(crate) mod unix {\n pub(crate) use super::imp::*;\n}\n\n#[path = \"windows.rs\"]\n#[cfg(windows)]\nmod imp;\n\nmod kill;\n\nuse crate::io::{AsyncRead, AsyncWrite, ReadBuf};\nuse crate::process::kill::Kill;\n\nuse std::ffi::OsStr;\nuse std::future::Future;\nuse std::io;\nuse std::path::Path;\nuse std::pin::Pin;\nuse std::process::{Child as StdChild, Command as StdCommand, ExitStatus, Output, Stdio};\nuse std::task::{ready, Context, Poll};\n\n#[cfg(unix)]\nuse std::os::unix::process::CommandExt;\n#[cfg(windows)]\nuse std::os::windows::process::CommandExt;\n\ncfg_windows! {\n use crate::os::windows::io::{AsRawHandle, RawHandle};\n}\n\n/// This structure mimics the API of [`std::process::Command`] found in the standard library, but\n/// replaces functions that create a process with an asynchronous variant. The main provided\n/// asynchronous functions are [spawn](Command::spawn), [status](Command::status), and\n/// [output](Command::output).\n///\n/// `Command` uses asynchronous versions of some `std` types (for example [`Child`]).\n///\n/// [`std::process::Command`]: std::process::Command\n/// [`Child`]: struct@Child\n#[derive(Debug)]\npub struct Command {\n std: StdCommand,\n kill_on_drop: bool,\n}\n\npub(crate) struct SpawnedChild {\n child: imp::Child,\n stdin: Option,\n stdout: Option,\n stderr: Option,\n}\n\nimpl Command {\n /// Constructs a new `Command` for launching the program at\n /// path `program`, with the following default configuration:\n ///\n /// * No arguments to the program\n /// * Inherit the current process's environment\n /// * Inherit the current process's working directory\n /// * Inherit stdin/stdout/stderr for `spawn` or `status`, but create pipes for `output`\n ///\n /// Builder methods are provided to change these defaults and\n /// otherwise configure the process.\n ///\n /// If `program` is not an absolute path, the `PATH` will be searched in\n /// an OS-defined way.\n ///\n /// The search path to be used may be controlled by setting the\n /// `PATH` environment variable on the Command,\n /// but this has some implementation limitations on Windows\n /// (see issue [rust-lang/rust#37519]).\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// use tokio::process::Command;\n /// let mut command = Command::new(\"sh\");\n /// # let _ = command.output(); // assert borrow checker\n /// ```\n ///\n /// [rust-lang/rust#37519]: https://github.com/rust-lang/rust/issues/37519\n pub fn new>(program: S) -> Command {\n Self::from(StdCommand::new(program))\n }\n\n /// Cheaply convert to a `&std::process::Command` for places where the type from the standard\n /// library is expected.\n pub fn as_std(&self) -> &StdCommand {\n &self.std\n }\n\n /// Cheaply convert to a `&mut std::process::Command` for places where the type from the\n /// standard library is expected.\n pub fn as_std_mut(&mut self) -> &mut StdCommand {\n &mut self.std\n }\n\n /// Cheaply convert into a `std::process::Command`.\n ///\n /// Note that Tokio specific options will be lost. Currently, this only applies to [`kill_on_drop`].\n ///\n /// [`kill_on_drop`]: Command::kill_on_drop\n pub fn into_std(self) -> StdCommand {\n self.std\n }\n\n /// Adds an argument to pass to the program.\n ///\n /// Only one argument can be passed per use. So instead of:\n ///\n /// ```no_run\n /// let mut command = tokio::process::Command::new(\"sh\");\n /// command.arg(\"-C /path/to/repo\");\n ///\n /// # let _ = command.output(); // assert borrow checker\n /// ```\n ///\n /// usage would be:\n ///\n /// ```no_run\n /// let mut command = tokio::process::Command::new(\"sh\");\n /// command.arg(\"-C\");\n /// command.arg(\"/path/to/repo\");\n ///\n /// # let _ = command.output(); // assert borrow checker\n /// ```\n ///\n /// To pass multiple arguments see [`args`].\n ///\n /// [`args`]: method@Self::args\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .arg(\"-l\")\n /// .arg(\"-a\")\n /// .output().await.unwrap();\n /// # }\n ///\n /// ```\n pub fn arg>(&mut self, arg: S) -> &mut Command {\n self.std.arg(arg);\n self\n }\n\n /// Adds multiple arguments to pass to the program.\n ///\n /// To pass a single argument see [`arg`].\n ///\n /// [`arg`]: method@Self::arg\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .args(&[\"-l\", \"-a\"])\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn args(&mut self, args: I) -> &mut Command\n where\n I: IntoIterator,\n S: AsRef,\n {\n self.std.args(args);\n self\n }\n\n cfg_windows! {\n /// Append literal text to the command line without any quoting or escaping.\n ///\n /// This is useful for passing arguments to `cmd.exe /c`, which doesn't follow\n /// `CommandLineToArgvW` escaping rules.\n pub fn raw_arg>(&mut self, text_to_append_as_is: S) -> &mut Command {\n self.std.raw_arg(text_to_append_as_is);\n self\n }\n }\n\n /// Inserts or updates an environment variable mapping.\n ///\n /// Note that environment variable names are case-insensitive (but case-preserving) on Windows,\n /// and case-sensitive on all other platforms.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .env(\"PATH\", \"/bin\")\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn env(&mut self, key: K, val: V) -> &mut Command\n where\n K: AsRef,\n V: AsRef,\n {\n self.std.env(key, val);\n self\n }\n\n /// Adds or updates multiple environment variable mappings.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n /// use std::process::{Stdio};\n /// use std::env;\n /// use std::collections::HashMap;\n ///\n /// let filtered_env : HashMap =\n /// env::vars().filter(|&(ref k, _)|\n /// k == \"TERM\" || k == \"TZ\" || k == \"LANG\" || k == \"PATH\"\n /// ).collect();\n ///\n /// let output = Command::new(\"printenv\")\n /// .stdin(Stdio::null())\n /// .stdout(Stdio::inherit())\n /// .env_clear()\n /// .envs(&filtered_env)\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn envs(&mut self, vars: I) -> &mut Command\n where\n I: IntoIterator,\n K: AsRef,\n V: AsRef,\n {\n self.std.envs(vars);\n self\n }\n\n /// Removes an environment variable mapping.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .env_remove(\"PATH\")\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn env_remove>(&mut self, key: K) -> &mut Command {\n self.std.env_remove(key);\n self\n }\n\n /// Clears the entire environment map for the child process.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .env_clear()\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn env_clear(&mut self) -> &mut Command {\n self.std.env_clear();\n self\n }\n\n /// Sets the working directory for the child process.\n ///\n /// # Platform-specific behavior\n ///\n /// If the program path is relative (e.g., `\"./script.sh\"`), it's ambiguous\n /// whether it should be interpreted relative to the parent's working\n /// directory or relative to `current_dir`. The behavior in this case is\n /// platform specific and unstable, and it's recommended to use\n /// [`canonicalize`] to get an absolute program path instead.\n ///\n /// [`canonicalize`]: crate::fs::canonicalize()\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .current_dir(\"/bin\")\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn current_dir>(&mut self, dir: P) -> &mut Command {\n self.std.current_dir(dir);\n self\n }\n\n /// Sets configuration for the child process's standard input (stdin) handle.\n ///\n /// Defaults to [`inherit`].\n ///\n /// [`inherit`]: std::process::Stdio::inherit\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use std::process::{Stdio};\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"ls\")\n /// .stdin(Stdio::null())\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn stdin>(&mut self, cfg: T) -> &mut Command {\n self.std.stdin(cfg);\n self\n }\n\n /// Sets configuration for the child process's standard output (stdout) handle.\n ///\n /// Defaults to [`inherit`] when used with `spawn` or `status`, and\n /// defaults to [`piped`] when used with `output`.\n ///\n /// [`inherit`]: std::process::Stdio::inherit\n /// [`piped`]: std::process::Stdio::piped\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n /// use std::process::Stdio;\n ///\n /// let output = Command::new(\"ls\")\n /// .stdout(Stdio::null())\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn stdout>(&mut self, cfg: T) -> &mut Command {\n self.std.stdout(cfg);\n self\n }\n\n /// Sets configuration for the child process's standard error (stderr) handle.\n ///\n /// Defaults to [`inherit`] when used with `spawn` or `status`, and\n /// defaults to [`piped`] when used with `output`.\n ///\n /// [`inherit`]: std::process::Stdio::inherit\n /// [`piped`]: std::process::Stdio::piped\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n /// use std::process::{Stdio};\n ///\n /// let output = Command::new(\"ls\")\n /// .stderr(Stdio::null())\n /// .output().await.unwrap();\n /// # }\n /// ```\n pub fn stderr>(&mut self, cfg: T) -> &mut Command {\n self.std.stderr(cfg);\n self\n }\n\n /// Controls whether a `kill` operation should be invoked on a spawned child\n /// process when its corresponding `Child` handle is dropped.\n ///\n /// By default, this value is assumed to be `false`, meaning the next spawned\n /// process will not be killed on drop, similar to the behavior of the standard\n /// library.\n ///\n /// # Caveats\n ///\n /// On Unix platforms processes must be \"reaped\" by their parent process after\n /// they have exited in order to release all OS resources. A child process which\n /// has exited, but has not yet been reaped by its parent is considered a \"zombie\"\n /// process. Such processes continue to count against limits imposed by the system,\n /// and having too many zombie processes present can prevent additional processes\n /// from being spawned.\n ///\n /// Although issuing a `kill` signal to the child process is a synchronous\n /// operation, the resulting zombie process cannot be `.await`ed inside of the\n /// destructor to avoid blocking other tasks. The tokio runtime will, on a\n /// best-effort basis, attempt to reap and clean up such processes in the\n /// background, but no additional guarantees are made with regard to\n /// how quickly or how often this procedure will take place.\n ///\n /// If stronger guarantees are required, it is recommended to avoid dropping\n /// a [`Child`] handle where possible, and instead utilize `child.wait().await`\n /// or `child.kill().await` where possible.\n pub fn kill_on_drop(&mut self, kill_on_drop: bool) -> &mut Command {\n self.kill_on_drop = kill_on_drop;\n self\n }\n\n cfg_windows! {\n /// Sets the [process creation flags][1] to be passed to `CreateProcess`.\n ///\n /// These will always be ORed with `CREATE_UNICODE_ENVIRONMENT`.\n ///\n /// [1]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx\n pub fn creation_flags(&mut self, flags: u32) -> &mut Command {\n self.std.creation_flags(flags);\n self\n }\n }\n\n /// Sets the child process's user ID. This translates to a\n /// `setuid` call in the child process. Failure in the `setuid`\n /// call will cause the spawn to fail.\n #[cfg(unix)]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n pub fn uid(&mut self, id: u32) -> &mut Command {\n #[cfg(target_os = \"nto\")]\n let id = id as i32;\n self.std.uid(id);\n self\n }\n\n /// Similar to `uid` but sets the group ID of the child process. This has\n /// the same semantics as the `uid` field.\n #[cfg(unix)]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n pub fn gid(&mut self, id: u32) -> &mut Command {\n #[cfg(target_os = \"nto\")]\n let id = id as i32;\n self.std.gid(id);\n self\n }\n\n /// Sets executable argument.\n ///\n /// Set the first process argument, `argv[0]`, to something other than the\n /// default executable path.\n #[cfg(unix)]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n pub fn arg0(&mut self, arg: S) -> &mut Command\n where\n S: AsRef,\n {\n self.std.arg0(arg);\n self\n }\n\n /// Schedules a closure to be run just before the `exec` function is\n /// invoked.\n ///\n /// The closure is allowed to return an I/O error whose OS error code will\n /// be communicated back to the parent and returned as an error from when\n /// the spawn was requested.\n ///\n /// Multiple closures can be registered and they will be called in order of\n /// their registration. If a closure returns `Err` then no further closures\n /// will be called and the spawn operation will immediately return with a\n /// failure.\n ///\n /// # Safety\n ///\n /// This closure will be run in the context of the child process after a\n /// `fork`. This primarily means that any modifications made to memory on\n /// behalf of this closure will **not** be visible to the parent process.\n /// This is often a very constrained environment where normal operations\n /// like `malloc` or acquiring a mutex are not guaranteed to work (due to\n /// other threads perhaps still running when the `fork` was run).\n ///\n /// This also means that all resources such as file descriptors and\n /// memory-mapped regions got duplicated. It is your responsibility to make\n /// sure that the closure does not violate library invariants by making\n /// invalid use of these duplicates.\n ///\n /// When this closure is run, aspects such as the stdio file descriptors and\n /// working directory have successfully been changed, so output to these\n /// locations may not appear where intended.\n #[cfg(unix)]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n pub unsafe fn pre_exec(&mut self, f: F) -> &mut Command\n where\n F: FnMut() -> io::Result<()> + Send + Sync + 'static,\n {\n unsafe { self.std.pre_exec(f) };\n self\n }\n\n /// Sets the process group ID (PGID) of the child process. Equivalent to a\n /// `setpgid` call in the child process, but may be more efficient.\n ///\n /// Process groups determine which processes receive signals.\n ///\n /// # Examples\n ///\n /// Pressing Ctrl-C in a terminal will send `SIGINT` to all processes\n /// in the current foreground process group. By spawning the `sleep`\n /// subprocess in a new process group, it will not receive `SIGINT`\n /// from the terminal.\n ///\n /// The parent process could install a [signal handler] and manage the\n /// process on its own terms.\n ///\n /// A process group ID of 0 will use the process ID as the PGID.\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use tokio::process::Command;\n ///\n /// let output = Command::new(\"sleep\")\n /// .arg(\"10\")\n /// .process_group(0)\n /// .output()\n /// .await\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// [signal handler]: crate::signal\n #[cfg(unix)]\n #[cfg_attr(docsrs, doc(cfg(unix)))]\n pub fn process_group(&mut self, pgroup: i32) -> &mut Command {\n self.std.process_group(pgroup);\n self\n }\n\n /// Executes the command as a child process, returning a handle to it.\n ///\n /// By default, stdin, stdout and stderr are inherited from the parent.\n ///\n /// This method will spawn the child process synchronously and return a\n /// handle to a future-aware child process. The `Child` returned implements\n /// `Future` itself to acquire the `ExitStatus` of the child, and otherwise\n /// the `Child` has methods to acquire handles to the stdin, stdout, and\n /// stderr streams.\n ///\n /// All I/O this child does will be associated with the current default\n /// event loop.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// use tokio::process::Command;\n ///\n /// async fn run_ls() -> std::process::ExitStatus {\n /// Command::new(\"ls\")\n /// .spawn()\n /// .expect(\"ls command failed to start\")\n /// .wait()\n /// .await\n /// .expect(\"ls command failed to run\")\n /// }\n /// ```\n ///\n /// # Caveats\n ///\n /// ## Dropping/Cancellation\n ///\n /// Similar to the behavior to the standard library, and unlike the futures\n /// paradigm of dropping-implies-cancellation, a spawned process will, by\n /// default, continue to execute even after the `Child` handle has been dropped.\n ///\n /// The [`Command::kill_on_drop`] method can be used to modify this behavior\n /// and kill the child process if the `Child` wrapper is dropped before it\n /// has exited.\n ///\n /// ## Unix Processes\n ///\n /// On Unix platforms processes must be \"reaped\" by their parent process after\n /// they have exited in order to release all OS resources. A child process which\n /// has exited, but has not yet been reaped by its parent is considered a \"zombie\"\n /// process. Such processes continue to count against limits imposed by the system,\n /// and having too many zombie processes present can prevent additional processes\n /// from being spawned.\n ///\n /// The tokio runtime will, on a best-effort basis, attempt to reap and clean up\n /// any process which it has spawned. No additional guarantees are made with regard to\n /// how quickly or how often this procedure will take place.\n ///\n /// It is recommended to avoid dropping a [`Child`] process handle before it has been\n /// fully `await`ed if stricter cleanup guarantees are required.\n ///\n /// [`Command`]: crate::process::Command\n /// [`Command::kill_on_drop`]: crate::process::Command::kill_on_drop\n /// [`Child`]: crate::process::Child\n ///\n /// # Errors\n ///\n /// On Unix platforms this method will fail with `std::io::ErrorKind::WouldBlock`\n /// if the system process limit is reached (which includes other applications\n /// running on the system).\n #[inline]\n pub fn spawn(&mut self) -> io::Result {\n // On two lines to circumvent a mutable borrow check failure.\n let child = self.std.spawn()?;\n self.build_child(child)\n }\n\n /// Executes the command as a child process with a custom spawning function,\n /// returning a handle to it.\n ///\n /// This is identical to [`Self::spawn`] in every aspect except the spawn:\n /// here, it is customizable through the `with` parameter instead of\n /// defaulting to the usual spawn. In fact, [`Self::spawn`] is just\n /// [`Self::spawn_with`] with [`StdCommand::spawn`].\n ///\n /// This is useful mostly under Windows for now, since the platform exposes\n /// special APIs to configure child processes when spawning them with various\n /// attributes that customize the exact behavior of the spawn operation.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// # async fn test() { // allow using await\n /// use std::process::Stdio;\n ///\n /// let output = tokio::process::Command::new(\"ls\")\n /// .stdin(Stdio::null())\n /// .stdout(Stdio::piped())\n /// .stderr(Stdio::piped())\n /// .spawn_with(std::process::Command::spawn)\n /// .unwrap()\n /// .wait_with_output()\n /// .await\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// Actually customizing the spawn under Windows:\n ///\n /// ```ignore\n /// #![feature(windows_process_extensions_raw_attribute)]\n /// # #[cfg(windows)] // Windows-only nightly APIs are used here.\n /// # async fn test() { // Allow using await.\n /// use std::os::windows::process::{CommandExt, ProcThreadAttributeList};\n /// use std::process::Stdio;\n /// use tokio::process::Command;\n ///\n /// let parent = Command::new(\"cmd\").spawn().unwrap();\n /// let parent_process_handle = parent.raw_handle();\n ///\n /// const PROC_THREAD_ATTRIBUTE_PARENT_PROCESS: usize = 0x00020000;\n /// let attribute_list = ProcThreadAttributeList::build()\n /// .attribute(PROC_THREAD_ATTRIBUTE_PARENT_PROCESS, &parent_process_handle)\n /// .finish()\n /// .unwrap();\n ///\n /// let _output = Command::new(\"ls\")\n /// .stdin(Stdio::null())\n /// .stdout(Stdio::piped())\n /// .stderr(Stdio::piped())\n /// .spawn_with(|cmd| cmd.spawn_with_attributes(&attribute_list))\n /// .unwrap()\n /// .wait_with_output()\n /// .await\n /// .unwrap();\n /// # }\n /// ```\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n #[inline]\n pub fn spawn_with(\n &mut self,\n with: impl FnOnce(&mut StdCommand) -> io::Result,\n ) -> io::Result {\n // On two lines to circumvent a mutable borrow check failure.\n let child = with(&mut self.std)?;\n self.build_child(child)\n }\n\n /// Small indirection for the spawn implementations.\n ///\n /// This is introduced for [`Self::spawn`] and [`Self::spawn_with`] to use:\n /// [`Self::spawn`] cannot depend directly on [`Self::spawn_with`] since\n /// it is behind `tokio_unstable`. It also serves as a way to reduce\n /// monomorphization bloat by taking in an already-spawned child process\n /// instead of a command and custom spawn function.\n fn build_child(&self, child: StdChild) -> io::Result {\n let spawned_child = imp::build_child(child)?;\n\n Ok(Child {\n child: FusedChild::Child(ChildDropGuard {\n inner: spawned_child.child,\n kill_on_drop: self.kill_on_drop,\n }),\n stdin: spawned_child.stdin.map(|inner| ChildStdin { inner }),\n stdout: spawned_child.stdout.map(|inner| ChildStdout { inner }),\n stderr: spawned_child.stderr.map(|inner| ChildStderr { inner }),\n })\n }\n\n /// Executes the command as a child process, waiting for it to finish and\n /// collecting its exit status.\n ///\n /// By default, stdin, stdout and stderr are inherited from the parent.\n /// If any input/output handles are set to a pipe then they will be immediately\n /// closed after the child is spawned.\n ///\n /// All I/O this child does will be associated with the current default\n /// event loop.\n ///\n /// The destructor of the future returned by this function will kill\n /// the child if [`kill_on_drop`] is set to true.\n ///\n /// [`kill_on_drop`]: fn@Self::kill_on_drop\n ///\n /// # Errors\n ///\n /// This future will return an error if the child process cannot be spawned\n /// or if there is an error while awaiting its status.\n ///\n /// On Unix platforms this method will fail with `std::io::ErrorKind::WouldBlock`\n /// if the system process limit is reached (which includes other applications\n /// running on the system).\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// use tokio::process::Command;\n ///\n /// async fn run_ls() -> std::process::ExitStatus {\n /// Command::new(\"ls\")\n /// .status()\n /// .await\n /// .expect(\"ls command failed to run\")\n /// }\n /// ```\n pub fn status(&mut self) -> impl Future> {\n let child = self.spawn();\n\n async {\n let mut child = child?;\n\n // Ensure we close any stdio handles so we can't deadlock\n // waiting on the child which may be waiting to read/write\n // to a pipe we're holding.\n child.stdin.take();\n child.stdout.take();\n child.stderr.take();\n\n child.wait().await\n }\n }\n\n /// Executes the command as a child process, waiting for it to finish and\n /// collecting all of its output.\n ///\n /// > **Note**: this method, unlike the standard library, will\n /// > unconditionally configure the stdout/stderr handles to be pipes, even\n /// > if they have been previously configured. If this is not desired then\n /// > the `spawn` method should be used in combination with the\n /// > `wait_with_output` method on child.\n ///\n /// This method will return a future representing the collection of the\n /// child process's stdout/stderr. It will resolve to\n /// the `Output` type in the standard library, containing `stdout` and\n /// `stderr` as `Vec` along with an `ExitStatus` representing how the\n /// process exited.\n ///\n /// All I/O this child does will be associated with the current default\n /// event loop.\n ///\n /// The destructor of the future returned by this function will kill\n /// the child if [`kill_on_drop`] is set to true.\n ///\n /// [`kill_on_drop`]: fn@Self::kill_on_drop\n ///\n /// # Errors\n ///\n /// This future will return an error if the child process cannot be spawned\n /// or if there is an error while awaiting its status.\n ///\n /// On Unix platforms this method will fail with `std::io::ErrorKind::WouldBlock`\n /// if the system process limit is reached (which includes other applications\n /// running on the system).\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```no_run\n /// use tokio::process::Command;\n ///\n /// async fn run_ls() {\n /// let output: std::process::Output = Command::new(\"ls\")\n /// .output()\n /// .await\n /// .expect(\"ls command failed to run\");\n /// println!(\"stderr of ls: {:?}\", output.stderr);\n /// }\n /// ```\n pub fn output(&mut self) -> impl Future> {\n self.std.stdout(Stdio::piped());\n self.std.stderr(Stdio::piped());\n\n let child = self.spawn();\n\n async { child?.wait_with_output().await }\n }\n\n /// Returns the boolean value that was previously set by [`Command::kill_on_drop`].\n ///\n /// Note that if you have not previously called [`Command::kill_on_drop`], the\n /// default value of `false` will be returned here.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::process::Command;\n ///\n /// let mut cmd = Command::new(\"echo\");\n /// assert!(!cmd.get_kill_on_drop());\n ///\n /// cmd.kill_on_drop(true);\n /// assert!(cmd.get_kill_on_drop());\n /// ```\n pub fn get_kill_on_drop(&self) -> bool {\n self.kill_on_drop\n }\n}\n\nimpl From for Command {\n fn from(std: StdCommand) -> Command {\n Command {\n std,\n kill_on_drop: false,\n }\n }\n}\n\n/// A drop guard which can ensure the child process is killed on drop if specified.\n#[derive(Debug)]\nstruct ChildDropGuard {\n inner: T,\n kill_on_drop: bool,\n}\n\nimpl Kill for ChildDropGuard {\n fn kill(&mut self) -> io::Result<()> {\n let ret = self.inner.kill();\n\n if ret.is_ok() {\n self.kill_on_drop = false;\n }\n\n ret\n }\n}\n\nimpl Drop for ChildDropGuard {\n fn drop(&mut self) {\n if self.kill_on_drop {\n drop(self.kill());\n }\n }\n}\n\nimpl Future for ChildDropGuard\nwhere\n F: Future> + Kill + Unpin,\n{\n type Output = Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n ready!(crate::trace::trace_leaf(cx));\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let ret = Pin::new(&mut self.inner).poll(cx);\n\n if let Poll::Ready(Ok(_)) = ret {\n // Avoid the overhead of trying to kill a reaped process\n self.kill_on_drop = false;\n }\n\n if ret.is_ready() {\n coop.made_progress();\n }\n\n ret\n }\n}\n\n/// Keeps track of the exit status of a child process without worrying about\n/// polling the underlying futures even after they have completed.\n#[derive(Debug)]\nenum FusedChild {\n Child(ChildDropGuard),\n Done(ExitStatus),\n}\n\n/// Representation of a child process spawned onto an event loop.\n///\n/// # Caveats\n/// Similar to the behavior to the standard library, and unlike the futures\n/// paradigm of dropping-implies-cancellation, a spawned process will, by\n/// default, continue to execute even after the `Child` handle has been dropped.\n///\n/// The `Command::kill_on_drop` method can be used to modify this behavior\n/// and kill the child process if the `Child` wrapper is dropped before it\n/// has exited.\n#[derive(Debug)]\npub struct Child {\n child: FusedChild,\n\n /// The handle for writing to the child's standard input (stdin), if it has\n /// been captured. To avoid partially moving the `child` and thus blocking\n /// yourself from calling functions on `child` while using `stdin`, you might\n /// find it helpful to do:\n ///\n /// ```no_run\n /// # let mut child = tokio::process::Command::new(\"echo\").spawn().unwrap();\n /// let stdin = child.stdin.take().unwrap();\n /// ```\n pub stdin: Option,\n\n /// The handle for reading from the child's standard output (stdout), if it\n /// has been captured. You might find it helpful to do\n ///\n /// ```no_run\n /// # let mut child = tokio::process::Command::new(\"echo\").spawn().unwrap();\n /// let stdout = child.stdout.take().unwrap();\n /// ```\n ///\n /// to avoid partially moving the `child` and thus blocking yourself from calling\n /// functions on `child` while using `stdout`.\n pub stdout: Option,\n\n /// The handle for reading from the child's standard error (stderr), if it\n /// has been captured. You might find it helpful to do\n ///\n /// ```no_run\n /// # let mut child = tokio::process::Command::new(\"echo\").spawn().unwrap();\n /// let stderr = child.stderr.take().unwrap();\n /// ```\n ///\n /// to avoid partially moving the `child` and thus blocking yourself from calling\n /// functions on `child` while using `stderr`.\n pub stderr: Option,\n}\n\nimpl Child {\n /// Returns the OS-assigned process identifier associated with this child\n /// while it is still running.\n ///\n /// Once the child has been polled to completion this will return `None`.\n /// This is done to avoid confusion on platforms like Unix where the OS\n /// identifier could be reused once the process has completed.\n pub fn id(&self) -> Option {\n match &self.child {\n FusedChild::Child(child) => Some(child.inner.id()),\n FusedChild::Done(_) => None,\n }\n }\n\n cfg_windows! {\n /// Extracts the raw handle of the process associated with this child while\n /// it is still running. Returns `None` if the child has exited.\n pub fn raw_handle(&self) -> Option {\n match &self.child {\n FusedChild::Child(c) => Some(c.inner.as_raw_handle()),\n FusedChild::Done(_) => None,\n }\n }\n }\n\n /// Attempts to force the child to exit, but does not wait for the request\n /// to take effect.\n ///\n /// On Unix platforms, this is the equivalent to sending a `SIGKILL`. Note\n /// that on Unix platforms it is possible for a zombie process to remain\n /// after a kill is sent; to avoid this, the caller should ensure that either\n /// `child.wait().await` or `child.try_wait()` is invoked successfully.\n pub fn start_kill(&mut self) -> io::Result<()> {\n match &mut self.child {\n FusedChild::Child(child) => child.kill(),\n FusedChild::Done(_) => Ok(()),\n }\n }\n\n /// Forces the child to exit.\n ///\n /// This is equivalent to sending a `SIGKILL` on unix platforms\n /// followed by [`wait`](Child::wait).\n ///\n /// Note: std version of [`Child::kill`](std::process::Child::kill) does not `wait`.\n /// For an equivalent of `Child::kill` in the standard library,\n /// use [`start_kill`](Child::start_kill).\n ///\n /// # Examples\n ///\n /// If the child has to be killed remotely, it is possible to do it using\n /// a combination of the select! macro and a `oneshot` channel. In the following\n /// example, the child will run until completion unless a message is sent on\n /// the `oneshot` channel. If that happens, the child is killed immediately\n /// using the `.kill()` method.\n ///\n /// ```no_run\n /// use tokio::process::Command;\n /// use tokio::sync::oneshot::channel;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (send, recv) = channel::<()>();\n /// let mut child = Command::new(\"sleep\").arg(\"1\").spawn().unwrap();\n /// tokio::spawn(async move { send.send(()) });\n /// tokio::select! {\n /// _ = child.wait() => {}\n /// _ = recv => child.kill().await.expect(\"kill failed\"),\n /// }\n /// }\n /// ```\n ///\n /// You can also interact with the child's standard I/O. For example, you can\n /// read its stdout while waiting for it to exit.\n ///\n /// ```no_run\n /// # use std::process::Stdio;\n /// #\n /// # use tokio::io::AsyncReadExt;\n /// # use tokio::process::Command;\n /// # use tokio::sync::oneshot::channel;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (_tx, rx) = channel::<()>();\n ///\n /// let mut child = Command::new(\"echo\")\n /// .arg(\"Hello World!\")\n /// .stdout(Stdio::piped())\n /// .spawn()\n /// .unwrap();\n ///\n /// let mut stdout = child.stdout.take().expect(\"stdout is not captured\");\n ///\n /// let read_stdout = tokio::spawn(async move {\n /// let mut buff = Vec::new();\n /// let _ = stdout.read_to_end(&mut buff).await;\n ///\n /// buff\n /// });\n ///\n /// tokio::select! {\n /// _ = child.wait() => {}\n /// _ = rx => { child.kill().await.expect(\"kill failed\") },\n /// }\n ///\n /// let buff = read_stdout.await.unwrap();\n ///\n /// assert_eq!(buff, b\"Hello World!\\n\");\n /// }\n /// ```\n pub async fn kill(&mut self) -> io::Result<()> {\n self.start_kill()?;\n self.wait().await?;\n Ok(())\n }\n\n /// Waits for the child to exit completely, returning the status that it\n /// exited with. This function will continue to have the same return value\n /// after it has been called at least once.\n ///\n /// The stdin handle to the child process, if any, will be closed\n /// before waiting. This helps avoid deadlock: it ensures that the\n /// child does not block waiting for input from the parent, while\n /// the parent waits for the child to exit.\n ///\n /// If the caller wishes to explicitly control when the child's stdin\n /// handle is closed, they may `.take()` it before calling `.wait()`:\n ///\n /// # Cancel safety\n ///\n /// This function is cancel safe.\n ///\n /// ```\n /// # #[cfg(not(unix))]fn main(){}\n /// # #[cfg(unix)]\n /// use tokio::io::AsyncWriteExt;\n /// # #[cfg(unix)]\n /// use tokio::process::Command;\n /// # #[cfg(unix)]\n /// use std::process::Stdio;\n ///\n /// # #[cfg(unix)]\n /// #[tokio::main]\n /// async fn main() {\n /// # if cfg!(miri) { return; } // No `pidfd_spawnp` in miri.\n /// let mut child = Command::new(\"cat\")\n /// .stdin(Stdio::piped())\n /// .spawn()\n /// .unwrap();\n ///\n /// let mut stdin = child.stdin.take().unwrap();\n /// tokio::spawn(async move {\n /// // do something with stdin here...\n /// stdin.write_all(b\"hello world\\n\").await.unwrap();\n ///\n /// // then drop when finished\n /// drop(stdin);\n /// });\n ///\n /// // wait for the process to complete\n /// let _ = child.wait().await;\n /// }\n /// ```\n pub async fn wait(&mut self) -> io::Result {\n // Ensure stdin is closed so the child isn't stuck waiting on\n // input while the parent is waiting for it to exit.\n drop(self.stdin.take());\n\n match &mut self.child {\n FusedChild::Done(exit) => Ok(*exit),\n FusedChild::Child(child) => {\n let ret = child.await;\n\n if let Ok(exit) = ret {\n self.child = FusedChild::Done(exit);\n }\n\n ret\n }\n }\n }\n\n /// Attempts to collect the exit status of the child if it has already\n /// exited.\n ///\n /// This function will not block the calling thread and will only\n /// check to see if the child process has exited or not. If the child has\n /// exited then on Unix the process ID is reaped. This function is\n /// guaranteed to repeatedly return a successful exit status so long as the\n /// child has already exited.\n ///\n /// If the child has exited, then `Ok(Some(status))` is returned. If the\n /// exit status is not available at this time then `Ok(None)` is returned.\n /// If an error occurs, then that error is returned.\n ///\n /// Note that unlike `wait`, this function will not attempt to drop stdin,\n /// nor will it wake the current task if the child exits.\n pub fn try_wait(&mut self) -> io::Result> {\n match &mut self.child {\n FusedChild::Done(exit) => Ok(Some(*exit)),\n FusedChild::Child(guard) => {\n let ret = guard.inner.try_wait();\n\n if let Ok(Some(exit)) = ret {\n // Avoid the overhead of trying to kill a reaped process\n guard.kill_on_drop = false;\n self.child = FusedChild::Done(exit);\n }\n\n ret\n }\n }\n }\n\n /// Returns a future that will resolve to an `Output`, containing the exit\n /// status, stdout, and stderr of the child process.\n ///\n /// The returned future will simultaneously waits for the child to exit and\n /// collect all remaining output on the stdout/stderr handles, returning an\n /// `Output` instance.\n ///\n /// The stdin handle to the child process, if any, will be closed before\n /// waiting. This helps avoid deadlock: it ensures that the child does not\n /// block waiting for input from the parent, while the parent waits for the\n /// child to exit.\n ///\n /// By default, stdin, stdout and stderr are inherited from the parent. In\n /// order to capture the output into this `Output` it is necessary to create\n /// new pipes between parent and child. Use `stdout(Stdio::piped())` or\n /// `stderr(Stdio::piped())`, respectively, when creating a `Command`.\n pub async fn wait_with_output(mut self) -> io::Result {\n use crate::future::try_join3;\n\n async fn read_to_end(io: &mut Option) -> io::Result> {\n let mut vec = Vec::new();\n if let Some(io) = io.as_mut() {\n crate::io::util::read_to_end(io, &mut vec).await?;\n }\n Ok(vec)\n }\n\n let mut stdout_pipe = self.stdout.take();\n let mut stderr_pipe = self.stderr.take();\n\n let stdout_fut = read_to_end(&mut stdout_pipe);\n let stderr_fut = read_to_end(&mut stderr_pipe);\n\n let (status, stdout, stderr) = try_join3(self.wait(), stdout_fut, stderr_fut).await?;\n\n // Drop happens after `try_join` due to \n drop(stdout_pipe);\n drop(stderr_pipe);\n\n Ok(Output {\n status,\n stdout,\n stderr,\n })\n }\n}\n\n/// The standard input stream for spawned children.\n///\n/// This type implements the `AsyncWrite` trait to pass data to the stdin\n/// handle of a child process asynchronously.\n#[derive(Debug)]\npub struct ChildStdin {\n inner: imp::ChildStdio,\n}\n\n/// The standard output stream for spawned children.\n///\n/// This type implements the `AsyncRead` trait to read data from the stdout\n/// handle of a child process asynchronously.\n#[derive(Debug)]\npub struct ChildStdout {\n inner: imp::ChildStdio,\n}\n\n/// The standard error stream for spawned children.\n///\n/// This type implements the `AsyncRead` trait to read data from the stderr\n/// handle of a child process asynchronously.\n#[derive(Debug)]\npub struct ChildStderr {\n inner: imp::ChildStdio,\n}\n\nimpl ChildStdin {\n /// Creates an asynchronous `ChildStdin` from a synchronous one.\n ///\n /// # Errors\n ///\n /// This method may fail if an error is encountered when setting the pipe to\n /// non-blocking mode, or when registering the pipe with the runtime's IO\n /// driver.\n pub fn from_std(inner: std::process::ChildStdin) -> io::Result {\n Ok(Self {\n inner: imp::stdio(inner)?,\n })\n }\n}\n\nimpl ChildStdout {\n /// Creates an asynchronous `ChildStdout` from a synchronous one.\n ///\n /// # Errors\n ///\n /// This method may fail if an error is encountered when setting the pipe to\n /// non-blocking mode, or when registering the pipe with the runtime's IO\n /// driver.\n pub fn from_std(inner: std::process::ChildStdout) -> io::Result {\n Ok(Self {\n inner: imp::stdio(inner)?,\n })\n }\n}\n\nimpl ChildStderr {\n /// Creates an asynchronous `ChildStderr` from a synchronous one.\n ///\n /// # Errors\n ///\n /// This method may fail if an error is encountered when setting the pipe to\n /// non-blocking mode, or when registering the pipe with the runtime's IO\n /// driver.\n pub fn from_std(inner: std::process::ChildStderr) -> io::Result {\n Ok(Self {\n inner: imp::stdio(inner)?,\n })\n }\n}\n\nimpl AsyncWrite for ChildStdin {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_write(cx, buf)\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_shutdown(cx)\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.inner.is_write_vectored()\n }\n}\n\nimpl AsyncRead for ChildStdout {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_read(cx, buf)\n }\n}\n\nimpl AsyncRead for ChildStderr {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_read(cx, buf)\n }\n}\n\nimpl TryInto for ChildStdin {\n type Error = io::Error;\n\n fn try_into(self) -> Result {\n imp::convert_to_stdio(self.inner)\n }\n}\n\nimpl TryInto for ChildStdout {\n type Error = io::Error;\n\n fn try_into(self) -> Result {\n imp::convert_to_stdio(self.inner)\n }\n}\n\nimpl TryInto for ChildStderr {\n type Error = io::Error;\n\n fn try_into(self) -> Result {\n imp::convert_to_stdio(self.inner)\n }\n}\n\n#[cfg(unix)]\n#[cfg_attr(docsrs, doc(cfg(unix)))]\nmod sys {\n use std::{\n io,\n os::unix::io::{AsFd, AsRawFd, BorrowedFd, OwnedFd, RawFd},\n };\n\n use super::{ChildStderr, ChildStdin, ChildStdout};\n\n macro_rules! impl_traits {\n ($type:ty) => {\n impl $type {\n /// Convert into [`OwnedFd`].\n pub fn into_owned_fd(self) -> io::Result {\n self.inner.into_owned_fd()\n }\n }\n\n impl AsRawFd for $type {\n fn as_raw_fd(&self) -> RawFd {\n self.inner.as_raw_fd()\n }\n }\n\n impl AsFd for $type {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n }\n };\n }\n\n impl_traits!(ChildStdin);\n impl_traits!(ChildStdout);\n impl_traits!(ChildStderr);\n}\n\n#[cfg(any(windows, docsrs))]\n#[cfg_attr(docsrs, doc(cfg(windows)))]\nmod windows {\n use super::*;\n use crate::os::windows::io::{AsHandle, AsRawHandle, BorrowedHandle, OwnedHandle, RawHandle};\n\n #[cfg(not(docsrs))]\n macro_rules! impl_traits {\n ($type:ty) => {\n impl $type {\n /// Convert into [`OwnedHandle`].\n pub fn into_owned_handle(self) -> io::Result {\n self.inner.into_owned_handle()\n }\n }\n\n impl AsRawHandle for $type {\n fn as_raw_handle(&self) -> RawHandle {\n self.inner.as_raw_handle()\n }\n }\n\n impl AsHandle for $type {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n unsafe { BorrowedHandle::borrow_raw(self.as_raw_handle()) }\n }\n }\n };\n }\n\n #[cfg(docsrs)]\n macro_rules! impl_traits {\n ($type:ty) => {\n impl $type {\n /// Convert into [`OwnedHandle`].\n pub fn into_owned_handle(self) -> io::Result {\n todo!(\"For doc generation only\")\n }\n }\n\n impl AsRawHandle for $type {\n fn as_raw_handle(&self) -> RawHandle {\n todo!(\"For doc generation only\")\n }\n }\n\n impl AsHandle for $type {\n fn as_handle(&self) -> BorrowedHandle<'_> {\n todo!(\"For doc generation only\")\n }\n }\n };\n }\n\n impl_traits!(ChildStdin);\n impl_traits!(ChildStdout);\n impl_traits!(ChildStderr);\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::kill::Kill;\n use super::ChildDropGuard;\n\n use futures::future::FutureExt;\n use std::future::Future;\n use std::io;\n use std::pin::Pin;\n use std::task::{Context, Poll};\n\n struct Mock {\n num_kills: usize,\n num_polls: usize,\n poll_result: Poll>,\n }\n\n impl Mock {\n fn new() -> Self {\n Self::with_result(Poll::Pending)\n }\n\n fn with_result(result: Poll>) -> Self {\n Self {\n num_kills: 0,\n num_polls: 0,\n poll_result: result,\n }\n }\n }\n\n impl Kill for Mock {\n fn kill(&mut self) -> io::Result<()> {\n self.num_kills += 1;\n Ok(())\n }\n }\n\n impl Future for Mock {\n type Output = Result<(), ()>;\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n let inner = Pin::get_mut(self);\n inner.num_polls += 1;\n inner.poll_result\n }\n }\n\n #[test]\n fn kills_on_drop_if_specified() {\n let mut mock = Mock::new();\n\n {\n let guard = ChildDropGuard {\n inner: &mut mock,\n kill_on_drop: true,\n };\n drop(guard);\n }\n\n assert_eq!(1, mock.num_kills);\n assert_eq!(0, mock.num_polls);\n }\n\n #[test]\n fn no_kill_on_drop_by_default() {\n let mut mock = Mock::new();\n\n {\n let guard = ChildDropGuard {\n inner: &mut mock,\n kill_on_drop: false,\n };\n drop(guard);\n }\n\n assert_eq!(0, mock.num_kills);\n assert_eq!(0, mock.num_polls);\n }\n\n #[test]\n fn no_kill_if_already_killed() {\n let mut mock = Mock::new();\n\n {\n let mut guard = ChildDropGuard {\n inner: &mut mock,\n kill_on_drop: true,\n };\n let _ = guard.kill();\n drop(guard);\n }\n\n assert_eq!(1, mock.num_kills);\n assert_eq!(0, mock.num_polls);\n }\n\n #[test]\n fn no_kill_if_reaped() {\n let mut mock_pending = Mock::with_result(Poll::Pending);\n let mut mock_reaped = Mock::with_result(Poll::Ready(Ok(())));\n let mut mock_err = Mock::with_result(Poll::Ready(Err(())));\n\n let waker = futures::task::noop_waker();\n let mut context = Context::from_waker(&waker);\n {\n let mut guard = ChildDropGuard {\n inner: &mut mock_pending,\n kill_on_drop: true,\n };\n let _ = guard.poll_unpin(&mut context);\n\n let mut guard = ChildDropGuard {\n inner: &mut mock_reaped,\n kill_on_drop: true,\n };\n let _ = guard.poll_unpin(&mut context);\n\n let mut guard = ChildDropGuard {\n inner: &mut mock_err,\n kill_on_drop: true,\n };\n let _ = guard.poll_unpin(&mut context);\n }\n\n assert_eq!(1, mock_pending.num_kills);\n assert_eq!(1, mock_pending.num_polls);\n\n assert_eq!(0, mock_reaped.num_kills);\n assert_eq!(1, mock_reaped.num_polls);\n\n assert_eq!(1, mock_err.num_kills);\n assert_eq!(1, mock_err.num_polls);\n }\n}\n" }, { "path": "tokio/src/process/unix/mod.rs", "content": "//! Unix handling of child processes.\n//!\n//! Right now the only \"fancy\" thing about this is how we implement the\n//! `Future` implementation on `Child` to get the exit status. Unix offers\n//! no way to register a child with epoll, and the only real way to get a\n//! notification when a process exits is the SIGCHLD signal.\n//!\n//! Signal handling in general is *super* hairy and complicated, and it's even\n//! more complicated here with the fact that signals are coalesced, so we may\n//! not get a SIGCHLD-per-child.\n//!\n//! Our best approximation here is to check *all spawned processes* for all\n//! SIGCHLD signals received. To do that we create a `Signal`, implemented in\n//! the `tokio-net` crate, which is a stream over signals being received.\n//!\n//! Later when we poll the process's exit status we simply check to see if a\n//! SIGCHLD has happened since we last checked, and while that returns \"yes\" we\n//! keep trying.\n//!\n//! Note that this means that this isn't really scalable, but then again\n//! processes in general aren't scalable (e.g. millions) so it shouldn't be that\n//! bad in theory...\n\npub(crate) mod orphan;\nuse orphan::{OrphanQueue, OrphanQueueImpl, Wait};\n\nmod reap;\nuse reap::Reaper;\n\n#[cfg(all(target_os = \"linux\", feature = \"rt\"))]\nmod pidfd_reaper;\n\nuse crate::io::{AsyncRead, AsyncWrite, PollEvented, ReadBuf};\nuse crate::process::kill::Kill;\nuse crate::process::SpawnedChild;\nuse crate::runtime::signal::Handle as SignalHandle;\nuse crate::signal::unix::{signal, Signal, SignalKind};\n\nuse mio::event::Source;\nuse mio::unix::SourceFd;\nuse std::fmt;\nuse std::fs::File;\nuse std::future::Future;\nuse std::io;\nuse std::os::unix::io::{AsFd, AsRawFd, BorrowedFd, FromRawFd, IntoRawFd, OwnedFd, RawFd};\nuse std::pin::Pin;\nuse std::process::{Child as StdChild, ExitStatus, Stdio};\nuse std::task::Context;\nuse std::task::Poll;\n\nimpl Wait for StdChild {\n fn id(&self) -> u32 {\n self.id()\n }\n\n fn try_wait(&mut self) -> io::Result> {\n self.try_wait()\n }\n}\n\nimpl Kill for StdChild {\n fn kill(&mut self) -> io::Result<()> {\n self.kill()\n }\n}\n\ncfg_not_has_const_mutex_new! {\n fn get_orphan_queue() -> &'static OrphanQueueImpl {\n use std::sync::OnceLock;\n\n static ORPHAN_QUEUE: OnceLock> = OnceLock::new();\n\n ORPHAN_QUEUE.get_or_init(OrphanQueueImpl::new)\n }\n}\n\ncfg_has_const_mutex_new! {\n fn get_orphan_queue() -> &'static OrphanQueueImpl {\n static ORPHAN_QUEUE: OrphanQueueImpl = OrphanQueueImpl::new();\n\n &ORPHAN_QUEUE\n }\n}\n\npub(crate) struct GlobalOrphanQueue;\n\nimpl fmt::Debug for GlobalOrphanQueue {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n get_orphan_queue().fmt(fmt)\n }\n}\n\nimpl GlobalOrphanQueue {\n pub(crate) fn reap_orphans(handle: &SignalHandle) {\n get_orphan_queue().reap_orphans(handle);\n }\n}\n\nimpl OrphanQueue for GlobalOrphanQueue {\n fn push_orphan(&self, orphan: StdChild) {\n get_orphan_queue().push_orphan(orphan);\n }\n}\n\n#[must_use = \"futures do nothing unless polled\"]\npub(crate) enum Child {\n SignalReaper(Reaper),\n #[cfg(all(target_os = \"linux\", feature = \"rt\"))]\n PidfdReaper(pidfd_reaper::PidfdReaper),\n}\n\nimpl fmt::Debug for Child {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Child\").field(\"pid\", &self.id()).finish()\n }\n}\n\npub(crate) fn build_child(mut child: StdChild) -> io::Result {\n let stdin = child.stdin.take().map(stdio).transpose()?;\n let stdout = child.stdout.take().map(stdio).transpose()?;\n let stderr = child.stderr.take().map(stdio).transpose()?;\n\n #[cfg(all(target_os = \"linux\", feature = \"rt\"))]\n match pidfd_reaper::PidfdReaper::new(child, GlobalOrphanQueue) {\n Ok(pidfd_reaper) => {\n return Ok(SpawnedChild {\n child: Child::PidfdReaper(pidfd_reaper),\n stdin,\n stdout,\n stderr,\n })\n }\n Err((Some(err), _child)) => return Err(err),\n Err((None, child_returned)) => child = child_returned,\n }\n\n let signal = signal(SignalKind::child())?;\n\n Ok(SpawnedChild {\n child: Child::SignalReaper(Reaper::new(child, GlobalOrphanQueue, signal)),\n stdin,\n stdout,\n stderr,\n })\n}\n\nimpl Child {\n pub(crate) fn id(&self) -> u32 {\n match self {\n Self::SignalReaper(signal_reaper) => signal_reaper.id(),\n #[cfg(all(target_os = \"linux\", feature = \"rt\"))]\n Self::PidfdReaper(pidfd_reaper) => pidfd_reaper.id(),\n }\n }\n\n fn std_child(&mut self) -> &mut StdChild {\n match self {\n Self::SignalReaper(signal_reaper) => signal_reaper.inner_mut(),\n #[cfg(all(target_os = \"linux\", feature = \"rt\"))]\n Self::PidfdReaper(pidfd_reaper) => pidfd_reaper.inner_mut(),\n }\n }\n\n pub(crate) fn try_wait(&mut self) -> io::Result> {\n self.std_child().try_wait()\n }\n}\n\nimpl Kill for Child {\n fn kill(&mut self) -> io::Result<()> {\n self.std_child().kill()\n }\n}\n\nimpl Future for Child {\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n match Pin::into_inner(self) {\n Self::SignalReaper(signal_reaper) => Pin::new(signal_reaper).poll(cx),\n #[cfg(all(target_os = \"linux\", feature = \"rt\"))]\n Self::PidfdReaper(pidfd_reaper) => Pin::new(pidfd_reaper).poll(cx),\n }\n }\n}\n\n#[derive(Debug)]\npub(crate) struct Pipe {\n // Actually a pipe is not a File. However, we are reusing `File` to get\n // close on drop. This is a similar trick as `mio`.\n fd: File,\n}\n\nimpl From for Pipe {\n fn from(fd: T) -> Self {\n let fd = unsafe { File::from_raw_fd(fd.into_raw_fd()) };\n Self { fd }\n }\n}\n\nimpl io::Read for &Pipe {\n fn read(&mut self, bytes: &mut [u8]) -> io::Result {\n (&self.fd).read(bytes)\n }\n}\n\nimpl io::Write for &Pipe {\n fn write(&mut self, bytes: &[u8]) -> io::Result {\n (&self.fd).write(bytes)\n }\n\n fn flush(&mut self) -> io::Result<()> {\n (&self.fd).flush()\n }\n\n fn write_vectored(&mut self, bufs: &[io::IoSlice<'_>]) -> io::Result {\n (&self.fd).write_vectored(bufs)\n }\n}\n\nimpl AsRawFd for Pipe {\n fn as_raw_fd(&self) -> RawFd {\n self.fd.as_raw_fd()\n }\n}\n\nimpl AsFd for Pipe {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\nfn convert_to_blocking_file(io: ChildStdio) -> io::Result {\n let mut fd = io.inner.into_inner()?.fd;\n\n // Ensure that the fd to be inherited is set to *blocking* mode, as this\n // is the default that virtually all programs expect to have. Those\n // programs that know how to work with nonblocking stdio will know how to\n // change it to nonblocking mode.\n set_nonblocking(&mut fd, false)?;\n\n Ok(fd)\n}\n\npub(crate) fn convert_to_stdio(io: ChildStdio) -> io::Result {\n convert_to_blocking_file(io).map(Stdio::from)\n}\n\nimpl Source for Pipe {\n fn register(\n &mut self,\n registry: &mio::Registry,\n token: mio::Token,\n interest: mio::Interest,\n ) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).register(registry, token, interest)\n }\n\n fn reregister(\n &mut self,\n registry: &mio::Registry,\n token: mio::Token,\n interest: mio::Interest,\n ) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).reregister(registry, token, interest)\n }\n\n fn deregister(&mut self, registry: &mio::Registry) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).deregister(registry)\n }\n}\n\npub(crate) struct ChildStdio {\n inner: PollEvented,\n}\n\nimpl ChildStdio {\n pub(super) fn into_owned_fd(self) -> io::Result {\n convert_to_blocking_file(self).map(OwnedFd::from)\n }\n}\n\nimpl fmt::Debug for ChildStdio {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.inner.fmt(fmt)\n }\n}\n\nimpl AsRawFd for ChildStdio {\n fn as_raw_fd(&self) -> RawFd {\n self.inner.as_raw_fd()\n }\n}\n\nimpl AsFd for ChildStdio {\n fn as_fd(&self) -> BorrowedFd<'_> {\n unsafe { BorrowedFd::borrow_raw(self.as_raw_fd()) }\n }\n}\n\nimpl AsyncWrite for ChildStdio {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.inner.poll_write(cx, buf)\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n self.inner.poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n}\n\nimpl AsyncRead for ChildStdio {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n // Safety: pipes support reading into uninitialized memory\n unsafe { self.inner.poll_read(cx, buf) }\n }\n}\n\nfn set_nonblocking(fd: &mut T, nonblocking: bool) -> io::Result<()> {\n unsafe {\n let fd = fd.as_raw_fd();\n let previous = libc::fcntl(fd, libc::F_GETFL);\n if previous == -1 {\n return Err(io::Error::last_os_error());\n }\n\n let new = if nonblocking {\n previous | libc::O_NONBLOCK\n } else {\n previous & !libc::O_NONBLOCK\n };\n\n let r = libc::fcntl(fd, libc::F_SETFL, new);\n if r == -1 {\n return Err(io::Error::last_os_error());\n }\n }\n\n Ok(())\n}\n\npub(super) fn stdio(io: T) -> io::Result\nwhere\n T: IntoRawFd,\n{\n // Set the fd to nonblocking before we pass it to the event loop\n let mut pipe = Pipe::from(io);\n set_nonblocking(&mut pipe, true)?;\n\n PollEvented::new(pipe).map(|inner| ChildStdio { inner })\n}\n" }, { "path": "tokio/src/process/unix/orphan.rs", "content": "use crate::loom::sync::{Mutex, MutexGuard};\nuse crate::runtime::signal::Handle as SignalHandle;\nuse crate::signal::unix::{signal_with_handle, SignalKind};\nuse crate::sync::watch;\nuse std::io;\nuse std::process::ExitStatus;\n\n/// An interface for waiting on a process to exit.\npub(crate) trait Wait {\n /// Get the identifier for this process or diagnostics.\n #[allow(dead_code)]\n fn id(&self) -> u32;\n /// Try waiting for a process to exit in a non-blocking manner.\n fn try_wait(&mut self) -> io::Result>;\n}\n\nimpl Wait for &mut T {\n fn id(&self) -> u32 {\n (**self).id()\n }\n\n fn try_wait(&mut self) -> io::Result> {\n (**self).try_wait()\n }\n}\n\n/// An interface for queueing up an orphaned process so that it can be reaped.\npub(crate) trait OrphanQueue {\n /// Adds an orphan to the queue.\n fn push_orphan(&self, orphan: T);\n}\n\nimpl> OrphanQueue for &O {\n fn push_orphan(&self, orphan: T) {\n (**self).push_orphan(orphan);\n }\n}\n\n/// An implementation of `OrphanQueue`.\n#[derive(Debug)]\npub(crate) struct OrphanQueueImpl {\n sigchild: Mutex>>,\n queue: Mutex>,\n}\n\nimpl OrphanQueueImpl {\n cfg_not_has_const_mutex_new! {\n pub(crate) fn new() -> Self {\n Self {\n sigchild: Mutex::new(None),\n queue: Mutex::new(Vec::new()),\n }\n }\n }\n\n cfg_has_const_mutex_new! {\n pub(crate) const fn new() -> Self {\n Self {\n sigchild: Mutex::const_new(None),\n queue: Mutex::const_new(Vec::new()),\n }\n }\n }\n\n #[cfg(test)]\n fn len(&self) -> usize {\n self.queue.lock().len()\n }\n\n pub(crate) fn push_orphan(&self, orphan: T)\n where\n T: Wait,\n {\n self.queue.lock().push(orphan);\n }\n\n /// Attempts to reap every process in the queue, ignoring any errors and\n /// enqueueing any orphans which have not yet exited.\n pub(crate) fn reap_orphans(&self, handle: &SignalHandle)\n where\n T: Wait,\n {\n // If someone else is holding the lock, they will be responsible for draining\n // the queue as necessary, so we can safely bail if that happens\n if let Some(mut sigchild_guard) = self.sigchild.try_lock() {\n match &mut *sigchild_guard {\n Some(sigchild) => {\n if sigchild.try_has_changed().and_then(Result::ok).is_some() {\n drain_orphan_queue(self.queue.lock());\n }\n }\n None => {\n let queue = self.queue.lock();\n\n // Be lazy and only initialize the SIGCHLD listener if there\n // are any orphaned processes in the queue.\n if !queue.is_empty() {\n // An errors shouldn't really happen here, but if it does it\n // means that the signal driver isn't running, in\n // which case there isn't anything we can\n // register/initialize here, so we can try again later\n if let Ok(sigchild) = signal_with_handle(SignalKind::child(), handle) {\n *sigchild_guard = Some(sigchild);\n drain_orphan_queue(queue);\n }\n }\n }\n }\n }\n }\n}\n\nfn drain_orphan_queue(mut queue: MutexGuard<'_, Vec>)\nwhere\n T: Wait,\n{\n for i in (0..queue.len()).rev() {\n match queue[i].try_wait() {\n Ok(None) => {}\n Ok(Some(_)) | Err(_) => {\n // The stdlib handles interruption errors (EINTR) when polling a child process.\n // All other errors represent invalid inputs or pids that have already been\n // reaped, so we can drop the orphan in case an error is raised.\n queue.swap_remove(i);\n }\n }\n }\n\n drop(queue);\n}\n\n#[cfg(all(test, not(loom)))]\npub(crate) mod test {\n use super::*;\n use crate::runtime::io::Driver as IoDriver;\n use crate::runtime::signal::{Driver as SignalDriver, Handle as SignalHandle};\n use crate::sync::watch;\n use std::cell::{Cell, RefCell};\n use std::io;\n use std::os::unix::process::ExitStatusExt;\n use std::process::ExitStatus;\n use std::rc::Rc;\n\n pub(crate) struct MockQueue {\n pub(crate) all_enqueued: RefCell>,\n }\n\n impl MockQueue {\n pub(crate) fn new() -> Self {\n Self {\n all_enqueued: RefCell::new(Vec::new()),\n }\n }\n }\n\n impl OrphanQueue for MockQueue {\n fn push_orphan(&self, orphan: W) {\n self.all_enqueued.borrow_mut().push(orphan);\n }\n }\n\n struct MockWait {\n total_waits: Rc>,\n num_wait_until_status: usize,\n return_err: bool,\n }\n\n impl MockWait {\n fn new(num_wait_until_status: usize) -> Self {\n Self {\n total_waits: Rc::new(Cell::new(0)),\n num_wait_until_status,\n return_err: false,\n }\n }\n\n fn with_err() -> Self {\n Self {\n total_waits: Rc::new(Cell::new(0)),\n num_wait_until_status: 0,\n return_err: true,\n }\n }\n }\n\n impl Wait for MockWait {\n fn id(&self) -> u32 {\n 42\n }\n\n fn try_wait(&mut self) -> io::Result> {\n let waits = self.total_waits.get();\n\n let ret = if self.num_wait_until_status == waits {\n if self.return_err {\n Ok(Some(ExitStatus::from_raw(0)))\n } else {\n Err(io::Error::new(io::ErrorKind::Other, \"mock err\"))\n }\n } else {\n Ok(None)\n };\n\n self.total_waits.set(waits + 1);\n ret\n }\n }\n\n #[test]\n fn drain_attempts_a_single_reap_of_all_queued_orphans() {\n let first_orphan = MockWait::new(0);\n let second_orphan = MockWait::new(1);\n let third_orphan = MockWait::new(2);\n let fourth_orphan = MockWait::with_err();\n\n let first_waits = first_orphan.total_waits.clone();\n let second_waits = second_orphan.total_waits.clone();\n let third_waits = third_orphan.total_waits.clone();\n let fourth_waits = fourth_orphan.total_waits.clone();\n\n let orphanage = OrphanQueueImpl::new();\n orphanage.push_orphan(first_orphan);\n orphanage.push_orphan(third_orphan);\n orphanage.push_orphan(second_orphan);\n orphanage.push_orphan(fourth_orphan);\n\n assert_eq!(orphanage.len(), 4);\n\n drain_orphan_queue(orphanage.queue.lock());\n assert_eq!(orphanage.len(), 2);\n assert_eq!(first_waits.get(), 1);\n assert_eq!(second_waits.get(), 1);\n assert_eq!(third_waits.get(), 1);\n assert_eq!(fourth_waits.get(), 1);\n\n drain_orphan_queue(orphanage.queue.lock());\n assert_eq!(orphanage.len(), 1);\n assert_eq!(first_waits.get(), 1);\n assert_eq!(second_waits.get(), 2);\n assert_eq!(third_waits.get(), 2);\n assert_eq!(fourth_waits.get(), 1);\n\n drain_orphan_queue(orphanage.queue.lock());\n assert_eq!(orphanage.len(), 0);\n assert_eq!(first_waits.get(), 1);\n assert_eq!(second_waits.get(), 2);\n assert_eq!(third_waits.get(), 3);\n assert_eq!(fourth_waits.get(), 1);\n\n // Safe to reap when empty\n drain_orphan_queue(orphanage.queue.lock());\n }\n\n #[test]\n fn no_reap_if_no_signal_received() {\n let (tx, rx) = watch::channel(());\n\n let handle = SignalHandle::default();\n\n let orphanage = OrphanQueueImpl::new();\n *orphanage.sigchild.lock() = Some(rx);\n\n let orphan = MockWait::new(2);\n let waits = orphan.total_waits.clone();\n orphanage.push_orphan(orphan);\n\n orphanage.reap_orphans(&handle);\n assert_eq!(waits.get(), 0);\n\n orphanage.reap_orphans(&handle);\n assert_eq!(waits.get(), 0);\n\n tx.send(()).unwrap();\n orphanage.reap_orphans(&handle);\n assert_eq!(waits.get(), 1);\n }\n\n #[test]\n fn no_reap_if_signal_lock_held() {\n let handle = SignalHandle::default();\n\n let orphanage = OrphanQueueImpl::new();\n let signal_guard = orphanage.sigchild.lock();\n\n let orphan = MockWait::new(2);\n let waits = orphan.total_waits.clone();\n orphanage.push_orphan(orphan);\n\n orphanage.reap_orphans(&handle);\n assert_eq!(waits.get(), 0);\n\n drop(signal_guard);\n }\n\n #[cfg_attr(miri, ignore)] // No `sigaction` on Miri\n #[test]\n fn does_not_register_signal_if_queue_empty() {\n let (io_driver, io_handle) = IoDriver::new(1024).unwrap();\n let signal_driver = SignalDriver::new(io_driver, &io_handle).unwrap();\n let handle = signal_driver.handle();\n\n let orphanage = OrphanQueueImpl::new();\n assert!(orphanage.sigchild.lock().is_none()); // Sanity\n\n // No register when queue empty\n orphanage.reap_orphans(&handle);\n assert!(orphanage.sigchild.lock().is_none());\n\n let orphan = MockWait::new(2);\n let waits = orphan.total_waits.clone();\n orphanage.push_orphan(orphan);\n\n orphanage.reap_orphans(&handle);\n assert!(orphanage.sigchild.lock().is_some());\n assert_eq!(waits.get(), 1); // Eager reap when registering listener\n }\n\n #[test]\n fn does_nothing_if_signal_could_not_be_registered() {\n let handle = SignalHandle::default();\n\n let orphanage = OrphanQueueImpl::new();\n assert!(orphanage.sigchild.lock().is_none());\n\n let orphan = MockWait::new(2);\n let waits = orphan.total_waits.clone();\n orphanage.push_orphan(orphan);\n\n // Signal handler has \"gone away\", nothing to register or reap\n orphanage.reap_orphans(&handle);\n assert!(orphanage.sigchild.lock().is_none());\n assert_eq!(waits.get(), 0);\n }\n}\n" }, { "path": "tokio/src/process/unix/pidfd_reaper.rs", "content": "use crate::{\n io::{interest::Interest, PollEvented},\n process::{\n imp::{orphan::Wait, OrphanQueue},\n kill::Kill,\n },\n};\n\nuse libc::{syscall, SYS_pidfd_open, ENOSYS, PIDFD_NONBLOCK};\nuse mio::{event::Source, unix::SourceFd};\nuse std::{\n fs::File,\n future::Future,\n io,\n marker::Unpin,\n ops::Deref,\n os::unix::io::{AsRawFd, FromRawFd, RawFd},\n pin::Pin,\n process::ExitStatus,\n sync::atomic::{AtomicBool, Ordering::Relaxed},\n task::{Context, Poll},\n};\n\n#[derive(Debug)]\nstruct Pidfd {\n fd: File,\n}\n\nimpl Pidfd {\n fn open(pid: u32) -> Option {\n // Store false (0) to reduce executable size\n static NO_PIDFD_SUPPORT: AtomicBool = AtomicBool::new(false);\n\n if NO_PIDFD_SUPPORT.load(Relaxed) {\n return None;\n }\n\n // Safety: The following function calls invovkes syscall pidfd_open,\n // which takes two parameter: pidfd_open(fd: c_int, flag: c_int)\n let fd = unsafe { syscall(SYS_pidfd_open, pid, PIDFD_NONBLOCK) };\n if fd == -1 {\n let errno = io::Error::last_os_error().raw_os_error().unwrap();\n\n if errno == ENOSYS {\n NO_PIDFD_SUPPORT.store(true, Relaxed)\n }\n\n None\n } else {\n // Safety: pidfd_open returns -1 on error or a valid fd with ownership.\n Some(Pidfd {\n fd: unsafe { File::from_raw_fd(fd as i32) },\n })\n }\n }\n}\n\nimpl AsRawFd for Pidfd {\n fn as_raw_fd(&self) -> RawFd {\n self.fd.as_raw_fd()\n }\n}\n\nimpl Source for Pidfd {\n fn register(\n &mut self,\n registry: &mio::Registry,\n token: mio::Token,\n interest: mio::Interest,\n ) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).register(registry, token, interest)\n }\n\n fn reregister(\n &mut self,\n registry: &mio::Registry,\n token: mio::Token,\n interest: mio::Interest,\n ) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).reregister(registry, token, interest)\n }\n\n fn deregister(&mut self, registry: &mio::Registry) -> io::Result<()> {\n SourceFd(&self.as_raw_fd()).deregister(registry)\n }\n}\n\n#[derive(Debug)]\nstruct PidfdReaperInner\nwhere\n W: Unpin,\n{\n inner: W,\n pidfd: PollEvented,\n}\n\nimpl Future for PidfdReaperInner\nwhere\n W: Wait + Unpin,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = Pin::into_inner(self);\n\n match this.pidfd.registration().poll_read_ready(cx) {\n Poll::Ready(Ok(evt)) => {\n if let Some(exit_code) = this.inner.try_wait()? {\n return Poll::Ready(Ok(exit_code));\n }\n this.pidfd.registration().clear_readiness(evt);\n }\n Poll::Ready(Err(err)) if crate::runtime::is_rt_shutdown_err(&err) => {}\n Poll::Ready(Err(err)) => return Poll::Ready(Err(err)),\n Poll::Pending => return Poll::Pending,\n };\n\n this.pidfd.reregister(Interest::READABLE)?;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n}\n\n#[derive(Debug)]\npub(crate) struct PidfdReaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n{\n inner: Option>,\n orphan_queue: Q,\n}\n\nimpl Deref for PidfdReaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n{\n type Target = W;\n\n fn deref(&self) -> &Self::Target {\n &self.inner.as_ref().expect(\"inner has gone away\").inner\n }\n}\n\nimpl PidfdReaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n{\n pub(crate) fn new(inner: W, orphan_queue: Q) -> Result, W)> {\n if let Some(pidfd) = Pidfd::open(inner.id()) {\n match PollEvented::new_with_interest(pidfd, Interest::READABLE) {\n Ok(pidfd) => Ok(Self {\n inner: Some(PidfdReaperInner { pidfd, inner }),\n orphan_queue,\n }),\n Err(io_error) => Err((Some(io_error), inner)),\n }\n } else {\n Err((None, inner))\n }\n }\n\n pub(crate) fn inner_mut(&mut self) -> &mut W {\n &mut self.inner.as_mut().expect(\"inner has gone away\").inner\n }\n}\n\nimpl Future for PidfdReaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n{\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n Pin::new(\n Pin::into_inner(self)\n .inner\n .as_mut()\n .expect(\"inner has gone away\"),\n )\n .poll(cx)\n }\n}\n\nimpl Kill for PidfdReaper\nwhere\n W: Wait + Unpin + Kill,\n Q: OrphanQueue + Unpin,\n{\n fn kill(&mut self) -> io::Result<()> {\n self.inner_mut().kill()\n }\n}\n\nimpl Drop for PidfdReaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n{\n fn drop(&mut self) {\n let mut orphan = self.inner.take().expect(\"inner has gone away\").inner;\n if let Ok(Some(_)) = orphan.try_wait() {\n return;\n }\n\n self.orphan_queue.push_orphan(orphan);\n }\n}\n\n#[cfg(all(test, not(loom), not(miri)))]\nmod test {\n use super::*;\n use crate::{\n process::unix::orphan::test::MockQueue,\n runtime::{Builder as RuntimeBuilder, Runtime},\n };\n use std::process::{Command, Output};\n\n fn create_runtime() -> Runtime {\n RuntimeBuilder::new_current_thread()\n .enable_io()\n .build()\n .unwrap()\n }\n\n fn run_test(fut: impl Future) {\n create_runtime().block_on(fut)\n }\n\n fn is_pidfd_available() -> bool {\n let Output { stdout, status, .. } = Command::new(\"uname\").arg(\"-r\").output().unwrap();\n assert!(status.success());\n let stdout = String::from_utf8_lossy(&stdout);\n\n let mut kernel_version_iter = match stdout.split_once('-') {\n Some((version, _)) => version,\n _ => &stdout,\n }\n .split('.');\n\n let major: u32 = kernel_version_iter.next().unwrap().parse().unwrap();\n let minor: u32 = kernel_version_iter.next().unwrap().trim().parse().unwrap();\n\n major >= 6 || (major == 5 && minor >= 10)\n }\n\n #[test]\n fn test_pidfd_reaper_poll() {\n if !is_pidfd_available() {\n eprintln!(\"pidfd is not available on this linux kernel, skip this test\");\n return;\n }\n\n let queue = MockQueue::new();\n\n run_test(async {\n let child = Command::new(\"true\").spawn().unwrap();\n let pidfd_reaper = PidfdReaper::new(child, &queue).unwrap();\n\n let exit_status = pidfd_reaper.await.unwrap();\n assert!(exit_status.success());\n });\n\n assert!(queue.all_enqueued.borrow().is_empty());\n }\n\n #[test]\n fn test_pidfd_reaper_kill() {\n if !is_pidfd_available() {\n eprintln!(\"pidfd is not available on this linux kernel, skip this test\");\n return;\n }\n\n let queue = MockQueue::new();\n\n run_test(async {\n let child = Command::new(\"sleep\").arg(\"1800\").spawn().unwrap();\n let mut pidfd_reaper = PidfdReaper::new(child, &queue).unwrap();\n\n pidfd_reaper.kill().unwrap();\n\n let exit_status = pidfd_reaper.await.unwrap();\n assert!(!exit_status.success());\n });\n\n assert!(queue.all_enqueued.borrow().is_empty());\n }\n\n #[test]\n fn test_pidfd_reaper_drop() {\n if !is_pidfd_available() {\n eprintln!(\"pidfd is not available on this linux kernel, skip this test\");\n return;\n }\n\n let queue = MockQueue::new();\n\n let mut child = Command::new(\"sleep\").arg(\"1800\").spawn().unwrap();\n\n run_test(async {\n let _pidfd_reaper = PidfdReaper::new(&mut child, &queue).unwrap();\n });\n\n assert_eq!(queue.all_enqueued.borrow().len(), 1);\n\n child.kill().unwrap();\n child.wait().unwrap();\n }\n}\n" }, { "path": "tokio/src/process/unix/reap.rs", "content": "use crate::process::imp::orphan::{OrphanQueue, Wait};\nuse crate::process::kill::Kill;\nuse crate::signal::unix::InternalStream;\n\nuse std::future::Future;\nuse std::io;\nuse std::ops::Deref;\nuse std::pin::Pin;\nuse std::process::ExitStatus;\nuse std::task::Context;\nuse std::task::Poll;\n\n/// Orchestrates between registering interest for receiving signals when a\n/// child process has exited, and attempting to poll for process completion.\n#[derive(Debug)]\npub(crate) struct Reaper\nwhere\n W: Wait,\n Q: OrphanQueue,\n{\n inner: Option,\n orphan_queue: Q,\n signal: S,\n}\n\nimpl Deref for Reaper\nwhere\n W: Wait,\n Q: OrphanQueue,\n{\n type Target = W;\n\n fn deref(&self) -> &Self::Target {\n self.inner()\n }\n}\n\nimpl Reaper\nwhere\n W: Wait,\n Q: OrphanQueue,\n{\n pub(crate) fn new(inner: W, orphan_queue: Q, signal: S) -> Self {\n Self {\n inner: Some(inner),\n orphan_queue,\n signal,\n }\n }\n\n fn inner(&self) -> &W {\n self.inner.as_ref().expect(\"inner has gone away\")\n }\n\n pub(crate) fn inner_mut(&mut self) -> &mut W {\n self.inner.as_mut().expect(\"inner has gone away\")\n }\n}\n\nimpl Future for Reaper\nwhere\n W: Wait + Unpin,\n Q: OrphanQueue + Unpin,\n S: InternalStream + Unpin,\n{\n type Output = io::Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n loop {\n // If the child hasn't exited yet, then it's our responsibility to\n // ensure the current task gets notified when it might be able to\n // make progress. We can use the delivery of a SIGCHLD signal as a\n // sign that we can potentially make progress.\n //\n // However, we will register for a notification on the next signal\n // BEFORE we poll the child. Otherwise it is possible that the child\n // can exit and the signal can arrive after we last polled the child,\n // but before we've registered for a notification on the next signal\n // (this can cause a deadlock if there are no more spawned children\n // which can generate a different signal for us). A side effect of\n // pre-registering for signal notifications is that when the child\n // exits, we will have already registered for an additional\n // notification we don't need to consume. If another signal arrives,\n // this future's task will be notified/woken up again. Since the\n // futures model allows for spurious wake ups this extra wakeup\n // should not cause significant issues with parent futures.\n let registered_interest = self.signal.poll_recv(cx).is_pending();\n\n if let Some(status) = self.inner_mut().try_wait()? {\n return Poll::Ready(Ok(status));\n }\n\n // If our attempt to poll for the next signal was not ready, then\n // we've arranged for our task to get notified and we can bail out.\n if registered_interest {\n return Poll::Pending;\n } else {\n // Otherwise, if the signal stream delivered a signal to us, we\n // won't get notified at the next signal, so we'll loop and try\n // again.\n continue;\n }\n }\n }\n}\n\nimpl Kill for Reaper\nwhere\n W: Kill + Wait,\n Q: OrphanQueue,\n{\n fn kill(&mut self) -> io::Result<()> {\n self.inner_mut().kill()\n }\n}\n\nimpl Drop for Reaper\nwhere\n W: Wait,\n Q: OrphanQueue,\n{\n fn drop(&mut self) {\n if let Ok(Some(_)) = self.inner_mut().try_wait() {\n return;\n }\n\n let orphan = self.inner.take().unwrap();\n self.orphan_queue.push_orphan(orphan);\n }\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n use crate::process::unix::orphan::test::MockQueue;\n use futures::future::FutureExt;\n use std::os::unix::process::ExitStatusExt;\n use std::process::ExitStatus;\n use std::task::Context;\n use std::task::Poll;\n\n #[derive(Debug)]\n struct MockWait {\n total_kills: usize,\n total_waits: usize,\n num_wait_until_status: usize,\n status: ExitStatus,\n }\n\n impl MockWait {\n fn new(status: ExitStatus, num_wait_until_status: usize) -> Self {\n Self {\n total_kills: 0,\n total_waits: 0,\n num_wait_until_status,\n status,\n }\n }\n }\n\n impl Wait for MockWait {\n fn id(&self) -> u32 {\n 0\n }\n\n fn try_wait(&mut self) -> io::Result> {\n let ret = if self.num_wait_until_status == self.total_waits {\n Some(self.status)\n } else {\n None\n };\n\n self.total_waits += 1;\n Ok(ret)\n }\n }\n\n impl Kill for MockWait {\n fn kill(&mut self) -> io::Result<()> {\n self.total_kills += 1;\n Ok(())\n }\n }\n\n struct MockStream {\n total_polls: usize,\n values: Vec>,\n }\n\n impl MockStream {\n fn new(values: Vec>) -> Self {\n Self {\n total_polls: 0,\n values,\n }\n }\n }\n\n impl InternalStream for MockStream {\n fn poll_recv(&mut self, _cx: &mut Context<'_>) -> Poll> {\n self.total_polls += 1;\n match self.values.remove(0) {\n Some(()) => Poll::Ready(Some(())),\n None => Poll::Pending,\n }\n }\n }\n\n #[test]\n fn reaper() {\n let exit = ExitStatus::from_raw(0);\n let mock = MockWait::new(exit, 3);\n let mut grim = Reaper::new(\n mock,\n MockQueue::new(),\n MockStream::new(vec![None, Some(()), None, None, None]),\n );\n\n let waker = futures::task::noop_waker();\n let mut context = Context::from_waker(&waker);\n\n // Not yet exited, interest registered\n assert!(grim.poll_unpin(&mut context).is_pending());\n assert_eq!(1, grim.signal.total_polls);\n assert_eq!(1, grim.total_waits);\n assert!(grim.orphan_queue.all_enqueued.borrow().is_empty());\n\n // Not yet exited, couldn't register interest the first time\n // but managed to register interest the second time around\n assert!(grim.poll_unpin(&mut context).is_pending());\n assert_eq!(3, grim.signal.total_polls);\n assert_eq!(3, grim.total_waits);\n assert!(grim.orphan_queue.all_enqueued.borrow().is_empty());\n\n // Exited\n if let Poll::Ready(r) = grim.poll_unpin(&mut context) {\n assert!(r.is_ok());\n let exit_code = r.unwrap();\n assert_eq!(exit_code, exit);\n } else {\n unreachable!();\n }\n assert_eq!(4, grim.signal.total_polls);\n assert_eq!(4, grim.total_waits);\n assert!(grim.orphan_queue.all_enqueued.borrow().is_empty());\n }\n\n #[test]\n fn kill() {\n let exit = ExitStatus::from_raw(0);\n let mut grim = Reaper::new(\n MockWait::new(exit, 0),\n MockQueue::new(),\n MockStream::new(vec![None]),\n );\n\n grim.kill().unwrap();\n assert_eq!(1, grim.total_kills);\n assert!(grim.orphan_queue.all_enqueued.borrow().is_empty());\n }\n\n #[test]\n fn drop_reaps_if_possible() {\n let exit = ExitStatus::from_raw(0);\n let mut mock = MockWait::new(exit, 0);\n\n {\n let queue = MockQueue::new();\n\n let grim = Reaper::new(&mut mock, &queue, MockStream::new(vec![]));\n\n drop(grim);\n\n assert!(queue.all_enqueued.borrow().is_empty());\n }\n\n assert_eq!(1, mock.total_waits);\n assert_eq!(0, mock.total_kills);\n }\n\n #[test]\n fn drop_enqueues_orphan_if_wait_fails() {\n let exit = ExitStatus::from_raw(0);\n let mut mock = MockWait::new(exit, 2);\n\n {\n let queue = MockQueue::<&mut MockWait>::new();\n let grim = Reaper::new(&mut mock, &queue, MockStream::new(vec![]));\n drop(grim);\n\n assert_eq!(1, queue.all_enqueued.borrow().len());\n }\n\n assert_eq!(1, mock.total_waits);\n assert_eq!(0, mock.total_kills);\n }\n}\n" }, { "path": "tokio/src/process/windows.rs", "content": "//! Windows asynchronous process handling.\n//!\n//! Like with Unix we don't actually have a way of registering a process with an\n//! IOCP object. As a result we similarly need another mechanism for getting a\n//! signal when a process has exited. For now this is implemented with the\n//! `RegisterWaitForSingleObject` function in the kernel32.dll.\n//!\n//! This strategy is the same that libuv takes and essentially just queues up a\n//! wait for the process in a kernel32-specific thread pool. Once the object is\n//! notified (e.g. the process exits) then we have a callback that basically\n//! just completes a `Oneshot`.\n//!\n//! The `poll_exit` implementation will attempt to wait for the process in a\n//! nonblocking fashion, but failing that it'll fire off a\n//! `RegisterWaitForSingleObject` and then wait on the other end of the oneshot\n//! from then on out.\n\nuse crate::io::{blocking::Blocking, AsyncRead, AsyncWrite, ReadBuf};\nuse crate::process::kill::Kill;\nuse crate::process::SpawnedChild;\nuse crate::sync::oneshot;\n\nuse std::fmt;\nuse std::fs::File as StdFile;\nuse std::future::Future;\nuse std::io;\nuse std::os::windows::prelude::{AsRawHandle, IntoRawHandle, OwnedHandle, RawHandle};\nuse std::pin::Pin;\nuse std::process::Stdio;\nuse std::process::{Child as StdChild, ExitStatus};\nuse std::ptr::null_mut;\nuse std::sync::Arc;\nuse std::task::{Context, Poll};\n\nuse windows_sys::{\n Win32::Foundation::{DuplicateHandle, DUPLICATE_SAME_ACCESS, HANDLE, INVALID_HANDLE_VALUE},\n Win32::System::Threading::{\n GetCurrentProcess, RegisterWaitForSingleObject, UnregisterWaitEx, INFINITE,\n WT_EXECUTEINWAITTHREAD, WT_EXECUTEONLYONCE,\n },\n};\n\n#[must_use = \"futures do nothing unless polled\"]\npub(crate) struct Child {\n child: StdChild,\n waiting: Option,\n}\n\nimpl fmt::Debug for Child {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Child\")\n .field(\"pid\", &self.id())\n .field(\"child\", &self.child)\n .field(\"waiting\", &\"..\")\n .finish()\n }\n}\n\nstruct Waiting {\n rx: oneshot::Receiver<()>,\n wait_object: HANDLE,\n tx: *mut Option>,\n}\n\nunsafe impl Sync for Waiting {}\nunsafe impl Send for Waiting {}\n\npub(crate) fn build_child(mut child: StdChild) -> io::Result {\n let stdin = child.stdin.take().map(stdio).transpose()?;\n let stdout = child.stdout.take().map(stdio).transpose()?;\n let stderr = child.stderr.take().map(stdio).transpose()?;\n\n Ok(SpawnedChild {\n child: Child {\n child,\n waiting: None,\n },\n stdin,\n stdout,\n stderr,\n })\n}\n\nimpl Child {\n pub(crate) fn id(&self) -> u32 {\n self.child.id()\n }\n\n pub(crate) fn try_wait(&mut self) -> io::Result> {\n self.child.try_wait()\n }\n}\n\nimpl Kill for Child {\n fn kill(&mut self) -> io::Result<()> {\n self.child.kill()\n }\n}\n\nimpl Future for Child {\n type Output = io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let inner = Pin::get_mut(self);\n loop {\n if let Some(ref mut w) = inner.waiting {\n match Pin::new(&mut w.rx).poll(cx) {\n Poll::Ready(Ok(())) => {}\n Poll::Ready(Err(_)) => panic!(\"should not be canceled\"),\n Poll::Pending => return Poll::Pending,\n }\n let status = inner.try_wait()?.expect(\"not ready yet\");\n return Poll::Ready(Ok(status));\n }\n\n if let Some(e) = inner.try_wait()? {\n return Poll::Ready(Ok(e));\n }\n let (tx, rx) = oneshot::channel();\n let ptr = Box::into_raw(Box::new(Some(tx)));\n let mut wait_object = null_mut();\n let rc = unsafe {\n RegisterWaitForSingleObject(\n &mut wait_object,\n inner.child.as_raw_handle() as _,\n Some(callback),\n ptr as *mut _,\n INFINITE,\n WT_EXECUTEINWAITTHREAD | WT_EXECUTEONLYONCE,\n )\n };\n if rc == 0 {\n let err = io::Error::last_os_error();\n drop(unsafe { Box::from_raw(ptr) });\n return Poll::Ready(Err(err));\n }\n inner.waiting = Some(Waiting {\n rx,\n wait_object,\n tx: ptr,\n });\n }\n }\n}\n\nimpl AsRawHandle for Child {\n fn as_raw_handle(&self) -> RawHandle {\n self.child.as_raw_handle()\n }\n}\n\nimpl Drop for Waiting {\n fn drop(&mut self) {\n unsafe {\n let rc = UnregisterWaitEx(self.wait_object, INVALID_HANDLE_VALUE);\n if rc == 0 {\n panic!(\"failed to unregister: {}\", io::Error::last_os_error());\n }\n drop(Box::from_raw(self.tx));\n }\n }\n}\n\nunsafe extern \"system\" fn callback(ptr: *mut std::ffi::c_void, _timer_fired: bool) {\n let complete = unsafe { &mut *(ptr as *mut Option>) };\n let _ = complete.take().unwrap().send(());\n}\n\n#[derive(Debug)]\nstruct ArcFile(Arc);\n\nimpl io::Read for ArcFile {\n fn read(&mut self, bytes: &mut [u8]) -> io::Result {\n (&*self.0).read(bytes)\n }\n}\n\nimpl io::Write for ArcFile {\n fn write(&mut self, bytes: &[u8]) -> io::Result {\n (&*self.0).write(bytes)\n }\n\n fn flush(&mut self) -> io::Result<()> {\n (&*self.0).flush()\n }\n}\n\n#[derive(Debug)]\npub(crate) struct ChildStdio {\n // Used for accessing the raw handle, even if the io version is busy\n raw: Arc,\n // For doing I/O operations asynchronously\n io: Blocking,\n}\n\nimpl ChildStdio {\n pub(super) fn into_owned_handle(self) -> io::Result {\n convert_to_file(self).map(OwnedHandle::from)\n }\n}\n\nimpl AsRawHandle for ChildStdio {\n fn as_raw_handle(&self) -> RawHandle {\n self.raw.as_raw_handle()\n }\n}\n\nimpl AsyncRead for ChildStdio {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut self.io).poll_read(cx, buf)\n }\n}\n\nimpl AsyncWrite for ChildStdio {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n Pin::new(&mut self.io).poll_write(cx, buf)\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.io).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.io).poll_shutdown(cx)\n }\n}\n\npub(super) fn stdio(io: T) -> io::Result\nwhere\n T: IntoRawHandle,\n{\n use std::os::windows::prelude::FromRawHandle;\n\n let raw = Arc::new(unsafe { StdFile::from_raw_handle(io.into_raw_handle()) });\n let io = ArcFile(raw.clone());\n // SAFETY: the `Read` implementation of `io` does not\n // read from the buffer it is borrowing and correctly\n // reports the length of the data written into the buffer.\n let io = unsafe { Blocking::new(io) };\n Ok(ChildStdio { raw, io })\n}\n\nfn convert_to_file(child_stdio: ChildStdio) -> io::Result {\n let ChildStdio { raw, io } = child_stdio;\n drop(io); // Try to drop the Arc count here\n\n Arc::try_unwrap(raw).or_else(|raw| duplicate_handle(&*raw))\n}\n\npub(crate) fn convert_to_stdio(child_stdio: ChildStdio) -> io::Result {\n convert_to_file(child_stdio).map(Stdio::from)\n}\n\nfn duplicate_handle(io: &T) -> io::Result {\n use std::os::windows::prelude::FromRawHandle;\n\n unsafe {\n let mut dup_handle = INVALID_HANDLE_VALUE;\n let cur_proc = GetCurrentProcess();\n\n let status = DuplicateHandle(\n cur_proc,\n io.as_raw_handle() as _,\n cur_proc,\n &mut dup_handle,\n 0,\n 0,\n DUPLICATE_SAME_ACCESS,\n );\n\n if status == 0 {\n return Err(io::Error::last_os_error());\n }\n\n Ok(StdFile::from_raw_handle(dup_handle as _))\n }\n}\n" }, { "path": "tokio/src/runtime/blocking/mod.rs", "content": "//! Abstracts out the APIs necessary to `Runtime` for integrating the blocking\n//! pool. When the `blocking` feature flag is **not** enabled, these APIs are\n//! shells. This isolates the complexity of dealing with conditional\n//! compilation.\n\nmod pool;\npub(crate) use pool::{spawn_blocking, BlockingPool, Spawner};\n\ncfg_fs! {\n pub(crate) use pool::spawn_mandatory_blocking;\n}\n\ncfg_trace! {\n pub(crate) use pool::Mandatory;\n}\n\nmod schedule;\nmod shutdown;\nmod task;\npub(crate) use task::BlockingTask;\n\nuse crate::runtime::Builder;\n\npub(crate) fn create_blocking_pool(builder: &Builder, thread_cap: usize) -> BlockingPool {\n BlockingPool::new(builder, thread_cap)\n}\n" }, { "path": "tokio/src/runtime/blocking/pool.rs", "content": "//! Thread pool for blocking operations\n\nuse crate::loom::sync::{Arc, Condvar, Mutex};\nuse crate::loom::thread;\nuse crate::runtime::blocking::schedule::BlockingSchedule;\nuse crate::runtime::blocking::{shutdown, BlockingTask};\nuse crate::runtime::builder::ThreadNameFn;\nuse crate::runtime::task::{self, JoinHandle};\nuse crate::runtime::{Builder, Callback, Handle, BOX_FUTURE_THRESHOLD};\nuse crate::util::metric_atomics::MetricAtomicUsize;\nuse crate::util::trace::{blocking_task, SpawnMeta};\n\nuse std::collections::{HashMap, VecDeque};\nuse std::fmt;\nuse std::io;\nuse std::sync::atomic::Ordering;\nuse std::time::Duration;\n\npub(crate) struct BlockingPool {\n spawner: Spawner,\n shutdown_rx: shutdown::Receiver,\n}\n\n#[derive(Clone)]\npub(crate) struct Spawner {\n inner: Arc,\n}\n\n#[derive(Default)]\npub(crate) struct SpawnerMetrics {\n num_threads: MetricAtomicUsize,\n num_idle_threads: MetricAtomicUsize,\n queue_depth: MetricAtomicUsize,\n}\n\nimpl SpawnerMetrics {\n fn num_threads(&self) -> usize {\n self.num_threads.load(Ordering::Relaxed)\n }\n\n fn num_idle_threads(&self) -> usize {\n self.num_idle_threads.load(Ordering::Relaxed)\n }\n\n cfg_unstable_metrics! {\n fn queue_depth(&self) -> usize {\n self.queue_depth.load(Ordering::Relaxed)\n }\n }\n\n fn inc_num_threads(&self) {\n self.num_threads.increment();\n }\n\n fn dec_num_threads(&self) {\n self.num_threads.decrement();\n }\n\n fn inc_num_idle_threads(&self) {\n self.num_idle_threads.increment();\n }\n\n fn dec_num_idle_threads(&self) -> usize {\n self.num_idle_threads.decrement()\n }\n\n fn inc_queue_depth(&self) {\n self.queue_depth.increment();\n }\n\n fn dec_queue_depth(&self) {\n self.queue_depth.decrement();\n }\n}\n\nstruct Inner {\n /// State shared between worker threads.\n shared: Mutex,\n\n /// Pool threads wait on this.\n condvar: Condvar,\n\n /// Spawned threads use this name.\n thread_name: ThreadNameFn,\n\n /// Spawned thread stack size.\n stack_size: Option,\n\n /// Call after a thread starts.\n after_start: Option,\n\n /// Call before a thread stops.\n before_stop: Option,\n\n // Maximum number of threads.\n thread_cap: usize,\n\n // Customizable wait timeout.\n keep_alive: Duration,\n\n // Metrics about the pool.\n metrics: SpawnerMetrics,\n}\n\nstruct Shared {\n queue: VecDeque,\n num_notify: u32,\n shutdown: bool,\n shutdown_tx: Option,\n /// Prior to shutdown, we clean up `JoinHandles` by having each timed-out\n /// thread join on the previous timed-out thread. This is not strictly\n /// necessary but helps avoid Valgrind false positives, see\n /// \n /// for more information.\n last_exiting_thread: Option>,\n /// This holds the `JoinHandles` for all running threads; on shutdown, the thread\n /// calling shutdown handles joining on these.\n worker_threads: HashMap>,\n /// This is a counter used to iterate `worker_threads` in a consistent order (for loom's\n /// benefit).\n worker_thread_index: usize,\n}\n\npub(crate) struct Task {\n task: task::UnownedTask,\n mandatory: Mandatory,\n}\n\n#[derive(PartialEq, Eq)]\npub(crate) enum Mandatory {\n #[cfg_attr(not(feature = \"fs\"), allow(dead_code))]\n Mandatory,\n NonMandatory,\n}\n\npub(crate) enum SpawnError {\n /// Pool is shutting down and the task was not scheduled\n ShuttingDown,\n /// There are no worker threads available to take the task\n /// and the OS failed to spawn a new one\n NoThreads(io::Error),\n}\n\nimpl From for io::Error {\n fn from(e: SpawnError) -> Self {\n match e {\n SpawnError::ShuttingDown => {\n io::Error::new(io::ErrorKind::Other, \"blocking pool shutting down\")\n }\n SpawnError::NoThreads(e) => e,\n }\n }\n}\n\nimpl Task {\n pub(crate) fn new(task: task::UnownedTask, mandatory: Mandatory) -> Task {\n Task { task, mandatory }\n }\n\n fn run(self) {\n self.task.run();\n }\n\n fn shutdown_or_run_if_mandatory(self) {\n match self.mandatory {\n Mandatory::NonMandatory => self.task.shutdown(),\n Mandatory::Mandatory => self.task.run(),\n }\n }\n}\n\nconst KEEP_ALIVE: Duration = Duration::from_secs(10);\n\n/// Runs the provided function on an executor dedicated to blocking operations.\n/// Tasks will be scheduled as non-mandatory, meaning they may not get executed\n/// in case of runtime shutdown.\n#[track_caller]\n#[cfg_attr(target_os = \"wasi\", allow(dead_code))]\npub(crate) fn spawn_blocking(func: F) -> JoinHandle\nwhere\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n{\n let rt = Handle::current();\n rt.spawn_blocking(func)\n}\n\ncfg_fs! {\n #[cfg_attr(any(\n all(loom, not(test)), // the function is covered by loom tests\n test\n ), allow(dead_code))]\n /// Runs the provided function on an executor dedicated to blocking\n /// operations. Tasks will be scheduled as mandatory, meaning they are\n /// guaranteed to run unless a shutdown is already taking place. In case a\n /// shutdown is already taking place, `None` will be returned.\n pub(crate) fn spawn_mandatory_blocking(func: F) -> Option>\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n let rt = Handle::current();\n rt.inner.blocking_spawner().spawn_mandatory_blocking(&rt, func)\n }\n}\n\n// ===== impl BlockingPool =====\n\nimpl BlockingPool {\n pub(crate) fn new(builder: &Builder, thread_cap: usize) -> BlockingPool {\n let (shutdown_tx, shutdown_rx) = shutdown::channel();\n let keep_alive = builder.keep_alive.unwrap_or(KEEP_ALIVE);\n\n BlockingPool {\n spawner: Spawner {\n inner: Arc::new(Inner {\n shared: Mutex::new(Shared {\n queue: VecDeque::new(),\n num_notify: 0,\n shutdown: false,\n shutdown_tx: Some(shutdown_tx),\n last_exiting_thread: None,\n worker_threads: HashMap::new(),\n worker_thread_index: 0,\n }),\n condvar: Condvar::new(),\n thread_name: builder.thread_name.clone(),\n stack_size: builder.thread_stack_size,\n after_start: builder.after_start.clone(),\n before_stop: builder.before_stop.clone(),\n thread_cap,\n keep_alive,\n metrics: SpawnerMetrics::default(),\n }),\n },\n shutdown_rx,\n }\n }\n\n pub(crate) fn spawner(&self) -> &Spawner {\n &self.spawner\n }\n\n pub(crate) fn shutdown(&mut self, timeout: Option) {\n let mut shared = self.spawner.inner.shared.lock();\n\n // The function can be called multiple times. First, by explicitly\n // calling `shutdown` then by the drop handler calling `shutdown`. This\n // prevents shutting down twice.\n if shared.shutdown {\n return;\n }\n\n shared.shutdown = true;\n shared.shutdown_tx = None;\n self.spawner.inner.condvar.notify_all();\n\n let last_exited_thread = std::mem::take(&mut shared.last_exiting_thread);\n let workers = std::mem::take(&mut shared.worker_threads);\n\n drop(shared);\n\n if self.shutdown_rx.wait(timeout) {\n let _ = last_exited_thread.map(thread::JoinHandle::join);\n\n // Loom requires that execution be deterministic, so sort by thread ID before joining.\n // (HashMaps use a randomly-seeded hash function, so the order is nondeterministic)\n #[cfg(loom)]\n let workers: Vec<(usize, thread::JoinHandle<()>)> = {\n let mut workers: Vec<_> = workers.into_iter().collect();\n workers.sort_by_key(|(id, _)| *id);\n workers\n };\n\n for (_id, handle) in workers {\n let _ = handle.join();\n }\n }\n }\n}\n\nimpl Drop for BlockingPool {\n fn drop(&mut self) {\n self.shutdown(None);\n }\n}\n\nimpl fmt::Debug for BlockingPool {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"BlockingPool\").finish()\n }\n}\n\n// ===== impl Spawner =====\n\nimpl Spawner {\n #[track_caller]\n pub(crate) fn spawn_blocking(&self, rt: &Handle, func: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n let fn_size = std::mem::size_of::();\n let (join_handle, spawn_result) = if fn_size > BOX_FUTURE_THRESHOLD {\n self.spawn_blocking_inner(\n Box::new(func),\n Mandatory::NonMandatory,\n SpawnMeta::new_unnamed(fn_size),\n rt,\n )\n } else {\n self.spawn_blocking_inner(\n func,\n Mandatory::NonMandatory,\n SpawnMeta::new_unnamed(fn_size),\n rt,\n )\n };\n\n match spawn_result {\n Ok(()) => join_handle,\n // Compat: do not panic here, return the join_handle even though it will never resolve\n Err(SpawnError::ShuttingDown) => join_handle,\n Err(SpawnError::NoThreads(e)) => {\n panic!(\"OS can't spawn worker thread: {e}\")\n }\n }\n }\n\n cfg_fs! {\n #[track_caller]\n #[cfg_attr(any(\n all(loom, not(test)), // the function is covered by loom tests\n test\n ), allow(dead_code))]\n pub(crate) fn spawn_mandatory_blocking(&self, rt: &Handle, func: F) -> Option>\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n let fn_size = std::mem::size_of::();\n let (join_handle, spawn_result) = if fn_size > BOX_FUTURE_THRESHOLD {\n self.spawn_blocking_inner(\n Box::new(func),\n Mandatory::Mandatory,\n SpawnMeta::new_unnamed(fn_size),\n rt,\n )\n } else {\n self.spawn_blocking_inner(\n func,\n Mandatory::Mandatory,\n SpawnMeta::new_unnamed(fn_size),\n rt,\n )\n };\n\n if spawn_result.is_ok() {\n Some(join_handle)\n } else {\n None\n }\n }\n }\n\n #[track_caller]\n pub(crate) fn spawn_blocking_inner(\n &self,\n func: F,\n is_mandatory: Mandatory,\n spawn_meta: SpawnMeta<'_>,\n rt: &Handle,\n ) -> (JoinHandle, Result<(), SpawnError>)\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n let id = task::Id::next();\n let fut =\n blocking_task::>(BlockingTask::new(func), spawn_meta, id.as_u64());\n\n let (task, handle) = task::unowned(\n fut,\n BlockingSchedule::new(rt),\n id,\n task::SpawnLocation::capture(),\n );\n\n let spawned = self.spawn_task(Task::new(task, is_mandatory), rt);\n (handle, spawned)\n }\n\n fn spawn_task(&self, task: Task, rt: &Handle) -> Result<(), SpawnError> {\n let mut shared = self.inner.shared.lock();\n\n if shared.shutdown {\n // Shutdown the task: it's fine to shutdown this task (even if\n // mandatory) because it was scheduled after the shutdown of the\n // runtime began.\n task.task.shutdown();\n\n // no need to even push this task; it would never get picked up\n return Err(SpawnError::ShuttingDown);\n }\n\n shared.queue.push_back(task);\n self.inner.metrics.inc_queue_depth();\n\n if self.inner.metrics.num_idle_threads() == 0 {\n // No threads are able to process the task.\n\n if self.inner.metrics.num_threads() == self.inner.thread_cap {\n // At max number of threads\n } else {\n assert!(shared.shutdown_tx.is_some());\n let shutdown_tx = shared.shutdown_tx.clone();\n\n if let Some(shutdown_tx) = shutdown_tx {\n let id = shared.worker_thread_index;\n\n match self.spawn_thread(shutdown_tx, rt, id) {\n Ok(handle) => {\n self.inner.metrics.inc_num_threads();\n shared.worker_thread_index += 1;\n shared.worker_threads.insert(id, handle);\n }\n Err(ref e)\n if is_temporary_os_thread_error(e)\n && self.inner.metrics.num_threads() > 0 =>\n {\n // OS temporarily failed to spawn a new thread.\n // The task will be picked up eventually by a currently\n // busy thread.\n }\n Err(e) => {\n // The OS refused to spawn the thread and there is no thread\n // to pick up the task that has just been pushed to the queue.\n return Err(SpawnError::NoThreads(e));\n }\n }\n }\n }\n } else {\n // Notify an idle worker thread. The notification counter\n // is used to count the needed amount of notifications\n // exactly. Thread libraries may generate spurious\n // wakeups, this counter is used to keep us in a\n // consistent state.\n self.inner.metrics.dec_num_idle_threads();\n shared.num_notify += 1;\n self.inner.condvar.notify_one();\n }\n\n Ok(())\n }\n\n fn spawn_thread(\n &self,\n shutdown_tx: shutdown::Sender,\n rt: &Handle,\n id: usize,\n ) -> io::Result> {\n let mut builder = thread::Builder::new().name((self.inner.thread_name)());\n\n if let Some(stack_size) = self.inner.stack_size {\n builder = builder.stack_size(stack_size);\n }\n\n let rt = rt.clone();\n\n builder.spawn(move || {\n // Only the reference should be moved into the closure\n let _enter = rt.enter();\n rt.inner.blocking_spawner().inner.run(id);\n drop(shutdown_tx);\n })\n }\n}\n\ncfg_unstable_metrics! {\n impl Spawner {\n pub(crate) fn num_threads(&self) -> usize {\n self.inner.metrics.num_threads()\n }\n\n pub(crate) fn num_idle_threads(&self) -> usize {\n self.inner.metrics.num_idle_threads()\n }\n\n pub(crate) fn queue_depth(&self) -> usize {\n self.inner.metrics.queue_depth()\n }\n }\n}\n\n// Tells whether the error when spawning a thread is temporary.\n#[inline]\nfn is_temporary_os_thread_error(error: &io::Error) -> bool {\n matches!(error.kind(), io::ErrorKind::WouldBlock)\n}\n\nimpl Inner {\n fn run(&self, worker_thread_id: usize) {\n if let Some(f) = &self.after_start {\n f();\n }\n\n let mut shared = self.shared.lock();\n let mut join_on_thread = None;\n // is this thread currently counted in `num_idle_threads`?\n let mut is_counted_idle;\n\n 'main: loop {\n // BUSY\n while let Some(task) = shared.queue.pop_front() {\n self.metrics.dec_queue_depth();\n drop(shared);\n task.run();\n\n shared = self.shared.lock();\n }\n\n // IDLE\n self.metrics.inc_num_idle_threads();\n // mark this thread as currently counted in `num_idle_threads`.\n is_counted_idle = true;\n\n while !shared.shutdown {\n let lock_result = self.condvar.wait_timeout(shared, self.keep_alive).unwrap();\n\n shared = lock_result.0;\n let timeout_result = lock_result.1;\n\n if shared.num_notify != 0 {\n // We have received a legitimate wakeup,\n // acknowledge it by decrementing the counter\n // and transition to the BUSY state.\n shared.num_notify -= 1;\n // since this is a legitimate wakeup,\n // the `Spawner::spawn_task` has already decremented `num_idle_threads`.\n is_counted_idle = false;\n break;\n }\n\n // Even if the condvar \"timed out\", if the pool is entering the\n // shutdown phase, we want to perform the cleanup logic.\n if !shared.shutdown && timeout_result.timed_out() {\n // We'll join the prior timed-out thread's JoinHandle after dropping the lock.\n // This isn't done when shutting down, because the thread calling shutdown will\n // handle joining everything.\n let my_handle = shared.worker_threads.remove(&worker_thread_id);\n join_on_thread = std::mem::replace(&mut shared.last_exiting_thread, my_handle);\n\n break 'main;\n }\n\n // Spurious wakeup detected, go back to sleep.\n }\n\n if shared.shutdown {\n // Drain the queue\n while let Some(task) = shared.queue.pop_front() {\n self.metrics.dec_queue_depth();\n drop(shared);\n\n task.shutdown_or_run_if_mandatory();\n\n shared = self.shared.lock();\n }\n\n break;\n }\n }\n\n // Thread exit\n self.metrics.dec_num_threads();\n\n // Is this thread currently counted in `num_idle_threads`?\n if is_counted_idle {\n // `num_idle_threads` should now be tracked exactly, panic\n // with a descriptive message if it is not the\n // case.\n let prev_idle = self.metrics.dec_num_idle_threads();\n assert_ne!(\n prev_idle, 0,\n \"`num_idle_threads` underflowed on thread exit\"\n );\n }\n\n if shared.shutdown && self.metrics.num_threads() == 0 {\n self.condvar.notify_one();\n }\n\n drop(shared);\n\n if let Some(f) = &self.before_stop {\n f();\n }\n\n if let Some(handle) = join_on_thread {\n let _ = handle.join();\n }\n }\n}\n\nimpl fmt::Debug for Spawner {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"blocking::Spawner\").finish()\n }\n}\n" }, { "path": "tokio/src/runtime/blocking/schedule.rs", "content": "#[cfg(feature = \"test-util\")]\nuse crate::runtime::scheduler;\nuse crate::runtime::task::{self, Task, TaskHarnessScheduleHooks};\nuse crate::runtime::Handle;\n\n/// `task::Schedule` implementation that does nothing (except some bookkeeping\n/// in test-util builds). This is unique to the blocking scheduler as tasks\n/// scheduled are not really futures but blocking operations.\n///\n/// We avoid storing the task by forgetting it in `bind` and re-materializing it\n/// in `release`.\npub(crate) struct BlockingSchedule {\n #[cfg(feature = \"test-util\")]\n handle: Handle,\n hooks: TaskHarnessScheduleHooks,\n}\n\nimpl BlockingSchedule {\n #[cfg_attr(not(feature = \"test-util\"), allow(unused_variables))]\n pub(crate) fn new(handle: &Handle) -> Self {\n #[cfg(feature = \"test-util\")]\n {\n match &handle.inner {\n scheduler::Handle::CurrentThread(handle) => {\n handle.driver.clock.inhibit_auto_advance();\n }\n #[cfg(feature = \"rt-multi-thread\")]\n scheduler::Handle::MultiThread(_) => {}\n }\n }\n BlockingSchedule {\n #[cfg(feature = \"test-util\")]\n handle: handle.clone(),\n hooks: TaskHarnessScheduleHooks {\n task_terminate_callback: handle.inner.hooks().task_terminate_callback.clone(),\n },\n }\n }\n}\n\nimpl task::Schedule for BlockingSchedule {\n fn release(&self, _task: &Task) -> Option> {\n #[cfg(feature = \"test-util\")]\n {\n match &self.handle.inner {\n scheduler::Handle::CurrentThread(handle) => {\n handle.driver.clock.allow_auto_advance();\n handle.driver.unpark();\n }\n #[cfg(feature = \"rt-multi-thread\")]\n scheduler::Handle::MultiThread(_) => {}\n }\n }\n None\n }\n\n fn schedule(&self, _task: task::Notified) {\n unreachable!();\n }\n\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: self.hooks.task_terminate_callback.clone(),\n }\n }\n}\n" }, { "path": "tokio/src/runtime/blocking/shutdown.rs", "content": "//! A shutdown channel.\n//!\n//! Each worker holds the `Sender` half. When all the `Sender` halves are\n//! dropped, the `Receiver` receives a notification.\n\nuse crate::loom::sync::Arc;\nuse crate::sync::oneshot;\n\nuse std::time::Duration;\n\n#[derive(Debug, Clone)]\npub(super) struct Sender {\n _tx: Arc>,\n}\n\n#[derive(Debug)]\npub(super) struct Receiver {\n rx: oneshot::Receiver<()>,\n}\n\npub(super) fn channel() -> (Sender, Receiver) {\n let (tx, rx) = oneshot::channel();\n let tx = Sender { _tx: Arc::new(tx) };\n let rx = Receiver { rx };\n\n (tx, rx)\n}\n\nimpl Receiver {\n /// Blocks the current thread until all `Sender` handles drop.\n ///\n /// If `timeout` is `Some`, the thread is blocked for **at most** `timeout`\n /// duration. If `timeout` is `None`, then the thread is blocked until the\n /// shutdown signal is received.\n ///\n /// If the timeout has elapsed, it returns `false`, otherwise it returns `true`.\n pub(crate) fn wait(&mut self, timeout: Option) -> bool {\n use crate::runtime::context::try_enter_blocking_region;\n\n if timeout == Some(Duration::from_nanos(0)) {\n return false;\n }\n\n let mut e = match try_enter_blocking_region() {\n Some(enter) => enter,\n _ => {\n if std::thread::panicking() {\n // Don't panic in a panic\n return false;\n } else {\n panic!(\n \"Cannot drop a runtime in a context where blocking is not allowed. \\\n This happens when a runtime is dropped from within an asynchronous context.\"\n );\n }\n }\n };\n\n // The oneshot completes with an Err\n //\n // If blocking fails to wait, this indicates a problem parking the\n // current thread (usually, shutting down a runtime stored in a\n // thread-local).\n if let Some(timeout) = timeout {\n e.block_on_timeout(&mut self.rx, timeout).is_ok()\n } else {\n let _ = e.block_on(&mut self.rx);\n true\n }\n }\n}\n" }, { "path": "tokio/src/runtime/blocking/task.rs", "content": "use std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n/// Converts a function to a future that completes on poll.\npub(crate) struct BlockingTask {\n func: Option,\n}\n\nimpl BlockingTask {\n /// Initializes a new blocking task from the given function.\n pub(crate) fn new(func: T) -> BlockingTask {\n BlockingTask { func: Some(func) }\n }\n}\n\n// The closure `F` is never pinned\nimpl Unpin for BlockingTask {}\n\nimpl Future for BlockingTask\nwhere\n T: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n{\n type Output = R;\n\n fn poll(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n let me = &mut *self;\n let func = me\n .func\n .take()\n .expect(\"[internal exception] blocking task ran twice.\");\n\n // This is a little subtle:\n // For convenience, we'd like _every_ call tokio ever makes to Task::poll() to be budgeted\n // using coop. However, the way things are currently modeled, even running a blocking task\n // currently goes through Task::poll(), and so is subject to budgeting. That isn't really\n // what we want; a blocking task may itself want to run tasks (it might be a Worker!), so\n // we want it to start without any budgeting.\n crate::task::coop::stop();\n\n Poll::Ready(func())\n }\n}\n" }, { "path": "tokio/src/runtime/builder.rs", "content": "#![cfg_attr(loom, allow(unused_imports))]\n\nuse crate::runtime::handle::Handle;\nuse crate::runtime::{\n blocking, driver, Callback, HistogramBuilder, Runtime, TaskCallback, TimerFlavor,\n};\n#[cfg(tokio_unstable)]\nuse crate::runtime::{metrics::HistogramConfiguration, LocalOptions, LocalRuntime, TaskMeta};\nuse crate::util::rand::{RngSeed, RngSeedGenerator};\n\nuse crate::runtime::blocking::BlockingPool;\nuse crate::runtime::scheduler::CurrentThread;\nuse std::fmt;\nuse std::io;\nuse std::thread::ThreadId;\nuse std::time::Duration;\n\n/// Builds Tokio Runtime with custom configuration values.\n///\n/// Methods can be chained in order to set the configuration values. The\n/// Runtime is constructed by calling [`build`].\n///\n/// New instances of `Builder` are obtained via [`Builder::new_multi_thread`]\n/// or [`Builder::new_current_thread`].\n///\n/// See function level documentation for details on the various configuration\n/// settings.\n///\n/// [`build`]: method@Self::build\n/// [`Builder::new_multi_thread`]: method@Self::new_multi_thread\n/// [`Builder::new_current_thread`]: method@Self::new_current_thread\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use tokio::runtime::Builder;\n///\n/// fn main() {\n/// // build runtime\n/// let runtime = Builder::new_multi_thread()\n/// .worker_threads(4)\n/// .thread_name(\"my-custom-name\")\n/// .thread_stack_size(3 * 1024 * 1024)\n/// .build()\n/// .unwrap();\n///\n/// // use runtime ...\n/// }\n/// # }\n/// ```\npub struct Builder {\n /// Runtime type\n kind: Kind,\n\n /// Whether or not to enable the I/O driver\n enable_io: bool,\n nevents: usize,\n\n /// Whether or not to enable the time driver\n enable_time: bool,\n\n /// Whether or not the clock should start paused.\n start_paused: bool,\n\n /// The number of worker threads, used by Runtime.\n ///\n /// Only used when not using the current-thread executor.\n worker_threads: Option,\n\n /// Cap on thread usage.\n max_blocking_threads: usize,\n\n /// Name fn used for threads spawned by the runtime.\n pub(super) thread_name: ThreadNameFn,\n\n /// Stack size used for threads spawned by the runtime.\n pub(super) thread_stack_size: Option,\n\n /// Callback to run after each thread starts.\n pub(super) after_start: Option,\n\n /// To run before each worker thread stops\n pub(super) before_stop: Option,\n\n /// To run before each worker thread is parked.\n pub(super) before_park: Option,\n\n /// To run after each thread is unparked.\n pub(super) after_unpark: Option,\n\n /// To run before each task is spawned.\n pub(super) before_spawn: Option,\n\n /// To run before each poll\n #[cfg(tokio_unstable)]\n pub(super) before_poll: Option,\n\n /// To run after each poll\n #[cfg(tokio_unstable)]\n pub(super) after_poll: Option,\n\n /// To run after each task is terminated.\n pub(super) after_termination: Option,\n\n /// Customizable keep alive timeout for `BlockingPool`\n pub(super) keep_alive: Option,\n\n /// How many ticks before pulling a task from the global/remote queue?\n ///\n /// When `None`, the value is unspecified and behavior details are left to\n /// the scheduler. Each scheduler flavor could choose to either pick its own\n /// default value or use some other strategy to decide when to poll from the\n /// global queue. For example, the multi-threaded scheduler uses a\n /// self-tuning strategy based on mean task poll times.\n pub(super) global_queue_interval: Option,\n\n /// How many ticks before yielding to the driver for timer and I/O events?\n pub(super) event_interval: u32,\n\n /// When true, the multi-threade scheduler LIFO slot should not be used.\n ///\n /// This option should only be exposed as unstable.\n pub(super) disable_lifo_slot: bool,\n\n /// Specify a random number generator seed to provide deterministic results\n pub(super) seed_generator: RngSeedGenerator,\n\n /// When true, enables task poll count histogram instrumentation.\n pub(super) metrics_poll_count_histogram_enable: bool,\n\n /// Configures the task poll count histogram\n pub(super) metrics_poll_count_histogram: HistogramBuilder,\n\n #[cfg(tokio_unstable)]\n pub(super) unhandled_panic: UnhandledPanic,\n\n timer_flavor: TimerFlavor,\n}\n\ncfg_unstable! {\n /// How the runtime should respond to unhandled panics.\n ///\n /// Instances of `UnhandledPanic` are passed to `Builder::unhandled_panic`\n /// to configure the runtime behavior when a spawned task panics.\n ///\n /// See [`Builder::unhandled_panic`] for more details.\n #[derive(Debug, Clone)]\n #[non_exhaustive]\n pub enum UnhandledPanic {\n /// The runtime should ignore panics on spawned tasks.\n ///\n /// The panic is forwarded to the task's [`JoinHandle`] and all spawned\n /// tasks continue running normally.\n ///\n /// This is the default behavior.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::{self, UnhandledPanic};\n ///\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_current_thread()\n /// .unhandled_panic(UnhandledPanic::Ignore)\n /// .build()\n /// .unwrap();\n ///\n /// let task1 = rt.spawn(async { panic!(\"boom\"); });\n /// let task2 = rt.spawn(async {\n /// // This task completes normally\n /// \"done\"\n /// });\n ///\n /// rt.block_on(async {\n /// // The panic on the first task is forwarded to the `JoinHandle`\n /// assert!(task1.await.is_err());\n ///\n /// // The second task completes normally\n /// assert!(task2.await.is_ok());\n /// })\n /// # }\n /// # }\n /// ```\n ///\n /// [`JoinHandle`]: struct@crate::task::JoinHandle\n Ignore,\n\n /// The runtime should immediately shutdown if a spawned task panics.\n ///\n /// The runtime will immediately shutdown even if the panicked task's\n /// [`JoinHandle`] is still available. All further spawned tasks will be\n /// immediately dropped and call to [`Runtime::block_on`] will panic.\n ///\n /// # Examples\n ///\n /// ```should_panic\n /// use tokio::runtime::{self, UnhandledPanic};\n ///\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_current_thread()\n /// .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n /// .build()\n /// .unwrap();\n ///\n /// rt.spawn(async { panic!(\"boom\"); });\n /// rt.spawn(async {\n /// // This task never completes.\n /// });\n ///\n /// rt.block_on(async {\n /// // Do some work\n /// # loop { tokio::task::yield_now().await; }\n /// })\n /// # }\n /// ```\n ///\n /// [`JoinHandle`]: struct@crate::task::JoinHandle\n ShutdownRuntime,\n }\n}\n\npub(crate) type ThreadNameFn = std::sync::Arc String + Send + Sync + 'static>;\n\n#[derive(Clone, Copy)]\npub(crate) enum Kind {\n CurrentThread,\n #[cfg(feature = \"rt-multi-thread\")]\n MultiThread,\n}\n\nimpl Builder {\n /// Returns a new builder with the current thread scheduler selected.\n ///\n /// Configuration methods can be chained on the return value.\n ///\n /// To spawn non-`Send` tasks on the resulting runtime, combine it with a\n /// [`LocalSet`], or call [`build_local`] to create a [`LocalRuntime`] (unstable).\n ///\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`LocalRuntime`]: crate::runtime::LocalRuntime\n /// [`build_local`]: crate::runtime::Builder::build_local\n pub fn new_current_thread() -> Builder {\n #[cfg(loom)]\n const EVENT_INTERVAL: u32 = 4;\n // The number `61` is fairly arbitrary. I believe this value was copied from golang.\n #[cfg(not(loom))]\n const EVENT_INTERVAL: u32 = 61;\n\n Builder::new(Kind::CurrentThread, EVENT_INTERVAL)\n }\n\n /// Returns a new builder with the multi thread scheduler selected.\n ///\n /// Configuration methods can be chained on the return value.\n #[cfg(feature = \"rt-multi-thread\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt-multi-thread\")))]\n pub fn new_multi_thread() -> Builder {\n // The number `61` is fairly arbitrary. I believe this value was copied from golang.\n Builder::new(Kind::MultiThread, 61)\n }\n\n /// Returns a new runtime builder initialized with default configuration\n /// values.\n ///\n /// Configuration methods can be chained on the return value.\n pub(crate) fn new(kind: Kind, event_interval: u32) -> Builder {\n Builder {\n kind,\n\n // I/O defaults to \"off\"\n enable_io: false,\n nevents: 1024,\n\n // Time defaults to \"off\"\n enable_time: false,\n\n // The clock starts not-paused\n start_paused: false,\n\n // Read from environment variable first in multi-threaded mode.\n // Default to lazy auto-detection (one thread per CPU core)\n worker_threads: None,\n\n max_blocking_threads: 512,\n\n // Default thread name\n thread_name: std::sync::Arc::new(|| \"tokio-rt-worker\".into()),\n\n // Do not set a stack size by default\n thread_stack_size: None,\n\n // No worker thread callbacks\n after_start: None,\n before_stop: None,\n before_park: None,\n after_unpark: None,\n\n before_spawn: None,\n after_termination: None,\n\n #[cfg(tokio_unstable)]\n before_poll: None,\n #[cfg(tokio_unstable)]\n after_poll: None,\n\n keep_alive: None,\n\n // Defaults for these values depend on the scheduler kind, so we get them\n // as parameters.\n global_queue_interval: None,\n event_interval,\n\n seed_generator: RngSeedGenerator::new(RngSeed::new()),\n\n #[cfg(tokio_unstable)]\n unhandled_panic: UnhandledPanic::Ignore,\n\n metrics_poll_count_histogram_enable: false,\n\n metrics_poll_count_histogram: HistogramBuilder::default(),\n\n disable_lifo_slot: false,\n\n timer_flavor: TimerFlavor::Traditional,\n }\n }\n\n /// Enables both I/O and time drivers.\n ///\n /// Doing this is a shorthand for calling `enable_io` and `enable_time`\n /// individually. If additional components are added to Tokio in the future,\n /// `enable_all` will include these future components.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n pub fn enable_all(&mut self) -> &mut Self {\n #[cfg(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\")\n ))]\n self.enable_io();\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n self.enable_io_uring();\n\n #[cfg(feature = \"time\")]\n self.enable_time();\n\n self\n }\n\n /// Enables the alternative timer implementation, which is disabled by default.\n ///\n /// The alternative timer implementation is an unstable feature that may\n /// provide better performance on multi-threaded runtimes with a large number\n /// of worker threads.\n ///\n /// This option only applies to multi-threaded runtimes. Attempting to use\n /// this option with any other runtime type will have no effect.\n ///\n /// [Click here to share your experience with the alternative timer](https://github.com/tokio-rs/tokio/issues/7745)\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_alt_timer()\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n #[cfg(all(tokio_unstable, feature = \"time\", feature = \"rt-multi-thread\"))]\n #[cfg_attr(\n docsrs,\n doc(cfg(all(tokio_unstable, feature = \"time\", feature = \"rt-multi-thread\")))\n )]\n pub fn enable_alt_timer(&mut self) -> &mut Self {\n self.enable_time();\n self.timer_flavor = TimerFlavor::Alternative;\n self\n }\n\n /// Sets the number of worker threads the `Runtime` will use.\n ///\n /// This can be any number above 0 though it is advised to keep this value\n /// on the smaller side.\n ///\n /// This will override the value read from environment variable `TOKIO_WORKER_THREADS`.\n ///\n /// # Default\n ///\n /// The default value is the number of cores available to the system.\n ///\n /// When using the `current_thread` runtime this method has no effect.\n ///\n /// # Examples\n ///\n /// ## Multi threaded runtime with 4 threads\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// // This will spawn a work-stealing runtime with 4 worker threads.\n /// let rt = runtime::Builder::new_multi_thread()\n /// .worker_threads(4)\n /// .build()\n /// .unwrap();\n ///\n /// rt.spawn(async move {});\n /// # }\n /// ```\n ///\n /// ## Current thread runtime (will only run on the current thread via `Runtime::block_on`)\n ///\n /// ```\n /// use tokio::runtime;\n ///\n /// // Create a runtime that _must_ be driven from a call\n /// // to `Runtime::block_on`.\n /// let rt = runtime::Builder::new_current_thread()\n /// .build()\n /// .unwrap();\n ///\n /// // This will run the runtime and future on the current thread\n /// rt.block_on(async move {});\n /// ```\n ///\n /// # Panics\n ///\n /// This will panic if `val` is not larger than `0`.\n #[track_caller]\n pub fn worker_threads(&mut self, val: usize) -> &mut Self {\n assert!(val > 0, \"Worker threads cannot be set to 0\");\n self.worker_threads = Some(val);\n self\n }\n\n /// Specifies the limit for additional threads spawned by the Runtime.\n ///\n /// These threads are used for blocking operations like tasks spawned\n /// through [`spawn_blocking`], this includes but is not limited to:\n /// - [`fs`] operations\n /// - dns resolution through [`ToSocketAddrs`]\n /// - writing to [`Stdout`] or [`Stderr`]\n /// - reading from [`Stdin`]\n ///\n /// Unlike the [`worker_threads`], they are not always active and will exit\n /// if left idle for too long. You can change this timeout duration with [`thread_keep_alive`].\n ///\n /// It's recommended to not set this limit too low in order to avoid hanging on operations\n /// requiring [`spawn_blocking`].\n ///\n /// The default value is 512.\n ///\n /// # Queue Behavior\n ///\n /// When a blocking task is submitted, it will be inserted into a queue. If available, one of\n /// the idle threads will be notified to run the task. Otherwise, if the threshold set by this\n /// method has not been reached, a new thread will be spawned. If no idle thread is available\n /// and no more threads are allowed to be spawned, the task will remain in the queue until one\n /// of the busy threads pick it up. Note that since the queue does not apply any backpressure,\n /// it could potentially grow unbounded.\n ///\n /// # Panics\n ///\n /// This will panic if `val` is not larger than `0`.\n ///\n /// # Upgrading from 0.x\n ///\n /// In old versions `max_threads` limited both blocking and worker threads, but the\n /// current `max_blocking_threads` does not include async worker threads in the count.\n ///\n /// [`spawn_blocking`]: fn@crate::task::spawn_blocking\n /// [`fs`]: mod@crate::fs\n /// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs\n /// [`Stdout`]: struct@crate::io::Stdout\n /// [`Stdin`]: struct@crate::io::Stdin\n /// [`Stderr`]: struct@crate::io::Stderr\n /// [`worker_threads`]: Self::worker_threads\n /// [`thread_keep_alive`]: Self::thread_keep_alive\n #[track_caller]\n #[cfg_attr(docsrs, doc(alias = \"max_threads\"))]\n pub fn max_blocking_threads(&mut self, val: usize) -> &mut Self {\n assert!(val > 0, \"Max blocking threads cannot be set to 0\");\n self.max_blocking_threads = val;\n self\n }\n\n /// Sets name of threads spawned by the `Runtime`'s thread pool.\n ///\n /// The default name is \"tokio-rt-worker\".\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n ///\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .thread_name(\"my-pool\")\n /// .build();\n /// # }\n /// # }\n /// ```\n pub fn thread_name(&mut self, val: impl Into) -> &mut Self {\n let val = val.into();\n self.thread_name = std::sync::Arc::new(move || val.clone());\n self\n }\n\n /// Sets a function used to generate the name of threads spawned by the `Runtime`'s thread pool.\n ///\n /// The default name fn is `|| \"tokio-rt-worker\".into()`.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # use std::sync::atomic::{AtomicUsize, Ordering};\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .thread_name_fn(|| {\n /// static ATOMIC_ID: AtomicUsize = AtomicUsize::new(0);\n /// let id = ATOMIC_ID.fetch_add(1, Ordering::SeqCst);\n /// format!(\"my-pool-{}\", id)\n /// })\n /// .build();\n /// # }\n /// # }\n /// ```\n pub fn thread_name_fn(&mut self, f: F) -> &mut Self\n where\n F: Fn() -> String + Send + Sync + 'static,\n {\n self.thread_name = std::sync::Arc::new(f);\n self\n }\n\n /// Sets the stack size (in bytes) for worker threads.\n ///\n /// The actual stack size may be greater than this value if the platform\n /// specifies minimal stack size.\n ///\n /// The default stack size for spawned threads is 2 MiB, though this\n /// particular stack size is subject to change in the future.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n ///\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .thread_stack_size(32 * 1024)\n /// .build();\n /// # }\n /// # }\n /// ```\n pub fn thread_stack_size(&mut self, val: usize) -> &mut Self {\n self.thread_stack_size = Some(val);\n self\n }\n\n /// Executes function `f` after each thread is started but before it starts\n /// doing work.\n ///\n /// This is intended for bookkeeping and monitoring use cases.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let runtime = runtime::Builder::new_multi_thread()\n /// .on_thread_start(|| {\n /// println!(\"thread started\");\n /// })\n /// .build();\n /// # }\n /// # }\n /// ```\n #[cfg(not(loom))]\n pub fn on_thread_start(&mut self, f: F) -> &mut Self\n where\n F: Fn() + Send + Sync + 'static,\n {\n self.after_start = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` before each thread stops.\n ///\n /// This is intended for bookkeeping and monitoring use cases.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// {\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let runtime = runtime::Builder::new_multi_thread()\n /// .on_thread_stop(|| {\n /// println!(\"thread stopping\");\n /// })\n /// .build();\n /// # }\n /// # }\n /// ```\n #[cfg(not(loom))]\n pub fn on_thread_stop(&mut self, f: F) -> &mut Self\n where\n F: Fn() + Send + Sync + 'static,\n {\n self.before_stop = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just before a thread is parked (goes idle).\n /// `f` is called within the Tokio context, so functions like [`tokio::spawn`](crate::spawn)\n /// can be called, and may result in this thread being unparked immediately.\n ///\n /// This can be used to start work only when the executor is idle, or for bookkeeping\n /// and monitoring purposes.\n ///\n /// Note: There can only be one park callback for a runtime; calling this function\n /// more than once replaces the last callback defined, rather than adding to it.\n ///\n /// # Examples\n ///\n /// ## Multithreaded executor\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use std::sync::Arc;\n /// # use std::sync::atomic::{AtomicBool, Ordering};\n /// # use tokio::runtime;\n /// # use tokio::sync::Barrier;\n /// # pub fn main() {\n /// let once = AtomicBool::new(true);\n /// let barrier = Arc::new(Barrier::new(2));\n ///\n /// let runtime = runtime::Builder::new_multi_thread()\n /// .worker_threads(1)\n /// .on_thread_park({\n /// let barrier = barrier.clone();\n /// move || {\n /// let barrier = barrier.clone();\n /// if once.swap(false, Ordering::Relaxed) {\n /// tokio::spawn(async move { barrier.wait().await; });\n /// }\n /// }\n /// })\n /// .build()\n /// .unwrap();\n ///\n /// runtime.block_on(async {\n /// barrier.wait().await;\n /// })\n /// # }\n /// # }\n /// ```\n /// ## Current thread executor\n /// ```\n /// # use std::sync::Arc;\n /// # use std::sync::atomic::{AtomicBool, Ordering};\n /// # use tokio::runtime;\n /// # use tokio::sync::Barrier;\n /// # pub fn main() {\n /// let once = AtomicBool::new(true);\n /// let barrier = Arc::new(Barrier::new(2));\n ///\n /// let runtime = runtime::Builder::new_current_thread()\n /// .on_thread_park({\n /// let barrier = barrier.clone();\n /// move || {\n /// let barrier = barrier.clone();\n /// if once.swap(false, Ordering::Relaxed) {\n /// tokio::spawn(async move { barrier.wait().await; });\n /// }\n /// }\n /// })\n /// .build()\n /// .unwrap();\n ///\n /// runtime.block_on(async {\n /// barrier.wait().await;\n /// })\n /// # }\n /// ```\n #[cfg(not(loom))]\n pub fn on_thread_park(&mut self, f: F) -> &mut Self\n where\n F: Fn() + Send + Sync + 'static,\n {\n self.before_park = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just after a thread unparks (starts executing tasks).\n ///\n /// This is intended for bookkeeping and monitoring use cases; note that work\n /// in this callback will increase latencies when the application has allowed one or\n /// more runtime threads to go idle.\n ///\n /// Note: There can only be one unpark callback for a runtime; calling this function\n /// more than once replaces the last callback defined, rather than adding to it.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let runtime = runtime::Builder::new_multi_thread()\n /// .on_thread_unpark(|| {\n /// println!(\"thread unparking\");\n /// })\n /// .build();\n ///\n /// runtime.unwrap().block_on(async {\n /// tokio::task::yield_now().await;\n /// println!(\"Hello from Tokio!\");\n /// })\n /// # }\n /// # }\n /// ```\n #[cfg(not(loom))]\n pub fn on_thread_unpark(&mut self, f: F) -> &mut Self\n where\n F: Fn() + Send + Sync + 'static,\n {\n self.after_unpark = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just before a task is spawned.\n ///\n /// `f` is called within the Tokio context, so functions like\n /// [`tokio::spawn`](crate::spawn) can be called, and may result in this callback being\n /// invoked immediately.\n ///\n /// This can be used for bookkeeping or monitoring purposes.\n ///\n /// Note: There can only be one spawn callback for a runtime; calling this function more\n /// than once replaces the last callback defined, rather than adding to it.\n ///\n /// This *does not* support [`LocalSet`](crate::task::LocalSet) at this time.\n ///\n /// **Note**: This is an [unstable API][unstable]. The public API of this type\n /// may break in 1.x releases. See [the documentation on unstable\n /// features][unstable] for details.\n ///\n /// [unstable]: crate#unstable-features\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let runtime = runtime::Builder::new_current_thread()\n /// .on_task_spawn(|_| {\n /// println!(\"spawning task\");\n /// })\n /// .build()\n /// .unwrap();\n ///\n /// runtime.block_on(async {\n /// tokio::task::spawn(std::future::ready(()));\n ///\n /// for _ in 0..64 {\n /// tokio::task::yield_now().await;\n /// }\n /// })\n /// # }\n /// ```\n #[cfg(all(not(loom), tokio_unstable))]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn on_task_spawn(&mut self, f: F) -> &mut Self\n where\n F: Fn(&TaskMeta<'_>) + Send + Sync + 'static,\n {\n self.before_spawn = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just before a task is polled\n ///\n /// `f` is called within the Tokio context, so functions like\n /// [`tokio::spawn`](crate::spawn) can be called, and may result in this callback being\n /// invoked immediately.\n ///\n /// **Note**: This is an [unstable API][unstable]. The public API of this type\n /// may break in 1.x releases. See [the documentation on unstable\n /// features][unstable] for details.\n ///\n /// [unstable]: crate#unstable-features\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use std::sync::{atomic::AtomicUsize, Arc};\n /// # use tokio::task::yield_now;\n /// # pub fn main() {\n /// let poll_start_counter = Arc::new(AtomicUsize::new(0));\n /// let poll_start = poll_start_counter.clone();\n /// let rt = tokio::runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .on_before_task_poll(move |meta| {\n /// println!(\"task {} is about to be polled\", meta.id())\n /// })\n /// .build()\n /// .unwrap();\n /// let task = rt.spawn(async {\n /// yield_now().await;\n /// });\n /// let _ = rt.block_on(task);\n ///\n /// # }\n /// # }\n /// ```\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn on_before_task_poll(&mut self, f: F) -> &mut Self\n where\n F: Fn(&TaskMeta<'_>) + Send + Sync + 'static,\n {\n self.before_poll = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just after a task is polled\n ///\n /// `f` is called within the Tokio context, so functions like\n /// [`tokio::spawn`](crate::spawn) can be called, and may result in this callback being\n /// invoked immediately.\n ///\n /// **Note**: This is an [unstable API][unstable]. The public API of this type\n /// may break in 1.x releases. See [the documentation on unstable\n /// features][unstable] for details.\n ///\n /// [unstable]: crate#unstable-features\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use std::sync::{atomic::AtomicUsize, Arc};\n /// # use tokio::task::yield_now;\n /// # pub fn main() {\n /// let poll_stop_counter = Arc::new(AtomicUsize::new(0));\n /// let poll_stop = poll_stop_counter.clone();\n /// let rt = tokio::runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .on_after_task_poll(move |meta| {\n /// println!(\"task {} completed polling\", meta.id());\n /// })\n /// .build()\n /// .unwrap();\n /// let task = rt.spawn(async {\n /// yield_now().await;\n /// });\n /// let _ = rt.block_on(task);\n ///\n /// # }\n /// # }\n /// ```\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn on_after_task_poll(&mut self, f: F) -> &mut Self\n where\n F: Fn(&TaskMeta<'_>) + Send + Sync + 'static,\n {\n self.after_poll = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Executes function `f` just after a task is terminated.\n ///\n /// `f` is called within the Tokio context, so functions like\n /// [`tokio::spawn`](crate::spawn) can be called.\n ///\n /// This can be used for bookkeeping or monitoring purposes.\n ///\n /// Note: There can only be one task termination callback for a runtime; calling this\n /// function more than once replaces the last callback defined, rather than adding to it.\n ///\n /// This *does not* support [`LocalSet`](crate::task::LocalSet) at this time.\n ///\n /// **Note**: This is an [unstable API][unstable]. The public API of this type\n /// may break in 1.x releases. See [the documentation on unstable\n /// features][unstable] for details.\n ///\n /// [unstable]: crate#unstable-features\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let runtime = runtime::Builder::new_current_thread()\n /// .on_task_terminate(|_| {\n /// println!(\"killing task\");\n /// })\n /// .build()\n /// .unwrap();\n ///\n /// runtime.block_on(async {\n /// tokio::task::spawn(std::future::ready(()));\n ///\n /// for _ in 0..64 {\n /// tokio::task::yield_now().await;\n /// }\n /// })\n /// # }\n /// ```\n #[cfg(all(not(loom), tokio_unstable))]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn on_task_terminate(&mut self, f: F) -> &mut Self\n where\n F: Fn(&TaskMeta<'_>) + Send + Sync + 'static,\n {\n self.after_termination = Some(std::sync::Arc::new(f));\n self\n }\n\n /// Creates the configured `Runtime`.\n ///\n /// The returned `Runtime` instance is ready to spawn tasks.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Builder;\n ///\n /// let rt = Builder::new_multi_thread().build().unwrap();\n ///\n /// rt.block_on(async {\n /// println!(\"Hello from the Tokio runtime\");\n /// });\n /// # }\n /// ```\n pub fn build(&mut self) -> io::Result {\n match &self.kind {\n Kind::CurrentThread => self.build_current_thread_runtime(),\n #[cfg(feature = \"rt-multi-thread\")]\n Kind::MultiThread => self.build_threaded_runtime(),\n }\n }\n\n /// Creates the configured [`LocalRuntime`].\n ///\n /// The returned [`LocalRuntime`] instance is ready to spawn tasks.\n ///\n /// # Panics\n ///\n /// This will panic if the runtime is configured with [`new_multi_thread()`].\n ///\n /// [`new_multi_thread()`]: Builder::new_multi_thread\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{Builder, LocalOptions};\n ///\n /// let rt = Builder::new_current_thread()\n /// .build_local(LocalOptions::default())\n /// .unwrap();\n ///\n /// rt.spawn_local(async {\n /// println!(\"Hello from the Tokio runtime\");\n /// });\n /// ```\n #[allow(unused_variables, unreachable_patterns)]\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn build_local(&mut self, options: LocalOptions) -> io::Result {\n match &self.kind {\n Kind::CurrentThread => self.build_current_thread_local_runtime(),\n #[cfg(feature = \"rt-multi-thread\")]\n Kind::MultiThread => panic!(\"multi_thread is not supported for LocalRuntime\"),\n }\n }\n\n fn get_cfg(&self) -> driver::Cfg {\n driver::Cfg {\n enable_pause_time: match self.kind {\n Kind::CurrentThread => true,\n #[cfg(feature = \"rt-multi-thread\")]\n Kind::MultiThread => false,\n },\n enable_io: self.enable_io,\n enable_time: self.enable_time,\n start_paused: self.start_paused,\n nevents: self.nevents,\n timer_flavor: self.timer_flavor,\n }\n }\n\n /// Sets a custom timeout for a thread in the blocking pool.\n ///\n /// By default, the timeout for a thread is set to 10 seconds. This can\n /// be overridden using `.thread_keep_alive()`.\n ///\n /// # Example\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # use std::time::Duration;\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .thread_keep_alive(Duration::from_millis(100))\n /// .build();\n /// # }\n /// # }\n /// ```\n pub fn thread_keep_alive(&mut self, duration: Duration) -> &mut Self {\n self.keep_alive = Some(duration);\n self\n }\n\n /// Sets the number of scheduler ticks after which the scheduler will poll the global\n /// task queue.\n ///\n /// A scheduler \"tick\" roughly corresponds to one `poll` invocation on a task.\n ///\n /// By default the global queue interval is 31 for the current-thread scheduler. Please see\n /// [the module documentation] for the default behavior of the multi-thread scheduler.\n ///\n /// Schedulers have a local queue of already-claimed tasks, and a global queue of incoming\n /// tasks. Setting the interval to a smaller value increases the fairness of the scheduler,\n /// at the cost of more synchronization overhead. That can be beneficial for prioritizing\n /// getting started on new work, especially if tasks frequently yield rather than complete\n /// or await on further I/O. Setting the interval to `1` will prioritize the global queue and\n /// tasks from the local queue will be executed only if the global queue is empty.\n /// Conversely, a higher value prioritizes existing work, and is a good choice when most\n /// tasks quickly complete polling.\n ///\n /// [the module documentation]: crate::runtime#multi-threaded-runtime-behavior-at-the-time-of-writing\n ///\n /// # Panics\n ///\n /// This function will panic if 0 is passed as an argument.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .global_queue_interval(31)\n /// .build();\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn global_queue_interval(&mut self, val: u32) -> &mut Self {\n assert!(val > 0, \"global_queue_interval must be greater than 0\");\n self.global_queue_interval = Some(val);\n self\n }\n\n /// Sets the number of scheduler ticks after which the scheduler will poll for\n /// external events (timers, I/O, and so on).\n ///\n /// A scheduler \"tick\" roughly corresponds to one `poll` invocation on a task.\n ///\n /// By default, the event interval is `61` for all scheduler types.\n ///\n /// Setting the event interval determines the effective \"priority\" of delivering\n /// these external events (which may wake up additional tasks), compared to\n /// executing tasks that are currently ready to run. A smaller value is useful\n /// when tasks frequently spend a long time in polling, or infrequently yield,\n /// which can result in overly long delays picking up I/O events. Conversely,\n /// picking up new events requires extra synchronization and syscall overhead,\n /// so if tasks generally complete their polling quickly, a higher event interval\n /// will minimize that overhead while still keeping the scheduler responsive to\n /// events.\n ///\n /// # Panics\n ///\n /// This function will panic if 0 is passed as an argument.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use tokio::runtime;\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_multi_thread()\n /// .event_interval(31)\n /// .build();\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn event_interval(&mut self, val: u32) -> &mut Self {\n assert!(val > 0, \"event_interval must be greater than 0\");\n self.event_interval = val;\n self\n }\n\n cfg_unstable! {\n /// Configure how the runtime responds to an unhandled panic on a\n /// spawned task.\n ///\n /// By default, an unhandled panic (i.e. a panic not caught by\n /// [`std::panic::catch_unwind`]) has no impact on the runtime's\n /// execution. The panic's error value is forwarded to the task's\n /// [`JoinHandle`] and all other spawned tasks continue running.\n ///\n /// The `unhandled_panic` option enables configuring this behavior.\n ///\n /// * `UnhandledPanic::Ignore` is the default behavior. Panics on\n /// spawned tasks have no impact on the runtime's execution.\n /// * `UnhandledPanic::ShutdownRuntime` will force the runtime to\n /// shutdown immediately when a spawned task panics even if that\n /// task's `JoinHandle` has not been dropped. All other spawned tasks\n /// will immediately terminate and further calls to\n /// [`Runtime::block_on`] will panic.\n ///\n /// # Panics\n /// This method panics if called with [`UnhandledPanic::ShutdownRuntime`]\n /// on a runtime other than the current thread runtime.\n ///\n /// # Unstable\n ///\n /// This option is currently unstable and its implementation is\n /// incomplete. The API may change or be removed in the future. See\n /// issue [tokio-rs/tokio#4516] for more details.\n ///\n /// # Examples\n ///\n /// The following demonstrates a runtime configured to shutdown on\n /// panic. The first spawned task panics and results in the runtime\n /// shutting down. The second spawned task never has a chance to\n /// execute. The call to `block_on` will panic due to the runtime being\n /// forcibly shutdown.\n ///\n /// ```should_panic\n /// use tokio::runtime::{self, UnhandledPanic};\n ///\n /// # pub fn main() {\n /// let rt = runtime::Builder::new_current_thread()\n /// .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n /// .build()\n /// .unwrap();\n ///\n /// rt.spawn(async { panic!(\"boom\"); });\n /// rt.spawn(async {\n /// // This task never completes.\n /// });\n ///\n /// rt.block_on(async {\n /// // Do some work\n /// # loop { tokio::task::yield_now().await; }\n /// })\n /// # }\n /// ```\n ///\n /// [`JoinHandle`]: struct@crate::task::JoinHandle\n /// [tokio-rs/tokio#4516]: https://github.com/tokio-rs/tokio/issues/4516\n pub fn unhandled_panic(&mut self, behavior: UnhandledPanic) -> &mut Self {\n if !matches!(self.kind, Kind::CurrentThread) && matches!(behavior, UnhandledPanic::ShutdownRuntime) {\n panic!(\"UnhandledPanic::ShutdownRuntime is only supported in current thread runtime\");\n }\n\n self.unhandled_panic = behavior;\n self\n }\n\n /// Disables the LIFO task scheduler heuristic.\n ///\n /// The multi-threaded scheduler includes a heuristic for optimizing\n /// message-passing patterns. This heuristic results in the **last**\n /// scheduled task being polled first.\n ///\n /// To implement this heuristic, each worker thread has a slot which\n /// holds the task that should be polled next. However, this slot cannot\n /// be stolen by other worker threads, which can result in lower total\n /// throughput when tasks tend to have longer poll times.\n ///\n /// This configuration option will disable this heuristic resulting in\n /// all scheduled tasks being pushed into the worker-local queue, which\n /// is stealable.\n ///\n /// Consider trying this option when the task \"scheduled\" time is high\n /// but the runtime is underutilized. Use [tokio-rs/tokio-metrics] to\n /// collect this data.\n ///\n /// # Unstable\n ///\n /// This configuration option is considered a workaround for the LIFO\n /// slot not being stealable. When the slot becomes stealable, we will\n /// revisit whether or not this option is necessary. See\n /// issue [tokio-rs/tokio#4941].\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .disable_lifo_slot()\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// [tokio-rs/tokio-metrics]: https://github.com/tokio-rs/tokio-metrics\n /// [tokio-rs/tokio#4941]: https://github.com/tokio-rs/tokio/issues/4941\n pub fn disable_lifo_slot(&mut self) -> &mut Self {\n self.disable_lifo_slot = true;\n self\n }\n\n /// Specifies the random number generation seed to use within all\n /// threads associated with the runtime being built.\n ///\n /// This option is intended to make certain parts of the runtime\n /// deterministic (e.g. the [`tokio::select!`] macro). In the case of\n /// [`tokio::select!`] it will ensure that the order that branches are\n /// polled is deterministic.\n ///\n /// In addition to the code specifying `rng_seed` and interacting with\n /// the runtime, the internals of Tokio and the Rust compiler may affect\n /// the sequences of random numbers. In order to ensure repeatable\n /// results, the version of Tokio, the versions of all other\n /// dependencies that interact with Tokio, and the Rust compiler version\n /// should also all remain constant.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::runtime::{self, RngSeed};\n /// # pub fn main() {\n /// let seed = RngSeed::from_bytes(b\"place your seed here\");\n /// let rt = runtime::Builder::new_current_thread()\n /// .rng_seed(seed)\n /// .build();\n /// # }\n /// ```\n ///\n /// [`tokio::select!`]: crate::select\n pub fn rng_seed(&mut self, seed: RngSeed) -> &mut Self {\n self.seed_generator = RngSeedGenerator::new(seed);\n self\n }\n }\n\n cfg_unstable_metrics! {\n /// Enables tracking the distribution of task poll times.\n ///\n /// Task poll times are not instrumented by default as doing so requires\n /// calling [`Instant::now()`] twice per task poll, which could add\n /// measurable overhead. Use the [`Handle::metrics()`] to access the\n /// metrics data.\n ///\n /// The histogram uses fixed bucket sizes. In other words, the histogram\n /// buckets are not dynamic based on input values. Use the\n /// `metrics_poll_time_histogram` builder methods to configure the\n /// histogram details.\n ///\n /// By default, a linear histogram with 10 buckets each 100 microseconds wide will be used.\n /// This has an extremely low memory footprint, but may not provide enough granularity. For\n /// better granularity with low memory usage, use [`metrics_poll_time_histogram_configuration()`]\n /// to select [`LogHistogram`] instead.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .build()\n /// .unwrap();\n /// # // Test default values here\n /// # fn us(n: u64) -> std::time::Duration { std::time::Duration::from_micros(n) }\n /// # let m = rt.handle().metrics();\n /// # assert_eq!(m.poll_time_histogram_num_buckets(), 10);\n /// # assert_eq!(m.poll_time_histogram_bucket_range(0), us(0)..us(100));\n /// # assert_eq!(m.poll_time_histogram_bucket_range(1), us(100)..us(200));\n /// # }\n /// ```\n ///\n /// [`Handle::metrics()`]: crate::runtime::Handle::metrics\n /// [`Instant::now()`]: std::time::Instant::now\n /// [`LogHistogram`]: crate::runtime::LogHistogram\n /// [`metrics_poll_time_histogram_configuration()`]: Builder::metrics_poll_time_histogram_configuration\n pub fn enable_metrics_poll_time_histogram(&mut self) -> &mut Self {\n self.metrics_poll_count_histogram_enable = true;\n self\n }\n\n /// Deprecated. Use [`enable_metrics_poll_time_histogram()`] instead.\n ///\n /// [`enable_metrics_poll_time_histogram()`]: Builder::enable_metrics_poll_time_histogram\n #[deprecated(note = \"`poll_count_histogram` related methods have been renamed `poll_time_histogram` to better reflect their functionality.\")]\n #[doc(hidden)]\n pub fn enable_metrics_poll_count_histogram(&mut self) -> &mut Self {\n self.enable_metrics_poll_time_histogram()\n }\n\n /// Sets the histogram scale for tracking the distribution of task poll\n /// times.\n ///\n /// Tracking the distribution of task poll times can be done using a\n /// linear or log scale. When using linear scale, each histogram bucket\n /// will represent the same range of poll times. When using log scale,\n /// each histogram bucket will cover a range twice as big as the\n /// previous bucket.\n ///\n /// **Default:** linear scale.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::{self, HistogramScale};\n ///\n /// # #[allow(deprecated)]\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_count_histogram_scale(HistogramScale::Log)\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n #[deprecated(note = \"use `metrics_poll_time_histogram_configuration`\")]\n pub fn metrics_poll_count_histogram_scale(&mut self, histogram_scale: crate::runtime::HistogramScale) -> &mut Self {\n self.metrics_poll_count_histogram.legacy_mut(|b|b.scale = histogram_scale);\n self\n }\n\n /// Configure the histogram for tracking poll times\n ///\n /// By default, a linear histogram with 10 buckets each 100 microseconds wide will be used.\n /// This has an extremely low memory footprint, but may not provide enough granularity. For\n /// better granularity with low memory usage, use [`LogHistogram`] instead.\n ///\n /// # Examples\n /// Configure a [`LogHistogram`] with [default configuration]:\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n /// use tokio::runtime::{HistogramConfiguration, LogHistogram};\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_time_histogram_configuration(\n /// HistogramConfiguration::log(LogHistogram::default())\n /// )\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// Configure a linear histogram with 100 buckets, each 10μs wide\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n /// use std::time::Duration;\n /// use tokio::runtime::HistogramConfiguration;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_time_histogram_configuration(\n /// HistogramConfiguration::linear(Duration::from_micros(10), 100)\n /// )\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// Configure a [`LogHistogram`] with the following settings:\n /// - Measure times from 100ns to 120s\n /// - Max error of 0.1\n /// - No more than 1024 buckets\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::time::Duration;\n /// use tokio::runtime;\n /// use tokio::runtime::{HistogramConfiguration, LogHistogram};\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_time_histogram_configuration(\n /// HistogramConfiguration::log(LogHistogram::builder()\n /// .max_value(Duration::from_secs(120))\n /// .min_value(Duration::from_nanos(100))\n /// .max_error(0.1)\n /// .max_buckets(1024)\n /// .expect(\"configuration uses 488 buckets\")\n /// )\n /// )\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n ///\n /// When migrating from the legacy histogram ([`HistogramScale::Log`]) and wanting\n /// to match the previous behavior, use `precision_exact(0)`. This creates a histogram\n /// where each bucket is twice the size of the previous bucket.\n /// ```rust\n /// use std::time::Duration;\n /// use tokio::runtime::{HistogramConfiguration, LogHistogram};\n /// let rt = tokio::runtime::Builder::new_current_thread()\n /// .enable_all()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_time_histogram_configuration(HistogramConfiguration::log(\n /// LogHistogram::builder()\n /// .min_value(Duration::from_micros(20))\n /// .max_value(Duration::from_millis(4))\n /// // Set `precision_exact` to `0` to match `HistogramScale::Log`\n /// .precision_exact(0)\n /// .max_buckets(10)\n /// .unwrap(),\n /// ))\n /// .build()\n /// .unwrap();\n /// ```\n ///\n /// [`LogHistogram`]: crate::runtime::LogHistogram\n /// [default configuration]: crate::runtime::LogHistogramBuilder\n /// [`HistogramScale::Log`]: crate::runtime::HistogramScale::Log\n pub fn metrics_poll_time_histogram_configuration(&mut self, configuration: HistogramConfiguration) -> &mut Self {\n self.metrics_poll_count_histogram.histogram_type = configuration.inner;\n self\n }\n\n /// Sets the histogram resolution for tracking the distribution of task\n /// poll times.\n ///\n /// The resolution is the histogram's first bucket's range. When using a\n /// linear histogram scale, each bucket will cover the same range. When\n /// using a log scale, each bucket will cover a range twice as big as\n /// the previous bucket. In the log case, the resolution represents the\n /// smallest bucket range.\n ///\n /// Note that, when using log scale, the resolution is rounded up to the\n /// nearest power of 2 in nanoseconds.\n ///\n /// **Default:** 100 microseconds.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n /// use std::time::Duration;\n ///\n /// # #[allow(deprecated)]\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_count_histogram_resolution(Duration::from_micros(100))\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n #[deprecated(note = \"use `metrics_poll_time_histogram_configuration`\")]\n pub fn metrics_poll_count_histogram_resolution(&mut self, resolution: Duration) -> &mut Self {\n assert!(resolution > Duration::from_secs(0));\n // Sanity check the argument and also make the cast below safe.\n assert!(resolution <= Duration::from_secs(1));\n\n let resolution = resolution.as_nanos() as u64;\n\n self.metrics_poll_count_histogram.legacy_mut(|b|b.resolution = resolution);\n self\n }\n\n /// Sets the number of buckets for the histogram tracking the\n /// distribution of task poll times.\n ///\n /// The last bucket tracks all greater values that fall out of other\n /// ranges. So, configuring the histogram using a linear scale,\n /// resolution of 50ms, and 10 buckets, the 10th bucket will track task\n /// polls that take more than 450ms to complete.\n ///\n /// **Default:** 10\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// # #[allow(deprecated)]\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .metrics_poll_count_histogram_buckets(15)\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n #[deprecated(note = \"use `metrics_poll_time_histogram_configuration`\")]\n pub fn metrics_poll_count_histogram_buckets(&mut self, buckets: usize) -> &mut Self {\n self.metrics_poll_count_histogram.legacy_mut(|b|b.num_buckets = buckets);\n self\n }\n }\n\n fn build_current_thread_runtime(&mut self) -> io::Result {\n use crate::runtime::runtime::Scheduler;\n\n let (scheduler, handle, blocking_pool) =\n self.build_current_thread_runtime_components(None)?;\n\n Ok(Runtime::from_parts(\n Scheduler::CurrentThread(scheduler),\n handle,\n blocking_pool,\n ))\n }\n\n #[cfg(tokio_unstable)]\n fn build_current_thread_local_runtime(&mut self) -> io::Result {\n use crate::runtime::local_runtime::LocalRuntimeScheduler;\n\n let tid = std::thread::current().id();\n\n let (scheduler, handle, blocking_pool) =\n self.build_current_thread_runtime_components(Some(tid))?;\n\n Ok(LocalRuntime::from_parts(\n LocalRuntimeScheduler::CurrentThread(scheduler),\n handle,\n blocking_pool,\n ))\n }\n\n fn build_current_thread_runtime_components(\n &mut self,\n local_tid: Option,\n ) -> io::Result<(CurrentThread, Handle, BlockingPool)> {\n use crate::runtime::scheduler;\n use crate::runtime::Config;\n\n let mut cfg = self.get_cfg();\n cfg.timer_flavor = TimerFlavor::Traditional;\n let (driver, driver_handle) = driver::Driver::new(cfg)?;\n\n // Blocking pool\n let blocking_pool = blocking::create_blocking_pool(self, self.max_blocking_threads);\n let blocking_spawner = blocking_pool.spawner().clone();\n\n // Generate a rng seed for this runtime.\n let seed_generator_1 = self.seed_generator.next_generator();\n let seed_generator_2 = self.seed_generator.next_generator();\n\n // And now put a single-threaded scheduler on top of the timer. When\n // there are no futures ready to do something, it'll let the timer or\n // the reactor to generate some new stimuli for the futures to continue\n // in their life.\n let (scheduler, handle) = CurrentThread::new(\n driver,\n driver_handle,\n blocking_spawner,\n seed_generator_2,\n Config {\n before_park: self.before_park.clone(),\n after_unpark: self.after_unpark.clone(),\n before_spawn: self.before_spawn.clone(),\n #[cfg(tokio_unstable)]\n before_poll: self.before_poll.clone(),\n #[cfg(tokio_unstable)]\n after_poll: self.after_poll.clone(),\n after_termination: self.after_termination.clone(),\n global_queue_interval: self.global_queue_interval,\n event_interval: self.event_interval,\n #[cfg(tokio_unstable)]\n unhandled_panic: self.unhandled_panic.clone(),\n disable_lifo_slot: self.disable_lifo_slot,\n seed_generator: seed_generator_1,\n metrics_poll_count_histogram: self.metrics_poll_count_histogram_builder(),\n },\n local_tid,\n );\n\n let handle = Handle {\n inner: scheduler::Handle::CurrentThread(handle),\n };\n\n Ok((scheduler, handle, blocking_pool))\n }\n\n fn metrics_poll_count_histogram_builder(&self) -> Option {\n if self.metrics_poll_count_histogram_enable {\n Some(self.metrics_poll_count_histogram.clone())\n } else {\n None\n }\n }\n}\n\ncfg_io_driver! {\n impl Builder {\n /// Enables the I/O driver.\n ///\n /// Doing this enables using net, process, signal, and some I/O types on\n /// the runtime.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_io()\n /// .build()\n /// .unwrap();\n /// ```\n pub fn enable_io(&mut self) -> &mut Self {\n self.enable_io = true;\n self\n }\n\n /// Enables the I/O driver and configures the max number of events to be\n /// processed per tick.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_current_thread()\n /// .enable_io()\n /// .max_io_events_per_tick(1024)\n /// .build()\n /// .unwrap();\n /// ```\n pub fn max_io_events_per_tick(&mut self, capacity: usize) -> &mut Self {\n self.nevents = capacity;\n self\n }\n }\n}\n\ncfg_time! {\n impl Builder {\n /// Enables the time driver.\n ///\n /// Doing this enables using `tokio::time` on the runtime.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_time()\n /// .build()\n /// .unwrap();\n /// # }\n /// ```\n pub fn enable_time(&mut self) -> &mut Self {\n self.enable_time = true;\n self\n }\n }\n}\n\ncfg_io_uring! {\n impl Builder {\n /// Enables the tokio's io_uring driver.\n ///\n /// Doing this enables using io_uring operations on the runtime.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_multi_thread()\n /// .enable_io_uring()\n /// .build()\n /// .unwrap();\n /// ```\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-uring\")))]\n pub fn enable_io_uring(&mut self) -> &mut Self {\n // Currently, the uring flag is equivalent to `enable_io`.\n self.enable_io = true;\n self\n }\n }\n}\n\ncfg_test_util! {\n impl Builder {\n /// Controls if the runtime's clock starts paused or advancing.\n ///\n /// Pausing time requires the current-thread runtime; construction of\n /// the runtime will panic otherwise.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime;\n ///\n /// let rt = runtime::Builder::new_current_thread()\n /// .enable_time()\n /// .start_paused(true)\n /// .build()\n /// .unwrap();\n /// ```\n pub fn start_paused(&mut self, start_paused: bool) -> &mut Self {\n self.start_paused = start_paused;\n self\n }\n }\n}\n\ncfg_rt_multi_thread! {\n impl Builder {\n fn build_threaded_runtime(&mut self) -> io::Result {\n use crate::loom::sys::num_cpus;\n use crate::runtime::{Config, runtime::Scheduler};\n use crate::runtime::scheduler::{self, MultiThread};\n\n let worker_threads = self.worker_threads.unwrap_or_else(num_cpus);\n\n let (driver, driver_handle) = driver::Driver::new(self.get_cfg())?;\n\n // Create the blocking pool\n let blocking_pool =\n blocking::create_blocking_pool(self, self.max_blocking_threads + worker_threads);\n let blocking_spawner = blocking_pool.spawner().clone();\n\n // Generate a rng seed for this runtime.\n let seed_generator_1 = self.seed_generator.next_generator();\n let seed_generator_2 = self.seed_generator.next_generator();\n\n let (scheduler, handle, launch) = MultiThread::new(\n worker_threads,\n driver,\n driver_handle,\n blocking_spawner,\n seed_generator_2,\n Config {\n before_park: self.before_park.clone(),\n after_unpark: self.after_unpark.clone(),\n before_spawn: self.before_spawn.clone(),\n #[cfg(tokio_unstable)]\n before_poll: self.before_poll.clone(),\n #[cfg(tokio_unstable)]\n after_poll: self.after_poll.clone(),\n after_termination: self.after_termination.clone(),\n global_queue_interval: self.global_queue_interval,\n event_interval: self.event_interval,\n #[cfg(tokio_unstable)]\n unhandled_panic: self.unhandled_panic.clone(),\n disable_lifo_slot: self.disable_lifo_slot,\n seed_generator: seed_generator_1,\n metrics_poll_count_histogram: self.metrics_poll_count_histogram_builder(),\n },\n self.timer_flavor,\n );\n\n let handle = Handle { inner: scheduler::Handle::MultiThread(handle) };\n\n // Spawn the thread pool workers\n let _enter = handle.enter();\n launch.launch();\n\n Ok(Runtime::from_parts(Scheduler::MultiThread(scheduler), handle, blocking_pool))\n }\n }\n}\n\nimpl fmt::Debug for Builder {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Builder\")\n .field(\"worker_threads\", &self.worker_threads)\n .field(\"max_blocking_threads\", &self.max_blocking_threads)\n .field(\n \"thread_name\",\n &\" String + Send + Sync + 'static>\",\n )\n .field(\"thread_stack_size\", &self.thread_stack_size)\n .field(\"after_start\", &self.after_start.as_ref().map(|_| \"...\"))\n .field(\"before_stop\", &self.before_stop.as_ref().map(|_| \"...\"))\n .field(\"before_park\", &self.before_park.as_ref().map(|_| \"...\"))\n .field(\"after_unpark\", &self.after_unpark.as_ref().map(|_| \"...\"))\n .finish()\n }\n}\n" }, { "path": "tokio/src/runtime/config.rs", "content": "#![cfg_attr(\n any(not(all(tokio_unstable, feature = \"full\")), target_family = \"wasm\"),\n allow(dead_code)\n)]\nuse crate::runtime::{Callback, TaskCallback};\nuse crate::util::RngSeedGenerator;\n\npub(crate) struct Config {\n /// How many ticks before pulling a task from the global/remote queue?\n pub(crate) global_queue_interval: Option,\n\n /// How many ticks before yielding to the driver for timer and I/O events?\n pub(crate) event_interval: u32,\n\n /// Callback for a worker parking itself\n pub(crate) before_park: Option,\n\n /// Callback for a worker unparking itself\n pub(crate) after_unpark: Option,\n\n /// To run before each task is spawned.\n pub(crate) before_spawn: Option,\n\n /// To run after each task is terminated.\n pub(crate) after_termination: Option,\n\n /// To run before each poll\n #[cfg(tokio_unstable)]\n pub(crate) before_poll: Option,\n\n /// To run after each poll\n #[cfg(tokio_unstable)]\n pub(crate) after_poll: Option,\n\n /// The multi-threaded scheduler includes a per-worker LIFO slot used to\n /// store the last scheduled task. This can improve certain usage patterns,\n /// especially message passing between tasks. However, this LIFO slot is not\n /// currently stealable.\n ///\n /// Eventually, the LIFO slot **will** become stealable, however as a\n /// stop-gap, this unstable option lets users disable the LIFO task.\n pub(crate) disable_lifo_slot: bool,\n\n /// Random number generator seed to configure runtimes to act in a\n /// deterministic way.\n pub(crate) seed_generator: RngSeedGenerator,\n\n /// How to build poll time histograms\n pub(crate) metrics_poll_count_histogram: Option,\n\n #[cfg(tokio_unstable)]\n /// How to respond to unhandled task panics.\n pub(crate) unhandled_panic: crate::runtime::UnhandledPanic,\n}\n" }, { "path": "tokio/src/runtime/context/blocking.rs", "content": "use super::{EnterRuntime, CONTEXT};\n\nuse crate::loom::thread::AccessError;\nuse crate::util::markers::NotSendOrSync;\n\nuse std::marker::PhantomData;\nuse std::time::Duration;\n\n/// Guard tracking that a caller has entered a blocking region.\n#[must_use]\npub(crate) struct BlockingRegionGuard {\n _p: PhantomData,\n}\n\npub(crate) struct DisallowBlockInPlaceGuard(bool);\n\npub(crate) fn try_enter_blocking_region() -> Option {\n CONTEXT\n .try_with(|c| {\n if c.runtime.get().is_entered() {\n None\n } else {\n Some(BlockingRegionGuard::new())\n }\n // If accessing the thread-local fails, the thread is terminating\n // and thread-locals are being destroyed. Because we don't know if\n // we are currently in a runtime or not, we default to being\n // permissive.\n })\n .unwrap_or_else(|_| Some(BlockingRegionGuard::new()))\n}\n\n/// Disallows blocking in the current runtime context until the guard is dropped.\npub(crate) fn disallow_block_in_place() -> DisallowBlockInPlaceGuard {\n let reset = CONTEXT.try_with(|c| {\n if let EnterRuntime::Entered {\n allow_block_in_place: true,\n } = c.runtime.get()\n {\n c.runtime.set(EnterRuntime::Entered {\n allow_block_in_place: false,\n });\n true\n } else {\n false\n }\n });\n\n DisallowBlockInPlaceGuard(reset.unwrap_or(false))\n}\n\nimpl BlockingRegionGuard {\n pub(super) fn new() -> BlockingRegionGuard {\n BlockingRegionGuard { _p: PhantomData }\n }\n\n /// Blocks the thread on the specified future, returning the value with\n /// which that future completes.\n pub(crate) fn block_on(&mut self, f: F) -> Result\n where\n F: std::future::Future,\n {\n use crate::runtime::park::CachedParkThread;\n\n let mut park = CachedParkThread::new();\n park.block_on(f)\n }\n\n /// Blocks the thread on the specified future for **at most** `timeout`\n ///\n /// If the future completes before `timeout`, the result is returned. If\n /// `timeout` elapses, then `Err` is returned.\n pub(crate) fn block_on_timeout(&mut self, f: F, timeout: Duration) -> Result\n where\n F: std::future::Future,\n {\n use crate::runtime::park::CachedParkThread;\n use std::task::Context;\n use std::task::Poll::Ready;\n use std::time::Instant;\n\n let mut park = CachedParkThread::new();\n let waker = park.waker().map_err(|_| ())?;\n let mut cx = Context::from_waker(&waker);\n\n pin!(f);\n let when = Instant::now() + timeout;\n\n loop {\n if let Ready(v) = crate::task::coop::budget(|| f.as_mut().poll(&mut cx)) {\n return Ok(v);\n }\n\n let now = Instant::now();\n\n if now >= when {\n return Err(());\n }\n\n park.park_timeout(when - now);\n }\n }\n}\n\nimpl Drop for DisallowBlockInPlaceGuard {\n fn drop(&mut self) {\n if self.0 {\n // XXX: Do we want some kind of assertion here, or is \"best effort\" okay?\n CONTEXT.with(|c| {\n if let EnterRuntime::Entered {\n allow_block_in_place: false,\n } = c.runtime.get()\n {\n c.runtime.set(EnterRuntime::Entered {\n allow_block_in_place: true,\n });\n }\n });\n }\n }\n}\n" }, { "path": "tokio/src/runtime/context/current.rs", "content": "use super::{Context, CONTEXT};\n\nuse crate::runtime::{scheduler, TryCurrentError};\nuse crate::util::markers::SyncNotSend;\n\nuse std::cell::{Cell, RefCell};\nuse std::marker::PhantomData;\n\n#[derive(Debug)]\n#[must_use]\npub(crate) struct SetCurrentGuard {\n // The previous handle\n prev: Option,\n\n // The depth for this guard\n depth: usize,\n\n // Don't let the type move across threads.\n _p: PhantomData,\n}\n\npub(super) struct HandleCell {\n /// Current handle\n handle: RefCell>,\n\n /// Tracks the number of nested calls to `try_set_current`.\n depth: Cell,\n}\n\n/// Sets this [`Handle`] as the current active [`Handle`].\n///\n/// [`Handle`]: crate::runtime::scheduler::Handle\npub(crate) fn try_set_current(handle: &scheduler::Handle) -> Option {\n CONTEXT.try_with(|ctx| ctx.set_current(handle)).ok()\n}\n\npub(crate) fn with_current(f: F) -> Result\nwhere\n F: FnOnce(&scheduler::Handle) -> R,\n{\n match CONTEXT.try_with(|ctx| ctx.current.handle.borrow().as_ref().map(f)) {\n Ok(Some(ret)) => Ok(ret),\n Ok(None) => Err(TryCurrentError::new_no_context()),\n Err(_access_error) => Err(TryCurrentError::new_thread_local_destroyed()),\n }\n}\n\nimpl Context {\n pub(super) fn set_current(&self, handle: &scheduler::Handle) -> SetCurrentGuard {\n let old_handle = self.current.handle.borrow_mut().replace(handle.clone());\n let depth = self.current.depth.get();\n\n assert!(depth != usize::MAX, \"reached max `enter` depth\");\n\n let depth = depth + 1;\n self.current.depth.set(depth);\n\n SetCurrentGuard {\n prev: old_handle,\n depth,\n _p: PhantomData,\n }\n }\n}\n\nimpl HandleCell {\n pub(super) const fn new() -> HandleCell {\n HandleCell {\n handle: RefCell::new(None),\n depth: Cell::new(0),\n }\n }\n}\n\nimpl Drop for SetCurrentGuard {\n fn drop(&mut self) {\n CONTEXT.with(|ctx| {\n let depth = ctx.current.depth.get();\n\n if depth != self.depth {\n if !std::thread::panicking() {\n panic!(\n \"`EnterGuard` values dropped out of order. Guards returned by \\\n `tokio::runtime::Handle::enter()` must be dropped in the reverse \\\n order as they were acquired.\"\n );\n } else {\n // Just return... this will leave handles in a wonky state though...\n return;\n }\n }\n\n *ctx.current.handle.borrow_mut() = self.prev.take();\n ctx.current.depth.set(depth - 1);\n });\n }\n}\n" }, { "path": "tokio/src/runtime/context/runtime.rs", "content": "use super::{BlockingRegionGuard, SetCurrentGuard, CONTEXT};\n\nuse crate::runtime::scheduler;\nuse crate::util::rand::{FastRand, RngSeed};\n\nuse std::fmt;\n\n#[derive(Debug, Clone, Copy)]\n#[must_use]\npub(crate) enum EnterRuntime {\n /// Currently in a runtime context.\n #[cfg_attr(not(feature = \"rt\"), allow(dead_code))]\n Entered { allow_block_in_place: bool },\n\n /// Not in a runtime context **or** a blocking region.\n NotEntered,\n}\n\n/// Guard tracking that a caller has entered a runtime context.\n#[must_use]\npub(crate) struct EnterRuntimeGuard {\n /// Tracks that the current thread has entered a blocking function call.\n pub(crate) blocking: BlockingRegionGuard,\n\n #[allow(dead_code)] // Only tracking the guard.\n pub(crate) handle: SetCurrentGuard,\n\n // Tracks the previous random number generator seed\n old_seed: RngSeed,\n}\n\n/// Marks the current thread as being within the dynamic extent of an\n/// executor.\n#[track_caller]\npub(crate) fn enter_runtime(handle: &scheduler::Handle, allow_block_in_place: bool, f: F) -> R\nwhere\n F: FnOnce(&mut BlockingRegionGuard) -> R,\n{\n let maybe_guard = CONTEXT.with(|c| {\n if c.runtime.get().is_entered() {\n None\n } else {\n // Set the entered flag\n c.runtime.set(EnterRuntime::Entered {\n allow_block_in_place,\n });\n\n // Generate a new seed\n let rng_seed = handle.seed_generator().next_seed();\n\n // Swap the RNG seed\n let mut rng = c.rng.get().unwrap_or_else(FastRand::new);\n let old_seed = rng.replace_seed(rng_seed);\n c.rng.set(Some(rng));\n\n Some(EnterRuntimeGuard {\n blocking: BlockingRegionGuard::new(),\n handle: c.set_current(handle),\n old_seed,\n })\n }\n });\n\n if let Some(mut guard) = maybe_guard {\n return f(&mut guard.blocking);\n }\n\n panic!(\n \"Cannot start a runtime from within a runtime. This happens \\\n because a function (like `block_on`) attempted to block the \\\n current thread while the thread is being used to drive \\\n asynchronous tasks.\"\n );\n}\n\nimpl fmt::Debug for EnterRuntimeGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Enter\").finish()\n }\n}\n\nimpl Drop for EnterRuntimeGuard {\n fn drop(&mut self) {\n CONTEXT.with(|c| {\n assert!(c.runtime.get().is_entered());\n c.runtime.set(EnterRuntime::NotEntered);\n // Replace the previous RNG seed\n let mut rng = c.rng.get().unwrap_or_else(FastRand::new);\n rng.replace_seed(self.old_seed.clone());\n c.rng.set(Some(rng));\n });\n }\n}\n\nimpl EnterRuntime {\n pub(crate) fn is_entered(self) -> bool {\n matches!(self, EnterRuntime::Entered { .. })\n }\n}\n" }, { "path": "tokio/src/runtime/context/runtime_mt.rs", "content": "use super::{EnterRuntime, CONTEXT};\n\n/// Returns true if in a runtime context.\npub(crate) fn current_enter_context() -> EnterRuntime {\n CONTEXT.with(|c| c.runtime.get())\n}\n\n/// Forces the current \"entered\" state to be cleared while the closure\n/// is executed.\npub(crate) fn exit_runtime R, R>(f: F) -> R {\n // Reset in case the closure panics\n struct Reset(EnterRuntime);\n\n impl Drop for Reset {\n fn drop(&mut self) {\n CONTEXT.with(|c| {\n assert!(\n !c.runtime.get().is_entered(),\n \"closure claimed permanent executor\"\n );\n c.runtime.set(self.0);\n });\n }\n }\n\n let was = CONTEXT.with(|c| {\n let e = c.runtime.get();\n assert!(e.is_entered(), \"asked to exit when not entered\");\n c.runtime.set(EnterRuntime::NotEntered);\n e\n });\n\n let _reset = Reset(was);\n // dropping _reset after f() will reset ENTERED\n f()\n}\n" }, { "path": "tokio/src/runtime/context/scoped.rs", "content": "use std::cell::Cell;\nuse std::ptr;\n\n/// Scoped thread-local storage\npub(super) struct Scoped {\n pub(super) inner: Cell<*const T>,\n}\n\nimpl Scoped {\n pub(super) const fn new() -> Scoped {\n Scoped {\n inner: Cell::new(ptr::null()),\n }\n }\n\n /// Inserts a value into the scoped cell for the duration of the closure\n pub(super) fn set(&self, t: &T, f: F) -> R\n where\n F: FnOnce() -> R,\n {\n struct Reset<'a, T> {\n cell: &'a Cell<*const T>,\n prev: *const T,\n }\n\n impl Drop for Reset<'_, T> {\n fn drop(&mut self) {\n self.cell.set(self.prev);\n }\n }\n\n let prev = self.inner.get();\n self.inner.set(t as *const _);\n\n let _reset = Reset {\n cell: &self.inner,\n prev,\n };\n\n f()\n }\n\n /// Gets the value out of the scoped cell;\n pub(super) fn with(&self, f: F) -> R\n where\n F: FnOnce(Option<&T>) -> R,\n {\n let val = self.inner.get();\n\n if val.is_null() {\n f(None)\n } else {\n unsafe { f(Some(&*val)) }\n }\n }\n}\n" }, { "path": "tokio/src/runtime/context.rs", "content": "use crate::loom::thread::AccessError;\nuse crate::task::coop;\n\nuse std::cell::Cell;\n\n#[cfg(any(feature = \"rt\", feature = \"macros\"))]\nuse crate::util::rand::FastRand;\n\ncfg_rt! {\n mod blocking;\n pub(crate) use blocking::{disallow_block_in_place, try_enter_blocking_region, BlockingRegionGuard};\n\n mod current;\n pub(crate) use current::{with_current, try_set_current, SetCurrentGuard};\n\n mod runtime;\n pub(crate) use runtime::{EnterRuntime, enter_runtime};\n\n mod scoped;\n use scoped::Scoped;\n\n use crate::runtime::{scheduler, task::Id};\n\n use std::task::Waker;\n\n cfg_taskdump! {\n use crate::runtime::task::trace;\n }\n}\n\ncfg_rt_multi_thread! {\n mod runtime_mt;\n pub(crate) use runtime_mt::{current_enter_context, exit_runtime};\n}\n\nstruct Context {\n /// Uniquely identifies the current thread\n #[cfg(feature = \"rt\")]\n thread_id: Cell>,\n\n /// Handle to the runtime scheduler running on the current thread.\n #[cfg(feature = \"rt\")]\n current: current::HandleCell,\n\n /// Handle to the scheduler's internal \"context\"\n #[cfg(feature = \"rt\")]\n scheduler: Scoped,\n\n #[cfg(feature = \"rt\")]\n current_task_id: Cell>,\n\n /// Tracks if the current thread is currently driving a runtime.\n /// Note, that if this is set to \"entered\", the current scheduler\n /// handle may not reference the runtime currently executing. This\n /// is because other runtime handles may be set to current from\n /// within a runtime.\n #[cfg(feature = \"rt\")]\n runtime: Cell,\n\n #[cfg(any(feature = \"rt\", feature = \"macros\"))]\n rng: Cell>,\n\n /// Tracks the amount of \"work\" a task may still do before yielding back to\n /// the scheduler\n budget: Cell,\n\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n trace: trace::Context,\n}\n\ntokio_thread_local! {\n static CONTEXT: Context = const {\n Context {\n #[cfg(feature = \"rt\")]\n thread_id: Cell::new(None),\n\n // Tracks the current runtime handle to use when spawning,\n // accessing drivers, etc...\n #[cfg(feature = \"rt\")]\n current: current::HandleCell::new(),\n\n // Tracks the current scheduler internal context\n #[cfg(feature = \"rt\")]\n scheduler: Scoped::new(),\n\n #[cfg(feature = \"rt\")]\n current_task_id: Cell::new(None),\n\n // Tracks if the current thread is currently driving a runtime.\n // Note, that if this is set to \"entered\", the current scheduler\n // handle may not reference the runtime currently executing. This\n // is because other runtime handles may be set to current from\n // within a runtime.\n #[cfg(feature = \"rt\")]\n runtime: Cell::new(EnterRuntime::NotEntered),\n\n #[cfg(any(feature = \"rt\", feature = \"macros\"))]\n rng: Cell::new(None),\n\n budget: Cell::new(coop::Budget::unconstrained()),\n\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(\n target_arch = \"aarch64\",\n target_arch = \"x86\",\n target_arch = \"x86_64\"\n )\n ))]\n trace: trace::Context::new(),\n }\n }\n}\n\n#[cfg(any(feature = \"macros\", all(feature = \"sync\", feature = \"rt\")))]\npub(crate) fn thread_rng_n(n: u32) -> u32 {\n CONTEXT.with(|ctx| {\n let mut rng = ctx.rng.get().unwrap_or_else(FastRand::new);\n let ret = rng.fastrand_n(n);\n ctx.rng.set(Some(rng));\n ret\n })\n}\n\npub(crate) fn budget(f: impl FnOnce(&Cell) -> R) -> Result {\n CONTEXT.try_with(|ctx| f(&ctx.budget))\n}\n\ncfg_rt! {\n use crate::runtime::ThreadId;\n\n pub(crate) fn thread_id() -> Result {\n CONTEXT.try_with(|ctx| {\n match ctx.thread_id.get() {\n Some(id) => id,\n None => {\n let id = ThreadId::next();\n ctx.thread_id.set(Some(id));\n id\n }\n }\n })\n }\n\n pub(crate) fn set_current_task_id(id: Option) -> Option {\n CONTEXT.try_with(|ctx| ctx.current_task_id.replace(id)).unwrap_or(None)\n }\n\n pub(crate) fn current_task_id() -> Option {\n CONTEXT.try_with(|ctx| ctx.current_task_id.get()).unwrap_or(None)\n }\n\n #[cfg(tokio_unstable)]\n pub(crate) fn worker_index() -> Option {\n with_scheduler(|ctx| ctx.and_then(|c| c.worker_index()))\n }\n\n #[track_caller]\n pub(crate) fn defer(waker: &Waker) {\n with_scheduler(|maybe_scheduler| {\n if let Some(scheduler) = maybe_scheduler {\n scheduler.defer(waker);\n } else {\n // Called from outside of the runtime, immediately wake the\n // task.\n waker.wake_by_ref();\n }\n });\n }\n\n pub(super) fn set_scheduler(v: &scheduler::Context, f: impl FnOnce() -> R) -> R {\n CONTEXT.with(|c| c.scheduler.set(v, f))\n }\n\n #[track_caller]\n pub(super) fn with_scheduler(f: impl FnOnce(Option<&scheduler::Context>) -> R) -> R {\n let mut f = Some(f);\n CONTEXT.try_with(|c| {\n let f = f.take().unwrap();\n if matches!(c.runtime.get(), EnterRuntime::Entered { .. }) {\n c.scheduler.with(f)\n } else {\n f(None)\n }\n })\n .unwrap_or_else(|_| (f.take().unwrap())(None))\n }\n\n cfg_taskdump! {\n /// SAFETY: Callers of this function must ensure that trace frames always\n /// form a valid linked list.\n pub(crate) unsafe fn with_trace(f: impl FnOnce(&trace::Context) -> R) -> Option {\n CONTEXT.try_with(|c| f(&c.trace)).ok()\n }\n }\n}\n" }, { "path": "tokio/src/runtime/driver/op.rs", "content": "use crate::io::uring::open::Open;\nuse crate::io::uring::read::Read;\nuse crate::io::uring::write::Write;\nuse crate::runtime::Handle;\n\nuse io_uring::cqueue;\nuse io_uring::squeue::Entry;\nuse std::future::Future;\nuse std::io::{self, Error};\nuse std::mem;\nuse std::pin::Pin;\nuse std::task::{Context, Poll, Waker};\n\n// This field isn't accessed directly, but it holds cancellation data,\n// so `#[allow(dead_code)]` is needed.\n#[allow(dead_code)]\n#[derive(Debug)]\npub(crate) enum CancelData {\n Open(Open),\n Write(Write),\n Read(Read),\n}\n\n#[derive(Debug)]\npub(crate) enum Lifecycle {\n /// The operation has been submitted to uring and is currently in-flight\n Submitted,\n\n /// The submitter is waiting for the completion of the operation\n Waiting(Waker),\n\n /// The submitter no longer has interest in the operation result. The state\n /// must be passed to the driver and held until the operation completes.\n Cancelled(\n // This field isn't accessed directly, but it holds cancellation data,\n // so `#[allow(dead_code)]` is needed.\n #[allow(dead_code)] CancelData,\n ),\n\n /// The operation has completed with a single cqe result\n Completed(io_uring::cqueue::Entry),\n}\n\npub(crate) enum State {\n Initialize(Option),\n Polled(usize),\n Complete,\n}\n\npub(crate) struct Op {\n // Handle to the runtime\n handle: Handle,\n // State of this Op\n state: State,\n // Per operation data.\n data: Option,\n}\n\nimpl Op {\n /// # Safety\n ///\n /// Callers must ensure that parameters of the entry (such as buffer) are valid and will\n /// be valid for the entire duration of the operation, otherwise it may cause memory problems.\n pub(crate) unsafe fn new(entry: Entry, data: T) -> Self {\n let handle = Handle::current();\n Self {\n handle,\n data: Some(data),\n state: State::Initialize(Some(entry)),\n }\n }\n pub(crate) fn take_data(&mut self) -> Option {\n self.data.take()\n }\n}\n\nimpl Drop for Op {\n fn drop(&mut self) {\n match self.state {\n // We've already dropped this Op.\n State::Complete => (),\n // We will cancel this Op.\n State::Polled(index) => {\n let data = self.take_data();\n let handle = &mut self.handle;\n handle.inner.driver().io().cancel_op(index, data);\n }\n // This Op has not been polled yet.\n // We don't need to do anything here.\n State::Initialize(_) => (),\n }\n }\n}\n\n/// A single CQE result\npub(crate) struct CqeResult {\n pub(crate) result: io::Result,\n}\n\nimpl From for CqeResult {\n fn from(cqe: cqueue::Entry) -> Self {\n let res = cqe.result();\n let result = if res >= 0 {\n Ok(res as u32)\n } else {\n Err(io::Error::from_raw_os_error(-res))\n };\n CqeResult { result }\n }\n}\n\n/// A trait that converts a CQE result into a usable value for each operation.\npub(crate) trait Completable {\n type Output;\n fn complete(self, cqe: CqeResult) -> Self::Output;\n\n // This is used when you want to terminate an operation with an error.\n //\n // The `Op` type that implements this trait can return the passed error\n // upstream by embedding it in the `Output`.\n fn complete_with_error(self, error: Error) -> Self::Output;\n}\n\n/// Extracts the `CancelData` needed to safely cancel an in-flight io_uring operation.\npub(crate) trait Cancellable {\n fn cancel(self) -> CancelData;\n}\n\nimpl Unpin for Op {}\n\nimpl Future for Op {\n type Output = T::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.get_mut();\n let handle = &mut this.handle;\n let driver = handle.inner.driver().io();\n\n match &mut this.state {\n State::Initialize(entry_opt) => {\n let entry = entry_opt.take().expect(\"Entry must be present\");\n let waker = cx.waker().clone();\n\n // SAFETY: entry is valid for the entire duration of the operation\n match unsafe { driver.register_op(entry, waker) } {\n Ok(idx) => this.state = State::Polled(idx),\n Err(err) => {\n let data = this\n .take_data()\n .expect(\"Data must be present on Initialization\");\n\n this.state = State::Complete;\n\n return Poll::Ready(data.complete_with_error(err));\n }\n };\n\n Poll::Pending\n }\n\n State::Polled(idx) => {\n let mut ctx = driver.get_uring().lock();\n let lifecycle = ctx.ops.get_mut(*idx).expect(\"Lifecycle must be present\");\n\n match mem::replace(lifecycle, Lifecycle::Submitted) {\n // Only replace the stored waker if it wouldn't wake the new one\n Lifecycle::Waiting(prev) if !prev.will_wake(cx.waker()) => {\n let waker = cx.waker().clone();\n *lifecycle = Lifecycle::Waiting(waker);\n Poll::Pending\n }\n\n Lifecycle::Waiting(prev) => {\n *lifecycle = Lifecycle::Waiting(prev);\n Poll::Pending\n }\n\n Lifecycle::Completed(cqe) => {\n // Clean up and complete the future\n ctx.remove_op(*idx);\n\n this.state = State::Complete;\n\n drop(ctx);\n\n let data = this\n .take_data()\n .expect(\"Data must be present on completion\");\n Poll::Ready(data.complete(cqe.into()))\n }\n\n Lifecycle::Submitted => {\n unreachable!(\"Submitted lifecycle should never be seen here\");\n }\n\n Lifecycle::Cancelled(_) => {\n unreachable!(\"Cancelled lifecycle should never be seen here\");\n }\n }\n }\n\n State::Complete => {\n panic!(\"Future polled after completion\");\n }\n }\n }\n}\n" }, { "path": "tokio/src/runtime/driver.rs", "content": "//! Abstracts out the entire chain of runtime sub-drivers into common types.\n\n// Eventually, this file will see significant refactoring / cleanup. For now, we\n// don't need to worry much about dead code with certain feature permutations.\n#![cfg_attr(\n any(not(all(tokio_unstable, feature = \"full\")), target_family = \"wasm\"),\n allow(dead_code)\n)]\n\nuse crate::runtime::park::{ParkThread, UnparkThread};\n\nuse std::io;\nuse std::time::Duration;\n\n#[derive(Debug)]\npub(crate) struct Driver {\n inner: TimeDriver,\n}\n\n#[derive(Debug)]\npub(crate) struct Handle {\n /// IO driver handle\n pub(crate) io: IoHandle,\n\n /// Signal driver handle\n #[cfg_attr(any(not(unix), loom), allow(dead_code))]\n pub(crate) signal: SignalHandle,\n\n /// Time driver handle\n pub(crate) time: TimeHandle,\n\n /// Source of `Instant::now()`\n #[cfg_attr(not(all(feature = \"time\", feature = \"test-util\")), allow(dead_code))]\n pub(crate) clock: Clock,\n}\n\npub(crate) struct Cfg {\n pub(crate) enable_io: bool,\n pub(crate) enable_time: bool,\n pub(crate) enable_pause_time: bool,\n pub(crate) start_paused: bool,\n pub(crate) nevents: usize,\n pub(crate) timer_flavor: crate::runtime::TimerFlavor,\n}\n\nimpl Driver {\n pub(crate) fn new(cfg: Cfg) -> io::Result<(Self, Handle)> {\n let (io_stack, io_handle, signal_handle) = create_io_stack(cfg.enable_io, cfg.nevents)?;\n\n let clock = create_clock(cfg.enable_pause_time, cfg.start_paused);\n\n let (time_driver, time_handle) =\n create_time_driver(cfg.enable_time, cfg.timer_flavor, io_stack, &clock);\n\n Ok((\n Self { inner: time_driver },\n Handle {\n io: io_handle,\n signal: signal_handle,\n time: time_handle,\n clock,\n },\n ))\n }\n\n pub(crate) fn park(&mut self, handle: &Handle) {\n self.inner.park(handle);\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &Handle, duration: Duration) {\n self.inner.park_timeout(handle, duration);\n }\n\n pub(crate) fn shutdown(&mut self, handle: &Handle) {\n self.inner.shutdown(handle);\n }\n}\n\nimpl Handle {\n pub(crate) fn unpark(&self) {\n #[cfg(feature = \"time\")]\n if let Some(handle) = &self.time {\n handle.unpark();\n }\n\n self.io.unpark();\n }\n\n cfg_io_driver! {\n #[track_caller]\n pub(crate) fn io(&self) -> &crate::runtime::io::Handle {\n self.io\n .as_ref()\n .expect(\"A Tokio 1.x context was found, but IO is disabled. Call `enable_io` on the runtime builder to enable IO.\")\n }\n }\n\n cfg_signal_internal_and_unix! {\n #[track_caller]\n pub(crate) fn signal(&self) -> &crate::runtime::signal::Handle {\n self.signal\n .as_ref()\n .expect(\"there is no signal driver running, must be called from the context of Tokio runtime\")\n }\n }\n\n cfg_time! {\n /// Returns a reference to the time driver handle.\n ///\n /// Panics if no time driver is present.\n #[track_caller]\n pub(crate) fn time(&self) -> &crate::runtime::time::Handle {\n self.time\n .as_ref()\n .expect(\"A Tokio 1.x context was found, but timers are disabled. Call `enable_time` on the runtime builder to enable timers.\")\n }\n\n #[cfg(tokio_unstable)]\n pub(crate) fn with_time(&self, f: F) -> R\n where\n F: FnOnce(Option<&crate::runtime::time::Handle>) -> R,\n {\n f(self.time.as_ref())\n }\n\n pub(crate) fn clock(&self) -> &Clock {\n &self.clock\n }\n }\n}\n\n// ===== io driver =====\n\ncfg_io_driver! {\n pub(crate) type IoDriver = crate::runtime::io::Driver;\n\n #[derive(Debug)]\n pub(crate) enum IoStack {\n Enabled(ProcessDriver),\n Disabled(ParkThread),\n }\n\n #[derive(Debug)]\n pub(crate) enum IoHandle {\n Enabled(crate::runtime::io::Handle),\n Disabled(UnparkThread),\n }\n\n fn create_io_stack(enabled: bool, nevents: usize) -> io::Result<(IoStack, IoHandle, SignalHandle)> {\n #[cfg(loom)]\n assert!(!enabled);\n\n let ret = if enabled {\n let (io_driver, io_handle) = crate::runtime::io::Driver::new(nevents)?;\n\n let (signal_driver, signal_handle) = create_signal_driver(io_driver, &io_handle)?;\n let process_driver = create_process_driver(signal_driver);\n\n (IoStack::Enabled(process_driver), IoHandle::Enabled(io_handle), signal_handle)\n } else {\n let park_thread = ParkThread::new();\n let unpark_thread = park_thread.unpark();\n (IoStack::Disabled(park_thread), IoHandle::Disabled(unpark_thread), Default::default())\n };\n\n Ok(ret)\n }\n\n impl IoStack {\n pub(crate) fn park(&mut self, handle: &Handle) {\n match self {\n IoStack::Enabled(v) => v.park(handle),\n IoStack::Disabled(v) => v.park(),\n }\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &Handle, duration: Duration) {\n match self {\n IoStack::Enabled(v) => v.park_timeout(handle, duration),\n IoStack::Disabled(v) => v.park_timeout(duration),\n }\n }\n\n pub(crate) fn shutdown(&mut self, handle: &Handle) {\n match self {\n IoStack::Enabled(v) => v.shutdown(handle),\n IoStack::Disabled(v) => v.shutdown(),\n }\n }\n }\n\n impl IoHandle {\n pub(crate) fn unpark(&self) {\n match self {\n IoHandle::Enabled(handle) => handle.unpark(),\n IoHandle::Disabled(handle) => handle.unpark(),\n }\n }\n\n pub(crate) fn as_ref(&self) -> Option<&crate::runtime::io::Handle> {\n match self {\n IoHandle::Enabled(v) => Some(v),\n IoHandle::Disabled(..) => None,\n }\n }\n }\n}\n\ncfg_not_io_driver! {\n pub(crate) type IoHandle = UnparkThread;\n\n #[derive(Debug)]\n pub(crate) struct IoStack(ParkThread);\n\n fn create_io_stack(_enabled: bool, _nevents: usize) -> io::Result<(IoStack, IoHandle, SignalHandle)> {\n let park_thread = ParkThread::new();\n let unpark_thread = park_thread.unpark();\n Ok((IoStack(park_thread), unpark_thread, Default::default()))\n }\n\n impl IoStack {\n pub(crate) fn park(&mut self, _handle: &Handle) {\n self.0.park();\n }\n\n pub(crate) fn park_timeout(&mut self, _handle: &Handle, duration: Duration) {\n self.0.park_timeout(duration);\n }\n\n pub(crate) fn shutdown(&mut self, _handle: &Handle) {\n self.0.shutdown();\n }\n\n /// This is not a \"real\" driver, so it is not considered enabled.\n pub(crate) fn is_enabled(&self) -> bool {\n false\n }\n }\n}\n\n// ===== signal driver =====\n\ncfg_signal_internal_and_unix! {\n type SignalDriver = crate::runtime::signal::Driver;\n pub(crate) type SignalHandle = Option;\n\n fn create_signal_driver(io_driver: IoDriver, io_handle: &crate::runtime::io::Handle) -> io::Result<(SignalDriver, SignalHandle)> {\n let driver = crate::runtime::signal::Driver::new(io_driver, io_handle)?;\n let handle = driver.handle();\n Ok((driver, Some(handle)))\n }\n}\n\ncfg_not_signal_internal! {\n pub(crate) type SignalHandle = ();\n\n cfg_io_driver! {\n type SignalDriver = IoDriver;\n\n fn create_signal_driver(io_driver: IoDriver, _io_handle: &crate::runtime::io::Handle) -> io::Result<(SignalDriver, SignalHandle)> {\n Ok((io_driver, ()))\n }\n }\n}\n\n// ===== process driver =====\n\ncfg_process_driver! {\n type ProcessDriver = crate::runtime::process::Driver;\n\n fn create_process_driver(signal_driver: SignalDriver) -> ProcessDriver {\n ProcessDriver::new(signal_driver)\n }\n}\n\ncfg_not_process_driver! {\n cfg_io_driver! {\n type ProcessDriver = SignalDriver;\n\n fn create_process_driver(signal_driver: SignalDriver) -> ProcessDriver {\n signal_driver\n }\n }\n}\n\n// ===== time driver =====\n\ncfg_time! {\n #[derive(Debug)]\n pub(crate) enum TimeDriver {\n Enabled {\n driver: crate::runtime::time::Driver,\n },\n EnabledAlt(IoStack),\n Disabled(IoStack),\n }\n\n pub(crate) type Clock = crate::time::Clock;\n pub(crate) type TimeHandle = Option;\n\n fn create_clock(enable_pausing: bool, start_paused: bool) -> Clock {\n crate::time::Clock::new(enable_pausing, start_paused)\n }\n\n fn create_time_driver(\n enable: bool,\n timer_flavor: crate::runtime::TimerFlavor,\n io_stack: IoStack,\n clock: &Clock,\n ) -> (TimeDriver, TimeHandle) {\n if enable {\n match timer_flavor {\n crate::runtime::TimerFlavor::Traditional => {\n let (driver, handle) = crate::runtime::time::Driver::new(io_stack, clock);\n (TimeDriver::Enabled { driver }, Some(handle))\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n crate::runtime::TimerFlavor::Alternative => {\n (TimeDriver::EnabledAlt(io_stack), Some(crate::runtime::time::Driver::new_alt(clock)))\n }\n }\n } else {\n (TimeDriver::Disabled(io_stack), None)\n }\n }\n\n impl TimeDriver {\n pub(crate) fn park(&mut self, handle: &Handle) {\n match self {\n TimeDriver::Enabled { driver, .. } => driver.park(handle),\n TimeDriver::EnabledAlt(v) => v.park(handle),\n TimeDriver::Disabled(v) => v.park(handle),\n }\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &Handle, duration: Duration) {\n match self {\n TimeDriver::Enabled { driver } => driver.park_timeout(handle, duration),\n TimeDriver::EnabledAlt(v) => v.park_timeout(handle, duration),\n TimeDriver::Disabled(v) => v.park_timeout(handle, duration),\n }\n }\n\n pub(crate) fn shutdown(&mut self, handle: &Handle) {\n match self {\n TimeDriver::Enabled { driver } => driver.shutdown(handle),\n TimeDriver::EnabledAlt(v) => v.shutdown(handle),\n TimeDriver::Disabled(v) => v.shutdown(handle),\n }\n }\n }\n}\n\ncfg_not_time! {\n type TimeDriver = IoStack;\n\n pub(crate) type Clock = ();\n pub(crate) type TimeHandle = ();\n\n fn create_clock(_enable_pausing: bool, _start_paused: bool) -> Clock {\n ()\n }\n\n fn create_time_driver(\n _enable: bool,\n _timer_flavor: crate::runtime::TimerFlavor,\n io_stack: IoStack,\n _clock: &Clock,\n ) -> (TimeDriver, TimeHandle) {\n (io_stack, ())\n }\n}\n\ncfg_io_uring! {\n pub(crate) mod op;\n}\n" }, { "path": "tokio/src/runtime/dump.rs", "content": "//! Snapshots of runtime state.\n//!\n//! See [`Handle::dump`][crate::runtime::Handle::dump].\n\nuse crate::task::Id;\nuse std::{fmt, future::Future, path::Path};\n\npub use crate::runtime::task::trace::Root;\n\n/// A snapshot of a runtime's state.\n///\n/// See [`Handle::dump`][crate::runtime::Handle::dump].\n#[derive(Debug)]\npub struct Dump {\n tasks: Tasks,\n}\n\n/// Snapshots of tasks.\n///\n/// See [`Handle::dump`][crate::runtime::Handle::dump].\n#[derive(Debug)]\npub struct Tasks {\n tasks: Vec,\n}\n\n/// A snapshot of a task.\n///\n/// See [`Handle::dump`][crate::runtime::Handle::dump].\n#[derive(Debug)]\npub struct Task {\n id: Id,\n trace: Trace,\n}\n\n/// Represents an address that should not be dereferenced.\n///\n/// This type exists to get the auto traits correct, the public API\n/// uses raw pointers to make life easier for users.\n#[derive(Copy, Clone, Debug)]\nstruct Address(*mut std::ffi::c_void);\n\n// Safe since Address should not be dereferenced\nunsafe impl Send for Address {}\nunsafe impl Sync for Address {}\n\n/// A backtrace symbol.\n///\n/// This struct provides accessors for backtrace symbols, similar to [`backtrace::BacktraceSymbol`].\n#[derive(Clone, Debug)]\npub struct BacktraceSymbol {\n name: Option>,\n name_demangled: Option>,\n addr: Option

,\n filename: Option,\n lineno: Option,\n colno: Option,\n}\n\nimpl BacktraceSymbol {\n pub(crate) fn from_backtrace_symbol(sym: &backtrace::BacktraceSymbol) -> Self {\n let name = sym.name();\n Self {\n name: name.as_ref().map(|name| name.as_bytes().into()),\n name_demangled: name.map(|name| format!(\"{name}\").into()),\n addr: sym.addr().map(Address),\n filename: sym.filename().map(From::from),\n lineno: sym.lineno(),\n colno: sym.colno(),\n }\n }\n\n /// Return the raw name of the symbol.\n pub fn name_raw(&self) -> Option<&[u8]> {\n self.name.as_deref()\n }\n\n /// Return the demangled name of the symbol.\n pub fn name_demangled(&self) -> Option<&str> {\n self.name_demangled.as_deref()\n }\n\n /// Returns the starting address of this symbol.\n pub fn addr(&self) -> Option<*mut std::ffi::c_void> {\n self.addr.map(|addr| addr.0)\n }\n\n /// Returns the file name where this function was defined. If debuginfo\n /// is missing, this is likely to return None.\n pub fn filename(&self) -> Option<&Path> {\n self.filename.as_deref()\n }\n\n /// Returns the line number for where this symbol is currently executing.\n ///\n /// If debuginfo is missing, this is likely to return `None`.\n pub fn lineno(&self) -> Option {\n self.lineno\n }\n\n /// Returns the column number for where this symbol is currently executing.\n ///\n /// If debuginfo is missing, this is likely to return `None`.\n pub fn colno(&self) -> Option {\n self.colno\n }\n}\n\n/// A backtrace frame.\n///\n/// This struct represents one stack frame in a captured backtrace, similar to [`backtrace::BacktraceFrame`].\n#[derive(Clone, Debug)]\npub struct BacktraceFrame {\n ip: Address,\n symbol_address: Address,\n symbols: Box<[BacktraceSymbol]>,\n}\n\nimpl BacktraceFrame {\n pub(crate) fn from_resolved_backtrace_frame(frame: &backtrace::BacktraceFrame) -> Self {\n Self {\n ip: Address(frame.ip()),\n symbol_address: Address(frame.symbol_address()),\n symbols: frame\n .symbols()\n .iter()\n .map(BacktraceSymbol::from_backtrace_symbol)\n .collect(),\n }\n }\n\n /// Return the instruction pointer of this frame.\n ///\n /// See the ABI docs for your platform for the exact meaning.\n pub fn ip(&self) -> *mut std::ffi::c_void {\n self.ip.0\n }\n\n /// Returns the starting symbol address of the frame of this function.\n pub fn symbol_address(&self) -> *mut std::ffi::c_void {\n self.symbol_address.0\n }\n\n /// Return an iterator over the symbols of this backtrace frame.\n ///\n /// Due to inlining, it is possible for there to be multiple [`BacktraceSymbol`] items relating\n /// to a single frame. The first symbol listed is the \"innermost function\",\n /// whereas the last symbol is the outermost (last caller).\n pub fn symbols(&self) -> impl Iterator {\n self.symbols.iter()\n }\n}\n\n/// A captured backtrace.\n///\n/// This struct provides access to each backtrace frame, similar to [`backtrace::Backtrace`].\n#[derive(Clone, Debug)]\npub struct Backtrace {\n frames: Box<[BacktraceFrame]>,\n}\n\nimpl Backtrace {\n /// Return the frames in this backtrace, innermost (in a task dump,\n /// likely to be a leaf future's poll function) first.\n pub fn frames(&self) -> impl Iterator {\n self.frames.iter()\n }\n}\n\n/// An execution trace of a task's last poll.\n///\n///
\n///\n/// Resolving a backtrace, either via the [`Display`][std::fmt::Display] impl or via\n/// [`resolve_backtraces`][Trace::resolve_backtraces], parses debuginfo, which is\n/// possibly a CPU-expensive operation that can take a platform-specific but\n/// long time to run - often over 100 milliseconds, especially if the current\n/// process's binary is big. In some cases, the platform might internally cache some of the\n/// debuginfo, so successive calls to `resolve_backtraces` might be faster than\n/// the first call, but all guarantees are platform-dependent.\n///\n/// To avoid blocking the runtime, it is recommended\n/// that you resolve backtraces inside of a [`spawn_blocking()`][crate::task::spawn_blocking]\n/// and to have some concurrency-limiting mechanism to avoid unexpected performance impact.\n///
\n///\n/// See [`Handle::dump`][crate::runtime::Handle::dump].\n#[derive(Debug)]\npub struct Trace {\n inner: super::task::trace::Trace,\n}\n\nimpl Trace {\n /// Resolve and return a list of backtraces that are involved in polls in this trace.\n ///\n /// The exact backtraces included here are unstable and might change in the future,\n /// but you can expect one [`Backtrace`] for every call to\n /// [`poll`] to a bottom-level Tokio future - so if something like [`join!`] is\n /// used, there will be a backtrace for each future in the join.\n ///\n /// [`poll`]: std::future::Future::poll\n /// [`join!`]: macro@join\n pub fn resolve_backtraces(&self) -> Vec {\n self.inner\n .backtraces()\n .iter()\n .map(|backtrace| {\n let mut backtrace = backtrace::Backtrace::from(backtrace.clone());\n backtrace.resolve();\n Backtrace {\n frames: backtrace\n .frames()\n .iter()\n .map(BacktraceFrame::from_resolved_backtrace_frame)\n .collect(),\n }\n })\n .collect()\n }\n\n /// Runs the function `f` in tracing mode, and returns its result along with the resulting [`Trace`].\n ///\n /// This is normally called with `f` being the poll function of a future, and will give you a backtrace\n /// that tells you what that one future is doing.\n ///\n /// Use [`Handle::dump`] instead if you want to know what *all the tasks* in your program are doing.\n /// Also see [`Handle::dump`] for more documentation about dumps, but unlike [`Handle::dump`], this function\n /// should not be much slower than calling `f` directly.\n ///\n /// Due to the way tracing is implemented, Tokio leaf futures will usually, instead of doing their\n /// actual work, do the equivalent of a `yield_now` (returning a `Poll::Pending` and scheduling the\n /// current context for execution), which means forward progress will probably not happen unless\n /// you eventually call your future outside of `capture`.\n ///\n /// [`Handle::dump`]: crate::runtime::Handle::dump\n ///\n /// Example usage:\n /// ```\n /// use std::future::Future;\n /// use std::task::Poll;\n /// use tokio::runtime::dump::Trace;\n ///\n /// # async fn test_fn() {\n /// // some future\n /// let mut test_future = std::pin::pin!(async move { tokio::task::yield_now().await; 0 });\n ///\n /// // trace it once, see what it's doing\n /// let (trace, res) = Trace::root(std::future::poll_fn(|cx| {\n /// let (res, trace) = Trace::capture(|| test_future.as_mut().poll(cx));\n /// Poll::Ready((trace, res))\n /// })).await;\n ///\n /// // await it to let it finish, outside of a `capture`\n /// let output = match res {\n /// Poll::Ready(output) => output,\n /// Poll::Pending => test_future.await,\n /// };\n ///\n /// println!(\"{trace}\");\n /// # }\n /// ```\n ///\n /// ### Nested calls\n ///\n /// Nested calls to `capture` might return partial traces, but will not do any other undesirable behavior (for\n /// example, they will not panic).\n pub fn capture(f: F) -> (R, Trace)\n where\n F: FnOnce() -> R,\n {\n let (res, trace) = super::task::trace::Trace::capture(f);\n (res, Trace { inner: trace })\n }\n\n /// Create a root for stack traces captured using [`Trace::capture`]. Stack frames above\n /// the root will not be captured.\n ///\n /// Nesting multiple [`Root`] futures is fine. Captures will stop at the first root. Not having\n /// a [`Root`] is fine as well, but there is no guarantee on where the capture will stop.\n pub fn root(f: F) -> Root\n where\n F: Future,\n {\n crate::runtime::task::trace::Trace::root(f)\n }\n}\n\nimpl Dump {\n pub(crate) fn new(tasks: Vec) -> Self {\n Self {\n tasks: Tasks { tasks },\n }\n }\n\n /// Tasks in this snapshot.\n pub fn tasks(&self) -> &Tasks {\n &self.tasks\n }\n}\n\nimpl Tasks {\n /// Iterate over tasks.\n pub fn iter(&self) -> impl Iterator {\n self.tasks.iter()\n }\n}\n\nimpl Task {\n pub(crate) fn new(id: Id, trace: super::task::trace::Trace) -> Self {\n Self {\n id,\n trace: Trace { inner: trace },\n }\n }\n\n /// Returns a [task ID] that uniquely identifies this task relative to other\n /// tasks spawned at the time of the dump.\n ///\n /// **Note**: This is an [unstable API][unstable]. The public API of this type\n /// may break in 1.x releases. See [the documentation on unstable\n /// features][unstable] for details.\n ///\n /// [task ID]: crate::task::Id\n /// [unstable]: crate#unstable-features\n #[cfg(tokio_unstable)]\n #[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\n pub fn id(&self) -> Id {\n self.id\n }\n\n /// A trace of this task's state.\n pub fn trace(&self) -> &Trace {\n &self.trace\n }\n}\n\nimpl fmt::Display for Trace {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.inner.fmt(f)\n }\n}\n" }, { "path": "tokio/src/runtime/handle.rs", "content": "use crate::runtime;\nuse crate::runtime::{context, scheduler, RuntimeFlavor, RuntimeMetrics};\n\n/// Handle to the runtime.\n///\n/// The handle is internally reference-counted and can be freely cloned. A handle can be\n/// obtained using the [`Runtime::handle`] method.\n///\n/// [`Runtime::handle`]: crate::runtime::Runtime::handle()\n#[derive(Debug, Clone)]\n// When the `rt` feature is *not* enabled, this type is still defined, but not\n// included in the public API.\npub struct Handle {\n pub(crate) inner: scheduler::Handle,\n}\n\nuse crate::runtime::task::JoinHandle;\nuse crate::runtime::BOX_FUTURE_THRESHOLD;\nuse crate::util::error::{CONTEXT_MISSING_ERROR, THREAD_LOCAL_DESTROYED_ERROR};\nuse crate::util::trace::SpawnMeta;\n\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::{error, fmt, mem};\n\n/// Runtime context guard.\n///\n/// Returned by [`Runtime::enter`] and [`Handle::enter`], the context guard exits\n/// the runtime context on drop.\n///\n/// [`Runtime::enter`]: fn@crate::runtime::Runtime::enter\n#[derive(Debug)]\n#[must_use = \"Creating and dropping a guard does nothing\"]\npub struct EnterGuard<'a> {\n _guard: context::SetCurrentGuard,\n _handle_lifetime: PhantomData<&'a Handle>,\n}\n\nimpl Handle {\n /// Enters the runtime context. This allows you to construct types that must\n /// have an executor available on creation such as [`Sleep`] or\n /// [`TcpStream`]. It will also allow you to call methods such as\n /// [`tokio::spawn`] and [`Handle::current`] without panicking.\n ///\n /// # Panics\n ///\n /// When calling `Handle::enter` multiple times, the returned guards\n /// **must** be dropped in the reverse order that they were acquired.\n /// Failure to do so will result in a panic and possible memory leaks.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// let rt = Runtime::new().unwrap();\n ///\n /// let _guard = rt.enter();\n /// tokio::spawn(async {\n /// println!(\"Hello world!\");\n /// });\n /// # }\n /// ```\n ///\n /// Do **not** do the following, this shows a scenario that will result in a\n /// panic and possible memory leak.\n ///\n /// ```should_panic,ignore-wasm\n /// use tokio::runtime::Runtime;\n ///\n /// let rt1 = Runtime::new().unwrap();\n /// let rt2 = Runtime::new().unwrap();\n ///\n /// let enter1 = rt1.enter();\n /// let enter2 = rt2.enter();\n ///\n /// drop(enter1);\n /// drop(enter2);\n /// ```\n ///\n /// [`Sleep`]: struct@crate::time::Sleep\n /// [`TcpStream`]: struct@crate::net::TcpStream\n /// [`tokio::spawn`]: fn@crate::spawn\n pub fn enter(&self) -> EnterGuard<'_> {\n EnterGuard {\n _guard: match context::try_set_current(&self.inner) {\n Some(guard) => guard,\n None => panic!(\"{}\", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),\n },\n _handle_lifetime: PhantomData,\n }\n }\n\n /// Returns a `Handle` view over the currently running `Runtime`.\n ///\n /// # Panics\n ///\n /// This will panic if called outside the context of a Tokio runtime. That means that you must\n /// call this on one of the threads **being run by the runtime**, or from a thread with an active\n /// `EnterGuard`. Calling this from within a thread created by `std::thread::spawn` (for example)\n /// will cause a panic unless that thread has an active `EnterGuard`.\n ///\n /// # Examples\n ///\n /// This can be used to obtain the handle of the surrounding runtime from an async\n /// block or function running on that runtime.\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// # use std::thread;\n /// # use tokio::runtime::Runtime;\n /// # fn dox() {\n /// # let rt = Runtime::new().unwrap();\n /// # rt.spawn(async {\n /// use tokio::runtime::Handle;\n ///\n /// // Inside an async block or function.\n /// let handle = Handle::current();\n /// handle.spawn(async {\n /// println!(\"now running in the existing Runtime\");\n /// });\n ///\n /// # let handle =\n /// thread::spawn(move || {\n /// // Notice that the handle is created outside of this thread and then moved in\n /// handle.spawn(async { /* ... */ });\n /// // This next line would cause a panic because we haven't entered the runtime\n /// // and created an EnterGuard\n /// // let handle2 = Handle::current(); // panic\n /// // So we create a guard here with Handle::enter();\n /// let _guard = handle.enter();\n /// // Now we can call Handle::current();\n /// let handle2 = Handle::current();\n /// });\n /// # handle.join().unwrap();\n /// # });\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn current() -> Self {\n Handle {\n inner: scheduler::Handle::current(),\n }\n }\n\n /// Returns a Handle view over the currently running Runtime\n ///\n /// Returns an error if no Runtime has been started\n ///\n /// Contrary to `current`, this never panics\n pub fn try_current() -> Result {\n context::with_current(|inner| Handle {\n inner: inner.clone(),\n })\n }\n\n /// Spawns a future onto the Tokio runtime.\n ///\n /// This spawns the given future onto the runtime's executor, usually a\n /// thread pool. The thread pool is then responsible for polling the future\n /// until it completes.\n ///\n /// The provided future will start running in the background immediately\n /// when `spawn` is called, even if you don't await the returned\n /// `JoinHandle` (assuming that the runtime [is running][running-runtime]).\n ///\n /// See [module level][mod] documentation for more details.\n ///\n /// [mod]: index.html\n /// [running-runtime]: index.html#driving-the-runtime\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n /// // Get a handle from this runtime\n /// let handle = rt.handle();\n ///\n /// // Spawn a future onto the runtime using the handle\n /// handle.spawn(async {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn(&self, future: F) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n let fut_size = mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n self.spawn_named(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n self.spawn_named(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n /// Runs the provided function on an executor dedicated to blocking\n /// operations.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n /// // Get a handle from this runtime\n /// let handle = rt.handle();\n ///\n /// // Spawn a blocking function onto the runtime using the handle\n /// handle.spawn_blocking(|| {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn_blocking(&self, func: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n self.inner.blocking_spawner().spawn_blocking(self, func)\n }\n\n /// Runs a future to completion on this `Handle`'s associated `Runtime`.\n ///\n /// This runs the given future on the current thread, blocking until it is\n /// complete, and yielding its resolved result. Any tasks or timers which\n /// the future spawns internally will be executed on the runtime.\n ///\n /// When this is used on a `current_thread` runtime, only the\n /// [`Runtime::block_on`] method can drive the IO and timer drivers, but the\n /// `Handle::block_on` method cannot drive them. This means that, when using\n /// this method on a `current_thread` runtime, anything that relies on IO or\n /// timers will not work unless there is another thread currently calling\n /// [`Runtime::block_on`] on the same runtime.\n ///\n /// # If the runtime has been shut down\n ///\n /// If the `Handle`'s associated `Runtime` has been shut down (through\n /// [`Runtime::shutdown_background`], [`Runtime::shutdown_timeout`], or by\n /// dropping it) and `Handle::block_on` is used it might return an error or\n /// panic. Specifically IO resources will return an error and timers will\n /// panic. Runtime independent futures will run as normal.\n ///\n /// # Panics\n ///\n /// This function will panic if any of the following conditions are met:\n /// - The provided future panics.\n /// - It is called from within an asynchronous context, such as inside\n /// [`Runtime::block_on`], `Handle::block_on`, or from a function annotated\n /// with [`tokio::main`].\n /// - A timer future is executed on a runtime that has been shut down.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n ///\n /// // Get a handle from this runtime\n /// let handle = rt.handle();\n ///\n /// // Execute the future, blocking the current thread until completion\n /// handle.block_on(async {\n /// println!(\"hello\");\n /// });\n /// # }\n /// ```\n ///\n /// Or using `Handle::current`:\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main]\n /// async fn main () {\n /// let handle = Handle::current();\n /// std::thread::spawn(move || {\n /// // Using Handle::block_on to run async code in the new thread.\n /// handle.block_on(async {\n /// println!(\"hello\");\n /// });\n /// });\n /// }\n /// # }\n /// ```\n ///\n /// `Handle::block_on` may be combined with [`task::block_in_place`] to\n /// re-enter the async context of a multi-thread scheduler runtime:\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::task;\n /// use tokio::runtime::Handle;\n ///\n /// # async fn docs() {\n /// task::block_in_place(move || {\n /// Handle::current().block_on(async move {\n /// // do something async\n /// });\n /// });\n /// # }\n /// # }\n /// ```\n ///\n /// [`JoinError`]: struct@crate::task::JoinError\n /// [`JoinHandle`]: struct@crate::task::JoinHandle\n /// [`Runtime::block_on`]: fn@crate::runtime::Runtime::block_on\n /// [`Runtime::shutdown_background`]: fn@crate::runtime::Runtime::shutdown_background\n /// [`Runtime::shutdown_timeout`]: fn@crate::runtime::Runtime::shutdown_timeout\n /// [`spawn_blocking`]: crate::task::spawn_blocking\n /// [`tokio::fs`]: crate::fs\n /// [`tokio::net`]: crate::net\n /// [`tokio::time`]: crate::time\n /// [`tokio::main`]: ../attr.main.html\n /// [`task::block_in_place`]: crate::task::block_in_place\n #[track_caller]\n pub fn block_on(&self, future: F) -> F::Output {\n let fut_size = mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n self.block_on_inner(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n self.block_on_inner(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n #[track_caller]\n fn block_on_inner(&self, future: F, _meta: SpawnMeta<'_>) -> F::Output {\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n let future = super::task::trace::Trace::root(future);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let future =\n crate::util::trace::task(future, \"block_on\", _meta, super::task::Id::next().as_u64());\n\n // Enter the runtime context. This sets the current driver handles and\n // prevents blocking an existing runtime.\n context::enter_runtime(&self.inner, true, |blocking| {\n blocking.block_on(future).expect(\"failed to park thread\")\n })\n }\n\n #[track_caller]\n pub(crate) fn spawn_named(&self, future: F, meta: SpawnMeta<'_>) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n let id = crate::runtime::task::Id::next();\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n let future = super::task::trace::Trace::root(future);\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let future = crate::util::trace::task(future, \"task\", meta, id.as_u64());\n self.inner.spawn(future, id, meta.spawned_at)\n }\n\n #[track_caller]\n #[allow(dead_code)]\n /// # Safety\n ///\n /// This must only be called in `LocalRuntime` if the runtime has been verified to be owned\n /// by the current thread.\n pub(crate) unsafe fn spawn_local_named(\n &self,\n future: F,\n meta: SpawnMeta<'_>,\n ) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let id = crate::runtime::task::Id::next();\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n let future = super::task::trace::Trace::root(future);\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let future = crate::util::trace::task(future, \"task\", meta, id.as_u64());\n unsafe { self.inner.spawn_local(future, id, meta.spawned_at) }\n }\n\n /// Returns the flavor of the current `Runtime`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{Handle, RuntimeFlavor};\n ///\n /// #[tokio::main(flavor = \"current_thread\")]\n /// async fn main() {\n /// assert_eq!(RuntimeFlavor::CurrentThread, Handle::current().runtime_flavor());\n /// }\n /// ```\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::{Handle, RuntimeFlavor};\n ///\n /// #[tokio::main(flavor = \"multi_thread\", worker_threads = 4)]\n /// async fn main() {\n /// assert_eq!(RuntimeFlavor::MultiThread, Handle::current().runtime_flavor());\n /// }\n /// # }\n /// ```\n pub fn runtime_flavor(&self) -> RuntimeFlavor {\n match self.inner {\n scheduler::Handle::CurrentThread(_) => RuntimeFlavor::CurrentThread,\n #[cfg(feature = \"rt-multi-thread\")]\n scheduler::Handle::MultiThread(_) => RuntimeFlavor::MultiThread,\n }\n }\n\n /// Returns the [`Id`] of the current `Runtime`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main(flavor = \"current_thread\")]\n /// async fn main() {\n /// println!(\"Current runtime id: {}\", Handle::current().id());\n /// }\n /// ```\n ///\n /// [`Id`]: struct@crate::runtime::Id\n pub fn id(&self) -> runtime::Id {\n let owned_id = match &self.inner {\n scheduler::Handle::CurrentThread(handle) => handle.owned_id(),\n #[cfg(feature = \"rt-multi-thread\")]\n scheduler::Handle::MultiThread(handle) => handle.owned_id(),\n };\n runtime::Id::new(owned_id)\n }\n\n /// Returns a view that lets you get information about how the runtime\n /// is performing.\n pub fn metrics(&self) -> RuntimeMetrics {\n RuntimeMetrics::new(self.clone())\n }\n}\n\nimpl std::panic::UnwindSafe for Handle {}\n\nimpl std::panic::RefUnwindSafe for Handle {}\n\ncfg_taskdump! {\n impl Handle {\n /// Captures a snapshot of the runtime's state.\n ///\n /// If you only want to capture a snapshot of a single future's state, you can use\n /// [`Trace::capture`][crate::runtime::dump::Trace].\n ///\n /// This functionality is experimental, and comes with a number of\n /// requirements and limitations.\n ///\n /// # Examples\n ///\n /// This can be used to get call traces of each task in the runtime.\n /// Calls to `Handle::dump` should usually be enclosed in a\n /// [timeout][crate::time::timeout], so that dumping does not escalate a\n /// single blocked runtime thread into an entirely blocked runtime.\n ///\n /// ```\n /// # use tokio::runtime::Runtime;\n /// # fn dox() {\n /// # let rt = Runtime::new().unwrap();\n /// # rt.spawn(async {\n /// use tokio::runtime::Handle;\n /// use tokio::time::{timeout, Duration};\n ///\n /// // Inside an async block or function.\n /// let handle = Handle::current();\n /// if let Ok(dump) = timeout(Duration::from_secs(2), handle.dump()).await {\n /// for (i, task) in dump.tasks().iter().enumerate() {\n /// let trace = task.trace();\n /// println!(\"TASK {i}:\");\n /// println!(\"{trace}\\n\");\n /// }\n /// }\n /// # });\n /// # }\n /// ```\n ///\n /// This produces highly detailed traces of tasks; e.g.:\n ///\n /// ```plain\n /// TASK 0:\n /// ╼ dump::main::{{closure}}::a::{{closure}} at /tokio/examples/dump.rs:18:20\n /// └╼ dump::main::{{closure}}::b::{{closure}} at /tokio/examples/dump.rs:23:20\n /// └╼ dump::main::{{closure}}::c::{{closure}} at /tokio/examples/dump.rs:28:24\n /// └╼ tokio::sync::barrier::Barrier::wait::{{closure}} at /tokio/tokio/src/sync/barrier.rs:129:10\n /// └╼ as core::future::future::Future>::poll at /tokio/tokio/src/util/trace.rs:77:46\n /// └╼ tokio::sync::barrier::Barrier::wait_internal::{{closure}} at /tokio/tokio/src/sync/barrier.rs:183:36\n /// └╼ tokio::sync::watch::Receiver::changed::{{closure}} at /tokio/tokio/src/sync/watch.rs:604:55\n /// └╼ tokio::sync::watch::changed_impl::{{closure}} at /tokio/tokio/src/sync/watch.rs:755:18\n /// └╼ ::poll at /tokio/tokio/src/sync/notify.rs:1103:9\n /// └╼ tokio::sync::notify::Notified::poll_notified at /tokio/tokio/src/sync/notify.rs:996:32\n /// ```\n ///\n /// # Requirements\n ///\n /// ## Debug Info Must Be Available\n ///\n /// To produce task traces, the application must **not** be compiled\n /// with `split debuginfo`. On Linux, including `debuginfo` within the\n /// application binary is the (correct) default. You can further ensure\n /// this behavior with the following directive in your `Cargo.toml`:\n ///\n /// ```toml\n /// [profile.*]\n /// split-debuginfo = \"off\"\n /// ```\n ///\n /// ## Unstable Features\n ///\n /// This functionality is **unstable**, and requires both the\n /// `--cfg tokio_unstable` and cargo feature `taskdump` to be set.\n ///\n /// You can do this by setting the `RUSTFLAGS` environment variable\n /// before invoking `cargo`; e.g.:\n /// ```bash\n /// RUSTFLAGS=\"--cfg tokio_unstable\" cargo run --example dump\n /// ```\n ///\n /// Or by [configuring][cargo-config] `rustflags` in\n /// `.cargo/config.toml`:\n /// ```text\n /// [build]\n /// rustflags = [\"--cfg\", \"tokio_unstable\"]\n /// ```\n ///\n /// [cargo-config]:\n /// https://doc.rust-lang.org/cargo/reference/config.html\n ///\n /// ## Platform Requirements\n ///\n /// Task dumps are supported on Linux atop `aarch64`, `x86` and `x86_64`.\n ///\n /// ## Current Thread Runtime Requirements\n ///\n /// On the `current_thread` runtime, task dumps may only be requested\n /// from *within* the context of the runtime being dumped. Do not, for\n /// example, await `Handle::dump()` on a different runtime.\n ///\n /// # Limitations\n ///\n /// ## Performance\n ///\n /// Although enabling the `taskdump` feature imposes virtually no\n /// additional runtime overhead, actually calling `Handle::dump` is\n /// expensive. The runtime must synchronize and pause its workers, then\n /// re-poll every task in a special tracing mode. Avoid requesting dumps\n /// often.\n ///\n /// ## Local Executors\n ///\n /// Tasks managed by local executors (e.g., `FuturesUnordered` and\n /// [`LocalSet`][crate::task::LocalSet]) may not appear in task dumps.\n ///\n /// ## Non-Termination When Workers Are Blocked\n ///\n /// The future produced by `Handle::dump` may never produce `Ready` if\n /// another runtime worker is blocked for more than 250ms. This may\n /// occur if a dump is requested during shutdown, or if another runtime\n /// worker is infinite looping or synchronously deadlocked. For these\n /// reasons, task dumping should usually be paired with an explicit\n /// [timeout][crate::time::timeout].\n pub async fn dump(&self) -> crate::runtime::Dump {\n match &self.inner {\n scheduler::Handle::CurrentThread(handle) => handle.dump(),\n #[cfg(all(feature = \"rt-multi-thread\", not(target_os = \"wasi\")))]\n scheduler::Handle::MultiThread(handle) => {\n // perform the trace in a separate thread so that the\n // trace itself does not appear in the taskdump.\n let handle = handle.clone();\n spawn_thread(async {\n let handle = handle;\n handle.dump().await\n }).await\n },\n }\n }\n\n /// Produces `true` if the current task is being traced for a dump;\n /// otherwise false. This function is only public for integration\n /// testing purposes. Do not rely on it.\n #[doc(hidden)]\n pub fn is_tracing() -> bool {\n super::task::trace::Context::is_tracing()\n }\n }\n\n cfg_rt_multi_thread! {\n /// Spawn a new thread and asynchronously await on its result.\n async fn spawn_thread(f: F) -> ::Output\n where\n F: Future + Send + 'static,\n ::Output: Send + 'static\n {\n let (tx, rx) = crate::sync::oneshot::channel();\n crate::loom::thread::spawn(|| {\n let rt = crate::runtime::Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let _ = tx.send(f.await);\n });\n });\n rx.await.unwrap()\n }\n }\n}\n\n/// Error returned by `try_current` when no Runtime has been started\n#[derive(Debug)]\npub struct TryCurrentError {\n kind: TryCurrentErrorKind,\n}\n\nimpl TryCurrentError {\n pub(crate) fn new_no_context() -> Self {\n Self {\n kind: TryCurrentErrorKind::NoContext,\n }\n }\n\n pub(crate) fn new_thread_local_destroyed() -> Self {\n Self {\n kind: TryCurrentErrorKind::ThreadLocalDestroyed,\n }\n }\n\n /// Returns true if the call failed because there is currently no runtime in\n /// the Tokio context.\n pub fn is_missing_context(&self) -> bool {\n matches!(self.kind, TryCurrentErrorKind::NoContext)\n }\n\n /// Returns true if the call failed because the Tokio context thread-local\n /// had been destroyed. This can usually only happen if in the destructor of\n /// other thread-locals.\n pub fn is_thread_local_destroyed(&self) -> bool {\n matches!(self.kind, TryCurrentErrorKind::ThreadLocalDestroyed)\n }\n}\n\nenum TryCurrentErrorKind {\n NoContext,\n ThreadLocalDestroyed,\n}\n\nimpl fmt::Debug for TryCurrentErrorKind {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n TryCurrentErrorKind::NoContext => f.write_str(\"NoContext\"),\n TryCurrentErrorKind::ThreadLocalDestroyed => f.write_str(\"ThreadLocalDestroyed\"),\n }\n }\n}\n\nimpl fmt::Display for TryCurrentError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n use TryCurrentErrorKind as E;\n match self.kind {\n E::NoContext => f.write_str(CONTEXT_MISSING_ERROR),\n E::ThreadLocalDestroyed => f.write_str(THREAD_LOCAL_DESTROYED_ERROR),\n }\n }\n}\n\nimpl error::Error for TryCurrentError {}\n" }, { "path": "tokio/src/runtime/id.rs", "content": "use std::fmt;\nuse std::num::NonZeroU64;\n\n/// An opaque ID that uniquely identifies a runtime relative to all other currently\n/// running runtimes.\n///\n/// # Notes\n///\n/// - Runtime IDs are unique relative to other *currently running* runtimes.\n/// When a runtime completes, the same ID may be used for another runtime.\n/// - Runtime IDs are *not* sequential, and do not indicate the order in which\n/// runtimes are started or any other data.\n/// - The runtime ID of the currently running task can be obtained from the\n/// Handle.\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use tokio::runtime::Handle;\n///\n/// #[tokio::main(flavor = \"multi_thread\", worker_threads = 4)]\n/// async fn main() {\n/// println!(\"Current runtime id: {}\", Handle::current().id());\n/// }\n/// # }\n/// ```\n#[derive(Clone, Copy, Debug, Hash, Eq, PartialEq)]\npub struct Id(NonZeroU64);\n\nimpl Id {\n pub(crate) fn new(integer: impl Into) -> Self {\n Self(integer.into())\n }\n}\n\nimpl fmt::Display for Id {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.0.fmt(f)\n }\n}\n" }, { "path": "tokio/src/runtime/io/driver/signal.rs", "content": "use super::{Driver, Handle, TOKEN_SIGNAL};\n\nuse std::io;\n\nimpl Handle {\n pub(crate) fn register_signal_receiver(\n &self,\n receiver: &mut mio::net::UnixStream,\n ) -> io::Result<()> {\n self.registry\n .register(receiver, TOKEN_SIGNAL, mio::Interest::READABLE)?;\n Ok(())\n }\n}\n\nimpl Driver {\n pub(crate) fn consume_signal_ready(&mut self) -> bool {\n let ret = self.signal_ready;\n self.signal_ready = false;\n ret\n }\n}\n" }, { "path": "tokio/src/runtime/io/driver/uring.rs", "content": "use io_uring::{squeue::Entry, IoUring, Probe};\nuse mio::unix::SourceFd;\nuse slab::Slab;\n\nuse crate::runtime::driver::op::{Cancellable, Lifecycle};\nuse crate::{io::Interest, loom::sync::Mutex};\n\nuse super::{Handle, TOKEN_WAKEUP};\n\nuse std::os::fd::{AsRawFd, RawFd};\nuse std::{io, mem, task::Waker};\n\nconst DEFAULT_RING_SIZE: u32 = 256;\n\npub(crate) struct UringContext {\n pub(crate) uring: Option,\n pub(crate) ops: slab::Slab,\n}\n\nimpl UringContext {\n pub(crate) fn new() -> Self {\n Self {\n ops: Slab::new(),\n uring: None,\n }\n }\n\n pub(crate) fn ring(&self) -> &io_uring::IoUring {\n self.uring.as_ref().expect(\"io_uring not initialized\")\n }\n\n pub(crate) fn ring_mut(&mut self) -> &mut io_uring::IoUring {\n self.uring.as_mut().expect(\"io_uring not initialized\")\n }\n\n /// Perform `io_uring_setup` system call, and Returns true if this\n /// actually initialized the io_uring.\n ///\n /// If the machine doesn't support io_uring, then this will return an\n /// `ENOSYS` error.\n pub(crate) fn try_init(&mut self, probe: &mut Probe) -> io::Result {\n if self.uring.is_some() {\n // Already initialized.\n return Ok(false);\n }\n\n let uring = IoUring::new(DEFAULT_RING_SIZE)?;\n\n match uring.submitter().register_probe(probe) {\n Ok(_) => {}\n Err(e) if e.raw_os_error() == Some(libc::EINVAL) => {\n // The kernel does not support IORING_REGISTER_PROBE.\n return Err(io::Error::from_raw_os_error(libc::ENOSYS));\n }\n Err(e) => return Err(e),\n }\n\n self.uring.replace(uring);\n\n Ok(true)\n }\n\n pub(crate) fn dispatch_completions(&mut self) {\n let ops = &mut self.ops;\n let Some(mut uring) = self.uring.take() else {\n // Uring is not initialized yet.\n return;\n };\n\n let cq = uring.completion();\n\n for cqe in cq {\n let idx = cqe.user_data() as usize;\n\n match ops.get_mut(idx) {\n Some(Lifecycle::Waiting(waker)) => {\n waker.wake_by_ref();\n *ops.get_mut(idx).unwrap() = Lifecycle::Completed(cqe);\n }\n Some(Lifecycle::Cancelled(_)) => {\n // Op future was cancelled, so we discard the result.\n // We just remove the entry from the slab.\n ops.remove(idx);\n }\n Some(other) => {\n panic!(\"unexpected lifecycle for slot {idx}: {other:?}\");\n }\n None => {\n panic!(\"no op at index {idx}\");\n }\n }\n }\n\n self.uring.replace(uring);\n\n // `cq`'s drop gets called here, updating the latest head pointer\n }\n\n pub(crate) fn submit(&mut self) -> io::Result<()> {\n loop {\n // Errors from io_uring_enter: https://man7.org/linux/man-pages/man2/io_uring_enter.2.html#ERRORS\n match self.ring().submit() {\n Ok(_) => {\n return Ok(());\n }\n\n // If the submission queue is full, we dispatch completions and try again.\n Err(ref e) if e.raw_os_error() == Some(libc::EBUSY) => {\n self.dispatch_completions();\n }\n // For other errors, we currently return the error as is.\n Err(e) => {\n return Err(e);\n }\n }\n }\n }\n\n pub(crate) fn remove_op(&mut self, index: usize) -> Lifecycle {\n self.ops.remove(index)\n }\n}\n\n/// Drop the driver, cancelling any in-progress ops and waiting for them to terminate.\nimpl Drop for UringContext {\n fn drop(&mut self) {\n if self.uring.is_none() {\n // Uring is not initialized or not supported.\n return;\n }\n\n // Make sure we flush the submission queue before dropping the driver.\n while !self.ring_mut().submission().is_empty() {\n self.submit().expect(\"Internal error when dropping driver\");\n }\n\n let mut ops = std::mem::take(&mut self.ops);\n\n // Remove all completed ops since we don't need to wait for them.\n ops.retain(|_, lifecycle| !matches!(lifecycle, Lifecycle::Completed(_)));\n\n while !ops.is_empty() {\n // Wait until at least one completion is available.\n self.ring_mut()\n .submit_and_wait(1)\n .expect(\"Internal error when dropping driver\");\n\n for cqe in self.ring_mut().completion() {\n let idx = cqe.user_data() as usize;\n ops.remove(idx);\n }\n }\n }\n}\n\nimpl Handle {\n fn add_uring_source(&self, uringfd: RawFd) -> io::Result<()> {\n let mut source = SourceFd(&uringfd);\n self.registry\n .register(&mut source, TOKEN_WAKEUP, Interest::READABLE.to_mio())\n }\n\n pub(crate) fn get_uring(&self) -> &Mutex {\n &self.uring_context\n }\n\n /// Check if the io_uring context is initialized. If not, it will try to initialize it.\n /// Then, check if the provided opcode is supported.\n ///\n /// If both the context initialization succeeds and the opcode is supported,\n /// this returns `Ok(true)`.\n /// If either io_uring is unsupported or the opcode is unsupported,\n /// this returns `Ok(false)`.\n /// An error is returned if an io_uring syscall returns an unexpected error value.\n ///\n /// TODO: This would like to be a synchronous function,\n /// but we require `OnceLock::get_or_try_init`.\n /// \n pub(crate) async fn check_and_init(&self, opcode: u8) -> io::Result {\n let probe = self\n .uring_probe\n .get_or_try_init(|| async {\n let mut probe = Probe::new();\n match self.try_init(&mut probe) {\n Ok(()) => Ok(Some(probe)),\n // If the system doesn't support io_uring, we set the probe to `None`.\n Err(e) if e.raw_os_error() == Some(libc::ENOSYS) => Ok(None),\n // If we get EPERM, io-uring syscalls may be blocked (for example, by seccomp).\n // In this case, we try to fall back to spawn_blocking for this and future operations.\n // See also: https://github.com/tokio-rs/tokio/issues/7691\n Err(e) if e.raw_os_error() == Some(libc::EPERM) => Ok(None),\n // For other system errors, we just return it.\n Err(e) => Err(e),\n }\n })\n .await?;\n\n Ok(probe\n .as_ref()\n .is_some_and(|probe| probe.is_supported(opcode)))\n }\n\n /// Initialize the io_uring context if it hasn't been initialized yet.\n fn try_init(&self, probe: &mut Probe) -> io::Result<()> {\n let mut guard = self.get_uring().lock();\n if guard.try_init(probe)? {\n self.add_uring_source(guard.ring().as_raw_fd())?;\n }\n\n Ok(())\n }\n\n /// Register an operation with the io_uring.\n ///\n /// If this is the first io_uring operation, it will also initialize the io_uring context.\n /// If io_uring isn't supported, this function returns an `ENOSYS` error, so the caller can\n /// perform custom handling, such as falling back to an alternative mechanism.\n ///\n /// # Safety\n ///\n /// Callers must ensure that parameters of the entry (such as buffer) are valid and will\n /// be valid for the entire duration of the operation, otherwise it may cause memory problems.\n pub(crate) unsafe fn register_op(&self, entry: Entry, waker: Waker) -> io::Result {\n assert!(self.uring_probe.initialized());\n\n // Uring is initialized.\n\n let mut guard = self.get_uring().lock();\n let ctx = &mut *guard;\n let index = ctx.ops.insert(Lifecycle::Waiting(waker));\n let entry = entry.user_data(index as u64);\n\n let submit_or_remove = |ctx: &mut UringContext| -> io::Result<()> {\n if let Err(e) = ctx.submit() {\n // Submission failed, remove the entry from the slab and return the error\n ctx.remove_op(index);\n return Err(e);\n }\n Ok(())\n };\n\n // SAFETY: entry is valid for the entire duration of the operation\n while unsafe { ctx.ring_mut().submission().push(&entry).is_err() } {\n // If the submission queue is full, flush it to the kernel\n submit_or_remove(ctx)?;\n }\n\n // Ensure that the completion queue is not full before submitting the entry.\n while ctx.ring_mut().completion().is_full() {\n ctx.dispatch_completions();\n }\n\n // Note: For now, we submit the entry immediately without utilizing batching.\n submit_or_remove(ctx)?;\n\n Ok(index)\n }\n\n pub(crate) fn cancel_op(&self, index: usize, data: Option) {\n let mut guard = self.get_uring().lock();\n let ctx = &mut *guard;\n let ops = &mut ctx.ops;\n let Some(lifecycle) = ops.get_mut(index) else {\n // The corresponding index doesn't exist anymore, so this Op is already complete.\n return;\n };\n\n // This Op will be cancelled. Here, we don't remove the lifecycle from the slab to keep\n // uring data alive until the operation completes.\n\n let cancel_data = data.expect(\"Data should be present\").cancel();\n match mem::replace(lifecycle, Lifecycle::Cancelled(cancel_data)) {\n Lifecycle::Submitted | Lifecycle::Waiting(_) => (),\n // The driver saw the completion, but it was never polled.\n Lifecycle::Completed(_) => {\n // We can safely remove the entry from the slab, as it has already been completed.\n ops.remove(index);\n }\n prev => panic!(\"Unexpected state: {prev:?}\"),\n };\n }\n}\n" }, { "path": "tokio/src/runtime/io/driver.rs", "content": "// Signal handling\ncfg_signal_internal_and_unix! {\n mod signal;\n}\ncfg_io_uring! {\n mod uring;\n use uring::UringContext;\n use crate::sync::OnceCell;\n}\n\nuse crate::io::interest::Interest;\nuse crate::io::ready::Ready;\nuse crate::loom::sync::Mutex;\nuse crate::runtime::driver;\nuse crate::runtime::io::registration_set;\nuse crate::runtime::io::{IoDriverMetrics, RegistrationSet, ScheduledIo};\n\nuse mio::event::Source;\nuse std::fmt;\nuse std::io;\nuse std::sync::Arc;\nuse std::time::Duration;\n\n/// I/O driver, backed by Mio.\npub(crate) struct Driver {\n /// True when an event with the signal token is received\n signal_ready: bool,\n\n /// Reuse the `mio::Events` value across calls to poll.\n events: mio::Events,\n\n /// The system event queue.\n poll: mio::Poll,\n}\n\n/// A reference to an I/O driver.\npub(crate) struct Handle {\n /// Registers I/O resources.\n registry: mio::Registry,\n\n /// Tracks all registrations\n registrations: RegistrationSet,\n\n /// State that should be synchronized\n synced: Mutex,\n\n /// Used to wake up the reactor from a call to `turn`.\n /// Not supported on `Wasi` due to lack of threading support.\n #[cfg(not(target_os = \"wasi\"))]\n waker: mio::Waker,\n\n pub(crate) metrics: IoDriverMetrics,\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n pub(crate) uring_context: Mutex,\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n pub(crate) uring_probe: OnceCell>,\n}\n\n#[derive(Debug)]\npub(crate) struct ReadyEvent {\n pub(super) tick: u8,\n pub(crate) ready: Ready,\n pub(super) is_shutdown: bool,\n}\n\ncfg_net_unix!(\n impl ReadyEvent {\n pub(crate) fn with_ready(&self, ready: Ready) -> Self {\n Self {\n ready,\n tick: self.tick,\n is_shutdown: self.is_shutdown,\n }\n }\n }\n);\n\n#[derive(Debug, Eq, PartialEq, Clone, Copy)]\npub(super) enum Direction {\n Read,\n Write,\n}\n\npub(super) enum Tick {\n Set,\n Clear(u8),\n}\n\nconst TOKEN_WAKEUP: mio::Token = mio::Token(0);\nconst TOKEN_SIGNAL: mio::Token = mio::Token(1);\n\nfn _assert_kinds() {\n fn _assert() {}\n\n _assert::();\n}\n\n// ===== impl Driver =====\n\nimpl Driver {\n /// Creates a new event loop, returning any error that happened during the\n /// creation.\n pub(crate) fn new(nevents: usize) -> io::Result<(Driver, Handle)> {\n let poll = mio::Poll::new()?;\n #[cfg(not(target_os = \"wasi\"))]\n let waker = mio::Waker::new(poll.registry(), TOKEN_WAKEUP)?;\n let registry = poll.registry().try_clone()?;\n\n let driver = Driver {\n signal_ready: false,\n events: mio::Events::with_capacity(nevents),\n poll,\n };\n\n let (registrations, synced) = RegistrationSet::new();\n\n let handle = Handle {\n registry,\n registrations,\n synced: Mutex::new(synced),\n #[cfg(not(target_os = \"wasi\"))]\n waker,\n metrics: IoDriverMetrics::default(),\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n uring_context: Mutex::new(UringContext::new()),\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n uring_probe: OnceCell::new(),\n };\n\n Ok((driver, handle))\n }\n\n pub(crate) fn park(&mut self, rt_handle: &driver::Handle) {\n let handle = rt_handle.io();\n self.turn(handle, None);\n }\n\n pub(crate) fn park_timeout(&mut self, rt_handle: &driver::Handle, duration: Duration) {\n let handle = rt_handle.io();\n self.turn(handle, Some(duration));\n }\n\n pub(crate) fn shutdown(&mut self, rt_handle: &driver::Handle) {\n let handle = rt_handle.io();\n let ios = handle.registrations.shutdown(&mut handle.synced.lock());\n\n // `shutdown()` must be called without holding the lock.\n for io in ios {\n io.shutdown();\n }\n }\n\n fn turn(&mut self, handle: &Handle, max_wait: Option) {\n debug_assert!(!handle.registrations.is_shutdown(&handle.synced.lock()));\n\n handle.release_pending_registrations();\n\n let events = &mut self.events;\n\n // Block waiting for an event to happen, peeling out how many events\n // happened.\n match self.poll.poll(events, max_wait) {\n Ok(()) => {}\n Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {}\n #[cfg(target_os = \"wasi\")]\n Err(e) if e.kind() == io::ErrorKind::InvalidInput => {\n // In case of wasm32_wasi this error happens, when trying to poll without subscriptions\n // just return from the park, as there would be nothing, which wakes us up.\n }\n Err(e) => panic!(\"unexpected error when polling the I/O driver: {e:?}\"),\n }\n\n // Process all the events that came in, dispatching appropriately\n let mut ready_count = 0;\n for event in events.iter() {\n let token = event.token();\n\n if token == TOKEN_WAKEUP {\n // Nothing to do, the event is used to unblock the I/O driver\n } else if token == TOKEN_SIGNAL {\n self.signal_ready = true;\n } else {\n let ready = Ready::from_mio(event);\n let ptr = super::EXPOSE_IO.from_exposed_addr(token.0);\n\n // Safety: we ensure that the pointers used as tokens are not freed\n // until they are both deregistered from mio **and** we know the I/O\n // driver is not concurrently polling. The I/O driver holds ownership of\n // an `Arc` so we can safely cast this to a ref.\n let io: &ScheduledIo = unsafe { &*ptr };\n\n io.set_readiness(Tick::Set, |curr| curr | ready);\n io.wake(ready);\n\n ready_count += 1;\n }\n }\n\n #[cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\",\n ))]\n {\n let mut guard = handle.get_uring().lock();\n let ctx = &mut *guard;\n ctx.dispatch_completions();\n }\n\n handle.metrics.incr_ready_count_by(ready_count);\n }\n}\n\nimpl fmt::Debug for Driver {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(f, \"Driver\")\n }\n}\n\nimpl Handle {\n /// Forces a reactor blocked in a call to `turn` to wakeup, or otherwise\n /// makes the next call to `turn` return immediately.\n ///\n /// This method is intended to be used in situations where a notification\n /// needs to otherwise be sent to the main reactor. If the reactor is\n /// currently blocked inside of `turn` then it will wake up and soon return\n /// after this method has been called. If the reactor is not currently\n /// blocked in `turn`, then the next call to `turn` will not block and\n /// return immediately.\n pub(crate) fn unpark(&self) {\n #[cfg(not(target_os = \"wasi\"))]\n self.waker.wake().expect(\"failed to wake I/O driver\");\n }\n\n /// Registers an I/O resource with the reactor for a given `mio::Ready` state.\n ///\n /// The registration token is returned.\n pub(super) fn add_source(\n &self,\n source: &mut impl mio::event::Source,\n interest: Interest,\n ) -> io::Result> {\n let scheduled_io = self.registrations.allocate(&mut self.synced.lock())?;\n let token = scheduled_io.token();\n\n // we should remove the `scheduled_io` from the `registrations` set if registering\n // the `source` with the OS fails. Otherwise it will leak the `scheduled_io`.\n if let Err(e) = self.registry.register(source, token, interest.to_mio()) {\n // safety: `scheduled_io` is part of the `registrations` set.\n unsafe {\n self.registrations\n .remove(&mut self.synced.lock(), &scheduled_io)\n };\n\n return Err(e);\n }\n\n // TODO: move this logic to `RegistrationSet` and use a `CountedLinkedList`\n self.metrics.incr_fd_count();\n\n Ok(scheduled_io)\n }\n\n /// Deregisters an I/O resource from the reactor.\n pub(super) fn deregister_source(\n &self,\n registration: &Arc,\n source: &mut impl Source,\n ) -> io::Result<()> {\n // Deregister the source with the OS poller **first**\n // Cleanup ALWAYS happens\n let os_result = self.registry.deregister(source);\n\n if self\n .registrations\n .deregister(&mut self.synced.lock(), registration)\n {\n self.unpark();\n }\n\n self.metrics.dec_fd_count();\n\n os_result // Return error after cleanup\n }\n\n fn release_pending_registrations(&self) {\n if self.registrations.needs_release() {\n self.registrations.release(&mut self.synced.lock());\n }\n }\n}\n\nimpl fmt::Debug for Handle {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(f, \"Handle\")\n }\n}\n\nimpl Direction {\n pub(super) fn mask(self) -> Ready {\n match self {\n Direction::Read => Ready::READABLE | Ready::READ_CLOSED,\n Direction::Write => Ready::WRITABLE | Ready::WRITE_CLOSED,\n }\n }\n}\n" }, { "path": "tokio/src/runtime/io/metrics.rs", "content": "//! This file contains mocks of the metrics types used in the I/O driver.\n//!\n//! The reason these mocks don't live in `src/runtime/mock.rs` is because\n//! these need to be available in the case when `net` is enabled but\n//! `rt` is not.\n\ncfg_not_rt_and_metrics_and_net! {\n #[derive(Default)]\n pub(crate) struct IoDriverMetrics {}\n\n impl IoDriverMetrics {\n pub(crate) fn incr_fd_count(&self) {}\n pub(crate) fn dec_fd_count(&self) {}\n pub(crate) fn incr_ready_count_by(&self, _amt: u64) {}\n }\n}\n\ncfg_net! {\n cfg_rt! {\n cfg_unstable_metrics! {\n pub(crate) use crate::runtime::IoDriverMetrics;\n }\n }\n}\n" }, { "path": "tokio/src/runtime/io/mod.rs", "content": "#![cfg_attr(\n not(all(feature = \"rt\", feature = \"net\", feature = \"io-uring\", tokio_unstable)),\n allow(dead_code)\n)]\nmod driver;\nuse driver::{Direction, Tick};\npub(crate) use driver::{Driver, Handle, ReadyEvent};\n\nmod registration;\npub(crate) use registration::Registration;\n\nmod registration_set;\nuse registration_set::RegistrationSet;\n\nmod scheduled_io;\nuse scheduled_io::ScheduledIo;\n\nmod metrics;\nuse metrics::IoDriverMetrics;\n\nuse crate::util::ptr_expose::PtrExposeDomain;\nstatic EXPOSE_IO: PtrExposeDomain = PtrExposeDomain::new();\n" }, { "path": "tokio/src/runtime/io/registration.rs", "content": "#![cfg_attr(not(feature = \"net\"), allow(dead_code))]\n\nuse crate::io::interest::Interest;\nuse crate::runtime::io::{Direction, Handle, ReadyEvent, ScheduledIo};\nuse crate::runtime::scheduler;\n\nuse mio::event::Source;\nuse std::io;\nuse std::sync::Arc;\nuse std::task::{ready, Context, Poll};\n\ncfg_io_driver! {\n /// Associates an I/O resource with the reactor instance that drives it.\n ///\n /// A registration represents an I/O resource registered with a Reactor such\n /// that it will receive task notifications on readiness. This is the lowest\n /// level API for integrating with a reactor.\n ///\n /// The association between an I/O resource is made by calling\n /// [`new_with_interest_and_handle`].\n /// Once the association is established, it remains established until the\n /// registration instance is dropped.\n ///\n /// A registration instance represents two separate readiness streams. One\n /// for the read readiness and one for write readiness. These streams are\n /// independent and can be consumed from separate tasks.\n ///\n /// **Note**: while `Registration` is `Sync`, the caller must ensure that\n /// there are at most two tasks that use a registration instance\n /// concurrently. One task for [`poll_read_ready`] and one task for\n /// [`poll_write_ready`]. While violating this requirement is \"safe\" from a\n /// Rust memory safety point of view, it will result in unexpected behavior\n /// in the form of lost notifications and tasks hanging.\n ///\n /// ## Platform-specific events\n ///\n /// `Registration` also allows receiving platform-specific `mio::Ready`\n /// events. These events are included as part of the read readiness event\n /// stream. The write readiness event stream is only for `Ready::writable()`\n /// events.\n ///\n /// [`new_with_interest_and_handle`]: method@Self::new_with_interest_and_handle\n /// [`poll_read_ready`]: method@Self::poll_read_ready`\n /// [`poll_write_ready`]: method@Self::poll_write_ready`\n #[derive(Debug)]\n pub(crate) struct Registration {\n /// Handle to the associated runtime.\n ///\n /// TODO: this can probably be moved into `ScheduledIo`.\n handle: scheduler::Handle,\n\n /// Reference to state stored by the driver.\n shared: Arc,\n }\n}\n\nunsafe impl Send for Registration {}\nunsafe impl Sync for Registration {}\n\n// ===== impl Registration =====\n\nimpl Registration {\n /// Registers the I/O resource with the reactor for the provided handle, for\n /// a specific `Interest`. This does not add `hup` or `error` so if you are\n /// interested in those states, you will need to add them to the readiness\n /// state passed to this function.\n ///\n /// # Return\n ///\n /// - `Ok` if the registration happened successfully\n /// - `Err` if an error was encountered during registration\n #[track_caller]\n pub(crate) fn new_with_interest_and_handle(\n io: &mut impl Source,\n interest: Interest,\n handle: scheduler::Handle,\n ) -> io::Result {\n let shared = handle.driver().io().add_source(io, interest)?;\n\n Ok(Registration { handle, shared })\n }\n\n /// Deregisters the I/O resource from the reactor it is associated with.\n ///\n /// This function must be called before the I/O resource associated with the\n /// registration is dropped.\n ///\n /// Note that deregistering does not guarantee that the I/O resource can be\n /// registered with a different reactor. Some I/O resource types can only be\n /// associated with a single reactor instance for their lifetime.\n ///\n /// # Return\n ///\n /// If the deregistration was successful, `Ok` is returned. Any calls to\n /// `Reactor::turn` that happen after a successful call to `deregister` will\n /// no longer result in notifications getting sent for this registration.\n ///\n /// `Err` is returned if an error is encountered.\n pub(crate) fn deregister(&mut self, io: &mut impl Source) -> io::Result<()> {\n self.handle().deregister_source(&self.shared, io)\n }\n\n pub(crate) fn clear_readiness(&self, event: ReadyEvent) {\n self.shared.clear_readiness(event);\n }\n\n // Uses the poll path, requiring the caller to ensure mutual exclusion for\n // correctness. Only the last task to call this function is notified.\n pub(crate) fn poll_read_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.poll_ready(cx, Direction::Read)\n }\n\n // Uses the poll path, requiring the caller to ensure mutual exclusion for\n // correctness. Only the last task to call this function is notified.\n pub(crate) fn poll_write_ready(&self, cx: &mut Context<'_>) -> Poll> {\n self.poll_ready(cx, Direction::Write)\n }\n\n // Uses the poll path, requiring the caller to ensure mutual exclusion for\n // correctness. Only the last task to call this function is notified.\n #[cfg(not(target_os = \"wasi\"))]\n pub(crate) fn poll_read_io(\n &self,\n cx: &mut Context<'_>,\n f: impl FnMut() -> io::Result,\n ) -> Poll> {\n self.poll_io(cx, Direction::Read, f)\n }\n\n // Uses the poll path, requiring the caller to ensure mutual exclusion for\n // correctness. Only the last task to call this function is notified.\n pub(crate) fn poll_write_io(\n &self,\n cx: &mut Context<'_>,\n f: impl FnMut() -> io::Result,\n ) -> Poll> {\n self.poll_io(cx, Direction::Write, f)\n }\n\n /// Polls for events on the I/O resource's `direction` readiness stream.\n ///\n /// If called with a task context, notify the task when a new event is\n /// received.\n fn poll_ready(\n &self,\n cx: &mut Context<'_>,\n direction: Direction,\n ) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n let ev = ready!(self.shared.poll_readiness(cx, direction));\n\n if ev.is_shutdown {\n return Poll::Ready(Err(gone()));\n }\n\n coop.made_progress();\n Poll::Ready(Ok(ev))\n }\n\n fn poll_io(\n &self,\n cx: &mut Context<'_>,\n direction: Direction,\n mut f: impl FnMut() -> io::Result,\n ) -> Poll> {\n loop {\n let ev = ready!(self.poll_ready(cx, direction))?;\n\n match f() {\n Ok(ret) => {\n return Poll::Ready(Ok(ret));\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.clear_readiness(ev);\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n\n pub(crate) fn try_io(\n &self,\n interest: Interest,\n f: impl FnOnce() -> io::Result,\n ) -> io::Result {\n let ev = self.shared.ready_event(interest);\n\n // Don't attempt the operation if the resource is not ready.\n if ev.ready.is_empty() {\n return Err(io::ErrorKind::WouldBlock.into());\n }\n\n match f() {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.clear_readiness(ev);\n Err(io::ErrorKind::WouldBlock.into())\n }\n res => res,\n }\n }\n\n pub(crate) async fn readiness(&self, interest: Interest) -> io::Result {\n let ev = self.shared.readiness(interest).await;\n\n if ev.is_shutdown {\n return Err(gone());\n }\n\n Ok(ev)\n }\n\n pub(crate) async fn async_io(\n &self,\n interest: Interest,\n mut f: impl FnMut() -> io::Result,\n ) -> io::Result {\n loop {\n let event = self.readiness(interest).await?;\n\n let coop = std::future::poll_fn(crate::task::coop::poll_proceed).await;\n\n match f() {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n self.clear_readiness(event);\n }\n x => {\n coop.made_progress();\n return x;\n }\n }\n }\n }\n\n fn handle(&self) -> &Handle {\n self.handle.driver().io()\n }\n}\n\nimpl Drop for Registration {\n fn drop(&mut self) {\n // It is possible for a cycle to be created between wakers stored in\n // `ScheduledIo` instances and `Arc`. To break this\n // cycle, wakers are cleared. This is an imperfect solution as it is\n // possible to store a `Registration` in a waker. In this case, the\n // cycle would remain.\n //\n // See tokio-rs/tokio#3481 for more details.\n self.shared.clear_wakers();\n }\n}\n\nfn gone() -> io::Error {\n io::Error::new(\n io::ErrorKind::Other,\n crate::util::error::RUNTIME_SHUTTING_DOWN_ERROR,\n )\n}\n" }, { "path": "tokio/src/runtime/io/registration_set.rs", "content": "use crate::loom::sync::atomic::AtomicUsize;\nuse crate::runtime::io::ScheduledIo;\nuse crate::util::linked_list::{self, LinkedList};\n\nuse std::io;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::{Acquire, Release};\nuse std::sync::Arc;\n\n// Kind of arbitrary, but buffering 16 `ScheduledIo`s doesn't seem like much\nconst NOTIFY_AFTER: usize = 16;\n\npub(super) struct RegistrationSet {\n num_pending_release: AtomicUsize,\n}\n\npub(super) struct Synced {\n // True when the I/O driver shutdown. At this point, no more registrations\n // should be added to the set.\n is_shutdown: bool,\n\n // List of all registrations tracked by the set\n registrations: LinkedList, ScheduledIo>,\n\n // Registrations that are pending drop. When a `Registration` is dropped, it\n // stores its `ScheduledIo` in this list. The I/O driver is responsible for\n // dropping it. This ensures the `ScheduledIo` is not freed while it can\n // still be included in an I/O event.\n pending_release: Vec>,\n}\n\nimpl RegistrationSet {\n pub(super) fn new() -> (RegistrationSet, Synced) {\n let set = RegistrationSet {\n num_pending_release: AtomicUsize::new(0),\n };\n\n let synced = Synced {\n is_shutdown: false,\n registrations: LinkedList::new(),\n pending_release: Vec::with_capacity(NOTIFY_AFTER),\n };\n\n (set, synced)\n }\n\n pub(super) fn is_shutdown(&self, synced: &Synced) -> bool {\n synced.is_shutdown\n }\n\n /// Returns `true` if there are registrations that need to be released\n pub(super) fn needs_release(&self) -> bool {\n self.num_pending_release.load(Acquire) != 0\n }\n\n pub(super) fn allocate(&self, synced: &mut Synced) -> io::Result> {\n if synced.is_shutdown {\n return Err(io::Error::new(\n io::ErrorKind::Other,\n crate::util::error::RUNTIME_SHUTTING_DOWN_ERROR,\n ));\n }\n\n let ret = Arc::new(ScheduledIo::default());\n\n // Push a ref into the list of all resources.\n synced.registrations.push_front(ret.clone());\n\n Ok(ret)\n }\n\n // Returns `true` if the caller should unblock the I/O driver to purge\n // registrations pending release.\n pub(super) fn deregister(&self, synced: &mut Synced, registration: &Arc) -> bool {\n synced.pending_release.push(registration.clone());\n\n let len = synced.pending_release.len();\n self.num_pending_release.store(len, Release);\n\n len == NOTIFY_AFTER\n }\n\n pub(super) fn shutdown(&self, synced: &mut Synced) -> Vec> {\n if synced.is_shutdown {\n return vec![];\n }\n\n synced.is_shutdown = true;\n synced.pending_release.clear();\n\n // Building a vec of all outstanding I/O handles could be expensive, but\n // this is the shutdown operation. In theory, shutdowns should be\n // \"clean\" with no outstanding I/O resources. Even if it is slow, we\n // aren't optimizing for shutdown.\n let mut ret = vec![];\n\n while let Some(io) = synced.registrations.pop_back() {\n ret.push(io);\n }\n\n ret\n }\n\n pub(super) fn release(&self, synced: &mut Synced) {\n let pending = std::mem::take(&mut synced.pending_release);\n\n for io in pending {\n // safety: the registration is part of our list\n unsafe { self.remove(synced, &io) }\n }\n\n self.num_pending_release.store(0, Release);\n }\n\n // This function is marked as unsafe, because the caller must make sure that\n // `io` is part of the registration set.\n pub(super) unsafe fn remove(&self, synced: &mut Synced, io: &Arc) {\n // SAFETY: Pointers into an Arc are never null.\n let io = unsafe { NonNull::new_unchecked(Arc::as_ptr(io).cast_mut()) };\n\n super::EXPOSE_IO.unexpose_provenance(io.as_ptr());\n // SAFETY: the caller guarantees that `io` is part of this list.\n let _ = unsafe { synced.registrations.remove(io) };\n }\n}\n\n// Safety: `Arc` pins the inner data\nunsafe impl linked_list::Link for Arc {\n type Handle = Arc;\n type Target = ScheduledIo;\n\n fn as_raw(handle: &Self::Handle) -> NonNull {\n // safety: Arc::as_ptr never returns null\n unsafe { NonNull::new_unchecked(Arc::as_ptr(handle) as *mut _) }\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Arc {\n // safety: the linked list currently owns a ref count\n unsafe { Arc::from_raw(ptr.as_ptr() as *const _) }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n // safety: `target.as_ref().linked_list_pointers` is a `UnsafeCell` that\n // always returns a non-null pointer.\n unsafe { NonNull::new_unchecked(target.as_ref().linked_list_pointers.get()) }\n }\n}\n" }, { "path": "tokio/src/runtime/io/scheduled_io.rs", "content": "use crate::io::interest::Interest;\nuse crate::io::ready::Ready;\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::Mutex;\nuse crate::runtime::io::{Direction, ReadyEvent, Tick};\nuse crate::util::bit;\nuse crate::util::linked_list::{self, LinkedList};\nuse crate::util::WakeList;\n\nuse std::cell::UnsafeCell;\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire};\nuse std::task::{Context, Poll, Waker};\n\n/// Stored in the I/O driver resource slab.\n#[derive(Debug)]\n// # This struct should be cache padded to avoid false sharing. The cache padding rules are copied\n// from crossbeam-utils/src/cache_padded.rs\n//\n// Starting from Intel's Sandy Bridge, spatial prefetcher is now pulling pairs of 64-byte cache\n// lines at a time, so we have to align to 128 bytes rather than 64.\n//\n// Sources:\n// - https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf\n// - https://github.com/facebook/folly/blob/1b5288e6eea6df074758f877c849b6e73bbb9fbb/folly/lang/Align.h#L107\n//\n// ARM's big.LITTLE architecture has asymmetric cores and \"big\" cores have 128-byte cache line size.\n//\n// Sources:\n// - https://www.mono-project.com/news/2016/09/12/arm64-icache/\n//\n// powerpc64 has 128-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_ppc64x.go#L9\n#[cfg_attr(\n any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n ),\n repr(align(128))\n)]\n// arm, mips, mips64, sparc, and hexagon have 32-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_arm.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mipsle.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips64x.go#L9\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/sparc/include/asm/cache.h#L17\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/hexagon/include/asm/cache.h#L12\n#[cfg_attr(\n any(\n target_arch = \"arm\",\n target_arch = \"mips\",\n target_arch = \"mips64\",\n target_arch = \"sparc\",\n target_arch = \"hexagon\",\n ),\n repr(align(32))\n)]\n// m68k has 16-byte cache line size.\n//\n// Sources:\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/m68k/include/asm/cache.h#L9\n#[cfg_attr(target_arch = \"m68k\", repr(align(16)))]\n// s390x has 256-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_s390x.go#L7\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/s390/include/asm/cache.h#L13\n#[cfg_attr(target_arch = \"s390x\", repr(align(256)))]\n// x86, riscv, wasm, and sparc64 have 64-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/dda2991c2ea0c5914714469c4defc2562a907230/src/internal/cpu/cpu_x86.go#L9\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_wasm.go#L7\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/sparc/include/asm/cache.h#L19\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/riscv/include/asm/cache.h#L10\n//\n// All others are assumed to have 64-byte cache line size.\n#[cfg_attr(\n not(any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n target_arch = \"arm\",\n target_arch = \"mips\",\n target_arch = \"mips64\",\n target_arch = \"sparc\",\n target_arch = \"hexagon\",\n target_arch = \"m68k\",\n target_arch = \"s390x\",\n )),\n repr(align(64))\n)]\npub(crate) struct ScheduledIo {\n pub(super) linked_list_pointers: UnsafeCell>,\n\n /// Packs the resource's readiness and I/O driver latest tick.\n readiness: AtomicUsize,\n\n waiters: Mutex,\n}\n\ntype WaitList = LinkedList::Target>;\n\n#[derive(Debug, Default)]\nstruct Waiters {\n /// List of all current waiters.\n list: WaitList,\n\n /// Waker used for `AsyncRead`.\n reader: Option,\n\n /// Waker used for `AsyncWrite`.\n writer: Option,\n}\n\n#[derive(Debug)]\nstruct Waiter {\n pointers: linked_list::Pointers,\n\n /// The waker for this task.\n waker: Option,\n\n /// The interest this waiter is waiting on.\n interest: Interest,\n\n is_ready: bool,\n\n /// Should never be `Unpin`.\n _p: PhantomPinned,\n}\n\ngenerate_addr_of_methods! {\n impl<> Waiter {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull> {\n &self.pointers\n }\n }\n}\n\n/// Future returned by `readiness()`.\nstruct Readiness<'a> {\n scheduled_io: &'a ScheduledIo,\n\n state: State,\n\n /// Entry in the waiter `LinkedList`.\n waiter: UnsafeCell,\n}\n\nenum State {\n Init,\n Waiting,\n Done,\n}\n\n// The `ScheduledIo::readiness` (`AtomicUsize`) is packed full of goodness.\n//\n// | shutdown | driver tick | readiness |\n// |----------+-------------+-----------|\n// | 1 bit | 15 bits | 16 bits |\n\nconst READINESS: bit::Pack = bit::Pack::least_significant(16);\n\nconst TICK: bit::Pack = READINESS.then(15);\n\nconst SHUTDOWN: bit::Pack = TICK.then(1);\n\n// ===== impl ScheduledIo =====\n\nimpl Default for ScheduledIo {\n fn default() -> ScheduledIo {\n ScheduledIo {\n linked_list_pointers: UnsafeCell::new(linked_list::Pointers::new()),\n readiness: AtomicUsize::new(0),\n waiters: Mutex::new(Waiters::default()),\n }\n }\n}\n\nimpl ScheduledIo {\n pub(crate) fn token(&self) -> mio::Token {\n mio::Token(super::EXPOSE_IO.expose_provenance(self))\n }\n\n /// Invoked when the IO driver is shut down; forces this `ScheduledIo` into a\n /// permanently shutdown state.\n pub(super) fn shutdown(&self) {\n let mask = SHUTDOWN.pack(1, 0);\n self.readiness.fetch_or(mask, AcqRel);\n self.wake(Ready::ALL);\n }\n\n /// Sets the readiness on this `ScheduledIo` by invoking the given closure on\n /// the current value, returning the previous readiness value.\n ///\n /// # Arguments\n /// - `tick`: whether setting the tick or trying to clear readiness for a\n /// specific tick.\n /// - `f`: a closure returning a new readiness value given the previous\n /// readiness.\n pub(super) fn set_readiness(&self, tick_op: Tick, f: impl Fn(Ready) -> Ready) {\n let _ = self.readiness.fetch_update(AcqRel, Acquire, |curr| {\n // If the io driver is shut down, then you are only allowed to clear readiness.\n debug_assert!(SHUTDOWN.unpack(curr) == 0 || matches!(tick_op, Tick::Clear(_)));\n\n const MAX_TICK: usize = TICK.max_value() + 1;\n let tick = TICK.unpack(curr);\n\n let new_tick = match tick_op {\n // Trying to clear readiness with an old event!\n Tick::Clear(t) if tick as u8 != t => return None,\n Tick::Clear(t) => t as usize,\n Tick::Set => tick.wrapping_add(1) % MAX_TICK,\n };\n let ready = Ready::from_usize(READINESS.unpack(curr));\n Some(TICK.pack(new_tick, f(ready).as_usize()))\n });\n }\n\n /// Notifies all pending waiters that have registered interest in `ready`.\n ///\n /// There may be many waiters to notify. Waking the pending task **must** be\n /// done from outside of the lock otherwise there is a potential for a\n /// deadlock.\n ///\n /// A stack array of wakers is created and filled with wakers to notify, the\n /// lock is released, and the wakers are notified. Because there may be more\n /// than 32 wakers to notify, if the stack array fills up, the lock is\n /// released, the array is cleared, and the iteration continues.\n pub(super) fn wake(&self, ready: Ready) {\n let mut wakers = WakeList::new();\n\n let mut waiters = self.waiters.lock();\n\n // check for AsyncRead slot\n if ready.is_readable() {\n if let Some(waker) = waiters.reader.take() {\n wakers.push(waker);\n }\n }\n\n // check for AsyncWrite slot\n if ready.is_writable() {\n if let Some(waker) = waiters.writer.take() {\n wakers.push(waker);\n }\n }\n\n 'outer: loop {\n let mut iter = waiters.list.drain_filter(|w| ready.satisfies(w.interest));\n\n while wakers.can_push() {\n match iter.next() {\n Some(waiter) => {\n let waiter = unsafe { &mut *waiter.as_ptr() };\n\n if let Some(waker) = waiter.waker.take() {\n waiter.is_ready = true;\n wakers.push(waker);\n }\n }\n None => {\n break 'outer;\n }\n }\n }\n\n drop(waiters);\n\n wakers.wake_all();\n\n // Acquire the lock again.\n waiters = self.waiters.lock();\n }\n\n // Release the lock before notifying\n drop(waiters);\n\n wakers.wake_all();\n }\n\n pub(super) fn ready_event(&self, interest: Interest) -> ReadyEvent {\n let curr = self.readiness.load(Acquire);\n\n ReadyEvent {\n tick: TICK.unpack(curr) as u8,\n ready: interest.mask() & Ready::from_usize(READINESS.unpack(curr)),\n is_shutdown: SHUTDOWN.unpack(curr) != 0,\n }\n }\n\n /// Polls for readiness events in a given direction.\n ///\n /// These are to support `AsyncRead` and `AsyncWrite` polling methods,\n /// which cannot use the `async fn` version. This uses reserved reader\n /// and writer slots.\n pub(super) fn poll_readiness(\n &self,\n cx: &mut Context<'_>,\n direction: Direction,\n ) -> Poll {\n let curr = self.readiness.load(Acquire);\n\n let ready = direction.mask() & Ready::from_usize(READINESS.unpack(curr));\n let is_shutdown = SHUTDOWN.unpack(curr) != 0;\n\n if ready.is_empty() && !is_shutdown {\n // Update the task info\n let mut waiters = self.waiters.lock();\n let waker = match direction {\n Direction::Read => &mut waiters.reader,\n Direction::Write => &mut waiters.writer,\n };\n\n // Avoid cloning the waker if one is already stored that matches the\n // current task.\n match waker {\n Some(waker) => waker.clone_from(cx.waker()),\n None => *waker = Some(cx.waker().clone()),\n }\n\n // Try again, in case the readiness was changed while we were\n // taking the waiters lock\n let curr = self.readiness.load(Acquire);\n let ready = direction.mask() & Ready::from_usize(READINESS.unpack(curr));\n let is_shutdown = SHUTDOWN.unpack(curr) != 0;\n if is_shutdown {\n Poll::Ready(ReadyEvent {\n tick: TICK.unpack(curr) as u8,\n ready: direction.mask(),\n is_shutdown,\n })\n } else if ready.is_empty() {\n Poll::Pending\n } else {\n Poll::Ready(ReadyEvent {\n tick: TICK.unpack(curr) as u8,\n ready,\n is_shutdown,\n })\n }\n } else {\n Poll::Ready(ReadyEvent {\n tick: TICK.unpack(curr) as u8,\n ready,\n is_shutdown,\n })\n }\n }\n\n pub(crate) fn clear_readiness(&self, event: ReadyEvent) {\n // This consumes the current readiness state **except** for closed\n // states. Closed states are excluded because they are final states.\n let mask_no_closed = event.ready - Ready::READ_CLOSED - Ready::WRITE_CLOSED;\n self.set_readiness(Tick::Clear(event.tick), |curr| curr - mask_no_closed);\n }\n\n pub(crate) fn clear_wakers(&self) {\n let mut waiters = self.waiters.lock();\n waiters.reader.take();\n waiters.writer.take();\n }\n}\n\nimpl Drop for ScheduledIo {\n fn drop(&mut self) {\n self.wake(Ready::ALL);\n }\n}\n\nunsafe impl Send for ScheduledIo {}\nunsafe impl Sync for ScheduledIo {}\n\nimpl ScheduledIo {\n /// An async version of `poll_readiness` which uses a linked list of wakers.\n pub(crate) async fn readiness(&self, interest: Interest) -> ReadyEvent {\n self.readiness_fut(interest).await\n }\n\n // This is in a separate function so that the borrow checker doesn't think\n // we are borrowing the `UnsafeCell` possibly over await boundaries.\n //\n // Go figure.\n fn readiness_fut(&self, interest: Interest) -> Readiness<'_> {\n Readiness {\n scheduled_io: self,\n state: State::Init,\n waiter: UnsafeCell::new(Waiter {\n pointers: linked_list::Pointers::new(),\n waker: None,\n is_ready: false,\n interest,\n _p: PhantomPinned,\n }),\n }\n }\n}\n\nunsafe impl linked_list::Link for Waiter {\n type Handle = NonNull;\n type Target = Waiter;\n\n fn as_raw(handle: &NonNull) -> NonNull {\n *handle\n }\n\n unsafe fn from_raw(ptr: NonNull) -> NonNull {\n ptr\n }\n\n unsafe fn pointers(target: NonNull) -> NonNull> {\n unsafe { Waiter::addr_of_pointers(target) }\n }\n}\n\n// ===== impl Readiness =====\n\nimpl Future for Readiness<'_> {\n type Output = ReadyEvent;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n use std::sync::atomic::Ordering::SeqCst;\n\n let (scheduled_io, state, waiter) = {\n // Safety: `Self` is `!Unpin`\n //\n // While we could use `pin_project!` to remove\n // this unsafe block, there are already unsafe blocks here,\n // so it wouldn't significantly ease the mental burden\n // and would actually complicate the code.\n // That's why we didn't use it.\n let me = unsafe { self.get_unchecked_mut() };\n (me.scheduled_io, &mut me.state, &me.waiter)\n };\n\n loop {\n match *state {\n State::Init => {\n // Optimistically check existing readiness\n let curr = scheduled_io.readiness.load(SeqCst);\n let is_shutdown = SHUTDOWN.unpack(curr) != 0;\n\n // Safety: `waiter.interest` never changes\n let interest = unsafe { (*waiter.get()).interest };\n let ready = Ready::from_usize(READINESS.unpack(curr)).intersection(interest);\n\n if !ready.is_empty() || is_shutdown {\n // Currently ready!\n let tick = TICK.unpack(curr) as u8;\n *state = State::Done;\n return Poll::Ready(ReadyEvent {\n tick,\n ready,\n is_shutdown,\n });\n }\n\n // Wasn't ready, take the lock (and check again while locked).\n let mut waiters = scheduled_io.waiters.lock();\n\n let curr = scheduled_io.readiness.load(SeqCst);\n let mut ready = Ready::from_usize(READINESS.unpack(curr));\n let is_shutdown = SHUTDOWN.unpack(curr) != 0;\n\n if is_shutdown {\n ready = Ready::ALL;\n }\n\n let ready = ready.intersection(interest);\n\n if !ready.is_empty() || is_shutdown {\n // Currently ready!\n let tick = TICK.unpack(curr) as u8;\n *state = State::Done;\n return Poll::Ready(ReadyEvent {\n tick,\n ready,\n is_shutdown,\n });\n }\n\n // Not ready even after locked, insert into list...\n\n // Safety: Since the `waiter` is not in the intrusive list yet,\n // we have exclusive access to it. The Mutex ensures\n // that this modification is visible to other threads that\n // acquire the same Mutex.\n let waker = unsafe { &mut (*waiter.get()).waker };\n let old = waker.replace(cx.waker().clone());\n debug_assert!(old.is_none(), \"waker should be None at the first poll\");\n\n // Insert the waiter into the linked list\n //\n // safety: pointers from `UnsafeCell` are never null.\n waiters\n .list\n .push_front(unsafe { NonNull::new_unchecked(waiter.get()) });\n *state = State::Waiting;\n }\n State::Waiting => {\n // Currently in the \"Waiting\" state, implying the caller has\n // a waiter stored in the waiter list (guarded by\n // `notify.waiters`). In order to access the waker fields,\n // we must hold the lock.\n\n let waiters = scheduled_io.waiters.lock();\n\n // Safety: With the lock held, we have exclusive access to\n // the waiter. In other words, `ScheduledIo::wake()`\n // cannot access the waiter concurrently.\n let w = unsafe { &mut *waiter.get() };\n\n if w.is_ready {\n // Our waker has been notified.\n *state = State::Done;\n } else {\n // Update the waker, if necessary.\n w.waker.as_mut().unwrap().clone_from(cx.waker());\n return Poll::Pending;\n }\n\n // Explicit drop of the lock to indicate the scope that the\n // lock is held. Because holding the lock is required to\n // ensure safe access to fields not held within the lock, it\n // is helpful to visualize the scope of the critical\n // section.\n drop(waiters);\n }\n State::Done => {\n let curr = scheduled_io.readiness.load(Acquire);\n let is_shutdown = SHUTDOWN.unpack(curr) != 0;\n\n // The returned tick might be newer than the event\n // which notified our waker. This is ok because the future\n // still didn't return `Poll::Ready`.\n let tick = TICK.unpack(curr) as u8;\n\n // Safety: We don't need to acquire the lock here because\n // 1. `State::Done`` means `waiter` is no longer shared,\n // this means no concurrent access to `waiter` can happen\n // at this point.\n // 2. `waiter.interest` is never changed, this means\n // no side effects need to be synchronized by the lock.\n let interest = unsafe { (*waiter.get()).interest };\n // The readiness state could have been cleared in the meantime,\n // but we allow the returned ready set to be empty.\n let ready = Ready::from_usize(READINESS.unpack(curr)).intersection(interest);\n\n return Poll::Ready(ReadyEvent {\n tick,\n ready,\n is_shutdown,\n });\n }\n }\n }\n }\n}\n\nimpl Drop for Readiness<'_> {\n fn drop(&mut self) {\n let mut waiters = self.scheduled_io.waiters.lock();\n\n // Safety: `waiter` is only ever stored in `waiters`\n unsafe {\n waiters\n .list\n .remove(NonNull::new_unchecked(self.waiter.get()))\n };\n }\n}\n\nunsafe impl Send for Readiness<'_> {}\nunsafe impl Sync for Readiness<'_> {}\n" }, { "path": "tokio/src/runtime/local_runtime/mod.rs", "content": "mod runtime;\n\nmod options;\n\npub use options::LocalOptions;\npub use runtime::LocalRuntime;\npub(super) use runtime::LocalRuntimeScheduler;\n" }, { "path": "tokio/src/runtime/local_runtime/options.rs", "content": "use std::marker::PhantomData;\n\n/// [`LocalRuntime`]-only config options\n///\n/// Currently, there are no such options, but in the future, things like `!Send + !Sync` hooks may\n/// be added.\n///\n/// Use `LocalOptions::default()` to create the default set of options. This type is used with\n/// [`Builder::build_local`].\n///\n/// [`Builder::build_local`]: crate::runtime::Builder::build_local\n/// [`LocalRuntime`]: crate::runtime::LocalRuntime\n#[derive(Default, Debug)]\n#[non_exhaustive]\npub struct LocalOptions {\n /// Marker used to make this !Send and !Sync.\n _phantom: PhantomData<*mut u8>,\n}\n" }, { "path": "tokio/src/runtime/local_runtime/runtime.rs", "content": "#![allow(irrefutable_let_patterns)]\n\nuse crate::runtime::blocking::BlockingPool;\nuse crate::runtime::scheduler::CurrentThread;\nuse crate::runtime::{context, Builder, EnterGuard, Handle, BOX_FUTURE_THRESHOLD};\nuse crate::task::JoinHandle;\n\nuse crate::util::trace::SpawnMeta;\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::mem;\nuse std::time::Duration;\n\n/// A local Tokio runtime.\n///\n/// This runtime is capable of driving tasks which are not `Send + Sync` without the use of a\n/// `LocalSet`, and thus supports `spawn_local` without the need for a `LocalSet` context.\n///\n/// This runtime cannot be moved between threads or driven from different threads.\n///\n/// This runtime is incompatible with `LocalSet`. You should not attempt to drive a `LocalSet` within a\n/// `LocalRuntime`.\n///\n/// Currently, this runtime supports one flavor, which is internally identical to `current_thread`,\n/// save for the aforementioned differences related to `spawn_local`.\n///\n/// For more general information on how to use runtimes, see the [module] docs.\n///\n/// [runtime]: crate::runtime::Runtime\n/// [module]: crate::runtime\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(tokio_unstable)))]\npub struct LocalRuntime {\n /// Task scheduler\n scheduler: LocalRuntimeScheduler,\n\n /// Handle to runtime, also contains driver handles\n handle: Handle,\n\n /// Blocking pool handle, used to signal shutdown\n blocking_pool: BlockingPool,\n\n /// Marker used to make this !Send and !Sync.\n _phantom: PhantomData<*mut u8>,\n}\n\n/// The runtime scheduler is always a `current_thread` scheduler right now.\n#[derive(Debug)]\npub(crate) enum LocalRuntimeScheduler {\n /// Execute all tasks on the current-thread.\n CurrentThread(CurrentThread),\n}\n\nimpl LocalRuntime {\n pub(crate) fn from_parts(\n scheduler: LocalRuntimeScheduler,\n handle: Handle,\n blocking_pool: BlockingPool,\n ) -> LocalRuntime {\n LocalRuntime {\n scheduler,\n handle,\n blocking_pool,\n _phantom: Default::default(),\n }\n }\n\n /// Creates a new local runtime instance with default configuration values.\n ///\n /// This results in the scheduler, I/O driver, and time driver being\n /// initialized.\n ///\n /// When a more complex configuration is necessary, the [runtime builder] may be used.\n ///\n /// See [module level][mod] documentation for more details.\n ///\n /// # Examples\n ///\n /// Creating a new `LocalRuntime` with default configuration values.\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n ///\n /// let rt = LocalRuntime::new()\n /// .unwrap();\n ///\n /// // Use the runtime...\n /// ```\n ///\n /// [mod]: crate::runtime\n /// [runtime builder]: crate::runtime::Builder\n pub fn new() -> std::io::Result {\n Builder::new_current_thread()\n .enable_all()\n .build_local(Default::default())\n }\n\n /// Returns a handle to the runtime's spawner.\n ///\n /// The returned handle can be used to spawn tasks that run on this runtime, and can\n /// be cloned to allow moving the `Handle` to other threads.\n ///\n /// As the handle can be sent to other threads, it can only be used to spawn tasks that are `Send`.\n ///\n /// Calling [`Handle::block_on`] on a handle to a `LocalRuntime` is error-prone.\n /// Refer to the documentation of [`Handle::block_on`] for more.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n ///\n /// let rt = LocalRuntime::new()\n /// .unwrap();\n ///\n /// let handle = rt.handle();\n ///\n /// // Use the handle...\n /// ```\n pub fn handle(&self) -> &Handle {\n &self.handle\n }\n\n /// Spawns a task on the runtime.\n ///\n /// This is analogous to the [`spawn`] method on the standard [`Runtime`], but works even if the task is not thread-safe.\n ///\n /// [`spawn`]: crate::runtime::Runtime::spawn\n /// [`Runtime`]: crate::runtime::Runtime\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = LocalRuntime::new().unwrap();\n ///\n /// // Spawn a future onto the runtime\n /// rt.spawn_local(async {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn_local(&self, future: F) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let fut_size = std::mem::size_of::();\n let meta = SpawnMeta::new_unnamed(fut_size);\n\n // safety: spawn_local can only be called from `LocalRuntime`, which this is\n unsafe {\n if std::mem::size_of::() > BOX_FUTURE_THRESHOLD {\n self.handle.spawn_local_named(Box::pin(future), meta)\n } else {\n self.handle.spawn_local_named(future, meta)\n }\n }\n }\n\n /// Runs the provided function on a thread from a dedicated blocking thread pool.\n ///\n /// This function _will_ be run on another thread.\n ///\n /// See the [documentation in the non-local runtime][Runtime] for more\n /// information.\n ///\n /// [Runtime]: crate::runtime::Runtime::spawn_blocking\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = LocalRuntime::new().unwrap();\n ///\n /// // Spawn a blocking function onto the runtime\n /// rt.spawn_blocking(|| {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn_blocking(&self, func: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n self.handle.spawn_blocking(func)\n }\n\n /// Runs a future to completion on the Tokio runtime. This is the\n /// runtime's entry point.\n ///\n /// See the documentation for [the equivalent method on Runtime][Runtime]\n /// for more information.\n ///\n /// [Runtime]: crate::runtime::Runtime::block_on\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::runtime::LocalRuntime;\n ///\n /// // Create the runtime\n /// let rt = LocalRuntime::new().unwrap();\n ///\n /// // Execute the future, blocking the current thread until completion\n /// rt.block_on(async {\n /// println!(\"hello\");\n /// });\n /// ```\n #[track_caller]\n pub fn block_on(&self, future: F) -> F::Output {\n let fut_size = mem::size_of::();\n let meta = SpawnMeta::new_unnamed(fut_size);\n\n if std::mem::size_of::() > BOX_FUTURE_THRESHOLD {\n self.block_on_inner(Box::pin(future), meta)\n } else {\n self.block_on_inner(future, meta)\n }\n }\n\n #[track_caller]\n fn block_on_inner(&self, future: F, _meta: SpawnMeta<'_>) -> F::Output {\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n let future = crate::runtime::task::trace::Trace::root(future);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let future = crate::util::trace::task(\n future,\n \"block_on\",\n _meta,\n crate::runtime::task::Id::next().as_u64(),\n );\n\n let _enter = self.enter();\n\n if let LocalRuntimeScheduler::CurrentThread(exec) = &self.scheduler {\n exec.block_on(&self.handle.inner, future)\n } else {\n unreachable!(\"LocalRuntime only supports current_thread\")\n }\n }\n\n /// Enters the runtime context.\n ///\n /// This allows you to construct types that must have an executor\n /// available on creation such as [`Sleep`] or [`TcpStream`]. It will\n /// also allow you to call methods such as [`tokio::spawn`].\n ///\n /// If this is a handle to a [`LocalRuntime`], and this function is being invoked from the same\n /// thread that the runtime was created on, you will also be able to call\n /// [`tokio::task::spawn_local`].\n ///\n /// [`Sleep`]: struct@crate::time::Sleep\n /// [`TcpStream`]: struct@crate::net::TcpStream\n /// [`tokio::spawn`]: fn@crate::spawn\n /// [`LocalRuntime`]: struct@crate::runtime::LocalRuntime\n /// [`tokio::task::spawn_local`]: fn@crate::task::spawn_local\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n /// use tokio::task::JoinHandle;\n ///\n /// fn function_that_spawns(msg: String) -> JoinHandle<()> {\n /// // Had we not used `rt.enter` below, this would panic.\n /// tokio::spawn(async move {\n /// println!(\"{}\", msg);\n /// })\n /// }\n ///\n /// fn main() {\n /// let rt = LocalRuntime::new().unwrap();\n ///\n /// let s = \"Hello World!\".to_string();\n ///\n /// // By entering the context, we tie `tokio::spawn` to this executor.\n /// let _guard = rt.enter();\n /// let handle = function_that_spawns(s);\n ///\n /// // Wait for the task before we end the test.\n /// rt.block_on(handle).unwrap();\n /// }\n /// ```\n pub fn enter(&self) -> EnterGuard<'_> {\n self.handle.enter()\n }\n\n /// Shuts down the runtime, waiting for at most `duration` for all spawned\n /// work to stop.\n ///\n /// Note that `spawn_blocking` tasks, and only `spawn_blocking` tasks, can get left behind if\n /// the timeout expires.\n ///\n /// See the [struct level documentation](LocalRuntime#shutdown) for more details.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::LocalRuntime;\n /// use tokio::task;\n ///\n /// use std::thread;\n /// use std::time::Duration;\n ///\n /// fn main() {\n /// let runtime = LocalRuntime::new().unwrap();\n ///\n /// runtime.block_on(async move {\n /// task::spawn_blocking(move || {\n /// thread::sleep(Duration::from_secs(10_000));\n /// });\n /// });\n ///\n /// runtime.shutdown_timeout(Duration::from_millis(100));\n /// }\n /// # }\n /// ```\n pub fn shutdown_timeout(mut self, duration: Duration) {\n // Wakeup and shutdown all the worker threads\n self.handle.inner.shutdown();\n self.blocking_pool.shutdown(Some(duration));\n }\n\n /// Shuts down the runtime, without waiting for any spawned work to stop.\n ///\n /// This can be useful if you want to drop a runtime from within another runtime.\n /// Normally, dropping a runtime will block indefinitely for spawned blocking tasks\n /// to complete, which would normally not be permitted within an asynchronous context.\n /// By calling `shutdown_background()`, you can drop the runtime from such a context.\n ///\n /// Note however, that because we do not wait for any blocking tasks to complete, this\n /// may result in a resource leak (in that any blocking tasks are still running until they\n /// return. No other tasks will leak.\n ///\n /// See the [struct level documentation](LocalRuntime#shutdown) for more details.\n ///\n /// This function is equivalent to calling `shutdown_timeout(Duration::from_nanos(0))`.\n ///\n /// ```\n /// use tokio::runtime::LocalRuntime;\n ///\n /// fn main() {\n /// let runtime = LocalRuntime::new().unwrap();\n ///\n /// runtime.block_on(async move {\n /// let inner_runtime = LocalRuntime::new().unwrap();\n /// // ...\n /// inner_runtime.shutdown_background();\n /// });\n /// }\n /// ```\n pub fn shutdown_background(self) {\n self.shutdown_timeout(Duration::from_nanos(0));\n }\n\n /// Returns a view that lets you get information about how the runtime\n /// is performing.\n pub fn metrics(&self) -> crate::runtime::RuntimeMetrics {\n self.handle.metrics()\n }\n}\n\nimpl Drop for LocalRuntime {\n fn drop(&mut self) {\n if let LocalRuntimeScheduler::CurrentThread(current_thread) = &mut self.scheduler {\n // This ensures that tasks spawned on the current-thread\n // runtime are dropped inside the runtime's context.\n let _guard = context::try_set_current(&self.handle.inner);\n current_thread.shutdown(&self.handle.inner);\n } else {\n unreachable!(\"LocalRuntime only supports current-thread\")\n }\n }\n}\n\nimpl std::panic::UnwindSafe for LocalRuntime {}\n\nimpl std::panic::RefUnwindSafe for LocalRuntime {}\n" }, { "path": "tokio/src/runtime/metrics/batch.rs", "content": "use crate::runtime::metrics::WorkerMetrics;\n\ncfg_unstable_metrics! {\n use crate::runtime::metrics::HistogramBatch;\n}\n\nuse std::sync::atomic::Ordering::Relaxed;\nuse std::time::{Duration, Instant};\n\npub(crate) struct MetricsBatch {\n /// The total busy duration in nanoseconds.\n busy_duration_total: u64,\n\n /// Instant at which work last resumed (continued after park).\n processing_scheduled_tasks_started_at: Option,\n\n /// Number of times the worker parked.\n park_count: u64,\n\n /// Number of times the worker parked and unparked.\n park_unpark_count: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of times the worker woke w/o doing work.\n noop_count: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks stolen.\n steal_count: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of times tasks where stolen.\n steal_operations: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks that were polled by the worker.\n poll_count: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks polled when the worker entered park. This is used to\n /// track the noop count.\n poll_count_on_last_park: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks that were scheduled locally on this worker.\n local_schedule_count: u64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks moved to the global queue to make space in the local\n /// queue\n overflow_count: u64,\n\n #[cfg(tokio_unstable)]\n /// If `Some`, tracks poll times in nanoseconds\n poll_timer: Option,\n}\n\ncfg_unstable_metrics! {\n struct PollTimer {\n /// Histogram of poll counts within each band.\n poll_counts: HistogramBatch,\n\n /// Instant when the most recent task started polling.\n poll_started_at: Instant,\n }\n}\n\nimpl MetricsBatch {\n pub(crate) fn new(worker_metrics: &WorkerMetrics) -> MetricsBatch {\n let maybe_now = now();\n Self::new_unstable(worker_metrics, maybe_now)\n }\n\n cfg_metrics_variant! {\n stable: {\n #[inline(always)]\n fn new_unstable(_worker_metrics: &WorkerMetrics, maybe_now: Option) -> MetricsBatch {\n MetricsBatch {\n busy_duration_total: 0,\n processing_scheduled_tasks_started_at: maybe_now,\n park_count: 0,\n park_unpark_count: 0,\n }\n }\n },\n unstable: {\n #[inline(always)]\n fn new_unstable(worker_metrics: &WorkerMetrics, maybe_now: Option) -> MetricsBatch {\n let poll_timer = maybe_now.and_then(|now| {\n worker_metrics\n .poll_count_histogram\n .as_ref()\n .map(|worker_poll_counts| PollTimer {\n poll_counts: HistogramBatch::from_histogram(worker_poll_counts),\n poll_started_at: now,\n })\n });\n MetricsBatch {\n park_count: 0,\n park_unpark_count: 0,\n noop_count: 0,\n steal_count: 0,\n steal_operations: 0,\n poll_count: 0,\n poll_count_on_last_park: 0,\n local_schedule_count: 0,\n overflow_count: 0,\n busy_duration_total: 0,\n processing_scheduled_tasks_started_at: maybe_now,\n poll_timer,\n }\n }\n }\n }\n\n pub(crate) fn submit(&mut self, worker: &WorkerMetrics, mean_poll_time: u64) {\n worker\n .busy_duration_total\n .store(self.busy_duration_total, Relaxed);\n\n self.submit_unstable(worker, mean_poll_time);\n }\n\n cfg_metrics_variant! {\n stable: {\n #[inline(always)]\n fn submit_unstable(&mut self, worker: &WorkerMetrics, _mean_poll_time: u64) {\n worker.park_count.store(self.park_count, Relaxed);\n worker\n .park_unpark_count\n .store(self.park_unpark_count, Relaxed);\n }\n },\n unstable: {\n #[inline(always)]\n fn submit_unstable(&mut self, worker: &WorkerMetrics, mean_poll_time: u64) {\n worker.mean_poll_time.store(mean_poll_time, Relaxed);\n worker.park_count.store(self.park_count, Relaxed);\n worker\n .park_unpark_count\n .store(self.park_unpark_count, Relaxed);\n worker.noop_count.store(self.noop_count, Relaxed);\n worker.steal_count.store(self.steal_count, Relaxed);\n worker\n .steal_operations\n .store(self.steal_operations, Relaxed);\n worker.poll_count.store(self.poll_count, Relaxed);\n\n worker\n .local_schedule_count\n .store(self.local_schedule_count, Relaxed);\n worker.overflow_count.store(self.overflow_count, Relaxed);\n\n if let Some(poll_timer) = &self.poll_timer {\n let dst = worker.poll_count_histogram.as_ref().unwrap();\n poll_timer.poll_counts.submit(dst);\n }\n }\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n /// The worker is about to park.\n pub(crate) fn about_to_park(&mut self) {\n self.park_count += 1;\n self.park_unpark_count += 1;\n }\n },\n unstable: {\n /// The worker is about to park.\n pub(crate) fn about_to_park(&mut self) {\n {\n self.park_count += 1;\n self.park_unpark_count += 1;\n\n if self.poll_count_on_last_park == self.poll_count {\n self.noop_count += 1;\n } else {\n self.poll_count_on_last_park = self.poll_count;\n }\n }\n }\n }\n }\n /// The worker was unparked.\n pub(crate) fn unparked(&mut self) {\n self.park_unpark_count += 1;\n }\n\n /// Start processing a batch of tasks\n pub(crate) fn start_processing_scheduled_tasks(&mut self) {\n self.processing_scheduled_tasks_started_at = now();\n }\n\n /// Stop processing a batch of tasks\n pub(crate) fn end_processing_scheduled_tasks(&mut self) {\n if let Some(processing_scheduled_tasks_started_at) =\n self.processing_scheduled_tasks_started_at\n {\n let busy_duration = processing_scheduled_tasks_started_at.elapsed();\n self.busy_duration_total += duration_as_u64(busy_duration);\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n /// Start polling an individual task\n pub(crate) fn start_poll(&mut self) {}\n },\n unstable: {\n /// Start polling an individual task\n pub(crate) fn start_poll(&mut self) {\n self.poll_count += 1;\n if let Some(poll_timer) = &mut self.poll_timer {\n poll_timer.poll_started_at = Instant::now();\n }\n }\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n /// Stop polling an individual task\n pub(crate) fn end_poll(&mut self) {}\n },\n unstable: {\n /// Stop polling an individual task\n pub(crate) fn end_poll(&mut self) {\n if let Some(poll_timer) = &mut self.poll_timer {\n let elapsed = duration_as_u64(poll_timer.poll_started_at.elapsed());\n poll_timer.poll_counts.measure(elapsed, 1);\n }\n }\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n pub(crate) fn inc_local_schedule_count(&mut self) {}\n },\n unstable: {\n pub(crate) fn inc_local_schedule_count(&mut self) {\n self.local_schedule_count += 1;\n }\n }\n }\n}\n\ncfg_rt_multi_thread! {\n impl MetricsBatch {\n cfg_metrics_variant! {\n stable: {\n pub(crate) fn incr_steal_count(&mut self, _by: u16) {}\n },\n unstable: {\n pub(crate) fn incr_steal_count(&mut self, by: u16) {\n self.steal_count += by as u64;\n }\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n pub(crate) fn incr_steal_operations(&mut self) {}\n },\n unstable: {\n pub(crate) fn incr_steal_operations(&mut self) {\n self.steal_operations += 1;\n }\n }\n }\n\n cfg_metrics_variant! {\n stable: {\n pub(crate) fn incr_overflow_count(&mut self) {}\n },\n unstable: {\n pub(crate) fn incr_overflow_count(&mut self) {\n self.overflow_count += 1;\n }\n }\n }\n }\n}\n\npub(crate) fn duration_as_u64(dur: Duration) -> u64 {\n u64::try_from(dur.as_nanos()).unwrap_or(u64::MAX)\n}\n\n/// Gate unsupported time metrics for `wasm32-unknown-unknown`\n/// \nfn now() -> Option {\n if cfg!(all(\n target_arch = \"wasm32\",\n target_os = \"unknown\",\n target_vendor = \"unknown\"\n )) {\n None\n } else {\n Some(Instant::now())\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/histogram/h2_histogram.rs", "content": "use crate::runtime::metrics::batch::duration_as_u64;\nuse std::cmp;\nuse std::error::Error;\nuse std::fmt::{Display, Formatter};\nuse std::time::Duration;\n\nconst DEFAULT_MIN_VALUE: Duration = Duration::from_nanos(100);\nconst DEFAULT_MAX_VALUE: Duration = Duration::from_secs(60);\n\n/// Default precision is 2^-2 = 25% max error\nconst DEFAULT_PRECISION: u32 = 2;\nconst MAX_PRECISION: u32 = 10;\n\n/// Log Histogram\n///\n/// This implements an [H2 Histogram](https://iop.systems/blog/h2-histogram/), a histogram similar\n/// to HdrHistogram, but with better performance. It guarantees an error bound of `2^-p`.\n///\n/// Unlike a traditional H2 histogram this has two small changes:\n/// 1. The 0th bucket runs for `0..min_value`. This allows truncating a large number of buckets that\n/// would cover extremely short timescales that customers usually don't care about.\n/// 2. The final bucket runs all the way to `u64::MAX` — traditional H2 histograms would truncate\n/// or reject these values.\n///\n/// For information about the default configuration, see [`LogHistogramBuilder`].\n#[derive(Debug, Copy, Clone, PartialEq, Eq)]\npub struct LogHistogram {\n /// Number of buckets in the histogram\n pub(crate) num_buckets: usize,\n\n /// Precision of histogram. Error is bounded to 2^-p.\n pub(crate) p: u32,\n\n /// All buckets `idx < bucket_offset` are grouped into bucket 0.\n ///\n /// This increases the smallest measurable value of the histogram.\n pub(crate) bucket_offset: usize,\n}\n\nimpl Default for LogHistogram {\n fn default() -> Self {\n LogHistogramBuilder::default().build()\n }\n}\n\nimpl LogHistogram {\n /// Create a Histogram configuration directly from values for `n` and `p`.\n ///\n /// # Panics\n /// - If `bucket_offset` is greater than the specified number of buckets, `(n - p + 1) * 2^p`\n fn from_n_p(n: u32, p: u32, bucket_offset: usize) -> Self {\n assert!(n >= p, \"{n} (n) must be at least as large as {p} (p)\");\n let num_buckets = ((n - p + 1) << p) as usize - bucket_offset;\n Self {\n num_buckets,\n p,\n bucket_offset,\n }\n }\n\n fn truncate_to_max_value(&self, max_value: u64) -> LogHistogram {\n let mut hist = *self;\n while hist.max_value() >= max_value {\n hist.num_buckets -= 1;\n }\n hist.num_buckets += 1;\n hist\n }\n\n /// Creates a builder for [`LogHistogram`]\n pub fn builder() -> LogHistogramBuilder {\n LogHistogramBuilder::default()\n }\n\n /// The maximum value that can be stored before truncation in this histogram\n pub fn max_value(&self) -> u64 {\n self.bucket_range(self.num_buckets - 2).end\n }\n\n pub(crate) fn value_to_bucket(&self, value: u64) -> usize {\n let index = bucket_index(value, self.p);\n let offset_bucket = index.saturating_sub(self.bucket_offset as u64);\n let max = self.num_buckets - 1;\n offset_bucket.min(max as u64) as usize\n }\n\n pub(crate) fn bucket_range(&self, bucket: usize) -> std::ops::Range {\n let LogHistogram {\n p,\n bucket_offset,\n num_buckets,\n } = self;\n let input_bucket = bucket;\n let bucket = bucket + bucket_offset;\n let range_start_0th_bucket = match input_bucket {\n 0 => Some(0_u64),\n _ => None,\n };\n let range_end_last_bucket = match input_bucket {\n n if n == num_buckets - 1 => Some(u64::MAX),\n _ => None,\n };\n if bucket < 1 << p {\n // The first set of buckets are all size 1\n let bucket = bucket as u64;\n range_start_0th_bucket.unwrap_or(bucket)..range_end_last_bucket.unwrap_or(bucket + 1)\n } else {\n // Determine which range of buckets we're in, then determine which bucket in the range it is\n let bucket = bucket as u64;\n let p = *p as u64;\n let w = (bucket >> p) - 1;\n let base_bucket = (w + 1) * (1_u64 << p);\n let offset = bucket - base_bucket;\n let s = 1_u64 << (w + p);\n let start = s + (offset << w);\n let end = s + ((offset + 1) << w);\n\n range_start_0th_bucket.unwrap_or(start)..range_end_last_bucket.unwrap_or(end)\n }\n }\n}\n\n/// Configuration for a [`LogHistogram`]\n///\n/// The log-scaled histogram implements an H2 histogram where the first bucket covers\n/// the range from 0 to [`LogHistogramBuilder::min_value`] and the final bucket covers\n/// [`LogHistogramBuilder::max_value`] to infinity. The precision is bounded to the specified\n/// [`LogHistogramBuilder::max_error`]. Specifically, the precision is the next smallest value\n/// of `2^-p` such that it is smaller than the requested max error. You can also select `p` directly\n/// with [`LogHistogramBuilder::precision_exact`].\n///\n/// Depending on the selected parameters, the number of buckets required is variable. To ensure\n/// that the histogram size is acceptable, callers may call [`LogHistogramBuilder::max_buckets`].\n/// If the resulting histogram would require more buckets, then the method will return an error.\n///\n/// ## Default values\n/// The default configuration provides the following settings:\n/// 1. `min_value`: 100ns\n/// 2. `max_value`: 68 seconds. The final bucket covers all values >68 seconds\n/// 3. `precision`: max error of 25%\n///\n/// This uses 237 64-bit buckets.\n#[derive(Default, Debug, Copy, Clone)]\npub struct LogHistogramBuilder {\n max_value: Option,\n min_value: Option,\n precision: Option,\n}\n\nimpl From for LogHistogram {\n fn from(value: LogHistogramBuilder) -> Self {\n value.build()\n }\n}\n\nimpl LogHistogramBuilder {\n /// Set the precision for this histogram\n ///\n /// This function determines the smallest value of `p` that would satisfy the requested precision\n /// such that `2^-p` is less than `precision`. To set `p` directly, use\n /// [`LogHistogramBuilder::precision_exact`].\n ///\n /// Precision controls the size of the \"bucket groups\" (consecutive buckets with identical\n /// ranges). When `p` is 0, each bucket will be twice the size of the previous bucket. To match\n /// the behavior of the legacy log histogram implementation, use `builder.precision_exact(0)`.\n ///\n /// The default value is 25% (2^-2)\n ///\n /// The highest supported precision is `0.0977%` `(2^-10)`. Provided values\n /// less than this will be truncated.\n ///\n /// # Panics\n /// - `max_error` <= 0\n /// - `max_error` >= 1\n pub fn max_error(mut self, max_error: f64) -> Self {\n assert!(max_error > 0.0, \"max_error must be greater than 0\");\n assert!(max_error < 1.0, \"max_error must be less than 1\");\n let mut p = 2;\n while 2_f64.powf(-(p as f64)) > max_error && p <= MAX_PRECISION {\n p += 1;\n }\n self.precision = Some(p);\n self\n }\n\n /// Sets the precision of this histogram directly.\n ///\n /// The precision (meaning: the ratio `n/bucket_range(n)` for some given `n`) will be `2^-p`.\n ///\n /// Precision controls the number consecutive buckets with identically sized ranges.\n /// When `p` is 0, each bucket will be twice the size of the previous bucket (bucket groups are\n /// only a single bucket wide).\n ///\n /// To match the behavior of the legacy implementation ([`HistogramScale::Log`]), use `builder.precision_exact(0)`.\n ///\n /// # Panics\n /// - `p` > 10\n ///\n /// [`HistogramScale::Log`]: [crate::runtime::HistogramScale]\n pub fn precision_exact(mut self, p: u32) -> Self {\n assert!(p <= MAX_PRECISION, \"precision must be <= {MAX_PRECISION}\");\n self.precision = Some(p);\n self\n }\n\n /// Sets the minimum duration that can be accurately stored by this histogram.\n ///\n /// This sets the resolution. The first bucket will be no larger than\n /// the provided duration. Setting this value will reduce the number of required buckets,\n /// sometimes quite significantly.\n pub fn min_value(mut self, duration: Duration) -> Self {\n self.min_value = Some(duration);\n self\n }\n\n /// Sets the maximum value that can by this histogram without truncation\n ///\n /// Values greater than this fall in the final bucket that stretches to `u64::MAX`.\n ///\n /// # Panics\n /// The provided value is 0\n pub fn max_value(mut self, duration: Duration) -> Self {\n if duration.is_zero() {\n panic!(\"max value must be greater than 0\");\n }\n self.max_value = Some(duration);\n self\n }\n\n /// Builds the log histogram, enforcing the max buckets requirement\n pub fn max_buckets(\n &mut self,\n max_buckets: usize,\n ) -> Result {\n let histogram = self.build();\n if histogram.num_buckets > max_buckets {\n return Err(InvalidHistogramConfiguration::TooManyBuckets {\n required_bucket_count: histogram.num_buckets,\n });\n }\n Ok(histogram)\n }\n\n /// Builds the log histogram\n pub fn build(&self) -> LogHistogram {\n let requested_max_value = duration_as_u64(self.max_value.unwrap_or(DEFAULT_MAX_VALUE));\n let max_value = requested_max_value.next_power_of_two();\n let min_value = duration_as_u64(self.min_value.unwrap_or(DEFAULT_MIN_VALUE));\n let p = self.precision.unwrap_or(DEFAULT_PRECISION);\n // determine the bucket offset by finding the bucket for the minimum value. We need to lower\n // this by one to ensure we are at least as granular as requested.\n let bucket_offset = cmp::max(bucket_index(min_value, p), 1) - 1;\n // n must be at least as large as p\n let n = max_value.ilog2().max(p) + 1;\n LogHistogram::from_n_p(n, p, bucket_offset as usize)\n .truncate_to_max_value(requested_max_value)\n }\n}\n\n/// Error constructing a histogram\n#[derive(Debug)]\npub enum InvalidHistogramConfiguration {\n /// This histogram required more than the specified number of buckets\n TooManyBuckets {\n /// The number of buckets that would have been required\n required_bucket_count: usize,\n },\n}\n\nimpl Display for InvalidHistogramConfiguration {\n fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {\n match self {\n InvalidHistogramConfiguration::TooManyBuckets { required_bucket_count } =>\n write!(f, \"The configuration for this histogram would have required {required_bucket_count} buckets\")\n }\n }\n}\n\nimpl Error for InvalidHistogramConfiguration {}\n\n/// Compute the index for a given value + p combination\n///\n/// This function does NOT enforce that the value is within the number of expected buckets.\nfn bucket_index(value: u64, p: u32) -> u64 {\n // Algorithm described here: https://iop.systems/blog/h2-histogram/\n // find the highest non-zero digit\n if value == 0 {\n return 0;\n }\n let h = 63 - value.leading_zeros();\n if h <= p {\n value\n } else {\n let w = h - p;\n ((w + 1) * (1_u32 << p)) as u64 + ((value - (1_u64 << h)) >> w)\n }\n}\n\n#[cfg(test)]\nmod test {\n use super::InvalidHistogramConfiguration;\n use crate::runtime::metrics::histogram::h2_histogram::LogHistogram;\n use crate::runtime::metrics::histogram::HistogramType;\n\n #[cfg(not(target_family = \"wasm\"))]\n mod proptests {\n use super::*;\n use crate::runtime::metrics::batch::duration_as_u64;\n use crate::runtime::metrics::histogram::h2_histogram::MAX_PRECISION;\n use proptest::prelude::*;\n use std::time::Duration;\n\n fn valid_log_histogram_strategy() -> impl Strategy {\n (1..=50u32, 0..=MAX_PRECISION, 0..100usize).prop_map(|(n, p, bucket_offset)| {\n let p = p.min(n);\n let base = LogHistogram::from_n_p(n, p, 0);\n LogHistogram::from_n_p(n, p, bucket_offset.min(base.num_buckets - 1))\n })\n }\n\n fn log_histogram_settings() -> impl Strategy {\n (\n duration_as_u64(Duration::from_nanos(1))..duration_as_u64(Duration::from_secs(20)),\n duration_as_u64(Duration::from_secs(1))..duration_as_u64(Duration::from_secs(1000)),\n 0..MAX_PRECISION,\n )\n }\n\n // test against a wide assortment of different histogram configurations to ensure invariants hold\n proptest! {\n #[test]\n fn log_histogram_settings_maintain_invariants((min_value, max_value, p) in log_histogram_settings()) {\n if max_value < min_value {\n return Ok(())\n }\n let (min_value, max_value) = (Duration::from_nanos(min_value), Duration::from_nanos(max_value));\n let histogram = LogHistogram::builder().min_value(min_value).max_value(max_value).precision_exact(p).build();\n let first_bucket_end = Duration::from_nanos(histogram.bucket_range(0).end);\n let last_bucket_start = Duration::from_nanos(histogram.bucket_range(histogram.num_buckets - 1).start);\n let second_last_bucket_start = Duration::from_nanos(histogram.bucket_range(histogram.num_buckets - 2).start);\n prop_assert!(\n first_bucket_end <= min_value,\n \"first bucket {first_bucket_end:?} must be less than {min_value:?}\"\n );\n prop_assert!(\n last_bucket_start > max_value,\n \"last bucket start ({last_bucket_start:?} must be at least as big as `max_value` ({max_value:?})\"\n );\n\n // We should have the exact right number of buckets. The second to last bucket should be strictly less than max value.\n prop_assert!(\n second_last_bucket_start < max_value,\n \"second last bucket end ({second_last_bucket_start:?} must be at least as big as `max_value` ({max_value:?})\"\n );\n }\n\n #[test]\n fn proptest_log_histogram_invariants(histogram in valid_log_histogram_strategy()) {\n // 1. Assert that the first bucket always starts at 0\n let first_range = histogram.bucket_range(0);\n prop_assert_eq!(first_range.start, 0, \"First bucket doesn't start at 0\");\n\n // Check that bucket ranges are disjoint and contiguous\n let mut prev_end = 0;\n let mut prev_size = 0;\n for bucket in 0..histogram.num_buckets {\n let range = histogram.bucket_range(bucket);\n prop_assert_eq!(range.start, prev_end, \"Bucket ranges are not contiguous\");\n prop_assert!(range.start < range.end, \"Bucket range is empty or reversed\");\n\n let size = range.end - range.start;\n\n // 2. Assert that the sizes of the buckets are always powers of 2\n if bucket > 0 && bucket < histogram.num_buckets - 1 {\n prop_assert!(size.is_power_of_two(), \"Bucket size is not a power of 2\");\n }\n\n if bucket > 1 {\n // Assert that the sizes of the buckets are monotonically increasing\n // (after the first bucket, which may be smaller than the 0 bucket)\n prop_assert!(size >= prev_size, \"Bucket sizes are not monotonically increasing: This size {size} (previous: {prev_size}). Bucket: {bucket}\");\n }\n\n\n // 4. Assert that the size of the buckets is always within the error bound of 2^-p\n if bucket > 0 && bucket < histogram.num_buckets - 1 {\n let p = histogram.p as f64;\n let error_bound = 2.0_f64.powf(-p);\n // the most it could be wrong is by the length of the range / 2\n let relative_error = ((size as f64 - 1.0) / 2.0) / range.start as f64;\n prop_assert!(\n relative_error <= error_bound,\n \"Bucket size error exceeds bound: {:?} > {:?} ({range:?})\",\n relative_error,\n error_bound\n );\n }\n\n prev_end = range.end;\n prev_size = size;\n }\n prop_assert_eq!(prev_end, u64::MAX, \"Last bucket should end at u64::MAX\");\n\n // Check bijection between value_to_bucket and bucket_range\n for bucket in 0..histogram.num_buckets {\n let range = histogram.bucket_range(bucket);\n for value in [range.start, range.end - 1] {\n prop_assert_eq!(\n histogram.value_to_bucket(value),\n bucket,\n \"value_to_bucket is not consistent with bucket_range\"\n );\n }\n }\n }\n }\n }\n\n #[test]\n fn bucket_ranges_are_correct() {\n let p = 2;\n let config = HistogramType::H2(LogHistogram {\n num_buckets: 1024,\n p,\n bucket_offset: 0,\n });\n\n // check precise buckets. There are 2^(p+1) precise buckets\n for i in 0..2_usize.pow(p + 1) {\n assert_eq!(\n config.value_to_bucket(i as u64),\n i,\n \"{i} should be in bucket {i}\"\n );\n }\n\n let mut value = 2_usize.pow(p + 1);\n let current_bucket = value;\n while value < current_bucket * 2 {\n assert_eq!(\n config.value_to_bucket(value as u64),\n current_bucket + ((value - current_bucket) / 2),\n \"bucket for {value}\"\n );\n value += 1;\n }\n }\n\n // test buckets against known values\n #[test]\n fn bucket_computation_spot_check() {\n let p = 9;\n let config = HistogramType::H2(LogHistogram {\n num_buckets: 4096,\n p,\n bucket_offset: 0,\n });\n struct T {\n v: u64,\n bucket: usize,\n }\n let tests = [\n T { v: 1, bucket: 1 },\n T {\n v: 1023,\n bucket: 1023,\n },\n T {\n v: 1024,\n bucket: 1024,\n },\n T {\n v: 2048,\n bucket: 1536,\n },\n T {\n v: 2052,\n bucket: 1537,\n },\n ];\n for test in tests {\n assert_eq!(config.value_to_bucket(test.v), test.bucket);\n }\n }\n\n #[test]\n fn last_bucket_goes_to_infinity() {\n let conf = HistogramType::H2(LogHistogram::from_n_p(16, 3, 10));\n assert_eq!(conf.bucket_range(conf.num_buckets() - 1).end, u64::MAX);\n }\n\n #[test]\n fn bucket_offset() {\n // skip the first 10 buckets\n let conf = HistogramType::H2(LogHistogram::from_n_p(16, 3, 10));\n for i in 0..10 {\n assert_eq!(conf.value_to_bucket(i), 0);\n }\n // There are 16 1-element buckets. We skipped 10 of them. The first 2 element bucket starts\n // at 16\n assert_eq!(conf.value_to_bucket(10), 0);\n assert_eq!(conf.value_to_bucket(16), 6);\n assert_eq!(conf.value_to_bucket(17), 6);\n assert_eq!(conf.bucket_range(6), 16..18);\n }\n\n #[test]\n fn max_buckets_enforcement() {\n let error = LogHistogram::builder()\n .max_error(0.001)\n .max_buckets(5)\n .expect_err(\"this produces way more than 5 buckets\");\n let num_buckets = match error {\n InvalidHistogramConfiguration::TooManyBuckets {\n required_bucket_count,\n } => required_bucket_count,\n };\n assert_eq!(num_buckets, 27291);\n }\n\n #[test]\n fn default_configuration_size() {\n let conf = LogHistogram::builder().build();\n assert_eq!(conf.num_buckets, 119);\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/histogram.rs", "content": "mod h2_histogram;\n\npub use h2_histogram::{InvalidHistogramConfiguration, LogHistogram, LogHistogramBuilder};\n\nuse crate::util::metric_atomics::MetricAtomicU64;\nuse std::sync::atomic::Ordering::Relaxed;\n\nuse crate::runtime::metrics::batch::duration_as_u64;\nuse std::cmp;\nuse std::ops::Range;\nuse std::time::Duration;\n\n#[derive(Debug)]\npub(crate) struct Histogram {\n /// The histogram buckets\n buckets: Box<[MetricAtomicU64]>,\n\n /// The type of the histogram\n ///\n /// This handles `fn(bucket) -> Range` and `fn(value) -> bucket`\n histogram_type: HistogramType,\n}\n\n#[derive(Debug, Clone)]\npub(crate) struct HistogramBuilder {\n pub(crate) histogram_type: HistogramType,\n pub(crate) legacy: Option,\n}\n\n#[derive(Debug, Clone)]\npub(crate) struct LegacyBuilder {\n pub(crate) resolution: u64,\n pub(crate) scale: HistogramScale,\n pub(crate) num_buckets: usize,\n}\n\nimpl Default for LegacyBuilder {\n fn default() -> Self {\n Self {\n resolution: 100_000,\n num_buckets: 10,\n scale: HistogramScale::Linear,\n }\n }\n}\n\n#[derive(Debug)]\npub(crate) struct HistogramBatch {\n buckets: Box<[u64]>,\n configuration: HistogramType,\n}\n\ncfg_unstable! {\n /// Whether the histogram used to aggregate a metric uses a linear or\n /// logarithmic scale.\n #[derive(Debug, Copy, Clone, Eq, PartialEq)]\n #[non_exhaustive]\n pub enum HistogramScale {\n /// Linear bucket scale\n Linear,\n\n /// Logarithmic bucket scale\n Log,\n }\n\n /// Configuration for the poll count histogram\n #[derive(Debug, Clone)]\n pub struct HistogramConfiguration {\n pub(crate) inner: HistogramType\n }\n\n impl HistogramConfiguration {\n /// Create a linear bucketed histogram\n ///\n /// # Arguments\n ///\n /// * `bucket_width`: The width of each bucket\n /// * `num_buckets`: The number of buckets\n pub fn linear(bucket_width: Duration, num_buckets: usize) -> Self {\n Self {\n inner: HistogramType::Linear(LinearHistogram {\n num_buckets,\n bucket_width: duration_as_u64(bucket_width),\n }),\n }\n }\n\n /// Creates a log-scaled bucketed histogram\n ///\n /// See [`LogHistogramBuilder`] for information about configuration & defaults\n pub fn log(configuration: impl Into) -> Self {\n Self {\n inner: HistogramType::H2(configuration.into()),\n }\n }\n }\n}\n\n#[derive(Debug, Clone, Copy, Eq, PartialEq)]\npub(crate) enum HistogramType {\n /// Linear histogram with fixed width buckets\n Linear(LinearHistogram),\n\n /// Old log histogram where each bucket doubles in size\n LogLegacy(LegacyLogHistogram),\n\n /// Log histogram implementation based on H2 Histograms\n H2(LogHistogram),\n}\n\nimpl HistogramType {\n pub(crate) fn num_buckets(&self) -> usize {\n match self {\n HistogramType::Linear(linear) => linear.num_buckets,\n HistogramType::LogLegacy(log) => log.num_buckets,\n HistogramType::H2(h2) => h2.num_buckets,\n }\n }\n fn value_to_bucket(&self, value: u64) -> usize {\n match self {\n HistogramType::Linear(LinearHistogram {\n num_buckets,\n bucket_width,\n }) => {\n let max = num_buckets - 1;\n cmp::min(value / *bucket_width, max as u64) as usize\n }\n HistogramType::LogLegacy(LegacyLogHistogram {\n num_buckets,\n first_bucket_width,\n }) => {\n let max = num_buckets - 1;\n if value < *first_bucket_width {\n 0\n } else {\n let significant_digits = 64 - value.leading_zeros();\n let bucket_digits = 64 - (first_bucket_width - 1).leading_zeros();\n cmp::min(significant_digits as usize - bucket_digits as usize, max)\n }\n }\n HistogramType::H2(log_histogram) => log_histogram.value_to_bucket(value),\n }\n }\n\n fn bucket_range(&self, bucket: usize) -> Range {\n match self {\n HistogramType::Linear(LinearHistogram {\n num_buckets,\n bucket_width,\n }) => Range {\n start: bucket_width * bucket as u64,\n end: if bucket == num_buckets - 1 {\n u64::MAX\n } else {\n bucket_width * (bucket as u64 + 1)\n },\n },\n HistogramType::LogLegacy(LegacyLogHistogram {\n num_buckets,\n first_bucket_width,\n }) => Range {\n start: if bucket == 0 {\n 0\n } else {\n first_bucket_width << (bucket - 1)\n },\n end: if bucket == num_buckets - 1 {\n u64::MAX\n } else {\n first_bucket_width << bucket\n },\n },\n HistogramType::H2(log) => log.bucket_range(bucket),\n }\n }\n}\n\n#[derive(Debug, Copy, Clone, Eq, PartialEq)]\npub(crate) struct LinearHistogram {\n num_buckets: usize,\n bucket_width: u64,\n}\n\n#[derive(Debug, Copy, Clone, Eq, PartialEq)]\npub(crate) struct LegacyLogHistogram {\n num_buckets: usize,\n first_bucket_width: u64,\n}\n\nimpl Histogram {\n pub(crate) fn num_buckets(&self) -> usize {\n self.buckets.len()\n }\n\n cfg_64bit_metrics! {\n pub(crate) fn get(&self, bucket: usize) -> u64 {\n self.buckets[bucket].load(Relaxed)\n }\n }\n\n pub(crate) fn bucket_range(&self, bucket: usize) -> Range {\n self.histogram_type.bucket_range(bucket)\n }\n}\n\nimpl HistogramBatch {\n pub(crate) fn from_histogram(histogram: &Histogram) -> HistogramBatch {\n let buckets = vec![0; histogram.buckets.len()].into_boxed_slice();\n\n HistogramBatch {\n buckets,\n configuration: histogram.histogram_type,\n }\n }\n\n pub(crate) fn measure(&mut self, value: u64, count: u64) {\n self.buckets[self.value_to_bucket(value)] += count;\n }\n\n pub(crate) fn submit(&self, histogram: &Histogram) {\n debug_assert_eq!(self.configuration, histogram.histogram_type);\n debug_assert_eq!(self.buckets.len(), histogram.buckets.len());\n\n for i in 0..self.buckets.len() {\n histogram.buckets[i].store(self.buckets[i], Relaxed);\n }\n }\n\n fn value_to_bucket(&self, value: u64) -> usize {\n self.configuration.value_to_bucket(value)\n }\n}\n\nimpl HistogramBuilder {\n pub(crate) fn new() -> HistogramBuilder {\n HistogramBuilder {\n histogram_type: HistogramType::Linear(LinearHistogram {\n num_buckets: 10,\n bucket_width: 100_000,\n }),\n legacy: None,\n }\n }\n\n pub(crate) fn legacy_mut(&mut self, f: impl Fn(&mut LegacyBuilder)) {\n let legacy = self.legacy.get_or_insert_with(LegacyBuilder::default);\n f(legacy);\n }\n\n pub(crate) fn build(&self) -> Histogram {\n let histogram_type = match &self.legacy {\n Some(legacy) => {\n assert!(legacy.resolution > 0);\n match legacy.scale {\n HistogramScale::Linear => HistogramType::Linear(LinearHistogram {\n num_buckets: legacy.num_buckets,\n bucket_width: legacy.resolution,\n }),\n HistogramScale::Log => HistogramType::LogLegacy(LegacyLogHistogram {\n num_buckets: legacy.num_buckets,\n first_bucket_width: legacy.resolution.next_power_of_two(),\n }),\n }\n }\n None => self.histogram_type,\n };\n let num_buckets = histogram_type.num_buckets();\n\n Histogram {\n buckets: (0..num_buckets)\n .map(|_| MetricAtomicU64::new(0))\n .collect::>()\n .into_boxed_slice(),\n histogram_type,\n }\n }\n}\n\nimpl Default for HistogramBuilder {\n fn default() -> HistogramBuilder {\n HistogramBuilder::new()\n }\n}\n\n#[cfg(all(test, target_has_atomic = \"64\"))]\nmod test {\n use super::*;\n\n macro_rules! assert_bucket_eq {\n ($h:expr, $bucket:expr, $val:expr) => {{\n assert_eq!($h.buckets[$bucket], $val);\n }};\n }\n\n fn linear(resolution: u64, num_buckets: usize) -> Histogram {\n HistogramBuilder {\n histogram_type: HistogramType::Linear(LinearHistogram {\n bucket_width: resolution,\n num_buckets,\n }),\n legacy: None,\n }\n .build()\n }\n\n #[test]\n fn test_legacy_builder() {\n let mut builder = HistogramBuilder::new();\n builder.legacy_mut(|b| b.num_buckets = 20);\n assert_eq!(builder.build().num_buckets(), 20);\n }\n\n #[test]\n fn log_scale_resolution_1() {\n let h = HistogramBuilder {\n histogram_type: HistogramType::LogLegacy(LegacyLogHistogram {\n first_bucket_width: 1,\n num_buckets: 10,\n }),\n legacy: None,\n }\n .build();\n\n assert_eq!(h.bucket_range(0), 0..1);\n assert_eq!(h.bucket_range(1), 1..2);\n assert_eq!(h.bucket_range(2), 2..4);\n assert_eq!(h.bucket_range(3), 4..8);\n assert_eq!(h.bucket_range(9), 256..u64::MAX);\n\n let mut b = HistogramBatch::from_histogram(&h);\n\n b.measure(0, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(1, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(2, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 1);\n\n b.measure(3, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 2);\n\n b.measure(4, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 2);\n assert_bucket_eq!(b, 3, 1);\n\n b.measure(100, 1);\n assert_bucket_eq!(b, 7, 1);\n\n b.measure(128, 1);\n assert_bucket_eq!(b, 8, 1);\n\n b.measure(4096, 1);\n assert_bucket_eq!(b, 9, 1);\n\n b.measure(u64::MAX, 1);\n assert_bucket_eq!(b, 9, 2);\n }\n\n #[test]\n fn log_scale_resolution_2() {\n let h = HistogramBuilder {\n histogram_type: HistogramType::LogLegacy(LegacyLogHistogram {\n num_buckets: 10,\n first_bucket_width: 2,\n }),\n legacy: None,\n }\n .build();\n\n assert_eq!(h.bucket_range(0), 0..2);\n assert_eq!(h.bucket_range(1), 2..4);\n assert_eq!(h.bucket_range(2), 4..8);\n assert_eq!(h.bucket_range(3), 8..16);\n assert_eq!(h.bucket_range(9), 512..u64::MAX);\n\n let mut b = HistogramBatch::from_histogram(&h);\n\n b.measure(0, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(1, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(2, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(3, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(4, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 1);\n\n b.measure(5, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 2);\n\n b.measure(6, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 3);\n\n b.measure(7, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 4);\n\n b.measure(8, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 4);\n assert_bucket_eq!(b, 3, 1);\n\n b.measure(100, 1);\n assert_bucket_eq!(b, 6, 1);\n\n b.measure(128, 1);\n assert_bucket_eq!(b, 7, 1);\n\n b.measure(4096, 1);\n assert_bucket_eq!(b, 9, 1);\n\n for bucket in h.buckets.iter() {\n assert_eq!(bucket.load(Relaxed), 0);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n }\n\n #[test]\n fn linear_scale_resolution_1() {\n let h = linear(1, 10);\n\n assert_eq!(h.bucket_range(0), 0..1);\n assert_eq!(h.bucket_range(1), 1..2);\n assert_eq!(h.bucket_range(2), 2..3);\n assert_eq!(h.bucket_range(3), 3..4);\n assert_eq!(h.bucket_range(9), 9..u64::MAX);\n\n let mut b = HistogramBatch::from_histogram(&h);\n\n b.measure(0, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(1, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(2, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 1);\n assert_bucket_eq!(b, 3, 0);\n\n b.measure(3, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 1);\n assert_bucket_eq!(b, 3, 1);\n\n b.measure(5, 1);\n assert_bucket_eq!(b, 5, 1);\n\n b.measure(4096, 1);\n assert_bucket_eq!(b, 9, 1);\n\n for bucket in h.buckets.iter() {\n assert_eq!(bucket.load(Relaxed), 0);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n }\n\n #[test]\n fn linear_scale_resolution_100() {\n let h = linear(100, 10);\n\n assert_eq!(h.bucket_range(0), 0..100);\n assert_eq!(h.bucket_range(1), 100..200);\n assert_eq!(h.bucket_range(2), 200..300);\n assert_eq!(h.bucket_range(3), 300..400);\n assert_eq!(h.bucket_range(9), 900..u64::MAX);\n\n let mut b = HistogramBatch::from_histogram(&h);\n\n b.measure(0, 1);\n assert_bucket_eq!(b, 0, 1);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(50, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(100, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 1);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(101, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(200, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 1);\n\n b.measure(299, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 2);\n\n b.measure(222, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 3);\n\n b.measure(300, 1);\n assert_bucket_eq!(b, 0, 2);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 3);\n assert_bucket_eq!(b, 3, 1);\n\n b.measure(888, 1);\n assert_bucket_eq!(b, 8, 1);\n\n b.measure(4096, 1);\n assert_bucket_eq!(b, 9, 1);\n\n for bucket in h.buckets.iter() {\n assert_eq!(bucket.load(Relaxed), 0);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n }\n\n #[test]\n fn inc_by_more_than_one() {\n let h = linear(100, 10);\n\n let mut b = HistogramBatch::from_histogram(&h);\n\n b.measure(0, 3);\n assert_bucket_eq!(b, 0, 3);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(50, 5);\n assert_bucket_eq!(b, 0, 8);\n assert_bucket_eq!(b, 1, 0);\n\n b.measure(100, 2);\n assert_bucket_eq!(b, 0, 8);\n assert_bucket_eq!(b, 1, 2);\n assert_bucket_eq!(b, 2, 0);\n\n b.measure(101, 19);\n assert_bucket_eq!(b, 0, 8);\n assert_bucket_eq!(b, 1, 21);\n assert_bucket_eq!(b, 2, 0);\n\n for bucket in h.buckets.iter() {\n assert_eq!(bucket.load(Relaxed), 0);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n\n b.submit(&h);\n\n for i in 0..h.buckets.len() {\n assert_eq!(h.buckets[i].load(Relaxed), b.buckets[i]);\n }\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/io.rs", "content": "#![cfg_attr(not(feature = \"net\"), allow(dead_code))]\n\nuse crate::util::metric_atomics::MetricAtomicU64;\nuse std::sync::atomic::Ordering::Relaxed;\n\n#[derive(Default)]\npub(crate) struct IoDriverMetrics {\n pub(super) fd_registered_count: MetricAtomicU64,\n pub(super) fd_deregistered_count: MetricAtomicU64,\n pub(super) ready_count: MetricAtomicU64,\n}\n\nimpl IoDriverMetrics {\n pub(crate) fn incr_fd_count(&self) {\n self.fd_registered_count.add(1, Relaxed);\n }\n\n pub(crate) fn dec_fd_count(&self) {\n self.fd_deregistered_count.add(1, Relaxed);\n }\n\n pub(crate) fn incr_ready_count_by(&self, amt: u64) {\n self.ready_count.add(amt, Relaxed);\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/mock.rs", "content": "//! This file contains mocks of the types in src/runtime/metrics\n\npub(crate) struct SchedulerMetrics {}\n\n#[derive(Clone, Default)]\npub(crate) struct HistogramBuilder {}\n\nimpl SchedulerMetrics {\n pub(crate) fn new() -> Self {\n Self {}\n }\n\n /// Increment the number of tasks scheduled externally\n pub(crate) fn inc_remote_schedule_count(&self) {}\n}\n" }, { "path": "tokio/src/runtime/metrics/mod.rs", "content": "//! This module contains information need to view information about how the\n//! runtime is performing.\n//!\n//! **Note**: This is an [unstable API][unstable]. The public API of types in\n//! this module may break in 1.x releases. See [the documentation on unstable\n//! features][unstable] for details.\n//!\n//! [unstable]: crate#unstable-features\n#![allow(clippy::module_inception)]\n\nmod runtime;\npub use runtime::RuntimeMetrics;\n\nmod batch;\npub(crate) use batch::MetricsBatch;\n\nmod worker;\npub(crate) use worker::WorkerMetrics;\n\ncfg_unstable_metrics! {\n\n mod histogram;\n pub(crate) use histogram::{Histogram, HistogramBatch, HistogramBuilder};\n\n #[allow(unreachable_pub)] // rust-lang/rust#57411\n pub use histogram::{HistogramScale, HistogramConfiguration, LogHistogram, LogHistogramBuilder, InvalidHistogramConfiguration};\n\n mod scheduler;\n pub(crate) use scheduler::SchedulerMetrics;\n\n cfg_net! {\n mod io;\n pub(crate) use io::IoDriverMetrics;\n }\n}\n\ncfg_not_unstable_metrics! {\n mod mock;\n pub(crate) use mock::{SchedulerMetrics, HistogramBuilder};\n}\n" }, { "path": "tokio/src/runtime/metrics/runtime.rs", "content": "use crate::runtime::Handle;\nuse std::time::Duration;\n\ncfg_64bit_metrics! {\n use std::sync::atomic::Ordering::Relaxed;\n}\n\ncfg_unstable_metrics! {\n use std::ops::Range;\n use std::thread::ThreadId;\n}\n\n/// Handle to the runtime's metrics.\n///\n/// This handle is internally reference-counted and can be freely cloned. A\n/// `RuntimeMetrics` handle is obtained using the [`Runtime::metrics`] method.\n///\n/// [`Runtime::metrics`]: crate::runtime::Runtime::metrics()\n#[derive(Clone, Debug)]\npub struct RuntimeMetrics {\n handle: Handle,\n}\n\nimpl RuntimeMetrics {\n pub(crate) fn new(handle: Handle) -> RuntimeMetrics {\n RuntimeMetrics { handle }\n }\n\n /// Returns the number of worker threads used by the runtime.\n ///\n /// The number of workers is set by configuring `worker_threads` on\n /// `runtime::Builder`. When using the `current_thread` runtime, the return\n /// value is always `1`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.num_workers();\n /// println!(\"Runtime is using {} workers\", n);\n /// # }\n /// ```\n pub fn num_workers(&self) -> usize {\n self.handle.inner.num_workers()\n }\n\n /// Returns the current number of alive tasks in the runtime.\n ///\n /// This counter increases when a task is spawned and decreases when a\n /// task exits.\n ///\n /// Note: When using the multi-threaded runtime this number may not\n /// not have strong consistency i.e. no tasks may be running but the metric\n /// reports otherwise.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.num_alive_tasks();\n /// println!(\"Runtime has {} alive tasks\", n);\n /// # }\n /// ```\n pub fn num_alive_tasks(&self) -> usize {\n self.handle.inner.num_alive_tasks()\n }\n\n /// Returns the number of tasks currently scheduled in the runtime's\n /// global queue.\n ///\n /// Tasks that are spawned or notified from a non-runtime thread are\n /// scheduled using the runtime's global queue. This metric returns the\n /// **current** number of tasks pending in the global queue. As such, the\n /// returned value may increase or decrease as new tasks are scheduled and\n /// processed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.global_queue_depth();\n /// println!(\"{} tasks currently pending in the runtime's global queue\", n);\n /// # }\n /// ```\n pub fn global_queue_depth(&self) -> usize {\n self.handle.inner.injection_queue_depth()\n }\n\n cfg_64bit_metrics! {\n /// Returns the amount of time the given worker thread has been busy.\n ///\n /// The worker busy duration starts at zero when the runtime is created and\n /// increases whenever the worker is spending time processing work. Using\n /// this value can indicate the load of the given worker. If a lot of time\n /// is spent busy, then the worker is under load and will check for inbound\n /// events less often.\n ///\n /// The timer is monotonically increasing. It is never decremented or reset\n /// to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_total_busy_duration(0);\n /// println!(\"worker 0 was busy for a total of {:?}\", n);\n /// # }\n /// ```\n pub fn worker_total_busy_duration(&self, worker: usize) -> Duration {\n let nanos = self\n .handle\n .inner\n .worker_metrics(worker)\n .busy_duration_total\n .load(Relaxed);\n Duration::from_nanos(nanos)\n }\n\n /// Returns the total number of times the given worker thread has parked.\n ///\n /// The worker park count starts at zero when the runtime is created and\n /// increases by one each time the worker parks the thread waiting for new\n /// inbound events to process. This usually means the worker has processed\n /// all pending work and is currently idle.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_park_count(0);\n /// println!(\"worker 0 parked {} times\", n);\n /// # }\n /// ```\n pub fn worker_park_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .park_count\n .load(Relaxed)\n }\n\n /// Returns the total number of times the given worker thread has parked\n /// and unparked.\n ///\n /// The worker park/unpark count starts at zero when the runtime is created\n /// and increases by one each time the worker parks the thread waiting for\n /// new inbound events to process. This usually means the worker has processed\n /// all pending work and is currently idle. When new work becomes available,\n /// the worker is unparked and the park/unpark count is again increased by one.\n ///\n /// An odd count means that the worker is currently parked.\n /// An even count means that the worker is currently active.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n /// let n = metrics.worker_park_unpark_count(0);\n ///\n /// println!(\"worker 0 parked and unparked {} times\", n);\n ///\n /// if n % 2 == 0 {\n /// println!(\"worker 0 is active\");\n /// } else {\n /// println!(\"worker 0 is parked\");\n /// }\n /// # }\n /// ```\n pub fn worker_park_unpark_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .park_unpark_count\n .load(Relaxed)\n }\n }\n\n cfg_unstable_metrics! {\n\n /// Returns the number of additional threads spawned by the runtime.\n ///\n /// The number of workers is set by configuring `max_blocking_threads` on\n /// `runtime::Builder`.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let _ = tokio::task::spawn_blocking(move || {\n /// // Stand-in for compute-heavy work or using synchronous APIs\n /// 1 + 1\n /// }).await;\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.num_blocking_threads();\n /// println!(\"Runtime has created {} threads\", n);\n /// # }\n /// # }\n /// ```\n pub fn num_blocking_threads(&self) -> usize {\n self.handle.inner.num_blocking_threads()\n }\n\n #[deprecated = \"Renamed to num_alive_tasks\"]\n /// Renamed to [`RuntimeMetrics::num_alive_tasks`]\n pub fn active_tasks_count(&self) -> usize {\n self.num_alive_tasks()\n }\n\n /// Returns the number of idle threads, which have spawned by the runtime\n /// for `spawn_blocking` calls.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let _ = tokio::task::spawn_blocking(move || {\n /// // Stand-in for compute-heavy work or using synchronous APIs\n /// 1 + 1\n /// }).await;\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.num_idle_blocking_threads();\n /// println!(\"Runtime has {} idle blocking thread pool threads\", n);\n /// }\n /// # }\n /// ```\n pub fn num_idle_blocking_threads(&self) -> usize {\n self.handle.inner.num_idle_blocking_threads()\n }\n\n /// Returns the thread id of the given worker thread.\n ///\n /// The returned value is `None` if the worker thread has not yet finished\n /// starting up.\n ///\n /// If additional information about the thread, such as its native id, are\n /// required, those can be collected in [`on_thread_start`] and correlated\n /// using the thread id.\n ///\n /// [`on_thread_start`]: crate::runtime::Builder::on_thread_start\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let id = metrics.worker_thread_id(0);\n /// println!(\"worker 0 has id {:?}\", id);\n /// # }\n /// ```\n pub fn worker_thread_id(&self, worker: usize) -> Option {\n self.handle\n .inner\n .worker_metrics(worker)\n .thread_id()\n }\n\n /// Renamed to [`RuntimeMetrics::global_queue_depth`]\n #[deprecated = \"Renamed to global_queue_depth\"]\n #[doc(hidden)]\n pub fn injection_queue_depth(&self) -> usize {\n self.handle.inner.injection_queue_depth()\n }\n\n /// Returns the number of tasks currently scheduled in the given worker's\n /// local queue.\n ///\n /// Tasks that are spawned or notified from within a runtime thread are\n /// scheduled using that worker's local queue. This metric returns the\n /// **current** number of tasks pending in the worker's local queue. As\n /// such, the returned value may increase or decrease as new tasks are\n /// scheduled and processed.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_local_queue_depth(0);\n /// println!(\"{} tasks currently pending in worker 0's local queue\", n);\n /// # }\n /// ```\n pub fn worker_local_queue_depth(&self, worker: usize) -> usize {\n self.handle.inner.worker_local_queue_depth(worker)\n }\n\n /// Returns `true` if the runtime is tracking the distribution of task poll\n /// times.\n ///\n /// Task poll times are not instrumented by default as doing so requires\n /// calling [`Instant::now()`] twice per task poll. The feature is enabled\n /// by calling [`enable_metrics_poll_time_histogram()`] when building the\n /// runtime.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{self, Handle};\n ///\n /// fn main() {\n /// runtime::Builder::new_current_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .build()\n /// .unwrap()\n /// .block_on(async {\n /// let metrics = Handle::current().metrics();\n /// let enabled = metrics.poll_time_histogram_enabled();\n ///\n /// println!(\"Tracking task poll time distribution: {:?}\", enabled);\n /// });\n /// }\n /// ```\n ///\n /// [`enable_metrics_poll_time_histogram()`]: crate::runtime::Builder::enable_metrics_poll_time_histogram\n /// [`Instant::now()`]: std::time::Instant::now\n pub fn poll_time_histogram_enabled(&self) -> bool {\n self.handle\n .inner\n .worker_metrics(0)\n .poll_count_histogram\n .is_some()\n }\n\n #[deprecated(note = \"Renamed to `poll_time_histogram_enabled`\")]\n #[doc(hidden)]\n pub fn poll_count_histogram_enabled(&self) -> bool {\n self.poll_time_histogram_enabled()\n }\n\n /// Returns the number of histogram buckets tracking the distribution of\n /// task poll times.\n ///\n /// This value is configured by calling\n /// [`metrics_poll_time_histogram_configuration()`] when building the runtime.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{self, Handle};\n ///\n /// fn main() {\n /// runtime::Builder::new_current_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .build()\n /// .unwrap()\n /// .block_on(async {\n /// let metrics = Handle::current().metrics();\n /// let buckets = metrics.poll_time_histogram_num_buckets();\n ///\n /// println!(\"Histogram buckets: {:?}\", buckets);\n /// });\n /// }\n /// ```\n ///\n /// [`metrics_poll_time_histogram_configuration()`]:\n /// crate::runtime::Builder::metrics_poll_time_histogram_configuration\n pub fn poll_time_histogram_num_buckets(&self) -> usize {\n self.handle\n .inner\n .worker_metrics(0)\n .poll_count_histogram\n .as_ref()\n .map(|histogram| histogram.num_buckets())\n .unwrap_or_default()\n }\n\n /// Deprecated. Use [`poll_time_histogram_num_buckets()`] instead.\n ///\n /// [`poll_time_histogram_num_buckets()`]: Self::poll_time_histogram_num_buckets\n #[doc(hidden)]\n #[deprecated(note = \"renamed to `poll_time_histogram_num_buckets`.\")]\n pub fn poll_count_histogram_num_buckets(&self) -> usize {\n self.poll_time_histogram_num_buckets()\n }\n\n /// Returns the range of task poll times tracked by the given bucket.\n ///\n /// This value is configured by calling\n /// [`metrics_poll_time_histogram_configuration()`] when building the runtime.\n ///\n /// # Panics\n ///\n /// The method panics if `bucket` represents an invalid bucket index, i.e.\n /// is greater than or equal to `poll_time_histogram_num_buckets()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{self, Handle};\n ///\n /// fn main() {\n /// runtime::Builder::new_current_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .build()\n /// .unwrap()\n /// .block_on(async {\n /// let metrics = Handle::current().metrics();\n /// let buckets = metrics.poll_time_histogram_num_buckets();\n ///\n /// for i in 0..buckets {\n /// let range = metrics.poll_time_histogram_bucket_range(i);\n /// println!(\"Histogram bucket {} range: {:?}\", i, range);\n /// }\n /// });\n /// }\n /// ```\n ///\n /// [`metrics_poll_time_histogram_configuration()`]:\n /// crate::runtime::Builder::metrics_poll_time_histogram_configuration\n #[track_caller]\n pub fn poll_time_histogram_bucket_range(&self, bucket: usize) -> Range {\n self.handle\n .inner\n .worker_metrics(0)\n .poll_count_histogram\n .as_ref()\n .map(|histogram| {\n let range = histogram.bucket_range(bucket);\n std::ops::Range {\n start: Duration::from_nanos(range.start),\n end: Duration::from_nanos(range.end),\n }\n })\n .unwrap_or_default()\n }\n\n /// Deprecated. Use [`poll_time_histogram_bucket_range()`] instead.\n ///\n /// [`poll_time_histogram_bucket_range()`]: Self::poll_time_histogram_bucket_range\n #[track_caller]\n #[doc(hidden)]\n #[deprecated(note = \"renamed to `poll_time_histogram_bucket_range`\")]\n pub fn poll_count_histogram_bucket_range(&self, bucket: usize) -> Range {\n self.poll_time_histogram_bucket_range(bucket)\n }\n\n /// Returns the number of tasks currently scheduled in the blocking\n /// thread pool, spawned using `spawn_blocking`.\n ///\n /// This metric returns the **current** number of tasks pending in\n /// blocking thread pool. As such, the returned value may increase\n /// or decrease as new tasks are scheduled and processed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.blocking_queue_depth();\n /// println!(\"{} tasks currently pending in the blocking thread pool\", n);\n /// # }\n /// ```\n pub fn blocking_queue_depth(&self) -> usize {\n self.handle.inner.blocking_queue_depth()\n }\n }\n\n feature! {\n #![all(\n tokio_unstable,\n target_has_atomic = \"64\"\n )]\n /// Returns the number of tasks spawned in this runtime since it was created.\n ///\n /// This count starts at zero when the runtime is created and increases by one each time a task is spawned.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.spawned_tasks_count();\n /// println!(\"Runtime has had {} tasks spawned\", n);\n /// # }\n /// ```\n pub fn spawned_tasks_count(&self) -> u64 {\n self.handle.inner.spawned_tasks_count()\n }\n\n /// Returns the number of tasks scheduled from **outside** of the runtime.\n ///\n /// The remote schedule count starts at zero when the runtime is created and\n /// increases by one each time a task is woken from **outside** of the\n /// runtime. This usually means that a task is spawned or notified from a\n /// non-runtime thread and must be queued using the Runtime's injection\n /// queue, which tends to be slower.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.remote_schedule_count();\n /// println!(\"{} tasks were scheduled from outside the runtime\", n);\n /// # }\n /// ```\n pub fn remote_schedule_count(&self) -> u64 {\n self.handle\n .inner\n .scheduler_metrics()\n .remote_schedule_count\n .load(Relaxed)\n }\n\n /// Returns the number of times that tasks have been forced to yield back to the scheduler\n /// after exhausting their task budgets.\n ///\n /// This count starts at zero when the runtime is created and increases by one each time a task yields due to exhausting its budget.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n pub fn budget_forced_yield_count(&self) -> u64 {\n self.handle\n .inner\n .scheduler_metrics()\n .budget_forced_yield_count\n .load(Relaxed)\n }\n\n /// Returns the number of times the given worker thread unparked but\n /// performed no work before parking again.\n ///\n /// The worker no-op count starts at zero when the runtime is created and\n /// increases by one each time the worker unparks the thread but finds no\n /// new work and goes back to sleep. This indicates a false-positive wake up.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_noop_count(0);\n /// println!(\"worker 0 had {} no-op unparks\", n);\n /// # }\n /// ```\n pub fn worker_noop_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .noop_count\n .load(Relaxed)\n }\n\n /// Returns the number of tasks the given worker thread stole from\n /// another worker thread.\n ///\n /// This metric only applies to the **multi-threaded** runtime and will\n /// always return `0` when using the current thread runtime.\n ///\n /// The worker steal count starts at zero when the runtime is created and\n /// increases by `N` each time the worker has processed its scheduled queue\n /// and successfully steals `N` more pending tasks from another worker.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_steal_count(0);\n /// println!(\"worker 0 has stolen {} tasks\", n);\n /// # }\n /// ```\n pub fn worker_steal_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .steal_count\n .load(Relaxed)\n }\n\n /// Returns the number of times the given worker thread stole tasks from\n /// another worker thread.\n ///\n /// This metric only applies to the **multi-threaded** runtime and will\n /// always return `0` when using the current thread runtime.\n ///\n /// The worker steal count starts at zero when the runtime is created and\n /// increases by one each time the worker has processed its scheduled queue\n /// and successfully steals more pending tasks from another worker.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_steal_operations(0);\n /// println!(\"worker 0 has stolen tasks {} times\", n);\n /// # }\n /// ```\n pub fn worker_steal_operations(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .steal_operations\n .load(Relaxed)\n }\n\n /// Returns the number of tasks the given worker thread has polled.\n ///\n /// The worker poll count starts at zero when the runtime is created and\n /// increases by one each time the worker polls a scheduled task.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_poll_count(0);\n /// println!(\"worker 0 has polled {} tasks\", n);\n /// # }\n /// ```\n pub fn worker_poll_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .poll_count\n .load(Relaxed)\n }\n\n /// Returns the number of tasks scheduled from **within** the runtime on the\n /// given worker's local queue.\n ///\n /// The local schedule count starts at zero when the runtime is created and\n /// increases by one each time a task is woken from **inside** of the\n /// runtime on the given worker. This usually means that a task is spawned\n /// or notified from within a runtime thread and will be queued on the\n /// worker-local queue.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_local_schedule_count(0);\n /// println!(\"{} tasks were scheduled on the worker's local queue\", n);\n /// # }\n /// ```\n pub fn worker_local_schedule_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .local_schedule_count\n .load(Relaxed)\n }\n\n /// Returns the number of times the given worker thread saturated its local\n /// queue.\n ///\n /// This metric only applies to the **multi-threaded** scheduler.\n ///\n /// The worker overflow count starts at zero when the runtime is created and\n /// increases by one each time the worker attempts to schedule a task\n /// locally, but its local queue is full. When this happens, half of the\n /// local queue is moved to the injection queue.\n ///\n /// The counter is monotonically increasing. It is never decremented or\n /// reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_overflow_count(0);\n /// println!(\"worker 0 has overflowed its queue {} times\", n);\n /// # }\n /// ```\n pub fn worker_overflow_count(&self, worker: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .overflow_count\n .load(Relaxed)\n }\n\n /// Returns the number of times the given worker polled tasks with a poll\n /// duration within the given bucket's range.\n ///\n /// Each worker maintains its own histogram and the counts for each bucket\n /// starts at zero when the runtime is created. Each time the worker polls a\n /// task, it tracks the duration the task poll time took and increments the\n /// associated bucket by 1.\n ///\n /// Each bucket is a monotonically increasing counter. It is never\n /// decremented or reset to zero.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// `bucket` is the index of the bucket being queried. The bucket is scoped\n /// to the worker. The range represented by the bucket can be queried by\n /// calling [`poll_time_histogram_bucket_range()`]. Each worker maintains\n /// identical bucket ranges.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()` or if `bucket` represents an\n /// invalid bucket.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::{self, Handle};\n ///\n /// fn main() {\n /// runtime::Builder::new_current_thread()\n /// .enable_metrics_poll_time_histogram()\n /// .build()\n /// .unwrap()\n /// .block_on(async {\n /// let metrics = Handle::current().metrics();\n /// let buckets = metrics.poll_time_histogram_num_buckets();\n ///\n /// for worker in 0..metrics.num_workers() {\n /// for i in 0..buckets {\n /// let count = metrics.poll_time_histogram_bucket_count(worker, i);\n /// println!(\"Poll count {}\", count);\n /// }\n /// }\n /// });\n /// }\n /// ```\n ///\n /// [`poll_time_histogram_bucket_range()`]: crate::runtime::RuntimeMetrics::poll_time_histogram_bucket_range\n #[track_caller]\n pub fn poll_time_histogram_bucket_count(&self, worker: usize, bucket: usize) -> u64 {\n self.handle\n .inner\n .worker_metrics(worker)\n .poll_count_histogram\n .as_ref()\n .map(|histogram| histogram.get(bucket))\n .unwrap_or_default()\n }\n\n #[doc(hidden)]\n #[deprecated(note = \"use `poll_time_histogram_bucket_count` instead\")]\n pub fn poll_count_histogram_bucket_count(&self, worker: usize, bucket: usize) -> u64 {\n self.poll_time_histogram_bucket_count(worker, bucket)\n }\n\n /// Returns the mean duration of task polls, in nanoseconds.\n ///\n /// This is an exponentially weighted moving average. Currently, this metric\n /// is only provided by the multi-threaded runtime.\n ///\n /// # Arguments\n ///\n /// `worker` is the index of the worker being queried. The given value must\n /// be between 0 and `num_workers()`. The index uniquely identifies a single\n /// worker and will continue to identify the worker throughout the lifetime\n /// of the runtime instance.\n ///\n /// # Panics\n ///\n /// The method panics when `worker` represents an invalid worker, i.e. is\n /// greater than or equal to `num_workers()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.worker_mean_poll_time(0);\n /// println!(\"worker 0 has a mean poll time of {:?}\", n);\n /// # }\n /// ```\n #[track_caller]\n pub fn worker_mean_poll_time(&self, worker: usize) -> Duration {\n let nanos = self\n .handle\n .inner\n .worker_metrics(worker)\n .mean_poll_time\n .load(Relaxed);\n Duration::from_nanos(nanos)\n }\n }\n\n feature! {\n #![all(\n tokio_unstable,\n target_has_atomic = \"64\",\n feature = \"net\"\n )]\n /// Returns the number of file descriptors that have been registered with the\n /// runtime's I/O driver.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let registered_fds = metrics.io_driver_fd_registered_count();\n /// println!(\"{} fds have been registered with the runtime's I/O driver.\", registered_fds);\n ///\n /// let deregistered_fds = metrics.io_driver_fd_deregistered_count();\n ///\n /// let current_fd_count = registered_fds - deregistered_fds;\n /// println!(\"{} fds are currently registered by the runtime's I/O driver.\", current_fd_count);\n /// }\n /// ```\n pub fn io_driver_fd_registered_count(&self) -> u64 {\n self.with_io_driver_metrics(|m| {\n m.fd_registered_count.load(Relaxed)\n })\n }\n\n /// Returns the number of file descriptors that have been deregistered by the\n /// runtime's I/O driver.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.io_driver_fd_deregistered_count();\n /// println!(\"{} fds have been deregistered by the runtime's I/O driver.\", n);\n /// }\n /// ```\n pub fn io_driver_fd_deregistered_count(&self) -> u64 {\n self.with_io_driver_metrics(|m| {\n m.fd_deregistered_count.load(Relaxed)\n })\n }\n\n /// Returns the number of ready events processed by the runtime's\n /// I/O driver.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::runtime::Handle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let metrics = Handle::current().metrics();\n ///\n /// let n = metrics.io_driver_ready_count();\n /// println!(\"{} ready events processed by the runtime's I/O driver.\", n);\n /// }\n /// ```\n pub fn io_driver_ready_count(&self) -> u64 {\n self.with_io_driver_metrics(|m| m.ready_count.load(Relaxed))\n }\n\n fn with_io_driver_metrics(&self, f: F) -> u64\n where\n F: Fn(&super::IoDriverMetrics) -> u64,\n {\n // TODO: Investigate if this should return 0, most of our metrics always increase\n // thus this breaks that guarantee.\n self.handle\n .inner\n .driver()\n .io\n .as_ref()\n .map(|h| f(&h.metrics))\n .unwrap_or(0)\n }\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/scheduler.rs", "content": "use crate::loom::sync::atomic::Ordering::Relaxed;\nuse crate::util::metric_atomics::MetricAtomicU64;\n\n/// Retrieves metrics from the Tokio runtime.\n///\n/// **Note**: This is an [unstable API][unstable]. The public API of this type\n/// may break in 1.x releases. See [the documentation on unstable\n/// features][unstable] for details.\n///\n/// [unstable]: crate#unstable-features\n#[derive(Debug)]\npub(crate) struct SchedulerMetrics {\n /// Number of tasks that are scheduled from outside the runtime.\n pub(super) remote_schedule_count: MetricAtomicU64,\n pub(super) budget_forced_yield_count: MetricAtomicU64,\n}\n\nimpl SchedulerMetrics {\n pub(crate) fn new() -> SchedulerMetrics {\n SchedulerMetrics {\n remote_schedule_count: MetricAtomicU64::new(0),\n budget_forced_yield_count: MetricAtomicU64::new(0),\n }\n }\n\n /// Increment the number of tasks scheduled externally\n pub(crate) fn inc_remote_schedule_count(&self) {\n self.remote_schedule_count.add(1, Relaxed);\n }\n\n /// Increment the number of tasks forced to yield due to budget exhaustion\n pub(crate) fn inc_budget_forced_yield_count(&self) {\n self.budget_forced_yield_count.add(1, Relaxed);\n }\n}\n" }, { "path": "tokio/src/runtime/metrics/worker.rs", "content": "use crate::runtime::Config;\nuse crate::util::metric_atomics::{MetricAtomicU64, MetricAtomicUsize};\nuse std::sync::atomic::Ordering::Relaxed;\nuse std::sync::Mutex;\nuse std::thread::ThreadId;\n\ncfg_unstable_metrics! {\n use crate::runtime::metrics::Histogram;\n}\n\n/// Retrieve runtime worker metrics.\n///\n/// **Note**: This is an [unstable API][unstable]. The public API of this type\n/// may break in 1.x releases. See [the documentation on unstable\n/// features][unstable] for details.\n///\n/// [unstable]: crate#unstable-features\n#[derive(Debug, Default)]\n#[repr(align(128))]\npub(crate) struct WorkerMetrics {\n /// Amount of time the worker spent doing work vs. parking.\n pub(crate) busy_duration_total: MetricAtomicU64,\n\n /// Number of tasks currently in the local queue. Used only by the\n /// current-thread scheduler.\n pub(crate) queue_depth: MetricAtomicUsize,\n\n /// Thread id of worker thread.\n thread_id: Mutex>,\n\n /// Number of times the worker parked.\n pub(crate) park_count: MetricAtomicU64,\n\n /// Number of times the worker parked and unparked.\n pub(crate) park_unpark_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of times the worker woke then parked again without doing work.\n pub(crate) noop_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks the worker stole.\n pub(crate) steal_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of times the worker stole\n pub(crate) steal_operations: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks the worker polled.\n pub(crate) poll_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// EWMA task poll time, in nanoseconds.\n pub(crate) mean_poll_time: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks scheduled for execution on the worker's local queue.\n pub(crate) local_schedule_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// Number of tasks moved from the local queue to the global queue to free space.\n pub(crate) overflow_count: MetricAtomicU64,\n\n #[cfg(tokio_unstable)]\n /// If `Some`, tracks the number of polls by duration range.\n pub(super) poll_count_histogram: Option,\n}\n\nimpl WorkerMetrics {\n pub(crate) fn new() -> WorkerMetrics {\n WorkerMetrics::default()\n }\n\n pub(crate) fn set_queue_depth(&self, len: usize) {\n self.queue_depth.store(len, Relaxed);\n }\n\n pub(crate) fn set_thread_id(&self, thread_id: ThreadId) {\n *self.thread_id.lock().unwrap() = Some(thread_id);\n }\n\n cfg_metrics_variant! {\n stable: {\n pub(crate) fn from_config(_: &Config) -> WorkerMetrics {\n WorkerMetrics::new()\n }\n },\n unstable: {\n pub(crate) fn from_config(config: &Config) -> WorkerMetrics {\n let mut worker_metrics = WorkerMetrics::new();\n worker_metrics.poll_count_histogram = config\n .metrics_poll_count_histogram\n .as_ref()\n .map(|histogram_builder| histogram_builder.build());\n worker_metrics\n }\n }\n }\n\n cfg_unstable_metrics! {\n pub(crate) fn queue_depth(&self) -> usize {\n self.queue_depth.load(Relaxed)\n }\n\n pub(crate) fn thread_id(&self) -> Option {\n *self.thread_id.lock().unwrap()\n }\n }\n}\n" }, { "path": "tokio/src/runtime/mod.rs", "content": "//! The Tokio runtime.\n//!\n//! Unlike other Rust programs, asynchronous applications require runtime\n//! support. In particular, the following runtime services are necessary:\n//!\n//! * An **I/O event loop**, called the driver, which drives I/O resources and\n//! dispatches I/O events to tasks that depend on them.\n//! * A **scheduler** to execute [tasks] that use these I/O resources.\n//! * A **timer** for scheduling work to run after a set period of time.\n//!\n//! Tokio's [`Runtime`] bundles all of these services as a single type, allowing\n//! them to be started, shut down, and configured together. However, often it is\n//! not required to configure a [`Runtime`] manually, and a user may just use the\n//! [`tokio::main`] attribute macro, which creates a [`Runtime`] under the hood.\n//!\n//! # Choose your runtime\n//!\n//! Here is the rules of thumb to choose the right runtime for your application.\n//!\n//! ```plaintext\n//! +------------------------------------------------------+\n//! | Do you want work-stealing or multi-thread scheduler? |\n//! +------------------------------------------------------+\n//! | Yes | No\n//! | |\n//! | |\n//! v |\n//! +------------------------+ |\n//! | Multi-threaded Runtime | |\n//! +------------------------+ |\n//! |\n//! V\n//! +--------------------------------+\n//! | Do you execute `!Send` Future? |\n//! +--------------------------------+\n//! | Yes | No\n//! | |\n//! V |\n//! +--------------------------+ |\n//! | Local Runtime (unstable) | |\n//! +--------------------------+ |\n//! |\n//! v\n//! +------------------------+\n//! | Current-thread Runtime |\n//! +------------------------+\n//! ```\n//!\n//! The above decision tree is not exhaustive. there are other factors that\n//! may influence your decision.\n//!\n//! ## Bridging with sync code\n//!\n//! See for details.\n//!\n//! ## NUMA awareness\n//!\n//! The tokio runtime is not NUMA (Non-Uniform Memory Access) aware.\n//! You may want to start multiple runtimes instead of a single runtime\n//! for better performance on NUMA systems.\n//!\n//! # Usage\n//!\n//! When no fine tuning is required, the [`tokio::main`] attribute macro can be\n//! used.\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::net::TcpListener;\n//! use tokio::io::{AsyncReadExt, AsyncWriteExt};\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n//!\n//! loop {\n//! let (mut socket, _) = listener.accept().await?;\n//!\n//! tokio::spawn(async move {\n//! let mut buf = [0; 1024];\n//!\n//! // In a loop, read data from the socket and write the data back.\n//! loop {\n//! let n = match socket.read(&mut buf).await {\n//! // socket closed\n//! Ok(0) => return,\n//! Ok(n) => n,\n//! Err(e) => {\n//! println!(\"failed to read from socket; err = {:?}\", e);\n//! return;\n//! }\n//! };\n//!\n//! // Write the data back\n//! if let Err(e) = socket.write_all(&buf[0..n]).await {\n//! println!(\"failed to write to socket; err = {:?}\", e);\n//! return;\n//! }\n//! }\n//! });\n//! }\n//! }\n//! # }\n//! ```\n//!\n//! From within the context of the runtime, additional tasks are spawned using\n//! the [`tokio::spawn`] function. Futures spawned using this function will be\n//! executed on the same thread pool used by the [`Runtime`].\n//!\n//! A [`Runtime`] instance can also be used directly.\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::net::TcpListener;\n//! use tokio::io::{AsyncReadExt, AsyncWriteExt};\n//! use tokio::runtime::Runtime;\n//!\n//! fn main() -> Result<(), Box> {\n//! // Create the runtime\n//! let rt = Runtime::new()?;\n//!\n//! // Spawn the root task\n//! rt.block_on(async {\n//! let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n//!\n//! loop {\n//! let (mut socket, _) = listener.accept().await?;\n//!\n//! tokio::spawn(async move {\n//! let mut buf = [0; 1024];\n//!\n//! // In a loop, read data from the socket and write the data back.\n//! loop {\n//! let n = match socket.read(&mut buf).await {\n//! // socket closed\n//! Ok(0) => return,\n//! Ok(n) => n,\n//! Err(e) => {\n//! println!(\"failed to read from socket; err = {:?}\", e);\n//! return;\n//! }\n//! };\n//!\n//! // Write the data back\n//! if let Err(e) = socket.write_all(&buf[0..n]).await {\n//! println!(\"failed to write to socket; err = {:?}\", e);\n//! return;\n//! }\n//! }\n//! });\n//! }\n//! })\n//! }\n//! # }\n//! ```\n//!\n//! ## Runtime Configurations\n//!\n//! Tokio provides multiple task scheduling strategies, suitable for different\n//! applications. The [runtime builder] or `#[tokio::main]` attribute may be\n//! used to select which scheduler to use.\n//!\n//! #### Multi-Thread Scheduler\n//!\n//! The multi-thread scheduler executes futures on a _thread pool_, using a\n//! work-stealing strategy. By default, it will start a worker thread for each\n//! CPU core available on the system. This tends to be the ideal configuration\n//! for most applications. The multi-thread scheduler requires the `rt-multi-thread`\n//! feature flag, and is selected by default:\n//! ```\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::runtime;\n//!\n//! # fn main() -> Result<(), Box> {\n//! let threaded_rt = runtime::Runtime::new()?;\n//! # Ok(()) }\n//! # }\n//! ```\n//!\n//! Most applications should use the multi-thread scheduler, except in some\n//! niche use-cases, such as when running only a single thread is required.\n//!\n//! #### Current-Thread Scheduler\n//!\n//! The current-thread scheduler provides a _single-threaded_ future executor.\n//! All tasks will be created and executed on the current thread. This requires\n//! the `rt` feature flag.\n//! ```\n//! use tokio::runtime;\n//!\n//! # fn main() -> Result<(), Box> {\n//! let rt = runtime::Builder::new_current_thread()\n//! .build()?;\n//! # Ok(()) }\n//! ```\n//!\n//! #### Resource drivers\n//!\n//! When configuring a runtime by hand, no resource drivers are enabled by\n//! default. In this case, attempting to use networking types or time types will\n//! fail. In order to enable these types, the resource drivers must be enabled.\n//! This is done with [`Builder::enable_io`] and [`Builder::enable_time`]. As a\n//! shorthand, [`Builder::enable_all`] enables both resource drivers.\n//!\n//! ## Driving the runtime\n//!\n//! A Tokio runtime can only execute tasks if the runtime is running. Normally\n//! this is not an issue as the default configuration of a runtime is always running,\n//! but alternate configurations such as the current-thread runtime require that\n//! [`Runtime::block_on`] is called.\n//!\n//! - A multi-threaded runtime is always running because it spawns its own worker\n//! threads.\n//! - A current-thread runtime does not spawn any worker threads, so it can only\n//! execute tasks when you provide a thread by calling [`Runtime::block_on`].\n//! - A [`LocalSet`](crate::task::LocalSet) only executes local tasks spawned on\n//! it when the `LocalSet` is `.awaited` or otherwise driven using one of its\n//! methods for this purpose.\n//!\n//! Please be aware that [`Handle::block_on`] does not drive the runtime.\n//! There must be at least one call to [`Runtime::block_on`] when using the current\n//! thread runtime. [`Handle::block_on`] is not enough.\n//!\n//! ## Lifetime of spawned threads\n//!\n//! The runtime may spawn threads depending on its configuration and usage. The\n//! multi-thread scheduler spawns threads to schedule tasks and for `spawn_blocking`\n//! calls.\n//!\n//! While the `Runtime` is active, threads may shut down after periods of being\n//! idle. Once `Runtime` is dropped, all runtime threads have usually been\n//! terminated, but in the presence of unstoppable spawned work are not\n//! guaranteed to have been terminated. See the\n//! [struct level documentation](Runtime#shutdown) for more details.\n//!\n//! [tasks]: crate::task\n//! [`Runtime`]: Runtime\n//! [`tokio::spawn`]: crate::spawn\n//! [`tokio::main`]: ../attr.main.html\n//! [runtime builder]: crate::runtime::Builder\n//! [`Runtime::new`]: crate::runtime::Runtime::new\n//! [`Builder::enable_io`]: crate::runtime::Builder::enable_io\n//! [`Builder::enable_time`]: crate::runtime::Builder::enable_time\n//! [`Builder::enable_all`]: crate::runtime::Builder::enable_all\n//!\n//! # Detailed runtime behavior\n//!\n//! This section gives more details into how the Tokio runtime will schedule\n//! tasks for execution.\n//!\n//! At its most basic level, a runtime has a collection of tasks that need to be\n//! scheduled. It will repeatedly remove a task from that collection and\n//! schedule it (by calling [`poll`]). When the collection is empty, the thread\n//! will go to sleep until a task is added to the collection.\n//!\n//! However, the above is not sufficient to guarantee a well-behaved runtime.\n//! For example, the runtime might have a single task that is always ready to be\n//! scheduled, and schedule that task every time. This is a problem because it\n//! starves other tasks by not scheduling them. To solve this, Tokio provides\n//! the following fairness guarantee:\n//!\n//! > If the total number of tasks does not grow without bound, and no task is\n//! > [blocking the thread], then it is guaranteed that tasks are scheduled\n//! > fairly.\n//!\n//! Or, more formally:\n//!\n//! > Under the following two assumptions:\n//! >\n//! > * There is some number `MAX_TASKS` such that the total number of tasks on\n//! > the runtime at any specific point in time never exceeds `MAX_TASKS`.\n//! > * There is some number `MAX_SCHEDULE` such that calling [`poll`] on any\n//! > task spawned on the runtime returns within `MAX_SCHEDULE` time units.\n//! >\n//! > Then, there is some number `MAX_DELAY` such that when a task is woken, it\n//! > will be scheduled by the runtime within `MAX_DELAY` time units.\n//!\n//! (Here, `MAX_TASKS` and `MAX_SCHEDULE` can be any number and the user of\n//! the runtime may choose them. The `MAX_DELAY` number is controlled by the\n//! runtime, and depends on the value of `MAX_TASKS` and `MAX_SCHEDULE`.)\n//!\n//! Other than the above fairness guarantee, there is no guarantee about the\n//! order in which tasks are scheduled. There is also no guarantee that the\n//! runtime is equally fair to all tasks. For example, if the runtime has two\n//! tasks A and B that are both ready, then the runtime may schedule A five\n//! times before it schedules B. This is the case even if A yields using\n//! [`yield_now`]. All that is guaranteed is that it will schedule B eventually.\n//!\n//! Normally, tasks are scheduled only if they have been woken by calling\n//! [`wake`] on their waker. However, this is not guaranteed, and Tokio may\n//! schedule tasks that have not been woken under some circumstances. This is\n//! called a spurious wakeup.\n//!\n//! ## IO and timers\n//!\n//! Beyond just scheduling tasks, the runtime must also manage IO resources and\n//! timers. It does this by periodically checking whether there are any IO\n//! resources or timers that are ready, and waking the relevant task so that\n//! it will be scheduled.\n//!\n//! These checks are performed periodically between scheduling tasks. Under the\n//! same assumptions as the previous fairness guarantee, Tokio guarantees that\n//! it will wake tasks with an IO or timer event within some maximum number of\n//! time units.\n//!\n//! ## Current thread runtime (behavior at the time of writing)\n//!\n//! This section describes how the [current thread runtime] behaves today. This\n//! behavior may change in future versions of Tokio.\n//!\n//! The current thread runtime maintains two FIFO queues of tasks that are ready\n//! to be scheduled: the global queue and the local queue. The runtime will prefer\n//! to choose the next task to schedule from the local queue, and will only pick a\n//! task from the global queue if the local queue is empty, or if it has picked\n//! a task from the local queue 31 times in a row. The number 31 can be\n//! changed using the [`global_queue_interval`] setting.\n//!\n//! The runtime will check for new IO or timer events whenever there are no\n//! tasks ready to be scheduled, or when it has scheduled 61 tasks in a row. The\n//! number 61 may be changed using the [`event_interval`] setting.\n//!\n//! When a task is woken from within a task running on the runtime, then the\n//! woken task is added directly to the local queue. Otherwise, the task is\n//! added to the global queue. The current thread runtime does not use [the lifo\n//! slot optimization].\n//!\n//! ## Multi threaded runtime (behavior at the time of writing)\n//!\n//! This section describes how the [multi thread runtime] behaves today. This\n//! behavior may change in future versions of Tokio.\n//!\n//! A multi thread runtime has a fixed number of worker threads, which are all\n//! created on startup. The multi thread runtime maintains one global queue, and\n//! a local queue for each worker thread. The local queue of a worker thread can\n//! fit at most 256 tasks. If more than 256 tasks are added to the local queue,\n//! then half of them are moved to the global queue to make space.\n//!\n//! The runtime will prefer to choose the next task to schedule from the local\n//! queue, and will only pick a task from the global queue if the local queue is\n//! empty, or if it has picked a task from the local queue\n//! [`global_queue_interval`] times in a row. If the value of\n//! [`global_queue_interval`] is not explicitly set using the runtime builder,\n//! then the runtime will dynamically compute it using a heuristic that targets\n//! 10ms intervals between each check of the global queue (based on the\n//! [`worker_mean_poll_time`] metric).\n//!\n//! If both the local queue and global queue is empty, then the worker thread\n//! will attempt to steal tasks from the local queue of another worker thread.\n//! Stealing is done by moving half of the tasks in one local queue to another\n//! local queue.\n//!\n//! The runtime will check for new IO or timer events whenever there are no\n//! tasks ready to be scheduled, or when it has scheduled 61 tasks in a row. The\n//! number 61 may be changed using the [`event_interval`] setting.\n//!\n//! The multi thread runtime uses [the lifo slot optimization]: Whenever a task\n//! wakes up another task, the other task is added to the worker thread's lifo\n//! slot instead of being added to a queue. If there was already a task in the\n//! lifo slot when this happened, then the lifo slot is replaced, and the task\n//! that used to be in the lifo slot is placed in the thread's local queue.\n//! When the runtime finishes scheduling a task, it will schedule the task in\n//! the lifo slot immediately, if any. When the lifo slot is used, the [coop\n//! budget] is not reset. Furthermore, if a worker thread uses the lifo slot\n//! three times in a row, it is temporarily disabled until the worker thread has\n//! scheduled a task that didn't come from the lifo slot. The lifo slot can be\n//! disabled using the [`disable_lifo_slot`] setting. The lifo slot is separate\n//! from the local queue, so other worker threads cannot steal the task in the\n//! lifo slot.\n//!\n//! When a task is woken from a thread that is not a worker thread, then the\n//! task is placed in the global queue.\n//!\n//! [`poll`]: std::future::Future::poll\n//! [`wake`]: std::task::Waker::wake\n//! [`yield_now`]: crate::task::yield_now\n//! [blocking the thread]: https://ryhl.io/blog/async-what-is-blocking/\n//! [current thread runtime]: crate::runtime::Builder::new_current_thread\n//! [multi thread runtime]: crate::runtime::Builder::new_multi_thread\n//! [`global_queue_interval`]: crate::runtime::Builder::global_queue_interval\n//! [`event_interval`]: crate::runtime::Builder::event_interval\n//! [`disable_lifo_slot`]: crate::runtime::Builder::disable_lifo_slot\n//! [the lifo slot optimization]: crate::runtime::Builder::disable_lifo_slot\n//! [coop budget]: crate::task::coop#cooperative-scheduling\n//! [`worker_mean_poll_time`]: crate::runtime::RuntimeMetrics::worker_mean_poll_time\n\n// At the top due to macros\n#[cfg(test)]\n#[cfg(not(target_family = \"wasm\"))]\n#[macro_use]\nmod tests;\n\npub(crate) mod context;\n\npub(crate) mod park;\n\npub(crate) mod driver;\n\npub(crate) mod scheduler;\n\ncfg_io_driver_impl! {\n pub(crate) mod io;\n}\n\ncfg_process_driver! {\n mod process;\n}\n\n#[cfg_attr(not(feature = \"time\"), allow(dead_code))]\n#[derive(Debug, Copy, Clone, PartialEq)]\npub(crate) enum TimerFlavor {\n Traditional,\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Alternative,\n}\n\ncfg_time! {\n pub(crate) mod time;\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n pub(crate) mod time_alt;\n\n use std::task::{Context, Poll};\n use std::pin::Pin;\n\n #[derive(Debug)]\n pub(crate) enum Timer {\n Traditional(time::TimerEntry),\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Alternative(time_alt::Timer),\n }\n\n impl Timer {\n #[track_caller]\n pub(crate) fn new(\n handle: crate::runtime::scheduler::Handle,\n deadline: crate::time::Instant,\n ) -> Self {\n match handle.timer_flavor() {\n crate::runtime::TimerFlavor::Traditional => {\n Timer::Traditional(time::TimerEntry::new(handle, deadline))\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n crate::runtime::TimerFlavor::Alternative => {\n Timer::Alternative(time_alt::Timer::new(handle, deadline))\n }\n }\n }\n\n pub(crate) fn deadline(&self) -> crate::time::Instant {\n match self {\n Timer::Traditional(entry) => entry.deadline(),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => entry.deadline(),\n }\n }\n\n pub(crate) fn is_elapsed(&self) -> bool {\n match self {\n Timer::Traditional(entry) => entry.is_elapsed(),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => entry.is_elapsed(),\n }\n }\n\n pub(crate) fn flavor(self: Pin<&Self>) -> TimerFlavor {\n match self.get_ref() {\n Timer::Traditional(_) => TimerFlavor::Traditional,\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(_) => TimerFlavor::Alternative,\n }\n }\n\n pub(crate) fn reset(\n self: Pin<&mut Self>,\n new_time: crate::time::Instant,\n reregister: bool\n ) {\n // Safety: we never move the inner entries.\n let this = unsafe { self.get_unchecked_mut() };\n match this {\n Timer::Traditional(entry) => {\n // Safety: we never move the inner entries.\n unsafe { Pin::new_unchecked(entry).reset(new_time, reregister); }\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(_) => panic!(\"not implemented yet\"),\n }\n }\n\n pub(crate) fn poll_elapsed(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n // Safety: we never move the inner entries.\n let this = unsafe { self.get_unchecked_mut() };\n match this {\n Timer::Traditional(entry) => {\n // Safety: we never move the inner entries.\n unsafe { Pin::new_unchecked(entry).poll_elapsed(cx) }\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => {\n // Safety: we never move the inner entries.\n unsafe { Pin::new_unchecked(entry).poll_elapsed(cx).map(Ok) }\n }\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n pub(crate) fn scheduler_handle(&self) -> &crate::runtime::scheduler::Handle {\n match self {\n Timer::Traditional(_) => unreachable!(\"we should not call this on Traditional Timer\"),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => entry.scheduler_handle(),\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) fn driver(self: Pin<&Self>) -> &crate::runtime::time::Handle {\n match self.get_ref() {\n Timer::Traditional(entry) => entry.driver(),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => entry.driver(),\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) fn clock(self: Pin<&Self>) -> &crate::time::Clock {\n match self.get_ref() {\n Timer::Traditional(entry) => entry.clock(),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Timer::Alternative(entry) => entry.clock(),\n }\n }\n }\n}\n\ncfg_signal_internal_and_unix! {\n pub(crate) mod signal;\n}\n\ncfg_rt! {\n pub(crate) mod task;\n\n mod config;\n use config::Config;\n\n mod blocking;\n #[cfg_attr(target_os = \"wasi\", allow(unused_imports))]\n pub(crate) use blocking::spawn_blocking;\n\n cfg_trace! {\n pub(crate) use blocking::Mandatory;\n }\n\n cfg_fs! {\n pub(crate) use blocking::spawn_mandatory_blocking;\n }\n\n mod builder;\n pub use self::builder::Builder;\n cfg_unstable! {\n pub use self::builder::UnhandledPanic;\n pub use crate::util::rand::RngSeed;\n\n mod local_runtime;\n pub use local_runtime::{LocalRuntime, LocalOptions};\n\n /// Returns the index of the current worker thread, if called from a\n /// runtime worker thread.\n ///\n /// The returned value is a 0-based index matching the worker indices\n /// used by [`RuntimeMetrics`] methods such as\n /// [`worker_total_busy_duration`](RuntimeMetrics::worker_total_busy_duration).\n ///\n /// Returns `None` when called from outside a runtime worker thread\n /// (for example, from a blocking thread or a non-Tokio thread). On the\n /// multi-thread runtime, the thread that calls [`Runtime::block_on`] is\n /// not a worker thread, so this also returns `None` there.\n ///\n /// For the current-thread runtime and [`LocalRuntime`], this always\n /// returns `Some(0)` (including inside `block_on`, since the calling\n /// thread *is* the worker thread).\n ///\n /// Note that the result may change across `.await` points, as the\n /// task may be moved to a different worker thread by the scheduler.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// #[tokio::main(flavor = \"multi_thread\", worker_threads = 4)]\n /// async fn main() {\n /// let index = tokio::spawn(async {\n /// tokio::runtime::worker_index()\n /// }).await.unwrap();\n /// println!(\"Task ran on worker {:?}\", index);\n /// }\n /// # }\n /// ```\n pub fn worker_index() -> Option {\n context::worker_index()\n }\n }\n\n cfg_taskdump! {\n pub mod dump;\n pub use dump::Dump;\n }\n\n mod task_hooks;\n pub(crate) use task_hooks::{TaskHooks, TaskCallback};\n cfg_unstable! {\n pub use task_hooks::TaskMeta;\n }\n #[cfg(not(tokio_unstable))]\n pub(crate) use task_hooks::TaskMeta;\n\n mod handle;\n pub use handle::{EnterGuard, Handle, TryCurrentError};\n\n mod runtime;\n pub use runtime::{Runtime, RuntimeFlavor, is_rt_shutdown_err};\n\n mod id;\n pub use id::Id;\n\n\n /// Boundary value to prevent stack overflow caused by a large-sized\n /// Future being placed in the stack.\n pub(crate) const BOX_FUTURE_THRESHOLD: usize = if cfg!(debug_assertions) {\n 2048\n } else {\n 16384\n };\n\n mod thread_id;\n pub(crate) use thread_id::ThreadId;\n\n pub(crate) mod metrics;\n pub use metrics::RuntimeMetrics;\n\n cfg_unstable_metrics! {\n pub use metrics::{HistogramScale, HistogramConfiguration, LogHistogram, LogHistogramBuilder, InvalidHistogramConfiguration} ;\n\n cfg_net! {\n pub(crate) use metrics::IoDriverMetrics;\n }\n }\n\n pub(crate) use metrics::{MetricsBatch, SchedulerMetrics, WorkerMetrics, HistogramBuilder};\n\n /// After thread starts / before thread stops\n type Callback = std::sync::Arc;\n}\n" }, { "path": "tokio/src/runtime/park.rs", "content": "#![cfg_attr(not(feature = \"full\"), allow(dead_code))]\n\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::{Arc, Condvar, Mutex};\n\nuse std::sync::atomic::Ordering::SeqCst;\nuse std::time::Duration;\n\n#[derive(Debug)]\npub(crate) struct ParkThread {\n inner: Arc,\n}\n\n/// Unblocks a thread that was blocked by `ParkThread`.\n#[derive(Clone, Debug)]\npub(crate) struct UnparkThread {\n inner: Arc,\n}\n\n#[derive(Debug)]\nstruct Inner {\n state: AtomicUsize,\n mutex: Mutex<()>,\n condvar: Condvar,\n}\n\nconst EMPTY: usize = 0;\nconst PARKED: usize = 1;\nconst NOTIFIED: usize = 2;\n\ntokio_thread_local! {\n static CURRENT_PARKER: ParkThread = ParkThread::new();\n}\n\n// Bit of a hack, but it is only for loom\n#[cfg(loom)]\ntokio_thread_local! {\n pub(crate) static CURRENT_THREAD_PARK_COUNT: AtomicUsize = AtomicUsize::new(0);\n}\n\n// ==== impl ParkThread ====\n\nimpl ParkThread {\n pub(crate) fn new() -> Self {\n Self {\n inner: Arc::new(Inner {\n state: AtomicUsize::new(EMPTY),\n mutex: Mutex::new(()),\n condvar: Condvar::new(),\n }),\n }\n }\n\n pub(crate) fn unpark(&self) -> UnparkThread {\n let inner = self.inner.clone();\n UnparkThread { inner }\n }\n\n pub(crate) fn park(&mut self) {\n #[cfg(loom)]\n CURRENT_THREAD_PARK_COUNT.with(|count| count.fetch_add(1, SeqCst));\n self.inner.park();\n }\n\n pub(crate) fn park_timeout(&mut self, duration: Duration) {\n #[cfg(loom)]\n CURRENT_THREAD_PARK_COUNT.with(|count| count.fetch_add(1, SeqCst));\n self.inner.park_timeout(duration);\n }\n\n pub(crate) fn shutdown(&mut self) {\n self.inner.shutdown();\n }\n}\n\n// ==== impl Inner ====\n\nimpl Inner {\n fn park(&self) {\n // If we were previously notified then we consume this notification and\n // return quickly.\n if self\n .state\n .compare_exchange(NOTIFIED, EMPTY, SeqCst, SeqCst)\n .is_ok()\n {\n return;\n }\n\n // Otherwise we need to coordinate going to sleep\n let mut m = self.mutex.lock();\n\n match self.state.compare_exchange(EMPTY, PARKED, SeqCst, SeqCst) {\n Ok(_) => {}\n Err(NOTIFIED) => {\n // We must read here, even though we know it will be `NOTIFIED`.\n // This is because `unpark` may have been called again since we read\n // `NOTIFIED` in the `compare_exchange` above. We must perform an\n // acquire operation that synchronizes with that `unpark` to observe\n // any writes it made before the call to unpark. To do that we must\n // read from the write it made to `state`.\n let old = self.state.swap(EMPTY, SeqCst);\n debug_assert_eq!(old, NOTIFIED, \"park state changed unexpectedly\");\n\n return;\n }\n Err(actual) => panic!(\"inconsistent park state; actual = {actual}\"),\n }\n\n loop {\n m = self.condvar.wait(m).unwrap();\n\n if self\n .state\n .compare_exchange(NOTIFIED, EMPTY, SeqCst, SeqCst)\n .is_ok()\n {\n // got a notification\n return;\n }\n\n // spurious wakeup, go back to sleep\n }\n }\n\n /// Parks the current thread for at most `dur`.\n fn park_timeout(&self, dur: Duration) {\n // Like `park` above we have a fast path for an already-notified thread,\n // and afterwards we start coordinating for a sleep. Return quickly.\n if self\n .state\n .compare_exchange(NOTIFIED, EMPTY, SeqCst, SeqCst)\n .is_ok()\n {\n return;\n }\n\n if dur == Duration::from_millis(0) {\n return;\n }\n\n let m = self.mutex.lock();\n\n match self.state.compare_exchange(EMPTY, PARKED, SeqCst, SeqCst) {\n Ok(_) => {}\n Err(NOTIFIED) => {\n // We must read again here, see `park`.\n let old = self.state.swap(EMPTY, SeqCst);\n debug_assert_eq!(old, NOTIFIED, \"park state changed unexpectedly\");\n\n return;\n }\n Err(actual) => panic!(\"inconsistent park_timeout state; actual = {actual}\"),\n }\n\n #[cfg(not(all(target_family = \"wasm\", not(target_feature = \"atomics\"))))]\n // Wait with a timeout, and if we spuriously wake up or otherwise wake up\n // from a notification, we just want to unconditionally set the state back to\n // empty, either consuming a notification or un-flagging ourselves as\n // parked.\n let (_m, _result) = self.condvar.wait_timeout(m, dur).unwrap();\n\n #[cfg(all(target_family = \"wasm\", not(target_feature = \"atomics\")))]\n // Wasm without atomics doesn't have threads, so just sleep.\n {\n let _m = m;\n std::thread::sleep(dur);\n }\n\n match self.state.swap(EMPTY, SeqCst) {\n NOTIFIED => {} // got a notification, hurray!\n PARKED => {} // no notification, alas\n n => panic!(\"inconsistent park_timeout state: {n}\"),\n }\n }\n\n fn unpark(&self) {\n // To ensure the unparked thread will observe any writes we made before\n // this call, we must perform a release operation that `park` can\n // synchronize with. To do that we must write `NOTIFIED` even if `state`\n // is already `NOTIFIED`. That is why this must be a swap rather than a\n // compare-and-swap that returns if it reads `NOTIFIED` on failure.\n match self.state.swap(NOTIFIED, SeqCst) {\n EMPTY => return, // no one was waiting\n NOTIFIED => return, // already unparked\n PARKED => {} // gotta go wake someone up\n _ => panic!(\"inconsistent state in unpark\"),\n }\n\n // There is a period between when the parked thread sets `state` to\n // `PARKED` (or last checked `state` in the case of a spurious wake\n // up) and when it actually waits on `cvar`. If we were to notify\n // during this period it would be ignored and then when the parked\n // thread went to sleep it would never wake up. Fortunately, it has\n // `lock` locked at this stage so we can acquire `lock` to wait until\n // it is ready to receive the notification.\n //\n // Releasing `lock` before the call to `notify_one` means that when the\n // parked thread wakes it doesn't get woken only to have to wait for us\n // to release `lock`.\n drop(self.mutex.lock());\n\n self.condvar.notify_one();\n }\n\n fn shutdown(&self) {\n self.condvar.notify_all();\n }\n}\n\nimpl Default for ParkThread {\n fn default() -> Self {\n Self::new()\n }\n}\n\n// ===== impl UnparkThread =====\n\nimpl UnparkThread {\n pub(crate) fn unpark(&self) {\n self.inner.unpark();\n }\n}\n\nuse crate::loom::thread::AccessError;\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::rc::Rc;\nuse std::task::{RawWaker, RawWakerVTable, Waker};\n\n/// Blocks the current thread using a condition variable.\n#[derive(Debug)]\npub(crate) struct CachedParkThread {\n _anchor: PhantomData>,\n}\n\nimpl CachedParkThread {\n /// Creates a new `ParkThread` handle for the current thread.\n ///\n /// This type cannot be moved to other threads, so it should be created on\n /// the thread that the caller intends to park.\n pub(crate) fn new() -> CachedParkThread {\n CachedParkThread {\n _anchor: PhantomData,\n }\n }\n\n pub(crate) fn waker(&self) -> Result {\n self.unpark().map(UnparkThread::into_waker)\n }\n\n fn unpark(&self) -> Result {\n self.with_current(ParkThread::unpark)\n }\n\n pub(crate) fn park(&mut self) {\n self.with_current(|park_thread| park_thread.inner.park())\n .unwrap();\n }\n\n pub(crate) fn park_timeout(&mut self, duration: Duration) {\n self.with_current(|park_thread| park_thread.inner.park_timeout(duration))\n .unwrap();\n }\n\n /// Gets a reference to the `ParkThread` handle for this thread.\n fn with_current(&self, f: F) -> Result\n where\n F: FnOnce(&ParkThread) -> R,\n {\n CURRENT_PARKER.try_with(|inner| f(inner))\n }\n\n pub(crate) fn block_on(&mut self, f: F) -> Result {\n use std::task::Context;\n use std::task::Poll::Ready;\n\n let waker = self.waker()?;\n let mut cx = Context::from_waker(&waker);\n\n pin!(f);\n\n loop {\n if let Ready(v) = crate::task::coop::budget(|| f.as_mut().poll(&mut cx)) {\n return Ok(v);\n }\n\n self.park();\n }\n }\n}\n\nimpl UnparkThread {\n pub(crate) fn into_waker(self) -> Waker {\n unsafe {\n let raw = unparker_to_raw_waker(self.inner);\n Waker::from_raw(raw)\n }\n }\n}\n\nimpl Inner {\n #[allow(clippy::wrong_self_convention)]\n fn into_raw(this: Arc) -> *const () {\n Arc::into_raw(this) as *const ()\n }\n\n /// # Safety\n ///\n /// The pointer must have been created by [`Self::into_raw`].\n unsafe fn from_raw(ptr: *const ()) -> Arc {\n unsafe { Arc::from_raw(ptr as *const Inner) }\n }\n}\n\n// TODO: Is this really an unsafe function?\nunsafe fn unparker_to_raw_waker(unparker: Arc) -> RawWaker {\n RawWaker::new(\n Inner::into_raw(unparker),\n &RawWakerVTable::new(clone, wake, wake_by_ref, drop_waker),\n )\n}\n\n/// # Safety\n///\n/// The pointer must have been created by [`Inner::into_raw`].\nunsafe fn clone(raw: *const ()) -> RawWaker {\n unsafe {\n Arc::increment_strong_count(raw as *const Inner);\n }\n unsafe { unparker_to_raw_waker(Inner::from_raw(raw)) }\n}\n\n/// # Safety\n///\n/// The pointer must have been created by [`Inner::into_raw`].\nunsafe fn drop_waker(raw: *const ()) {\n drop(unsafe { Inner::from_raw(raw) });\n}\n\n/// # Safety\n///\n/// The pointer must have been created by [`Inner::into_raw`].\nunsafe fn wake(raw: *const ()) {\n let unparker = unsafe { Inner::from_raw(raw) };\n unparker.unpark();\n}\n\n/// # Safety\n///\n/// The pointer must have been created by [`Inner::into_raw`].\nunsafe fn wake_by_ref(raw: *const ()) {\n let raw = raw as *const Inner;\n unsafe {\n (*raw).unpark();\n }\n}\n\n#[cfg(loom)]\npub(crate) fn current_thread_park_count() -> usize {\n CURRENT_THREAD_PARK_COUNT.with(|count| count.load(SeqCst))\n}\n" }, { "path": "tokio/src/runtime/process.rs", "content": "#![cfg_attr(not(feature = \"rt\"), allow(dead_code))]\n\n//! Process driver.\n\nuse crate::process::unix::GlobalOrphanQueue;\nuse crate::runtime::driver;\nuse crate::runtime::signal::{Driver as SignalDriver, Handle as SignalHandle};\n\nuse std::time::Duration;\n\n/// Responsible for cleaning up orphaned child processes on Unix platforms.\n#[derive(Debug)]\npub(crate) struct Driver {\n park: SignalDriver,\n signal_handle: SignalHandle,\n}\n\n// ===== impl Driver =====\n\nimpl Driver {\n /// Creates a new signal `Driver` instance that delegates wakeups to `park`.\n pub(crate) fn new(park: SignalDriver) -> Self {\n let signal_handle = park.handle();\n\n Self {\n park,\n signal_handle,\n }\n }\n\n pub(crate) fn park(&mut self, handle: &driver::Handle) {\n self.park.park(handle);\n GlobalOrphanQueue::reap_orphans(&self.signal_handle);\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &driver::Handle, duration: Duration) {\n self.park.park_timeout(handle, duration);\n GlobalOrphanQueue::reap_orphans(&self.signal_handle);\n }\n\n pub(crate) fn shutdown(&mut self, handle: &driver::Handle) {\n self.park.shutdown(handle);\n }\n}\n" }, { "path": "tokio/src/runtime/runtime.rs", "content": "use super::BOX_FUTURE_THRESHOLD;\nuse crate::runtime::blocking::BlockingPool;\nuse crate::runtime::scheduler::CurrentThread;\nuse crate::runtime::{context, EnterGuard, Handle};\nuse crate::task::JoinHandle;\nuse crate::util::error::RUNTIME_SHUTTING_DOWN_ERROR;\nuse crate::util::trace::SpawnMeta;\n\nuse std::future::Future;\nuse std::io;\nuse std::mem;\nuse std::time::Duration;\n\ncfg_rt_multi_thread! {\n use crate::runtime::Builder;\n use crate::runtime::scheduler::MultiThread;\n}\n\n/// The Tokio runtime.\n///\n/// The runtime provides an I/O driver, task scheduler, [timer], and\n/// blocking pool, necessary for running asynchronous tasks.\n///\n/// Instances of `Runtime` can be created using [`new`], or [`Builder`].\n/// However, most users will use the [`#[tokio::main]`][main] annotation on\n/// their entry point instead.\n///\n/// See [module level][mod] documentation for more details.\n///\n/// # Shutdown\n///\n/// Shutting down the runtime is done by dropping the value, or calling\n/// [`shutdown_background`] or [`shutdown_timeout`].\n///\n/// Tasks spawned through [`Runtime::spawn`] keep running until they yield.\n/// Then they are dropped. They are not *guaranteed* to run to completion, but\n/// *might* do so if they do not yield until completion.\n///\n/// Blocking functions spawned through [`Runtime::spawn_blocking`] keep running\n/// until they return.\n///\n/// The thread initiating the shutdown blocks until all spawned work has been\n/// stopped. This can take an indefinite amount of time. The `Drop`\n/// implementation waits forever for this.\n///\n/// The [`shutdown_background`] and [`shutdown_timeout`] methods can be used if\n/// waiting forever is undesired. When the timeout is reached, spawned work that\n/// did not stop in time and threads running it are leaked. The work continues\n/// to run until one of the stopping conditions is fulfilled, but the thread\n/// initiating the shutdown is unblocked.\n///\n/// Once the runtime has been dropped, any outstanding I/O resources bound to\n/// it will no longer function. Calling any method on them will result in an\n/// error.\n///\n/// # Sharing\n///\n/// There are several ways to establish shared access to a Tokio runtime:\n///\n/// * Using an [Arc]\\.\n/// * Using a [`Handle`].\n/// * Entering the runtime context.\n///\n/// Using an [Arc]\\ or [`Handle`] allows you to do various\n/// things with the runtime such as spawning new tasks or entering the runtime\n/// context. Both types can be cloned to create a new handle that allows access\n/// to the same runtime. By passing clones into different tasks or threads, you\n/// will be able to access the runtime from those tasks or threads.\n///\n/// The difference between [Arc]\\ and [`Handle`] is that\n/// an [Arc]\\ will prevent the runtime from shutting down,\n/// whereas a [`Handle`] does not prevent that. This is because shutdown of the\n/// runtime happens when the destructor of the `Runtime` object runs.\n///\n/// Calls to [`shutdown_background`] and [`shutdown_timeout`] require exclusive\n/// ownership of the `Runtime` type. When using an [Arc]\\,\n/// this can be achieved via [`Arc::try_unwrap`] when only one strong count\n/// reference is left over.\n///\n/// The runtime context is entered using the [`Runtime::enter`] or\n/// [`Handle::enter`] methods, which use a thread-local variable to store the\n/// current runtime. Whenever you are inside the runtime context, methods such\n/// as [`tokio::spawn`] will use the runtime whose context you are inside.\n///\n/// [timer]: crate::time\n/// [mod]: index.html\n/// [`new`]: method@Self::new\n/// [`Builder`]: struct@Builder\n/// [`Handle`]: struct@Handle\n/// [main]: macro@crate::main\n/// [`tokio::spawn`]: crate::spawn\n/// [`Arc::try_unwrap`]: std::sync::Arc::try_unwrap\n/// [Arc]: std::sync::Arc\n/// [`shutdown_background`]: method@Runtime::shutdown_background\n/// [`shutdown_timeout`]: method@Runtime::shutdown_timeout\n#[derive(Debug)]\npub struct Runtime {\n /// Task scheduler\n scheduler: Scheduler,\n\n /// Handle to runtime, also contains driver handles\n handle: Handle,\n\n /// Blocking pool handle, used to signal shutdown\n blocking_pool: BlockingPool,\n}\n\n/// The flavor of a `Runtime`.\n///\n/// This is the return type for [`Handle::runtime_flavor`](crate::runtime::Handle::runtime_flavor()).\n#[derive(Debug, PartialEq, Eq)]\n#[non_exhaustive]\npub enum RuntimeFlavor {\n /// The flavor that executes all tasks on the current thread.\n CurrentThread,\n /// The flavor that executes tasks across multiple threads.\n MultiThread,\n}\n\n/// The runtime scheduler is either a multi-thread or a current-thread executor.\n#[derive(Debug)]\npub(super) enum Scheduler {\n /// Execute all tasks on the current-thread.\n CurrentThread(CurrentThread),\n\n /// Execute tasks across multiple threads.\n #[cfg(feature = \"rt-multi-thread\")]\n MultiThread(MultiThread),\n}\n\nimpl Runtime {\n pub(super) fn from_parts(\n scheduler: Scheduler,\n handle: Handle,\n blocking_pool: BlockingPool,\n ) -> Runtime {\n Runtime {\n scheduler,\n handle,\n blocking_pool,\n }\n }\n\n /// Creates a new runtime instance with default configuration values.\n ///\n /// This results in the multi threaded scheduler, I/O driver, and time driver being\n /// initialized.\n ///\n /// Most applications will not need to call this function directly. Instead,\n /// they will use the [`#[tokio::main]` attribute][main]. When a more complex\n /// configuration is necessary, the [runtime builder] may be used.\n ///\n /// See [module level][mod] documentation for more details.\n ///\n /// # Examples\n ///\n /// Creating a new `Runtime` with default configuration values.\n ///\n /// ```\n /// use tokio::runtime::Runtime;\n ///\n /// let rt = Runtime::new()\n /// .unwrap();\n ///\n /// // Use the runtime...\n /// ```\n ///\n /// [mod]: index.html\n /// [main]: ../attr.main.html\n /// [threaded scheduler]: index.html#threaded-scheduler\n /// [runtime builder]: crate::runtime::Builder\n #[cfg(feature = \"rt-multi-thread\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt-multi-thread\")))]\n pub fn new() -> std::io::Result {\n Builder::new_multi_thread().enable_all().build()\n }\n\n /// Returns a handle to the runtime's spawner.\n ///\n /// The returned handle can be used to spawn tasks that run on this runtime, and can\n /// be cloned to allow moving the `Handle` to other threads.\n ///\n /// Calling [`Handle::block_on`] on a handle to a `current_thread` runtime is error-prone.\n /// Refer to the documentation of [`Handle::block_on`] for more.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// let rt = Runtime::new()\n /// .unwrap();\n ///\n /// let handle = rt.handle();\n ///\n /// // Use the handle...\n /// # }\n /// ```\n pub fn handle(&self) -> &Handle {\n &self.handle\n }\n\n /// Spawns a future onto the Tokio runtime.\n ///\n /// This spawns the given future onto the runtime's executor, usually a\n /// thread pool. The thread pool is then responsible for polling the future\n /// until it completes.\n ///\n /// The provided future will start running in the background immediately\n /// when `spawn` is called, even if you don't await the returned\n /// `JoinHandle` (assuming that the runtime [is running][running-runtime]).\n ///\n /// See [module level][mod] documentation for more details.\n ///\n /// [mod]: index.html\n /// [running-runtime]: index.html#driving-the-runtime\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n ///\n /// // Spawn a future onto the runtime\n /// rt.spawn(async {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn(&self, future: F) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n let fut_size = mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n self.handle\n .spawn_named(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n self.handle\n .spawn_named(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n /// Runs the provided function on an executor dedicated to blocking operations.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// # fn dox() {\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n ///\n /// // Spawn a blocking function onto the runtime\n /// rt.spawn_blocking(|| {\n /// println!(\"now running on a worker thread\");\n /// });\n /// # }\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn_blocking(&self, func: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n self.handle.spawn_blocking(func)\n }\n\n /// Runs a future to completion on the Tokio runtime. This is the\n /// runtime's entry point.\n ///\n /// This runs the given future on the current thread, blocking until it is\n /// complete, and yielding its resolved result. Any tasks or timers\n /// which the future spawns internally will be executed on the runtime.\n ///\n /// # Non-worker future\n ///\n /// Note that the future required by this function does not run as a\n /// worker. The expectation is that other tasks are spawned by the future here.\n /// Awaiting on other futures from the future provided here will not\n /// perform as fast as those spawned as workers.\n ///\n /// # Multi thread scheduler\n ///\n /// When the multi thread scheduler is used this will allow futures\n /// to run within the io driver and timer context of the overall runtime.\n ///\n /// Any spawned tasks will continue running after `block_on` returns.\n ///\n /// # Current thread scheduler\n ///\n /// When the current thread scheduler is enabled `block_on`\n /// can be called concurrently from multiple threads. The first call\n /// will take ownership of the io and timer drivers. This means\n /// other threads which do not own the drivers will hook into that one.\n /// When the first `block_on` completes, other threads will be able to\n /// \"steal\" the driver to allow continued execution of their futures.\n ///\n /// Any spawned tasks will be suspended after `block_on` returns. Calling\n /// `block_on` again will resume previously spawned tasks.\n ///\n /// # Panics\n ///\n /// This function panics if the provided future panics, or if called within an\n /// asynchronous execution context.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// // Create the runtime\n /// let rt = Runtime::new().unwrap();\n ///\n /// // Execute the future, blocking the current thread until completion\n /// rt.block_on(async {\n /// println!(\"hello\");\n /// });\n /// # }\n /// ```\n ///\n /// [handle]: fn@Handle::block_on\n #[track_caller]\n pub fn block_on(&self, future: F) -> F::Output {\n let fut_size = mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n self.block_on_inner(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n self.block_on_inner(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n #[track_caller]\n fn block_on_inner(&self, future: F, _meta: SpawnMeta<'_>) -> F::Output {\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n let future = super::task::trace::Trace::root(future);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let future = crate::util::trace::task(\n future,\n \"block_on\",\n _meta,\n crate::runtime::task::Id::next().as_u64(),\n );\n\n let _enter = self.enter();\n\n match &self.scheduler {\n Scheduler::CurrentThread(exec) => exec.block_on(&self.handle.inner, future),\n #[cfg(feature = \"rt-multi-thread\")]\n Scheduler::MultiThread(exec) => exec.block_on(&self.handle.inner, future),\n }\n }\n\n /// Enters the runtime context.\n ///\n /// This allows you to construct types that must have an executor\n /// available on creation such as [`Sleep`] or [`TcpStream`]. It will\n /// also allow you to call methods such as [`tokio::spawn`].\n ///\n /// [`Sleep`]: struct@crate::time::Sleep\n /// [`TcpStream`]: struct@crate::net::TcpStream\n /// [`tokio::spawn`]: fn@crate::spawn\n ///\n /// # Example\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n /// use tokio::task::JoinHandle;\n ///\n /// fn function_that_spawns(msg: String) -> JoinHandle<()> {\n /// // Had we not used `rt.enter` below, this would panic.\n /// tokio::spawn(async move {\n /// println!(\"{}\", msg);\n /// })\n /// }\n ///\n /// fn main() {\n /// let rt = Runtime::new().unwrap();\n ///\n /// let s = \"Hello World!\".to_string();\n ///\n /// // By entering the context, we tie `tokio::spawn` to this executor.\n /// let _guard = rt.enter();\n /// let handle = function_that_spawns(s);\n ///\n /// // Wait for the task before we end the test.\n /// rt.block_on(handle).unwrap();\n /// }\n /// # }\n /// ```\n pub fn enter(&self) -> EnterGuard<'_> {\n self.handle.enter()\n }\n\n /// Shuts down the runtime, waiting for at most `duration` for all spawned\n /// work to stop.\n ///\n /// See the [struct level documentation](Runtime#shutdown) for more details.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n /// use tokio::task;\n ///\n /// use std::thread;\n /// use std::time::Duration;\n ///\n /// fn main() {\n /// let runtime = Runtime::new().unwrap();\n ///\n /// runtime.block_on(async move {\n /// task::spawn_blocking(move || {\n /// thread::sleep(Duration::from_secs(10_000));\n /// });\n /// });\n ///\n /// runtime.shutdown_timeout(Duration::from_millis(100));\n /// }\n /// # }\n /// ```\n pub fn shutdown_timeout(mut self, duration: Duration) {\n // Wakeup and shutdown all the worker threads\n self.handle.inner.shutdown();\n self.blocking_pool.shutdown(Some(duration));\n }\n\n /// Shuts down the runtime, without waiting for any spawned work to stop.\n ///\n /// This can be useful if you want to drop a runtime from within another runtime.\n /// Normally, dropping a runtime will block indefinitely for spawned blocking tasks\n /// to complete, which would normally not be permitted within an asynchronous context.\n /// By calling `shutdown_background()`, you can drop the runtime from such a context.\n ///\n /// Note however, that because we do not wait for any blocking tasks to complete, this\n /// may result in a resource leak (in that any blocking tasks are still running until they\n /// return.\n ///\n /// See the [struct level documentation](Runtime#shutdown) for more details.\n ///\n /// This function is equivalent to calling `shutdown_timeout(Duration::from_nanos(0))`.\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n ///\n /// fn main() {\n /// let runtime = Runtime::new().unwrap();\n ///\n /// runtime.block_on(async move {\n /// let inner_runtime = Runtime::new().unwrap();\n /// // ...\n /// inner_runtime.shutdown_background();\n /// });\n /// }\n /// # }\n /// ```\n pub fn shutdown_background(self) {\n self.shutdown_timeout(Duration::from_nanos(0));\n }\n\n /// Returns a view that lets you get information about how the runtime\n /// is performing.\n pub fn metrics(&self) -> crate::runtime::RuntimeMetrics {\n self.handle.metrics()\n }\n}\n\nimpl Drop for Runtime {\n fn drop(&mut self) {\n match &mut self.scheduler {\n Scheduler::CurrentThread(current_thread) => {\n // This ensures that tasks spawned on the current-thread\n // runtime are dropped inside the runtime's context.\n let _guard = context::try_set_current(&self.handle.inner);\n current_thread.shutdown(&self.handle.inner);\n }\n #[cfg(feature = \"rt-multi-thread\")]\n Scheduler::MultiThread(multi_thread) => {\n // The threaded scheduler drops its tasks on its worker threads, which is\n // already in the runtime's context.\n multi_thread.shutdown(&self.handle.inner);\n }\n }\n }\n}\n\nimpl std::panic::UnwindSafe for Runtime {}\n\nimpl std::panic::RefUnwindSafe for Runtime {}\n\nfn display_eq(d: impl std::fmt::Display, s: &str) -> bool {\n use std::fmt::Write;\n\n struct FormatEq<'r> {\n remainder: &'r str,\n unequal: bool,\n }\n\n impl<'r> Write for FormatEq<'r> {\n fn write_str(&mut self, s: &str) -> std::fmt::Result {\n if !self.unequal {\n if let Some(new_remainder) = self.remainder.strip_prefix(s) {\n self.remainder = new_remainder;\n } else {\n self.unequal = true;\n }\n }\n Ok(())\n }\n }\n\n let mut fmt_eq = FormatEq {\n remainder: s,\n unequal: false,\n };\n let _ = write!(fmt_eq, \"{d}\");\n fmt_eq.remainder.is_empty() && !fmt_eq.unequal\n}\n\n/// Checks whether the given error was emitted by Tokio when shutting down its runtime.\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use tokio::runtime::Runtime;\n/// use tokio::net::TcpListener;\n///\n/// fn main() {\n/// let rt1 = Runtime::new().unwrap();\n/// let rt2 = Runtime::new().unwrap();\n///\n/// let listener = rt1.block_on(async {\n/// TcpListener::bind(\"127.0.0.1:0\").await.unwrap()\n/// });\n///\n/// drop(rt1);\n///\n/// rt2.block_on(async {\n/// let res = listener.accept().await;\n/// assert!(res.is_err());\n/// assert!(tokio::runtime::is_rt_shutdown_err(res.as_ref().unwrap_err()));\n/// });\n/// }\n/// # }\n/// ```\npub fn is_rt_shutdown_err(err: &io::Error) -> bool {\n if let Some(inner) = err.get_ref() {\n err.kind() == io::ErrorKind::Other\n && inner.source().is_none()\n && display_eq(inner, RUNTIME_SHUTTING_DOWN_ERROR)\n } else {\n false\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/block_in_place.rs", "content": "use crate::runtime::scheduler;\n\n#[track_caller]\npub(crate) fn block_in_place(f: F) -> R\nwhere\n F: FnOnce() -> R,\n{\n scheduler::multi_thread::block_in_place(f)\n}\n" }, { "path": "tokio/src/runtime/scheduler/current_thread/mod.rs", "content": "use crate::loom::sync::atomic::AtomicBool;\nuse crate::loom::sync::Arc;\nuse crate::runtime::driver::{self, Driver};\nuse crate::runtime::scheduler::{self, Defer, Inject};\nuse crate::runtime::task::{\n self, JoinHandle, OwnedTasks, Schedule, SpawnLocation, Task, TaskHarnessScheduleHooks,\n};\nuse crate::runtime::{\n blocking, context, Config, MetricsBatch, SchedulerMetrics, TaskHooks, TaskMeta, WorkerMetrics,\n};\nuse crate::sync::notify::Notify;\nuse crate::util::atomic_cell::AtomicCell;\nuse crate::util::{waker_ref, RngSeedGenerator, Wake, WakerRef};\n\nuse std::cell::RefCell;\nuse std::collections::VecDeque;\nuse std::future::{poll_fn, Future};\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Release};\nuse std::task::Poll::{Pending, Ready};\nuse std::task::Waker;\nuse std::thread::ThreadId;\nuse std::time::Duration;\nuse std::{fmt, thread};\n\n/// Executes tasks on the current thread\npub(crate) struct CurrentThread {\n /// Core scheduler data is acquired by a thread entering `block_on`.\n core: AtomicCell,\n\n /// Notifier for waking up other threads to steal the\n /// driver.\n notify: Notify,\n}\n\n/// Handle to the current thread scheduler\npub(crate) struct Handle {\n /// Scheduler state shared across threads\n shared: Shared,\n\n /// Resource driver handles\n pub(crate) driver: driver::Handle,\n\n /// Blocking pool spawner\n pub(crate) blocking_spawner: blocking::Spawner,\n\n /// Current random number generator seed\n pub(crate) seed_generator: RngSeedGenerator,\n\n /// User-supplied hooks to invoke for things\n pub(crate) task_hooks: TaskHooks,\n\n /// If this is a `LocalRuntime`, flags the owning thread ID.\n pub(crate) local_tid: Option,\n}\n\n/// Data required for executing the scheduler. The struct is passed around to\n/// a function that will perform the scheduling work and acts as a capability token.\nstruct Core {\n /// Scheduler run queue\n tasks: VecDeque,\n\n /// Current tick\n tick: u32,\n\n /// Runtime driver\n ///\n /// The driver is removed before starting to park the thread\n driver: Option,\n\n /// Metrics batch\n metrics: MetricsBatch,\n\n /// How often to check the global queue\n global_queue_interval: u32,\n\n /// True if a task panicked without being handled and the runtime is\n /// configured to shutdown on unhandled panic.\n unhandled_panic: bool,\n}\n\n/// Scheduler state shared between threads.\nstruct Shared {\n /// Remote run queue\n inject: Inject>,\n\n /// Collection of all active tasks spawned onto this executor.\n owned: OwnedTasks>,\n\n /// Indicates whether the blocked on thread was woken.\n woken: AtomicBool,\n\n /// Scheduler configuration options\n config: Config,\n\n /// Keeps track of various runtime metrics.\n scheduler_metrics: SchedulerMetrics,\n\n /// This scheduler only has one worker.\n worker_metrics: WorkerMetrics,\n}\n\n/// Thread-local context.\n///\n/// pub(crate) to store in `runtime::context`.\npub(crate) struct Context {\n /// Scheduler handle\n handle: Arc,\n\n /// Scheduler core, enabling the holder of `Context` to execute the\n /// scheduler.\n core: RefCell>>,\n\n /// Deferred tasks, usually ones that called `task::yield_now()`.\n pub(crate) defer: Defer,\n}\n\ntype Notified = task::Notified>;\n\n/// Initial queue capacity.\nconst INITIAL_CAPACITY: usize = 64;\n\n/// Used if none is specified. This is a temporary constant and will be removed\n/// as we unify tuning logic between the multi-thread and current-thread\n/// schedulers.\nconst DEFAULT_GLOBAL_QUEUE_INTERVAL: u32 = 31;\n\nimpl CurrentThread {\n pub(crate) fn new(\n driver: Driver,\n driver_handle: driver::Handle,\n blocking_spawner: blocking::Spawner,\n seed_generator: RngSeedGenerator,\n config: Config,\n local_tid: Option,\n ) -> (CurrentThread, Arc) {\n let worker_metrics = WorkerMetrics::from_config(&config);\n worker_metrics.set_thread_id(thread::current().id());\n\n // Get the configured global queue interval, or use the default.\n let global_queue_interval = config\n .global_queue_interval\n .unwrap_or(DEFAULT_GLOBAL_QUEUE_INTERVAL);\n\n let handle = Arc::new(Handle {\n task_hooks: TaskHooks {\n task_spawn_callback: config.before_spawn.clone(),\n task_terminate_callback: config.after_termination.clone(),\n #[cfg(tokio_unstable)]\n before_poll_callback: config.before_poll.clone(),\n #[cfg(tokio_unstable)]\n after_poll_callback: config.after_poll.clone(),\n },\n shared: Shared {\n inject: Inject::new(),\n owned: OwnedTasks::new(1),\n woken: AtomicBool::new(false),\n config,\n scheduler_metrics: SchedulerMetrics::new(),\n worker_metrics,\n },\n driver: driver_handle,\n blocking_spawner,\n seed_generator,\n local_tid,\n });\n\n let core = AtomicCell::new(Some(Box::new(Core {\n tasks: VecDeque::with_capacity(INITIAL_CAPACITY),\n tick: 0,\n driver: Some(driver),\n metrics: MetricsBatch::new(&handle.shared.worker_metrics),\n global_queue_interval,\n unhandled_panic: false,\n })));\n\n let scheduler = CurrentThread {\n core,\n notify: Notify::new(),\n };\n\n (scheduler, handle)\n }\n\n #[track_caller]\n pub(crate) fn block_on(&self, handle: &scheduler::Handle, future: F) -> F::Output {\n pin!(future);\n\n crate::runtime::context::enter_runtime(handle, false, |blocking| {\n let handle = handle.as_current_thread();\n\n // Attempt to steal the scheduler core and block_on the future if we can\n // there, otherwise, lets select on a notification that the core is\n // available or the future is complete.\n loop {\n if let Some(core) = self.take_core(handle) {\n handle\n .shared\n .worker_metrics\n .set_thread_id(thread::current().id());\n return core.block_on(future);\n } else {\n let notified = self.notify.notified();\n pin!(notified);\n\n if let Some(out) = blocking\n .block_on(poll_fn(|cx| {\n if notified.as_mut().poll(cx).is_ready() {\n return Ready(None);\n }\n\n if let Ready(out) = future.as_mut().poll(cx) {\n return Ready(Some(out));\n }\n\n Pending\n }))\n .expect(\"Failed to `Enter::block_on`\")\n {\n return out;\n }\n }\n }\n })\n }\n\n fn take_core(&self, handle: &Arc) -> Option> {\n let core = self.core.take()?;\n\n Some(CoreGuard {\n context: scheduler::Context::CurrentThread(Context {\n handle: handle.clone(),\n core: RefCell::new(Some(core)),\n defer: Defer::new(),\n }),\n scheduler: self,\n })\n }\n\n pub(crate) fn shutdown(&mut self, handle: &scheduler::Handle) {\n let handle = handle.as_current_thread();\n\n // Avoid a double panic if we are currently panicking and\n // the lock may be poisoned.\n\n let core = match self.take_core(handle) {\n Some(core) => core,\n None if std::thread::panicking() => return,\n None => panic!(\"Oh no! We never placed the Core back, this is a bug!\"),\n };\n\n // Check that the thread-local is not being destroyed\n let tls_available = context::with_current(|_| ()).is_ok();\n\n if tls_available {\n core.enter(|core, _context| {\n let core = shutdown2(core, handle);\n (core, ())\n });\n } else {\n // Shutdown without setting the context. `tokio::spawn` calls will\n // fail, but those will fail either way because the thread-local is\n // not available anymore.\n let context = core.context.expect_current_thread();\n let core = context.core.borrow_mut().take().unwrap();\n\n let core = shutdown2(core, handle);\n *context.core.borrow_mut() = Some(core);\n }\n }\n}\n\nfn shutdown2(mut core: Box, handle: &Handle) -> Box {\n // Drain the OwnedTasks collection. This call also closes the\n // collection, ensuring that no tasks are ever pushed after this\n // call returns.\n handle.shared.owned.close_and_shutdown_all(0);\n\n // Drain local queue\n // We already shut down every task, so we just need to drop the task.\n while let Some(task) = core.next_local_task(handle) {\n drop(task);\n }\n\n // Close the injection queue\n handle.shared.inject.close();\n\n // Drain remote queue\n while let Some(task) = handle.shared.inject.pop() {\n drop(task);\n }\n\n assert!(handle.shared.owned.is_empty());\n\n // Submit metrics\n core.submit_metrics(handle);\n\n // Shutdown the resource drivers\n if let Some(driver) = core.driver.as_mut() {\n driver.shutdown(&handle.driver);\n }\n\n core\n}\n\nimpl fmt::Debug for CurrentThread {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"CurrentThread\").finish()\n }\n}\n\n// ===== impl Core =====\n\nimpl Core {\n /// Get and increment the current tick\n fn tick(&mut self) {\n self.tick = self.tick.wrapping_add(1);\n }\n\n fn next_task(&mut self, handle: &Handle) -> Option {\n if self.tick % self.global_queue_interval == 0 {\n handle\n .next_remote_task()\n .or_else(|| self.next_local_task(handle))\n } else {\n self.next_local_task(handle)\n .or_else(|| handle.next_remote_task())\n }\n }\n\n fn next_local_task(&mut self, handle: &Handle) -> Option {\n let ret = self.tasks.pop_front();\n handle\n .shared\n .worker_metrics\n .set_queue_depth(self.tasks.len());\n ret\n }\n\n fn push_task(&mut self, handle: &Handle, task: Notified) {\n self.tasks.push_back(task);\n self.metrics.inc_local_schedule_count();\n handle\n .shared\n .worker_metrics\n .set_queue_depth(self.tasks.len());\n }\n\n fn submit_metrics(&mut self, handle: &Handle) {\n self.metrics.submit(&handle.shared.worker_metrics, 0);\n }\n}\n\n#[cfg(feature = \"taskdump\")]\nfn wake_deferred_tasks_and_free(context: &Context) {\n let wakers = context.defer.take_deferred();\n for waker in wakers {\n waker.wake();\n }\n}\n\n// ===== impl Context =====\n\nimpl Context {\n /// Execute the closure with the given scheduler core stored in the\n /// thread-local context.\n fn run_task(&self, mut core: Box, f: impl FnOnce() -> R) -> (Box, R) {\n core.metrics.start_poll();\n let mut ret = self.enter(core, || crate::task::coop::budget(f));\n ret.0.metrics.end_poll();\n ret\n }\n\n /// Blocks the current thread until an event is received by the driver,\n /// including I/O events, timer events, ...\n fn park(&self, mut core: Box, handle: &Handle) -> Box {\n let mut driver = core.driver.take().expect(\"driver missing\");\n\n if let Some(f) = &handle.shared.config.before_park {\n let (c, ()) = self.enter(core, || f());\n core = c;\n }\n\n // If `before_park` spawns a task (or otherwise schedules work for us), then we should not\n // park the thread.\n if !self.has_pending_work(&core) {\n // Park until the thread is signaled\n core.metrics.about_to_park();\n core.submit_metrics(handle);\n\n core = self.park_internal(core, handle, &mut driver, None);\n\n core.metrics.unparked();\n core.submit_metrics(handle);\n }\n\n if let Some(f) = &handle.shared.config.after_unpark {\n let (c, ()) = self.enter(core, || f());\n core = c;\n }\n\n core.driver = Some(driver);\n core\n }\n\n /// Checks the driver for new events without blocking the thread.\n fn park_yield(&self, mut core: Box, handle: &Handle) -> Box {\n let mut driver = core.driver.take().expect(\"driver missing\");\n\n core.submit_metrics(handle);\n\n core = self.park_internal(core, handle, &mut driver, Some(Duration::from_millis(0)));\n\n core.driver = Some(driver);\n core\n }\n\n fn has_pending_work(&self, core: &Core) -> bool {\n !core.tasks.is_empty() || !self.defer.is_empty() || self.handle.shared.woken.load(Acquire)\n }\n\n fn park_internal(\n &self,\n core: Box,\n handle: &Handle,\n driver: &mut Driver,\n duration: Option,\n ) -> Box {\n let (core, ()) = self.enter(core, || {\n match duration {\n Some(dur) => driver.park_timeout(&handle.driver, dur),\n None => driver.park(&handle.driver),\n }\n self.defer.wake();\n });\n\n core\n }\n\n fn enter(&self, core: Box, f: impl FnOnce() -> R) -> (Box, R) {\n // Store the scheduler core in the thread-local context\n //\n // A drop-guard is employed at a higher level.\n *self.core.borrow_mut() = Some(core);\n\n // Execute the closure while tracking the execution budget\n let ret = f();\n\n // Take the scheduler core back\n let core = self.core.borrow_mut().take().expect(\"core missing\");\n (core, ret)\n }\n\n pub(crate) fn defer(&self, waker: &Waker) {\n self.defer.defer(waker);\n }\n}\n\n// ===== impl Handle =====\n\nimpl Handle {\n /// Spawns a future onto the `CurrentThread` scheduler\n #[track_caller]\n pub(crate) fn spawn(\n me: &Arc,\n future: F,\n id: crate::runtime::task::Id,\n spawned_at: SpawnLocation,\n ) -> JoinHandle\n where\n F: crate::future::Future + Send + 'static,\n F::Output: Send + 'static,\n {\n let (handle, notified) = me.shared.owned.bind(future, me.clone(), id, spawned_at);\n\n me.task_hooks.spawn(&TaskMeta {\n id,\n spawned_at,\n _phantom: Default::default(),\n });\n\n if let Some(notified) = notified {\n me.schedule(notified);\n }\n\n handle\n }\n\n /// Spawn a task which isn't safe to send across thread boundaries onto the runtime.\n ///\n /// # Safety\n ///\n /// This should only be used when this is a `LocalRuntime` or in another case where the runtime\n /// provably cannot be driven from or moved to different threads from the one on which the task\n /// is spawned.\n #[track_caller]\n pub(crate) unsafe fn spawn_local(\n me: &Arc,\n future: F,\n id: crate::runtime::task::Id,\n spawned_at: SpawnLocation,\n ) -> JoinHandle\n where\n F: crate::future::Future + 'static,\n F::Output: 'static,\n {\n // Safety: the caller guarantees that this is only called on a `LocalRuntime`.\n let (handle, notified) = unsafe {\n me.shared\n .owned\n .bind_local(future, me.clone(), id, spawned_at)\n };\n\n me.task_hooks.spawn(&TaskMeta {\n id,\n spawned_at,\n _phantom: Default::default(),\n });\n\n if let Some(notified) = notified {\n me.schedule(notified);\n }\n\n handle\n }\n\n /// Capture a snapshot of this runtime's state.\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n pub(crate) fn dump(&self) -> crate::runtime::Dump {\n use crate::runtime::dump;\n use task::trace::trace_current_thread;\n\n let mut traces = vec![];\n\n // todo: how to make this work outside of a runtime context?\n context::with_scheduler(|maybe_context| {\n // drain the local queue\n let context = if let Some(context) = maybe_context {\n context.expect_current_thread()\n } else {\n return;\n };\n let mut maybe_core = context.core.borrow_mut();\n let core = if let Some(core) = maybe_core.as_mut() {\n core\n } else {\n return;\n };\n let local = &mut core.tasks;\n\n if self.shared.inject.is_closed() {\n return;\n }\n\n traces = trace_current_thread(&self.shared.owned, local, &self.shared.inject)\n .into_iter()\n .map(|(id, trace)| dump::Task::new(id, trace))\n .collect();\n\n // Avoid double borrow panic\n drop(maybe_core);\n\n // Taking a taskdump could wakes every task, but we probably don't want\n // the `yield_now` vector to be that large under normal circumstances.\n // Therefore, we free its allocation.\n wake_deferred_tasks_and_free(context);\n });\n\n dump::Dump::new(traces)\n }\n\n fn next_remote_task(&self) -> Option {\n self.shared.inject.pop()\n }\n\n fn waker_ref(me: &Arc) -> WakerRef<'_> {\n // Set woken to true when enter block_on, ensure outer future\n // be polled for the first time when enter loop\n me.shared.woken.store(true, Release);\n waker_ref(me)\n }\n\n // reset woken to false and return original value\n pub(crate) fn reset_woken(&self) -> bool {\n self.shared.woken.swap(false, AcqRel)\n }\n\n pub(crate) fn num_alive_tasks(&self) -> usize {\n self.shared.owned.num_alive_tasks()\n }\n\n pub(crate) fn injection_queue_depth(&self) -> usize {\n self.shared.inject.len()\n }\n\n pub(crate) fn worker_metrics(&self, worker: usize) -> &WorkerMetrics {\n assert_eq!(0, worker);\n &self.shared.worker_metrics\n }\n}\n\ncfg_unstable_metrics! {\n impl Handle {\n pub(crate) fn scheduler_metrics(&self) -> &SchedulerMetrics {\n &self.shared.scheduler_metrics\n }\n\n pub(crate) fn worker_local_queue_depth(&self, worker: usize) -> usize {\n self.worker_metrics(worker).queue_depth()\n }\n\n pub(crate) fn num_blocking_threads(&self) -> usize {\n self.blocking_spawner.num_threads()\n }\n\n pub(crate) fn num_idle_blocking_threads(&self) -> usize {\n self.blocking_spawner.num_idle_threads()\n }\n\n pub(crate) fn blocking_queue_depth(&self) -> usize {\n self.blocking_spawner.queue_depth()\n }\n\n cfg_64bit_metrics! {\n pub(crate) fn spawned_tasks_count(&self) -> u64 {\n self.shared.owned.spawned_tasks_count()\n }\n }\n }\n}\n\nuse std::num::NonZeroU64;\n\nimpl Handle {\n pub(crate) fn owned_id(&self) -> NonZeroU64 {\n self.shared.owned.id\n }\n}\n\nimpl fmt::Debug for Handle {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"current_thread::Handle { ... }\").finish()\n }\n}\n\n// ===== impl Shared =====\n\nimpl Schedule for Arc {\n fn release(&self, task: &Task) -> Option> {\n self.shared.owned.remove(task)\n }\n\n fn schedule(&self, task: task::Notified) {\n use scheduler::Context::CurrentThread;\n\n context::with_scheduler(|maybe_cx| match maybe_cx {\n Some(CurrentThread(cx)) if Arc::ptr_eq(self, &cx.handle) => {\n let mut core = cx.core.borrow_mut();\n\n // If `None`, the runtime is shutting down, so there is no need\n // to schedule the task.\n if let Some(core) = core.as_mut() {\n core.push_task(self, task);\n }\n }\n _ => {\n // Track that a task was scheduled from **outside** of the runtime.\n self.shared.scheduler_metrics.inc_remote_schedule_count();\n\n // Schedule the task\n self.shared.inject.push(task);\n self.driver.unpark();\n }\n });\n }\n\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: self.task_hooks.task_terminate_callback.clone(),\n }\n }\n\n cfg_unstable! {\n fn unhandled_panic(&self) {\n use crate::runtime::UnhandledPanic;\n\n match self.shared.config.unhandled_panic {\n UnhandledPanic::Ignore => {\n // Do nothing\n }\n UnhandledPanic::ShutdownRuntime => {\n use scheduler::Context::CurrentThread;\n\n // This hook is only called from within the runtime, so\n // `context::with_scheduler` should match with `&self`, i.e.\n // there is no opportunity for a nested scheduler to be\n // called.\n context::with_scheduler(|maybe_cx| match maybe_cx {\n Some(CurrentThread(cx)) if Arc::ptr_eq(self, &cx.handle) => {\n let mut core = cx.core.borrow_mut();\n\n // If `None`, the runtime is shutting down, so there is no need to signal shutdown\n if let Some(core) = core.as_mut() {\n core.unhandled_panic = true;\n self.shared.owned.close_and_shutdown_all(0);\n }\n }\n _ => unreachable!(\"runtime core not set in CURRENT thread-local\"),\n })\n }\n }\n }\n }\n}\n\nimpl Wake for Handle {\n fn wake(arc_self: Arc) {\n Wake::wake_by_ref(&arc_self);\n }\n\n /// Wake by reference\n fn wake_by_ref(arc_self: &Arc) {\n let already_woken = arc_self.shared.woken.swap(true, Release);\n\n if !already_woken {\n use scheduler::Context::CurrentThread;\n\n // If we are already running on the runtime, then it's not required to wake up the\n // runtime.\n context::with_scheduler(|maybe_cx| match maybe_cx {\n Some(CurrentThread(cx)) if Arc::ptr_eq(arc_self, &cx.handle) => {}\n _ => {\n arc_self.driver.unpark();\n }\n });\n }\n }\n}\n\n// ===== CoreGuard =====\n\n/// Used to ensure we always place the `Core` value back into its slot in\n/// `CurrentThread`, even if the future panics.\nstruct CoreGuard<'a> {\n context: scheduler::Context,\n scheduler: &'a CurrentThread,\n}\n\nimpl CoreGuard<'_> {\n #[track_caller]\n fn block_on(self, future: F) -> F::Output {\n let ret = self.enter(|mut core, context| {\n let waker = Handle::waker_ref(&context.handle);\n let mut cx = std::task::Context::from_waker(&waker);\n\n pin!(future);\n\n core.metrics.start_processing_scheduled_tasks();\n\n 'outer: loop {\n let handle = &context.handle;\n\n if handle.reset_woken() {\n let (c, res) = context.enter(core, || {\n crate::task::coop::budget(|| future.as_mut().poll(&mut cx))\n });\n\n core = c;\n\n if let Ready(v) = res {\n return (core, Some(v));\n }\n }\n\n for _ in 0..handle.shared.config.event_interval {\n // Make sure we didn't hit an unhandled_panic\n if core.unhandled_panic {\n return (core, None);\n }\n\n core.tick();\n\n let entry = core.next_task(handle);\n\n let task = match entry {\n Some(entry) => entry,\n None => {\n core.metrics.end_processing_scheduled_tasks();\n\n core = if context.has_pending_work(&core) {\n context.park_yield(core, handle)\n } else {\n context.park(core, handle)\n };\n\n core.metrics.start_processing_scheduled_tasks();\n\n // Try polling the `block_on` future next\n continue 'outer;\n }\n };\n\n let task = context.handle.shared.owned.assert_owner(task);\n\n #[cfg(tokio_unstable)]\n let task_meta = task.task_meta();\n\n let (c, ()) = context.run_task(core, || {\n #[cfg(tokio_unstable)]\n context.handle.task_hooks.poll_start_callback(&task_meta);\n\n task.run();\n\n #[cfg(tokio_unstable)]\n context.handle.task_hooks.poll_stop_callback(&task_meta);\n });\n\n core = c;\n }\n\n core.metrics.end_processing_scheduled_tasks();\n\n // Yield to the driver, this drives the timer and pulls any\n // pending I/O events.\n core = context.park_yield(core, handle);\n\n core.metrics.start_processing_scheduled_tasks();\n }\n });\n\n match ret {\n Some(ret) => ret,\n None => {\n // `block_on` panicked.\n panic!(\"a spawned task panicked and the runtime is configured to shut down on unhandled panic\");\n }\n }\n }\n\n /// Enters the scheduler context. This sets the queue and other necessary\n /// scheduler state in the thread-local.\n fn enter(self, f: F) -> R\n where\n F: FnOnce(Box, &Context) -> (Box, R),\n {\n let context = self.context.expect_current_thread();\n\n // Remove `core` from `context` to pass into the closure.\n let core = context.core.borrow_mut().take().expect(\"core missing\");\n\n // Call the closure and place `core` back\n let (core, ret) = context::set_scheduler(&self.context, || f(core, context));\n\n *context.core.borrow_mut() = Some(core);\n\n ret\n }\n}\n\nimpl Drop for CoreGuard<'_> {\n fn drop(&mut self) {\n let context = self.context.expect_current_thread();\n\n if let Some(core) = context.core.borrow_mut().take() {\n // Replace old scheduler back into the state to allow\n // other threads to pick it up and drive it.\n self.scheduler.core.set(core);\n\n // Wake up other possible threads that could steal the driver.\n self.scheduler.notify.notify_one();\n }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/defer.rs", "content": "use std::cell::RefCell;\nuse std::task::Waker;\n\npub(crate) struct Defer {\n deferred: RefCell>,\n}\n\nimpl Defer {\n pub(crate) fn new() -> Defer {\n Defer {\n deferred: RefCell::default(),\n }\n }\n\n pub(crate) fn defer(&self, waker: &Waker) {\n let mut deferred = self.deferred.borrow_mut();\n\n // If the same task adds itself a bunch of times, then only add it once.\n if let Some(last) = deferred.last() {\n if last.will_wake(waker) {\n return;\n }\n }\n\n deferred.push(waker.clone());\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.deferred.borrow().is_empty()\n }\n\n pub(crate) fn wake(&self) {\n while let Some(waker) = self.deferred.borrow_mut().pop() {\n waker.wake();\n }\n }\n\n #[cfg(feature = \"taskdump\")]\n pub(crate) fn take_deferred(&self) -> Vec {\n let mut deferred = self.deferred.borrow_mut();\n std::mem::take(&mut *deferred)\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject/metrics.rs", "content": "use super::Inject;\n\nimpl Inject {\n pub(crate) fn len(&self) -> usize {\n self.shared.len()\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject/pop.rs", "content": "use super::Synced;\n\nuse crate::runtime::task;\n\nuse std::marker::PhantomData;\n\npub(crate) struct Pop<'a, T: 'static> {\n len: usize,\n synced: &'a mut Synced,\n _p: PhantomData,\n}\n\nimpl<'a, T: 'static> Pop<'a, T> {\n pub(super) fn new(len: usize, synced: &'a mut Synced) -> Pop<'a, T> {\n Pop {\n len,\n synced,\n _p: PhantomData,\n }\n }\n}\n\nimpl<'a, T: 'static> Iterator for Pop<'a, T> {\n type Item = task::Notified;\n\n fn next(&mut self) -> Option {\n if self.len == 0 {\n return None;\n }\n\n let ret = self.synced.pop();\n\n // Should be `Some` when `len > 0`\n debug_assert!(ret.is_some());\n\n self.len -= 1;\n ret\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (self.len, Some(self.len))\n }\n}\n\nimpl<'a, T: 'static> ExactSizeIterator for Pop<'a, T> {\n fn len(&self) -> usize {\n self.len\n }\n}\n\nimpl<'a, T: 'static> Drop for Pop<'a, T> {\n fn drop(&mut self) {\n for _ in self.by_ref() {}\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject/rt_multi_thread.rs", "content": "use super::{Shared, Synced};\n\nuse crate::runtime::scheduler::Lock;\nuse crate::runtime::task;\n\nuse std::sync::atomic::Ordering::Release;\n\nimpl<'a> Lock for &'a mut Synced {\n type Handle = &'a mut Synced;\n\n fn lock(self) -> Self::Handle {\n self\n }\n}\n\nimpl AsMut for Synced {\n fn as_mut(&mut self) -> &mut Synced {\n self\n }\n}\n\nimpl Shared {\n /// Pushes several values into the queue.\n ///\n /// # Safety\n ///\n /// Must be called with the same `Synced` instance returned by `Inject::new`\n #[inline]\n pub(crate) unsafe fn push_batch(&self, shared: L, mut iter: I)\n where\n L: Lock,\n I: Iterator>,\n {\n let first = match iter.next() {\n Some(first) => first.into_raw(),\n None => return,\n };\n\n // Link up all the tasks.\n let mut prev = first;\n let mut counter = 1;\n\n // We are going to be called with an `std::iter::Chain`, and that\n // iterator overrides `for_each` to something that is easier for the\n // compiler to optimize than a loop.\n iter.for_each(|next| {\n let next = next.into_raw();\n\n // safety: Holding the Notified for a task guarantees exclusive\n // access to the `queue_next` field.\n unsafe { prev.set_queue_next(Some(next)) };\n prev = next;\n counter += 1;\n });\n\n // Now that the tasks are linked together, insert them into the\n // linked list.\n //\n // Safety: exactly the same safety requirements as `push_batch` method.\n unsafe {\n self.push_batch_inner(shared, first, prev, counter);\n }\n }\n\n /// Inserts several tasks that have been linked together into the queue.\n ///\n /// The provided head and tail may be the same task. In this case, a\n /// single task is inserted.\n ///\n /// # Safety\n ///\n /// Must be called with the same `Synced` instance returned by `Inject::new`\n #[inline]\n unsafe fn push_batch_inner(\n &self,\n shared: L,\n batch_head: task::RawTask,\n batch_tail: task::RawTask,\n num: usize,\n ) where\n L: Lock,\n {\n debug_assert!(unsafe { batch_tail.get_queue_next().is_none() });\n\n let mut synced = shared.lock();\n\n if synced.as_mut().is_closed {\n drop(synced);\n\n let mut curr = Some(batch_head);\n\n while let Some(task) = curr {\n // Safety: exactly the same safety requirements as `push_batch_inner`.\n curr = unsafe { task.get_queue_next() };\n\n let _ = unsafe { task::Notified::::from_raw(task) };\n }\n\n return;\n }\n\n let synced = synced.as_mut();\n\n if let Some(tail) = synced.tail {\n unsafe {\n tail.set_queue_next(Some(batch_head));\n }\n } else {\n synced.head = Some(batch_head);\n }\n\n synced.tail = Some(batch_tail);\n\n // Increment the count.\n //\n // safety: All updates to the len atomic are guarded by the mutex. As\n // such, a non-atomic load followed by a store is safe.\n let len = unsafe { self.len.unsync_load() };\n\n self.len.store(len + num, Release);\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject/shared.rs", "content": "use super::{Pop, Synced};\n\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::runtime::task;\n\nuse std::marker::PhantomData;\nuse std::sync::atomic::Ordering::{Acquire, Release};\n\npub(crate) struct Shared {\n /// Number of pending tasks in the queue. This helps prevent unnecessary\n /// locking in the hot path.\n pub(super) len: AtomicUsize,\n\n _p: PhantomData,\n}\n\nunsafe impl Send for Shared {}\nunsafe impl Sync for Shared {}\n\nimpl Shared {\n pub(crate) fn new() -> (Shared, Synced) {\n let inject = Shared {\n len: AtomicUsize::new(0),\n _p: PhantomData,\n };\n\n let synced = Synced {\n is_closed: false,\n head: None,\n tail: None,\n };\n\n (inject, synced)\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n // Kind of annoying to have to include the cfg here\n #[cfg(any(feature = \"taskdump\", feature = \"rt-multi-thread\"))]\n pub(crate) fn is_closed(&self, synced: &Synced) -> bool {\n synced.is_closed\n }\n\n /// Closes the injection queue, returns `true` if the queue is open when the\n /// transition is made.\n pub(crate) fn close(&self, synced: &mut Synced) -> bool {\n if synced.is_closed {\n return false;\n }\n\n synced.is_closed = true;\n true\n }\n\n pub(crate) fn len(&self) -> usize {\n self.len.load(Acquire)\n }\n\n /// Pushes a value into the queue.\n ///\n /// This does nothing if the queue is closed.\n ///\n /// # Safety\n ///\n /// Must be called with the same `Synced` instance returned by `Inject::new`\n pub(crate) unsafe fn push(&self, synced: &mut Synced, task: task::Notified) {\n if synced.is_closed {\n return;\n }\n\n // safety: only mutated with the lock held\n let len = unsafe { self.len.unsync_load() };\n let task = task.into_raw();\n\n // The next pointer should already be null\n debug_assert!(unsafe { task.get_queue_next().is_none() });\n\n if let Some(tail) = synced.tail {\n // safety: Holding the Notified for a task guarantees exclusive\n // access to the `queue_next` field.\n unsafe { tail.set_queue_next(Some(task)) };\n } else {\n synced.head = Some(task);\n }\n\n synced.tail = Some(task);\n self.len.store(len + 1, Release);\n }\n\n /// Pop a value from the queue.\n ///\n /// # Safety\n ///\n /// Must be called with the same `Synced` instance returned by `Inject::new`\n pub(crate) unsafe fn pop(&self, synced: &mut Synced) -> Option> {\n unsafe { self.pop_n(synced, 1).next() }\n }\n\n /// Pop `n` values from the queue\n ///\n /// # Safety\n ///\n /// Must be called with the same `Synced` instance returned by `Inject::new`\n pub(crate) unsafe fn pop_n<'a>(&'a self, synced: &'a mut Synced, n: usize) -> Pop<'a, T> {\n use std::cmp;\n\n debug_assert!(n > 0);\n\n // safety: All updates to the len atomic are guarded by the mutex. As\n // such, a non-atomic load followed by a store is safe.\n let len = unsafe { self.len.unsync_load() };\n let n = cmp::min(n, len);\n\n // Decrement the count.\n self.len.store(len - n, Release);\n\n Pop::new(n, synced)\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject/synced.rs", "content": "#![cfg_attr(\n any(not(all(tokio_unstable, feature = \"full\")), target_family = \"wasm\"),\n allow(dead_code)\n)]\n\nuse crate::runtime::task;\n\npub(crate) struct Synced {\n /// True if the queue is closed.\n pub(super) is_closed: bool,\n\n /// Linked-list head.\n pub(super) head: Option,\n\n /// Linked-list tail.\n pub(super) tail: Option,\n}\n\nunsafe impl Send for Synced {}\nunsafe impl Sync for Synced {}\n\nimpl Synced {\n pub(super) fn pop(&mut self) -> Option> {\n let task = self.head?;\n\n self.head = unsafe { task.get_queue_next() };\n\n if self.head.is_none() {\n self.tail = None;\n }\n\n unsafe { task.set_queue_next(None) };\n\n // safety: a `Notified` is pushed into the queue and now it is popped!\n Some(unsafe { task::Notified::from_raw(task) })\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/inject.rs", "content": "//! Inject queue used to send wakeups to a work-stealing scheduler\n\nuse crate::loom::sync::Mutex;\nuse crate::runtime::task;\n\nmod pop;\npub(crate) use pop::Pop;\n\nmod shared;\npub(crate) use shared::Shared;\n\nmod synced;\npub(crate) use synced::Synced;\n\ncfg_rt_multi_thread! {\n mod rt_multi_thread;\n}\n\nmod metrics;\n\n/// Growable, MPMC queue used to inject new tasks into the scheduler and as an\n/// overflow queue when the local, fixed-size, array queue overflows.\npub(crate) struct Inject {\n shared: Shared,\n synced: Mutex,\n}\n\nimpl Inject {\n pub(crate) fn new() -> Inject {\n let (shared, synced) = Shared::new();\n\n Inject {\n shared,\n synced: Mutex::new(synced),\n }\n }\n\n // Kind of annoying to have to include the cfg here\n #[cfg(feature = \"taskdump\")]\n pub(crate) fn is_closed(&self) -> bool {\n let synced = self.synced.lock();\n self.shared.is_closed(&synced)\n }\n\n /// Closes the injection queue, returns `true` if the queue is open when the\n /// transition is made.\n pub(crate) fn close(&self) -> bool {\n let mut synced = self.synced.lock();\n self.shared.close(&mut synced)\n }\n\n /// Pushes a value into the queue.\n ///\n /// This does nothing if the queue is closed.\n pub(crate) fn push(&self, task: task::Notified) {\n let mut synced = self.synced.lock();\n // safety: passing correct `Synced`\n unsafe { self.shared.push(&mut synced, task) }\n }\n\n pub(crate) fn pop(&self) -> Option> {\n if self.shared.is_empty() {\n return None;\n }\n\n let mut synced = self.synced.lock();\n // safety: passing correct `Synced`\n unsafe { self.shared.pop(&mut synced) }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/lock.rs", "content": "/// A lock (mutex) yielding generic data.\npub(crate) trait Lock {\n type Handle: AsMut;\n\n fn lock(self) -> Self::Handle;\n}\n" }, { "path": "tokio/src/runtime/scheduler/mod.rs", "content": "cfg_rt! {\n pub(crate) mod current_thread;\n pub(crate) use current_thread::CurrentThread;\n\n mod defer;\n use defer::Defer;\n\n pub(crate) mod inject;\n pub(crate) use inject::Inject;\n\n use crate::runtime::TaskHooks;\n\n use crate::runtime::WorkerMetrics;\n}\n\ncfg_rt_multi_thread! {\n mod block_in_place;\n pub(crate) use block_in_place::block_in_place;\n\n mod lock;\n use lock::Lock;\n\n pub(crate) mod multi_thread;\n pub(crate) use multi_thread::MultiThread;\n}\n\npub(super) mod util;\n\nuse crate::runtime::driver;\n\n#[derive(Debug, Clone)]\npub(crate) enum Handle {\n #[cfg(feature = \"rt\")]\n CurrentThread(Arc),\n\n #[cfg(feature = \"rt-multi-thread\")]\n MultiThread(Arc),\n\n // TODO: This is to avoid triggering \"dead code\" warnings many other places\n // in the codebase. Remove this during a later cleanup\n #[cfg(not(feature = \"rt\"))]\n #[allow(dead_code)]\n Disabled,\n}\n\n#[cfg(feature = \"rt\")]\npub(super) enum Context {\n CurrentThread(current_thread::Context),\n\n #[cfg(feature = \"rt-multi-thread\")]\n MultiThread(multi_thread::Context),\n}\n\nimpl Handle {\n #[cfg_attr(not(feature = \"full\"), allow(dead_code))]\n pub(crate) fn driver(&self) -> &driver::Handle {\n match *self {\n #[cfg(feature = \"rt\")]\n Handle::CurrentThread(ref h) => &h.driver,\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(ref h) => &h.driver,\n\n #[cfg(not(feature = \"rt\"))]\n Handle::Disabled => unreachable!(),\n }\n }\n}\n\ncfg_rt! {\n use crate::future::Future;\n use crate::loom::sync::Arc;\n use crate::runtime::{blocking, task::{Id, SpawnLocation}};\n use crate::runtime::context;\n use crate::task::JoinHandle;\n use crate::util::RngSeedGenerator;\n use std::task::Waker;\n\n macro_rules! match_flavor {\n ($self:expr, $ty:ident($h:ident) => $e:expr) => {\n match $self {\n $ty::CurrentThread($h) => $e,\n\n #[cfg(feature = \"rt-multi-thread\")]\n $ty::MultiThread($h) => $e,\n }\n }\n }\n\n impl Handle {\n #[track_caller]\n pub(crate) fn current() -> Handle {\n match context::with_current(Clone::clone) {\n Ok(handle) => handle,\n Err(e) => panic!(\"{}\", e),\n }\n }\n\n pub(crate) fn blocking_spawner(&self) -> &blocking::Spawner {\n match_flavor!(self, Handle(h) => &h.blocking_spawner)\n }\n\n pub(crate) fn is_local(&self) -> bool {\n match self {\n Handle::CurrentThread(h) => h.local_tid.is_some(),\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(_) => false,\n }\n }\n\n #[cfg(feature = \"time\")]\n pub(crate) fn timer_flavor(&self) -> crate::runtime::TimerFlavor {\n match self {\n Handle::CurrentThread(_) => crate::runtime::TimerFlavor::Traditional,\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(h) => h.timer_flavor,\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\", feature = \"time\"))]\n /// Returns true if both handles belong to the same runtime instance.\n pub(crate) fn is_same_runtime(&self, other: &Handle) -> bool {\n match (self, other) {\n (Handle::CurrentThread(a), Handle::CurrentThread(b)) => Arc::ptr_eq(a, b),\n #[cfg(feature = \"rt-multi-thread\")]\n (Handle::MultiThread(a), Handle::MultiThread(b)) => Arc::ptr_eq(a, b),\n #[cfg(feature = \"rt-multi-thread\")]\n _ => false, // different runtime types\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\", feature = \"time\"))]\n /// Returns true if the runtime is shutting down.\n pub(crate) fn is_shutdown(&self) -> bool {\n match self {\n Handle::CurrentThread(_) => panic!(\"the alternative timer implementation is not supported on CurrentThread runtime\"),\n Handle::MultiThread(h) => h.is_shutdown(),\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\", feature = \"time\"))]\n /// Push a timer entry that was created outside of this runtime\n /// into the runtime-global queue. The pushed timer will be\n /// processed by a random worker thread.\n pub(crate) fn push_remote_timer(&self, entry_hdl: crate::runtime::time_alt::EntryHandle) {\n match self {\n Handle::CurrentThread(_) => panic!(\"the alternative timer implementation is not supported on CurrentThread runtime\"),\n Handle::MultiThread(h) => h.push_remote_timer(entry_hdl),\n }\n }\n\n /// Returns true if this is a local runtime and the runtime is owned by the current thread.\n pub(crate) fn can_spawn_local_on_local_runtime(&self) -> bool {\n match self {\n Handle::CurrentThread(h) => h.local_tid.map(|x| std::thread::current().id() == x).unwrap_or(false),\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(_) => false,\n }\n }\n\n pub(crate) fn spawn(&self, future: F, id: Id, spawned_at: SpawnLocation) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n match self {\n Handle::CurrentThread(h) => current_thread::Handle::spawn(h, future, id, spawned_at),\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(h) => multi_thread::Handle::spawn(h, future, id, spawned_at),\n }\n }\n\n /// Spawn a local task\n ///\n /// # Safety\n ///\n /// This should only be called in `LocalRuntime` if the runtime has been verified to be owned\n /// by the current thread.\n #[allow(irrefutable_let_patterns)]\n #[track_caller]\n pub(crate) unsafe fn spawn_local(&self, future: F, id: Id, spawned_at: SpawnLocation) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n if let Handle::CurrentThread(h) = self {\n // Safety: caller guarantees that this is a `LocalRuntime`.\n unsafe { current_thread::Handle::spawn_local(h, future, id, spawned_at) }\n } else {\n panic!(\"Only current_thread and LocalSet have spawn_local internals implemented\")\n }\n }\n\n pub(crate) fn shutdown(&self) {\n match *self {\n Handle::CurrentThread(_) => {},\n\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(ref h) => h.shutdown(),\n }\n }\n\n pub(crate) fn seed_generator(&self) -> &RngSeedGenerator {\n match_flavor!(self, Handle(h) => &h.seed_generator)\n }\n\n pub(crate) fn as_current_thread(&self) -> &Arc {\n match self {\n Handle::CurrentThread(handle) => handle,\n #[cfg(feature = \"rt-multi-thread\")]\n _ => panic!(\"not a CurrentThread handle\"),\n }\n }\n\n pub(crate) fn hooks(&self) -> &TaskHooks {\n match self {\n Handle::CurrentThread(h) => &h.task_hooks,\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(h) => &h.task_hooks,\n }\n }\n }\n\n impl Handle {\n pub(crate) fn num_workers(&self) -> usize {\n match self {\n Handle::CurrentThread(_) => 1,\n #[cfg(feature = \"rt-multi-thread\")]\n Handle::MultiThread(handle) => handle.num_workers(),\n }\n }\n\n pub(crate) fn num_alive_tasks(&self) -> usize {\n match_flavor!(self, Handle(handle) => handle.num_alive_tasks())\n }\n\n pub(crate) fn injection_queue_depth(&self) -> usize {\n match_flavor!(self, Handle(handle) => handle.injection_queue_depth())\n }\n\n pub(crate) fn worker_metrics(&self, worker: usize) -> &WorkerMetrics {\n match_flavor!(self, Handle(handle) => handle.worker_metrics(worker))\n }\n }\n\n cfg_unstable_metrics! {\n use crate::runtime::SchedulerMetrics;\n\n impl Handle {\n cfg_64bit_metrics! {\n pub(crate) fn spawned_tasks_count(&self) -> u64 {\n match_flavor!(self, Handle(handle) => handle.spawned_tasks_count())\n }\n }\n\n pub(crate) fn num_blocking_threads(&self) -> usize {\n match_flavor!(self, Handle(handle) => handle.num_blocking_threads())\n }\n\n pub(crate) fn num_idle_blocking_threads(&self) -> usize {\n match_flavor!(self, Handle(handle) => handle.num_idle_blocking_threads())\n }\n\n pub(crate) fn scheduler_metrics(&self) -> &SchedulerMetrics {\n match_flavor!(self, Handle(handle) => handle.scheduler_metrics())\n }\n\n pub(crate) fn worker_local_queue_depth(&self, worker: usize) -> usize {\n match_flavor!(self, Handle(handle) => handle.worker_local_queue_depth(worker))\n }\n\n pub(crate) fn blocking_queue_depth(&self) -> usize {\n match_flavor!(self, Handle(handle) => handle.blocking_queue_depth())\n }\n }\n }\n\n impl Context {\n #[track_caller]\n pub(crate) fn expect_current_thread(&self) -> ¤t_thread::Context {\n match self {\n Context::CurrentThread(context) => context,\n #[cfg(feature = \"rt-multi-thread\")]\n _ => panic!(\"expected `CurrentThread::Context`\")\n }\n }\n\n pub(crate) fn defer(&self, waker: &Waker) {\n match_flavor!(self, Context(context) => context.defer(waker));\n }\n\n #[cfg(tokio_unstable)]\n pub(crate) fn worker_index(&self) -> Option {\n match self {\n Context::CurrentThread(_) => Some(0),\n #[cfg(feature = \"rt-multi-thread\")]\n Context::MultiThread(context) => Some(context.worker_index()),\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\", feature = \"rt-multi-thread\"))]\n pub(crate) fn with_time_temp_local_context(&self, f: F) -> R\n where\n F: FnOnce(Option>) -> R,\n {\n match self {\n Context::CurrentThread(_) => panic!(\"the alternative timer implementation is not supported on CurrentThread runtime\"),\n Context::MultiThread(context) => context.with_time_temp_local_context(f),\n }\n }\n\n cfg_rt_multi_thread! {\n #[track_caller]\n pub(crate) fn expect_multi_thread(&self) -> &multi_thread::Context {\n match self {\n Context::MultiThread(context) => context,\n _ => panic!(\"expected `MultiThread::Context`\")\n }\n }\n }\n }\n}\n\ncfg_not_rt! {\n #[cfg(any(\n feature = \"net\",\n all(unix, feature = \"process\"),\n all(unix, feature = \"signal\"),\n feature = \"time\",\n ))]\n impl Handle {\n #[track_caller]\n pub(crate) fn current() -> Handle {\n panic!(\"{}\", crate::util::error::CONTEXT_MISSING_ERROR)\n }\n\n #[cfg_attr(not(feature = \"time\"), allow(dead_code))]\n #[track_caller]\n pub(crate) fn timer_flavor(&self) -> crate::runtime::TimerFlavor {\n panic!(\"{}\", crate::util::error::CONTEXT_MISSING_ERROR)\n }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/counters.rs", "content": "#[cfg(tokio_internal_mt_counters)]\nmod imp {\n use std::sync::atomic::AtomicUsize;\n use std::sync::atomic::Ordering::Relaxed;\n\n static NUM_MAINTENANCE: AtomicUsize = AtomicUsize::new(0);\n static NUM_NOTIFY_LOCAL: AtomicUsize = AtomicUsize::new(0);\n static NUM_UNPARKS_LOCAL: AtomicUsize = AtomicUsize::new(0);\n static NUM_LIFO_SCHEDULES: AtomicUsize = AtomicUsize::new(0);\n static NUM_LIFO_CAPPED: AtomicUsize = AtomicUsize::new(0);\n\n impl Drop for super::Counters {\n fn drop(&mut self) {\n let notifies_local = NUM_NOTIFY_LOCAL.load(Relaxed);\n let unparks_local = NUM_UNPARKS_LOCAL.load(Relaxed);\n let maintenance = NUM_MAINTENANCE.load(Relaxed);\n let lifo_scheds = NUM_LIFO_SCHEDULES.load(Relaxed);\n let lifo_capped = NUM_LIFO_CAPPED.load(Relaxed);\n\n println!(\"---\");\n println!(\"notifies (local): {}\", notifies_local);\n println!(\" unparks (local): {}\", unparks_local);\n println!(\" maintenance: {}\", maintenance);\n println!(\" LIFO schedules: {}\", lifo_scheds);\n println!(\" LIFO capped: {}\", lifo_capped);\n }\n }\n\n pub(crate) fn inc_num_inc_notify_local() {\n NUM_NOTIFY_LOCAL.fetch_add(1, Relaxed);\n }\n\n pub(crate) fn inc_num_unparks_local() {\n NUM_UNPARKS_LOCAL.fetch_add(1, Relaxed);\n }\n\n pub(crate) fn inc_num_maintenance() {\n NUM_MAINTENANCE.fetch_add(1, Relaxed);\n }\n\n pub(crate) fn inc_lifo_schedules() {\n NUM_LIFO_SCHEDULES.fetch_add(1, Relaxed);\n }\n\n pub(crate) fn inc_lifo_capped() {\n NUM_LIFO_CAPPED.fetch_add(1, Relaxed);\n }\n}\n\n#[cfg(not(tokio_internal_mt_counters))]\nmod imp {\n pub(crate) fn inc_num_inc_notify_local() {}\n pub(crate) fn inc_num_unparks_local() {}\n pub(crate) fn inc_num_maintenance() {}\n pub(crate) fn inc_lifo_schedules() {}\n pub(crate) fn inc_lifo_capped() {}\n}\n\n#[derive(Debug)]\npub(crate) struct Counters;\n\npub(super) use imp::*;\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/handle/metrics.rs", "content": "use super::Handle;\nuse crate::runtime::WorkerMetrics;\n\ncfg_unstable_metrics! {\n use crate::runtime::SchedulerMetrics;\n}\n\nimpl Handle {\n pub(crate) fn num_workers(&self) -> usize {\n self.shared.worker_metrics.len()\n }\n\n pub(crate) fn num_alive_tasks(&self) -> usize {\n self.shared.owned.num_alive_tasks()\n }\n\n pub(crate) fn injection_queue_depth(&self) -> usize {\n self.shared.injection_queue_depth()\n }\n\n pub(crate) fn worker_metrics(&self, worker: usize) -> &WorkerMetrics {\n &self.shared.worker_metrics[worker]\n }\n\n cfg_unstable_metrics! {\n cfg_64bit_metrics! {\n pub(crate) fn spawned_tasks_count(&self) -> u64 {\n self.shared.owned.spawned_tasks_count()\n }\n }\n\n pub(crate) fn num_blocking_threads(&self) -> usize {\n // workers are currently spawned using spawn_blocking\n self.blocking_spawner\n .num_threads()\n .saturating_sub(self.num_workers())\n }\n\n pub(crate) fn num_idle_blocking_threads(&self) -> usize {\n self.blocking_spawner.num_idle_threads()\n }\n\n pub(crate) fn scheduler_metrics(&self) -> &SchedulerMetrics {\n &self.shared.scheduler_metrics\n }\n\n pub(crate) fn worker_local_queue_depth(&self, worker: usize) -> usize {\n self.shared.worker_local_queue_depth(worker)\n }\n\n pub(crate) fn blocking_queue_depth(&self) -> usize {\n self.blocking_spawner.queue_depth()\n }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/handle/taskdump.rs", "content": "use super::Handle;\n\nuse crate::runtime::Dump;\n\nimpl Handle {\n pub(crate) async fn dump(&self) -> Dump {\n let trace_status = &self.shared.trace_status;\n\n // If a dump is in progress, block.\n trace_status.start_trace_request(self).await;\n\n let result = loop {\n if let Some(result) = trace_status.take_result() {\n break result;\n } else {\n self.notify_all();\n trace_status.result_ready.notified().await;\n }\n };\n\n // Allow other queued dumps to proceed.\n trace_status.end_trace_request(self).await;\n\n result\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/handle.rs", "content": "use crate::future::Future;\nuse crate::loom::sync::Arc;\nuse crate::runtime::scheduler::multi_thread::worker;\nuse crate::runtime::task::{Notified, Task, TaskHarnessScheduleHooks};\nuse crate::runtime::{\n blocking, driver,\n task::{self, JoinHandle, SpawnLocation},\n TaskHooks, TaskMeta, TimerFlavor,\n};\nuse crate::util::RngSeedGenerator;\n\nuse std::fmt;\nuse std::num::NonZeroU64;\n\nmod metrics;\n\ncfg_taskdump! {\n mod taskdump;\n}\n\n#[cfg(all(tokio_unstable, feature = \"time\"))]\nuse crate::loom::sync::atomic::{AtomicBool, Ordering::SeqCst};\n\n/// Handle to the multi thread scheduler\npub(crate) struct Handle {\n /// Task spawner\n pub(super) shared: worker::Shared,\n\n /// Resource driver handles\n pub(crate) driver: driver::Handle,\n\n /// Blocking pool spawner\n pub(crate) blocking_spawner: blocking::Spawner,\n\n /// Current random number generator seed\n pub(crate) seed_generator: RngSeedGenerator,\n\n /// User-supplied hooks to invoke for things\n pub(crate) task_hooks: TaskHooks,\n\n #[cfg_attr(not(feature = \"time\"), allow(dead_code))]\n /// Timer flavor used by the runtime\n pub(crate) timer_flavor: TimerFlavor,\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n /// Indicates that the runtime is shutting down.\n pub(crate) is_shutdown: AtomicBool,\n}\n\nimpl Handle {\n /// Spawns a future onto the thread pool\n pub(crate) fn spawn(\n me: &Arc,\n future: F,\n id: task::Id,\n spawned_at: SpawnLocation,\n ) -> JoinHandle\n where\n F: crate::future::Future + Send + 'static,\n F::Output: Send + 'static,\n {\n Self::bind_new_task(me, future, id, spawned_at)\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n pub(crate) fn is_shutdown(&self) -> bool {\n self.is_shutdown\n .load(crate::loom::sync::atomic::Ordering::SeqCst)\n }\n\n pub(crate) fn shutdown(&self) {\n self.close();\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n self.is_shutdown.store(true, SeqCst);\n }\n\n #[track_caller]\n pub(super) fn bind_new_task(\n me: &Arc,\n future: T,\n id: task::Id,\n spawned_at: SpawnLocation,\n ) -> JoinHandle\n where\n T: Future + Send + 'static,\n T::Output: Send + 'static,\n {\n let (handle, notified) = me.shared.owned.bind(future, me.clone(), id, spawned_at);\n\n me.task_hooks.spawn(&TaskMeta {\n id,\n spawned_at,\n _phantom: Default::default(),\n });\n\n me.schedule_option_task_without_yield(notified);\n\n handle\n }\n}\n\nimpl task::Schedule for Arc {\n fn release(&self, task: &Task) -> Option> {\n self.shared.owned.remove(task)\n }\n\n fn schedule(&self, task: Notified) {\n self.schedule_task(task, false);\n }\n\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: self.task_hooks.task_terminate_callback.clone(),\n }\n }\n\n fn yield_now(&self, task: Notified) {\n self.schedule_task(task, true);\n }\n}\n\nimpl Handle {\n pub(crate) fn owned_id(&self) -> NonZeroU64 {\n self.shared.owned.id\n }\n}\n\nimpl fmt::Debug for Handle {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"multi_thread::Handle { ... }\").finish()\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/idle.rs", "content": "//! Coordinates idling workers\n\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::runtime::scheduler::multi_thread::Shared;\n\nuse std::fmt;\nuse std::sync::atomic::Ordering::{self, SeqCst};\n\npub(super) struct Idle {\n /// Tracks both the number of searching workers and the number of unparked\n /// workers.\n ///\n /// Used as a fast-path to avoid acquiring the lock when needed.\n state: AtomicUsize,\n\n /// Total number of workers.\n num_workers: usize,\n}\n\n/// Data synchronized by the scheduler mutex\npub(super) struct Synced {\n /// Sleeping workers\n sleepers: Vec,\n}\n\nconst UNPARK_SHIFT: usize = 16;\nconst UNPARK_MASK: usize = !SEARCH_MASK;\nconst SEARCH_MASK: usize = (1 << UNPARK_SHIFT) - 1;\n\n#[derive(Copy, Clone)]\nstruct State(usize);\n\nimpl Idle {\n pub(super) fn new(num_workers: usize) -> (Idle, Synced) {\n let init = State::new(num_workers);\n\n let idle = Idle {\n state: AtomicUsize::new(init.into()),\n num_workers,\n };\n\n let synced = Synced {\n sleepers: Vec::with_capacity(num_workers),\n };\n\n (idle, synced)\n }\n\n /// If there are no workers actively searching, returns the index of a\n /// worker currently sleeping.\n pub(super) fn worker_to_notify(&self, shared: &Shared) -> Option {\n // If at least one worker is spinning, work being notified will\n // eventually be found. A searching thread will find **some** work and\n // notify another worker, eventually leading to our work being found.\n //\n // For this to happen, this load must happen before the thread\n // transitioning `num_searching` to zero. Acquire / Release does not\n // provide sufficient guarantees, so this load is done with `SeqCst` and\n // will pair with the `fetch_sub(1)` when transitioning out of\n // searching.\n if !self.notify_should_wakeup() {\n return None;\n }\n\n // Acquire the lock\n let mut lock = shared.synced.lock();\n\n // Check again, now that the lock is acquired\n if !self.notify_should_wakeup() {\n return None;\n }\n\n // A worker should be woken up, atomically increment the number of\n // searching workers as well as the number of unparked workers.\n State::unpark_one(&self.state, 1);\n\n // Get the worker to unpark\n let ret = lock.idle.sleepers.pop();\n debug_assert!(ret.is_some());\n\n ret\n }\n\n /// Returns `true` if the worker needs to do a final check for submitted\n /// work.\n pub(super) fn transition_worker_to_parked(\n &self,\n shared: &Shared,\n worker: usize,\n is_searching: bool,\n ) -> bool {\n // Acquire the lock\n let mut lock = shared.synced.lock();\n\n // Decrement the number of unparked threads\n let ret = State::dec_num_unparked(&self.state, is_searching);\n\n // Track the sleeping worker\n lock.idle.sleepers.push(worker);\n\n ret\n }\n\n pub(super) fn transition_worker_to_searching(&self) -> bool {\n let state = State::load(&self.state, SeqCst);\n if 2 * state.num_searching() >= self.num_workers {\n return false;\n }\n\n // It is possible for this routine to allow more than 50% of the workers\n // to search. That is OK. Limiting searchers is only an optimization to\n // prevent too much contention.\n State::inc_num_searching(&self.state, SeqCst);\n true\n }\n\n /// A lightweight transition from searching -> running.\n ///\n /// Returns `true` if this is the final searching worker. The caller\n /// **must** notify a new worker.\n pub(super) fn transition_worker_from_searching(&self) -> bool {\n State::dec_num_searching(&self.state)\n }\n\n /// Unpark a specific worker. This happens if tasks are submitted from\n /// within the worker's park routine.\n ///\n /// Returns `true` if the worker was parked before calling the method.\n pub(super) fn unpark_worker_by_id(&self, shared: &Shared, worker_id: usize) -> bool {\n let mut lock = shared.synced.lock();\n let sleepers = &mut lock.idle.sleepers;\n\n for index in 0..sleepers.len() {\n if sleepers[index] == worker_id {\n sleepers.swap_remove(index);\n\n // Update the state accordingly while the lock is held.\n State::unpark_one(&self.state, 0);\n\n return true;\n }\n }\n\n false\n }\n\n /// Returns `true` if `worker_id` is contained in the sleep set.\n pub(super) fn is_parked(&self, shared: &Shared, worker_id: usize) -> bool {\n let lock = shared.synced.lock();\n lock.idle.sleepers.contains(&worker_id)\n }\n\n fn notify_should_wakeup(&self) -> bool {\n let state = State(self.state.fetch_add(0, SeqCst));\n state.num_searching() == 0 && state.num_unparked() < self.num_workers\n }\n}\n\nimpl State {\n fn new(num_workers: usize) -> State {\n // All workers start in the unparked state\n let ret = State(num_workers << UNPARK_SHIFT);\n debug_assert_eq!(num_workers, ret.num_unparked());\n debug_assert_eq!(0, ret.num_searching());\n ret\n }\n\n fn load(cell: &AtomicUsize, ordering: Ordering) -> State {\n State(cell.load(ordering))\n }\n\n fn unpark_one(cell: &AtomicUsize, num_searching: usize) {\n cell.fetch_add(num_searching | (1 << UNPARK_SHIFT), SeqCst);\n }\n\n fn inc_num_searching(cell: &AtomicUsize, ordering: Ordering) {\n cell.fetch_add(1, ordering);\n }\n\n /// Returns `true` if this is the final searching worker\n fn dec_num_searching(cell: &AtomicUsize) -> bool {\n let state = State(cell.fetch_sub(1, SeqCst));\n state.num_searching() == 1\n }\n\n /// Track a sleeping worker\n ///\n /// Returns `true` if this is the final searching worker.\n fn dec_num_unparked(cell: &AtomicUsize, is_searching: bool) -> bool {\n let mut dec = 1 << UNPARK_SHIFT;\n\n if is_searching {\n dec += 1;\n }\n\n let prev = State(cell.fetch_sub(dec, SeqCst));\n is_searching && prev.num_searching() == 1\n }\n\n /// Number of workers currently searching\n fn num_searching(self) -> usize {\n self.0 & SEARCH_MASK\n }\n\n /// Number of workers currently unparked\n fn num_unparked(self) -> usize {\n (self.0 & UNPARK_MASK) >> UNPARK_SHIFT\n }\n}\n\nimpl From for State {\n fn from(src: usize) -> State {\n State(src)\n }\n}\n\nimpl From for usize {\n fn from(src: State) -> usize {\n src.0\n }\n}\n\nimpl fmt::Debug for State {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"worker::State\")\n .field(\"num_unparked\", &self.num_unparked())\n .field(\"num_searching\", &self.num_searching())\n .finish()\n }\n}\n\n#[test]\nfn test_state() {\n assert_eq!(0, UNPARK_MASK & SEARCH_MASK);\n assert_eq!(0, !(UNPARK_MASK | SEARCH_MASK));\n\n let state = State::new(10);\n assert_eq!(10, state.num_unparked());\n assert_eq!(0, state.num_searching());\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/mod.rs", "content": "//! Multi-threaded runtime\n\nmod counters;\nuse counters::Counters;\n\nmod handle;\npub(crate) use handle::Handle;\n\nmod overflow;\npub(crate) use overflow::Overflow;\n\nmod idle;\nuse self::idle::Idle;\n\nmod stats;\npub(crate) use stats::Stats;\n\nmod park;\npub(crate) use park::{Parker, Unparker};\n\npub(crate) mod queue;\n\nmod worker;\npub(crate) use worker::{Context, Launch, Shared};\n\ncfg_taskdump! {\n mod trace;\n use trace::TraceStatus;\n\n pub(crate) use worker::Synced;\n}\n\ncfg_not_taskdump! {\n mod trace_mock;\n use trace_mock::TraceStatus;\n}\n\npub(crate) use worker::block_in_place;\n\nuse crate::loom::sync::Arc;\nuse crate::runtime::{\n blocking,\n driver::{self, Driver},\n scheduler, Config, TimerFlavor,\n};\nuse crate::util::RngSeedGenerator;\n\nuse std::fmt;\nuse std::future::Future;\n\n/// Work-stealing based thread pool for executing futures.\npub(crate) struct MultiThread;\n\n// ===== impl MultiThread =====\n\nimpl MultiThread {\n pub(crate) fn new(\n size: usize,\n driver: Driver,\n driver_handle: driver::Handle,\n blocking_spawner: blocking::Spawner,\n seed_generator: RngSeedGenerator,\n config: Config,\n timer_flavor: TimerFlavor,\n ) -> (MultiThread, Arc, Launch) {\n let parker = Parker::new(driver);\n let (handle, launch) = worker::create(\n size,\n parker,\n driver_handle,\n blocking_spawner,\n seed_generator,\n config,\n timer_flavor,\n );\n\n (MultiThread, handle, launch)\n }\n\n /// Blocks the current thread waiting for the future to complete.\n ///\n /// The future will execute on the current thread, but all spawned tasks\n /// will be executed on the thread pool.\n pub(crate) fn block_on(&self, handle: &scheduler::Handle, future: F) -> F::Output\n where\n F: Future,\n {\n crate::runtime::context::enter_runtime(handle, true, |blocking| {\n blocking.block_on(future).expect(\"failed to park thread\")\n })\n }\n\n pub(crate) fn shutdown(&mut self, handle: &scheduler::Handle) {\n match handle {\n scheduler::Handle::MultiThread(handle) => handle.shutdown(),\n _ => panic!(\"expected MultiThread scheduler\"),\n }\n }\n}\n\nimpl fmt::Debug for MultiThread {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"MultiThread\").finish()\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/overflow.rs", "content": "use crate::runtime::task;\n\n#[cfg(test)]\nuse std::cell::RefCell;\n\npub(crate) trait Overflow {\n fn push(&self, task: task::Notified);\n\n fn push_batch(&self, iter: I)\n where\n I: Iterator>;\n}\n\n#[cfg(test)]\nimpl Overflow for RefCell>> {\n fn push(&self, task: task::Notified) {\n self.borrow_mut().push(task);\n }\n\n fn push_batch(&self, iter: I)\n where\n I: Iterator>,\n {\n self.borrow_mut().extend(iter);\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/park.rs", "content": "//! Parks the runtime.\n//!\n//! A combination of the various resource driver park handles.\n\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::{Arc, Condvar, Mutex};\nuse crate::runtime::driver::{self, Driver};\nuse crate::util::TryLock;\n\nuse std::sync::atomic::Ordering::SeqCst;\nuse std::time::{Duration, Instant};\n\n#[cfg(loom)]\nuse crate::runtime::park::CURRENT_THREAD_PARK_COUNT;\n\npub(crate) struct Parker {\n inner: Arc,\n}\n\npub(crate) struct Unparker {\n inner: Arc,\n}\n\nstruct Inner {\n /// Avoids entering the park if possible\n state: AtomicUsize,\n\n /// Used to coordinate access to the driver / `condvar`\n mutex: Mutex<()>,\n\n /// `Condvar` to block on if the driver is unavailable.\n condvar: Condvar,\n\n /// Resource (I/O, time, ...) driver\n shared: Arc,\n}\n\nconst EMPTY: usize = 0;\nconst PARKED_CONDVAR: usize = 1;\nconst PARKED_DRIVER: usize = 2;\nconst NOTIFIED: usize = 3;\n\n/// Shared across multiple Parker handles\nstruct Shared {\n /// Shared driver. Only one thread at a time can use this\n driver: TryLock,\n}\n\nimpl Parker {\n pub(crate) fn new(driver: Driver) -> Parker {\n Parker {\n inner: Arc::new(Inner {\n state: AtomicUsize::new(EMPTY),\n mutex: Mutex::new(()),\n condvar: Condvar::new(),\n shared: Arc::new(Shared {\n driver: TryLock::new(driver),\n }),\n }),\n }\n }\n\n pub(crate) fn unpark(&self) -> Unparker {\n Unparker {\n inner: self.inner.clone(),\n }\n }\n\n pub(crate) fn park(&mut self, handle: &driver::Handle) {\n self.inner.park(handle);\n }\n\n /// Parks the current thread for up to `duration`.\n ///\n /// This function tries to acquire the driver lock. If it succeeds, it\n /// parks using the driver. Otherwise, it fails back to using a condvar,\n /// unless the duration is zero, in which case it returns immediately.\n pub(crate) fn park_timeout(&mut self, handle: &driver::Handle, duration: Duration) {\n if let Some(mut driver) = self.inner.shared.driver.try_lock() {\n self.inner.park_driver(&mut driver, handle, Some(duration));\n } else if !duration.is_zero() {\n self.inner.park_condvar(Some(duration));\n } else {\n // https://github.com/tokio-rs/tokio/issues/6536\n // Hacky, but it's just for loom tests. The counter gets incremented during\n // `park_timeout`, but we still have to increment the counter if we can't acquire the\n // lock.\n #[cfg(loom)]\n CURRENT_THREAD_PARK_COUNT.with(|count| count.fetch_add(1, SeqCst));\n }\n }\n\n pub(crate) fn shutdown(&mut self, handle: &driver::Handle) {\n self.inner.shutdown(handle);\n }\n}\n\nimpl Clone for Parker {\n fn clone(&self) -> Parker {\n Parker {\n inner: Arc::new(Inner {\n state: AtomicUsize::new(EMPTY),\n mutex: Mutex::new(()),\n condvar: Condvar::new(),\n shared: self.inner.shared.clone(),\n }),\n }\n }\n}\n\nimpl Unparker {\n pub(crate) fn unpark(&self, driver: &driver::Handle) {\n self.inner.unpark(driver);\n }\n}\n\nimpl Inner {\n /// Parks the current thread for at most `dur`.\n fn park(&self, handle: &driver::Handle) {\n // If we were previously notified then we consume this notification and\n // return quickly.\n if self\n .state\n .compare_exchange(NOTIFIED, EMPTY, SeqCst, SeqCst)\n .is_ok()\n {\n return;\n }\n\n if let Some(mut driver) = self.shared.driver.try_lock() {\n self.park_driver(&mut driver, handle, None);\n } else {\n self.park_condvar(None);\n }\n }\n\n /// Parks the current thread using a condvar for up to `duration`.\n ///\n /// If `duration` is `None`, parks indefinitely until notified.\n ///\n /// # Panics\n ///\n /// Panics if `duration` is `Some` and the duration is zero.\n fn park_condvar(&self, duration: Option) {\n // Otherwise we need to coordinate going to sleep\n let mut m = self.mutex.lock();\n\n match self\n .state\n .compare_exchange(EMPTY, PARKED_CONDVAR, SeqCst, SeqCst)\n {\n Ok(_) => {}\n Err(NOTIFIED) => {\n // We must read here, even though we know it will be `NOTIFIED`.\n // This is because `unpark` may have been called again since we read\n // `NOTIFIED` in the `compare_exchange` above. We must perform an\n // acquire operation that synchronizes with that `unpark` to observe\n // any writes it made before the call to unpark. To do that we must\n // read from the write it made to `state`.\n let old = self.state.swap(EMPTY, SeqCst);\n debug_assert_eq!(old, NOTIFIED, \"park state changed unexpectedly\");\n\n return;\n }\n Err(actual) => panic!(\"inconsistent park state; actual = {actual}\"),\n }\n\n let timeout_at = duration.map(|d| {\n Instant::now()\n .checked_add(d)\n // best effort to avoid overflow and still provide a usable timeout\n .unwrap_or(Instant::now() + Duration::from_secs(1))\n });\n\n loop {\n let is_timeout;\n (m, is_timeout) = match timeout_at {\n Some(timeout_at) => {\n let dur = timeout_at.saturating_duration_since(Instant::now());\n if !dur.is_zero() {\n // Ideally, we would use `condvar.wait_timeout_until` here, but it is not available\n // in `loom`. So we manually compute the timeout.\n let (m, res) = self.condvar.wait_timeout(m, dur).unwrap();\n (m, res.timed_out())\n } else {\n (m, true)\n }\n }\n None => (self.condvar.wait(m).unwrap(), false),\n };\n\n if is_timeout {\n match self.state.swap(EMPTY, SeqCst) {\n PARKED_CONDVAR => return, // timed out, and no notification received\n NOTIFIED => return, // notification and timeout happened concurrently\n actual @ (PARKED_DRIVER | EMPTY) => {\n panic!(\"inconsistent park_timeout state, actual = {actual}\")\n }\n invalid => panic!(\"invalid park_timeout state, actual = {invalid}\"),\n }\n } else if self\n .state\n .compare_exchange(NOTIFIED, EMPTY, SeqCst, SeqCst)\n .is_ok()\n {\n // got a notification\n return;\n }\n\n // spurious wakeup, go back to sleep\n }\n }\n\n fn park_driver(\n &self,\n driver: &mut Driver,\n handle: &driver::Handle,\n duration: Option,\n ) {\n if duration.as_ref().is_some_and(Duration::is_zero) {\n // zero duration doesn't actually park the thread, it just\n // polls the I/O events, timers, etc.\n driver.park_timeout(handle, Duration::ZERO);\n return;\n }\n\n match self\n .state\n .compare_exchange(EMPTY, PARKED_DRIVER, SeqCst, SeqCst)\n {\n Ok(_) => {}\n Err(NOTIFIED) => {\n // We must read here, even though we know it will be `NOTIFIED`.\n // This is because `unpark` may have been called again since we read\n // `NOTIFIED` in the `compare_exchange` above. We must perform an\n // acquire operation that synchronizes with that `unpark` to observe\n // any writes it made before the call to unpark. To do that we must\n // read from the write it made to `state`.\n let old = self.state.swap(EMPTY, SeqCst);\n debug_assert_eq!(old, NOTIFIED, \"park state changed unexpectedly\");\n\n return;\n }\n Err(actual) => panic!(\"inconsistent park state; actual = {actual}\"),\n }\n\n if let Some(duration) = duration {\n debug_assert_ne!(duration, Duration::ZERO);\n driver.park_timeout(handle, duration);\n } else {\n driver.park(handle);\n }\n\n match self.state.swap(EMPTY, SeqCst) {\n NOTIFIED => {} // got a notification, hurray!\n PARKED_DRIVER => {} // no notification, alas\n n => panic!(\"inconsistent park_timeout state: {n}\"),\n }\n }\n\n fn unpark(&self, driver: &driver::Handle) {\n // To ensure the unparked thread will observe any writes we made before\n // this call, we must perform a release operation that `park` can\n // synchronize with. To do that we must write `NOTIFIED` even if `state`\n // is already `NOTIFIED`. That is why this must be a swap rather than a\n // compare-and-swap that returns if it reads `NOTIFIED` on failure.\n match self.state.swap(NOTIFIED, SeqCst) {\n EMPTY => {} // no one was waiting\n NOTIFIED => {} // already unparked\n PARKED_CONDVAR => self.unpark_condvar(),\n PARKED_DRIVER => driver.unpark(),\n actual => panic!(\"inconsistent state in unpark; actual = {actual}\"),\n }\n }\n\n fn unpark_condvar(&self) {\n // There is a period between when the parked thread sets `state` to\n // `PARKED` (or last checked `state` in the case of a spurious wake\n // up) and when it actually waits on `cvar`. If we were to notify\n // during this period it would be ignored and then when the parked\n // thread went to sleep it would never wake up. Fortunately, it has\n // `lock` locked at this stage so we can acquire `lock` to wait until\n // it is ready to receive the notification.\n //\n // Releasing `lock` before the call to `notify_one` means that when the\n // parked thread wakes it doesn't get woken only to have to wait for us\n // to release `lock`.\n drop(self.mutex.lock());\n\n self.condvar.notify_one();\n }\n\n fn shutdown(&self, handle: &driver::Handle) {\n if let Some(mut driver) = self.shared.driver.try_lock() {\n driver.shutdown(handle);\n }\n\n self.condvar.notify_all();\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/queue.rs", "content": "//! Run-queue structures to support a work-stealing scheduler\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::Arc;\nuse crate::runtime::scheduler::multi_thread::{Overflow, Stats};\nuse crate::runtime::task;\n\nuse std::mem::{self, MaybeUninit};\nuse std::ptr;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release};\n\n// Use wider integers when possible to increase ABA resilience.\n//\n// See issue #5041: .\ncfg_has_atomic_u64! {\n type UnsignedShort = u32;\n type UnsignedLong = u64;\n type AtomicUnsignedShort = crate::loom::sync::atomic::AtomicU32;\n type AtomicUnsignedLong = crate::loom::sync::atomic::AtomicU64;\n}\ncfg_not_has_atomic_u64! {\n type UnsignedShort = u16;\n type UnsignedLong = u32;\n type AtomicUnsignedShort = crate::loom::sync::atomic::AtomicU16;\n type AtomicUnsignedLong = crate::loom::sync::atomic::AtomicU32;\n}\n\n/// Producer handle. May only be used from a single thread.\npub(crate) struct Local {\n inner: Arc>,\n}\n\n/// Consumer handle. May be used from many threads.\npub(crate) struct Steal(Arc>);\n\npub(crate) struct Inner {\n /// Concurrently updated by many threads.\n ///\n /// Contains two `UnsignedShort` values. The `LSB` byte is the \"real\" head of\n /// the queue. The `UnsignedShort` in the `MSB` is set by a stealer in process\n /// of stealing values. It represents the first value being stolen in the\n /// batch. The `UnsignedShort` indices are intentionally wider than strictly\n /// required for buffer indexing in order to provide ABA mitigation and make\n /// it possible to distinguish between full and empty buffers.\n ///\n /// When both `UnsignedShort` values are the same, there is no active\n /// stealer.\n ///\n /// Tracking an in-progress stealer prevents a wrapping scenario.\n head: AtomicUnsignedLong,\n\n /// Only updated by producer thread but read by many threads.\n tail: AtomicUnsignedShort,\n\n /// Elements\n buffer: Box<[UnsafeCell>>; LOCAL_QUEUE_CAPACITY]>,\n}\n\nunsafe impl Send for Inner {}\nunsafe impl Sync for Inner {}\n\n#[cfg(not(loom))]\nconst LOCAL_QUEUE_CAPACITY: usize = 256;\n\n// Shrink the size of the local queue when using loom. This shouldn't impact\n// logic, but allows loom to test more edge cases in a reasonable a mount of\n// time.\n#[cfg(loom)]\nconst LOCAL_QUEUE_CAPACITY: usize = 4;\n\nconst MASK: usize = LOCAL_QUEUE_CAPACITY - 1;\n\n// Constructing the fixed size array directly is very awkward. The only way to\n// do it is to repeat `UnsafeCell::new(MaybeUninit::uninit())` 256 times, as\n// the contents are not Copy. The trick with defining a const doesn't work for\n// generic types.\nfn make_fixed_size(buffer: Box<[T]>) -> Box<[T; LOCAL_QUEUE_CAPACITY]> {\n assert_eq!(buffer.len(), LOCAL_QUEUE_CAPACITY);\n\n // safety: We check that the length is correct.\n unsafe { Box::from_raw(Box::into_raw(buffer).cast()) }\n}\n\n/// Create a new local run-queue\npub(crate) fn local() -> (Steal, Local) {\n let mut buffer = Vec::with_capacity(LOCAL_QUEUE_CAPACITY);\n\n for _ in 0..LOCAL_QUEUE_CAPACITY {\n buffer.push(UnsafeCell::new(MaybeUninit::uninit()));\n }\n\n let inner = Arc::new(Inner {\n head: AtomicUnsignedLong::new(0),\n tail: AtomicUnsignedShort::new(0),\n buffer: make_fixed_size(buffer.into_boxed_slice()),\n });\n\n let local = Local {\n inner: inner.clone(),\n };\n\n let remote = Steal(inner);\n\n (remote, local)\n}\n\nimpl Local {\n /// Returns the number of entries in the queue\n pub(crate) fn len(&self) -> usize {\n let (_, head) = unpack(self.inner.head.load(Acquire));\n // safety: this is the **only** thread that updates this cell.\n let tail = unsafe { self.inner.tail.unsync_load() };\n len(head, tail)\n }\n\n /// How many tasks can be pushed into the queue\n pub(crate) fn remaining_slots(&self) -> usize {\n let (steal, _) = unpack(self.inner.head.load(Acquire));\n // safety: this is the **only** thread that updates this cell.\n let tail = unsafe { self.inner.tail.unsync_load() };\n\n LOCAL_QUEUE_CAPACITY - len(steal, tail)\n }\n\n pub(crate) fn max_capacity(&self) -> usize {\n LOCAL_QUEUE_CAPACITY\n }\n\n /// Returns false if there are any entries in the queue\n ///\n /// Separate to `is_stealable` so that refactors of `is_stealable` to \"protect\"\n /// some tasks from stealing won't affect this\n pub(crate) fn has_tasks(&self) -> bool {\n self.len() != 0\n }\n\n /// Pushes a batch of tasks to the back of the queue. All tasks must fit in\n /// the local queue.\n ///\n /// # Panics\n ///\n /// The method panics if there is not enough capacity to fit in the queue.\n pub(crate) fn push_back(&mut self, tasks: impl ExactSizeIterator>) {\n let len = tasks.len();\n assert!(len <= LOCAL_QUEUE_CAPACITY);\n\n if len == 0 {\n // Nothing to do\n return;\n }\n\n let head = self.inner.head.load(Acquire);\n let (steal, _) = unpack(head);\n\n // safety: this is the **only** thread that updates this cell.\n let mut tail = unsafe { self.inner.tail.unsync_load() };\n\n if tail.wrapping_sub(steal) <= (LOCAL_QUEUE_CAPACITY - len) as UnsignedShort {\n // Yes, this if condition is structured a bit weird (first block\n // does nothing, second returns an error). It is this way to match\n // `push_back_or_overflow`.\n } else {\n panic!()\n }\n\n for task in tasks {\n let idx = tail as usize & MASK;\n\n self.inner.buffer[idx].with_mut(|ptr| {\n // Write the task to the slot\n //\n // Safety: There is only one producer and the above `if`\n // condition ensures we don't touch a cell if there is a\n // value, thus no consumer.\n unsafe {\n ptr::write((*ptr).as_mut_ptr(), task);\n }\n });\n\n tail = tail.wrapping_add(1);\n }\n\n self.inner.tail.store(tail, Release);\n }\n\n /// Pushes a task to the back of the local queue, if there is not enough\n /// capacity in the queue, this triggers the overflow operation.\n ///\n /// When the queue overflows, half of the current contents of the queue is\n /// moved to the given Injection queue. This frees up capacity for more\n /// tasks to be pushed into the local queue.\n pub(crate) fn push_back_or_overflow>(\n &mut self,\n mut task: task::Notified,\n overflow: &O,\n stats: &mut Stats,\n ) {\n let tail = loop {\n let head = self.inner.head.load(Acquire);\n let (steal, real) = unpack(head);\n\n // safety: this is the **only** thread that updates this cell.\n let tail = unsafe { self.inner.tail.unsync_load() };\n\n if tail.wrapping_sub(steal) < LOCAL_QUEUE_CAPACITY as UnsignedShort {\n // There is capacity for the task\n break tail;\n } else if steal != real {\n // Concurrently stealing, this will free up capacity, so only\n // push the task onto the inject queue\n overflow.push(task);\n return;\n } else {\n // Push the current task and half of the queue into the\n // inject queue.\n match self.push_overflow(task, real, tail, overflow, stats) {\n Ok(_) => return,\n // Lost the race, try again\n Err(v) => {\n task = v;\n }\n }\n }\n };\n\n self.push_back_finish(task, tail);\n }\n\n // Second half of `push_back`\n fn push_back_finish(&self, task: task::Notified, tail: UnsignedShort) {\n // Map the position to a slot index.\n let idx = tail as usize & MASK;\n\n self.inner.buffer[idx].with_mut(|ptr| {\n // Write the task to the slot\n //\n // Safety: There is only one producer and the above `if`\n // condition ensures we don't touch a cell if there is a\n // value, thus no consumer.\n unsafe {\n ptr::write((*ptr).as_mut_ptr(), task);\n }\n });\n\n // Make the task available. Synchronizes with a load in\n // `steal_into2`.\n self.inner.tail.store(tail.wrapping_add(1), Release);\n }\n\n /// Moves a batch of tasks into the inject queue.\n ///\n /// This will temporarily make some of the tasks unavailable to stealers.\n /// Once `push_overflow` is done, a notification is sent out, so if other\n /// workers \"missed\" some of the tasks during a steal, they will get\n /// another opportunity.\n #[inline(never)]\n fn push_overflow>(\n &mut self,\n task: task::Notified,\n head: UnsignedShort,\n tail: UnsignedShort,\n overflow: &O,\n stats: &mut Stats,\n ) -> Result<(), task::Notified> {\n /// How many elements are we taking from the local queue.\n ///\n /// This is one less than the number of tasks pushed to the inject\n /// queue as we are also inserting the `task` argument.\n const NUM_TASKS_TAKEN: UnsignedShort = (LOCAL_QUEUE_CAPACITY / 2) as UnsignedShort;\n\n assert_eq!(\n tail.wrapping_sub(head) as usize,\n LOCAL_QUEUE_CAPACITY,\n \"queue is not full; tail = {tail}; head = {head}\"\n );\n\n let prev = pack(head, head);\n\n // Claim a bunch of tasks\n //\n // We are claiming the tasks **before** reading them out of the buffer.\n // This is safe because only the **current** thread is able to push new\n // tasks.\n //\n // There isn't really any need for memory ordering... Relaxed would\n // work. This is because all tasks are pushed into the queue from the\n // current thread (or memory has been acquired if the local queue handle\n // moved).\n if self\n .inner\n .head\n .compare_exchange(\n prev,\n pack(\n head.wrapping_add(NUM_TASKS_TAKEN),\n head.wrapping_add(NUM_TASKS_TAKEN),\n ),\n Release,\n Relaxed,\n )\n .is_err()\n {\n // We failed to claim the tasks, losing the race. Return out of\n // this function and try the full `push` routine again. The queue\n // may not be full anymore.\n return Err(task);\n }\n\n /// An iterator that takes elements out of the run queue.\n struct BatchTaskIter<'a, T: 'static> {\n buffer: &'a [UnsafeCell>>; LOCAL_QUEUE_CAPACITY],\n head: UnsignedLong,\n i: UnsignedLong,\n }\n impl<'a, T: 'static> Iterator for BatchTaskIter<'a, T> {\n type Item = task::Notified;\n\n #[inline]\n fn next(&mut self) -> Option> {\n if self.i == UnsignedLong::from(NUM_TASKS_TAKEN) {\n None\n } else {\n let i_idx = self.i.wrapping_add(self.head) as usize & MASK;\n let slot = &self.buffer[i_idx];\n\n // safety: Our CAS from before has assumed exclusive ownership\n // of the task pointers in this range.\n let task = slot.with(|ptr| unsafe { ptr::read((*ptr).as_ptr()) });\n\n self.i += 1;\n Some(task)\n }\n }\n }\n\n // safety: The CAS above ensures that no consumer will look at these\n // values again, and we are the only producer.\n let batch_iter = BatchTaskIter {\n buffer: &self.inner.buffer,\n head: head as UnsignedLong,\n i: 0,\n };\n overflow.push_batch(batch_iter.chain(std::iter::once(task)));\n\n // Add 1 to factor in the task currently being scheduled.\n stats.incr_overflow_count();\n\n Ok(())\n }\n\n /// Pops a task from the local queue.\n pub(crate) fn pop(&mut self) -> Option> {\n let mut head = self.inner.head.load(Acquire);\n\n let idx = loop {\n let (steal, real) = unpack(head);\n\n // safety: this is the **only** thread that updates this cell.\n let tail = unsafe { self.inner.tail.unsync_load() };\n\n if real == tail {\n // queue is empty\n return None;\n }\n\n let next_real = real.wrapping_add(1);\n\n // If `steal == real` there are no concurrent stealers. Both `steal`\n // and `real` are updated.\n let next = if steal == real {\n pack(next_real, next_real)\n } else {\n assert_ne!(steal, next_real);\n pack(steal, next_real)\n };\n\n // Attempt to claim a task.\n let res = self\n .inner\n .head\n .compare_exchange(head, next, AcqRel, Acquire);\n\n match res {\n Ok(_) => break real as usize & MASK,\n Err(actual) => head = actual,\n }\n };\n\n Some(self.inner.buffer[idx].with(|ptr| unsafe { ptr::read(ptr).assume_init() }))\n }\n}\n\nimpl Steal {\n /// Returns the number of entries in the queue\n pub(crate) fn len(&self) -> usize {\n let (_, head) = unpack(self.0.head.load(Acquire));\n let tail = self.0.tail.load(Acquire);\n len(head, tail)\n }\n\n /// Return true if the queue is empty,\n /// false if there are any entries in the queue\n pub(crate) fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n /// Steals half the tasks from self and place them into `dst`.\n pub(crate) fn steal_into(\n &self,\n dst: &mut Local,\n dst_stats: &mut Stats,\n ) -> Option> {\n // Safety: the caller is the only thread that mutates `dst.tail` and\n // holds a mutable reference.\n let dst_tail = unsafe { dst.inner.tail.unsync_load() };\n\n // To the caller, `dst` may **look** empty but still have values\n // contained in the buffer. If another thread is concurrently stealing\n // from `dst` there may not be enough capacity to steal.\n let (steal, _) = unpack(dst.inner.head.load(Acquire));\n\n if dst_tail.wrapping_sub(steal) > LOCAL_QUEUE_CAPACITY as UnsignedShort / 2 {\n // we *could* try to steal less here, but for simplicity, we're just\n // going to abort.\n return None;\n }\n\n // Steal the tasks into `dst`'s buffer. This does not yet expose the\n // tasks in `dst`.\n let mut n = self.steal_into2(dst, dst_tail);\n\n if n == 0 {\n // No tasks were stolen\n return None;\n }\n\n dst_stats.incr_steal_count(n as u16);\n dst_stats.incr_steal_operations();\n\n // We are returning a task here\n n -= 1;\n\n let ret_pos = dst_tail.wrapping_add(n);\n let ret_idx = ret_pos as usize & MASK;\n\n // safety: the value was written as part of `steal_into2` and not\n // exposed to stealers, so no other thread can access it.\n let ret = dst.inner.buffer[ret_idx].with(|ptr| unsafe { ptr::read((*ptr).as_ptr()) });\n\n if n == 0 {\n // The `dst` queue is empty, but a single task was stolen\n return Some(ret);\n }\n\n // Make the stolen items available to consumers\n dst.inner.tail.store(dst_tail.wrapping_add(n), Release);\n\n Some(ret)\n }\n\n // Steal tasks from `self`, placing them into `dst`. Returns the number of\n // tasks that were stolen.\n fn steal_into2(&self, dst: &mut Local, dst_tail: UnsignedShort) -> UnsignedShort {\n let mut prev_packed = self.0.head.load(Acquire);\n let mut next_packed;\n\n let n = loop {\n let (src_head_steal, src_head_real) = unpack(prev_packed);\n let src_tail = self.0.tail.load(Acquire);\n\n // If these two do not match, another thread is concurrently\n // stealing from the queue.\n if src_head_steal != src_head_real {\n return 0;\n }\n\n // Number of available tasks to steal\n let n = src_tail.wrapping_sub(src_head_real);\n let n = n - n / 2;\n\n if n == 0 {\n // No tasks available to steal\n return 0;\n }\n\n // Update the real head index to acquire the tasks.\n let steal_to = src_head_real.wrapping_add(n);\n assert_ne!(src_head_steal, steal_to);\n next_packed = pack(src_head_steal, steal_to);\n\n // Claim all those tasks. This is done by incrementing the \"real\"\n // head but not the steal. By doing this, no other thread is able to\n // steal from this queue until the current thread completes.\n let res = self\n .0\n .head\n .compare_exchange(prev_packed, next_packed, AcqRel, Acquire);\n\n match res {\n Ok(_) => break n,\n Err(actual) => prev_packed = actual,\n }\n };\n\n assert!(\n n <= LOCAL_QUEUE_CAPACITY as UnsignedShort / 2,\n \"actual = {n}\"\n );\n\n let (first, _) = unpack(next_packed);\n\n // Take all the tasks\n for i in 0..n {\n // Compute the positions\n let src_pos = first.wrapping_add(i);\n let dst_pos = dst_tail.wrapping_add(i);\n\n // Map to slots\n let src_idx = src_pos as usize & MASK;\n let dst_idx = dst_pos as usize & MASK;\n\n // Read the task\n //\n // safety: We acquired the task with the atomic exchange above.\n let task = self.0.buffer[src_idx].with(|ptr| unsafe { ptr::read((*ptr).as_ptr()) });\n\n // Write the task to the new slot\n //\n // safety: `dst` queue is empty and we are the only producer to\n // this queue.\n dst.inner.buffer[dst_idx]\n .with_mut(|ptr| unsafe { ptr::write((*ptr).as_mut_ptr(), task) });\n }\n\n let mut prev_packed = next_packed;\n\n // Update `src_head_steal` to match `src_head_real` signalling that the\n // stealing routine is complete.\n loop {\n let head = unpack(prev_packed).1;\n next_packed = pack(head, head);\n\n let res = self\n .0\n .head\n .compare_exchange(prev_packed, next_packed, AcqRel, Acquire);\n\n match res {\n Ok(_) => return n,\n Err(actual) => {\n let (actual_steal, actual_real) = unpack(actual);\n\n assert_ne!(actual_steal, actual_real);\n\n prev_packed = actual;\n }\n }\n }\n }\n}\n\nimpl Clone for Steal {\n fn clone(&self) -> Steal {\n Steal(self.0.clone())\n }\n}\n\nimpl Drop for Local {\n fn drop(&mut self) {\n if !std::thread::panicking() {\n assert!(self.pop().is_none(), \"queue not empty\");\n }\n }\n}\n\n/// Calculate the length of the queue using the head and tail.\n/// The `head` can be the `steal` or `real` head.\nfn len(head: UnsignedShort, tail: UnsignedShort) -> usize {\n tail.wrapping_sub(head) as usize\n}\n\n/// Split the head value into the real head and the index a stealer is working\n/// on.\nfn unpack(n: UnsignedLong) -> (UnsignedShort, UnsignedShort) {\n let real = n & UnsignedShort::MAX as UnsignedLong;\n let steal = n >> (mem::size_of::() * 8);\n\n (steal as UnsignedShort, real as UnsignedShort)\n}\n\n/// Join the two head values\nfn pack(steal: UnsignedShort, real: UnsignedShort) -> UnsignedLong {\n (real as UnsignedLong) | ((steal as UnsignedLong) << (mem::size_of::() * 8))\n}\n\n#[test]\nfn test_local_queue_capacity() {\n assert!(LOCAL_QUEUE_CAPACITY - 1 <= u8::MAX as usize);\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/stats.rs", "content": "use crate::runtime::{Config, MetricsBatch, WorkerMetrics};\n\nuse std::time::{Duration, Instant};\n\n/// Per-worker statistics. This is used for both tuning the scheduler and\n/// reporting runtime-level metrics/stats.\npub(crate) struct Stats {\n /// The metrics batch used to report runtime-level metrics/stats to the\n /// user.\n batch: MetricsBatch,\n\n /// Instant at which work last resumed (continued after park).\n ///\n /// This duplicates the value stored in `MetricsBatch`. We will unify\n /// `Stats` and `MetricsBatch` when we stabilize metrics.\n processing_scheduled_tasks_started_at: Instant,\n\n /// Number of tasks polled in the batch of scheduled tasks\n tasks_polled_in_batch: usize,\n\n /// Exponentially-weighted moving average of time spent polling scheduled a\n /// task.\n ///\n /// Tracked in nanoseconds, stored as a `f64` since that is what we use with\n /// the EWMA calculations\n task_poll_time_ewma: f64,\n}\n\n/// How to weigh each individual poll time, value is plucked from thin air.\nconst TASK_POLL_TIME_EWMA_ALPHA: f64 = 0.1;\n\n/// Ideally, we wouldn't go above this, value is plucked from thin air.\nconst TARGET_GLOBAL_QUEUE_INTERVAL: f64 = Duration::from_micros(200).as_nanos() as f64;\n\n/// Max value for the global queue interval. This is 2x the previous default\nconst MAX_TASKS_POLLED_PER_GLOBAL_QUEUE_INTERVAL: u32 = 127;\n\n/// This is the previous default\nconst TARGET_TASKS_POLLED_PER_GLOBAL_QUEUE_INTERVAL: u32 = 61;\n\nimpl Stats {\n pub(crate) fn new(worker_metrics: &WorkerMetrics) -> Stats {\n // Seed the value with what we hope to see.\n let task_poll_time_ewma =\n TARGET_GLOBAL_QUEUE_INTERVAL / TARGET_TASKS_POLLED_PER_GLOBAL_QUEUE_INTERVAL as f64;\n\n Stats {\n batch: MetricsBatch::new(worker_metrics),\n processing_scheduled_tasks_started_at: Instant::now(),\n tasks_polled_in_batch: 0,\n task_poll_time_ewma,\n }\n }\n\n pub(crate) fn tuned_global_queue_interval(&self, config: &Config) -> u32 {\n // If an interval is explicitly set, don't tune.\n if let Some(configured) = config.global_queue_interval {\n return configured;\n }\n\n // As of Rust 1.45, casts from f64 -> u32 are saturating, which is fine here.\n let tasks_per_interval = (TARGET_GLOBAL_QUEUE_INTERVAL / self.task_poll_time_ewma) as u32;\n\n // If we are using self-tuning, we don't want to return less than 2 as that would result in the\n // global queue always getting checked first.\n tasks_per_interval.clamp(2, MAX_TASKS_POLLED_PER_GLOBAL_QUEUE_INTERVAL)\n }\n\n pub(crate) fn submit(&mut self, to: &WorkerMetrics) {\n self.batch.submit(to, self.task_poll_time_ewma as u64);\n }\n\n pub(crate) fn about_to_park(&mut self) {\n self.batch.about_to_park();\n }\n\n pub(crate) fn unparked(&mut self) {\n self.batch.unparked();\n }\n\n pub(crate) fn inc_local_schedule_count(&mut self) {\n self.batch.inc_local_schedule_count();\n }\n\n pub(crate) fn start_processing_scheduled_tasks(&mut self) {\n self.batch.start_processing_scheduled_tasks();\n\n self.processing_scheduled_tasks_started_at = Instant::now();\n self.tasks_polled_in_batch = 0;\n }\n\n pub(crate) fn end_processing_scheduled_tasks(&mut self) {\n self.batch.end_processing_scheduled_tasks();\n\n // Update the EWMA task poll time\n if self.tasks_polled_in_batch > 0 {\n let now = Instant::now();\n\n // If we \"overflow\" this conversion, we have bigger problems than\n // slightly off stats.\n let elapsed = (now - self.processing_scheduled_tasks_started_at).as_nanos() as f64;\n let num_polls = self.tasks_polled_in_batch as f64;\n\n // Calculate the mean poll duration for a single task in the batch\n let mean_poll_duration = elapsed / num_polls;\n\n // Compute the alpha weighted by the number of tasks polled this batch.\n let weighted_alpha = 1.0 - (1.0 - TASK_POLL_TIME_EWMA_ALPHA).powf(num_polls);\n\n // Now compute the new weighted average task poll time.\n self.task_poll_time_ewma = weighted_alpha * mean_poll_duration\n + (1.0 - weighted_alpha) * self.task_poll_time_ewma;\n }\n }\n\n pub(crate) fn start_poll(&mut self) {\n self.batch.start_poll();\n\n self.tasks_polled_in_batch += 1;\n }\n\n pub(crate) fn end_poll(&mut self) {\n self.batch.end_poll();\n }\n\n pub(crate) fn incr_steal_count(&mut self, by: u16) {\n self.batch.incr_steal_count(by);\n }\n\n pub(crate) fn incr_steal_operations(&mut self) {\n self.batch.incr_steal_operations();\n }\n\n pub(crate) fn incr_overflow_count(&mut self) {\n self.batch.incr_overflow_count();\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/trace.rs", "content": "use crate::loom::sync::atomic::{AtomicBool, Ordering};\nuse crate::loom::sync::{Barrier, Mutex};\nuse crate::runtime::dump::Dump;\nuse crate::runtime::scheduler::multi_thread::Handle;\nuse crate::sync::notify::Notify;\n\n/// Tracing status of the worker.\npub(super) struct TraceStatus {\n pub(super) trace_requested: AtomicBool,\n pub(super) trace_start: Barrier,\n pub(super) trace_end: Barrier,\n pub(super) result_ready: Notify,\n pub(super) trace_result: Mutex>,\n}\n\nimpl TraceStatus {\n pub(super) fn new(remotes_len: usize) -> Self {\n Self {\n trace_requested: AtomicBool::new(false),\n trace_start: Barrier::new(remotes_len),\n trace_end: Barrier::new(remotes_len),\n result_ready: Notify::new(),\n trace_result: Mutex::new(None),\n }\n }\n\n pub(super) fn trace_requested(&self) -> bool {\n self.trace_requested.load(Ordering::Relaxed)\n }\n\n pub(super) async fn start_trace_request(&self, handle: &Handle) {\n while self\n .trace_requested\n .compare_exchange(false, true, Ordering::Acquire, Ordering::Relaxed)\n .is_err()\n {\n handle.notify_all();\n crate::task::yield_now().await;\n }\n }\n\n pub(super) fn stash_result(&self, dump: Dump) {\n let _ = self.trace_result.lock().insert(dump);\n self.result_ready.notify_one();\n }\n\n pub(super) fn take_result(&self) -> Option {\n self.trace_result.lock().take()\n }\n\n pub(super) async fn end_trace_request(&self, handle: &Handle) {\n while self\n .trace_requested\n .compare_exchange(true, false, Ordering::Acquire, Ordering::Relaxed)\n .is_err()\n {\n handle.notify_all();\n crate::task::yield_now().await;\n }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/trace_mock.rs", "content": "pub(super) struct TraceStatus {}\n\nimpl TraceStatus {\n pub(super) fn new(_: usize) -> Self {\n Self {}\n }\n\n pub(super) fn trace_requested(&self) -> bool {\n false\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/worker/metrics.rs", "content": "use super::Shared;\n\nimpl Shared {\n pub(crate) fn injection_queue_depth(&self) -> usize {\n self.inject.len()\n }\n}\n\ncfg_unstable_metrics! {\n impl Shared {\n pub(crate) fn worker_local_queue_depth(&self, worker: usize) -> usize {\n self.remotes[worker].steal.len()\n }\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/worker/taskdump.rs", "content": "use super::{Core, Handle, Shared};\n\nuse crate::loom::sync::Arc;\nuse crate::runtime::scheduler::multi_thread::Stats;\nuse crate::runtime::task::trace::trace_multi_thread;\nuse crate::runtime::{dump, WorkerMetrics};\n\nuse std::time::Duration;\n\nimpl Handle {\n pub(super) fn trace_core(&self, mut core: Box) -> Box {\n core.is_traced = false;\n\n if core.is_shutdown {\n return core;\n }\n\n // wait for other workers, or timeout without tracing\n let timeout = Duration::from_millis(250); // a _very_ generous timeout\n let barrier =\n if let Some(barrier) = self.shared.trace_status.trace_start.wait_timeout(timeout) {\n barrier\n } else {\n // don't attempt to trace\n return core;\n };\n\n if !barrier.is_leader() {\n // wait for leader to finish tracing\n self.shared.trace_status.trace_end.wait();\n return core;\n }\n\n // trace\n\n let owned = &self.shared.owned;\n let mut local = self.shared.steal_all();\n let synced = &self.shared.synced;\n let injection = &self.shared.inject;\n\n // safety: `trace_multi_thread` is invoked with the same `synced` that `injection`\n // was created with.\n let traces = unsafe { trace_multi_thread(owned, &mut local, synced, injection) }\n .into_iter()\n .map(|(id, trace)| dump::Task::new(id, trace))\n .collect();\n\n let result = dump::Dump::new(traces);\n\n // stash the result\n self.shared.trace_status.stash_result(result);\n\n // allow other workers to proceed\n self.shared.trace_status.trace_end.wait();\n\n core\n }\n}\n\nimpl Shared {\n /// Steal all tasks from remotes into a single local queue.\n pub(super) fn steal_all(&self) -> super::queue::Local> {\n let (_steal, mut local) = super::queue::local();\n\n let worker_metrics = WorkerMetrics::new();\n let mut stats = Stats::new(&worker_metrics);\n\n for remote in self.remotes.iter() {\n let steal = &remote.steal;\n while !steal.is_empty() {\n if let Some(task) = steal.steal_into(&mut local, &mut stats) {\n local.push_back([task].into_iter());\n }\n }\n }\n\n local\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/worker/taskdump_mock.rs", "content": "use super::{Core, Handle};\n\nimpl Handle {\n pub(super) fn trace_core(&self, core: Box) -> Box {\n core\n }\n}\n" }, { "path": "tokio/src/runtime/scheduler/multi_thread/worker.rs", "content": "//! A scheduler is initialized with a fixed number of workers. Each worker is\n//! driven by a thread. Each worker has a \"core\" which contains data such as the\n//! run queue and other state. When `block_in_place` is called, the worker's\n//! \"core\" is handed off to a new thread allowing the scheduler to continue to\n//! make progress while the originating thread blocks.\n//!\n//! # Shutdown\n//!\n//! Shutting down the runtime involves the following steps:\n//!\n//! 1. The Shared::close method is called. This closes the inject queue and\n//! `OwnedTasks` instance and wakes up all worker threads.\n//!\n//! 2. Each worker thread observes the close signal next time it runs\n//! Core::maintenance by checking whether the inject queue is closed.\n//! The `Core::is_shutdown` flag is set to true.\n//!\n//! 3. The worker thread calls `pre_shutdown` in parallel. Here, the worker\n//! will keep removing tasks from `OwnedTasks` until it is empty. No new\n//! tasks can be pushed to the `OwnedTasks` during or after this step as it\n//! was closed in step 1.\n//!\n//! 5. The workers call Shared::shutdown to enter the single-threaded phase of\n//! shutdown. These calls will push their core to `Shared::shutdown_cores`,\n//! and the last thread to push its core will finish the shutdown procedure.\n//!\n//! 6. The local run queue of each core is emptied, then the inject queue is\n//! emptied.\n//!\n//! At this point, shutdown has completed. It is not possible for any of the\n//! collections to contain any tasks at this point, as each collection was\n//! closed first, then emptied afterwards.\n//!\n//! ## Spawns during shutdown\n//!\n//! When spawning tasks during shutdown, there are two cases:\n//!\n//! * The spawner observes the `OwnedTasks` being open, and the inject queue is\n//! closed.\n//! * The spawner observes the `OwnedTasks` being closed and doesn't check the\n//! inject queue.\n//!\n//! The first case can only happen if the `OwnedTasks::bind` call happens before\n//! or during step 1 of shutdown. In this case, the runtime will clean up the\n//! task in step 3 of shutdown.\n//!\n//! In the latter case, the task was not spawned and the task is immediately\n//! cancelled by the spawner.\n//!\n//! The correctness of shutdown requires both the inject queue and `OwnedTasks`\n//! collection to have a closed bit. With a close bit on only the inject queue,\n//! spawning could run in to a situation where a task is successfully bound long\n//! after the runtime has shut down. With a close bit on only the `OwnedTasks`,\n//! the first spawning situation could result in the notification being pushed\n//! to the inject queue after step 6 of shutdown, which would leave a task in\n//! the inject queue indefinitely. This would be a ref-count cycle and a memory\n//! leak.\n\nuse crate::loom::sync::{Arc, Mutex};\nuse crate::runtime;\nuse crate::runtime::scheduler::multi_thread::{\n idle, queue, Counters, Handle, Idle, Overflow, Parker, Stats, TraceStatus, Unparker,\n};\nuse crate::runtime::scheduler::{inject, Defer, Lock};\nuse crate::runtime::task::OwnedTasks;\nuse crate::runtime::{\n blocking, driver, scheduler, task, Config, SchedulerMetrics, TimerFlavor, WorkerMetrics,\n};\nuse crate::runtime::{context, TaskHooks};\nuse crate::task::coop;\nuse crate::util::atomic_cell::AtomicCell;\nuse crate::util::rand::{FastRand, RngSeedGenerator};\n\nuse std::cell::RefCell;\nuse std::task::Waker;\nuse std::thread;\nuse std::time::Duration;\n\nmod metrics;\n\ncfg_taskdump! {\n mod taskdump;\n}\n\ncfg_not_taskdump! {\n mod taskdump_mock;\n}\n\n#[cfg(all(tokio_unstable, feature = \"time\"))]\nuse crate::loom::sync::atomic::AtomicBool;\n\n#[cfg(all(tokio_unstable, feature = \"time\"))]\nuse crate::runtime::time_alt;\n\n#[cfg(all(tokio_unstable, feature = \"time\"))]\nuse crate::runtime::scheduler::util;\n\n/// A scheduler worker\npub(super) struct Worker {\n /// Reference to scheduler's handle\n handle: Arc,\n\n /// Index holding this worker's remote state\n index: usize,\n\n /// Used to hand-off a worker's core to another thread.\n core: AtomicCell,\n}\n\n/// Core data\nstruct Core {\n /// Used to schedule bookkeeping tasks every so often.\n tick: u32,\n\n /// When a task is scheduled from a worker, it is stored in this slot. The\n /// worker will check this slot for a task **before** checking the run\n /// queue. This effectively results in the **last** scheduled task to be run\n /// next (LIFO). This is an optimization for improving locality which\n /// benefits message passing patterns and helps to reduce latency.\n lifo_slot: Option,\n\n /// When `true`, locally scheduled tasks go to the LIFO slot. When `false`,\n /// they go to the back of the `run_queue`.\n lifo_enabled: bool,\n\n /// The worker-local run queue.\n run_queue: queue::Local>,\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n time_context: time_alt::LocalContext,\n\n /// True if the worker is currently searching for more work. Searching\n /// involves attempting to steal from other workers.\n is_searching: bool,\n\n /// True if the scheduler is being shutdown\n is_shutdown: bool,\n\n /// True if the scheduler is being traced\n is_traced: bool,\n\n /// Parker\n ///\n /// Stored in an `Option` as the parker is added / removed to make the\n /// borrow checker happy.\n park: Option,\n\n /// Per-worker runtime stats\n stats: Stats,\n\n /// How often to check the global queue\n global_queue_interval: u32,\n\n /// Fast random number generator.\n rand: FastRand,\n}\n\n/// State shared across all workers\npub(crate) struct Shared {\n /// Per-worker remote state. All other workers have access to this and is\n /// how they communicate between each other.\n remotes: Box<[Remote]>,\n\n /// Global task queue used for:\n /// 1. Submit work to the scheduler while **not** currently on a worker thread.\n /// 2. Submit work to the scheduler when a worker run queue is saturated\n pub(super) inject: inject::Shared>,\n\n /// Coordinates idle workers\n idle: Idle,\n\n /// Collection of all active tasks spawned onto this executor.\n pub(crate) owned: OwnedTasks>,\n\n /// Data synchronized by the scheduler mutex\n pub(super) synced: Mutex,\n\n /// Cores that have observed the shutdown signal\n ///\n /// The core is **not** placed back in the worker to avoid it from being\n /// stolen by a thread that was spawned as part of `block_in_place`.\n #[allow(clippy::vec_box)] // we're moving an already-boxed value\n shutdown_cores: Mutex>>,\n\n /// The number of cores that have observed the trace signal.\n pub(super) trace_status: TraceStatus,\n\n /// Scheduler configuration options\n config: Config,\n\n /// Collects metrics from the runtime.\n pub(super) scheduler_metrics: SchedulerMetrics,\n\n pub(super) worker_metrics: Box<[WorkerMetrics]>,\n\n /// Only held to trigger some code on drop. This is used to get internal\n /// runtime metrics that can be useful when doing performance\n /// investigations. This does nothing (empty struct, no drop impl) unless\n /// the `tokio_internal_mt_counters` `cfg` flag is set.\n _counters: Counters,\n}\n\n/// Data synchronized by the scheduler mutex\npub(crate) struct Synced {\n /// Synchronized state for `Idle`.\n pub(super) idle: idle::Synced,\n\n /// Synchronized state for `Inject`.\n pub(crate) inject: inject::Synced,\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n /// Timers pending to be registered.\n /// This is used to register a timer but the [`Core`]\n /// is not available in the current thread.\n inject_timers: Vec,\n}\n\n/// Used to communicate with a worker from other threads.\nstruct Remote {\n /// Steals tasks from this worker.\n pub(super) steal: queue::Steal>,\n\n /// Unparks the associated worker thread\n unpark: Unparker,\n}\n\n/// Thread-local context\npub(crate) struct Context {\n /// Worker\n worker: Arc,\n\n /// Core data\n core: RefCell>>,\n\n /// Tasks to wake after resource drivers are polled. This is mostly to\n /// handle yielded tasks.\n pub(crate) defer: Defer,\n}\n\n/// Starts the workers\npub(crate) struct Launch(Vec>);\n\n/// Running a task may consume the core. If the core is still available when\n/// running the task completes, it is returned. Otherwise, the worker will need\n/// to stop processing.\ntype RunResult = Result, ()>;\n\n/// A notified task handle\ntype Notified = task::Notified>;\n\n/// Value picked out of thin-air. Running the LIFO slot a handful of times\n/// seems sufficient to benefit from locality. More than 3 times probably is\n/// over-weighting. The value can be tuned in the future with data that shows\n/// improvements.\nconst MAX_LIFO_POLLS_PER_TICK: usize = 3;\n\npub(super) fn create(\n size: usize,\n park: Parker,\n driver_handle: driver::Handle,\n blocking_spawner: blocking::Spawner,\n seed_generator: RngSeedGenerator,\n config: Config,\n timer_flavor: TimerFlavor,\n) -> (Arc, Launch) {\n let mut cores = Vec::with_capacity(size);\n let mut remotes = Vec::with_capacity(size);\n let mut worker_metrics = Vec::with_capacity(size);\n\n // Create the local queues\n for _ in 0..size {\n let (steal, run_queue) = queue::local();\n\n let park = park.clone();\n let unpark = park.unpark();\n let metrics = WorkerMetrics::from_config(&config);\n let stats = Stats::new(&metrics);\n\n cores.push(Box::new(Core {\n tick: 0,\n lifo_slot: None,\n lifo_enabled: !config.disable_lifo_slot,\n run_queue,\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n time_context: time_alt::LocalContext::new(),\n is_searching: false,\n is_shutdown: false,\n is_traced: false,\n park: Some(park),\n global_queue_interval: stats.tuned_global_queue_interval(&config),\n stats,\n rand: FastRand::from_seed(config.seed_generator.next_seed()),\n }));\n\n remotes.push(Remote { steal, unpark });\n worker_metrics.push(metrics);\n }\n\n let (idle, idle_synced) = Idle::new(size);\n let (inject, inject_synced) = inject::Shared::new();\n\n let remotes_len = remotes.len();\n let handle = Arc::new(Handle {\n task_hooks: TaskHooks::from_config(&config),\n shared: Shared {\n remotes: remotes.into_boxed_slice(),\n inject,\n idle,\n owned: OwnedTasks::new(size),\n synced: Mutex::new(Synced {\n idle: idle_synced,\n inject: inject_synced,\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n inject_timers: Vec::new(),\n }),\n shutdown_cores: Mutex::new(vec![]),\n trace_status: TraceStatus::new(remotes_len),\n config,\n scheduler_metrics: SchedulerMetrics::new(),\n worker_metrics: worker_metrics.into_boxed_slice(),\n _counters: Counters,\n },\n driver: driver_handle,\n blocking_spawner,\n seed_generator,\n timer_flavor,\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n is_shutdown: AtomicBool::new(false),\n });\n\n let mut launch = Launch(vec![]);\n\n for (index, core) in cores.drain(..).enumerate() {\n launch.0.push(Arc::new(Worker {\n handle: handle.clone(),\n index,\n core: AtomicCell::new(Some(core)),\n }));\n }\n\n (handle, launch)\n}\n\n#[track_caller]\npub(crate) fn block_in_place(f: F) -> R\nwhere\n F: FnOnce() -> R,\n{\n // Try to steal the worker core back\n struct Reset {\n take_core: bool,\n budget: coop::Budget,\n }\n\n impl Drop for Reset {\n fn drop(&mut self) {\n with_current(|maybe_cx| {\n if let Some(cx) = maybe_cx {\n if self.take_core {\n let core = cx.worker.core.take();\n\n if core.is_some() {\n cx.worker.handle.shared.worker_metrics[cx.worker.index]\n .set_thread_id(thread::current().id());\n }\n\n let mut cx_core = cx.core.borrow_mut();\n assert!(cx_core.is_none());\n *cx_core = core;\n }\n\n // Reset the task budget as we are re-entering the\n // runtime.\n coop::set(self.budget);\n }\n });\n }\n }\n\n let mut had_entered = false;\n let mut take_core = false;\n\n let setup_result = with_current(|maybe_cx| {\n match (\n crate::runtime::context::current_enter_context(),\n maybe_cx.is_some(),\n ) {\n (context::EnterRuntime::Entered { .. }, true) => {\n // We are on a thread pool runtime thread, so we just need to\n // set up blocking.\n had_entered = true;\n }\n (\n context::EnterRuntime::Entered {\n allow_block_in_place,\n },\n false,\n ) => {\n // We are on an executor, but _not_ on the thread pool. That is\n // _only_ okay if we are in a thread pool runtime's block_on\n // method:\n if allow_block_in_place {\n had_entered = true;\n return Ok(());\n } else {\n // This probably means we are on the current_thread runtime or in a\n // LocalSet, where it is _not_ okay to block.\n return Err(\n \"can call blocking only when running on the multi-threaded runtime\",\n );\n }\n }\n (context::EnterRuntime::NotEntered, true) => {\n // This is a nested call to block_in_place (we already exited).\n // All the necessary setup has already been done.\n return Ok(());\n }\n (context::EnterRuntime::NotEntered, false) => {\n // We are outside of the tokio runtime, so blocking is fine.\n // We can also skip all of the thread pool blocking setup steps.\n return Ok(());\n }\n }\n\n let cx = maybe_cx.expect(\"no .is_some() == false cases above should lead here\");\n\n // Since deferred tasks don't stay on `core`, make sure to wake them\n // before blocking.\n cx.defer.wake();\n\n // Get the worker core. If none is set, then blocking is fine!\n let mut core = match cx.core.borrow_mut().take() {\n Some(core) => core,\n None => return Ok(()),\n };\n\n // If we heavily call `spawn_blocking`, there might be no available thread to\n // run this core. Except for the task in the lifo_slot, all tasks can be\n // stolen, so we move the task out of the lifo_slot to the run_queue.\n if let Some(task) = core.lifo_slot.take() {\n core.run_queue\n .push_back_or_overflow(task, &*cx.worker.handle, &mut core.stats);\n }\n\n // We are taking the core from the context and sending it to another\n // thread.\n take_core = true;\n\n // The parker should be set here\n assert!(core.park.is_some());\n\n // In order to block, the core must be sent to another thread for\n // execution.\n //\n // First, move the core back into the worker's shared core slot.\n cx.worker.core.set(core);\n\n // Next, clone the worker handle and send it to a new thread for\n // processing.\n //\n // Once the blocking task is done executing, we will attempt to\n // steal the core back.\n let worker = cx.worker.clone();\n runtime::spawn_blocking(move || run(worker));\n Ok(())\n });\n\n if let Err(panic_message) = setup_result {\n panic!(\"{}\", panic_message);\n }\n\n if had_entered {\n // Unset the current task's budget. Blocking sections are not\n // constrained by task budgets.\n let _reset = Reset {\n take_core,\n budget: coop::stop(),\n };\n\n crate::runtime::context::exit_runtime(f)\n } else {\n f()\n }\n}\n\nimpl Launch {\n pub(crate) fn launch(mut self) {\n for worker in self.0.drain(..) {\n runtime::spawn_blocking(move || run(worker));\n }\n }\n}\n\nfn run(worker: Arc) {\n #[allow(dead_code)]\n struct AbortOnPanic;\n\n impl Drop for AbortOnPanic {\n fn drop(&mut self) {\n if std::thread::panicking() {\n eprintln!(\"worker thread panicking; aborting process\");\n std::process::abort();\n }\n }\n }\n\n // Catching panics on worker threads in tests is quite tricky. Instead, when\n // debug assertions are enabled, we just abort the process.\n #[cfg(debug_assertions)]\n let _abort_on_panic = AbortOnPanic;\n\n // Acquire a core. If this fails, then another thread is running this\n // worker and there is nothing further to do.\n let core = match worker.core.take() {\n Some(core) => core,\n None => return,\n };\n\n worker.handle.shared.worker_metrics[worker.index].set_thread_id(thread::current().id());\n\n let handle = scheduler::Handle::MultiThread(worker.handle.clone());\n\n crate::runtime::context::enter_runtime(&handle, true, |_| {\n // Set the worker context.\n let cx = scheduler::Context::MultiThread(Context {\n worker,\n core: RefCell::new(None),\n defer: Defer::new(),\n });\n\n context::set_scheduler(&cx, || {\n let cx = cx.expect_multi_thread();\n\n // This should always be an error. It only returns a `Result` to support\n // using `?` to short circuit.\n assert!(cx.run(core).is_err());\n\n // Check if there are any deferred tasks to notify. This can happen when\n // the worker core is lost due to `block_in_place()` being called from\n // within the task.\n cx.defer.wake();\n });\n });\n}\n\nimpl Context {\n fn run(&self, mut core: Box) -> RunResult {\n // Reset `lifo_enabled` here in case the core was previously stolen from\n // a task that had the LIFO slot disabled.\n self.reset_lifo_enabled(&mut core);\n\n // Start as \"processing\" tasks as polling tasks from the local queue\n // will be one of the first things we do.\n core.stats.start_processing_scheduled_tasks();\n\n while !core.is_shutdown {\n self.assert_lifo_enabled_is_correct(&core);\n\n if core.is_traced {\n core = self.worker.handle.trace_core(core);\n }\n\n // Increment the tick\n core.tick();\n\n // Run maintenance, if needed\n core = self.maintenance(core);\n\n // First, check work available to the current worker.\n if let Some(task) = core.next_task(&self.worker) {\n core = self.run_task(task, core)?;\n continue;\n }\n\n // We consumed all work in the queues and will start searching for work.\n core.stats.end_processing_scheduled_tasks();\n\n // There is no more **local** work to process, try to steal work\n // from other workers.\n if let Some(task) = core.steal_work(&self.worker) {\n // Found work, switch back to processing\n core.stats.start_processing_scheduled_tasks();\n core = self.run_task(task, core)?;\n } else {\n // Wait for work\n core = if !self.defer.is_empty() {\n self.park_yield(core)\n } else {\n self.park(core)\n };\n core.stats.start_processing_scheduled_tasks();\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n {\n match self.worker.handle.timer_flavor {\n TimerFlavor::Traditional => {}\n TimerFlavor::Alternative => {\n util::time_alt::shutdown_local_timers(\n &mut core.time_context.wheel,\n &mut core.time_context.canc_rx,\n self.worker.handle.take_remote_timers(),\n &self.worker.handle.driver,\n );\n }\n }\n }\n\n core.pre_shutdown(&self.worker);\n // Signal shutdown\n self.worker.handle.shutdown_core(core);\n Err(())\n }\n\n fn run_task(&self, task: Notified, mut core: Box) -> RunResult {\n #[cfg(tokio_unstable)]\n let task_meta = task.task_meta();\n\n let task = self.worker.handle.shared.owned.assert_owner(task);\n\n // Make sure the worker is not in the **searching** state. This enables\n // another idle worker to try to steal work.\n core.transition_from_searching(&self.worker);\n\n self.assert_lifo_enabled_is_correct(&core);\n\n // Measure the poll start time. Note that we may end up polling other\n // tasks under this measurement. In this case, the tasks came from the\n // LIFO slot and are considered part of the current task for scheduling\n // purposes. These tasks inherent the \"parent\"'s limits.\n core.stats.start_poll();\n\n // Make the core available to the runtime context\n *self.core.borrow_mut() = Some(core);\n\n // Run the task\n coop::budget(|| {\n // Unlike the poll time above, poll start callback is attached to the task id,\n // so it is tightly associated with the actual poll invocation.\n #[cfg(tokio_unstable)]\n self.worker\n .handle\n .task_hooks\n .poll_start_callback(&task_meta);\n\n task.run();\n\n #[cfg(tokio_unstable)]\n self.worker.handle.task_hooks.poll_stop_callback(&task_meta);\n\n let mut lifo_polls = 0;\n\n // As long as there is budget remaining and a task exists in the\n // `lifo_slot`, then keep running.\n loop {\n // Check if we still have the core. If not, the core was stolen\n // by another worker.\n let mut core = match self.core.borrow_mut().take() {\n Some(core) => core,\n None => {\n // In this case, we cannot call `reset_lifo_enabled()`\n // because the core was stolen. The stealer will handle\n // that at the top of `Context::run`\n return Err(());\n }\n };\n\n // Check for a task in the LIFO slot\n let task = match core.lifo_slot.take() {\n Some(task) => task,\n None => {\n self.reset_lifo_enabled(&mut core);\n core.stats.end_poll();\n return Ok(core);\n }\n };\n\n if !coop::has_budget_remaining() {\n core.stats.end_poll();\n\n // Not enough budget left to run the LIFO task, push it to\n // the back of the queue and return.\n core.run_queue.push_back_or_overflow(\n task,\n &*self.worker.handle,\n &mut core.stats,\n );\n // If we hit this point, the LIFO slot should be enabled.\n // There is no need to reset it.\n debug_assert!(core.lifo_enabled);\n return Ok(core);\n }\n\n // Track that we are about to run a task from the LIFO slot.\n lifo_polls += 1;\n super::counters::inc_lifo_schedules();\n\n // Disable the LIFO slot if we reach our limit\n //\n // In ping-ping style workloads where task A notifies task B,\n // which notifies task A again, continuously prioritizing the\n // LIFO slot can cause starvation as these two tasks will\n // repeatedly schedule the other. To mitigate this, we limit the\n // number of times the LIFO slot is prioritized.\n if lifo_polls >= MAX_LIFO_POLLS_PER_TICK {\n core.lifo_enabled = false;\n super::counters::inc_lifo_capped();\n }\n\n // Run the LIFO task, then loop\n *self.core.borrow_mut() = Some(core);\n let task = self.worker.handle.shared.owned.assert_owner(task);\n\n #[cfg(tokio_unstable)]\n let task_meta = task.task_meta();\n\n #[cfg(tokio_unstable)]\n self.worker\n .handle\n .task_hooks\n .poll_start_callback(&task_meta);\n\n task.run();\n\n #[cfg(tokio_unstable)]\n self.worker.handle.task_hooks.poll_stop_callback(&task_meta);\n }\n })\n }\n\n fn reset_lifo_enabled(&self, core: &mut Core) {\n core.lifo_enabled = !self.worker.handle.shared.config.disable_lifo_slot;\n }\n\n fn assert_lifo_enabled_is_correct(&self, core: &Core) {\n debug_assert_eq!(\n core.lifo_enabled,\n !self.worker.handle.shared.config.disable_lifo_slot\n );\n }\n\n fn maintenance(&self, mut core: Box) -> Box {\n if core.tick % self.worker.handle.shared.config.event_interval == 0 {\n super::counters::inc_num_maintenance();\n\n core.stats.end_processing_scheduled_tasks();\n\n // Call `park` with a 0 timeout. This enables the I/O driver, timer, ...\n // to run without actually putting the thread to sleep.\n core = self.park_yield(core);\n\n // Run regularly scheduled maintenance\n core.maintenance(&self.worker);\n\n core.stats.start_processing_scheduled_tasks();\n }\n\n core\n }\n\n /// Parks the worker thread while waiting for tasks to execute.\n ///\n /// This function checks if indeed there's no more work left to be done before parking.\n /// Also important to notice that, before parking, the worker thread will try to take\n /// ownership of the Driver (IO/Time) and dispatch any events that might have fired.\n /// Whenever a worker thread executes the Driver loop, all waken tasks are scheduled\n /// in its own local queue until the queue saturates (ntasks > `LOCAL_QUEUE_CAPACITY`).\n /// When the local queue is saturated, the overflow tasks are added to the injection queue\n /// from where other workers can pick them up.\n /// Also, we rely on the workstealing algorithm to spread the tasks amongst workers\n /// after all the IOs get dispatched\n fn park(&self, mut core: Box) -> Box {\n if let Some(f) = &self.worker.handle.shared.config.before_park {\n f();\n }\n\n if core.transition_to_parked(&self.worker) {\n while !core.is_shutdown && !core.is_traced {\n core.stats.about_to_park();\n core.stats\n .submit(&self.worker.handle.shared.worker_metrics[self.worker.index]);\n\n core = self.park_internal(core, None);\n\n core.stats.unparked();\n\n // Run regularly scheduled maintenance\n core.maintenance(&self.worker);\n\n if core.transition_from_parked(&self.worker) {\n break;\n }\n }\n }\n\n if let Some(f) = &self.worker.handle.shared.config.after_unpark {\n f();\n }\n core\n }\n\n fn park_yield(&self, core: Box) -> Box {\n self.park_internal(core, Some(Duration::from_millis(0)))\n }\n\n fn park_internal(&self, mut core: Box, duration: Option) -> Box {\n self.assert_lifo_enabled_is_correct(&core);\n\n // Take the parker out of core\n let mut park = core.park.take().expect(\"park missing\");\n // Store `core` in context\n *self.core.borrow_mut() = Some(core);\n\n #[cfg(feature = \"time\")]\n let (duration, auto_advance_duration) = match self.worker.handle.timer_flavor {\n TimerFlavor::Traditional => (duration, None::),\n #[cfg(tokio_unstable)]\n TimerFlavor::Alternative => {\n // Must happens after taking out the parker, as the `Handle::schedule_local`\n // will delay the notify if the parker taken out.\n //\n // See comments in `Handle::schedule_local` for more details.\n let MaintainLocalTimer {\n park_duration: duration,\n auto_advance_duration,\n } = self.maintain_local_timers_before_parking(duration);\n (duration, auto_advance_duration)\n }\n };\n\n // Park thread\n if let Some(timeout) = duration {\n park.park_timeout(&self.worker.handle.driver, timeout);\n } else {\n park.park(&self.worker.handle.driver);\n }\n\n self.defer.wake();\n\n #[cfg(feature = \"time\")]\n match self.worker.handle.timer_flavor {\n TimerFlavor::Traditional => {\n // suppress unused variable warning\n let _ = auto_advance_duration;\n }\n #[cfg(tokio_unstable)]\n TimerFlavor::Alternative => {\n // Must happens before placing back the parker, as the `Handle::schedule_local`\n // will delay the notify if the parker is still in `core`.\n //\n // See comments in `Handle::schedule_local` for more details.\n self.maintain_local_timers_after_parking(auto_advance_duration);\n }\n }\n\n // Remove `core` from context\n core = self.core.borrow_mut().take().expect(\"core missing\");\n\n // Place `park` back in `core`\n core.park = Some(park);\n if core.should_notify_others() {\n self.worker.handle.notify_parked_local();\n }\n core\n }\n\n pub(crate) fn defer(&self, waker: &Waker) {\n if self.core.borrow().is_none() {\n // If there is no core, then the worker is currently in a block_in_place. In this case,\n // we cannot use the defer queue as we aren't really in the current runtime.\n waker.wake_by_ref();\n } else {\n self.defer.defer(waker);\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n /// Maintain local timers before parking the resource driver.\n ///\n /// * Remove cancelled timers from the local timer wheel.\n /// * Register remote timers to the local timer wheel.\n /// * Adjust the park duration based on\n /// * the next timer expiration time.\n /// * whether auto-advancing is required (feature = \"test-util\").\n ///\n /// # Returns\n ///\n /// `(Box, park_duration, auto_advance_duration)`\n fn maintain_local_timers_before_parking(\n &self,\n park_duration: Option,\n ) -> MaintainLocalTimer {\n let handle = &self.worker.handle;\n let mut wake_queue = time_alt::WakeQueue::new();\n\n let (should_yield, next_timer) = with_current(|maybe_cx| {\n let cx = maybe_cx.expect(\"function should be called when core is present\");\n assert_eq!(\n Arc::as_ptr(&cx.worker.handle),\n Arc::as_ptr(&self.worker.handle),\n \"function should be called on the exact same worker\"\n );\n\n let mut maybe_core = cx.core.borrow_mut();\n let core = maybe_core.as_mut().expect(\"core missing\");\n let time_cx = &mut core.time_context;\n\n util::time_alt::process_registration_queue(\n &mut time_cx.registration_queue,\n &mut time_cx.wheel,\n &time_cx.canc_tx,\n &mut wake_queue,\n );\n util::time_alt::insert_inject_timers(\n &mut time_cx.wheel,\n &time_cx.canc_tx,\n handle.take_remote_timers(),\n &mut wake_queue,\n );\n util::time_alt::remove_cancelled_timers(&mut time_cx.wheel, &mut time_cx.canc_rx);\n let should_yield = !wake_queue.is_empty();\n\n let next_timer = util::time_alt::next_expiration_time(&time_cx.wheel, &handle.driver);\n\n (should_yield, next_timer)\n });\n\n wake_queue.wake_all();\n\n if should_yield {\n MaintainLocalTimer {\n park_duration: Some(Duration::from_millis(0)),\n auto_advance_duration: None,\n }\n } else {\n // get the minimum duration\n let dur = util::time_alt::min_duration(park_duration, next_timer);\n if util::time_alt::pre_auto_advance(&handle.driver, dur) {\n MaintainLocalTimer {\n park_duration: Some(Duration::ZERO),\n auto_advance_duration: dur,\n }\n } else {\n MaintainLocalTimer {\n park_duration: dur,\n auto_advance_duration: None,\n }\n }\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n /// Maintain local timers after unparking the resource driver.\n ///\n /// * Auto-advance time, if required (feature = \"test-util\").\n /// * Process expired timers.\n fn maintain_local_timers_after_parking(&self, auto_advance_duration: Option) {\n let handle = &self.worker.handle;\n let mut wake_queue = time_alt::WakeQueue::new();\n\n with_current(|maybe_cx| {\n let cx = maybe_cx.expect(\"function should be called when core is present\");\n assert_eq!(\n Arc::as_ptr(&cx.worker.handle),\n Arc::as_ptr(&self.worker.handle),\n \"function should be called on the exact same worker\"\n );\n\n let mut maybe_core = cx.core.borrow_mut();\n let core = maybe_core.as_mut().expect(\"core missing\");\n let time_cx = &mut core.time_context;\n\n util::time_alt::post_auto_advance(&handle.driver, auto_advance_duration);\n util::time_alt::process_expired_timers(\n &mut time_cx.wheel,\n &handle.driver,\n &mut wake_queue,\n );\n });\n\n wake_queue.wake_all();\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n fn with_core(&self, f: F) -> R\n where\n F: FnOnce(Option<&mut Core>) -> R,\n {\n match self.core.borrow_mut().as_mut() {\n Some(core) => f(Some(core)),\n None => f(None),\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n pub(crate) fn with_time_temp_local_context(&self, f: F) -> R\n where\n F: FnOnce(Option>) -> R,\n {\n self.with_core(|maybe_core| match maybe_core {\n Some(core) if core.is_shutdown => f(Some(time_alt::TempLocalContext::new_shutdown())),\n Some(core) => f(Some(time_alt::TempLocalContext::new_running(\n &mut core.time_context,\n ))),\n None => f(None),\n })\n }\n\n #[cfg(tokio_unstable)]\n pub(crate) fn worker_index(&self) -> usize {\n self.worker.index\n }\n}\n\nimpl Core {\n /// Increment the tick\n fn tick(&mut self) {\n self.tick = self.tick.wrapping_add(1);\n }\n\n /// Return the next notified task available to this worker.\n fn next_task(&mut self, worker: &Worker) -> Option {\n if self.tick % self.global_queue_interval == 0 {\n // Update the global queue interval, if needed\n self.tune_global_queue_interval(worker);\n\n worker\n .handle\n .next_remote_task()\n .or_else(|| self.next_local_task())\n } else {\n let maybe_task = self.next_local_task();\n\n if maybe_task.is_some() {\n return maybe_task;\n }\n\n if worker.inject().is_empty() {\n return None;\n }\n\n // Other threads can only **remove** tasks from the current worker's\n // `run_queue`. So, we can be confident that by the time we call\n // `run_queue.push_back` below, there will be *at least* `cap`\n // available slots in the queue.\n let cap = usize::min(\n self.run_queue.remaining_slots(),\n self.run_queue.max_capacity() / 2,\n );\n\n // The worker is currently idle, pull a batch of work from the\n // injection queue. We don't want to pull *all* the work so other\n // workers can also get some.\n let n = usize::min(\n worker.inject().len() / worker.handle.shared.remotes.len() + 1,\n cap,\n );\n\n // Take at least one task since the first task is returned directly\n // and not pushed onto the local queue.\n let n = usize::max(1, n);\n\n let mut synced = worker.handle.shared.synced.lock();\n // safety: passing in the correct `inject::Synced`.\n let mut tasks = unsafe { worker.inject().pop_n(&mut synced.inject, n) };\n\n // Pop the first task to return immediately\n let ret = tasks.next();\n\n // Push the rest of the on the run queue\n self.run_queue.push_back(tasks);\n\n ret\n }\n }\n\n fn next_local_task(&mut self) -> Option {\n self.lifo_slot.take().or_else(|| self.run_queue.pop())\n }\n\n /// Function responsible for stealing tasks from another worker\n ///\n /// Note: Only if less than half the workers are searching for tasks to steal\n /// a new worker will actually try to steal. The idea is to make sure not all\n /// workers will be trying to steal at the same time.\n fn steal_work(&mut self, worker: &Worker) -> Option {\n if !self.transition_to_searching(worker) {\n return None;\n }\n\n let num = worker.handle.shared.remotes.len();\n // Start from a random worker\n let start = self.rand.fastrand_n(num as u32) as usize;\n\n for i in 0..num {\n let i = (start + i) % num;\n\n // Don't steal from ourself! We know we don't have work.\n if i == worker.index {\n continue;\n }\n\n let target = &worker.handle.shared.remotes[i];\n if let Some(task) = target\n .steal\n .steal_into(&mut self.run_queue, &mut self.stats)\n {\n return Some(task);\n }\n }\n\n // Fallback on checking the global queue\n worker.handle.next_remote_task()\n }\n\n fn transition_to_searching(&mut self, worker: &Worker) -> bool {\n if !self.is_searching {\n self.is_searching = worker.handle.shared.idle.transition_worker_to_searching();\n }\n\n self.is_searching\n }\n\n fn transition_from_searching(&mut self, worker: &Worker) {\n if !self.is_searching {\n return;\n }\n\n self.is_searching = false;\n worker.handle.transition_worker_from_searching();\n }\n\n fn has_tasks(&self) -> bool {\n self.lifo_slot.is_some() || self.run_queue.has_tasks()\n }\n\n fn should_notify_others(&self) -> bool {\n // If there are tasks available to steal, but this worker is not\n // looking for tasks to steal, notify another worker.\n if self.is_searching {\n return false;\n }\n self.lifo_slot.is_some() as usize + self.run_queue.len() > 1\n }\n\n /// Prepares the worker state for parking.\n ///\n /// Returns true if the transition happened, false if there is work to do first.\n fn transition_to_parked(&mut self, worker: &Worker) -> bool {\n // Workers should not park if they have work to do\n if self.has_tasks() || self.is_traced {\n return false;\n }\n\n // When the final worker transitions **out** of searching to parked, it\n // must check all the queues one last time in case work materialized\n // between the last work scan and transitioning out of searching.\n let is_last_searcher = worker.handle.shared.idle.transition_worker_to_parked(\n &worker.handle.shared,\n worker.index,\n self.is_searching,\n );\n\n // The worker is no longer searching. Setting this is the local cache\n // only.\n self.is_searching = false;\n\n if is_last_searcher {\n worker.handle.notify_if_work_pending();\n }\n\n true\n }\n\n /// Returns `true` if the transition happened.\n fn transition_from_parked(&mut self, worker: &Worker) -> bool {\n // If a task is in the lifo slot/run queue, then we must unpark regardless of\n // being notified\n if self.has_tasks() {\n // When a worker wakes, it should only transition to the \"searching\"\n // state when the wake originates from another worker *or* a new task\n // is pushed. We do *not* want the worker to transition to \"searching\"\n // when it wakes when the I/O driver receives new events.\n self.is_searching = !worker\n .handle\n .shared\n .idle\n .unpark_worker_by_id(&worker.handle.shared, worker.index);\n return true;\n }\n\n if worker\n .handle\n .shared\n .idle\n .is_parked(&worker.handle.shared, worker.index)\n {\n return false;\n }\n\n // When unparked, the worker is in the searching state.\n self.is_searching = true;\n true\n }\n\n /// Runs maintenance work such as checking the pool's state.\n fn maintenance(&mut self, worker: &Worker) {\n self.stats\n .submit(&worker.handle.shared.worker_metrics[worker.index]);\n\n if !self.is_shutdown {\n // Check if the scheduler has been shutdown\n let synced = worker.handle.shared.synced.lock();\n self.is_shutdown = worker.inject().is_closed(&synced.inject);\n }\n\n if !self.is_traced {\n // Check if the worker should be tracing.\n self.is_traced = worker.handle.shared.trace_status.trace_requested();\n }\n }\n\n /// Signals all tasks to shut down, and waits for them to complete. Must run\n /// before we enter the single-threaded phase of shutdown processing.\n fn pre_shutdown(&mut self, worker: &Worker) {\n // Start from a random inner list\n let start = self\n .rand\n .fastrand_n(worker.handle.shared.owned.get_shard_size() as u32);\n // Signal to all tasks to shut down.\n worker\n .handle\n .shared\n .owned\n .close_and_shutdown_all(start as usize);\n\n self.stats\n .submit(&worker.handle.shared.worker_metrics[worker.index]);\n }\n\n /// Shuts down the core.\n fn shutdown(&mut self, handle: &Handle) {\n // Take the core\n let mut park = self.park.take().expect(\"park missing\");\n\n // Drain the queue\n while self.next_local_task().is_some() {}\n\n park.shutdown(&handle.driver);\n }\n\n fn tune_global_queue_interval(&mut self, worker: &Worker) {\n let next = self\n .stats\n .tuned_global_queue_interval(&worker.handle.shared.config);\n\n // Smooth out jitter\n if u32::abs_diff(self.global_queue_interval, next) > 2 {\n self.global_queue_interval = next;\n }\n }\n}\n\nimpl Worker {\n /// Returns a reference to the scheduler's injection queue.\n fn inject(&self) -> &inject::Shared> {\n &self.handle.shared.inject\n }\n}\n\nimpl Handle {\n pub(super) fn schedule_task(&self, task: Notified, is_yield: bool) {\n with_current(|maybe_cx| {\n if let Some(cx) = maybe_cx {\n // Make sure the task is part of the **current** scheduler.\n if self.ptr_eq(&cx.worker.handle) {\n // And the current thread still holds a core\n if let Some(core) = cx.core.borrow_mut().as_mut() {\n self.schedule_local(core, task, is_yield);\n return;\n }\n }\n }\n\n // Otherwise, use the inject queue.\n self.push_remote_task(task);\n self.notify_parked_remote();\n });\n }\n\n // Separated case to reduce LLVM codegen in `Handle::bind_new_task`.\n pub(super) fn schedule_option_task_without_yield(&self, task: Option) {\n if let Some(task) = task {\n self.schedule_task(task, false);\n }\n }\n\n fn schedule_local(&self, core: &mut Core, task: Notified, is_yield: bool) {\n core.stats.inc_local_schedule_count();\n\n // Spawning from the worker thread. If scheduling a \"yield\" then the\n // task must always be pushed to the back of the queue, enabling other\n // tasks to be executed. If **not** a yield, then there is more\n // flexibility and the task may go to the front of the queue.\n let should_notify = if is_yield || !core.lifo_enabled {\n core.run_queue\n .push_back_or_overflow(task, self, &mut core.stats);\n true\n } else {\n // Push to the LIFO slot\n let prev = core.lifo_slot.take();\n let ret = prev.is_some();\n\n if let Some(prev) = prev {\n core.run_queue\n .push_back_or_overflow(prev, self, &mut core.stats);\n }\n\n core.lifo_slot = Some(task);\n\n ret\n };\n\n // Only notify if not currently parked. If `park` is `None`, then the\n // scheduling is from a resource driver. As notifications often come in\n // batches, the notification is delayed until the park is complete.\n if should_notify && core.park.is_some() {\n self.notify_parked_local();\n }\n }\n\n fn next_remote_task(&self) -> Option {\n if self.shared.inject.is_empty() {\n return None;\n }\n\n let mut synced = self.shared.synced.lock();\n // safety: passing in correct `idle::Synced`\n unsafe { self.shared.inject.pop(&mut synced.inject) }\n }\n\n fn push_remote_task(&self, task: Notified) {\n self.shared.scheduler_metrics.inc_remote_schedule_count();\n\n let mut synced = self.shared.synced.lock();\n // safety: passing in correct `idle::Synced`\n unsafe {\n self.shared.inject.push(&mut synced.inject, task);\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n pub(crate) fn push_remote_timer(&self, hdl: time_alt::EntryHandle) {\n assert_eq!(self.timer_flavor, TimerFlavor::Alternative);\n {\n let mut synced = self.shared.synced.lock();\n synced.inject_timers.push(hdl);\n }\n self.notify_parked_remote();\n }\n\n #[cfg(all(tokio_unstable, feature = \"time\"))]\n pub(crate) fn take_remote_timers(&self) -> Vec {\n assert_eq!(self.timer_flavor, TimerFlavor::Alternative);\n // It's ok to lost the race, as another worker is\n // draining the inject_timers.\n match self.shared.synced.try_lock() {\n Some(mut synced) => std::mem::take(&mut synced.inject_timers),\n None => Vec::new(),\n }\n }\n\n pub(super) fn close(&self) {\n if self\n .shared\n .inject\n .close(&mut self.shared.synced.lock().inject)\n {\n self.notify_all();\n }\n }\n\n fn notify_parked_local(&self) {\n super::counters::inc_num_inc_notify_local();\n\n if let Some(index) = self.shared.idle.worker_to_notify(&self.shared) {\n super::counters::inc_num_unparks_local();\n self.shared.remotes[index].unpark.unpark(&self.driver);\n }\n }\n\n fn notify_parked_remote(&self) {\n if let Some(index) = self.shared.idle.worker_to_notify(&self.shared) {\n self.shared.remotes[index].unpark.unpark(&self.driver);\n }\n }\n\n pub(super) fn notify_all(&self) {\n for remote in &self.shared.remotes[..] {\n remote.unpark.unpark(&self.driver);\n }\n }\n\n fn notify_if_work_pending(&self) {\n for remote in &self.shared.remotes[..] {\n if !remote.steal.is_empty() {\n self.notify_parked_local();\n return;\n }\n }\n\n if !self.shared.inject.is_empty() {\n self.notify_parked_local();\n }\n }\n\n fn transition_worker_from_searching(&self) {\n if self.shared.idle.transition_worker_from_searching() {\n // We are the final searching worker. Because work was found, we\n // need to notify another worker.\n self.notify_parked_local();\n }\n }\n\n /// Signals that a worker has observed the shutdown signal and has replaced\n /// its core back into its handle.\n ///\n /// If all workers have reached this point, the final cleanup is performed.\n fn shutdown_core(&self, core: Box) {\n let mut cores = self.shared.shutdown_cores.lock();\n cores.push(core);\n\n if cores.len() != self.shared.remotes.len() {\n return;\n }\n\n debug_assert!(self.shared.owned.is_empty());\n\n for mut core in cores.drain(..) {\n core.shutdown(self);\n }\n\n // Drain the injection queue\n //\n // We already shut down every task, so we can simply drop the tasks.\n while let Some(task) = self.next_remote_task() {\n drop(task);\n }\n }\n\n fn ptr_eq(&self, other: &Handle) -> bool {\n std::ptr::eq(self, other)\n }\n}\n\nimpl Overflow> for Handle {\n fn push(&self, task: task::Notified>) {\n self.push_remote_task(task);\n }\n\n fn push_batch(&self, iter: I)\n where\n I: Iterator>>,\n {\n unsafe {\n self.shared.inject.push_batch(self, iter);\n }\n }\n}\n\npub(crate) struct InjectGuard<'a> {\n lock: crate::loom::sync::MutexGuard<'a, Synced>,\n}\n\nimpl<'a> AsMut for InjectGuard<'a> {\n fn as_mut(&mut self) -> &mut inject::Synced {\n &mut self.lock.inject\n }\n}\n\nimpl<'a> Lock for &'a Handle {\n type Handle = InjectGuard<'a>;\n\n fn lock(self) -> Self::Handle {\n InjectGuard {\n lock: self.shared.synced.lock(),\n }\n }\n}\n\n#[cfg(all(tokio_unstable, feature = \"time\"))]\n/// Returned by [`Context::maintain_local_timers_before_parking`].\nstruct MaintainLocalTimer {\n park_duration: Option,\n auto_advance_duration: Option,\n}\n\n#[track_caller]\nfn with_current(f: impl FnOnce(Option<&Context>) -> R) -> R {\n use scheduler::Context::MultiThread;\n\n context::with_scheduler(|ctx| match ctx {\n Some(MultiThread(ctx)) => f(Some(ctx)),\n _ => f(None),\n })\n}\n" }, { "path": "tokio/src/runtime/scheduler/util/mod.rs", "content": "#[cfg(all(tokio_unstable, feature = \"time\", feature = \"rt-multi-thread\"))]\npub(in crate::runtime) mod time_alt;\n" }, { "path": "tokio/src/runtime/scheduler/util/time_alt.rs", "content": "use crate::runtime::scheduler::driver;\nuse crate::runtime::time_alt::cancellation_queue::{Receiver, Sender};\nuse crate::runtime::time_alt::{EntryHandle, RegistrationQueue, WakeQueue, Wheel};\nuse std::time::Duration;\n\npub(crate) fn min_duration(a: Option, b: Option) -> Option {\n match (a, b) {\n (Some(dur_a), Some(dur_b)) => Some(std::cmp::min(dur_a, dur_b)),\n (Some(dur_a), None) => Some(dur_a),\n (None, Some(dur_b)) => Some(dur_b),\n (None, None) => None,\n }\n}\n\npub(crate) fn process_registration_queue(\n registration_queue: &mut RegistrationQueue,\n wheel: &mut Wheel,\n tx: &Sender,\n wake_queue: &mut WakeQueue,\n) {\n while let Some(hdl) = registration_queue.pop_front() {\n if hdl.deadline() <= wheel.elapsed() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n } else {\n // Safety: the entry is not registered yet\n unsafe {\n wheel.insert(hdl, tx.clone());\n }\n }\n }\n}\n\npub(crate) fn insert_inject_timers(\n wheel: &mut Wheel,\n tx: &Sender,\n inject: Vec,\n wake_queue: &mut WakeQueue,\n) {\n for hdl in inject {\n if hdl.deadline() <= wheel.elapsed() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n } else {\n // Safety: the entry is not registered yet\n unsafe {\n wheel.insert(hdl, tx.clone());\n }\n }\n }\n}\n\npub(crate) fn remove_cancelled_timers(wheel: &mut Wheel, rx: &mut Receiver) {\n for hdl in rx.recv_all() {\n debug_assert!(hdl.is_cancelled());\n\n if hdl.deadline() > wheel.elapsed() {\n // Safety: the entry is registered in THIS wheel\n unsafe {\n wheel.remove(hdl);\n }\n }\n }\n}\n\npub(crate) fn next_expiration_time(wheel: &Wheel, drv_hdl: &driver::Handle) -> Option {\n drv_hdl.with_time(|maybe_time_hdl| {\n let Some(time_hdl) = maybe_time_hdl else {\n // time driver is not enabled, nothing to do.\n return None;\n };\n\n let clock = drv_hdl.clock();\n let time_source = time_hdl.time_source();\n\n wheel.next_expiration_time().map(|tick| {\n let now = time_source.now(clock);\n time_source.tick_to_duration(tick.saturating_sub(now))\n })\n })\n}\n\n#[cfg(feature = \"test-util\")]\npub(crate) fn pre_auto_advance(drv_hdl: &driver::Handle, duration: Option) -> bool {\n drv_hdl.with_time(|maybe_time_hdl| {\n if maybe_time_hdl.is_none() {\n // time driver is not enabled, nothing to do.\n return false;\n }\n\n if duration.is_some() {\n let clock = drv_hdl.clock();\n if clock.can_auto_advance() {\n return true;\n }\n\n false\n } else {\n false\n }\n })\n}\n\npub(crate) fn process_expired_timers(\n wheel: &mut Wheel,\n drv_hdl: &driver::Handle,\n wake_queue: &mut WakeQueue,\n) {\n drv_hdl.with_time(|maybe_time_hdl| {\n let Some(time_hdl) = maybe_time_hdl else {\n // time driver is not enabled, nothing to do.\n return;\n };\n\n let clock = drv_hdl.clock();\n let time_source = time_hdl.time_source();\n\n let now = time_source.now(clock);\n time_hdl.process_at_time_alt(wheel, now, wake_queue);\n });\n}\n\npub(crate) fn shutdown_local_timers(\n wheel: &mut Wheel,\n rx: &mut Receiver,\n inject: Vec,\n drv_hdl: &driver::Handle,\n) {\n drv_hdl.with_time(|maybe_time_hdl| {\n let Some(time_hdl) = maybe_time_hdl else {\n // time driver is not enabled, nothing to do.\n return;\n };\n\n remove_cancelled_timers(wheel, rx);\n time_hdl.shutdown_alt(wheel);\n\n let mut wake_queue = WakeQueue::new();\n // simply wake all unregistered timers\n for hdl in inject.into_iter().filter(|hdl| !hdl.is_cancelled()) {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n\n wake_queue.wake_all();\n });\n}\n\n#[cfg(feature = \"test-util\")]\npub(crate) fn post_auto_advance(drv_hdl: &driver::Handle, duration: Option) {\n drv_hdl.with_time(|maybe_time_hdl| {\n let Some(time_hdl) = maybe_time_hdl else {\n // time driver is not enabled, nothing to do.\n return;\n };\n\n if let Some(park_duration) = duration {\n let clock = drv_hdl.clock();\n if clock.can_auto_advance() && !time_hdl.did_wake() {\n if let Err(msg) = clock.advance(park_duration) {\n panic!(\"{msg}\");\n }\n }\n }\n })\n}\n\n#[cfg(not(feature = \"test-util\"))]\npub(crate) fn pre_auto_advance(_drv_hdl: &driver::Handle, _duration: Option) -> bool {\n false\n}\n\n#[cfg(not(feature = \"test-util\"))]\npub(crate) fn post_auto_advance(_drv_hdl: &driver::Handle, _duration: Option) {\n // No-op in non-test util builds\n}\n" }, { "path": "tokio/src/runtime/signal/mod.rs", "content": "#![cfg_attr(not(feature = \"rt\"), allow(dead_code))]\n\n//! Signal driver\n\nuse crate::runtime::{driver, io};\nuse crate::signal::registry::globals;\n\nuse mio::net::UnixStream;\nuse std::io::{self as std_io, Read};\nuse std::sync::{Arc, Weak};\nuse std::time::Duration;\n\n/// Responsible for registering wakeups when an OS signal is received, and\n/// subsequently dispatching notifications to any signal listeners as appropriate.\n///\n/// Note: this driver relies on having an enabled IO driver in order to listen to\n/// pipe write wakeups.\n#[derive(Debug)]\npub(crate) struct Driver {\n /// Thread parker. The `Driver` park implementation delegates to this.\n io: io::Driver,\n\n /// A pipe for receiving wake events from the signal handler\n receiver: UnixStream,\n\n /// Shared state. The driver keeps a strong ref and the handle keeps a weak\n /// ref. The weak ref is used to check if the driver is still active before\n /// trying to register a signal handler.\n inner: Arc<()>,\n}\n\n#[derive(Debug, Default)]\npub(crate) struct Handle {\n /// Paired w/ the `Arc` above and is used to check if the driver is still\n /// around before attempting to register a signal handler.\n inner: Weak<()>,\n}\n\n// ===== impl Driver =====\n\nimpl Driver {\n /// Creates a new signal `Driver` instance that delegates wakeups to `park`.\n pub(crate) fn new(io: io::Driver, io_handle: &io::Handle) -> std_io::Result {\n use std::mem::ManuallyDrop;\n use std::os::unix::io::{AsRawFd, FromRawFd};\n\n // NB: We give each driver a \"fresh\" receiver file descriptor to avoid\n // the issues described in alexcrichton/tokio-process#42.\n //\n // In the past we would reuse the actual receiver file descriptor and\n // swallow any errors around double registration of the same descriptor.\n // I'm not sure if the second (failed) registration simply doesn't end\n // up receiving wake up notifications, or there could be some race\n // condition when consuming readiness events, but having distinct\n // descriptors appears to mitigate this.\n //\n // Unfortunately we cannot just use a single global UnixStream instance\n // either, since we can't assume they will always be registered with the\n // exact same reactor.\n //\n // Mio 0.7 removed `try_clone()` as an API due to unexpected behavior\n // with registering dups with the same reactor. In this case, duping is\n // safe as each dup is registered with separate reactors **and** we\n // only expect at least one dup to receive the notification.\n\n // Manually drop as we don't actually own this instance of UnixStream.\n let receiver_fd = globals().receiver.as_raw_fd();\n\n // safety: there is nothing unsafe about this, but the `from_raw_fd` fn is marked as unsafe.\n let original =\n ManuallyDrop::new(unsafe { std::os::unix::net::UnixStream::from_raw_fd(receiver_fd) });\n let mut receiver = UnixStream::from_std(original.try_clone()?);\n\n io_handle.register_signal_receiver(&mut receiver)?;\n\n Ok(Self {\n io,\n receiver,\n inner: Arc::new(()),\n })\n }\n\n /// Returns a handle to this event loop which can be sent across threads\n /// and can be used as a proxy to the event loop itself.\n pub(crate) fn handle(&self) -> Handle {\n Handle {\n inner: Arc::downgrade(&self.inner),\n }\n }\n\n pub(crate) fn park(&mut self, handle: &driver::Handle) {\n self.io.park(handle);\n self.process();\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &driver::Handle, duration: Duration) {\n self.io.park_timeout(handle, duration);\n self.process();\n }\n\n pub(crate) fn shutdown(&mut self, handle: &driver::Handle) {\n self.io.shutdown(handle);\n }\n\n fn process(&mut self) {\n // If the signal pipe has not received a readiness event, then there is\n // nothing else to do.\n if !self.io.consume_signal_ready() {\n return;\n }\n\n // Drain the pipe completely so we can receive a new readiness event\n // if another signal has come in.\n let mut buf = [0; 128];\n #[allow(clippy::unused_io_amount)]\n loop {\n match self.receiver.read(&mut buf) {\n Ok(0) => panic!(\"EOF on self-pipe\"),\n Ok(_) => continue, // Keep reading\n Err(e) if e.kind() == std_io::ErrorKind::WouldBlock => break,\n Err(e) => panic!(\"Bad read on self-pipe: {e}\"),\n }\n }\n\n // Broadcast any signals which were received\n globals().broadcast();\n }\n}\n\n// ===== impl Handle =====\n\nimpl Handle {\n pub(crate) fn check_inner(&self) -> std_io::Result<()> {\n if self.inner.strong_count() > 0 {\n Ok(())\n } else {\n Err(std_io::Error::new(\n std_io::ErrorKind::Other,\n \"signal driver gone\",\n ))\n }\n }\n}\n" }, { "path": "tokio/src/runtime/task/abort.rs", "content": "use crate::runtime::task::{Header, RawTask};\nuse std::fmt;\nuse std::panic::{RefUnwindSafe, UnwindSafe};\n\n/// An owned permission to abort a spawned task, without awaiting its completion.\n///\n/// Unlike a [`JoinHandle`], an `AbortHandle` does *not* represent the\n/// permission to await the task's completion, only to terminate it.\n///\n/// The task may be aborted by calling the [`AbortHandle::abort`] method.\n/// Dropping an `AbortHandle` releases the permission to terminate the task\n/// --- it does *not* abort the task.\n///\n/// Be aware that tasks spawned using [`spawn_blocking`] cannot be aborted\n/// because they are not async. If you call `abort` on a `spawn_blocking` task,\n/// then this *will not have any effect*, and the task will continue running\n/// normally. The exception is if the task has not started running yet; in that\n/// case, calling `abort` may prevent the task from starting.\n///\n/// [`JoinHandle`]: crate::task::JoinHandle\n/// [`spawn_blocking`]: crate::task::spawn_blocking\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub struct AbortHandle {\n raw: RawTask,\n}\n\nimpl AbortHandle {\n pub(super) fn new(raw: RawTask) -> Self {\n Self { raw }\n }\n\n /// Abort the task associated with the handle.\n ///\n /// Awaiting a cancelled task might complete as usual if the task was\n /// already completed at the time it was cancelled, but most likely it\n /// will fail with a [cancelled] `JoinError`.\n ///\n /// If the task was already cancelled, such as by [`JoinHandle::abort`],\n /// this method will do nothing.\n ///\n /// Be aware that tasks spawned using [`spawn_blocking`] cannot be aborted\n /// because they are not async. If you call `abort` on a `spawn_blocking`\n /// task, then this *will not have any effect*, and the task will continue\n /// running normally. The exception is if the task has not started running\n /// yet; in that case, calling `abort` may prevent the task from starting.\n ///\n /// See also [the module level docs] for more information on cancellation.\n ///\n /// [cancelled]: method@super::error::JoinError::is_cancelled\n /// [`JoinHandle::abort`]: method@super::JoinHandle::abort\n /// [the module level docs]: crate::task#cancellation\n /// [`spawn_blocking`]: crate::task::spawn_blocking\n pub fn abort(&self) {\n self.raw.remote_abort();\n }\n\n /// Checks if the task associated with this `AbortHandle` has finished.\n ///\n /// Please note that this method can return `false` even if `abort` has been\n /// called on the task. This is because the cancellation process may take\n /// some time, and this method does not return `true` until it has\n /// completed.\n pub fn is_finished(&self) -> bool {\n let state = self.raw.state().load();\n state.is_complete()\n }\n\n /// Returns a [task ID] that uniquely identifies this task relative to other\n /// currently spawned tasks.\n ///\n /// [task ID]: crate::task::Id\n pub fn id(&self) -> super::Id {\n // Safety: The header pointer is valid.\n unsafe { Header::get_id(self.raw.header_ptr()) }\n }\n}\n\nunsafe impl Send for AbortHandle {}\nunsafe impl Sync for AbortHandle {}\n\nimpl UnwindSafe for AbortHandle {}\nimpl RefUnwindSafe for AbortHandle {}\n\nimpl fmt::Debug for AbortHandle {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n // Safety: The header pointer is valid.\n let id_ptr = unsafe { Header::get_id_ptr(self.raw.header_ptr()) };\n let id = unsafe { id_ptr.as_ref() };\n fmt.debug_struct(\"AbortHandle\").field(\"id\", id).finish()\n }\n}\n\nimpl Drop for AbortHandle {\n fn drop(&mut self) {\n self.raw.drop_abort_handle();\n }\n}\n\nimpl Clone for AbortHandle {\n /// Returns a cloned `AbortHandle` that can be used to remotely abort this task.\n fn clone(&self) -> Self {\n self.raw.ref_inc();\n Self::new(self.raw)\n }\n}\n" }, { "path": "tokio/src/runtime/task/core.rs", "content": "//! Core task module.\n//!\n//! # Safety\n//!\n//! The functions in this module are private to the `task` module. All of them\n//! should be considered `unsafe` to use, but are not marked as such since it\n//! would be too noisy.\n//!\n//! Make sure to consult the relevant safety section of each function before\n//! use.\n\n// It doesn't make sense to enforce `unsafe_op_in_unsafe_fn` for this module because\n//\n// * This module is doing the low-level task management that requires tons of unsafe\n// operations.\n// * Excessive `unsafe {}` blocks hurt readability significantly.\n// TODO: replace with `#[expect(unsafe_op_in_unsafe_fn)]` after bumpping\n// the MSRV to 1.81.0.\n#![allow(unsafe_op_in_unsafe_fn)]\n\nuse crate::future::Future;\nuse crate::loom::cell::UnsafeCell;\nuse crate::runtime::context;\nuse crate::runtime::task::raw::{self, Vtable};\nuse crate::runtime::task::state::State;\nuse crate::runtime::task::{Id, Schedule, TaskHarnessScheduleHooks};\nuse crate::util::linked_list;\n\nuse std::num::NonZeroU64;\n#[cfg(tokio_unstable)]\nuse std::panic::Location;\nuse std::pin::Pin;\nuse std::ptr::NonNull;\nuse std::task::{Context, Poll, Waker};\n\n/// The task cell. Contains the components of the task.\n///\n/// It is critical for `Header` to be the first field as the task structure will\n/// be referenced by both *mut Cell and *mut Header.\n///\n/// Any changes to the layout of this struct _must_ also be reflected in the\n/// `const` fns in raw.rs.\n///\n// # This struct should be cache padded to avoid false sharing. The cache padding rules are copied\n// from crossbeam-utils/src/cache_padded.rs\n//\n// Starting from Intel's Sandy Bridge, spatial prefetcher is now pulling pairs of 64-byte cache\n// lines at a time, so we have to align to 128 bytes rather than 64.\n//\n// Sources:\n// - https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf\n// - https://github.com/facebook/folly/blob/1b5288e6eea6df074758f877c849b6e73bbb9fbb/folly/lang/Align.h#L107\n//\n// ARM's big.LITTLE architecture has asymmetric cores and \"big\" cores have 128-byte cache line size.\n//\n// Sources:\n// - https://www.mono-project.com/news/2016/09/12/arm64-icache/\n//\n// powerpc64 has 128-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_ppc64x.go#L9\n#[cfg_attr(\n any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n ),\n repr(align(128))\n)]\n// arm, mips, mips64, sparc, and hexagon have 32-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_arm.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mipsle.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips64x.go#L9\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/sparc/include/asm/cache.h#L17\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/hexagon/include/asm/cache.h#L12\n#[cfg_attr(\n any(\n target_arch = \"arm\",\n target_arch = \"mips\",\n target_arch = \"mips64\",\n target_arch = \"sparc\",\n target_arch = \"hexagon\",\n ),\n repr(align(32))\n)]\n// m68k has 16-byte cache line size.\n//\n// Sources:\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/m68k/include/asm/cache.h#L9\n#[cfg_attr(target_arch = \"m68k\", repr(align(16)))]\n// s390x has 256-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_s390x.go#L7\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/s390/include/asm/cache.h#L13\n#[cfg_attr(target_arch = \"s390x\", repr(align(256)))]\n// x86, riscv, wasm, and sparc64 have 64-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/dda2991c2ea0c5914714469c4defc2562a907230/src/internal/cpu/cpu_x86.go#L9\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_wasm.go#L7\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/sparc/include/asm/cache.h#L19\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/riscv/include/asm/cache.h#L10\n//\n// All others are assumed to have 64-byte cache line size.\n#[cfg_attr(\n not(any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n target_arch = \"arm\",\n target_arch = \"mips\",\n target_arch = \"mips64\",\n target_arch = \"sparc\",\n target_arch = \"hexagon\",\n target_arch = \"m68k\",\n target_arch = \"s390x\",\n )),\n repr(align(64))\n)]\n#[repr(C)]\npub(super) struct Cell {\n /// Hot task state data\n pub(super) header: Header,\n\n /// Either the future or output, depending on the execution stage.\n pub(super) core: Core,\n\n /// Cold data\n pub(super) trailer: Trailer,\n}\n\npub(super) struct CoreStage {\n stage: UnsafeCell>,\n}\n\n/// The core of the task.\n///\n/// Holds the future or output, depending on the stage of execution.\n///\n/// Any changes to the layout of this struct _must_ also be reflected in the\n/// `const` fns in raw.rs.\n#[repr(C)]\npub(super) struct Core {\n /// Scheduler used to drive this future.\n pub(super) scheduler: S,\n\n /// The task's ID, used for populating `JoinError`s.\n pub(super) task_id: Id,\n\n /// The source code location where the task was spawned.\n ///\n /// This is used for populating the `TaskMeta` passed to the task runtime\n /// hooks.\n #[cfg(tokio_unstable)]\n pub(super) spawned_at: &'static Location<'static>,\n\n /// Either the future or the output.\n pub(super) stage: CoreStage,\n}\n\n/// Crate public as this is also needed by the pool.\n#[repr(C)]\npub(crate) struct Header {\n /// Task state.\n pub(super) state: State,\n\n /// Pointer to next task, used with the injection queue.\n pub(super) queue_next: UnsafeCell>>,\n\n /// Table of function pointers for executing actions on the task.\n pub(super) vtable: &'static Vtable,\n\n /// This integer contains the id of the `OwnedTasks` or `LocalOwnedTasks`\n /// that this task is stored in. If the task is not in any list, should be\n /// the id of the list that it was previously in, or `None` if it has never\n /// been in any list.\n ///\n /// Once a task has been bound to a list, it can never be bound to another\n /// list, even if removed from the first list.\n ///\n /// The id is not unset when removed from a list because we want to be able\n /// to read the id without synchronization, even if it is concurrently being\n /// removed from the list.\n pub(super) owner_id: UnsafeCell>,\n\n /// The tracing ID for this instrumented task.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) tracing_id: Option,\n}\n\nunsafe impl Send for Header {}\nunsafe impl Sync for Header {}\n\n/// Cold data is stored after the future. Data is considered cold if it is only\n/// used during creation or shutdown of the task.\npub(super) struct Trailer {\n /// Pointers for the linked list in the `OwnedTasks` that owns this task.\n pub(super) owned: linked_list::Pointers
,\n /// Consumer task waiting on completion of this task.\n pub(super) waker: UnsafeCell>,\n /// Optional hooks needed in the harness.\n #[cfg_attr(not(tokio_unstable), allow(dead_code))] //TODO: remove when hooks are stabilized\n pub(super) hooks: TaskHarnessScheduleHooks,\n}\n\ngenerate_addr_of_methods! {\n impl<> Trailer {\n pub(super) unsafe fn addr_of_owned(self: NonNull) -> NonNull> {\n &self.owned\n }\n }\n}\n\n/// Either the future or the output.\n#[repr(C)] // https://github.com/rust-lang/miri/issues/3780\npub(super) enum Stage {\n Running(T),\n Finished(super::Result),\n Consumed,\n}\n\nimpl Cell {\n /// Allocates a new task cell, containing the header, trailer, and core\n /// structures.\n pub(super) fn new(\n future: T,\n scheduler: S,\n state: State,\n task_id: Id,\n #[cfg(tokio_unstable)] spawned_at: &'static Location<'static>,\n ) -> Box> {\n // Separated into a non-generic function to reduce LLVM codegen\n fn new_header(\n state: State,\n vtable: &'static Vtable,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))] tracing_id: Option,\n ) -> Header {\n Header {\n state,\n queue_next: UnsafeCell::new(None),\n vtable,\n owner_id: UnsafeCell::new(None),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n tracing_id,\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let tracing_id = future.id();\n let vtable = raw::vtable::();\n let result = Box::new(Cell {\n trailer: Trailer::new(scheduler.hooks()),\n header: new_header(\n state,\n vtable,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n tracing_id,\n ),\n core: Core {\n scheduler,\n stage: CoreStage {\n stage: UnsafeCell::new(Stage::Running(future)),\n },\n task_id,\n #[cfg(tokio_unstable)]\n spawned_at,\n },\n });\n\n #[cfg(debug_assertions)]\n {\n // Using a separate function for this code avoids instantiating it separately for every `T`.\n unsafe fn check(\n header: &Header,\n trailer: &Trailer,\n scheduler: &S,\n task_id: &Id,\n #[cfg(tokio_unstable)] spawn_location: &&'static Location<'static>,\n ) {\n let trailer_addr = trailer as *const Trailer as usize;\n let trailer_ptr = unsafe { Header::get_trailer(NonNull::from(header)) };\n assert_eq!(trailer_addr, trailer_ptr.as_ptr() as usize);\n\n let scheduler_addr = scheduler as *const S as usize;\n let scheduler_ptr = unsafe { Header::get_scheduler::(NonNull::from(header)) };\n assert_eq!(scheduler_addr, scheduler_ptr.as_ptr() as usize);\n\n let id_addr = task_id as *const Id as usize;\n let id_ptr = unsafe { Header::get_id_ptr(NonNull::from(header)) };\n assert_eq!(id_addr, id_ptr.as_ptr() as usize);\n\n #[cfg(tokio_unstable)]\n {\n let spawn_location_addr =\n spawn_location as *const &'static Location<'static> as usize;\n let spawn_location_ptr =\n unsafe { Header::get_spawn_location_ptr(NonNull::from(header)) };\n assert_eq!(spawn_location_addr, spawn_location_ptr.as_ptr() as usize);\n }\n }\n unsafe {\n check(\n &result.header,\n &result.trailer,\n &result.core.scheduler,\n &result.core.task_id,\n #[cfg(tokio_unstable)]\n &result.core.spawned_at,\n );\n }\n }\n\n result\n }\n}\n\nimpl CoreStage {\n pub(super) fn with_mut(&self, f: impl FnOnce(*mut Stage) -> R) -> R {\n self.stage.with_mut(f)\n }\n}\n\n/// Set and clear the task id in the context when the future is executed or\n/// dropped, or when the output produced by the future is dropped.\npub(crate) struct TaskIdGuard {\n parent_task_id: Option,\n}\n\nimpl TaskIdGuard {\n fn enter(id: Id) -> Self {\n TaskIdGuard {\n parent_task_id: context::set_current_task_id(Some(id)),\n }\n }\n}\n\nimpl Drop for TaskIdGuard {\n fn drop(&mut self) {\n context::set_current_task_id(self.parent_task_id);\n }\n}\n\nimpl Core {\n /// Polls the future.\n ///\n /// # Safety\n ///\n /// The caller must ensure it is safe to mutate the `state` field. This\n /// requires ensuring mutual exclusion between any concurrent thread that\n /// might modify the future or output field.\n ///\n /// The mutual exclusion is implemented by `Harness` and the `Lifecycle`\n /// component of the task state.\n ///\n /// `self` must also be pinned. This is handled by storing the task on the\n /// heap.\n pub(super) fn poll(&self, mut cx: Context<'_>) -> Poll {\n let res = {\n self.stage.stage.with_mut(|ptr| {\n // Safety: The caller ensures mutual exclusion to the field.\n let future = match unsafe { &mut *ptr } {\n Stage::Running(future) => future,\n _ => unreachable!(\"unexpected stage\"),\n };\n\n // Safety: The caller ensures the future is pinned.\n let future = unsafe { Pin::new_unchecked(future) };\n\n let _guard = TaskIdGuard::enter(self.task_id);\n future.poll(&mut cx)\n })\n };\n\n if res.is_ready() {\n self.drop_future_or_output();\n }\n\n res\n }\n\n /// Drops the future.\n ///\n /// # Safety\n ///\n /// The caller must ensure it is safe to mutate the `stage` field.\n pub(super) fn drop_future_or_output(&self) {\n // Safety: the caller ensures mutual exclusion to the field.\n unsafe {\n self.set_stage(Stage::Consumed);\n }\n }\n\n /// Stores the task output.\n ///\n /// # Safety\n ///\n /// The caller must ensure it is safe to mutate the `stage` field.\n pub(super) fn store_output(&self, output: super::Result) {\n // Safety: the caller ensures mutual exclusion to the field.\n unsafe {\n self.set_stage(Stage::Finished(output));\n }\n }\n\n /// Takes the task output.\n ///\n /// # Safety\n ///\n /// The caller must ensure it is safe to mutate the `stage` field.\n pub(super) fn take_output(&self) -> super::Result {\n use std::mem;\n\n self.stage.stage.with_mut(|ptr| {\n // Safety:: the caller ensures mutual exclusion to the field.\n match mem::replace(unsafe { &mut *ptr }, Stage::Consumed) {\n Stage::Finished(output) => output,\n _ => panic!(\"JoinHandle polled after completion\"),\n }\n })\n }\n\n unsafe fn set_stage(&self, stage: Stage) {\n let _guard = TaskIdGuard::enter(self.task_id);\n self.stage.stage.with_mut(|ptr| *ptr = stage);\n }\n}\n\nimpl Header {\n pub(super) unsafe fn set_next(&self, next: Option>) {\n self.queue_next.with_mut(|ptr| *ptr = next);\n }\n\n // safety: The caller must guarantee exclusive access to this field, and\n // must ensure that the id is either `None` or the id of the OwnedTasks\n // containing this task.\n pub(super) unsafe fn set_owner_id(&self, owner: NonZeroU64) {\n self.owner_id.with_mut(|ptr| *ptr = Some(owner));\n }\n\n pub(super) fn get_owner_id(&self) -> Option {\n // safety: If there are concurrent writes, then that write has violated\n // the safety requirements on `set_owner_id`.\n unsafe { self.owner_id.with(|ptr| *ptr) }\n }\n\n /// Gets a pointer to the `Trailer` of the task containing this `Header`.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n pub(super) unsafe fn get_trailer(me: NonNull
) -> NonNull {\n let offset = me.as_ref().vtable.trailer_offset;\n let trailer = me.as_ptr().cast::().add(offset).cast::();\n NonNull::new_unchecked(trailer)\n }\n\n /// Gets a pointer to the scheduler of the task containing this `Header`.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n ///\n /// The generic type S must be set to the correct scheduler type for this\n /// task.\n pub(super) unsafe fn get_scheduler(me: NonNull
) -> NonNull {\n let offset = me.as_ref().vtable.scheduler_offset;\n let scheduler = me.as_ptr().cast::().add(offset).cast::();\n NonNull::new_unchecked(scheduler)\n }\n\n /// Gets a pointer to the id of the task containing this `Header`.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n pub(super) unsafe fn get_id_ptr(me: NonNull
) -> NonNull {\n let offset = me.as_ref().vtable.id_offset;\n let id = me.as_ptr().cast::().add(offset).cast::();\n NonNull::new_unchecked(id)\n }\n\n /// Gets the id of the task containing this `Header`.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n pub(super) unsafe fn get_id(me: NonNull
) -> Id {\n let ptr = Header::get_id_ptr(me).as_ptr();\n *ptr\n }\n\n /// Gets a pointer to the source code location where the task containing\n /// this `Header` was spawned.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n #[cfg(tokio_unstable)]\n pub(super) unsafe fn get_spawn_location_ptr(\n me: NonNull
,\n ) -> NonNull<&'static Location<'static>> {\n let offset = me.as_ref().vtable.spawn_location_offset;\n let spawned_at = me\n .as_ptr()\n .cast::()\n .add(offset)\n .cast::<&'static Location<'static>>();\n NonNull::new_unchecked(spawned_at)\n }\n\n /// Gets the source code location where the task containing\n /// this `Header` was spawned\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n #[cfg(tokio_unstable)]\n pub(super) unsafe fn get_spawn_location(me: NonNull
) -> &'static Location<'static> {\n let ptr = Header::get_spawn_location_ptr(me).as_ptr();\n *ptr\n }\n\n /// Gets the tracing id of the task containing this `Header`.\n ///\n /// # Safety\n ///\n /// The provided raw pointer must point at the header of a task.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) unsafe fn get_tracing_id(me: &NonNull
) -> Option<&tracing::Id> {\n me.as_ref().tracing_id.as_ref()\n }\n}\n\nimpl Trailer {\n fn new(hooks: TaskHarnessScheduleHooks) -> Self {\n Trailer {\n waker: UnsafeCell::new(None),\n owned: linked_list::Pointers::new(),\n hooks,\n }\n }\n\n pub(super) unsafe fn set_waker(&self, waker: Option) {\n self.waker.with_mut(|ptr| {\n *ptr = waker;\n });\n }\n\n pub(super) unsafe fn will_wake(&self, waker: &Waker) -> bool {\n self.waker\n .with(|ptr| (*ptr).as_ref().unwrap().will_wake(waker))\n }\n\n pub(super) fn wake_join(&self) {\n self.waker.with(|ptr| match unsafe { &*ptr } {\n Some(waker) => waker.wake_by_ref(),\n None => panic!(\"waker missing\"),\n });\n }\n}\n\n#[test]\n#[cfg(not(loom))]\nfn header_lte_cache_line() {\n assert!(std::mem::size_of::
() <= 8 * std::mem::size_of::<*const ()>());\n}\n" }, { "path": "tokio/src/runtime/task/error.rs", "content": "use std::any::Any;\nuse std::fmt;\nuse std::io;\n\nuse super::Id;\nuse crate::util::SyncWrapper;\ncfg_rt! {\n /// Task failed to execute to completion.\n pub struct JoinError {\n repr: Repr,\n id: Id,\n }\n}\n\nenum Repr {\n Cancelled,\n Panic(SyncWrapper>),\n}\n\nimpl JoinError {\n pub(crate) fn cancelled(id: Id) -> JoinError {\n JoinError {\n repr: Repr::Cancelled,\n id,\n }\n }\n\n pub(crate) fn panic(id: Id, err: Box) -> JoinError {\n JoinError {\n repr: Repr::Panic(SyncWrapper::new(err)),\n id,\n }\n }\n\n /// Returns true if the error was caused by the task being cancelled.\n ///\n /// See [the module level docs] for more information on cancellation.\n ///\n /// [the module level docs]: crate::task#cancellation\n pub fn is_cancelled(&self) -> bool {\n matches!(&self.repr, Repr::Cancelled)\n }\n\n /// Returns true if the error was caused by the task panicking.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::panic;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let err = tokio::spawn(async {\n /// panic!(\"boom\");\n /// }).await.unwrap_err();\n ///\n /// assert!(err.is_panic());\n /// }\n /// # }\n /// ```\n pub fn is_panic(&self) -> bool {\n matches!(&self.repr, Repr::Panic(_))\n }\n\n /// Consumes the join error, returning the object with which the task panicked.\n ///\n /// # Panics\n ///\n /// `into_panic()` panics if the `Error` does not represent the underlying\n /// task terminating with a panic. Use `is_panic` to check the error reason\n /// or `try_into_panic` for a variant that does not panic.\n ///\n /// # Examples\n ///\n /// ```should_panic,ignore-wasm\n /// use std::panic;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let err = tokio::spawn(async {\n /// panic!(\"boom\");\n /// }).await.unwrap_err();\n ///\n /// if err.is_panic() {\n /// // Resume the panic on the main task\n /// panic::resume_unwind(err.into_panic());\n /// }\n /// }\n /// ```\n #[track_caller]\n pub fn into_panic(self) -> Box {\n self.try_into_panic()\n .expect(\"`JoinError` reason is not a panic.\")\n }\n\n /// Consumes the join error, returning the object with which the task\n /// panicked if the task terminated due to a panic. Otherwise, `self` is\n /// returned.\n ///\n /// # Examples\n ///\n /// ```should_panic,ignore-wasm\n /// use std::panic;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let err = tokio::spawn(async {\n /// panic!(\"boom\");\n /// }).await.unwrap_err();\n ///\n /// if let Ok(reason) = err.try_into_panic() {\n /// // Resume the panic on the main task\n /// panic::resume_unwind(reason);\n /// }\n /// }\n /// ```\n pub fn try_into_panic(self) -> Result, JoinError> {\n match self.repr {\n Repr::Panic(p) => Ok(p.into_inner()),\n _ => Err(self),\n }\n }\n\n /// Returns a [task ID] that identifies the task which errored relative to\n /// other currently spawned tasks.\n ///\n /// [task ID]: crate::task::Id\n pub fn id(&self) -> Id {\n self.id\n }\n}\n\nimpl fmt::Display for JoinError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n match &self.repr {\n Repr::Cancelled => write!(fmt, \"task {} was cancelled\", self.id),\n Repr::Panic(p) => match panic_payload_as_str(p) {\n Some(panic_str) => {\n write!(\n fmt,\n \"task {} panicked with message {:?}\",\n self.id, panic_str\n )\n }\n None => {\n write!(fmt, \"task {} panicked\", self.id)\n }\n },\n }\n }\n}\n\nimpl fmt::Debug for JoinError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n match &self.repr {\n Repr::Cancelled => write!(fmt, \"JoinError::Cancelled({:?})\", self.id),\n Repr::Panic(p) => match panic_payload_as_str(p) {\n Some(panic_str) => {\n write!(fmt, \"JoinError::Panic({:?}, {:?}, ...)\", self.id, panic_str)\n }\n None => write!(fmt, \"JoinError::Panic({:?}, ...)\", self.id),\n },\n }\n }\n}\n\nimpl std::error::Error for JoinError {}\n\nimpl From for io::Error {\n fn from(src: JoinError) -> io::Error {\n io::Error::new(\n io::ErrorKind::Other,\n match src.repr {\n Repr::Cancelled => \"task was cancelled\",\n Repr::Panic(_) => \"task panicked\",\n },\n )\n }\n}\n\nfn panic_payload_as_str(payload: &SyncWrapper>) -> Option<&str> {\n // Panic payloads are almost always `String` (if invoked with formatting arguments)\n // or `&'static str` (if invoked with a string literal).\n //\n // Non-string panic payloads have niche use-cases,\n // so we don't really need to worry about those.\n if let Some(s) = payload.downcast_ref_sync::() {\n return Some(s);\n }\n\n if let Some(s) = payload.downcast_ref_sync::<&'static str>() {\n return Some(s);\n }\n\n None\n}\n" }, { "path": "tokio/src/runtime/task/harness.rs", "content": "use crate::future::Future;\nuse crate::runtime::task::core::{Cell, Core, Header, Trailer};\nuse crate::runtime::task::state::{Snapshot, State};\nuse crate::runtime::task::waker::waker_ref;\nuse crate::runtime::task::{Id, JoinError, Notified, RawTask, Schedule, Task};\n\n#[cfg(tokio_unstable)]\nuse crate::runtime::TaskMeta;\nuse std::any::Any;\nuse std::mem;\nuse std::mem::ManuallyDrop;\nuse std::panic;\nuse std::ptr::NonNull;\nuse std::task::{Context, Poll, Waker};\n\n/// Typed raw task handle.\npub(super) struct Harness {\n cell: NonNull>,\n}\n\nimpl Harness\nwhere\n T: Future,\n S: 'static,\n{\n pub(super) unsafe fn from_raw(ptr: NonNull
) -> Harness {\n Harness {\n cell: ptr.cast::>(),\n }\n }\n\n fn header_ptr(&self) -> NonNull
{\n self.cell.cast()\n }\n\n fn header(&self) -> &Header {\n unsafe { &*self.header_ptr().as_ptr() }\n }\n\n fn state(&self) -> &State {\n &self.header().state\n }\n\n fn trailer(&self) -> &Trailer {\n unsafe { &self.cell.as_ref().trailer }\n }\n\n fn core(&self) -> &Core {\n unsafe { &self.cell.as_ref().core }\n }\n}\n\n/// Task operations that can be implemented without being generic over the\n/// scheduler or task. Only one version of these methods should exist in the\n/// final binary.\nimpl RawTask {\n pub(super) fn drop_reference(self) {\n if self.state().ref_dec() {\n self.dealloc();\n }\n }\n\n /// This call consumes a ref-count and notifies the task. This will create a\n /// new Notified and submit it if necessary.\n ///\n /// The caller does not need to hold a ref-count besides the one that was\n /// passed to this call.\n pub(super) fn wake_by_val(&self) {\n use super::state::TransitionToNotifiedByVal;\n\n match self.state().transition_to_notified_by_val() {\n TransitionToNotifiedByVal::Submit => {\n // The caller has given us a ref-count, and the transition has\n // created a new ref-count, so we now hold two. We turn the new\n // ref-count Notified and pass it to the call to `schedule`.\n //\n // The old ref-count is retained for now to ensure that the task\n // is not dropped during the call to `schedule` if the call\n // drops the task it was given.\n self.schedule();\n\n // Now that we have completed the call to schedule, we can\n // release our ref-count.\n self.drop_reference();\n }\n TransitionToNotifiedByVal::Dealloc => {\n self.dealloc();\n }\n TransitionToNotifiedByVal::DoNothing => {}\n }\n }\n\n /// This call notifies the task. It will not consume any ref-counts, but the\n /// caller should hold a ref-count. This will create a new Notified and\n /// submit it if necessary.\n pub(super) fn wake_by_ref(&self) {\n use super::state::TransitionToNotifiedByRef;\n\n match self.state().transition_to_notified_by_ref() {\n TransitionToNotifiedByRef::Submit => {\n // The transition above incremented the ref-count for a new task\n // and the caller also holds a ref-count. The caller's ref-count\n // ensures that the task is not destroyed even if the new task\n // is dropped before `schedule` returns.\n self.schedule();\n }\n TransitionToNotifiedByRef::DoNothing => {}\n }\n }\n\n /// Remotely aborts the task.\n ///\n /// The caller should hold a ref-count, but we do not consume it.\n ///\n /// This is similar to `shutdown` except that it asks the runtime to perform\n /// the shutdown. This is necessary to avoid the shutdown happening in the\n /// wrong thread for non-Send tasks.\n pub(super) fn remote_abort(&self) {\n if self.state().transition_to_notified_and_cancel() {\n // The transition has created a new ref-count, which we turn into\n // a Notified and pass to the task.\n //\n // Since the caller holds a ref-count, the task cannot be destroyed\n // before the call to `schedule` returns even if the call drops the\n // `Notified` internally.\n self.schedule();\n }\n }\n\n /// Try to set the waker notified when the task is complete. Returns true if\n /// the task has already completed. If this call returns false, then the\n /// waker will not be notified.\n pub(super) fn try_set_join_waker(&self, waker: &Waker) -> bool {\n can_read_output(self.header(), self.trailer(), waker)\n }\n}\n\nimpl Harness\nwhere\n T: Future,\n S: Schedule,\n{\n pub(super) fn drop_reference(self) {\n if self.state().ref_dec() {\n self.dealloc();\n }\n }\n\n /// Polls the inner future. A ref-count is consumed.\n ///\n /// All necessary state checks and transitions are performed.\n /// Panics raised while polling the future are handled.\n pub(super) fn poll(self) {\n // We pass our ref-count to `poll_inner`.\n match self.poll_inner() {\n PollFuture::Notified => {\n // The `poll_inner` call has given us two ref-counts back.\n // We give one of them to a new task and call `yield_now`.\n self.core()\n .scheduler\n .yield_now(Notified(self.get_new_task()));\n\n // The remaining ref-count is now dropped. We kept the extra\n // ref-count until now to ensure that even if the `yield_now`\n // call drops the provided task, the task isn't deallocated\n // before after `yield_now` returns.\n self.drop_reference();\n }\n PollFuture::Complete => {\n self.complete();\n }\n PollFuture::Dealloc => {\n self.dealloc();\n }\n PollFuture::Done => (),\n }\n }\n\n /// Polls the task and cancel it if necessary. This takes ownership of a\n /// ref-count.\n ///\n /// If the return value is Notified, the caller is given ownership of two\n /// ref-counts.\n ///\n /// If the return value is Complete, the caller is given ownership of a\n /// single ref-count, which should be passed on to `complete`.\n ///\n /// If the return value is `Dealloc`, then this call consumed the last\n /// ref-count and the caller should call `dealloc`.\n ///\n /// Otherwise the ref-count is consumed and the caller should not access\n /// `self` again.\n fn poll_inner(&self) -> PollFuture {\n use super::state::{TransitionToIdle, TransitionToRunning};\n\n match self.state().transition_to_running() {\n TransitionToRunning::Success => {\n // Separated to reduce LLVM codegen\n fn transition_result_to_poll_future(result: TransitionToIdle) -> PollFuture {\n match result {\n TransitionToIdle::Ok => PollFuture::Done,\n TransitionToIdle::OkNotified => PollFuture::Notified,\n TransitionToIdle::OkDealloc => PollFuture::Dealloc,\n TransitionToIdle::Cancelled => PollFuture::Complete,\n }\n }\n let header_ptr = self.header_ptr();\n let waker_ref = waker_ref::(&header_ptr);\n let cx = Context::from_waker(&waker_ref);\n let res = poll_future(self.core(), cx);\n\n if res == Poll::Ready(()) {\n // The future completed. Move on to complete the task.\n return PollFuture::Complete;\n }\n\n let transition_res = self.state().transition_to_idle();\n if let TransitionToIdle::Cancelled = transition_res {\n // The transition to idle failed because the task was\n // cancelled during the poll.\n cancel_task(self.core());\n }\n transition_result_to_poll_future(transition_res)\n }\n TransitionToRunning::Cancelled => {\n cancel_task(self.core());\n PollFuture::Complete\n }\n TransitionToRunning::Failed => PollFuture::Done,\n TransitionToRunning::Dealloc => PollFuture::Dealloc,\n }\n }\n\n /// Forcibly shuts down the task.\n ///\n /// Attempt to transition to `Running` in order to forcibly shutdown the\n /// task. If the task is currently running or in a state of completion, then\n /// there is nothing further to do. When the task completes running, it will\n /// notice the `CANCELLED` bit and finalize the task.\n pub(super) fn shutdown(self) {\n if !self.state().transition_to_shutdown() {\n // The task is concurrently running. No further work needed.\n self.drop_reference();\n return;\n }\n\n // By transitioning the lifecycle to `Running`, we have permission to\n // drop the future.\n cancel_task(self.core());\n self.complete();\n }\n\n pub(super) fn dealloc(self) {\n // Observe that we expect to have mutable access to these objects\n // because we are going to drop them. This only matters when running\n // under loom.\n self.trailer().waker.with_mut(|_| ());\n self.core().stage.with_mut(|_| ());\n\n // Safety: The caller of this method just transitioned our ref-count to\n // zero, so it is our responsibility to release the allocation.\n //\n // We don't hold any references into the allocation at this point, but\n // it is possible for another thread to still hold a `&State` into the\n // allocation if that other thread has decremented its last ref-count,\n // but has not yet returned from the relevant method on `State`.\n //\n // However, the `State` type consists of just an `AtomicUsize`, and an\n // `AtomicUsize` wraps the entirety of its contents in an `UnsafeCell`.\n // As explained in the documentation for `UnsafeCell`, such references\n // are allowed to be dangling after their last use, even if the\n // reference has not yet gone out of scope.\n unsafe {\n drop(Box::from_raw(self.cell.as_ptr()));\n }\n }\n\n // ===== join handle =====\n\n /// Read the task output into `dst`.\n pub(super) fn try_read_output(self, dst: &mut Poll>, waker: &Waker) {\n if can_read_output(self.header(), self.trailer(), waker) {\n *dst = Poll::Ready(self.core().take_output());\n }\n }\n\n pub(super) fn drop_join_handle_slow(self) {\n // Try to unset `JOIN_INTEREST` and `JOIN_WAKER`. This must be done as a first step in\n // case the task concurrently completed.\n let transition = self.state().transition_to_join_handle_dropped();\n\n if transition.drop_output {\n // It is our responsibility to drop the output. This is critical as\n // the task output may not be `Send` and as such must remain with\n // the scheduler or `JoinHandle`. i.e. if the output remains in the\n // task structure until the task is deallocated, it may be dropped\n // by a Waker on any arbitrary thread.\n //\n // Panics are delivered to the user via the `JoinHandle`. Given that\n // they are dropping the `JoinHandle`, we assume they are not\n // interested in the panic and swallow it.\n let _ = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n self.core().drop_future_or_output();\n }));\n }\n\n if transition.drop_waker {\n // If the JOIN_WAKER flag is unset at this point, the task is either\n // already terminal or not complete so the `JoinHandle` is responsible\n // for dropping the waker.\n // Safety:\n // If the JOIN_WAKER bit is not set the join handle has exclusive\n // access to the waker as per rule 2 in task/mod.rs.\n // This can only be the case at this point in two scenarios:\n // 1. The task completed and the runtime unset `JOIN_WAKER` flag\n // after accessing the waker during task completion. So the\n // `JoinHandle` is the only one to access the join waker here.\n // 2. The task is not completed so the `JoinHandle` was able to unset\n // `JOIN_WAKER` bit itself to get mutable access to the waker.\n // The runtime will not access the waker when this flag is unset.\n unsafe { self.trailer().set_waker(None) };\n }\n\n // Drop the `JoinHandle` reference, possibly deallocating the task\n self.drop_reference();\n }\n\n // ====== internal ======\n\n /// Completes the task. This method assumes that the state is RUNNING.\n fn complete(self) {\n // The future has completed and its output has been written to the task\n // stage. We transition from running to complete.\n let snapshot = self.state().transition_to_complete();\n\n // We catch panics here in case dropping the future or waking the\n // JoinHandle panics.\n let _ = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n if !snapshot.is_join_interested() {\n // The `JoinHandle` is not interested in the output of\n // this task. It is our responsibility to drop the\n // output. The join waker was already dropped by the\n // `JoinHandle` before.\n self.core().drop_future_or_output();\n } else if snapshot.is_join_waker_set() {\n // Notify the waker. Reading the waker field is safe per rule 4\n // in task/mod.rs, since the JOIN_WAKER bit is set and the call\n // to transition_to_complete() above set the COMPLETE bit.\n self.trailer().wake_join();\n\n // Inform the `JoinHandle` that we are done waking the waker by\n // unsetting the `JOIN_WAKER` bit. If the `JoinHandle` has\n // already been dropped and `JOIN_INTEREST` is unset, then we must\n // drop the waker ourselves.\n if !self\n .state()\n .unset_waker_after_complete()\n .is_join_interested()\n {\n // SAFETY: We have COMPLETE=1 and JOIN_INTEREST=0, so\n // we have exclusive access to the waker.\n unsafe { self.trailer().set_waker(None) };\n }\n }\n }));\n\n // We catch panics here in case invoking a hook panics.\n //\n // We call this in a separate block so that it runs after the task appears to have\n // completed and will still run if the destructor panics.\n #[cfg(tokio_unstable)]\n if let Some(f) = self.trailer().hooks.task_terminate_callback.as_ref() {\n let _ = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n f(&TaskMeta {\n id: self.core().task_id,\n spawned_at: self.core().spawned_at.into(),\n _phantom: Default::default(),\n })\n }));\n }\n\n // The task has completed execution and will no longer be scheduled.\n let num_release = self.release();\n\n if self.state().transition_to_terminal(num_release) {\n self.dealloc();\n }\n }\n\n /// Releases the task from the scheduler. Returns the number of ref-counts\n /// that should be decremented.\n fn release(&self) -> usize {\n // We don't actually increment the ref-count here, but the new task is\n // never destroyed, so that's ok.\n let me = ManuallyDrop::new(self.get_new_task());\n\n if let Some(task) = self.core().scheduler.release(&me) {\n mem::forget(task);\n 2\n } else {\n 1\n }\n }\n\n /// Creates a new task that holds its own ref-count.\n ///\n /// # Safety\n ///\n /// Any use of `self` after this call must ensure that a ref-count to the\n /// task holds the task alive until after the use of `self`. Passing the\n /// returned Task to any method on `self` is unsound if dropping the Task\n /// could drop `self` before the call on `self` returned.\n fn get_new_task(&self) -> Task {\n // safety: The header is at the beginning of the cell, so this cast is\n // safe.\n unsafe { Task::from_raw(self.cell.cast()) }\n }\n}\n\nfn can_read_output(header: &Header, trailer: &Trailer, waker: &Waker) -> bool {\n // Load a snapshot of the current task state\n let snapshot = header.state.load();\n\n debug_assert!(snapshot.is_join_interested());\n\n if !snapshot.is_complete() {\n // If the task is not complete, try storing the provided waker in the\n // task's waker field.\n\n let res = if snapshot.is_join_waker_set() {\n // If JOIN_WAKER is set, then JoinHandle has previously stored a\n // waker in the waker field per step (iii) of rule 5 in task/mod.rs.\n\n // Optimization: if the stored waker and the provided waker wake the\n // same task, then return without touching the waker field. (Reading\n // the waker field below is safe per rule 3 in task/mod.rs.)\n if unsafe { trailer.will_wake(waker) } {\n return false;\n }\n\n // Otherwise swap the stored waker with the provided waker by\n // following the rule 5 in task/mod.rs.\n header\n .state\n .unset_waker()\n .and_then(|snapshot| set_join_waker(header, trailer, waker.clone(), snapshot))\n } else {\n // If JOIN_WAKER is unset, then JoinHandle has mutable access to the\n // waker field per rule 2 in task/mod.rs; therefore, skip step (i)\n // of rule 5 and try to store the provided waker in the waker field.\n set_join_waker(header, trailer, waker.clone(), snapshot)\n };\n\n match res {\n Ok(_) => return false,\n Err(snapshot) => {\n assert!(snapshot.is_complete());\n }\n }\n }\n true\n}\n\nfn set_join_waker(\n header: &Header,\n trailer: &Trailer,\n waker: Waker,\n snapshot: Snapshot,\n) -> Result {\n assert!(snapshot.is_join_interested());\n assert!(!snapshot.is_join_waker_set());\n\n // Safety: Only the `JoinHandle` may set the `waker` field. When\n // `JOIN_INTEREST` is **not** set, nothing else will touch the field.\n unsafe {\n trailer.set_waker(Some(waker));\n }\n\n // Update the `JoinWaker` state accordingly\n let res = header.state.set_join_waker();\n\n // If the state could not be updated, then clear the join waker\n if res.is_err() {\n unsafe {\n trailer.set_waker(None);\n }\n }\n\n res\n}\n\nenum PollFuture {\n Complete,\n Notified,\n Done,\n Dealloc,\n}\n\n/// Cancels the task and store the appropriate error in the stage field.\nfn cancel_task(core: &Core) {\n // Drop the future from a panic guard.\n let res = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n core.drop_future_or_output();\n }));\n\n core.store_output(Err(panic_result_to_join_error(core.task_id, res)));\n}\n\nfn panic_result_to_join_error(\n task_id: Id,\n res: Result<(), Box>,\n) -> JoinError {\n match res {\n Ok(()) => JoinError::cancelled(task_id),\n Err(panic) => JoinError::panic(task_id, panic),\n }\n}\n\n/// Polls the future. If the future completes, the output is written to the\n/// stage field.\nfn poll_future(core: &Core, cx: Context<'_>) -> Poll<()> {\n // Poll the future.\n let output = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n struct Guard<'a, T: Future, S: Schedule> {\n core: &'a Core,\n }\n impl<'a, T: Future, S: Schedule> Drop for Guard<'a, T, S> {\n fn drop(&mut self) {\n // If the future panics on poll, we drop it inside the panic\n // guard.\n self.core.drop_future_or_output();\n }\n }\n let guard = Guard { core };\n let res = guard.core.poll(cx);\n mem::forget(guard);\n res\n }));\n\n // Prepare output for being placed in the core stage.\n let output = match output {\n Ok(Poll::Pending) => return Poll::Pending,\n Ok(Poll::Ready(output)) => Ok(output),\n Err(panic) => Err(panic_to_error(&core.scheduler, core.task_id, panic)),\n };\n\n // Catch and ignore panics if the future panics on drop.\n let res = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n core.store_output(output);\n }));\n\n if res.is_err() {\n core.scheduler.unhandled_panic();\n }\n\n Poll::Ready(())\n}\n\n#[cold]\nfn panic_to_error(\n scheduler: &S,\n task_id: Id,\n panic: Box,\n) -> JoinError {\n scheduler.unhandled_panic();\n JoinError::panic(task_id, panic)\n}\n" }, { "path": "tokio/src/runtime/task/id.rs", "content": "use crate::runtime::context;\n\nuse std::{fmt, num::NonZeroU64};\n\n/// An opaque ID that uniquely identifies a task relative to all other currently\n/// running tasks.\n///\n/// A task's ID may be re-used for another task only once *both* of the\n/// following happen:\n/// 1. The task itself exits.\n/// 2. There is no active [`JoinHandle`] associated with this task.\n///\n/// A [`JoinHandle`] is considered active in the following situations:\n/// - You are explicitly holding a [`JoinHandle`], [`AbortHandle`], or\n/// `tokio_util::task::AbortOnDropHandle`.\n/// - The task is being tracked by a [`JoinSet`] or `tokio_util::task::JoinMap`.\n///\n/// # Notes\n///\n/// - Task IDs are *not* sequential, and do not indicate the order in which\n/// tasks are spawned, what runtime a task is spawned on, or any other data.\n/// - The task ID of the currently running task can be obtained from inside the\n/// task via the [`task::try_id()`](crate::task::try_id()) and\n/// [`task::id()`](crate::task::id()) functions and from outside the task via\n/// the [`JoinHandle::id()`](crate::task::JoinHandle::id()) function.\n///\n/// [`JoinHandle`]: crate::task::JoinHandle\n/// [`AbortHandle`]: crate::task::AbortHandle\n/// [`JoinSet`]: crate::task::JoinSet\n#[cfg_attr(docsrs, doc(cfg(all(feature = \"rt\"))))]\n#[derive(Clone, Copy, Debug, Hash, Eq, PartialEq, PartialOrd, Ord)]\npub struct Id(pub(crate) NonZeroU64);\n\n/// Returns the [`Id`] of the currently running task.\n///\n/// # Panics\n///\n/// This function panics if called from outside a task. Please note that calls\n/// to `block_on` do not have task IDs, so the method will panic if called from\n/// within a call to `block_on`. For a version of this function that doesn't\n/// panic, see [`task::try_id()`](crate::runtime::task::try_id()).\n///\n/// [task ID]: crate::task::Id\n#[track_caller]\npub fn id() -> Id {\n context::current_task_id().expect(\"Can't get a task id when not inside a task\")\n}\n\n/// Returns the [`Id`] of the currently running task, or `None` if called outside\n/// of a task.\n///\n/// This function is similar to [`task::id()`](crate::runtime::task::id()), except\n/// that it returns `None` rather than panicking if called outside of a task\n/// context.\n///\n/// [task ID]: crate::task::Id\n#[track_caller]\npub fn try_id() -> Option {\n context::current_task_id()\n}\n\nimpl fmt::Display for Id {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.0.fmt(f)\n }\n}\n\nimpl Id {\n pub(crate) fn next() -> Self {\n use crate::loom::sync::atomic::{AtomicU64, Ordering::Relaxed};\n\n #[cfg(all(test, loom))]\n crate::loom::lazy_static! {\n static ref NEXT_ID: AtomicU64 = AtomicU64::new(1);\n }\n\n #[cfg(not(all(test, loom)))]\n static NEXT_ID: AtomicU64 = AtomicU64::new(1);\n\n loop {\n let id = NEXT_ID.fetch_add(1, Relaxed);\n if let Some(id) = NonZeroU64::new(id) {\n return Self(id);\n }\n }\n }\n\n pub(crate) fn as_u64(&self) -> u64 {\n self.0.get()\n }\n}\n" }, { "path": "tokio/src/runtime/task/join.rs", "content": "use crate::runtime::task::{AbortHandle, Header, RawTask};\n\nuse std::fmt;\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::panic::{RefUnwindSafe, UnwindSafe};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll, Waker};\n\ncfg_rt! {\n /// An owned permission to join on a task (await its termination).\n ///\n /// This can be thought of as the equivalent of [`std::thread::JoinHandle`]\n /// for a Tokio task rather than a thread. Note that the background task\n /// associated with this `JoinHandle` started running immediately when you\n /// called spawn, even if you have not yet awaited the `JoinHandle`.\n ///\n /// A `JoinHandle` *detaches* the associated task when it is dropped, which\n /// means that there is no longer any handle to the task, and no way to `join`\n /// on it.\n ///\n /// This `struct` is created by the [`task::spawn`] and [`task::spawn_blocking`]\n /// functions.\n ///\n /// It is guaranteed that the destructor of the spawned task has finished\n /// before task completion is observed via `JoinHandle` `await`,\n /// [`JoinHandle::is_finished`] or [`AbortHandle::is_finished`].\n ///\n /// # Cancel safety\n ///\n /// The `&mut JoinHandle` type is cancel safe. If it is used as the event\n /// in a `tokio::select!` statement and some other branch completes first,\n /// then it is guaranteed that the output of the task is not lost.\n ///\n /// If a `JoinHandle` is dropped, then the task continues running in the\n /// background and its return value is lost.\n ///\n /// # Examples\n ///\n /// Creation from [`task::spawn`]:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn doc() {\n /// let join_handle: task::JoinHandle<_> = task::spawn(async {\n /// // some work here\n /// });\n /// # }\n /// ```\n ///\n /// Creation from [`task::spawn_blocking`]:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn doc() {\n /// let join_handle: task::JoinHandle<_> = task::spawn_blocking(|| {\n /// // some blocking work here\n /// });\n /// # }\n /// ```\n ///\n /// The generic parameter `T` in `JoinHandle` is the return type of the spawned task.\n /// If the return value is an `i32`, the join handle has type `JoinHandle`:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn doc() {\n /// let join_handle: task::JoinHandle = task::spawn(async {\n /// 5 + 3\n /// });\n /// # }\n ///\n /// ```\n ///\n /// If the task does not have a return value, the join handle has type `JoinHandle<()>`:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn doc() {\n /// let join_handle: task::JoinHandle<()> = task::spawn(async {\n /// println!(\"I return nothing.\");\n /// });\n /// # }\n /// ```\n ///\n /// Note that `handle.await` doesn't give you the return type directly. It is wrapped in a\n /// `Result` because panics in the spawned task are caught by Tokio. The `?` operator has\n /// to be double chained to extract the returned value:\n ///\n /// ```\n /// use tokio::task;\n /// use std::io;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> io::Result<()> {\n /// let join_handle: task::JoinHandle> = tokio::spawn(async {\n /// Ok(5 + 3)\n /// });\n ///\n /// let result = join_handle.await??;\n /// assert_eq!(result, 8);\n /// Ok(())\n /// # }\n /// ```\n ///\n /// If the task panics, the error is a [`JoinError`] that contains the panic:\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::task;\n /// use std::io;\n /// use std::panic;\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let join_handle: task::JoinHandle> = tokio::spawn(async {\n /// panic!(\"boom\");\n /// });\n ///\n /// let err = join_handle.await.unwrap_err();\n /// assert!(err.is_panic());\n /// Ok(())\n /// }\n /// # }\n /// ```\n /// Child being detached and outliving its parent:\n ///\n /// ```no_run\n /// use tokio::task;\n /// use tokio::time;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let original_task = task::spawn(async {\n /// let _detached_task = task::spawn(async {\n /// // Here we sleep to make sure that the first task returns before.\n /// time::sleep(Duration::from_millis(10)).await;\n /// // This will be called, even though the JoinHandle is dropped.\n /// println!(\"♫ Still alive ♫\");\n /// });\n /// });\n ///\n /// original_task.await.expect(\"The task being joined has panicked\");\n /// println!(\"Original task is joined.\");\n ///\n /// // We make sure that the new task has time to run, before the main\n /// // task returns.\n ///\n /// time::sleep(Duration::from_millis(1000)).await;\n /// # }\n /// ```\n ///\n /// [`task::spawn`]: crate::task::spawn()\n /// [`task::spawn_blocking`]: crate::task::spawn_blocking\n /// [`std::thread::JoinHandle`]: std::thread::JoinHandle\n /// [`JoinError`]: crate::task::JoinError\n pub struct JoinHandle {\n raw: RawTask,\n _p: PhantomData,\n }\n}\n\nunsafe impl Send for JoinHandle {}\nunsafe impl Sync for JoinHandle {}\n\nimpl UnwindSafe for JoinHandle {}\nimpl RefUnwindSafe for JoinHandle {}\n\nimpl JoinHandle {\n pub(super) fn new(raw: RawTask) -> JoinHandle {\n JoinHandle {\n raw,\n _p: PhantomData,\n }\n }\n\n /// Abort the task associated with the handle.\n ///\n /// Awaiting a cancelled task might complete as usual if the task was\n /// already completed at the time it was cancelled, but most likely it\n /// will fail with a [cancelled] `JoinError`.\n ///\n /// Be aware that tasks spawned using [`spawn_blocking`] cannot be aborted\n /// because they are not async. If you call `abort` on a `spawn_blocking`\n /// task, then this *will not have any effect*, and the task will continue\n /// running normally. The exception is if the task has not started running\n /// yet; in that case, calling `abort` may prevent the task from starting.\n ///\n /// See also [the module level docs] for more information on cancellation.\n ///\n /// ```rust\n /// use tokio::time;\n ///\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// # async fn main() {\n /// let mut handles = Vec::new();\n ///\n /// handles.push(tokio::spawn(async {\n /// time::sleep(time::Duration::from_secs(10)).await;\n /// true\n /// }));\n ///\n /// handles.push(tokio::spawn(async {\n /// time::sleep(time::Duration::from_secs(10)).await;\n /// false\n /// }));\n ///\n /// for handle in &handles {\n /// handle.abort();\n /// }\n ///\n /// for handle in handles {\n /// assert!(handle.await.unwrap_err().is_cancelled());\n /// }\n /// # }\n /// ```\n ///\n /// [cancelled]: method@super::error::JoinError::is_cancelled\n /// [the module level docs]: crate::task#cancellation\n /// [`spawn_blocking`]: crate::task::spawn_blocking\n pub fn abort(&self) {\n self.raw.remote_abort();\n }\n\n /// Checks if the task associated with this `JoinHandle` has finished.\n ///\n /// Please note that this method can return `false` even if [`abort`] has been\n /// called on the task. This is because the cancellation process may take\n /// some time, and this method does not return `true` until it has\n /// completed.\n ///\n /// ```rust\n /// use tokio::time;\n ///\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// # async fn main() {\n /// let handle1 = tokio::spawn(async {\n /// // do some stuff here\n /// });\n /// let handle2 = tokio::spawn(async {\n /// // do some other stuff here\n /// time::sleep(time::Duration::from_secs(10)).await;\n /// });\n /// // Wait for the task to finish\n /// handle2.abort();\n /// time::sleep(time::Duration::from_secs(1)).await;\n /// assert!(handle1.is_finished());\n /// assert!(handle2.is_finished());\n /// # }\n /// ```\n /// [`abort`]: method@JoinHandle::abort\n pub fn is_finished(&self) -> bool {\n let state = self.raw.header().state.load();\n state.is_complete()\n }\n\n /// Set the waker that is notified when the task completes.\n pub(crate) fn set_join_waker(&mut self, waker: &Waker) {\n if self.raw.try_set_join_waker(waker) {\n // In this case the task has already completed. We wake the waker immediately.\n waker.wake_by_ref();\n }\n }\n\n /// Returns a new `AbortHandle` that can be used to remotely abort this task.\n ///\n /// Awaiting a task cancelled by the `AbortHandle` might complete as usual if the task was\n /// already completed at the time it was cancelled, but most likely it\n /// will fail with a [cancelled] `JoinError`.\n ///\n /// ```rust\n /// use tokio::{time, task};\n ///\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// # async fn main() {\n /// let mut handles = Vec::new();\n ///\n /// handles.push(tokio::spawn(async {\n /// time::sleep(time::Duration::from_secs(10)).await;\n /// true\n /// }));\n ///\n /// handles.push(tokio::spawn(async {\n /// time::sleep(time::Duration::from_secs(10)).await;\n /// false\n /// }));\n ///\n /// let abort_handles: Vec = handles.iter().map(|h| h.abort_handle()).collect();\n ///\n /// for handle in abort_handles {\n /// handle.abort();\n /// }\n ///\n /// for handle in handles {\n /// assert!(handle.await.unwrap_err().is_cancelled());\n /// }\n /// # }\n /// ```\n /// [cancelled]: method@super::error::JoinError::is_cancelled\n #[must_use = \"abort handles do nothing unless `.abort` is called\"]\n pub fn abort_handle(&self) -> AbortHandle {\n self.raw.ref_inc();\n AbortHandle::new(self.raw)\n }\n\n /// Returns a [task ID] that uniquely identifies this task relative to other\n /// currently spawned tasks.\n ///\n /// [task ID]: crate::task::Id\n pub fn id(&self) -> super::Id {\n // Safety: The header pointer is valid.\n unsafe { Header::get_id(self.raw.header_ptr()) }\n }\n}\n\nimpl Unpin for JoinHandle {}\n\nimpl Future for JoinHandle {\n type Output = super::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n ready!(crate::trace::trace_leaf(cx));\n let mut ret = Poll::Pending;\n\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n // Try to read the task output. If the task is not yet complete, the\n // waker is stored and is notified once the task does complete.\n //\n // The function must go via the vtable, which requires erasing generic\n // types. To do this, the function \"return\" is placed on the stack\n // **before** calling the function and is passed into the function using\n // `*mut ()`.\n //\n // Safety:\n //\n // The type of `T` must match the task's output type.\n unsafe {\n self.raw.try_read_output(&mut ret, cx.waker());\n }\n\n if ret.is_ready() {\n coop.made_progress();\n }\n\n ret\n }\n}\n\nimpl Drop for JoinHandle {\n fn drop(&mut self) {\n if self.raw.state().drop_join_handle_fast().is_ok() {\n return;\n }\n\n self.raw.drop_join_handle_slow();\n }\n}\n\nimpl fmt::Debug for JoinHandle\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n // Safety: The header pointer is valid.\n let id_ptr = unsafe { Header::get_id_ptr(self.raw.header_ptr()) };\n let id = unsafe { id_ptr.as_ref() };\n fmt.debug_struct(\"JoinHandle\").field(\"id\", id).finish()\n }\n}\n" }, { "path": "tokio/src/runtime/task/list.rs", "content": "//! This module has containers for storing the tasks spawned on a scheduler. The\n//! `OwnedTasks` container is thread-safe but can only store tasks that\n//! implement Send. The `LocalOwnedTasks` container is not thread safe, but can\n//! store non-Send tasks.\n//!\n//! The collections can be closed to prevent adding new tasks during shutdown of\n//! the scheduler with the collection.\n\nuse crate::future::Future;\nuse crate::loom::cell::UnsafeCell;\nuse crate::runtime::task::{JoinHandle, LocalNotified, Notified, Schedule, SpawnLocation, Task};\nuse crate::util::linked_list::{Link, LinkedList};\nuse crate::util::sharded_list;\n\nuse crate::loom::sync::atomic::{AtomicBool, Ordering};\nuse std::marker::PhantomData;\nuse std::num::NonZeroU64;\n\n// The id from the module below is used to verify whether a given task is stored\n// in this OwnedTasks, or some other task. The counter starts at one so we can\n// use `None` for tasks not owned by any list.\n//\n// The safety checks in this file can technically be violated if the counter is\n// overflown, but the checks are not supposed to ever fail unless there is a\n// bug in Tokio, so we accept that certain bugs would not be caught if the two\n// mixed up runtimes happen to have the same id.\n\ncfg_has_atomic_u64! {\n use std::sync::atomic::AtomicU64;\n\n static NEXT_OWNED_TASKS_ID: AtomicU64 = AtomicU64::new(1);\n\n fn get_next_id() -> NonZeroU64 {\n loop {\n let id = NEXT_OWNED_TASKS_ID.fetch_add(1, Ordering::Relaxed);\n if let Some(id) = NonZeroU64::new(id) {\n return id;\n }\n }\n }\n}\n\ncfg_not_has_atomic_u64! {\n use std::sync::atomic::AtomicU32;\n\n static NEXT_OWNED_TASKS_ID: AtomicU32 = AtomicU32::new(1);\n\n fn get_next_id() -> NonZeroU64 {\n loop {\n let id = NEXT_OWNED_TASKS_ID.fetch_add(1, Ordering::Relaxed);\n if let Some(id) = NonZeroU64::new(u64::from(id)) {\n return id;\n }\n }\n }\n}\n\npub(crate) struct OwnedTasks {\n list: List,\n pub(crate) id: NonZeroU64,\n closed: AtomicBool,\n}\n\ntype List = sharded_list::ShardedList, as Link>::Target>;\n\npub(crate) struct LocalOwnedTasks {\n inner: UnsafeCell>,\n pub(crate) id: NonZeroU64,\n _not_send_or_sync: PhantomData<*const ()>,\n}\n\nstruct OwnedTasksInner {\n list: LinkedList, as Link>::Target>,\n closed: bool,\n}\n\nimpl OwnedTasks {\n pub(crate) fn new(num_cores: usize) -> Self {\n let shard_size = Self::gen_shared_list_size(num_cores);\n Self {\n list: List::new(shard_size),\n closed: AtomicBool::new(false),\n id: get_next_id(),\n }\n }\n\n /// Binds the provided task to this `OwnedTasks` instance. This fails if the\n /// `OwnedTasks` has been closed.\n pub(crate) fn bind(\n &self,\n task: T,\n scheduler: S,\n id: super::Id,\n spawned_at: SpawnLocation,\n ) -> (JoinHandle, Option>)\n where\n S: Schedule,\n T: Future + Send + 'static,\n T::Output: Send + 'static,\n {\n let (task, notified, join) = super::new_task(task, scheduler, id, spawned_at);\n let notified = unsafe { self.bind_inner(task, notified) };\n (join, notified)\n }\n\n /// Bind a task that isn't safe to transfer across thread boundaries.\n ///\n /// # Safety\n ///\n /// Only use this in `LocalRuntime` where the task cannot move\n pub(crate) unsafe fn bind_local(\n &self,\n task: T,\n scheduler: S,\n id: super::Id,\n spawned_at: SpawnLocation,\n ) -> (JoinHandle, Option>)\n where\n S: Schedule,\n T: Future + 'static,\n T::Output: 'static,\n {\n let (task, notified, join) = super::new_task(task, scheduler, id, spawned_at);\n let notified = unsafe { self.bind_inner(task, notified) };\n (join, notified)\n }\n\n /// The part of `bind` that's the same for every type of future.\n unsafe fn bind_inner(&self, task: Task, notified: Notified) -> Option>\n where\n S: Schedule,\n {\n unsafe {\n // safety: We just created the task, so we have exclusive access\n // to the field.\n task.header().set_owner_id(self.id);\n }\n\n let shard = self.list.lock_shard(&task);\n // Check the closed flag in the lock for ensuring all that tasks\n // will shut down after the OwnedTasks has been closed.\n if self.closed.load(Ordering::Acquire) {\n drop(shard);\n task.shutdown();\n return None;\n }\n shard.push(task);\n Some(notified)\n }\n\n /// Asserts that the given task is owned by this `OwnedTasks` and convert it to\n /// a `LocalNotified`, giving the thread permission to poll this task.\n #[inline]\n pub(crate) fn assert_owner(&self, task: Notified) -> LocalNotified {\n debug_assert_eq!(task.header().get_owner_id(), Some(self.id));\n // safety: All tasks bound to this OwnedTasks are Send, so it is safe\n // to poll it on this thread no matter what thread we are on.\n LocalNotified {\n task: task.0,\n _not_send: PhantomData,\n }\n }\n\n /// Shuts down all tasks in the collection. This call also closes the\n /// collection, preventing new items from being added.\n ///\n /// The parameter start determines which shard this method will start at.\n /// Using different values for each worker thread reduces contention.\n pub(crate) fn close_and_shutdown_all(&self, start: usize)\n where\n S: Schedule,\n {\n self.closed.store(true, Ordering::Release);\n for i in start..self.get_shard_size() + start {\n loop {\n let task = self.list.pop_back(i);\n match task {\n Some(task) => {\n task.shutdown();\n }\n None => break,\n }\n }\n }\n }\n\n #[inline]\n pub(crate) fn get_shard_size(&self) -> usize {\n self.list.shard_size()\n }\n\n pub(crate) fn num_alive_tasks(&self) -> usize {\n self.list.len()\n }\n\n cfg_unstable_metrics! {\n cfg_64bit_metrics! {\n pub(crate) fn spawned_tasks_count(&self) -> u64 {\n self.list.added()\n }\n }\n }\n\n pub(crate) fn remove(&self, task: &Task) -> Option> {\n // If the task's owner ID is `None` then it is not part of any list and\n // doesn't need removing.\n let task_id = task.header().get_owner_id()?;\n\n assert_eq!(task_id, self.id);\n\n // safety: We just checked that the provided task is not in some other\n // linked list.\n unsafe { self.list.remove(task.header_ptr()) }\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.list.is_empty()\n }\n\n /// Generates the size of the sharded list based on the number of worker threads.\n ///\n /// The sharded lock design can effectively alleviate\n /// lock contention performance problems caused by high concurrency.\n ///\n /// However, as the number of shards increases, the memory continuity between\n /// nodes in the intrusive linked list will diminish. Furthermore,\n /// the construction time of the sharded list will also increase with a higher number of shards.\n ///\n /// Due to the above reasons, we set a maximum value for the shared list size,\n /// denoted as `MAX_SHARED_LIST_SIZE`.\n fn gen_shared_list_size(num_cores: usize) -> usize {\n const MAX_SHARED_LIST_SIZE: usize = 1 << 16;\n usize::min(MAX_SHARED_LIST_SIZE, num_cores.next_power_of_two() * 4)\n }\n}\n\ncfg_taskdump! {\n impl OwnedTasks {\n /// Locks the tasks, and calls `f` on an iterator over them.\n pub(crate) fn for_each(&self, f: F)\n where\n F: FnMut(&Task),\n {\n self.list.for_each(f);\n }\n }\n}\n\nimpl LocalOwnedTasks {\n pub(crate) fn new() -> Self {\n Self {\n inner: UnsafeCell::new(OwnedTasksInner {\n list: LinkedList::new(),\n closed: false,\n }),\n id: get_next_id(),\n _not_send_or_sync: PhantomData,\n }\n }\n\n pub(crate) fn bind(\n &self,\n task: T,\n scheduler: S,\n id: super::Id,\n spawned_at: SpawnLocation,\n ) -> (JoinHandle, Option>)\n where\n S: Schedule,\n T: Future + 'static,\n T::Output: 'static,\n {\n let (task, notified, join) = super::new_task(task, scheduler, id, spawned_at);\n\n unsafe {\n // safety: We just created the task, so we have exclusive access\n // to the field.\n task.header().set_owner_id(self.id);\n }\n\n if self.is_closed() {\n drop(notified);\n task.shutdown();\n (join, None)\n } else {\n self.with_inner(|inner| {\n inner.list.push_front(task);\n });\n (join, Some(notified))\n }\n }\n\n /// Shuts down all tasks in the collection. This call also closes the\n /// collection, preventing new items from being added.\n pub(crate) fn close_and_shutdown_all(&self)\n where\n S: Schedule,\n {\n self.with_inner(|inner| inner.closed = true);\n\n while let Some(task) = self.with_inner(|inner| inner.list.pop_back()) {\n task.shutdown();\n }\n }\n\n pub(crate) fn remove(&self, task: &Task) -> Option> {\n // If the task's owner ID is `None` then it is not part of any list and\n // doesn't need removing.\n let task_id = task.header().get_owner_id()?;\n\n assert_eq!(task_id, self.id);\n\n self.with_inner(|inner|\n // safety: We just checked that the provided task is not in some\n // other linked list.\n unsafe { inner.list.remove(task.header_ptr()) })\n }\n\n /// Asserts that the given task is owned by this `LocalOwnedTasks` and convert\n /// it to a `LocalNotified`, giving the thread permission to poll this task.\n #[inline]\n pub(crate) fn assert_owner(&self, task: Notified) -> LocalNotified {\n assert_eq!(task.header().get_owner_id(), Some(self.id));\n\n // safety: The task was bound to this LocalOwnedTasks, and the\n // LocalOwnedTasks is not Send or Sync, so we are on the right thread\n // for polling this task.\n LocalNotified {\n task: task.0,\n _not_send: PhantomData,\n }\n }\n\n #[inline]\n fn with_inner(&self, f: F) -> T\n where\n F: FnOnce(&mut OwnedTasksInner) -> T,\n {\n // safety: This type is not Sync, so concurrent calls of this method\n // can't happen. Furthermore, all uses of this method in this file make\n // sure that they don't call `with_inner` recursively.\n self.inner.with_mut(|ptr| unsafe { f(&mut *ptr) })\n }\n\n pub(crate) fn is_closed(&self) -> bool {\n self.with_inner(|inner| inner.closed)\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.with_inner(|inner| inner.list.is_empty())\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n // This test may run in parallel with other tests, so we only test that ids\n // come in increasing order.\n #[test]\n fn test_id_not_broken() {\n let mut last_id = get_next_id();\n\n for _ in 0..1000 {\n let next_id = get_next_id();\n assert!(last_id < next_id);\n last_id = next_id;\n }\n }\n}\n" }, { "path": "tokio/src/runtime/task/mod.rs", "content": "//! The task module.\n//!\n//! The task module contains the code that manages spawned tasks and provides a\n//! safe API for the rest of the runtime to use. Each task in a runtime is\n//! stored in an `OwnedTasks` or `LocalOwnedTasks` object.\n//!\n//! # Task reference types\n//!\n//! A task is usually referenced by multiple handles, and there are several\n//! types of handles.\n//!\n//! * `OwnedTask` - tasks stored in an `OwnedTasks` or `LocalOwnedTasks` are of this\n//! reference type.\n//!\n//! * `JoinHandle` - each task has a `JoinHandle` that allows access to the output\n//! of the task.\n//!\n//! * `Waker` - every waker for a task has this reference type. There can be any\n//! number of waker references.\n//!\n//! * `Notified` - tracks whether the task is notified.\n//!\n//! * `Unowned` - this task reference type is used for tasks not stored in any\n//! runtime. Mainly used for blocking tasks, but also in tests.\n//!\n//! The task uses a reference count to keep track of how many active references\n//! exist. The `Unowned` reference type takes up two ref-counts. All other\n//! reference types take up a single ref-count.\n//!\n//! Besides the waker type, each task has at most one of each reference type.\n//!\n//! # State\n//!\n//! The task stores its state in an atomic `usize` with various bitfields for the\n//! necessary information. The state has the following bitfields:\n//!\n//! * `RUNNING` - Tracks whether the task is currently being polled or cancelled.\n//! This bit functions as a lock around the task.\n//!\n//! * `COMPLETE` - Is one once the future has fully completed and has been\n//! dropped. Never unset once set. Never set together with RUNNING.\n//!\n//! * `NOTIFIED` - Tracks whether a Notified object currently exists.\n//!\n//! * `CANCELLED` - Is set to one for tasks that should be cancelled as soon as\n//! possible. May take any value for completed tasks.\n//!\n//! * `JOIN_INTEREST` - Is set to one if there exists a `JoinHandle`.\n//!\n//! * `JOIN_WAKER` - Acts as an access control bit for the join handle waker. The\n//! protocol for its usage is described below.\n//!\n//! The rest of the bits are used for the ref-count.\n//!\n//! # Fields in the task\n//!\n//! The task has various fields. This section describes how and when it is safe\n//! to access a field.\n//!\n//! * The state field is accessed with atomic instructions.\n//!\n//! * The `OwnedTask` reference has exclusive access to the `owned` field.\n//!\n//! * The Notified reference has exclusive access to the `queue_next` field.\n//!\n//! * The `owner_id` field can be set as part of construction of the task, but\n//! is otherwise immutable and anyone can access the field immutably without\n//! synchronization.\n//!\n//! * If COMPLETE is one, then the `JoinHandle` has exclusive access to the\n//! stage field. If COMPLETE is zero, then the RUNNING bitfield functions as\n//! a lock for the stage field, and it can be accessed only by the thread\n//! that set RUNNING to one.\n//!\n//! * The waker field may be concurrently accessed by different threads: in one\n//! thread the runtime may complete a task and *read* the waker field to\n//! invoke the waker, and in another thread the task's `JoinHandle` may be\n//! polled, and if the task hasn't yet completed, the `JoinHandle` may *write*\n//! a waker to the waker field. The `JOIN_WAKER` bit ensures safe access by\n//! multiple threads to the waker field using the following rules:\n//!\n//! 1. `JOIN_WAKER` is initialized to zero.\n//!\n//! 2. If `JOIN_WAKER` is zero, then the `JoinHandle` has exclusive (mutable)\n//! access to the waker field.\n//!\n//! 3. If `JOIN_WAKER` is one, then the `JoinHandle` has shared (read-only)\n//! access to the waker field.\n//!\n//! 4. If `JOIN_WAKER` is one and COMPLETE is one, then the runtime has shared\n//! (read-only) access to the waker field.\n//!\n//! 5. If the `JoinHandle` needs to write to the waker field, then the\n//! `JoinHandle` needs to (i) successfully set `JOIN_WAKER` to zero if it is\n//! not already zero to gain exclusive access to the waker field per rule\n//! 2, (ii) write a waker, and (iii) successfully set `JOIN_WAKER` to one.\n//! If the `JoinHandle` unsets `JOIN_WAKER` in the process of being dropped\n//! to clear the waker field, only steps (i) and (ii) are relevant.\n//!\n//! 6. The `JoinHandle` can change `JOIN_WAKER` only if COMPLETE is zero (i.e.\n//! the task hasn't yet completed). The runtime can change `JOIN_WAKER` only\n//! if COMPLETE is one.\n//!\n//! 7. If `JOIN_INTEREST` is zero and COMPLETE is one, then the runtime has\n//! exclusive (mutable) access to the waker field. This might happen if the\n//! `JoinHandle` gets dropped right after the task completes and the runtime\n//! sets the `COMPLETE` bit. In this case the runtime needs the mutable access\n//! to the waker field to drop it.\n//!\n//! Rule 6 implies that the steps (i) or (iii) of rule 5 may fail due to a\n//! race. If step (i) fails, then the attempt to write a waker is aborted. If\n//! step (iii) fails because COMPLETE is set to one by another thread after\n//! step (i), then the waker field is cleared. Once COMPLETE is one (i.e.\n//! task has completed), the `JoinHandle` will not modify `JOIN_WAKER`. After the\n//! runtime sets COMPLETE to one, it invokes the waker if there is one so in this\n//! case when a task completes the `JOIN_WAKER` bit implicates to the runtime\n//! whether it should invoke the waker or not. After the runtime is done with\n//! using the waker during task completion, it unsets the `JOIN_WAKER` bit to give\n//! the `JoinHandle` exclusive access again so that it is able to drop the waker\n//! at a later point.\n//!\n//! All other fields are immutable and can be accessed immutably without\n//! synchronization by anyone.\n//!\n//! # Safety\n//!\n//! This section goes through various situations and explains why the API is\n//! safe in that situation.\n//!\n//! ## Polling or dropping the future\n//!\n//! Any mutable access to the future happens after obtaining a lock by modifying\n//! the RUNNING field, so exclusive access is ensured.\n//!\n//! When the task completes, exclusive access to the output is transferred to\n//! the `JoinHandle`. If the `JoinHandle` is already dropped when the transition to\n//! complete happens, the thread performing that transition retains exclusive\n//! access to the output and should immediately drop it.\n//!\n//! ## Non-Send futures\n//!\n//! If a future is not Send, then it is bound to a `LocalOwnedTasks`. The future\n//! will only ever be polled or dropped given a `LocalNotified` or inside a call\n//! to `LocalOwnedTasks::shutdown_all`. In either case, it is guaranteed that the\n//! future is on the right thread.\n//!\n//! If the task is never removed from the `LocalOwnedTasks`, then it is leaked, so\n//! there is no risk that the task is dropped on some other thread when the last\n//! ref-count drops.\n//!\n//! ## Non-Send output\n//!\n//! When a task completes, the output is placed in the stage of the task. Then,\n//! a transition that sets COMPLETE to true is performed, and the value of\n//! `JOIN_INTEREST` when this transition happens is read.\n//!\n//! If `JOIN_INTEREST` is zero when the transition to COMPLETE happens, then the\n//! output is immediately dropped.\n//!\n//! If `JOIN_INTEREST` is one when the transition to COMPLETE happens, then the\n//! `JoinHandle` is responsible for cleaning up the output. If the output is not\n//! Send, then this happens:\n//!\n//! 1. The output is created on the thread that the future was polled on. Since\n//! only non-Send futures can have non-Send output, the future was polled on\n//! the thread that the future was spawned from.\n//! 2. Since `JoinHandle` is not Send if Output is not Send, the\n//! `JoinHandle` is also on the thread that the future was spawned from.\n//! 3. Thus, the `JoinHandle` will not move the output across threads when it\n//! takes or drops the output.\n//!\n//! ## Recursive poll/shutdown\n//!\n//! Calling poll from inside a shutdown call or vice-versa is not prevented by\n//! the API exposed by the task module, so this has to be safe. In either case,\n//! the lock in the RUNNING bitfield makes the inner call return immediately. If\n//! the inner call is a `shutdown` call, then the CANCELLED bit is set, and the\n//! poll call will notice it when the poll finishes, and the task is cancelled\n//! at that point.\n\nmod core;\nuse self::core::Cell;\nuse self::core::Header;\n\nmod error;\npub use self::error::JoinError;\n\nmod harness;\nuse self::harness::Harness;\n\nmod id;\npub use id::{id, try_id, Id};\n\n#[cfg(feature = \"rt\")]\nmod abort;\nmod join;\n\n#[cfg(feature = \"rt\")]\npub use self::abort::AbortHandle;\n\npub use self::join::JoinHandle;\n\nmod list;\npub(crate) use self::list::{LocalOwnedTasks, OwnedTasks};\n\nmod raw;\npub(crate) use self::raw::RawTask;\n\nmod state;\nuse self::state::State;\n\nmod waker;\n\npub(crate) use self::spawn_location::SpawnLocation;\n\ncfg_taskdump! {\n pub(crate) mod trace;\n}\n\nuse crate::future::Future;\nuse crate::util::linked_list;\nuse crate::util::sharded_list;\n\nuse crate::runtime::TaskCallback;\nuse std::marker::PhantomData;\nuse std::panic::Location;\nuse std::ptr::NonNull;\nuse std::{fmt, mem};\n\n/// An owned handle to the task, tracked by ref count.\n#[repr(transparent)]\npub(crate) struct Task {\n raw: RawTask,\n _p: PhantomData,\n}\n\nunsafe impl Send for Task {}\nunsafe impl Sync for Task {}\n\n/// A task was notified.\n#[repr(transparent)]\npub(crate) struct Notified(Task);\n\nimpl Notified {\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n #[inline]\n pub(crate) fn task_meta<'meta>(&self) -> crate::runtime::TaskMeta<'meta> {\n self.0.task_meta()\n }\n}\n\n// safety: This type cannot be used to touch the task without first verifying\n// that the value is on a thread where it is safe to poll the task.\nunsafe impl Send for Notified {}\nunsafe impl Sync for Notified {}\n\n/// A non-Send variant of Notified with the invariant that it is on a thread\n/// where it is safe to poll it.\n#[repr(transparent)]\npub(crate) struct LocalNotified {\n task: Task,\n _not_send: PhantomData<*const ()>,\n}\n\nimpl LocalNotified {\n #[cfg(tokio_unstable)]\n #[inline]\n pub(crate) fn task_meta<'meta>(&self) -> crate::runtime::TaskMeta<'meta> {\n self.task.task_meta()\n }\n}\n\n/// A task that is not owned by any `OwnedTasks`. Used for blocking tasks.\n/// This type holds two ref-counts.\npub(crate) struct UnownedTask {\n raw: RawTask,\n _p: PhantomData,\n}\n\n// safety: This type can only be created given a Send task.\nunsafe impl Send for UnownedTask {}\nunsafe impl Sync for UnownedTask {}\n\n/// Task result sent back.\npub(crate) type Result = std::result::Result;\n\n/// Hooks for scheduling tasks which are needed in the task harness.\n#[derive(Clone)]\npub(crate) struct TaskHarnessScheduleHooks {\n pub(crate) task_terminate_callback: Option,\n}\n\npub(crate) trait Schedule: Sync + Sized + 'static {\n /// The task has completed work and is ready to be released. The scheduler\n /// should release it immediately and return it. The task module will batch\n /// the ref-dec with setting other options.\n ///\n /// If the scheduler has already released the task, then None is returned.\n fn release(&self, task: &Task) -> Option>;\n\n /// Schedule the task\n fn schedule(&self, task: Notified);\n\n fn hooks(&self) -> TaskHarnessScheduleHooks;\n\n /// Schedule the task to run in the near future, yielding the thread to\n /// other tasks.\n fn yield_now(&self, task: Notified) {\n self.schedule(task);\n }\n\n /// Polling the task resulted in a panic. Should the runtime shutdown?\n fn unhandled_panic(&self) {\n // By default, do nothing. This maintains the 1.0 behavior.\n }\n}\n\ncfg_rt! {\n /// This is the constructor for a new task. Three references to the task are\n /// created. The first task reference is usually put into an `OwnedTasks`\n /// immediately. The Notified is sent to the scheduler as an ordinary\n /// notification.\n fn new_task(\n task: T,\n scheduler: S,\n id: Id,\n spawned_at: SpawnLocation,\n ) -> (Task, Notified, JoinHandle)\n where\n S: Schedule,\n T: Future + 'static,\n T::Output: 'static,\n {\n let raw = RawTask::new::(\n task,\n scheduler,\n id,\n spawned_at,\n );\n let task = Task {\n raw,\n _p: PhantomData,\n };\n let notified = Notified(Task {\n raw,\n _p: PhantomData,\n });\n let join = JoinHandle::new(raw);\n\n (task, notified, join)\n }\n\n /// Creates a new task with an associated join handle. This method is used\n /// only when the task is not going to be stored in an `OwnedTasks` list.\n ///\n /// Currently only blocking tasks use this method.\n pub(crate) fn unowned(\n task: T,\n scheduler: S,\n id: Id,\n spawned_at: SpawnLocation,\n ) -> (UnownedTask, JoinHandle)\n where\n S: Schedule,\n T: Send + Future + 'static,\n T::Output: Send + 'static,\n {\n let (task, notified, join) = new_task(\n task,\n scheduler,\n id,\n spawned_at,\n );\n\n // This transfers the ref-count of task and notified into an UnownedTask.\n // This is valid because an UnownedTask holds two ref-counts.\n let unowned = UnownedTask {\n raw: task.raw,\n _p: PhantomData,\n };\n std::mem::forget(task);\n std::mem::forget(notified);\n\n (unowned, join)\n }\n}\n\nimpl Task {\n unsafe fn new(raw: RawTask) -> Task {\n Task {\n raw,\n _p: PhantomData,\n }\n }\n\n /// # Safety\n ///\n /// `ptr` must be a valid pointer to a [`Header`].\n unsafe fn from_raw(ptr: NonNull
) -> Task {\n unsafe { Task::new(RawTask::from_raw(ptr)) }\n }\n\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n pub(super) fn as_raw(&self) -> RawTask {\n self.raw\n }\n\n fn header(&self) -> &Header {\n self.raw.header()\n }\n\n fn header_ptr(&self) -> NonNull
{\n self.raw.header_ptr()\n }\n\n /// Returns a [task ID] that uniquely identifies this task relative to other\n /// currently spawned tasks.\n ///\n /// [task ID]: crate::task::Id\n #[cfg(tokio_unstable)]\n pub(crate) fn id(&self) -> crate::task::Id {\n // Safety: The header pointer is valid.\n unsafe { Header::get_id(self.raw.header_ptr()) }\n }\n\n #[cfg(tokio_unstable)]\n pub(crate) fn spawned_at(&self) -> &'static Location<'static> {\n // Safety: The header pointer is valid.\n unsafe { Header::get_spawn_location(self.raw.header_ptr()) }\n }\n\n // Explicit `'task` and `'meta` lifetimes are necessary here, as otherwise,\n // the compiler infers the lifetimes to be the same, and considers the task\n // to be borrowed for the lifetime of the returned `TaskMeta`.\n #[cfg(tokio_unstable)]\n pub(crate) fn task_meta<'meta>(&self) -> crate::runtime::TaskMeta<'meta> {\n crate::runtime::TaskMeta {\n id: self.id(),\n spawned_at: self.spawned_at().into(),\n _phantom: PhantomData,\n }\n }\n\n cfg_taskdump! {\n /// Notify the task for task dumping.\n ///\n /// Returns `None` if the task has already been notified.\n pub(super) fn notify_for_tracing(&self) -> Option> {\n if self.as_raw().state().transition_to_notified_for_tracing() {\n // SAFETY: `transition_to_notified_for_tracing` increments the\n // refcount.\n Some(unsafe { Notified(Task::new(self.raw)) })\n } else {\n None\n }\n }\n\n }\n}\n\nimpl Notified {\n fn header(&self) -> &Header {\n self.0.header()\n }\n\n #[cfg(tokio_unstable)]\n #[allow(dead_code)]\n pub(crate) fn task_id(&self) -> crate::task::Id {\n self.0.id()\n }\n}\n\nimpl Notified {\n /// # Safety\n ///\n /// [`RawTask::ptr`] must be a valid pointer to a [`Header`].\n pub(crate) unsafe fn from_raw(ptr: RawTask) -> Notified {\n Notified(unsafe { Task::new(ptr) })\n }\n}\n\nimpl Notified {\n pub(crate) fn into_raw(self) -> RawTask {\n let raw = self.0.raw;\n mem::forget(self);\n raw\n }\n}\n\nimpl Task {\n /// Preemptively cancels the task as part of the shutdown process.\n pub(crate) fn shutdown(self) {\n let raw = self.raw;\n mem::forget(self);\n raw.shutdown();\n }\n}\n\nimpl LocalNotified {\n /// Runs the task.\n pub(crate) fn run(self) {\n let raw = self.task.raw;\n mem::forget(self);\n raw.poll();\n }\n}\n\nimpl UnownedTask {\n // Used in test of the inject queue.\n #[cfg(test)]\n #[cfg_attr(target_family = \"wasm\", allow(dead_code))]\n pub(super) fn into_notified(self) -> Notified {\n Notified(self.into_task())\n }\n\n fn into_task(self) -> Task {\n // Convert into a task.\n let task = Task {\n raw: self.raw,\n _p: PhantomData,\n };\n mem::forget(self);\n\n // Drop a ref-count since an UnownedTask holds two.\n task.header().state.ref_dec();\n\n task\n }\n\n pub(crate) fn run(self) {\n let raw = self.raw;\n mem::forget(self);\n\n // Transfer one ref-count to a Task object.\n let task = Task:: {\n raw,\n _p: PhantomData,\n };\n\n // Use the other ref-count to poll the task.\n raw.poll();\n // Decrement our extra ref-count\n drop(task);\n }\n\n pub(crate) fn shutdown(self) {\n self.into_task().shutdown();\n }\n}\n\nimpl Drop for Task {\n fn drop(&mut self) {\n // Decrement the ref count\n if self.header().state.ref_dec() {\n // Deallocate if this is the final ref count\n self.raw.dealloc();\n }\n }\n}\n\nimpl Drop for UnownedTask {\n fn drop(&mut self) {\n // Decrement the ref count\n if self.raw.header().state.ref_dec_twice() {\n // Deallocate if this is the final ref count\n self.raw.dealloc();\n }\n }\n}\n\nimpl fmt::Debug for Task {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"Task({:p})\", self.header())\n }\n}\n\nimpl fmt::Debug for Notified {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"task::Notified({:p})\", self.0.header())\n }\n}\n\n/// # Safety\n///\n/// Tasks are pinned.\nunsafe impl linked_list::Link for Task {\n type Handle = Task;\n type Target = Header;\n\n fn as_raw(handle: &Task) -> NonNull
{\n handle.raw.header_ptr()\n }\n\n unsafe fn from_raw(ptr: NonNull
) -> Task {\n unsafe { Task::from_raw(ptr) }\n }\n\n unsafe fn pointers(target: NonNull
) -> NonNull> {\n unsafe { self::core::Trailer::addr_of_owned(Header::get_trailer(target)) }\n }\n}\n\n/// # Safety\n///\n/// The id of a task is never changed after creation of the task, so the return value of\n/// `get_shard_id` will not change. (The cast may throw away the upper 32 bits of the task id, but\n/// the shard id still won't change from call to call.)\nunsafe impl sharded_list::ShardedListItem for Task {\n unsafe fn get_shard_id(target: NonNull) -> usize {\n // SAFETY: The caller guarantees that `target` points at a valid task.\n let task_id = unsafe { Header::get_id(target) };\n task_id.0.get() as usize\n }\n}\n\n/// Wrapper around [`std::panic::Location`] that's conditionally compiled out\n/// when `tokio_unstable` is not enabled.\n#[cfg(tokio_unstable)]\nmod spawn_location {\n\n use std::panic::Location;\n\n #[derive(Copy, Clone)]\n pub(crate) struct SpawnLocation(pub &'static Location<'static>);\n\n impl From<&'static Location<'static>> for SpawnLocation {\n fn from(location: &'static Location<'static>) -> Self {\n Self(location)\n }\n }\n}\n\n#[cfg(not(tokio_unstable))]\nmod spawn_location {\n use std::panic::Location;\n\n #[derive(Copy, Clone)]\n pub(crate) struct SpawnLocation();\n\n impl From<&'static Location<'static>> for SpawnLocation {\n fn from(_: &'static Location<'static>) -> Self {\n Self()\n }\n }\n\n #[cfg(test)]\n #[test]\n fn spawn_location_is_zero_sized() {\n assert_eq!(std::mem::size_of::(), 0);\n }\n}\n\nimpl SpawnLocation {\n #[track_caller]\n #[inline]\n pub(crate) fn capture() -> Self {\n Self::from(Location::caller())\n }\n}\n" }, { "path": "tokio/src/runtime/task/raw.rs", "content": "// It doesn't make sense to enforce `unsafe_op_in_unsafe_fn` for this module because\n//\n// * This module is doing the low-level task management that requires tons of unsafe\n// operations.\n// * Excessive `unsafe {}` blocks hurt readability significantly.\n// TODO: replace with `#[expect(unsafe_op_in_unsafe_fn)]` after bumpping\n// the MSRV to 1.81.0.\n#![allow(unsafe_op_in_unsafe_fn)]\n\nuse crate::future::Future;\nuse crate::runtime::task::core::{Core, Trailer};\nuse crate::runtime::task::{Cell, Harness, Header, Id, Schedule, State};\n#[cfg(tokio_unstable)]\nuse std::panic::Location;\nuse std::ptr::NonNull;\nuse std::task::{Poll, Waker};\n\n/// Raw task handle\n#[derive(Clone)]\npub(crate) struct RawTask {\n ptr: NonNull
,\n}\n\npub(super) struct Vtable {\n /// Polls the future.\n pub(super) poll: unsafe fn(NonNull
),\n\n /// Schedules the task for execution on the runtime.\n pub(super) schedule: unsafe fn(NonNull
),\n\n /// Deallocates the memory.\n pub(super) dealloc: unsafe fn(NonNull
),\n\n /// Reads the task output, if complete.\n pub(super) try_read_output: unsafe fn(NonNull
, *mut (), &Waker),\n\n /// The join handle has been dropped.\n pub(super) drop_join_handle_slow: unsafe fn(NonNull
),\n\n /// An abort handle has been dropped.\n pub(super) drop_abort_handle: unsafe fn(NonNull
),\n\n /// Scheduler is being shutdown.\n pub(super) shutdown: unsafe fn(NonNull
),\n\n /// The number of bytes that the `trailer` field is offset from the header.\n pub(super) trailer_offset: usize,\n\n /// The number of bytes that the `scheduler` field is offset from the header.\n pub(super) scheduler_offset: usize,\n\n /// The number of bytes that the `id` field is offset from the header.\n pub(super) id_offset: usize,\n\n /// The number of bytes that the `spawned_at` field is offset from the header.\n #[cfg(tokio_unstable)]\n pub(super) spawn_location_offset: usize,\n}\n\n/// Get the vtable for the requested `T` and `S` generics.\npub(super) fn vtable() -> &'static Vtable {\n &Vtable {\n poll: poll::,\n schedule: schedule::,\n dealloc: dealloc::,\n try_read_output: try_read_output::,\n drop_join_handle_slow: drop_join_handle_slow::,\n drop_abort_handle: drop_abort_handle::,\n shutdown: shutdown::,\n trailer_offset: OffsetHelper::::TRAILER_OFFSET,\n scheduler_offset: OffsetHelper::::SCHEDULER_OFFSET,\n id_offset: OffsetHelper::::ID_OFFSET,\n #[cfg(tokio_unstable)]\n spawn_location_offset: OffsetHelper::::SPAWN_LOCATION_OFFSET,\n }\n}\n\n/// Calling `get_trailer_offset` directly in vtable doesn't work because it\n/// prevents the vtable from being promoted to a static reference.\n///\n/// See this thread for more info:\n/// \nstruct OffsetHelper(T, S);\nimpl OffsetHelper {\n // Pass `size_of`/`align_of` as arguments rather than calling them directly\n // inside `get_trailer_offset` because trait bounds on generic parameters\n // of const fn are unstable on our MSRV.\n const TRAILER_OFFSET: usize = get_trailer_offset(\n std::mem::size_of::
(),\n std::mem::size_of::>(),\n std::mem::align_of::>(),\n std::mem::align_of::(),\n );\n\n // The `scheduler` is the first field of `Core`, so it has the same\n // offset as `Core`.\n const SCHEDULER_OFFSET: usize = get_core_offset(\n std::mem::size_of::
(),\n std::mem::align_of::>(),\n );\n\n const ID_OFFSET: usize = get_id_offset(\n std::mem::size_of::
(),\n std::mem::align_of::>(),\n std::mem::size_of::(),\n std::mem::align_of::(),\n );\n\n #[cfg(tokio_unstable)]\n const SPAWN_LOCATION_OFFSET: usize = get_spawn_location_offset(\n std::mem::size_of::
(),\n std::mem::align_of::>(),\n std::mem::size_of::(),\n std::mem::align_of::(),\n std::mem::size_of::(),\n std::mem::align_of::<&'static Location<'static>>(),\n );\n}\n\n/// Compute the offset of the `Trailer` field in `Cell` using the\n/// `#[repr(C)]` algorithm.\n///\n/// Pseudo-code for the `#[repr(C)]` algorithm can be found here:\n/// \nconst fn get_trailer_offset(\n header_size: usize,\n core_size: usize,\n core_align: usize,\n trailer_align: usize,\n) -> usize {\n let mut offset = header_size;\n\n let core_misalign = offset % core_align;\n if core_misalign > 0 {\n offset += core_align - core_misalign;\n }\n offset += core_size;\n\n let trailer_misalign = offset % trailer_align;\n if trailer_misalign > 0 {\n offset += trailer_align - trailer_misalign;\n }\n\n offset\n}\n\n/// Compute the offset of the `Core` field in `Cell` using the\n/// `#[repr(C)]` algorithm.\n///\n/// Pseudo-code for the `#[repr(C)]` algorithm can be found here:\n/// \nconst fn get_core_offset(header_size: usize, core_align: usize) -> usize {\n let mut offset = header_size;\n\n let core_misalign = offset % core_align;\n if core_misalign > 0 {\n offset += core_align - core_misalign;\n }\n\n offset\n}\n\n/// Compute the offset of the `Id` field in `Cell` using the\n/// `#[repr(C)]` algorithm.\n///\n/// Pseudo-code for the `#[repr(C)]` algorithm can be found here:\n/// \nconst fn get_id_offset(\n header_size: usize,\n core_align: usize,\n scheduler_size: usize,\n id_align: usize,\n) -> usize {\n let mut offset = get_core_offset(header_size, core_align);\n offset += scheduler_size;\n\n let id_misalign = offset % id_align;\n if id_misalign > 0 {\n offset += id_align - id_misalign;\n }\n\n offset\n}\n\n/// Compute the offset of the `&'static Location<'static>` field in `Cell`\n/// using the `#[repr(C)]` algorithm.\n///\n/// Pseudo-code for the `#[repr(C)]` algorithm can be found here:\n/// \n#[cfg(tokio_unstable)]\nconst fn get_spawn_location_offset(\n header_size: usize,\n core_align: usize,\n scheduler_size: usize,\n id_align: usize,\n id_size: usize,\n spawn_location_align: usize,\n) -> usize {\n let mut offset = get_id_offset(header_size, core_align, scheduler_size, id_align);\n offset += id_size;\n\n let spawn_location_misalign = offset % spawn_location_align;\n if spawn_location_misalign > 0 {\n offset += spawn_location_align - spawn_location_misalign;\n }\n\n offset\n}\n\nimpl RawTask {\n pub(super) fn new(\n task: T,\n scheduler: S,\n id: Id,\n _spawned_at: super::SpawnLocation,\n ) -> RawTask\n where\n T: Future,\n S: Schedule,\n {\n let ptr = Box::into_raw(Cell::<_, S>::new(\n task,\n scheduler,\n State::new(),\n id,\n #[cfg(tokio_unstable)]\n _spawned_at.0,\n ));\n let ptr = unsafe { NonNull::new_unchecked(ptr.cast()) };\n\n RawTask { ptr }\n }\n\n /// # Safety\n ///\n /// `ptr` must be a valid pointer to a [`Header`].\n pub(super) unsafe fn from_raw(ptr: NonNull
) -> RawTask {\n RawTask { ptr }\n }\n\n pub(super) fn header_ptr(&self) -> NonNull
{\n self.ptr\n }\n\n pub(super) fn trailer_ptr(&self) -> NonNull {\n unsafe { Header::get_trailer(self.ptr) }\n }\n\n /// Returns a reference to the task's header.\n pub(super) fn header(&self) -> &Header {\n unsafe { self.ptr.as_ref() }\n }\n\n /// Returns a reference to the task's trailer.\n pub(super) fn trailer(&self) -> &Trailer {\n unsafe { &*self.trailer_ptr().as_ptr() }\n }\n\n /// Returns a reference to the task's state.\n pub(super) fn state(&self) -> &State {\n &self.header().state\n }\n\n /// Safety: mutual exclusion is required to call this function.\n pub(crate) fn poll(self) {\n let vtable = self.header().vtable;\n unsafe { (vtable.poll)(self.ptr) }\n }\n\n pub(super) fn schedule(self) {\n let vtable = self.header().vtable;\n unsafe { (vtable.schedule)(self.ptr) }\n }\n\n pub(super) fn dealloc(self) {\n let vtable = self.header().vtable;\n unsafe {\n (vtable.dealloc)(self.ptr);\n }\n }\n\n /// Safety: `dst` must be a `*mut Poll>` where `T`\n /// is the future stored by the task.\n pub(super) unsafe fn try_read_output(self, dst: *mut Poll>, waker: &Waker) {\n let vtable = self.header().vtable;\n (vtable.try_read_output)(self.ptr, dst as *mut _, waker);\n }\n\n pub(super) fn drop_join_handle_slow(self) {\n let vtable = self.header().vtable;\n unsafe { (vtable.drop_join_handle_slow)(self.ptr) }\n }\n\n pub(super) fn drop_abort_handle(self) {\n let vtable = self.header().vtable;\n unsafe { (vtable.drop_abort_handle)(self.ptr) }\n }\n\n pub(super) fn shutdown(self) {\n let vtable = self.header().vtable;\n unsafe { (vtable.shutdown)(self.ptr) }\n }\n\n /// Increment the task's reference count.\n ///\n /// Currently, this is used only when creating an `AbortHandle`.\n pub(super) fn ref_inc(self) {\n self.header().state.ref_inc();\n }\n\n /// Get the queue-next pointer\n ///\n /// This is for usage by the injection queue\n ///\n /// Safety: make sure only one queue uses this and access is synchronized.\n pub(crate) unsafe fn get_queue_next(self) -> Option {\n self.header()\n .queue_next\n .with(|ptr| *ptr)\n .map(|p| RawTask::from_raw(p))\n }\n\n /// Sets the queue-next pointer\n ///\n /// This is for usage by the injection queue\n ///\n /// Safety: make sure only one queue uses this and access is synchronized.\n pub(crate) unsafe fn set_queue_next(self, val: Option) {\n self.header().set_next(val.map(|task| task.ptr));\n }\n}\n\nimpl Copy for RawTask {}\n\nunsafe fn poll(ptr: NonNull
) {\n let harness = Harness::::from_raw(ptr);\n harness.poll();\n}\n\nunsafe fn schedule(ptr: NonNull
) {\n use crate::runtime::task::{Notified, Task};\n\n let scheduler = Header::get_scheduler::(ptr);\n scheduler\n .as_ref()\n .schedule(Notified(Task::from_raw(ptr.cast())));\n}\n\nunsafe fn dealloc(ptr: NonNull
) {\n let harness = Harness::::from_raw(ptr);\n harness.dealloc();\n}\n\nunsafe fn try_read_output(\n ptr: NonNull
,\n dst: *mut (),\n waker: &Waker,\n) {\n let out = &mut *(dst as *mut Poll>);\n\n let harness = Harness::::from_raw(ptr);\n harness.try_read_output(out, waker);\n}\n\nunsafe fn drop_join_handle_slow(ptr: NonNull
) {\n let harness = Harness::::from_raw(ptr);\n harness.drop_join_handle_slow();\n}\n\nunsafe fn drop_abort_handle(ptr: NonNull
) {\n let harness = Harness::::from_raw(ptr);\n harness.drop_reference();\n}\n\nunsafe fn shutdown(ptr: NonNull
) {\n let harness = Harness::::from_raw(ptr);\n harness.shutdown();\n}\n" }, { "path": "tokio/src/runtime/task/state.rs", "content": "use crate::loom::sync::atomic::AtomicUsize;\n\nuse std::fmt;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Release};\n\npub(super) struct State {\n val: AtomicUsize,\n}\n\n/// Current state value.\n#[derive(Copy, Clone)]\npub(super) struct Snapshot(usize);\n\ntype UpdateResult = Result;\n\n/// The task is currently being run.\nconst RUNNING: usize = 0b0001;\n\n/// The task is complete.\n///\n/// Once this bit is set, it is never unset.\nconst COMPLETE: usize = 0b0010;\n\n/// Extracts the task's lifecycle value from the state.\nconst LIFECYCLE_MASK: usize = 0b11;\n\n/// Flag tracking if the task has been pushed into a run queue.\nconst NOTIFIED: usize = 0b100;\n\n/// The join handle is still around.\nconst JOIN_INTEREST: usize = 0b1_000;\n\n/// A join handle waker has been set.\nconst JOIN_WAKER: usize = 0b10_000;\n\n/// The task has been forcibly cancelled.\nconst CANCELLED: usize = 0b100_000;\n\n/// All bits.\nconst STATE_MASK: usize = LIFECYCLE_MASK | NOTIFIED | JOIN_INTEREST | JOIN_WAKER | CANCELLED;\n\n/// Bits used by the ref count portion of the state.\nconst REF_COUNT_MASK: usize = !STATE_MASK;\n\n/// Number of positions to shift the ref count.\nconst REF_COUNT_SHIFT: usize = REF_COUNT_MASK.count_zeros() as usize;\n\n/// One ref count.\nconst REF_ONE: usize = 1 << REF_COUNT_SHIFT;\n\n/// State a task is initialized with.\n///\n/// A task is initialized with three references:\n///\n/// * A reference that will be stored in an `OwnedTasks` or `LocalOwnedTasks`.\n/// * A reference that will be sent to the scheduler as an ordinary notification.\n/// * A reference for the `JoinHandle`.\n///\n/// As the task starts with a `JoinHandle`, `JOIN_INTEREST` is set.\n/// As the task starts with a `Notified`, `NOTIFIED` is set.\nconst INITIAL_STATE: usize = (REF_ONE * 3) | JOIN_INTEREST | NOTIFIED;\n\n#[must_use]\npub(super) enum TransitionToRunning {\n Success,\n Cancelled,\n Failed,\n Dealloc,\n}\n\n#[must_use]\npub(super) enum TransitionToIdle {\n Ok,\n OkNotified,\n OkDealloc,\n Cancelled,\n}\n\n#[must_use]\npub(super) enum TransitionToNotifiedByVal {\n DoNothing,\n Submit,\n Dealloc,\n}\n\n#[must_use]\npub(crate) enum TransitionToNotifiedByRef {\n DoNothing,\n Submit,\n}\n\n#[must_use]\npub(super) struct TransitionToJoinHandleDrop {\n pub(super) drop_waker: bool,\n pub(super) drop_output: bool,\n}\n\n/// All transitions are performed via RMW operations. This establishes an\n/// unambiguous modification order.\nimpl State {\n /// Returns a task's initial state.\n pub(super) fn new() -> State {\n // The raw task returned by this method has a ref-count of three. See\n // the comment on INITIAL_STATE for more.\n State {\n val: AtomicUsize::new(INITIAL_STATE),\n }\n }\n\n /// Loads the current state, establishes `Acquire` ordering.\n pub(super) fn load(&self) -> Snapshot {\n Snapshot(self.val.load(Acquire))\n }\n\n /// Attempts to transition the lifecycle to `Running`. This sets the\n /// notified bit to false so notifications during the poll can be detected.\n pub(super) fn transition_to_running(&self) -> TransitionToRunning {\n self.fetch_update_action(|mut next| {\n let action;\n assert!(next.is_notified());\n\n if !next.is_idle() {\n // This happens if the task is either currently running or if it\n // has already completed, e.g. if it was cancelled during\n // shutdown. Consume the ref-count and return.\n next.ref_dec();\n if next.ref_count() == 0 {\n action = TransitionToRunning::Dealloc;\n } else {\n action = TransitionToRunning::Failed;\n }\n } else {\n // We are able to lock the RUNNING bit.\n next.set_running();\n next.unset_notified();\n\n if next.is_cancelled() {\n action = TransitionToRunning::Cancelled;\n } else {\n action = TransitionToRunning::Success;\n }\n }\n (action, Some(next))\n })\n }\n\n /// Transitions the task from `Running` -> `Idle`.\n ///\n /// The transition to `Idle` fails if the task has been flagged to be\n /// cancelled.\n pub(super) fn transition_to_idle(&self) -> TransitionToIdle {\n self.fetch_update_action(|curr| {\n assert!(curr.is_running());\n\n if curr.is_cancelled() {\n return (TransitionToIdle::Cancelled, None);\n }\n\n let mut next = curr;\n let action;\n next.unset_running();\n\n if !next.is_notified() {\n // Polling the future consumes the ref-count of the Notified.\n next.ref_dec();\n if next.ref_count() == 0 {\n action = TransitionToIdle::OkDealloc;\n } else {\n action = TransitionToIdle::Ok;\n }\n } else {\n // The caller will schedule a new notification, so we create a\n // new ref-count for the notification. Our own ref-count is kept\n // for now, and the caller will drop it shortly.\n next.ref_inc();\n action = TransitionToIdle::OkNotified;\n }\n\n (action, Some(next))\n })\n }\n\n /// Transitions the task from `Running` -> `Complete`.\n pub(super) fn transition_to_complete(&self) -> Snapshot {\n const DELTA: usize = RUNNING | COMPLETE;\n\n let prev = Snapshot(self.val.fetch_xor(DELTA, AcqRel));\n assert!(prev.is_running());\n assert!(!prev.is_complete());\n\n Snapshot(prev.0 ^ DELTA)\n }\n\n /// Transitions from `Complete` -> `Terminal`, decrementing the reference\n /// count the specified number of times.\n ///\n /// Returns true if the task should be deallocated.\n pub(super) fn transition_to_terminal(&self, count: usize) -> bool {\n let prev = Snapshot(self.val.fetch_sub(count * REF_ONE, AcqRel));\n assert!(\n prev.ref_count() >= count,\n \"current: {}, sub: {}\",\n prev.ref_count(),\n count\n );\n prev.ref_count() == count\n }\n\n /// Transitions the state to `NOTIFIED`.\n ///\n /// If no task needs to be submitted, a ref-count is consumed.\n ///\n /// If a task needs to be submitted, the ref-count is incremented for the\n /// new Notified.\n pub(super) fn transition_to_notified_by_val(&self) -> TransitionToNotifiedByVal {\n self.fetch_update_action(|mut snapshot| {\n let action;\n\n if snapshot.is_running() {\n // If the task is running, we mark it as notified, but we should\n // not submit anything as the thread currently running the\n // future is responsible for that.\n snapshot.set_notified();\n snapshot.ref_dec();\n\n // The thread that set the running bit also holds a ref-count.\n assert!(snapshot.ref_count() > 0);\n\n action = TransitionToNotifiedByVal::DoNothing;\n } else if snapshot.is_complete() || snapshot.is_notified() {\n // We do not need to submit any notifications, but we have to\n // decrement the ref-count.\n snapshot.ref_dec();\n\n if snapshot.ref_count() == 0 {\n action = TransitionToNotifiedByVal::Dealloc;\n } else {\n action = TransitionToNotifiedByVal::DoNothing;\n }\n } else {\n // We create a new notified that we can submit. The caller\n // retains ownership of the ref-count they passed in.\n snapshot.set_notified();\n snapshot.ref_inc();\n action = TransitionToNotifiedByVal::Submit;\n }\n\n (action, Some(snapshot))\n })\n }\n\n /// Transitions the state to `NOTIFIED`.\n pub(super) fn transition_to_notified_by_ref(&self) -> TransitionToNotifiedByRef {\n self.fetch_update_action(|mut snapshot| {\n if snapshot.is_complete() {\n // The complete state is final\n (TransitionToNotifiedByRef::DoNothing, None)\n } else if snapshot.is_notified() {\n // Even hough we have nothing to do in this branch,\n // wake_by_ref() should synchronize-with the task starting execution,\n // therefore we must use an Release store (with the same value),\n // to pair with the Acquire in transition_to_running.\n (TransitionToNotifiedByRef::DoNothing, Some(snapshot))\n } else if snapshot.is_running() {\n // If the task is running, we mark it as notified, but we should\n // not submit as the thread currently running the future is\n // responsible for that.\n snapshot.set_notified();\n (TransitionToNotifiedByRef::DoNothing, Some(snapshot))\n } else {\n // The task is idle and not notified. We should submit a\n // notification.\n snapshot.set_notified();\n snapshot.ref_inc();\n (TransitionToNotifiedByRef::Submit, Some(snapshot))\n }\n })\n }\n\n /// Transitions the state to `NOTIFIED`, unconditionally increasing the ref\n /// count.\n ///\n /// Returns `true` if the notified bit was transitioned from `0` to `1`;\n /// otherwise `false.`\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n ))]\n pub(super) fn transition_to_notified_for_tracing(&self) -> bool {\n self.fetch_update_action(|mut snapshot| {\n if snapshot.is_notified() {\n (false, None)\n } else {\n snapshot.set_notified();\n snapshot.ref_inc();\n (true, Some(snapshot))\n }\n })\n }\n\n /// Sets the cancelled bit and transitions the state to `NOTIFIED` if idle.\n ///\n /// Returns `true` if the task needs to be submitted to the pool for\n /// execution.\n pub(super) fn transition_to_notified_and_cancel(&self) -> bool {\n self.fetch_update_action(|mut snapshot| {\n if snapshot.is_cancelled() || snapshot.is_complete() {\n // Aborts to completed or cancelled tasks are no-ops.\n (false, None)\n } else if snapshot.is_running() {\n // If the task is running, we mark it as cancelled. The thread\n // running the task will notice the cancelled bit when it\n // stops polling and it will kill the task.\n //\n // The set_notified() call is not strictly necessary but it will\n // in some cases let a wake_by_ref call return without having\n // to perform a compare_exchange.\n snapshot.set_notified();\n snapshot.set_cancelled();\n (false, Some(snapshot))\n } else {\n // The task is idle. We set the cancelled and notified bits and\n // submit a notification if the notified bit was not already\n // set.\n snapshot.set_cancelled();\n if !snapshot.is_notified() {\n snapshot.set_notified();\n snapshot.ref_inc();\n (true, Some(snapshot))\n } else {\n (false, Some(snapshot))\n }\n }\n })\n }\n\n /// Sets the `CANCELLED` bit and attempts to transition to `Running`.\n ///\n /// Returns `true` if the transition to `Running` succeeded.\n pub(super) fn transition_to_shutdown(&self) -> bool {\n let mut prev = Snapshot(0);\n\n let _ = self.fetch_update(|mut snapshot| {\n prev = snapshot;\n\n if snapshot.is_idle() {\n snapshot.set_running();\n }\n\n // If the task was not idle, the thread currently running the task\n // will notice the cancelled bit and cancel it once the poll\n // completes.\n snapshot.set_cancelled();\n Some(snapshot)\n });\n\n prev.is_idle()\n }\n\n /// Optimistically tries to swap the state assuming the join handle is\n /// __immediately__ dropped on spawn.\n pub(super) fn drop_join_handle_fast(&self) -> Result<(), ()> {\n use std::sync::atomic::Ordering::Relaxed;\n\n // Relaxed is acceptable as if this function is called and succeeds,\n // then nothing has been done w/ the join handle.\n //\n // The moment the join handle is used (polled), the `JOIN_WAKER` flag is\n // set, at which point the CAS will fail.\n //\n // Given this, there is no risk if this operation is reordered.\n self.val\n .compare_exchange_weak(\n INITIAL_STATE,\n (INITIAL_STATE - REF_ONE) & !JOIN_INTEREST,\n Release,\n Relaxed,\n )\n .map(|_| ())\n .map_err(|_| ())\n }\n\n /// Unsets the `JOIN_INTEREST` flag. If `COMPLETE` is not set, the `JOIN_WAKER`\n /// flag is also unset.\n /// The returned `TransitionToJoinHandleDrop` indicates whether the `JoinHandle` should drop\n /// the output of the future or the join waker after the transition.\n pub(super) fn transition_to_join_handle_dropped(&self) -> TransitionToJoinHandleDrop {\n self.fetch_update_action(|mut snapshot| {\n assert!(snapshot.is_join_interested());\n\n let mut transition = TransitionToJoinHandleDrop {\n drop_waker: false,\n drop_output: false,\n };\n\n snapshot.unset_join_interested();\n\n if !snapshot.is_complete() {\n // If `COMPLETE` is unset we also unset `JOIN_WAKER` to give the\n // `JoinHandle` exclusive access to the waker following rule 6 in task/mod.rs.\n // The `JoinHandle` will drop the waker if it has exclusive access\n // to drop it.\n snapshot.unset_join_waker();\n } else {\n // If `COMPLETE` is set the task is completed so the `JoinHandle` is responsible\n // for dropping the output.\n transition.drop_output = true;\n }\n\n if !snapshot.is_join_waker_set() {\n // If the `JOIN_WAKER` bit is unset and the `JOIN_HANDLE` has exclusive access to\n // the join waker and should drop it following this transition.\n // This might happen in two situations:\n // 1. The task is not completed and we just unset the `JOIN_WAKer` above in this\n // function.\n // 2. The task is completed. In that case the `JOIN_WAKER` bit was already unset\n // by the runtime during completion.\n transition.drop_waker = true;\n }\n\n (transition, Some(snapshot))\n })\n }\n\n /// Sets the `JOIN_WAKER` bit.\n ///\n /// Returns `Ok` if the bit is set, `Err` otherwise. This operation fails if\n /// the task has completed.\n pub(super) fn set_join_waker(&self) -> UpdateResult {\n self.fetch_update(|curr| {\n assert!(curr.is_join_interested());\n assert!(!curr.is_join_waker_set());\n\n if curr.is_complete() {\n return None;\n }\n\n let mut next = curr;\n next.set_join_waker();\n\n Some(next)\n })\n }\n\n /// Unsets the `JOIN_WAKER` bit.\n ///\n /// Returns `Ok` has been unset, `Err` otherwise. This operation fails if\n /// the task has completed.\n pub(super) fn unset_waker(&self) -> UpdateResult {\n self.fetch_update(|curr| {\n assert!(curr.is_join_interested());\n\n if curr.is_complete() {\n return None;\n }\n\n // If the task is completed, this bit may have been unset by\n // `unset_waker_after_complete`.\n assert!(curr.is_join_waker_set());\n\n let mut next = curr;\n next.unset_join_waker();\n\n Some(next)\n })\n }\n\n /// Unsets the `JOIN_WAKER` bit unconditionally after task completion.\n ///\n /// This operation requires the task to be completed.\n pub(super) fn unset_waker_after_complete(&self) -> Snapshot {\n let prev = Snapshot(self.val.fetch_and(!JOIN_WAKER, AcqRel));\n assert!(prev.is_complete());\n assert!(prev.is_join_waker_set());\n Snapshot(prev.0 & !JOIN_WAKER)\n }\n\n pub(super) fn ref_inc(&self) {\n use std::process;\n use std::sync::atomic::Ordering::Relaxed;\n\n // Using a relaxed ordering is alright here, as knowledge of the\n // original reference prevents other threads from erroneously deleting\n // the object.\n //\n // As explained in the [Boost documentation][1], Increasing the\n // reference counter can always be done with memory_order_relaxed: New\n // references to an object can only be formed from an existing\n // reference, and passing an existing reference from one thread to\n // another must already provide any required synchronization.\n //\n // [1]: (www.boost.org/doc/libs/1_55_0/doc/html/atomic/usage_examples.html)\n let prev = self.val.fetch_add(REF_ONE, Relaxed);\n\n // If the reference count overflowed, abort.\n if prev > isize::MAX as usize {\n process::abort();\n }\n }\n\n /// Returns `true` if the task should be released.\n pub(super) fn ref_dec(&self) -> bool {\n let prev = Snapshot(self.val.fetch_sub(REF_ONE, AcqRel));\n assert!(prev.ref_count() >= 1);\n prev.ref_count() == 1\n }\n\n /// Returns `true` if the task should be released.\n pub(super) fn ref_dec_twice(&self) -> bool {\n let prev = Snapshot(self.val.fetch_sub(2 * REF_ONE, AcqRel));\n assert!(prev.ref_count() >= 2);\n prev.ref_count() == 2\n }\n\n fn fetch_update_action(&self, mut f: F) -> T\n where\n F: FnMut(Snapshot) -> (T, Option),\n {\n let mut curr = self.load();\n\n loop {\n let (output, next) = f(curr);\n let next = match next {\n Some(next) => next,\n None => return output,\n };\n\n let res = self.val.compare_exchange(curr.0, next.0, AcqRel, Acquire);\n\n match res {\n Ok(_) => return output,\n Err(actual) => curr = Snapshot(actual),\n }\n }\n }\n\n fn fetch_update(&self, mut f: F) -> Result\n where\n F: FnMut(Snapshot) -> Option,\n {\n let mut curr = self.load();\n\n loop {\n let next = match f(curr) {\n Some(next) => next,\n None => return Err(curr),\n };\n\n let res = self.val.compare_exchange(curr.0, next.0, AcqRel, Acquire);\n\n match res {\n Ok(_) => return Ok(next),\n Err(actual) => curr = Snapshot(actual),\n }\n }\n }\n}\n\n// ===== impl Snapshot =====\n\nimpl Snapshot {\n /// Returns `true` if the task is in an idle state.\n pub(super) fn is_idle(self) -> bool {\n self.0 & (RUNNING | COMPLETE) == 0\n }\n\n /// Returns `true` if the task has been flagged as notified.\n pub(super) fn is_notified(self) -> bool {\n self.0 & NOTIFIED == NOTIFIED\n }\n\n fn unset_notified(&mut self) {\n self.0 &= !NOTIFIED;\n }\n\n fn set_notified(&mut self) {\n self.0 |= NOTIFIED;\n }\n\n pub(super) fn is_running(self) -> bool {\n self.0 & RUNNING == RUNNING\n }\n\n fn set_running(&mut self) {\n self.0 |= RUNNING;\n }\n\n fn unset_running(&mut self) {\n self.0 &= !RUNNING;\n }\n\n pub(super) fn is_cancelled(self) -> bool {\n self.0 & CANCELLED == CANCELLED\n }\n\n fn set_cancelled(&mut self) {\n self.0 |= CANCELLED;\n }\n\n /// Returns `true` if the task's future has completed execution.\n pub(super) fn is_complete(self) -> bool {\n self.0 & COMPLETE == COMPLETE\n }\n\n pub(super) fn is_join_interested(self) -> bool {\n self.0 & JOIN_INTEREST == JOIN_INTEREST\n }\n\n fn unset_join_interested(&mut self) {\n self.0 &= !JOIN_INTEREST;\n }\n\n pub(super) fn is_join_waker_set(self) -> bool {\n self.0 & JOIN_WAKER == JOIN_WAKER\n }\n\n fn set_join_waker(&mut self) {\n self.0 |= JOIN_WAKER;\n }\n\n fn unset_join_waker(&mut self) {\n self.0 &= !JOIN_WAKER;\n }\n\n pub(super) fn ref_count(self) -> usize {\n (self.0 & REF_COUNT_MASK) >> REF_COUNT_SHIFT\n }\n\n fn ref_inc(&mut self) {\n assert!(self.0 <= isize::MAX as usize);\n self.0 += REF_ONE;\n }\n\n pub(super) fn ref_dec(&mut self) {\n assert!(self.ref_count() > 0);\n self.0 -= REF_ONE;\n }\n}\n\nimpl fmt::Debug for State {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n let snapshot = self.load();\n snapshot.fmt(fmt)\n }\n}\n\nimpl fmt::Debug for Snapshot {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Snapshot\")\n .field(\"is_running\", &self.is_running())\n .field(\"is_complete\", &self.is_complete())\n .field(\"is_notified\", &self.is_notified())\n .field(\"is_cancelled\", &self.is_cancelled())\n .field(\"is_join_interested\", &self.is_join_interested())\n .field(\"is_join_waker_set\", &self.is_join_waker_set())\n .field(\"ref_count\", &self.ref_count())\n .finish()\n }\n}\n" }, { "path": "tokio/src/runtime/task/trace/mod.rs", "content": "use crate::loom::sync::Arc;\nuse crate::runtime::context;\nuse crate::runtime::scheduler::{self, current_thread, Inject};\nuse crate::task::Id;\n\nuse backtrace::BacktraceFrame;\nuse std::cell::Cell;\nuse std::collections::VecDeque;\nuse std::ffi::c_void;\nuse std::fmt;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::ptr::{self, NonNull};\nuse std::task::{self, Poll};\n\nmod symbol;\nmod tree;\n\nuse symbol::Symbol;\nuse tree::Tree;\n\nuse super::{Notified, OwnedTasks, Schedule};\n\ntype Backtrace = Vec;\ntype SymbolTrace = Vec;\n\n/// The ambient backtracing context.\npub(crate) struct Context {\n /// The address of [`Trace::root`] establishes an upper unwinding bound on\n /// the backtraces in `Trace`.\n active_frame: Cell>>,\n /// The place to stash backtraces.\n collector: Cell>,\n}\n\n/// A [`Frame`] in an intrusive, doubly-linked tree of [`Frame`]s.\nstruct Frame {\n /// The location associated with this frame.\n inner_addr: *const c_void,\n\n /// The parent frame, if any.\n parent: Option>,\n}\n\n/// An tree execution trace.\n///\n/// Traces are captured with [`Trace::capture`], rooted with [`Trace::root`]\n/// and leaved with [`trace_leaf`].\n#[derive(Clone, Debug)]\npub(crate) struct Trace {\n // The linear backtraces that comprise this trace. These linear traces can\n // be re-knitted into a tree.\n backtraces: Vec,\n}\n\npin_project_lite::pin_project! {\n #[derive(Debug, Clone)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n /// A future wrapper that roots traces (captured with [`Trace::capture`]).\n pub struct Root {\n #[pin]\n future: T,\n }\n}\n\nconst FAIL_NO_THREAD_LOCAL: &str = \"The Tokio thread-local has been destroyed \\\n as part of shutting down the current \\\n thread, so collecting a taskdump is not \\\n possible.\";\n\nimpl Context {\n pub(crate) const fn new() -> Self {\n Context {\n active_frame: Cell::new(None),\n collector: Cell::new(None),\n }\n }\n\n /// SAFETY: Callers of this function must ensure that trace frames always\n /// form a valid linked list.\n unsafe fn try_with_current(f: F) -> Option\n where\n F: FnOnce(&Self) -> R,\n {\n unsafe { crate::runtime::context::with_trace(f) }\n }\n\n /// SAFETY: Callers of this function must ensure that trace frames always\n /// form a valid linked list.\n unsafe fn with_current_frame(f: F) -> R\n where\n F: FnOnce(&Cell>>) -> R,\n {\n unsafe {\n Self::try_with_current(|context| f(&context.active_frame)).expect(FAIL_NO_THREAD_LOCAL)\n }\n }\n\n fn with_current_collector(f: F) -> R\n where\n F: FnOnce(&Cell>) -> R,\n {\n // SAFETY: This call can only access the collector field, so it cannot\n // break the trace frame linked list.\n unsafe {\n Self::try_with_current(|context| f(&context.collector)).expect(FAIL_NO_THREAD_LOCAL)\n }\n }\n\n /// Produces `true` if the current task is being traced; otherwise false.\n pub(crate) fn is_tracing() -> bool {\n Self::with_current_collector(|maybe_collector| {\n let collector = maybe_collector.take();\n let result = collector.is_some();\n maybe_collector.set(collector);\n result\n })\n }\n}\n\nimpl Trace {\n /// Invokes `f`, returning both its result and the collection of backtraces\n /// captured at each sub-invocation of [`trace_leaf`].\n #[inline(never)]\n pub(crate) fn capture(f: F) -> (R, Trace)\n where\n F: FnOnce() -> R,\n {\n let collector = Trace { backtraces: vec![] };\n\n let previous = Context::with_current_collector(|current| current.replace(Some(collector)));\n\n let result = f();\n\n let collector =\n Context::with_current_collector(|current| current.replace(previous)).unwrap();\n\n (result, collector)\n }\n\n /// The root of a trace.\n #[inline(never)]\n pub(crate) fn root(future: F) -> Root {\n Root { future }\n }\n\n pub(crate) fn backtraces(&self) -> &[Backtrace] {\n &self.backtraces\n }\n}\n\n/// If this is a sub-invocation of [`Trace::capture`], capture a backtrace.\n///\n/// The captured backtrace will be returned by [`Trace::capture`].\n///\n/// Invoking this function does nothing when it is not a sub-invocation\n/// [`Trace::capture`].\n// This function is marked `#[inline(never)]` to ensure that it gets a distinct `Frame` in the\n// backtrace, below which frames should not be included in the backtrace (since they reflect the\n// internal implementation details of this crate).\n#[inline(never)]\npub(crate) fn trace_leaf(cx: &mut task::Context<'_>) -> Poll<()> {\n // Safety: We don't manipulate the current context's active frame.\n let did_trace = unsafe {\n Context::try_with_current(|context_cell| {\n if let Some(mut collector) = context_cell.collector.take() {\n let mut frames = vec![];\n let mut above_leaf = false;\n\n if let Some(active_frame) = context_cell.active_frame.get() {\n let active_frame = active_frame.as_ref();\n\n backtrace::trace(|frame| {\n let below_root = !ptr::eq(frame.symbol_address(), active_frame.inner_addr);\n\n // only capture frames above `Trace::leaf` and below\n // `Trace::root`.\n if above_leaf && below_root {\n frames.push(frame.to_owned().into());\n }\n\n if ptr::eq(frame.symbol_address(), trace_leaf as *const _) {\n above_leaf = true;\n }\n\n // only continue unwinding if we're below `Trace::root`\n below_root\n });\n }\n collector.backtraces.push(frames);\n context_cell.collector.set(Some(collector));\n true\n } else {\n false\n }\n })\n .unwrap_or(false)\n };\n\n if did_trace {\n // Use the same logic that `yield_now` uses to send out wakeups after\n // the task yields.\n context::with_scheduler(|scheduler| {\n if let Some(scheduler) = scheduler {\n match scheduler {\n scheduler::Context::CurrentThread(s) => s.defer.defer(cx.waker()),\n #[cfg(feature = \"rt-multi-thread\")]\n scheduler::Context::MultiThread(s) => s.defer.defer(cx.waker()),\n }\n }\n });\n\n Poll::Pending\n } else {\n Poll::Ready(())\n }\n}\n\nimpl fmt::Display for Trace {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n Tree::from_trace(self.clone()).fmt(f)\n }\n}\n\nfn defer R, R>(f: F) -> impl Drop {\n use std::mem::ManuallyDrop;\n\n struct Defer R, R>(ManuallyDrop);\n\n impl R, R> Drop for Defer {\n #[inline(always)]\n fn drop(&mut self) {\n unsafe {\n ManuallyDrop::take(&mut self.0)();\n }\n }\n }\n\n Defer(ManuallyDrop::new(f))\n}\n\nimpl Future for Root {\n type Output = T::Output;\n\n #[inline(never)]\n fn poll(self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll {\n // SAFETY: The context's current frame is restored to its original state\n // before `frame` is dropped.\n unsafe {\n let mut frame = Frame {\n inner_addr: Self::poll as *const c_void,\n parent: None,\n };\n\n Context::with_current_frame(|current| {\n frame.parent = current.take();\n current.set(Some(NonNull::from(&frame)));\n });\n\n let _restore = defer(|| {\n Context::with_current_frame(|current| {\n current.set(frame.parent);\n });\n });\n\n let this = self.project();\n this.future.poll(cx)\n }\n }\n}\n\n/// Trace and poll all tasks of the `current_thread` runtime.\npub(in crate::runtime) fn trace_current_thread(\n owned: &OwnedTasks>,\n local: &mut VecDeque>>,\n injection: &Inject>,\n) -> Vec<(Id, Trace)> {\n // clear the local and injection queues\n\n let mut dequeued = Vec::new();\n\n while let Some(task) = local.pop_back() {\n dequeued.push(task);\n }\n\n while let Some(task) = injection.pop() {\n dequeued.push(task);\n }\n\n // precondition: We have drained the tasks from the injection queue.\n trace_owned(owned, dequeued)\n}\n\ncfg_rt_multi_thread! {\n use crate::loom::sync::Mutex;\n use crate::runtime::scheduler::multi_thread;\n use crate::runtime::scheduler::multi_thread::Synced;\n use crate::runtime::scheduler::inject::Shared;\n\n /// Trace and poll all tasks of the `current_thread` runtime.\n ///\n /// ## Safety\n ///\n /// Must be called with the same `synced` that `injection` was created with.\n pub(in crate::runtime) unsafe fn trace_multi_thread(\n owned: &OwnedTasks>,\n local: &mut multi_thread::queue::Local>,\n synced: &Mutex,\n injection: &Shared>,\n ) -> Vec<(Id, Trace)> {\n let mut dequeued = Vec::new();\n\n // clear the local queue\n while let Some(notified) = local.pop() {\n dequeued.push(notified);\n }\n\n // clear the injection queue\n let mut synced = synced.lock();\n // Safety: exactly the same safety requirements as `trace_multi_thread` function.\n while let Some(notified) = unsafe { injection.pop(&mut synced.inject) } {\n dequeued.push(notified);\n }\n\n drop(synced);\n\n // precondition: we have drained the tasks from the local and injection\n // queues.\n trace_owned(owned, dequeued)\n }\n}\n\n/// Trace the `OwnedTasks`.\n///\n/// # Preconditions\n///\n/// This helper presumes exclusive access to each task. The tasks must not exist\n/// in any other queue.\nfn trace_owned(owned: &OwnedTasks, dequeued: Vec>) -> Vec<(Id, Trace)> {\n let mut tasks = dequeued;\n // Notify and trace all un-notified tasks. The dequeued tasks are already\n // notified and so do not need to be re-notified.\n owned.for_each(|task| {\n // Notify the task (and thus make it poll-able) and stash it. This fails\n // if the task is already notified. In these cases, we skip tracing the\n // task.\n if let Some(notified) = task.notify_for_tracing() {\n tasks.push(notified);\n }\n // We do not poll tasks here, since we hold a lock on `owned` and the\n // task may complete and need to remove itself from `owned`. Polling\n // such a task here would result in a deadlock.\n });\n\n tasks\n .into_iter()\n .map(|task| {\n let local_notified = owned.assert_owner(task);\n let id = local_notified.task.id();\n let ((), trace) = Trace::capture(|| local_notified.run());\n (id, trace)\n })\n .collect()\n}\n" }, { "path": "tokio/src/runtime/task/trace/symbol.rs", "content": "use backtrace::BacktraceSymbol;\nuse std::fmt;\nuse std::hash::{Hash, Hasher};\nuse std::ptr;\n\n/// A symbol in a backtrace.\n///\n/// This wrapper type serves two purposes. The first is that it provides a\n/// representation of a symbol that can be inserted into hashmaps and hashsets;\n/// the [`backtrace`] crate does not define [`Hash`], [`PartialEq`], or [`Eq`]\n/// on [`BacktraceSymbol`], and recommends that users define their own wrapper\n/// which implements these traits.\n///\n/// Second, this wrapper includes a `parent_hash` field that uniquely\n/// identifies this symbol's position in its trace. Otherwise, e.g., our code\n/// would not be able to distinguish between recursive calls of a function at\n/// different depths.\n#[derive(Clone)]\npub(super) struct Symbol {\n pub(super) symbol: BacktraceSymbol,\n pub(super) parent_hash: u64,\n}\n\nimpl Hash for Symbol {\n fn hash(&self, state: &mut H)\n where\n H: Hasher,\n {\n if let Some(name) = self.symbol.name() {\n name.as_bytes().hash(state);\n }\n\n if let Some(addr) = self.symbol.addr() {\n ptr::hash(addr, state);\n }\n\n self.symbol.filename().hash(state);\n self.symbol.lineno().hash(state);\n self.symbol.colno().hash(state);\n self.parent_hash.hash(state);\n }\n}\n\nimpl PartialEq for Symbol {\n fn eq(&self, other: &Self) -> bool {\n (self.parent_hash == other.parent_hash)\n && match (self.symbol.name(), other.symbol.name()) {\n (None, None) => true,\n (Some(lhs_name), Some(rhs_name)) => lhs_name.as_bytes() == rhs_name.as_bytes(),\n _ => false,\n }\n && match (self.symbol.addr(), other.symbol.addr()) {\n (None, None) => true,\n (Some(lhs_addr), Some(rhs_addr)) => ptr::eq(lhs_addr, rhs_addr),\n _ => false,\n }\n && (self.symbol.filename() == other.symbol.filename())\n && (self.symbol.lineno() == other.symbol.lineno())\n && (self.symbol.colno() == other.symbol.colno())\n }\n}\n\nimpl Eq for Symbol {}\n\nimpl fmt::Display for Symbol {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n if let Some(name) = self.symbol.name() {\n let name = name.to_string();\n let name = if let Some((name, _)) = name.rsplit_once(\"::\") {\n name\n } else {\n &name\n };\n fmt::Display::fmt(&name, f)?;\n }\n\n if let Some(filename) = self.symbol.filename() {\n f.write_str(\" at \")?;\n filename.to_string_lossy().fmt(f)?;\n if let Some(lineno) = self.symbol.lineno() {\n f.write_str(\":\")?;\n fmt::Display::fmt(&lineno, f)?;\n if let Some(colno) = self.symbol.colno() {\n f.write_str(\":\")?;\n fmt::Display::fmt(&colno, f)?;\n }\n }\n }\n\n Ok(())\n }\n}\n" }, { "path": "tokio/src/runtime/task/trace/tree.rs", "content": "use std::collections::{hash_map::DefaultHasher, HashMap, HashSet};\nuse std::fmt;\nuse std::hash::{Hash, Hasher};\n\nuse super::{Backtrace, Symbol, SymbolTrace, Trace};\n\n/// An adjacency list representation of an execution tree.\n///\n/// This tree provides a convenient intermediate representation for formatting\n/// [`Trace`] as a tree.\npub(super) struct Tree {\n /// The roots of the trees.\n ///\n /// There should only be one root, but the code is robust to multiple roots.\n roots: HashSet,\n\n /// The adjacency list of symbols in the execution tree(s).\n edges: HashMap>,\n}\n\nimpl Tree {\n /// Constructs a [`Tree`] from [`Trace`]\n pub(super) fn from_trace(trace: Trace) -> Self {\n let mut roots: HashSet = HashSet::default();\n let mut edges: HashMap> = HashMap::default();\n\n for trace in trace.backtraces {\n let trace = to_symboltrace(trace);\n\n if let Some(first) = trace.first() {\n roots.insert(first.to_owned());\n }\n\n let mut trace = trace.into_iter().peekable();\n while let Some(frame) = trace.next() {\n let subframes = edges.entry(frame).or_default();\n if let Some(subframe) = trace.peek() {\n subframes.insert(subframe.clone());\n }\n }\n }\n\n Tree { roots, edges }\n }\n\n /// Produces the sub-symbols of a given symbol.\n fn consequences(&self, frame: &Symbol) -> Option> {\n Some(self.edges.get(frame)?.iter())\n }\n\n /// Format this [`Tree`] as a textual tree.\n fn display(\n &self,\n f: &mut W,\n root: &Symbol,\n is_last: bool,\n prefix: &str,\n ) -> fmt::Result {\n let root_fmt = format!(\"{root}\");\n\n let current;\n let next;\n\n if is_last {\n current = format!(\"{prefix}└╼\\u{a0}{root_fmt}\");\n next = format!(\"{prefix}\\u{a0}\\u{a0}\\u{a0}\");\n } else {\n current = format!(\"{prefix}├╼\\u{a0}{root_fmt}\");\n next = format!(\"{prefix}│\\u{a0}\\u{a0}\");\n }\n\n write!(f, \"{}\", {\n let mut current = current.chars();\n current.next().unwrap();\n current.next().unwrap();\n ¤t.as_str()\n })?;\n\n if let Some(consequences) = self.consequences(root) {\n let len = consequences.len();\n for (i, consequence) in consequences.enumerate() {\n let is_last = i == len - 1;\n writeln!(f)?;\n self.display(f, consequence, is_last, &next)?;\n }\n }\n\n Ok(())\n }\n}\n\nimpl fmt::Display for Tree {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n for root in &self.roots {\n self.display(f, root, true, \" \")?;\n }\n Ok(())\n }\n}\n\n/// Resolve a sequence of [`backtrace::BacktraceFrame`]s into a sequence of\n/// [`Symbol`]s.\nfn to_symboltrace(backtrace: Backtrace) -> SymbolTrace {\n // Resolve the backtrace frames to symbols.\n let backtrace: Backtrace = {\n let mut backtrace = backtrace::Backtrace::from(backtrace);\n backtrace.resolve();\n backtrace.into()\n };\n\n // Accumulate the symbols in descending order into `symboltrace`.\n let mut symboltrace: SymbolTrace = vec![];\n let mut state = DefaultHasher::new();\n for frame in backtrace.into_iter().rev() {\n for symbol in frame.symbols().iter().rev() {\n let symbol = Symbol {\n symbol: symbol.clone(),\n parent_hash: state.finish(),\n };\n symbol.hash(&mut state);\n symboltrace.push(symbol);\n }\n }\n\n symboltrace\n}\n" }, { "path": "tokio/src/runtime/task/waker.rs", "content": "use crate::runtime::task::{Header, RawTask, Schedule};\n\nuse std::marker::PhantomData;\nuse std::mem::ManuallyDrop;\nuse std::ops;\nuse std::ptr::NonNull;\nuse std::task::{RawWaker, RawWakerVTable, Waker};\n\npub(super) struct WakerRef<'a, S: 'static> {\n waker: ManuallyDrop,\n _p: PhantomData<(&'a Header, S)>,\n}\n\n/// Returns a `WakerRef` which avoids having to preemptively increase the\n/// refcount if there is no need to do so.\npub(super) fn waker_ref(header: &NonNull
) -> WakerRef<'_, S>\nwhere\n S: Schedule,\n{\n // `Waker::will_wake` uses the VTABLE pointer as part of the check. This\n // means that `will_wake` will always return false when using the current\n // task's waker. (discussion at rust-lang/rust#66281).\n //\n // To fix this, we use a single vtable. Since we pass in a reference at this\n // point and not an *owned* waker, we must ensure that `drop` is never\n // called on this waker instance. This is done by wrapping it with\n // `ManuallyDrop` and then never calling drop.\n let waker = unsafe { ManuallyDrop::new(Waker::from_raw(raw_waker(*header))) };\n\n WakerRef {\n waker,\n _p: PhantomData,\n }\n}\n\nimpl ops::Deref for WakerRef<'_, S> {\n type Target = Waker;\n\n fn deref(&self) -> &Waker {\n &self.waker\n }\n}\n\ncfg_trace! {\n /// # Safety\n ///\n /// `$header` must be a valid pointer to a [`Header`].\n macro_rules! trace {\n ($header:expr, $op:expr) => {\n if let Some(id) = Header::get_tracing_id(&$header) {\n tracing::trace!(\n target: \"tokio::task::waker\",\n op = $op,\n task.id = id.into_u64(),\n );\n }\n }\n }\n}\n\ncfg_not_trace! {\n macro_rules! trace {\n ($header:expr, $op:expr) => {\n // noop\n let _ = &$header;\n }\n }\n}\n\nunsafe fn clone_waker(ptr: *const ()) -> RawWaker {\n // Safety: `ptr` was created from a `Header` pointer in function `waker_ref`.\n let header = unsafe { NonNull::new_unchecked(ptr as *mut Header) };\n #[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_unsafe))]\n unsafe {\n trace!(header, \"waker.clone\");\n }\n unsafe { header.as_ref() }.state.ref_inc();\n raw_waker(header)\n}\n\nunsafe fn drop_waker(ptr: *const ()) {\n // Safety: `ptr` was created from a `Header` pointer in function `waker_ref`.\n let ptr = unsafe { NonNull::new_unchecked(ptr as *mut Header) };\n // TODO; replace to #[expect(unused_unsafe)] after bumping MSRV to 1.81.0.\n #[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_unsafe))]\n unsafe {\n trace!(ptr, \"waker.drop\");\n }\n let raw = unsafe { RawTask::from_raw(ptr) };\n raw.drop_reference();\n}\n\nunsafe fn wake_by_val(ptr: *const ()) {\n // Safety: `ptr` was created from a `Header` pointer in function `waker_ref`.\n let ptr = unsafe { NonNull::new_unchecked(ptr as *mut Header) };\n // TODO; replace to #[expect(unused_unsafe)] after bumping MSRV to 1.81.0.\n #[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_unsafe))]\n unsafe {\n trace!(ptr, \"waker.wake\");\n }\n let raw = unsafe { RawTask::from_raw(ptr) };\n raw.wake_by_val();\n}\n\n// Wake without consuming the waker\nunsafe fn wake_by_ref(ptr: *const ()) {\n // Safety: `ptr` was created from a `Header` pointer in function `waker_ref`.\n let ptr = unsafe { NonNull::new_unchecked(ptr as *mut Header) };\n // TODO; replace to #[expect(unused_unsafe)] after bumping MSRV to 1.81.0.\n #[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_unsafe))]\n unsafe {\n trace!(ptr, \"waker.wake_by_ref\");\n }\n let raw = unsafe { RawTask::from_raw(ptr) };\n raw.wake_by_ref();\n}\n\nstatic WAKER_VTABLE: RawWakerVTable =\n RawWakerVTable::new(clone_waker, wake_by_val, wake_by_ref, drop_waker);\n\nfn raw_waker(header: NonNull
) -> RawWaker {\n let ptr = header.as_ptr() as *const ();\n RawWaker::new(ptr, &WAKER_VTABLE)\n}\n" }, { "path": "tokio/src/runtime/task_hooks.rs", "content": "use super::Config;\nuse std::marker::PhantomData;\n\nimpl TaskHooks {\n pub(crate) fn spawn(&self, meta: &TaskMeta<'_>) {\n if let Some(f) = self.task_spawn_callback.as_ref() {\n f(meta)\n }\n }\n\n #[allow(dead_code)]\n pub(crate) fn from_config(config: &Config) -> Self {\n Self {\n task_spawn_callback: config.before_spawn.clone(),\n task_terminate_callback: config.after_termination.clone(),\n #[cfg(tokio_unstable)]\n before_poll_callback: config.before_poll.clone(),\n #[cfg(tokio_unstable)]\n after_poll_callback: config.after_poll.clone(),\n }\n }\n\n #[cfg(tokio_unstable)]\n #[inline]\n pub(crate) fn poll_start_callback(&self, meta: &TaskMeta<'_>) {\n if let Some(poll_start) = &self.before_poll_callback {\n (poll_start)(meta);\n }\n }\n\n #[cfg(tokio_unstable)]\n #[inline]\n pub(crate) fn poll_stop_callback(&self, meta: &TaskMeta<'_>) {\n if let Some(poll_stop) = &self.after_poll_callback {\n (poll_stop)(meta);\n }\n }\n}\n\n#[derive(Clone)]\npub(crate) struct TaskHooks {\n pub(crate) task_spawn_callback: Option,\n pub(crate) task_terminate_callback: Option,\n #[cfg(tokio_unstable)]\n pub(crate) before_poll_callback: Option,\n #[cfg(tokio_unstable)]\n pub(crate) after_poll_callback: Option,\n}\n\n/// Task metadata supplied to user-provided hooks for task events.\n///\n/// **Note**: This is an [unstable API][unstable]. The public API of this type\n/// may break in 1.x releases. See [the documentation on unstable\n/// features][unstable] for details.\n///\n/// [unstable]: crate#unstable-features\n#[allow(missing_debug_implementations)]\n#[cfg_attr(not(tokio_unstable), allow(unreachable_pub))]\npub struct TaskMeta<'a> {\n /// The opaque ID of the task.\n pub(crate) id: super::task::Id,\n /// The location where the task was spawned.\n #[cfg_attr(not(tokio_unstable), allow(unreachable_pub, dead_code))]\n pub(crate) spawned_at: crate::runtime::task::SpawnLocation,\n pub(crate) _phantom: PhantomData<&'a ()>,\n}\n\nimpl<'a> TaskMeta<'a> {\n /// Return the opaque ID of the task.\n #[cfg_attr(not(tokio_unstable), allow(unreachable_pub, dead_code))]\n pub fn id(&self) -> super::task::Id {\n self.id\n }\n\n /// Return the source code location where the task was spawned.\n #[cfg(tokio_unstable)]\n pub fn spawned_at(&self) -> &'static std::panic::Location<'static> {\n self.spawned_at.0\n }\n}\n\n/// Runs on specific task-related events\npub(crate) type TaskCallback = std::sync::Arc) + Send + Sync>;\n" }, { "path": "tokio/src/runtime/tests/inject.rs", "content": "use crate::runtime::scheduler::inject;\n\n#[test]\nfn push_and_pop() {\n const N: usize = 2;\n\n let (inject, mut synced) = inject::Shared::new();\n\n for i in 0..N {\n assert_eq!(inject.len(), i);\n let (task, _) = super::unowned(async {});\n unsafe { inject.push(&mut synced, task) };\n }\n\n for i in 0..N {\n assert_eq!(inject.len(), N - i);\n assert!(unsafe { inject.pop(&mut synced) }.is_some());\n }\n\n println!(\"--------------\");\n\n assert!(unsafe { inject.pop(&mut synced) }.is_none());\n}\n\n#[test]\nfn push_batch_and_pop() {\n let (inject, mut inject_synced) = inject::Shared::new();\n\n unsafe {\n inject.push_batch(\n &mut inject_synced,\n (0..10).map(|_| super::unowned(async {}).0),\n );\n\n assert_eq!(5, inject.pop_n(&mut inject_synced, 5).count());\n assert_eq!(5, inject.pop_n(&mut inject_synced, 5).count());\n assert_eq!(0, inject.pop_n(&mut inject_synced, 5).count());\n }\n}\n\n#[test]\nfn pop_n_drains_on_drop() {\n let (inject, mut inject_synced) = inject::Shared::new();\n\n unsafe {\n inject.push_batch(\n &mut inject_synced,\n (0..10).map(|_| super::unowned(async {}).0),\n );\n let _ = inject.pop_n(&mut inject_synced, 10);\n\n assert_eq!(inject.len(), 0);\n }\n}\n" }, { "path": "tokio/src/runtime/tests/loom_blocking.rs", "content": "use crate::runtime::{self, Runtime};\n\nuse std::sync::Arc;\nuse std::time::Duration;\n\n#[test]\nfn blocking_shutdown() {\n loom::model(|| {\n let v = Arc::new(());\n\n let rt = mk_runtime(1);\n {\n let _enter = rt.enter();\n for _ in 0..2 {\n let v = v.clone();\n crate::task::spawn_blocking(move || {\n assert!(1 < Arc::strong_count(&v));\n });\n }\n }\n\n drop(rt);\n assert_eq!(1, Arc::strong_count(&v));\n });\n}\n\n#[test]\nfn spawn_mandatory_blocking_should_always_run() {\n use crate::runtime::tests::loom_oneshot;\n loom::model(|| {\n let rt = runtime::Builder::new_current_thread().build().unwrap();\n\n let (tx, rx) = loom_oneshot::channel();\n let _enter = rt.enter();\n runtime::spawn_blocking(|| {});\n runtime::spawn_mandatory_blocking(move || {\n let _ = tx.send(());\n })\n .unwrap();\n\n drop(rt);\n\n // This call will deadlock if `spawn_mandatory_blocking` doesn't run.\n let () = rx.recv();\n });\n}\n\n#[test]\nfn spawn_mandatory_blocking_should_run_even_when_shutting_down_from_other_thread() {\n use crate::runtime::tests::loom_oneshot;\n loom::model(|| {\n let rt = runtime::Builder::new_current_thread().build().unwrap();\n let handle = rt.handle().clone();\n\n // Drop the runtime in a different thread\n {\n loom::thread::spawn(move || {\n drop(rt);\n });\n }\n\n let _enter = handle.enter();\n let (tx, rx) = loom_oneshot::channel();\n let handle = runtime::spawn_mandatory_blocking(move || {\n let _ = tx.send(());\n });\n\n // handle.is_some() means that `spawn_mandatory_blocking`\n // promised us to run the blocking task\n if handle.is_some() {\n // This call will deadlock if `spawn_mandatory_blocking` doesn't run.\n let () = rx.recv();\n }\n });\n}\n\n#[test]\nfn spawn_blocking_when_paused() {\n loom::model(|| {\n let rt = crate::runtime::Builder::new_current_thread()\n .enable_time()\n .start_paused(true)\n .build()\n .unwrap();\n let handle = rt.handle();\n let _enter = handle.enter();\n let a = crate::task::spawn_blocking(|| {});\n let b = crate::task::spawn_blocking(|| {});\n rt.block_on(crate::time::timeout(Duration::from_millis(1), async move {\n a.await.expect(\"blocking task should finish\");\n b.await.expect(\"blocking task should finish\");\n }))\n .expect(\"timeout should not trigger\");\n });\n}\n\n#[test]\n/// See \nfn spawn_blocking_then_shutdown() {\n loom::model(|| {\n let rt = crate::runtime::Builder::new_current_thread()\n .max_blocking_threads(1)\n .thread_keep_alive(Duration::from_secs(7200)) // don't let the thread exit on its own\n .build()\n .unwrap();\n let rt_hdl = rt.handle().clone();\n\n // Currently, there is no live blocking thread,\n // so `spawn_blocking` will spawn a new blocking thread.\n let jh0 = rt_hdl.spawn_blocking(|| {});\n loom::future::block_on(jh0).unwrap();\n\n // Now, there is a idle blocking threads park on the condvar,\n // so the following `spawn_blocking` will decrease the `num_idle_threads`\n // and then notify one of the idle threads to run the task.\n\n // this will decrease the `num_idle_threads`\n // and then notify one of the idle threads to run the task.\n let jh3 = rt_hdl.spawn_blocking(|| {});\n\n // shutdown the runtime, which also shutdown the blocking pool\n drop(rt);\n\n // loom will emulate two parrel operations:\n //\n // 1. the blocking thread is woken up on the condvar\n // 2. the main thread is waiting for the blocking thread to finish the task\n //\n // So, if the `num_idle_threads` is not counted correctly,\n // it will trigger the assertions inside the `Inner::run` function.\n let _ = loom::future::block_on(jh3);\n });\n}\n\nfn mk_runtime(num_threads: usize) -> Runtime {\n runtime::Builder::new_multi_thread()\n .worker_threads(num_threads)\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/src/runtime/tests/loom_current_thread/yield_now.rs", "content": "use crate::runtime::park;\nuse crate::runtime::{self, Runtime};\n\n#[test]\nfn yield_calls_park_before_scheduling_again() {\n // Don't need to check all permutations\n let mut loom = loom::model::Builder::default();\n loom.max_permutations = Some(1);\n loom.check(|| {\n let rt = mk_runtime();\n\n let jh = rt.spawn(async {\n let tid = loom::thread::current().id();\n let park_count = park::current_thread_park_count();\n\n crate::task::yield_now().await;\n\n if tid == loom::thread::current().id() {\n let new_park_count = park::current_thread_park_count();\n assert_eq!(park_count + 1, new_park_count);\n }\n });\n\n rt.block_on(jh).unwrap();\n });\n}\n\nfn mk_runtime() -> Runtime {\n runtime::Builder::new_current_thread().build().unwrap()\n}\n" }, { "path": "tokio/src/runtime/tests/loom_current_thread.rs", "content": "mod yield_now;\n\nuse crate::loom::sync::atomic::{AtomicUsize, Ordering};\nuse crate::loom::sync::Arc;\nuse crate::loom::thread;\nuse crate::runtime::{Builder, Runtime};\nuse crate::sync::oneshot::{self, Receiver};\nuse crate::task;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::atomic::Ordering::{Acquire, Release};\nuse std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};\n\nfn assert_at_most_num_polls(rt: Arc, at_most_polls: usize) {\n let (tx, rx) = oneshot::channel();\n let num_polls = Arc::new(AtomicUsize::new(0));\n rt.spawn(async move {\n for _ in 0..12 {\n task::yield_now().await;\n }\n tx.send(()).unwrap();\n });\n\n rt.block_on(async {\n BlockedFuture {\n rx,\n num_polls: num_polls.clone(),\n }\n .await;\n });\n\n let polls = num_polls.load(Acquire);\n assert!(polls <= at_most_polls);\n}\n\n#[test]\nfn block_on_num_polls() {\n loom::model(|| {\n // we expect at most 4 number of polls because there are three points at\n // which we poll the future and an opportunity for a false-positive.. At\n // any of these points it can be ready:\n //\n // - when we fail to steal the parker and we block on a notification\n // that it is available.\n //\n // - when we steal the parker and we schedule the future\n //\n // - when the future is woken up and we have ran the max number of tasks\n // for the current tick or there are no more tasks to run.\n //\n // - a thread is notified that the parker is available but a third\n // thread acquires it before the notified thread can.\n //\n let at_most = 4;\n\n let rt1 = Arc::new(Builder::new_current_thread().build().unwrap());\n let rt2 = rt1.clone();\n let rt3 = rt1.clone();\n\n let th1 = thread::spawn(move || assert_at_most_num_polls(rt1, at_most));\n let th2 = thread::spawn(move || assert_at_most_num_polls(rt2, at_most));\n let th3 = thread::spawn(move || assert_at_most_num_polls(rt3, at_most));\n\n th1.join().unwrap();\n th2.join().unwrap();\n th3.join().unwrap();\n });\n}\n\n#[test]\nfn assert_no_unnecessary_polls() {\n loom::model(|| {\n // // After we poll outer future, woken should reset to false\n let rt = Builder::new_current_thread().build().unwrap();\n let (tx, rx) = oneshot::channel();\n let pending_cnt = Arc::new(AtomicUsize::new(0));\n\n rt.spawn(async move {\n for _ in 0..24 {\n task::yield_now().await;\n }\n tx.send(()).unwrap();\n });\n\n let pending_cnt_clone = pending_cnt.clone();\n rt.block_on(async move {\n // use task::yield_now() to ensure woken set to true\n // ResetFuture will be polled at most once\n // Here comes two cases\n // 1. recv no message from channel, ResetFuture will be polled\n // but get Pending and we record ResetFuture.pending_cnt ++.\n // Then when message arrive, ResetFuture returns Ready. So we\n // expect ResetFuture.pending_cnt = 1\n // 2. recv message from channel, ResetFuture returns Ready immediately.\n // We expect ResetFuture.pending_cnt = 0\n task::yield_now().await;\n ResetFuture {\n rx,\n pending_cnt: pending_cnt_clone,\n }\n .await;\n });\n\n let pending_cnt = pending_cnt.load(Acquire);\n assert!(pending_cnt <= 1);\n });\n}\n\n#[test]\nfn drop_jh_during_schedule() {\n unsafe fn waker_clone(ptr: *const ()) -> RawWaker {\n let atomic = unsafe { &*(ptr as *const AtomicUsize) };\n atomic.fetch_add(1, Ordering::Relaxed);\n RawWaker::new(ptr, &VTABLE)\n }\n unsafe fn waker_drop(ptr: *const ()) {\n let atomic = unsafe { &*(ptr as *const AtomicUsize) };\n atomic.fetch_sub(1, Ordering::Relaxed);\n }\n unsafe fn waker_nop(_ptr: *const ()) {}\n\n static VTABLE: RawWakerVTable =\n RawWakerVTable::new(waker_clone, waker_drop, waker_nop, waker_drop);\n\n loom::model(|| {\n let rt = Builder::new_current_thread().build().unwrap();\n\n let mut jh = rt.spawn(async {});\n // Using AbortHandle to increment task refcount. This ensures that the waker is not\n // destroyed due to the refcount hitting zero.\n let task_refcnt = jh.abort_handle();\n\n let waker_refcnt = AtomicUsize::new(1);\n {\n // Set up the join waker.\n use std::future::Future;\n use std::pin::Pin;\n\n // SAFETY: Before `waker_refcnt` goes out of scope, this test asserts that the refcnt\n // has dropped to zero.\n let join_waker = unsafe {\n Waker::from_raw(RawWaker::new(\n (&waker_refcnt) as *const AtomicUsize as *const (),\n &VTABLE,\n ))\n };\n\n assert!(Pin::new(&mut jh)\n .poll(&mut Context::from_waker(&join_waker))\n .is_pending());\n }\n assert_eq!(waker_refcnt.load(Ordering::Relaxed), 1);\n\n let bg_thread = loom::thread::spawn(move || drop(jh));\n rt.block_on(crate::task::yield_now());\n bg_thread.join().unwrap();\n\n assert_eq!(waker_refcnt.load(Ordering::Relaxed), 0);\n drop(task_refcnt);\n });\n}\n\nstruct BlockedFuture {\n rx: Receiver<()>,\n num_polls: Arc,\n}\n\nimpl Future for BlockedFuture {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n self.num_polls.fetch_add(1, Release);\n\n match Pin::new(&mut self.rx).poll(cx) {\n Poll::Pending => Poll::Pending,\n _ => Poll::Ready(()),\n }\n }\n}\n\nstruct ResetFuture {\n rx: Receiver<()>,\n pending_cnt: Arc,\n}\n\nimpl Future for ResetFuture {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n match Pin::new(&mut self.rx).poll(cx) {\n Poll::Pending => {\n self.pending_cnt.fetch_add(1, Release);\n Poll::Pending\n }\n _ => Poll::Ready(()),\n }\n }\n}\n" }, { "path": "tokio/src/runtime/tests/loom_join_set.rs", "content": "use crate::runtime::Builder;\nuse crate::task::JoinSet;\n\n#[test]\nfn test_join_set() {\n loom::model(|| {\n let rt = Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n let mut set = JoinSet::new();\n\n rt.block_on(async {\n assert_eq!(set.len(), 0);\n set.spawn(async { () });\n assert_eq!(set.len(), 1);\n set.spawn(async { () });\n assert_eq!(set.len(), 2);\n let () = set.join_next().await.unwrap().unwrap();\n assert_eq!(set.len(), 1);\n set.spawn(async { () });\n assert_eq!(set.len(), 2);\n let () = set.join_next().await.unwrap().unwrap();\n assert_eq!(set.len(), 1);\n let () = set.join_next().await.unwrap().unwrap();\n assert_eq!(set.len(), 0);\n set.spawn(async { () });\n assert_eq!(set.len(), 1);\n });\n\n drop(set);\n drop(rt);\n });\n}\n\n#[test]\nfn abort_all_during_completion() {\n use std::sync::{\n atomic::{AtomicBool, Ordering::SeqCst},\n Arc,\n };\n\n // These booleans assert that at least one execution had the task complete first, and that at\n // least one execution had the task be cancelled before it completed.\n let complete_happened = Arc::new(AtomicBool::new(false));\n let cancel_happened = Arc::new(AtomicBool::new(false));\n\n {\n let complete_happened = complete_happened.clone();\n let cancel_happened = cancel_happened.clone();\n loom::model(move || {\n let rt = Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n let mut set = JoinSet::new();\n\n rt.block_on(async {\n set.spawn(async { () });\n set.abort_all();\n\n match set.join_next().await {\n Some(Ok(())) => complete_happened.store(true, SeqCst),\n Some(Err(err)) if err.is_cancelled() => cancel_happened.store(true, SeqCst),\n Some(Err(err)) => panic!(\"fail: {}\", err),\n None => {\n unreachable!(\"Aborting the task does not remove it from the JoinSet.\")\n }\n }\n\n assert!(matches!(set.join_next().await, None));\n });\n\n drop(set);\n drop(rt);\n });\n }\n\n assert!(complete_happened.load(SeqCst));\n assert!(cancel_happened.load(SeqCst));\n}\n" }, { "path": "tokio/src/runtime/tests/loom_local.rs", "content": "use crate::runtime::tests::loom_oneshot as oneshot;\nuse crate::runtime::Builder;\nuse crate::task::LocalSet;\n\nuse std::task::Poll;\n\n/// Waking a runtime will attempt to push a task into a queue of notifications\n/// in the runtime, however the tasks in such a queue usually have a reference\n/// to the runtime itself. This means that if they are not properly removed at\n/// runtime shutdown, this will cause a memory leak.\n///\n/// This test verifies that waking something during shutdown of a `LocalSet` does\n/// not result in tasks lingering in the queue once shutdown is complete. This\n/// is verified using loom's leak finder.\n#[test]\nfn wake_during_shutdown() {\n loom::model(|| {\n let rt = Builder::new_current_thread().build().unwrap();\n let ls = LocalSet::new();\n\n let (send, recv) = oneshot::channel();\n\n ls.spawn_local(async move {\n let mut send = Some(send);\n\n let () = std::future::poll_fn(|cx| {\n if let Some(send) = send.take() {\n send.send(cx.waker().clone());\n }\n\n Poll::Pending\n })\n .await;\n });\n\n let handle = loom::thread::spawn(move || {\n let waker = recv.recv();\n waker.wake();\n });\n\n ls.block_on(&rt, crate::task::yield_now());\n\n drop(ls);\n handle.join().unwrap();\n drop(rt);\n });\n}\n" }, { "path": "tokio/src/runtime/tests/loom_multi_thread/queue.rs", "content": "use crate::runtime::scheduler::multi_thread::{queue, Stats};\nuse crate::runtime::tests::{unowned, NoopSchedule};\n\nuse loom::thread;\nuse std::cell::RefCell;\n\nfn new_stats() -> Stats {\n Stats::new(&crate::runtime::WorkerMetrics::new())\n}\n\n#[test]\nfn basic() {\n loom::model(|| {\n let (steal, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n let mut stats = new_stats();\n\n let th = thread::spawn(move || {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n let mut n = 0;\n\n for _ in 0..3 {\n if steal.steal_into(&mut local, &mut stats).is_some() {\n n += 1;\n }\n\n while local.pop().is_some() {\n n += 1;\n }\n }\n\n n\n });\n\n let mut n = 0;\n\n for _ in 0..2 {\n for _ in 0..2 {\n let (task, _) = unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n if local.pop().is_some() {\n n += 1;\n }\n\n // Push another task\n let (task, _) = unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n\n while local.pop().is_some() {\n n += 1;\n }\n }\n\n n += inject.borrow_mut().drain(..).count();\n\n n += th.join().unwrap();\n\n assert_eq!(6, n);\n });\n}\n\n#[test]\nfn steal_overflow() {\n loom::model(|| {\n let (steal, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n let mut stats = new_stats();\n\n let th = thread::spawn(move || {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n let mut n = 0;\n\n if steal.steal_into(&mut local, &mut stats).is_some() {\n n += 1;\n }\n\n while local.pop().is_some() {\n n += 1;\n }\n\n n\n });\n\n let mut n = 0;\n\n // push a task, pop a task\n let (task, _) = unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n\n if local.pop().is_some() {\n n += 1;\n }\n\n for _ in 0..6 {\n let (task, _) = unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n n += th.join().unwrap();\n\n while local.pop().is_some() {\n n += 1;\n }\n\n n += inject.borrow_mut().drain(..).count();\n\n assert_eq!(7, n);\n });\n}\n\n#[test]\nfn multi_stealer() {\n const NUM_TASKS: usize = 5;\n\n fn steal_tasks(steal: queue::Steal) -> usize {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n\n if steal.steal_into(&mut local, &mut stats).is_none() {\n return 0;\n }\n\n let mut n = 1;\n\n while local.pop().is_some() {\n n += 1;\n }\n\n n\n }\n\n loom::model(|| {\n let (steal, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n let mut stats = new_stats();\n\n // Push work\n for _ in 0..NUM_TASKS {\n let (task, _) = unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n let th1 = {\n let steal = steal.clone();\n thread::spawn(move || steal_tasks(steal))\n };\n\n let th2 = thread::spawn(move || steal_tasks(steal));\n\n let mut n = 0;\n\n while local.pop().is_some() {\n n += 1;\n }\n\n n += inject.borrow_mut().drain(..).count();\n\n n += th1.join().unwrap();\n n += th2.join().unwrap();\n\n assert_eq!(n, NUM_TASKS);\n });\n}\n\n#[test]\nfn chained_steal() {\n loom::model(|| {\n let mut stats = new_stats();\n let (s1, mut l1) = queue::local();\n let (s2, mut l2) = queue::local();\n let inject = RefCell::new(vec![]);\n\n // Load up some tasks\n for _ in 0..4 {\n let (task, _) = unowned(async {});\n l1.push_back_or_overflow(task, &inject, &mut stats);\n\n let (task, _) = unowned(async {});\n l2.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n // Spawn a task to steal from **our** queue\n let th = thread::spawn(move || {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n s1.steal_into(&mut local, &mut stats);\n\n while local.pop().is_some() {}\n });\n\n // Drain our tasks, then attempt to steal\n while l1.pop().is_some() {}\n\n s2.steal_into(&mut l1, &mut stats);\n\n th.join().unwrap();\n\n while l1.pop().is_some() {}\n while l2.pop().is_some() {}\n });\n}\n" }, { "path": "tokio/src/runtime/tests/loom_multi_thread/shutdown.rs", "content": "use crate::runtime::{Builder, Handle};\n\n#[test]\nfn join_handle_cancel_on_shutdown() {\n let mut builder = loom::model::Builder::new();\n builder.preemption_bound = Some(2);\n builder.check(|| {\n use futures::future::FutureExt;\n\n let rt = Builder::new_multi_thread()\n .worker_threads(2)\n .build()\n .unwrap();\n\n let handle = rt.block_on(async move { Handle::current() });\n\n let jh1 = handle.spawn(futures::future::pending::<()>());\n\n drop(rt);\n\n let jh2 = handle.spawn(futures::future::pending::<()>());\n\n let err1 = jh1.now_or_never().unwrap().unwrap_err();\n let err2 = jh2.now_or_never().unwrap().unwrap_err();\n assert!(err1.is_cancelled());\n assert!(err2.is_cancelled());\n });\n}\n" }, { "path": "tokio/src/runtime/tests/loom_multi_thread/yield_now.rs", "content": "use crate::runtime::park;\nuse crate::runtime::tests::loom_oneshot as oneshot;\nuse crate::runtime::{self, Runtime};\n\n#[test]\nfn yield_calls_park_before_scheduling_again() {\n // Don't need to check all permutations\n let mut loom = loom::model::Builder::default();\n loom.max_permutations = Some(1);\n loom.check(|| {\n let rt = mk_runtime(2);\n let (tx, rx) = oneshot::channel::<()>();\n\n rt.spawn(async {\n let tid = loom::thread::current().id();\n let park_count = park::current_thread_park_count();\n\n crate::task::yield_now().await;\n\n if tid == loom::thread::current().id() {\n let new_park_count = park::current_thread_park_count();\n assert_eq!(park_count + 1, new_park_count);\n }\n\n tx.send(());\n });\n\n rx.recv();\n });\n}\n\nfn mk_runtime(num_threads: usize) -> Runtime {\n runtime::Builder::new_multi_thread()\n .worker_threads(num_threads)\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/src/runtime/tests/loom_multi_thread.rs", "content": "mod queue;\nmod shutdown;\nmod yield_now;\n\n/// Full runtime loom tests. These are heavy tests and take significant time to\n/// run on CI.\n///\n/// Use `LOOM_MAX_PREEMPTIONS=1` to do a \"quick\" run as a smoke test.\n///\n/// In order to speed up the C\nuse crate::runtime::tests::loom_oneshot as oneshot;\nuse crate::runtime::{self, Runtime};\nuse crate::{spawn, task};\nuse tokio_test::assert_ok;\n\nuse loom::sync::atomic::{AtomicBool, AtomicUsize};\nuse loom::sync::Arc;\n\nuse pin_project_lite::pin_project;\nuse std::future::{poll_fn, Future};\nuse std::pin::Pin;\nuse std::sync::atomic::Ordering::{Relaxed, SeqCst};\nuse std::task::{ready, Context, Poll};\n\nmod atomic_take {\n use loom::sync::atomic::AtomicBool;\n use std::mem::MaybeUninit;\n use std::sync::atomic::Ordering::SeqCst;\n\n pub(super) struct AtomicTake {\n inner: MaybeUninit,\n taken: AtomicBool,\n }\n\n impl AtomicTake {\n pub(super) fn new(value: T) -> Self {\n Self {\n inner: MaybeUninit::new(value),\n taken: AtomicBool::new(false),\n }\n }\n\n pub(super) fn take(&self) -> Option {\n // safety: Only one thread will see the boolean change from false\n // to true, so that thread is able to take the value.\n match self.taken.fetch_or(true, SeqCst) {\n false => unsafe { Some(std::ptr::read(self.inner.as_ptr())) },\n true => None,\n }\n }\n }\n\n impl Drop for AtomicTake {\n fn drop(&mut self) {\n drop(self.take());\n }\n }\n}\n\n#[derive(Clone)]\nstruct AtomicOneshot {\n value: std::sync::Arc>>,\n}\nimpl AtomicOneshot {\n fn new(sender: oneshot::Sender) -> Self {\n Self {\n value: std::sync::Arc::new(atomic_take::AtomicTake::new(sender)),\n }\n }\n\n fn assert_send(&self, value: T) {\n self.value.take().unwrap().send(value);\n }\n}\n\n/// Tests are divided into groups to make the runs faster on CI.\nmod group_a {\n use super::*;\n\n #[test]\n fn racy_shutdown() {\n loom::model(|| {\n let pool = mk_pool(1);\n\n // here's the case we want to exercise:\n //\n // a worker that still has tasks in its local queue gets sent to the blocking pool (due to\n // block_in_place). the blocking pool is shut down, so drops the worker. the worker's\n // shutdown method never gets run.\n //\n // we do this by spawning two tasks on one worker, the first of which does block_in_place,\n // and then immediately drop the pool.\n\n pool.spawn(track(async {\n crate::task::block_in_place(|| {});\n }));\n pool.spawn(track(async {}));\n drop(pool);\n });\n }\n\n #[test]\n fn pool_multi_spawn() {\n loom::model(|| {\n let pool = mk_pool(2);\n let c1 = Arc::new(AtomicUsize::new(0));\n\n let (tx, rx) = oneshot::channel();\n let tx1 = AtomicOneshot::new(tx);\n\n // Spawn a task\n let c2 = c1.clone();\n let tx2 = tx1.clone();\n pool.spawn(track(async move {\n spawn(track(async move {\n if 1 == c1.fetch_add(1, Relaxed) {\n tx1.assert_send(());\n }\n }));\n }));\n\n // Spawn a second task\n pool.spawn(track(async move {\n spawn(track(async move {\n if 1 == c2.fetch_add(1, Relaxed) {\n tx2.assert_send(());\n }\n }));\n }));\n\n rx.recv();\n });\n }\n\n fn only_blocking_inner(first_pending: bool) {\n loom::model(move || {\n let pool = mk_pool(1);\n let (block_tx, block_rx) = oneshot::channel();\n\n pool.spawn(track(async move {\n crate::task::block_in_place(move || {\n block_tx.send(());\n });\n if first_pending {\n task::yield_now().await\n }\n }));\n\n block_rx.recv();\n drop(pool);\n });\n }\n\n #[test]\n fn only_blocking_without_pending() {\n only_blocking_inner(false)\n }\n\n #[test]\n fn only_blocking_with_pending() {\n only_blocking_inner(true)\n }\n}\n\nmod group_b {\n use super::*;\n\n fn blocking_and_regular_inner(first_pending: bool) {\n const NUM: usize = 3;\n loom::model(move || {\n let pool = mk_pool(1);\n let cnt = Arc::new(AtomicUsize::new(0));\n\n let (block_tx, block_rx) = oneshot::channel();\n let (done_tx, done_rx) = oneshot::channel();\n let done_tx = AtomicOneshot::new(done_tx);\n\n pool.spawn(track(async move {\n crate::task::block_in_place(move || {\n block_tx.send(());\n });\n if first_pending {\n task::yield_now().await\n }\n }));\n\n for _ in 0..NUM {\n let cnt = cnt.clone();\n let done_tx = done_tx.clone();\n\n pool.spawn(track(async move {\n if NUM == cnt.fetch_add(1, Relaxed) + 1 {\n done_tx.assert_send(());\n }\n }));\n }\n\n done_rx.recv();\n block_rx.recv();\n\n drop(pool);\n });\n }\n\n #[test]\n fn blocking_and_regular() {\n blocking_and_regular_inner(false);\n }\n\n #[test]\n fn blocking_and_regular_with_pending() {\n blocking_and_regular_inner(true);\n }\n\n #[test]\n fn join_output() {\n loom::model(|| {\n let rt = mk_pool(1);\n\n rt.block_on(async {\n let t = crate::spawn(track(async { \"hello\" }));\n\n let out = assert_ok!(t.await);\n assert_eq!(\"hello\", out.into_inner());\n });\n });\n }\n\n #[test]\n fn poll_drop_handle_then_drop() {\n loom::model(|| {\n let rt = mk_pool(1);\n\n rt.block_on(async move {\n let mut t = crate::spawn(track(async { \"hello\" }));\n\n poll_fn(|cx| {\n let _ = Pin::new(&mut t).poll(cx);\n Poll::Ready(())\n })\n .await;\n });\n })\n }\n\n #[test]\n fn complete_block_on_under_load() {\n loom::model(|| {\n let pool = mk_pool(1);\n\n pool.block_on(async {\n // Trigger a re-schedule\n crate::spawn(track(async {\n for _ in 0..2 {\n task::yield_now().await;\n }\n }));\n\n gated2(true).await\n });\n });\n }\n\n #[test]\n fn shutdown_with_notification() {\n use crate::sync::oneshot;\n\n loom::model(|| {\n let rt = mk_pool(2);\n let (done_tx, done_rx) = oneshot::channel::<()>();\n\n rt.spawn(track(async move {\n let (tx, rx) = oneshot::channel::<()>();\n\n crate::spawn(async move {\n crate::task::spawn_blocking(move || {\n let _ = tx.send(());\n });\n\n let _ = done_rx.await;\n });\n\n let _ = rx.await;\n\n let _ = done_tx.send(());\n }));\n });\n }\n}\n\nmod group_c {\n use super::*;\n\n #[test]\n fn pool_shutdown() {\n loom::model(|| {\n let pool = mk_pool(2);\n\n pool.spawn(track(async move {\n gated2(true).await;\n }));\n\n pool.spawn(track(async move {\n gated2(false).await;\n }));\n\n drop(pool);\n });\n }\n}\n\nmod group_d {\n use super::*;\n\n #[test]\n fn pool_multi_notify() {\n loom::model(|| {\n let pool = mk_pool(2);\n\n let c1 = Arc::new(AtomicUsize::new(0));\n\n let (done_tx, done_rx) = oneshot::channel();\n let done_tx1 = AtomicOneshot::new(done_tx);\n let done_tx2 = done_tx1.clone();\n\n // Spawn a task\n let c2 = c1.clone();\n pool.spawn(track(async move {\n multi_gated().await;\n\n if 1 == c1.fetch_add(1, Relaxed) {\n done_tx1.assert_send(());\n }\n }));\n\n // Spawn a second task\n pool.spawn(track(async move {\n multi_gated().await;\n\n if 1 == c2.fetch_add(1, Relaxed) {\n done_tx2.assert_send(());\n }\n }));\n\n done_rx.recv();\n });\n }\n}\n\nfn mk_pool(num_threads: usize) -> Runtime {\n runtime::Builder::new_multi_thread()\n .worker_threads(num_threads)\n // Set the intervals to avoid tuning logic\n .event_interval(2)\n .build()\n .unwrap()\n}\n\nfn gated2(thread: bool) -> impl Future {\n use loom::thread;\n use std::sync::Arc;\n\n let gate = Arc::new(AtomicBool::new(false));\n let mut fired = false;\n\n poll_fn(move |cx| {\n if !fired {\n let gate = gate.clone();\n let waker = cx.waker().clone();\n\n if thread {\n thread::spawn(move || {\n gate.store(true, SeqCst);\n waker.wake_by_ref();\n });\n } else {\n spawn(track(async move {\n gate.store(true, SeqCst);\n waker.wake_by_ref();\n }));\n }\n\n fired = true;\n\n return Poll::Pending;\n }\n\n if gate.load(SeqCst) {\n Poll::Ready(\"hello world\")\n } else {\n Poll::Pending\n }\n })\n}\n\nasync fn multi_gated() {\n struct Gate {\n waker: loom::future::AtomicWaker,\n count: AtomicUsize,\n }\n\n let gate = Arc::new(Gate {\n waker: loom::future::AtomicWaker::new(),\n count: AtomicUsize::new(0),\n });\n\n {\n let gate = gate.clone();\n spawn(track(async move {\n for i in 1..3 {\n gate.count.store(i, SeqCst);\n gate.waker.wake();\n }\n }));\n }\n\n poll_fn(move |cx| {\n gate.waker.register_by_ref(cx.waker());\n if gate.count.load(SeqCst) < 2 {\n Poll::Pending\n } else {\n Poll::Ready(())\n }\n })\n .await;\n}\n\nfn track(f: T) -> Track {\n Track {\n inner: f,\n arc: Arc::new(()),\n }\n}\n\npin_project! {\n struct Track {\n #[pin]\n inner: T,\n // Arc is used to hook into loom's leak tracking.\n arc: Arc<()>,\n }\n}\n\nimpl Track {\n fn into_inner(self) -> T {\n self.inner\n }\n}\n\nimpl Future for Track {\n type Output = Track;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n\n Poll::Ready(Track {\n inner: ready!(me.inner.poll(cx)),\n arc: me.arc.clone(),\n })\n }\n}\n" }, { "path": "tokio/src/runtime/tests/loom_oneshot.rs", "content": "use crate::loom::sync::{Arc, Mutex};\nuse loom::sync::Notify;\n\npub(crate) fn channel() -> (Sender, Receiver) {\n let inner = Arc::new(Inner {\n notify: Notify::new(),\n value: Mutex::new(None),\n });\n\n let tx = Sender {\n inner: inner.clone(),\n };\n let rx = Receiver { inner };\n\n (tx, rx)\n}\n\npub(crate) struct Sender {\n inner: Arc>,\n}\n\npub(crate) struct Receiver {\n inner: Arc>,\n}\n\nstruct Inner {\n notify: Notify,\n value: Mutex>,\n}\n\nimpl Sender {\n pub(crate) fn send(self, value: T) {\n *self.inner.value.lock() = Some(value);\n self.inner.notify.notify();\n }\n}\n\nimpl Receiver {\n pub(crate) fn recv(self) -> T {\n loop {\n if let Some(v) = self.inner.value.lock().take() {\n return v;\n }\n\n self.inner.notify.wait();\n }\n }\n}\n" }, { "path": "tokio/src/runtime/tests/mod.rs", "content": "// Enable dead_code / unreachable_pub here. It has been disabled in lib.rs for\n// other code when running loom tests.\n#![cfg_attr(loom, warn(dead_code, unreachable_pub))]\n\nuse self::noop_scheduler::NoopSchedule;\nuse self::unowned_wrapper::unowned;\n\nmod noop_scheduler {\n use crate::runtime::task::{self, Task, TaskHarnessScheduleHooks};\n\n /// `task::Schedule` implementation that does nothing, for testing.\n pub(crate) struct NoopSchedule;\n\n impl task::Schedule for NoopSchedule {\n fn release(&self, _task: &Task) -> Option> {\n None\n }\n\n fn schedule(&self, _task: task::Notified) {\n unreachable!();\n }\n\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: None,\n }\n }\n }\n}\n\nmod unowned_wrapper {\n use crate::runtime::task::{Id, JoinHandle, Notified, SpawnLocation};\n use crate::runtime::tests::NoopSchedule;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n #[track_caller]\n pub(crate) fn unowned(task: T) -> (Notified, JoinHandle)\n where\n T: std::future::Future + Send + 'static,\n T::Output: Send + 'static,\n {\n use tracing::Instrument;\n let span = tracing::trace_span!(\"test_span\");\n let task = task.instrument(span);\n let (task, handle) =\n crate::runtime::task::unowned(task, NoopSchedule, Id::next(), SpawnLocation::capture());\n (task.into_notified(), handle)\n }\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n #[track_caller]\n pub(crate) fn unowned(task: T) -> (Notified, JoinHandle)\n where\n T: std::future::Future + Send + 'static,\n T::Output: Send + 'static,\n {\n let (task, handle) =\n crate::runtime::task::unowned(task, NoopSchedule, Id::next(), SpawnLocation::capture());\n (task.into_notified(), handle)\n }\n}\n\ncfg_loom! {\n mod loom_blocking;\n mod loom_current_thread;\n mod loom_join_set;\n mod loom_local;\n mod loom_multi_thread;\n mod loom_oneshot;\n\n // Make sure debug assertions are enabled\n #[cfg(not(debug_assertions))]\n compile_error!(\"these tests require debug assertions to be enabled\");\n}\n\ncfg_not_loom! {\n mod inject;\n mod queue;\n\n #[cfg(not(miri))] // takes a really long time with miri\n mod task_combinations;\n\n #[cfg(miri)]\n mod task;\n}\n" }, { "path": "tokio/src/runtime/tests/queue.rs", "content": "use crate::runtime::scheduler::multi_thread::{queue, Stats};\n\nuse std::cell::RefCell;\nuse std::thread;\nuse std::time::Duration;\n\n#[allow(unused)]\nmacro_rules! assert_metrics {\n ($stats:ident, $field:ident == $v:expr) => {\n #[cfg(target_has_atomic = \"64\")]\n {\n use crate::runtime::WorkerMetrics;\n use std::sync::atomic::Ordering::Relaxed;\n\n let worker = WorkerMetrics::new();\n $stats.submit(&worker);\n\n let expect = $v;\n let actual = worker.$field.load(Relaxed);\n\n assert!(actual == expect, \"expect = {}; actual = {}\", expect, actual)\n }\n };\n}\n\nfn new_stats() -> Stats {\n use crate::runtime::WorkerMetrics;\n Stats::new(&WorkerMetrics::new())\n}\n\n#[test]\nfn fits_256_one_at_a_time() {\n let (_, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n let mut stats = new_stats();\n\n for _ in 0..256 {\n let (task, _) = super::unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n cfg_unstable_metrics! {\n assert_metrics!(stats, overflow_count == 0);\n }\n\n assert!(inject.borrow_mut().pop().is_none());\n\n while local.pop().is_some() {}\n}\n\n#[test]\nfn fits_256_all_at_once() {\n let (_, mut local) = queue::local();\n\n let mut tasks = (0..256)\n .map(|_| super::unowned(async {}).0)\n .collect::>();\n local.push_back(tasks.drain(..));\n\n let mut i = 0;\n while local.pop().is_some() {\n i += 1;\n }\n\n assert_eq!(i, 256);\n}\n\n#[test]\nfn fits_256_all_in_chunks() {\n let (_, mut local) = queue::local();\n\n let mut tasks = (0..256)\n .map(|_| super::unowned(async {}).0)\n .collect::>();\n\n local.push_back(tasks.drain(..10));\n local.push_back(tasks.drain(..100));\n local.push_back(tasks.drain(..46));\n local.push_back(tasks.drain(..100));\n\n let mut i = 0;\n while local.pop().is_some() {\n i += 1;\n }\n\n assert_eq!(i, 256);\n}\n\n#[test]\nfn overflow() {\n let (_, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n let mut stats = new_stats();\n\n for _ in 0..257 {\n let (task, _) = super::unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n cfg_unstable_metrics! {\n assert_metrics!(stats, overflow_count == 1);\n }\n\n let mut n = 0;\n\n n += inject.borrow_mut().drain(..).count();\n\n while local.pop().is_some() {\n n += 1;\n }\n\n assert_eq!(n, 257);\n}\n\n#[test]\nfn steal_batch() {\n let mut stats = new_stats();\n\n let (steal1, mut local1) = queue::local();\n let (_, mut local2) = queue::local();\n let inject = RefCell::new(vec![]);\n\n for _ in 0..4 {\n let (task, _) = super::unowned(async {});\n local1.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n assert!(steal1.steal_into(&mut local2, &mut stats).is_some());\n\n cfg_unstable_metrics! {\n assert_metrics!(stats, steal_count == 2);\n }\n\n for _ in 0..1 {\n assert!(local2.pop().is_some());\n }\n\n assert!(local2.pop().is_none());\n\n for _ in 0..2 {\n assert!(local1.pop().is_some());\n }\n\n assert!(local1.pop().is_none());\n}\n\nconst fn normal_or_miri(normal: usize, miri: usize) -> usize {\n if cfg!(miri) {\n miri\n } else {\n normal\n }\n}\n\n#[test]\nfn stress1() {\n const NUM_ITER: usize = 5;\n const NUM_STEAL: usize = normal_or_miri(1_000, 10);\n const NUM_LOCAL: usize = normal_or_miri(1_000, 10);\n const NUM_PUSH: usize = normal_or_miri(500, 10);\n const NUM_POP: usize = normal_or_miri(250, 10);\n\n let mut stats = new_stats();\n\n for _ in 0..NUM_ITER {\n let (steal, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n\n let th = thread::spawn(move || {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n let mut n = 0;\n\n for _ in 0..NUM_STEAL {\n if steal.steal_into(&mut local, &mut stats).is_some() {\n n += 1;\n }\n\n while local.pop().is_some() {\n n += 1;\n }\n\n thread::yield_now();\n }\n\n cfg_unstable_metrics! {\n assert_metrics!(stats, steal_count == n as _);\n }\n\n n\n });\n\n let mut n = 0;\n\n for _ in 0..NUM_LOCAL {\n for _ in 0..NUM_PUSH {\n let (task, _) = super::unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n }\n\n for _ in 0..NUM_POP {\n if local.pop().is_some() {\n n += 1;\n } else {\n break;\n }\n }\n }\n\n n += inject.borrow_mut().drain(..).count();\n\n n += th.join().unwrap();\n\n assert_eq!(n, NUM_LOCAL * NUM_PUSH);\n }\n}\n\n#[test]\nfn stress2() {\n const NUM_ITER: usize = 1;\n const NUM_TASKS: usize = normal_or_miri(1_000_000, 50);\n const NUM_STEAL: usize = normal_or_miri(1_000, 10);\n\n let mut stats = new_stats();\n\n for _ in 0..NUM_ITER {\n let (steal, mut local) = queue::local();\n let inject = RefCell::new(vec![]);\n\n let th = thread::spawn(move || {\n let mut stats = new_stats();\n let (_, mut local) = queue::local();\n let mut n = 0;\n\n for _ in 0..NUM_STEAL {\n if steal.steal_into(&mut local, &mut stats).is_some() {\n n += 1;\n }\n\n while local.pop().is_some() {\n n += 1;\n }\n\n thread::sleep(Duration::from_micros(10));\n }\n\n n\n });\n\n let mut num_pop = 0;\n\n for i in 0..NUM_TASKS {\n let (task, _) = super::unowned(async {});\n local.push_back_or_overflow(task, &inject, &mut stats);\n\n if i % 128 == 0 && local.pop().is_some() {\n num_pop += 1;\n }\n\n num_pop += inject.borrow_mut().drain(..).count();\n }\n\n num_pop += th.join().unwrap();\n\n while local.pop().is_some() {\n num_pop += 1;\n }\n\n num_pop += inject.borrow_mut().drain(..).count();\n\n assert_eq!(num_pop, NUM_TASKS);\n }\n}\n" }, { "path": "tokio/src/runtime/tests/task.rs", "content": "use crate::runtime::task::{\n self, unowned, Id, JoinHandle, OwnedTasks, Schedule, SpawnLocation, Task,\n TaskHarnessScheduleHooks,\n};\nuse crate::runtime::tests::NoopSchedule;\n\nuse std::collections::VecDeque;\nuse std::future::Future;\n#[cfg(tokio_unstable)]\nuse std::panic::Location;\nuse std::sync::atomic::{AtomicBool, Ordering};\nuse std::sync::{Arc, Mutex};\n\nstruct AssertDropHandle {\n is_dropped: Arc,\n}\nimpl AssertDropHandle {\n #[track_caller]\n fn assert_dropped(&self) {\n assert!(self.is_dropped.load(Ordering::SeqCst));\n }\n\n #[track_caller]\n fn assert_not_dropped(&self) {\n assert!(!self.is_dropped.load(Ordering::SeqCst));\n }\n}\n\nstruct AssertDrop {\n is_dropped: Arc,\n}\nimpl AssertDrop {\n fn new() -> (Self, AssertDropHandle) {\n let shared = Arc::new(AtomicBool::new(false));\n (\n AssertDrop {\n is_dropped: shared.clone(),\n },\n AssertDropHandle {\n is_dropped: shared.clone(),\n },\n )\n }\n}\nimpl Drop for AssertDrop {\n fn drop(&mut self) {\n self.is_dropped.store(true, Ordering::SeqCst);\n }\n}\n\n// A Notified does not shut down on drop, but it is dropped once the ref-count\n// hits zero.\n#[test]\nfn create_drop1() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n drop(notified);\n handle.assert_not_dropped();\n drop(join);\n handle.assert_dropped();\n}\n\n#[test]\nfn create_drop2() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n drop(join);\n handle.assert_not_dropped();\n drop(notified);\n handle.assert_dropped();\n}\n\n#[test]\nfn drop_abort_handle1() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n let abort = join.abort_handle();\n drop(join);\n handle.assert_not_dropped();\n drop(notified);\n handle.assert_not_dropped();\n drop(abort);\n handle.assert_dropped();\n}\n\n#[test]\nfn drop_abort_handle2() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n let abort = join.abort_handle();\n drop(notified);\n handle.assert_not_dropped();\n drop(abort);\n handle.assert_not_dropped();\n drop(join);\n handle.assert_dropped();\n}\n\n#[test]\nfn drop_abort_handle_clone() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n let abort = join.abort_handle();\n let abort_clone = abort.clone();\n drop(join);\n handle.assert_not_dropped();\n drop(notified);\n handle.assert_not_dropped();\n drop(abort);\n handle.assert_not_dropped();\n drop(abort_clone);\n handle.assert_dropped();\n}\n\n// Shutting down through Notified works\n#[test]\nfn create_shutdown1() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n drop(join);\n handle.assert_not_dropped();\n notified.shutdown();\n handle.assert_dropped();\n}\n\n#[test]\nfn create_shutdown2() {\n let (ad, handle) = AssertDrop::new();\n let (notified, join) = unowned(\n async {\n drop(ad);\n unreachable!()\n },\n NoopSchedule,\n Id::next(),\n SpawnLocation::capture(),\n );\n handle.assert_not_dropped();\n notified.shutdown();\n handle.assert_dropped();\n drop(join);\n}\n\n#[test]\nfn unowned_poll() {\n let (task, _) = unowned(async {}, NoopSchedule, Id::next(), SpawnLocation::capture());\n task.run();\n}\n\n#[test]\nfn schedule() {\n with(|rt| {\n rt.spawn(async {\n crate::task::yield_now().await;\n });\n\n assert_eq!(2, rt.tick());\n rt.shutdown();\n })\n}\n\n#[test]\nfn shutdown() {\n with(|rt| {\n rt.spawn(async {\n loop {\n crate::task::yield_now().await;\n }\n });\n\n rt.tick_max(1);\n\n rt.shutdown();\n })\n}\n\n#[test]\nfn shutdown_immediately() {\n with(|rt| {\n rt.spawn(async {\n loop {\n crate::task::yield_now().await;\n }\n });\n\n rt.shutdown();\n })\n}\n\n// Test for https://github.com/tokio-rs/tokio/issues/6729\n#[test]\nfn spawn_niche_in_task() {\n use std::future::poll_fn;\n use std::task::{Context, Poll, Waker};\n\n with(|rt| {\n let state = Arc::new(Mutex::new(State::new()));\n\n let mut subscriber = Subscriber::new(Arc::clone(&state), 1);\n rt.spawn(async move {\n subscriber.wait().await;\n subscriber.wait().await;\n });\n\n rt.spawn(async move {\n state.lock().unwrap().set_version(2);\n state.lock().unwrap().set_version(0);\n });\n\n rt.tick_max(10);\n assert!(rt.is_empty());\n rt.shutdown();\n });\n\n pub(crate) struct Subscriber {\n state: Arc>,\n observed_version: u64,\n waker_key: Option,\n }\n\n impl Subscriber {\n pub(crate) fn new(state: Arc>, version: u64) -> Self {\n Self {\n state,\n observed_version: version,\n waker_key: None,\n }\n }\n\n pub(crate) async fn wait(&mut self) {\n poll_fn(|cx| {\n self.state\n .lock()\n .unwrap()\n .poll_update(&mut self.observed_version, &mut self.waker_key, cx)\n .map(|_| ())\n })\n .await;\n }\n }\n\n struct State {\n version: u64,\n wakers: Vec,\n }\n\n impl State {\n pub(crate) fn new() -> Self {\n Self {\n version: 1,\n wakers: Vec::new(),\n }\n }\n\n pub(crate) fn poll_update(\n &mut self,\n observed_version: &mut u64,\n waker_key: &mut Option,\n cx: &Context<'_>,\n ) -> Poll> {\n if self.version == 0 {\n *waker_key = None;\n Poll::Ready(None)\n } else if *observed_version < self.version {\n *waker_key = None;\n *observed_version = self.version;\n Poll::Ready(Some(()))\n } else {\n self.wakers.push(cx.waker().clone());\n *waker_key = Some(self.wakers.len());\n Poll::Pending\n }\n }\n\n pub(crate) fn set_version(&mut self, version: u64) {\n self.version = version;\n for waker in self.wakers.drain(..) {\n waker.wake();\n }\n }\n }\n}\n\n#[test]\nfn spawn_during_shutdown() {\n static DID_SPAWN: AtomicBool = AtomicBool::new(false);\n\n struct SpawnOnDrop(Runtime);\n impl Drop for SpawnOnDrop {\n fn drop(&mut self) {\n DID_SPAWN.store(true, Ordering::SeqCst);\n self.0.spawn(async {});\n }\n }\n\n with(|rt| {\n let rt2 = rt.clone();\n rt.spawn(async move {\n let _spawn_on_drop = SpawnOnDrop(rt2);\n\n loop {\n crate::task::yield_now().await;\n }\n });\n\n rt.tick_max(1);\n rt.shutdown();\n });\n\n assert!(DID_SPAWN.load(Ordering::SeqCst));\n}\n\nfn with(f: impl FnOnce(Runtime)) {\n struct Reset;\n\n impl Drop for Reset {\n fn drop(&mut self) {\n let _rt = CURRENT.try_lock().unwrap().take();\n }\n }\n\n let _reset = Reset;\n\n let rt = Runtime(Arc::new(Inner {\n owned: OwnedTasks::new(16),\n core: Mutex::new(Core {\n queue: VecDeque::new(),\n }),\n }));\n\n *CURRENT.try_lock().unwrap() = Some(rt.clone());\n f(rt)\n}\n\n#[derive(Clone)]\nstruct Runtime(Arc);\n\nstruct Inner {\n core: Mutex,\n owned: OwnedTasks,\n}\n\nstruct Core {\n queue: VecDeque>,\n}\n\nstatic CURRENT: Mutex> = Mutex::new(None);\n\nimpl Runtime {\n #[track_caller]\n fn spawn(&self, future: T) -> JoinHandle\n where\n T: 'static + Send + Future,\n T::Output: 'static + Send,\n {\n let (handle, notified) =\n self.0\n .owned\n .bind(future, self.clone(), Id::next(), SpawnLocation::capture());\n\n if let Some(notified) = notified {\n self.schedule(notified);\n }\n\n handle\n }\n\n fn tick(&self) -> usize {\n self.tick_max(usize::MAX)\n }\n\n fn tick_max(&self, max: usize) -> usize {\n let mut n = 0;\n\n while !self.is_empty() && n < max {\n let task = self.next_task();\n n += 1;\n let task = self.0.owned.assert_owner(task);\n task.run();\n }\n\n n\n }\n\n fn is_empty(&self) -> bool {\n self.0.core.try_lock().unwrap().queue.is_empty()\n }\n\n fn next_task(&self) -> task::Notified {\n self.0.core.try_lock().unwrap().queue.pop_front().unwrap()\n }\n\n fn shutdown(&self) {\n let mut core = self.0.core.try_lock().unwrap();\n\n self.0.owned.close_and_shutdown_all(0);\n\n while let Some(task) = core.queue.pop_back() {\n drop(task);\n }\n\n drop(core);\n assert!(self.0.owned.is_empty());\n }\n}\n\nimpl Schedule for Runtime {\n fn release(&self, task: &Task) -> Option> {\n self.0.owned.remove(task)\n }\n\n fn schedule(&self, task: task::Notified) {\n self.0.core.try_lock().unwrap().queue.push_back(task);\n }\n\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: None,\n }\n }\n}\n" }, { "path": "tokio/src/runtime/tests/task_combinations.rs", "content": "use std::fmt;\nuse std::future::Future;\nuse std::panic;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nuse crate::runtime::task::AbortHandle;\nuse crate::runtime::Builder;\nuse crate::sync::oneshot;\nuse crate::task::JoinHandle;\n\nuse futures::future::FutureExt;\n\n// Enums for each option in the combinations being tested\n\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiRuntime {\n CurrentThread,\n Multi1,\n Multi2,\n}\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiLocalSet {\n Yes,\n No,\n}\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiTask {\n PanicOnRun,\n PanicOnDrop,\n PanicOnRunAndDrop,\n NoPanic,\n}\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiOutput {\n PanicOnDrop,\n NoPanic,\n}\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiJoinInterest {\n Polled,\n NotPolled,\n}\n#[allow(clippy::enum_variant_names)] // we aren't using glob imports\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiJoinHandle {\n DropImmediately = 1,\n DropFirstPoll = 2,\n DropAfterNoConsume = 3,\n DropAfterConsume = 4,\n}\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiAbort {\n NotAborted = 0,\n AbortedImmediately = 1,\n AbortedFirstPoll = 2,\n AbortedAfterFinish = 3,\n AbortedAfterConsumeOutput = 4,\n}\n\n#[derive(Copy, Clone, Debug, PartialEq)]\nenum CombiAbortSource {\n JoinHandle,\n AbortHandle,\n}\n\n#[test]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn test_combinations() {\n let mut rt = &[\n CombiRuntime::CurrentThread,\n CombiRuntime::Multi1,\n CombiRuntime::Multi2,\n ][..];\n\n if cfg!(miri) {\n rt = &[CombiRuntime::CurrentThread];\n }\n\n let ls = [CombiLocalSet::Yes, CombiLocalSet::No];\n let task = [\n CombiTask::NoPanic,\n CombiTask::PanicOnRun,\n CombiTask::PanicOnDrop,\n CombiTask::PanicOnRunAndDrop,\n ];\n let output = [CombiOutput::NoPanic, CombiOutput::PanicOnDrop];\n let ji = [CombiJoinInterest::Polled, CombiJoinInterest::NotPolled];\n let jh = [\n CombiJoinHandle::DropImmediately,\n CombiJoinHandle::DropFirstPoll,\n CombiJoinHandle::DropAfterNoConsume,\n CombiJoinHandle::DropAfterConsume,\n ];\n let abort = [\n CombiAbort::NotAborted,\n CombiAbort::AbortedImmediately,\n CombiAbort::AbortedFirstPoll,\n CombiAbort::AbortedAfterFinish,\n CombiAbort::AbortedAfterConsumeOutput,\n ];\n let ah = [\n None,\n Some(CombiJoinHandle::DropImmediately),\n Some(CombiJoinHandle::DropFirstPoll),\n Some(CombiJoinHandle::DropAfterNoConsume),\n Some(CombiJoinHandle::DropAfterConsume),\n ];\n\n for rt in rt.iter().copied() {\n for ls in ls.iter().copied() {\n for task in task.iter().copied() {\n for output in output.iter().copied() {\n for ji in ji.iter().copied() {\n for jh in jh.iter().copied() {\n for abort in abort.iter().copied() {\n // abort via join handle --- abort handles\n // may be dropped at any point\n for ah in ah.iter().copied() {\n test_combination(\n rt,\n ls,\n task,\n output,\n ji,\n jh,\n ah,\n abort,\n CombiAbortSource::JoinHandle,\n );\n }\n // if aborting via AbortHandle, it will\n // never be dropped.\n test_combination(\n rt,\n ls,\n task,\n output,\n ji,\n jh,\n None,\n abort,\n CombiAbortSource::AbortHandle,\n );\n }\n }\n }\n }\n }\n }\n }\n}\n\nfn is_debug(_: &T) {}\n\n#[allow(clippy::too_many_arguments)]\nfn test_combination(\n rt: CombiRuntime,\n ls: CombiLocalSet,\n task: CombiTask,\n output: CombiOutput,\n ji: CombiJoinInterest,\n jh: CombiJoinHandle,\n ah: Option,\n abort: CombiAbort,\n abort_src: CombiAbortSource,\n) {\n match (abort_src, ah) {\n (CombiAbortSource::JoinHandle, _) if (jh as usize) < (abort as usize) => {\n // join handle dropped prior to abort\n return;\n }\n (CombiAbortSource::AbortHandle, Some(_)) => {\n // abort handle dropped, we can't abort through the\n // abort handle\n return;\n }\n\n _ => {}\n }\n\n if (task == CombiTask::PanicOnDrop) && (output == CombiOutput::PanicOnDrop) {\n // this causes double panic\n return;\n }\n if (task == CombiTask::PanicOnRunAndDrop) && (abort != CombiAbort::AbortedImmediately) {\n // this causes double panic\n return;\n }\n\n is_debug(&rt);\n is_debug(&ls);\n is_debug(&task);\n is_debug(&output);\n is_debug(&ji);\n is_debug(&jh);\n is_debug(&ah);\n is_debug(&abort);\n is_debug(&abort_src);\n\n // A runtime optionally with a LocalSet\n struct Rt {\n rt: crate::runtime::Runtime,\n ls: Option,\n }\n impl Rt {\n fn new(rt: CombiRuntime, ls: CombiLocalSet) -> Self {\n let rt = match rt {\n CombiRuntime::CurrentThread => Builder::new_current_thread().build().unwrap(),\n CombiRuntime::Multi1 => Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap(),\n CombiRuntime::Multi2 => Builder::new_multi_thread()\n .worker_threads(2)\n .build()\n .unwrap(),\n };\n\n let ls = match ls {\n CombiLocalSet::Yes => Some(crate::task::LocalSet::new()),\n CombiLocalSet::No => None,\n };\n\n Self { rt, ls }\n }\n fn block_on(&self, task: T) -> T::Output\n where\n T: Future,\n {\n match &self.ls {\n Some(ls) => ls.block_on(&self.rt, task),\n None => self.rt.block_on(task),\n }\n }\n fn spawn(&self, task: T) -> JoinHandle\n where\n T: Future + Send + 'static,\n T::Output: Send + 'static,\n {\n match &self.ls {\n Some(ls) => ls.spawn_local(task),\n None => self.rt.spawn(task),\n }\n }\n }\n\n // The type used for the output of the future\n struct Output {\n panic_on_drop: bool,\n on_drop: Option>,\n }\n impl Output {\n fn disarm(&mut self) {\n self.panic_on_drop = false;\n }\n }\n impl Drop for Output {\n fn drop(&mut self) {\n let _ = self.on_drop.take().unwrap().send(());\n if self.panic_on_drop {\n panic!(\"Panicking in Output\");\n }\n }\n }\n\n // A wrapper around the future that is spawned\n struct FutWrapper {\n inner: F,\n on_drop: Option>,\n panic_on_drop: bool,\n }\n impl Future for FutWrapper {\n type Output = F::Output;\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n unsafe {\n let me = Pin::into_inner_unchecked(self);\n let inner = Pin::new_unchecked(&mut me.inner);\n inner.poll(cx)\n }\n }\n }\n impl Drop for FutWrapper {\n fn drop(&mut self) {\n let _: Result<(), ()> = self.on_drop.take().unwrap().send(());\n if self.panic_on_drop {\n panic!(\"Panicking in FutWrapper\");\n }\n }\n }\n\n // The channels passed to the task\n struct Signals {\n on_first_poll: Option>,\n wait_complete: Option>,\n on_output_drop: Option>,\n }\n\n // The task we will spawn\n async fn my_task(mut signal: Signals, task: CombiTask, out: CombiOutput) -> Output {\n // Signal that we have been polled once\n let _ = signal.on_first_poll.take().unwrap().send(());\n\n // Wait for a signal, then complete the future\n let _ = signal.wait_complete.take().unwrap().await;\n\n // If the task gets past wait_complete without yielding, then aborts\n // may not be caught without this yield_now.\n crate::task::yield_now().await;\n\n if task == CombiTask::PanicOnRun || task == CombiTask::PanicOnRunAndDrop {\n panic!(\"Panicking in my_task on {:?}\", std::thread::current().id());\n }\n\n Output {\n panic_on_drop: out == CombiOutput::PanicOnDrop,\n on_drop: signal.on_output_drop.take(),\n }\n }\n\n let rt = Rt::new(rt, ls);\n\n let (on_first_poll, wait_first_poll) = oneshot::channel();\n let (on_complete, wait_complete) = oneshot::channel();\n let (on_future_drop, wait_future_drop) = oneshot::channel();\n let (on_output_drop, wait_output_drop) = oneshot::channel();\n let signal = Signals {\n on_first_poll: Some(on_first_poll),\n wait_complete: Some(wait_complete),\n on_output_drop: Some(on_output_drop),\n };\n\n // === Spawn task ===\n let mut handle = Some(rt.spawn(FutWrapper {\n inner: my_task(signal, task, output),\n on_drop: Some(on_future_drop),\n panic_on_drop: task == CombiTask::PanicOnDrop || task == CombiTask::PanicOnRunAndDrop,\n }));\n\n // Keep track of whether the task has been killed with an abort\n let mut aborted = false;\n\n // If we want to poll the JoinHandle, do it now\n if ji == CombiJoinInterest::Polled {\n assert!(\n handle.as_mut().unwrap().now_or_never().is_none(),\n \"Polling handle succeeded\"\n );\n }\n\n // If we are either aborting the task via an abort handle, or dropping via\n // an abort handle, do that now.\n let mut abort_handle = if ah.is_some() || abort_src == CombiAbortSource::AbortHandle {\n handle.as_ref().map(JoinHandle::abort_handle)\n } else {\n None\n };\n\n let do_abort = |abort_handle: &mut Option,\n join_handle: Option<&mut JoinHandle<_>>| {\n match abort_src {\n CombiAbortSource::AbortHandle => abort_handle.take().unwrap().abort(),\n CombiAbortSource::JoinHandle => join_handle.unwrap().abort(),\n }\n };\n\n if abort == CombiAbort::AbortedImmediately {\n do_abort(&mut abort_handle, handle.as_mut());\n aborted = true;\n }\n if jh == CombiJoinHandle::DropImmediately {\n drop(handle.take().unwrap());\n }\n\n // === Wait for first poll ===\n let got_polled = rt.block_on(wait_first_poll).is_ok();\n if !got_polled {\n // it's possible that we are aborted but still got polled\n assert!(\n aborted,\n \"Task completed without ever being polled but was not aborted.\"\n );\n }\n\n if abort == CombiAbort::AbortedFirstPoll {\n do_abort(&mut abort_handle, handle.as_mut());\n aborted = true;\n }\n if jh == CombiJoinHandle::DropFirstPoll {\n drop(handle.take().unwrap());\n }\n if ah == Some(CombiJoinHandle::DropFirstPoll) {\n drop(abort_handle.take().unwrap());\n }\n\n // Signal the future that it can return now\n let _ = on_complete.send(());\n // === Wait for future to be dropped ===\n assert!(\n rt.block_on(wait_future_drop).is_ok(),\n \"The future should always be dropped.\"\n );\n\n if abort == CombiAbort::AbortedAfterFinish {\n // Don't set aborted to true here as the task already finished\n do_abort(&mut abort_handle, handle.as_mut());\n }\n if jh == CombiJoinHandle::DropAfterNoConsume {\n if ah == Some(CombiJoinHandle::DropAfterNoConsume) {\n drop(handle.take().unwrap());\n // The runtime will usually have dropped every ref-count at this point,\n // in which case dropping the AbortHandle drops the output.\n //\n // (But it might race and still hold a ref-count)\n let panic = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n drop(abort_handle.take().unwrap());\n }));\n if panic.is_err() {\n assert!(\n (output == CombiOutput::PanicOnDrop)\n && (!matches!(task, CombiTask::PanicOnRun | CombiTask::PanicOnRunAndDrop))\n && !aborted,\n \"Dropping AbortHandle shouldn't panic here\"\n );\n }\n } else {\n // The runtime will usually have dropped every ref-count at this point,\n // in which case dropping the JoinHandle drops the output.\n //\n // (But it might race and still hold a ref-count)\n let panic = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n drop(handle.take().unwrap());\n }));\n if panic.is_err() {\n assert!(\n (output == CombiOutput::PanicOnDrop)\n && (!matches!(task, CombiTask::PanicOnRun | CombiTask::PanicOnRunAndDrop))\n && !aborted,\n \"Dropping JoinHandle shouldn't panic here\"\n );\n }\n }\n }\n\n // Check whether we drop after consuming the output\n if jh == CombiJoinHandle::DropAfterConsume {\n // Using as_mut() to not immediately drop the handle\n let result = rt.block_on(handle.as_mut().unwrap());\n\n match result {\n Ok(mut output) => {\n // Don't panic here.\n output.disarm();\n assert!(!aborted, \"Task was aborted but returned output\");\n }\n Err(err) if err.is_cancelled() => assert!(aborted, \"Cancelled output but not aborted\"),\n Err(err) if err.is_panic() => {\n assert!(\n (task == CombiTask::PanicOnRun)\n || (task == CombiTask::PanicOnDrop)\n || (task == CombiTask::PanicOnRunAndDrop)\n || (output == CombiOutput::PanicOnDrop),\n \"Panic but nothing should panic\"\n );\n }\n _ => unreachable!(),\n }\n\n let mut handle = handle.take().unwrap();\n if abort == CombiAbort::AbortedAfterConsumeOutput {\n do_abort(&mut abort_handle, Some(&mut handle));\n }\n drop(handle);\n\n if ah == Some(CombiJoinHandle::DropAfterConsume) {\n drop(abort_handle.take());\n }\n }\n\n // The output should have been dropped now. Check whether the output\n // object was created at all.\n let output_created = rt.block_on(wait_output_drop).is_ok();\n assert_eq!(\n output_created,\n (!matches!(task, CombiTask::PanicOnRun | CombiTask::PanicOnRunAndDrop)) && !aborted,\n \"Creation of output object\"\n );\n}\n" }, { "path": "tokio/src/runtime/thread_id.rs", "content": "use std::num::NonZeroU64;\n\n#[derive(Eq, PartialEq, Clone, Copy, Hash, Debug)]\npub(crate) struct ThreadId(NonZeroU64);\n\nimpl ThreadId {\n pub(crate) fn next() -> Self {\n use crate::loom::sync::atomic::{AtomicU64, Ordering::Relaxed};\n\n #[cfg(all(test, loom))]\n crate::loom::lazy_static! {\n static ref NEXT_ID: AtomicU64 = AtomicU64::new(0);\n }\n\n #[cfg(not(all(test, loom)))]\n static NEXT_ID: AtomicU64 = AtomicU64::new(0);\n\n let mut last = NEXT_ID.load(Relaxed);\n loop {\n let id = match last.checked_add(1) {\n Some(id) => id,\n None => exhausted(),\n };\n\n match NEXT_ID.compare_exchange_weak(last, id, Relaxed, Relaxed) {\n Ok(_) => return ThreadId(NonZeroU64::new(id).unwrap()),\n Err(id) => last = id,\n }\n }\n }\n}\n\n#[cold]\n#[allow(dead_code)]\nfn exhausted() -> ! {\n panic!(\"failed to generate unique thread ID: bitspace exhausted\")\n}\n" }, { "path": "tokio/src/runtime/time/entry.rs", "content": "//! Timer state structures.\n//!\n//! This module contains the heart of the intrusive timer implementation, and as\n//! such the structures inside are full of tricky concurrency and unsafe code.\n//!\n//! # Ground rules\n//!\n//! The heart of the timer implementation here is the [`TimerShared`] structure,\n//! shared between the [`TimerEntry`] and the driver. Generally, we permit access\n//! to [`TimerShared`] ONLY via either 1) a mutable reference to [`TimerEntry`] or\n//! 2) a held driver lock.\n//!\n//! It follows from this that any changes made while holding BOTH 1 and 2 will\n//! be reliably visible, regardless of ordering. This is because of the `acq/rel`\n//! fences on the driver lock ensuring ordering with 2, and rust mutable\n//! reference rules for 1 (a mutable reference to an object can't be passed\n//! between threads without an `acq/rel` barrier, and same-thread we have local\n//! happens-before ordering).\n//!\n//! # State field\n//!\n//! Each timer has a state field associated with it. This field contains either\n//! the current scheduled time, or a special flag value indicating its state.\n//! This state can either indicate that the timer is on the 'pending' queue (and\n//! thus will be fired with an `Ok(())` result soon) or that it has already been\n//! fired/deregistered.\n//!\n//! This single state field allows for code that is firing the timer to\n//! synchronize with any racing `reset` calls reliably.\n//!\n//! # Registered vs true timeouts\n//!\n//! To allow for the use case of a timeout that is periodically reset before\n//! expiration to be as lightweight as possible, we support optimistically\n//! lock-free timer resets, in the case where a timer is rescheduled to a later\n//! point than it was originally scheduled for.\n//!\n//! This is accomplished by lazily rescheduling timers. That is, we update the\n//! state field with the true expiration of the timer from the holder of\n//! the [`TimerEntry`]. When the driver services timers (ie, whenever it's\n//! walking lists of timers), it checks this \"true when\" value, and reschedules\n//! based on it.\n//!\n//! We do, however, also need to track what the expiration time was when we\n//! originally registered the timer; this is used to locate the right linked\n//! list when the timer is being cancelled.\n//! This is referred to as the `registered_when` internally.\n//!\n//! There is of course a race condition between timer reset and timer\n//! expiration. If the driver fails to observe the updated expiration time, it\n//! could trigger expiration of the timer too early. However, because\n//! [`mark_pending`][mark_pending] performs a compare-and-swap, it will identify this race and\n//! refuse to mark the timer as pending.\n//!\n//! [mark_pending]: TimerHandle::mark_pending\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::AtomicU64;\nuse crate::loom::sync::atomic::Ordering;\n\nuse crate::runtime::scheduler;\nuse crate::sync::AtomicWaker;\nuse crate::time::Instant;\nuse crate::util::linked_list;\n\nuse pin_project_lite::pin_project;\nuse std::task::{Context, Poll, Waker};\nuse std::{marker::PhantomPinned, pin::Pin, ptr::NonNull};\n\ntype TimerResult = Result<(), crate::time::error::Error>;\n\npub(in crate::runtime::time) const STATE_DEREGISTERED: u64 = u64::MAX;\nconst STATE_PENDING_FIRE: u64 = STATE_DEREGISTERED - 1;\nconst STATE_MIN_VALUE: u64 = STATE_PENDING_FIRE;\n/// The largest safe integer to use for ticks.\n///\n/// This value should be updated if any other signal values are added above.\npub(super) const MAX_SAFE_MILLIS_DURATION: u64 = STATE_MIN_VALUE - 1;\n\n/// This structure holds the current shared state of the timer - its scheduled\n/// time (if registered), or otherwise the result of the timer completing, as\n/// well as the registered waker.\n///\n/// Generally, the `StateCell` is only permitted to be accessed from two contexts:\n/// Either a thread holding the corresponding `&mut TimerEntry`, or a thread\n/// holding the timer driver lock. The write actions on the `StateCell` amount to\n/// passing \"ownership\" of the `StateCell` between these contexts; moving a timer\n/// from the `TimerEntry` to the driver requires _both_ holding the `&mut\n/// TimerEntry` and the driver lock, while moving it back (firing the timer)\n/// requires only the driver lock.\npub(super) struct StateCell {\n /// Holds either the scheduled expiration time for this timer, or (if the\n /// timer has been fired and is unregistered), `u64::MAX`.\n state: AtomicU64,\n /// If the timer is fired (an Acquire order read on state shows\n /// `u64::MAX`), holds the result that should be returned from\n /// polling the timer. Otherwise, the contents are unspecified and reading\n /// without holding the driver lock is undefined behavior.\n result: UnsafeCell,\n /// The currently-registered waker\n waker: AtomicWaker,\n}\n\nimpl Default for StateCell {\n fn default() -> Self {\n Self::new()\n }\n}\n\nimpl std::fmt::Debug for StateCell {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n write!(f, \"StateCell({:?})\", self.read_state())\n }\n}\n\nimpl StateCell {\n fn new() -> Self {\n Self {\n state: AtomicU64::new(STATE_DEREGISTERED),\n result: UnsafeCell::new(Ok(())),\n waker: AtomicWaker::new(),\n }\n }\n\n fn is_pending(&self) -> bool {\n self.state.load(Ordering::Relaxed) == STATE_PENDING_FIRE\n }\n\n /// Returns the current expiration time, or None if not currently scheduled.\n fn when(&self) -> Option {\n let cur_state = self.state.load(Ordering::Relaxed);\n\n if cur_state == STATE_DEREGISTERED {\n None\n } else {\n Some(cur_state)\n }\n }\n\n /// If the timer is completed, returns the result of the timer. Otherwise,\n /// returns None and registers the waker.\n fn poll(&self, waker: &Waker) -> Poll {\n // We must register first. This ensures that either `fire` will\n // observe the new waker, or we will observe a racing fire to have set\n // the state, or both.\n self.waker.register_by_ref(waker);\n\n self.read_state()\n }\n\n fn read_state(&self) -> Poll {\n let cur_state = self.state.load(Ordering::Acquire);\n\n if cur_state == STATE_DEREGISTERED {\n // SAFETY: The driver has fired this timer; this involves writing\n // the result, and then writing (with release ordering) the state\n // field.\n Poll::Ready(unsafe { self.result.with(|p| *p) })\n } else {\n Poll::Pending\n }\n }\n\n /// Marks this timer as being moved to the pending list, if its scheduled\n /// time is not after `not_after`.\n ///\n /// If the timer is scheduled for a time after `not_after`, returns an Err\n /// containing the current scheduled time.\n ///\n /// SAFETY: Must hold the driver lock.\n unsafe fn mark_pending(&self, not_after: u64) -> Result<(), u64> {\n // Quick initial debug check to see if the timer is already fired. Since\n // firing the timer can only happen with the driver lock held, we know\n // we shouldn't be able to \"miss\" a transition to a fired state, even\n // with relaxed ordering.\n let mut cur_state = self.state.load(Ordering::Relaxed);\n\n loop {\n // improve the error message for things like\n // https://github.com/tokio-rs/tokio/issues/3675\n assert!(\n cur_state < STATE_MIN_VALUE,\n \"mark_pending called when the timer entry is in an invalid state\"\n );\n\n if cur_state > not_after {\n break Err(cur_state);\n }\n\n match self.state.compare_exchange_weak(\n cur_state,\n STATE_PENDING_FIRE,\n Ordering::AcqRel,\n Ordering::Acquire,\n ) {\n Ok(_) => break Ok(()),\n Err(actual_state) => cur_state = actual_state,\n }\n }\n }\n\n /// Fires the timer, setting the result to the provided result.\n ///\n /// Returns:\n /// * `Some(waker)` - if fired and a waker needs to be invoked once the\n /// driver lock is released\n /// * `None` - if fired and a waker does not need to be invoked, or if\n /// already fired\n ///\n /// SAFETY: The driver lock must be held.\n unsafe fn fire(&self, result: TimerResult) -> Option {\n // Quick initial check to see if the timer is already fired. Since\n // firing the timer can only happen with the driver lock held, we know\n // we shouldn't be able to \"miss\" a transition to a fired state, even\n // with relaxed ordering.\n let cur_state = self.state.load(Ordering::Relaxed);\n if cur_state == STATE_DEREGISTERED {\n return None;\n }\n\n // SAFETY: We assume the driver lock is held and the timer is not\n // fired, so only the driver is accessing this field.\n //\n // We perform a release-ordered store to state below, to ensure this\n // write is visible before the state update is visible.\n unsafe { self.result.with_mut(|p| *p = result) };\n\n self.state.store(STATE_DEREGISTERED, Ordering::Release);\n\n self.waker.take_waker()\n }\n\n /// Marks the timer as registered (poll will return None) and sets the\n /// expiration time.\n ///\n /// While this function is memory-safe, it should only be called from a\n /// context holding both `&mut TimerEntry` and the driver lock.\n fn set_expiration(&self, timestamp: u64) {\n debug_assert!(timestamp < STATE_MIN_VALUE);\n\n // We can use relaxed ordering because we hold the driver lock and will\n // fence when we release the lock.\n self.state.store(timestamp, Ordering::Relaxed);\n }\n\n /// Attempts to adjust the timer to a new timestamp.\n ///\n /// If the timer has already been fired, is pending firing, or the new\n /// timestamp is earlier than the old timestamp, (or occasionally\n /// spuriously) returns Err without changing the timer's state. In this\n /// case, the timer must be deregistered and re-registered.\n fn extend_expiration(&self, new_timestamp: u64) -> Result<(), ()> {\n let mut prior = self.state.load(Ordering::Relaxed);\n loop {\n if new_timestamp < prior || prior >= STATE_MIN_VALUE {\n return Err(());\n }\n\n match self.state.compare_exchange_weak(\n prior,\n new_timestamp,\n Ordering::AcqRel,\n Ordering::Acquire,\n ) {\n Ok(_) => return Ok(()),\n Err(true_prior) => prior = true_prior,\n }\n }\n }\n\n /// Returns true if the state of this timer indicates that the timer might\n /// be registered with the driver. This check is performed with relaxed\n /// ordering, but is conservative - if it returns false, the timer is\n /// definitely _not_ registered.\n pub(super) fn might_be_registered(&self) -> bool {\n self.state.load(Ordering::Relaxed) != STATE_DEREGISTERED\n }\n}\n\npin_project! {\n // A timer entry.\n //\n // This is the handle to a timer that is controlled by the requester of the\n // timer. As this participates in intrusive data structures, it must be pinned\n // before polling.\n #[derive(Debug)]\n pub(crate) struct TimerEntry {\n // Arc reference to the runtime handle. We can only free the driver after\n // deregistering everything from their respective timer wheels.\n driver: scheduler::Handle,\n // Shared inner structure; this is part of an intrusive linked list, and\n // therefore other references can exist to it while mutable references to\n // Entry exist.\n //\n // This is manipulated only under the inner mutex.\n #[pin]\n inner: Option,\n // Deadline for the timer. This is used to register on the first\n // poll, as we can't register prior to being pinned.\n deadline: Instant,\n // Whether the deadline has been registered.\n registered: bool,\n }\n\n impl PinnedDrop for TimerEntry {\n fn drop(this: Pin<&mut Self>) {\n this.cancel();\n }\n }\n}\n\nunsafe impl Send for TimerEntry {}\nunsafe impl Sync for TimerEntry {}\n\n/// An `TimerHandle` is the (non-enforced) \"unique\" pointer from the driver to the\n/// timer entry. Generally, at most one `TimerHandle` exists for a timer at a time\n/// (enforced by the timer state machine).\n///\n/// SAFETY: An `TimerHandle` is essentially a raw pointer, and the usual caveats\n/// of pointer safety apply. In particular, `TimerHandle` does not itself enforce\n/// that the timer does still exist; however, normally an `TimerHandle` is created\n/// immediately before registering the timer, and is consumed when firing the\n/// timer, to help minimize mistakes. Still, because `TimerHandle` cannot enforce\n/// memory safety, all operations are unsafe.\n#[derive(Debug)]\npub(crate) struct TimerHandle {\n inner: NonNull,\n}\n\npub(super) type EntryList = crate::util::linked_list::LinkedList;\n\n/// The shared state structure of a timer. This structure is shared between the\n/// frontend (`Entry`) and driver backend.\n///\n/// Note that this structure is located inside the `TimerEntry` structure.\npub(crate) struct TimerShared {\n /// A link within the doubly-linked list of timers on a particular level and\n /// slot. Valid only if state is equal to Registered.\n ///\n /// Only accessed under the entry lock.\n pointers: linked_list::Pointers,\n\n /// The time when the [`TimerEntry`] was registered into the Wheel,\n /// [`STATE_DEREGISTERED`] means it is not registered.\n ///\n /// Generally owned by the driver, but is accessed by the entry when not\n /// registered.\n ///\n /// We use relaxed ordering for both loading and storing since this value\n /// is only accessed either when holding the driver lock or through mutable\n /// references to [`TimerEntry`].\n registered_when: AtomicU64,\n\n /// Current state. This records whether the timer entry is currently under\n /// the ownership of the driver, and if not, its current state (not\n /// complete, fired, error, etc).\n state: StateCell,\n\n _p: PhantomPinned,\n}\n\nunsafe impl Send for TimerShared {}\nunsafe impl Sync for TimerShared {}\n\nimpl std::fmt::Debug for TimerShared {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"TimerShared\")\n .field(\n \"registered_when\",\n &self.registered_when.load(Ordering::Relaxed),\n )\n .field(\"state\", &self.state)\n .finish()\n }\n}\n\ngenerate_addr_of_methods! {\n impl<> TimerShared {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull> {\n &self.pointers\n }\n }\n}\n\nimpl TimerShared {\n pub(super) fn new() -> Self {\n Self {\n registered_when: AtomicU64::new(0),\n pointers: linked_list::Pointers::new(),\n state: StateCell::default(),\n _p: PhantomPinned,\n }\n }\n\n /// Gets the cached time-of-expiration value.\n pub(super) fn registered_when(&self) -> u64 {\n // Cached-when is only accessed under the driver lock, so we can use relaxed\n self.registered_when.load(Ordering::Relaxed)\n }\n\n /// Gets the true time-of-expiration value, and copies it into the cached\n /// time-of-expiration value.\n ///\n /// SAFETY: Must be called with the driver lock held, and when this entry is\n /// not in any timer wheel lists.\n pub(super) unsafe fn sync_when(&self) -> u64 {\n let true_when = self.true_when();\n\n self.registered_when.store(true_when, Ordering::Relaxed);\n\n true_when\n }\n\n /// Sets the cached time-of-expiration value.\n ///\n /// SAFETY: Must be called with the driver lock held, and when this entry is\n /// not in any timer wheel lists.\n unsafe fn set_registered_when(&self, when: u64) {\n self.registered_when.store(when, Ordering::Relaxed);\n }\n\n /// Returns the true time-of-expiration value, with relaxed memory ordering.\n pub(super) fn true_when(&self) -> u64 {\n self.state.when().expect(\"Timer already fired\")\n }\n\n /// Sets the true time-of-expiration value, even if it is less than the\n /// current expiration or the timer is deregistered.\n ///\n /// SAFETY: Must only be called with the driver lock held and the entry not\n /// in the timer wheel.\n pub(super) unsafe fn set_expiration(&self, t: u64) {\n self.state.set_expiration(t);\n self.registered_when.store(t, Ordering::Relaxed);\n }\n\n /// Sets the true time-of-expiration only if it is after the current.\n pub(super) fn extend_expiration(&self, t: u64) -> Result<(), ()> {\n self.state.extend_expiration(t)\n }\n\n /// Returns a `TimerHandle` for this timer.\n pub(super) fn handle(&self) -> TimerHandle {\n TimerHandle {\n inner: NonNull::from(self),\n }\n }\n\n /// Returns true if the state of this timer indicates that the timer might\n /// be registered with the driver. This check is performed with relaxed\n /// ordering, but is conservative - if it returns false, the timer is\n /// definitely _not_ registered.\n pub(super) fn might_be_registered(&self) -> bool {\n self.state.might_be_registered()\n }\n}\n\nunsafe impl linked_list::Link for TimerShared {\n type Handle = TimerHandle;\n\n type Target = TimerShared;\n\n fn as_raw(handle: &Self::Handle) -> NonNull {\n handle.inner\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle {\n TimerHandle { inner: ptr }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n unsafe { TimerShared::addr_of_pointers(target) }\n }\n}\n\n// ===== impl Entry =====\n\nimpl TimerEntry {\n #[track_caller]\n pub(crate) fn new(handle: scheduler::Handle, deadline: Instant) -> Self {\n // Panic if the time driver is not enabled\n let _ = handle.driver().time();\n\n Self {\n driver: handle,\n inner: None,\n deadline,\n registered: false,\n }\n }\n\n fn inner(&self) -> Option<&TimerShared> {\n self.inner.as_ref()\n }\n\n fn init_inner(self: Pin<&mut Self>) {\n match self.inner {\n Some(_) => {}\n None => self.project().inner.set(Some(TimerShared::new())),\n }\n }\n\n pub(crate) fn deadline(&self) -> Instant {\n self.deadline\n }\n\n pub(crate) fn is_elapsed(&self) -> bool {\n let Some(inner) = self.inner() else {\n return false;\n };\n\n // Is this timer still in the timer wheel?\n let deregistered = !inner.might_be_registered();\n\n // Once the timer has expired,\n // it will be taken out of the wheel and be fired.\n //\n // So if we have already registered the timer into the wheel,\n // but now it is not in the wheel, it means that it has been\n // fired.\n //\n // +--------------+-----------------+----------+\n // | deregistered | self.registered | output |\n // +--------------+-----------------+----------+\n // | true | false | false | <- never been registered\n // +--------------+-----------------+----------+\n // | false | false | false | <- never been registered\n // +--------------+-----------------+----------+\n // | true | true | true | <- registered into the wheel,\n // | | | | and then taken out of the wheel.\n // +--------------+-----------------+----------+\n // | false | true | false | <- still registered in the wheel\n // +--------------+-----------------+----------+\n deregistered && self.registered\n }\n\n /// Cancels and deregisters the timer. This operation is irreversible.\n pub(crate) fn cancel(self: Pin<&mut Self>) {\n // Avoid calling the `clear_entry` method, because it has not been initialized yet.\n let Some(inner) = self.inner() else {\n return;\n };\n\n // We need to perform an acq/rel fence with the driver thread, and the\n // simplest way to do so is to grab the driver lock.\n //\n // Why is this necessary? We're about to release this timer's memory for\n // some other non-timer use. However, we've been doing a bunch of\n // relaxed (or even non-atomic) writes from the driver thread, and we'll\n // be doing more from _this thread_ (as this memory is interpreted as\n // something else).\n //\n // It is critical to ensure that, from the point of view of the driver,\n // those future non-timer writes happen-after the timer is fully fired,\n // and from the purpose of this thread, the driver's writes all\n // happen-before we drop the timer. This in turn requires us to perform\n // an acquire-release barrier in _both_ directions between the driver\n // and dropping thread.\n //\n // The lock acquisition in clear_entry serves this purpose. All of the\n // driver manipulations happen with the lock held, so we can just take\n // the lock and be sure that this drop happens-after everything the\n // driver did so far and happens-before everything the driver does in\n // the future. While we have the lock held, we also go ahead and\n // deregister the entry if necessary.\n unsafe { self.driver().clear_entry(NonNull::from(inner)) };\n }\n\n pub(crate) fn reset(mut self: Pin<&mut Self>, new_time: Instant, reregister: bool) {\n let this = self.as_mut().project();\n *this.deadline = new_time;\n *this.registered = reregister;\n\n let tick = self.driver().time_source().deadline_to_tick(new_time);\n let inner = match self.inner() {\n Some(inner) => inner,\n None => {\n self.as_mut().init_inner();\n self.inner()\n .expect(\"inner should already be initialized by `this.init_inner()`\")\n }\n };\n\n if inner.extend_expiration(tick).is_ok() {\n return;\n }\n\n if reregister {\n unsafe {\n self.driver()\n .reregister(&self.driver.driver().io, tick, inner.into());\n }\n }\n }\n\n pub(crate) fn poll_elapsed(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n assert!(\n !self.driver().is_shutdown(),\n \"{}\",\n crate::util::error::RUNTIME_SHUTTING_DOWN_ERROR\n );\n\n if !self.registered {\n let deadline = self.deadline;\n self.as_mut().reset(deadline, true);\n }\n\n let inner = self\n .inner()\n .expect(\"inner should already be initialized by `self.reset()`\");\n inner.state.poll(cx.waker())\n }\n\n pub(crate) fn driver(&self) -> &super::Handle {\n self.driver.driver().time()\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) fn clock(&self) -> &super::Clock {\n self.driver.driver().clock()\n }\n}\n\nimpl TimerHandle {\n pub(super) unsafe fn registered_when(&self) -> u64 {\n unsafe { self.inner.as_ref().registered_when() }\n }\n\n pub(super) unsafe fn sync_when(&self) -> u64 {\n unsafe { self.inner.as_ref().sync_when() }\n }\n\n pub(super) unsafe fn is_pending(&self) -> bool {\n unsafe { self.inner.as_ref().state.is_pending() }\n }\n\n /// Forcibly sets the true and cached expiration times to the given tick.\n ///\n /// SAFETY: The caller must ensure that the handle remains valid, the driver\n /// lock is held, and that the timer is not in any wheel linked lists.\n pub(super) unsafe fn set_expiration(&self, tick: u64) {\n unsafe {\n self.inner.as_ref().set_expiration(tick);\n }\n }\n\n /// Attempts to mark this entry as pending. If the expiration time is after\n /// `not_after`, however, returns an Err with the current expiration time.\n ///\n /// If an `Err` is returned, the `registered_when` value will be updated to this\n /// new expiration time.\n ///\n /// SAFETY: The caller must ensure that the handle remains valid, the driver\n /// lock is held, and that the timer is not in any wheel linked lists.\n /// After returning Ok, the entry must be added to the pending list.\n pub(super) unsafe fn mark_pending(&self, not_after: u64) -> Result<(), u64> {\n match unsafe { self.inner.as_ref().state.mark_pending(not_after) } {\n Ok(()) => {\n // mark this as being on the pending queue in registered_when\n unsafe {\n self.inner.as_ref().set_registered_when(STATE_DEREGISTERED);\n }\n Ok(())\n }\n Err(tick) => {\n unsafe {\n self.inner.as_ref().set_registered_when(tick);\n }\n Err(tick)\n }\n }\n }\n\n /// Attempts to transition to a terminal state. If the state is already a\n /// terminal state, does nothing.\n ///\n /// Because the entry might be dropped after the state is moved to a\n /// terminal state, this function consumes the handle to ensure we don't\n /// access the entry afterwards.\n ///\n /// Returns the last-registered waker, if any.\n ///\n /// SAFETY: The driver lock must be held while invoking this function, and\n /// the entry must not be in any wheel linked lists.\n pub(super) unsafe fn fire(self, completed_state: TimerResult) -> Option {\n unsafe { self.inner.as_ref().state.fire(completed_state) }\n }\n}\n" }, { "path": "tokio/src/runtime/time/handle.rs", "content": "use crate::runtime::time::TimeSource;\nuse std::fmt;\n\n/// Handle to time driver instance.\npub(crate) struct Handle {\n pub(super) time_source: TimeSource,\n pub(super) inner: super::Inner,\n}\n\nimpl Handle {\n /// Returns the time source associated with this handle.\n pub(crate) fn time_source(&self) -> &TimeSource {\n &self.time_source\n }\n\n /// Checks whether the driver has been shutdown.\n pub(super) fn is_shutdown(&self) -> bool {\n self.inner.is_shutdown()\n }\n\n /// Track that the driver is being unparked\n pub(crate) fn unpark(&self) {\n #[cfg(feature = \"test-util\")]\n match self.inner {\n super::Inner::Traditional { ref did_wake, .. } => {\n did_wake.store(true, std::sync::atomic::Ordering::SeqCst);\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n super::Inner::Alternative { ref did_wake, .. } => {\n did_wake.store(true, std::sync::atomic::Ordering::SeqCst);\n }\n }\n }\n}\n\ncfg_not_rt! {\n impl Handle {\n /// Tries to get a handle to the current timer.\n ///\n /// # Panics\n ///\n /// This function panics if there is no current timer set.\n ///\n /// It can be triggered when [`Builder::enable_time`] or\n /// [`Builder::enable_all`] are not included in the builder.\n ///\n /// It can also panic whenever a timer is created outside of a\n /// Tokio runtime. That is why `rt.block_on(sleep(...))` will panic,\n /// since the function is executed outside of the runtime.\n /// Whereas `rt.block_on(async {sleep(...).await})` doesn't panic.\n /// And this is because wrapping the function on an async makes it lazy,\n /// and so gets executed inside the runtime successfully without\n /// panicking.\n ///\n /// [`Builder::enable_time`]: crate::runtime::Builder::enable_time\n /// [`Builder::enable_all`]: crate::runtime::Builder::enable_all\n #[track_caller]\n pub(crate) fn current() -> Self {\n panic!(\"{}\", crate::util::error::CONTEXT_MISSING_ERROR)\n }\n }\n}\n\nimpl fmt::Debug for Handle {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(f, \"Handle\")\n }\n}\n" }, { "path": "tokio/src/runtime/time/mod.rs", "content": "// Currently, rust warns when an unsafe fn contains an unsafe {} block. However,\n// in the future, this will change to the reverse. For now, suppress this\n// warning and generally stick with being explicit about unsafety.\n#![allow(unused_unsafe)]\n#![cfg_attr(not(feature = \"rt\"), allow(dead_code))]\n\n//! Time driver.\n\nmod entry;\npub(crate) use entry::TimerEntry;\nuse entry::{EntryList, TimerHandle, TimerShared, MAX_SAFE_MILLIS_DURATION};\n\nmod handle;\npub(crate) use self::handle::Handle;\n\nmod source;\npub(crate) use source::TimeSource;\n\nmod wheel;\n\n#[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\nuse super::time_alt;\n\nuse crate::loom::sync::atomic::{AtomicBool, Ordering};\nuse crate::loom::sync::Mutex;\nuse crate::runtime::driver::{self, IoHandle, IoStack};\nuse crate::time::error::Error;\nuse crate::time::{Clock, Duration};\nuse crate::util::WakeList;\n\nuse std::fmt;\nuse std::{num::NonZeroU64, ptr::NonNull};\n\n/// Time implementation that drives [`Sleep`][sleep], [`Interval`][interval], and [`Timeout`][timeout].\n///\n/// A `Driver` instance tracks the state necessary for managing time and\n/// notifying the [`Sleep`][sleep] instances once their deadlines are reached.\n///\n/// It is expected that a single instance manages many individual [`Sleep`][sleep]\n/// instances. The `Driver` implementation is thread-safe and, as such, is able\n/// to handle callers from across threads.\n///\n/// After creating the `Driver` instance, the caller must repeatedly call `park`\n/// or `park_timeout`. The time driver will perform no work unless `park` or\n/// `park_timeout` is called repeatedly.\n///\n/// The driver has a resolution of one millisecond. Any unit of time that falls\n/// between milliseconds are rounded up to the next millisecond.\n///\n/// When an instance is dropped, any outstanding [`Sleep`][sleep] instance that has not\n/// elapsed will be notified with an error. At this point, calling `poll` on the\n/// [`Sleep`][sleep] instance will result in panic.\n///\n/// # Implementation\n///\n/// The time driver is based on the [paper by Varghese and Lauck][paper].\n///\n/// A hashed timing wheel is a vector of slots, where each slot handles a time\n/// slice. As time progresses, the timer walks over the slot for the current\n/// instant, and processes each entry for that slot. When the timer reaches the\n/// end of the wheel, it starts again at the beginning.\n///\n/// The implementation maintains six wheels arranged in a set of levels. As the\n/// levels go up, the slots of the associated wheel represent larger intervals\n/// of time. At each level, the wheel has 64 slots. Each slot covers a range of\n/// time equal to the wheel at the lower level. At level zero, each slot\n/// represents one millisecond of time.\n///\n/// The wheels are:\n///\n/// * Level 0: 64 x 1 millisecond slots.\n/// * Level 1: 64 x 64 millisecond slots.\n/// * Level 2: 64 x ~4 second slots.\n/// * Level 3: 64 x ~4 minute slots.\n/// * Level 4: 64 x ~4 hour slots.\n/// * Level 5: 64 x ~12 day slots.\n///\n/// When the timer processes entries at level zero, it will notify all the\n/// `Sleep` instances as their deadlines have been reached. For all higher\n/// levels, all entries will be redistributed across the wheel at the next level\n/// down. Eventually, as time progresses, entries with [`Sleep`][sleep] instances will\n/// either be canceled (dropped) or their associated entries will reach level\n/// zero and be notified.\n///\n/// [paper]: http://www.cs.columbia.edu/~nahum/w6998/papers/ton97-timing-wheels.pdf\n/// [sleep]: crate::time::Sleep\n/// [timeout]: crate::time::Timeout\n/// [interval]: crate::time::Interval\n#[derive(Debug)]\npub(crate) struct Driver {\n /// Parker to delegate to.\n park: IoStack,\n}\n\nenum Inner {\n Traditional {\n // The state is split like this so `Handle` can access `is_shutdown` without locking the mutex\n state: Mutex,\n\n /// True if the driver is being shutdown.\n is_shutdown: AtomicBool,\n\n // When `true`, a call to `park_timeout` should immediately return and time\n // should not advance. One reason for this to be `true` is if the task\n // passed to `Runtime::block_on` called `task::yield_now()`.\n //\n // While it may look racy, it only has any effect when the clock is paused\n // and pausing the clock is restricted to a single-threaded runtime.\n #[cfg(feature = \"test-util\")]\n did_wake: AtomicBool,\n },\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Alternative {\n /// True if the driver is being shutdown.\n is_shutdown: AtomicBool,\n\n // When `true`, a call to `park_timeout` should immediately return and time\n // should not advance. One reason for this to be `true` is if the task\n // passed to `Runtime::block_on` called `task::yield_now()`.\n //\n // While it may look racy, it only has any effect when the clock is paused\n // and pausing the clock is restricted to a single-threaded runtime.\n #[cfg(feature = \"test-util\")]\n did_wake: AtomicBool,\n },\n}\n\n/// Time state shared which must be protected by a `Mutex`\nstruct InnerState {\n /// The earliest time at which we promise to wake up without unparking.\n next_wake: Option,\n\n /// Timer wheel.\n wheel: wheel::Wheel,\n}\n\n// ===== impl Driver =====\n\nimpl Driver {\n /// Creates a new `Driver` instance that uses `park` to block the current\n /// thread and `time_source` to get the current time and convert to ticks.\n ///\n /// Specifying the source of time is useful when testing.\n pub(crate) fn new(park: IoStack, clock: &Clock) -> (Driver, Handle) {\n let time_source = TimeSource::new(clock);\n\n let handle = Handle {\n time_source,\n inner: Inner::Traditional {\n state: Mutex::new(InnerState {\n next_wake: None,\n wheel: wheel::Wheel::new(),\n }),\n is_shutdown: AtomicBool::new(false),\n\n #[cfg(feature = \"test-util\")]\n did_wake: AtomicBool::new(false),\n },\n };\n\n let driver = Driver { park };\n\n (driver, handle)\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n pub(crate) fn new_alt(clock: &Clock) -> Handle {\n let time_source = TimeSource::new(clock);\n\n Handle {\n time_source,\n inner: Inner::Alternative {\n is_shutdown: AtomicBool::new(false),\n #[cfg(feature = \"test-util\")]\n did_wake: AtomicBool::new(false),\n },\n }\n }\n\n pub(crate) fn park(&mut self, handle: &driver::Handle) {\n self.park_internal(handle, None);\n }\n\n pub(crate) fn park_timeout(&mut self, handle: &driver::Handle, duration: Duration) {\n self.park_internal(handle, Some(duration));\n }\n\n pub(crate) fn shutdown(&mut self, rt_handle: &driver::Handle) {\n let handle = rt_handle.time();\n\n if handle.is_shutdown() {\n return;\n }\n\n match &handle.inner {\n Inner::Traditional { is_shutdown, .. } => {\n is_shutdown.store(true, Ordering::SeqCst);\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Inner::Alternative { is_shutdown, .. } => {\n is_shutdown.store(true, Ordering::SeqCst);\n }\n }\n\n // Advance time forward to the end of time.\n\n handle.process_at_time(u64::MAX);\n\n self.park.shutdown(rt_handle);\n }\n\n fn park_internal(&mut self, rt_handle: &driver::Handle, limit: Option) {\n let handle = rt_handle.time();\n let mut lock = handle.inner.lock();\n\n assert!(!handle.is_shutdown());\n\n let next_wake = lock.wheel.next_expiration_time();\n lock.next_wake =\n next_wake.map(|t| NonZeroU64::new(t).unwrap_or_else(|| NonZeroU64::new(1).unwrap()));\n\n drop(lock);\n\n match next_wake {\n Some(when) => {\n let now = handle.time_source.now(rt_handle.clock());\n // Note that we effectively round up to 1ms here - this avoids\n // very short-duration microsecond-resolution sleeps that the OS\n // might treat as zero-length.\n let mut duration = handle\n .time_source\n .tick_to_duration(when.saturating_sub(now));\n\n if duration > Duration::from_millis(0) {\n if let Some(limit) = limit {\n duration = std::cmp::min(limit, duration);\n }\n\n self.park_thread_timeout(rt_handle, duration);\n } else {\n self.park.park_timeout(rt_handle, Duration::from_secs(0));\n }\n }\n None => {\n if let Some(duration) = limit {\n self.park_thread_timeout(rt_handle, duration);\n } else {\n self.park.park(rt_handle);\n }\n }\n }\n\n // Process pending timers after waking up\n handle.process(rt_handle.clock());\n }\n\n cfg_test_util! {\n fn park_thread_timeout(&mut self, rt_handle: &driver::Handle, duration: Duration) {\n let handle = rt_handle.time();\n let clock = rt_handle.clock();\n\n if clock.can_auto_advance() {\n self.park.park_timeout(rt_handle, Duration::from_secs(0));\n\n // If the time driver was woken, then the park completed\n // before the \"duration\" elapsed (usually caused by a\n // yield in `Runtime::block_on`). In this case, we don't\n // advance the clock.\n if !handle.did_wake() {\n // Simulate advancing time\n if let Err(msg) = clock.advance(duration) {\n panic!(\"{}\", msg);\n }\n }\n } else {\n self.park.park_timeout(rt_handle, duration);\n }\n }\n }\n\n cfg_not_test_util! {\n fn park_thread_timeout(&mut self, rt_handle: &driver::Handle, duration: Duration) {\n self.park.park_timeout(rt_handle, duration);\n }\n }\n}\n\nimpl Handle {\n pub(self) fn process(&self, clock: &Clock) {\n let now = self.time_source().now(clock);\n\n self.process_at_time(now);\n }\n\n pub(self) fn process_at_time(&self, mut now: u64) {\n let mut waker_list = WakeList::new();\n\n let mut lock = self.inner.lock();\n\n if now < lock.wheel.elapsed() {\n // Time went backwards! This normally shouldn't happen as the Rust language\n // guarantees that an Instant is monotonic, but can happen when running\n // Linux in a VM on a Windows host due to std incorrectly trusting the\n // hardware clock to be monotonic.\n //\n // See for more information.\n now = lock.wheel.elapsed();\n }\n\n while let Some(entry) = lock.wheel.poll(now) {\n debug_assert!(unsafe { entry.is_pending() });\n\n // SAFETY: We hold the driver lock, and just removed the entry from any linked lists.\n if let Some(waker) = unsafe { entry.fire(Ok(())) } {\n waker_list.push(waker);\n\n if !waker_list.can_push() {\n // Wake a batch of wakers. To avoid deadlock, we must do this with the lock temporarily dropped.\n drop(lock);\n\n waker_list.wake_all();\n\n lock = self.inner.lock();\n }\n }\n }\n\n lock.next_wake = lock\n .wheel\n .poll_at()\n .map(|t| NonZeroU64::new(t).unwrap_or_else(|| NonZeroU64::new(1).unwrap()));\n\n drop(lock);\n\n waker_list.wake_all();\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n pub(crate) fn process_at_time_alt(\n &self,\n wheel: &mut time_alt::Wheel,\n mut now: u64,\n wake_queue: &mut time_alt::WakeQueue,\n ) {\n if now < wheel.elapsed() {\n // Time went backwards! This normally shouldn't happen as the Rust language\n // guarantees that an Instant is monotonic, but can happen when running\n // Linux in a VM on a Windows host due to std incorrectly trusting the\n // hardware clock to be monotonic.\n //\n // See for more information.\n now = wheel.elapsed();\n }\n\n wheel.take_expired(now, wake_queue);\n }\n\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n pub(crate) fn shutdown_alt(&self, wheel: &mut time_alt::Wheel) {\n // self.is_shutdown.store(true, Ordering::SeqCst);\n // Advance time forward to the end of time.\n // This will ensure that all timers are fired.\n let max_tick = u64::MAX;\n let mut wake_queue = time_alt::WakeQueue::new();\n self.process_at_time_alt(wheel, max_tick, &mut wake_queue);\n wake_queue.wake_all();\n }\n\n /// Removes a registered timer from the driver.\n ///\n /// The timer will be moved to the cancelled state. Wakers will _not_ be\n /// invoked. If the timer is already completed, this function is a no-op.\n ///\n /// This function always acquires the driver lock, even if the entry does\n /// not appear to be registered.\n ///\n /// SAFETY: The timer must not be registered with some other driver, and\n /// `add_entry` must not be called concurrently.\n pub(self) unsafe fn clear_entry(&self, entry: NonNull) {\n unsafe {\n let mut lock = self.inner.lock();\n\n if entry.as_ref().might_be_registered() {\n lock.wheel.remove(entry);\n }\n\n entry.as_ref().handle().fire(Ok(()));\n }\n }\n\n /// Removes and re-adds an entry to the driver.\n ///\n /// SAFETY: The timer must be either unregistered, or registered with this\n /// driver. No other threads are allowed to concurrently manipulate the\n /// timer at all (the current thread should hold an exclusive reference to\n /// the `TimerEntry`)\n pub(self) unsafe fn reregister(\n &self,\n unpark: &IoHandle,\n new_tick: u64,\n entry: NonNull,\n ) {\n let waker = unsafe {\n let mut lock = self.inner.lock();\n\n // We may have raced with a firing/deregistration, so check before\n // deregistering.\n if unsafe { entry.as_ref().might_be_registered() } {\n lock.wheel.remove(entry);\n }\n\n // Now that we have exclusive control of this entry, mint a handle to reinsert it.\n let entry = entry.as_ref().handle();\n\n if self.is_shutdown() {\n unsafe { entry.fire(Err(crate::time::error::Error::shutdown())) }\n } else {\n entry.set_expiration(new_tick);\n\n // Note: We don't have to worry about racing with some other resetting\n // thread, because add_entry and reregister require exclusive control of\n // the timer entry.\n match unsafe { lock.wheel.insert(entry) } {\n Ok(when) => {\n if lock\n .next_wake\n .map(|next_wake| when < next_wake.get())\n .unwrap_or(true)\n {\n unpark.unpark();\n }\n\n None\n }\n Err((entry, crate::time::error::InsertError::Elapsed)) => unsafe {\n entry.fire(Ok(()))\n },\n }\n }\n\n // Must release lock before invoking waker to avoid the risk of deadlock.\n };\n\n // The timer was fired synchronously as a result of the reregistration.\n // Wake the waker; this is needed because we might reset _after_ a poll,\n // and otherwise the task won't be awoken to poll again.\n if let Some(waker) = waker {\n waker.wake();\n }\n }\n\n cfg_test_util! {\n pub(super) fn did_wake(&self) -> bool {\n match &self.inner {\n Inner::Traditional { did_wake, .. } => did_wake.swap(false, Ordering::SeqCst),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Inner::Alternative { did_wake, .. } => did_wake.swap(false, Ordering::SeqCst),\n }\n }\n }\n}\n\n// ===== impl Inner =====\n\nimpl Inner {\n /// Locks the driver's inner structure\n pub(super) fn lock(&self) -> crate::loom::sync::MutexGuard<'_, InnerState> {\n match self {\n Inner::Traditional { state, .. } => state.lock(),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Inner::Alternative { .. } => unreachable!(\"unreachable in alternative timer\"),\n }\n }\n\n // Check whether the driver has been shutdown\n pub(super) fn is_shutdown(&self) -> bool {\n match self {\n Inner::Traditional { is_shutdown, .. } => is_shutdown.load(Ordering::SeqCst),\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n Inner::Alternative { is_shutdown, .. } => is_shutdown.load(Ordering::SeqCst),\n }\n }\n}\n\nimpl fmt::Debug for Inner {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Inner\").finish()\n }\n}\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/runtime/time/source.rs", "content": "use super::MAX_SAFE_MILLIS_DURATION;\nuse crate::time::{Clock, Duration, Instant};\n\n/// A structure which handles conversion from Instants to `u64` timestamps.\n#[derive(Debug)]\npub(crate) struct TimeSource {\n start_time: Instant,\n}\n\nimpl TimeSource {\n pub(crate) fn new(clock: &Clock) -> Self {\n Self {\n start_time: clock.now(),\n }\n }\n\n pub(crate) fn deadline_to_tick(&self, t: Instant) -> u64 {\n // Round up to the end of a ms\n self.instant_to_tick(t + Duration::from_nanos(999_999))\n }\n\n pub(crate) fn instant_to_tick(&self, t: Instant) -> u64 {\n // round up\n let dur: Duration = t.saturating_duration_since(self.start_time);\n let ms = dur\n .as_millis()\n .try_into()\n .unwrap_or(MAX_SAFE_MILLIS_DURATION);\n ms.min(MAX_SAFE_MILLIS_DURATION)\n }\n\n pub(crate) fn tick_to_duration(&self, t: u64) -> Duration {\n Duration::from_millis(t)\n }\n\n pub(crate) fn now(&self, clock: &Clock) -> u64 {\n self.instant_to_tick(clock.now())\n }\n\n #[cfg(test)]\n #[allow(dead_code)]\n pub(super) fn start_time(&self) -> Instant {\n self.start_time\n }\n}\n" }, { "path": "tokio/src/runtime/time/tests/mod.rs", "content": "#![cfg(not(target_os = \"wasi\"))]\n\nuse std::{task::Context, time::Duration};\n\n#[cfg(not(loom))]\nuse futures::task::noop_waker_ref;\n\nuse crate::loom::sync::atomic::{AtomicBool, Ordering};\nuse crate::loom::sync::Arc;\nuse crate::loom::thread;\n\nuse super::TimerEntry;\n\nfn block_on(f: impl std::future::Future) -> T {\n #[cfg(loom)]\n return loom::future::block_on(f);\n\n #[cfg(not(loom))]\n {\n let rt = crate::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.block_on(f)\n }\n}\n\nfn model(f: impl Fn() + Send + Sync + 'static) {\n #[cfg(loom)]\n loom::model(f);\n\n #[cfg(not(loom))]\n f();\n}\n\nfn rt(start_paused: bool) -> crate::runtime::Runtime {\n crate::runtime::Builder::new_current_thread()\n .enable_time()\n .start_paused(start_paused)\n .build()\n .unwrap()\n}\n\n#[test]\nfn single_timer() {\n model(|| {\n let rt = rt(false);\n let handle = rt.handle();\n\n let handle_ = handle.clone();\n let jh = thread::spawn(move || {\n let entry = TimerEntry::new(\n handle_.inner.clone(),\n handle_.inner.driver().clock().now() + Duration::from_secs(1),\n );\n pin!(entry);\n\n block_on(std::future::poll_fn(|cx| entry.as_mut().poll_elapsed(cx))).unwrap();\n });\n\n thread::yield_now();\n\n let time = handle.inner.driver().time();\n let clock = handle.inner.driver().clock();\n\n // advance 2s\n time.process_at_time(time.time_source().now(clock) + 2_000_000_000);\n\n jh.join().unwrap();\n })\n}\n\n#[test]\nfn drop_timer() {\n model(|| {\n let rt = rt(false);\n let handle = rt.handle();\n\n let handle_ = handle.clone();\n let jh = thread::spawn(move || {\n let entry = TimerEntry::new(\n handle_.inner.clone(),\n handle_.inner.driver().clock().now() + Duration::from_secs(1),\n );\n pin!(entry);\n\n let _ = entry\n .as_mut()\n .poll_elapsed(&mut Context::from_waker(futures::task::noop_waker_ref()));\n let _ = entry\n .as_mut()\n .poll_elapsed(&mut Context::from_waker(futures::task::noop_waker_ref()));\n });\n\n thread::yield_now();\n\n let time = handle.inner.driver().time();\n let clock = handle.inner.driver().clock();\n\n // advance 2s in the future.\n time.process_at_time(time.time_source().now(clock) + 2_000_000_000);\n\n jh.join().unwrap();\n })\n}\n\n#[test]\nfn change_waker() {\n model(|| {\n let rt = rt(false);\n let handle = rt.handle();\n\n let handle_ = handle.clone();\n let jh = thread::spawn(move || {\n let entry = TimerEntry::new(\n handle_.inner.clone(),\n handle_.inner.driver().clock().now() + Duration::from_secs(1),\n );\n pin!(entry);\n\n let _ = entry\n .as_mut()\n .poll_elapsed(&mut Context::from_waker(futures::task::noop_waker_ref()));\n\n block_on(std::future::poll_fn(|cx| entry.as_mut().poll_elapsed(cx))).unwrap();\n });\n\n thread::yield_now();\n\n let time = handle.inner.driver().time();\n let clock = handle.inner.driver().clock();\n\n // advance 2s\n time.process_at_time(time.time_source().now(clock) + 2_000_000_000);\n\n jh.join().unwrap();\n })\n}\n\n#[test]\nfn reset_future() {\n model(|| {\n let finished_early = Arc::new(AtomicBool::new(false));\n\n let rt = rt(false);\n let handle = rt.handle();\n\n let handle_ = handle.clone();\n let finished_early_ = finished_early.clone();\n let start = handle.inner.driver().clock().now();\n\n let jh = thread::spawn(move || {\n let entry = TimerEntry::new(handle_.inner.clone(), start + Duration::from_secs(1));\n pin!(entry);\n\n let _ = entry\n .as_mut()\n .poll_elapsed(&mut Context::from_waker(futures::task::noop_waker_ref()));\n\n entry.as_mut().reset(start + Duration::from_secs(2), true);\n\n // shouldn't complete before 2s\n block_on(std::future::poll_fn(|cx| entry.as_mut().poll_elapsed(cx))).unwrap();\n\n finished_early_.store(true, Ordering::Relaxed);\n });\n\n thread::yield_now();\n\n let handle = handle.inner.driver().time();\n\n handle.process_at_time(\n handle\n .time_source()\n .instant_to_tick(start + Duration::from_millis(1500)),\n );\n\n assert!(!finished_early.load(Ordering::Relaxed));\n\n handle.process_at_time(\n handle\n .time_source()\n .instant_to_tick(start + Duration::from_millis(2500)),\n );\n\n jh.join().unwrap();\n\n assert!(finished_early.load(Ordering::Relaxed));\n })\n}\n\n#[cfg(not(loom))]\nfn normal_or_miri(normal: T, miri: T) -> T {\n if cfg!(miri) {\n miri\n } else {\n normal\n }\n}\n\n#[test]\n#[cfg(not(loom))]\nfn poll_process_levels() {\n let rt = rt(true);\n let handle = rt.handle();\n\n let mut entries = vec![];\n\n for i in 0..normal_or_miri(1024, 64) {\n let mut entry = Box::pin(TimerEntry::new(\n handle.inner.clone(),\n handle.inner.driver().clock().now() + Duration::from_millis(i),\n ));\n\n let _ = entry\n .as_mut()\n .poll_elapsed(&mut Context::from_waker(noop_waker_ref()));\n\n entries.push(entry);\n }\n\n for t in 1..normal_or_miri(1024, 64) {\n handle.inner.driver().time().process_at_time(t as u64);\n\n for (deadline, future) in entries.iter_mut().enumerate() {\n let mut context = Context::from_waker(noop_waker_ref());\n if deadline <= t {\n assert!(future.as_mut().poll_elapsed(&mut context).is_ready());\n } else {\n assert!(future.as_mut().poll_elapsed(&mut context).is_pending());\n }\n }\n }\n}\n\n#[test]\n#[cfg(not(loom))]\nfn poll_process_levels_targeted() {\n let mut context = Context::from_waker(noop_waker_ref());\n\n let rt = rt(true);\n let handle = rt.handle();\n\n let e1 = TimerEntry::new(\n handle.inner.clone(),\n handle.inner.driver().clock().now() + Duration::from_millis(193),\n );\n pin!(e1);\n\n let handle = handle.inner.driver().time();\n\n handle.process_at_time(62);\n assert!(e1.as_mut().poll_elapsed(&mut context).is_pending());\n handle.process_at_time(192);\n handle.process_at_time(192);\n}\n\n#[test]\n#[cfg(not(loom))]\nfn instant_to_tick_max() {\n use crate::runtime::time::entry::MAX_SAFE_MILLIS_DURATION;\n\n let rt = rt(true);\n let handle = rt.handle().inner.driver().time();\n\n let start_time = handle.time_source.start_time();\n let long_future = start_time + std::time::Duration::from_millis(MAX_SAFE_MILLIS_DURATION + 1);\n\n assert!(handle.time_source.instant_to_tick(long_future) <= MAX_SAFE_MILLIS_DURATION);\n}\n" }, { "path": "tokio/src/runtime/time/wheel/level.rs", "content": "use crate::runtime::time::{EntryList, TimerHandle, TimerShared};\n\nuse std::{array, fmt, ptr::NonNull};\n\n/// Wheel for a single level in the timer. This wheel contains 64 slots.\npub(crate) struct Level {\n level: usize,\n\n /// Bit field tracking which slots currently contain entries.\n ///\n /// Using a bit field to track slots that contain entries allows avoiding a\n /// scan to find entries. This field is updated when entries are added or\n /// removed from a slot.\n ///\n /// The least-significant bit represents slot zero.\n occupied: u64,\n\n /// Slots. We access these via the EntryInner `current_list` as well, so this needs to be an `UnsafeCell`.\n slot: [EntryList; LEVEL_MULT],\n}\n\n/// Indicates when a slot must be processed next.\n#[derive(Debug)]\npub(crate) struct Expiration {\n /// The level containing the slot.\n pub(crate) level: usize,\n\n /// The slot index.\n pub(crate) slot: usize,\n\n /// The instant at which the slot needs to be processed.\n pub(crate) deadline: u64,\n}\n\n/// Level multiplier.\n///\n/// Being a power of 2 is very important.\nconst LEVEL_MULT: usize = 64;\n\nimpl Level {\n pub(crate) fn new(level: usize) -> Level {\n Level {\n level,\n occupied: 0,\n slot: array::from_fn(|_| EntryList::default()),\n }\n }\n\n /// Finds the slot that needs to be processed next and returns the slot and\n /// `Instant` at which this slot must be processed.\n pub(crate) fn next_expiration(&self, now: u64) -> Option {\n // Use the `occupied` bit field to get the index of the next slot that\n // needs to be processed.\n let slot = self.next_occupied_slot(now)?;\n\n // From the slot index, calculate the `Instant` at which it needs to be\n // processed. This value *must* be in the future with respect to `now`.\n\n let level_range = level_range(self.level);\n let slot_range = slot_range(self.level);\n\n // Compute the start date of the current level by masking the low bits\n // of `now` (`level_range` is a power of 2).\n let level_start = now & !(level_range - 1);\n let mut deadline = level_start + slot as u64 * slot_range;\n\n if deadline <= now {\n // A timer is in a slot \"prior\" to the current time. This can occur\n // because we do not have an infinite hierarchy of timer levels, and\n // eventually a timer scheduled for a very distant time might end up\n // being placed in a slot that is beyond the end of all of the\n // arrays.\n //\n // To deal with this, we first limit timers to being scheduled no\n // more than MAX_DURATION ticks in the future; that is, they're at\n // most one rotation of the top level away. Then, we force timers\n // that logically would go into the top+1 level, to instead go into\n // the top level's slots.\n //\n // What this means is that the top level's slots act as a\n // pseudo-ring buffer, and we rotate around them indefinitely. If we\n // compute a deadline before now, and it's the top level, it\n // therefore means we're actually looking at a slot in the future.\n debug_assert_eq!(self.level, super::NUM_LEVELS - 1);\n\n deadline += level_range;\n }\n\n debug_assert!(\n deadline >= now,\n \"deadline={:016X}; now={:016X}; level={}; lr={:016X}, sr={:016X}, slot={}; occupied={:b}\",\n deadline,\n now,\n self.level,\n level_range,\n slot_range,\n slot,\n self.occupied\n );\n\n Some(Expiration {\n level: self.level,\n slot,\n deadline,\n })\n }\n\n fn next_occupied_slot(&self, now: u64) -> Option {\n if self.occupied == 0 {\n return None;\n }\n\n // Get the slot for now using Maths\n let now_slot = (now / slot_range(self.level)) as usize;\n let occupied = self.occupied.rotate_right(now_slot as u32);\n let zeros = occupied.trailing_zeros() as usize;\n let slot = (zeros + now_slot) % LEVEL_MULT;\n\n Some(slot)\n }\n\n pub(crate) unsafe fn add_entry(&mut self, item: TimerHandle) {\n let slot = slot_for(unsafe { item.registered_when() }, self.level);\n\n self.slot[slot].push_front(item);\n\n self.occupied |= occupied_bit(slot);\n }\n\n pub(crate) unsafe fn remove_entry(&mut self, item: NonNull) {\n let slot = slot_for(unsafe { item.as_ref().registered_when() }, self.level);\n\n unsafe { self.slot[slot].remove(item) };\n if self.slot[slot].is_empty() {\n // The bit is currently set\n debug_assert!(self.occupied & occupied_bit(slot) != 0);\n\n // Unset the bit\n self.occupied ^= occupied_bit(slot);\n }\n }\n\n pub(crate) fn take_slot(&mut self, slot: usize) -> EntryList {\n self.occupied &= !occupied_bit(slot);\n\n std::mem::take(&mut self.slot[slot])\n }\n}\n\nimpl fmt::Debug for Level {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Level\")\n .field(\"occupied\", &self.occupied)\n .finish()\n }\n}\n\nfn occupied_bit(slot: usize) -> u64 {\n 1 << slot\n}\n\nfn slot_range(level: usize) -> u64 {\n LEVEL_MULT.pow(level as u32) as u64\n}\n\nfn level_range(level: usize) -> u64 {\n LEVEL_MULT as u64 * slot_range(level)\n}\n\n/// Converts a duration (milliseconds) and a level to a slot position.\nfn slot_for(duration: u64, level: usize) -> usize {\n ((duration >> (level * 6)) % LEVEL_MULT as u64) as usize\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_slot_for() {\n for pos in 0..64 {\n assert_eq!(pos as usize, slot_for(pos, 0));\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(pos, slot_for(a as u64, level));\n }\n }\n }\n}\n" }, { "path": "tokio/src/runtime/time/wheel/mod.rs", "content": "use crate::runtime::time::{TimerHandle, TimerShared};\nuse crate::time::error::InsertError;\n\nmod level;\npub(crate) use self::level::Expiration;\nuse self::level::Level;\n\nuse std::{array, ptr::NonNull};\n\nuse super::entry::STATE_DEREGISTERED;\nuse super::EntryList;\n\n/// Timing wheel implementation.\n///\n/// This type provides the hashed timing wheel implementation that backs\n/// [`Driver`].\n///\n/// See [`Driver`] documentation for some implementation notes.\n///\n/// [`Driver`]: crate::runtime::time::Driver\n#[derive(Debug)]\npub(crate) struct Wheel {\n /// The number of milliseconds elapsed since the wheel started.\n elapsed: u64,\n\n /// Timer wheel.\n ///\n /// Levels:\n ///\n /// * 1 ms slots / 64 ms range\n /// * 64 ms slots / ~ 4 sec range\n /// * ~ 4 sec slots / ~ 4 min range\n /// * ~ 4 min slots / ~ 4 hr range\n /// * ~ 4 hr slots / ~ 12 day range\n /// * ~ 12 day slots / ~ 2 yr range\n levels: Box<[Level; NUM_LEVELS]>,\n\n /// Entries queued for firing\n pending: EntryList,\n}\n\n/// Number of levels. Each level has 64 slots. By using 6 levels with 64 slots\n/// each, the timer is able to track time up to 2 years into the future with a\n/// precision of 1 millisecond.\nconst NUM_LEVELS: usize = 6;\n\n/// The maximum duration of a `Sleep`.\npub(super) const MAX_DURATION: u64 = (1 << (6 * NUM_LEVELS)) - 1;\n\nimpl Wheel {\n /// Creates a new timing wheel.\n pub(crate) fn new() -> Wheel {\n Wheel {\n elapsed: 0,\n levels: Box::new(array::from_fn(Level::new)),\n pending: EntryList::new(),\n }\n }\n\n /// Returns the number of milliseconds that have elapsed since the timing\n /// wheel's creation.\n pub(crate) fn elapsed(&self) -> u64 {\n self.elapsed\n }\n\n /// Inserts an entry into the timing wheel.\n ///\n /// # Arguments\n ///\n /// * `item`: The item to insert into the wheel.\n ///\n /// # Return\n ///\n /// Returns `Ok` when the item is successfully inserted, `Err` otherwise.\n ///\n /// `Err(Elapsed)` indicates that `when` represents an instant that has\n /// already passed. In this case, the caller should fire the timeout\n /// immediately.\n ///\n /// `Err(Invalid)` indicates an invalid `when` argument as been supplied.\n ///\n /// # Safety\n ///\n /// This function registers item into an intrusive linked list. The caller\n /// must ensure that `item` is pinned and will not be dropped without first\n /// being deregistered.\n pub(crate) unsafe fn insert(\n &mut self,\n item: TimerHandle,\n ) -> Result {\n let when = unsafe { item.sync_when() };\n\n if when <= self.elapsed {\n return Err((item, InsertError::Elapsed));\n }\n\n // Get the level at which the entry should be stored\n let level = self.level_for(when);\n\n unsafe {\n self.levels[level].add_entry(item);\n }\n\n debug_assert!({\n self.levels[level]\n .next_expiration(self.elapsed)\n .map(|e| e.deadline >= self.elapsed)\n .unwrap_or(true)\n });\n\n Ok(when)\n }\n\n /// Removes `item` from the timing wheel.\n pub(crate) unsafe fn remove(&mut self, item: NonNull) {\n unsafe {\n let when = item.as_ref().registered_when();\n if when == STATE_DEREGISTERED {\n self.pending.remove(item);\n } else {\n debug_assert!(\n self.elapsed <= when,\n \"elapsed={}; when={}\",\n self.elapsed,\n when\n );\n\n let level = self.level_for(when);\n self.levels[level].remove_entry(item);\n }\n }\n }\n\n /// Instant at which to poll.\n pub(crate) fn poll_at(&self) -> Option {\n self.next_expiration().map(|expiration| expiration.deadline)\n }\n\n /// Advances the timer up to the instant represented by `now`.\n pub(crate) fn poll(&mut self, now: u64) -> Option {\n loop {\n if let Some(handle) = self.pending.pop_back() {\n return Some(handle);\n }\n\n match self.next_expiration() {\n Some(ref expiration) if expiration.deadline <= now => {\n self.process_expiration(expiration);\n\n self.set_elapsed(expiration.deadline);\n }\n _ => {\n // in this case the poll did not indicate an expiration\n // _and_ we were not able to find a next expiration in\n // the current list of timers. advance to the poll's\n // current time and do nothing else.\n self.set_elapsed(now);\n break;\n }\n }\n }\n\n self.pending.pop_back()\n }\n\n /// Returns the instant at which the next timeout expires.\n fn next_expiration(&self) -> Option {\n if !self.pending.is_empty() {\n // Expire immediately as we have things pending firing\n return Some(Expiration {\n level: 0,\n slot: 0,\n deadline: self.elapsed,\n });\n }\n\n // Check all levels\n for (level_num, level) in self.levels.iter().enumerate() {\n if let Some(expiration) = level.next_expiration(self.elapsed) {\n // There cannot be any expirations at a higher level that happen\n // before this one.\n debug_assert!(self.no_expirations_before(level_num + 1, expiration.deadline));\n\n return Some(expiration);\n }\n }\n\n None\n }\n\n /// Returns the tick at which this timer wheel next needs to perform some\n /// processing, or None if there are no timers registered.\n pub(super) fn next_expiration_time(&self) -> Option {\n self.next_expiration().map(|ex| ex.deadline)\n }\n\n /// Used for debug assertions\n fn no_expirations_before(&self, start_level: usize, before: u64) -> bool {\n let mut res = true;\n\n for level in &self.levels[start_level..] {\n if let Some(e2) = level.next_expiration(self.elapsed) {\n if e2.deadline < before {\n res = false;\n }\n }\n }\n\n res\n }\n\n /// iteratively find entries that are between the wheel's current\n /// time and the expiration time. for each in that population either\n /// queue it for notification (in the case of the last level) or tier\n /// it down to the next level (in all other cases).\n pub(crate) fn process_expiration(&mut self, expiration: &Expiration) {\n // Note that we need to take _all_ of the entries off the list before\n // processing any of them. This is important because it's possible that\n // those entries might need to be reinserted into the same slot.\n //\n // This happens only on the highest level, when an entry is inserted\n // more than MAX_DURATION into the future. When this happens, we wrap\n // around, and process some entries a multiple of MAX_DURATION before\n // they actually need to be dropped down a level. We then reinsert them\n // back into the same position; we must make sure we don't then process\n // those entries again or we'll end up in an infinite loop.\n let mut entries = self.take_entries(expiration);\n\n while let Some(item) = entries.pop_back() {\n if expiration.level == 0 {\n debug_assert_eq!(unsafe { item.registered_when() }, expiration.deadline);\n }\n\n // Try to expire the entry; this is cheap (doesn't synchronize) if\n // the timer is not expired, and updates registered_when.\n match unsafe { item.mark_pending(expiration.deadline) } {\n Ok(()) => {\n // Item was expired\n self.pending.push_front(item);\n }\n Err(expiration_tick) => {\n let level = level_for(expiration.deadline, expiration_tick);\n unsafe {\n self.levels[level].add_entry(item);\n }\n }\n }\n }\n }\n\n fn set_elapsed(&mut self, when: u64) {\n assert!(\n self.elapsed <= when,\n \"elapsed={:?}; when={:?}\",\n self.elapsed,\n when\n );\n\n if when > self.elapsed {\n self.elapsed = when;\n }\n }\n\n /// Obtains the list of entries that need processing for the given expiration.\n fn take_entries(&mut self, expiration: &Expiration) -> EntryList {\n self.levels[expiration.level].take_slot(expiration.slot)\n }\n\n fn level_for(&self, when: u64) -> usize {\n level_for(self.elapsed, when)\n }\n}\n\nfn level_for(elapsed: u64, when: u64) -> usize {\n const SLOT_MASK: u64 = (1 << 6) - 1;\n\n // Mask in the trailing bits ignored by the level calculation in order to cap\n // the possible leading zeros\n let mut masked = elapsed ^ when | SLOT_MASK;\n\n if masked >= MAX_DURATION {\n // Fudge the timer into the top level\n masked = MAX_DURATION - 1;\n }\n\n let leading_zeros = masked.leading_zeros() as usize;\n let significant = 63 - leading_zeros;\n\n significant / NUM_LEVELS\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_level_for() {\n for pos in 0..64 {\n assert_eq!(0, level_for(0, pos), \"level_for({pos}) -- binary = {pos:b}\");\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n\n if pos > level {\n let a = a - 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n\n if pos < 64 {\n let a = a + 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n }\n }\n }\n}\n" }, { "path": "tokio/src/runtime/time_alt/cancellation_queue/tests.rs", "content": "use super::*;\n\nuse futures::task::noop_waker;\n\n#[cfg(loom)]\nconst NUM_ITEMS: usize = 16;\n\n#[cfg(not(loom))]\nconst NUM_ITEMS: usize = 64;\n\nfn new_handle() -> EntryHandle {\n EntryHandle::new(0, noop_waker())\n}\n\nfn model(f: F) {\n #[cfg(loom)]\n loom::model(f);\n\n #[cfg(not(loom))]\n f();\n}\n\n#[test]\nfn single_thread() {\n model(|| {\n for i in 0..NUM_ITEMS {\n let (tx, mut rx) = new();\n\n for _ in 0..i {\n unsafe { tx.send(new_handle()) };\n }\n\n assert_eq!(rx.recv_all().count(), i);\n }\n });\n}\n\n#[test]\n#[cfg(not(target_os = \"wasi\"))] // No thread on wasi.\nfn multi_thread() {\n use crate::loom::sync::atomic::{AtomicUsize, Ordering::SeqCst};\n use crate::loom::sync::Arc;\n use crate::loom::thread;\n\n #[cfg(loom)]\n const NUM_THREADS: usize = 3;\n #[cfg(not(loom))]\n const NUM_THREADS: usize = 8;\n\n model(|| {\n let (tx, mut rx) = new();\n let mut jhs = Vec::new();\n let sent = Arc::new(AtomicUsize::new(0));\n\n for _ in 0..NUM_THREADS {\n let tx = tx.clone();\n let sent = sent.clone();\n jhs.push(thread::spawn(move || {\n for _ in 0..NUM_ITEMS {\n unsafe { tx.send(new_handle()) };\n sent.fetch_add(1, SeqCst);\n }\n }));\n }\n\n let mut count = 0;\n loop {\n count += rx.recv_all().count();\n if sent.fetch_add(0, SeqCst) == NUM_ITEMS * NUM_THREADS {\n jhs.into_iter().for_each(|jh| {\n jh.join().unwrap();\n });\n count += rx.recv_all().count();\n break;\n }\n thread::yield_now();\n }\n\n assert_eq!(count, NUM_ITEMS * NUM_THREADS);\n })\n}\n\n#[test]\nfn drop_iter_should_not_leak_memory() {\n model(|| {\n let (tx, mut rx) = new();\n\n let hdls = (0..NUM_ITEMS).map(|_| new_handle()).collect::>();\n for hdl in hdls.iter() {\n unsafe { tx.send(hdl.clone()) };\n }\n\n drop(rx.recv_all());\n\n assert!(hdls.into_iter().all(|hdl| hdl.inner_strong_count() == 1));\n });\n}\n" }, { "path": "tokio/src/runtime/time_alt/cancellation_queue.rs", "content": "use super::{CancellationQueueEntry, Entry, EntryHandle};\nuse crate::loom::sync::{Arc, Mutex};\nuse crate::util::linked_list;\n\ntype EntryList = linked_list::LinkedList;\n\n#[derive(Debug, Default)]\nstruct Inner {\n list: EntryList,\n}\n\nimpl Drop for Inner {\n fn drop(&mut self) {\n // consume all entries\n while let Some(hdl) = self.list.pop_front() {\n drop(hdl)\n }\n }\n}\n\nimpl Inner {\n fn new() -> Self {\n Self {\n list: EntryList::new(),\n }\n }\n\n /// # Safety\n ///\n /// Behavior is undefined if any of the following conditions are violated:\n ///\n /// - `hdl` must not in any [`super::cancellation_queue`], and also mus not in any [`super::WakeQueue`].\n unsafe fn push_front(&mut self, hdl: EntryHandle) {\n self.list.push_front(hdl);\n }\n\n fn into_iter(self) -> impl Iterator {\n struct Iter(Inner);\n\n impl Iterator for Iter {\n type Item = EntryHandle;\n\n fn next(&mut self) -> Option {\n self.0.list.pop_front()\n }\n }\n\n Iter(self)\n }\n}\n\n#[derive(Debug, Clone)]\npub(crate) struct Sender {\n inner: Arc>,\n}\n\nimpl Sender {\n /// # Safety\n ///\n /// Behavior is undefined if any of the following conditions are violated:\n ///\n /// - `hdl` must not in any cancellation queue.\n pub(crate) unsafe fn send(&self, hdl: EntryHandle) {\n unsafe {\n self.inner.lock().push_front(hdl);\n }\n }\n}\n\n#[derive(Debug)]\npub(crate) struct Receiver {\n inner: Arc>,\n}\n\nimpl Receiver {\n pub(crate) fn recv_all(&mut self) -> impl Iterator {\n std::mem::take(&mut *self.inner.lock()).into_iter()\n }\n}\n\npub(crate) fn new() -> (Sender, Receiver) {\n let inner = Arc::new(Mutex::new(Inner::new()));\n (\n Sender {\n inner: inner.clone(),\n },\n Receiver { inner },\n )\n}\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/runtime/time_alt/context.rs", "content": "use super::{cancellation_queue, RegistrationQueue, Wheel};\n\n/// Local context for the time driver, used when the runtime wants to\n/// fire/cancel timers.\npub(crate) struct LocalContext {\n pub(crate) wheel: Wheel,\n pub(crate) registration_queue: RegistrationQueue,\n pub(crate) canc_tx: cancellation_queue::Sender,\n pub(crate) canc_rx: cancellation_queue::Receiver,\n}\n\nimpl LocalContext {\n pub(crate) fn new() -> Self {\n let (canc_tx, canc_rx) = cancellation_queue::new();\n Self {\n wheel: Wheel::new(),\n registration_queue: RegistrationQueue::new(),\n canc_tx,\n canc_rx,\n }\n }\n}\n\npub(crate) enum TempLocalContext<'a> {\n /// The runtime is running, we can access it.\n Running {\n registration_queue: &'a mut RegistrationQueue,\n elapsed: u64,\n },\n #[cfg(feature = \"rt-multi-thread\")]\n /// The runtime is shutting down, no timers can be registered.\n Shutdown,\n}\n\nimpl<'a> TempLocalContext<'a> {\n pub(crate) fn new_running(cx: &'a mut LocalContext) -> Self {\n TempLocalContext::Running {\n registration_queue: &mut cx.registration_queue,\n elapsed: cx.wheel.elapsed(),\n }\n }\n\n #[cfg(feature = \"rt-multi-thread\")]\n pub(crate) fn new_shutdown() -> Self {\n TempLocalContext::Shutdown\n }\n}\n" }, { "path": "tokio/src/runtime/time_alt/entry.rs", "content": "use super::cancellation_queue::Sender;\nuse crate::loom::sync::{Arc, Mutex};\nuse crate::util::linked_list;\n\nuse std::marker::PhantomPinned;\nuse std::ptr::NonNull;\nuse std::task::Waker;\n\npub(super) type EntryList = linked_list::LinkedList;\n\n#[derive(Debug)]\nstruct State {\n cancelled: bool,\n woken_up: bool,\n waker: Option,\n cancel_tx: Option,\n}\n\n#[derive(Debug)]\npub(crate) struct Entry {\n /// The intrusive pointer used by [`super::cancellation_queue`].\n cancel_pointers: linked_list::Pointers,\n\n /// The intrusive pointer used by any of the following queues:\n ///\n /// - [`Wheel`]\n /// - [`RegistrationQueue`]\n /// - [`WakeQueue`]\n ///\n /// We can guarantee that this pointer is only used by one of the above\n /// at any given time. See below for the journey of this pointer.\n ///\n /// Initially, this pointer is used by the [`RegistrationQueue`].\n ///\n /// And then, before parking the resource driver,\n /// the scheduler removes the entry from the [`RegistrationQueue`]\n /// [`RegistrationQueue`] and insert it into the [`Wheel`].\n ///\n /// Finally, after parking the resource driver, the scheduler removes\n /// the entry from the [`Wheel`] and insert it into the [`WakeQueue`].\n ///\n /// [`RegistrationQueue`]: super::RegistrationQueue\n /// [`Wheel`]: super::Wheel\n /// [`WakeQueue`]: super::WakeQueue\n extra_pointers: linked_list::Pointers,\n\n /// The tick when this entry is scheduled to expire.\n deadline: u64,\n\n state: Mutex,\n\n /// Make the type `!Unpin` to prevent LLVM from emitting\n /// the `noalias` attribute for mutable references.\n ///\n /// See .\n _pin: PhantomPinned,\n}\n\n// Safety: `Entry` is always in an `Arc`.\nunsafe impl linked_list::Link for Entry {\n type Handle = Handle;\n type Target = Entry;\n\n fn as_raw(hdl: &Self::Handle) -> NonNull {\n unsafe { NonNull::new_unchecked(Arc::as_ptr(&hdl.entry).cast_mut()) }\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle {\n Handle {\n entry: unsafe { Arc::from_raw(ptr.as_ptr()) },\n }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n let this = target.as_ptr();\n let field = unsafe { std::ptr::addr_of_mut!((*this).extra_pointers) };\n unsafe { NonNull::new_unchecked(field) }\n }\n}\n\n/// An ZST to allow [`super::registration_queue`] to utilize the [`Entry::extra_pointers`]\n/// by impl [`linked_list::Link`] as we cannot impl it on [`Entry`]\n/// directly due to the conflicting implementations.\n///\n/// This type should never be constructed.\npub(super) struct RegistrationQueueEntry;\n\n// Safety: `Entry` is always in an `Arc`.\nunsafe impl linked_list::Link for RegistrationQueueEntry {\n type Handle = Handle;\n type Target = Entry;\n\n fn as_raw(hdl: &Self::Handle) -> NonNull {\n unsafe { NonNull::new_unchecked(Arc::as_ptr(&hdl.entry).cast_mut()) }\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle {\n Handle {\n entry: unsafe { Arc::from_raw(ptr.as_ptr()) },\n }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n let this = target.as_ptr();\n let field = unsafe { std::ptr::addr_of_mut!((*this).extra_pointers) };\n unsafe { NonNull::new_unchecked(field) }\n }\n}\n\n/// An ZST to allow [`super::cancellation_queue`] to utilize the [`Entry::cancel_pointers`]\n/// by impl [`linked_list::Link`] as we cannot impl it on [`Entry`]\n/// directly due to the conflicting implementations.\n///\n/// This type should never be constructed.\npub(super) struct CancellationQueueEntry;\n\n// Safety: `Entry` is always in an `Arc`.\nunsafe impl linked_list::Link for CancellationQueueEntry {\n type Handle = Handle;\n type Target = Entry;\n\n fn as_raw(hdl: &Self::Handle) -> NonNull {\n unsafe { NonNull::new_unchecked(Arc::as_ptr(&hdl.entry).cast_mut()) }\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle {\n Handle {\n entry: unsafe { Arc::from_raw(ptr.as_ptr()) },\n }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n let this = target.as_ptr();\n let field = unsafe { std::ptr::addr_of_mut!((*this).cancel_pointers) };\n unsafe { NonNull::new_unchecked(field) }\n }\n}\n\n/// An ZST to allow [`super::WakeQueue`] to utilize the [`Entry::extra_pointers`]\n/// by impl [`linked_list::Link`] as we cannot impl it on [`Entry`]\n/// directly due to the conflicting implementations.\n///\n/// This type should never be constructed.\npub(super) struct WakeQueueEntry;\n\n// Safety: `Entry` is always in an `Arc`.\nunsafe impl linked_list::Link for WakeQueueEntry {\n type Handle = Handle;\n type Target = Entry;\n\n fn as_raw(hdl: &Self::Handle) -> NonNull {\n unsafe { NonNull::new_unchecked(Arc::as_ptr(&hdl.entry).cast_mut()) }\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle {\n Handle {\n entry: unsafe { Arc::from_raw(ptr.as_ptr()) },\n }\n }\n\n unsafe fn pointers(\n target: NonNull,\n ) -> NonNull> {\n let this = target.as_ptr();\n let field = unsafe { std::ptr::addr_of_mut!((*this).extra_pointers) };\n unsafe { NonNull::new_unchecked(field) }\n }\n}\n\n#[derive(Debug, Clone)]\npub(crate) struct Handle {\n pub(crate) entry: Arc,\n}\n\nimpl From<&Handle> for NonNull {\n fn from(hdl: &Handle) -> Self {\n // Safety: entry is in an `Arc`, so the pointer is valid.\n unsafe { NonNull::new_unchecked(Arc::as_ptr(&hdl.entry) as *mut Entry) }\n }\n}\n\nimpl Handle {\n pub(crate) fn new(deadline: u64, waker: Waker) -> Self {\n let state = State {\n cancelled: false,\n woken_up: false,\n waker: Some(waker),\n cancel_tx: None,\n };\n\n let entry = Arc::new(Entry {\n cancel_pointers: linked_list::Pointers::new(),\n extra_pointers: linked_list::Pointers::new(),\n deadline,\n state: Mutex::new(state),\n _pin: PhantomPinned,\n });\n\n Handle { entry }\n }\n\n /// Wake the entry if it is already in the pending queue of the timer wheel.\n pub(crate) fn wake(&self) {\n let mut lock = self.entry.state.lock();\n\n if !lock.cancelled {\n lock.woken_up = true;\n if let Some(waker) = lock.waker.take() {\n // unlock before calling waker\n drop(lock);\n waker.wake();\n }\n }\n }\n\n pub(crate) fn register_cancel_tx(&self, cancel_tx: Sender) {\n let mut lock = self.entry.state.lock();\n if !lock.cancelled && !lock.woken_up {\n let old_tx = lock.cancel_tx.replace(cancel_tx);\n // don't unlock — poisoning the `Mutex` stops others from using the bad state.\n assert!(old_tx.is_none(), \"cancel_tx is already registered\");\n }\n }\n\n pub(crate) fn register_waker(&self, waker: Waker) {\n let mut lock = self.entry.state.lock();\n if !lock.cancelled && !lock.woken_up {\n let maybe_old_waker = lock.waker.replace(waker);\n // unlock before calling waker\n drop(lock);\n drop(maybe_old_waker);\n }\n }\n\n pub(crate) fn cancel(&self) {\n let mut lock = self.entry.state.lock();\n if !lock.cancelled {\n lock.cancelled = true;\n if let Some(cancel_tx) = lock.cancel_tx.take() {\n drop(lock);\n\n // Safety: we can guarantee that `self` is not in any cancellation queue\n // because the `self.cancelled` was just set to `true`.\n unsafe {\n cancel_tx.send(self.clone());\n }\n }\n }\n }\n\n pub(crate) fn deadline(&self) -> u64 {\n self.entry.deadline\n }\n\n pub(crate) fn is_woken_up(&self) -> bool {\n let lock = self.entry.state.lock();\n lock.woken_up\n }\n\n pub(crate) fn is_cancelled(&self) -> bool {\n let lock = self.entry.state.lock();\n lock.cancelled\n }\n\n #[cfg(test)]\n /// Only used for unit tests.\n pub(crate) fn inner_strong_count(&self) -> usize {\n Arc::strong_count(&self.entry)\n }\n}\n" }, { "path": "tokio/src/runtime/time_alt/mod.rs", "content": "pub(crate) mod context;\npub(super) use context::{LocalContext, TempLocalContext};\n\npub(crate) mod cancellation_queue;\n\nmod entry;\npub(crate) use entry::Handle as EntryHandle;\nuse entry::{CancellationQueueEntry, RegistrationQueueEntry, WakeQueueEntry};\nuse entry::{Entry, EntryList};\n\nmod registration_queue;\npub(crate) use registration_queue::RegistrationQueue;\n\nmod timer;\npub(crate) use timer::Timer;\n\nmod wheel;\npub(super) use wheel::Wheel;\n\nmod wake_queue;\npub(crate) use wake_queue::WakeQueue;\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/runtime/time_alt/registration_queue/tests.rs", "content": "use super::*;\n\nuse futures::task::noop_waker;\n\n#[cfg(loom)]\nconst NUM_ITEMS: usize = 16;\n\n#[cfg(not(loom))]\nconst NUM_ITEMS: usize = 64;\n\nfn new_handle() -> EntryHandle {\n EntryHandle::new(0, noop_waker())\n}\n\nfn model(f: F) {\n #[cfg(loom)]\n loom::model(f);\n\n #[cfg(not(loom))]\n f();\n}\n\n#[test]\nfn sanity() {\n model(|| {\n let mut queue = RegistrationQueue::new();\n for _ in 0..NUM_ITEMS {\n unsafe {\n queue.push_front(new_handle());\n }\n }\n for _ in 0..NUM_ITEMS {\n assert!(queue.pop_front().is_some());\n }\n assert!(queue.pop_front().is_none());\n });\n}\n\n#[test]\nfn drop_should_not_leak_memory() {\n model(|| {\n let mut queue = RegistrationQueue::new();\n\n let hdls = (0..NUM_ITEMS).map(|_| new_handle()).collect::>();\n for hdl in hdls.iter() {\n unsafe { queue.push_front(hdl.clone()) };\n }\n\n drop(queue);\n\n assert!(hdls.into_iter().all(|hdl| hdl.inner_strong_count() == 1));\n });\n}\n" }, { "path": "tokio/src/runtime/time_alt/registration_queue.rs", "content": "use super::{Entry, EntryHandle, RegistrationQueueEntry};\nuse crate::util::linked_list;\n\ntype EntryList = linked_list::LinkedList;\n\n/// A queue of entries that need to be registered in the timer wheel.\n#[derive(Debug)]\npub(crate) struct RegistrationQueue {\n list: EntryList,\n}\n\nimpl Drop for RegistrationQueue {\n fn drop(&mut self) {\n // drain all entries without waking them up\n while let Some(hdl) = self.list.pop_front() {\n drop(hdl);\n }\n }\n}\n\nimpl RegistrationQueue {\n pub(crate) fn new() -> Self {\n Self {\n list: EntryList::new(),\n }\n }\n\n /// # Safety\n ///\n /// Behavior is undefined if any of the following conditions are violated:\n ///\n /// - [`Entry::extra_pointers`] of `hdl` must not being used.\n pub(crate) unsafe fn push_front(&mut self, hdl: EntryHandle) {\n self.list.push_front(hdl);\n }\n\n pub(crate) fn pop_front(&mut self) -> Option {\n self.list.pop_front()\n }\n}\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/runtime/time_alt/tests.rs", "content": "use super::*;\nuse crate::loom::thread;\n\nuse futures_test::task::{new_count_waker, AwokenCount};\n\n#[cfg(loom)]\nconst NUM_ITEMS: usize = 16;\n\n#[cfg(not(loom))]\nconst NUM_ITEMS: usize = 64;\n\nfn new_handle() -> (EntryHandle, AwokenCount) {\n let (waker, count) = new_count_waker();\n (EntryHandle::new(0, waker), count)\n}\n\nfn model(f: F) {\n #[cfg(loom)]\n loom::model(f);\n\n #[cfg(not(loom))]\n f();\n}\n\n#[test]\nfn wake_up_in_the_same_thread() {\n model(|| {\n let mut counts = Vec::new();\n\n let mut reg_queue = RegistrationQueue::new();\n\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n counts.push(count);\n unsafe {\n reg_queue.push_front(hdl);\n }\n }\n\n let mut wake_queue = WakeQueue::new();\n for _ in 0..NUM_ITEMS {\n if let Some(hdl) = reg_queue.pop_front() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n }\n assert!(reg_queue.pop_front().is_none());\n wake_queue.wake_all();\n\n assert!(counts.into_iter().all(|c| c.get() == 1));\n });\n}\n\n#[test]\nfn cancel_in_the_same_thread() {\n model(|| {\n let mut counts = Vec::new();\n let (cancel_tx, mut cancel_rx) = cancellation_queue::new();\n\n let mut reg_queue = RegistrationQueue::new();\n\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n hdl.register_cancel_tx(cancel_tx.clone());\n counts.push(count);\n unsafe {\n reg_queue.push_front(hdl.clone());\n }\n hdl.cancel();\n }\n\n // drain the registration queue\n while let Some(hdl) = reg_queue.pop_front() {\n drop(hdl);\n }\n\n let mut wake_queue = WakeQueue::new();\n for hdl in cancel_rx.recv_all() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n wake_queue.wake_all();\n\n assert!(counts.into_iter().all(|c| c.get() == 0));\n });\n}\n\n#[test]\nfn wake_up_in_the_different_thread() {\n model(|| {\n let mut counts = Vec::new();\n\n let mut hdls = Vec::new();\n let mut reg_queue = RegistrationQueue::new();\n\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n counts.push(count);\n hdls.push(hdl.clone());\n unsafe {\n reg_queue.push_front(hdl);\n }\n }\n\n // wake up all handles in a different thread\n thread::spawn(move || {\n let mut wake_queue = WakeQueue::new();\n for _ in 0..NUM_ITEMS {\n if let Some(hdl) = reg_queue.pop_front() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n }\n assert!(reg_queue.pop_front().is_none());\n wake_queue.wake_all();\n assert!(counts.into_iter().all(|c| c.get() == 1));\n })\n .join()\n .unwrap();\n });\n}\n\n#[test]\nfn cancel_in_the_different_thread() {\n model(|| {\n let mut counts = Vec::new();\n let (cancel_tx, mut cancel_rx) = cancellation_queue::new();\n let mut hdls = Vec::new();\n let mut reg_queue = RegistrationQueue::new();\n\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n hdl.register_cancel_tx(cancel_tx.clone());\n counts.push(count);\n hdls.push(hdl.clone());\n unsafe {\n reg_queue.push_front(hdl);\n }\n }\n\n // this thread cancel all handles concurrently\n let jh = thread::spawn(move || {\n // cancel all handles\n for hdl in hdls {\n hdl.cancel();\n }\n });\n\n // cancellation queue concurrently\n while let Some(hdl) = reg_queue.pop_front() {\n drop(hdl);\n }\n\n let mut wake_queue = WakeQueue::new();\n for hdl in cancel_rx.recv_all() {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n wake_queue.wake_all();\n assert!(counts.into_iter().all(|c| c.get() == 0));\n\n jh.join().unwrap();\n })\n}\n" }, { "path": "tokio/src/runtime/time_alt/timer.rs", "content": "use super::{EntryHandle, TempLocalContext};\nuse crate::runtime::scheduler::Handle as SchedulerHandle;\nuse crate::time::Instant;\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[cfg(any(feature = \"rt\", feature = \"rt-multi-thread\"))]\nuse crate::util::error::RUNTIME_SHUTTING_DOWN_ERROR;\n\npub(crate) struct Timer {\n sched_handle: SchedulerHandle,\n\n /// The entry in the timing wheel.\n ///\n /// - `Some` if the timer is registered / pending / woken up / cancelling.\n /// - `None` if the timer is unregistered.\n entry: Option,\n\n /// The deadline for the timer.\n deadline: Instant,\n}\n\nimpl std::fmt::Debug for Timer {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"Timer\")\n .field(\"deadline\", &self.deadline)\n .finish()\n }\n}\n\nimpl Drop for Timer {\n fn drop(&mut self) {\n if let Some(entry) = self.entry.take() {\n entry.cancel();\n }\n }\n}\n\nimpl Timer {\n #[track_caller]\n pub(crate) fn new(sched_hdl: SchedulerHandle, deadline: Instant) -> Self {\n // Panic if the time driver is not enabled\n let _ = sched_hdl.driver().time();\n Timer {\n sched_handle: sched_hdl,\n entry: None,\n deadline,\n }\n }\n\n pub(crate) fn deadline(&self) -> Instant {\n self.deadline\n }\n\n pub(crate) fn is_elapsed(&self) -> bool {\n self.entry.as_ref().is_some_and(|entry| entry.is_woken_up())\n }\n\n fn register(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n let this = self.get_mut();\n\n with_current_temp_local_context(&this.sched_handle, |maybe_time_cx| {\n let deadline = deadline_to_tick(&this.sched_handle, this.deadline);\n\n match maybe_time_cx {\n Some(TempLocalContext::Running {\n registration_queue: _,\n elapsed,\n }) if deadline <= elapsed => Poll::Ready(()),\n\n Some(TempLocalContext::Running {\n registration_queue,\n elapsed: _,\n }) => {\n let hdl = EntryHandle::new(deadline, cx.waker().clone());\n this.entry = Some(hdl.clone());\n unsafe {\n registration_queue.push_front(hdl);\n }\n Poll::Pending\n }\n #[cfg(feature = \"rt-multi-thread\")]\n Some(TempLocalContext::Shutdown) => panic!(\"{RUNTIME_SHUTTING_DOWN_ERROR}\"),\n\n _ => {\n let hdl = EntryHandle::new(deadline, cx.waker().clone());\n this.entry = Some(hdl.clone());\n push_from_remote(&this.sched_handle, hdl);\n Poll::Pending\n }\n }\n })\n }\n\n pub(crate) fn poll_elapsed(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n match self.entry.as_ref() {\n Some(entry) if entry.is_woken_up() => Poll::Ready(()),\n Some(entry) => {\n entry.register_waker(cx.waker().clone());\n Poll::Pending\n }\n None => self.register(cx),\n }\n }\n\n pub(crate) fn scheduler_handle(&self) -> &SchedulerHandle {\n &self.sched_handle\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) fn driver(&self) -> &crate::runtime::time::Handle {\n self.sched_handle.driver().time()\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) fn clock(&self) -> &crate::time::Clock {\n self.sched_handle.driver().clock()\n }\n}\n\nfn with_current_temp_local_context(hdl: &SchedulerHandle, f: F) -> R\nwhere\n F: FnOnce(Option>) -> R,\n{\n #[cfg(not(feature = \"rt\"))]\n {\n let (_, _) = (hdl, f);\n panic!(\"Tokio runtime is not enabled, cannot access the current wheel\");\n }\n\n #[cfg(feature = \"rt\")]\n {\n use crate::runtime::context;\n\n let is_same_rt =\n context::with_current(|cur_hdl| cur_hdl.is_same_runtime(hdl)).unwrap_or_default();\n\n if !is_same_rt {\n // We don't want to create the timer in one runtime,\n // but register it in a different runtime's timer wheel.\n f(None)\n } else {\n context::with_scheduler(|maybe_cx| match maybe_cx {\n Some(cx) => cx.with_time_temp_local_context(f),\n None => f(None),\n })\n }\n }\n}\n\nfn push_from_remote(sched_hdl: &SchedulerHandle, entry_hdl: EntryHandle) {\n #[cfg(not(feature = \"rt\"))]\n {\n let (_, _) = (sched_hdl, entry_hdl);\n panic!(\"Tokio runtime is not enabled, cannot access the current wheel\");\n }\n\n #[cfg(feature = \"rt\")]\n {\n assert!(!sched_hdl.is_shutdown(), \"{RUNTIME_SHUTTING_DOWN_ERROR}\");\n sched_hdl.push_remote_timer(entry_hdl);\n }\n}\n\nfn deadline_to_tick(sched_hdl: &SchedulerHandle, deadline: Instant) -> u64 {\n let time_hdl = sched_hdl.driver().time();\n time_hdl.time_source().deadline_to_tick(deadline)\n}\n" }, { "path": "tokio/src/runtime/time_alt/wake_queue/tests.rs", "content": "use super::*;\n\nuse futures_test::task::{new_count_waker, AwokenCount};\n\n#[cfg(loom)]\nconst NUM_ITEMS: usize = 16;\n\n#[cfg(not(loom))]\nconst NUM_ITEMS: usize = 64;\n\nfn new_handle() -> (EntryHandle, AwokenCount) {\n let (waker, count) = new_count_waker();\n (EntryHandle::new(0, waker), count)\n}\n\nfn model(f: F) {\n #[cfg(loom)]\n loom::model(f);\n\n #[cfg(not(loom))]\n f();\n}\n\n#[test]\nfn sanity() {\n model(|| {\n let mut queue = WakeQueue::new();\n let mut counts = Vec::new();\n\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n counts.push(count);\n unsafe {\n queue.push_front(hdl);\n }\n }\n assert!(!queue.is_empty());\n queue.wake_all();\n assert!(counts.into_iter().all(|c| c.get() == 1));\n });\n}\n\n#[test]\nfn drop_should_not_leak_memory() {\n model(|| {\n let mut queue = WakeQueue::new();\n\n let mut hdls = vec![];\n let mut counts = vec![];\n for _ in 0..NUM_ITEMS {\n let (hdl, count) = new_handle();\n hdls.push(hdl);\n counts.push(count);\n }\n\n for hdl in hdls.iter() {\n unsafe { queue.push_front(hdl.clone()) };\n }\n\n drop(queue);\n\n assert!(hdls.into_iter().all(|hdl| hdl.inner_strong_count() == 1));\n // drop should not wake any entries\n assert!(counts.into_iter().all(|count| count.get() == 0));\n });\n}\n" }, { "path": "tokio/src/runtime/time_alt/wake_queue.rs", "content": "use super::{Entry, EntryHandle, WakeQueueEntry};\nuse crate::util::linked_list;\n\ntype EntryList = linked_list::LinkedList;\n\n/// A queue of entries that need to be woken up.\n#[derive(Debug)]\npub(crate) struct WakeQueue {\n list: EntryList,\n}\n\nimpl Drop for WakeQueue {\n fn drop(&mut self) {\n // drain all entries without waking them up\n while let Some(hdl) = self.list.pop_front() {\n drop(hdl);\n }\n }\n}\n\nimpl WakeQueue {\n pub(crate) fn new() -> Self {\n Self {\n list: EntryList::new(),\n }\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.list.is_empty()\n }\n\n /// # Safety\n ///\n /// Behavior is undefined if any of the following conditions are violated:\n ///\n /// - [`Entry::extra_pointers`] of `hdl` must not being used.\n pub(crate) unsafe fn push_front(&mut self, hdl: EntryHandle) {\n self.list.push_front(hdl);\n }\n\n /// Wakes all entries in the wake queue.\n pub(crate) fn wake_all(mut self) {\n while let Some(hdl) = self.list.pop_front() {\n hdl.wake();\n }\n }\n}\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/runtime/time_alt/wheel/level.rs", "content": "use super::{EntryHandle, EntryList};\nuse std::ptr::NonNull;\nuse std::{array, fmt};\n\n/// Wheel for a single level in the timer. This wheel contains 64 slots.\npub(crate) struct Level {\n level: usize,\n\n /// Bit field tracking which slots currently contain entries.\n ///\n /// Using a bit field to track slots that contain entries allows avoiding a\n /// scan to find entries. This field is updated when entries are added or\n /// removed from a slot.\n ///\n /// The least-significant bit represents slot zero.\n occupied: u64,\n\n /// Slots. We access these via the EntryInner `current_list` as well, so this needs to be an `UnsafeCell`.\n slot: [EntryList; LEVEL_MULT],\n}\n\n/// Indicates when a slot must be processed next.\n#[derive(Debug)]\npub(crate) struct Expiration {\n /// The level containing the slot.\n pub(crate) level: usize,\n\n /// The slot index.\n pub(crate) slot: usize,\n\n /// The instant at which the slot needs to be processed.\n pub(crate) deadline: u64,\n}\n\n/// Level multiplier.\n///\n/// Being a power of 2 is very important.\nconst LEVEL_MULT: usize = 64;\n\nimpl Level {\n pub(crate) fn new(level: usize) -> Level {\n Level {\n level,\n occupied: 0,\n slot: array::from_fn(|_| EntryList::default()),\n }\n }\n\n /// Finds the slot that needs to be processed next and returns the slot and\n /// `Instant` at which this slot must be processed.\n pub(crate) fn next_expiration(&self, now: u64) -> Option {\n // Use the `occupied` bit field to get the index of the next slot that\n // needs to be processed.\n let slot = self.next_occupied_slot(now)?;\n\n // From the slot index, calculate the `Instant` at which it needs to be\n // processed. This value *must* be in the future with respect to `now`.\n\n let level_range = level_range(self.level);\n let slot_range = slot_range(self.level);\n\n // Compute the start date of the current level by masking the low bits\n // of `now` (`level_range` is a power of 2).\n let level_start = now & !(level_range - 1);\n let mut deadline = level_start + slot as u64 * slot_range;\n\n if deadline <= now {\n // A timer is in a slot \"prior\" to the current time. This can occur\n // because we do not have an infinite hierarchy of timer levels, and\n // eventually a timer scheduled for a very distant time might end up\n // being placed in a slot that is beyond the end of all of the\n // arrays.\n //\n // To deal with this, we first limit timers to being scheduled no\n // more than MAX_DURATION ticks in the future; that is, they're at\n // most one rotation of the top level away. Then, we force timers\n // that logically would go into the top+1 level, to instead go into\n // the top level's slots.\n //\n // What this means is that the top level's slots act as a\n // pseudo-ring buffer, and we rotate around them indefinitely. If we\n // compute a deadline before now, and it's the top level, it\n // therefore means we're actually looking at a slot in the future.\n debug_assert_eq!(self.level, super::NUM_LEVELS - 1);\n\n deadline += level_range;\n }\n\n debug_assert!(\n deadline >= now,\n \"deadline={:016X}; now={:016X}; level={}; lr={:016X}, sr={:016X}, slot={}; occupied={:b}\",\n deadline,\n now,\n self.level,\n level_range,\n slot_range,\n slot,\n self.occupied\n );\n\n Some(Expiration {\n level: self.level,\n slot,\n deadline,\n })\n }\n\n fn next_occupied_slot(&self, now: u64) -> Option {\n if self.occupied == 0 {\n return None;\n }\n\n // Get the slot for now using Maths\n let now_slot = (now / slot_range(self.level)) as usize;\n let occupied = self.occupied.rotate_right(now_slot as u32);\n let zeros = occupied.trailing_zeros() as usize;\n let slot = (zeros + now_slot) % LEVEL_MULT;\n\n Some(slot)\n }\n\n pub(crate) unsafe fn add_entry(&mut self, hdl: EntryHandle) {\n // Safety: the associated entry must be valid.\n let deadline = hdl.deadline();\n let slot = slot_for(deadline, self.level);\n\n self.slot[slot].push_front(hdl);\n\n self.occupied |= occupied_bit(slot);\n }\n\n pub(crate) unsafe fn remove_entry(&mut self, hdl: EntryHandle) {\n let slot = slot_for(hdl.deadline(), self.level);\n\n unsafe { self.slot[slot].remove(NonNull::from(&hdl)) };\n if self.slot[slot].is_empty() {\n // The bit is currently set\n debug_assert!(self.occupied & occupied_bit(slot) != 0);\n\n // Unset the bit\n self.occupied ^= occupied_bit(slot);\n }\n }\n\n pub(crate) fn take_slot(&mut self, slot: usize) -> EntryList {\n self.occupied &= !occupied_bit(slot);\n\n std::mem::take(&mut self.slot[slot])\n }\n}\n\nimpl fmt::Debug for Level {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Level\")\n .field(\"occupied\", &self.occupied)\n .finish()\n }\n}\n\nfn occupied_bit(slot: usize) -> u64 {\n 1 << slot\n}\n\nfn slot_range(level: usize) -> u64 {\n LEVEL_MULT.pow(level as u32) as u64\n}\n\nfn level_range(level: usize) -> u64 {\n LEVEL_MULT as u64 * slot_range(level)\n}\n\n/// Converts a duration (milliseconds) and a level to a slot position.\nfn slot_for(duration: u64, level: usize) -> usize {\n ((duration >> (level * 6)) % LEVEL_MULT as u64) as usize\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_slot_for() {\n for pos in 0..64 {\n assert_eq!(pos as usize, slot_for(pos, 0));\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(pos, slot_for(a as u64, level));\n }\n }\n }\n}\n" }, { "path": "tokio/src/runtime/time_alt/wheel/mod.rs", "content": "mod level;\npub(crate) use self::level::Expiration;\nuse self::level::Level;\n\nuse super::cancellation_queue::Sender;\nuse super::{EntryHandle, EntryList, WakeQueue};\n\nuse std::array;\n\n/// Hashed timing wheel implementation.\n///\n/// See [`Driver`] documentation for some implementation notes.\n///\n/// [`Driver`]: crate::runtime::time::Driver\n#[derive(Debug)]\npub(crate) struct Wheel {\n /// The number of milliseconds elapsed since the wheel started.\n elapsed: u64,\n\n /// Timer wheel.\n ///\n /// Levels:\n ///\n /// * 1 ms slots / 64 ms range\n /// * 64 ms slots / ~ 4 sec range\n /// * ~ 4 sec slots / ~ 4 min range\n /// * ~ 4 min slots / ~ 4 hr range\n /// * ~ 4 hr slots / ~ 12 day range\n /// * ~ 12 day slots / ~ 2 yr range\n levels: Box<[Level; NUM_LEVELS]>,\n}\n\n/// Number of levels. Each level has 64 slots. By using 6 levels with 64 slots\n/// each, the timer is able to track time up to 2 years into the future with a\n/// precision of 1 millisecond.\nconst NUM_LEVELS: usize = 6;\n\n/// The maximum duration of a `Sleep`.\npub(super) const MAX_DURATION: u64 = (1 << (6 * NUM_LEVELS)) - 1;\n\nimpl Wheel {\n /// Creates a new timing wheel.\n pub(crate) fn new() -> Wheel {\n Wheel {\n elapsed: 0,\n levels: Box::new(array::from_fn(Level::new)),\n }\n }\n\n /// Returns the number of milliseconds that have elapsed since the timing\n /// wheel's creation.\n pub(crate) fn elapsed(&self) -> u64 {\n self.elapsed\n }\n\n /// Inserts an entry into the timing wheel.\n ///\n /// # Arguments\n ///\n /// * `hdl`: The entry handle to insert into the wheel.\n ///\n /// # Safety\n ///\n /// The caller must ensure:\n ///\n /// * The entry is not already registered in ANY wheel.\n pub(crate) unsafe fn insert(&mut self, hdl: EntryHandle, cancel_tx: Sender) {\n let deadline = hdl.deadline();\n\n assert!(deadline > self.elapsed);\n\n hdl.register_cancel_tx(cancel_tx);\n\n // Get the level at which the entry should be stored\n let level = self.level_for(deadline);\n unsafe {\n self.levels[level].add_entry(hdl);\n }\n\n debug_assert!({\n self.levels[level]\n .next_expiration(self.elapsed)\n .map(|e| e.deadline >= self.elapsed)\n .unwrap_or(true)\n });\n }\n\n /// Removes `item` from the timing wheel.\n ///\n /// # Safety\n ///\n /// The caller must ensure:\n ///\n /// * The entry is already registered in THIS wheel.\n pub(crate) unsafe fn remove(&mut self, hdl: EntryHandle) {\n let deadline = hdl.deadline();\n debug_assert!(\n self.elapsed <= deadline,\n \"elapsed={}; deadline={}\",\n self.elapsed,\n deadline\n );\n\n let level = self.level_for(deadline);\n unsafe { self.levels[level].remove_entry(hdl.clone()) };\n }\n\n /// Advances the timer up to the instant represented by `now`.\n pub(crate) fn take_expired(&mut self, now: u64, wake_queue: &mut WakeQueue) {\n while let Some(expiration) = self\n .next_expiration()\n .filter(|expiration| expiration.deadline <= now)\n {\n self.process_expiration(&expiration, wake_queue);\n\n self.set_elapsed(expiration.deadline);\n }\n self.set_elapsed(now);\n }\n\n /// Returns the instant at which the next timeout expires.\n fn next_expiration(&self) -> Option {\n // Check all levels\n self.levels\n .iter()\n .enumerate()\n .find_map(|(level_num, level)| {\n let expiration = level.next_expiration(self.elapsed)?;\n // There cannot be any expirations at a higher level that happen\n // before this one.\n debug_assert!(self.no_expirations_before(level_num + 1, expiration.deadline));\n\n Some(expiration)\n })\n }\n\n /// Returns the tick at which this timer wheel next needs to perform some\n /// processing, or None if there are no timers registered.\n pub(crate) fn next_expiration_time(&self) -> Option {\n self.next_expiration().map(|ex| ex.deadline)\n }\n\n /// Used for debug assertions\n fn no_expirations_before(&self, start_level: usize, before: u64) -> bool {\n self.levels[start_level..]\n .iter()\n .flat_map(|level| level.next_expiration(self.elapsed))\n .all(|e2| before <= e2.deadline)\n }\n\n /// iteratively find entries that are between the wheel's current\n /// time and the expiration time. for each in that population either\n /// queue it for notification (in the case of the last level) or tier\n /// it down to the next level (in all other cases).\n pub(crate) fn process_expiration(\n &mut self,\n expiration: &Expiration,\n wake_queue: &mut WakeQueue,\n ) {\n // Note that we need to take _all_ of the entries off the list before\n // processing any of them. This is important because it's possible that\n // those entries might need to be reinserted into the same slot.\n //\n // This happens only on the highest level, when an entry is inserted\n // more than MAX_DURATION into the future. When this happens, we wrap\n // around, and process some entries a multiple of MAX_DURATION before\n // they actually need to be dropped down a level. We then reinsert them\n // back into the same position; we must make sure we don't then process\n // those entries again or we'll end up in an infinite loop.\n let mut entries = self.take_entries(expiration);\n\n while let Some(hdl) = entries.pop_back() {\n if expiration.level == 0 {\n debug_assert_eq!(hdl.deadline(), expiration.deadline);\n }\n\n let deadline = hdl.deadline();\n\n if deadline > expiration.deadline {\n let level = level_for(expiration.deadline, deadline);\n unsafe {\n self.levels[level].add_entry(hdl);\n }\n } else {\n unsafe {\n wake_queue.push_front(hdl);\n }\n }\n }\n }\n\n fn set_elapsed(&mut self, when: u64) {\n assert!(\n self.elapsed <= when,\n \"elapsed={:?}; when={:?}\",\n self.elapsed,\n when\n );\n\n if when > self.elapsed {\n self.elapsed = when;\n }\n }\n\n /// Obtains the list of entries that need processing for the given expiration.\n fn take_entries(&mut self, expiration: &Expiration) -> EntryList {\n self.levels[expiration.level].take_slot(expiration.slot)\n }\n\n fn level_for(&self, when: u64) -> usize {\n level_for(self.elapsed, when)\n }\n}\n\nfn level_for(elapsed: u64, when: u64) -> usize {\n const SLOT_MASK: u64 = (1 << 6) - 1;\n\n // Mask in the trailing bits ignored by the level calculation in order to cap\n // the possible leading zeros\n let mut masked = elapsed ^ when | SLOT_MASK;\n\n if masked >= MAX_DURATION {\n // Fudge the timer into the top level\n masked = MAX_DURATION - 1;\n }\n\n let leading_zeros = masked.leading_zeros() as usize;\n let significant = 63 - leading_zeros;\n\n significant / NUM_LEVELS\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_level_for() {\n for pos in 0..64 {\n assert_eq!(0, level_for(0, pos), \"level_for({pos}) -- binary = {pos:b}\");\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n\n if pos > level {\n let a = a - 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n\n if pos < 64 {\n let a = a + 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n }\n }\n }\n}\n" }, { "path": "tokio/src/signal/ctrl_c.rs", "content": "#[cfg(unix)]\nuse super::unix::{self as os_impl};\n#[cfg(windows)]\nuse super::windows::{self as os_impl};\n\nuse std::io;\n\n/// Completes when a \"ctrl-c\" notification is sent to the process.\n///\n/// While signals are handled very differently between Unix and Windows, both\n/// platforms support receiving a signal on \"ctrl-c\". This function provides a\n/// portable API for receiving this notification.\n///\n/// Once the returned future is polled, a listener is registered. The future\n/// will complete on the first received `ctrl-c` **after** the initial call to\n/// either `Future::poll` or `.await`.\n///\n/// # Caveats\n///\n/// On Unix platforms, the first time that a `Signal` instance is registered for a\n/// particular signal kind, an OS signal-handler is installed which replaces the\n/// default platform behavior when that signal is received, **for the duration of\n/// the entire process**.\n///\n/// For example, Unix systems will terminate a process by default when it\n/// receives a signal generated by `\"CTRL+C\"` on the terminal. But, when a\n/// `ctrl_c` stream is created to listen for this signal, the time it arrives,\n/// it will be translated to a stream event, and the process will continue to\n/// execute. **Even if this `Signal` instance is dropped, subsequent `SIGINT`\n/// deliveries will end up captured by Tokio, and the default platform behavior\n/// will NOT be reset**.\n///\n/// Thus, applications should take care to ensure the expected signal behavior\n/// occurs as expected after listening for specific signals.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal;\n///\n/// #[tokio::main]\n/// async fn main() {\n/// println!(\"waiting for ctrl-c\");\n///\n/// signal::ctrl_c().await.expect(\"failed to listen for event\");\n///\n/// println!(\"received ctrl-c event\");\n/// }\n/// ```\n///\n/// Listen in the background:\n///\n/// ```rust,no_run\n/// tokio::spawn(async move {\n/// tokio::signal::ctrl_c().await.unwrap();\n/// // Your handler here\n/// });\n/// ```\npub async fn ctrl_c() -> io::Result<()> {\n os_impl::ctrl_c()?.recv().await;\n Ok(())\n}\n" }, { "path": "tokio/src/signal/mod.rs", "content": "//! Asynchronous signal handling for Tokio.\n//!\n//! Note that signal handling is in general a very tricky topic and should be\n//! used with great care. This crate attempts to implement 'best practice' for\n//! signal handling, but it should be evaluated for your own applications' needs\n//! to see if it's suitable.\n//!\n//! There are some fundamental limitations of this crate documented on the OS\n//! specific structures, as well.\n//!\n//! # Examples\n//!\n//! Print on \"ctrl-c\" notification.\n//!\n//! ```rust,no_run\n//! use tokio::signal;\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! signal::ctrl_c().await?;\n//! println!(\"ctrl-c received!\");\n//! Ok(())\n//! }\n//! ```\n//!\n//! Wait for `SIGHUP` on Unix\n//!\n//! ```rust,no_run\n//! # #[cfg(unix)] {\n//! use tokio::signal::unix::{signal, SignalKind};\n//!\n//! #[tokio::main]\n//! async fn main() -> Result<(), Box> {\n//! // An infinite stream of hangup signals.\n//! let mut stream = signal(SignalKind::hangup())?;\n//!\n//! // Print whenever a HUP signal is received\n//! loop {\n//! stream.recv().await;\n//! println!(\"got signal HUP\");\n//! }\n//! }\n//! # }\n//! ```\nuse crate::sync::watch::Receiver;\nuse std::task::{Context, Poll};\n\n#[cfg(feature = \"signal\")]\nmod ctrl_c;\n#[cfg(feature = \"signal\")]\npub use ctrl_c::ctrl_c;\n\n#[cfg(unix)]\npub(crate) mod registry;\n\npub mod unix;\npub mod windows;\n\nmod reusable_box;\nuse self::reusable_box::ReusableBoxFuture;\n\n#[derive(Debug)]\nstruct RxFuture {\n inner: ReusableBoxFuture>,\n}\n\nasync fn make_future(mut rx: Receiver<()>) -> Receiver<()> {\n rx.changed().await.expect(\"signal sender went away\");\n rx\n}\n\nimpl RxFuture {\n fn new(rx: Receiver<()>) -> Self {\n Self {\n inner: ReusableBoxFuture::new(make_future(rx)),\n }\n }\n\n async fn recv(&mut self) {\n use std::future::poll_fn;\n poll_fn(|cx| self.poll_recv(cx)).await\n }\n\n fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll<()> {\n self.inner\n .poll(cx)\n .map(|rx| self.inner.set(make_future(rx)))\n }\n}\n" }, { "path": "tokio/src/signal/registry.rs", "content": "use crate::signal::unix::{OsExtraData, OsStorage};\nuse crate::sync::watch;\n\nuse std::ops;\nuse std::sync::atomic::{AtomicBool, Ordering};\nuse std::sync::OnceLock;\n\npub(crate) type EventId = usize;\n\n/// State for a specific event, whether a notification is pending delivery,\n/// and what listeners are registered.\n#[derive(Debug)]\npub(crate) struct EventInfo {\n pending: AtomicBool,\n tx: watch::Sender<()>,\n}\n\nimpl Default for EventInfo {\n fn default() -> Self {\n let (tx, _rx) = watch::channel(());\n\n Self {\n pending: AtomicBool::new(false),\n tx,\n }\n }\n}\n\n/// An interface for retrieving the `EventInfo` for a particular `eventId`.\npub(crate) trait Storage {\n /// Gets the `EventInfo` for `id` if it exists.\n fn event_info(&self, id: EventId) -> Option<&EventInfo>;\n\n /// Invokes `f` once for each defined `EventInfo` in this storage.\n fn for_each<'a, F>(&'a self, f: F)\n where\n F: FnMut(&'a EventInfo);\n}\n\nimpl Storage for Vec {\n fn event_info(&self, id: EventId) -> Option<&EventInfo> {\n self.get(id)\n }\n\n fn for_each<'a, F>(&'a self, f: F)\n where\n F: FnMut(&'a EventInfo),\n {\n self.iter().for_each(f);\n }\n}\n\n/// Manages and distributes event notifications to any registered listeners.\n///\n/// Generic over the underlying storage to allow for domain specific\n/// optimizations (e.g. `eventIds` may or may not be contiguous).\n#[derive(Debug)]\npub(crate) struct Registry {\n storage: S,\n}\n\nimpl Registry {\n fn new(storage: S) -> Self {\n Self { storage }\n }\n}\n\nimpl Registry {\n /// Registers a new listener for `event_id`.\n fn register_listener(&self, event_id: EventId) -> watch::Receiver<()> {\n self.storage\n .event_info(event_id)\n .unwrap_or_else(|| panic!(\"invalid event_id: {event_id}\"))\n .tx\n .subscribe()\n }\n\n /// Marks `event_id` as having been delivered, without broadcasting it to\n /// any listeners.\n fn record_event(&self, event_id: EventId) {\n if let Some(event_info) = self.storage.event_info(event_id) {\n event_info.pending.store(true, Ordering::SeqCst);\n }\n }\n\n /// Broadcasts all previously recorded events to their respective listeners.\n ///\n /// Returns `true` if an event was delivered to at least one listener.\n fn broadcast(&self) -> bool {\n let mut did_notify = false;\n self.storage.for_each(|event_info| {\n // Any signal of this kind arrived since we checked last?\n if !event_info.pending.swap(false, Ordering::SeqCst) {\n return;\n }\n\n // Ignore errors if there are no listeners\n if event_info.tx.send(()).is_ok() {\n did_notify = true;\n }\n });\n\n did_notify\n }\n}\n\npub(crate) struct Globals {\n extra: OsExtraData,\n registry: Registry,\n}\n\nimpl ops::Deref for Globals {\n type Target = OsExtraData;\n\n fn deref(&self) -> &Self::Target {\n &self.extra\n }\n}\n\nimpl Globals {\n /// Registers a new listener for `event_id`.\n pub(crate) fn register_listener(&self, event_id: EventId) -> watch::Receiver<()> {\n self.registry.register_listener(event_id)\n }\n\n /// Marks `event_id` as having been delivered, without broadcasting it to\n /// any listeners.\n pub(crate) fn record_event(&self, event_id: EventId) {\n self.registry.record_event(event_id);\n }\n\n /// Broadcasts all previously recorded events to their respective listeners.\n ///\n /// Returns `true` if an event was delivered to at least one listener.\n pub(crate) fn broadcast(&self) -> bool {\n self.registry.broadcast()\n }\n\n #[cfg(unix)]\n pub(crate) fn storage(&self) -> &OsStorage {\n &self.registry.storage\n }\n}\n\nfn globals_init() -> Globals\nwhere\n OsExtraData: 'static + Send + Sync + Default,\n OsStorage: 'static + Send + Sync + Default,\n{\n Globals {\n extra: OsExtraData::default(),\n registry: Registry::new(OsStorage::default()),\n }\n}\n\npub(crate) fn globals() -> &'static Globals\nwhere\n OsExtraData: 'static + Send + Sync + Default,\n OsStorage: 'static + Send + Sync + Default,\n{\n static GLOBALS: OnceLock = OnceLock::new();\n\n GLOBALS.get_or_init(globals_init)\n}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use super::*;\n use crate::runtime::{self, Runtime};\n use crate::sync::{oneshot, watch};\n\n use futures::future;\n\n #[test]\n fn smoke() {\n let rt = rt();\n rt.block_on(async move {\n let registry = Registry::new(vec![\n EventInfo::default(),\n EventInfo::default(),\n EventInfo::default(),\n ]);\n\n let first = registry.register_listener(0);\n let second = registry.register_listener(1);\n let third = registry.register_listener(2);\n\n let (fire, wait) = oneshot::channel();\n\n crate::spawn(async {\n wait.await.expect(\"wait failed\");\n\n // Record some events which should get coalesced\n registry.record_event(0);\n registry.record_event(0);\n registry.record_event(1);\n registry.record_event(1);\n registry.broadcast();\n\n // Yield so the previous broadcast can get received\n //\n // This yields many times since the block_on task is only polled every 61\n // ticks.\n for _ in 0..100 {\n crate::task::yield_now().await;\n }\n\n // Send subsequent signal\n registry.record_event(0);\n registry.broadcast();\n\n drop(registry);\n });\n\n let _ = fire.send(());\n let all = future::join3(collect(first), collect(second), collect(third));\n\n let (first_results, second_results, third_results) = all.await;\n assert_eq!(2, first_results.len());\n assert_eq!(1, second_results.len());\n assert_eq!(0, third_results.len());\n });\n }\n\n #[test]\n #[should_panic = \"invalid event_id: 1\"]\n fn register_panics_on_invalid_input() {\n let registry = Registry::new(vec![EventInfo::default()]);\n\n registry.register_listener(1);\n }\n\n #[test]\n fn record_invalid_event_does_nothing() {\n let registry = Registry::new(vec![EventInfo::default()]);\n registry.record_event(1302);\n }\n\n #[test]\n fn broadcast_returns_if_at_least_one_event_fired() {\n let registry = Registry::new(vec![EventInfo::default(), EventInfo::default()]);\n\n registry.record_event(0);\n assert!(!registry.broadcast());\n\n let first = registry.register_listener(0);\n let second = registry.register_listener(1);\n\n registry.record_event(0);\n assert!(registry.broadcast());\n\n drop(first);\n registry.record_event(0);\n assert!(!registry.broadcast());\n\n drop(second);\n }\n\n fn rt() -> Runtime {\n runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap()\n }\n\n async fn collect(mut rx: watch::Receiver<()>) -> Vec<()> {\n let mut ret = vec![];\n\n while let Ok(v) = rx.changed().await {\n ret.push(v);\n }\n\n ret\n }\n}\n" }, { "path": "tokio/src/signal/reusable_box.rs", "content": "use std::alloc::Layout;\nuse std::future::Future;\nuse std::panic::AssertUnwindSafe;\nuse std::pin::Pin;\nuse std::ptr::{self, NonNull};\nuse std::task::{Context, Poll};\nuse std::{fmt, panic};\n\n/// A reusable `Pin + Send>>`.\n///\n/// This type lets you replace the future stored in the box without\n/// reallocating when the size and alignment permits this.\npub(crate) struct ReusableBoxFuture {\n boxed: NonNull + Send>,\n}\n\nimpl ReusableBoxFuture {\n /// Create a new `ReusableBoxFuture` containing the provided future.\n pub(crate) fn new(future: F) -> Self\n where\n F: Future + Send + 'static,\n {\n let boxed: Box + Send> = Box::new(future);\n\n let boxed = Box::into_raw(boxed);\n\n // SAFETY: Box::into_raw does not return null pointers.\n let boxed = unsafe { NonNull::new_unchecked(boxed) };\n\n Self { boxed }\n }\n\n /// Replaces the future currently stored in this box.\n ///\n /// This reallocates if and only if the layout of the provided future is\n /// different from the layout of the currently stored future.\n pub(crate) fn set(&mut self, future: F)\n where\n F: Future + Send + 'static,\n {\n if let Err(future) = self.try_set(future) {\n *self = Self::new(future);\n }\n }\n\n /// Replaces the future currently stored in this box.\n ///\n /// This function never reallocates, but returns an error if the provided\n /// future has a different size or alignment from the currently stored\n /// future.\n pub(crate) fn try_set(&mut self, future: F) -> Result<(), F>\n where\n F: Future + Send + 'static,\n {\n // SAFETY: The pointer is not dangling.\n let self_layout = {\n let dyn_future: &(dyn Future + Send) = unsafe { self.boxed.as_ref() };\n Layout::for_value(dyn_future)\n };\n\n if Layout::new::() == self_layout {\n // SAFETY: We just checked that the layout of F is correct.\n unsafe {\n self.set_same_layout(future);\n }\n\n Ok(())\n } else {\n Err(future)\n }\n }\n\n /// Sets the current future.\n ///\n /// # Safety\n ///\n /// This function requires that the layout of the provided future is the\n /// same as `self.layout`.\n unsafe fn set_same_layout(&mut self, future: F)\n where\n F: Future + Send + 'static,\n {\n // Drop the existing future, catching any panics.\n let result = panic::catch_unwind(AssertUnwindSafe(|| unsafe {\n ptr::drop_in_place(self.boxed.as_ptr());\n }));\n\n // Overwrite the future behind the pointer. This is safe because the\n // allocation was allocated with the same size and alignment as the type F.\n let self_ptr: *mut F = self.boxed.as_ptr() as *mut F;\n // SAFETY: The pointer is valid and the layout is exactly same.\n unsafe {\n ptr::write(self_ptr, future);\n }\n\n // Update the vtable of self.boxed. The pointer is not null because we\n // just got it from self.boxed, which is not null.\n self.boxed = unsafe { NonNull::new_unchecked(self_ptr) };\n\n // If the old future's destructor panicked, resume unwinding.\n match result {\n Ok(()) => {}\n Err(payload) => {\n panic::resume_unwind(payload);\n }\n }\n }\n\n /// Gets a pinned reference to the underlying future.\n pub(crate) fn get_pin(&mut self) -> Pin<&mut (dyn Future + Send)> {\n // SAFETY: The user of this box cannot move the box, and we do not move it\n // either.\n unsafe { Pin::new_unchecked(self.boxed.as_mut()) }\n }\n\n /// Polls the future stored inside this box.\n pub(crate) fn poll(&mut self, cx: &mut Context<'_>) -> Poll {\n self.get_pin().poll(cx)\n }\n}\n\nimpl Future for ReusableBoxFuture {\n type Output = T;\n\n /// Polls the future stored inside this box.\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n Pin::into_inner(self).get_pin().poll(cx)\n }\n}\n\n// The future stored inside ReusableBoxFuture must be Send.\nunsafe impl Send for ReusableBoxFuture {}\n\n// The only method called on self.boxed is poll, which takes &mut self, so this\n// struct being Sync does not permit any invalid access to the Future, even if\n// the future is not Sync.\nunsafe impl Sync for ReusableBoxFuture {}\n\n// Just like a Pin> is always Unpin, so is this type.\nimpl Unpin for ReusableBoxFuture {}\n\nimpl Drop for ReusableBoxFuture {\n fn drop(&mut self) {\n unsafe {\n drop(Box::from_raw(self.boxed.as_ptr()));\n }\n }\n}\n\nimpl fmt::Debug for ReusableBoxFuture {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"ReusableBoxFuture\").finish()\n }\n}\n\n#[cfg(test)]\nmod test {\n use super::ReusableBoxFuture;\n use futures::future::FutureExt;\n use std::alloc::Layout;\n use std::future::Future;\n use std::pin::Pin;\n use std::task::{Context, Poll};\n\n #[test]\n fn test_different_futures() {\n let fut = async move { 10 };\n // Not zero sized!\n assert_eq!(Layout::for_value(&fut).size(), 1);\n\n let mut b = ReusableBoxFuture::new(fut);\n\n assert_eq!(b.get_pin().now_or_never(), Some(10));\n\n b.try_set(async move { 20 })\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(20));\n\n b.try_set(async move { 30 })\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(30));\n }\n\n #[test]\n fn test_different_sizes() {\n let fut1 = async move { 10 };\n let val = [0u32; 1000];\n let fut2 = async move { val[0] };\n let fut3 = ZeroSizedFuture {};\n\n assert_eq!(Layout::for_value(&fut1).size(), 1);\n assert_eq!(Layout::for_value(&fut2).size(), 4004);\n assert_eq!(Layout::for_value(&fut3).size(), 0);\n\n let mut b = ReusableBoxFuture::new(fut1);\n assert_eq!(b.get_pin().now_or_never(), Some(10));\n b.set(fut2);\n assert_eq!(b.get_pin().now_or_never(), Some(0));\n b.set(fut3);\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n }\n\n struct ZeroSizedFuture {}\n impl Future for ZeroSizedFuture {\n type Output = u32;\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n Poll::Ready(5)\n }\n }\n\n #[test]\n fn test_zero_sized() {\n let fut = ZeroSizedFuture {};\n // Zero sized!\n assert_eq!(Layout::for_value(&fut).size(), 0);\n\n let mut b = ReusableBoxFuture::new(fut);\n\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n\n b.try_set(ZeroSizedFuture {})\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n }\n}\n" }, { "path": "tokio/src/signal/unix.rs", "content": "//! Unix-specific types for signal handling.\n//!\n//! This module is only defined on Unix platforms and contains the primary\n//! `Signal` type for receiving notifications of signals.\n\n#![cfg(unix)]\n#![cfg_attr(docsrs, doc(cfg(all(unix, feature = \"signal\"))))]\n\nuse crate::runtime::scheduler;\nuse crate::runtime::signal::Handle;\nuse crate::signal::registry::{globals, EventId, EventInfo, Globals, Storage};\nuse crate::signal::RxFuture;\nuse crate::sync::watch;\n\nuse mio::net::UnixStream;\nuse std::io::{self, Error, ErrorKind, Write};\nuse std::sync::OnceLock;\nuse std::task::{Context, Poll};\n\n#[cfg(not(any(target_os = \"linux\", target_os = \"illumos\")))]\npub(crate) struct OsStorage([SignalInfo; 33]);\n\n#[cfg(any(target_os = \"linux\", target_os = \"illumos\"))]\npub(crate) struct OsStorage(Box<[SignalInfo]>);\n\nimpl OsStorage {\n fn get(&self, id: EventId) -> Option<&SignalInfo> {\n self.0.get(id - 1)\n }\n}\n\nimpl Default for OsStorage {\n fn default() -> Self {\n // There are reliable signals ranging from 1 to 33 available on every Unix platform.\n #[cfg(not(any(target_os = \"linux\", target_os = \"illumos\")))]\n let inner = std::array::from_fn(|_| SignalInfo::default());\n\n // On Linux and illumos, there are additional real-time signals\n // available. (This is also likely true on Solaris, but this should be\n // verified before being enabled.)\n #[cfg(any(target_os = \"linux\", target_os = \"illumos\"))]\n let inner = std::iter::repeat_with(SignalInfo::default)\n .take(libc::SIGRTMAX() as usize)\n .collect();\n\n Self(inner)\n }\n}\n\nimpl Storage for OsStorage {\n fn event_info(&self, id: EventId) -> Option<&EventInfo> {\n self.get(id).map(|si| &si.event_info)\n }\n\n fn for_each<'a, F>(&'a self, f: F)\n where\n F: FnMut(&'a EventInfo),\n {\n self.0.iter().map(|si| &si.event_info).for_each(f);\n }\n}\n\n#[derive(Debug)]\npub(crate) struct OsExtraData {\n sender: UnixStream,\n pub(crate) receiver: UnixStream,\n}\n\nimpl Default for OsExtraData {\n fn default() -> Self {\n let (receiver, sender) = UnixStream::pair().expect(\"failed to create UnixStream\");\n\n Self { sender, receiver }\n }\n}\n\n/// Represents the specific kind of signal to listen for.\n#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]\npub struct SignalKind(libc::c_int);\n\nimpl SignalKind {\n /// Allows for listening to any valid OS signal.\n ///\n /// For example, this can be used for listening for platform-specific\n /// signals.\n /// ```rust,no_run\n /// # use tokio::signal::unix::SignalKind;\n /// # let signum = -1;\n /// // let signum = libc::OS_SPECIFIC_SIGNAL;\n /// let kind = SignalKind::from_raw(signum);\n /// ```\n // Use `std::os::raw::c_int` on public API to prevent leaking a non-stable\n // type alias from libc.\n // `libc::c_int` and `std::os::raw::c_int` are currently the same type, and are\n // unlikely to change to other types, but technically libc can change this\n // in the future minor version.\n // See https://github.com/tokio-rs/tokio/issues/3767 for more.\n pub const fn from_raw(signum: std::os::raw::c_int) -> Self {\n Self(signum as libc::c_int)\n }\n\n /// Get the signal's numeric value.\n ///\n /// ```rust\n /// # use tokio::signal::unix::SignalKind;\n /// let kind = SignalKind::interrupt();\n /// assert_eq!(kind.as_raw_value(), libc::SIGINT);\n /// ```\n pub const fn as_raw_value(&self) -> std::os::raw::c_int {\n self.0\n }\n\n /// Represents the `SIGALRM` signal.\n ///\n /// On Unix systems this signal is sent when a real-time timer has expired.\n /// By default, the process is terminated by this signal.\n pub const fn alarm() -> Self {\n Self(libc::SIGALRM)\n }\n\n /// Represents the `SIGCHLD` signal.\n ///\n /// On Unix systems this signal is sent when the status of a child process\n /// has changed. By default, this signal is ignored.\n pub const fn child() -> Self {\n Self(libc::SIGCHLD)\n }\n\n /// Represents the `SIGHUP` signal.\n ///\n /// On Unix systems this signal is sent when the terminal is disconnected.\n /// By default, the process is terminated by this signal.\n pub const fn hangup() -> Self {\n Self(libc::SIGHUP)\n }\n\n /// Represents the `SIGINFO` signal.\n ///\n /// On Unix systems this signal is sent to request a status update from the\n /// process. By default, this signal is ignored.\n #[cfg(any(\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"illumos\"\n ))]\n pub const fn info() -> Self {\n Self(libc::SIGINFO)\n }\n\n /// Represents the `SIGINT` signal.\n ///\n /// On Unix systems this signal is sent to interrupt a program.\n /// By default, the process is terminated by this signal.\n pub const fn interrupt() -> Self {\n Self(libc::SIGINT)\n }\n\n #[cfg(target_os = \"haiku\")]\n /// Represents the `SIGPOLL` signal.\n ///\n /// On POSIX systems this signal is sent when I/O operations are possible\n /// on some file descriptor. By default, this signal is ignored.\n pub const fn io() -> Self {\n Self(libc::SIGPOLL)\n }\n #[cfg(not(target_os = \"haiku\"))]\n /// Represents the `SIGIO` signal.\n ///\n /// On Unix systems this signal is sent when I/O operations are possible\n /// on some file descriptor. By default, this signal is ignored.\n pub const fn io() -> Self {\n Self(libc::SIGIO)\n }\n\n /// Represents the `SIGPIPE` signal.\n ///\n /// On Unix systems this signal is sent when the process attempts to write\n /// to a pipe which has no reader. By default, the process is terminated by\n /// this signal.\n pub const fn pipe() -> Self {\n Self(libc::SIGPIPE)\n }\n\n /// Represents the `SIGQUIT` signal.\n ///\n /// On Unix systems this signal is sent to issue a shutdown of the\n /// process, after which the OS will dump the process core.\n /// By default, the process is terminated by this signal.\n pub const fn quit() -> Self {\n Self(libc::SIGQUIT)\n }\n\n /// Represents the `SIGTERM` signal.\n ///\n /// On Unix systems this signal is sent to issue a shutdown of the\n /// process. By default, the process is terminated by this signal.\n pub const fn terminate() -> Self {\n Self(libc::SIGTERM)\n }\n\n /// Represents the `SIGUSR1` signal.\n ///\n /// On Unix systems this is a user defined signal.\n /// By default, the process is terminated by this signal.\n pub const fn user_defined1() -> Self {\n Self(libc::SIGUSR1)\n }\n\n /// Represents the `SIGUSR2` signal.\n ///\n /// On Unix systems this is a user defined signal.\n /// By default, the process is terminated by this signal.\n pub const fn user_defined2() -> Self {\n Self(libc::SIGUSR2)\n }\n\n /// Represents the `SIGWINCH` signal.\n ///\n /// On Unix systems this signal is sent when the terminal window is resized.\n /// By default, this signal is ignored.\n pub const fn window_change() -> Self {\n Self(libc::SIGWINCH)\n }\n}\n\nimpl From for SignalKind {\n fn from(signum: std::os::raw::c_int) -> Self {\n Self::from_raw(signum as libc::c_int)\n }\n}\n\nimpl From for std::os::raw::c_int {\n fn from(kind: SignalKind) -> Self {\n kind.as_raw_value()\n }\n}\n\n#[derive(Default)]\npub(crate) struct SignalInfo {\n event_info: EventInfo,\n init: OnceLock>>,\n}\n\n/// Our global signal handler for all signals registered by this module.\n///\n/// The purpose of this signal handler is to primarily:\n///\n/// 1. Flag that our specific signal was received (e.g. store an atomic flag)\n/// 2. Wake up the driver by writing a byte to a pipe\n///\n/// Those two operations should both be async-signal safe.\nfn action(globals: &'static Globals, signal: libc::c_int) {\n globals.record_event(signal as EventId);\n\n // Send a wakeup, ignore any errors (anything reasonably possible is\n // full pipe and then it will wake up anyway).\n let mut sender = &globals.sender;\n drop(sender.write(&[1]));\n}\n\n/// Enables this module to receive signal notifications for the `signal`\n/// provided.\n///\n/// This will register the signal handler if it hasn't already been registered,\n/// returning any error along the way if that fails.\nfn signal_enable(signal: SignalKind, handle: &Handle) -> io::Result<()> {\n let signal = signal.0;\n if signal <= 0 || signal_hook_registry::FORBIDDEN.contains(&signal) {\n return Err(Error::new(\n ErrorKind::Other,\n format!(\"Refusing to register signal {signal}\"),\n ));\n }\n\n // Check that we have a signal driver running\n handle.check_inner()?;\n\n let globals = globals();\n let siginfo = match globals.storage().get(signal as EventId) {\n Some(slot) => slot,\n None => return Err(io::Error::new(io::ErrorKind::Other, \"signal too large\")),\n };\n\n siginfo\n .init\n .get_or_init(|| {\n unsafe { signal_hook_registry::register(signal, move || action(globals, signal)) }\n .map(|_| ())\n .map_err(|e| e.raw_os_error())\n })\n .map_err(|e| {\n e.map_or_else(\n || Error::new(ErrorKind::Other, \"registering signal handler failed\"),\n Error::from_raw_os_error,\n )\n })\n}\n\n/// An listener for receiving a particular type of OS signal.\n///\n/// The listener can be turned into a `Stream` using [`SignalStream`].\n///\n/// [`SignalStream`]: https://docs.rs/tokio-stream/latest/tokio_stream/wrappers/struct.SignalStream.html\n///\n/// In general signal handling on Unix is a pretty tricky topic, and this\n/// structure is no exception! There are some important limitations to keep in\n/// mind when using `Signal` streams:\n///\n/// * Signals handling in Unix already necessitates coalescing signals\n/// together sometimes. This `Signal` stream is also no exception here in\n/// that it will also coalesce signals. That is, even if the signal handler\n/// for this process runs multiple times, the `Signal` stream may only return\n/// one signal notification. Specifically, before `poll` is called, all\n/// signal notifications are coalesced into one item returned from `poll`.\n/// Once `poll` has been called, however, a further signal is guaranteed to\n/// be yielded as an item.\n///\n/// Put another way, any element pulled off the returned listener corresponds to\n/// *at least one* signal, but possibly more.\n///\n/// * Signal handling in general is relatively inefficient. Although some\n/// improvements are possible in this crate, it's recommended to not plan on\n/// having millions of signal channels open.\n///\n/// If you've got any questions about this feel free to open an issue on the\n/// repo! New approaches to alleviate some of these limitations are always\n/// appreciated!\n///\n/// # Caveats\n///\n/// The first time that a `Signal` instance is registered for a particular\n/// signal kind, an OS signal-handler is installed which replaces the default\n/// platform behavior when that signal is received, **for the duration of the\n/// entire process**.\n///\n/// For example, Unix systems will terminate a process by default when it\n/// receives `SIGINT`. But, when a `Signal` instance is created to listen for\n/// this signal, the next `SIGINT` that arrives will be translated to a stream\n/// event, and the process will continue to execute. **Even if this `Signal`\n/// instance is dropped, subsequent `SIGINT` deliveries will end up captured by\n/// Tokio, and the default platform behavior will NOT be reset**.\n///\n/// Thus, applications should take care to ensure the expected signal behavior\n/// occurs as expected after listening for specific signals.\n///\n/// # Examples\n///\n/// Wait for `SIGHUP`\n///\n/// ```rust,no_run\n/// use tokio::signal::unix::{signal, SignalKind};\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // An infinite stream of hangup signals.\n/// let mut sig = signal(SignalKind::hangup())?;\n///\n/// // Print whenever a HUP signal is received\n/// loop {\n/// sig.recv().await;\n/// println!(\"got signal HUP\");\n/// }\n/// }\n/// ```\n#[must_use = \"streams do nothing unless polled\"]\n#[derive(Debug)]\npub struct Signal {\n inner: RxFuture,\n}\n\n/// Creates a new listener which will receive notifications when the current\n/// process receives the specified signal `kind`.\n///\n/// This function will create a new stream which binds to the default reactor.\n/// The `Signal` stream is an infinite stream which will receive\n/// notifications whenever a signal is received. More documentation can be\n/// found on `Signal` itself, but to reiterate:\n///\n/// * Signals may be coalesced beyond what the kernel already does.\n/// * Once a signal handler is registered with the process the underlying\n/// libc signal handler is never unregistered.\n///\n/// A `Signal` stream can be created for a particular signal number\n/// multiple times. When a signal is received then all the associated\n/// channels will receive the signal notification.\n///\n/// # Errors\n///\n/// * If the lower-level C functions fail for some reason.\n/// * If the previous initialization of this specific signal failed.\n/// * If the signal is one of\n/// [`signal_hook::FORBIDDEN`](fn@signal_hook_registry::register#panics)\n///\n/// # Panics\n///\n/// This function panics if there is no current reactor set, or if the `rt`\n/// feature flag is not enabled.\n#[track_caller]\npub fn signal(kind: SignalKind) -> io::Result {\n let handle = scheduler::Handle::current();\n let rx = signal_with_handle(kind, handle.driver().signal())?;\n\n Ok(Signal {\n inner: RxFuture::new(rx),\n })\n}\n\npub(crate) fn signal_with_handle(\n kind: SignalKind,\n handle: &Handle,\n) -> io::Result> {\n // Turn the signal delivery on once we are ready for it\n signal_enable(kind, handle)?;\n\n Ok(globals().register_listener(kind.0 as EventId))\n}\n\nimpl Signal {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no signal is lost.\n ///\n /// # Examples\n ///\n /// Wait for `SIGHUP`\n ///\n /// ```rust,no_run\n /// use tokio::signal::unix::{signal, SignalKind};\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // An infinite stream of hangup signals.\n /// let mut stream = signal(SignalKind::hangup())?;\n ///\n /// // Print whenever a HUP signal is received\n /// loop {\n /// stream.recv().await;\n /// println!(\"got signal HUP\");\n /// }\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::unix::Signal;\n ///\n /// struct MyFuture {\n /// signal: Signal,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.signal.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n\n// Work around for abstracting streams internally\n#[cfg(feature = \"process\")]\npub(crate) trait InternalStream {\n fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll>;\n}\n\n#[cfg(feature = \"process\")]\nimpl InternalStream for Signal {\n fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.poll_recv(cx)\n }\n}\n\npub(crate) fn ctrl_c() -> io::Result {\n signal(SignalKind::interrupt())\n}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use super::*;\n\n #[test]\n fn signal_enable_error_on_invalid_input() {\n let inputs = [-1, 0];\n\n for input in inputs {\n assert_eq!(\n signal_enable(SignalKind::from_raw(input), &Handle::default())\n .unwrap_err()\n .kind(),\n ErrorKind::Other,\n );\n }\n }\n\n #[test]\n fn signal_enable_error_on_forbidden_input() {\n let inputs = signal_hook_registry::FORBIDDEN;\n\n for &input in inputs {\n assert_eq!(\n signal_enable(SignalKind::from_raw(input), &Handle::default())\n .unwrap_err()\n .kind(),\n ErrorKind::Other,\n );\n }\n }\n\n #[test]\n fn from_c_int() {\n assert_eq!(SignalKind::from(2), SignalKind::interrupt());\n }\n\n #[test]\n fn into_c_int() {\n let value: std::os::raw::c_int = SignalKind::interrupt().into();\n assert_eq!(value, libc::SIGINT as _);\n }\n}\n" }, { "path": "tokio/src/signal/windows/stub.rs", "content": "//! Stub implementations for the platform API so that rustdoc can build linkable\n//! documentation on non-windows platforms.\n\nuse crate::signal::RxFuture;\nuse std::io;\n\npub(super) fn ctrl_break() -> io::Result {\n panic!()\n}\n\npub(super) fn ctrl_close() -> io::Result {\n panic!()\n}\n\npub(super) fn ctrl_c() -> io::Result {\n panic!()\n}\n\npub(super) fn ctrl_logoff() -> io::Result {\n panic!()\n}\n\npub(super) fn ctrl_shutdown() -> io::Result {\n panic!()\n}\n" }, { "path": "tokio/src/signal/windows/sys.rs", "content": "use std::io;\nuse std::sync::OnceLock;\n\nuse crate::signal::RxFuture;\nuse crate::sync::watch;\n\nuse windows_sys::core::BOOL;\nuse windows_sys::Win32::System::Console as console;\n\ntype EventInfo = watch::Sender<()>;\n\npub(super) fn ctrl_break() -> io::Result {\n new(®istry().ctrl_break)\n}\n\npub(super) fn ctrl_close() -> io::Result {\n new(®istry().ctrl_close)\n}\n\npub(super) fn ctrl_c() -> io::Result {\n new(®istry().ctrl_c)\n}\n\npub(super) fn ctrl_logoff() -> io::Result {\n new(®istry().ctrl_logoff)\n}\n\npub(super) fn ctrl_shutdown() -> io::Result {\n new(®istry().ctrl_shutdown)\n}\n\nfn new(event_info: &EventInfo) -> io::Result {\n global_init()?;\n let rx = event_info.subscribe();\n Ok(RxFuture::new(rx))\n}\n\nfn event_requires_infinite_sleep_in_handler(signum: u32) -> bool {\n // Returning from the handler function of those events immediately terminates the process.\n // So for async systems, the easiest solution is to simply never return from\n // the handler function.\n //\n // For more information, see:\n // https://learn.microsoft.com/en-us/windows/console/handlerroutine#remarks\n matches!(\n signum,\n console::CTRL_CLOSE_EVENT | console::CTRL_LOGOFF_EVENT | console::CTRL_SHUTDOWN_EVENT\n )\n}\n\n#[derive(Debug, Default)]\nstruct Registry {\n ctrl_break: EventInfo,\n ctrl_close: EventInfo,\n ctrl_c: EventInfo,\n ctrl_logoff: EventInfo,\n ctrl_shutdown: EventInfo,\n}\n\nimpl Registry {\n fn event_info(&self, signum: u32) -> Option<&EventInfo> {\n match signum {\n console::CTRL_BREAK_EVENT => Some(&self.ctrl_break),\n console::CTRL_CLOSE_EVENT => Some(&self.ctrl_close),\n console::CTRL_C_EVENT => Some(&self.ctrl_c),\n console::CTRL_LOGOFF_EVENT => Some(&self.ctrl_logoff),\n console::CTRL_SHUTDOWN_EVENT => Some(&self.ctrl_shutdown),\n _ => None,\n }\n }\n}\n\nfn registry() -> &'static Registry {\n static REGISTRY: OnceLock = OnceLock::new();\n\n REGISTRY.get_or_init(Default::default)\n}\n\nfn global_init() -> io::Result<()> {\n static INIT: OnceLock>> = OnceLock::new();\n\n INIT.get_or_init(|| {\n let rc = unsafe { console::SetConsoleCtrlHandler(Some(handler), 1) };\n if rc == 0 {\n Err(io::Error::last_os_error().raw_os_error())\n } else {\n Ok(())\n }\n })\n .map_err(|e| {\n e.map_or_else(\n || io::Error::new(io::ErrorKind::Other, \"registering signal handler failed\"),\n io::Error::from_raw_os_error,\n )\n })\n}\n\nunsafe extern \"system\" fn handler(ty: u32) -> BOOL {\n // Ignore unknown control signal types.\n let Some(event_info) = registry().event_info(ty) else {\n return 0;\n };\n\n // According to https://learn.microsoft.com/en-us/windows/console/handlerroutine\n // the handler routine is always invoked in a new thread, thus we don't\n // have the same restrictions as in Unix signal handlers, meaning we can\n // go ahead and perform the broadcast here.\n match event_info.send(()) {\n Ok(_) if event_requires_infinite_sleep_in_handler(ty) => loop {\n std::thread::park();\n },\n Ok(_) => 1,\n // No one is listening for this notification any more\n // let the OS fire the next (possibly the default) handler.\n Err(_) => 0,\n }\n}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use super::*;\n use crate::runtime::Runtime;\n\n use tokio_test::{assert_ok, assert_pending, assert_ready_ok, task};\n\n unsafe fn raise_event(signum: u32) {\n if event_requires_infinite_sleep_in_handler(signum) {\n // Those events will enter an infinite loop in `handler`, so\n // we need to run them on a separate thread\n std::thread::spawn(move || unsafe { super::handler(signum) });\n } else {\n unsafe { super::handler(signum) };\n }\n }\n\n #[test]\n fn ctrl_c() {\n let rt = rt();\n let _enter = rt.enter();\n\n let mut ctrl_c = task::spawn(crate::signal::ctrl_c());\n\n assert_pending!(ctrl_c.poll());\n\n // Windows doesn't have a good programmatic way of sending events\n // like sending signals on Unix, so we'll stub out the actual OS\n // integration and test that our handling works.\n unsafe {\n raise_event(console::CTRL_C_EVENT);\n }\n\n assert_ready_ok!(ctrl_c.poll());\n }\n\n #[test]\n fn ctrl_break() {\n let rt = rt();\n\n rt.block_on(async {\n let mut ctrl_break = assert_ok!(crate::signal::windows::ctrl_break());\n\n // Windows doesn't have a good programmatic way of sending events\n // like sending signals on Unix, so we'll stub out the actual OS\n // integration and test that our handling works.\n unsafe {\n raise_event(console::CTRL_BREAK_EVENT);\n }\n\n ctrl_break.recv().await.unwrap();\n });\n }\n\n #[test]\n fn ctrl_close() {\n let rt = rt();\n\n rt.block_on(async {\n let mut ctrl_close = assert_ok!(crate::signal::windows::ctrl_close());\n\n // Windows doesn't have a good programmatic way of sending events\n // like sending signals on Unix, so we'll stub out the actual OS\n // integration and test that our handling works.\n unsafe {\n raise_event(console::CTRL_CLOSE_EVENT);\n }\n\n ctrl_close.recv().await.unwrap();\n });\n }\n\n #[test]\n fn ctrl_shutdown() {\n let rt = rt();\n\n rt.block_on(async {\n let mut ctrl_shutdown = assert_ok!(crate::signal::windows::ctrl_shutdown());\n\n // Windows doesn't have a good programmatic way of sending events\n // like sending signals on Unix, so we'll stub out the actual OS\n // integration and test that our handling works.\n unsafe {\n raise_event(console::CTRL_SHUTDOWN_EVENT);\n }\n\n ctrl_shutdown.recv().await.unwrap();\n });\n }\n\n #[test]\n fn ctrl_logoff() {\n let rt = rt();\n\n rt.block_on(async {\n let mut ctrl_logoff = assert_ok!(crate::signal::windows::ctrl_logoff());\n\n // Windows doesn't have a good programmatic way of sending events\n // like sending signals on Unix, so we'll stub out the actual OS\n // integration and test that our handling works.\n unsafe {\n raise_event(console::CTRL_LOGOFF_EVENT);\n }\n\n ctrl_logoff.recv().await.unwrap();\n });\n }\n\n fn rt() -> Runtime {\n crate::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n }\n}\n" }, { "path": "tokio/src/signal/windows.rs", "content": "//! Windows-specific types for signal handling.\n//!\n//! This module is only defined on Windows and allows receiving \"ctrl-c\",\n//! \"ctrl-break\", \"ctrl-logoff\", \"ctrl-shutdown\", and \"ctrl-close\"\n//! notifications. These events are listened for via the `SetConsoleCtrlHandler`\n//! function which receives the corresponding `windows_sys` event type.\n\n#![cfg(any(windows, docsrs))]\n#![cfg_attr(docsrs, doc(cfg(all(windows, feature = \"signal\"))))]\n\nuse crate::signal::RxFuture;\nuse std::io;\nuse std::task::{Context, Poll};\n\n#[cfg(windows)]\n#[path = \"windows/sys.rs\"]\nmod imp;\n\n// For building documentation on Unix machines when the `docsrs` flag is set.\n#[cfg(not(windows))]\n#[path = \"windows/stub.rs\"]\nmod imp;\n\n/// Creates a new listener which receives \"ctrl-c\" notifications sent to the\n/// process.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal::windows::ctrl_c;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // A listener of CTRL-C events.\n/// let mut signal = ctrl_c()?;\n///\n/// // Print whenever a CTRL-C event is received.\n/// for countdown in (0..3).rev() {\n/// signal.recv().await;\n/// println!(\"got CTRL-C. {} more to exit\", countdown);\n/// }\n///\n/// Ok(())\n/// }\n/// ```\npub fn ctrl_c() -> io::Result {\n Ok(CtrlC {\n inner: self::imp::ctrl_c()?,\n })\n}\n\n/// Represents a listener which receives \"ctrl-c\" notifications sent to the process\n/// via `SetConsoleCtrlHandler`.\n///\n/// This event can be turned into a `Stream` using [`CtrlCStream`].\n///\n/// [`CtrlCStream`]: https://docs.rs/tokio-stream/latest/tokio_stream/wrappers/struct.CtrlCStream.html\n///\n/// A notification to this process notifies *all* receivers for\n/// this event. Moreover, the notifications **are coalesced** if they aren't processed\n/// quickly enough. This means that if two notifications are received back-to-back,\n/// then the listener may only receive one item about the two notifications.\n#[must_use = \"listeners do nothing unless polled\"]\n#[derive(Debug)]\npub struct CtrlC {\n inner: RxFuture,\n}\n\nimpl CtrlC {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::signal::windows::ctrl_c;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// let mut signal = ctrl_c()?;\n ///\n /// // Print whenever a CTRL-C event is received.\n /// for countdown in (0..3).rev() {\n /// signal.recv().await;\n /// println!(\"got CTRL-C. {} more to exit\", countdown);\n /// }\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::windows::CtrlC;\n ///\n /// struct MyFuture {\n /// ctrl_c: CtrlC,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.ctrl_c.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n\n/// Represents a listener which receives \"ctrl-break\" notifications sent to the process\n/// via `SetConsoleCtrlHandler`.\n///\n/// This listener can be turned into a `Stream` using [`CtrlBreakStream`].\n///\n/// [`CtrlBreakStream`]: https://docs.rs/tokio-stream/latest/tokio_stream/wrappers/struct.CtrlBreakStream.html\n///\n/// A notification to this process notifies *all* receivers for\n/// this event. Moreover, the notifications **are coalesced** if they aren't processed\n/// quickly enough. This means that if two notifications are received back-to-back,\n/// then the listener may only receive one item about the two notifications.\n#[must_use = \"listeners do nothing unless polled\"]\n#[derive(Debug)]\npub struct CtrlBreak {\n inner: RxFuture,\n}\n\nimpl CtrlBreak {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::signal::windows::ctrl_break;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // A listener of CTRL-BREAK events.\n /// let mut signal = ctrl_break()?;\n ///\n /// // Print whenever a CTRL-BREAK event is received.\n /// loop {\n /// signal.recv().await;\n /// println!(\"got signal CTRL-BREAK\");\n /// }\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::windows::CtrlBreak;\n ///\n /// struct MyFuture {\n /// ctrl_break: CtrlBreak,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.ctrl_break.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n\n/// Creates a new listener which receives \"ctrl-break\" notifications sent to the\n/// process.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal::windows::ctrl_break;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // A listener of CTRL-BREAK events.\n/// let mut signal = ctrl_break()?;\n///\n/// // Print whenever a CTRL-BREAK event is received.\n/// loop {\n/// signal.recv().await;\n/// println!(\"got signal CTRL-BREAK\");\n/// }\n/// }\n/// ```\npub fn ctrl_break() -> io::Result {\n Ok(CtrlBreak {\n inner: self::imp::ctrl_break()?,\n })\n}\n\n/// Creates a new listener which receives \"ctrl-close\" notifications sent to the\n/// process.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal::windows::ctrl_close;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // A listener of CTRL-CLOSE events.\n/// let mut signal = ctrl_close()?;\n///\n/// // Print whenever a CTRL-CLOSE event is received.\n/// for countdown in (0..3).rev() {\n/// signal.recv().await;\n/// println!(\"got CTRL-CLOSE. {} more to exit\", countdown);\n/// }\n///\n/// Ok(())\n/// }\n/// ```\npub fn ctrl_close() -> io::Result {\n Ok(CtrlClose {\n inner: self::imp::ctrl_close()?,\n })\n}\n\n/// Represents a listener which receives \"ctrl-close\" notifications sent to the process\n/// via `SetConsoleCtrlHandler`.\n///\n/// A notification to this process notifies *all* listeners listening for\n/// this event. Moreover, the notifications **are coalesced** if they aren't processed\n/// quickly enough. This means that if two notifications are received back-to-back,\n/// then the listener may only receive one item about the two notifications.\n#[must_use = \"listeners do nothing unless polled\"]\n#[derive(Debug)]\npub struct CtrlClose {\n inner: RxFuture,\n}\n\nimpl CtrlClose {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::signal::windows::ctrl_close;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // A listener of CTRL-CLOSE events.\n /// let mut signal = ctrl_close()?;\n ///\n /// // Print whenever a CTRL-CLOSE event is received.\n /// signal.recv().await;\n /// println!(\"got CTRL-CLOSE. Cleaning up before exiting\");\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::windows::CtrlClose;\n ///\n /// struct MyFuture {\n /// ctrl_close: CtrlClose,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.ctrl_close.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n\n/// Creates a new listener which receives \"ctrl-shutdown\" notifications sent to the\n/// process.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal::windows::ctrl_shutdown;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // A listener of CTRL-SHUTDOWN events.\n/// let mut signal = ctrl_shutdown()?;\n///\n/// signal.recv().await;\n/// println!(\"got CTRL-SHUTDOWN. Cleaning up before exiting\");\n///\n/// Ok(())\n/// }\n/// ```\npub fn ctrl_shutdown() -> io::Result {\n Ok(CtrlShutdown {\n inner: self::imp::ctrl_shutdown()?,\n })\n}\n\n/// Represents a listener which receives \"ctrl-shutdown\" notifications sent to the process\n/// via `SetConsoleCtrlHandler`.\n///\n/// A notification to this process notifies *all* listeners listening for\n/// this event. Moreover, the notifications **are coalesced** if they aren't processed\n/// quickly enough. This means that if two notifications are received back-to-back,\n/// then the listener may only receive one item about the two notifications.\n#[must_use = \"listeners do nothing unless polled\"]\n#[derive(Debug)]\npub struct CtrlShutdown {\n inner: RxFuture,\n}\n\nimpl CtrlShutdown {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::signal::windows::ctrl_shutdown;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // A listener of CTRL-SHUTDOWN events.\n /// let mut signal = ctrl_shutdown()?;\n ///\n /// // Print whenever a CTRL-SHUTDOWN event is received.\n /// signal.recv().await;\n /// println!(\"got CTRL-SHUTDOWN. Cleaning up before exiting\");\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::windows::CtrlShutdown;\n ///\n /// struct MyFuture {\n /// ctrl_shutdown: CtrlShutdown,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.ctrl_shutdown.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n\n/// Creates a new listener which receives \"ctrl-logoff\" notifications sent to the\n/// process.\n///\n/// # Examples\n///\n/// ```rust,no_run\n/// use tokio::signal::windows::ctrl_logoff;\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), Box> {\n/// // A listener of CTRL-LOGOFF events.\n/// let mut signal = ctrl_logoff()?;\n///\n/// signal.recv().await;\n/// println!(\"got CTRL-LOGOFF. Cleaning up before exiting\");\n///\n/// Ok(())\n/// }\n/// ```\npub fn ctrl_logoff() -> io::Result {\n Ok(CtrlLogoff {\n inner: self::imp::ctrl_logoff()?,\n })\n}\n\n/// Represents a listener which receives \"ctrl-logoff\" notifications sent to the process\n/// via `SetConsoleCtrlHandler`.\n///\n/// A notification to this process notifies *all* listeners listening for\n/// this event. Moreover, the notifications **are coalesced** if they aren't processed\n/// quickly enough. This means that if two notifications are received back-to-back,\n/// then the listener may only receive one item about the two notifications.\n#[must_use = \"listeners do nothing unless polled\"]\n#[derive(Debug)]\npub struct CtrlLogoff {\n inner: RxFuture,\n}\n\nimpl CtrlLogoff {\n /// Receives the next signal notification event.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// ```rust,no_run\n /// use tokio::signal::windows::ctrl_logoff;\n ///\n /// #[tokio::main]\n /// async fn main() -> Result<(), Box> {\n /// // An listener of CTRL-LOGOFF events.\n /// let mut signal = ctrl_logoff()?;\n ///\n /// // Print whenever a CTRL-LOGOFF event is received.\n /// signal.recv().await;\n /// println!(\"got CTRL-LOGOFF. Cleaning up before exiting\");\n ///\n /// Ok(())\n /// }\n /// ```\n pub async fn recv(&mut self) -> Option<()> {\n self.inner.recv().await;\n Some(())\n }\n\n /// Polls to receive the next signal notification event, outside of an\n /// `async` context.\n ///\n /// Although this returns `Option<()>`, it will never actually return `None`.\n /// This was accidentally exposed and would be a breaking change to be removed.\n ///\n /// # Examples\n ///\n /// Polling from a manually implemented future\n ///\n /// ```rust,no_run\n /// use std::pin::Pin;\n /// use std::future::Future;\n /// use std::task::{Context, Poll};\n /// use tokio::signal::windows::CtrlLogoff;\n ///\n /// struct MyFuture {\n /// ctrl_logoff: CtrlLogoff,\n /// }\n ///\n /// impl Future for MyFuture {\n /// type Output = Option<()>;\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// println!(\"polling MyFuture\");\n /// self.ctrl_logoff.poll_recv(cx)\n /// }\n /// }\n /// ```\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx).map(Some)\n }\n}\n" }, { "path": "tokio/src/sync/barrier.rs", "content": "use crate::loom::sync::Mutex;\nuse crate::sync::watch;\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\n\n/// A barrier enables multiple tasks to synchronize the beginning of some computation.\n///\n/// ```\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// use tokio::sync::Barrier;\n/// use std::sync::Arc;\n///\n/// let mut handles = Vec::with_capacity(10);\n/// let barrier = Arc::new(Barrier::new(10));\n/// for _ in 0..10 {\n/// let c = barrier.clone();\n/// // The same messages will be printed together.\n/// // You will NOT see any interleaving.\n/// handles.push(tokio::spawn(async move {\n/// println!(\"before wait\");\n/// let wait_result = c.wait().await;\n/// println!(\"after wait\");\n/// wait_result\n/// }));\n/// }\n///\n/// // Will not resolve until all \"after wait\" messages have been printed\n/// let mut num_leaders = 0;\n/// for handle in handles {\n/// let wait_result = handle.await.unwrap();\n/// if wait_result.is_leader() {\n/// num_leaders += 1;\n/// }\n/// }\n///\n/// // Exactly one barrier will resolve as the \"leader\"\n/// assert_eq!(num_leaders, 1);\n/// # }\n/// ```\n#[derive(Debug)]\npub struct Barrier {\n state: Mutex,\n wait: watch::Receiver,\n n: usize,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n}\n\n#[derive(Debug)]\nstruct BarrierState {\n waker: watch::Sender,\n arrived: usize,\n generation: usize,\n}\n\nimpl Barrier {\n /// Creates a new barrier that can block a given number of tasks.\n ///\n /// A barrier will block `n`-1 tasks which call [`Barrier::wait`] and then wake up all\n /// tasks at once when the `n`th task calls `wait`.\n #[track_caller]\n pub fn new(mut n: usize) -> Barrier {\n let (waker, wait) = crate::sync::watch::channel(0);\n\n if n == 0 {\n // if n is 0, it's not clear what behavior the user wants.\n // in std::sync::Barrier, an n of 0 exhibits the same behavior as n == 1, where every\n // .wait() immediately unblocks, so we adopt that here as well.\n n = 1;\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Barrier\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n );\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n size = n,\n );\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n arrived = 0,\n )\n });\n resource_span\n };\n\n Barrier {\n state: Mutex::new(BarrierState {\n waker,\n arrived: 0,\n generation: 1,\n }),\n n,\n wait,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Does not resolve until all tasks have rendezvoused here.\n ///\n /// Barriers are re-usable after all tasks have rendezvoused once, and can\n /// be used continuously.\n ///\n /// A single (arbitrary) future will receive a [`BarrierWaitResult`] that returns `true` from\n /// [`BarrierWaitResult::is_leader`] when returning from this function, and all other tasks\n /// will receive a result that will return `false` from `is_leader`.\n ///\n /// # Cancel safety\n ///\n /// This method is not cancel safe.\n pub async fn wait(&self) -> BarrierWaitResult {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n return trace::async_op(\n || self.wait_internal(),\n self.resource_span.clone(),\n \"Barrier::wait\",\n \"poll\",\n false,\n )\n .await;\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n return self.wait_internal().await;\n }\n async fn wait_internal(&self) -> BarrierWaitResult {\n crate::trace::async_trace_leaf().await;\n\n // NOTE: we are taking a _synchronous_ lock here.\n // It is okay to do so because the critical section is fast and never yields, so it cannot\n // deadlock even if another future is concurrently holding the lock.\n // It is _desirable_ to do so as synchronous Mutexes are, at least in theory, faster than\n // the asynchronous counter-parts, so we should use them where possible [citation needed].\n // NOTE: the extra scope here is so that the compiler doesn't think `state` is held across\n // a yield point, and thus marks the returned future as !Send.\n let generation = {\n let mut state = self.state.lock();\n let generation = state.generation;\n state.arrived += 1;\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n arrived = 1,\n arrived.op = \"add\",\n );\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n arrived = true,\n );\n if state.arrived == self.n {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n is_leader = true,\n );\n // we are the leader for this generation\n // wake everyone, increment the generation, and return\n state\n .waker\n .send(state.generation)\n .expect(\"there is at least one receiver\");\n state.arrived = 0;\n state.generation += 1;\n return BarrierWaitResult(true);\n }\n\n generation\n };\n\n // we're going to have to wait for the last of the generation to arrive\n let mut wait = self.wait.clone();\n\n loop {\n let _ = wait.changed().await;\n\n // note that the first time through the loop, this _will_ yield a generation\n // immediately, since we cloned a receiver that has never seen any values.\n if *wait.borrow() >= generation {\n break;\n }\n }\n\n BarrierWaitResult(false)\n }\n}\n\n/// A `BarrierWaitResult` is returned by `wait` when all tasks in the `Barrier` have rendezvoused.\n#[derive(Debug, Clone)]\npub struct BarrierWaitResult(bool);\n\nimpl BarrierWaitResult {\n /// Returns `true` if this task from wait is the \"leader task\".\n ///\n /// Only one task will have `true` returned from their result, all other tasks will have\n /// `false` returned.\n pub fn is_leader(&self) -> bool {\n self.0\n }\n}\n" }, { "path": "tokio/src/sync/batch_semaphore.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(unreachable_pub, dead_code))]\n//! # Implementation Details.\n//!\n//! The semaphore is implemented using an intrusive linked list of waiters. An\n//! atomic counter tracks the number of available permits. If the semaphore does\n//! not contain the required number of permits, the task attempting to acquire\n//! permits places its waker at the end of a queue. When new permits are made\n//! available (such as by releasing an initial acquisition), they are assigned\n//! to the task at the front of the queue, waking that task if its requested\n//! number of permits is met.\n//!\n//! Because waiters are enqueued at the back of the linked list and dequeued\n//! from the front, the semaphore is fair. Tasks trying to acquire large numbers\n//! of permits at a time will always be woken eventually, even if many other\n//! tasks are acquiring smaller numbers of permits. This means that in a\n//! use-case like tokio's read-write lock, writers will not be starved by\n//! readers.\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::{Mutex, MutexGuard};\nuse crate::util::linked_list::{self, LinkedList};\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\nuse crate::util::WakeList;\n\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::*;\nuse std::task::{ready, Context, Poll, Waker};\nuse std::{cmp, fmt};\n\n/// An asynchronous counting semaphore which permits waiting on multiple permits at once.\npub(crate) struct Semaphore {\n waiters: Mutex,\n /// The current number of available permits in the semaphore.\n permits: AtomicUsize,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n}\n\nstruct Waitlist {\n queue: LinkedList::Target>,\n closed: bool,\n}\n\n/// Error returned from the [`Semaphore::try_acquire`] function.\n///\n/// [`Semaphore::try_acquire`]: crate::sync::Semaphore::try_acquire\n#[derive(Debug, PartialEq, Eq)]\npub enum TryAcquireError {\n /// The semaphore has been [closed] and cannot issue new permits.\n ///\n /// [closed]: crate::sync::Semaphore::close\n Closed,\n\n /// The semaphore has no available permits.\n NoPermits,\n}\n/// Error returned from the [`Semaphore::acquire`] function.\n///\n/// An `acquire` operation can only fail if the semaphore has been\n/// [closed].\n///\n/// [closed]: crate::sync::Semaphore::close\n/// [`Semaphore::acquire`]: crate::sync::Semaphore::acquire\n#[derive(Debug)]\npub struct AcquireError(());\n\npub(crate) struct Acquire<'a> {\n node: Waiter,\n semaphore: &'a Semaphore,\n num_permits: usize,\n queued: bool,\n}\n\n/// An entry in the wait queue.\nstruct Waiter {\n /// The current state of the waiter.\n ///\n /// This is either the number of remaining permits required by\n /// the waiter, or a flag indicating that the waiter is not yet queued.\n state: AtomicUsize,\n\n /// The waker to notify the task awaiting permits.\n ///\n /// # Safety\n ///\n /// This may only be accessed while the wait queue is locked.\n waker: UnsafeCell>,\n\n /// Intrusive linked-list pointers.\n ///\n /// # Safety\n ///\n /// This may only be accessed while the wait queue is locked.\n ///\n /// TODO: Ideally, we would be able to use loom to enforce that\n /// this isn't accessed concurrently. However, it is difficult to\n /// use a `UnsafeCell` here, since the `Link` trait requires _returning_\n /// references to `Pointers`, and `UnsafeCell` requires that checked access\n /// take place inside a closure. We should consider changing `Pointers` to\n /// use `UnsafeCell` internally.\n pointers: linked_list::Pointers,\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n ctx: trace::AsyncOpTracingCtx,\n\n /// Should not be `Unpin`.\n _p: PhantomPinned,\n}\n\ngenerate_addr_of_methods! {\n impl<> Waiter {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull> {\n &self.pointers\n }\n }\n}\n\nimpl Semaphore {\n /// The maximum number of permits which a semaphore can hold.\n ///\n /// Note that this reserves three bits of flags in the permit counter, but\n /// we only actually use one of them. However, the previous semaphore\n /// implementation used three bits, so we will continue to reserve them to\n /// avoid a breaking change if additional flags need to be added in the\n /// future.\n pub(crate) const MAX_PERMITS: usize = usize::MAX >> 3;\n const CLOSED: usize = 1;\n // The least-significant bit in the number of permits is reserved to use\n // as a flag indicating that the semaphore has been closed. Consequently\n // PERMIT_SHIFT is used to leave that bit for that purpose.\n const PERMIT_SHIFT: usize = 1;\n\n /// Creates a new semaphore with the initial number of permits\n ///\n /// Maximum number of permits on 32-bit platforms is `1<<29`.\n pub(crate) fn new(permits: usize) -> Self {\n assert!(\n permits <= Self::MAX_PERMITS,\n \"a semaphore may not have more than MAX_PERMITS permits ({})\",\n Self::MAX_PERMITS\n );\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Semaphore\",\n kind = \"Sync\",\n is_internal = true\n );\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n permits = permits,\n permits.op = \"override\",\n )\n });\n resource_span\n };\n\n Self {\n permits: AtomicUsize::new(permits << Self::PERMIT_SHIFT),\n waiters: Mutex::new(Waitlist {\n queue: LinkedList::new(),\n closed: false,\n }),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Creates a new semaphore with the initial number of permits.\n ///\n /// Maximum number of permits on 32-bit platforms is `1<<29`.\n #[cfg(not(all(loom, test)))]\n pub(crate) const fn const_new(permits: usize) -> Self {\n assert!(permits <= Self::MAX_PERMITS);\n\n Self {\n permits: AtomicUsize::new(permits << Self::PERMIT_SHIFT),\n waiters: Mutex::const_new(Waitlist {\n queue: LinkedList::new(),\n closed: false,\n }),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Creates a new closed semaphore with 0 permits.\n pub(crate) fn new_closed() -> Self {\n Self {\n permits: AtomicUsize::new(Self::CLOSED),\n waiters: Mutex::new(Waitlist {\n queue: LinkedList::new(),\n closed: true,\n }),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Creates a new closed semaphore with 0 permits.\n #[cfg(not(all(loom, test)))]\n pub(crate) const fn const_new_closed() -> Self {\n Self {\n permits: AtomicUsize::new(Self::CLOSED),\n waiters: Mutex::const_new(Waitlist {\n queue: LinkedList::new(),\n closed: true,\n }),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Returns the current number of available permits.\n pub(crate) fn available_permits(&self) -> usize {\n self.permits.load(Acquire) >> Self::PERMIT_SHIFT\n }\n\n /// Adds `added` new permits to the semaphore.\n ///\n /// The maximum number of permits is `usize::MAX >> 3`, and this function will panic if the limit is exceeded.\n pub(crate) fn release(&self, added: usize) {\n if added == 0 {\n return;\n }\n\n // Assign permits to the wait queue\n self.add_permits_locked(added, self.waiters.lock());\n }\n\n /// Closes the semaphore. This prevents the semaphore from issuing new\n /// permits and notifies all pending waiters.\n pub(crate) fn close(&self) {\n let mut waiters = self.waiters.lock();\n // If the semaphore's permits counter has enough permits for an\n // unqueued waiter to acquire all the permits it needs immediately,\n // it won't touch the wait list. Therefore, we have to set a bit on\n // the permit counter as well. However, we must do this while\n // holding the lock --- otherwise, if we set the bit and then wait\n // to acquire the lock we'll enter an inconsistent state where the\n // permit counter is closed, but the wait list is not.\n self.permits.fetch_or(Self::CLOSED, Release);\n waiters.closed = true;\n while let Some(mut waiter) = waiters.queue.pop_back() {\n let waker = unsafe { waiter.as_mut().waker.with_mut(|waker| (*waker).take()) };\n if let Some(waker) = waker {\n waker.wake();\n }\n }\n }\n\n /// Returns true if the semaphore is closed.\n pub(crate) fn is_closed(&self) -> bool {\n self.permits.load(Acquire) & Self::CLOSED == Self::CLOSED\n }\n\n pub(crate) fn try_acquire(&self, num_permits: usize) -> Result<(), TryAcquireError> {\n assert!(\n num_permits <= Self::MAX_PERMITS,\n \"a semaphore may not have more than MAX_PERMITS permits ({})\",\n Self::MAX_PERMITS\n );\n let num_permits = num_permits << Self::PERMIT_SHIFT;\n let mut curr = self.permits.load(Acquire);\n loop {\n // Has the semaphore closed?\n if curr & Self::CLOSED == Self::CLOSED {\n return Err(TryAcquireError::Closed);\n }\n\n // Are there enough permits remaining?\n if curr < num_permits {\n return Err(TryAcquireError::NoPermits);\n }\n\n let next = curr - num_permits;\n\n match self.permits.compare_exchange(curr, next, AcqRel, Acquire) {\n Ok(_) => {\n // TODO: Instrument once issue has been solved\n return Ok(());\n }\n Err(actual) => curr = actual,\n }\n }\n }\n\n pub(crate) fn acquire(&self, num_permits: usize) -> Acquire<'_> {\n Acquire::new(self, num_permits)\n }\n\n /// Release `rem` permits to the semaphore's wait list, starting from the\n /// end of the queue.\n ///\n /// If `rem` exceeds the number of permits needed by the wait list, the\n /// remainder are assigned back to the semaphore.\n fn add_permits_locked(&self, mut rem: usize, waiters: MutexGuard<'_, Waitlist>) {\n let mut wakers = WakeList::new();\n let mut lock = Some(waiters);\n let mut is_empty = false;\n while rem > 0 {\n let mut waiters = lock.take().unwrap_or_else(|| self.waiters.lock());\n 'inner: while wakers.can_push() {\n // Was the waiter assigned enough permits to wake it?\n match waiters.queue.last() {\n Some(waiter) => {\n if !waiter.assign_permits(&mut rem) {\n break 'inner;\n }\n }\n None => {\n is_empty = true;\n // If we assigned permits to all the waiters in the queue, and there are\n // still permits left over, assign them back to the semaphore.\n break 'inner;\n }\n };\n let mut waiter = waiters.queue.pop_back().unwrap();\n if let Some(waker) =\n unsafe { waiter.as_mut().waker.with_mut(|waker| (*waker).take()) }\n {\n wakers.push(waker);\n }\n }\n\n if rem > 0 && is_empty {\n let permits = rem;\n assert!(\n permits <= Self::MAX_PERMITS,\n \"cannot add more than MAX_PERMITS permits ({})\",\n Self::MAX_PERMITS\n );\n let prev = self.permits.fetch_add(rem << Self::PERMIT_SHIFT, Release);\n let prev = prev >> Self::PERMIT_SHIFT;\n assert!(\n prev + permits <= Self::MAX_PERMITS,\n \"number of added permits ({}) would overflow MAX_PERMITS ({})\",\n rem,\n Self::MAX_PERMITS\n );\n\n // add remaining permits back\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n permits = rem,\n permits.op = \"add\",\n )\n });\n\n rem = 0;\n }\n\n drop(waiters); // release the lock\n\n wakers.wake_all();\n }\n\n assert_eq!(rem, 0);\n }\n\n /// Decrease a semaphore's permits by a maximum of `n`.\n ///\n /// If there are insufficient permits and it's not possible to reduce by `n`,\n /// return the number of permits that were actually reduced.\n pub(crate) fn forget_permits(&self, n: usize) -> usize {\n if n == 0 {\n return 0;\n }\n\n let mut curr_bits = self.permits.load(Acquire);\n loop {\n let curr = curr_bits >> Self::PERMIT_SHIFT;\n let new = curr.saturating_sub(n);\n match self.permits.compare_exchange_weak(\n curr_bits,\n new << Self::PERMIT_SHIFT,\n AcqRel,\n Acquire,\n ) {\n Ok(_) => return std::cmp::min(curr, n),\n Err(actual) => curr_bits = actual,\n };\n }\n }\n\n fn poll_acquire(\n &self,\n cx: &mut Context<'_>,\n num_permits: usize,\n node: Pin<&mut Waiter>,\n queued: bool,\n ) -> Poll> {\n let mut acquired = 0;\n\n let needed = if queued {\n node.state.load(Acquire) << Self::PERMIT_SHIFT\n } else {\n num_permits << Self::PERMIT_SHIFT\n };\n\n let mut lock = None;\n // First, try to take the requested number of permits from the\n // semaphore.\n let mut curr = self.permits.load(Acquire);\n let mut waiters = loop {\n // Has the semaphore closed?\n if curr & Self::CLOSED > 0 {\n return Poll::Ready(Err(AcquireError::closed()));\n }\n\n let mut remaining = 0;\n let total = curr\n .checked_add(acquired)\n .expect(\"number of permits must not overflow\");\n let (next, acq) = if total >= needed {\n let next = curr - (needed - acquired);\n (next, needed >> Self::PERMIT_SHIFT)\n } else {\n remaining = (needed - acquired) - curr;\n (0, curr >> Self::PERMIT_SHIFT)\n };\n\n if remaining > 0 && lock.is_none() {\n // No permits were immediately available, so this permit will\n // (probably) need to wait. We'll need to acquire a lock on the\n // wait queue before continuing. We need to do this _before_ the\n // CAS that sets the new value of the semaphore's `permits`\n // counter. Otherwise, if we subtract the permits and then\n // acquire the lock, we might miss additional permits being\n // added while waiting for the lock.\n lock = Some(self.waiters.lock());\n }\n\n match self.permits.compare_exchange(curr, next, AcqRel, Acquire) {\n Ok(_) => {\n acquired += acq;\n if remaining == 0 {\n if !queued {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n permits = acquired,\n permits.op = \"sub\",\n );\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n permits_obtained = acquired,\n permits.op = \"add\",\n )\n });\n\n return Poll::Ready(Ok(()));\n } else if lock.is_none() {\n break self.waiters.lock();\n }\n }\n break lock.expect(\"lock must be acquired before waiting\");\n }\n Err(actual) => curr = actual,\n }\n };\n\n if waiters.closed {\n return Poll::Ready(Err(AcquireError::closed()));\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n permits = acquired,\n permits.op = \"sub\",\n )\n });\n\n if node.assign_permits(&mut acquired) {\n self.add_permits_locked(acquired, waiters);\n return Poll::Ready(Ok(()));\n }\n\n assert_eq!(acquired, 0);\n let mut old_waker = None;\n\n // Otherwise, register the waker & enqueue the node.\n node.waker.with_mut(|waker| {\n // Safety: the wait list is locked, so we may modify the waker.\n let waker = unsafe { &mut *waker };\n // Do we need to register the new waker?\n if waker\n .as_ref()\n .map_or(true, |waker| !waker.will_wake(cx.waker()))\n {\n old_waker = waker.replace(cx.waker().clone());\n }\n });\n\n // If the waiter is not already in the wait queue, enqueue it.\n if !queued {\n let node = unsafe {\n let node = Pin::into_inner_unchecked(node) as *mut _;\n NonNull::new_unchecked(node)\n };\n\n waiters.queue.push_front(node);\n }\n drop(waiters);\n drop(old_waker);\n\n Poll::Pending\n }\n}\n\nimpl fmt::Debug for Semaphore {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Semaphore\")\n .field(\"permits\", &self.available_permits())\n .finish()\n }\n}\n\nimpl Waiter {\n fn new(\n num_permits: usize,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))] ctx: trace::AsyncOpTracingCtx,\n ) -> Self {\n Waiter {\n waker: UnsafeCell::new(None),\n state: AtomicUsize::new(num_permits),\n pointers: linked_list::Pointers::new(),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n ctx,\n _p: PhantomPinned,\n }\n }\n\n /// Assign permits to the waiter.\n ///\n /// Returns `true` if the waiter should be removed from the queue\n fn assign_permits(&self, n: &mut usize) -> bool {\n let mut curr = self.state.load(Acquire);\n loop {\n let assign = cmp::min(curr, *n);\n let next = curr - assign;\n match self.state.compare_exchange(curr, next, AcqRel, Acquire) {\n Ok(_) => {\n *n -= assign;\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.ctx.async_op_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n permits_obtained = assign,\n permits.op = \"add\",\n );\n });\n return next == 0;\n }\n Err(actual) => curr = actual,\n }\n }\n }\n}\n\nimpl Future for Acquire<'_> {\n type Output = Result<(), AcquireError>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n ready!(crate::trace::trace_leaf(cx));\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _resource_span = self.node.ctx.resource_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _async_op_span = self.node.ctx.async_op_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _async_op_poll_span = self.node.ctx.async_op_poll_span.clone().entered();\n\n let (node, semaphore, needed, queued) = self.project();\n\n // First, ensure the current task has enough budget to proceed.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let coop = ready!(trace_poll_op!(\n \"poll_acquire\",\n crate::task::coop::poll_proceed(cx),\n ));\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let result = match semaphore.poll_acquire(cx, needed, node, *queued) {\n Poll::Pending => {\n *queued = true;\n Poll::Pending\n }\n Poll::Ready(r) => {\n coop.made_progress();\n r?;\n *queued = false;\n Poll::Ready(Ok(()))\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n return trace_poll_op!(\"poll_acquire\", result);\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n return result;\n }\n}\n\nimpl<'a> Acquire<'a> {\n fn new(semaphore: &'a Semaphore, num_permits: usize) -> Self {\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n return Self {\n node: Waiter::new(num_permits),\n semaphore,\n num_permits,\n queued: false,\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n return semaphore.resource_span.in_scope(|| {\n let async_op_span =\n tracing::trace_span!(\"runtime.resource.async_op\", source = \"Acquire::new\");\n let async_op_poll_span = async_op_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n permits_requested = num_permits,\n permits.op = \"override\",\n );\n\n tracing::trace!(\n target: \"runtime::resource::async_op::state_update\",\n permits_obtained = 0usize,\n permits.op = \"override\",\n );\n\n tracing::trace_span!(\"runtime.resource.async_op.poll\")\n });\n\n let ctx = trace::AsyncOpTracingCtx {\n async_op_span,\n async_op_poll_span,\n resource_span: semaphore.resource_span.clone(),\n };\n\n Self {\n node: Waiter::new(num_permits, ctx),\n semaphore,\n num_permits,\n queued: false,\n }\n });\n }\n\n fn project(self: Pin<&mut Self>) -> (Pin<&mut Waiter>, &Semaphore, usize, &mut bool) {\n fn is_unpin() {}\n unsafe {\n // Safety: all fields other than `node` are `Unpin`\n\n is_unpin::<&Semaphore>();\n is_unpin::<&mut bool>();\n is_unpin::();\n\n let this = self.get_unchecked_mut();\n (\n Pin::new_unchecked(&mut this.node),\n this.semaphore,\n this.num_permits,\n &mut this.queued,\n )\n }\n }\n}\n\nimpl Drop for Acquire<'_> {\n fn drop(&mut self) {\n // If the future is completed, there is no node in the wait list, so we\n // can skip acquiring the lock.\n if !self.queued {\n return;\n }\n\n // This is where we ensure safety. The future is being dropped,\n // which means we must ensure that the waiter entry is no longer stored\n // in the linked list.\n let mut waiters = self.semaphore.waiters.lock();\n\n // remove the entry from the list\n let node = NonNull::from(&mut self.node);\n // Safety: we have locked the wait list.\n unsafe { waiters.queue.remove(node) };\n\n let acquired_permits = self.num_permits - self.node.state.load(Acquire);\n if acquired_permits > 0 {\n self.semaphore.add_permits_locked(acquired_permits, waiters);\n }\n }\n}\n\n// Safety: the `Acquire` future is not `Sync` automatically because it contains\n// a `Waiter`, which, in turn, contains an `UnsafeCell`. However, the\n// `UnsafeCell` is only accessed when the future is borrowed mutably (either in\n// `poll` or in `drop`). Therefore, it is safe (although not particularly\n// _useful_) for the future to be borrowed immutably across threads.\nunsafe impl Sync for Acquire<'_> {}\n\n// ===== impl AcquireError ====\n\nimpl AcquireError {\n fn closed() -> AcquireError {\n AcquireError(())\n }\n}\n\nimpl fmt::Display for AcquireError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"semaphore closed\")\n }\n}\n\nimpl std::error::Error for AcquireError {}\n\n// ===== impl TryAcquireError =====\n\nimpl TryAcquireError {\n /// Returns `true` if the error was caused by a closed semaphore.\n #[allow(dead_code)] // may be used later!\n pub(crate) fn is_closed(&self) -> bool {\n matches!(self, TryAcquireError::Closed)\n }\n\n /// Returns `true` if the error was caused by calling `try_acquire` on a\n /// semaphore with no available permits.\n #[allow(dead_code)] // may be used later!\n pub(crate) fn is_no_permits(&self) -> bool {\n matches!(self, TryAcquireError::NoPermits)\n }\n}\n\nimpl fmt::Display for TryAcquireError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n TryAcquireError::Closed => write!(fmt, \"semaphore closed\"),\n TryAcquireError::NoPermits => write!(fmt, \"no permits available\"),\n }\n }\n}\n\nimpl std::error::Error for TryAcquireError {}\n\n/// # Safety\n///\n/// `Waiter` is forced to be !Unpin.\nunsafe impl linked_list::Link for Waiter {\n type Handle = NonNull;\n type Target = Waiter;\n\n fn as_raw(handle: &Self::Handle) -> NonNull {\n *handle\n }\n\n unsafe fn from_raw(ptr: NonNull) -> NonNull {\n ptr\n }\n\n unsafe fn pointers(target: NonNull) -> NonNull> {\n unsafe { Waiter::addr_of_pointers(target) }\n }\n}\n" }, { "path": "tokio/src/sync/broadcast.rs", "content": "//! A multi-producer, multi-consumer broadcast queue. Each sent value is seen by\n//! all consumers.\n//!\n//! A [`Sender`] is used to broadcast values to **all** connected [`Receiver`]\n//! values. [`Sender`] handles are clone-able, allowing concurrent send and\n//! receive actions. [`Sender`] and [`Receiver`] are both `Send` and `Sync` as\n//! long as `T` is `Send`.\n//!\n//! When a value is sent, **all** [`Receiver`] handles are notified and will\n//! receive the value. The value is stored once inside the channel and cloned on\n//! demand for each receiver. Once all receivers have received a clone of the\n//! value, the value is released from the channel.\n//!\n//! A channel is created by calling [`channel`], specifying the maximum number\n//! of messages the channel can retain at any given time.\n//!\n//! New [`Receiver`] handles are created by calling [`Sender::subscribe`]. The\n//! returned [`Receiver`] will receive values sent **after** the call to\n//! `subscribe`.\n//!\n//! This channel is also suitable for the single-producer multi-consumer\n//! use-case, where a single sender broadcasts values to many receivers.\n//!\n//! ## Lagging\n//!\n//! As sent messages must be retained until **all** [`Receiver`] handles receive\n//! a clone, broadcast channels are susceptible to the \"slow receiver\" problem.\n//! In this case, all but one receiver are able to receive values at the rate\n//! they are sent. Because one receiver is stalled, the channel starts to fill\n//! up.\n//!\n//! This broadcast channel implementation handles this case by setting a hard\n//! upper bound on the number of values the channel may retain at any given\n//! time. This upper bound is passed to the [`channel`] function as an argument.\n//!\n//! If a value is sent when the channel is at capacity, the oldest value\n//! currently held by the channel is released. This frees up space for the new\n//! value. Any receiver that has not yet seen the released value will return\n//! [`RecvError::Lagged`] the next time [`recv`] is called.\n//!\n//! Once [`RecvError::Lagged`] is returned, the lagging receiver's position is\n//! updated to the oldest value contained by the channel. The next call to\n//! [`recv`] will return this value.\n//!\n//! This behavior enables a receiver to detect when it has lagged so far behind\n//! that data has been dropped. The caller may decide how to respond to this:\n//! either by aborting its task or by tolerating lost messages and resuming\n//! consumption of the channel.\n//!\n//! ## Closing\n//!\n//! When **all** [`Sender`] handles have been dropped, no new values may be\n//! sent. At this point, the channel is \"closed\". Once a receiver has received\n//! all values retained by the channel, the next call to [`recv`] will return\n//! with [`RecvError::Closed`].\n//!\n//! When a [`Receiver`] handle is dropped, any messages not read by the receiver\n//! will be marked as read. If this receiver was the only one not to have read\n//! that message, the message will be dropped at this point.\n//!\n//! [`Sender`]: crate::sync::broadcast::Sender\n//! [`Sender::subscribe`]: crate::sync::broadcast::Sender::subscribe\n//! [`Receiver`]: crate::sync::broadcast::Receiver\n//! [`channel`]: crate::sync::broadcast::channel\n//! [`RecvError::Lagged`]: crate::sync::broadcast::error::RecvError::Lagged\n//! [`RecvError::Closed`]: crate::sync::broadcast::error::RecvError::Closed\n//! [`recv`]: crate::sync::broadcast::Receiver::recv\n//!\n//! # Examples\n//!\n//! Basic usage\n//!\n//! ```\n//! use tokio::sync::broadcast;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, mut rx1) = broadcast::channel(16);\n//! let mut rx2 = tx.subscribe();\n//!\n//! tokio::spawn(async move {\n//! assert_eq!(rx1.recv().await.unwrap(), 10);\n//! assert_eq!(rx1.recv().await.unwrap(), 20);\n//! });\n//!\n//! tokio::spawn(async move {\n//! assert_eq!(rx2.recv().await.unwrap(), 10);\n//! assert_eq!(rx2.recv().await.unwrap(), 20);\n//! });\n//!\n//! tx.send(10).unwrap();\n//! tx.send(20).unwrap();\n//! # }\n//! ```\n//!\n//! Handling lag\n//!\n//! ```\n//! use tokio::sync::broadcast;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, mut rx) = broadcast::channel(2);\n//!\n//! tx.send(10).unwrap();\n//! tx.send(20).unwrap();\n//! tx.send(30).unwrap();\n//!\n//! // The receiver lagged behind\n//! assert!(rx.recv().await.is_err());\n//!\n//! // At this point, we can abort or continue with lost messages\n//!\n//! assert_eq!(20, rx.recv().await.unwrap());\n//! assert_eq!(30, rx.recv().await.unwrap());\n//! # }\n//! ```\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::{AtomicBool, AtomicUsize};\nuse crate::loom::sync::{Arc, Mutex, MutexGuard};\nuse crate::task::coop::cooperative;\nuse crate::util::linked_list::{self, GuardedLinkedList, LinkedList};\nuse crate::util::WakeList;\n\nuse std::fmt;\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release, SeqCst};\nuse std::task::{ready, Context, Poll, Waker};\n\n/// Sending-half of the [`broadcast`] channel.\n///\n/// May be used from many threads. Messages can be sent with\n/// [`send`][Sender::send].\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::broadcast;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, mut rx1) = broadcast::channel(16);\n/// let mut rx2 = tx.subscribe();\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx1.recv().await.unwrap(), 10);\n/// assert_eq!(rx1.recv().await.unwrap(), 20);\n/// });\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx2.recv().await.unwrap(), 10);\n/// assert_eq!(rx2.recv().await.unwrap(), 20);\n/// });\n///\n/// tx.send(10).unwrap();\n/// tx.send(20).unwrap();\n/// # }\n/// ```\n///\n/// [`broadcast`]: crate::sync::broadcast\npub struct Sender {\n shared: Arc>,\n}\n\n/// A sender that does not prevent the channel from being closed.\n///\n/// If all [`Sender`] instances of a channel were dropped and only `WeakSender`\n/// instances remain, the channel is closed.\n///\n/// In order to send messages, the `WeakSender` needs to be upgraded using\n/// [`WeakSender::upgrade`], which returns `Option`. It returns `None`\n/// if all `Sender`s have been dropped, and otherwise it returns a `Sender`.\n///\n/// [`Sender`]: Sender\n/// [`WeakSender::upgrade`]: WeakSender::upgrade\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::broadcast::channel;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, _rx) = channel::(15);\n/// let tx_weak = tx.downgrade();\n///\n/// // Upgrading will succeed because `tx` still exists.\n/// assert!(tx_weak.upgrade().is_some());\n///\n/// // If we drop `tx`, then it will fail.\n/// drop(tx);\n/// assert!(tx_weak.clone().upgrade().is_none());\n/// # }\n/// ```\npub struct WeakSender {\n shared: Arc>,\n}\n\n/// Receiving-half of the [`broadcast`] channel.\n///\n/// Must not be used concurrently. Messages may be retrieved using\n/// [`recv`][Receiver::recv].\n///\n/// To turn this receiver into a `Stream`, you can use the [`BroadcastStream`]\n/// wrapper.\n///\n/// [`BroadcastStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.BroadcastStream.html\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::broadcast;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, mut rx1) = broadcast::channel(16);\n/// let mut rx2 = tx.subscribe();\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx1.recv().await.unwrap(), 10);\n/// assert_eq!(rx1.recv().await.unwrap(), 20);\n/// });\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx2.recv().await.unwrap(), 10);\n/// assert_eq!(rx2.recv().await.unwrap(), 20);\n/// });\n///\n/// tx.send(10).unwrap();\n/// tx.send(20).unwrap();\n/// # }\n/// ```\n///\n/// [`broadcast`]: crate::sync::broadcast\npub struct Receiver {\n /// State shared with all receivers and senders.\n shared: Arc>,\n\n /// Next position to read from\n next: u64,\n}\n\npub mod error {\n //! Broadcast error types\n\n use std::fmt;\n\n /// Error returned by the [`send`] function on a [`Sender`].\n ///\n /// A **send** operation can only fail if there are no active receivers,\n /// implying that the message could never be received. The error contains the\n /// message being sent as a payload so it can be recovered.\n ///\n /// [`send`]: crate::sync::broadcast::Sender::send\n /// [`Sender`]: crate::sync::broadcast::Sender\n #[derive(Debug)]\n pub struct SendError(pub T);\n\n impl fmt::Display for SendError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(f, \"channel closed\")\n }\n }\n\n impl std::error::Error for SendError {}\n\n /// An error returned from the [`recv`] function on a [`Receiver`].\n ///\n /// [`recv`]: crate::sync::broadcast::Receiver::recv\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n #[derive(Debug, PartialEq, Eq, Clone)]\n pub enum RecvError {\n /// There are no more active senders implying no further messages will ever\n /// be sent.\n Closed,\n\n /// The receiver lagged too far behind. Attempting to receive again will\n /// return the oldest message still retained by the channel.\n ///\n /// Includes the number of skipped messages.\n Lagged(u64),\n }\n\n impl fmt::Display for RecvError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n RecvError::Closed => write!(f, \"channel closed\"),\n RecvError::Lagged(amt) => write!(f, \"channel lagged by {amt}\"),\n }\n }\n }\n\n impl std::error::Error for RecvError {}\n\n /// An error returned from the [`try_recv`] function on a [`Receiver`].\n ///\n /// [`try_recv`]: crate::sync::broadcast::Receiver::try_recv\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n #[derive(Debug, PartialEq, Eq, Clone)]\n pub enum TryRecvError {\n /// The channel is currently empty. There are still active\n /// [`Sender`] handles, so data may yet become available.\n ///\n /// [`Sender`]: crate::sync::broadcast::Sender\n Empty,\n\n /// There are no more active senders implying no further messages will ever\n /// be sent.\n Closed,\n\n /// The receiver lagged too far behind and has been forcibly disconnected.\n /// Attempting to receive again will return the oldest message still\n /// retained by the channel.\n ///\n /// Includes the number of skipped messages.\n Lagged(u64),\n }\n\n impl fmt::Display for TryRecvError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n TryRecvError::Empty => write!(f, \"channel empty\"),\n TryRecvError::Closed => write!(f, \"channel closed\"),\n TryRecvError::Lagged(amt) => write!(f, \"channel lagged by {amt}\"),\n }\n }\n }\n\n impl std::error::Error for TryRecvError {}\n}\n\nuse self::error::{RecvError, SendError, TryRecvError};\n\nuse super::Notify;\n\n/// Data shared between senders and receivers.\nstruct Shared {\n /// slots in the channel.\n buffer: Box<[Mutex>]>,\n\n /// Mask a position -> index.\n mask: usize,\n\n /// Tail of the queue. Includes the rx wait list.\n tail: Mutex,\n\n /// Number of outstanding Sender handles.\n num_tx: AtomicUsize,\n\n /// Number of outstanding weak Sender handles.\n num_weak_tx: AtomicUsize,\n\n /// Notify when the last subscribed [`Receiver`] drops.\n notify_last_rx_drop: Notify,\n}\n\n/// Next position to write a value.\nstruct Tail {\n /// Next position to write to.\n pos: u64,\n\n /// Number of active receivers.\n rx_cnt: usize,\n\n /// True if the channel is closed.\n closed: bool,\n\n /// Receivers waiting for a value.\n waiters: LinkedList::Target>,\n}\n\n/// Slot in the buffer.\nstruct Slot {\n /// Remaining number of receivers that are expected to see this value.\n ///\n /// When this goes to zero, the value is released.\n ///\n /// An atomic is used as it is mutated concurrently with the slot read lock\n /// acquired.\n rem: AtomicUsize,\n\n /// Uniquely identifies the `send` stored in the slot.\n pos: u64,\n\n /// The value being broadcast.\n ///\n /// The value is set by `send` when the write lock is held. When a reader\n /// drops, `rem` is decremented. When it hits zero, the value is dropped.\n val: Option,\n}\n\n/// An entry in the wait queue.\nstruct Waiter {\n /// True if queued.\n queued: AtomicBool,\n\n /// Task waiting on the broadcast channel.\n waker: Option,\n\n /// Intrusive linked-list pointers.\n pointers: linked_list::Pointers,\n\n /// Should not be `Unpin`.\n _p: PhantomPinned,\n}\n\nimpl Waiter {\n fn new() -> Self {\n Self {\n queued: AtomicBool::new(false),\n waker: None,\n pointers: linked_list::Pointers::new(),\n _p: PhantomPinned,\n }\n }\n}\n\ngenerate_addr_of_methods! {\n impl<> Waiter {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull> {\n &self.pointers\n }\n }\n}\n\nstruct RecvGuard<'a, T> {\n slot: MutexGuard<'a, Slot>,\n}\n\n/// Receive a value future.\nstruct Recv<'a, T> {\n /// Receiver being waited on.\n receiver: &'a mut Receiver,\n\n /// Entry in the waiter `LinkedList`.\n waiter: WaiterCell,\n}\n\n// The wrapper around `UnsafeCell` isolates the unsafe impl `Send` and `Sync`\n// from `Recv`.\nstruct WaiterCell(UnsafeCell);\n\nunsafe impl Send for WaiterCell {}\nunsafe impl Sync for WaiterCell {}\n\n/// Max number of receivers. Reserve space to lock.\nconst MAX_RECEIVERS: usize = usize::MAX >> 2;\n\n/// Create a bounded, multi-producer, multi-consumer channel where each sent\n/// value is broadcasted to all active receivers.\n///\n/// **Note:** The actual capacity may be greater than the provided `capacity`.\n///\n/// All data sent on [`Sender`] will become available on every active\n/// [`Receiver`] in the same order as it was sent.\n///\n/// The `Sender` can be cloned to `send` to the same channel from multiple\n/// points in the process or it can be used concurrently from an `Arc`. New\n/// `Receiver` handles are created by calling [`Sender::subscribe`].\n///\n/// If all [`Receiver`] handles are dropped, the `send` method will return a\n/// [`SendError`]. Similarly, if all [`Sender`] handles are dropped, the [`recv`]\n/// method will return a [`RecvError`].\n///\n/// [`Sender`]: crate::sync::broadcast::Sender\n/// [`Sender::subscribe`]: crate::sync::broadcast::Sender::subscribe\n/// [`Receiver`]: crate::sync::broadcast::Receiver\n/// [`recv`]: crate::sync::broadcast::Receiver::recv\n/// [`SendError`]: crate::sync::broadcast::error::SendError\n/// [`RecvError`]: crate::sync::broadcast::error::RecvError\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::broadcast;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, mut rx1) = broadcast::channel(16);\n/// let mut rx2 = tx.subscribe();\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx1.recv().await.unwrap(), 10);\n/// assert_eq!(rx1.recv().await.unwrap(), 20);\n/// });\n///\n/// tokio::spawn(async move {\n/// assert_eq!(rx2.recv().await.unwrap(), 10);\n/// assert_eq!(rx2.recv().await.unwrap(), 20);\n/// });\n///\n/// tx.send(10).unwrap();\n/// tx.send(20).unwrap();\n/// # }\n/// ```\n///\n/// # Panics\n///\n/// This will panic if `capacity` is equal to `0`.\n///\n/// This pre-allocates space for `capacity` messages. Allocation failure may result in a panic or\n/// [an allocation error](std::alloc::handle_alloc_error).\n#[track_caller]\npub fn channel(capacity: usize) -> (Sender, Receiver) {\n // SAFETY: In the line below we are creating one extra receiver, so there will be 1 in total.\n let tx = unsafe { Sender::new_with_receiver_count(1, capacity) };\n let rx = Receiver {\n shared: tx.shared.clone(),\n next: 0,\n };\n (tx, rx)\n}\n\nimpl Sender {\n /// Creates the sending-half of the [`broadcast`] channel.\n ///\n /// See the documentation of [`broadcast::channel`] for more information on this method.\n ///\n /// [`broadcast`]: crate::sync::broadcast\n /// [`broadcast::channel`]: crate::sync::broadcast::channel\n #[track_caller]\n pub fn new(capacity: usize) -> Self {\n // SAFETY: We don't create extra receivers, so there are 0.\n unsafe { Self::new_with_receiver_count(0, capacity) }\n }\n\n /// Creates the sending-half of the [`broadcast`](self) channel, and provide the receiver\n /// count.\n ///\n /// See the documentation of [`broadcast::channel`](self::channel) for more errors when\n /// calling this function.\n ///\n /// # Safety:\n ///\n /// The caller must ensure that the amount of receivers for this Sender is correct before\n /// the channel functionalities are used, the count is zero by default, as this function\n /// does not create any receivers by itself.\n #[track_caller]\n unsafe fn new_with_receiver_count(receiver_count: usize, mut capacity: usize) -> Self {\n assert!(capacity > 0, \"broadcast channel capacity cannot be zero\");\n assert!(\n capacity <= usize::MAX >> 1,\n \"broadcast channel capacity exceeded `usize::MAX / 2`\"\n );\n\n // Round to a power of two\n capacity = capacity.next_power_of_two();\n\n let mut buffer = Vec::with_capacity(capacity);\n\n for i in 0..capacity {\n buffer.push(Mutex::new(Slot {\n rem: AtomicUsize::new(0),\n pos: (i as u64).wrapping_sub(capacity as u64),\n val: None,\n }));\n }\n\n let shared = Arc::new(Shared {\n buffer: buffer.into_boxed_slice(),\n mask: capacity - 1,\n tail: Mutex::new(Tail {\n pos: 0,\n rx_cnt: receiver_count,\n closed: receiver_count == 0,\n waiters: LinkedList::new(),\n }),\n num_tx: AtomicUsize::new(1),\n num_weak_tx: AtomicUsize::new(0),\n notify_last_rx_drop: Notify::new(),\n });\n\n Sender { shared }\n }\n\n /// Attempts to send a value to all active [`Receiver`] handles, returning\n /// it back if it could not be sent.\n ///\n /// A successful send occurs when there is at least one active [`Receiver`]\n /// handle. An unsuccessful send would be one where all associated\n /// [`Receiver`] handles have already been dropped.\n ///\n /// # Return\n ///\n /// On success, the number of subscribed [`Receiver`] handles is returned.\n /// This does not mean that this number of receivers will see the message as\n /// a receiver may drop or lag ([see lagging](self#lagging)) before receiving\n /// the message.\n ///\n /// # Note\n ///\n /// A return value of `Ok` **does not** mean that the sent value will be\n /// observed by all or any of the active [`Receiver`] handles. [`Receiver`]\n /// handles may be dropped before receiving the sent message.\n ///\n /// A return value of `Err` **does not** mean that future calls to `send`\n /// will fail. New [`Receiver`] handles may be created by calling\n /// [`subscribe`].\n ///\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n /// [`subscribe`]: crate::sync::broadcast::Sender::subscribe\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n /// let mut rx2 = tx.subscribe();\n ///\n /// tokio::spawn(async move {\n /// assert_eq!(rx1.recv().await.unwrap(), 10);\n /// assert_eq!(rx1.recv().await.unwrap(), 20);\n /// });\n ///\n /// tokio::spawn(async move {\n /// assert_eq!(rx2.recv().await.unwrap(), 10);\n /// assert_eq!(rx2.recv().await.unwrap(), 20);\n /// });\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n /// # }\n /// ```\n pub fn send(&self, value: T) -> Result> {\n let mut tail = self.shared.tail.lock();\n\n if tail.rx_cnt == 0 {\n return Err(SendError(value));\n }\n\n // Position to write into\n let pos = tail.pos;\n let rem = tail.rx_cnt;\n let idx = (pos & self.shared.mask as u64) as usize;\n\n // Update the tail position\n tail.pos = tail.pos.wrapping_add(1);\n\n // Get the slot\n let mut slot = self.shared.buffer[idx].lock();\n\n // Track the position\n slot.pos = pos;\n\n // Set remaining receivers\n slot.rem.with_mut(|v| *v = rem);\n\n // Write the value\n slot.val = Some(value);\n\n // Release the slot lock before notifying the receivers.\n drop(slot);\n\n // Notify and release the mutex. This must happen after the slot lock is\n // released, otherwise the writer lock bit could be cleared while another\n // thread is in the critical section.\n self.shared.notify_rx(tail);\n\n Ok(rem)\n }\n\n /// Creates a new [`Receiver`] handle that will receive values sent **after**\n /// this call to `subscribe`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx) = broadcast::channel(16);\n ///\n /// // Will not be seen\n /// tx.send(10).unwrap();\n ///\n /// let mut rx = tx.subscribe();\n ///\n /// tx.send(20).unwrap();\n ///\n /// let value = rx.recv().await.unwrap();\n /// assert_eq!(20, value);\n /// # }\n /// ```\n pub fn subscribe(&self) -> Receiver {\n let shared = self.shared.clone();\n new_receiver(shared)\n }\n\n /// Converts the `Sender` to a [`WeakSender`] that does not count\n /// towards RAII semantics, i.e. if all `Sender` instances of the\n /// channel were dropped and only `WeakSender` instances remain,\n /// the channel is closed.\n #[must_use = \"Downgrade creates a WeakSender without destroying the original non-weak sender.\"]\n pub fn downgrade(&self) -> WeakSender {\n self.shared.num_weak_tx.fetch_add(1, Relaxed);\n WeakSender {\n shared: self.shared.clone(),\n }\n }\n\n /// Returns the number of queued values.\n ///\n /// A value is queued until it has either been seen by all receivers that were alive at the time\n /// it was sent, or has been evicted from the queue by subsequent sends that exceeded the\n /// queue's capacity.\n ///\n /// # Note\n ///\n /// In contrast to [`Receiver::len`], this method only reports queued values and not values that\n /// have been evicted from the queue before being seen by all receivers.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n /// let mut rx2 = tx.subscribe();\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n /// tx.send(30).unwrap();\n ///\n /// assert_eq!(tx.len(), 3);\n ///\n /// rx1.recv().await.unwrap();\n ///\n /// // The len is still 3 since rx2 hasn't seen the first value yet.\n /// assert_eq!(tx.len(), 3);\n ///\n /// rx2.recv().await.unwrap();\n ///\n /// assert_eq!(tx.len(), 2);\n /// # }\n /// ```\n pub fn len(&self) -> usize {\n let tail = self.shared.tail.lock();\n\n let base_idx = (tail.pos & self.shared.mask as u64) as usize;\n let mut low = 0;\n let mut high = self.shared.buffer.len();\n while low < high {\n let mid = low + (high - low) / 2;\n let idx = base_idx.wrapping_add(mid) & self.shared.mask;\n if self.shared.buffer[idx].lock().rem.load(SeqCst) == 0 {\n low = mid + 1;\n } else {\n high = mid;\n }\n }\n\n self.shared.buffer.len() - low\n }\n\n /// Returns true if there are no queued values.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n /// let mut rx2 = tx.subscribe();\n ///\n /// assert!(tx.is_empty());\n ///\n /// tx.send(10).unwrap();\n ///\n /// assert!(!tx.is_empty());\n ///\n /// rx1.recv().await.unwrap();\n ///\n /// // The queue is still not empty since rx2 hasn't seen the value.\n /// assert!(!tx.is_empty());\n ///\n /// rx2.recv().await.unwrap();\n ///\n /// assert!(tx.is_empty());\n /// # }\n /// ```\n pub fn is_empty(&self) -> bool {\n let tail = self.shared.tail.lock();\n\n let idx = (tail.pos.wrapping_sub(1) & self.shared.mask as u64) as usize;\n self.shared.buffer[idx].lock().rem.load(SeqCst) == 0\n }\n\n /// Returns the number of active receivers.\n ///\n /// An active receiver is a [`Receiver`] handle returned from [`channel`] or\n /// [`subscribe`]. These are the handles that will receive values sent on\n /// this [`Sender`].\n ///\n /// # Note\n ///\n /// It is not guaranteed that a sent message will reach this number of\n /// receivers. Active receivers may never call [`recv`] again before\n /// dropping.\n ///\n /// [`recv`]: crate::sync::broadcast::Receiver::recv\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n /// [`Sender`]: crate::sync::broadcast::Sender\n /// [`subscribe`]: crate::sync::broadcast::Sender::subscribe\n /// [`channel`]: crate::sync::broadcast::channel\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx1) = broadcast::channel(16);\n ///\n /// assert_eq!(1, tx.receiver_count());\n ///\n /// let mut _rx2 = tx.subscribe();\n ///\n /// assert_eq!(2, tx.receiver_count());\n ///\n /// tx.send(10).unwrap();\n /// # }\n /// ```\n pub fn receiver_count(&self) -> usize {\n let tail = self.shared.tail.lock();\n tail.rx_cnt\n }\n\n /// Returns `true` if senders belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx) = broadcast::channel::<()>(16);\n /// let tx2 = tx.clone();\n ///\n /// assert!(tx.same_channel(&tx2));\n ///\n /// let (tx3, _rx3) = broadcast::channel::<()>(16);\n ///\n /// assert!(!tx3.same_channel(&tx2));\n /// # }\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n Arc::ptr_eq(&self.shared, &other.shared)\n }\n\n /// A future which completes when the number of [Receiver]s subscribed to this `Sender` reaches\n /// zero.\n ///\n /// # Examples\n ///\n /// ```\n /// use futures::FutureExt;\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel::(16);\n /// let mut rx2 = tx.subscribe();\n ///\n /// let _ = tx.send(10);\n ///\n /// assert_eq!(rx1.recv().await.unwrap(), 10);\n /// drop(rx1);\n /// assert!(tx.closed().now_or_never().is_none());\n ///\n /// assert_eq!(rx2.recv().await.unwrap(), 10);\n /// drop(rx2);\n /// assert!(tx.closed().now_or_never().is_some());\n /// # }\n /// ```\n pub async fn closed(&self) {\n loop {\n let notified = self.shared.notify_last_rx_drop.notified();\n\n {\n // Ensure the lock drops if the channel isn't closed\n let tail = self.shared.tail.lock();\n if tail.closed {\n return;\n }\n }\n\n notified.await;\n }\n }\n\n fn close_channel(&self) {\n let mut tail = self.shared.tail.lock();\n tail.closed = true;\n\n self.shared.notify_rx(tail);\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn strong_count(&self) -> usize {\n self.shared.num_tx.load(Acquire)\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.shared.num_weak_tx.load(Acquire)\n }\n}\n\n/// Create a new `Receiver` which reads starting from the tail.\nfn new_receiver(shared: Arc>) -> Receiver {\n let mut tail = shared.tail.lock();\n\n assert!(tail.rx_cnt != MAX_RECEIVERS, \"max receivers\");\n\n if tail.rx_cnt == 0 {\n // Potentially need to re-open the channel, if a new receiver has been added between calls\n // to poll(). Note that we use rx_cnt == 0 instead of is_closed since is_closed also\n // applies if the sender has been dropped\n tail.closed = false;\n }\n\n tail.rx_cnt = tail.rx_cnt.checked_add(1).expect(\"overflow\");\n let next = tail.pos;\n\n drop(tail);\n\n Receiver { shared, next }\n}\n\n/// List used in `Shared::notify_rx`. It wraps a guarded linked list\n/// and gates the access to it on the `Shared.tail` mutex. It also empties\n/// the list on drop.\nstruct WaitersList<'a, T> {\n list: GuardedLinkedList::Target>,\n is_empty: bool,\n shared: &'a Shared,\n}\n\nimpl<'a, T> Drop for WaitersList<'a, T> {\n fn drop(&mut self) {\n // If the list is not empty, we unlink all waiters from it.\n // We do not wake the waiters to avoid double panics.\n if !self.is_empty {\n let _lock_guard = self.shared.tail.lock();\n while self.list.pop_back().is_some() {}\n }\n }\n}\n\nimpl<'a, T> WaitersList<'a, T> {\n fn new(\n unguarded_list: LinkedList::Target>,\n guard: Pin<&'a Waiter>,\n shared: &'a Shared,\n ) -> Self {\n let guard_ptr = NonNull::from(guard.get_ref());\n let list = unguarded_list.into_guarded(guard_ptr);\n WaitersList {\n list,\n is_empty: false,\n shared,\n }\n }\n\n /// Removes the last element from the guarded list. Modifying this list\n /// requires an exclusive access to the main list in `Notify`.\n fn pop_back_locked(&mut self, _tail: &mut Tail) -> Option> {\n let result = self.list.pop_back();\n if result.is_none() {\n // Save information about emptiness to avoid waiting for lock\n // in the destructor.\n self.is_empty = true;\n }\n result\n }\n}\n\nimpl Shared {\n fn notify_rx<'a, 'b: 'a>(&'b self, mut tail: MutexGuard<'a, Tail>) {\n // It is critical for `GuardedLinkedList` safety that the guard node is\n // pinned in memory and is not dropped until the guarded list is dropped.\n let guard = Waiter::new();\n pin!(guard);\n\n // We move all waiters to a secondary list. It uses a `GuardedLinkedList`\n // underneath to allow every waiter to safely remove itself from it.\n //\n // * This list will be still guarded by the `waiters` lock.\n // `NotifyWaitersList` wrapper makes sure we hold the lock to modify it.\n // * This wrapper will empty the list on drop. It is critical for safety\n // that we will not leave any list entry with a pointer to the local\n // guard node after this function returns / panics.\n let mut list = WaitersList::new(std::mem::take(&mut tail.waiters), guard.as_ref(), self);\n\n let mut wakers = WakeList::new();\n 'outer: loop {\n while wakers.can_push() {\n match list.pop_back_locked(&mut tail) {\n Some(waiter) => {\n unsafe {\n // Safety: accessing `waker` is safe because\n // the tail lock is held.\n if let Some(waker) = (*waiter.as_ptr()).waker.take() {\n wakers.push(waker);\n }\n\n // Safety: `queued` is atomic.\n let queued = &(*waiter.as_ptr()).queued;\n // `Relaxed` suffices because the tail lock is held.\n assert!(queued.load(Relaxed));\n // `Release` is needed to synchronize with `Recv::drop`.\n // It is critical to set this variable **after** waker\n // is extracted, otherwise we may data race with `Recv::drop`.\n queued.store(false, Release);\n }\n }\n None => {\n break 'outer;\n }\n }\n }\n\n // Release the lock before waking.\n drop(tail);\n\n // Before we acquire the lock again all sorts of things can happen:\n // some waiters may remove themselves from the list and new waiters\n // may be added. This is fine since at worst we will unnecessarily\n // wake up waiters which will then queue themselves again.\n\n wakers.wake_all();\n\n // Acquire the lock again.\n tail = self.tail.lock();\n }\n\n // Release the lock before waking.\n drop(tail);\n\n wakers.wake_all();\n }\n}\n\nimpl Clone for Sender {\n fn clone(&self) -> Sender {\n let shared = self.shared.clone();\n shared.num_tx.fetch_add(1, Relaxed);\n\n Sender { shared }\n }\n}\n\nimpl Drop for Sender {\n fn drop(&mut self) {\n if 1 == self.shared.num_tx.fetch_sub(1, AcqRel) {\n self.close_channel();\n }\n }\n}\n\nimpl WeakSender {\n /// Tries to convert a `WeakSender` into a [`Sender`].\n ///\n /// This will return `Some` if there are other `Sender` instances alive and\n /// the channel wasn't previously dropped, otherwise `None` is returned.\n #[must_use]\n pub fn upgrade(&self) -> Option> {\n let mut tx_count = self.shared.num_tx.load(Acquire);\n\n loop {\n if tx_count == 0 {\n // channel is closed so this WeakSender can not be upgraded\n return None;\n }\n\n match self\n .shared\n .num_tx\n .compare_exchange_weak(tx_count, tx_count + 1, Relaxed, Acquire)\n {\n Ok(_) => {\n return Some(Sender {\n shared: self.shared.clone(),\n })\n }\n Err(prev_count) => tx_count = prev_count,\n }\n }\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn strong_count(&self) -> usize {\n self.shared.num_tx.load(Acquire)\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.shared.num_weak_tx.load(Acquire)\n }\n}\n\nimpl Clone for WeakSender {\n fn clone(&self) -> WeakSender {\n let shared = self.shared.clone();\n shared.num_weak_tx.fetch_add(1, Relaxed);\n\n Self { shared }\n }\n}\n\nimpl Drop for WeakSender {\n fn drop(&mut self) {\n self.shared.num_weak_tx.fetch_sub(1, AcqRel);\n }\n}\n\nimpl Receiver {\n /// Returns the number of messages that were sent into the channel and that\n /// this [`Receiver`] has yet to receive.\n ///\n /// If the returned value from `len` is larger than the next largest power of 2\n /// of the capacity of the channel any call to [`recv`] will return an\n /// `Err(RecvError::Lagged)` and any call to [`try_recv`] will return an\n /// `Err(TryRecvError::Lagged)`, e.g. if the capacity of the channel is 10,\n /// [`recv`] will start to return `Err(RecvError::Lagged)` once `len` returns\n /// values larger than 16.\n ///\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n /// [`recv`]: crate::sync::broadcast::Receiver::recv\n /// [`try_recv`]: crate::sync::broadcast::Receiver::try_recv\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n ///\n /// assert_eq!(rx1.len(), 2);\n /// assert_eq!(rx1.recv().await.unwrap(), 10);\n /// assert_eq!(rx1.len(), 1);\n /// assert_eq!(rx1.recv().await.unwrap(), 20);\n /// assert_eq!(rx1.len(), 0);\n /// # }\n /// ```\n pub fn len(&self) -> usize {\n let next_send_pos = self.shared.tail.lock().pos;\n (next_send_pos - self.next) as usize\n }\n\n /// Returns true if there aren't any messages in the channel that the [`Receiver`]\n /// has yet to receive.\n ///\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n ///\n /// assert!(rx1.is_empty());\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n ///\n /// assert!(!rx1.is_empty());\n /// assert_eq!(rx1.recv().await.unwrap(), 10);\n /// assert_eq!(rx1.recv().await.unwrap(), 20);\n /// assert!(rx1.is_empty());\n /// # }\n /// ```\n pub fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n /// Returns `true` if receivers belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = broadcast::channel::<()>(16);\n /// let rx2 = tx.subscribe();\n ///\n /// assert!(rx.same_channel(&rx2));\n ///\n /// let (_tx3, rx3) = broadcast::channel::<()>(16);\n ///\n /// assert!(!rx3.same_channel(&rx2));\n /// # }\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n Arc::ptr_eq(&self.shared, &other.shared)\n }\n\n /// Locks the next value if there is one.\n fn recv_ref(\n &mut self,\n waiter: Option<(&UnsafeCell, &Waker)>,\n ) -> Result, TryRecvError> {\n let idx = (self.next & self.shared.mask as u64) as usize;\n\n // The slot holding the next value to read\n let mut slot = self.shared.buffer[idx].lock();\n\n if slot.pos != self.next {\n // Release the `slot` lock before attempting to acquire the `tail`\n // lock. This is required because `send2` acquires the tail lock\n // first followed by the slot lock. Acquiring the locks in reverse\n // order here would result in a potential deadlock: `recv_ref`\n // acquires the `slot` lock and attempts to acquire the `tail` lock\n // while `send2` acquired the `tail` lock and attempts to acquire\n // the slot lock.\n drop(slot);\n\n let mut old_waker = None;\n\n let mut tail = self.shared.tail.lock();\n\n // Acquire slot lock again\n slot = self.shared.buffer[idx].lock();\n\n // Make sure the position did not change. This could happen in the\n // unlikely event that the buffer is wrapped between dropping the\n // read lock and acquiring the tail lock.\n if slot.pos != self.next {\n let next_pos = slot.pos.wrapping_add(self.shared.buffer.len() as u64);\n\n if next_pos == self.next {\n // At this point the channel is empty for *this* receiver. If\n // it's been closed, then that's what we return, otherwise we\n // set a waker and return empty.\n if tail.closed {\n return Err(TryRecvError::Closed);\n }\n\n // Store the waker\n if let Some((waiter, waker)) = waiter {\n // Safety: called while locked.\n unsafe {\n // Only queue if not already queued\n waiter.with_mut(|ptr| {\n // If there is no waker **or** if the currently\n // stored waker references a **different** task,\n // track the tasks' waker to be notified on\n // receipt of a new value.\n match (*ptr).waker {\n Some(ref w) if w.will_wake(waker) => {}\n _ => {\n old_waker = (*ptr).waker.replace(waker.clone());\n }\n }\n\n // If the waiter is not already queued, enqueue it.\n // `Relaxed` order suffices: we have synchronized with\n // all writers through the tail lock that we hold.\n if !(*ptr).queued.load(Relaxed) {\n // `Relaxed` order suffices: all the readers will\n // synchronize with this write through the tail lock.\n (*ptr).queued.store(true, Relaxed);\n tail.waiters.push_front(NonNull::new_unchecked(&mut *ptr));\n }\n });\n }\n }\n\n // Drop the old waker after releasing the locks.\n drop(slot);\n drop(tail);\n drop(old_waker);\n\n return Err(TryRecvError::Empty);\n }\n\n // At this point, the receiver has lagged behind the sender by\n // more than the channel capacity. The receiver will attempt to\n // catch up by skipping dropped messages and setting the\n // internal cursor to the **oldest** message stored by the\n // channel.\n let next = tail.pos.wrapping_sub(self.shared.buffer.len() as u64);\n\n let missed = next.wrapping_sub(self.next);\n\n drop(tail);\n\n // The receiver is slow but no values have been missed\n if missed == 0 {\n self.next = self.next.wrapping_add(1);\n\n return Ok(RecvGuard { slot });\n }\n\n self.next = next;\n\n return Err(TryRecvError::Lagged(missed));\n }\n }\n\n self.next = self.next.wrapping_add(1);\n\n Ok(RecvGuard { slot })\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn sender_strong_count(&self) -> usize {\n self.shared.num_tx.load(Acquire)\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn sender_weak_count(&self) -> usize {\n self.shared.num_weak_tx.load(Acquire)\n }\n\n /// Checks if a channel is closed.\n ///\n /// This method returns `true` if the channel has been closed. The channel is closed\n /// when all [`Sender`] have been dropped.\n ///\n /// [`Sender`]: crate::sync::broadcast::Sender\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = broadcast::channel::<()>(10);\n /// assert!(!rx.is_closed());\n ///\n /// drop(tx);\n ///\n /// assert!(rx.is_closed());\n /// # }\n /// ```\n pub fn is_closed(&self) -> bool {\n // Channel is closed when there are no strong senders left active\n self.shared.num_tx.load(Acquire) == 0\n }\n}\n\nimpl Receiver {\n /// Re-subscribes to the channel starting from the current tail element.\n ///\n /// This [`Receiver`] handle will receive a clone of all values sent\n /// **after** it has resubscribed. This will not include elements that are\n /// in the queue of the current receiver. Consider the following example.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = broadcast::channel(2);\n ///\n /// tx.send(1).unwrap();\n /// let mut rx2 = rx.resubscribe();\n /// tx.send(2).unwrap();\n ///\n /// assert_eq!(rx2.recv().await.unwrap(), 2);\n /// assert_eq!(rx.recv().await.unwrap(), 1);\n /// # }\n /// ```\n pub fn resubscribe(&self) -> Self {\n let shared = self.shared.clone();\n new_receiver(shared)\n }\n /// Receives the next value for this receiver.\n ///\n /// Each [`Receiver`] handle will receive a clone of all values sent\n /// **after** it has subscribed.\n ///\n /// `Err(RecvError::Closed)` is returned when all `Sender` halves have\n /// dropped, indicating that no further values can be sent on the channel.\n ///\n /// If the [`Receiver`] handle falls behind, once the channel is full, newly\n /// sent values will overwrite old values. At this point, a call to [`recv`]\n /// will return with `Err(RecvError::Lagged)` and the [`Receiver`]'s\n /// internal cursor is updated to point to the oldest value still held by\n /// the channel. A subsequent call to [`recv`] will return this value\n /// **unless** it has been since overwritten.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// channel.\n ///\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n /// [`recv`]: crate::sync::broadcast::Receiver::recv\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx1) = broadcast::channel(16);\n /// let mut rx2 = tx.subscribe();\n ///\n /// tokio::spawn(async move {\n /// assert_eq!(rx1.recv().await.unwrap(), 10);\n /// assert_eq!(rx1.recv().await.unwrap(), 20);\n /// });\n ///\n /// tokio::spawn(async move {\n /// assert_eq!(rx2.recv().await.unwrap(), 10);\n /// assert_eq!(rx2.recv().await.unwrap(), 20);\n /// });\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n /// # }\n /// ```\n ///\n /// Handling lag\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = broadcast::channel(2);\n ///\n /// tx.send(10).unwrap();\n /// tx.send(20).unwrap();\n /// tx.send(30).unwrap();\n ///\n /// // The receiver lagged behind\n /// assert!(rx.recv().await.is_err());\n ///\n /// // At this point, we can abort or continue with lost messages\n ///\n /// assert_eq!(20, rx.recv().await.unwrap());\n /// assert_eq!(30, rx.recv().await.unwrap());\n /// # }\n /// ```\n pub async fn recv(&mut self) -> Result {\n cooperative(Recv::new(self)).await\n }\n\n /// Attempts to return a pending value on this receiver without awaiting.\n ///\n /// This is useful for a flavor of \"optimistic check\" before deciding to\n /// await on a receiver.\n ///\n /// Compared with [`recv`], this function has three failure cases instead of two\n /// (one for closed, one for an empty buffer, one for a lagging receiver).\n ///\n /// `Err(TryRecvError::Closed)` is returned when all `Sender` halves have\n /// dropped, indicating that no further values can be sent on the channel.\n ///\n /// If the [`Receiver`] handle falls behind, once the channel is full, newly\n /// sent values will overwrite old values. At this point, a call to [`recv`]\n /// will return with `Err(TryRecvError::Lagged)` and the [`Receiver`]'s\n /// internal cursor is updated to point to the oldest value still held by\n /// the channel. A subsequent call to [`try_recv`] will return this value\n /// **unless** it has been since overwritten. If there are no values to\n /// receive, `Err(TryRecvError::Empty)` is returned.\n ///\n /// [`recv`]: crate::sync::broadcast::Receiver::recv\n /// [`try_recv`]: crate::sync::broadcast::Receiver::try_recv\n /// [`Receiver`]: crate::sync::broadcast::Receiver\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::broadcast;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = broadcast::channel(16);\n ///\n /// assert!(rx.try_recv().is_err());\n ///\n /// tx.send(10).unwrap();\n ///\n /// let value = rx.try_recv().unwrap();\n /// assert_eq!(10, value);\n /// # }\n /// ```\n pub fn try_recv(&mut self) -> Result {\n let guard = self.recv_ref(None)?;\n guard.clone_value().ok_or(TryRecvError::Closed)\n }\n\n /// Blocking receive to call outside of asynchronous contexts.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution\n /// context.\n ///\n /// # Examples\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::thread;\n /// use tokio::sync::broadcast;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (tx, mut rx) = broadcast::channel(16);\n ///\n /// let sync_code = thread::spawn(move || {\n /// assert_eq!(rx.blocking_recv(), Ok(10));\n /// });\n ///\n /// let _ = tx.send(10);\n /// sync_code.join().unwrap();\n /// }\n /// # }\n /// ```\n pub fn blocking_recv(&mut self) -> Result {\n crate::future::block_on(self.recv())\n }\n}\n\nimpl Drop for Receiver {\n fn drop(&mut self) {\n let mut tail = self.shared.tail.lock();\n\n tail.rx_cnt -= 1;\n let until = tail.pos;\n let remaining_rx = tail.rx_cnt;\n\n if remaining_rx == 0 {\n self.shared.notify_last_rx_drop.notify_waiters();\n tail.closed = true;\n }\n\n drop(tail);\n\n while self.next < until {\n match self.recv_ref(None) {\n Ok(_) => {}\n // The channel is closed\n Err(TryRecvError::Closed) => break,\n // Ignore lagging, we will catch up\n Err(TryRecvError::Lagged(..)) => {}\n // Can't be empty\n Err(TryRecvError::Empty) => panic!(\"unexpected empty broadcast channel\"),\n }\n }\n }\n}\n\nimpl<'a, T> Recv<'a, T> {\n fn new(receiver: &'a mut Receiver) -> Recv<'a, T> {\n Recv {\n receiver,\n waiter: WaiterCell(UnsafeCell::new(Waiter {\n queued: AtomicBool::new(false),\n waker: None,\n pointers: linked_list::Pointers::new(),\n _p: PhantomPinned,\n })),\n }\n }\n\n /// A custom `project` implementation is used in place of `pin-project-lite`\n /// as a custom drop implementation is needed.\n fn project(self: Pin<&mut Self>) -> (&mut Receiver, &UnsafeCell) {\n unsafe {\n // Safety: Receiver is Unpin\n is_unpin::<&mut Receiver>();\n\n let me = self.get_unchecked_mut();\n (me.receiver, &me.waiter.0)\n }\n }\n}\n\nimpl<'a, T> Future for Recv<'a, T>\nwhere\n T: Clone,\n{\n type Output = Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n\n let (receiver, waiter) = self.project();\n\n let guard = match receiver.recv_ref(Some((waiter, cx.waker()))) {\n Ok(value) => value,\n Err(TryRecvError::Empty) => return Poll::Pending,\n Err(TryRecvError::Lagged(n)) => return Poll::Ready(Err(RecvError::Lagged(n))),\n Err(TryRecvError::Closed) => return Poll::Ready(Err(RecvError::Closed)),\n };\n\n Poll::Ready(guard.clone_value().ok_or(RecvError::Closed))\n }\n}\n\nimpl<'a, T> Drop for Recv<'a, T> {\n fn drop(&mut self) {\n // Safety: `waiter.queued` is atomic.\n // Acquire ordering is required to synchronize with\n // `Shared::notify_rx` before we drop the object.\n let queued = self\n .waiter\n .0\n .with(|ptr| unsafe { (*ptr).queued.load(Acquire) });\n\n // If the waiter is queued, we need to unlink it from the waiters list.\n // If not, no further synchronization is required, since the waiter\n // is not in the list and, as such, is not shared with any other threads.\n if queued {\n // Acquire the tail lock. This is required for safety before accessing\n // the waiter node.\n let mut tail = self.receiver.shared.tail.lock();\n\n // Safety: tail lock is held.\n // `Relaxed` order suffices because we hold the tail lock.\n let queued = self\n .waiter\n .0\n .with_mut(|ptr| unsafe { (*ptr).queued.load(Relaxed) });\n\n if queued {\n // Remove the node\n //\n // safety: tail lock is held and the wait node is verified to be in\n // the list.\n unsafe {\n self.waiter.0.with_mut(|ptr| {\n tail.waiters.remove((&mut *ptr).into());\n });\n }\n }\n }\n }\n}\n\n/// # Safety\n///\n/// `Waiter` is forced to be !Unpin.\nunsafe impl linked_list::Link for Waiter {\n type Handle = NonNull;\n type Target = Waiter;\n\n fn as_raw(handle: &NonNull) -> NonNull {\n *handle\n }\n\n unsafe fn from_raw(ptr: NonNull) -> NonNull {\n ptr\n }\n\n unsafe fn pointers(target: NonNull) -> NonNull> {\n unsafe { Waiter::addr_of_pointers(target) }\n }\n}\n\nimpl fmt::Debug for Sender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"broadcast::Sender\")\n }\n}\n\nimpl fmt::Debug for WeakSender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"broadcast::WeakSender\")\n }\n}\n\nimpl fmt::Debug for Receiver {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"broadcast::Receiver\")\n }\n}\n\nimpl<'a, T> RecvGuard<'a, T> {\n fn clone_value(&self) -> Option\n where\n T: Clone,\n {\n self.slot.val.clone()\n }\n}\n\nimpl<'a, T> Drop for RecvGuard<'a, T> {\n fn drop(&mut self) {\n // Decrement the remaining counter\n if 1 == self.slot.rem.fetch_sub(1, SeqCst) {\n self.slot.val = None;\n }\n }\n}\n\nfn is_unpin() {}\n\n#[cfg(not(loom))]\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn receiver_count_on_sender_constructor() {\n let sender = Sender::::new(16);\n assert_eq!(sender.receiver_count(), 0);\n\n let rx_1 = sender.subscribe();\n assert_eq!(sender.receiver_count(), 1);\n\n let rx_2 = rx_1.resubscribe();\n assert_eq!(sender.receiver_count(), 2);\n\n let rx_3 = sender.subscribe();\n assert_eq!(sender.receiver_count(), 3);\n\n drop(rx_3);\n drop(rx_1);\n assert_eq!(sender.receiver_count(), 1);\n\n drop(rx_2);\n assert_eq!(sender.receiver_count(), 0);\n }\n\n #[cfg(not(loom))]\n #[test]\n fn receiver_count_on_channel_constructor() {\n let (sender, rx) = channel::(16);\n assert_eq!(sender.receiver_count(), 1);\n\n let _rx_2 = rx.resubscribe();\n assert_eq!(sender.receiver_count(), 2);\n }\n}\n" }, { "path": "tokio/src/sync/mod.rs", "content": "#![cfg_attr(loom, allow(dead_code, unreachable_pub, unused_imports))]\n\n//! Synchronization primitives for use in asynchronous contexts.\n//!\n//! Tokio programs tend to be organized as a set of [tasks] where each task\n//! operates independently and may be executed on separate physical threads. The\n//! synchronization primitives provided in this module permit these independent\n//! tasks to communicate together.\n//!\n//! [tasks]: crate::task\n//!\n//! # Message passing\n//!\n//! The most common form of synchronization in a Tokio program is message\n//! passing. Two tasks operate independently and send messages to each other to\n//! synchronize. Doing so has the advantage of avoiding shared state.\n//!\n//! Message passing is implemented using channels. A channel supports sending a\n//! message from one producer task to one or more consumer tasks. There are a\n//! few flavors of channels provided by Tokio. Each channel flavor supports\n//! different message passing patterns. When a channel supports multiple\n//! producers, many separate tasks may **send** messages. When a channel\n//! supports multiple consumers, many different separate tasks may **receive**\n//! messages.\n//!\n//! Tokio provides many different channel flavors as different message passing\n//! patterns are best handled with different implementations.\n//!\n//! ## `oneshot` channel\n//!\n//! The [`oneshot` channel][oneshot] supports sending a **single** value from a\n//! single producer to a single consumer. This channel is usually used to send\n//! the result of a computation to a waiter.\n//!\n//! **Example:** using a [`oneshot` channel][oneshot] to receive the result of a\n//! computation.\n//!\n//! ```\n//! use tokio::sync::oneshot;\n//!\n//! async fn some_computation() -> String {\n//! \"represents the result of the computation\".to_string()\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, rx) = oneshot::channel();\n//!\n//! tokio::spawn(async move {\n//! let res = some_computation().await;\n//! tx.send(res).unwrap();\n//! });\n//!\n//! // Do other work while the computation is happening in the background\n//!\n//! // Wait for the computation result\n//! let res = rx.await.unwrap();\n//! # }\n//! ```\n//!\n//! Note, if the task produces a computation result as its final\n//! action before terminating, the [`JoinHandle`] can be used to\n//! receive that value instead of allocating resources for the\n//! `oneshot` channel. Awaiting on [`JoinHandle`] returns `Result`. If\n//! the task panics, the `Joinhandle` yields `Err` with the panic\n//! cause.\n//!\n//! **Example:**\n//!\n//! ```\n//! async fn some_computation() -> String {\n//! \"the result of the computation\".to_string()\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let join_handle = tokio::spawn(async move {\n//! some_computation().await\n//! });\n//!\n//! // Do other work while the computation is happening in the background\n//!\n//! // Wait for the computation result\n//! let res = join_handle.await.unwrap();\n//! # }\n//! ```\n//!\n//! [`JoinHandle`]: crate::task::JoinHandle\n//!\n//! ## `mpsc` channel\n//!\n//! The [`mpsc` channel][mpsc] supports sending **many** values from **many**\n//! producers to a single consumer. This channel is often used to send work to a\n//! task or to receive the result of many computations.\n//!\n//! This is also the channel you should use if you want to send many messages\n//! from a single producer to a single consumer. There is no dedicated spsc\n//! channel.\n//!\n//! **Example:** using an mpsc to incrementally stream the results of a series\n//! of computations.\n//!\n//! ```\n//! use tokio::sync::mpsc;\n//!\n//! async fn some_computation(input: u32) -> String {\n//! format!(\"the result of computation {}\", input)\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, mut rx) = mpsc::channel(100);\n//!\n//! tokio::spawn(async move {\n//! for i in 0..10 {\n//! let res = some_computation(i).await;\n//! tx.send(res).await.unwrap();\n//! }\n//! });\n//!\n//! while let Some(res) = rx.recv().await {\n//! println!(\"got = {}\", res);\n//! }\n//! # }\n//! ```\n//!\n//! The argument to `mpsc::channel` is the channel capacity. This is the maximum\n//! number of values that can be stored in the channel pending receipt at any\n//! given time. Properly setting this value is key in implementing robust\n//! programs as the channel capacity plays a critical part in handling back\n//! pressure.\n//!\n//! A common concurrency pattern for resource management is to spawn a task\n//! dedicated to managing that resource and using message passing between other\n//! tasks to interact with the resource. The resource may be anything that may\n//! not be concurrently used. Some examples include a socket and program state.\n//! For example, if multiple tasks need to send data over a single socket, spawn\n//! a task to manage the socket and use a channel to synchronize.\n//!\n//! **Example:** sending data from many tasks over a single socket using message\n//! passing.\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::io::{self, AsyncWriteExt};\n//! use tokio::net::TcpStream;\n//! use tokio::sync::mpsc;\n//!\n//! #[tokio::main]\n//! async fn main() -> io::Result<()> {\n//! let mut socket = TcpStream::connect(\"www.example.com:1234\").await?;\n//! let (tx, mut rx) = mpsc::channel(100);\n//!\n//! for _ in 0..10 {\n//! // Each task needs its own `tx` handle. This is done by cloning the\n//! // original handle.\n//! let tx = tx.clone();\n//!\n//! tokio::spawn(async move {\n//! tx.send(&b\"data to write\"[..]).await.unwrap();\n//! });\n//! }\n//!\n//! // The `rx` half of the channel returns `None` once **all** `tx` clones\n//! // drop. To ensure `None` is returned, drop the handle owned by the\n//! // current task. If this `tx` handle is not dropped, there will always\n//! // be a single outstanding `tx` handle.\n//! drop(tx);\n//!\n//! while let Some(res) = rx.recv().await {\n//! socket.write_all(res).await?;\n//! }\n//!\n//! Ok(())\n//! }\n//! # }\n//! ```\n//!\n//! The [`mpsc`] and [`oneshot`] channels can be combined to provide a request /\n//! response type synchronization pattern with a shared resource. A task is\n//! spawned to synchronize a resource and waits on commands received on a\n//! [`mpsc`] channel. Each command includes a [`oneshot`] `Sender` on which the\n//! result of the command is sent.\n//!\n//! **Example:** use a task to synchronize a `u64` counter. Each task sends an\n//! \"fetch and increment\" command. The counter value **before** the increment is\n//! sent over the provided `oneshot` channel.\n//!\n//! ```\n//! use tokio::sync::{oneshot, mpsc};\n//! use Command::Increment;\n//!\n//! enum Command {\n//! Increment,\n//! // Other commands can be added here\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (cmd_tx, mut cmd_rx) = mpsc::channel::<(Command, oneshot::Sender)>(100);\n//!\n//! // Spawn a task to manage the counter\n//! tokio::spawn(async move {\n//! let mut counter: u64 = 0;\n//!\n//! while let Some((cmd, response)) = cmd_rx.recv().await {\n//! match cmd {\n//! Increment => {\n//! let prev = counter;\n//! counter += 1;\n//! response.send(prev).unwrap();\n//! }\n//! }\n//! }\n//! });\n//!\n//! let mut join_handles = vec![];\n//!\n//! // Spawn tasks that will send the increment command.\n//! for _ in 0..10 {\n//! let cmd_tx = cmd_tx.clone();\n//!\n//! join_handles.push(tokio::spawn(async move {\n//! let (resp_tx, resp_rx) = oneshot::channel();\n//!\n//! cmd_tx.send((Increment, resp_tx)).await.ok().unwrap();\n//! let res = resp_rx.await.unwrap();\n//!\n//! println!(\"previous value = {}\", res);\n//! }));\n//! }\n//!\n//! // Wait for all tasks to complete\n//! for join_handle in join_handles.drain(..) {\n//! join_handle.await.unwrap();\n//! }\n//! # }\n//! ```\n//!\n//! ## `broadcast` channel\n//!\n//! The [`broadcast` channel] supports sending **many** values from\n//! **many** producers to **many** consumers. Each consumer will receive\n//! **each** value. This channel can be used to implement \"fan out\" style\n//! patterns common with pub / sub or \"chat\" systems.\n//!\n//! This channel tends to be used less often than `oneshot` and `mpsc` but still\n//! has its use cases.\n//!\n//! This is also the channel you should use if you want to broadcast values from\n//! a single producer to many consumers. There is no dedicated spmc broadcast\n//! channel.\n//!\n//! Basic usage\n//!\n//! ```\n//! use tokio::sync::broadcast;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, mut rx1) = broadcast::channel(16);\n//! let mut rx2 = tx.subscribe();\n//!\n//! tokio::spawn(async move {\n//! assert_eq!(rx1.recv().await.unwrap(), 10);\n//! assert_eq!(rx1.recv().await.unwrap(), 20);\n//! });\n//!\n//! tokio::spawn(async move {\n//! assert_eq!(rx2.recv().await.unwrap(), 10);\n//! assert_eq!(rx2.recv().await.unwrap(), 20);\n//! });\n//!\n//! tx.send(10).unwrap();\n//! tx.send(20).unwrap();\n//! # }\n//! ```\n//!\n//! [`broadcast` channel]: crate::sync::broadcast\n//!\n//! ## `watch` channel\n//!\n//! The [`watch` channel] supports sending **many** values from **many**\n//! producers to **many** consumers. However, only the **most recent** value is\n//! stored in the channel. Consumers are notified when a new value is sent, but\n//! there is no guarantee that consumers will see **all** values.\n//!\n//! The [`watch` channel] is similar to a [`broadcast` channel] with capacity 1.\n//!\n//! Use cases for the [`watch` channel] include broadcasting configuration\n//! changes or signalling program state changes, such as transitioning to\n//! shutdown.\n//!\n//! **Example:** use a [`watch` channel] to notify tasks of configuration\n//! changes. In this example, a configuration file is checked periodically. When\n//! the file changes, the configuration changes are signalled to consumers.\n//!\n//! ```\n//! use tokio::sync::watch;\n//! use tokio::time::{self, Duration, Instant};\n//!\n//! use std::io;\n//!\n//! #[derive(Debug, Clone, Eq, PartialEq)]\n//! struct Config {\n//! timeout: Duration,\n//! }\n//!\n//! impl Config {\n//! async fn load_from_file() -> io::Result {\n//! // file loading and deserialization logic here\n//! # Ok(Config { timeout: Duration::from_secs(1) })\n//! }\n//! }\n//!\n//! async fn my_async_operation() {\n//! // Do something here\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! // Load initial configuration value\n//! let mut config = Config::load_from_file().await.unwrap();\n//!\n//! // Create the watch channel, initialized with the loaded configuration\n//! let (tx, rx) = watch::channel(config.clone());\n//!\n//! // Spawn a task to monitor the file.\n//! tokio::spawn(async move {\n//! loop {\n//! // Wait 10 seconds between checks\n//! time::sleep(Duration::from_secs(10)).await;\n//!\n//! // Load the configuration file\n//! let new_config = Config::load_from_file().await.unwrap();\n//!\n//! // If the configuration changed, send the new config value\n//! // on the watch channel.\n//! if new_config != config {\n//! tx.send(new_config.clone()).unwrap();\n//! config = new_config;\n//! }\n//! }\n//! });\n//!\n//! let mut handles = vec![];\n//!\n//! // Spawn tasks that runs the async operation for at most `timeout`. If\n//! // the timeout elapses, restart the operation.\n//! //\n//! // The task simultaneously watches the `Config` for changes. When the\n//! // timeout duration changes, the timeout is updated without restarting\n//! // the in-flight operation.\n//! for _ in 0..5 {\n//! // Clone a config watch handle for use in this task\n//! let mut rx = rx.clone();\n//!\n//! let handle = tokio::spawn(async move {\n//! // Start the initial operation and pin the future to the stack.\n//! // Pinning to the stack is required to resume the operation\n//! // across multiple calls to `select!`\n//! let op = my_async_operation();\n//! tokio::pin!(op);\n//!\n//! // Get the initial config value\n//! let mut conf = rx.borrow().clone();\n//!\n//! let mut op_start = Instant::now();\n//! let sleep = time::sleep_until(op_start + conf.timeout);\n//! tokio::pin!(sleep);\n//!\n//! loop {\n//! tokio::select! {\n//! _ = &mut sleep => {\n//! // The operation elapsed. Restart it\n//! op.set(my_async_operation());\n//!\n//! // Track the new start time\n//! op_start = Instant::now();\n//!\n//! // Restart the timeout\n//! sleep.set(time::sleep_until(op_start + conf.timeout));\n//! }\n//! _ = rx.changed() => {\n//! conf = rx.borrow_and_update().clone();\n//!\n//! // The configuration has been updated. Update the\n//! // `sleep` using the new `timeout` value.\n//! sleep.as_mut().reset(op_start + conf.timeout);\n//! }\n//! _ = &mut op => {\n//! // The operation completed!\n//! return\n//! }\n//! }\n//! }\n//! });\n//!\n//! handles.push(handle);\n//! }\n//!\n//! for handle in handles.drain(..) {\n//! handle.await.unwrap();\n//! }\n//! # }\n//! ```\n//!\n//! [`watch` channel]: mod@crate::sync::watch\n//! [`broadcast` channel]: mod@crate::sync::broadcast\n//!\n//! # State synchronization\n//!\n//! The remaining synchronization primitives focus on synchronizing state.\n//! These are asynchronous equivalents to versions provided by `std`. They\n//! operate in a similar way as their `std` counterparts but will wait\n//! asynchronously instead of blocking the thread.\n//!\n//! * [`Barrier`] Ensures multiple tasks will wait for each other to reach a\n//! point in the program, before continuing execution all together.\n//!\n//! * [`Mutex`] Mutual Exclusion mechanism, which ensures that at most one\n//! thread at a time is able to access some data.\n//!\n//! * [`Notify`] Basic task notification. `Notify` supports notifying a\n//! receiving task without sending data. In this case, the task wakes up and\n//! resumes processing.\n//!\n//! * [`RwLock`] Provides a mutual exclusion mechanism which allows multiple\n//! readers at the same time, while allowing only one writer at a time. In\n//! some cases, this can be more efficient than a mutex.\n//!\n//! * [`Semaphore`] Limits the amount of concurrency. A semaphore holds a\n//! number of permits, which tasks may request in order to enter a critical\n//! section. Semaphores are useful for implementing limiting or bounding of\n//! any kind.\n//!\n//! # Runtime compatibility\n//!\n//! All synchronization primitives provided in this module are runtime agnostic.\n//! You can freely move them between different instances of the Tokio runtime\n//! or even use them from non-Tokio runtimes.\n//!\n//! When used in a Tokio runtime, the synchronization primitives participate in\n//! [cooperative scheduling](crate::task::coop#cooperative-scheduling) to avoid\n//! starvation. This feature does not apply when used from non-Tokio runtimes.\n//!\n//! As an exception, methods ending in `_timeout` are not runtime agnostic\n//! because they require access to the Tokio timer. See the documentation of\n//! each `*_timeout` method for more information on its use.\n\ncfg_sync! {\n /// Named future types.\n pub mod futures {\n pub use super::notify::{Notified, OwnedNotified};\n }\n\n mod barrier;\n pub use barrier::{Barrier, BarrierWaitResult};\n\n pub mod broadcast;\n\n pub mod mpsc;\n\n mod mutex;\n pub use mutex::{Mutex, MutexGuard, TryLockError, OwnedMutexGuard, MappedMutexGuard, OwnedMappedMutexGuard};\n\n pub(crate) mod notify;\n pub use notify::Notify;\n\n pub mod oneshot;\n\n pub(crate) mod batch_semaphore;\n pub use batch_semaphore::{AcquireError, TryAcquireError};\n\n mod semaphore;\n pub use semaphore::{Semaphore, SemaphorePermit, OwnedSemaphorePermit};\n\n mod rwlock;\n pub use rwlock::RwLock;\n pub use rwlock::owned_read_guard::OwnedRwLockReadGuard;\n pub use rwlock::owned_write_guard::OwnedRwLockWriteGuard;\n pub use rwlock::owned_write_guard_mapped::OwnedRwLockMappedWriteGuard;\n pub use rwlock::read_guard::RwLockReadGuard;\n pub use rwlock::write_guard::RwLockWriteGuard;\n pub use rwlock::write_guard_mapped::RwLockMappedWriteGuard;\n\n mod task;\n pub(crate) use task::AtomicWaker;\n\n mod once_cell;\n pub use self::once_cell::{OnceCell, SetError};\n\n mod set_once;\n pub use self::set_once::{SetOnce, SetOnceError};\n\n pub mod watch;\n}\n\ncfg_not_sync! {\n cfg_fs! {\n pub(crate) mod batch_semaphore;\n mod mutex;\n pub(crate) use mutex::Mutex;\n }\n\n #[cfg(any(feature = \"rt\", feature = \"signal\", all(unix, feature = \"process\")))]\n pub(crate) mod notify;\n\n #[cfg(any(feature = \"rt\", all(windows, feature = \"process\")))]\n pub(crate) mod oneshot;\n\n cfg_atomic_waker_impl! {\n mod task;\n pub(crate) use task::AtomicWaker;\n }\n\n #[cfg(any(feature = \"signal\", all(unix, feature = \"process\")))]\n pub(crate) mod watch;\n}\n\n/// Unit tests\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio/src/sync/mpsc/block.rs", "content": "use crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::{AtomicPtr, AtomicUsize};\n\nuse std::alloc::Layout;\nuse std::mem::MaybeUninit;\nuse std::ops;\nuse std::ptr::{self, NonNull};\nuse std::sync::atomic::Ordering::{self, AcqRel, Acquire, Release};\n\n/// A block in a linked list.\n///\n/// Each block in the list can hold up to `BLOCK_CAP` messages.\npub(crate) struct Block {\n /// The header fields.\n header: BlockHeader,\n\n /// Array containing values pushed into the block. Values are stored in a\n /// continuous array in order to improve cache line behavior when reading.\n /// The values must be manually dropped.\n values: Values,\n}\n\n/// Extra fields for a `Block`.\nstruct BlockHeader {\n /// The start index of this block.\n ///\n /// Slots in this block have indices in `start_index .. start_index + BLOCK_CAP`.\n start_index: usize,\n\n /// The next block in the linked list.\n next: AtomicPtr>,\n\n /// Bitfield tracking slots that are ready to have their values consumed.\n ready_slots: AtomicUsize,\n\n /// The observed `tail_position` value *after* the block has been passed by\n /// `block_tail`.\n observed_tail_position: UnsafeCell,\n}\n\npub(crate) enum Read {\n Value(T),\n Closed,\n}\n\n#[repr(transparent)]\nstruct Values([UnsafeCell>; BLOCK_CAP]);\n\nuse super::BLOCK_CAP;\n\n/// Masks an index to get the block identifier.\nconst BLOCK_MASK: usize = !(BLOCK_CAP - 1);\n\n/// Masks an index to get the value offset in a block.\nconst SLOT_MASK: usize = BLOCK_CAP - 1;\n\n/// Flag tracking that a block has gone through the sender's release routine.\n///\n/// When this is set, the receiver may consider freeing the block.\nconst RELEASED: usize = 1 << BLOCK_CAP;\n\n/// Flag tracking all senders dropped.\n///\n/// When this flag is set, the send half of the channel has closed.\nconst TX_CLOSED: usize = RELEASED << 1;\n\n/// Mask covering all bits used to track slot readiness.\nconst READY_MASK: usize = RELEASED - 1;\n\n/// Returns the index of the first slot in the block referenced by `slot_index`.\n#[inline(always)]\npub(crate) fn start_index(slot_index: usize) -> usize {\n BLOCK_MASK & slot_index\n}\n\n/// Returns the offset into the block referenced by `slot_index`.\n#[inline(always)]\npub(crate) fn offset(slot_index: usize) -> usize {\n SLOT_MASK & slot_index\n}\n\ngenerate_addr_of_methods! {\n impl Block {\n unsafe fn addr_of_header(self: NonNull) -> NonNull> {\n &self.header\n }\n\n unsafe fn addr_of_values(self: NonNull) -> NonNull> {\n &self.values\n }\n }\n}\n\nimpl Block {\n pub(crate) fn new(start_index: usize) -> Box> {\n unsafe {\n // Allocate the block on the heap.\n // SAFETY: The size of the Block is non-zero, since it is at least the size of the header.\n let block = std::alloc::alloc(Layout::new::>()) as *mut Block;\n let block = match NonNull::new(block) {\n Some(block) => block,\n None => std::alloc::handle_alloc_error(Layout::new::>()),\n };\n\n // Write the header to the block.\n Block::addr_of_header(block).as_ptr().write(BlockHeader {\n // The absolute index in the channel of the first slot in the block.\n start_index,\n\n // Pointer to the next block in the linked list.\n next: AtomicPtr::new(ptr::null_mut()),\n\n ready_slots: AtomicUsize::new(0),\n\n observed_tail_position: UnsafeCell::new(0),\n });\n\n // Initialize the values array.\n Values::initialize(Block::addr_of_values(block));\n\n // Convert the pointer to a `Box`.\n // Safety: The raw pointer was allocated using the global allocator, and with\n // the layout for a `Block`, so it's valid to convert it to box.\n Box::from_raw(block.as_ptr())\n }\n }\n\n /// Returns `true` if the block matches the given index.\n pub(crate) fn is_at_index(&self, index: usize) -> bool {\n debug_assert!(offset(index) == 0);\n self.header.start_index == index\n }\n\n /// Returns the number of blocks between `self` and the block at the\n /// specified index.\n ///\n /// `start_index` must represent a block *after* `self`.\n pub(crate) fn distance(&self, other_index: usize) -> usize {\n debug_assert!(offset(other_index) == 0);\n other_index.wrapping_sub(self.header.start_index) / BLOCK_CAP\n }\n\n /// Reads the value at the given offset.\n ///\n /// Returns `None` if the slot is empty.\n ///\n /// # Safety\n ///\n /// To maintain safety, the caller must ensure:\n ///\n /// * No concurrent access to the slot.\n pub(crate) unsafe fn read(&self, slot_index: usize) -> Option> {\n let offset = offset(slot_index);\n\n let ready_bits = self.header.ready_slots.load(Acquire);\n\n if !is_ready(ready_bits, offset) {\n if is_tx_closed(ready_bits) {\n return Some(Read::Closed);\n }\n\n return None;\n }\n\n // Get the value\n //\n // Safety:\n //\n // 1. The caller guarantees that there is no concurrent access to the slot.\n // 2. The `UnsafeCell` always give us a valid pointer to the value.\n let value = self.values[offset].with(|ptr| unsafe { ptr::read(ptr) });\n\n // Safety: the ready bit is set, so the value has been initialized.\n Some(Read::Value(unsafe { value.assume_init() }))\n }\n\n /// Returns true if *this* block has a value in the given slot.\n ///\n /// Always returns false when given an index from a different block.\n pub(crate) fn has_value(&self, slot_index: usize) -> bool {\n if slot_index < self.header.start_index {\n return false;\n }\n if slot_index >= self.header.start_index + super::BLOCK_CAP {\n return false;\n }\n\n let offset = offset(slot_index);\n let ready_bits = self.header.ready_slots.load(Acquire);\n is_ready(ready_bits, offset)\n }\n\n /// Writes a value to the block at the given offset.\n ///\n /// # Safety\n ///\n /// To maintain safety, the caller must ensure:\n ///\n /// * The slot is empty.\n /// * No concurrent access to the slot.\n pub(crate) unsafe fn write(&self, slot_index: usize, value: T) {\n // Get the offset into the block\n let slot_offset = offset(slot_index);\n\n self.values[slot_offset].with_mut(|ptr| {\n // Safety: the caller guarantees that there is no concurrent access to the slot\n unsafe {\n ptr::write(ptr, MaybeUninit::new(value));\n }\n });\n\n // Release the value. After this point, the slot ref may no longer\n // be used. It is possible for the receiver to free the memory at\n // any point.\n self.set_ready(slot_offset);\n }\n\n /// Signal to the receiver that the sender half of the list is closed.\n pub(crate) unsafe fn tx_close(&self) {\n self.header.ready_slots.fetch_or(TX_CLOSED, Release);\n }\n\n pub(crate) unsafe fn is_closed(&self) -> bool {\n let ready_bits = self.header.ready_slots.load(Acquire);\n is_tx_closed(ready_bits)\n }\n\n /// Resets the block to a blank state. This enables reusing blocks in the\n /// channel.\n ///\n /// # Safety\n ///\n /// To maintain safety, the caller must ensure:\n ///\n /// * All slots are empty.\n /// * The caller holds a unique pointer to the block.\n pub(crate) unsafe fn reclaim(&mut self) {\n self.header.start_index = 0;\n self.header.next = AtomicPtr::new(ptr::null_mut());\n self.header.ready_slots = AtomicUsize::new(0);\n }\n\n /// Releases the block to the rx half for freeing.\n ///\n /// This function is called by the tx half once it can be guaranteed that no\n /// more senders will attempt to access the block.\n ///\n /// # Safety\n ///\n /// To maintain safety, the caller must ensure:\n ///\n /// * The block will no longer be accessed by any sender.\n pub(crate) unsafe fn tx_release(&self, tail_position: usize) {\n // Track the observed tail_position. Any sender targeting a greater\n // tail_position is guaranteed to not access this block.\n self.header\n .observed_tail_position\n // Safety:\n //\n // 1. The caller guarantees unique access to the block.\n // 2. The `UnsafeCell` always gives us a valid pointer.\n .with_mut(|ptr| unsafe { *ptr = tail_position });\n\n // Set the released bit, signalling to the receiver that it is safe to\n // free the block's memory as soon as all slots **prior** to\n // `observed_tail_position` have been filled.\n self.header.ready_slots.fetch_or(RELEASED, Release);\n }\n\n /// Mark a slot as ready\n fn set_ready(&self, slot: usize) {\n let mask = 1 << slot;\n self.header.ready_slots.fetch_or(mask, Release);\n }\n\n /// Returns `true` when all slots have their `ready` bits set.\n ///\n /// This indicates that the block is in its final state and will no longer\n /// be mutated.\n pub(crate) fn is_final(&self) -> bool {\n self.header.ready_slots.load(Acquire) & READY_MASK == READY_MASK\n }\n\n /// Returns the `observed_tail_position` value, if set\n pub(crate) fn observed_tail_position(&self) -> Option {\n if 0 == RELEASED & self.header.ready_slots.load(Acquire) {\n None\n } else {\n Some(\n self.header\n .observed_tail_position\n .with(|ptr| unsafe { *ptr }),\n )\n }\n }\n\n /// Loads the next block\n pub(crate) fn load_next(&self, ordering: Ordering) -> Option>> {\n let ret = NonNull::new(self.header.next.load(ordering));\n\n debug_assert!(unsafe {\n ret.map_or(true, |block| {\n block.as_ref().header.start_index == self.header.start_index.wrapping_add(BLOCK_CAP)\n })\n });\n\n ret\n }\n\n /// Pushes `block` as the next block in the link.\n ///\n /// Returns Ok if successful, otherwise, a pointer to the next block in\n /// the list is returned.\n ///\n /// This requires that the next pointer is null.\n ///\n /// # Ordering\n ///\n /// This performs a compare-and-swap on `next` using `AcqRel` ordering.\n ///\n /// # Safety\n ///\n /// To maintain safety, the caller must ensure:\n ///\n /// * `block` is not freed until it has been removed from the list.\n pub(crate) unsafe fn try_push(\n &self,\n block: &mut NonNull>,\n success: Ordering,\n failure: Ordering,\n ) -> Result<(), NonNull>> {\n // Safety: caller guarantees that `block` is valid.\n unsafe { block.as_mut() }.header.start_index =\n self.header.start_index.wrapping_add(BLOCK_CAP);\n\n let next_ptr = self\n .header\n .next\n .compare_exchange(ptr::null_mut(), block.as_ptr(), success, failure)\n .unwrap_or_else(|x| x);\n\n match NonNull::new(next_ptr) {\n Some(next_ptr) => Err(next_ptr),\n None => Ok(()),\n }\n }\n\n /// Grows the `Block` linked list by allocating and appending a new block.\n ///\n /// The next block in the linked list is returned. This may or may not be\n /// the one allocated by the function call.\n ///\n /// # Implementation\n ///\n /// It is assumed that `self.next` is null. A new block is allocated with\n /// `start_index` set to be the next block. A compare-and-swap is performed\n /// with `AcqRel` memory ordering. If the compare-and-swap is successful, the\n /// newly allocated block is released to other threads walking the block\n /// linked list. If the compare-and-swap fails, the current thread acquires\n /// the next block in the linked list, allowing the current thread to access\n /// the slots.\n pub(crate) fn grow(&self) -> NonNull> {\n // Create the new block. It is assumed that the block will become the\n // next one after `&self`. If this turns out to not be the case,\n // `start_index` is updated accordingly.\n let new_block = Block::new(self.header.start_index + BLOCK_CAP);\n\n let mut new_block = unsafe { NonNull::new_unchecked(Box::into_raw(new_block)) };\n\n // Attempt to store the block. The first compare-and-swap attempt is\n // \"unrolled\" due to minor differences in logic\n //\n // `AcqRel` is used as the ordering **only** when attempting the\n // compare-and-swap on self.next.\n //\n // If the compare-and-swap fails, then the actual value of the cell is\n // returned from this function and accessed by the caller. Given this,\n // the memory must be acquired.\n //\n // `Release` ensures that the newly allocated block is available to\n // other threads acquiring the next pointer.\n let next = NonNull::new(\n self.header\n .next\n .compare_exchange(ptr::null_mut(), new_block.as_ptr(), AcqRel, Acquire)\n .unwrap_or_else(|x| x),\n );\n\n let next = match next {\n Some(next) => next,\n None => {\n // The compare-and-swap succeeded and the newly allocated block\n // is successfully pushed.\n return new_block;\n }\n };\n\n // There already is a next block in the linked list. The newly allocated\n // block could be dropped and the discovered next block returned;\n // however, that would be wasteful. Instead, the linked list is walked\n // by repeatedly attempting to compare-and-swap the pointer into the\n // `next` register until the compare-and-swap succeed.\n //\n // Care is taken to update new_block's start_index field as appropriate.\n\n let mut curr = next;\n\n // TODO: Should this iteration be capped?\n loop {\n let actual = unsafe { curr.as_ref().try_push(&mut new_block, AcqRel, Acquire) };\n\n curr = match actual {\n Ok(()) => {\n return next;\n }\n Err(curr) => curr,\n };\n\n crate::loom::thread::yield_now();\n }\n }\n}\n\n/// Returns `true` if the specified slot has a value ready to be consumed.\nfn is_ready(bits: usize, slot: usize) -> bool {\n let mask = 1 << slot;\n mask == mask & bits\n}\n\n/// Returns `true` if the closed flag has been set.\nfn is_tx_closed(bits: usize) -> bool {\n TX_CLOSED == bits & TX_CLOSED\n}\n\nimpl Values {\n /// Initialize a `Values` struct from a pointer.\n ///\n /// # Safety\n ///\n /// The raw pointer must be valid for writing a `Values`.\n unsafe fn initialize(_value: NonNull>) {\n // When fuzzing, `UnsafeCell` needs to be initialized.\n if_loom! {\n let p = _value.as_ptr() as *mut UnsafeCell>;\n for i in 0..BLOCK_CAP {\n unsafe {\n p.add(i).write(UnsafeCell::new(MaybeUninit::uninit()));\n }\n }\n }\n }\n}\n\nimpl ops::Index for Values {\n type Output = UnsafeCell>;\n\n fn index(&self, index: usize) -> &Self::Output {\n self.0.index(index)\n }\n}\n\n#[cfg(all(test, not(loom)))]\n#[test]\nfn assert_no_stack_overflow() {\n // https://github.com/tokio-rs/tokio/issues/5293\n\n struct Foo {\n _a: [u8; 2_000_000],\n }\n\n assert_eq!(\n Layout::new::>>(),\n Layout::new::>()\n );\n\n let _block = Block::::new(0);\n}\n" }, { "path": "tokio/src/sync/mpsc/bounded.rs", "content": "use crate::loom::sync::Arc;\nuse crate::sync::batch_semaphore::{self as semaphore, TryAcquireError};\nuse crate::sync::mpsc::chan;\nuse crate::sync::mpsc::error::{SendError, TryRecvError, TrySendError};\n\ncfg_time! {\n use crate::sync::mpsc::error::SendTimeoutError;\n use crate::time::Duration;\n}\n\nuse std::fmt;\nuse std::task::{Context, Poll};\n\n/// Sends values to the associated `Receiver`.\n///\n/// Instances are created by the [`channel`] function.\n///\n/// To convert the `Sender` into a `Sink` or use it in a poll function, you can\n/// use the [`PollSender`] utility.\n///\n/// [`PollSender`]: https://docs.rs/tokio-util/latest/tokio_util/sync/struct.PollSender.html\npub struct Sender {\n chan: chan::Tx,\n}\n\n/// A sender that does not prevent the channel from being closed.\n///\n/// If all [`Sender`] instances of a channel were dropped and only `WeakSender`\n/// instances remain, the channel is closed.\n///\n/// In order to send messages, the `WeakSender` needs to be upgraded using\n/// [`WeakSender::upgrade`], which returns `Option`. It returns `None`\n/// if all `Sender`s have been dropped, and otherwise it returns a `Sender`.\n///\n/// [`Sender`]: Sender\n/// [`WeakSender::upgrade`]: WeakSender::upgrade\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::mpsc::channel;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, _rx) = channel::(15);\n/// let tx_weak = tx.downgrade();\n///\n/// // Upgrading will succeed because `tx` still exists.\n/// assert!(tx_weak.upgrade().is_some());\n///\n/// // If we drop `tx`, then it will fail.\n/// drop(tx);\n/// assert!(tx_weak.clone().upgrade().is_none());\n/// # }\n/// ```\npub struct WeakSender {\n chan: Arc>,\n}\n\n/// Permits to send one value into the channel.\n///\n/// `Permit` values are returned by [`Sender::reserve()`] and [`Sender::try_reserve()`]\n/// and are used to guarantee channel capacity before generating a message to send.\n///\n/// [`Sender::reserve()`]: Sender::reserve\n/// [`Sender::try_reserve()`]: Sender::try_reserve\npub struct Permit<'a, T> {\n chan: &'a chan::Tx,\n}\n\n/// An [`Iterator`] of [`Permit`] that can be used to hold `n` slots in the channel.\n///\n/// `PermitIterator` values are returned by [`Sender::reserve_many()`] and [`Sender::try_reserve_many()`]\n/// and are used to guarantee channel capacity before generating `n` messages to send.\n///\n/// [`Sender::reserve_many()`]: Sender::reserve_many\n/// [`Sender::try_reserve_many()`]: Sender::try_reserve_many\npub struct PermitIterator<'a, T> {\n chan: &'a chan::Tx,\n n: usize,\n}\n\n/// Owned permit to send one value into the channel.\n///\n/// This is identical to the [`Permit`] type, except that it moves the sender\n/// rather than borrowing it.\n///\n/// `OwnedPermit` values are returned by [`Sender::reserve_owned()`] and\n/// [`Sender::try_reserve_owned()`] and are used to guarantee channel capacity\n/// before generating a message to send.\n///\n/// [`Permit`]: Permit\n/// [`Sender::reserve_owned()`]: Sender::reserve_owned\n/// [`Sender::try_reserve_owned()`]: Sender::try_reserve_owned\npub struct OwnedPermit {\n chan: Option>,\n}\n\n/// Receives values from the associated `Sender`.\n///\n/// Instances are created by the [`channel`] function.\n///\n/// This receiver can be turned into a `Stream` using [`ReceiverStream`].\n///\n/// [`ReceiverStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.ReceiverStream.html\npub struct Receiver {\n /// The channel receiver.\n chan: chan::Rx,\n}\n\n/// Creates a bounded mpsc channel for communicating between asynchronous tasks\n/// with backpressure.\n///\n/// The channel will buffer up to the provided number of messages. Once the\n/// buffer is full, attempts to send new messages will wait until a message is\n/// received from the channel. The provided buffer capacity must be at least 1.\n///\n/// All data sent on `Sender` will become available on `Receiver` in the same\n/// order as it was sent.\n///\n/// The `Sender` can be cloned to `send` to the same channel from multiple code\n/// locations. Only one `Receiver` is supported.\n///\n/// If the `Receiver` is disconnected while trying to `send`, the `send` method\n/// will return a `SendError`. Similarly, if `Sender` is disconnected while\n/// trying to `recv`, the `recv` method will return `None`.\n///\n/// # Panics\n///\n/// Panics if the buffer capacity is 0, or too large. Currently the maximum\n/// capacity is [`Semaphore::MAX_PERMITS`].\n///\n/// [`Semaphore::MAX_PERMITS`]: crate::sync::Semaphore::MAX_PERMITS\n///\n/// # Examples\n///\n/// ```rust\n/// use tokio::sync::mpsc;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, mut rx) = mpsc::channel(100);\n///\n/// tokio::spawn(async move {\n/// for i in 0..10 {\n/// if let Err(_) = tx.send(i).await {\n/// println!(\"receiver dropped\");\n/// return;\n/// }\n/// }\n/// });\n///\n/// while let Some(i) = rx.recv().await {\n/// println!(\"got = {}\", i);\n/// }\n/// # }\n/// ```\n#[track_caller]\npub fn channel(buffer: usize) -> (Sender, Receiver) {\n assert!(buffer > 0, \"mpsc bounded channel requires buffer > 0\");\n let semaphore = Semaphore {\n semaphore: semaphore::Semaphore::new(buffer),\n bound: buffer,\n };\n let (tx, rx) = chan::channel(semaphore);\n\n let tx = Sender::new(tx);\n let rx = Receiver::new(rx);\n\n (tx, rx)\n}\n\n/// Channel semaphore is a tuple of the semaphore implementation and a `usize`\n/// representing the channel bound.\n#[derive(Debug)]\npub(crate) struct Semaphore {\n pub(crate) semaphore: semaphore::Semaphore,\n pub(crate) bound: usize,\n}\n\nimpl Receiver {\n pub(crate) fn new(chan: chan::Rx) -> Receiver {\n Receiver { chan }\n }\n\n /// Receives the next value for this receiver.\n ///\n /// This method returns `None` if the channel has been closed and there are\n /// no remaining messages in the channel's buffer. This indicates that no\n /// further values can ever be received from this `Receiver`. The channel is\n /// closed when all senders have been dropped, or when [`close`] is called.\n ///\n /// If there are no messages in the channel's buffer, but the channel has\n /// not yet been closed, this method will sleep until a message is sent or\n /// the channel is closed. Note that if [`close`] is called, but there are\n /// still outstanding [`Permits`] from before it was closed, the channel is\n /// not considered closed by `recv` until the permits are released.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// channel.\n ///\n /// [`close`]: Self::close\n /// [`Permits`]: struct@crate::sync::mpsc::Permit\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(100);\n ///\n /// tokio::spawn(async move {\n /// tx.send(\"hello\").await.unwrap();\n /// });\n ///\n /// assert_eq!(Some(\"hello\"), rx.recv().await);\n /// assert_eq!(None, rx.recv().await);\n /// # }\n /// ```\n ///\n /// Values are buffered:\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(100);\n ///\n /// tx.send(\"hello\").await.unwrap();\n /// tx.send(\"world\").await.unwrap();\n ///\n /// assert_eq!(Some(\"hello\"), rx.recv().await);\n /// assert_eq!(Some(\"world\"), rx.recv().await);\n /// # }\n /// ```\n pub async fn recv(&mut self) -> Option {\n use std::future::poll_fn;\n poll_fn(|cx| self.chan.recv(cx)).await\n }\n\n /// Receives the next values for this receiver and extends `buffer`.\n ///\n /// This method extends `buffer` by no more than a fixed number of values\n /// as specified by `limit`. If `limit` is zero, the function immediately\n /// returns `0`. The return value is the number of values added to `buffer`.\n ///\n /// For `limit > 0`, if there are no messages in the channel's queue, but\n /// the channel has not yet been closed, this method will sleep until a\n /// message is sent or the channel is closed. Note that if [`close`] is\n /// called, but there are still outstanding [`Permits`] from before it was\n /// closed, the channel is not considered closed by `recv_many` until the\n /// permits are released.\n ///\n /// For non-zero values of `limit`, this method will never return `0` unless\n /// the channel has been closed and there are no remaining messages in the\n /// channel's queue. This indicates that no further values can ever be\n /// received from this `Receiver`. The channel is closed when all senders\n /// have been dropped, or when [`close`] is called.\n ///\n /// The capacity of `buffer` is increased as needed.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv_many` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// channel.\n ///\n /// [`close`]: Self::close\n /// [`Permits`]: struct@crate::sync::mpsc::Permit\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut buffer: Vec<&str> = Vec::with_capacity(2);\n /// let limit = 2;\n /// let (tx, mut rx) = mpsc::channel(100);\n /// let tx2 = tx.clone();\n /// tx2.send(\"first\").await.unwrap();\n /// tx2.send(\"second\").await.unwrap();\n /// tx2.send(\"third\").await.unwrap();\n ///\n /// // Call `recv_many` to receive up to `limit` (2) values.\n /// assert_eq!(2, rx.recv_many(&mut buffer, limit).await);\n /// assert_eq!(vec![\"first\", \"second\"], buffer);\n ///\n /// // If the buffer is full, the next call to `recv_many`\n /// // reserves additional capacity.\n /// assert_eq!(1, rx.recv_many(&mut buffer, 1).await);\n ///\n /// tokio::spawn(async move {\n /// tx.send(\"fourth\").await.unwrap();\n /// });\n ///\n /// // 'tx' is dropped, but `recv_many`\n /// // is guaranteed not to return 0 as the channel\n /// // is not yet closed.\n /// assert_eq!(1, rx.recv_many(&mut buffer, 1).await);\n /// assert_eq!(vec![\"first\", \"second\", \"third\", \"fourth\"], buffer);\n ///\n /// // Once the last sender is dropped, the channel is\n /// // closed and `recv_many` returns 0, capacity unchanged.\n /// drop(tx2);\n /// assert_eq!(0, rx.recv_many(&mut buffer, limit).await);\n /// assert_eq!(vec![\"first\", \"second\", \"third\", \"fourth\"], buffer);\n /// # }\n /// ```\n pub async fn recv_many(&mut self, buffer: &mut Vec, limit: usize) -> usize {\n use std::future::poll_fn;\n poll_fn(|cx| self.chan.recv_many(cx, buffer, limit)).await\n }\n\n /// Tries to receive the next value for this receiver.\n ///\n /// This method returns the [`Empty`] error if the channel is currently\n /// empty, but there are still outstanding [senders] or [permits].\n ///\n /// This method returns the [`Disconnected`] error if the channel is\n /// currently empty, and there are no outstanding [senders] or [permits].\n ///\n /// Unlike the [`poll_recv`] method, this method will never return an\n /// [`Empty`] error spuriously.\n ///\n /// [`Empty`]: crate::sync::mpsc::error::TryRecvError::Empty\n /// [`Disconnected`]: crate::sync::mpsc::error::TryRecvError::Disconnected\n /// [`poll_recv`]: Self::poll_recv\n /// [senders]: crate::sync::mpsc::Sender\n /// [permits]: crate::sync::mpsc::Permit\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n /// use tokio::sync::mpsc::error::TryRecvError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(100);\n ///\n /// tx.send(\"hello\").await.unwrap();\n ///\n /// assert_eq!(Ok(\"hello\"), rx.try_recv());\n /// assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n ///\n /// tx.send(\"hello\").await.unwrap();\n /// // Drop the last sender, closing the channel.\n /// drop(tx);\n ///\n /// assert_eq!(Ok(\"hello\"), rx.try_recv());\n /// assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n /// # }\n /// ```\n pub fn try_recv(&mut self) -> Result {\n self.chan.try_recv()\n }\n\n /// Blocking receive to call outside of asynchronous contexts.\n ///\n /// This method returns `None` if the channel has been closed and there are\n /// no remaining messages in the channel's buffer. This indicates that no\n /// further values can ever be received from this `Receiver`. The channel is\n /// closed when all senders have been dropped, or when [`close`] is called.\n ///\n /// If there are no messages in the channel's buffer, but the channel has\n /// not yet been closed, this method will block until a message is sent or\n /// the channel is closed.\n ///\n /// This method is intended for use cases where you are sending from\n /// asynchronous code to synchronous code, and will work even if the sender\n /// is not using [`blocking_send`] to send the message.\n ///\n /// Note that if [`close`] is called, but there are still outstanding\n /// [`Permits`] from before it was closed, the channel is not considered\n /// closed by `blocking_recv` until the permits are released.\n ///\n /// [`close`]: Self::close\n /// [`Permits`]: struct@crate::sync::mpsc::Permit\n /// [`blocking_send`]: fn@crate::sync::mpsc::Sender::blocking_send\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution\n /// context.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::thread;\n /// use tokio::runtime::Runtime;\n /// use tokio::sync::mpsc;\n ///\n /// fn main() {\n /// let (tx, mut rx) = mpsc::channel::(10);\n ///\n /// let sync_code = thread::spawn(move || {\n /// assert_eq!(Some(10), rx.blocking_recv());\n /// });\n ///\n /// Runtime::new()\n /// .unwrap()\n /// .block_on(async move {\n /// let _ = tx.send(10).await;\n /// });\n /// sync_code.join().unwrap()\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"recv_blocking\"))]\n pub fn blocking_recv(&mut self) -> Option {\n crate::future::block_on(self.recv())\n }\n\n /// Variant of [`Self::recv_many`] for blocking contexts.\n ///\n /// The same conditions as in [`Self::blocking_recv`] apply.\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"recv_many_blocking\"))]\n pub fn blocking_recv_many(&mut self, buffer: &mut Vec, limit: usize) -> usize {\n crate::future::block_on(self.recv_many(buffer, limit))\n }\n\n /// Closes the receiving half of a channel without dropping it.\n ///\n /// This prevents any further messages from being sent on the channel while\n /// still enabling the receiver to drain messages that are buffered. Any\n /// outstanding [`Permit`] values will still be able to send messages.\n ///\n /// To guarantee that no messages are dropped, after calling `close()`,\n /// `recv()` must be called until `None` is returned. If there are\n /// outstanding [`Permit`] or [`OwnedPermit`] values, the `recv` method will\n /// not return `None` until those are released.\n ///\n /// [`Permit`]: Permit\n /// [`OwnedPermit`]: OwnedPermit\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(20);\n ///\n /// tokio::spawn(async move {\n /// let mut i = 0;\n /// while let Ok(permit) = tx.reserve().await {\n /// permit.send(i);\n /// i += 1;\n /// }\n /// });\n ///\n /// rx.close();\n ///\n /// while let Some(msg) = rx.recv().await {\n /// println!(\"got {}\", msg);\n /// }\n ///\n /// // Channel closed and no messages are lost.\n /// # }\n /// ```\n pub fn close(&mut self) {\n self.chan.close();\n }\n\n /// Checks if a channel is closed.\n ///\n /// This method returns `true` if the channel has been closed. The channel is closed\n /// when all [`Sender`] have been dropped, or when [`Receiver::close`] is called.\n ///\n /// [`Sender`]: crate::sync::mpsc::Sender\n /// [`Receiver::close`]: crate::sync::mpsc::Receiver::close\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (_tx, mut rx) = mpsc::channel::<()>(10);\n /// assert!(!rx.is_closed());\n ///\n /// rx.close();\n ///\n /// assert!(rx.is_closed());\n /// # }\n /// ```\n pub fn is_closed(&self) -> bool {\n self.chan.is_closed()\n }\n\n /// Checks if a channel is empty.\n ///\n /// This method returns `true` if the channel has no messages.\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel(10);\n /// assert!(rx.is_empty());\n ///\n /// tx.send(0).await.unwrap();\n /// assert!(!rx.is_empty());\n /// # }\n ///\n /// ```\n pub fn is_empty(&self) -> bool {\n self.chan.is_empty()\n }\n\n /// Returns the number of messages in the channel.\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel(10);\n /// assert_eq!(0, rx.len());\n ///\n /// tx.send(0).await.unwrap();\n /// assert_eq!(1, rx.len());\n /// # }\n /// ```\n pub fn len(&self) -> usize {\n self.chan.len()\n }\n\n /// Returns the current capacity of the channel.\n ///\n /// The capacity goes down when the sender sends a value by calling [`Sender::send`] or by reserving\n /// capacity with [`Sender::reserve`]. The capacity goes up when values are received.\n /// This is distinct from [`max_capacity`], which always returns buffer capacity initially\n /// specified when calling [`channel`].\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel::<()>(5);\n ///\n /// assert_eq!(rx.capacity(), 5);\n ///\n /// // Making a reservation drops the capacity by one.\n /// let permit = tx.reserve().await.unwrap();\n /// assert_eq!(rx.capacity(), 4);\n /// assert_eq!(rx.len(), 0);\n ///\n /// // Sending and receiving a value increases the capacity by one.\n /// permit.send(());\n /// assert_eq!(rx.len(), 1);\n /// rx.recv().await.unwrap();\n /// assert_eq!(rx.capacity(), 5);\n ///\n /// // Directly sending a message drops the capacity by one.\n /// tx.send(()).await.unwrap();\n /// assert_eq!(rx.capacity(), 4);\n /// assert_eq!(rx.len(), 1);\n ///\n /// // Receiving the message increases the capacity by one.\n /// rx.recv().await.unwrap();\n /// assert_eq!(rx.capacity(), 5);\n /// assert_eq!(rx.len(), 0);\n /// # }\n /// ```\n /// [`capacity`]: Receiver::capacity\n /// [`max_capacity`]: Receiver::max_capacity\n pub fn capacity(&self) -> usize {\n self.chan.semaphore().semaphore.available_permits()\n }\n\n /// Returns the maximum buffer capacity of the channel.\n ///\n /// The maximum capacity is the buffer capacity initially specified when calling\n /// [`channel`]. This is distinct from [`capacity`], which returns the *current*\n /// available buffer capacity: as messages are sent and received, the value\n /// returned by [`capacity`] will go up or down, whereas the value\n /// returned by [`max_capacity`] will remain constant.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel::<()>(5);\n ///\n /// // both max capacity and capacity are the same at first\n /// assert_eq!(rx.max_capacity(), 5);\n /// assert_eq!(rx.capacity(), 5);\n ///\n /// // Making a reservation doesn't change the max capacity.\n /// let permit = tx.reserve().await.unwrap();\n /// assert_eq!(rx.max_capacity(), 5);\n /// // but drops the capacity by one\n /// assert_eq!(rx.capacity(), 4);\n /// # }\n /// ```\n /// [`capacity`]: Receiver::capacity\n /// [`max_capacity`]: Receiver::max_capacity\n pub fn max_capacity(&self) -> usize {\n self.chan.semaphore().bound\n }\n\n /// Polls to receive the next message on this channel.\n ///\n /// This method returns:\n ///\n /// * `Poll::Pending` if no messages are available but the channel is not\n /// closed, or if a spurious failure happens.\n /// * `Poll::Ready(Some(message))` if a message is available.\n /// * `Poll::Ready(None)` if the channel has been closed and all messages\n /// sent before it was closed have been received.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when a message is sent on any\n /// receiver, or when the channel is closed. Note that on multiple calls to\n /// `poll_recv` or `poll_recv_many`, only the `Waker` from the `Context`\n /// passed to the most recent call is scheduled to receive a wakeup.\n ///\n /// If this method returns `Poll::Pending` due to a spurious failure, then\n /// the `Waker` will be notified when the situation causing the spurious\n /// failure has been resolved. Note that receiving such a wakeup does not\n /// guarantee that the next call will succeed — it could fail with another\n /// spurious failure.\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.chan.recv(cx)\n }\n\n /// Polls to receive multiple messages on this channel, extending the provided buffer.\n ///\n /// This method returns:\n /// * `Poll::Pending` if no messages are available but the channel is not closed, or if a\n /// spurious failure happens.\n /// * `Poll::Ready(count)` where `count` is the number of messages successfully received and\n /// stored in `buffer`. This can be less than, or equal to, `limit`.\n /// * `Poll::Ready(0)` if `limit` is set to zero or when the channel is closed.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when a message is sent on any\n /// receiver, or when the channel is closed. Note that on multiple calls to\n /// `poll_recv` or `poll_recv_many`, only the `Waker` from the `Context`\n /// passed to the most recent call is scheduled to receive a wakeup.\n ///\n /// Note that this method does not guarantee that exactly `limit` messages\n /// are received. Rather, if at least one message is available, it returns\n /// as many messages as it can up to the given limit. This method returns\n /// zero only if the channel is closed (or if `limit` is zero).\n ///\n /// # Examples\n ///\n /// ```\n /// use std::task::{Context, Poll};\n /// use std::pin::Pin;\n /// use tokio::sync::mpsc;\n /// use futures::Future;\n ///\n /// struct MyReceiverFuture<'a> {\n /// receiver: mpsc::Receiver,\n /// buffer: &'a mut Vec,\n /// limit: usize,\n /// }\n ///\n /// impl<'a> Future for MyReceiverFuture<'a> {\n /// type Output = usize; // Number of messages received\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// let MyReceiverFuture { receiver, buffer, limit } = &mut *self;\n ///\n /// // Now `receiver` and `buffer` are mutable references, and `limit` is copied\n /// match receiver.poll_recv_many(cx, *buffer, *limit) {\n /// Poll::Pending => Poll::Pending,\n /// Poll::Ready(count) => Poll::Ready(count),\n /// }\n /// }\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel(32);\n /// let mut buffer = Vec::new();\n ///\n /// let my_receiver_future = MyReceiverFuture {\n /// receiver: rx,\n /// buffer: &mut buffer,\n /// limit: 3,\n /// };\n ///\n /// for i in 0..10 {\n /// tx.send(i).await.unwrap();\n /// }\n ///\n /// let count = my_receiver_future.await;\n /// assert_eq!(count, 3);\n /// assert_eq!(buffer, vec![0,1,2])\n /// # }\n /// ```\n pub fn poll_recv_many(\n &mut self,\n cx: &mut Context<'_>,\n buffer: &mut Vec,\n limit: usize,\n ) -> Poll {\n self.chan.recv_many(cx, buffer, limit)\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn sender_strong_count(&self) -> usize {\n self.chan.sender_strong_count()\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn sender_weak_count(&self) -> usize {\n self.chan.sender_weak_count()\n }\n}\n\nimpl fmt::Debug for Receiver {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Receiver\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n\nimpl Unpin for Receiver {}\n\nimpl Sender {\n pub(crate) fn new(chan: chan::Tx) -> Sender {\n Sender { chan }\n }\n\n /// Sends a value, waiting until there is capacity.\n ///\n /// A successful send occurs when it is determined that the other end of the\n /// channel has not hung up already. An unsuccessful send would be one where\n /// the corresponding receiver has already been closed. Note that a return\n /// value of `Err` means that the data will never be received, but a return\n /// value of `Ok` does not mean that the data will be received. It is\n /// possible for the corresponding receiver to hang up immediately after\n /// this function returns `Ok`.\n ///\n /// # Errors\n ///\n /// If the receive half of the channel is closed, either due to [`close`]\n /// being called or the [`Receiver`] handle dropping, the function returns\n /// an error. The error includes the value passed to `send`.\n ///\n /// [`close`]: Receiver::close\n /// [`Receiver`]: Receiver\n ///\n /// # Cancel safety\n ///\n /// If `send` is used as the event in a [`tokio::select!`](crate::select)\n /// statement and some other branch completes first, then it is guaranteed\n /// that the message was not sent. **However, in that case, the message\n /// is dropped and will be lost.**\n ///\n /// To avoid losing messages, use [`reserve`](Self::reserve) to reserve\n /// capacity, then use the returned [`Permit`] to send the message.\n ///\n /// This channel uses a queue to ensure that calls to `send` and `reserve`\n /// complete in the order they were requested. Cancelling a call to\n /// `send` makes you lose your place in the queue.\n ///\n /// # Examples\n ///\n /// In the following example, each call to `send` will block until the\n /// previously sent value was received.\n ///\n /// ```rust\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// tokio::spawn(async move {\n /// for i in 0..10 {\n /// if let Err(_) = tx.send(i).await {\n /// println!(\"receiver dropped\");\n /// return;\n /// }\n /// }\n /// });\n ///\n /// while let Some(i) = rx.recv().await {\n /// println!(\"got = {}\", i);\n /// }\n /// # }\n /// ```\n pub async fn send(&self, value: T) -> Result<(), SendError> {\n match self.reserve().await {\n Ok(permit) => {\n permit.send(value);\n Ok(())\n }\n Err(_) => Err(SendError(value)),\n }\n }\n\n /// Completes when the receiver has dropped.\n ///\n /// This allows the producers to get notified when interest in the produced\n /// values is canceled and immediately stop doing work.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once the channel is closed, it stays closed\n /// forever and all future calls to `closed` will return immediately.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx1, rx) = mpsc::channel::<()>(1);\n /// let tx2 = tx1.clone();\n /// let tx3 = tx1.clone();\n /// let tx4 = tx1.clone();\n /// let tx5 = tx1.clone();\n /// tokio::spawn(async move {\n /// drop(rx);\n /// });\n ///\n /// futures::join!(\n /// tx1.closed(),\n /// tx2.closed(),\n /// tx3.closed(),\n /// tx4.closed(),\n /// tx5.closed()\n /// );\n /// println!(\"Receiver dropped\");\n /// # }\n /// ```\n pub async fn closed(&self) {\n self.chan.closed().await;\n }\n\n /// Attempts to immediately send a message on this `Sender`.\n ///\n /// This method differs from [`send`] by returning immediately if the channel's\n /// buffer is full or no receiver is waiting to acquire some data. Compared\n /// with [`send`], this function has two failure cases instead of one (one for\n /// disconnection, one for a full buffer).\n ///\n /// # Errors\n ///\n /// If the channel capacity has been reached, i.e., the channel has `n`\n /// buffered values where `n` is the argument passed to [`channel`], then an\n /// error is returned.\n ///\n /// If the receive half of the channel is closed, either due to [`close`]\n /// being called or the [`Receiver`] handle dropping, the function returns\n /// an error. The error includes the value passed to `send`.\n ///\n /// [`send`]: Sender::send\n /// [`channel`]: channel\n /// [`close`]: Receiver::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// // Create a channel with buffer size 1\n /// let (tx1, mut rx) = mpsc::channel(1);\n /// let tx2 = tx1.clone();\n ///\n /// tokio::spawn(async move {\n /// tx1.send(1).await.unwrap();\n /// tx1.send(2).await.unwrap();\n /// // task waits until the receiver receives a value.\n /// });\n ///\n /// tokio::spawn(async move {\n /// // This will return an error and send\n /// // no message if the buffer is full\n /// let _ = tx2.try_send(3);\n /// });\n ///\n /// let mut msg;\n /// msg = rx.recv().await.unwrap();\n /// println!(\"message {} received\", msg);\n ///\n /// msg = rx.recv().await.unwrap();\n /// println!(\"message {} received\", msg);\n ///\n /// // Third message may have never been sent\n /// match rx.recv().await {\n /// Some(msg) => println!(\"message {} received\", msg),\n /// None => println!(\"the third message was never sent\"),\n /// }\n /// # }\n /// ```\n pub fn try_send(&self, message: T) -> Result<(), TrySendError> {\n match self.chan.semaphore().semaphore.try_acquire(1) {\n Ok(()) => {}\n Err(TryAcquireError::Closed) => return Err(TrySendError::Closed(message)),\n Err(TryAcquireError::NoPermits) => return Err(TrySendError::Full(message)),\n }\n\n // Send the message\n self.chan.send(message);\n Ok(())\n }\n\n /// Sends a value, waiting until there is capacity, but only for a limited time.\n ///\n /// Shares the same success and error conditions as [`send`], adding one more\n /// condition for an unsuccessful send, which is when the provided timeout has\n /// elapsed, and there is no capacity available.\n ///\n /// [`send`]: Sender::send\n ///\n /// # Errors\n ///\n /// If the receive half of the channel is closed, either due to [`close`]\n /// being called or the [`Receiver`] having been dropped,\n /// the function returns an error. The error includes the value passed to `send`.\n ///\n /// [`close`]: Receiver::close\n /// [`Receiver`]: Receiver\n ///\n /// # Panics\n ///\n /// This function panics if it is called outside the context of a Tokio\n /// runtime [with time enabled](crate::runtime::Builder::enable_time).\n ///\n /// # Examples\n ///\n /// In the following example, each call to `send_timeout` will block until the\n /// previously sent value was received, unless the timeout has elapsed.\n ///\n /// ```rust\n /// use tokio::sync::mpsc;\n /// use tokio::time::{sleep, Duration};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// tokio::spawn(async move {\n /// for i in 0..10 {\n /// if let Err(e) = tx.send_timeout(i, Duration::from_millis(100)).await {\n /// println!(\"send error: #{:?}\", e);\n /// return;\n /// }\n /// }\n /// });\n ///\n /// while let Some(i) = rx.recv().await {\n /// println!(\"got = {}\", i);\n /// sleep(Duration::from_millis(200)).await;\n /// }\n /// # }\n /// ```\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n pub async fn send_timeout(\n &self,\n value: T,\n timeout: Duration,\n ) -> Result<(), SendTimeoutError> {\n let permit = match crate::time::timeout(timeout, self.reserve()).await {\n Err(_) => {\n return Err(SendTimeoutError::Timeout(value));\n }\n Ok(Err(_)) => {\n return Err(SendTimeoutError::Closed(value));\n }\n Ok(Ok(permit)) => permit,\n };\n\n permit.send(value);\n Ok(())\n }\n\n /// Blocking send to call outside of asynchronous contexts.\n ///\n /// This method is intended for use cases where you are sending from\n /// synchronous code to asynchronous code, and will work even if the\n /// receiver is not using [`blocking_recv`] to receive the message.\n ///\n /// [`blocking_recv`]: fn@crate::sync::mpsc::Receiver::blocking_recv\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution\n /// context.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::thread;\n /// use tokio::runtime::Runtime;\n /// use tokio::sync::mpsc;\n ///\n /// fn main() {\n /// let (tx, mut rx) = mpsc::channel::(1);\n ///\n /// let sync_code = thread::spawn(move || {\n /// tx.blocking_send(10).unwrap();\n /// });\n ///\n /// Runtime::new().unwrap().block_on(async move {\n /// assert_eq!(Some(10), rx.recv().await);\n /// });\n /// sync_code.join().unwrap()\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"send_blocking\"))]\n pub fn blocking_send(&self, value: T) -> Result<(), SendError> {\n crate::future::block_on(self.send(value))\n }\n\n /// Checks if the channel has been closed. This happens when the\n /// [`Receiver`] is dropped, or when the [`Receiver::close`] method is\n /// called.\n ///\n /// [`Receiver`]: crate::sync::mpsc::Receiver\n /// [`Receiver::close`]: crate::sync::mpsc::Receiver::close\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::mpsc::channel::<()>(42);\n /// assert!(!tx.is_closed());\n ///\n /// let tx2 = tx.clone();\n /// assert!(!tx2.is_closed());\n ///\n /// drop(rx);\n /// assert!(tx.is_closed());\n /// assert!(tx2.is_closed());\n /// ```\n pub fn is_closed(&self) -> bool {\n self.chan.is_closed()\n }\n\n /// Waits for channel capacity. Once capacity to send one message is\n /// available, it is reserved for the caller.\n ///\n /// If the channel is full, the function waits for the number of unreceived\n /// messages to become less than the channel capacity. Capacity to send one\n /// message is reserved for the caller. A [`Permit`] is returned to track\n /// the reserved capacity. The [`send`] function on [`Permit`] consumes the\n /// reserved capacity.\n ///\n /// Dropping [`Permit`] without sending a message releases the capacity back\n /// to the channel.\n ///\n /// [`Permit`]: Permit\n /// [`send`]: Permit::send\n ///\n /// # Cancel safety\n ///\n /// This channel uses a queue to ensure that calls to `send` and `reserve`\n /// complete in the order they were requested. Cancelling a call to\n /// `reserve` makes you lose your place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity\n /// let permit = tx.reserve().await.unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Sending on the permit succeeds\n /// permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n /// # }\n /// ```\n pub async fn reserve(&self) -> Result, SendError<()>> {\n self.reserve_inner(1).await?;\n Ok(Permit { chan: &self.chan })\n }\n\n /// Waits for channel capacity. Once capacity to send `n` messages is\n /// available, it is reserved for the caller.\n ///\n /// If the channel is full or if there are fewer than `n` permits available, the function waits\n /// for the number of unreceived messages to become `n` less than the channel capacity.\n /// Capacity to send `n` message is then reserved for the caller.\n ///\n /// A [`PermitIterator`] is returned to track the reserved capacity.\n /// You can call this [`Iterator`] until it is exhausted to\n /// get a [`Permit`] and then call [`Permit::send`]. This function is similar to\n /// [`try_reserve_many`] except it waits for the slots to become available.\n ///\n /// If the channel is closed, the function returns a [`SendError`].\n ///\n /// Dropping [`PermitIterator`] without consuming it entirely releases the remaining\n /// permits back to the channel.\n ///\n /// [`PermitIterator`]: PermitIterator\n /// [`Permit`]: Permit\n /// [`send`]: Permit::send\n /// [`try_reserve_many`]: Sender::try_reserve_many\n ///\n /// # Cancel safety\n ///\n /// This channel uses a queue to ensure that calls to `send` and `reserve_many`\n /// complete in the order they were requested. Cancelling a call to\n /// `reserve_many` makes you lose your place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(2);\n ///\n /// // Reserve capacity\n /// let mut permit = tx.reserve_many(2).await.unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Sending with the permit iterator succeeds\n /// permit.next().unwrap().send(456);\n /// permit.next().unwrap().send(457);\n ///\n /// // The iterator should now be exhausted\n /// assert!(permit.next().is_none());\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n /// assert_eq!(rx.recv().await.unwrap(), 457);\n /// # }\n /// ```\n pub async fn reserve_many(&self, n: usize) -> Result, SendError<()>> {\n self.reserve_inner(n).await?;\n Ok(PermitIterator {\n chan: &self.chan,\n n,\n })\n }\n\n /// Waits for channel capacity, moving the `Sender` and returning an owned\n /// permit. Once capacity to send one message is available, it is reserved\n /// for the caller.\n ///\n /// This moves the sender _by value_, and returns an owned permit that can\n /// be used to send a message into the channel. Unlike [`Sender::reserve`],\n /// this method may be used in cases where the permit must be valid for the\n /// `'static` lifetime. `Sender`s may be cloned cheaply (`Sender::clone` is\n /// essentially a reference count increment, comparable to [`Arc::clone`]),\n /// so when multiple [`OwnedPermit`]s are needed or the `Sender` cannot be\n /// moved, it can be cloned prior to calling `reserve_owned`.\n ///\n /// If the channel is full, the function waits for the number of unreceived\n /// messages to become less than the channel capacity. Capacity to send one\n /// message is reserved for the caller. An [`OwnedPermit`] is returned to\n /// track the reserved capacity. The [`send`] function on [`OwnedPermit`]\n /// consumes the reserved capacity.\n ///\n /// Dropping the [`OwnedPermit`] without sending a message releases the\n /// capacity back to the channel.\n ///\n /// # Cancel safety\n ///\n /// This channel uses a queue to ensure that calls to `send` and `reserve`\n /// complete in the order they were requested. Cancelling a call to\n /// `reserve_owned` makes you lose your place in the queue.\n ///\n /// # Examples\n /// Sending a message using an [`OwnedPermit`]:\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity, moving the sender.\n /// let permit = tx.reserve_owned().await.unwrap();\n ///\n /// // Send a message, consuming the permit and returning\n /// // the moved sender.\n /// let tx = permit.send(123);\n ///\n /// // The value sent on the permit is received.\n /// assert_eq!(rx.recv().await.unwrap(), 123);\n ///\n /// // The sender can now be used again.\n /// tx.send(456).await.unwrap();\n /// # }\n /// ```\n ///\n /// When multiple [`OwnedPermit`]s are needed, or the sender cannot be moved\n /// by value, it can be inexpensively cloned before calling `reserve_owned`:\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Clone the sender and reserve capacity.\n /// let permit = tx.clone().reserve_owned().await.unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Sending on the permit succeeds.\n /// permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n /// # }\n /// ```\n ///\n /// [`Sender::reserve`]: Sender::reserve\n /// [`OwnedPermit`]: OwnedPermit\n /// [`send`]: OwnedPermit::send\n /// [`Arc::clone`]: std::sync::Arc::clone\n pub async fn reserve_owned(self) -> Result, SendError<()>> {\n self.reserve_inner(1).await?;\n Ok(OwnedPermit {\n chan: Some(self.chan),\n })\n }\n\n async fn reserve_inner(&self, n: usize) -> Result<(), SendError<()>> {\n crate::trace::async_trace_leaf().await;\n\n if n > self.max_capacity() {\n return Err(SendError(()));\n }\n match self.chan.semaphore().semaphore.acquire(n).await {\n Ok(()) => Ok(()),\n Err(_) => Err(SendError(())),\n }\n }\n\n /// Tries to acquire a slot in the channel without waiting for the slot to become\n /// available.\n ///\n /// If the channel is full this function will return [`TrySendError`], otherwise\n /// if there is a slot available it will return a [`Permit`] that will then allow you\n /// to [`send`] on the channel with a guaranteed slot. This function is similar to\n /// [`reserve`] except it does not await for the slot to become available.\n ///\n /// Dropping [`Permit`] without sending a message releases the capacity back\n /// to the channel.\n ///\n /// [`Permit`]: Permit\n /// [`send`]: Permit::send\n /// [`reserve`]: Sender::reserve\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity\n /// let permit = tx.try_reserve().unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Trying to reserve an additional slot on the `tx` will\n /// // fail because there is no capacity.\n /// assert!(tx.try_reserve().is_err());\n ///\n /// // Sending on the permit succeeds\n /// permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n ///\n /// # }\n /// ```\n pub fn try_reserve(&self) -> Result, TrySendError<()>> {\n match self.chan.semaphore().semaphore.try_acquire(1) {\n Ok(()) => {}\n Err(TryAcquireError::Closed) => return Err(TrySendError::Closed(())),\n Err(TryAcquireError::NoPermits) => return Err(TrySendError::Full(())),\n }\n\n Ok(Permit { chan: &self.chan })\n }\n\n /// Tries to acquire `n` slots in the channel without waiting for the slot to become\n /// available.\n ///\n /// A [`PermitIterator`] is returned to track the reserved capacity.\n /// You can call this [`Iterator`] until it is exhausted to\n /// get a [`Permit`] and then call [`Permit::send`]. This function is similar to\n /// [`reserve_many`] except it does not await for the slots to become available.\n ///\n /// If there are fewer than `n` permits available on the channel, then\n /// this function will return a [`TrySendError::Full`]. If the channel is closed\n /// this function will return a [`TrySendError::Closed`].\n ///\n /// Dropping [`PermitIterator`] without consuming it entirely releases the remaining\n /// permits back to the channel.\n ///\n /// [`PermitIterator`]: PermitIterator\n /// [`send`]: Permit::send\n /// [`reserve_many`]: Sender::reserve_many\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(2);\n ///\n /// // Reserve capacity\n /// let mut permit = tx.try_reserve_many(2).unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Trying to reserve an additional slot on the `tx` will\n /// // fail because there is no capacity.\n /// assert!(tx.try_reserve().is_err());\n ///\n /// // Sending with the permit iterator succeeds\n /// permit.next().unwrap().send(456);\n /// permit.next().unwrap().send(457);\n ///\n /// // The iterator should now be exhausted\n /// assert!(permit.next().is_none());\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n /// assert_eq!(rx.recv().await.unwrap(), 457);\n ///\n /// // Trying to call try_reserve_many with 0 will return an empty iterator\n /// let mut permit = tx.try_reserve_many(0).unwrap();\n /// assert!(permit.next().is_none());\n ///\n /// // Trying to call try_reserve_many with a number greater than the channel\n /// // capacity will return an error\n /// let permit = tx.try_reserve_many(3);\n /// assert!(permit.is_err());\n ///\n /// // Trying to call try_reserve_many on a closed channel will return an error\n /// drop(rx);\n /// let permit = tx.try_reserve_many(1);\n /// assert!(permit.is_err());\n ///\n /// let permit = tx.try_reserve_many(0);\n /// assert!(permit.is_err());\n /// # }\n /// ```\n pub fn try_reserve_many(&self, n: usize) -> Result, TrySendError<()>> {\n if n > self.max_capacity() {\n return Err(TrySendError::Full(()));\n }\n\n match self.chan.semaphore().semaphore.try_acquire(n) {\n Ok(()) => {}\n Err(TryAcquireError::Closed) => return Err(TrySendError::Closed(())),\n Err(TryAcquireError::NoPermits) => return Err(TrySendError::Full(())),\n }\n\n Ok(PermitIterator {\n chan: &self.chan,\n n,\n })\n }\n\n /// Tries to acquire a slot in the channel without waiting for the slot to become\n /// available, returning an owned permit.\n ///\n /// This moves the sender _by value_, and returns an owned permit that can\n /// be used to send a message into the channel. Unlike [`Sender::try_reserve`],\n /// this method may be used in cases where the permit must be valid for the\n /// `'static` lifetime. `Sender`s may be cloned cheaply (`Sender::clone` is\n /// essentially a reference count increment, comparable to [`Arc::clone`]),\n /// so when multiple [`OwnedPermit`]s are needed or the `Sender` cannot be\n /// moved, it can be cloned prior to calling `try_reserve_owned`.\n ///\n /// If the channel is full this function will return a [`TrySendError`].\n /// Since the sender is taken by value, the `TrySendError` returned in this\n /// case contains the sender, so that it may be used again. Otherwise, if\n /// there is a slot available, this method will return an [`OwnedPermit`]\n /// that can then be used to [`send`] on the channel with a guaranteed slot.\n /// This function is similar to [`reserve_owned`] except it does not await\n /// for the slot to become available.\n ///\n /// Dropping the [`OwnedPermit`] without sending a message releases the capacity back\n /// to the channel.\n ///\n /// [`OwnedPermit`]: OwnedPermit\n /// [`send`]: OwnedPermit::send\n /// [`reserve_owned`]: Sender::reserve_owned\n /// [`Arc::clone`]: std::sync::Arc::clone\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity\n /// let permit = tx.clone().try_reserve_owned().unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Trying to reserve an additional slot on the `tx` will\n /// // fail because there is no capacity.\n /// assert!(tx.try_reserve().is_err());\n ///\n /// // Sending on the permit succeeds\n /// permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n ///\n /// # }\n /// ```\n pub fn try_reserve_owned(self) -> Result, TrySendError> {\n match self.chan.semaphore().semaphore.try_acquire(1) {\n Ok(()) => {}\n Err(TryAcquireError::Closed) => return Err(TrySendError::Closed(self)),\n Err(TryAcquireError::NoPermits) => return Err(TrySendError::Full(self)),\n }\n\n Ok(OwnedPermit {\n chan: Some(self.chan),\n })\n }\n\n /// Returns `true` if senders belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::mpsc::channel::<()>(1);\n /// let tx2 = tx.clone();\n /// assert!(tx.same_channel(&tx2));\n ///\n /// let (tx3, rx3) = tokio::sync::mpsc::channel::<()>(1);\n /// assert!(!tx3.same_channel(&tx2));\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n self.chan.same_channel(&other.chan)\n }\n\n /// Returns the current capacity of the channel.\n ///\n /// The capacity goes down when sending a value by calling [`send`] or by reserving capacity\n /// with [`reserve`]. The capacity goes up when values are received by the [`Receiver`].\n /// This is distinct from [`max_capacity`], which always returns buffer capacity initially\n /// specified when calling [`channel`]\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel::<()>(5);\n ///\n /// assert_eq!(tx.capacity(), 5);\n ///\n /// // Making a reservation drops the capacity by one.\n /// let permit = tx.reserve().await.unwrap();\n /// assert_eq!(tx.capacity(), 4);\n ///\n /// // Sending and receiving a value increases the capacity by one.\n /// permit.send(());\n /// rx.recv().await.unwrap();\n /// assert_eq!(tx.capacity(), 5);\n /// # }\n /// ```\n ///\n /// [`send`]: Sender::send\n /// [`reserve`]: Sender::reserve\n /// [`channel`]: channel\n /// [`max_capacity`]: Sender::max_capacity\n pub fn capacity(&self) -> usize {\n self.chan.semaphore().semaphore.available_permits()\n }\n\n /// Converts the `Sender` to a [`WeakSender`] that does not count\n /// towards RAII semantics, i.e. if all `Sender` instances of the\n /// channel were dropped and only `WeakSender` instances remain,\n /// the channel is closed.\n #[must_use = \"Downgrade creates a WeakSender without destroying the original non-weak sender.\"]\n pub fn downgrade(&self) -> WeakSender {\n WeakSender {\n chan: self.chan.downgrade(),\n }\n }\n\n /// Returns the maximum buffer capacity of the channel.\n ///\n /// The maximum capacity is the buffer capacity initially specified when calling\n /// [`channel`]. This is distinct from [`capacity`], which returns the *current*\n /// available buffer capacity: as messages are sent and received, the\n /// value returned by [`capacity`] will go up or down, whereas the value\n /// returned by [`max_capacity`] will remain constant.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx) = mpsc::channel::<()>(5);\n ///\n /// // both max capacity and capacity are the same at first\n /// assert_eq!(tx.max_capacity(), 5);\n /// assert_eq!(tx.capacity(), 5);\n ///\n /// // Making a reservation doesn't change the max capacity.\n /// let permit = tx.reserve().await.unwrap();\n /// assert_eq!(tx.max_capacity(), 5);\n /// // but drops the capacity by one\n /// assert_eq!(tx.capacity(), 4);\n /// # }\n /// ```\n ///\n /// [`channel`]: channel\n /// [`max_capacity`]: Sender::max_capacity\n /// [`capacity`]: Sender::capacity\n pub fn max_capacity(&self) -> usize {\n self.chan.semaphore().bound\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn strong_count(&self) -> usize {\n self.chan.strong_count()\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.chan.weak_count()\n }\n}\n\nimpl Clone for Sender {\n fn clone(&self) -> Self {\n Sender {\n chan: self.chan.clone(),\n }\n }\n}\n\nimpl fmt::Debug for Sender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Sender\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n\nimpl Clone for WeakSender {\n fn clone(&self) -> Self {\n self.chan.increment_weak_count();\n\n WeakSender {\n chan: self.chan.clone(),\n }\n }\n}\n\nimpl Drop for WeakSender {\n fn drop(&mut self) {\n self.chan.decrement_weak_count();\n }\n}\n\nimpl WeakSender {\n /// Tries to convert a `WeakSender` into a [`Sender`]. This will return `Some`\n /// if there are other `Sender` instances alive and the channel wasn't\n /// previously dropped, otherwise `None` is returned.\n pub fn upgrade(&self) -> Option> {\n chan::Tx::upgrade(self.chan.clone()).map(Sender::new)\n }\n\n /// Returns the number of [`Sender`] handles.\n pub fn strong_count(&self) -> usize {\n self.chan.strong_count()\n }\n\n /// Returns the number of [`WeakSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.chan.weak_count()\n }\n}\n\nimpl fmt::Debug for WeakSender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"WeakSender\").finish()\n }\n}\n\n// ===== impl Permit =====\n\nimpl Permit<'_, T> {\n /// Sends a value using the reserved capacity.\n ///\n /// Capacity for the message has already been reserved. The message is sent\n /// to the receiver and the permit is consumed. The operation will succeed\n /// even if the receiver half has been closed. See [`Receiver::close`] for\n /// more details on performing a clean shutdown.\n ///\n /// [`Receiver::close`]: Receiver::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity\n /// let permit = tx.reserve().await.unwrap();\n ///\n /// // Trying to send directly on the `tx` will fail due to no\n /// // available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Send a message on the permit\n /// permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n /// # }\n /// ```\n pub fn send(self, value: T) {\n use std::mem;\n\n self.chan.send(value);\n\n // Avoid the drop logic\n mem::forget(self);\n }\n}\n\nimpl Drop for Permit<'_, T> {\n fn drop(&mut self) {\n use chan::Semaphore;\n\n let semaphore = self.chan.semaphore();\n\n // Add the permit back to the semaphore\n semaphore.add_permit();\n\n // If this is the last sender for this channel, wake the receiver so\n // that it can be notified that the channel is closed.\n if semaphore.is_closed() && semaphore.is_idle() {\n self.chan.wake_rx();\n }\n }\n}\n\nimpl fmt::Debug for Permit<'_, T> {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Permit\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n\n// ===== impl PermitIterator =====\n\nimpl<'a, T> Iterator for PermitIterator<'a, T> {\n type Item = Permit<'a, T>;\n\n fn next(&mut self) -> Option {\n if self.n == 0 {\n return None;\n }\n\n self.n -= 1;\n Some(Permit { chan: self.chan })\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let n = self.n;\n (n, Some(n))\n }\n}\nimpl ExactSizeIterator for PermitIterator<'_, T> {}\nimpl std::iter::FusedIterator for PermitIterator<'_, T> {}\n\nimpl Drop for PermitIterator<'_, T> {\n fn drop(&mut self) {\n use chan::Semaphore;\n\n if self.n == 0 {\n return;\n }\n\n let semaphore = self.chan.semaphore();\n\n // Add the remaining permits back to the semaphore\n semaphore.add_permits(self.n);\n\n // If this is the last sender for this channel, wake the receiver so\n // that it can be notified that the channel is closed.\n if semaphore.is_closed() && semaphore.is_idle() {\n self.chan.wake_rx();\n }\n }\n}\n\nimpl fmt::Debug for PermitIterator<'_, T> {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"PermitIterator\")\n .field(\"chan\", &self.chan)\n .field(\"capacity\", &self.n)\n .finish()\n }\n}\n\n// ===== impl Permit =====\n\nimpl OwnedPermit {\n /// Sends a value using the reserved capacity.\n ///\n /// Capacity for the message has already been reserved. The message is sent\n /// to the receiver and the permit is consumed. The operation will succeed\n /// even if the receiver half has been closed. See [`Receiver::close`] for\n /// more details on performing a clean shutdown.\n ///\n /// Unlike [`Permit::send`], this method returns the [`Sender`] from which\n /// the `OwnedPermit` was reserved.\n ///\n /// [`Receiver::close`]: Receiver::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::channel(1);\n ///\n /// // Reserve capacity\n /// let permit = tx.reserve_owned().await.unwrap();\n ///\n /// // Send a message on the permit, returning the sender.\n /// let tx = permit.send(456);\n ///\n /// // The value sent on the permit is received\n /// assert_eq!(rx.recv().await.unwrap(), 456);\n ///\n /// // We may now reuse `tx` to send another message.\n /// tx.send(789).await.unwrap();\n /// # }\n /// ```\n pub fn send(mut self, value: T) -> Sender {\n let chan = self.chan.take().unwrap_or_else(|| {\n unreachable!(\"OwnedPermit channel is only taken when the permit is moved\")\n });\n chan.send(value);\n\n Sender { chan }\n }\n\n /// Releases the reserved capacity *without* sending a message, returning the\n /// [`Sender`].\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel(1);\n ///\n /// // Clone the sender and reserve capacity\n /// let permit = tx.clone().reserve_owned().await.unwrap();\n ///\n /// // Trying to send on the original `tx` will fail, since the `permit`\n /// // has reserved all the available capacity.\n /// assert!(tx.try_send(123).is_err());\n ///\n /// // Release the permit without sending a message, returning the clone\n /// // of the sender.\n /// let tx2 = permit.release();\n ///\n /// // We may now reuse `tx` to send another message.\n /// tx.send(789).await.unwrap();\n /// # drop(rx); drop(tx2);\n /// # }\n /// ```\n ///\n /// [`Sender`]: Sender\n pub fn release(mut self) -> Sender {\n use chan::Semaphore;\n\n let chan = self.chan.take().unwrap_or_else(|| {\n unreachable!(\"OwnedPermit channel is only taken when the permit is moved\")\n });\n\n // Add the permit back to the semaphore\n chan.semaphore().add_permit();\n Sender { chan }\n }\n\n /// Returns `true` if permits belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel::<()>(2);\n ///\n /// let permit1 = tx.clone().reserve_owned().await.unwrap();\n /// let permit2 = tx.clone().reserve_owned().await.unwrap();\n /// assert!(permit1.same_channel(&permit2));\n ///\n /// let (tx2, rx2) = mpsc::channel::<()>(1);\n ///\n /// let permit3 = tx2.clone().reserve_owned().await.unwrap();\n /// assert!(!permit3.same_channel(&permit2));\n /// # }\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n self.chan\n .as_ref()\n .zip(other.chan.as_ref())\n .is_some_and(|(a, b)| a.same_channel(b))\n }\n\n /// Returns `true` if this permit belongs to the same channel as the given [`Sender`].\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::channel::<()>(1);\n ///\n /// let permit = tx.clone().reserve_owned().await.unwrap();\n /// assert!(permit.same_channel_as_sender(&tx));\n ///\n /// let (tx2, rx2) = mpsc::channel::<()>(1);\n /// assert!(!permit.same_channel_as_sender(&tx2));\n /// # }\n /// ```\n pub fn same_channel_as_sender(&self, sender: &Sender) -> bool {\n self.chan\n .as_ref()\n .is_some_and(|chan| chan.same_channel(&sender.chan))\n }\n}\n\nimpl Drop for OwnedPermit {\n fn drop(&mut self) {\n use chan::Semaphore;\n\n // Are we still holding onto the sender?\n if let Some(chan) = self.chan.take() {\n let semaphore = chan.semaphore();\n\n // Add the permit back to the semaphore\n semaphore.add_permit();\n\n // If this `OwnedPermit` is holding the last sender for this\n // channel, wake the receiver so that it can be notified that the\n // channel is closed.\n if semaphore.is_closed() && semaphore.is_idle() {\n chan.wake_rx();\n }\n }\n\n // Otherwise, do nothing.\n }\n}\n\nimpl fmt::Debug for OwnedPermit {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"OwnedPermit\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n" }, { "path": "tokio/src/sync/mpsc/chan.rs", "content": "use crate::loom::cell::UnsafeCell;\nuse crate::loom::future::AtomicWaker;\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::Arc;\nuse crate::runtime::park::CachedParkThread;\nuse crate::sync::mpsc::error::TryRecvError;\nuse crate::sync::mpsc::{bounded, list, unbounded};\nuse crate::sync::notify::Notify;\nuse crate::util::cacheline::CachePadded;\n\nuse std::fmt;\nuse std::panic;\nuse std::process;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release};\nuse std::task::Poll::{Pending, Ready};\nuse std::task::{ready, Context, Poll};\n\n/// Channel sender.\npub(crate) struct Tx {\n inner: Arc>,\n}\n\nimpl fmt::Debug for Tx {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Tx\").field(\"inner\", &self.inner).finish()\n }\n}\n\n/// Channel receiver.\npub(crate) struct Rx {\n inner: Arc>,\n}\n\nimpl fmt::Debug for Rx {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Rx\").field(\"inner\", &self.inner).finish()\n }\n}\n\npub(crate) trait Semaphore {\n fn is_idle(&self) -> bool;\n\n fn add_permit(&self);\n\n fn add_permits(&self, n: usize);\n\n fn close(&self);\n\n fn is_closed(&self) -> bool;\n}\n\npub(super) struct Chan {\n /// Handle to the push half of the lock-free list.\n tx: CachePadded>,\n\n /// Receiver waker. Notified when a value is pushed into the channel.\n rx_waker: CachePadded,\n\n /// Notifies all tasks listening for the receiver being dropped.\n notify_rx_closed: Notify,\n\n /// Coordinates access to channel's capacity.\n semaphore: S,\n\n /// Tracks the number of outstanding sender handles.\n ///\n /// When this drops to zero, the send half of the channel is closed.\n tx_count: AtomicUsize,\n\n /// Tracks the number of outstanding weak sender handles.\n tx_weak_count: AtomicUsize,\n\n /// Only accessed by `Rx` handle.\n rx_fields: UnsafeCell>,\n}\n\nimpl fmt::Debug for Chan\nwhere\n S: fmt::Debug,\n{\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Chan\")\n .field(\"tx\", &*self.tx)\n .field(\"semaphore\", &self.semaphore)\n .field(\"rx_waker\", &*self.rx_waker)\n .field(\"tx_count\", &self.tx_count)\n .field(\"rx_fields\", &\"...\")\n .finish()\n }\n}\n\n/// Fields only accessed by `Rx` handle.\nstruct RxFields {\n /// Channel receiver. This field is only accessed by the `Receiver` type.\n list: list::Rx,\n\n /// `true` if `Rx::close` is called.\n rx_closed: bool,\n}\n\nimpl fmt::Debug for RxFields {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"RxFields\")\n .field(\"list\", &self.list)\n .field(\"rx_closed\", &self.rx_closed)\n .finish()\n }\n}\n\nunsafe impl Send for Chan {}\nunsafe impl Sync for Chan {}\nimpl panic::RefUnwindSafe for Chan {}\nimpl panic::UnwindSafe for Chan {}\n\npub(crate) fn channel(semaphore: S) -> (Tx, Rx) {\n let (tx, rx) = list::channel();\n\n let chan = Arc::new(Chan {\n notify_rx_closed: Notify::new(),\n tx: CachePadded::new(tx),\n semaphore,\n rx_waker: CachePadded::new(AtomicWaker::new()),\n tx_count: AtomicUsize::new(1),\n tx_weak_count: AtomicUsize::new(0),\n rx_fields: UnsafeCell::new(RxFields {\n list: rx,\n rx_closed: false,\n }),\n });\n\n (Tx::new(chan.clone()), Rx::new(chan))\n}\n\n// ===== impl Tx =====\n\nimpl Tx {\n fn new(chan: Arc>) -> Tx {\n Tx { inner: chan }\n }\n\n pub(super) fn strong_count(&self) -> usize {\n self.inner.tx_count.load(Acquire)\n }\n\n pub(super) fn weak_count(&self) -> usize {\n self.inner.tx_weak_count.load(Relaxed)\n }\n\n pub(super) fn downgrade(&self) -> Arc> {\n self.inner.increment_weak_count();\n\n self.inner.clone()\n }\n\n // Returns the upgraded channel or None if the upgrade failed.\n pub(super) fn upgrade(chan: Arc>) -> Option {\n let mut tx_count = chan.tx_count.load(Acquire);\n\n loop {\n if tx_count == 0 {\n // channel is closed\n return None;\n }\n\n match chan\n .tx_count\n .compare_exchange_weak(tx_count, tx_count + 1, AcqRel, Acquire)\n {\n Ok(_) => return Some(Tx { inner: chan }),\n Err(prev_count) => tx_count = prev_count,\n }\n }\n }\n\n pub(super) fn semaphore(&self) -> &S {\n &self.inner.semaphore\n }\n\n /// Send a message and notify the receiver.\n pub(crate) fn send(&self, value: T) {\n self.inner.send(value);\n }\n\n /// Wake the receive half\n pub(crate) fn wake_rx(&self) {\n self.inner.rx_waker.wake();\n }\n\n /// Returns `true` if senders belong to the same channel.\n pub(crate) fn same_channel(&self, other: &Self) -> bool {\n Arc::ptr_eq(&self.inner, &other.inner)\n }\n}\n\nimpl Tx {\n pub(crate) fn is_closed(&self) -> bool {\n self.inner.semaphore.is_closed()\n }\n\n pub(crate) async fn closed(&self) {\n // In order to avoid a race condition, we first request a notification,\n // **then** check whether the semaphore is closed. If the semaphore is\n // closed the notification request is dropped.\n let notified = self.inner.notify_rx_closed.notified();\n\n if self.inner.semaphore.is_closed() {\n return;\n }\n notified.await;\n }\n}\n\nimpl Clone for Tx {\n fn clone(&self) -> Tx {\n // Using a Relaxed ordering here is sufficient as the caller holds a\n // strong ref to `self`, preventing a concurrent decrement to zero.\n self.inner.tx_count.fetch_add(1, Relaxed);\n\n Tx {\n inner: self.inner.clone(),\n }\n }\n}\n\nimpl Drop for Tx {\n fn drop(&mut self) {\n if self.inner.tx_count.fetch_sub(1, AcqRel) != 1 {\n return;\n }\n\n // Close the list, which sends a `Close` message\n self.inner.tx.close();\n\n // Notify the receiver\n self.wake_rx();\n }\n}\n\n// ===== impl Rx =====\n\nimpl Rx {\n fn new(chan: Arc>) -> Rx {\n Rx { inner: chan }\n }\n\n pub(crate) fn close(&mut self) {\n self.inner.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n\n if rx_fields.rx_closed {\n return;\n }\n\n rx_fields.rx_closed = true;\n });\n\n self.inner.semaphore.close();\n self.inner.notify_rx_closed.notify_waiters();\n }\n\n pub(crate) fn is_closed(&self) -> bool {\n // There two internal states that can represent a closed channel\n //\n // 1. When `close` is called.\n // In this case, the inner semaphore will be closed.\n //\n // 2. When all senders are dropped.\n // In this case, the semaphore remains unclosed, and the `index` in the list won't\n // reach the tail position. It is necessary to check the list if the last block is\n // `closed`.\n self.inner.semaphore.is_closed() || self.inner.tx_count.load(Acquire) == 0\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.inner.rx_fields.with(|rx_fields_ptr| {\n let rx_fields = unsafe { &*rx_fields_ptr };\n rx_fields.list.is_empty(&self.inner.tx)\n })\n }\n\n pub(crate) fn len(&self) -> usize {\n self.inner.rx_fields.with(|rx_fields_ptr| {\n let rx_fields = unsafe { &*rx_fields_ptr };\n rx_fields.list.len(&self.inner.tx)\n })\n }\n\n /// Receive the next value\n pub(crate) fn recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n use super::block::Read;\n\n ready!(crate::trace::trace_leaf(cx));\n\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n self.inner.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n\n macro_rules! try_recv {\n () => {\n match rx_fields.list.pop(&self.inner.tx) {\n Some(Read::Value(value)) => {\n self.inner.semaphore.add_permit();\n coop.made_progress();\n return Ready(Some(value));\n }\n Some(Read::Closed) => {\n // TODO: This check may not be required as it most\n // likely can only return `true` at this point. A\n // channel is closed when all tx handles are\n // dropped. Dropping a tx handle releases memory,\n // which ensures that if dropping the tx handle is\n // visible, then all messages sent are also visible.\n assert!(self.inner.semaphore.is_idle());\n coop.made_progress();\n return Ready(None);\n }\n None => {} // fall through\n }\n };\n }\n\n try_recv!();\n\n self.inner.rx_waker.register_by_ref(cx.waker());\n\n // It is possible that a value was pushed between attempting to read\n // and registering the task, so we have to check the channel a\n // second time here.\n try_recv!();\n\n if rx_fields.rx_closed && self.inner.semaphore.is_idle() {\n coop.made_progress();\n Ready(None)\n } else {\n Pending\n }\n })\n }\n\n /// Receives up to `limit` values into `buffer`\n ///\n /// For `limit > 0`, receives up to limit values into `buffer`.\n /// For `limit == 0`, immediately returns Ready(0).\n pub(crate) fn recv_many(\n &mut self,\n cx: &mut Context<'_>,\n buffer: &mut Vec,\n limit: usize,\n ) -> Poll {\n use super::block::Read;\n\n ready!(crate::trace::trace_leaf(cx));\n\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n if limit == 0 {\n coop.made_progress();\n return Ready(0usize);\n }\n\n let mut remaining = limit;\n let initial_length = buffer.len();\n\n self.inner.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n macro_rules! try_recv {\n () => {\n while remaining > 0 {\n match rx_fields.list.pop(&self.inner.tx) {\n Some(Read::Value(value)) => {\n remaining -= 1;\n buffer.push(value);\n }\n\n Some(Read::Closed) => {\n let number_added = buffer.len() - initial_length;\n if number_added > 0 {\n self.inner.semaphore.add_permits(number_added);\n }\n // TODO: This check may not be required as it most\n // likely can only return `true` at this point. A\n // channel is closed when all tx handles are\n // dropped. Dropping a tx handle releases memory,\n // which ensures that if dropping the tx handle is\n // visible, then all messages sent are also visible.\n assert!(self.inner.semaphore.is_idle());\n coop.made_progress();\n return Ready(number_added);\n }\n\n None => {\n break; // fall through\n }\n }\n }\n let number_added = buffer.len() - initial_length;\n if number_added > 0 {\n self.inner.semaphore.add_permits(number_added);\n coop.made_progress();\n return Ready(number_added);\n }\n };\n }\n\n try_recv!();\n\n self.inner.rx_waker.register_by_ref(cx.waker());\n\n // It is possible that a value was pushed between attempting to read\n // and registering the task, so we have to check the channel a\n // second time here.\n try_recv!();\n\n if rx_fields.rx_closed && self.inner.semaphore.is_idle() {\n assert!(buffer.is_empty());\n coop.made_progress();\n Ready(0usize)\n } else {\n Pending\n }\n })\n }\n\n /// Try to receive the next value.\n pub(crate) fn try_recv(&mut self) -> Result {\n use super::list::TryPopResult;\n\n self.inner.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n\n macro_rules! try_recv {\n () => {\n match rx_fields.list.try_pop(&self.inner.tx) {\n TryPopResult::Ok(value) => {\n self.inner.semaphore.add_permit();\n return Ok(value);\n }\n TryPopResult::Closed => return Err(TryRecvError::Disconnected),\n // If close() was called, an empty queue should report Disconnected.\n TryPopResult::Empty if rx_fields.rx_closed => {\n return Err(TryRecvError::Disconnected)\n }\n TryPopResult::Empty => return Err(TryRecvError::Empty),\n TryPopResult::Busy => {} // fall through\n }\n };\n }\n\n try_recv!();\n\n // If a previous `poll_recv` call has set a waker, we wake it here.\n // This allows us to put our own CachedParkThread waker in the\n // AtomicWaker slot instead.\n //\n // This is not a spurious wakeup to `poll_recv` since we just got a\n // Busy from `try_pop`, which only happens if there are messages in\n // the queue.\n self.inner.rx_waker.wake();\n\n // Park the thread until the problematic send has completed.\n let mut park = CachedParkThread::new();\n let waker = park.waker().unwrap();\n loop {\n self.inner.rx_waker.register_by_ref(&waker);\n // It is possible that the problematic send has now completed,\n // so we have to check for messages again.\n try_recv!();\n park.park();\n }\n })\n }\n\n pub(super) fn semaphore(&self) -> &S {\n &self.inner.semaphore\n }\n\n pub(super) fn sender_strong_count(&self) -> usize {\n self.inner.tx_count.load(Acquire)\n }\n\n pub(super) fn sender_weak_count(&self) -> usize {\n self.inner.tx_weak_count.load(Relaxed)\n }\n}\n\nimpl Drop for Rx {\n fn drop(&mut self) {\n use super::block::Read::Value;\n\n self.close();\n\n self.inner.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n struct Guard<'a, T, S: Semaphore> {\n list: &'a mut list::Rx,\n tx: &'a list::Tx,\n sem: &'a S,\n }\n\n impl<'a, T, S: Semaphore> Guard<'a, T, S> {\n fn drain(&mut self) {\n // call T's destructor.\n while let Some(Value(_)) = self.list.pop(self.tx) {\n self.sem.add_permit();\n }\n }\n }\n\n impl<'a, T, S: Semaphore> Drop for Guard<'a, T, S> {\n fn drop(&mut self) {\n self.drain();\n }\n }\n\n let mut guard = Guard {\n list: &mut rx_fields.list,\n tx: &self.inner.tx,\n sem: &self.inner.semaphore,\n };\n\n guard.drain();\n });\n }\n}\n\n// ===== impl Chan =====\n\nimpl Chan {\n fn send(&self, value: T) {\n // Push the value\n self.tx.push(value);\n\n // Notify the rx task\n self.rx_waker.wake();\n }\n\n pub(super) fn decrement_weak_count(&self) {\n self.tx_weak_count.fetch_sub(1, Relaxed);\n }\n\n pub(super) fn increment_weak_count(&self) {\n self.tx_weak_count.fetch_add(1, Relaxed);\n }\n\n pub(super) fn strong_count(&self) -> usize {\n self.tx_count.load(Acquire)\n }\n\n pub(super) fn weak_count(&self) -> usize {\n self.tx_weak_count.load(Relaxed)\n }\n}\n\nimpl Drop for Chan {\n fn drop(&mut self) {\n use super::block::Read::Value;\n\n // Safety: the only owner of the rx fields is Chan, and being\n // inside its own Drop means we're the last ones to touch it.\n self.rx_fields.with_mut(|rx_fields_ptr| {\n let rx_fields = unsafe { &mut *rx_fields_ptr };\n\n while let Some(Value(_)) = rx_fields.list.pop(&self.tx) {}\n unsafe { rx_fields.list.free_blocks() };\n });\n }\n}\n\n// ===== impl Semaphore for (::Semaphore, capacity) =====\n\nimpl Semaphore for bounded::Semaphore {\n fn add_permit(&self) {\n self.semaphore.release(1);\n }\n\n fn add_permits(&self, n: usize) {\n self.semaphore.release(n)\n }\n\n fn is_idle(&self) -> bool {\n self.semaphore.available_permits() == self.bound\n }\n\n fn close(&self) {\n self.semaphore.close();\n }\n\n fn is_closed(&self) -> bool {\n self.semaphore.is_closed()\n }\n}\n\n// ===== impl Semaphore for AtomicUsize =====\n\nimpl Semaphore for unbounded::Semaphore {\n fn add_permit(&self) {\n let prev = self.0.fetch_sub(2, Release);\n\n if prev >> 1 == 0 {\n // Something went wrong\n process::abort();\n }\n }\n\n fn add_permits(&self, n: usize) {\n let prev = self.0.fetch_sub(n << 1, Release);\n\n if (prev >> 1) < n {\n // Something went wrong\n process::abort();\n }\n }\n\n fn is_idle(&self) -> bool {\n self.0.load(Acquire) >> 1 == 0\n }\n\n fn close(&self) {\n self.0.fetch_or(1, Release);\n }\n\n fn is_closed(&self) -> bool {\n self.0.load(Acquire) & 1 == 1\n }\n}\n" }, { "path": "tokio/src/sync/mpsc/error.rs", "content": "//! Channel error types.\n\nuse std::error::Error;\nuse std::fmt;\n\n/// Error returned by [`Sender::send`](super::Sender::send).\n#[derive(PartialEq, Eq, Clone, Copy)]\npub struct SendError(pub T);\n\nimpl fmt::Debug for SendError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"SendError\").finish_non_exhaustive()\n }\n}\n\nimpl fmt::Display for SendError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n}\n\nimpl Error for SendError {}\n\n// ===== TrySendError =====\n\n/// Error returned by [`Sender::try_send`](super::Sender::try_send).\n#[derive(PartialEq, Eq, Clone, Copy)]\npub enum TrySendError {\n /// The data could not be sent on the channel because the channel is\n /// currently full and sending would require blocking.\n Full(T),\n\n /// The receive half of the channel was explicitly closed or has been\n /// dropped.\n Closed(T),\n}\n\nimpl TrySendError {\n /// Consume the `TrySendError`, returning the unsent value.\n pub fn into_inner(self) -> T {\n match self {\n TrySendError::Full(val) => val,\n TrySendError::Closed(val) => val,\n }\n }\n}\n\nimpl fmt::Debug for TrySendError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match *self {\n TrySendError::Full(..) => \"Full(..)\".fmt(f),\n TrySendError::Closed(..) => \"Closed(..)\".fmt(f),\n }\n }\n}\n\nimpl fmt::Display for TrySendError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(\n fmt,\n \"{}\",\n match self {\n TrySendError::Full(..) => \"no available capacity\",\n TrySendError::Closed(..) => \"channel closed\",\n }\n )\n }\n}\n\nimpl Error for TrySendError {}\n\nimpl From> for TrySendError {\n fn from(src: SendError) -> TrySendError {\n TrySendError::Closed(src.0)\n }\n}\n\n// ===== TryRecvError =====\n\n/// Error returned by [`Receiver::try_recv`](super::Receiver::try_recv).\n#[derive(PartialEq, Eq, Clone, Copy, Debug)]\npub enum TryRecvError {\n /// This **channel** is currently empty, but the **Sender**(s) have not yet\n /// disconnected, so data may yet become available.\n Empty,\n /// The **channel**'s sending half has become disconnected, and there will\n /// never be any more data received on it.\n Disconnected,\n}\n\nimpl fmt::Display for TryRecvError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n match *self {\n TryRecvError::Empty => \"receiving on an empty channel\".fmt(fmt),\n TryRecvError::Disconnected => \"receiving on a closed channel\".fmt(fmt),\n }\n }\n}\n\nimpl Error for TryRecvError {}\n\n// ===== RecvError =====\n\n/// Error returned by `Receiver`.\n#[derive(Debug, Clone)]\n#[doc(hidden)]\n#[deprecated(note = \"This type is unused because recv returns an Option.\")]\npub struct RecvError(());\n\n#[allow(deprecated)]\nimpl fmt::Display for RecvError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n}\n\n#[allow(deprecated)]\nimpl Error for RecvError {}\n\ncfg_time! {\n // ===== SendTimeoutError =====\n\n #[derive(PartialEq, Eq, Clone, Copy)]\n /// Error returned by [`Sender::send_timeout`](super::Sender::send_timeout).\n pub enum SendTimeoutError {\n /// The data could not be sent on the channel because the channel is\n /// full, and the timeout to send has elapsed.\n Timeout(T),\n\n /// The receive half of the channel was explicitly closed or has been\n /// dropped.\n Closed(T),\n }\n\n impl SendTimeoutError {\n /// Consume the `SendTimeoutError`, returning the unsent value.\n pub fn into_inner(self) -> T {\n match self {\n SendTimeoutError::Timeout(val) => val,\n SendTimeoutError::Closed(val) => val,\n }\n }\n }\n\n impl fmt::Debug for SendTimeoutError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match *self {\n SendTimeoutError::Timeout(..) => \"Timeout(..)\".fmt(f),\n SendTimeoutError::Closed(..) => \"Closed(..)\".fmt(f),\n }\n }\n }\n\n impl fmt::Display for SendTimeoutError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(\n fmt,\n \"{}\",\n match self {\n SendTimeoutError::Timeout(..) => \"timed out waiting on send operation\",\n SendTimeoutError::Closed(..) => \"channel closed\",\n }\n )\n }\n }\n\n impl Error for SendTimeoutError {}\n}\n" }, { "path": "tokio/src/sync/mpsc/list.rs", "content": "//! A concurrent, lock-free, FIFO list.\n\nuse crate::loom::sync::atomic::{AtomicPtr, AtomicUsize};\nuse crate::loom::thread;\nuse crate::sync::mpsc::block::{self, Block};\n\nuse std::fmt;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release};\n\n/// List queue transmit handle.\npub(crate) struct Tx {\n /// Tail in the `Block` mpmc list.\n block_tail: AtomicPtr>,\n\n /// Position to push the next message. This references a block and offset\n /// into the block.\n tail_position: AtomicUsize,\n}\n\n/// List queue receive handle\npub(crate) struct Rx {\n /// Pointer to the block being processed.\n head: NonNull>,\n\n /// Next slot index to process.\n index: usize,\n\n /// Pointer to the next block pending release.\n free_head: NonNull>,\n}\n\n/// Return value of `Rx::try_pop`.\npub(crate) enum TryPopResult {\n /// Successfully popped a value.\n Ok(T),\n /// The channel is empty.\n ///\n /// Note that `list.rs` only tracks the close state set by senders. If the\n /// channel is closed by `Rx::close()`, then `TryPopResult::Empty` is still\n /// returned, and the close state needs to be handled by `chan.rs`.\n Empty,\n /// The channel is empty and closed.\n ///\n /// Returned when the send half is closed (all senders dropped).\n Closed,\n /// The channel is not empty, but the first value is being written.\n Busy,\n}\n\npub(crate) fn channel() -> (Tx, Rx) {\n // Create the initial block shared between the tx and rx halves.\n let initial_block = Block::new(0);\n let initial_block_ptr = Box::into_raw(initial_block);\n\n let tx = Tx {\n block_tail: AtomicPtr::new(initial_block_ptr),\n tail_position: AtomicUsize::new(0),\n };\n\n let head = NonNull::new(initial_block_ptr).unwrap();\n\n let rx = Rx {\n head,\n index: 0,\n free_head: head,\n };\n\n (tx, rx)\n}\n\nimpl Tx {\n /// Pushes a value into the list.\n pub(crate) fn push(&self, value: T) {\n // First, claim a slot for the value. `Acquire` is used here to\n // synchronize with the `fetch_add` in `reclaim_blocks`.\n let slot_index = self.tail_position.fetch_add(1, Acquire);\n\n // Load the current block and write the value\n let block = self.find_block(slot_index);\n\n unsafe {\n // Write the value to the block\n block.as_ref().write(slot_index, value);\n }\n }\n\n /// Closes the send half of the list.\n ///\n /// Similar process as pushing a value, but instead of writing the value &\n /// setting the ready flag, the `TX_CLOSED` flag is set on the block.\n pub(crate) fn close(&self) {\n // First, claim a slot for the value. This is the last slot that will be\n // claimed.\n let slot_index = self.tail_position.fetch_add(1, Acquire);\n\n let block = self.find_block(slot_index);\n\n unsafe { block.as_ref().tx_close() }\n }\n\n fn find_block(&self, slot_index: usize) -> NonNull> {\n // The start index of the block that contains `index`.\n let start_index = block::start_index(slot_index);\n\n // The index offset into the block\n let offset = block::offset(slot_index);\n\n // Load the current head of the block\n let mut block_ptr = self.block_tail.load(Acquire);\n\n let block = unsafe { &*block_ptr };\n\n // Calculate the distance between the tail ptr and the target block\n let distance = block.distance(start_index);\n\n // Decide if this call to `find_block` should attempt to update the\n // `block_tail` pointer.\n //\n // Updating `block_tail` is not always performed in order to reduce\n // contention.\n //\n // When set, as the routine walks the linked list, it attempts to update\n // `block_tail`. If the update cannot be performed, `try_updating_tail`\n // is unset.\n let mut try_updating_tail = distance > offset;\n\n // Walk the linked list of blocks until the block with `start_index` is\n // found.\n loop {\n let block = unsafe { &(*block_ptr) };\n\n if block.is_at_index(start_index) {\n return unsafe { NonNull::new_unchecked(block_ptr) };\n }\n\n let next_block = block\n .load_next(Acquire)\n // There is no allocated next block, grow the linked list.\n .unwrap_or_else(|| block.grow());\n\n // If the block is **not** final, then the tail pointer cannot be\n // advanced any more.\n try_updating_tail &= block.is_final();\n\n if try_updating_tail {\n // Advancing `block_tail` must happen when walking the linked\n // list. `block_tail` may not advance passed any blocks that are\n // not \"final\". At the point a block is finalized, it is unknown\n // if there are any prior blocks that are unfinalized, which\n // makes it impossible to advance `block_tail`.\n //\n // While walking the linked list, `block_tail` can be advanced\n // as long as finalized blocks are traversed.\n //\n // Release ordering is used to ensure that any subsequent reads\n // are able to see the memory pointed to by `block_tail`.\n //\n // Acquire is not needed as any \"actual\" value is not accessed.\n // At this point, the linked list is walked to acquire blocks.\n if self\n .block_tail\n .compare_exchange(block_ptr, next_block.as_ptr(), Release, Relaxed)\n .is_ok()\n {\n // Synchronize with any senders\n let tail_position = self.tail_position.fetch_add(0, Release);\n\n unsafe {\n block.tx_release(tail_position);\n }\n } else {\n // A concurrent sender is also working on advancing\n // `block_tail` and this thread is falling behind.\n //\n // Stop trying to advance the tail pointer\n try_updating_tail = false;\n }\n }\n\n block_ptr = next_block.as_ptr();\n\n thread::yield_now();\n }\n }\n\n /// # Safety\n ///\n /// Behavior is undefined if any of the following conditions are violated:\n ///\n /// - The `block` was created by [`Box::into_raw`].\n /// - The `block` is not currently part of any linked list.\n /// - The `block` is a valid pointer to a [`Block`].\n pub(crate) unsafe fn reclaim_block(&self, mut block: NonNull>) {\n // The block has been removed from the linked list and ownership\n // is reclaimed.\n //\n // Before dropping the block, see if it can be reused by\n // inserting it back at the end of the linked list.\n //\n // First, reset the data\n //\n // Safety: caller guarantees the block is valid and not in any list.\n unsafe {\n block.as_mut().reclaim();\n }\n\n let mut reused = false;\n\n // Attempt to insert the block at the end\n //\n // Walk at most three times\n let curr_ptr = self.block_tail.load(Acquire);\n\n // The pointer can never be null\n debug_assert!(!curr_ptr.is_null());\n\n // Safety: curr_ptr is never null.\n let mut curr = unsafe { NonNull::new_unchecked(curr_ptr) };\n\n // TODO: Unify this logic with Block::grow\n for _ in 0..3 {\n match unsafe { curr.as_ref().try_push(&mut block, AcqRel, Acquire) } {\n Ok(()) => {\n reused = true;\n break;\n }\n Err(next) => {\n curr = next;\n }\n }\n }\n\n if !reused {\n // Safety:\n //\n // 1. Caller guarantees the block is valid and not in any list.\n // 2. The block was created by `Box::into_raw`.\n let _ = unsafe { Box::from_raw(block.as_ptr()) };\n }\n }\n\n pub(crate) fn is_closed(&self) -> bool {\n let tail = self.block_tail.load(Acquire);\n\n unsafe {\n let tail_block = &*tail;\n tail_block.is_closed()\n }\n }\n}\n\nimpl fmt::Debug for Tx {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Tx\")\n .field(\"block_tail\", &self.block_tail.load(Relaxed))\n .field(\"tail_position\", &self.tail_position.load(Relaxed))\n .finish()\n }\n}\n\nimpl Rx {\n pub(crate) fn is_empty(&self, tx: &Tx) -> bool {\n let block = unsafe { self.head.as_ref() };\n if block.has_value(self.index) {\n return false;\n }\n\n // It is possible that a block has no value \"now\" but the list is still not empty.\n // To be sure, it is necessary to check the length of the list.\n self.len(tx) == 0\n }\n\n pub(crate) fn len(&self, tx: &Tx) -> usize {\n // When all the senders are dropped, there will be a last block in the tail position,\n // but it will be closed\n let tail_position = tx.tail_position.load(Acquire);\n tail_position - self.index - (tx.is_closed() as usize)\n }\n\n /// Pops the next value off the queue.\n pub(crate) fn pop(&mut self, tx: &Tx) -> Option> {\n // Advance `head`, if needed\n if !self.try_advancing_head() {\n return None;\n }\n\n self.reclaim_blocks(tx);\n\n unsafe {\n let block = self.head.as_ref();\n\n let ret = block.read(self.index);\n\n if let Some(block::Read::Value(..)) = ret {\n self.index = self.index.wrapping_add(1);\n }\n\n ret\n }\n }\n\n /// Pops the next value off the queue, detecting whether the block\n /// is busy or empty on failure.\n ///\n /// This function exists because `Rx::pop` can return `None` even if the\n /// channel's queue contains a message that has been completely written.\n /// This can happen if the fully delivered message is behind another message\n /// that is in the middle of being written to the block, since the channel\n /// can't return the messages out of order.\n pub(crate) fn try_pop(&mut self, tx: &Tx) -> TryPopResult {\n let tail_position = tx.tail_position.load(Acquire);\n let result = self.pop(tx);\n\n match result {\n Some(block::Read::Value(t)) => TryPopResult::Ok(t),\n Some(block::Read::Closed) => TryPopResult::Closed,\n None if tail_position == self.index => TryPopResult::Empty,\n None => TryPopResult::Busy,\n }\n }\n\n /// Tries advancing the block pointer to the block referenced by `self.index`.\n ///\n /// Returns `true` if successful, `false` if there is no next block to load.\n fn try_advancing_head(&mut self) -> bool {\n let block_index = block::start_index(self.index);\n\n loop {\n let next_block = {\n let block = unsafe { self.head.as_ref() };\n\n if block.is_at_index(block_index) {\n return true;\n }\n\n block.load_next(Acquire)\n };\n\n let next_block = match next_block {\n Some(next_block) => next_block,\n None => {\n return false;\n }\n };\n\n self.head = next_block;\n\n thread::yield_now();\n }\n }\n\n fn reclaim_blocks(&mut self, tx: &Tx) {\n while self.free_head != self.head {\n unsafe {\n // Get a handle to the block that will be freed and update\n // `free_head` to point to the next block.\n let block = self.free_head;\n\n let observed_tail_position = block.as_ref().observed_tail_position();\n\n let required_index = match observed_tail_position {\n Some(i) => i,\n None => return,\n };\n\n if required_index > self.index {\n return;\n }\n\n // We may read the next pointer with `Relaxed` ordering as it is\n // guaranteed that the `reclaim_blocks` routine trails the `recv`\n // routine. Any memory accessed by `reclaim_blocks` has already\n // been acquired by `recv`.\n let next_block = block.as_ref().load_next(Relaxed);\n\n // Update the free list head\n self.free_head = next_block.unwrap();\n\n // Push the emptied block onto the back of the queue, making it\n // available to senders.\n tx.reclaim_block(block);\n }\n\n thread::yield_now();\n }\n }\n\n /// Effectively `Drop` all the blocks. Should only be called once, when\n /// the list is dropping.\n pub(super) unsafe fn free_blocks(&mut self) {\n debug_assert_ne!(self.free_head, NonNull::dangling());\n\n let mut cur = Some(self.free_head);\n\n #[cfg(debug_assertions)]\n {\n // to trigger the debug assert above so as to catch that we\n // don't call `free_blocks` more than once.\n self.free_head = NonNull::dangling();\n self.head = NonNull::dangling();\n }\n\n while let Some(block) = cur {\n cur = unsafe { block.as_ref() }.load_next(Relaxed);\n drop(unsafe { Box::from_raw(block.as_ptr()) });\n }\n }\n}\n\nimpl fmt::Debug for Rx {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Rx\")\n .field(\"head\", &self.head)\n .field(\"index\", &self.index)\n .field(\"free_head\", &self.free_head)\n .finish()\n }\n}\n" }, { "path": "tokio/src/sync/mpsc/mod.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(dead_code, unreachable_pub))]\n\n//! A multi-producer, single-consumer queue for sending values between\n//! asynchronous tasks.\n//!\n//! This module provides two variants of the channel: bounded and unbounded. The\n//! bounded variant has a limit on the number of messages that the channel can\n//! store, and if this limit is reached, trying to send another message will\n//! wait until a message is received from the channel. An unbounded channel has\n//! an infinite capacity, so the `send` method will always complete immediately.\n//! This makes the [`UnboundedSender`] usable from both synchronous and\n//! asynchronous code.\n//!\n//! Similar to the `mpsc` channels provided by `std`, the channel constructor\n//! functions provide separate send and receive handles, [`Sender`] and\n//! [`Receiver`] for the bounded channel, [`UnboundedSender`] and\n//! [`UnboundedReceiver`] for the unbounded channel. If there is no message to read,\n//! the current task will be notified when a new value is sent. [`Sender`] and\n//! [`UnboundedSender`] allow sending values into the channel. If the bounded\n//! channel is at capacity, the send is rejected and the task will be notified\n//! when additional capacity is available. In other words, the channel provides\n//! backpressure.\n//!\n//! This channel is also suitable for the single-producer single-consumer\n//! use-case. (Unless you only need to send one message, in which case you\n//! should use the [oneshot] channel.)\n//!\n//! # Disconnection\n//!\n//! When all [`Sender`] handles have been dropped, it is no longer\n//! possible to send values into the channel. This is considered the termination\n//! event of the stream. Once all senders have been dropped and any remaining\n//! buffered values have been received, `Receiver::recv` returns `None`\n//! (and `Receiver::poll_recv` returns `Poll::Ready(None)`).\n//!\n//! If the [`Receiver`] handle is dropped, then messages can no longer\n//! be read out of the channel. In this case, all further attempts to send will\n//! result in an error. Additionally, all unread messages will be drained from the\n//! channel and dropped.\n//!\n//! # Clean Shutdown\n//!\n//! When the [`Receiver`] is dropped, it is possible for unprocessed messages to\n//! remain in the channel. Instead, it is usually desirable to perform a \"clean\"\n//! shutdown. To do this, the receiver first calls `close`, which will prevent\n//! any further messages to be sent into the channel. Then, the receiver\n//! consumes the channel to completion, at which point the receiver can be\n//! dropped.\n//!\n//! # Communicating between sync and async code\n//!\n//! When you want to communicate between synchronous and asynchronous code, there\n//! are two situations to consider:\n//!\n//! **Bounded channel**: If you need a bounded channel, you should use a bounded\n//! Tokio `mpsc` channel for both directions of communication. Instead of calling\n//! the async [`send`][bounded-send] or [`recv`][bounded-recv] methods, in\n//! synchronous code you will need to use the [`blocking_send`][blocking-send] or\n//! [`blocking_recv`][blocking-recv] methods.\n//!\n//! **Unbounded channel**: You should use the kind of channel that matches where\n//! the receiver is. So for sending a message _from async to sync_, you should\n//! use [the standard library unbounded channel][std-unbounded] or\n//! [crossbeam][crossbeam-unbounded]. Similarly, for sending a message _from sync\n//! to async_, you should use an unbounded Tokio `mpsc` channel.\n//!\n//! Please be aware that the above remarks were written with the `mpsc` channel\n//! in mind, but they can also be generalized to other kinds of channels. In\n//! general, any channel method that isn't marked async can be called anywhere,\n//! including outside of the runtime. For example, sending a message on a\n//! [oneshot] channel from outside the runtime is perfectly fine.\n//!\n//! # Multiple runtimes\n//!\n//! The `mpsc` channel is runtime agnostic. You can freely move it between\n//! different instances of the Tokio runtime or even use it from non-Tokio\n//! runtimes.\n//!\n//! When used in a Tokio runtime, it participates in\n//! [cooperative scheduling](crate::task::coop#cooperative-scheduling) to avoid\n//! starvation. This feature does not apply when used from non-Tokio runtimes.\n//!\n//! As an exception, methods ending in `_timeout` are not runtime agnostic\n//! because they require access to the Tokio timer. See the documentation of\n//! each `*_timeout` method for more information on its use.\n//!\n//! # Allocation behavior\n//!\n//!
The implementation details described in this section may change in future\n//! Tokio releases.
\n//!\n//! The mpsc channel stores elements in blocks. Blocks are organized in a linked list. Sending\n//! pushes new elements onto the block at the front of the list, and receiving pops them off the\n//! one at the back. A block can hold 32 messages on a 64-bit target and 16 messages on a 32-bit\n//! target. This number is independent of channel and message size. Each block also stores 4\n//! pointer-sized values for bookkeeping (so on a 64-bit machine, each message has 1 byte of\n//! overhead).\n//!\n//! When all values in a block have been received, it becomes empty. It will then be freed, unless\n//! the channel's first block (where newly-sent elements are being stored) has no next block. In\n//! that case, the empty block is reused as the next block.\n//!\n//! [`Sender`]: crate::sync::mpsc::Sender\n//! [`Receiver`]: crate::sync::mpsc::Receiver\n//! [bounded-send]: crate::sync::mpsc::Sender::send()\n//! [bounded-recv]: crate::sync::mpsc::Receiver::recv()\n//! [blocking-send]: crate::sync::mpsc::Sender::blocking_send()\n//! [blocking-recv]: crate::sync::mpsc::Receiver::blocking_recv()\n//! [`UnboundedSender`]: crate::sync::mpsc::UnboundedSender\n//! [`UnboundedReceiver`]: crate::sync::mpsc::UnboundedReceiver\n//! [oneshot]: crate::sync::oneshot\n//! [`Handle::block_on`]: crate::runtime::Handle::block_on()\n//! [std-unbounded]: std::sync::mpsc::channel\n//! [crossbeam-unbounded]: https://docs.rs/crossbeam/*/crossbeam/channel/fn.unbounded.html\n//! [`send_timeout`]: crate::sync::mpsc::Sender::send_timeout\n\npub(super) mod block;\n\nmod bounded;\npub use self::bounded::{\n channel, OwnedPermit, Permit, PermitIterator, Receiver, Sender, WeakSender,\n};\n\nmod chan;\n\npub(super) mod list;\n\nmod unbounded;\npub use self::unbounded::{\n unbounded_channel, UnboundedReceiver, UnboundedSender, WeakUnboundedSender,\n};\n\npub mod error;\n\n/// The number of values a block can contain.\n///\n/// This value must be a power of 2. It also must be smaller than the number of\n/// bits in `usize`.\n#[cfg(all(target_pointer_width = \"64\", not(loom)))]\nconst BLOCK_CAP: usize = 32;\n\n#[cfg(all(not(target_pointer_width = \"64\"), not(loom)))]\nconst BLOCK_CAP: usize = 16;\n\n#[cfg(loom)]\nconst BLOCK_CAP: usize = 2;\n" }, { "path": "tokio/src/sync/mpsc/unbounded.rs", "content": "use crate::loom::sync::{atomic::AtomicUsize, Arc};\nuse crate::sync::mpsc::chan;\nuse crate::sync::mpsc::error::{SendError, TryRecvError};\n\nuse std::fmt;\nuse std::task::{Context, Poll};\n\n/// Send values to the associated `UnboundedReceiver`.\n///\n/// Instances are created by the [`unbounded_channel`] function.\npub struct UnboundedSender {\n chan: chan::Tx,\n}\n\n/// An unbounded sender that does not prevent the channel from being closed.\n///\n/// If all [`UnboundedSender`] instances of a channel were dropped and only\n/// `WeakUnboundedSender` instances remain, the channel is closed.\n///\n/// In order to send messages, the `WeakUnboundedSender` needs to be upgraded using\n/// [`WeakUnboundedSender::upgrade`], which returns `Option`. It returns `None`\n/// if all `UnboundedSender`s have been dropped, and otherwise it returns an `UnboundedSender`.\n///\n/// [`UnboundedSender`]: UnboundedSender\n/// [`WeakUnboundedSender::upgrade`]: WeakUnboundedSender::upgrade\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::mpsc::unbounded_channel;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, _rx) = unbounded_channel::();\n/// let tx_weak = tx.downgrade();\n///\n/// // Upgrading will succeed because `tx` still exists.\n/// assert!(tx_weak.upgrade().is_some());\n///\n/// // If we drop `tx`, then it will fail.\n/// drop(tx);\n/// assert!(tx_weak.clone().upgrade().is_none());\n/// # }\n/// ```\npub struct WeakUnboundedSender {\n chan: Arc>,\n}\n\nimpl Clone for UnboundedSender {\n fn clone(&self) -> Self {\n UnboundedSender {\n chan: self.chan.clone(),\n }\n }\n}\n\nimpl fmt::Debug for UnboundedSender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"UnboundedSender\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n\n/// Receive values from the associated `UnboundedSender`.\n///\n/// Instances are created by the [`unbounded_channel`] function.\n///\n/// This receiver can be turned into a `Stream` using [`UnboundedReceiverStream`].\n///\n/// [`UnboundedReceiverStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.UnboundedReceiverStream.html\npub struct UnboundedReceiver {\n /// The channel receiver\n chan: chan::Rx,\n}\n\nimpl fmt::Debug for UnboundedReceiver {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"UnboundedReceiver\")\n .field(\"chan\", &self.chan)\n .finish()\n }\n}\n\n/// Creates an unbounded mpsc channel for communicating between asynchronous\n/// tasks without backpressure.\n///\n/// A `send` on this channel will always succeed as long as the receive half has\n/// not been closed. If the receiver falls behind, messages will be arbitrarily\n/// buffered.\n///\n/// **Note** that the amount of available system memory is an implicit bound to\n/// the channel. Using an `unbounded` channel has the ability of causing the\n/// process to run out of memory. In this case, the process will be aborted.\npub fn unbounded_channel() -> (UnboundedSender, UnboundedReceiver) {\n let (tx, rx) = chan::channel(Semaphore(AtomicUsize::new(0)));\n\n let tx = UnboundedSender::new(tx);\n let rx = UnboundedReceiver::new(rx);\n\n (tx, rx)\n}\n\n/// No capacity\n#[derive(Debug)]\npub(crate) struct Semaphore(pub(crate) AtomicUsize);\n\nimpl UnboundedReceiver {\n pub(crate) fn new(chan: chan::Rx) -> UnboundedReceiver {\n UnboundedReceiver { chan }\n }\n\n /// Receives the next value for this receiver.\n ///\n /// This method returns `None` if the channel has been closed and there are\n /// no remaining messages in the channel's buffer. This indicates that no\n /// further values can ever be received from this `Receiver`. The channel is\n /// closed when all senders have been dropped, or when [`close`] is called.\n ///\n /// If there are no messages in the channel's buffer, but the channel has\n /// not yet been closed, this method will sleep until a message is sent or\n /// the channel is closed.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// channel.\n ///\n /// [`close`]: Self::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::unbounded_channel();\n ///\n /// tokio::spawn(async move {\n /// tx.send(\"hello\").unwrap();\n /// });\n ///\n /// assert_eq!(Some(\"hello\"), rx.recv().await);\n /// assert_eq!(None, rx.recv().await);\n /// # }\n /// ```\n ///\n /// Values are buffered:\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::unbounded_channel();\n ///\n /// tx.send(\"hello\").unwrap();\n /// tx.send(\"world\").unwrap();\n ///\n /// assert_eq!(Some(\"hello\"), rx.recv().await);\n /// assert_eq!(Some(\"world\"), rx.recv().await);\n /// # }\n /// ```\n pub async fn recv(&mut self) -> Option {\n use std::future::poll_fn;\n\n poll_fn(|cx| self.poll_recv(cx)).await\n }\n\n /// Receives the next values for this receiver and extends `buffer`.\n ///\n /// This method extends `buffer` by no more than a fixed number of values\n /// as specified by `limit`. If `limit` is zero, the function returns\n /// immediately with `0`. The return value is the number of values added to\n /// `buffer`.\n ///\n /// For `limit > 0`, if there are no messages in the channel's queue,\n /// but the channel has not yet been closed, this method will sleep\n /// until a message is sent or the channel is closed.\n ///\n /// For non-zero values of `limit`, this method will never return `0` unless\n /// the channel has been closed and there are no remaining messages in the\n /// channel's queue. This indicates that no further values can ever be\n /// received from this `Receiver`. The channel is closed when all senders\n /// have been dropped, or when [`close`] is called.\n ///\n /// The capacity of `buffer` is increased as needed.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `recv_many` is used as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, it is guaranteed that no messages were received on this\n /// channel.\n ///\n /// [`close`]: Self::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut buffer: Vec<&str> = Vec::with_capacity(2);\n /// let limit = 2;\n /// let (tx, mut rx) = mpsc::unbounded_channel();\n /// let tx2 = tx.clone();\n /// tx2.send(\"first\").unwrap();\n /// tx2.send(\"second\").unwrap();\n /// tx2.send(\"third\").unwrap();\n ///\n /// // Call `recv_many` to receive up to `limit` (2) values.\n /// assert_eq!(2, rx.recv_many(&mut buffer, limit).await);\n /// assert_eq!(vec![\"first\", \"second\"], buffer);\n ///\n /// // If the buffer is full, the next call to `recv_many`\n /// // reserves additional capacity.\n /// assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n ///\n /// tokio::spawn(async move {\n /// tx.send(\"fourth\").unwrap();\n /// });\n ///\n /// // 'tx' is dropped, but `recv_many`\n /// // is guaranteed not to return 0 as the channel\n /// // is not yet closed.\n /// assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n /// assert_eq!(vec![\"first\", \"second\", \"third\", \"fourth\"], buffer);\n ///\n /// // Once the last sender is dropped, the channel is\n /// // closed and `recv_many` returns 0, capacity unchanged.\n /// drop(tx2);\n /// assert_eq!(0, rx.recv_many(&mut buffer, limit).await);\n /// assert_eq!(vec![\"first\", \"second\", \"third\", \"fourth\"], buffer);\n /// # }\n /// ```\n pub async fn recv_many(&mut self, buffer: &mut Vec, limit: usize) -> usize {\n use std::future::poll_fn;\n poll_fn(|cx| self.chan.recv_many(cx, buffer, limit)).await\n }\n\n /// Tries to receive the next value for this receiver.\n ///\n /// This method returns the [`Empty`] error if the channel is currently\n /// empty, but there are still outstanding [senders] or [permits].\n ///\n /// This method returns the [`Disconnected`] error if the channel is\n /// currently empty, and there are no outstanding [senders] or [permits].\n ///\n /// Unlike the [`poll_recv`] method, this method will never return an\n /// [`Empty`] error spuriously.\n ///\n /// [`Empty`]: crate::sync::mpsc::error::TryRecvError::Empty\n /// [`Disconnected`]: crate::sync::mpsc::error::TryRecvError::Disconnected\n /// [`poll_recv`]: Self::poll_recv\n /// [senders]: crate::sync::mpsc::Sender\n /// [permits]: crate::sync::mpsc::Permit\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n /// use tokio::sync::mpsc::error::TryRecvError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = mpsc::unbounded_channel();\n ///\n /// tx.send(\"hello\").unwrap();\n ///\n /// assert_eq!(Ok(\"hello\"), rx.try_recv());\n /// assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n ///\n /// tx.send(\"hello\").unwrap();\n /// // Drop the last sender, closing the channel.\n /// drop(tx);\n ///\n /// assert_eq!(Ok(\"hello\"), rx.try_recv());\n /// assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n /// # }\n /// ```\n pub fn try_recv(&mut self) -> Result {\n self.chan.try_recv()\n }\n\n /// Blocking receive to call outside of asynchronous contexts.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution\n /// context.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::thread;\n /// use tokio::sync::mpsc;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (tx, mut rx) = mpsc::unbounded_channel::();\n ///\n /// let sync_code = thread::spawn(move || {\n /// assert_eq!(Some(10), rx.blocking_recv());\n /// });\n ///\n /// let _ = tx.send(10);\n /// sync_code.join().unwrap();\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"recv_blocking\"))]\n pub fn blocking_recv(&mut self) -> Option {\n crate::future::block_on(self.recv())\n }\n\n /// Variant of [`Self::recv_many`] for blocking contexts.\n ///\n /// The same conditions as in [`Self::blocking_recv`] apply.\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"recv_many_blocking\"))]\n pub fn blocking_recv_many(&mut self, buffer: &mut Vec, limit: usize) -> usize {\n crate::future::block_on(self.recv_many(buffer, limit))\n }\n\n /// Closes the receiving half of a channel, without dropping it.\n ///\n /// This prevents any further messages from being sent on the channel while\n /// still enabling the receiver to drain messages that are buffered.\n ///\n /// To guarantee that no messages are dropped, after calling `close()`,\n /// `recv()` must be called until `None` is returned.\n pub fn close(&mut self) {\n self.chan.close();\n }\n\n /// Checks if a channel is closed.\n ///\n /// This method returns `true` if the channel has been closed. The channel is closed\n /// when all [`UnboundedSender`] have been dropped, or when [`UnboundedReceiver::close`] is called.\n ///\n /// [`UnboundedSender`]: crate::sync::mpsc::UnboundedSender\n /// [`UnboundedReceiver::close`]: crate::sync::mpsc::UnboundedReceiver::close\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (_tx, mut rx) = mpsc::unbounded_channel::<()>();\n /// assert!(!rx.is_closed());\n ///\n /// rx.close();\n ///\n /// assert!(rx.is_closed());\n /// # }\n /// ```\n pub fn is_closed(&self) -> bool {\n self.chan.is_closed()\n }\n\n /// Checks if a channel is empty.\n ///\n /// This method returns `true` if the channel has no messages.\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::unbounded_channel();\n /// assert!(rx.is_empty());\n ///\n /// tx.send(0).unwrap();\n /// assert!(!rx.is_empty());\n /// # }\n ///\n /// ```\n pub fn is_empty(&self) -> bool {\n self.chan.is_empty()\n }\n\n /// Returns the number of messages in the channel.\n ///\n /// # Examples\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::unbounded_channel();\n /// assert_eq!(0, rx.len());\n ///\n /// tx.send(0).unwrap();\n /// assert_eq!(1, rx.len());\n /// # }\n /// ```\n pub fn len(&self) -> usize {\n self.chan.len()\n }\n\n /// Polls to receive the next message on this channel.\n ///\n /// This method returns:\n ///\n /// * `Poll::Pending` if no messages are available but the channel is not\n /// closed, or if a spurious failure happens.\n /// * `Poll::Ready(Some(message))` if a message is available.\n /// * `Poll::Ready(None)` if the channel has been closed and all messages\n /// sent before it was closed have been received.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when a message is sent on any\n /// receiver, or when the channel is closed. Note that on multiple calls to\n /// `poll_recv` or `poll_recv_many`, only the `Waker` from the `Context`\n /// passed to the most recent call is scheduled to receive a wakeup.\n ///\n /// If this method returns `Poll::Pending` due to a spurious failure, then\n /// the `Waker` will be notified when the situation causing the spurious\n /// failure has been resolved. Note that receiving such a wakeup does not\n /// guarantee that the next call will succeed — it could fail with another\n /// spurious failure.\n pub fn poll_recv(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.chan.recv(cx)\n }\n\n /// Polls to receive multiple messages on this channel, extending the provided buffer.\n ///\n /// This method returns:\n /// * `Poll::Pending` if no messages are available but the channel is not closed, or if a\n /// spurious failure happens.\n /// * `Poll::Ready(count)` where `count` is the number of messages successfully received and\n /// stored in `buffer`. This can be less than, or equal to, `limit`.\n /// * `Poll::Ready(0)` if `limit` is set to zero or when the channel is closed.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided\n /// `Context` is scheduled to receive a wakeup when a message is sent on any\n /// receiver, or when the channel is closed. Note that on multiple calls to\n /// `poll_recv` or `poll_recv_many`, only the `Waker` from the `Context`\n /// passed to the most recent call is scheduled to receive a wakeup.\n ///\n /// Note that this method does not guarantee that exactly `limit` messages\n /// are received. Rather, if at least one message is available, it returns\n /// as many messages as it can up to the given limit. This method returns\n /// zero only if the channel is closed (or if `limit` is zero).\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::task::{Context, Poll};\n /// use std::pin::Pin;\n /// use tokio::sync::mpsc;\n /// use futures::Future;\n ///\n /// struct MyReceiverFuture<'a> {\n /// receiver: mpsc::UnboundedReceiver,\n /// buffer: &'a mut Vec,\n /// limit: usize,\n /// }\n ///\n /// impl<'a> Future for MyReceiverFuture<'a> {\n /// type Output = usize; // Number of messages received\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n /// let MyReceiverFuture { receiver, buffer, limit } = &mut *self;\n ///\n /// // Now `receiver` and `buffer` are mutable references, and `limit` is copied\n /// match receiver.poll_recv_many(cx, *buffer, *limit) {\n /// Poll::Pending => Poll::Pending,\n /// Poll::Ready(count) => Poll::Ready(count),\n /// }\n /// }\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = mpsc::unbounded_channel::();\n /// let mut buffer = Vec::new();\n ///\n /// let my_receiver_future = MyReceiverFuture {\n /// receiver: rx,\n /// buffer: &mut buffer,\n /// limit: 3,\n /// };\n ///\n /// for i in 0..10 {\n /// tx.send(i).expect(\"Unable to send integer\");\n /// }\n ///\n /// let count = my_receiver_future.await;\n /// assert_eq!(count, 3);\n /// assert_eq!(buffer, vec![0,1,2])\n /// # }\n /// # }\n /// ```\n pub fn poll_recv_many(\n &mut self,\n cx: &mut Context<'_>,\n buffer: &mut Vec,\n limit: usize,\n ) -> Poll {\n self.chan.recv_many(cx, buffer, limit)\n }\n\n /// Returns the number of [`UnboundedSender`] handles.\n pub fn sender_strong_count(&self) -> usize {\n self.chan.sender_strong_count()\n }\n\n /// Returns the number of [`WeakUnboundedSender`] handles.\n pub fn sender_weak_count(&self) -> usize {\n self.chan.sender_weak_count()\n }\n}\n\nimpl UnboundedSender {\n pub(crate) fn new(chan: chan::Tx) -> UnboundedSender {\n UnboundedSender { chan }\n }\n\n /// Attempts to send a message on this `UnboundedSender` without blocking.\n ///\n /// This method is not marked as `async` because sending a message to an unbounded channel\n /// never requires any form of waiting. This is due to the channel's infinite capacity,\n /// allowing the `send` operation to complete immediately. As a result, the `send` method can be\n /// used in both synchronous and asynchronous code without issues.\n ///\n /// If the receive half of the channel is closed, either due to [`close`]\n /// being called or the [`UnboundedReceiver`] having been dropped, this\n /// function returns an error. The error includes the value passed to `send`.\n ///\n /// [`close`]: UnboundedReceiver::close\n /// [`UnboundedReceiver`]: UnboundedReceiver\n pub fn send(&self, message: T) -> Result<(), SendError> {\n if !self.inc_num_messages() {\n return Err(SendError(message));\n }\n\n self.chan.send(message);\n Ok(())\n }\n\n fn inc_num_messages(&self) -> bool {\n use std::process;\n use std::sync::atomic::Ordering::{AcqRel, Acquire};\n\n let mut curr = self.chan.semaphore().0.load(Acquire);\n\n loop {\n if curr & 1 == 1 {\n return false;\n }\n\n if curr == usize::MAX ^ 1 {\n // Overflowed the ref count. There is no safe way to recover, so\n // abort the process. In practice, this should never happen.\n process::abort()\n }\n\n match self\n .chan\n .semaphore()\n .0\n .compare_exchange(curr, curr + 2, AcqRel, Acquire)\n {\n Ok(_) => return true,\n Err(actual) => {\n curr = actual;\n }\n }\n }\n }\n\n /// Completes when the receiver has dropped.\n ///\n /// This allows the producers to get notified when interest in the produced\n /// values is canceled and immediately stop doing work.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. Once the channel is closed, it stays closed\n /// forever and all future calls to `closed` will return immediately.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::mpsc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx1, rx) = mpsc::unbounded_channel::<()>();\n /// let tx2 = tx1.clone();\n /// let tx3 = tx1.clone();\n /// let tx4 = tx1.clone();\n /// let tx5 = tx1.clone();\n /// tokio::spawn(async move {\n /// drop(rx);\n /// });\n ///\n /// futures::join!(\n /// tx1.closed(),\n /// tx2.closed(),\n /// tx3.closed(),\n /// tx4.closed(),\n /// tx5.closed()\n /// );\n /// println!(\"Receiver dropped\");\n /// # }\n /// ```\n pub async fn closed(&self) {\n self.chan.closed().await;\n }\n\n /// Checks if the channel has been closed. This happens when the\n /// [`UnboundedReceiver`] is dropped, or when the\n /// [`UnboundedReceiver::close`] method is called.\n ///\n /// [`UnboundedReceiver`]: crate::sync::mpsc::UnboundedReceiver\n /// [`UnboundedReceiver::close`]: crate::sync::mpsc::UnboundedReceiver::close\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::mpsc::unbounded_channel::<()>();\n /// assert!(!tx.is_closed());\n ///\n /// let tx2 = tx.clone();\n /// assert!(!tx2.is_closed());\n ///\n /// drop(rx);\n /// assert!(tx.is_closed());\n /// assert!(tx2.is_closed());\n /// ```\n pub fn is_closed(&self) -> bool {\n self.chan.is_closed()\n }\n\n /// Returns `true` if senders belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::mpsc::unbounded_channel::<()>();\n /// let tx2 = tx.clone();\n /// assert!(tx.same_channel(&tx2));\n ///\n /// let (tx3, rx3) = tokio::sync::mpsc::unbounded_channel::<()>();\n /// assert!(!tx3.same_channel(&tx2));\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n self.chan.same_channel(&other.chan)\n }\n\n /// Converts the `UnboundedSender` to a [`WeakUnboundedSender`] that does not count\n /// towards RAII semantics, i.e. if all `UnboundedSender` instances of the\n /// channel were dropped and only `WeakUnboundedSender` instances remain,\n /// the channel is closed.\n #[must_use = \"Downgrade creates a WeakSender without destroying the original non-weak sender.\"]\n pub fn downgrade(&self) -> WeakUnboundedSender {\n WeakUnboundedSender {\n chan: self.chan.downgrade(),\n }\n }\n\n /// Returns the number of [`UnboundedSender`] handles.\n pub fn strong_count(&self) -> usize {\n self.chan.strong_count()\n }\n\n /// Returns the number of [`WeakUnboundedSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.chan.weak_count()\n }\n}\n\nimpl Clone for WeakUnboundedSender {\n fn clone(&self) -> Self {\n self.chan.increment_weak_count();\n\n WeakUnboundedSender {\n chan: self.chan.clone(),\n }\n }\n}\n\nimpl Drop for WeakUnboundedSender {\n fn drop(&mut self) {\n self.chan.decrement_weak_count();\n }\n}\n\nimpl WeakUnboundedSender {\n /// Tries to convert a `WeakUnboundedSender` into an [`UnboundedSender`].\n /// This will return `Some` if there are other `Sender` instances alive and\n /// the channel wasn't previously dropped, otherwise `None` is returned.\n pub fn upgrade(&self) -> Option> {\n chan::Tx::upgrade(self.chan.clone()).map(UnboundedSender::new)\n }\n\n /// Returns the number of [`UnboundedSender`] handles.\n pub fn strong_count(&self) -> usize {\n self.chan.strong_count()\n }\n\n /// Returns the number of [`WeakUnboundedSender`] handles.\n pub fn weak_count(&self) -> usize {\n self.chan.weak_count()\n }\n}\n\nimpl fmt::Debug for WeakUnboundedSender {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"WeakUnboundedSender\").finish()\n }\n}\n" }, { "path": "tokio/src/sync/mutex.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(unreachable_pub, dead_code))]\n\nuse crate::sync::batch_semaphore as semaphore;\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\n\nuse std::cell::UnsafeCell;\nuse std::error::Error;\nuse std::marker::PhantomData;\nuse std::ops::{Deref, DerefMut};\nuse std::sync::Arc;\nuse std::{fmt, mem, ptr};\n\n/// An asynchronous `Mutex`-like type.\n///\n/// This type acts similarly to [`std::sync::Mutex`], with two major\n/// differences: [`lock`] is an async method so does not block, and the lock\n/// guard is designed to be held across `.await` points.\n///\n/// Tokio's Mutex operates on a guaranteed FIFO basis.\n/// This means that the order in which tasks call the [`lock`] method is\n/// the exact order in which they will acquire the lock.\n///\n/// # Which kind of mutex should you use?\n///\n/// Contrary to popular belief, it is ok and often preferred to use the ordinary\n/// [`Mutex`][std] from the standard library in asynchronous code.\n///\n/// The feature that the async mutex offers over the blocking mutex is the\n/// ability to keep it locked across an `.await` point. This makes the async\n/// mutex more expensive than the blocking mutex, so the blocking mutex should\n/// be preferred in the cases where it can be used. The primary use case for the\n/// async mutex is to provide shared mutable access to IO resources such as a\n/// database connection. If the value behind the mutex is just data, it's\n/// usually appropriate to use a blocking mutex such as the one in the standard\n/// library or [`parking_lot`].\n///\n/// Note that, although the compiler will not prevent the std `Mutex` from holding\n/// its guard across `.await` points in situations where the task is not movable\n/// between threads, this virtually never leads to correct concurrent code in\n/// practice as it can easily lead to deadlocks.\n///\n/// A common pattern is to wrap the `Arc>` in a struct that provides\n/// non-async methods for performing operations on the data within, and only\n/// lock the mutex inside these methods. The [mini-redis] example provides an\n/// illustration of this pattern.\n///\n/// Additionally, when you _do_ want shared access to an IO resource, it is\n/// often better to spawn a task to manage the IO resource, and to use message\n/// passing to communicate with that task.\n///\n/// [std]: std::sync::Mutex\n/// [`parking_lot`]: https://docs.rs/parking_lot\n/// [mini-redis]: https://github.com/tokio-rs/mini-redis/blob/master/src/db.rs\n///\n/// # Examples:\n///\n/// ```rust,no_run\n/// use tokio::sync::Mutex;\n/// use std::sync::Arc;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let data1 = Arc::new(Mutex::new(0));\n/// let data2 = Arc::clone(&data1);\n///\n/// tokio::spawn(async move {\n/// let mut lock = data2.lock().await;\n/// *lock += 1;\n/// });\n///\n/// let mut lock = data1.lock().await;\n/// *lock += 1;\n/// # }\n/// ```\n///\n///\n/// ```rust,no_run\n/// use tokio::sync::Mutex;\n/// use std::sync::Arc;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let count = Arc::new(Mutex::new(0));\n///\n/// for i in 0..5 {\n/// let my_count = Arc::clone(&count);\n/// tokio::spawn(async move {\n/// for j in 0..10 {\n/// let mut lock = my_count.lock().await;\n/// *lock += 1;\n/// println!(\"{} {} {}\", i, j, lock);\n/// }\n/// });\n/// }\n///\n/// loop {\n/// if *count.lock().await >= 50 {\n/// break;\n/// }\n/// }\n/// println!(\"Count hit 50.\");\n/// # }\n/// ```\n/// There are a few things of note here to pay attention to in this example.\n/// 1. The mutex is wrapped in an [`Arc`] to allow it to be shared across\n/// threads.\n/// 2. Each spawned task obtains a lock and releases it on every iteration.\n/// 3. Mutation of the data protected by the Mutex is done by de-referencing\n/// the obtained lock as seen on lines 13 and 20.\n///\n/// Tokio's Mutex works in a simple FIFO (first in, first out) style where all\n/// calls to [`lock`] complete in the order they were performed. In that way the\n/// Mutex is \"fair\" and predictable in how it distributes the locks to inner\n/// data. Locks are released and reacquired after every iteration, so basically,\n/// each thread goes to the back of the line after it increments the value once.\n/// Note that there's some unpredictability to the timing between when the\n/// threads are started, but once they are going they alternate predictably.\n/// Finally, since there is only a single valid lock at any given time, there is\n/// no possibility of a race condition when mutating the inner value.\n///\n/// Note that in contrast to [`std::sync::Mutex`], this implementation does not\n/// poison the mutex when a thread holding the [`MutexGuard`] panics. In such a\n/// case, the mutex will be unlocked. If the panic is caught, this might leave\n/// the data protected by the mutex in an inconsistent state.\n///\n/// [`Mutex`]: struct@Mutex\n/// [`MutexGuard`]: struct@MutexGuard\n/// [`Arc`]: struct@std::sync::Arc\n/// [`std::sync::Mutex`]: struct@std::sync::Mutex\n/// [`Send`]: trait@std::marker::Send\n/// [`lock`]: method@Mutex::lock\npub struct Mutex {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n s: semaphore::Semaphore,\n c: UnsafeCell,\n}\n\n/// A handle to a held `Mutex`. The guard can be held across any `.await` point\n/// as it is [`Send`].\n///\n/// As long as you have this guard, you have exclusive access to the underlying\n/// `T`. The guard internally borrows the `Mutex`, so the mutex will not be\n/// dropped while a guard exists.\n///\n/// The lock is automatically released whenever the guard is dropped, at which\n/// point `lock` will succeed yet again.\n#[clippy::has_significant_drop]\n#[must_use = \"if unused the Mutex will immediately unlock\"]\npub struct MutexGuard<'a, T: ?Sized> {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n lock: &'a Mutex,\n}\n\n/// An owned handle to a held `Mutex`.\n///\n/// This guard is only available from a `Mutex` that is wrapped in an [`Arc`]. It\n/// is identical to `MutexGuard`, except that rather than borrowing the `Mutex`,\n/// it clones the `Arc`, incrementing the reference count. This means that\n/// unlike `MutexGuard`, it will have the `'static` lifetime.\n///\n/// As long as you have this guard, you have exclusive access to the underlying\n/// `T`. The guard internally keeps a reference-counted pointer to the original\n/// `Mutex`, so even if the lock goes away, the guard remains valid.\n///\n/// The lock is automatically released whenever the guard is dropped, at which\n/// point `lock` will succeed yet again.\n///\n/// [`Arc`]: std::sync::Arc\n#[clippy::has_significant_drop]\npub struct OwnedMutexGuard {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n lock: Arc>,\n}\n\n/// A handle to a held `Mutex` that has had a function applied to it via [`MutexGuard::map`].\n///\n/// This can be used to hold a subfield of the protected data.\n///\n/// [`MutexGuard::map`]: method@MutexGuard::map\n#[clippy::has_significant_drop]\n#[must_use = \"if unused the Mutex will immediately unlock\"]\npub struct MappedMutexGuard<'a, T: ?Sized> {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n s: &'a semaphore::Semaphore,\n data: *mut T,\n // Needed to tell the borrow checker that we are holding a `&mut T`\n marker: PhantomData<&'a mut T>,\n}\n\n/// A owned handle to a held `Mutex` that has had a function applied to it via\n/// [`OwnedMutexGuard::map`].\n///\n/// This can be used to hold a subfield of the protected data.\n///\n/// [`OwnedMutexGuard::map`]: method@OwnedMutexGuard::map\n#[clippy::has_significant_drop]\n#[must_use = \"if unused the Mutex will immediately unlock\"]\npub struct OwnedMappedMutexGuard {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n data: *mut U,\n lock: Arc>,\n}\n\n/// A helper type used when taking apart a `MutexGuard` without running its\n/// Drop implementation.\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct MutexGuardInner<'a, T: ?Sized> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n lock: &'a Mutex,\n}\n\n/// A helper type used when taking apart a `OwnedMutexGuard` without running\n/// its Drop implementation.\nstruct OwnedMutexGuardInner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n lock: Arc>,\n}\n\n/// A helper type used when taking apart a `MappedMutexGuard` without running\n/// its Drop implementation.\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct MappedMutexGuardInner<'a, T: ?Sized> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n s: &'a semaphore::Semaphore,\n data: *mut T,\n}\n\n/// A helper type used when taking apart a `OwnedMappedMutexGuard` without running\n/// its Drop implementation.\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct OwnedMappedMutexGuardInner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n data: *mut U,\n lock: Arc>,\n}\n\n// As long as T: Send, it's fine to send and share Mutex between threads.\n// If T was not Send, sending and sharing a Mutex would be bad, since you can\n// access T through Mutex.\nunsafe impl Send for Mutex where T: ?Sized + Send {}\nunsafe impl Sync for Mutex where T: ?Sized + Send {}\nunsafe impl Sync for MutexGuard<'_, T> where T: ?Sized + Send + Sync {}\nunsafe impl Sync for OwnedMutexGuard where T: ?Sized + Send + Sync {}\nunsafe impl<'a, T> Sync for MappedMutexGuard<'a, T> where T: ?Sized + Sync + 'a {}\nunsafe impl<'a, T> Send for MappedMutexGuard<'a, T> where T: ?Sized + Send + 'a {}\n\nunsafe impl Sync for OwnedMappedMutexGuard\nwhere\n T: ?Sized + Send + Sync,\n U: ?Sized + Send + Sync,\n{\n}\nunsafe impl Send for OwnedMappedMutexGuard\nwhere\n T: ?Sized + Send,\n U: ?Sized + Send,\n{\n}\n\n/// Error returned from the [`Mutex::try_lock`], [`RwLock::try_read`] and\n/// [`RwLock::try_write`] functions.\n///\n/// `Mutex::try_lock` operation will only fail if the mutex is already locked.\n///\n/// `RwLock::try_read` operation will only fail if the lock is currently held\n/// by an exclusive writer.\n///\n/// `RwLock::try_write` operation will only fail if the lock is currently held\n/// by any reader or by an exclusive writer.\n///\n/// [`Mutex::try_lock`]: Mutex::try_lock\n/// [`RwLock::try_read`]: fn@super::RwLock::try_read\n/// [`RwLock::try_write`]: fn@super::RwLock::try_write\n#[derive(Debug)]\npub struct TryLockError(pub(super) ());\n\nimpl fmt::Display for TryLockError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"operation would block\")\n }\n}\n\nimpl Error for TryLockError {}\n\n#[test]\n#[cfg(not(loom))]\nfn bounds() {\n fn check_send() {}\n fn check_unpin() {}\n // This has to take a value, since the async fn's return type is unnameable.\n fn check_send_sync_val(_t: T) {}\n fn check_send_sync() {}\n fn check_static() {}\n fn check_static_val(_t: T) {}\n\n check_send::>();\n check_send::>();\n check_unpin::>();\n check_send_sync::>();\n check_static::>();\n\n let mutex = Mutex::new(1);\n check_send_sync_val(mutex.lock());\n let arc_mutex = Arc::new(Mutex::new(1));\n check_send_sync_val(arc_mutex.clone().lock_owned());\n check_static_val(arc_mutex.lock_owned());\n}\n\nimpl Mutex {\n /// Creates a new lock in an unlocked state ready for use.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n ///\n /// let lock = Mutex::new(5);\n /// ```\n #[track_caller]\n pub fn new(t: T) -> Self\n where\n T: Sized,\n {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n\n tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Mutex\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n )\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let s = resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = false,\n );\n semaphore::Semaphore::new(1)\n });\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let s = semaphore::Semaphore::new(1);\n\n Self {\n c: UnsafeCell::new(t),\n s,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Creates a new lock in an unlocked state ready for use.\n ///\n /// When using the `tracing` [unstable feature], a `Mutex` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`Mutex::new`] should be used to create\n /// an instrumented object if that is needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n ///\n /// static LOCK: Mutex = Mutex::const_new(5);\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new(t: T) -> Self\n where\n T: Sized,\n {\n Self {\n c: UnsafeCell::new(t),\n s: semaphore::Semaphore::const_new(1),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Locks this mutex, causing the current task to yield until the lock has\n /// been acquired. When the lock has been acquired, function returns a\n /// [`MutexGuard`].\n ///\n /// If the mutex is available to be acquired immediately, then this call\n /// will typically not yield to the runtime. However, this is not guaranteed\n /// under all circumstances.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `lock` makes you lose your place in\n /// the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mutex = Mutex::new(1);\n ///\n /// let mut n = mutex.lock().await;\n /// *n = 2;\n /// # }\n /// ```\n pub async fn lock(&self) -> MutexGuard<'_, T> {\n let acquire_fut = async {\n self.acquire().await;\n\n MutexGuard {\n lock: self,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n self.resource_span.clone(),\n \"Mutex::lock\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = true,\n );\n });\n\n guard\n }\n\n /// Blockingly locks this `Mutex`. When the lock has been acquired, function returns a\n /// [`MutexGuard`].\n ///\n /// This method is intended for use cases where you\n /// need to use this mutex in asynchronous code as well as in synchronous code.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution context.\n ///\n /// - If you find yourself in an asynchronous execution context and needing\n /// to call some (synchronous) function which performs one of these\n /// `blocking_` operations, then consider wrapping that call inside\n /// [`spawn_blocking()`][crate::runtime::Handle::spawn_blocking]\n /// (or [`block_in_place()`][crate::task::block_in_place]).\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::sync::Arc;\n /// use tokio::sync::Mutex;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let mutex = Arc::new(Mutex::new(1));\n /// let lock = mutex.lock().await;\n ///\n /// let mutex1 = Arc::clone(&mutex);\n /// let blocking_task = tokio::task::spawn_blocking(move || {\n /// // This shall block until the `lock` is released.\n /// let mut n = mutex1.blocking_lock();\n /// *n = 2;\n /// });\n ///\n /// assert_eq!(*lock, 1);\n /// // Release the lock.\n /// drop(lock);\n ///\n /// // Await the completion of the blocking task.\n /// blocking_task.await.unwrap();\n ///\n /// // Assert uncontended.\n /// let n = mutex.try_lock().unwrap();\n /// assert_eq!(*n, 2);\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"lock_blocking\"))]\n pub fn blocking_lock(&self) -> MutexGuard<'_, T> {\n crate::future::block_on(self.lock())\n }\n\n /// Blockingly locks this `Mutex`. When the lock has been acquired, function returns an\n /// [`OwnedMutexGuard`].\n ///\n /// This method is identical to [`Mutex::blocking_lock`], except that the returned\n /// guard references the `Mutex` with an [`Arc`] rather than by borrowing\n /// it. Therefore, the `Mutex` must be wrapped in an `Arc` to call this\n /// method, and the guard will live for the `'static` lifetime, as it keeps\n /// the `Mutex` alive by holding an `Arc`.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution context.\n ///\n /// - If you find yourself in an asynchronous execution context and needing\n /// to call some (synchronous) function which performs one of these\n /// `blocking_` operations, then consider wrapping that call inside\n /// [`spawn_blocking()`][crate::runtime::Handle::spawn_blocking]\n /// (or [`block_in_place()`][crate::task::block_in_place]).\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::sync::Arc;\n /// use tokio::sync::Mutex;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let mutex = Arc::new(Mutex::new(1));\n /// let lock = mutex.lock().await;\n ///\n /// let mutex1 = Arc::clone(&mutex);\n /// let blocking_task = tokio::task::spawn_blocking(move || {\n /// // This shall block until the `lock` is released.\n /// let mut n = mutex1.blocking_lock_owned();\n /// *n = 2;\n /// });\n ///\n /// assert_eq!(*lock, 1);\n /// // Release the lock.\n /// drop(lock);\n ///\n /// // Await the completion of the blocking task.\n /// blocking_task.await.unwrap();\n ///\n /// // Assert uncontended.\n /// let n = mutex.try_lock().unwrap();\n /// assert_eq!(*n, 2);\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n pub fn blocking_lock_owned(self: Arc) -> OwnedMutexGuard {\n crate::future::block_on(self.lock_owned())\n }\n\n /// Locks this mutex, causing the current task to yield until the lock has\n /// been acquired. When the lock has been acquired, this returns an\n /// [`OwnedMutexGuard`].\n ///\n /// If the mutex is available to be acquired immediately, then this call\n /// will typically not yield to the runtime. However, this is not guaranteed\n /// under all circumstances.\n ///\n /// This method is identical to [`Mutex::lock`], except that the returned\n /// guard references the `Mutex` with an [`Arc`] rather than by borrowing\n /// it. Therefore, the `Mutex` must be wrapped in an `Arc` to call this\n /// method, and the guard will live for the `'static` lifetime, as it keeps\n /// the `Mutex` alive by holding an `Arc`.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `lock_owned` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n /// use std::sync::Arc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mutex = Arc::new(Mutex::new(1));\n ///\n /// let mut n = mutex.clone().lock_owned().await;\n /// *n = 2;\n /// # }\n /// ```\n ///\n /// [`Arc`]: std::sync::Arc\n pub async fn lock_owned(self: Arc) -> OwnedMutexGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = self.resource_span.clone();\n\n let acquire_fut = async {\n self.acquire().await;\n\n OwnedMutexGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n lock: self,\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n resource_span,\n \"Mutex::lock_owned\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = true,\n );\n });\n\n guard\n }\n\n async fn acquire(&self) {\n crate::trace::async_trace_leaf().await;\n\n self.s.acquire(1).await.unwrap_or_else(|_| {\n // The semaphore was closed. but, we never explicitly close it, and\n // we own it exclusively, which means that this can never happen.\n unreachable!()\n });\n }\n\n /// Attempts to acquire the lock, and returns [`TryLockError`] if the\n /// lock is currently held somewhere else.\n ///\n /// [`TryLockError`]: TryLockError\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n /// # async fn dox() -> Result<(), tokio::sync::TryLockError> {\n ///\n /// let mutex = Mutex::new(1);\n ///\n /// let n = mutex.try_lock()?;\n /// assert_eq!(*n, 1);\n /// # Ok(())\n /// # }\n /// ```\n pub fn try_lock(&self) -> Result, TryLockError> {\n match self.s.try_acquire(1) {\n Ok(()) => {\n let guard = MutexGuard {\n lock: self,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = true,\n );\n });\n\n Ok(guard)\n }\n Err(_) => Err(TryLockError(())),\n }\n }\n\n /// Returns a mutable reference to the underlying data.\n ///\n /// Since this call borrows the `Mutex` mutably, no actual locking needs to\n /// take place -- the mutable borrow statically guarantees no locks exist.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n ///\n /// fn main() {\n /// let mut mutex = Mutex::new(1);\n ///\n /// let n = mutex.get_mut();\n /// *n = 2;\n /// }\n /// ```\n pub fn get_mut(&mut self) -> &mut T {\n self.c.get_mut()\n }\n\n /// Attempts to acquire the lock, and returns [`TryLockError`] if the lock\n /// is currently held somewhere else.\n ///\n /// This method is identical to [`Mutex::try_lock`], except that the\n /// returned guard references the `Mutex` with an [`Arc`] rather than by\n /// borrowing it. Therefore, the `Mutex` must be wrapped in an `Arc` to call\n /// this method, and the guard will live for the `'static` lifetime, as it\n /// keeps the `Mutex` alive by holding an `Arc`.\n ///\n /// [`TryLockError`]: TryLockError\n /// [`Arc`]: std::sync::Arc\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n /// use std::sync::Arc;\n /// # async fn dox() -> Result<(), tokio::sync::TryLockError> {\n ///\n /// let mutex = Arc::new(Mutex::new(1));\n ///\n /// let n = mutex.clone().try_lock_owned()?;\n /// assert_eq!(*n, 1);\n /// # Ok(())\n /// # }\n pub fn try_lock_owned(self: Arc) -> Result, TryLockError> {\n match self.s.try_acquire(1) {\n Ok(()) => {\n let guard = OwnedMutexGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n lock: self,\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = true,\n );\n });\n\n Ok(guard)\n }\n Err(_) => Err(TryLockError(())),\n }\n }\n\n /// Consumes the mutex, returning the underlying data.\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Mutex;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mutex = Mutex::new(1);\n ///\n /// let n = mutex.into_inner();\n /// assert_eq!(n, 1);\n /// # }\n /// ```\n pub fn into_inner(self) -> T\n where\n T: Sized,\n {\n self.c.into_inner()\n }\n}\n\nimpl From for Mutex {\n fn from(s: T) -> Self {\n Self::new(s)\n }\n}\n\nimpl Default for Mutex\nwhere\n T: Default,\n{\n fn default() -> Self {\n Self::new(T::default())\n }\n}\n\nimpl std::fmt::Debug for Mutex\nwhere\n T: std::fmt::Debug,\n{\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n let mut d = f.debug_struct(\"Mutex\");\n match self.try_lock() {\n Ok(inner) => d.field(\"data\", &&*inner),\n Err(_) => d.field(\"data\", &format_args!(\"\")),\n };\n d.finish()\n }\n}\n\n// === impl MutexGuard ===\n\nimpl<'a, T: ?Sized> MutexGuard<'a, T> {\n fn skip_drop(self) -> MutexGuardInner<'a, T> {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the `resource_span` and then forgets the\n // original. In the end, we have not duplicated or forgotten any values.\n MutexGuardInner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: unsafe { std::ptr::read(&me.resource_span) },\n lock: me.lock,\n }\n }\n\n /// Makes a new [`MappedMutexGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the [`MutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `MutexGuard::map(...)`. A method\n /// would interfere with methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Mutex, MutexGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let foo = Mutex::new(Foo(1));\n ///\n /// {\n /// let mut mapped = MutexGuard::map(foo.lock().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *foo.lock().await);\n /// # }\n /// ```\n ///\n /// [`MutexGuard`]: struct@MutexGuard\n /// [`MappedMutexGuard`]: struct@MappedMutexGuard\n #[inline]\n pub fn map(mut this: Self, f: F) -> MappedMutexGuard<'a, U>\n where\n U: ?Sized,\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let inner = this.skip_drop();\n MappedMutexGuard {\n s: &inner.lock.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n }\n }\n\n /// Attempts to make a new [`MappedMutexGuard`] for a component of the locked data. The\n /// original guard is returned if the closure returns `None`.\n ///\n /// This operation cannot fail as the [`MutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `MutexGuard::try_map(...)`. A\n /// method would interfere with methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Mutex, MutexGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let foo = Mutex::new(Foo(1));\n ///\n /// {\n /// let mut mapped = MutexGuard::try_map(foo.lock().await, |f| Some(&mut f.0))\n /// .expect(\"should not fail\");\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *foo.lock().await);\n /// # }\n /// ```\n ///\n /// [`MutexGuard`]: struct@MutexGuard\n /// [`MappedMutexGuard`]: struct@MappedMutexGuard\n #[inline]\n pub fn try_map(mut this: Self, f: F) -> Result, Self>\n where\n U: ?Sized,\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let inner = this.skip_drop();\n Ok(MappedMutexGuard {\n s: &inner.lock.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n })\n }\n\n /// Returns a reference to the original `Mutex`.\n ///\n /// ```\n /// use tokio::sync::{Mutex, MutexGuard};\n ///\n /// async fn unlock_and_relock<'l>(guard: MutexGuard<'l, u32>) -> MutexGuard<'l, u32> {\n /// println!(\"1. contains: {:?}\", *guard);\n /// let mutex = MutexGuard::mutex(&guard);\n /// drop(guard);\n /// let guard = mutex.lock().await;\n /// println!(\"2. contains: {:?}\", *guard);\n /// guard\n /// }\n /// #\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// # let mutex = Mutex::new(0u32);\n /// # let guard = mutex.lock().await;\n /// # let _guard = unlock_and_relock(guard).await;\n /// # }\n /// ```\n #[inline]\n pub fn mutex(this: &Self) -> &'a Mutex {\n this.lock\n }\n}\n\nimpl Drop for MutexGuard<'_, T> {\n fn drop(&mut self) {\n self.lock.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = false,\n );\n });\n }\n}\n\nimpl Deref for MutexGuard<'_, T> {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n unsafe { &*self.lock.c.get() }\n }\n}\n\nimpl DerefMut for MutexGuard<'_, T> {\n fn deref_mut(&mut self) -> &mut Self::Target {\n unsafe { &mut *self.lock.c.get() }\n }\n}\n\nimpl fmt::Debug for MutexGuard<'_, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for MutexGuard<'_, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\n// === impl OwnedMutexGuard ===\n\nimpl OwnedMutexGuard {\n fn skip_drop(self) -> OwnedMutexGuardInner {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n unsafe {\n OwnedMutexGuardInner {\n lock: ptr::read(&me.lock),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: ptr::read(&me.resource_span),\n }\n }\n }\n\n /// Makes a new [`OwnedMappedMutexGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the [`OwnedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `OwnedMutexGuard::map(...)`. A method\n /// would interfere with methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Mutex, OwnedMutexGuard};\n /// use std::sync::Arc;\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let foo = Arc::new(Mutex::new(Foo(1)));\n ///\n /// {\n /// let mut mapped = OwnedMutexGuard::map(foo.clone().lock_owned().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *foo.lock().await);\n /// # }\n /// ```\n ///\n /// [`OwnedMutexGuard`]: struct@OwnedMutexGuard\n /// [`OwnedMappedMutexGuard`]: struct@OwnedMappedMutexGuard\n #[inline]\n pub fn map(mut this: Self, f: F) -> OwnedMappedMutexGuard\n where\n U: ?Sized,\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let inner = this.skip_drop();\n OwnedMappedMutexGuard {\n data,\n lock: inner.lock,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n }\n }\n\n /// Attempts to make a new [`OwnedMappedMutexGuard`] for a component of the locked data. The\n /// original guard is returned if the closure returns `None`.\n ///\n /// This operation cannot fail as the [`OwnedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `OwnedMutexGuard::try_map(...)`. A\n /// method would interfere with methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Mutex, OwnedMutexGuard};\n /// use std::sync::Arc;\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let foo = Arc::new(Mutex::new(Foo(1)));\n ///\n /// {\n /// let mut mapped = OwnedMutexGuard::try_map(foo.clone().lock_owned().await, |f| Some(&mut f.0))\n /// .expect(\"should not fail\");\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *foo.lock().await);\n /// # }\n /// ```\n ///\n /// [`OwnedMutexGuard`]: struct@OwnedMutexGuard\n /// [`OwnedMappedMutexGuard`]: struct@OwnedMappedMutexGuard\n #[inline]\n pub fn try_map(mut this: Self, f: F) -> Result, Self>\n where\n U: ?Sized,\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let inner = this.skip_drop();\n Ok(OwnedMappedMutexGuard {\n data,\n lock: inner.lock,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n })\n }\n\n /// Returns a reference to the original `Arc`.\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{Mutex, OwnedMutexGuard};\n ///\n /// async fn unlock_and_relock(guard: OwnedMutexGuard) -> OwnedMutexGuard {\n /// println!(\"1. contains: {:?}\", *guard);\n /// let mutex: Arc> = OwnedMutexGuard::mutex(&guard).clone();\n /// drop(guard);\n /// let guard = mutex.lock_owned().await;\n /// println!(\"2. contains: {:?}\", *guard);\n /// guard\n /// }\n /// #\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// # let mutex = Arc::new(Mutex::new(0u32));\n /// # let guard = mutex.lock_owned().await;\n /// # unlock_and_relock(guard).await;\n /// # }\n /// ```\n #[inline]\n pub fn mutex(this: &Self) -> &Arc> {\n &this.lock\n }\n}\n\nimpl Drop for OwnedMutexGuard {\n fn drop(&mut self) {\n self.lock.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = false,\n );\n });\n }\n}\n\nimpl Deref for OwnedMutexGuard {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n unsafe { &*self.lock.c.get() }\n }\n}\n\nimpl DerefMut for OwnedMutexGuard {\n fn deref_mut(&mut self) -> &mut Self::Target {\n unsafe { &mut *self.lock.c.get() }\n }\n}\n\nimpl fmt::Debug for OwnedMutexGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for OwnedMutexGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\n// === impl MappedMutexGuard ===\n\nimpl<'a, T: ?Sized> MappedMutexGuard<'a, T> {\n fn skip_drop(self) -> MappedMutexGuardInner<'a, T> {\n let me = mem::ManuallyDrop::new(self);\n MappedMutexGuardInner {\n s: me.s,\n data: me.data,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: unsafe { std::ptr::read(&me.resource_span) },\n }\n }\n\n /// Makes a new [`MappedMutexGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the [`MappedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `MappedMutexGuard::map(...)`. A\n /// method would interfere with methods of the same name on the contents of the locked data.\n ///\n /// [`MappedMutexGuard`]: struct@MappedMutexGuard\n #[inline]\n pub fn map(mut this: Self, f: F) -> MappedMutexGuard<'a, U>\n where\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let inner = this.skip_drop();\n MappedMutexGuard {\n s: inner.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n }\n }\n\n /// Attempts to make a new [`MappedMutexGuard`] for a component of the locked data. The\n /// original guard is returned if the closure returns `None`.\n ///\n /// This operation cannot fail as the [`MappedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `MappedMutexGuard::try_map(...)`. A\n /// method would interfere with methods of the same name on the contents of the locked data.\n ///\n /// [`MappedMutexGuard`]: struct@MappedMutexGuard\n #[inline]\n pub fn try_map(mut this: Self, f: F) -> Result, Self>\n where\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let inner = this.skip_drop();\n Ok(MappedMutexGuard {\n s: inner.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n })\n }\n}\n\nimpl<'a, T: ?Sized> Drop for MappedMutexGuard<'a, T> {\n fn drop(&mut self) {\n self.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = false,\n );\n });\n }\n}\n\nimpl<'a, T: ?Sized> Deref for MappedMutexGuard<'a, T> {\n type Target = T;\n fn deref(&self) -> &Self::Target {\n unsafe { &*self.data }\n }\n}\n\nimpl<'a, T: ?Sized> DerefMut for MappedMutexGuard<'a, T> {\n fn deref_mut(&mut self) -> &mut Self::Target {\n unsafe { &mut *self.data }\n }\n}\n\nimpl<'a, T: ?Sized + fmt::Debug> fmt::Debug for MappedMutexGuard<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized + fmt::Display> fmt::Display for MappedMutexGuard<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\n// === impl OwnedMappedMutexGuard ===\n\nimpl OwnedMappedMutexGuard {\n fn skip_drop(self) -> OwnedMappedMutexGuardInner {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n unsafe {\n OwnedMappedMutexGuardInner {\n data: me.data,\n lock: ptr::read(&me.lock),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: ptr::read(&me.resource_span),\n }\n }\n }\n\n /// Makes a new [`OwnedMappedMutexGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the [`OwnedMappedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `OwnedMappedMutexGuard::map(...)`. A method\n /// would interfere with methods of the same name on the contents of the locked data.\n ///\n /// [`OwnedMappedMutexGuard`]: struct@OwnedMappedMutexGuard\n #[inline]\n pub fn map(mut this: Self, f: F) -> OwnedMappedMutexGuard\n where\n F: FnOnce(&mut U) -> &mut S,\n {\n let data = f(&mut *this) as *mut S;\n let inner = this.skip_drop();\n OwnedMappedMutexGuard {\n data,\n lock: inner.lock,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n }\n }\n\n /// Attempts to make a new [`OwnedMappedMutexGuard`] for a component of the locked data. The\n /// original guard is returned if the closure returns `None`.\n ///\n /// This operation cannot fail as the [`OwnedMutexGuard`] passed in already locked the mutex.\n ///\n /// This is an associated function that needs to be used as `OwnedMutexGuard::try_map(...)`. A\n /// method would interfere with methods of the same name on the contents of the locked data.\n ///\n /// [`OwnedMutexGuard`]: struct@OwnedMutexGuard\n /// [`OwnedMappedMutexGuard`]: struct@OwnedMappedMutexGuard\n #[inline]\n pub fn try_map(mut this: Self, f: F) -> Result, Self>\n where\n F: FnOnce(&mut U) -> Option<&mut S>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut S,\n None => return Err(this),\n };\n let inner = this.skip_drop();\n Ok(OwnedMappedMutexGuard {\n data,\n lock: inner.lock,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: inner.resource_span,\n })\n }\n}\n\nimpl Drop for OwnedMappedMutexGuard {\n fn drop(&mut self) {\n self.lock.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n locked = false,\n );\n });\n }\n}\n\nimpl Deref for OwnedMappedMutexGuard {\n type Target = U;\n fn deref(&self) -> &Self::Target {\n unsafe { &*self.data }\n }\n}\n\nimpl DerefMut for OwnedMappedMutexGuard {\n fn deref_mut(&mut self) -> &mut Self::Target {\n unsafe { &mut *self.data }\n }\n}\n\nimpl fmt::Debug for OwnedMappedMutexGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for OwnedMappedMutexGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n" }, { "path": "tokio/src/sync/notify.rs", "content": "// Allow `unreachable_pub` warnings when sync is not enabled\n// due to the usage of `Notify` within the `rt` feature set.\n// When this module is compiled with `sync` enabled we will warn on\n// this lint. When `rt` is enabled we use `pub(crate)` which\n// triggers this warning but it is safe to ignore in this case.\n#![cfg_attr(not(feature = \"sync\"), allow(unreachable_pub, dead_code))]\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::Mutex;\nuse crate::util::linked_list::{self, GuardedLinkedList, LinkedList};\nuse crate::util::WakeList;\n\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::panic::{RefUnwindSafe, UnwindSafe};\nuse std::pin::Pin;\nuse std::ptr::NonNull;\nuse std::sync::atomic::Ordering::{self, Acquire, Relaxed, Release, SeqCst};\nuse std::sync::Arc;\nuse std::task::{Context, Poll, Waker};\n\ntype WaitList = LinkedList::Target>;\ntype GuardedWaitList = GuardedLinkedList::Target>;\n\n/// Notifies a single task to wake up.\n///\n/// `Notify` provides a basic mechanism to notify a single task of an event.\n/// `Notify` itself does not carry any data. Instead, it is to be used to signal\n/// another task to perform an operation.\n///\n/// A `Notify` can be thought of as a [`Semaphore`] starting with 0 permits. The\n/// [`notified().await`] method waits for a permit to become available, and\n/// [`notify_one()`] sets a permit **if there currently are no available\n/// permits**.\n///\n/// The synchronization details of `Notify` are similar to\n/// [`thread::park`][park] and [`Thread::unpark`][unpark] from std. A [`Notify`]\n/// value contains a single permit. [`notified().await`] waits for the permit to\n/// be made available, consumes the permit, and resumes. [`notify_one()`] sets\n/// the permit, waking a pending task if there is one.\n///\n/// If `notify_one()` is called **before** `notified().await`, then the next\n/// call to `notified().await` will complete immediately, consuming the permit.\n/// Any subsequent calls to `notified().await` will wait for a new permit.\n///\n/// If `notify_one()` is called **multiple** times before `notified().await`,\n/// only a **single** permit is stored. The next call to `notified().await` will\n/// complete immediately, but the one after will wait for a new permit.\n///\n/// # Examples\n///\n/// Basic usage.\n///\n/// ```\n/// use tokio::sync::Notify;\n/// use std::sync::Arc;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let notify = Arc::new(Notify::new());\n/// let notify2 = notify.clone();\n///\n/// let handle = tokio::spawn(async move {\n/// notify2.notified().await;\n/// println!(\"received notification\");\n/// });\n///\n/// println!(\"sending notification\");\n/// notify.notify_one();\n///\n/// // Wait for task to receive notification.\n/// handle.await.unwrap();\n/// # }\n/// ```\n///\n/// Unbound multi-producer single-consumer (mpsc) channel.\n///\n/// No wakeups can be lost when using this channel because the call to\n/// `notify_one()` will store a permit in the `Notify`, which the following call\n/// to `notified()` will consume.\n///\n/// ```\n/// use tokio::sync::Notify;\n///\n/// use std::collections::VecDeque;\n/// use std::sync::Mutex;\n///\n/// struct Channel {\n/// values: Mutex>,\n/// notify: Notify,\n/// }\n///\n/// impl Channel {\n/// pub fn send(&self, value: T) {\n/// self.values.lock().unwrap()\n/// .push_back(value);\n///\n/// // Notify the consumer a value is available\n/// self.notify.notify_one();\n/// }\n///\n/// // This is a single-consumer channel, so several concurrent calls to\n/// // `recv` are not allowed.\n/// pub async fn recv(&self) -> T {\n/// loop {\n/// // Drain values\n/// if let Some(value) = self.values.lock().unwrap().pop_front() {\n/// return value;\n/// }\n///\n/// // Wait for values to be available\n/// self.notify.notified().await;\n/// }\n/// }\n/// }\n/// ```\n///\n/// Unbound multi-producer multi-consumer (mpmc) channel.\n///\n/// The call to [`enable`] is important because otherwise if you have two\n/// calls to `recv` and two calls to `send` in parallel, the following could\n/// happen:\n///\n/// 1. Both calls to `try_recv` return `None`.\n/// 2. Both new elements are added to the vector.\n/// 3. The `notify_one` method is called twice, adding only a single\n/// permit to the `Notify`.\n/// 4. Both calls to `recv` reach the `Notified` future. One of them\n/// consumes the permit, and the other sleeps forever.\n///\n/// By adding the `Notified` futures to the list by calling `enable` before\n/// `try_recv`, the `notify_one` calls in step three would remove the\n/// futures from the list and mark them notified instead of adding a permit\n/// to the `Notify`. This ensures that both futures are woken.\n///\n/// Notice that this failure can only happen if there are two concurrent calls\n/// to `recv`. This is why the mpsc example above does not require a call to\n/// `enable`.\n///\n/// ```\n/// use tokio::sync::Notify;\n///\n/// use std::collections::VecDeque;\n/// use std::sync::Mutex;\n///\n/// struct Channel {\n/// messages: Mutex>,\n/// notify_on_sent: Notify,\n/// }\n///\n/// impl Channel {\n/// pub fn send(&self, msg: T) {\n/// let mut locked_queue = self.messages.lock().unwrap();\n/// locked_queue.push_back(msg);\n/// drop(locked_queue);\n///\n/// // Send a notification to one of the calls currently\n/// // waiting in a call to `recv`.\n/// self.notify_on_sent.notify_one();\n/// }\n///\n/// pub fn try_recv(&self) -> Option {\n/// let mut locked_queue = self.messages.lock().unwrap();\n/// locked_queue.pop_front()\n/// }\n///\n/// pub async fn recv(&self) -> T {\n/// let future = self.notify_on_sent.notified();\n/// tokio::pin!(future);\n///\n/// loop {\n/// // Make sure that no wakeup is lost if we get\n/// // `None` from `try_recv`.\n/// future.as_mut().enable();\n///\n/// if let Some(msg) = self.try_recv() {\n/// return msg;\n/// }\n///\n/// // Wait for a call to `notify_one`.\n/// //\n/// // This uses `.as_mut()` to avoid consuming the future,\n/// // which lets us call `Pin::set` below.\n/// future.as_mut().await;\n///\n/// // Reset the future in case another call to\n/// // `try_recv` got the message before us.\n/// future.set(self.notify_on_sent.notified());\n/// }\n/// }\n/// }\n/// ```\n///\n/// [park]: std::thread::park\n/// [unpark]: std::thread::Thread::unpark\n/// [`notified().await`]: Notify::notified()\n/// [`notify_one()`]: Notify::notify_one()\n/// [`enable`]: Notified::enable()\n/// [`Semaphore`]: crate::sync::Semaphore\n#[derive(Debug)]\npub struct Notify {\n // `state` uses 2 bits to store one of `EMPTY`,\n // `WAITING` or `NOTIFIED`. The rest of the bits\n // are used to store the number of times `notify_waiters`\n // was called.\n //\n // Throughout the code there are two assumptions:\n // - state can be transitioned *from* `WAITING` only if\n // `waiters` lock is held\n // - number of times `notify_waiters` was called can\n // be modified only if `waiters` lock is held\n state: AtomicUsize,\n waiters: Mutex,\n}\n\n#[derive(Debug)]\nstruct Waiter {\n /// Intrusive linked-list pointers.\n pointers: linked_list::Pointers,\n\n /// Waiting task's waker. Depending on the value of `notification`,\n /// this field is either protected by the `waiters` lock in\n /// `Notify`, or it is exclusively owned by the enclosing `Waiter`.\n waker: UnsafeCell>,\n\n /// Notification for this waiter. Uses 2 bits to store if and how was\n /// notified, 1 bit for storing if it was woken up using FIFO or LIFO, and\n /// the rest of it is unused.\n /// * if it's `None`, then `waker` is protected by the `waiters` lock.\n /// * if it's `Some`, then `waker` is exclusively owned by the\n /// enclosing `Waiter` and can be accessed without locking.\n notification: AtomicNotification,\n\n /// Should not be `Unpin`.\n _p: PhantomPinned,\n}\n\nimpl Waiter {\n fn new() -> Waiter {\n Waiter {\n pointers: linked_list::Pointers::new(),\n waker: UnsafeCell::new(None),\n notification: AtomicNotification::none(),\n _p: PhantomPinned,\n }\n }\n}\n\ngenerate_addr_of_methods! {\n impl<> Waiter {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull> {\n &self.pointers\n }\n }\n}\n\n// No notification.\nconst NOTIFICATION_NONE: usize = 0b000;\n\n// Notification type used by `notify_one`.\nconst NOTIFICATION_ONE: usize = 0b001;\n\n// Notification type used by `notify_last`.\nconst NOTIFICATION_LAST: usize = 0b101;\n\n// Notification type used by `notify_waiters`.\nconst NOTIFICATION_ALL: usize = 0b010;\n\n/// Notification for a `Waiter`.\n/// This struct is equivalent to `Option`, but uses\n/// `AtomicUsize` inside for atomic operations.\n#[derive(Debug)]\nstruct AtomicNotification(AtomicUsize);\n\nimpl AtomicNotification {\n fn none() -> Self {\n AtomicNotification(AtomicUsize::new(NOTIFICATION_NONE))\n }\n\n /// Store-release a notification.\n /// This method should be called exactly once.\n fn store_release(&self, notification: Notification) {\n let data: usize = match notification {\n Notification::All => NOTIFICATION_ALL,\n Notification::One(NotifyOneStrategy::Fifo) => NOTIFICATION_ONE,\n Notification::One(NotifyOneStrategy::Lifo) => NOTIFICATION_LAST,\n };\n self.0.store(data, Release);\n }\n\n fn load(&self, ordering: Ordering) -> Option {\n let data = self.0.load(ordering);\n match data {\n NOTIFICATION_NONE => None,\n NOTIFICATION_ONE => Some(Notification::One(NotifyOneStrategy::Fifo)),\n NOTIFICATION_LAST => Some(Notification::One(NotifyOneStrategy::Lifo)),\n NOTIFICATION_ALL => Some(Notification::All),\n _ => unreachable!(),\n }\n }\n\n /// Clears the notification.\n /// This method is used by a `Notified` future to consume the\n /// notification. It uses relaxed ordering and should be only\n /// used once the atomic notification is no longer shared.\n fn clear(&self) {\n self.0.store(NOTIFICATION_NONE, Relaxed);\n }\n}\n\n#[derive(Debug, PartialEq, Eq)]\n#[repr(usize)]\nenum NotifyOneStrategy {\n Fifo,\n Lifo,\n}\n\n#[derive(Debug, PartialEq, Eq)]\n#[repr(usize)]\nenum Notification {\n One(NotifyOneStrategy),\n All,\n}\n\n/// List used in `Notify::notify_waiters`. It wraps a guarded linked list\n/// and gates the access to it on `notify.waiters` mutex. It also empties\n/// the list on drop.\nstruct NotifyWaitersList<'a> {\n list: GuardedWaitList,\n is_empty: bool,\n notify: &'a Notify,\n}\n\nimpl<'a> NotifyWaitersList<'a> {\n fn new(\n unguarded_list: WaitList,\n guard: Pin<&'a Waiter>,\n notify: &'a Notify,\n ) -> NotifyWaitersList<'a> {\n let guard_ptr = NonNull::from(guard.get_ref());\n let list = unguarded_list.into_guarded(guard_ptr);\n NotifyWaitersList {\n list,\n is_empty: false,\n notify,\n }\n }\n\n /// Removes the last element from the guarded list. Modifying this list\n /// requires an exclusive access to the main list in `Notify`.\n fn pop_back_locked(&mut self, _waiters: &mut WaitList) -> Option> {\n let result = self.list.pop_back();\n if result.is_none() {\n // Save information about emptiness to avoid waiting for lock\n // in the destructor.\n self.is_empty = true;\n }\n result\n }\n}\n\nimpl Drop for NotifyWaitersList<'_> {\n fn drop(&mut self) {\n // If the list is not empty, we unlink all waiters from it.\n // We do not wake the waiters to avoid double panics.\n if !self.is_empty {\n let _lock_guard = self.notify.waiters.lock();\n while let Some(waiter) = self.list.pop_back() {\n // Safety: we never make mutable references to waiters.\n let waiter = unsafe { waiter.as_ref() };\n waiter.notification.store_release(Notification::All);\n }\n }\n }\n}\n\n/// Future returned from [`Notify::notified()`].\n///\n/// This future is fused, so once it has completed, any future calls to poll\n/// will immediately return `Poll::Ready`.\n#[derive(Debug)]\n#[must_use = \"futures do nothing unless you `.await` or poll them\"]\npub struct Notified<'a> {\n /// The `Notify` being received on.\n notify: &'a Notify,\n\n /// The current state of the receiving process.\n state: State,\n\n /// Number of calls to `notify_waiters` at the time of creation.\n notify_waiters_calls: usize,\n\n /// Entry in the waiter `LinkedList`.\n waiter: Waiter,\n}\n\nunsafe impl<'a> Send for Notified<'a> {}\nunsafe impl<'a> Sync for Notified<'a> {}\n\n/// Future returned from [`Notify::notified_owned()`].\n///\n/// This future is fused, so once it has completed, any future calls to poll\n/// will immediately return `Poll::Ready`.\n#[derive(Debug)]\n#[must_use = \"futures do nothing unless you `.await` or poll them\"]\npub struct OwnedNotified {\n /// The `Notify` being received on.\n notify: Arc,\n\n /// The current state of the receiving process.\n state: State,\n\n /// Number of calls to `notify_waiters` at the time of creation.\n notify_waiters_calls: usize,\n\n /// Entry in the waiter `LinkedList`.\n waiter: Waiter,\n}\n\nunsafe impl Sync for OwnedNotified {}\n\n/// A custom `project` implementation is used in place of `pin-project-lite`\n/// as a custom drop for [`Notified`] and [`OwnedNotified`] implementation\n/// is needed.\nstruct NotifiedProject<'a> {\n notify: &'a Notify,\n state: &'a mut State,\n notify_waiters_calls: &'a usize,\n waiter: &'a Waiter,\n}\n\n#[derive(Debug)]\nenum State {\n Init,\n Waiting,\n Done,\n}\n\nconst NOTIFY_WAITERS_SHIFT: usize = 2;\nconst STATE_MASK: usize = (1 << NOTIFY_WAITERS_SHIFT) - 1;\nconst NOTIFY_WAITERS_CALLS_MASK: usize = !STATE_MASK;\n\n/// Initial \"idle\" state.\nconst EMPTY: usize = 0;\n\n/// One or more threads are currently waiting to be notified.\nconst WAITING: usize = 1;\n\n/// Pending notification.\nconst NOTIFIED: usize = 2;\n\nfn set_state(data: usize, state: usize) -> usize {\n (data & NOTIFY_WAITERS_CALLS_MASK) | (state & STATE_MASK)\n}\n\nfn get_state(data: usize) -> usize {\n data & STATE_MASK\n}\n\nfn get_num_notify_waiters_calls(data: usize) -> usize {\n (data & NOTIFY_WAITERS_CALLS_MASK) >> NOTIFY_WAITERS_SHIFT\n}\n\nfn inc_num_notify_waiters_calls(data: usize) -> usize {\n data + (1 << NOTIFY_WAITERS_SHIFT)\n}\n\nfn atomic_inc_num_notify_waiters_calls(data: &AtomicUsize) {\n data.fetch_add(1 << NOTIFY_WAITERS_SHIFT, SeqCst);\n}\n\nimpl Notify {\n /// Create a new `Notify`, initialized without a permit.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Notify;\n ///\n /// let notify = Notify::new();\n /// ```\n pub fn new() -> Notify {\n Notify {\n state: AtomicUsize::new(0),\n waiters: Mutex::new(LinkedList::new()),\n }\n }\n\n /// Create a new `Notify`, initialized without a permit.\n ///\n /// When using the `tracing` [unstable feature], a `Notify` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`Notify::new`] should be used to create\n /// an instrumented object if that is needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Notify;\n ///\n /// static NOTIFY: Notify = Notify::const_new();\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new() -> Notify {\n Notify {\n state: AtomicUsize::new(0),\n waiters: Mutex::const_new(LinkedList::new()),\n }\n }\n\n /// Wait for a notification.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn notified(&self);\n /// ```\n ///\n /// Each `Notify` value holds a single permit. If a permit is available from\n /// an earlier call to [`notify_one()`], then `notified().await` will complete\n /// immediately, consuming that permit. Otherwise, `notified().await` waits\n /// for a permit to be made available by the next call to `notify_one()`.\n ///\n /// The `Notified` future is not guaranteed to receive wakeups from calls to\n /// `notify_one()` if it has not yet been polled. See the documentation for\n /// [`Notified::enable()`] for more details.\n ///\n /// The `Notified` future is guaranteed to receive wakeups from\n /// `notify_waiters()` as soon as it has been created, even if it has not\n /// yet been polled.\n ///\n /// [`notify_one()`]: Notify::notify_one\n /// [`Notified::enable()`]: Notified::enable\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute notifications in the order\n /// they were requested. Cancelling a call to `notified` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Notify;\n /// use std::sync::Arc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let notify = Arc::new(Notify::new());\n /// let notify2 = notify.clone();\n ///\n /// tokio::spawn(async move {\n /// notify2.notified().await;\n /// println!(\"received notification\");\n /// });\n ///\n /// println!(\"sending notification\");\n /// notify.notify_one();\n /// # }\n /// ```\n pub fn notified(&self) -> Notified<'_> {\n // we load the number of times notify_waiters\n // was called and store that in the future.\n let state = self.state.load(SeqCst);\n Notified {\n notify: self,\n state: State::Init,\n notify_waiters_calls: get_num_notify_waiters_calls(state),\n waiter: Waiter::new(),\n }\n }\n\n /// Wait for a notification with an owned `Future`.\n ///\n /// Unlike [`Self::notified`] which returns a future tied to the `Notify`'s\n /// lifetime, `notified_owned` creates a self-contained future that owns its\n /// notification state, making it safe to move between threads.\n ///\n /// See [`Self::notified`] for more details.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute notifications in the order\n /// they were requested. Cancelling a call to `notified_owned` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Notify;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let notify = Arc::new(Notify::new());\n ///\n /// for _ in 0..10 {\n /// let notified = notify.clone().notified_owned();\n /// tokio::spawn(async move {\n /// notified.await;\n /// println!(\"received notification\");\n /// });\n /// }\n ///\n /// println!(\"sending notification\");\n /// notify.notify_waiters();\n /// # }\n /// ```\n pub fn notified_owned(self: Arc) -> OwnedNotified {\n // we load the number of times notify_waiters\n // was called and store that in the future.\n let state = self.state.load(SeqCst);\n OwnedNotified {\n notify: self,\n state: State::Init,\n notify_waiters_calls: get_num_notify_waiters_calls(state),\n waiter: Waiter::new(),\n }\n }\n /// Notifies the first waiting task.\n ///\n /// If a task is currently waiting, that task is notified. Otherwise, a\n /// permit is stored in this `Notify` value and the **next** call to\n /// [`notified().await`] will complete immediately consuming the permit made\n /// available by this call to `notify_one()`.\n ///\n /// At most one permit may be stored by `Notify`. Many sequential calls to\n /// `notify_one` will result in a single permit being stored. The next call to\n /// `notified().await` will complete immediately, but the one after that\n /// will wait.\n ///\n /// [`notified().await`]: Notify::notified()\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Notify;\n /// use std::sync::Arc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let notify = Arc::new(Notify::new());\n /// let notify2 = notify.clone();\n ///\n /// tokio::spawn(async move {\n /// notify2.notified().await;\n /// println!(\"received notification\");\n /// });\n ///\n /// println!(\"sending notification\");\n /// notify.notify_one();\n /// # }\n /// ```\n // Alias for old name in 0.x\n #[cfg_attr(docsrs, doc(alias = \"notify\"))]\n pub fn notify_one(&self) {\n self.notify_with_strategy(NotifyOneStrategy::Fifo);\n }\n\n /// Notifies the last waiting task.\n ///\n /// This function behaves similar to `notify_one`. The only difference is that it wakes\n /// the most recently added waiter instead of the oldest waiter.\n ///\n /// Check the [`notify_one()`] documentation for more info and\n /// examples.\n ///\n /// [`notify_one()`]: Notify::notify_one\n pub fn notify_last(&self) {\n self.notify_with_strategy(NotifyOneStrategy::Lifo);\n }\n\n fn notify_with_strategy(&self, strategy: NotifyOneStrategy) {\n // Load the current state\n let mut curr = self.state.load(SeqCst);\n\n // If the state is `EMPTY`, transition to `NOTIFIED` and return.\n while let EMPTY | NOTIFIED = get_state(curr) {\n // The compare-exchange from `NOTIFIED` -> `NOTIFIED` is intended. A\n // happens-before synchronization must happen between this atomic\n // operation and a task calling `notified().await`.\n let new = set_state(curr, NOTIFIED);\n let res = self.state.compare_exchange(curr, new, SeqCst, SeqCst);\n\n match res {\n // No waiters, no further work to do\n Ok(_) => return,\n Err(actual) => {\n curr = actual;\n }\n }\n }\n\n // There are waiters, the lock must be acquired to notify.\n let mut waiters = self.waiters.lock();\n\n // The state must be reloaded while the lock is held. The state may only\n // transition out of WAITING while the lock is held.\n curr = self.state.load(SeqCst);\n\n if let Some(waker) = notify_locked(&mut waiters, &self.state, curr, strategy) {\n drop(waiters);\n waker.wake();\n }\n }\n\n /// Notifies all waiting tasks.\n ///\n /// If a task is currently waiting, that task is notified. Unlike with\n /// `notify_one()`, no permit is stored to be used by the next call to\n /// `notified().await`. The purpose of this method is to notify all\n /// already registered waiters. Registering for notification is done by\n /// acquiring an instance of the `Notified` future via calling `notified()`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Notify;\n /// use std::sync::Arc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let notify = Arc::new(Notify::new());\n /// let notify2 = notify.clone();\n ///\n /// let notified1 = notify.notified();\n /// let notified2 = notify.notified();\n ///\n /// let handle = tokio::spawn(async move {\n /// println!(\"sending notifications\");\n /// notify2.notify_waiters();\n /// });\n ///\n /// notified1.await;\n /// notified2.await;\n /// println!(\"received notifications\");\n /// # }\n /// ```\n pub fn notify_waiters(&self) {\n self.lock_waiter_list().notify_waiters();\n }\n\n fn inner_notify_waiters<'a>(\n &'a self,\n curr: usize,\n mut waiters: crate::loom::sync::MutexGuard<'a, LinkedList>,\n ) {\n if matches!(get_state(curr), EMPTY | NOTIFIED) {\n // There are no waiting tasks. All we need to do is increment the\n // number of times this method was called.\n atomic_inc_num_notify_waiters_calls(&self.state);\n return;\n }\n\n // Increment the number of times this method was called\n // and transition to empty.\n let new_state = set_state(inc_num_notify_waiters_calls(curr), EMPTY);\n self.state.store(new_state, SeqCst);\n\n // It is critical for `GuardedLinkedList` safety that the guard node is\n // pinned in memory and is not dropped until the guarded list is dropped.\n let guard = Waiter::new();\n pin!(guard);\n\n // We move all waiters to a secondary list. It uses a `GuardedLinkedList`\n // underneath to allow every waiter to safely remove itself from it.\n //\n // * This list will be still guarded by the `waiters` lock.\n // `NotifyWaitersList` wrapper makes sure we hold the lock to modify it.\n // * This wrapper will empty the list on drop. It is critical for safety\n // that we will not leave any list entry with a pointer to the local\n // guard node after this function returns / panics.\n let mut list = NotifyWaitersList::new(std::mem::take(&mut *waiters), guard.as_ref(), self);\n\n let mut wakers = WakeList::new();\n 'outer: loop {\n while wakers.can_push() {\n match list.pop_back_locked(&mut waiters) {\n Some(waiter) => {\n // Safety: we never make mutable references to waiters.\n let waiter = unsafe { waiter.as_ref() };\n\n // Safety: we hold the lock, so we can access the waker.\n if let Some(waker) =\n unsafe { waiter.waker.with_mut(|waker| (*waker).take()) }\n {\n wakers.push(waker);\n }\n\n // This waiter is unlinked and will not be shared ever again, release it.\n waiter.notification.store_release(Notification::All);\n }\n None => {\n break 'outer;\n }\n }\n }\n\n // Release the lock before notifying.\n drop(waiters);\n\n // One of the wakers may panic, but the remaining waiters will still\n // be unlinked from the list in `NotifyWaitersList` destructor.\n wakers.wake_all();\n\n // Acquire the lock again.\n waiters = self.waiters.lock();\n }\n\n // Release the lock before notifying\n drop(waiters);\n\n wakers.wake_all();\n }\n\n pub(crate) fn lock_waiter_list(&self) -> NotifyGuard<'_> {\n let guarded_waiters = self.waiters.lock();\n\n // The state must be loaded while the lock is held. The state may only\n // transition out of WAITING while the lock is held.\n let current_state = self.state.load(SeqCst);\n\n NotifyGuard {\n guarded_notify: self,\n guarded_waiters,\n current_state,\n }\n }\n}\n\nimpl Default for Notify {\n fn default() -> Notify {\n Notify::new()\n }\n}\n\nimpl UnwindSafe for Notify {}\nimpl RefUnwindSafe for Notify {}\n\nfn notify_locked(\n waiters: &mut WaitList,\n state: &AtomicUsize,\n curr: usize,\n strategy: NotifyOneStrategy,\n) -> Option {\n match get_state(curr) {\n EMPTY | NOTIFIED => {\n let res = state.compare_exchange(curr, set_state(curr, NOTIFIED), SeqCst, SeqCst);\n\n match res {\n Ok(_) => None,\n Err(actual) => {\n let actual_state = get_state(actual);\n assert!(actual_state == EMPTY || actual_state == NOTIFIED);\n state.store(set_state(actual, NOTIFIED), SeqCst);\n None\n }\n }\n }\n WAITING => {\n // At this point, it is guaranteed that the state will not\n // concurrently change as holding the lock is required to\n // transition **out** of `WAITING`.\n //\n // Get a pending waiter using one of the available dequeue strategies.\n let waiter = match strategy {\n NotifyOneStrategy::Fifo => waiters.pop_back().unwrap(),\n NotifyOneStrategy::Lifo => waiters.pop_front().unwrap(),\n };\n\n // Safety: we never make mutable references to waiters.\n let waiter = unsafe { waiter.as_ref() };\n\n // Safety: we hold the lock, so we can access the waker.\n let waker = unsafe { waiter.waker.with_mut(|waker| (*waker).take()) };\n\n // This waiter is unlinked and will not be shared ever again, release it.\n waiter\n .notification\n .store_release(Notification::One(strategy));\n\n if waiters.is_empty() {\n // As this the **final** waiter in the list, the state\n // must be transitioned to `EMPTY`. As transitioning\n // **from** `WAITING` requires the lock to be held, a\n // `store` is sufficient.\n state.store(set_state(curr, EMPTY), SeqCst);\n }\n waker\n }\n _ => unreachable!(),\n }\n}\n\n// ===== impl Notified =====\n\nimpl Notified<'_> {\n /// Adds this future to the list of futures that are ready to receive\n /// wakeups from calls to [`notify_one`].\n ///\n /// Polling the future also adds it to the list, so this method should only\n /// be used if you want to add the future to the list before the first call\n /// to `poll`. (In fact, this method is equivalent to calling `poll` except\n /// that no `Waker` is registered.)\n ///\n /// This has no effect on notifications sent using [`notify_waiters`], which\n /// are received as long as they happen after the creation of the `Notified`\n /// regardless of whether `enable` or `poll` has been called.\n ///\n /// This method returns true if the `Notified` is ready. This happens in the\n /// following situations:\n ///\n /// 1. The `notify_waiters` method was called between the creation of the\n /// `Notified` and the call to this method.\n /// 2. This is the first call to `enable` or `poll` on this future, and the\n /// `Notify` was holding a permit from a previous call to `notify_one`.\n /// The call consumes the permit in that case.\n /// 3. The future has previously been enabled or polled, and it has since\n /// then been marked ready by either consuming a permit from the\n /// `Notify`, or by a call to `notify_one` or `notify_waiters` that\n /// removed it from the list of futures ready to receive wakeups.\n ///\n /// If this method returns true, any future calls to poll on the same future\n /// will immediately return `Poll::Ready`.\n ///\n /// # Examples\n ///\n /// Unbound multi-producer multi-consumer (mpmc) channel.\n ///\n /// The call to `enable` is important because otherwise if you have two\n /// calls to `recv` and two calls to `send` in parallel, the following could\n /// happen:\n ///\n /// 1. Both calls to `try_recv` return `None`.\n /// 2. Both new elements are added to the vector.\n /// 3. The `notify_one` method is called twice, adding only a single\n /// permit to the `Notify`.\n /// 4. Both calls to `recv` reach the `Notified` future. One of them\n /// consumes the permit, and the other sleeps forever.\n ///\n /// By adding the `Notified` futures to the list by calling `enable` before\n /// `try_recv`, the `notify_one` calls in step three would remove the\n /// futures from the list and mark them notified instead of adding a permit\n /// to the `Notify`. This ensures that both futures are woken.\n ///\n /// ```\n /// use tokio::sync::Notify;\n ///\n /// use std::collections::VecDeque;\n /// use std::sync::Mutex;\n ///\n /// struct Channel {\n /// messages: Mutex>,\n /// notify_on_sent: Notify,\n /// }\n ///\n /// impl Channel {\n /// pub fn send(&self, msg: T) {\n /// let mut locked_queue = self.messages.lock().unwrap();\n /// locked_queue.push_back(msg);\n /// drop(locked_queue);\n ///\n /// // Send a notification to one of the calls currently\n /// // waiting in a call to `recv`.\n /// self.notify_on_sent.notify_one();\n /// }\n ///\n /// pub fn try_recv(&self) -> Option {\n /// let mut locked_queue = self.messages.lock().unwrap();\n /// locked_queue.pop_front()\n /// }\n ///\n /// pub async fn recv(&self) -> T {\n /// let future = self.notify_on_sent.notified();\n /// tokio::pin!(future);\n ///\n /// loop {\n /// // Make sure that no wakeup is lost if we get\n /// // `None` from `try_recv`.\n /// future.as_mut().enable();\n ///\n /// if let Some(msg) = self.try_recv() {\n /// return msg;\n /// }\n ///\n /// // Wait for a call to `notify_one`.\n /// //\n /// // This uses `.as_mut()` to avoid consuming the future,\n /// // which lets us call `Pin::set` below.\n /// future.as_mut().await;\n ///\n /// // Reset the future in case another call to\n /// // `try_recv` got the message before us.\n /// future.set(self.notify_on_sent.notified());\n /// }\n /// }\n /// }\n /// ```\n ///\n /// [`notify_one`]: Notify::notify_one()\n /// [`notify_waiters`]: Notify::notify_waiters()\n pub fn enable(self: Pin<&mut Self>) -> bool {\n self.poll_notified(None).is_ready()\n }\n\n fn project(self: Pin<&mut Self>) -> NotifiedProject<'_> {\n unsafe {\n // Safety: `notify`, `state` and `notify_waiters_calls` are `Unpin`.\n\n is_unpin::<&Notify>();\n is_unpin::();\n is_unpin::();\n\n let me = self.get_unchecked_mut();\n NotifiedProject {\n notify: me.notify,\n state: &mut me.state,\n notify_waiters_calls: &me.notify_waiters_calls,\n waiter: &me.waiter,\n }\n }\n }\n\n fn poll_notified(self: Pin<&mut Self>, waker: Option<&Waker>) -> Poll<()> {\n self.project().poll_notified(waker)\n }\n}\n\nimpl Future for Notified<'_> {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n self.poll_notified(Some(cx.waker()))\n }\n}\n\nimpl Drop for Notified<'_> {\n fn drop(&mut self) {\n // Safety: The type only transitions to a \"Waiting\" state when pinned.\n unsafe { Pin::new_unchecked(self) }\n .project()\n .drop_notified();\n }\n}\n\n// ===== impl OwnedNotified =====\n\nimpl OwnedNotified {\n /// Adds this future to the list of futures that are ready to receive\n /// wakeups from calls to [`notify_one`].\n ///\n /// See [`Notified::enable`] for more details.\n ///\n /// [`notify_one`]: Notify::notify_one()\n pub fn enable(self: Pin<&mut Self>) -> bool {\n self.poll_notified(None).is_ready()\n }\n\n /// A custom `project` implementation is used in place of `pin-project-lite`\n /// as a custom drop implementation is needed.\n fn project(self: Pin<&mut Self>) -> NotifiedProject<'_> {\n unsafe {\n // Safety: `notify`, `state` and `notify_waiters_calls` are `Unpin`.\n\n is_unpin::<&Notify>();\n is_unpin::();\n is_unpin::();\n\n let me = self.get_unchecked_mut();\n NotifiedProject {\n notify: &me.notify,\n state: &mut me.state,\n notify_waiters_calls: &me.notify_waiters_calls,\n waiter: &me.waiter,\n }\n }\n }\n\n fn poll_notified(self: Pin<&mut Self>, waker: Option<&Waker>) -> Poll<()> {\n self.project().poll_notified(waker)\n }\n}\n\nimpl Future for OwnedNotified {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n self.poll_notified(Some(cx.waker()))\n }\n}\n\nimpl Drop for OwnedNotified {\n fn drop(&mut self) {\n // Safety: The type only transitions to a \"Waiting\" state when pinned.\n unsafe { Pin::new_unchecked(self) }\n .project()\n .drop_notified();\n }\n}\n\n// ===== impl NotifiedProject =====\n\nimpl NotifiedProject<'_> {\n fn poll_notified(self, waker: Option<&Waker>) -> Poll<()> {\n let NotifiedProject {\n notify,\n state,\n notify_waiters_calls,\n waiter,\n } = self;\n\n 'outer_loop: loop {\n match *state {\n State::Init => {\n let curr = notify.state.load(SeqCst);\n\n // Optimistically try acquiring a pending notification\n let res = notify.state.compare_exchange(\n set_state(curr, NOTIFIED),\n set_state(curr, EMPTY),\n SeqCst,\n SeqCst,\n );\n\n if res.is_ok() {\n // Acquired the notification\n *state = State::Done;\n continue 'outer_loop;\n }\n\n // Clone the waker before locking, a waker clone can be\n // triggering arbitrary code.\n let waker = waker.cloned();\n\n // Acquire the lock and attempt to transition to the waiting\n // state.\n let mut waiters = notify.waiters.lock();\n\n // Reload the state with the lock held\n let mut curr = notify.state.load(SeqCst);\n\n // if notify_waiters has been called after the future\n // was created, then we are done\n if get_num_notify_waiters_calls(curr) != *notify_waiters_calls {\n *state = State::Done;\n continue 'outer_loop;\n }\n\n // Transition the state to WAITING.\n loop {\n match get_state(curr) {\n EMPTY => {\n // Transition to WAITING\n let res = notify.state.compare_exchange(\n set_state(curr, EMPTY),\n set_state(curr, WAITING),\n SeqCst,\n SeqCst,\n );\n\n if let Err(actual) = res {\n assert_eq!(get_state(actual), NOTIFIED);\n curr = actual;\n } else {\n break;\n }\n }\n WAITING => break,\n NOTIFIED => {\n // Try consuming the notification\n let res = notify.state.compare_exchange(\n set_state(curr, NOTIFIED),\n set_state(curr, EMPTY),\n SeqCst,\n SeqCst,\n );\n\n match res {\n Ok(_) => {\n // Acquired the notification\n *state = State::Done;\n continue 'outer_loop;\n }\n Err(actual) => {\n assert_eq!(get_state(actual), EMPTY);\n curr = actual;\n }\n }\n }\n _ => unreachable!(),\n }\n }\n\n let mut old_waker = None;\n if waker.is_some() {\n // Safety: called while locked.\n //\n // The use of `old_waiter` here is not necessary, as the field is always\n // None when we reach this line.\n unsafe {\n old_waker =\n waiter.waker.with_mut(|v| std::mem::replace(&mut *v, waker));\n }\n }\n\n // Insert the waiter into the linked list\n waiters.push_front(NonNull::from(waiter));\n\n *state = State::Waiting;\n\n drop(waiters);\n drop(old_waker);\n\n return Poll::Pending;\n }\n State::Waiting => {\n #[cfg(feature = \"taskdump\")]\n if let Some(waker) = waker {\n let mut ctx = Context::from_waker(waker);\n std::task::ready!(crate::trace::trace_leaf(&mut ctx));\n }\n\n if waiter.notification.load(Acquire).is_some() {\n // Safety: waiter is already unlinked and will not be shared again,\n // so we have an exclusive access to `waker`.\n drop(unsafe { waiter.waker.with_mut(|waker| (*waker).take()) });\n\n waiter.notification.clear();\n *state = State::Done;\n return Poll::Ready(());\n }\n\n // Our waiter was not notified, implying it is still stored in a waiter\n // list (guarded by `notify.waiters`). In order to access the waker\n // fields, we must acquire the lock.\n\n let mut old_waker = None;\n let mut waiters = notify.waiters.lock();\n\n // We hold the lock and notifications are set only with the lock held,\n // so this can be relaxed, because the happens-before relationship is\n // established through the mutex.\n if waiter.notification.load(Relaxed).is_some() {\n // Safety: waiter is already unlinked and will not be shared again,\n // so we have an exclusive access to `waker`.\n old_waker = unsafe { waiter.waker.with_mut(|waker| (*waker).take()) };\n\n waiter.notification.clear();\n\n // Drop the old waker after releasing the lock.\n drop(waiters);\n drop(old_waker);\n\n *state = State::Done;\n return Poll::Ready(());\n }\n\n // Load the state with the lock held.\n let curr = notify.state.load(SeqCst);\n\n if get_num_notify_waiters_calls(curr) != *notify_waiters_calls {\n // Before we add a waiter to the list we check if these numbers are\n // different while holding the lock. If these numbers are different now,\n // it means that there is a call to `notify_waiters` in progress and this\n // waiter must be contained by a guarded list used in `notify_waiters`.\n // We can treat the waiter as notified and remove it from the list, as\n // it would have been notified in the `notify_waiters` call anyways.\n\n // Safety: we hold the lock, so we can modify the waker.\n old_waker = unsafe { waiter.waker.with_mut(|waker| (*waker).take()) };\n\n // Safety: we hold the lock, so we have an exclusive access to the list.\n // The list is used in `notify_waiters`, so it must be guarded.\n unsafe { waiters.remove(NonNull::from(waiter)) };\n\n *state = State::Done;\n } else {\n // Safety: we hold the lock, so we can modify the waker.\n unsafe {\n waiter.waker.with_mut(|v| {\n if let Some(waker) = waker {\n let should_update = match &*v {\n Some(current_waker) => !current_waker.will_wake(waker),\n None => true,\n };\n if should_update {\n old_waker = (*v).replace(waker.clone());\n }\n }\n });\n }\n\n // Drop the old waker after releasing the lock.\n drop(waiters);\n drop(old_waker);\n\n return Poll::Pending;\n }\n\n // Explicit drop of the lock to indicate the scope that the\n // lock is held. Because holding the lock is required to\n // ensure safe access to fields not held within the lock, it\n // is helpful to visualize the scope of the critical\n // section.\n drop(waiters);\n\n // Drop the old waker after releasing the lock.\n drop(old_waker);\n }\n State::Done => {\n #[cfg(feature = \"taskdump\")]\n if let Some(waker) = waker {\n let mut ctx = Context::from_waker(waker);\n std::task::ready!(crate::trace::trace_leaf(&mut ctx));\n }\n return Poll::Ready(());\n }\n }\n }\n }\n\n fn drop_notified(self) {\n let NotifiedProject {\n notify,\n state,\n waiter,\n ..\n } = self;\n\n // This is where we ensure safety. The `Notified` value is being\n // dropped, which means we must ensure that the waiter entry is no\n // longer stored in the linked list.\n if matches!(*state, State::Waiting) {\n let mut waiters = notify.waiters.lock();\n let mut notify_state = notify.state.load(SeqCst);\n\n // We hold the lock, so this field is not concurrently accessed by\n // `notify_*` functions and we can use the relaxed ordering.\n let notification = waiter.notification.load(Relaxed);\n\n // remove the entry from the list (if not already removed)\n //\n // Safety: we hold the lock, so we have an exclusive access to every list the\n // waiter may be contained in. If the node is not contained in the `waiters`\n // list, then it is contained by a guarded list used by `notify_waiters`.\n unsafe { waiters.remove(NonNull::from(waiter)) };\n\n if waiters.is_empty() && get_state(notify_state) == WAITING {\n notify_state = set_state(notify_state, EMPTY);\n notify.state.store(notify_state, SeqCst);\n }\n\n // See if the node was notified but not received. In this case, if\n // the notification was triggered via `notify_one`, it must be sent\n // to the next waiter.\n if let Some(Notification::One(strategy)) = notification {\n if let Some(waker) =\n notify_locked(&mut waiters, ¬ify.state, notify_state, strategy)\n {\n drop(waiters);\n waker.wake();\n }\n }\n }\n }\n}\n\n/// # Safety\n///\n/// `Waiter` is forced to be !Unpin.\nunsafe impl linked_list::Link for Waiter {\n type Handle = NonNull;\n type Target = Waiter;\n\n fn as_raw(handle: &NonNull) -> NonNull {\n *handle\n }\n\n unsafe fn from_raw(ptr: NonNull) -> NonNull {\n ptr\n }\n\n unsafe fn pointers(target: NonNull) -> NonNull> {\n unsafe { Waiter::addr_of_pointers(target) }\n }\n}\n\nfn is_unpin() {}\n\n/// A guard that provides exclusive access to a `Notify`'s internal\n/// waiters list.\n///\n/// While this guard is held, the `Notify` instance's waiter list is locked.\npub(crate) struct NotifyGuard<'a> {\n guarded_notify: &'a Notify,\n guarded_waiters: crate::loom::sync::MutexGuard<'a, WaitList>,\n current_state: usize,\n}\n\nimpl NotifyGuard<'_> {\n pub(crate) fn notify_waiters(self) {\n self.guarded_notify\n .inner_notify_waiters(self.current_state, self.guarded_waiters);\n }\n}\n" }, { "path": "tokio/src/sync/once_cell.rs", "content": "use super::{Semaphore, SemaphorePermit, TryAcquireError};\nuse crate::loom::cell::UnsafeCell;\nuse std::error::Error;\nuse std::fmt;\nuse std::future::Future;\nuse std::mem::MaybeUninit;\nuse std::ops::Drop;\nuse std::ptr;\nuse std::sync::atomic::{AtomicBool, Ordering};\n\n// This file contains an implementation of an OnceCell. The principle\n// behind the safety of the cell is that any thread with an `&OnceCell` may\n// access the `value` field according the following rules:\n//\n// 1. When `value_set` is false, the `value` field may be modified by the\n// thread holding the permit on the semaphore.\n// 2. When `value_set` is true, the `value` field may be accessed immutably by\n// any thread.\n//\n// It is an invariant that if the semaphore is closed, then `value_set` is true.\n// The reverse does not necessarily hold — but if not, the semaphore may not\n// have any available permits.\n//\n// A thread with a `&mut OnceCell` may modify the value in any way it wants as\n// long as the invariants are upheld.\n\n/// A thread-safe cell that can be written to only once.\n///\n/// A `OnceCell` is typically used for global variables that need to be\n/// initialized once on first use, but need no further changes. The `OnceCell`\n/// in Tokio allows the initialization procedure to be asynchronous.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::OnceCell;\n///\n/// async fn some_computation() -> u32 {\n/// 1 + 1\n/// }\n///\n/// static ONCE: OnceCell = OnceCell::const_new();\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let result = ONCE.get_or_init(some_computation).await;\n/// assert_eq!(*result, 2);\n/// # }\n/// ```\n///\n/// It is often useful to write a wrapper method for accessing the value.\n///\n/// ```\n/// use tokio::sync::OnceCell;\n///\n/// static ONCE: OnceCell = OnceCell::const_new();\n///\n/// async fn get_global_integer() -> &'static u32 {\n/// ONCE.get_or_init(|| async {\n/// 1 + 1\n/// }).await\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let result = get_global_integer().await;\n/// assert_eq!(*result, 2);\n/// # }\n/// ```\npub struct OnceCell {\n value_set: AtomicBool,\n value: UnsafeCell>,\n semaphore: Semaphore,\n}\n\nimpl Default for OnceCell {\n fn default() -> OnceCell {\n OnceCell::new()\n }\n}\n\nimpl fmt::Debug for OnceCell {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"OnceCell\")\n .field(\"value\", &self.get())\n .finish()\n }\n}\n\nimpl Clone for OnceCell {\n fn clone(&self) -> OnceCell {\n OnceCell::new_with(self.get().cloned())\n }\n}\n\nimpl PartialEq for OnceCell {\n fn eq(&self, other: &OnceCell) -> bool {\n self.get() == other.get()\n }\n}\n\nimpl Eq for OnceCell {}\n\nimpl Drop for OnceCell {\n fn drop(&mut self) {\n if self.initialized_mut() {\n unsafe {\n self.value\n .with_mut(|ptr| ptr::drop_in_place((*ptr).as_mut_ptr()));\n };\n }\n }\n}\n\nimpl From for OnceCell {\n fn from(value: T) -> Self {\n OnceCell {\n value_set: AtomicBool::new(true),\n value: UnsafeCell::new(MaybeUninit::new(value)),\n semaphore: Semaphore::new_closed(),\n }\n }\n}\n\nimpl OnceCell {\n /// Creates a new empty `OnceCell` instance.\n pub fn new() -> Self {\n OnceCell {\n value_set: AtomicBool::new(false),\n value: UnsafeCell::new(MaybeUninit::uninit()),\n semaphore: Semaphore::new(1),\n }\n }\n\n /// Creates a new empty `OnceCell` instance.\n ///\n /// Equivalent to `OnceCell::new`, except that it can be used in static\n /// variables.\n ///\n /// When using the `tracing` [unstable feature], a `OnceCell` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`OnceCell::new`] should be used to\n /// create an instrumented object if that is needed.\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::sync::OnceCell;\n ///\n /// static ONCE: OnceCell = OnceCell::const_new();\n ///\n /// async fn get_global_integer() -> &'static u32 {\n /// ONCE.get_or_init(|| async {\n /// 1 + 1\n /// }).await\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let result = get_global_integer().await;\n /// assert_eq!(*result, 2);\n /// # }\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new() -> Self {\n OnceCell {\n value_set: AtomicBool::new(false),\n value: UnsafeCell::new(MaybeUninit::uninit()),\n semaphore: Semaphore::const_new(1),\n }\n }\n\n /// Creates a new `OnceCell` that contains the provided value, if any.\n ///\n /// If the `Option` is `None`, this is equivalent to `OnceCell::new`.\n ///\n /// [`OnceCell::new`]: crate::sync::OnceCell::new\n // Once https://github.com/rust-lang/rust/issues/73255 lands\n // and tokio MSRV is bumped to the rustc version with it stabilised,\n // we can make this function available in const context,\n // by creating `Semaphore::const_new_closed`.\n pub fn new_with(value: Option) -> Self {\n if let Some(v) = value {\n OnceCell::from(v)\n } else {\n OnceCell::new()\n }\n }\n\n /// Creates a new `OnceCell` that contains the provided value.\n ///\n /// # Example\n ///\n /// When using the `tracing` [unstable feature], a `OnceCell` created with\n /// `const_new_with` will not be instrumented. As such, it will not be\n /// visible in [`tokio-console`]. Instead, [`OnceCell::new_with`] should be\n /// used to create an instrumented object if that is needed.\n ///\n /// ```\n /// use tokio::sync::OnceCell;\n ///\n /// static ONCE: OnceCell = OnceCell::const_new_with(1);\n ///\n /// async fn get_global_integer() -> &'static u32 {\n /// ONCE.get_or_init(|| async {\n /// 1 + 1\n /// }).await\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let result = get_global_integer().await;\n /// assert_eq!(*result, 1);\n /// # }\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new_with(value: T) -> Self {\n OnceCell {\n value_set: AtomicBool::new(true),\n value: UnsafeCell::new(MaybeUninit::new(value)),\n semaphore: Semaphore::const_new_closed(),\n }\n }\n\n /// Returns `true` if the `OnceCell` currently contains a value, and `false`\n /// otherwise.\n pub fn initialized(&self) -> bool {\n // Using acquire ordering so any threads that read a true from this\n // atomic is able to read the value.\n self.value_set.load(Ordering::Acquire)\n }\n\n /// Returns `true` if the `OnceCell` currently contains a value, and `false`\n /// otherwise.\n fn initialized_mut(&mut self) -> bool {\n *self.value_set.get_mut()\n }\n\n // SAFETY: The OnceCell must not be empty.\n unsafe fn get_unchecked(&self) -> &T {\n unsafe { &*self.value.with(|ptr| (*ptr).as_ptr()) }\n }\n\n // SAFETY: The OnceCell must not be empty.\n unsafe fn get_unchecked_mut(&mut self) -> &mut T {\n // SAFETY:\n //\n // 1. The caller guarantees that the OnceCell is initialized.\n // 2. The `&mut self` guarantees that there are no other references to the value.\n unsafe { &mut *self.value.with_mut(|ptr| (*ptr).as_mut_ptr()) }\n }\n\n fn set_value(&self, value: T, permit: SemaphorePermit<'_>) -> &T {\n // SAFETY: We are holding the only permit on the semaphore.\n unsafe {\n self.value.with_mut(|ptr| (*ptr).as_mut_ptr().write(value));\n }\n\n // Using release ordering so any threads that read a true from this\n // atomic is able to read the value we just stored.\n self.value_set.store(true, Ordering::Release);\n self.semaphore.close();\n permit.forget();\n\n // SAFETY: We just initialized the cell.\n unsafe { self.get_unchecked() }\n }\n\n /// Returns a reference to the value currently stored in the `OnceCell`, or\n /// `None` if the `OnceCell` is empty.\n pub fn get(&self) -> Option<&T> {\n if self.initialized() {\n Some(unsafe { self.get_unchecked() })\n } else {\n None\n }\n }\n\n /// Returns a mutable reference to the value currently stored in the\n /// `OnceCell`, or `None` if the `OnceCell` is empty.\n ///\n /// Since this call borrows the `OnceCell` mutably, it is safe to mutate the\n /// value inside the `OnceCell` — the mutable borrow statically guarantees\n /// no other references exist.\n pub fn get_mut(&mut self) -> Option<&mut T> {\n if self.initialized_mut() {\n Some(unsafe { self.get_unchecked_mut() })\n } else {\n None\n }\n }\n\n /// Sets the value of the `OnceCell` to the given value if the `OnceCell` is\n /// empty.\n ///\n /// If the `OnceCell` already has a value, this call will fail with an\n /// [`SetError::AlreadyInitializedError`].\n ///\n /// If the `OnceCell` is empty, but some other task is currently trying to\n /// set the value, this call will fail with [`SetError::InitializingError`].\n ///\n /// [`SetError::AlreadyInitializedError`]: crate::sync::SetError::AlreadyInitializedError\n /// [`SetError::InitializingError`]: crate::sync::SetError::InitializingError\n pub fn set(&self, value: T) -> Result<(), SetError> {\n if self.initialized() {\n return Err(SetError::AlreadyInitializedError(value));\n }\n\n // Another task might be initializing the cell, in which case\n // `try_acquire` will return an error. If we succeed to acquire the\n // permit, then we can set the value.\n match self.semaphore.try_acquire() {\n Ok(permit) => {\n debug_assert!(!self.initialized());\n self.set_value(value, permit);\n Ok(())\n }\n Err(TryAcquireError::NoPermits) => {\n // Some other task is holding the permit. That task is\n // currently trying to initialize the value.\n Err(SetError::InitializingError(value))\n }\n Err(TryAcquireError::Closed) => {\n // The semaphore was closed. Some other task has initialized\n // the value.\n Err(SetError::AlreadyInitializedError(value))\n }\n }\n }\n\n /// Gets the value currently in the `OnceCell`, or initialize it with the\n /// given asynchronous operation.\n ///\n /// If some other task is currently working on initializing the `OnceCell`,\n /// this call will wait for that other task to finish, then return the value\n /// that the other task produced.\n ///\n /// If the provided operation is cancelled or panics, the initialization\n /// attempt is cancelled. If there are other tasks waiting for the value to\n /// be initialized, one of them will start another attempt at initializing\n /// the value.\n ///\n /// This will deadlock if `f` tries to initialize the cell recursively.\n pub async fn get_or_init(&self, f: F) -> &T\n where\n F: FnOnce() -> Fut,\n Fut: Future,\n {\n crate::trace::async_trace_leaf().await;\n\n if self.initialized() {\n // SAFETY: The OnceCell has been fully initialized.\n unsafe { self.get_unchecked() }\n } else {\n // Here we try to acquire the semaphore permit. Holding the permit\n // will allow us to set the value of the OnceCell, and prevents\n // other tasks from initializing the OnceCell while we are holding\n // it.\n match self.semaphore.acquire().await {\n Ok(permit) => {\n debug_assert!(!self.initialized());\n\n // If `f()` panics or `select!` is called, this\n // `get_or_init` call is aborted and the semaphore permit is\n // dropped.\n let value = f().await;\n\n self.set_value(value, permit)\n }\n Err(_) => {\n debug_assert!(self.initialized());\n\n // SAFETY: The semaphore has been closed. This only happens\n // when the OnceCell is fully initialized.\n unsafe { self.get_unchecked() }\n }\n }\n }\n }\n\n /// Gets the value currently in the `OnceCell`, or initialize it with the\n /// given asynchronous operation.\n ///\n /// If some other task is currently working on initializing the `OnceCell`,\n /// this call will wait for that other task to finish, then return the value\n /// that the other task produced.\n ///\n /// If the provided operation returns an error, is cancelled or panics, the\n /// initialization attempt is cancelled. If there are other tasks waiting\n /// for the value to be initialized, one of them will start another attempt\n /// at initializing the value.\n ///\n /// This will deadlock if `f` tries to initialize the cell recursively.\n pub async fn get_or_try_init(&self, f: F) -> Result<&T, E>\n where\n F: FnOnce() -> Fut,\n Fut: Future>,\n {\n crate::trace::async_trace_leaf().await;\n\n if self.initialized() {\n // SAFETY: The OnceCell has been fully initialized.\n unsafe { Ok(self.get_unchecked()) }\n } else {\n // Here we try to acquire the semaphore permit. Holding the permit\n // will allow us to set the value of the OnceCell, and prevents\n // other tasks from initializing the OnceCell while we are holding\n // it.\n match self.semaphore.acquire().await {\n Ok(permit) => {\n debug_assert!(!self.initialized());\n\n // If `f()` panics or `select!` is called, this\n // `get_or_try_init` call is aborted and the semaphore\n // permit is dropped.\n let value = f().await;\n\n match value {\n Ok(value) => Ok(self.set_value(value, permit)),\n Err(e) => Err(e),\n }\n }\n Err(_) => {\n debug_assert!(self.initialized());\n\n // SAFETY: The semaphore has been closed. This only happens\n // when the OnceCell is fully initialized.\n unsafe { Ok(self.get_unchecked()) }\n }\n }\n }\n }\n\n /// Takes the value from the cell, destroying the cell in the process.\n /// Returns `None` if the cell is empty.\n pub fn into_inner(mut self) -> Option {\n if self.initialized_mut() {\n // Set to uninitialized for the destructor of `OnceCell` to work properly\n *self.value_set.get_mut() = false;\n Some(unsafe { self.value.with(|ptr| ptr::read(ptr).assume_init()) })\n } else {\n None\n }\n }\n\n /// Takes ownership of the current value, leaving the cell empty. Returns\n /// `None` if the cell is empty.\n pub fn take(&mut self) -> Option {\n std::mem::take(self).into_inner()\n }\n}\n\n// Since `get` gives us access to immutable references of the OnceCell, OnceCell\n// can only be Sync if T is Sync, otherwise OnceCell would allow sharing\n// references of !Sync values across threads. We need T to be Send in order for\n// OnceCell to by Sync because we can use `set` on `&OnceCell` to send values\n// (of type T) across threads.\nunsafe impl Sync for OnceCell {}\n\n// Access to OnceCell's value is guarded by the semaphore permit\n// and atomic operations on `value_set`, so as long as T itself is Send\n// it's safe to send it to another thread\nunsafe impl Send for OnceCell {}\n\n/// Errors that can be returned from [`OnceCell::set`].\n///\n/// [`OnceCell::set`]: crate::sync::OnceCell::set\n#[derive(Debug, PartialEq, Eq)]\npub enum SetError {\n /// The cell was already initialized when [`OnceCell::set`] was called.\n ///\n /// [`OnceCell::set`]: crate::sync::OnceCell::set\n AlreadyInitializedError(T),\n\n /// The cell is currently being initialized.\n InitializingError(T),\n}\n\nimpl fmt::Display for SetError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n SetError::AlreadyInitializedError(_) => write!(f, \"AlreadyInitializedError\"),\n SetError::InitializingError(_) => write!(f, \"InitializingError\"),\n }\n }\n}\n\nimpl Error for SetError {}\n\nimpl SetError {\n /// Whether `SetError` is `SetError::AlreadyInitializedError`.\n pub fn is_already_init_err(&self) -> bool {\n match self {\n SetError::AlreadyInitializedError(_) => true,\n SetError::InitializingError(_) => false,\n }\n }\n\n /// Whether `SetError` is `SetError::InitializingError`\n pub fn is_initializing_err(&self) -> bool {\n match self {\n SetError::AlreadyInitializedError(_) => false,\n SetError::InitializingError(_) => true,\n }\n }\n}\n" }, { "path": "tokio/src/sync/oneshot.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(dead_code, unreachable_pub))]\n\n//! A one-shot channel is used for sending a single message between\n//! asynchronous tasks. The [`channel`] function is used to create a\n//! [`Sender`] and [`Receiver`] handle pair that form the channel.\n//!\n//! The `Sender` handle is used by the producer to send the value.\n//! The `Receiver` handle is used by the consumer to receive the value.\n//!\n//! Each handle can be used on separate tasks.\n//!\n//! Since the `send` method is not async, it can be used anywhere. This includes\n//! sending between two runtimes, and using it from non-async code.\n//!\n//! If the [`Receiver`] is closed before receiving a message which has already\n//! been sent, the message will remain in the channel until the receiver is\n//! dropped, at which point the message will be dropped immediately.\n//!\n//! # Examples\n//!\n//! ```\n//! use tokio::sync::oneshot;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, rx) = oneshot::channel();\n//!\n//! tokio::spawn(async move {\n//! if let Err(_) = tx.send(3) {\n//! println!(\"the receiver dropped\");\n//! }\n//! });\n//!\n//! match rx.await {\n//! Ok(v) => println!(\"got = {:?}\", v),\n//! Err(_) => println!(\"the sender dropped\"),\n//! }\n//! # }\n//! ```\n//!\n//! If the sender is dropped without sending, the receiver will fail with\n//! [`error::RecvError`]:\n//!\n//! ```\n//! use tokio::sync::oneshot;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, rx) = oneshot::channel::();\n//!\n//! tokio::spawn(async move {\n//! drop(tx);\n//! });\n//!\n//! match rx.await {\n//! Ok(_) => panic!(\"This doesn't happen\"),\n//! Err(_) => println!(\"the sender dropped\"),\n//! }\n//! # }\n//! ```\n//!\n//! To use a `oneshot` channel in a `tokio::select!` loop, add `&mut` in front of\n//! the channel.\n//!\n//! ```\n//! use tokio::sync::oneshot;\n//! use tokio::time::{interval, sleep, Duration};\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn _doc() {}\n//! # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n//! # async fn main() {\n//! let (send, mut recv) = oneshot::channel();\n//! let mut interval = interval(Duration::from_millis(100));\n//!\n//! # let handle =\n//! tokio::spawn(async move {\n//! sleep(Duration::from_secs(1)).await;\n//! send.send(\"shut down\").unwrap();\n//! });\n//!\n//! loop {\n//! tokio::select! {\n//! _ = interval.tick() => println!(\"Another 100ms\"),\n//! msg = &mut recv => {\n//! println!(\"Got message: {}\", msg.unwrap());\n//! break;\n//! }\n//! }\n//! }\n//! # handle.await.unwrap();\n//! # }\n//! ```\n//!\n//! To use a `Sender` from a destructor, put it in an [`Option`] and call\n//! [`Option::take`].\n//!\n//! ```\n//! use tokio::sync::oneshot;\n//!\n//! struct SendOnDrop {\n//! sender: Option>,\n//! }\n//! impl Drop for SendOnDrop {\n//! fn drop(&mut self) {\n//! if let Some(sender) = self.sender.take() {\n//! // Using `let _ =` to ignore send errors.\n//! let _ = sender.send(\"I got dropped!\");\n//! }\n//! }\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn _doc() {}\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (send, recv) = oneshot::channel();\n//!\n//! let send_on_drop = SendOnDrop { sender: Some(send) };\n//! drop(send_on_drop);\n//!\n//! assert_eq!(recv.await, Ok(\"I got dropped!\"));\n//! # }\n//! ```\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::Arc;\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\n\nuse std::fmt;\nuse std::future::Future;\nuse std::mem::MaybeUninit;\nuse std::pin::Pin;\nuse std::sync::atomic::Ordering::{self, AcqRel, Acquire};\nuse std::task::Poll::{Pending, Ready};\nuse std::task::{ready, Context, Poll, Waker};\n\n/// Sends a value to the associated [`Receiver`].\n///\n/// A pair of both a [`Sender`] and a [`Receiver`] are created by the\n/// [`channel`](fn@channel) function.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, rx) = oneshot::channel();\n///\n/// tokio::spawn(async move {\n/// if let Err(_) = tx.send(3) {\n/// println!(\"the receiver dropped\");\n/// }\n/// });\n///\n/// match rx.await {\n/// Ok(v) => println!(\"got = {:?}\", v),\n/// Err(_) => println!(\"the sender dropped\"),\n/// }\n/// # }\n/// ```\n///\n/// If the sender is dropped without sending, the receiver will fail with\n/// [`error::RecvError`]:\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, rx) = oneshot::channel::();\n///\n/// tokio::spawn(async move {\n/// drop(tx);\n/// });\n///\n/// match rx.await {\n/// Ok(_) => panic!(\"This doesn't happen\"),\n/// Err(_) => println!(\"the sender dropped\"),\n/// }\n/// # }\n/// ```\n///\n/// To use a `Sender` from a destructor, put it in an [`Option`] and call\n/// [`Option::take`].\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// struct SendOnDrop {\n/// sender: Option>,\n/// }\n/// impl Drop for SendOnDrop {\n/// fn drop(&mut self) {\n/// if let Some(sender) = self.sender.take() {\n/// // Using `let _ =` to ignore send errors.\n/// let _ = sender.send(\"I got dropped!\");\n/// }\n/// }\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn _doc() {}\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (send, recv) = oneshot::channel();\n///\n/// let send_on_drop = SendOnDrop { sender: Some(send) };\n/// drop(send_on_drop);\n///\n/// assert_eq!(recv.await, Ok(\"I got dropped!\"));\n/// # }\n/// ```\n///\n/// [`Option`]: std::option::Option\n/// [`Option::take`]: std::option::Option::take\n#[derive(Debug)]\npub struct Sender {\n inner: Option>>,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n}\n\n/// Receives a value from the associated [`Sender`].\n///\n/// A pair of both a [`Sender`] and a [`Receiver`] are created by the\n/// [`channel`](fn@channel) function.\n///\n/// This channel has no `recv` method because the receiver itself implements the\n/// [`Future`] trait. To receive a `Result`, `.await` the `Receiver` object directly.\n///\n/// The `poll` method on the `Future` trait is allowed to spuriously return\n/// `Poll::Pending` even if the message has been sent. If such a spurious\n/// failure happens, then the caller will be woken when the spurious failure has\n/// been resolved so that the caller can attempt to receive the message again.\n/// Note that receiving such a wakeup does not guarantee that the next call will\n/// succeed — it could fail with another spurious failure. (A spurious failure\n/// does not mean that the message is lost. It is just delayed.)\n///\n/// [`Future`]: trait@std::future::Future\n///\n/// # Cancellation safety\n///\n/// The `Receiver` is cancel safe. If it is used as the event in a\n/// [`tokio::select!`](crate::select) statement and some other branch\n/// completes first, it is guaranteed that no message was received on this\n/// channel.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, rx) = oneshot::channel();\n///\n/// tokio::spawn(async move {\n/// if let Err(_) = tx.send(3) {\n/// println!(\"the receiver dropped\");\n/// }\n/// });\n///\n/// match rx.await {\n/// Ok(v) => println!(\"got = {:?}\", v),\n/// Err(_) => println!(\"the sender dropped\"),\n/// }\n/// # }\n/// ```\n///\n/// If the sender is dropped without sending, the receiver will fail with\n/// [`error::RecvError`]:\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, rx) = oneshot::channel::();\n///\n/// tokio::spawn(async move {\n/// drop(tx);\n/// });\n///\n/// match rx.await {\n/// Ok(_) => panic!(\"This doesn't happen\"),\n/// Err(_) => println!(\"the sender dropped\"),\n/// }\n/// # }\n/// ```\n///\n/// To use a `Receiver` in a `tokio::select!` loop, add `&mut` in front of the\n/// channel.\n///\n/// ```\n/// use tokio::sync::oneshot;\n/// use tokio::time::{interval, sleep, Duration};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn _doc() {}\n/// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n/// # async fn main() {\n/// let (send, mut recv) = oneshot::channel();\n/// let mut interval = interval(Duration::from_millis(100));\n///\n/// # let handle =\n/// tokio::spawn(async move {\n/// sleep(Duration::from_secs(1)).await;\n/// send.send(\"shut down\").unwrap();\n/// });\n///\n/// loop {\n/// tokio::select! {\n/// _ = interval.tick() => println!(\"Another 100ms\"),\n/// msg = &mut recv => {\n/// println!(\"Got message: {}\", msg.unwrap());\n/// break;\n/// }\n/// }\n/// }\n/// # handle.await.unwrap();\n/// # }\n/// ```\n#[derive(Debug)]\npub struct Receiver {\n inner: Option>>,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n async_op_span: tracing::Span,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n async_op_poll_span: tracing::Span,\n}\n\npub mod error {\n //! `Oneshot` error types.\n\n use std::fmt;\n\n /// Error returned by the `Future` implementation for `Receiver`.\n ///\n /// This error is returned by the receiver when the sender is dropped without sending.\n #[derive(Debug, Eq, PartialEq, Clone)]\n pub struct RecvError(pub(super) ());\n\n /// Error returned by the `try_recv` function on `Receiver`.\n #[derive(Debug, Eq, PartialEq, Clone)]\n pub enum TryRecvError {\n /// The send half of the channel has not yet sent a value.\n Empty,\n\n /// The send half of the channel was dropped without sending a value.\n Closed,\n }\n\n // ===== impl RecvError =====\n\n impl fmt::Display for RecvError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n }\n\n impl std::error::Error for RecvError {}\n\n // ===== impl TryRecvError =====\n\n impl fmt::Display for TryRecvError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n TryRecvError::Empty => write!(fmt, \"channel empty\"),\n TryRecvError::Closed => write!(fmt, \"channel closed\"),\n }\n }\n }\n\n impl std::error::Error for TryRecvError {}\n}\n\nuse self::error::*;\n\nstruct Inner {\n /// Manages the state of the inner cell.\n state: AtomicUsize,\n\n /// The value. This is set by `Sender` and read by `Receiver`. The state of\n /// the cell is tracked by `state`.\n value: UnsafeCell>,\n\n /// The task to notify when the receiver drops without consuming the value.\n ///\n /// ## Safety\n ///\n /// The `TX_TASK_SET` bit in the `state` field is set if this field is\n /// initialized. If that bit is unset, this field may be uninitialized.\n tx_task: Task,\n\n /// The task to notify when the value is sent.\n ///\n /// ## Safety\n ///\n /// The `RX_TASK_SET` bit in the `state` field is set if this field is\n /// initialized. If that bit is unset, this field may be uninitialized.\n rx_task: Task,\n}\n\nstruct Task(UnsafeCell>);\n\nimpl Task {\n /// # Safety\n ///\n /// The caller must do the necessary synchronization to ensure that\n /// the [`Self::0`] contains the valid [`Waker`] during the call.\n unsafe fn will_wake(&self, cx: &mut Context<'_>) -> bool {\n unsafe { self.with_task(|w| w.will_wake(cx.waker())) }\n }\n\n /// # Safety\n ///\n /// The caller must do the necessary synchronization to ensure that\n /// the [`Self::0`] contains the valid [`Waker`] during the call.\n unsafe fn with_task(&self, f: F) -> R\n where\n F: FnOnce(&Waker) -> R,\n {\n self.0.with(|ptr| {\n let waker: *const Waker = unsafe { (*ptr).as_ptr() };\n f(unsafe { &*waker })\n })\n }\n\n /// # Safety\n ///\n /// The caller must do the necessary synchronization to ensure that\n /// the [`Self::0`] contains the valid [`Waker`] during the call.\n unsafe fn drop_task(&self) {\n self.0.with_mut(|ptr| {\n let ptr: *mut Waker = unsafe { (*ptr).as_mut_ptr() };\n unsafe {\n ptr.drop_in_place();\n }\n });\n }\n\n /// # Safety\n ///\n /// The caller must do the necessary synchronization to ensure that\n /// the [`Self::0`] contains the valid [`Waker`] during the call.\n unsafe fn set_task(&self, cx: &mut Context<'_>) {\n self.0.with_mut(|ptr| {\n let ptr: *mut Waker = unsafe { (*ptr).as_mut_ptr() };\n unsafe {\n ptr.write(cx.waker().clone());\n }\n });\n }\n}\n\n#[derive(Clone, Copy)]\nstruct State(usize);\n\n/// Creates a new one-shot channel for sending single values across asynchronous\n/// tasks.\n///\n/// The function returns separate \"send\" and \"receive\" handles. The `Sender`\n/// handle is used by the producer to send the value. The `Receiver` handle is\n/// used by the consumer to receive the value.\n///\n/// Each handle can be used on separate tasks.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::oneshot;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx, rx) = oneshot::channel();\n///\n/// tokio::spawn(async move {\n/// if let Err(_) = tx.send(3) {\n/// println!(\"the receiver dropped\");\n/// }\n/// });\n///\n/// match rx.await {\n/// Ok(v) => println!(\"got = {:?}\", v),\n/// Err(_) => println!(\"the sender dropped\"),\n/// }\n/// # }\n/// ```\n#[track_caller]\npub fn channel() -> (Sender, Receiver) {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Sender|Receiver\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n );\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n tx_dropped = false,\n tx_dropped.op = \"override\",\n )\n });\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n rx_dropped = false,\n rx_dropped.op = \"override\",\n )\n });\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n value_sent = false,\n value_sent.op = \"override\",\n )\n });\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n value_received = false,\n value_received.op = \"override\",\n )\n });\n\n resource_span\n };\n\n let inner = Arc::new(Inner {\n state: AtomicUsize::new(State::new().as_usize()),\n value: UnsafeCell::new(None),\n tx_task: Task(UnsafeCell::new(MaybeUninit::uninit())),\n rx_task: Task(UnsafeCell::new(MaybeUninit::uninit())),\n });\n\n let tx = Sender {\n inner: Some(inner.clone()),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: resource_span.clone(),\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let async_op_span = resource_span\n .in_scope(|| tracing::trace_span!(\"runtime.resource.async_op\", source = \"Receiver::await\"));\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let async_op_poll_span =\n async_op_span.in_scope(|| tracing::trace_span!(\"runtime.resource.async_op.poll\"));\n\n let rx = Receiver {\n inner: Some(inner),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n async_op_span,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n async_op_poll_span,\n };\n\n (tx, rx)\n}\n\nimpl Sender {\n /// Attempts to send a value on this channel, returning it back if it could\n /// not be sent.\n ///\n /// This method consumes `self` as only one value may ever be sent on a `oneshot`\n /// channel. It is not marked async because sending a message to an `oneshot`\n /// channel never requires any form of waiting. Because of this, the `send`\n /// method can be used in both synchronous and asynchronous code without\n /// problems.\n ///\n /// A successful send occurs when it is determined that the other end of the\n /// channel has not hung up already. An unsuccessful send would be one where\n /// the corresponding receiver has already been deallocated. Note that a\n /// return value of `Err` means that the data will never be received, but\n /// a return value of `Ok` does *not* mean that the data will be received.\n /// It is possible for the corresponding receiver to hang up immediately\n /// after this function returns `Ok`.\n ///\n /// # Examples\n ///\n /// Send a value to another task\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = oneshot::channel();\n ///\n /// tokio::spawn(async move {\n /// if let Err(_) = tx.send(3) {\n /// println!(\"the receiver dropped\");\n /// }\n /// });\n ///\n /// match rx.await {\n /// Ok(v) => println!(\"got = {:?}\", v),\n /// Err(_) => println!(\"the sender dropped\"),\n /// }\n /// # }\n /// ```\n pub fn send(mut self, t: T) -> Result<(), T> {\n let inner = self.inner.take().unwrap();\n\n inner.value.with_mut(|ptr| unsafe {\n // SAFETY: The receiver will not access the `UnsafeCell` unless the\n // channel has been marked as \"complete\" (the `VALUE_SENT` state bit\n // is set).\n // That bit is only set by the sender later on in this method, and\n // calling this method consumes `self`. Therefore, if it was possible to\n // call this method, we know that the `VALUE_SENT` bit is unset, and\n // the receiver is not currently accessing the `UnsafeCell`.\n *ptr = Some(t);\n });\n\n if !inner.complete() {\n unsafe {\n // SAFETY: The receiver will not access the `UnsafeCell` unless\n // the channel has been marked as \"complete\". Calling\n // `complete()` will return true if this bit is set, and false\n // if it is not set. Thus, if `complete()` returned false, it is\n // safe for us to access the value, because we know that the\n // receiver will not.\n return Err(inner.consume_value().unwrap());\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n value_sent = true,\n value_sent.op = \"override\",\n )\n });\n\n Ok(())\n }\n\n /// Waits for the associated [`Receiver`] handle to close.\n ///\n /// A [`Receiver`] is closed by either calling [`close`] explicitly or the\n /// [`Receiver`] value is dropped.\n ///\n /// This function is useful when paired with `select!` to abort a\n /// computation when the receiver is no longer interested in the result.\n ///\n /// # Return\n ///\n /// Returns a `Future` which must be awaited on.\n ///\n /// [`Receiver`]: Receiver\n /// [`close`]: Receiver::close\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (mut tx, rx) = oneshot::channel::<()>();\n ///\n /// tokio::spawn(async move {\n /// drop(rx);\n /// });\n ///\n /// tx.closed().await;\n /// println!(\"the receiver dropped\");\n /// # }\n /// ```\n ///\n /// Paired with select\n ///\n /// ```\n /// use tokio::sync::oneshot;\n /// use tokio::time::{self, Duration};\n ///\n /// async fn compute() -> String {\n /// // Complex computation returning a `String`\n /// # \"hello\".to_string()\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (mut tx, rx) = oneshot::channel();\n ///\n /// tokio::spawn(async move {\n /// tokio::select! {\n /// _ = tx.closed() => {\n /// // The receiver dropped, no need to do any further work\n /// }\n /// value = compute() => {\n /// // The send can fail if the channel was closed at the exact same\n /// // time as when compute() finished, so just ignore the failure.\n /// let _ = tx.send(value);\n /// }\n /// }\n /// });\n ///\n /// // Wait for up to 10 seconds\n /// let _ = time::timeout(Duration::from_secs(10), rx).await;\n /// # }\n /// ```\n pub async fn closed(&mut self) {\n use std::future::poll_fn;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = self.resource_span.clone();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let closed = trace::async_op(\n || poll_fn(|cx| self.poll_closed(cx)),\n resource_span,\n \"Sender::closed\",\n \"poll_closed\",\n false,\n );\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let closed = poll_fn(|cx| self.poll_closed(cx));\n\n closed.await;\n }\n\n /// Returns `true` if the associated [`Receiver`] handle has been dropped.\n ///\n /// A [`Receiver`] is closed by either calling [`close`] explicitly or the\n /// [`Receiver`] value is dropped.\n ///\n /// If `true` is returned, a call to `send` will always result in an error.\n ///\n /// [`Receiver`]: Receiver\n /// [`close`]: Receiver::close\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = oneshot::channel();\n ///\n /// assert!(!tx.is_closed());\n ///\n /// drop(rx);\n ///\n /// assert!(tx.is_closed());\n /// assert!(tx.send(\"never received\").is_err());\n /// # }\n /// ```\n pub fn is_closed(&self) -> bool {\n let inner = self.inner.as_ref().unwrap();\n\n let state = State::load(&inner.state, Acquire);\n state.is_closed()\n }\n\n /// Checks whether the `oneshot` channel has been closed, and if not, schedules the\n /// `Waker` in the provided `Context` to receive a notification when the channel is\n /// closed.\n ///\n /// A [`Receiver`] is closed by either calling [`close`] explicitly, or when the\n /// [`Receiver`] value is dropped.\n ///\n /// Note that on multiple calls to poll, only the `Waker` from the `Context` passed\n /// to the most recent call will be scheduled to receive a wakeup.\n ///\n /// [`Receiver`]: struct@crate::sync::oneshot::Receiver\n /// [`close`]: fn@crate::sync::oneshot::Receiver::close\n ///\n /// # Return value\n ///\n /// This function returns:\n ///\n /// * `Poll::Pending` if the channel is still open.\n /// * `Poll::Ready(())` if the channel is closed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// use std::future::poll_fn;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (mut tx, mut rx) = oneshot::channel::<()>();\n ///\n /// tokio::spawn(async move {\n /// rx.close();\n /// });\n ///\n /// poll_fn(|cx| tx.poll_closed(cx)).await;\n ///\n /// println!(\"the receiver dropped\");\n /// # }\n /// ```\n pub fn poll_closed(&mut self, cx: &mut Context<'_>) -> Poll<()> {\n ready!(crate::trace::trace_leaf(cx));\n\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let inner = self.inner.as_ref().unwrap();\n\n let mut state = State::load(&inner.state, Acquire);\n\n if state.is_closed() {\n coop.made_progress();\n return Ready(());\n }\n\n if state.is_tx_task_set() {\n let will_notify = unsafe { inner.tx_task.will_wake(cx) };\n\n if !will_notify {\n state = State::unset_tx_task(&inner.state);\n\n if state.is_closed() {\n // Set the flag again so that the waker is released in drop\n State::set_tx_task(&inner.state);\n coop.made_progress();\n return Ready(());\n } else {\n unsafe { inner.tx_task.drop_task() };\n }\n }\n }\n\n if !state.is_tx_task_set() {\n // Attempt to set the task\n unsafe {\n inner.tx_task.set_task(cx);\n }\n\n // Update the state\n state = State::set_tx_task(&inner.state);\n\n if state.is_closed() {\n coop.made_progress();\n return Ready(());\n }\n }\n\n Pending\n }\n}\n\nimpl Drop for Sender {\n fn drop(&mut self) {\n if let Some(inner) = self.inner.as_ref() {\n inner.complete();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n tx_dropped = true,\n tx_dropped.op = \"override\",\n )\n });\n }\n }\n}\n\nimpl Receiver {\n /// Prevents the associated [`Sender`] handle from sending a value.\n ///\n /// Any `send` operation which happens after calling `close` is guaranteed\n /// to fail. After calling `close`, [`try_recv`] should be called to\n /// receive a value if one was sent **before** the call to `close`\n /// completed.\n ///\n /// This function is useful to perform a graceful shutdown and ensure that a\n /// value will not be sent into the channel and never received.\n ///\n /// `close` is no-op if a message is already received or the channel\n /// is already closed.\n ///\n /// [`Sender`]: Sender\n /// [`try_recv`]: Receiver::try_recv\n ///\n /// # Examples\n ///\n /// Prevent a value from being sent\n ///\n /// ```\n /// use tokio::sync::oneshot;\n /// use tokio::sync::oneshot::error::TryRecvError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n ///\n /// assert!(!tx.is_closed());\n ///\n /// rx.close();\n ///\n /// assert!(tx.is_closed());\n /// assert!(tx.send(\"never received\").is_err());\n ///\n /// match rx.try_recv() {\n /// Err(TryRecvError::Closed) => {}\n /// _ => unreachable!(),\n /// }\n /// # }\n /// ```\n ///\n /// Receive a value sent **before** calling `close`\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n ///\n /// assert!(tx.send(\"will receive\").is_ok());\n ///\n /// rx.close();\n ///\n /// let msg = rx.try_recv().unwrap();\n /// assert_eq!(msg, \"will receive\");\n /// # }\n /// ```\n pub fn close(&mut self) {\n if let Some(inner) = self.inner.as_ref() {\n inner.close();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n rx_dropped = true,\n rx_dropped.op = \"override\",\n )\n });\n }\n }\n\n /// Checks if this receiver is terminated.\n ///\n /// This function returns true if this receiver has already yielded a [`Poll::Ready`] result.\n /// If so, this receiver should no longer be polled.\n ///\n /// # Examples\n ///\n /// Sending a value and polling it.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// use std::task::Poll;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n ///\n /// // A receiver is not terminated when it is initialized.\n /// assert!(!rx.is_terminated());\n ///\n /// // A receiver is not terminated it is polled and is still pending.\n /// let poll = futures::poll!(&mut rx);\n /// assert_eq!(poll, Poll::Pending);\n /// assert!(!rx.is_terminated());\n ///\n /// // A receiver is not terminated if a value has been sent, but not yet read.\n /// tx.send(0).unwrap();\n /// assert!(!rx.is_terminated());\n ///\n /// // A receiver *is* terminated after it has been polled and yielded a value.\n /// assert_eq!((&mut rx).await, Ok(0));\n /// assert!(rx.is_terminated());\n /// # }\n /// ```\n ///\n /// Dropping the sender.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel::<()>();\n ///\n /// // A receiver is not immediately terminated when the sender is dropped.\n /// drop(tx);\n /// assert!(!rx.is_terminated());\n ///\n /// // A receiver *is* terminated after it has been polled and yielded an error.\n /// let _ = (&mut rx).await.unwrap_err();\n /// assert!(rx.is_terminated());\n /// # }\n /// ```\n pub fn is_terminated(&self) -> bool {\n self.inner.is_none()\n }\n\n /// Checks if a channel is empty.\n ///\n /// This method returns `true` if the channel has no messages.\n ///\n /// It is not necessarily safe to poll an empty receiver, which may have\n /// already yielded a value. Use [`is_terminated()`][Self::is_terminated]\n /// to check whether or not a receiver can be safely polled, instead.\n ///\n /// # Examples\n ///\n /// Sending a value.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n /// assert!(rx.is_empty());\n ///\n /// tx.send(0).unwrap();\n /// assert!(!rx.is_empty());\n ///\n /// let _ = (&mut rx).await;\n /// assert!(rx.is_empty());\n /// # }\n /// ```\n ///\n /// Dropping the sender.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel::<()>();\n ///\n /// // A channel is empty if the sender is dropped.\n /// drop(tx);\n /// assert!(rx.is_empty());\n ///\n /// // A closed channel still yields an error, however.\n /// (&mut rx).await.expect_err(\"should yield an error\");\n /// assert!(rx.is_empty());\n /// # }\n /// ```\n ///\n /// Terminated channels are empty.\n ///\n /// ```should_panic,ignore-wasm\n /// use tokio::sync::oneshot;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n /// tx.send(0).unwrap();\n /// let _ = (&mut rx).await;\n ///\n /// // NB: an empty channel is not necessarily safe to poll!\n /// assert!(rx.is_empty());\n /// let _ = (&mut rx).await;\n /// }\n /// ```\n pub fn is_empty(&self) -> bool {\n let Some(inner) = self.inner.as_ref() else {\n // The channel has already terminated.\n return true;\n };\n\n let state = State::load(&inner.state, Acquire);\n if state.is_complete() {\n // SAFETY: If `state.is_complete()` returns true, then the\n // `VALUE_SENT` bit has been set and the sender side of the\n // channel will no longer attempt to access the inner\n // `UnsafeCell`. Therefore, it is now safe for us to access the\n // cell.\n //\n // The channel is empty if it does not have a value.\n unsafe { !inner.has_value() }\n } else {\n // The receiver closed the channel or no value has been sent yet.\n true\n }\n }\n\n /// Attempts to receive a value.\n ///\n /// If a pending value exists in the channel, it is returned. If no value\n /// has been sent, the current task **will not** be registered for\n /// future notification.\n ///\n /// This function is useful to call from outside the context of an\n /// asynchronous task.\n ///\n /// Note that unlike the `poll` method, the `try_recv` method cannot fail\n /// spuriously. Any send or close event that happens before this call to\n /// `try_recv` will be correctly returned to the caller.\n ///\n /// # Return\n ///\n /// - `Ok(T)` if a value is pending in the channel.\n /// - `Err(TryRecvError::Empty)` if no value has been sent yet.\n /// - `Err(TryRecvError::Closed)` if the sender has dropped without sending\n /// a value, or if the message has already been received.\n ///\n /// # Examples\n ///\n /// `try_recv` before a value is sent, then after.\n ///\n /// ```\n /// use tokio::sync::oneshot;\n /// use tokio::sync::oneshot::error::TryRecvError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel();\n ///\n /// match rx.try_recv() {\n /// // The channel is currently empty\n /// Err(TryRecvError::Empty) => {}\n /// _ => unreachable!(),\n /// }\n ///\n /// // Send a value\n /// tx.send(\"hello\").unwrap();\n ///\n /// match rx.try_recv() {\n /// Ok(value) => assert_eq!(value, \"hello\"),\n /// _ => unreachable!(),\n /// }\n /// # }\n /// ```\n ///\n /// `try_recv` when the sender dropped before sending a value\n ///\n /// ```\n /// use tokio::sync::oneshot;\n /// use tokio::sync::oneshot::error::TryRecvError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = oneshot::channel::<()>();\n ///\n /// drop(tx);\n ///\n /// match rx.try_recv() {\n /// // The channel will never receive a value.\n /// Err(TryRecvError::Closed) => {}\n /// _ => unreachable!(),\n /// }\n /// # }\n /// ```\n pub fn try_recv(&mut self) -> Result {\n let result = if let Some(inner) = self.inner.as_ref() {\n let state = State::load(&inner.state, Acquire);\n\n if state.is_complete() {\n // SAFETY: If `state.is_complete()` returns true, then the\n // `VALUE_SENT` bit has been set and the sender side of the\n // channel will no longer attempt to access the inner\n // `UnsafeCell`. Therefore, it is now safe for us to access the\n // cell.\n match unsafe { inner.consume_value() } {\n Some(value) => {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n value_received = true,\n value_received.op = \"override\",\n )\n });\n Ok(value)\n }\n None => Err(TryRecvError::Closed),\n }\n } else if state.is_closed() {\n Err(TryRecvError::Closed)\n } else {\n // Not ready, this does not clear `inner`\n return Err(TryRecvError::Empty);\n }\n } else {\n Err(TryRecvError::Closed)\n };\n\n self.inner = None;\n result\n }\n\n /// Blocking receive to call outside of asynchronous contexts.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution\n /// context.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::thread;\n /// use tokio::sync::oneshot;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let (tx, rx) = oneshot::channel::();\n ///\n /// let sync_code = thread::spawn(move || {\n /// assert_eq!(Ok(10), rx.blocking_recv());\n /// });\n ///\n /// let _ = tx.send(10);\n /// sync_code.join().unwrap();\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(alias = \"recv_blocking\"))]\n pub fn blocking_recv(self) -> Result {\n crate::future::block_on(self)\n }\n}\n\nimpl Drop for Receiver {\n fn drop(&mut self) {\n if let Some(inner) = self.inner.as_ref() {\n let state = inner.close();\n\n if state.is_complete() {\n // SAFETY: we have ensured that the `VALUE_SENT` bit has been set,\n // so only the receiver can access the value.\n drop(unsafe { inner.consume_value() });\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n rx_dropped = true,\n rx_dropped.op = \"override\",\n )\n });\n }\n }\n}\n\nimpl Future for Receiver {\n type Output = Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n // If `inner` is `None`, then `poll()` has already completed.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _res_span = self.resource_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _ao_span = self.async_op_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _ao_poll_span = self.async_op_poll_span.clone().entered();\n\n let ret = if let Some(inner) = self.as_ref().get_ref().inner.as_ref() {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let res = ready!(trace_poll_op!(\"poll_recv\", inner.poll_recv(cx))).map_err(Into::into);\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let res = ready!(inner.poll_recv(cx)).map_err(Into::into);\n\n res\n } else {\n panic!(\"called after complete\");\n };\n\n self.inner = None;\n Ready(ret)\n }\n}\n\nimpl Inner {\n fn complete(&self) -> bool {\n let prev = State::set_complete(&self.state);\n\n if prev.is_closed() {\n return false;\n }\n\n if prev.is_rx_task_set() {\n // TODO: Consume waker?\n unsafe {\n self.rx_task.with_task(Waker::wake_by_ref);\n }\n }\n\n true\n }\n\n fn poll_recv(&self, cx: &mut Context<'_>) -> Poll> {\n ready!(crate::trace::trace_leaf(cx));\n // Keep track of task budget\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n // Load the state\n let mut state = State::load(&self.state, Acquire);\n\n if state.is_complete() {\n coop.made_progress();\n match unsafe { self.consume_value() } {\n Some(value) => Ready(Ok(value)),\n None => Ready(Err(RecvError(()))),\n }\n } else if state.is_closed() {\n coop.made_progress();\n Ready(Err(RecvError(())))\n } else {\n if state.is_rx_task_set() {\n let will_notify = unsafe { self.rx_task.will_wake(cx) };\n\n // Check if the task is still the same\n if !will_notify {\n // Unset the task\n state = State::unset_rx_task(&self.state);\n if state.is_complete() {\n // Set the flag again so that the waker is released in drop\n State::set_rx_task(&self.state);\n\n coop.made_progress();\n // SAFETY: If `state.is_complete()` returns true, then the\n // `VALUE_SENT` bit has been set and the sender side of the\n // channel will no longer attempt to access the inner\n // `UnsafeCell`. Therefore, it is now safe for us to access the\n // cell.\n return match unsafe { self.consume_value() } {\n Some(value) => Ready(Ok(value)),\n None => Ready(Err(RecvError(()))),\n };\n } else {\n unsafe { self.rx_task.drop_task() };\n }\n }\n }\n\n if !state.is_rx_task_set() {\n // Attempt to set the task\n unsafe {\n self.rx_task.set_task(cx);\n }\n\n // Update the state\n state = State::set_rx_task(&self.state);\n\n if state.is_complete() {\n coop.made_progress();\n match unsafe { self.consume_value() } {\n Some(value) => Ready(Ok(value)),\n None => Ready(Err(RecvError(()))),\n }\n } else {\n Pending\n }\n } else {\n Pending\n }\n }\n }\n\n /// Called by `Receiver` to indicate that the value will never be received.\n fn close(&self) -> State {\n let prev = State::set_closed(&self.state);\n\n if prev.is_tx_task_set() && !prev.is_complete() {\n unsafe {\n self.tx_task.with_task(Waker::wake_by_ref);\n }\n }\n\n if prev.is_rx_task_set() && !prev.is_complete() {\n State::unset_rx_task(&self.state);\n // SAFETY: The sender only accesses `rx_task` (via\n // `wake_by_ref`) in `complete()` after successfully setting\n // `VALUE_SENT`. But `set_complete` will not set `VALUE_SENT`\n // if `CLOSED` is already set (its CAS loop breaks early).\n // Since `prev` shows that `VALUE_SENT` was not set before we\n // set `CLOSED`, the sender can no longer set `VALUE_SENT` and\n // will never access `rx_task`. Therefore, we have exclusive\n // access here.\n unsafe { self.rx_task.drop_task() };\n }\n\n prev\n }\n\n /// Consumes the value. This function does not check `state`.\n ///\n /// # Safety\n ///\n /// Calling this method concurrently on multiple threads will result in a\n /// data race. The `VALUE_SENT` state bit is used to ensure that only the\n /// sender *or* the receiver will call this method at a given point in time.\n /// If `VALUE_SENT` is not set, then only the sender may call this method;\n /// if it is set, then only the receiver may call this method.\n unsafe fn consume_value(&self) -> Option {\n self.value.with_mut(|ptr| unsafe { (*ptr).take() })\n }\n\n /// Returns true if there is a value. This function does not check `state`.\n ///\n /// # Safety\n ///\n /// Calling this method concurrently on multiple threads will result in a\n /// data race. The `VALUE_SENT` state bit is used to ensure that only the\n /// sender *or* the receiver will call this method at a given point in time.\n /// If `VALUE_SENT` is not set, then only the sender may call this method;\n /// if it is set, then only the receiver may call this method.\n unsafe fn has_value(&self) -> bool {\n self.value.with(|ptr| unsafe { (*ptr).is_some() })\n }\n}\n\nunsafe impl Send for Inner {}\nunsafe impl Sync for Inner {}\n\nfn mut_load(this: &mut AtomicUsize) -> usize {\n this.with_mut(|v| *v)\n}\n\nimpl Drop for Inner {\n fn drop(&mut self) {\n let state = State(mut_load(&mut self.state));\n\n if state.is_rx_task_set() {\n unsafe {\n self.rx_task.drop_task();\n }\n }\n\n if state.is_tx_task_set() {\n unsafe {\n self.tx_task.drop_task();\n }\n }\n\n // SAFETY: we have `&mut self`, and therefore we have\n // exclusive access to the value.\n unsafe {\n // Note: the assertion holds because if the value has been sent by sender,\n // we must ensure that the value must have been consumed by the receiver before\n // dropping the `Inner`.\n debug_assert!(self.consume_value().is_none());\n }\n }\n}\n\nimpl fmt::Debug for Inner {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n use std::sync::atomic::Ordering::Relaxed;\n\n fmt.debug_struct(\"Inner\")\n .field(\"state\", &State::load(&self.state, Relaxed))\n .finish()\n }\n}\n\n/// Indicates that a waker for the receiving task has been set.\n///\n/// # Safety\n///\n/// If this bit is not set, the `rx_task` field may be uninitialized.\nconst RX_TASK_SET: usize = 0b00001;\n/// Indicates that a value has been stored in the channel's inner `UnsafeCell`.\n///\n/// # Safety\n///\n/// This bit controls which side of the channel is permitted to access the\n/// `UnsafeCell`. If it is set, the `UnsafeCell` may ONLY be accessed by the\n/// receiver. If this bit is NOT set, the `UnsafeCell` may ONLY be accessed by\n/// the sender.\nconst VALUE_SENT: usize = 0b00010;\nconst CLOSED: usize = 0b00100;\n\n/// Indicates that a waker for the sending task has been set.\n///\n/// # Safety\n///\n/// If this bit is not set, the `tx_task` field may be uninitialized.\nconst TX_TASK_SET: usize = 0b01000;\n\nimpl State {\n fn new() -> State {\n State(0)\n }\n\n fn is_complete(self) -> bool {\n self.0 & VALUE_SENT == VALUE_SENT\n }\n\n fn set_complete(cell: &AtomicUsize) -> State {\n // This method is a compare-and-swap loop rather than a fetch-or like\n // other `set_$WHATEVER` methods on `State`. This is because we must\n // check if the state has been closed before setting the `VALUE_SENT`\n // bit.\n //\n // We don't want to set both the `VALUE_SENT` bit if the `CLOSED`\n // bit is already set, because `VALUE_SENT` will tell the receiver that\n // it's okay to access the inner `UnsafeCell`. Immediately after calling\n // `set_complete`, if the channel was closed, the sender will _also_\n // access the `UnsafeCell` to take the value back out, so if a\n // `poll_recv` or `try_recv` call is occurring concurrently, both\n // threads may try to access the `UnsafeCell` if we were to set the\n // `VALUE_SENT` bit on a closed channel.\n let mut state = cell.load(Ordering::Relaxed);\n loop {\n if State(state).is_closed() {\n break;\n }\n // TODO: This could be `Release`, followed by an `Acquire` fence *if*\n // the `RX_TASK_SET` flag is set. However, `loom` does not support\n // fences yet.\n match cell.compare_exchange_weak(\n state,\n state | VALUE_SENT,\n Ordering::AcqRel,\n Ordering::Acquire,\n ) {\n Ok(_) => break,\n Err(actual) => state = actual,\n }\n }\n State(state)\n }\n\n fn is_rx_task_set(self) -> bool {\n self.0 & RX_TASK_SET == RX_TASK_SET\n }\n\n fn set_rx_task(cell: &AtomicUsize) -> State {\n let val = cell.fetch_or(RX_TASK_SET, AcqRel);\n State(val | RX_TASK_SET)\n }\n\n fn unset_rx_task(cell: &AtomicUsize) -> State {\n let val = cell.fetch_and(!RX_TASK_SET, AcqRel);\n State(val & !RX_TASK_SET)\n }\n\n fn is_closed(self) -> bool {\n self.0 & CLOSED == CLOSED\n }\n\n fn set_closed(cell: &AtomicUsize) -> State {\n // Acquire because we want all later writes (attempting to poll) to be\n // ordered after this.\n let val = cell.fetch_or(CLOSED, Acquire);\n State(val)\n }\n\n fn set_tx_task(cell: &AtomicUsize) -> State {\n let val = cell.fetch_or(TX_TASK_SET, AcqRel);\n State(val | TX_TASK_SET)\n }\n\n fn unset_tx_task(cell: &AtomicUsize) -> State {\n let val = cell.fetch_and(!TX_TASK_SET, AcqRel);\n State(val & !TX_TASK_SET)\n }\n\n fn is_tx_task_set(self) -> bool {\n self.0 & TX_TASK_SET == TX_TASK_SET\n }\n\n fn as_usize(self) -> usize {\n self.0\n }\n\n fn load(cell: &AtomicUsize, order: Ordering) -> State {\n let val = cell.load(order);\n State(val)\n }\n}\n\nimpl fmt::Debug for State {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"State\")\n .field(\"is_complete\", &self.is_complete())\n .field(\"is_closed\", &self.is_closed())\n .field(\"is_rx_task_set\", &self.is_rx_task_set())\n .field(\"is_tx_task_set\", &self.is_tx_task_set())\n .finish()\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/owned_read_guard.rs", "content": "use crate::sync::rwlock::RwLock;\nuse std::marker::PhantomData;\nuse std::sync::Arc;\nuse std::{fmt, mem, ops, ptr};\n\n/// Owned RAII structure used to release the shared read access of a lock when\n/// dropped.\n///\n/// This structure is created by the [`read_owned`] method on\n/// [`RwLock`].\n///\n/// [`read_owned`]: method@crate::sync::RwLock::read_owned\n/// [`RwLock`]: struct@crate::sync::RwLock\n#[clippy::has_significant_drop]\npub struct OwnedRwLockReadGuard {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) lock: Arc>,\n pub(super) data: *const U,\n pub(super) _p: PhantomData,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n lock: Arc>,\n data: *const U,\n}\n\nimpl OwnedRwLockReadGuard {\n fn skip_drop(self) -> Inner {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n unsafe {\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: ptr::read(&me.resource_span),\n lock: ptr::read(&me.lock),\n data: me.data,\n }\n }\n }\n\n /// Makes a new `OwnedRwLockReadGuard` for a component of the locked data.\n /// This operation cannot fail as the `OwnedRwLockReadGuard` passed in\n /// already locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `OwnedRwLockReadGuard::map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockReadGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// let guard = lock.read_owned().await;\n /// let guard = OwnedRwLockReadGuard::map(guard, |f| &f.0);\n ///\n /// assert_eq!(1, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn map(this: Self, f: F) -> OwnedRwLockReadGuard\n where\n F: FnOnce(&U) -> &V,\n {\n let data = f(&*this) as *const V;\n let this = this.skip_drop();\n\n OwnedRwLockReadGuard {\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Attempts to make a new [`OwnedRwLockReadGuard`] for a component of the\n /// locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `OwnedRwLockReadGuard` passed in\n /// already locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `OwnedRwLockReadGuard::try_map(..)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockReadGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// let guard = lock.read_owned().await;\n /// let guard = OwnedRwLockReadGuard::try_map(guard, |f| Some(&f.0)).expect(\"should not fail\");\n ///\n /// assert_eq!(1, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(this: Self, f: F) -> Result, Self>\n where\n F: FnOnce(&U) -> Option<&V>,\n {\n let data = match f(&*this) {\n Some(data) => data as *const V,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(OwnedRwLockReadGuard {\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n\n /// Returns a reference to the original `Arc`.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockReadGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// let guard = lock.clone().read_owned().await;\n /// assert!(Arc::ptr_eq(&lock, OwnedRwLockReadGuard::rwlock(&guard)));\n ///\n /// let guard = OwnedRwLockReadGuard::map(guard, |f| &f.0);\n /// assert!(Arc::ptr_eq(&lock, OwnedRwLockReadGuard::rwlock(&guard)));\n /// # }\n /// ```\n pub fn rwlock(this: &Self) -> &Arc> {\n &this.lock\n }\n}\n\nimpl ops::Deref for OwnedRwLockReadGuard {\n type Target = U;\n\n fn deref(&self) -> &U {\n unsafe { &*self.data }\n }\n}\n\nimpl fmt::Debug for OwnedRwLockReadGuard\nwhere\n U: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for OwnedRwLockReadGuard\nwhere\n U: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl Drop for OwnedRwLockReadGuard {\n fn drop(&mut self) {\n self.lock.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"sub\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/owned_write_guard.rs", "content": "use crate::sync::rwlock::owned_read_guard::OwnedRwLockReadGuard;\nuse crate::sync::rwlock::owned_write_guard_mapped::OwnedRwLockMappedWriteGuard;\nuse crate::sync::rwlock::RwLock;\nuse std::marker::PhantomData;\nuse std::sync::Arc;\nuse std::{fmt, mem, ops, ptr};\n\n/// Owned RAII structure used to release the exclusive write access of a lock when\n/// dropped.\n///\n/// This structure is created by the [`write_owned`] method\n/// on [`RwLock`].\n///\n/// [`write_owned`]: method@crate::sync::RwLock::write_owned\n/// [`RwLock`]: struct@crate::sync::RwLock\n#[clippy::has_significant_drop]\npub struct OwnedRwLockWriteGuard {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) permits_acquired: u32,\n pub(super) lock: Arc>,\n pub(super) data: *mut T,\n pub(super) _p: PhantomData,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n permits_acquired: u32,\n lock: Arc>,\n data: *const T,\n}\n\nimpl OwnedRwLockWriteGuard {\n fn skip_drop(self) -> Inner {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n unsafe {\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: ptr::read(&me.resource_span),\n permits_acquired: me.permits_acquired,\n lock: ptr::read(&me.lock),\n data: me.data,\n }\n }\n }\n\n /// Makes a new [`OwnedRwLockMappedWriteGuard`] for a component of the locked\n /// data.\n ///\n /// This operation cannot fail as the `OwnedRwLockWriteGuard` passed in\n /// already locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `OwnedRwLockWriteGuard::map(..)`. A method would interfere with methods\n /// of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// {\n /// let lock = Arc::clone(&lock);\n /// let mut mapped = OwnedRwLockWriteGuard::map(lock.write_owned().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn map(mut this: Self, f: F) -> OwnedRwLockMappedWriteGuard\n where\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let this = this.skip_drop();\n\n OwnedRwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Makes a new [`OwnedRwLockReadGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the `OwnedRwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `OwnedRwLockWriteGuard::downgrade_map(..)`. A method would interfere with methods of\n /// the same name on the contents of the locked data.\n ///\n /// Inside of `f`, you retain exclusive access to the data, despite only being given a `&T`. Handing out a\n /// `&mut T` would result in unsoundness, as you could use interior mutability.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// let guard = Arc::clone(&lock).write_owned().await;\n /// let mapped = OwnedRwLockWriteGuard::downgrade_map(guard, |f| &f.0);\n /// let foo = lock.read_owned().await;\n /// assert_eq!(foo.0, *mapped);\n /// # }\n /// ```\n #[inline]\n pub fn downgrade_map(this: Self, f: F) -> OwnedRwLockReadGuard\n where\n F: FnOnce(&T) -> &U,\n {\n let data = f(&*this) as *const U;\n let this = this.skip_drop();\n let guard = OwnedRwLockReadGuard {\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n guard.lock.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n\n /// Attempts to make a new [`OwnedRwLockMappedWriteGuard`] for a component\n /// of the locked data. The original guard is returned if the closure\n /// returns `None`.\n ///\n /// This operation cannot fail as the `OwnedRwLockWriteGuard` passed in\n /// already locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `OwnedRwLockWriteGuard::try_map(...)`. A method would interfere\n /// with methods of the same name on the contents of the locked data.\n ///\n /// [`RwLockMappedWriteGuard`]: struct@crate::sync::RwLockMappedWriteGuard\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// {\n /// let guard = Arc::clone(&lock).write_owned().await;\n /// let mut guard = OwnedRwLockWriteGuard::try_map(guard, |f| Some(&mut f.0)).expect(\"should not fail\");\n /// *guard = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(\n mut this: Self,\n f: F,\n ) -> Result, Self>\n where\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(OwnedRwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n\n /// Attempts to make a new [`OwnedRwLockReadGuard`] for a component of\n /// the locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `OwnedRwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `OwnedRwLockWriteGuard::try_downgrade_map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// Inside of `f`, you retain exclusive access to the data, despite only being given a `&T`. Handing out a\n /// `&mut T` would result in unsoundness, as you could use interior mutability.\n ///\n /// If this function returns `Err(...)`, the lock is never unlocked nor downgraded.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// let guard = Arc::clone(&lock).write_owned().await;\n /// let guard = OwnedRwLockWriteGuard::try_downgrade_map(guard, |f| Some(&f.0)).expect(\"should not fail\");\n /// let foo = lock.read_owned().await;\n /// assert_eq!(foo.0, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn try_downgrade_map(\n this: Self,\n f: F,\n ) -> Result, Self>\n where\n F: FnOnce(&T) -> Option<&U>,\n {\n let data = match f(&*this) {\n Some(data) => data as *const U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n let guard = OwnedRwLockReadGuard {\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n guard.lock.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n Ok(guard)\n }\n\n /// Converts this `OwnedRwLockWriteGuard` into an\n /// `OwnedRwLockMappedWriteGuard`. This method can be used to store a\n /// non-mapped guard in a struct field that expects a mapped guard.\n ///\n /// This is equivalent to calling `OwnedRwLockWriteGuard::map(guard, |me| me)`.\n #[inline]\n pub fn into_mapped(this: Self) -> OwnedRwLockMappedWriteGuard {\n Self::map(this, |me| me)\n }\n\n /// Atomically downgrades a write lock into a read lock without allowing\n /// any writers to take exclusive access of the lock in the meantime.\n ///\n /// **Note:** This won't *necessarily* allow any additional readers to acquire\n /// locks, since [`RwLock`] is fair and it is possible that a writer is next\n /// in line.\n ///\n /// Returns an RAII guard which will drop this read access of the `RwLock`\n /// when dropped.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::sync::RwLock;\n /// # use std::sync::Arc;\n /// #\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n ///\n /// let n = lock.clone().write_owned().await;\n ///\n /// let cloned_lock = lock.clone();\n /// let handle = tokio::spawn(async move {\n /// *cloned_lock.write_owned().await = 2;\n /// });\n ///\n /// let n = n.downgrade();\n /// assert_eq!(*n, 1, \"downgrade is atomic\");\n ///\n /// drop(n);\n /// handle.await.unwrap();\n /// assert_eq!(*lock.read().await, 2, \"second writer obtained write lock\");\n /// # }\n /// ```\n pub fn downgrade(self) -> OwnedRwLockReadGuard {\n let this = self.skip_drop();\n let guard = OwnedRwLockReadGuard {\n lock: this.lock,\n data: this.data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n guard.lock.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n\n /// Returns a reference to the original `Arc`.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n ///\n /// let guard = lock.clone().write_owned().await;\n /// assert!(Arc::ptr_eq(&lock, OwnedRwLockWriteGuard::rwlock(&guard)));\n /// # }\n /// ```\n pub fn rwlock(this: &Self) -> &Arc> {\n &this.lock\n }\n}\n\nimpl ops::Deref for OwnedRwLockWriteGuard {\n type Target = T;\n\n fn deref(&self) -> &T {\n unsafe { &*self.data }\n }\n}\n\nimpl ops::DerefMut for OwnedRwLockWriteGuard {\n fn deref_mut(&mut self) -> &mut T {\n unsafe { &mut *self.data }\n }\n}\n\nimpl fmt::Debug for OwnedRwLockWriteGuard\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for OwnedRwLockWriteGuard\nwhere\n T: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl Drop for OwnedRwLockWriteGuard {\n fn drop(&mut self) {\n self.lock.s.release(self.permits_acquired as usize);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/owned_write_guard_mapped.rs", "content": "use crate::sync::rwlock::RwLock;\nuse std::marker::PhantomData;\nuse std::sync::Arc;\nuse std::{fmt, mem, ops, ptr};\n\n/// Owned RAII structure used to release the exclusive write access of a lock when\n/// dropped.\n///\n/// This structure is created by [mapping] an [`OwnedRwLockWriteGuard`]. It is a\n/// separate type from `OwnedRwLockWriteGuard` to disallow downgrading a mapped\n/// guard, since doing so can cause undefined behavior.\n///\n/// [mapping]: method@crate::sync::OwnedRwLockWriteGuard::map\n/// [`OwnedRwLockWriteGuard`]: struct@crate::sync::OwnedRwLockWriteGuard\n#[clippy::has_significant_drop]\npub struct OwnedRwLockMappedWriteGuard {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) permits_acquired: u32,\n pub(super) lock: Arc>,\n pub(super) data: *mut U,\n pub(super) _p: PhantomData,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n permits_acquired: u32,\n lock: Arc>,\n data: *const U,\n}\n\nimpl OwnedRwLockMappedWriteGuard {\n fn skip_drop(self) -> Inner {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n unsafe {\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: ptr::read(&me.resource_span),\n permits_acquired: me.permits_acquired,\n lock: ptr::read(&me.lock),\n data: me.data,\n }\n }\n }\n\n /// Makes a new `OwnedRwLockMappedWriteGuard` for a component of the locked\n /// data.\n ///\n /// This operation cannot fail as the `OwnedRwLockMappedWriteGuard` passed\n /// in already locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `OwnedRwLockWriteGuard::map(..)`. A method would interfere with methods\n /// of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// {\n /// let lock = Arc::clone(&lock);\n /// let mut mapped = OwnedRwLockWriteGuard::map(lock.write_owned().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn map(mut this: Self, f: F) -> OwnedRwLockMappedWriteGuard\n where\n F: FnOnce(&mut U) -> &mut V,\n {\n let data = f(&mut *this) as *mut V;\n let this = this.skip_drop();\n\n OwnedRwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Attempts to make a new `OwnedRwLockMappedWriteGuard` for a component\n /// of the locked data. The original guard is returned if the closure\n /// returns `None`.\n ///\n /// This operation cannot fail as the `OwnedRwLockMappedWriteGuard` passed\n /// in already locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `OwnedRwLockMappedWriteGuard::try_map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{RwLock, OwnedRwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(Foo(1)));\n ///\n /// {\n /// let guard = Arc::clone(&lock).write_owned().await;\n /// let mut guard = OwnedRwLockWriteGuard::try_map(guard, |f| Some(&mut f.0)).expect(\"should not fail\");\n /// *guard = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(\n mut this: Self,\n f: F,\n ) -> Result, Self>\n where\n F: FnOnce(&mut U) -> Option<&mut V>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut V,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(OwnedRwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n lock: this.lock,\n data,\n _p: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n\n /// Returns a reference to the original `Arc`.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{\n /// RwLock,\n /// OwnedRwLockWriteGuard,\n /// OwnedRwLockMappedWriteGuard,\n /// };\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n ///\n /// let guard = lock.clone().write_owned().await;\n /// let guard = OwnedRwLockWriteGuard::map(guard, |x| x);\n /// assert!(Arc::ptr_eq(&lock, OwnedRwLockMappedWriteGuard::rwlock(&guard)));\n /// # }\n /// ```\n pub fn rwlock(this: &Self) -> &Arc> {\n &this.lock\n }\n}\n\nimpl ops::Deref for OwnedRwLockMappedWriteGuard {\n type Target = U;\n\n fn deref(&self) -> &U {\n unsafe { &*self.data }\n }\n}\n\nimpl ops::DerefMut for OwnedRwLockMappedWriteGuard {\n fn deref_mut(&mut self) -> &mut U {\n unsafe { &mut *self.data }\n }\n}\n\nimpl fmt::Debug for OwnedRwLockMappedWriteGuard\nwhere\n U: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl fmt::Display for OwnedRwLockMappedWriteGuard\nwhere\n U: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl Drop for OwnedRwLockMappedWriteGuard {\n fn drop(&mut self) {\n self.lock.s.release(self.permits_acquired as usize);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/read_guard.rs", "content": "use crate::sync::batch_semaphore::Semaphore;\nuse std::marker::PhantomData;\nuse std::{fmt, mem, ops};\n\n/// RAII structure used to release the shared read access of a lock when\n/// dropped.\n///\n/// This structure is created by the [`read`] method on\n/// [`RwLock`].\n///\n/// [`read`]: method@crate::sync::RwLock::read\n/// [`RwLock`]: struct@crate::sync::RwLock\n#[clippy::has_significant_drop]\n#[must_use = \"if unused the RwLock will immediately unlock\"]\npub struct RwLockReadGuard<'a, T: ?Sized> {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) s: &'a Semaphore,\n pub(super) data: *const T,\n pub(super) marker: PhantomData<&'a T>,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner<'a, T: ?Sized> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n s: &'a Semaphore,\n data: *const T,\n}\n\nimpl<'a, T: ?Sized> RwLockReadGuard<'a, T> {\n fn skip_drop(self) -> Inner<'a, T> {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: unsafe { std::ptr::read(&me.resource_span) },\n s: me.s,\n data: me.data,\n }\n }\n\n /// Makes a new `RwLockReadGuard` for a component of the locked data.\n ///\n /// This operation cannot fail as the `RwLockReadGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `RwLockReadGuard::map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockReadGuard::map`] from the\n /// [`parking_lot` crate].\n ///\n /// [`RwLockReadGuard::map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockReadGuard.html#method.map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockReadGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// let guard = lock.read().await;\n /// let guard = RwLockReadGuard::map(guard, |f| &f.0);\n ///\n /// assert_eq!(1, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn map(this: Self, f: F) -> RwLockReadGuard<'a, U>\n where\n F: FnOnce(&T) -> &U,\n {\n let data = f(&*this) as *const U;\n let this = this.skip_drop();\n\n RwLockReadGuard {\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Attempts to make a new [`RwLockReadGuard`] for a component of the\n /// locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `RwLockReadGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `RwLockReadGuard::try_map(..)`. A method would interfere with methods of the\n /// same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockReadGuard::try_map`] from the\n /// [`parking_lot` crate].\n ///\n /// [`RwLockReadGuard::try_map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockReadGuard.html#method.try_map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockReadGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// let guard = lock.read().await;\n /// let guard = RwLockReadGuard::try_map(guard, |f| Some(&f.0)).expect(\"should not fail\");\n ///\n /// assert_eq!(1, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(this: Self, f: F) -> Result, Self>\n where\n F: FnOnce(&T) -> Option<&U>,\n {\n let data = match f(&*this) {\n Some(data) => data as *const U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(RwLockReadGuard {\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n}\n\nimpl ops::Deref for RwLockReadGuard<'_, T> {\n type Target = T;\n\n fn deref(&self) -> &T {\n unsafe { &*self.data }\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Debug for RwLockReadGuard<'a, T>\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Display for RwLockReadGuard<'a, T>\nwhere\n T: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> Drop for RwLockReadGuard<'a, T> {\n fn drop(&mut self) {\n self.s.release(1);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"sub\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/write_guard.rs", "content": "use crate::sync::batch_semaphore::Semaphore;\nuse crate::sync::rwlock::read_guard::RwLockReadGuard;\nuse crate::sync::rwlock::write_guard_mapped::RwLockMappedWriteGuard;\nuse std::marker::PhantomData;\nuse std::{fmt, mem, ops};\n\n/// RAII structure used to release the exclusive write access of a lock when\n/// dropped.\n///\n/// This structure is created by the [`write`] method\n/// on [`RwLock`].\n///\n/// [`write`]: method@crate::sync::RwLock::write\n/// [`RwLock`]: struct@crate::sync::RwLock\n#[clippy::has_significant_drop]\n#[must_use = \"if unused the RwLock will immediately unlock\"]\npub struct RwLockWriteGuard<'a, T: ?Sized> {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) permits_acquired: u32,\n pub(super) s: &'a Semaphore,\n pub(super) data: *mut T,\n pub(super) marker: PhantomData<&'a mut T>,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner<'a, T: ?Sized> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n permits_acquired: u32,\n s: &'a Semaphore,\n data: *mut T,\n}\n\nimpl<'a, T: ?Sized> RwLockWriteGuard<'a, T> {\n fn skip_drop(self) -> Inner<'a, T> {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: unsafe { std::ptr::read(&me.resource_span) },\n permits_acquired: me.permits_acquired,\n s: me.s,\n data: me.data,\n }\n }\n\n /// Makes a new [`RwLockMappedWriteGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the `RwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `RwLockWriteGuard::map(..)`. A method would interfere with methods of\n /// the same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockWriteGuard::map`] from the\n /// [`parking_lot` crate].\n ///\n /// [`RwLockMappedWriteGuard`]: struct@crate::sync::RwLockMappedWriteGuard\n /// [`RwLockWriteGuard::map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// {\n /// let mut mapped = RwLockWriteGuard::map(lock.write().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn map(mut this: Self, f: F) -> RwLockMappedWriteGuard<'a, U>\n where\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let this = this.skip_drop();\n\n RwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Makes a new [`RwLockReadGuard`] for a component of the locked data.\n ///\n /// This operation cannot fail as the `RwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `RwLockWriteGuard::downgrade_map(..)`. A method would interfere with methods of\n /// the same name on the contents of the locked data.\n ///\n /// This is equivalent to a combination of asynchronous [`RwLockWriteGuard::map`] and [`RwLockWriteGuard::downgrade`]\n /// from the [`parking_lot` crate].\n ///\n /// Inside of `f`, you retain exclusive access to the data, despite only being given a `&T`. Handing out a\n /// `&mut T` would result in unsoundness, as you could use interior mutability.\n ///\n /// [`RwLockMappedWriteGuard`]: struct@crate::sync::RwLockMappedWriteGuard\n /// [`RwLockWriteGuard::map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.map\n /// [`RwLockWriteGuard::downgrade`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.downgrade\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// let mapped = RwLockWriteGuard::downgrade_map(lock.write().await, |f| &f.0);\n /// let foo = lock.read().await;\n /// assert_eq!(foo.0, *mapped);\n /// # }\n /// ```\n #[inline]\n pub fn downgrade_map(this: Self, f: F) -> RwLockReadGuard<'a, U>\n where\n F: FnOnce(&T) -> &U,\n {\n let data = f(&*this) as *const U;\n let this = this.skip_drop();\n let guard = RwLockReadGuard {\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n this.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n\n /// Attempts to make a new [`RwLockMappedWriteGuard`] for a component of\n /// the locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `RwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `RwLockWriteGuard::try_map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockWriteGuard::try_map`] from\n /// the [`parking_lot` crate].\n ///\n /// [`RwLockMappedWriteGuard`]: struct@crate::sync::RwLockMappedWriteGuard\n /// [`RwLockWriteGuard::try_map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.try_map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// {\n /// let guard = lock.write().await;\n /// let mut guard = RwLockWriteGuard::try_map(guard, |f| Some(&mut f.0)).expect(\"should not fail\");\n /// *guard = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(\n mut this: Self,\n f: F,\n ) -> Result, Self>\n where\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(RwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n\n /// Attempts to make a new [`RwLockReadGuard`] for a component of\n /// the locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `RwLockWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `RwLockWriteGuard::try_downgrade_map(...)`. A method would interfere with\n /// methods of the same name on the contents of the locked data.\n ///\n /// This is equivalent to a combination of asynchronous [`RwLockWriteGuard::try_map`] and [`RwLockWriteGuard::downgrade`]\n /// from the [`parking_lot` crate].\n ///\n /// Inside of `f`, you retain exclusive access to the data, despite only being given a `&T`. Handing out a\n /// `&mut T` would result in unsoundness, as you could use interior mutability.\n ///\n /// If this function returns `Err(...)`, the lock is never unlocked nor downgraded.\n ///\n /// [`RwLockMappedWriteGuard`]: struct@crate::sync::RwLockMappedWriteGuard\n /// [`RwLockWriteGuard::map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.map\n /// [`RwLockWriteGuard::downgrade`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.downgrade\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// let guard = RwLockWriteGuard::try_downgrade_map(lock.write().await, |f| Some(&f.0)).expect(\"should not fail\");\n /// let foo = lock.read().await;\n /// assert_eq!(foo.0, *guard);\n /// # }\n /// ```\n #[inline]\n pub fn try_downgrade_map(this: Self, f: F) -> Result, Self>\n where\n F: FnOnce(&T) -> Option<&U>,\n {\n let data = match f(&*this) {\n Some(data) => data as *const U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n let guard = RwLockReadGuard {\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n this.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n Ok(guard)\n }\n\n /// Converts this `RwLockWriteGuard` into an `RwLockMappedWriteGuard`. This\n /// method can be used to store a non-mapped guard in a struct field that\n /// expects a mapped guard.\n ///\n /// This is equivalent to calling `RwLockWriteGuard::map(guard, |me| me)`.\n #[inline]\n pub fn into_mapped(this: Self) -> RwLockMappedWriteGuard<'a, T> {\n RwLockWriteGuard::map(this, |me| me)\n }\n\n /// Atomically downgrades a write lock into a read lock without allowing\n /// any writers to take exclusive access of the lock in the meantime.\n ///\n /// **Note:** This won't *necessarily* allow any additional readers to acquire\n /// locks, since [`RwLock`] is fair and it is possible that a writer is next\n /// in line.\n ///\n /// Returns an RAII guard which will drop this read access of the `RwLock`\n /// when dropped.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::sync::RwLock;\n /// # use std::sync::Arc;\n /// #\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n ///\n /// let n = lock.write().await;\n ///\n /// let cloned_lock = lock.clone();\n /// let handle = tokio::spawn(async move {\n /// *cloned_lock.write().await = 2;\n /// });\n ///\n /// let n = n.downgrade();\n /// assert_eq!(*n, 1, \"downgrade is atomic\");\n ///\n /// drop(n);\n /// handle.await.unwrap();\n /// assert_eq!(*lock.read().await, 2, \"second writer obtained write lock\");\n /// # }\n /// ```\n ///\n /// [`RwLock`]: struct@crate::sync::RwLock\n pub fn downgrade(self) -> RwLockReadGuard<'a, T> {\n let this = self.skip_drop();\n let guard = RwLockReadGuard {\n s: this.s,\n data: this.data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n };\n\n // Release all but one of the permits held by the write guard\n let to_release = (this.permits_acquired - 1) as usize;\n this.s.release(to_release);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n}\n\nimpl ops::Deref for RwLockWriteGuard<'_, T> {\n type Target = T;\n\n fn deref(&self) -> &T {\n unsafe { &*self.data }\n }\n}\n\nimpl ops::DerefMut for RwLockWriteGuard<'_, T> {\n fn deref_mut(&mut self) -> &mut T {\n unsafe { &mut *self.data }\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Debug for RwLockWriteGuard<'a, T>\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Display for RwLockWriteGuard<'a, T>\nwhere\n T: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> Drop for RwLockWriteGuard<'a, T> {\n fn drop(&mut self) {\n self.s.release(self.permits_acquired as usize);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock/write_guard_mapped.rs", "content": "use crate::sync::batch_semaphore::Semaphore;\nuse std::marker::PhantomData;\nuse std::{fmt, mem, ops};\n\n/// RAII structure used to release the exclusive write access of a lock when\n/// dropped.\n///\n/// This structure is created by [mapping] an [`RwLockWriteGuard`]. It is a\n/// separate type from `RwLockWriteGuard` to disallow downgrading a mapped\n/// guard, since doing so can cause undefined behavior.\n///\n/// [mapping]: method@crate::sync::RwLockWriteGuard::map\n/// [`RwLockWriteGuard`]: struct@crate::sync::RwLockWriteGuard\n#[clippy::has_significant_drop]\npub struct RwLockMappedWriteGuard<'a, T: ?Sized> {\n // When changing the fields in this struct, make sure to update the\n // `skip_drop` method.\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(super) resource_span: tracing::Span,\n pub(super) permits_acquired: u32,\n pub(super) s: &'a Semaphore,\n pub(super) data: *mut T,\n pub(super) marker: PhantomData<&'a mut T>,\n}\n\n#[allow(dead_code)] // Unused fields are still used in Drop.\nstruct Inner<'a, T: ?Sized> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n permits_acquired: u32,\n s: &'a Semaphore,\n data: *mut T,\n}\n\nimpl<'a, T: ?Sized> RwLockMappedWriteGuard<'a, T> {\n fn skip_drop(self) -> Inner<'a, T> {\n let me = mem::ManuallyDrop::new(self);\n // SAFETY: This duplicates the values in every field of the guard, then\n // forgets the originals, so in the end no value is duplicated.\n Inner {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: unsafe { std::ptr::read(&me.resource_span) },\n permits_acquired: me.permits_acquired,\n s: me.s,\n data: me.data,\n }\n }\n\n /// Makes a new `RwLockMappedWriteGuard` for a component of the locked data.\n ///\n /// This operation cannot fail as the `RwLockMappedWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be used as\n /// `RwLockMappedWriteGuard::map(..)`. A method would interfere with methods\n /// of the same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockWriteGuard::map`] from the\n /// [`parking_lot` crate].\n ///\n /// [`RwLockWriteGuard::map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// {\n /// let mut mapped = RwLockWriteGuard::map(lock.write().await, |f| &mut f.0);\n /// *mapped = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn map(mut this: Self, f: F) -> RwLockMappedWriteGuard<'a, U>\n where\n F: FnOnce(&mut T) -> &mut U,\n {\n let data = f(&mut *this) as *mut U;\n let this = this.skip_drop();\n\n RwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n }\n }\n\n /// Attempts to make a new [`RwLockMappedWriteGuard`] for a component of\n /// the locked data. The original guard is returned if the closure returns\n /// `None`.\n ///\n /// This operation cannot fail as the `RwLockMappedWriteGuard` passed in already\n /// locked the data.\n ///\n /// This is an associated function that needs to be\n /// used as `RwLockMappedWriteGuard::try_map(...)`. A method would interfere\n /// with methods of the same name on the contents of the locked data.\n ///\n /// This is an asynchronous version of [`RwLockWriteGuard::try_map`] from\n /// the [`parking_lot` crate].\n ///\n /// [`RwLockWriteGuard::try_map`]: https://docs.rs/lock_api/latest/lock_api/struct.RwLockWriteGuard.html#method.try_map\n /// [`parking_lot` crate]: https://crates.io/crates/parking_lot\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{RwLock, RwLockWriteGuard};\n ///\n /// #[derive(Debug, Clone, Copy, PartialEq, Eq)]\n /// struct Foo(u32);\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(Foo(1));\n ///\n /// {\n /// let guard = lock.write().await;\n /// let mut guard = RwLockWriteGuard::try_map(guard, |f| Some(&mut f.0)).expect(\"should not fail\");\n /// *guard = 2;\n /// }\n ///\n /// assert_eq!(Foo(2), *lock.read().await);\n /// # }\n /// ```\n #[inline]\n pub fn try_map(\n mut this: Self,\n f: F,\n ) -> Result, Self>\n where\n F: FnOnce(&mut T) -> Option<&mut U>,\n {\n let data = match f(&mut *this) {\n Some(data) => data as *mut U,\n None => return Err(this),\n };\n let this = this.skip_drop();\n\n Ok(RwLockMappedWriteGuard {\n permits_acquired: this.permits_acquired,\n s: this.s,\n data,\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: this.resource_span,\n })\n }\n\n // Note: No `downgrade`, `downgrade_map` nor `try_downgrade_map` because they would be unsound, as we're already\n // potentially been mapped with internal mutability.\n}\n\nimpl ops::Deref for RwLockMappedWriteGuard<'_, T> {\n type Target = T;\n\n fn deref(&self) -> &T {\n unsafe { &*self.data }\n }\n}\n\nimpl ops::DerefMut for RwLockMappedWriteGuard<'_, T> {\n fn deref_mut(&mut self) -> &mut T {\n unsafe { &mut *self.data }\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Debug for RwLockMappedWriteGuard<'a, T>\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Debug::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> fmt::Display for RwLockMappedWriteGuard<'a, T>\nwhere\n T: fmt::Display,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(&**self, f)\n }\n}\n\nimpl<'a, T: ?Sized> Drop for RwLockMappedWriteGuard<'a, T> {\n fn drop(&mut self) {\n self.s.release(self.permits_acquired as usize);\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n write_locked.op = \"override\",\n )\n });\n }\n}\n" }, { "path": "tokio/src/sync/rwlock.rs", "content": "use crate::sync::batch_semaphore::{Semaphore, TryAcquireError};\nuse crate::sync::mutex::TryLockError;\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\nuse std::cell::UnsafeCell;\nuse std::marker;\nuse std::marker::PhantomData;\nuse std::sync::Arc;\n\npub(crate) mod owned_read_guard;\npub(crate) mod owned_write_guard;\npub(crate) mod owned_write_guard_mapped;\npub(crate) mod read_guard;\npub(crate) mod write_guard;\npub(crate) mod write_guard_mapped;\npub(crate) use owned_read_guard::OwnedRwLockReadGuard;\npub(crate) use owned_write_guard::OwnedRwLockWriteGuard;\npub(crate) use owned_write_guard_mapped::OwnedRwLockMappedWriteGuard;\npub(crate) use read_guard::RwLockReadGuard;\npub(crate) use write_guard::RwLockWriteGuard;\npub(crate) use write_guard_mapped::RwLockMappedWriteGuard;\n\n#[cfg(not(loom))]\nconst MAX_READS: u32 = u32::MAX >> 3;\n\n#[cfg(loom)]\nconst MAX_READS: u32 = 10;\n\n/// An asynchronous reader-writer lock.\n///\n/// This type of lock allows a number of readers or at most one writer at any\n/// point in time. The write portion of this lock typically allows modification\n/// of the underlying data (exclusive access) and the read portion of this lock\n/// typically allows for read-only access (shared access).\n///\n/// In comparison, a [`Mutex`] does not distinguish between readers or writers\n/// that acquire the lock, therefore causing any tasks waiting for the lock to\n/// become available to yield. An `RwLock` will allow any number of readers to\n/// acquire the lock as long as a writer is not holding the lock.\n///\n/// The priority policy of Tokio's read-write lock is _fair_ (or\n/// [_write-preferring_]), in order to ensure that readers cannot starve\n/// writers. Fairness is ensured using a first-in, first-out queue for the tasks\n/// awaiting the lock; a read lock will not be given out until all write lock\n/// requests that were queued before it have been acquired and released. This is\n/// in contrast to the Rust standard library's `std::sync::RwLock`, where the\n/// priority policy is dependent on the operating system's implementation.\n///\n/// The type parameter `T` represents the data that this lock protects. It is\n/// required that `T` satisfies [`Send`] to be shared across threads. The RAII guards\n/// returned from the locking methods implement [`Deref`](trait@std::ops::Deref)\n/// (and [`DerefMut`](trait@std::ops::DerefMut)\n/// for the `write` methods) to allow access to the content of the lock.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::sync::RwLock;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let lock = RwLock::new(5);\n///\n/// // many reader locks can be held at once\n/// {\n/// let r1 = lock.read().await;\n/// let r2 = lock.read().await;\n/// assert_eq!(*r1, 5);\n/// assert_eq!(*r2, 5);\n/// } // read locks are dropped at this point\n///\n/// // only one write lock may be held, however\n/// {\n/// let mut w = lock.write().await;\n/// *w += 1;\n/// assert_eq!(*w, 6);\n/// } // write lock is dropped here\n/// # }\n/// ```\n///\n/// [`Mutex`]: struct@super::Mutex\n/// [`RwLock`]: struct@RwLock\n/// [`RwLockReadGuard`]: struct@RwLockReadGuard\n/// [`RwLockWriteGuard`]: struct@RwLockWriteGuard\n/// [`Send`]: trait@std::marker::Send\n/// [_write-preferring_]: https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock#Priority_policies\npub struct RwLock {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n\n // maximum number of concurrent readers\n mr: u32,\n\n //semaphore to coordinate read and write access to T\n s: Semaphore,\n\n //inner data T\n c: UnsafeCell,\n}\n\n#[test]\n#[cfg(not(loom))]\nfn bounds() {\n fn check_send() {}\n fn check_sync() {}\n fn check_unpin() {}\n // This has to take a value, since the async fn's return type is unnameable.\n fn check_send_sync_val(_t: T) {}\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n check_send::>();\n check_sync::>();\n check_unpin::>();\n\n let rwlock = Arc::new(RwLock::new(0));\n check_send_sync_val(rwlock.read());\n check_send_sync_val(Arc::clone(&rwlock).read_owned());\n check_send_sync_val(rwlock.write());\n check_send_sync_val(Arc::clone(&rwlock).write_owned());\n}\n\n// As long as T: Send + Sync, it's fine to send and share RwLock between threads.\n// If T were not Send, sending and sharing a RwLock would be bad, since you can access T through\n// RwLock.\nunsafe impl Send for RwLock where T: ?Sized + Send {}\nunsafe impl Sync for RwLock where T: ?Sized + Send + Sync {}\n// NB: These impls need to be explicit since we're storing a raw pointer.\n// Safety: Stores a raw pointer to `T`, so if `T` is `Sync`, the lock guard over\n// `T` is `Send`.\nunsafe impl Send for RwLockReadGuard<'_, T> where T: ?Sized + Sync {}\nunsafe impl Sync for RwLockReadGuard<'_, T> where T: ?Sized + Send + Sync {}\n// T is required to be `Send` because an OwnedRwLockReadGuard can be used to drop the value held in\n// the RwLock, unlike RwLockReadGuard.\nunsafe impl Send for OwnedRwLockReadGuard\nwhere\n T: ?Sized + Send + Sync,\n U: ?Sized + Sync,\n{\n}\nunsafe impl Sync for OwnedRwLockReadGuard\nwhere\n T: ?Sized + Send + Sync,\n U: ?Sized + Send + Sync,\n{\n}\nunsafe impl Sync for RwLockWriteGuard<'_, T> where T: ?Sized + Send + Sync {}\nunsafe impl Sync for OwnedRwLockWriteGuard where T: ?Sized + Send + Sync {}\nunsafe impl Sync for RwLockMappedWriteGuard<'_, T> where T: ?Sized + Send + Sync {}\nunsafe impl Sync for OwnedRwLockMappedWriteGuard\nwhere\n T: ?Sized + Send + Sync,\n U: ?Sized + Send + Sync,\n{\n}\n// Safety: Stores a raw pointer to `T`, so if `T` is `Sync`, the lock guard over\n// `T` is `Send` - but since this is also provides mutable access, we need to\n// make sure that `T` is `Send` since its value can be sent across thread\n// boundaries.\nunsafe impl Send for RwLockWriteGuard<'_, T> where T: ?Sized + Send + Sync {}\nunsafe impl Send for OwnedRwLockWriteGuard where T: ?Sized + Send + Sync {}\nunsafe impl Send for RwLockMappedWriteGuard<'_, T> where T: ?Sized + Send + Sync {}\nunsafe impl Send for OwnedRwLockMappedWriteGuard\nwhere\n T: ?Sized + Send + Sync,\n U: ?Sized + Send + Sync,\n{\n}\n\nimpl RwLock {\n /// Creates a new instance of an `RwLock` which is unlocked.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// let lock = RwLock::new(5);\n /// ```\n #[track_caller]\n pub fn new(value: T) -> RwLock\n where\n T: Sized,\n {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"RwLock\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n );\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n max_readers = MAX_READS,\n );\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n );\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 0,\n );\n });\n\n resource_span\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let s = resource_span.in_scope(|| Semaphore::new(MAX_READS as usize));\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let s = Semaphore::new(MAX_READS as usize);\n\n RwLock {\n mr: MAX_READS,\n c: UnsafeCell::new(value),\n s,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Creates a new instance of an `RwLock` which is unlocked\n /// and allows a maximum of `max_reads` concurrent readers.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// let lock = RwLock::with_max_readers(5, 1024);\n /// ```\n ///\n /// # Panics\n ///\n /// Panics if `max_reads` is more than `u32::MAX >> 3`.\n #[track_caller]\n pub fn with_max_readers(value: T, max_reads: u32) -> RwLock\n where\n T: Sized,\n {\n assert!(\n max_reads <= MAX_READS,\n \"a RwLock may not be created with more than {MAX_READS} readers\"\n );\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"RwLock\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n );\n\n resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n max_readers = max_reads,\n );\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = false,\n );\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 0,\n );\n });\n\n resource_span\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let s = resource_span.in_scope(|| Semaphore::new(max_reads as usize));\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let s = Semaphore::new(max_reads as usize);\n\n RwLock {\n mr: max_reads,\n c: UnsafeCell::new(value),\n s,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Creates a new instance of an `RwLock` which is unlocked.\n ///\n /// When using the `tracing` [unstable feature], a `RwLock` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`RwLock::new`] should be used to create\n /// an instrumented object if that is needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// static LOCK: RwLock = RwLock::const_new(5);\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new(value: T) -> RwLock\n where\n T: Sized,\n {\n RwLock {\n mr: MAX_READS,\n c: UnsafeCell::new(value),\n s: Semaphore::const_new(MAX_READS as usize),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Creates a new instance of an `RwLock` which is unlocked\n /// and allows a maximum of `max_reads` concurrent readers.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// static LOCK: RwLock = RwLock::const_with_max_readers(5, 1024);\n /// ```\n #[cfg(not(all(loom, test)))]\n pub const fn const_with_max_readers(value: T, max_reads: u32) -> RwLock\n where\n T: Sized,\n {\n assert!(max_reads <= MAX_READS);\n\n RwLock {\n mr: max_reads,\n c: UnsafeCell::new(value),\n s: Semaphore::const_new(max_reads as usize),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Locks this `RwLock` with shared read access, causing the current task\n /// to yield until the lock has been acquired.\n ///\n /// The calling task will yield until there are no writers which hold the\n /// lock. There may be other readers inside the lock when the task resumes.\n ///\n /// Note that under the priority policy of [`RwLock`], read locks are not\n /// granted until prior write locks, to prevent starvation. Therefore\n /// deadlock may occur if a read lock is held by the current task, a write\n /// lock attempt is made, and then a subsequent read lock attempt is made\n /// by the current task.\n ///\n /// Returns an RAII guard which will drop this read access of the `RwLock`\n /// when dropped.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `read` makes you lose your place in\n /// the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n /// let c_lock = lock.clone();\n ///\n /// let n = lock.read().await;\n /// assert_eq!(*n, 1);\n ///\n /// tokio::spawn(async move {\n /// // While main has an active read lock, we acquire one too.\n /// let r = c_lock.read().await;\n /// assert_eq!(*r, 1);\n /// }).await.expect(\"The spawned task has panicked\");\n ///\n /// // Drop the guard after the spawned task finishes.\n /// drop(n);\n /// # }\n /// ```\n pub async fn read(&self) -> RwLockReadGuard<'_, T> {\n let acquire_fut = async {\n self.s.acquire(1).await.unwrap_or_else(|_| {\n // The semaphore was closed. but, we never explicitly close it, and we have a\n // handle to it through the Arc, which means that this can never happen.\n unreachable!()\n });\n\n RwLockReadGuard {\n s: &self.s,\n data: self.c.get(),\n marker: PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n self.resource_span.clone(),\n \"RwLock::read\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n\n /// Blockingly locks this `RwLock` with shared read access.\n ///\n /// This method is intended for use cases where you\n /// need to use this rwlock in asynchronous code as well as in synchronous code.\n ///\n /// Returns an RAII guard which will drop the read access of this `RwLock` when dropped.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution context.\n ///\n /// - If you find yourself in an asynchronous execution context and needing\n /// to call some (synchronous) function which performs one of these\n /// `blocking_` operations, then consider wrapping that call inside\n /// [`spawn_blocking()`][crate::runtime::Handle::spawn_blocking]\n /// (or [`block_in_place()`][crate::task::block_in_place]).\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let rwlock = Arc::new(RwLock::new(1));\n /// let mut write_lock = rwlock.write().await;\n ///\n /// let blocking_task = tokio::task::spawn_blocking({\n /// let rwlock = Arc::clone(&rwlock);\n /// move || {\n /// // This shall block until the `write_lock` is released.\n /// let read_lock = rwlock.blocking_read();\n /// assert_eq!(*read_lock, 0);\n /// }\n /// });\n ///\n /// *write_lock -= 1;\n /// drop(write_lock); // release the lock.\n ///\n /// // Await the completion of the blocking task.\n /// blocking_task.await.unwrap();\n ///\n /// // Assert uncontended.\n /// assert!(rwlock.try_write().is_ok());\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n pub fn blocking_read(&self) -> RwLockReadGuard<'_, T> {\n crate::future::block_on(self.read())\n }\n\n /// Locks this `RwLock` with shared read access, causing the current task\n /// to yield until the lock has been acquired.\n ///\n /// The calling task will yield until there are no writers which hold the\n /// lock. There may be other readers inside the lock when the task resumes.\n ///\n /// This method is identical to [`RwLock::read`], except that the returned\n /// guard references the `RwLock` with an [`Arc`] rather than by borrowing\n /// it. Therefore, the `RwLock` must be wrapped in an `Arc` to call this\n /// method, and the guard will live for the `'static` lifetime, as it keeps\n /// the `RwLock` alive by holding an `Arc`.\n ///\n /// Note that under the priority policy of [`RwLock`], read locks are not\n /// granted until prior write locks, to prevent starvation. Therefore\n /// deadlock may occur if a read lock is held by the current task, a write\n /// lock attempt is made, and then a subsequent read lock attempt is made\n /// by the current task.\n ///\n /// Returns an RAII guard which will drop this read access of the `RwLock`\n /// when dropped.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `read_owned` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n /// let c_lock = lock.clone();\n ///\n /// let n = lock.read_owned().await;\n /// assert_eq!(*n, 1);\n ///\n /// tokio::spawn(async move {\n /// // While main has an active read lock, we acquire one too.\n /// let r = c_lock.read_owned().await;\n /// assert_eq!(*r, 1);\n /// }).await.expect(\"The spawned task has panicked\");\n ///\n /// // Drop the guard after the spawned task finishes.\n /// drop(n);\n ///}\n /// ```\n pub async fn read_owned(self: Arc) -> OwnedRwLockReadGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = self.resource_span.clone();\n\n let acquire_fut = async {\n self.s.acquire(1).await.unwrap_or_else(|_| {\n // The semaphore was closed. but, we never explicitly close it, and we have a\n // handle to it through the Arc, which means that this can never happen.\n unreachable!()\n });\n\n OwnedRwLockReadGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n data: self.c.get(),\n lock: self,\n _p: PhantomData,\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n resource_span,\n \"RwLock::read_owned\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n guard\n }\n\n /// Attempts to acquire this `RwLock` with shared read access.\n ///\n /// If the access couldn't be acquired immediately, returns [`TryLockError`].\n /// Otherwise, an RAII guard is returned which will release read access\n /// when dropped.\n ///\n /// [`TryLockError`]: TryLockError\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n /// let c_lock = lock.clone();\n ///\n /// let v = lock.try_read().unwrap();\n /// assert_eq!(*v, 1);\n ///\n /// tokio::spawn(async move {\n /// // While main has an active read lock, we acquire one too.\n /// let n = c_lock.read().await;\n /// assert_eq!(*n, 1);\n /// }).await.expect(\"The spawned task has panicked\");\n ///\n /// // Drop the guard when spawned task finishes.\n /// drop(v);\n /// # }\n /// ```\n pub fn try_read(&self) -> Result, TryLockError> {\n match self.s.try_acquire(1) {\n Ok(permit) => permit,\n Err(TryAcquireError::NoPermits) => return Err(TryLockError(())),\n Err(TryAcquireError::Closed) => unreachable!(),\n }\n\n let guard = RwLockReadGuard {\n s: &self.s,\n data: self.c.get(),\n marker: marker::PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n Ok(guard)\n }\n\n /// Attempts to acquire this `RwLock` with shared read access.\n ///\n /// If the access couldn't be acquired immediately, returns [`TryLockError`].\n /// Otherwise, an RAII guard is returned which will release read access\n /// when dropped.\n ///\n /// This method is identical to [`RwLock::try_read`], except that the\n /// returned guard references the `RwLock` with an [`Arc`] rather than by\n /// borrowing it. Therefore, the `RwLock` must be wrapped in an `Arc` to\n /// call this method, and the guard will live for the `'static` lifetime,\n /// as it keeps the `RwLock` alive by holding an `Arc`.\n ///\n /// [`TryLockError`]: TryLockError\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n /// let c_lock = lock.clone();\n ///\n /// let v = lock.try_read_owned().unwrap();\n /// assert_eq!(*v, 1);\n ///\n /// tokio::spawn(async move {\n /// // While main has an active read lock, we acquire one too.\n /// let n = c_lock.read_owned().await;\n /// assert_eq!(*n, 1);\n /// }).await.expect(\"The spawned task has panicked\");\n ///\n /// // Drop the guard when spawned task finishes.\n /// drop(v);\n /// # }\n /// ```\n pub fn try_read_owned(self: Arc) -> Result, TryLockError> {\n match self.s.try_acquire(1) {\n Ok(permit) => permit,\n Err(TryAcquireError::NoPermits) => return Err(TryLockError(())),\n Err(TryAcquireError::Closed) => unreachable!(),\n }\n\n let guard = OwnedRwLockReadGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n data: self.c.get(),\n lock: self,\n _p: PhantomData,\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n current_readers = 1,\n current_readers.op = \"add\",\n )\n });\n\n Ok(guard)\n }\n\n /// Locks this `RwLock` with exclusive write access, causing the current\n /// task to yield until the lock has been acquired.\n ///\n /// The calling task will yield while other writers or readers currently\n /// have access to the lock.\n ///\n /// Returns an RAII guard which will drop the write access of this `RwLock`\n /// when dropped.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `write` makes you lose your place\n /// in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = RwLock::new(1);\n ///\n /// let mut n = lock.write().await;\n /// *n = 2;\n /// # }\n /// ```\n pub async fn write(&self) -> RwLockWriteGuard<'_, T> {\n let acquire_fut = async {\n self.s.acquire(self.mr as usize).await.unwrap_or_else(|_| {\n // The semaphore was closed. but, we never explicitly close it, and we have a\n // handle to it through the Arc, which means that this can never happen.\n unreachable!()\n });\n\n RwLockWriteGuard {\n permits_acquired: self.mr,\n s: &self.s,\n data: self.c.get(),\n marker: marker::PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n self.resource_span.clone(),\n \"RwLock::write\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = true,\n write_locked.op = \"override\",\n )\n });\n\n guard\n }\n\n /// Blockingly locks this `RwLock` with exclusive write access.\n ///\n /// This method is intended for use cases where you\n /// need to use this rwlock in asynchronous code as well as in synchronous code.\n ///\n /// Returns an RAII guard which will drop the write access of this `RwLock` when dropped.\n ///\n /// # Panics\n ///\n /// This function panics if called within an asynchronous execution context.\n ///\n /// - If you find yourself in an asynchronous execution context and needing\n /// to call some (synchronous) function which performs one of these\n /// `blocking_` operations, then consider wrapping that call inside\n /// [`spawn_blocking()`][crate::runtime::Handle::spawn_blocking]\n /// (or [`block_in_place()`][crate::task::block_in_place]).\n ///\n /// # Examples\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::sync::Arc;\n /// use tokio::{sync::RwLock};\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let rwlock = Arc::new(RwLock::new(1));\n /// let read_lock = rwlock.read().await;\n ///\n /// let blocking_task = tokio::task::spawn_blocking({\n /// let rwlock = Arc::clone(&rwlock);\n /// move || {\n /// // This shall block until the `read_lock` is released.\n /// let mut write_lock = rwlock.blocking_write();\n /// *write_lock = 2;\n /// }\n /// });\n ///\n /// assert_eq!(*read_lock, 1);\n /// // Release the last outstanding read lock.\n /// drop(read_lock);\n ///\n /// // Await the completion of the blocking task.\n /// blocking_task.await.unwrap();\n ///\n /// // Assert uncontended.\n /// let read_lock = rwlock.try_read().unwrap();\n /// assert_eq!(*read_lock, 2);\n /// }\n /// # }\n /// ```\n #[track_caller]\n #[cfg(feature = \"sync\")]\n pub fn blocking_write(&self) -> RwLockWriteGuard<'_, T> {\n crate::future::block_on(self.write())\n }\n\n /// Locks this `RwLock` with exclusive write access, causing the current\n /// task to yield until the lock has been acquired.\n ///\n /// The calling task will yield while other writers or readers currently\n /// have access to the lock.\n ///\n /// This method is identical to [`RwLock::write`], except that the returned\n /// guard references the `RwLock` with an [`Arc`] rather than by borrowing\n /// it. Therefore, the `RwLock` must be wrapped in an `Arc` to call this\n /// method, and the guard will live for the `'static` lifetime, as it keeps\n /// the `RwLock` alive by holding an `Arc`.\n ///\n /// Returns an RAII guard which will drop the write access of this `RwLock`\n /// when dropped.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute locks in the order they\n /// were requested. Cancelling a call to `write_owned` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let lock = Arc::new(RwLock::new(1));\n ///\n /// let mut n = lock.write_owned().await;\n /// *n = 2;\n ///}\n /// ```\n pub async fn write_owned(self: Arc) -> OwnedRwLockWriteGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = self.resource_span.clone();\n\n let acquire_fut = async {\n self.s.acquire(self.mr as usize).await.unwrap_or_else(|_| {\n // The semaphore was closed. but, we never explicitly close it, and we have a\n // handle to it through the Arc, which means that this can never happen.\n unreachable!()\n });\n\n OwnedRwLockWriteGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n permits_acquired: self.mr,\n data: self.c.get(),\n lock: self,\n _p: PhantomData,\n }\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let acquire_fut = trace::async_op(\n move || acquire_fut,\n resource_span,\n \"RwLock::write_owned\",\n \"poll\",\n false,\n );\n\n #[allow(clippy::let_and_return)] // this lint triggers when disabling tracing\n let guard = acquire_fut.await;\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = true,\n write_locked.op = \"override\",\n )\n });\n\n guard\n }\n\n /// Attempts to acquire this `RwLock` with exclusive write access.\n ///\n /// If the access couldn't be acquired immediately, returns [`TryLockError`].\n /// Otherwise, an RAII guard is returned which will release write access\n /// when dropped.\n ///\n /// [`TryLockError`]: TryLockError\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let rw = RwLock::new(1);\n ///\n /// let v = rw.read().await;\n /// assert_eq!(*v, 1);\n ///\n /// assert!(rw.try_write().is_err());\n /// # }\n /// ```\n pub fn try_write(&self) -> Result, TryLockError> {\n match self.s.try_acquire(self.mr as usize) {\n Ok(permit) => permit,\n Err(TryAcquireError::NoPermits) => return Err(TryLockError(())),\n Err(TryAcquireError::Closed) => unreachable!(),\n }\n\n let guard = RwLockWriteGuard {\n permits_acquired: self.mr,\n s: &self.s,\n data: self.c.get(),\n marker: marker::PhantomData,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n self.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = true,\n write_locked.op = \"override\",\n )\n });\n\n Ok(guard)\n }\n\n /// Attempts to acquire this `RwLock` with exclusive write access.\n ///\n /// If the access couldn't be acquired immediately, returns [`TryLockError`].\n /// Otherwise, an RAII guard is returned which will release write access\n /// when dropped.\n ///\n /// This method is identical to [`RwLock::try_write`], except that the\n /// returned guard references the `RwLock` with an [`Arc`] rather than by\n /// borrowing it. Therefore, the `RwLock` must be wrapped in an `Arc` to\n /// call this method, and the guard will live for the `'static` lifetime,\n /// as it keeps the `RwLock` alive by holding an `Arc`.\n ///\n /// [`TryLockError`]: TryLockError\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::RwLock;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let rw = Arc::new(RwLock::new(1));\n ///\n /// let v = Arc::clone(&rw).read_owned().await;\n /// assert_eq!(*v, 1);\n ///\n /// assert!(rw.try_write_owned().is_err());\n /// # }\n /// ```\n pub fn try_write_owned(self: Arc) -> Result, TryLockError> {\n match self.s.try_acquire(self.mr as usize) {\n Ok(permit) => permit,\n Err(TryAcquireError::NoPermits) => return Err(TryLockError(())),\n Err(TryAcquireError::Closed) => unreachable!(),\n }\n\n let guard = OwnedRwLockWriteGuard {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: self.resource_span.clone(),\n permits_acquired: self.mr,\n data: self.c.get(),\n lock: self,\n _p: PhantomData,\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n guard.resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n write_locked = true,\n write_locked.op = \"override\",\n )\n });\n\n Ok(guard)\n }\n\n /// Returns a mutable reference to the underlying data.\n ///\n /// Since this call borrows the `RwLock` mutably, no actual locking needs to\n /// take place -- the mutable borrow statically guarantees no locks exist.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::RwLock;\n ///\n /// fn main() {\n /// let mut lock = RwLock::new(1);\n ///\n /// let n = lock.get_mut();\n /// *n = 2;\n /// }\n /// ```\n pub fn get_mut(&mut self) -> &mut T {\n self.c.get_mut()\n }\n\n /// Consumes the lock, returning the underlying data.\n pub fn into_inner(self) -> T\n where\n T: Sized,\n {\n self.c.into_inner()\n }\n}\n\nimpl From for RwLock {\n fn from(s: T) -> Self {\n Self::new(s)\n }\n}\n\nimpl Default for RwLock\nwhere\n T: Default,\n{\n fn default() -> Self {\n Self::new(T::default())\n }\n}\n\nimpl std::fmt::Debug for RwLock\nwhere\n T: std::fmt::Debug,\n{\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n let mut d = f.debug_struct(\"RwLock\");\n match self.try_read() {\n Ok(inner) => d.field(\"data\", &&*inner),\n Err(_) => d.field(\"data\", &format_args!(\"\")),\n };\n d.finish()\n }\n}\n" }, { "path": "tokio/src/sync/semaphore.rs", "content": "use super::batch_semaphore as ll; // low level implementation\nuse super::{AcquireError, TryAcquireError};\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\nuse crate::util::trace;\nuse std::sync::Arc;\n\n/// Counting semaphore performing asynchronous permit acquisition.\n///\n/// A semaphore maintains a set of permits. Permits are used to synchronize\n/// access to a shared resource. A semaphore differs from a mutex in that it\n/// can allow more than one concurrent caller to access the shared resource at a\n/// time.\n///\n/// When `acquire` is called and the semaphore has remaining permits, the\n/// function immediately returns a permit. However, if no remaining permits are\n/// available, `acquire` (asynchronously) waits until an outstanding permit is\n/// dropped. At this point, the freed permit is assigned to the caller.\n///\n/// This `Semaphore` is fair, which means that permits are given out in the order\n/// they were requested. This fairness is also applied when `acquire_many` gets\n/// involved, so if a call to `acquire_many` at the front of the queue requests\n/// more permits than currently available, this can prevent a call to `acquire`\n/// from completing, even if the semaphore has enough permits complete the call\n/// to `acquire`.\n///\n/// To use the `Semaphore` in a poll function, you can use the [`PollSemaphore`]\n/// utility.\n///\n/// # Examples\n///\n/// Basic usage:\n///\n/// ```\n/// use tokio::sync::{Semaphore, TryAcquireError};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let semaphore = Semaphore::new(3);\n///\n/// let a_permit = semaphore.acquire().await.unwrap();\n/// let two_permits = semaphore.acquire_many(2).await.unwrap();\n///\n/// assert_eq!(semaphore.available_permits(), 0);\n///\n/// let permit_attempt = semaphore.try_acquire();\n/// assert_eq!(permit_attempt.err(), Some(TryAcquireError::NoPermits));\n/// # }\n/// ```\n///\n/// ## Limit the number of simultaneously opened files in your program\n///\n/// Most operating systems have limits on the number of open file\n/// handles. Even in systems without explicit limits, resource constraints\n/// implicitly set an upper bound on the number of open files. If your\n/// program attempts to open a large number of files and exceeds this\n/// limit, it will result in an error.\n///\n/// This example uses a Semaphore with 100 permits. By acquiring a permit from\n/// the Semaphore before accessing a file, you ensure that your program opens\n/// no more than 100 files at a time. When trying to open the 101st\n/// file, the program will wait until a permit becomes available before\n/// proceeding to open another file.\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use std::io::Result;\n/// use tokio::fs::File;\n/// use tokio::sync::Semaphore;\n/// use tokio::io::AsyncWriteExt;\n///\n/// static PERMITS: Semaphore = Semaphore::const_new(100);\n///\n/// async fn write_to_file(message: &[u8]) -> Result<()> {\n/// let _permit = PERMITS.acquire().await.unwrap();\n/// let mut buffer = File::create(\"example.txt\").await?;\n/// buffer.write_all(message).await?;\n/// Ok(()) // Permit goes out of scope here, and is available again for acquisition\n/// }\n/// # }\n/// ```\n///\n/// ## Limit the number of outgoing requests being sent at the same time\n///\n/// In some scenarios, it might be required to limit the number of outgoing\n/// requests being sent in parallel. This could be due to limits of a consumed\n/// API or the network resources of the system the application is running on.\n///\n/// This example uses an `Arc` with 10 permits. Each task spawned is\n/// given a reference to the semaphore by cloning the `Arc`. Before\n/// a task sends a request, it must acquire a permit from the semaphore by\n/// calling [`Semaphore::acquire`]. This ensures that at most 10 requests are\n/// sent in parallel at any given time. After a task has sent a request, it\n/// drops the permit to allow other tasks to send requests.\n///\n/// ```\n/// use std::sync::Arc;\n/// use tokio::sync::Semaphore;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// // Define maximum number of parallel requests.\n/// let semaphore = Arc::new(Semaphore::new(5));\n/// // Spawn many tasks that will send requests.\n/// let mut jhs = Vec::new();\n/// for task_id in 0..50 {\n/// let semaphore = semaphore.clone();\n/// let jh = tokio::spawn(async move {\n/// // Acquire permit before sending request.\n/// let _permit = semaphore.acquire().await.unwrap();\n/// // Send the request.\n/// let response = send_request(task_id).await;\n/// // Drop the permit after the request has been sent.\n/// drop(_permit);\n/// // Handle response.\n/// // ...\n///\n/// response\n/// });\n/// jhs.push(jh);\n/// }\n/// // Collect responses from tasks.\n/// let mut responses = Vec::new();\n/// for jh in jhs {\n/// let response = jh.await.unwrap();\n/// responses.push(response);\n/// }\n/// // Process responses.\n/// // ...\n/// # }\n/// # async fn send_request(task_id: usize) {\n/// # // Send request.\n/// # }\n/// ```\n///\n/// ## Limit the number of incoming requests being handled at the same time\n///\n/// Similar to limiting the number of simultaneously opened files, network handles\n/// are a limited resource. Allowing an unbounded amount of requests to be processed\n/// could result in a denial-of-service, among many other issues.\n///\n/// This example uses an `Arc` instead of a global variable.\n/// To limit the number of requests that can be processed at the time,\n/// we acquire a permit for each task before spawning it. Once acquired,\n/// a new task is spawned; and once finished, the permit is dropped inside\n/// of the task to allow others to spawn. Permits must be acquired via\n/// [`Semaphore::acquire_owned`] to be movable across the task boundary.\n/// (Since our semaphore is not a global variable — if it was, then `acquire` would be enough.)\n///\n/// ```no_run\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use std::sync::Arc;\n/// use tokio::sync::Semaphore;\n/// use tokio::net::TcpListener;\n///\n/// #[tokio::main]\n/// async fn main() -> std::io::Result<()> {\n/// let semaphore = Arc::new(Semaphore::new(3));\n/// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n///\n/// loop {\n/// // Acquire permit before accepting the next socket.\n/// //\n/// // We use `acquire_owned` so that we can move `permit` into\n/// // other tasks.\n/// let permit = semaphore.clone().acquire_owned().await.unwrap();\n/// let (mut socket, _) = listener.accept().await?;\n///\n/// tokio::spawn(async move {\n/// // Do work using the socket.\n/// handle_connection(&mut socket).await;\n/// // Drop socket while the permit is still live.\n/// drop(socket);\n/// // Drop the permit, so more tasks can be created.\n/// drop(permit);\n/// });\n/// }\n/// }\n/// # async fn handle_connection(_socket: &mut tokio::net::TcpStream) {\n/// # // Do work\n/// # }\n/// # }\n/// ```\n///\n/// ## Prevent tests from running in parallel\n///\n/// By default, Rust runs tests in the same file in parallel. However, in some\n/// cases, running two tests in parallel may lead to problems. For example, this\n/// can happen when tests use the same database.\n///\n/// Consider the following scenario:\n/// 1. `test_insert`: Inserts a key-value pair into the database, then retrieves\n/// the value using the same key to verify the insertion.\n/// 2. `test_update`: Inserts a key, then updates the key to a new value and\n/// verifies that the value has been accurately updated.\n/// 3. `test_others`: A third test that doesn't modify the database state. It\n/// can run in parallel with the other tests.\n///\n/// In this example, `test_insert` and `test_update` need to run in sequence to\n/// work, but it doesn't matter which test runs first. We can leverage a\n/// semaphore with a single permit to address this challenge.\n///\n/// ```\n/// # use tokio::sync::Mutex;\n/// # use std::collections::BTreeMap;\n/// # struct Database {\n/// # map: Mutex>,\n/// # }\n/// # impl Database {\n/// # pub const fn setup() -> Database {\n/// # Database {\n/// # map: Mutex::const_new(BTreeMap::new()),\n/// # }\n/// # }\n/// # pub async fn insert(&self, key: &str, value: i32) {\n/// # self.map.lock().await.insert(key.to_string(), value);\n/// # }\n/// # pub async fn update(&self, key: &str, value: i32) {\n/// # self.map.lock().await\n/// # .entry(key.to_string())\n/// # .and_modify(|origin| *origin = value);\n/// # }\n/// # pub async fn delete(&self, key: &str) {\n/// # self.map.lock().await.remove(key);\n/// # }\n/// # pub async fn get(&self, key: &str) -> i32 {\n/// # *self.map.lock().await.get(key).unwrap()\n/// # }\n/// # }\n/// use tokio::sync::Semaphore;\n///\n/// // Initialize a static semaphore with only one permit, which is used to\n/// // prevent test_insert and test_update from running in parallel.\n/// static PERMIT: Semaphore = Semaphore::const_new(1);\n///\n/// // Initialize the database that will be used by the subsequent tests.\n/// static DB: Database = Database::setup();\n///\n/// #[tokio::test]\n/// # async fn fake_test_insert() {}\n/// async fn test_insert() {\n/// // Acquire permit before proceeding. Since the semaphore has only one permit,\n/// // the test will wait if the permit is already acquired by other tests.\n/// let permit = PERMIT.acquire().await.unwrap();\n///\n/// // Do the actual test stuff with database\n///\n/// // Insert a key-value pair to database\n/// let (key, value) = (\"name\", 0);\n/// DB.insert(key, value).await;\n///\n/// // Verify that the value has been inserted correctly.\n/// assert_eq!(DB.get(key).await, value);\n///\n/// // Undo the insertion, so the database is empty at the end of the test.\n/// DB.delete(key).await;\n///\n/// // Drop permit. This allows the other test to start running.\n/// drop(permit);\n/// }\n///\n/// #[tokio::test]\n/// # async fn fake_test_update() {}\n/// async fn test_update() {\n/// // Acquire permit before proceeding. Since the semaphore has only one permit,\n/// // the test will wait if the permit is already acquired by other tests.\n/// let permit = PERMIT.acquire().await.unwrap();\n///\n/// // Do the same insert.\n/// let (key, value) = (\"name\", 0);\n/// DB.insert(key, value).await;\n///\n/// // Update the existing value with a new one.\n/// let new_value = 1;\n/// DB.update(key, new_value).await;\n///\n/// // Verify that the value has been updated correctly.\n/// assert_eq!(DB.get(key).await, new_value);\n///\n/// // Undo any modificattion.\n/// DB.delete(key).await;\n///\n/// // Drop permit. This allows the other test to start running.\n/// drop(permit);\n/// }\n///\n/// #[tokio::test]\n/// # async fn fake_test_others() {}\n/// async fn test_others() {\n/// // This test can run in parallel with test_insert and test_update,\n/// // so it does not use PERMIT.\n/// }\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// # test_insert().await;\n/// # test_update().await;\n/// # test_others().await;\n/// # }\n/// ```\n///\n/// ## Rate limiting using a token bucket\n///\n/// This example showcases the [`add_permits`] and [`SemaphorePermit::forget`] methods.\n///\n/// Many applications and systems have constraints on the rate at which certain\n/// operations should occur. Exceeding this rate can result in suboptimal\n/// performance or even errors.\n///\n/// This example implements rate limiting using a [token bucket]. A token bucket is a form of rate\n/// limiting that doesn't kick in immediately, to allow for short bursts of incoming requests that\n/// arrive at the same time.\n///\n/// With a token bucket, each incoming request consumes a token, and the tokens are refilled at a\n/// certain rate that defines the rate limit. When a burst of requests arrives, tokens are\n/// immediately given out until the bucket is empty. Once the bucket is empty, requests will have to\n/// wait for new tokens to be added.\n///\n/// Unlike the example that limits how many requests can be handled at the same time, we do not add\n/// tokens back when we finish handling a request. Instead, tokens are added only by a timer task.\n///\n/// Note that this implementation is suboptimal when the duration is small, because it consumes a\n/// lot of cpu constantly looping and sleeping.\n///\n/// [token bucket]: https://en.wikipedia.org/wiki/Token_bucket\n/// [`add_permits`]: crate::sync::Semaphore::add_permits\n/// [`SemaphorePermit::forget`]: crate::sync::SemaphorePermit::forget\n/// ```\n/// use std::sync::Arc;\n/// use tokio::sync::Semaphore;\n/// use tokio::time::{interval, Duration};\n///\n/// struct TokenBucket {\n/// sem: Arc,\n/// jh: tokio::task::JoinHandle<()>,\n/// }\n///\n/// impl TokenBucket {\n/// fn new(duration: Duration, capacity: usize) -> Self {\n/// let sem = Arc::new(Semaphore::new(capacity));\n///\n/// // refills the tokens at the end of each interval\n/// let jh = tokio::spawn({\n/// let sem = sem.clone();\n/// let mut interval = interval(duration);\n/// interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);\n///\n/// async move {\n/// loop {\n/// interval.tick().await;\n///\n/// if sem.available_permits() < capacity {\n/// sem.add_permits(1);\n/// }\n/// }\n/// }\n/// });\n///\n/// Self { jh, sem }\n/// }\n///\n/// async fn acquire(&self) {\n/// // This can return an error if the semaphore is closed, but we\n/// // never close it, so this error can never happen.\n/// let permit = self.sem.acquire().await.unwrap();\n/// // To avoid releasing the permit back to the semaphore, we use\n/// // the `SemaphorePermit::forget` method.\n/// permit.forget();\n/// }\n/// }\n///\n/// impl Drop for TokenBucket {\n/// fn drop(&mut self) {\n/// // Kill the background task so it stops taking up resources when we\n/// // don't need it anymore.\n/// self.jh.abort();\n/// }\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn _hidden() {}\n/// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n/// # async fn main() {\n/// let capacity = 5;\n/// let update_interval = Duration::from_secs_f32(1.0 / capacity as f32);\n/// let bucket = TokenBucket::new(update_interval, capacity);\n///\n/// for _ in 0..5 {\n/// bucket.acquire().await;\n///\n/// // do the operation\n/// }\n/// # }\n/// ```\n///\n/// [`PollSemaphore`]: https://docs.rs/tokio-util/latest/tokio_util/sync/struct.PollSemaphore.html\n/// [`Semaphore::acquire_owned`]: crate::sync::Semaphore::acquire_owned\n#[derive(Debug)]\npub struct Semaphore {\n /// The low level semaphore\n ll_sem: ll::Semaphore,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n}\n\n/// A permit from the semaphore.\n///\n/// This type is created by the [`acquire`] method.\n///\n/// [`acquire`]: crate::sync::Semaphore::acquire()\n#[must_use]\n#[clippy::has_significant_drop]\n#[derive(Debug)]\npub struct SemaphorePermit<'a> {\n sem: &'a Semaphore,\n permits: u32,\n}\n\n/// An owned permit from the semaphore.\n///\n/// This type is created by the [`acquire_owned`] method.\n///\n/// [`acquire_owned`]: crate::sync::Semaphore::acquire_owned()\n#[must_use]\n#[clippy::has_significant_drop]\n#[derive(Debug)]\npub struct OwnedSemaphorePermit {\n sem: Arc,\n permits: u32,\n}\n\n#[test]\n#[cfg(not(loom))]\nfn bounds() {\n fn check_unpin() {}\n // This has to take a value, since the async fn's return type is unnameable.\n fn check_send_sync_val(_t: T) {}\n fn check_send_sync() {}\n check_unpin::();\n check_unpin::>();\n check_send_sync::();\n\n let semaphore = Semaphore::new(0);\n check_send_sync_val(semaphore.acquire());\n}\n\nimpl Semaphore {\n /// The maximum number of permits which a semaphore can hold. It is `usize::MAX >> 3`.\n ///\n /// Exceeding this limit typically results in a panic.\n pub const MAX_PERMITS: usize = super::batch_semaphore::Semaphore::MAX_PERMITS;\n\n /// Creates a new semaphore with the initial number of permits.\n ///\n /// Panics if `permits` exceeds [`Semaphore::MAX_PERMITS`].\n #[track_caller]\n pub fn new(permits: usize) -> Self {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = std::panic::Location::caller();\n\n tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Semaphore\",\n kind = \"Sync\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n inherits_child_attrs = true,\n )\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let ll_sem = resource_span.in_scope(|| ll::Semaphore::new(permits));\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let ll_sem = ll::Semaphore::new(permits);\n\n Self {\n ll_sem,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n }\n\n /// Creates a new semaphore with the initial number of permits.\n ///\n /// When using the `tracing` [unstable feature], a `Semaphore` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`Semaphore::new`] should be used to\n /// create an instrumented object if that is needed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Semaphore;\n ///\n /// static SEM: Semaphore = Semaphore::const_new(10);\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new(permits: usize) -> Self {\n Self {\n ll_sem: ll::Semaphore::const_new(permits),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Creates a new closed semaphore with 0 permits.\n pub(crate) fn new_closed() -> Self {\n Self {\n ll_sem: ll::Semaphore::new_closed(),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Creates a new closed semaphore with 0 permits.\n #[cfg(not(all(loom, test)))]\n pub(crate) const fn const_new_closed() -> Self {\n Self {\n ll_sem: ll::Semaphore::const_new_closed(),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span::none(),\n }\n }\n\n /// Returns the current number of available permits.\n pub fn available_permits(&self) -> usize {\n self.ll_sem.available_permits()\n }\n\n /// Adds `n` new permits to the semaphore.\n ///\n /// The maximum number of permits is [`Semaphore::MAX_PERMITS`], and this function will panic if the limit is exceeded.\n pub fn add_permits(&self, n: usize) {\n self.ll_sem.release(n);\n }\n\n /// Decrease a semaphore's permits by a maximum of `n`.\n ///\n /// If there are insufficient permits and it's not possible to reduce by `n`,\n /// return the number of permits that were actually reduced.\n pub fn forget_permits(&self, n: usize) -> usize {\n self.ll_sem.forget_permits(n)\n }\n\n /// Acquires a permit from the semaphore.\n ///\n /// If the semaphore has been closed, this returns an [`AcquireError`].\n /// Otherwise, this returns a [`SemaphorePermit`] representing the\n /// acquired permit.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute permits in the order they\n /// were requested. Cancelling a call to `acquire` makes you lose your place\n /// in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Semaphore;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let semaphore = Semaphore::new(2);\n ///\n /// let permit_1 = semaphore.acquire().await.unwrap();\n /// assert_eq!(semaphore.available_permits(), 1);\n ///\n /// let permit_2 = semaphore.acquire().await.unwrap();\n /// assert_eq!(semaphore.available_permits(), 0);\n ///\n /// drop(permit_1);\n /// assert_eq!(semaphore.available_permits(), 1);\n /// # }\n /// ```\n ///\n /// [`AcquireError`]: crate::sync::AcquireError\n /// [`SemaphorePermit`]: crate::sync::SemaphorePermit\n pub async fn acquire(&self) -> Result, AcquireError> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let inner = trace::async_op(\n || self.ll_sem.acquire(1),\n self.resource_span.clone(),\n \"Semaphore::acquire\",\n \"poll\",\n true,\n );\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let inner = self.ll_sem.acquire(1);\n\n inner.await?;\n Ok(SemaphorePermit {\n sem: self,\n permits: 1,\n })\n }\n\n /// Acquires `n` permits from the semaphore.\n ///\n /// If the semaphore has been closed, this returns an [`AcquireError`].\n /// Otherwise, this returns a [`SemaphorePermit`] representing the\n /// acquired permits.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute permits in the order they\n /// were requested. Cancelling a call to `acquire_many` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Semaphore;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let semaphore = Semaphore::new(5);\n ///\n /// let permit = semaphore.acquire_many(3).await.unwrap();\n /// assert_eq!(semaphore.available_permits(), 2);\n /// # }\n /// ```\n ///\n /// [`AcquireError`]: crate::sync::AcquireError\n /// [`SemaphorePermit`]: crate::sync::SemaphorePermit\n pub async fn acquire_many(&self, n: u32) -> Result, AcquireError> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n trace::async_op(\n || self.ll_sem.acquire(n as usize),\n self.resource_span.clone(),\n \"Semaphore::acquire_many\",\n \"poll\",\n true,\n )\n .await?;\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n self.ll_sem.acquire(n as usize).await?;\n\n Ok(SemaphorePermit {\n sem: self,\n permits: n,\n })\n }\n\n /// Tries to acquire a permit from the semaphore.\n ///\n /// If the semaphore has been closed, this returns a [`TryAcquireError::Closed`]\n /// and a [`TryAcquireError::NoPermits`] if there are no permits left. Otherwise,\n /// this returns a [`SemaphorePermit`] representing the acquired permits.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Semaphore, TryAcquireError};\n ///\n /// # fn main() {\n /// let semaphore = Semaphore::new(2);\n ///\n /// let permit_1 = semaphore.try_acquire().unwrap();\n /// assert_eq!(semaphore.available_permits(), 1);\n ///\n /// let permit_2 = semaphore.try_acquire().unwrap();\n /// assert_eq!(semaphore.available_permits(), 0);\n ///\n /// let permit_3 = semaphore.try_acquire();\n /// assert_eq!(permit_3.err(), Some(TryAcquireError::NoPermits));\n /// # }\n /// ```\n ///\n /// [`TryAcquireError::Closed`]: crate::sync::TryAcquireError::Closed\n /// [`TryAcquireError::NoPermits`]: crate::sync::TryAcquireError::NoPermits\n /// [`SemaphorePermit`]: crate::sync::SemaphorePermit\n pub fn try_acquire(&self) -> Result, TryAcquireError> {\n match self.ll_sem.try_acquire(1) {\n Ok(()) => Ok(SemaphorePermit {\n sem: self,\n permits: 1,\n }),\n Err(e) => Err(e),\n }\n }\n\n /// Tries to acquire `n` permits from the semaphore.\n ///\n /// If the semaphore has been closed, this returns a [`TryAcquireError::Closed`]\n /// and a [`TryAcquireError::NoPermits`] if there are not enough permits left.\n /// Otherwise, this returns a [`SemaphorePermit`] representing the acquired permits.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::{Semaphore, TryAcquireError};\n ///\n /// # fn main() {\n /// let semaphore = Semaphore::new(4);\n ///\n /// let permit_1 = semaphore.try_acquire_many(3).unwrap();\n /// assert_eq!(semaphore.available_permits(), 1);\n ///\n /// let permit_2 = semaphore.try_acquire_many(2);\n /// assert_eq!(permit_2.err(), Some(TryAcquireError::NoPermits));\n /// # }\n /// ```\n ///\n /// [`TryAcquireError::Closed`]: crate::sync::TryAcquireError::Closed\n /// [`TryAcquireError::NoPermits`]: crate::sync::TryAcquireError::NoPermits\n /// [`SemaphorePermit`]: crate::sync::SemaphorePermit\n pub fn try_acquire_many(&self, n: u32) -> Result, TryAcquireError> {\n match self.ll_sem.try_acquire(n as usize) {\n Ok(()) => Ok(SemaphorePermit {\n sem: self,\n permits: n,\n }),\n Err(e) => Err(e),\n }\n }\n\n /// Acquires a permit from the semaphore.\n ///\n /// The semaphore must be wrapped in an [`Arc`] to call this method.\n /// If the semaphore has been closed, this returns an [`AcquireError`].\n /// Otherwise, this returns a [`OwnedSemaphorePermit`] representing the\n /// acquired permit.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute permits in the order they\n /// were requested. Cancelling a call to `acquire_owned` makes you lose your\n /// place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let semaphore = Arc::new(Semaphore::new(3));\n /// let mut join_handles = Vec::new();\n ///\n /// for _ in 0..5 {\n /// let permit = semaphore.clone().acquire_owned().await.unwrap();\n /// join_handles.push(tokio::spawn(async move {\n /// // perform task...\n /// // explicitly own `permit` in the task\n /// drop(permit);\n /// }));\n /// }\n ///\n /// for handle in join_handles {\n /// handle.await.unwrap();\n /// }\n /// # }\n /// ```\n ///\n /// [`Arc`]: std::sync::Arc\n /// [`AcquireError`]: crate::sync::AcquireError\n /// [`OwnedSemaphorePermit`]: crate::sync::OwnedSemaphorePermit\n pub async fn acquire_owned(self: Arc) -> Result {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let inner = trace::async_op(\n || self.ll_sem.acquire(1),\n self.resource_span.clone(),\n \"Semaphore::acquire_owned\",\n \"poll\",\n true,\n );\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let inner = self.ll_sem.acquire(1);\n\n inner.await?;\n Ok(OwnedSemaphorePermit {\n sem: self,\n permits: 1,\n })\n }\n\n /// Acquires `n` permits from the semaphore.\n ///\n /// The semaphore must be wrapped in an [`Arc`] to call this method.\n /// If the semaphore has been closed, this returns an [`AcquireError`].\n /// Otherwise, this returns a [`OwnedSemaphorePermit`] representing the\n /// acquired permit.\n ///\n /// # Cancel safety\n ///\n /// This method uses a queue to fairly distribute permits in the order they\n /// were requested. Cancelling a call to `acquire_many_owned` makes you lose\n /// your place in the queue.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let semaphore = Arc::new(Semaphore::new(10));\n /// let mut join_handles = Vec::new();\n ///\n /// for _ in 0..5 {\n /// let permit = semaphore.clone().acquire_many_owned(2).await.unwrap();\n /// join_handles.push(tokio::spawn(async move {\n /// // perform task...\n /// // explicitly own `permit` in the task\n /// drop(permit);\n /// }));\n /// }\n ///\n /// for handle in join_handles {\n /// handle.await.unwrap();\n /// }\n /// # }\n /// ```\n ///\n /// [`Arc`]: std::sync::Arc\n /// [`AcquireError`]: crate::sync::AcquireError\n /// [`OwnedSemaphorePermit`]: crate::sync::OwnedSemaphorePermit\n pub async fn acquire_many_owned(\n self: Arc,\n n: u32,\n ) -> Result {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let inner = trace::async_op(\n || self.ll_sem.acquire(n as usize),\n self.resource_span.clone(),\n \"Semaphore::acquire_many_owned\",\n \"poll\",\n true,\n );\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let inner = self.ll_sem.acquire(n as usize);\n\n inner.await?;\n Ok(OwnedSemaphorePermit {\n sem: self,\n permits: n,\n })\n }\n\n /// Tries to acquire a permit from the semaphore.\n ///\n /// The semaphore must be wrapped in an [`Arc`] to call this method. If\n /// the semaphore has been closed, this returns a [`TryAcquireError::Closed`]\n /// and a [`TryAcquireError::NoPermits`] if there are no permits left.\n /// Otherwise, this returns a [`OwnedSemaphorePermit`] representing the\n /// acquired permit.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{Semaphore, TryAcquireError};\n ///\n /// # fn main() {\n /// let semaphore = Arc::new(Semaphore::new(2));\n ///\n /// let permit_1 = Arc::clone(&semaphore).try_acquire_owned().unwrap();\n /// assert_eq!(semaphore.available_permits(), 1);\n ///\n /// let permit_2 = Arc::clone(&semaphore).try_acquire_owned().unwrap();\n /// assert_eq!(semaphore.available_permits(), 0);\n ///\n /// let permit_3 = semaphore.try_acquire_owned();\n /// assert_eq!(permit_3.err(), Some(TryAcquireError::NoPermits));\n /// # }\n /// ```\n ///\n /// [`Arc`]: std::sync::Arc\n /// [`TryAcquireError::Closed`]: crate::sync::TryAcquireError::Closed\n /// [`TryAcquireError::NoPermits`]: crate::sync::TryAcquireError::NoPermits\n /// [`OwnedSemaphorePermit`]: crate::sync::OwnedSemaphorePermit\n pub fn try_acquire_owned(self: Arc) -> Result {\n match self.ll_sem.try_acquire(1) {\n Ok(()) => Ok(OwnedSemaphorePermit {\n sem: self,\n permits: 1,\n }),\n Err(e) => Err(e),\n }\n }\n\n /// Tries to acquire `n` permits from the semaphore.\n ///\n /// The semaphore must be wrapped in an [`Arc`] to call this method. If\n /// the semaphore has been closed, this returns a [`TryAcquireError::Closed`]\n /// and a [`TryAcquireError::NoPermits`] if there are no permits left.\n /// Otherwise, this returns a [`OwnedSemaphorePermit`] representing the\n /// acquired permit.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::{Semaphore, TryAcquireError};\n ///\n /// # fn main() {\n /// let semaphore = Arc::new(Semaphore::new(4));\n ///\n /// let permit_1 = Arc::clone(&semaphore).try_acquire_many_owned(3).unwrap();\n /// assert_eq!(semaphore.available_permits(), 1);\n ///\n /// let permit_2 = semaphore.try_acquire_many_owned(2);\n /// assert_eq!(permit_2.err(), Some(TryAcquireError::NoPermits));\n /// # }\n /// ```\n ///\n /// [`Arc`]: std::sync::Arc\n /// [`TryAcquireError::Closed`]: crate::sync::TryAcquireError::Closed\n /// [`TryAcquireError::NoPermits`]: crate::sync::TryAcquireError::NoPermits\n /// [`OwnedSemaphorePermit`]: crate::sync::OwnedSemaphorePermit\n pub fn try_acquire_many_owned(\n self: Arc,\n n: u32,\n ) -> Result {\n match self.ll_sem.try_acquire(n as usize) {\n Ok(()) => Ok(OwnedSemaphorePermit {\n sem: self,\n permits: n,\n }),\n Err(e) => Err(e),\n }\n }\n\n /// Closes the semaphore.\n ///\n /// This prevents the semaphore from issuing new permits and notifies all pending waiters.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::Semaphore;\n /// use std::sync::Arc;\n /// use tokio::sync::TryAcquireError;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let semaphore = Arc::new(Semaphore::new(1));\n /// let semaphore2 = semaphore.clone();\n ///\n /// tokio::spawn(async move {\n /// let permit = semaphore.acquire_many(2).await;\n /// assert!(permit.is_err());\n /// println!(\"waiter received error\");\n /// });\n ///\n /// println!(\"closing semaphore\");\n /// semaphore2.close();\n ///\n /// // Cannot obtain more permits\n /// assert_eq!(semaphore2.try_acquire().err(), Some(TryAcquireError::Closed))\n /// # }\n /// ```\n pub fn close(&self) {\n self.ll_sem.close();\n }\n\n /// Returns true if the semaphore is closed\n pub fn is_closed(&self) -> bool {\n self.ll_sem.is_closed()\n }\n}\n\nimpl<'a> SemaphorePermit<'a> {\n /// Forgets the permit **without** releasing it back to the semaphore.\n /// This can be used to reduce the amount of permits available from a\n /// semaphore.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(10));\n /// {\n /// let permit = sem.try_acquire_many(5).unwrap();\n /// assert_eq!(sem.available_permits(), 5);\n /// permit.forget();\n /// }\n ///\n /// // Since we forgot the permit, available permits won't go back to its initial value\n /// // even after the permit is dropped.\n /// assert_eq!(sem.available_permits(), 5);\n /// ```\n pub fn forget(mut self) {\n self.permits = 0;\n }\n\n /// Merge two [`SemaphorePermit`] instances together, consuming `other`\n /// without releasing the permits it holds.\n ///\n /// Permits held by both `self` and `other` are released when `self` drops.\n ///\n /// # Panics\n ///\n /// This function panics if permits from different [`Semaphore`] instances\n /// are merged.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(10));\n /// let mut permit = sem.try_acquire().unwrap();\n ///\n /// for _ in 0..9 {\n /// let _permit = sem.try_acquire().unwrap();\n /// // Merge individual permits into a single one.\n /// permit.merge(_permit)\n /// }\n ///\n /// assert_eq!(sem.available_permits(), 0);\n ///\n /// // Release all permits in a single batch.\n /// drop(permit);\n ///\n /// assert_eq!(sem.available_permits(), 10);\n /// ```\n #[track_caller]\n pub fn merge(&mut self, mut other: Self) {\n assert!(\n std::ptr::eq(self.sem, other.sem),\n \"merging permits from different semaphore instances\"\n );\n self.permits += other.permits;\n other.permits = 0;\n }\n\n /// Splits `n` permits from `self` and returns a new [`SemaphorePermit`] instance that holds `n` permits.\n ///\n /// If there are insufficient permits and it's not possible to reduce by `n`, returns `None`.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(3));\n ///\n /// let mut p1 = sem.try_acquire_many(3).unwrap();\n /// let p2 = p1.split(1).unwrap();\n ///\n /// assert_eq!(p1.num_permits(), 2);\n /// assert_eq!(p2.num_permits(), 1);\n /// ```\n pub fn split(&mut self, n: usize) -> Option {\n let n = u32::try_from(n).ok()?;\n\n if n > self.permits {\n return None;\n }\n\n self.permits -= n;\n\n Some(Self {\n sem: self.sem,\n permits: n,\n })\n }\n\n /// Returns the number of permits held by `self`.\n pub fn num_permits(&self) -> usize {\n self.permits as usize\n }\n}\n\nimpl OwnedSemaphorePermit {\n /// Forgets the permit **without** releasing it back to the semaphore.\n /// This can be used to reduce the amount of permits available from a\n /// semaphore.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(10));\n /// {\n /// let permit = sem.clone().try_acquire_many_owned(5).unwrap();\n /// assert_eq!(sem.available_permits(), 5);\n /// permit.forget();\n /// }\n ///\n /// // Since we forgot the permit, available permits won't go back to its initial value\n /// // even after the permit is dropped.\n /// assert_eq!(sem.available_permits(), 5);\n /// ```\n pub fn forget(mut self) {\n self.permits = 0;\n }\n\n /// Merge two [`OwnedSemaphorePermit`] instances together, consuming `other`\n /// without releasing the permits it holds.\n ///\n /// Permits held by both `self` and `other` are released when `self` drops.\n ///\n /// # Panics\n ///\n /// This function panics if permits from different [`Semaphore`] instances\n /// are merged.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(10));\n /// let mut permit = sem.clone().try_acquire_owned().unwrap();\n ///\n /// for _ in 0..9 {\n /// let _permit = sem.clone().try_acquire_owned().unwrap();\n /// // Merge individual permits into a single one.\n /// permit.merge(_permit)\n /// }\n ///\n /// assert_eq!(sem.available_permits(), 0);\n ///\n /// // Release all permits in a single batch.\n /// drop(permit);\n ///\n /// assert_eq!(sem.available_permits(), 10);\n /// ```\n #[track_caller]\n pub fn merge(&mut self, mut other: Self) {\n assert!(\n Arc::ptr_eq(&self.sem, &other.sem),\n \"merging permits from different semaphore instances\"\n );\n self.permits += other.permits;\n other.permits = 0;\n }\n\n /// Splits `n` permits from `self` and returns a new [`OwnedSemaphorePermit`] instance that holds `n` permits.\n ///\n /// If there are insufficient permits and it's not possible to reduce by `n`, returns `None`.\n ///\n /// # Note\n ///\n /// It will clone the owned `Arc` to construct the new instance.\n ///\n /// # Examples\n ///\n /// ```\n /// use std::sync::Arc;\n /// use tokio::sync::Semaphore;\n ///\n /// let sem = Arc::new(Semaphore::new(3));\n ///\n /// let mut p1 = sem.try_acquire_many_owned(3).unwrap();\n /// let p2 = p1.split(1).unwrap();\n ///\n /// assert_eq!(p1.num_permits(), 2);\n /// assert_eq!(p2.num_permits(), 1);\n /// ```\n pub fn split(&mut self, n: usize) -> Option {\n let n = u32::try_from(n).ok()?;\n\n if n > self.permits {\n return None;\n }\n\n self.permits -= n;\n\n Some(Self {\n sem: self.sem.clone(),\n permits: n,\n })\n }\n\n /// Returns the [`Semaphore`] from which this permit was acquired.\n pub fn semaphore(&self) -> &Arc {\n &self.sem\n }\n\n /// Returns the number of permits held by `self`.\n pub fn num_permits(&self) -> usize {\n self.permits as usize\n }\n}\n\nimpl Drop for SemaphorePermit<'_> {\n fn drop(&mut self) {\n self.sem.add_permits(self.permits as usize);\n }\n}\n\nimpl Drop for OwnedSemaphorePermit {\n fn drop(&mut self) {\n self.sem.add_permits(self.permits as usize);\n }\n}\n" }, { "path": "tokio/src/sync/set_once.rs", "content": "use super::Notify;\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::atomic::AtomicBool;\n\nuse std::error::Error;\nuse std::fmt;\nuse std::future::{poll_fn, Future};\nuse std::mem::MaybeUninit;\nuse std::ops::Drop;\nuse std::ptr;\nuse std::sync::atomic::Ordering;\nuse std::task::Poll;\n\n// This file contains an implementation of an SetOnce. The value of SetOnce\n// can only be modified once during initialization.\n//\n// 1. When `value_set` is false, the `value` is not initialized and wait()\n// future will keep on waiting.\n// 2. When `value_set` is true, the wait() future completes, get() will return\n// Some(&T)\n//\n// The value cannot be changed after set() is called. Subsequent calls to set()\n// will return a `SetOnceError`.\n\n/// A thread-safe cell that can be written to only once.\n///\n/// A `SetOnce` is inspired from python's [`asyncio.Event`] type. It can be\n/// used to wait until the value of the `SetOnce` is set like a \"Event\" mechanism.\n///\n/// # Example\n///\n/// ```\n/// use tokio::sync::{SetOnce, SetOnceError};\n///\n/// static ONCE: SetOnce = SetOnce::const_new();\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), SetOnceError> {\n///\n/// // set the value inside a task somewhere...\n/// tokio::spawn(async move { ONCE.set(20) });\n///\n/// // checking with .get doesn't block main thread\n/// println!(\"{:?}\", ONCE.get());\n///\n/// // wait until the value is set, blocks the thread\n/// println!(\"{:?}\", ONCE.wait().await);\n///\n/// Ok(())\n/// # }\n/// ```\n///\n/// A `SetOnce` is typically used for global variables that need to be\n/// initialized once on first use, but need no further changes. The `SetOnce`\n/// in Tokio allows the initialization procedure to be asynchronous.\n///\n/// # Example\n///\n/// ```\n/// use tokio::sync::{SetOnce, SetOnceError};\n/// use std::sync::Arc;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), SetOnceError> {\n/// let once = SetOnce::new();\n///\n/// let arc = Arc::new(once);\n/// let first_cl = Arc::clone(&arc);\n/// let second_cl = Arc::clone(&arc);\n///\n/// // set the value inside a task\n/// tokio::spawn(async move { first_cl.set(20) }).await.unwrap()?;\n///\n/// // wait inside task to not block the main thread\n/// tokio::spawn(async move {\n/// // wait inside async context for the value to be set\n/// assert_eq!(*second_cl.wait().await, 20);\n/// }).await.unwrap();\n///\n/// // subsequent set calls will fail\n/// assert!(arc.set(30).is_err());\n///\n/// println!(\"{:?}\", arc.get());\n///\n/// Ok(())\n/// # }\n/// ```\n///\n/// [`asyncio.Event`]: https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event\npub struct SetOnce {\n value_set: AtomicBool,\n value: UnsafeCell>,\n notify: Notify,\n}\n\nimpl Default for SetOnce {\n fn default() -> SetOnce {\n SetOnce::new()\n }\n}\n\nimpl fmt::Debug for SetOnce {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"SetOnce\")\n .field(\"value\", &self.get())\n .finish()\n }\n}\n\nimpl Clone for SetOnce {\n fn clone(&self) -> SetOnce {\n SetOnce::new_with(self.get().cloned())\n }\n}\n\nimpl PartialEq for SetOnce {\n fn eq(&self, other: &SetOnce) -> bool {\n self.get() == other.get()\n }\n}\n\nimpl Eq for SetOnce {}\n\nimpl Drop for SetOnce {\n fn drop(&mut self) {\n // TODO: Use get_mut()\n if self.value_set.load(Ordering::Relaxed) {\n // SAFETY: If the value_set is true, then the value is initialized\n // then there is a value to be dropped and this is safe\n unsafe { self.value.with_mut(|ptr| ptr::drop_in_place(ptr as *mut T)) }\n }\n }\n}\n\nimpl From for SetOnce {\n fn from(value: T) -> Self {\n SetOnce {\n value_set: AtomicBool::new(true),\n value: UnsafeCell::new(MaybeUninit::new(value)),\n notify: Notify::new(),\n }\n }\n}\n\nimpl SetOnce {\n /// Creates a new empty `SetOnce` instance.\n pub fn new() -> Self {\n Self {\n value_set: AtomicBool::new(false),\n value: UnsafeCell::new(MaybeUninit::uninit()),\n notify: Notify::new(),\n }\n }\n\n /// Creates a new empty `SetOnce` instance.\n ///\n /// Equivalent to `SetOnce::new`, except that it can be used in static\n /// variables.\n ///\n /// When using the `tracing` [unstable feature], a `SetOnce` created with\n /// `const_new` will not be instrumented. As such, it will not be visible\n /// in [`tokio-console`]. Instead, [`SetOnce::new`] should be used to\n /// create an instrumented object if that is needed.\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::sync::{SetOnce, SetOnceError};\n ///\n /// static ONCE: SetOnce = SetOnce::const_new();\n ///\n /// fn get_global_integer() -> Result, SetOnceError> {\n /// ONCE.set(2)?;\n /// Ok(ONCE.get())\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> Result<(), SetOnceError> {\n /// let result = get_global_integer()?;\n ///\n /// assert_eq!(result, Some(&2));\n /// Ok(())\n /// # }\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new() -> Self {\n Self {\n value_set: AtomicBool::new(false),\n value: UnsafeCell::new(MaybeUninit::uninit()),\n notify: Notify::const_new(),\n }\n }\n\n /// Creates a new `SetOnce` that contains the provided value, if any.\n ///\n /// If the `Option` is `None`, this is equivalent to `SetOnce::new`.\n ///\n /// [`SetOnce::new`]: crate::sync::SetOnce::new\n pub fn new_with(value: Option) -> Self {\n if let Some(v) = value {\n SetOnce::from(v)\n } else {\n SetOnce::new()\n }\n }\n\n /// Creates a new `SetOnce` that contains the provided value.\n ///\n /// # Example\n ///\n /// When using the `tracing` [unstable feature], a `SetOnce` created with\n /// `const_new_with` will not be instrumented. As such, it will not be\n /// visible in [`tokio-console`]. Instead, [`SetOnce::new_with`] should be\n /// used to create an instrumented object if that is needed.\n ///\n /// ```\n /// use tokio::sync::SetOnce;\n ///\n /// static ONCE: SetOnce = SetOnce::const_new_with(1);\n ///\n /// fn get_global_integer() -> Option<&'static u32> {\n /// ONCE.get()\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let result = get_global_integer();\n ///\n /// assert_eq!(result, Some(&1));\n /// # }\n /// ```\n ///\n /// [`tokio-console`]: https://github.com/tokio-rs/console\n /// [unstable feature]: crate#unstable-features\n #[cfg(not(all(loom, test)))]\n pub const fn const_new_with(value: T) -> Self {\n Self {\n value_set: AtomicBool::new(true),\n value: UnsafeCell::new(MaybeUninit::new(value)),\n notify: Notify::const_new(),\n }\n }\n\n /// Returns `true` if the `SetOnce` currently contains a value, and `false`\n /// otherwise.\n pub fn initialized(&self) -> bool {\n // Using acquire ordering so we're able to read/catch any writes that\n // are done with `Ordering::Release`\n self.value_set.load(Ordering::Acquire)\n }\n\n // SAFETY: The SetOnce must not be empty.\n unsafe fn get_unchecked(&self) -> &T {\n unsafe { &*self.value.with(|ptr| (*ptr).as_ptr()) }\n }\n\n /// Returns a reference to the value currently stored in the `SetOnce`, or\n /// `None` if the `SetOnce` is empty.\n pub fn get(&self) -> Option<&T> {\n if self.initialized() {\n // SAFETY: the SetOnce is initialized, so we can safely\n // call get_unchecked and return the value\n Some(unsafe { self.get_unchecked() })\n } else {\n None\n }\n }\n\n /// Sets the value of the `SetOnce` to the given value if the `SetOnce` is\n /// empty.\n ///\n /// If the `SetOnce` already has a value, this call will fail with an\n /// [`SetOnceError`].\n ///\n /// [`SetOnceError`]: crate::sync::SetOnceError\n pub fn set(&self, value: T) -> Result<(), SetOnceError> {\n if self.initialized() {\n return Err(SetOnceError(value));\n }\n\n // SAFETY: lock notify to ensure only one caller of set\n // can run at a time.\n let guard = self.notify.lock_waiter_list();\n\n if self.initialized() {\n return Err(SetOnceError(value));\n }\n\n // SAFETY: We have locked the mutex and checked if the value is\n // initialized or not, so we can safely write to the value\n unsafe {\n self.value.with_mut(|ptr| (*ptr).as_mut_ptr().write(value));\n }\n\n // Using release ordering so any threads that read a true from this\n // atomic is able to read the value we just stored.\n self.value_set.store(true, Ordering::Release);\n\n // notify the waiting wakers that the value is set\n guard.notify_waiters();\n\n Ok(())\n }\n\n /// Takes the value from the cell, destroying the cell in the process.\n /// Returns `None` if the cell is empty.\n pub fn into_inner(self) -> Option {\n // TODO: Use get_mut()\n let value_set = self.value_set.load(Ordering::Relaxed);\n\n if value_set {\n // Since we have taken ownership of self, its drop implementation\n // will be called by the end of this function, to prevent a double\n // free we will set the value_set to false so that the drop\n // implementation does not try to drop the value again.\n self.value_set.store(false, Ordering::Relaxed);\n\n // SAFETY: The SetOnce is currently initialized, we can assume the\n // value is initialized and return that, when we return the value\n // we give the drop handler to the return scope.\n Some(unsafe { self.value.with_mut(|ptr| ptr::read(ptr).assume_init()) })\n } else {\n None\n }\n }\n\n /// Waits until the value is set.\n ///\n /// If the `SetOnce` is already initialized, it will return the value\n /// immediately.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe.\n pub async fn wait(&self) -> &T {\n loop {\n if let Some(val) = self.get() {\n return val;\n }\n\n let notify_fut = self.notify.notified();\n pin!(notify_fut);\n\n poll_fn(|cx| {\n // Register under the notify's internal lock.\n let ret = notify_fut.as_mut().poll(cx);\n if self.value_set.load(Ordering::Relaxed) {\n return Poll::Ready(());\n }\n ret\n })\n .await;\n }\n }\n}\n\n// Since `get` gives us access to immutable references of the SetOnce, SetOnce\n// can only be Sync if T is Sync, otherwise SetOnce would allow sharing\n// references of !Sync values across threads. We need T to be Send in order for\n// SetOnce to by Sync because we can use `set` on `&SetOnce` to send values\n// (of type T) across threads.\nunsafe impl Sync for SetOnce {}\n\n// Access to SetOnce's value is guarded by the Atomic boolean flag\n// and atomic operations on `value_set`, so as long as T itself is Send\n// it's safe to send it to another thread\nunsafe impl Send for SetOnce {}\n\n/// Error that can be returned from [`SetOnce::set`].\n///\n/// This error means that the `SetOnce` was already initialized when\n/// set was called\n///\n/// [`SetOnce::set`]: crate::sync::SetOnce::set\n#[derive(Debug, PartialEq, Eq)]\npub struct SetOnceError(pub T);\n\nimpl fmt::Display for SetOnceError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(f, \"SetOnceError\")\n }\n}\n\nimpl Error for SetOnceError {}\n" }, { "path": "tokio/src/sync/task/atomic_waker.rs", "content": "#![cfg_attr(any(loom, not(feature = \"sync\")), allow(dead_code, unreachable_pub))]\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::hint;\nuse crate::loom::sync::atomic::AtomicUsize;\n\nuse std::fmt;\nuse std::panic::{resume_unwind, AssertUnwindSafe, RefUnwindSafe, UnwindSafe};\nuse std::sync::atomic::Ordering::{AcqRel, Acquire, Release};\nuse std::task::Waker;\n\n/// A synchronization primitive for task waking.\n///\n/// `AtomicWaker` will coordinate concurrent wakes with the consumer\n/// potentially \"waking\" the underlying task. This is useful in scenarios\n/// where a computation completes in another thread and wants to wake the\n/// consumer, but the consumer is in the process of being migrated to a new\n/// logical task.\n///\n/// Consumers should call `register` before checking the result of a computation\n/// and producers should call `wake` after producing the computation (this\n/// differs from the usual `thread::park` pattern). It is also permitted for\n/// `wake` to be called **before** `register`. This results in a no-op.\n///\n/// A single `AtomicWaker` may be reused for any number of calls to `register` or\n/// `wake`.\npub(crate) struct AtomicWaker {\n state: AtomicUsize,\n waker: UnsafeCell>,\n}\n\nimpl RefUnwindSafe for AtomicWaker {}\nimpl UnwindSafe for AtomicWaker {}\n\n// `AtomicWaker` is a multi-consumer, single-producer transfer cell. The cell\n// stores a `Waker` value produced by calls to `register` and many threads can\n// race to take the waker by calling `wake`.\n//\n// If a new `Waker` instance is produced by calling `register` before an existing\n// one is consumed, then the existing one is overwritten.\n//\n// While `AtomicWaker` is single-producer, the implementation ensures memory\n// safety. In the event of concurrent calls to `register`, there will be a\n// single winner whose waker will get stored in the cell. The losers will not\n// have their tasks woken. As such, callers should ensure to add synchronization\n// to calls to `register`.\n//\n// The implementation uses a single `AtomicUsize` value to coordinate access to\n// the `Waker` cell. There are two bits that are operated on independently. These\n// are represented by `REGISTERING` and `WAKING`.\n//\n// The `REGISTERING` bit is set when a producer enters the critical section. The\n// `WAKING` bit is set when a consumer enters the critical section. Neither\n// bit being set is represented by `WAITING`.\n//\n// A thread obtains an exclusive lock on the waker cell by transitioning the\n// state from `WAITING` to `REGISTERING` or `WAKING`, depending on the\n// operation the thread wishes to perform. When this transition is made, it is\n// guaranteed that no other thread will access the waker cell.\n//\n// # Registering\n//\n// On a call to `register`, an attempt to transition the state from WAITING to\n// REGISTERING is made. On success, the caller obtains a lock on the waker cell.\n//\n// If the lock is obtained, then the thread sets the waker cell to the waker\n// provided as an argument. Then it attempts to transition the state back from\n// `REGISTERING` -> `WAITING`.\n//\n// If this transition is successful, then the registering process is complete\n// and the next call to `wake` will observe the waker.\n//\n// If the transition fails, then there was a concurrent call to `wake` that\n// was unable to access the waker cell (due to the registering thread holding the\n// lock). To handle this, the registering thread removes the waker it just set\n// from the cell and calls `wake` on it. This call to wake represents the\n// attempt to wake by the other thread (that set the `WAKING` bit). The\n// state is then transitioned from `REGISTERING | WAKING` back to `WAITING`.\n// This transition must succeed because, at this point, the state cannot be\n// transitioned by another thread.\n//\n// # Waking\n//\n// On a call to `wake`, an attempt to transition the state from `WAITING` to\n// `WAKING` is made. On success, the caller obtains a lock on the waker cell.\n//\n// If the lock is obtained, then the thread takes ownership of the current value\n// in the waker cell, and calls `wake` on it. The state is then transitioned\n// back to `WAITING`. This transition must succeed as, at this point, the state\n// cannot be transitioned by another thread.\n//\n// If the thread is unable to obtain the lock, the `WAKING` bit is still set.\n// This is because it has either been set by the current thread but the previous\n// value included the `REGISTERING` bit **or** a concurrent thread is in the\n// `WAKING` critical section. Either way, no action must be taken.\n//\n// If the current thread is the only concurrent call to `wake` and another\n// thread is in the `register` critical section, when the other thread **exits**\n// the `register` critical section, it will observe the `WAKING` bit and\n// handle the waker itself.\n//\n// If another thread is in the `waker` critical section, then it will handle\n// waking the caller task.\n//\n// # A potential race (is safely handled).\n//\n// Imagine the following situation:\n//\n// * Thread A obtains the `wake` lock and wakes a task.\n//\n// * Before thread A releases the `wake` lock, the woken task is scheduled.\n//\n// * Thread B attempts to wake the task. In theory this should result in the\n// task being woken, but it cannot because thread A still holds the wake\n// lock.\n//\n// This case is handled by requiring users of `AtomicWaker` to call `register`\n// **before** attempting to observe the application state change that resulted\n// in the task being woken. The wakers also change the application state\n// before calling wake.\n//\n// Because of this, the task will do one of two things.\n//\n// 1) Observe the application state change that Thread B is waking on. In\n// this case, it is OK for Thread B's wake to be lost.\n//\n// 2) Call register before attempting to observe the application state. Since\n// Thread A still holds the `wake` lock, the call to `register` will result\n// in the task waking itself and get scheduled again.\n\n/// Idle state.\nconst WAITING: usize = 0;\n\n/// A new waker value is being registered with the `AtomicWaker` cell.\nconst REGISTERING: usize = 0b01;\n\n/// The task currently registered with the `AtomicWaker` cell is being woken.\nconst WAKING: usize = 0b10;\n\nimpl AtomicWaker {\n /// Create an `AtomicWaker`\n pub(crate) fn new() -> AtomicWaker {\n AtomicWaker {\n state: AtomicUsize::new(WAITING),\n waker: UnsafeCell::new(None),\n }\n }\n\n /*\n /// Registers the current waker to be notified on calls to `wake`.\n pub(crate) fn register(&self, waker: Waker) {\n self.do_register(waker);\n }\n */\n\n /// Registers the provided waker to be notified on calls to `wake`.\n ///\n /// The new waker will take place of any previous wakers that were registered\n /// by previous calls to `register`. Any calls to `wake` that happen after\n /// a call to `register` (as defined by the memory ordering rules), will\n /// wake the `register` caller's task.\n ///\n /// It is safe to call `register` with multiple other threads concurrently\n /// calling `wake`. This will result in the `register` caller's current\n /// task being woken once.\n ///\n /// This function is safe to call concurrently, but this is generally a bad\n /// idea. Concurrent calls to `register` will attempt to register different\n /// tasks to be woken. One of the callers will win and have its task set,\n /// but there is no guarantee as to which caller will succeed.\n pub(crate) fn register_by_ref(&self, waker: &Waker) {\n self.do_register(waker);\n }\n\n fn do_register(&self, waker: W)\n where\n W: WakerRef,\n {\n fn catch_unwind R, R>(f: F) -> std::thread::Result {\n std::panic::catch_unwind(AssertUnwindSafe(f))\n }\n\n match self\n .state\n .compare_exchange(WAITING, REGISTERING, Acquire, Acquire)\n .unwrap_or_else(|x| x)\n {\n WAITING => {\n unsafe {\n // If `into_waker` panics (because it's code outside of\n // AtomicWaker) we need to prime a guard that is called on\n // unwind to restore the waker to a WAITING state. Otherwise\n // any future calls to register will incorrectly be stuck\n // believing it's being updated by someone else.\n let new_waker_or_panic = catch_unwind(move || waker.into_waker());\n\n // Set the field to contain the new waker, or if\n // `into_waker` panicked, leave the old value.\n let mut maybe_panic = None;\n let mut old_waker = None;\n match new_waker_or_panic {\n Ok(new_waker) => {\n old_waker = self.waker.with_mut(|t| (*t).take());\n self.waker.with_mut(|t| *t = Some(new_waker));\n }\n Err(panic) => maybe_panic = Some(panic),\n }\n\n // Release the lock. If the state transitioned to include\n // the `WAKING` bit, this means that a wake has been\n // called concurrently, so we have to remove the waker and\n // wake it.`\n //\n // Start by assuming that the state is `REGISTERING` as this\n // is what we jut set it to.\n let res = self\n .state\n .compare_exchange(REGISTERING, WAITING, AcqRel, Acquire);\n\n match res {\n Ok(_) => {\n // We don't want to give the caller the panic if it\n // was someone else who put in that waker.\n let _ = catch_unwind(move || {\n drop(old_waker);\n });\n }\n Err(actual) => {\n // This branch can only be reached if a\n // concurrent thread called `wake`. In this\n // case, `actual` **must** be `REGISTERING |\n // WAKING`.\n debug_assert_eq!(actual, REGISTERING | WAKING);\n\n // Take the waker to wake once the atomic operation has\n // completed.\n let mut waker = self.waker.with_mut(|t| (*t).take());\n\n // Just swap, because no one could change state\n // while state == `Registering | `Waking`\n self.state.swap(WAITING, AcqRel);\n\n // If `into_waker` panicked, then the waker in the\n // waker slot is actually the old waker.\n if maybe_panic.is_some() {\n old_waker = waker.take();\n }\n\n // We don't want to give the caller the panic if it\n // was someone else who put in that waker.\n if let Some(old_waker) = old_waker {\n let _ = catch_unwind(move || {\n old_waker.wake();\n });\n }\n\n // The atomic swap was complete, now wake the waker\n // and return.\n //\n // If this panics, we end up in a consumed state and\n // return the panic to the caller.\n if let Some(waker) = waker {\n debug_assert!(maybe_panic.is_none());\n waker.wake();\n }\n }\n }\n\n if let Some(panic) = maybe_panic {\n // If `into_waker` panicked, return the panic to the caller.\n resume_unwind(panic);\n }\n }\n }\n WAKING => {\n // Currently in the process of waking the task, i.e.,\n // `wake` is currently being called on the old waker.\n // So, we call wake on the new waker.\n //\n // If this panics, someone else is responsible for restoring the\n // state of the waker.\n waker.wake();\n\n // This is equivalent to a spin lock, so use a spin hint.\n hint::spin_loop();\n }\n state => {\n // In this case, a concurrent thread is holding the\n // \"registering\" lock. This probably indicates a bug in the\n // caller's code as racing to call `register` doesn't make much\n // sense.\n //\n // We just want to maintain memory safety. It is ok to drop the\n // call to `register`.\n debug_assert!(state == REGISTERING || state == REGISTERING | WAKING);\n }\n }\n }\n\n /// Wakes the task that last called `register`.\n ///\n /// If `register` has not been called yet, then this does nothing.\n pub(crate) fn wake(&self) {\n if let Some(waker) = self.take_waker() {\n // If wake panics, we've consumed the waker which is a legitimate\n // outcome.\n waker.wake();\n }\n }\n\n /// Attempts to take the `Waker` value out of the `AtomicWaker` with the\n /// intention that the caller will wake the task later.\n pub(crate) fn take_waker(&self) -> Option {\n // AcqRel ordering is used in order to acquire the value of the `waker`\n // cell as well as to establish a `release` ordering with whatever\n // memory the `AtomicWaker` is associated with.\n match self.state.fetch_or(WAKING, AcqRel) {\n WAITING => {\n // SAFETY: the waking lock has been acquired.\n let waker = unsafe { self.waker.with_mut(|t| (*t).take()) };\n\n // Release the lock.\n let old_state = self.state.swap(WAITING, Release);\n debug_assert!(old_state == WAKING);\n\n waker\n }\n state => {\n // There is a concurrent thread currently updating the\n // associated waker.\n //\n // Nothing more to do as the `WAKING` bit has been set. It\n // doesn't matter if there are concurrent registering threads or\n // not.\n //\n debug_assert!(\n state == REGISTERING || state == REGISTERING | WAKING || state == WAKING\n );\n None\n }\n }\n }\n}\n\nimpl Default for AtomicWaker {\n fn default() -> Self {\n AtomicWaker::new()\n }\n}\n\nimpl fmt::Debug for AtomicWaker {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"AtomicWaker\")\n }\n}\n\nunsafe impl Send for AtomicWaker {}\nunsafe impl Sync for AtomicWaker {}\n\ntrait WakerRef {\n fn wake(self);\n fn into_waker(self) -> Waker;\n}\n\nimpl WakerRef for Waker {\n fn wake(self) {\n self.wake();\n }\n\n fn into_waker(self) -> Waker {\n self\n }\n}\n\nimpl WakerRef for &Waker {\n fn wake(self) {\n self.wake_by_ref();\n }\n\n fn into_waker(self) -> Waker {\n self.clone()\n }\n}\n" }, { "path": "tokio/src/sync/task/mod.rs", "content": "//! Thread-safe task notification primitives.\n\nmod atomic_waker;\npub(crate) use self::atomic_waker::AtomicWaker;\n" }, { "path": "tokio/src/sync/tests/atomic_waker.rs", "content": "use crate::sync::AtomicWaker;\nuse tokio_test::task;\n\nuse std::task::Waker;\n\n#[allow(unused)]\ntrait AssertSend: Send {}\n\n#[allow(unused)]\ntrait AssertSync: Sync {}\n\nimpl AssertSend for AtomicWaker {}\nimpl AssertSync for AtomicWaker {}\n\nimpl AssertSend for Waker {}\nimpl AssertSync for Waker {}\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\n#[test]\nfn basic_usage() {\n let mut waker = task::spawn(AtomicWaker::new());\n\n waker.enter(|cx, waker| waker.register_by_ref(cx.waker()));\n waker.wake();\n\n assert!(waker.is_woken());\n}\n\n#[test]\nfn wake_without_register() {\n let mut waker = task::spawn(AtomicWaker::new());\n waker.wake();\n\n // Registering should not result in a notification\n waker.enter(|cx, waker| waker.register_by_ref(cx.waker()));\n\n assert!(!waker.is_woken());\n}\n\n#[test]\n#[cfg_attr(target_family = \"wasm\", ignore)] // threads not supported\nfn failed_wake_synchronizes() {\n for _ in 0..1000 {\n failed_wake_synchronizes_inner();\n }\n}\n\nfn failed_wake_synchronizes_inner() {\n use futures::task::noop_waker_ref;\n use std::sync::atomic::{AtomicBool, Ordering};\n static DID_SYNCHRONIZE: AtomicBool = AtomicBool::new(false);\n DID_SYNCHRONIZE.store(false, Ordering::Relaxed);\n\n let waker = AtomicWaker::new();\n waker.register_by_ref(noop_waker_ref());\n\n std::thread::scope(|s| {\n let jh = s.spawn(|| {\n DID_SYNCHRONIZE.store(true, Ordering::Relaxed);\n waker.take_waker()\n });\n\n waker.take_waker();\n waker.register_by_ref(noop_waker_ref());\n\n let did_synchronize = DID_SYNCHRONIZE.load(Ordering::Relaxed);\n let did_take = jh.join().unwrap().is_some();\n assert!(did_synchronize || did_take);\n });\n}\n\n#[cfg(panic = \"unwind\")]\n#[test]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn atomic_waker_panic_safe() {\n use std::panic;\n use std::ptr;\n use std::task::{RawWaker, RawWakerVTable, Waker};\n\n static PANICKING_VTABLE: RawWakerVTable = RawWakerVTable::new(\n |_| panic!(\"clone\"),\n |_| unimplemented!(\"wake\"),\n |_| unimplemented!(\"wake_by_ref\"),\n |_| (),\n );\n\n static NONPANICKING_VTABLE: RawWakerVTable = RawWakerVTable::new(\n |_| RawWaker::new(ptr::null(), &NONPANICKING_VTABLE),\n |_| unimplemented!(\"wake\"),\n |_| unimplemented!(\"wake_by_ref\"),\n |_| (),\n );\n\n let panicking = unsafe { Waker::from_raw(RawWaker::new(ptr::null(), &PANICKING_VTABLE)) };\n let nonpanicking = unsafe { Waker::from_raw(RawWaker::new(ptr::null(), &NONPANICKING_VTABLE)) };\n\n let atomic_waker = AtomicWaker::new();\n\n let panicking = panic::AssertUnwindSafe(&panicking);\n\n let result = panic::catch_unwind(|| {\n let panic::AssertUnwindSafe(panicking) = panicking;\n atomic_waker.register_by_ref(panicking);\n });\n\n assert!(result.is_err());\n assert!(atomic_waker.take_waker().is_none());\n\n atomic_waker.register_by_ref(&nonpanicking);\n assert!(atomic_waker.take_waker().is_some());\n}\n" }, { "path": "tokio/src/sync/tests/loom_atomic_waker.rs", "content": "use crate::sync::task::AtomicWaker;\n\nuse loom::future::block_on;\nuse loom::sync::atomic::AtomicUsize;\nuse loom::thread;\nuse std::future::poll_fn;\nuse std::sync::atomic::Ordering::Relaxed;\nuse std::sync::Arc;\nuse std::task::Poll::{Pending, Ready};\n\nstruct Chan {\n num: AtomicUsize,\n task: AtomicWaker,\n}\n\n#[test]\nfn basic_notification() {\n const NUM_NOTIFY: usize = 2;\n\n loom::model(|| {\n let chan = Arc::new(Chan {\n num: AtomicUsize::new(0),\n task: AtomicWaker::new(),\n });\n\n for _ in 0..NUM_NOTIFY {\n let chan = chan.clone();\n\n thread::spawn(move || {\n chan.num.fetch_add(1, Relaxed);\n chan.task.wake();\n });\n }\n\n block_on(poll_fn(move |cx| {\n chan.task.register_by_ref(cx.waker());\n\n if NUM_NOTIFY == chan.num.load(Relaxed) {\n return Ready(());\n }\n\n Pending\n }));\n });\n}\n\n#[test]\nfn test_panicky_waker() {\n use std::panic;\n use std::ptr;\n use std::task::{RawWaker, RawWakerVTable, Waker};\n\n static PANICKING_VTABLE: RawWakerVTable =\n RawWakerVTable::new(|_| panic!(\"clone\"), |_| (), |_| (), |_| ());\n\n let panicking = unsafe { Waker::from_raw(RawWaker::new(ptr::null(), &PANICKING_VTABLE)) };\n\n // If you're working with this test (and I sure hope you never have to!),\n // uncomment the following section because there will be a lot of panics\n // which would otherwise log.\n //\n // We can't however leaved it uncommented, because it's global.\n // panic::set_hook(Box::new(|_| ()));\n\n const NUM_NOTIFY: usize = 2;\n\n loom::model(move || {\n let chan = Arc::new(Chan {\n num: AtomicUsize::new(0),\n task: AtomicWaker::new(),\n });\n\n for _ in 0..NUM_NOTIFY {\n let chan = chan.clone();\n\n thread::spawn(move || {\n chan.num.fetch_add(1, Relaxed);\n chan.task.wake();\n });\n }\n\n // Note: this panic should have no effect on the overall state of the\n // waker and it should proceed as normal.\n //\n // A thread above might race to flag a wakeup, and a WAKING state will\n // be preserved if this expected panic races with that so the below\n // procedure should be allowed to continue uninterrupted.\n let _ = panic::catch_unwind(|| chan.task.register_by_ref(&panicking));\n\n block_on(poll_fn(move |cx| {\n chan.task.register_by_ref(cx.waker());\n\n if NUM_NOTIFY == chan.num.load(Relaxed) {\n return Ready(());\n }\n\n Pending\n }));\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_broadcast.rs", "content": "use crate::sync::broadcast;\nuse crate::sync::broadcast::error::RecvError::{Closed, Lagged};\n\nuse loom::future::block_on;\nuse loom::sync::Arc;\nuse loom::thread;\nuse tokio_test::{assert_err, assert_ok};\n\n#[test]\nfn broadcast_send() {\n loom::model(|| {\n let (tx1, mut rx) = broadcast::channel(2);\n let tx1 = Arc::new(tx1);\n let tx2 = tx1.clone();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n assert_ok!(tx1.send(\"one\"));\n assert_ok!(tx1.send(\"two\"));\n assert_ok!(tx1.send(\"three\"));\n });\n });\n\n let th2 = thread::spawn(move || {\n block_on(async {\n assert_ok!(tx2.send(\"eins\"));\n assert_ok!(tx2.send(\"zwei\"));\n assert_ok!(tx2.send(\"drei\"));\n });\n });\n\n block_on(async {\n let mut num = 0;\n loop {\n match rx.recv().await {\n Ok(_) => num += 1,\n Err(Closed) => break,\n Err(Lagged(n)) => num += n as usize,\n }\n }\n assert_eq!(num, 6);\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n// An `Arc` is used as the value in order to detect memory leaks.\n#[test]\nfn broadcast_two() {\n loom::model(|| {\n let (tx, mut rx1) = broadcast::channel::>(16);\n let mut rx2 = tx.subscribe();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n let v = assert_ok!(rx1.recv().await);\n assert_eq!(*v, \"hello\");\n\n let v = assert_ok!(rx1.recv().await);\n assert_eq!(*v, \"world\");\n\n match assert_err!(rx1.recv().await) {\n Closed => {}\n _ => panic!(),\n }\n });\n });\n\n let th2 = thread::spawn(move || {\n block_on(async {\n let v = assert_ok!(rx2.recv().await);\n assert_eq!(*v, \"hello\");\n\n let v = assert_ok!(rx2.recv().await);\n assert_eq!(*v, \"world\");\n\n match assert_err!(rx2.recv().await) {\n Closed => {}\n _ => panic!(),\n }\n });\n });\n\n assert_ok!(tx.send(Arc::new(\"hello\")));\n assert_ok!(tx.send(Arc::new(\"world\")));\n drop(tx);\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n#[test]\nfn broadcast_wrap() {\n loom::model(|| {\n let (tx, mut rx1) = broadcast::channel(2);\n let mut rx2 = tx.subscribe();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n let mut num = 0;\n\n loop {\n match rx1.recv().await {\n Ok(_) => num += 1,\n Err(Closed) => break,\n Err(Lagged(n)) => num += n as usize,\n }\n }\n\n assert_eq!(num, 3);\n });\n });\n\n let th2 = thread::spawn(move || {\n block_on(async {\n let mut num = 0;\n\n loop {\n match rx2.recv().await {\n Ok(_) => num += 1,\n Err(Closed) => break,\n Err(Lagged(n)) => num += n as usize,\n }\n }\n\n assert_eq!(num, 3);\n });\n });\n\n assert_ok!(tx.send(\"one\"));\n assert_ok!(tx.send(\"two\"));\n assert_ok!(tx.send(\"three\"));\n\n drop(tx);\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n#[test]\nfn drop_rx() {\n loom::model(|| {\n let (tx, mut rx1) = broadcast::channel(16);\n let rx2 = tx.subscribe();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n let v = assert_ok!(rx1.recv().await);\n assert_eq!(v, \"one\");\n\n let v = assert_ok!(rx1.recv().await);\n assert_eq!(v, \"two\");\n\n let v = assert_ok!(rx1.recv().await);\n assert_eq!(v, \"three\");\n\n match assert_err!(rx1.recv().await) {\n Closed => {}\n _ => panic!(),\n }\n });\n });\n\n let th2 = thread::spawn(move || {\n drop(rx2);\n });\n\n assert_ok!(tx.send(\"one\"));\n assert_ok!(tx.send(\"two\"));\n assert_ok!(tx.send(\"three\"));\n drop(tx);\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n#[test]\nfn drop_multiple_rx_with_overflow() {\n loom::model(move || {\n // It is essential to have multiple senders and receivers in this test case.\n let (tx, mut rx) = broadcast::channel(1);\n let _rx2 = tx.subscribe();\n\n let _ = tx.send(());\n let tx2 = tx.clone();\n let th1 = thread::spawn(move || {\n block_on(async {\n for _ in 0..100 {\n let _ = tx2.send(());\n }\n });\n });\n let _ = tx.send(());\n\n let th2 = thread::spawn(move || {\n block_on(async { while let Ok(_) = rx.recv().await {} });\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_list.rs", "content": "use crate::sync::mpsc::list;\n\nuse loom::thread;\nuse std::sync::Arc;\n\n#[test]\nfn smoke() {\n use crate::sync::mpsc::block::Read;\n\n const NUM_TX: usize = 2;\n const NUM_MSG: usize = 2;\n\n loom::model(|| {\n let (tx, mut rx) = list::channel();\n let tx = Arc::new(tx);\n\n for th in 0..NUM_TX {\n let tx = tx.clone();\n\n thread::spawn(move || {\n for i in 0..NUM_MSG {\n tx.push((th, i));\n }\n });\n }\n\n let mut next = vec![0; NUM_TX];\n\n loop {\n match rx.pop(&tx) {\n Some(Read::Value((th, v))) => {\n assert_eq!(v, next[th]);\n next[th] += 1;\n\n if next.iter().all(|&i| i == NUM_MSG) {\n break;\n }\n }\n Some(Read::Closed) => {\n panic!();\n }\n None => {\n thread::yield_now();\n }\n }\n }\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_mpsc.rs", "content": "use crate::sync::mpsc;\n\nuse loom::future::block_on;\nuse loom::sync::Arc;\nuse loom::thread;\nuse std::future::poll_fn;\nuse tokio_test::assert_ok;\n\n#[test]\nfn closing_tx() {\n loom::model(|| {\n let (tx, mut rx) = mpsc::channel(16);\n\n thread::spawn(move || {\n tx.try_send(()).unwrap();\n drop(tx);\n });\n\n let v = block_on(rx.recv());\n assert!(v.is_some());\n\n let v = block_on(rx.recv());\n assert!(v.is_none());\n });\n}\n\n#[test]\nfn closing_unbounded_tx() {\n loom::model(|| {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n thread::spawn(move || {\n tx.send(()).unwrap();\n drop(tx);\n });\n\n let v = block_on(rx.recv());\n assert!(v.is_some());\n\n let v = block_on(rx.recv());\n assert!(v.is_none());\n });\n}\n\n#[test]\nfn closing_bounded_rx() {\n loom::model(|| {\n let (tx1, rx) = mpsc::channel::<()>(16);\n let tx2 = tx1.clone();\n thread::spawn(move || {\n drop(rx);\n });\n\n block_on(tx1.closed());\n block_on(tx2.closed());\n });\n}\n\n#[test]\nfn closing_and_sending() {\n loom::model(|| {\n let (tx1, mut rx) = mpsc::channel::<()>(16);\n let tx1 = Arc::new(tx1);\n let tx2 = tx1.clone();\n\n let th1 = thread::spawn(move || {\n tx1.try_send(()).unwrap();\n });\n\n let th2 = thread::spawn(move || {\n block_on(tx2.closed());\n });\n\n let th3 = thread::spawn(move || {\n let v = block_on(rx.recv());\n assert!(v.is_some());\n drop(rx);\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n\n#[test]\nfn closing_unbounded_rx() {\n loom::model(|| {\n let (tx1, rx) = mpsc::unbounded_channel::<()>();\n let tx2 = tx1.clone();\n thread::spawn(move || {\n drop(rx);\n });\n\n block_on(tx1.closed());\n block_on(tx2.closed());\n });\n}\n\n#[test]\nfn dropping_tx() {\n loom::model(|| {\n let (tx, mut rx) = mpsc::channel::<()>(16);\n\n for _ in 0..2 {\n let tx = tx.clone();\n thread::spawn(move || {\n drop(tx);\n });\n }\n drop(tx);\n\n let v = block_on(rx.recv());\n assert!(v.is_none());\n });\n}\n\n#[test]\nfn dropping_unbounded_tx() {\n loom::model(|| {\n let (tx, mut rx) = mpsc::unbounded_channel::<()>();\n\n for _ in 0..2 {\n let tx = tx.clone();\n thread::spawn(move || {\n drop(tx);\n });\n }\n drop(tx);\n\n let v = block_on(rx.recv());\n assert!(v.is_none());\n });\n}\n\n#[test]\nfn try_recv() {\n loom::model(|| {\n use crate::sync::{mpsc, Semaphore};\n use loom::sync::{Arc, Mutex};\n\n const PERMITS: usize = 2;\n const TASKS: usize = 2;\n const CYCLES: usize = 1;\n\n struct Context {\n sem: Arc,\n tx: mpsc::Sender<()>,\n rx: Mutex>,\n }\n\n fn run(ctx: &Context) {\n block_on(async {\n let permit = ctx.sem.acquire().await;\n assert_ok!(ctx.rx.lock().unwrap().try_recv());\n crate::task::yield_now().await;\n assert_ok!(ctx.tx.clone().try_send(()));\n drop(permit);\n });\n }\n\n let (tx, rx) = mpsc::channel(PERMITS);\n let sem = Arc::new(Semaphore::new(PERMITS));\n let ctx = Arc::new(Context {\n sem,\n tx,\n rx: Mutex::new(rx),\n });\n\n for _ in 0..PERMITS {\n assert_ok!(ctx.tx.clone().try_send(()));\n }\n\n let mut threads = Vec::new();\n\n for _ in 0..TASKS {\n let ctx = ctx.clone();\n\n threads.push(thread::spawn(move || {\n run(&ctx);\n }));\n }\n\n run(&ctx);\n\n for thread in threads {\n thread.join().unwrap();\n }\n });\n}\n\n#[test]\nfn len_nonzero_after_send() {\n loom::model(|| {\n let (send, recv) = mpsc::channel(10);\n let send2 = send.clone();\n\n let join = thread::spawn(move || {\n block_on(send2.send(\"message2\")).unwrap();\n });\n\n block_on(send.send(\"message1\")).unwrap();\n assert!(recv.len() != 0);\n\n join.join().unwrap();\n });\n}\n\n#[test]\nfn nonempty_after_send() {\n loom::model(|| {\n let (send, recv) = mpsc::channel(10);\n let send2 = send.clone();\n\n let join = thread::spawn(move || {\n block_on(send2.send(\"message2\")).unwrap();\n });\n\n block_on(send.send(\"message1\")).unwrap();\n assert!(!recv.is_empty());\n\n join.join().unwrap();\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_notify.rs", "content": "use crate::sync::Notify;\n\nuse loom::future::block_on;\nuse loom::sync::Arc;\nuse loom::thread;\n\nuse tokio_test::{assert_pending, assert_ready};\n\n/// `util::wake_list::NUM_WAKERS`\nconst WAKE_LIST_SIZE: usize = 32;\n\n#[test]\nfn notify_one() {\n loom::model(|| {\n let tx = Arc::new(Notify::new());\n let rx = tx.clone();\n\n let th = thread::spawn(move || {\n block_on(async {\n rx.notified().await;\n });\n });\n\n tx.notify_one();\n th.join().unwrap();\n });\n}\n\n#[test]\nfn notify_waiters() {\n loom::model(|| {\n let notify = Arc::new(Notify::new());\n let tx = notify.clone();\n let notified1 = notify.notified();\n let notified2 = notify.notified();\n\n let th = thread::spawn(move || {\n tx.notify_waiters();\n });\n\n block_on(async {\n notified1.await;\n notified2.await;\n });\n\n th.join().unwrap();\n });\n}\n\n#[test]\nfn notify_waiters_and_one() {\n loom::model(|| {\n let notify = Arc::new(Notify::new());\n let tx1 = notify.clone();\n let tx2 = notify.clone();\n\n let th1 = thread::spawn(move || {\n tx1.notify_waiters();\n });\n\n let th2 = thread::spawn(move || {\n tx2.notify_one();\n });\n\n let th3 = thread::spawn(move || {\n let notified = notify.notified();\n\n block_on(async {\n notified.await;\n });\n });\n\n th1.join().unwrap();\n th2.join().unwrap();\n th3.join().unwrap();\n });\n}\n\n#[test]\nfn notify_multi() {\n loom::model(|| {\n let notify = Arc::new(Notify::new());\n\n let mut threads = vec![];\n\n for _ in 0..2 {\n let notify = notify.clone();\n\n threads.push(thread::spawn(move || {\n block_on(async {\n notify.notified().await;\n notify.notify_one();\n })\n }));\n }\n\n notify.notify_one();\n\n for thread in threads.drain(..) {\n thread.join().unwrap();\n }\n\n block_on(async {\n notify.notified().await;\n });\n });\n}\n\n#[test]\nfn notify_drop() {\n use std::future::{poll_fn, Future};\n use std::task::Poll;\n\n loom::model(|| {\n let notify = Arc::new(Notify::new());\n let rx1 = notify.clone();\n let rx2 = notify.clone();\n\n let th1 = thread::spawn(move || {\n let mut recv = Box::pin(rx1.notified());\n\n block_on(poll_fn(|cx| {\n if recv.as_mut().poll(cx).is_ready() {\n rx1.notify_one();\n }\n Poll::Ready(())\n }));\n });\n\n let th2 = thread::spawn(move || {\n block_on(async {\n rx2.notified().await;\n // Trigger second notification\n rx2.notify_one();\n rx2.notified().await;\n });\n });\n\n notify.notify_one();\n\n th1.join().unwrap();\n th2.join().unwrap();\n });\n}\n\n/// Polls two `Notified` futures and checks if poll results are consistent\n/// with each other. If the first future is notified by a `notify_waiters`\n/// call, then the second one must be notified as well.\n#[test]\nfn notify_waiters_poll_consistency() {\n fn notify_waiters_poll_consistency_variant(poll_setting: [bool; 2]) {\n let notify = Arc::new(Notify::new());\n let mut notified = [\n tokio_test::task::spawn(notify.notified()),\n tokio_test::task::spawn(notify.notified()),\n ];\n for i in 0..2 {\n if poll_setting[i] {\n assert_pending!(notified[i].poll());\n }\n }\n\n let tx = notify.clone();\n let th = thread::spawn(move || {\n tx.notify_waiters();\n });\n\n let res1 = notified[0].poll();\n let res2 = notified[1].poll();\n\n // If res1 is ready, then res2 must also be ready.\n assert!(res1.is_pending() || res2.is_ready());\n\n th.join().unwrap();\n }\n\n // We test different scenarios in which pending futures had or had not\n // been polled before the call to `notify_waiters`.\n loom::model(|| notify_waiters_poll_consistency_variant([false, false]));\n loom::model(|| notify_waiters_poll_consistency_variant([true, false]));\n loom::model(|| notify_waiters_poll_consistency_variant([false, true]));\n loom::model(|| notify_waiters_poll_consistency_variant([true, true]));\n}\n\n/// Polls two `Notified` futures and checks if poll results are consistent\n/// with each other. If the first future is notified by a `notify_waiters`\n/// call, then the second one must be notified as well.\n///\n/// Here we also add other `Notified` futures in between to force the two\n/// tested futures to end up in different chunks.\n#[test]\nfn notify_waiters_poll_consistency_many() {\n fn notify_waiters_poll_consistency_many_variant(order: [usize; 2]) {\n let notify = Arc::new(Notify::new());\n\n let mut futs = (0..WAKE_LIST_SIZE + 1)\n .map(|_| tokio_test::task::spawn(notify.notified()))\n .collect::>();\n\n assert_pending!(futs[order[0]].poll());\n for i in 2..futs.len() {\n assert_pending!(futs[i].poll());\n }\n assert_pending!(futs[order[1]].poll());\n\n let tx = notify.clone();\n let th = thread::spawn(move || {\n tx.notify_waiters();\n });\n\n let res1 = futs[0].poll();\n let res2 = futs[1].poll();\n\n // If res1 is ready, then res2 must also be ready.\n assert!(res1.is_pending() || res2.is_ready());\n\n th.join().unwrap();\n }\n\n // We test different scenarios in which futures are polled in different order.\n loom::model(|| notify_waiters_poll_consistency_many_variant([0, 1]));\n loom::model(|| notify_waiters_poll_consistency_many_variant([1, 0]));\n}\n\n/// Checks if a call to `notify_waiters` is observed as atomic when combined\n/// with a concurrent call to `notify_one`.\n#[test]\nfn notify_waiters_is_atomic() {\n fn notify_waiters_is_atomic_variant(tested_fut_index: usize) {\n let notify = Arc::new(Notify::new());\n\n let mut futs = (0..WAKE_LIST_SIZE + 1)\n .map(|_| tokio_test::task::spawn(notify.notified()))\n .collect::>();\n\n for fut in &mut futs {\n assert_pending!(fut.poll());\n }\n\n let tx = notify.clone();\n let th = thread::spawn(move || {\n tx.notify_waiters();\n });\n\n block_on(async {\n // If awaiting one of the futures completes, then we should be\n // able to assume that all pending futures are notified. Therefore\n // a notification from a subsequent `notify_one` call should not\n // be consumed by an old future.\n futs.remove(tested_fut_index).await;\n\n let mut new_fut = tokio_test::task::spawn(notify.notified());\n assert_pending!(new_fut.poll());\n\n notify.notify_one();\n\n // `new_fut` must consume the notification from `notify_one`.\n assert_ready!(new_fut.poll());\n });\n\n th.join().unwrap();\n }\n\n // We test different scenarios in which the tested future is at the beginning\n // or at the end of the waiters queue used by `Notify`.\n loom::model(|| notify_waiters_is_atomic_variant(0));\n loom::model(|| notify_waiters_is_atomic_variant(32));\n}\n\n/// Checks if a single call to `notify_waiters` does not get through two `Notified`\n/// futures created and awaited sequentially like this:\n/// ```ignore\n/// notify.notified().await;\n/// notify.notified().await;\n/// ```\n#[test]\nfn notify_waiters_sequential_notified_await() {\n use crate::sync::oneshot;\n\n loom::model(|| {\n let notify = Arc::new(Notify::new());\n\n let (tx_fst, rx_fst) = oneshot::channel();\n let (tx_snd, rx_snd) = oneshot::channel();\n\n let receiver = thread::spawn({\n let notify = notify.clone();\n move || {\n block_on(async {\n // Poll the first `Notified` to put it as the first waiter\n // in the queue.\n let mut first_notified = tokio_test::task::spawn(notify.notified());\n assert_pending!(first_notified.poll());\n\n // Create additional waiters to force `notify_waiters` to\n // release the lock at least once.\n let _task_pile = (0..WAKE_LIST_SIZE + 1)\n .map(|_| {\n let mut fut = tokio_test::task::spawn(notify.notified());\n assert_pending!(fut.poll());\n fut\n })\n .collect::>();\n\n // We are ready for the notify_waiters call.\n tx_fst.send(()).unwrap();\n\n first_notified.await;\n\n // Poll the second `Notified` future to try to insert\n // it to the waiters queue.\n let mut second_notified = tokio_test::task::spawn(notify.notified());\n assert_pending!(second_notified.poll());\n\n // Wait for the `notify_waiters` to end and check if we\n // are woken up.\n rx_snd.await.unwrap();\n assert_pending!(second_notified.poll());\n });\n }\n });\n\n // Wait for the signal and call `notify_waiters`.\n block_on(rx_fst).unwrap();\n notify.notify_waiters();\n tx_snd.send(()).unwrap();\n\n receiver.join().unwrap();\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_oneshot.rs", "content": "use crate::sync::oneshot;\n\nuse loom::future::block_on;\nuse loom::thread;\nuse std::future::poll_fn;\nuse std::pin::Pin;\nuse std::task::Poll::{Pending, Ready};\n\n#[test]\nfn smoke() {\n loom::model(|| {\n let (tx, rx) = oneshot::channel();\n\n thread::spawn(move || {\n tx.send(1).unwrap();\n });\n\n let value = block_on(rx).unwrap();\n assert_eq!(1, value);\n });\n}\n\n#[test]\nfn changing_rx_task() {\n loom::model(|| {\n let (tx, mut rx) = oneshot::channel();\n\n thread::spawn(move || {\n tx.send(1).unwrap();\n });\n\n let rx = thread::spawn(move || {\n let ready = block_on(poll_fn(|cx| match Pin::new(&mut rx).poll(cx) {\n Ready(Ok(value)) => {\n assert_eq!(1, value);\n Ready(true)\n }\n Ready(Err(_)) => unimplemented!(),\n Pending => Ready(false),\n }));\n\n if ready {\n None\n } else {\n Some(rx)\n }\n })\n .join()\n .unwrap();\n\n if let Some(rx) = rx {\n // Previous task parked, use a new task...\n let value = block_on(rx).unwrap();\n assert_eq!(1, value);\n }\n });\n}\n\n#[test]\nfn try_recv_close() {\n // reproduces https://github.com/tokio-rs/tokio/issues/4225\n loom::model(|| {\n let (tx, mut rx) = oneshot::channel();\n thread::spawn(move || {\n let _ = tx.send(());\n });\n\n rx.close();\n let _ = rx.try_recv();\n })\n}\n\n#[test]\nfn recv_closed() {\n // reproduces https://github.com/tokio-rs/tokio/issues/4225\n loom::model(|| {\n let (tx, mut rx) = oneshot::channel();\n\n thread::spawn(move || {\n let _ = tx.send(1);\n });\n\n rx.close();\n let _ = block_on(rx);\n });\n}\n\n// TODO: Move this into `oneshot` proper.\n\nuse std::future::Future;\nuse std::task::{Context, Poll};\n\nstruct OnClose<'a> {\n tx: &'a mut oneshot::Sender,\n}\n\nimpl<'a> OnClose<'a> {\n fn new(tx: &'a mut oneshot::Sender) -> Self {\n OnClose { tx }\n }\n}\n\nimpl Future for OnClose<'_> {\n type Output = bool;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let fut = self.get_mut().tx.closed();\n crate::pin!(fut);\n\n Ready(fut.poll(cx).is_ready())\n }\n}\n\n#[test]\nfn changing_tx_task() {\n loom::model(|| {\n let (mut tx, rx) = oneshot::channel::();\n\n thread::spawn(move || {\n drop(rx);\n });\n\n let tx = thread::spawn(move || {\n let t1 = block_on(OnClose::new(&mut tx));\n\n if t1 {\n None\n } else {\n Some(tx)\n }\n })\n .join()\n .unwrap();\n\n if let Some(mut tx) = tx {\n // Previous task parked, use a new task...\n block_on(OnClose::new(&mut tx));\n }\n });\n}\n\n#[test]\nfn checking_tx_send_ok_not_drop() {\n use std::borrow::Borrow;\n use std::cell::Cell;\n\n loom::thread_local! {\n static IS_RX: Cell = Cell::new(true);\n }\n\n struct Msg;\n\n impl Drop for Msg {\n fn drop(&mut self) {\n IS_RX.with(|is_rx: &Cell<_>| {\n // On `tx.send(msg)` returning `Err(msg)`,\n // we call `std::mem::forget(msg)`, so that\n // `drop` is not expected to be called in the\n // tx thread.\n assert!(is_rx.get());\n });\n }\n }\n\n let mut builder = loom::model::Builder::new();\n builder.preemption_bound = Some(2);\n\n builder.check(|| {\n let (tx, rx) = oneshot::channel();\n\n // tx thread\n let tx_thread_join_handle = thread::spawn(move || {\n // Ensure that `Msg::drop` in this thread will see is_rx == false\n IS_RX.with(|is_rx: &Cell<_>| {\n is_rx.set(false);\n });\n if let Err(msg) = tx.send(Msg) {\n std::mem::forget(msg);\n }\n });\n\n // main thread is the rx thread\n drop(rx);\n\n tx_thread_join_handle.join().unwrap();\n });\n}\n\n#[test]\nfn drop_rx_after_poll() {\n // Test that rx_task is properly deallocated when the receiver is dropped\n // after being polled (which sets rx_task), while the sender is concurrently\n // trying to send.\n loom::model(|| {\n let (tx, mut rx) = oneshot::channel::();\n\n // Poll once to set rx_task before entering the parallel part of the\n // test.\n let _ = block_on(poll_fn(|cx| {\n let _ = Pin::new(&mut rx).poll(cx);\n Ready(())\n }));\n\n // Drop the receiver concurrently with the sender trying to send.\n let rx_thread = thread::spawn(move || {\n drop(rx);\n });\n\n let _ = tx.send(1);\n\n rx_thread.join().unwrap();\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_rwlock.rs", "content": "use crate::sync::rwlock::*;\n\nuse loom::future::block_on;\nuse loom::thread;\nuse std::sync::Arc;\n\n#[test]\nfn concurrent_write() {\n let b = loom::model::Builder::new();\n\n b.check(|| {\n let rwlock = Arc::new(RwLock::::new(0));\n\n let rwclone = rwlock.clone();\n let t1 = thread::spawn(move || {\n block_on(async {\n let mut guard = rwclone.write().await;\n *guard += 5;\n });\n });\n\n let rwclone = rwlock.clone();\n let t2 = thread::spawn(move || {\n block_on(async {\n let mut guard = rwclone.write_owned().await;\n *guard += 5;\n });\n });\n\n t1.join().expect(\"thread 1 write should not panic\");\n t2.join().expect(\"thread 2 write should not panic\");\n //when all threads have finished the value on the lock should be 10\n let guard = block_on(rwlock.read());\n assert_eq!(10, *guard);\n });\n}\n\n#[test]\nfn concurrent_read_write() {\n let b = loom::model::Builder::new();\n\n b.check(|| {\n let rwlock = Arc::new(RwLock::::new(0));\n\n let rwclone = rwlock.clone();\n let t1 = thread::spawn(move || {\n block_on(async {\n let mut guard = rwclone.write().await;\n *guard += 5;\n });\n });\n\n let rwclone = rwlock.clone();\n let t2 = thread::spawn(move || {\n block_on(async {\n let mut guard = rwclone.write_owned().await;\n *guard += 5;\n });\n });\n\n let rwclone = rwlock.clone();\n let t3 = thread::spawn(move || {\n block_on(async {\n let guard = rwclone.read().await;\n //at this state the value on the lock may either be 0, 5, or 10\n assert!(*guard == 0 || *guard == 5 || *guard == 10);\n });\n });\n\n {\n let guard = block_on(rwlock.clone().read_owned());\n //at this state the value on the lock may either be 0, 5, or 10\n assert!(*guard == 0 || *guard == 5 || *guard == 10);\n }\n\n t1.join().expect(\"thread 1 write should not panic\");\n t2.join().expect(\"thread 2 write should not panic\");\n t3.join().expect(\"thread 3 read should not panic\");\n\n let guard = block_on(rwlock.read());\n //when all threads have finished the value on the lock should be 10\n assert_eq!(10, *guard);\n });\n}\n#[test]\nfn downgrade() {\n loom::model(|| {\n let lock = Arc::new(RwLock::new(1));\n\n let n = block_on(lock.write());\n\n let cloned_lock = lock.clone();\n let handle = thread::spawn(move || {\n let mut guard = block_on(cloned_lock.write());\n *guard = 2;\n });\n\n let n = n.downgrade();\n assert_eq!(*n, 1);\n\n drop(n);\n handle.join().unwrap();\n assert_eq!(*block_on(lock.read()), 2);\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_semaphore_batch.rs", "content": "use crate::sync::batch_semaphore::*;\n\nuse loom::future::block_on;\nuse loom::sync::atomic::AtomicUsize;\nuse loom::thread;\nuse std::future::{poll_fn, Future};\nuse std::pin::Pin;\nuse std::sync::atomic::Ordering::SeqCst;\nuse std::sync::Arc;\nuse std::task::Poll::Ready;\nuse std::task::{Context, Poll};\n\n#[test]\nfn basic_usage() {\n const NUM: usize = 2;\n\n struct Shared {\n semaphore: Semaphore,\n active: AtomicUsize,\n }\n\n async fn actor(shared: Arc) {\n shared.semaphore.acquire(1).await.unwrap();\n let actual = shared.active.fetch_add(1, SeqCst);\n assert!(actual <= NUM - 1);\n\n let actual = shared.active.fetch_sub(1, SeqCst);\n assert!(actual <= NUM);\n shared.semaphore.release(1);\n }\n\n loom::model(|| {\n let shared = Arc::new(Shared {\n semaphore: Semaphore::new(NUM),\n active: AtomicUsize::new(0),\n });\n\n for _ in 0..NUM {\n let shared = shared.clone();\n\n thread::spawn(move || {\n block_on(actor(shared));\n });\n }\n\n block_on(actor(shared));\n });\n}\n\n#[test]\nfn release() {\n loom::model(|| {\n let semaphore = Arc::new(Semaphore::new(1));\n\n {\n let semaphore = semaphore.clone();\n thread::spawn(move || {\n block_on(semaphore.acquire(1)).unwrap();\n semaphore.release(1);\n });\n }\n\n block_on(semaphore.acquire(1)).unwrap();\n\n semaphore.release(1);\n });\n}\n\n#[test]\nfn basic_closing() {\n const NUM: usize = 2;\n\n loom::model(|| {\n let semaphore = Arc::new(Semaphore::new(1));\n\n for _ in 0..NUM {\n let semaphore = semaphore.clone();\n\n thread::spawn(move || {\n for _ in 0..2 {\n block_on(semaphore.acquire(1)).map_err(|_| ())?;\n\n semaphore.release(1);\n }\n\n Ok::<(), ()>(())\n });\n }\n\n semaphore.close();\n });\n}\n\n#[test]\nfn concurrent_close() {\n const NUM: usize = 3;\n\n loom::model(|| {\n let semaphore = Arc::new(Semaphore::new(1));\n\n for _ in 0..NUM {\n let semaphore = semaphore.clone();\n\n thread::spawn(move || {\n block_on(semaphore.acquire(1)).map_err(|_| ())?;\n semaphore.release(1);\n semaphore.close();\n\n Ok::<(), ()>(())\n });\n }\n });\n}\n\n#[test]\nfn concurrent_cancel() {\n async fn poll_and_cancel(semaphore: Arc) {\n let mut acquire1 = Some(semaphore.acquire(1));\n let mut acquire2 = Some(semaphore.acquire(1));\n poll_fn(|cx| {\n // poll the acquire future once, and then immediately throw\n // it away. this simulates a situation where a future is\n // polled and then cancelled, such as by a timeout.\n if let Some(acquire) = acquire1.take() {\n pin!(acquire);\n let _ = acquire.poll(cx);\n }\n if let Some(acquire) = acquire2.take() {\n pin!(acquire);\n let _ = acquire.poll(cx);\n }\n Poll::Ready(())\n })\n .await\n }\n\n loom::model(|| {\n let semaphore = Arc::new(Semaphore::new(0));\n let t1 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || block_on(poll_and_cancel(semaphore)))\n };\n let t2 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || block_on(poll_and_cancel(semaphore)))\n };\n let t3 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || block_on(poll_and_cancel(semaphore)))\n };\n\n t1.join().unwrap();\n semaphore.release(10);\n t2.join().unwrap();\n t3.join().unwrap();\n });\n}\n\n#[test]\nfn batch() {\n let mut b = loom::model::Builder::new();\n b.preemption_bound = Some(1);\n\n b.check(|| {\n let semaphore = Arc::new(Semaphore::new(10));\n let active = Arc::new(AtomicUsize::new(0));\n let mut threads = vec![];\n\n for _ in 0..2 {\n let semaphore = semaphore.clone();\n let active = active.clone();\n\n threads.push(thread::spawn(move || {\n for n in &[4, 10, 8] {\n block_on(semaphore.acquire(*n)).unwrap();\n\n active.fetch_add(*n as usize, SeqCst);\n\n let num_active = active.load(SeqCst);\n assert!(num_active <= 10);\n\n thread::yield_now();\n\n active.fetch_sub(*n as usize, SeqCst);\n\n semaphore.release(*n as usize);\n }\n }));\n }\n\n for thread in threads.into_iter() {\n thread.join().unwrap();\n }\n\n assert_eq!(10, semaphore.available_permits());\n });\n}\n\n#[test]\nfn release_during_acquire() {\n loom::model(|| {\n let semaphore = Arc::new(Semaphore::new(10));\n semaphore\n .try_acquire(8)\n .expect(\"try_acquire should succeed; semaphore uncontended\");\n let semaphore2 = semaphore.clone();\n let thread = thread::spawn(move || block_on(semaphore2.acquire(4)).unwrap());\n\n semaphore.release(8);\n thread.join().unwrap();\n semaphore.release(4);\n assert_eq!(10, semaphore.available_permits());\n })\n}\n\n#[test]\nfn concurrent_permit_updates() {\n loom::model(move || {\n let semaphore = Arc::new(Semaphore::new(5));\n let t1 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || semaphore.release(3))\n };\n let t2 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || {\n semaphore\n .try_acquire(1)\n .expect(\"try_acquire should succeed\")\n })\n };\n let t3 = {\n let semaphore = semaphore.clone();\n thread::spawn(move || semaphore.forget_permits(2))\n };\n\n t1.join().unwrap();\n t2.join().unwrap();\n t3.join().unwrap();\n assert_eq!(semaphore.available_permits(), 5);\n })\n}\n" }, { "path": "tokio/src/sync/tests/loom_set_once.rs", "content": "use crate::sync::SetOnce;\n\nuse loom::future::block_on;\nuse loom::sync::atomic::AtomicU32;\nuse loom::thread;\nuse std::sync::atomic::Ordering;\nuse std::sync::Arc;\n\n#[derive(Clone)]\nstruct DropCounter {\n pub drops: Arc,\n}\n\nimpl DropCounter {\n pub fn new() -> Self {\n Self {\n drops: Arc::new(AtomicU32::new(0)),\n }\n }\n\n fn assert_num_drops(&self, value: u32) {\n assert_eq!(value, self.drops.load(Ordering::Relaxed));\n }\n}\n\nimpl Drop for DropCounter {\n fn drop(&mut self) {\n self.drops.fetch_add(1, Ordering::Relaxed);\n }\n}\n\n#[test]\nfn set_once_drop_test() {\n loom::model(|| {\n let set_once = Arc::new(SetOnce::new());\n let set_once_clone = Arc::clone(&set_once);\n\n let drop_counter = DropCounter::new();\n let counter_cl = drop_counter.clone();\n\n let thread = thread::spawn(move || set_once_clone.set(counter_cl).is_ok());\n\n let foo = drop_counter.clone();\n\n let set = set_once.set(foo).is_ok();\n let res = thread.join().unwrap();\n\n drop(set_once);\n\n drop_counter.assert_num_drops(2);\n assert!(res != set);\n });\n}\n\n#[test]\nfn set_once_wait_test() {\n loom::model(|| {\n let tx = Arc::new(SetOnce::new());\n let rx_one = tx.clone();\n let rx_two = tx.clone();\n\n let thread = thread::spawn(move || {\n assert!(rx_one.set(2).is_ok());\n });\n\n block_on(async {\n assert_eq!(*rx_two.wait().await, 2);\n });\n\n thread.join().unwrap();\n });\n}\n" }, { "path": "tokio/src/sync/tests/loom_watch.rs", "content": "use crate::sync::watch;\n\nuse loom::future::block_on;\nuse loom::thread;\nuse std::sync::Arc;\n\n#[test]\nfn smoke() {\n loom::model(|| {\n let (tx, mut rx1) = watch::channel(1);\n let mut rx2 = rx1.clone();\n let mut rx3 = rx1.clone();\n let mut rx4 = rx1.clone();\n let mut rx5 = rx1.clone();\n\n let th = thread::spawn(move || {\n tx.send(2).unwrap();\n });\n\n block_on(rx1.changed()).unwrap();\n assert_eq!(*rx1.borrow(), 2);\n\n block_on(rx2.changed()).unwrap();\n assert_eq!(*rx2.borrow(), 2);\n\n block_on(rx3.changed()).unwrap();\n assert_eq!(*rx3.borrow(), 2);\n\n block_on(rx4.changed()).unwrap();\n assert_eq!(*rx4.borrow(), 2);\n\n block_on(rx5.changed()).unwrap();\n assert_eq!(*rx5.borrow(), 2);\n\n th.join().unwrap();\n })\n}\n\n#[test]\nfn wait_for_test() {\n loom::model(move || {\n let (tx, mut rx) = watch::channel(false);\n\n let tx_arc = Arc::new(tx);\n let tx1 = tx_arc.clone();\n let tx2 = tx_arc.clone();\n\n let th1 = thread::spawn(move || {\n for _ in 0..2 {\n tx1.send_modify(|_x| {});\n }\n });\n\n let th2 = thread::spawn(move || {\n tx2.send(true).unwrap();\n });\n\n assert_eq!(*block_on(rx.wait_for(|x| *x)).unwrap(), true);\n\n th1.join().unwrap();\n th2.join().unwrap();\n });\n}\n\n#[test]\nfn wait_for_returns_correct_value() {\n loom::model(move || {\n let (tx, mut rx) = watch::channel(0);\n\n let jh = thread::spawn(move || {\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n tx.send(3).unwrap();\n });\n\n // Stop at the first value we are called at.\n let mut stopped_at = usize::MAX;\n let returned = *block_on(rx.wait_for(|x| {\n stopped_at = *x;\n true\n }))\n .unwrap();\n\n // Check that it returned the same value as the one we returned\n // `true` for.\n assert_eq!(stopped_at, returned);\n\n jh.join().unwrap();\n });\n}\n\n#[test]\nfn multiple_sender_drop_concurrently() {\n loom::model(move || {\n let (tx1, rx) = watch::channel(0);\n let tx2 = tx1.clone();\n\n let jh = thread::spawn(move || {\n drop(tx2);\n });\n assert!(rx.has_changed().is_ok());\n\n drop(tx1);\n\n jh.join().unwrap();\n\n // Check if all sender are dropped and closed flag is set.\n assert!(rx.has_changed().is_err());\n });\n}\n" }, { "path": "tokio/src/sync/tests/mod.rs", "content": "cfg_not_loom! {\n mod atomic_waker;\n mod notify;\n mod semaphore_batch;\n}\n\ncfg_loom! {\n mod loom_atomic_waker;\n mod loom_broadcast;\n mod loom_list;\n mod loom_mpsc;\n mod loom_notify;\n mod loom_oneshot;\n mod loom_semaphore_batch;\n mod loom_watch;\n mod loom_rwlock;\n mod loom_set_once;\n}\n" }, { "path": "tokio/src/sync/tests/notify.rs", "content": "use crate::sync::Notify;\nuse std::future::Future;\nuse std::sync::Arc;\nuse std::task::{Context, RawWaker, RawWakerVTable, Waker};\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\n#[test]\nfn notify_clones_waker_before_lock() {\n const VTABLE: &RawWakerVTable = &RawWakerVTable::new(clone_w, wake, wake_by_ref, drop_w);\n\n unsafe fn clone_w(data: *const ()) -> RawWaker {\n let ptr = data as *const Notify;\n unsafe {\n Arc::::increment_strong_count(ptr);\n }\n // Or some other arbitrary code that shouldn't be executed while the\n // Notify wait list is locked.\n unsafe { (*ptr).notify_one() };\n RawWaker::new(data, VTABLE)\n }\n\n unsafe fn drop_w(data: *const ()) {\n drop(unsafe { Arc::::from_raw(data as *const Notify) });\n }\n\n unsafe fn wake(_data: *const ()) {\n unreachable!()\n }\n\n unsafe fn wake_by_ref(_data: *const ()) {\n unreachable!()\n }\n\n let notify = Arc::new(Notify::new());\n let notify2 = notify.clone();\n\n let waker =\n unsafe { Waker::from_raw(RawWaker::new(Arc::into_raw(notify2) as *const _, VTABLE)) };\n let mut cx = Context::from_waker(&waker);\n\n let future = notify.notified();\n pin!(future);\n\n // The result doesn't matter, we're just testing that we don't deadlock.\n let _ = future.poll(&mut cx);\n}\n\n#[cfg(panic = \"unwind\")]\n#[test]\nfn notify_waiters_handles_panicking_waker() {\n use futures::task::ArcWake;\n\n let notify = Arc::new(Notify::new());\n\n struct PanickingWaker(#[allow(dead_code)] Arc);\n\n impl ArcWake for PanickingWaker {\n fn wake_by_ref(_arc_self: &Arc) {\n panic!(\"waker panicked\");\n }\n }\n\n let bad_fut = notify.notified();\n pin!(bad_fut);\n\n let waker = futures::task::waker(Arc::new(PanickingWaker(notify.clone())));\n let mut cx = Context::from_waker(&waker);\n let _ = bad_fut.poll(&mut cx);\n\n let mut futs = Vec::new();\n for _ in 0..32 {\n let mut fut = tokio_test::task::spawn(notify.notified());\n assert!(fut.poll().is_pending());\n futs.push(fut);\n }\n\n assert!(std::panic::catch_unwind(|| {\n notify.notify_waiters();\n })\n .is_err());\n\n for mut fut in futs {\n assert!(fut.poll().is_ready());\n }\n}\n\n#[test]\nfn notify_simple() {\n let notify = Notify::new();\n\n let mut fut1 = tokio_test::task::spawn(notify.notified());\n assert!(fut1.poll().is_pending());\n\n let mut fut2 = tokio_test::task::spawn(notify.notified());\n assert!(fut2.poll().is_pending());\n\n notify.notify_waiters();\n\n assert!(fut1.poll().is_ready());\n assert!(fut2.poll().is_ready());\n}\n\n#[test]\n#[cfg(not(target_family = \"wasm\"))]\nfn watch_test() {\n let rt = crate::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n rt.block_on(async {\n let (tx, mut rx) = crate::sync::watch::channel(());\n\n crate::spawn(async move {\n let _ = tx.send(());\n });\n\n let _ = rx.changed().await;\n });\n}\n" }, { "path": "tokio/src/sync/tests/semaphore_batch.rs", "content": "use crate::sync::batch_semaphore::Semaphore;\nuse tokio_test::*;\n\nconst MAX_PERMITS: usize = crate::sync::Semaphore::MAX_PERMITS;\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\n#[test]\nfn poll_acquire_one_available() {\n let s = Semaphore::new(100);\n assert_eq!(s.available_permits(), 100);\n\n // Polling for a permit succeeds immediately\n assert_ready_ok!(task::spawn(s.acquire(1)).poll());\n assert_eq!(s.available_permits(), 99);\n}\n\n#[test]\nfn poll_acquire_many_available() {\n let s = Semaphore::new(100);\n assert_eq!(s.available_permits(), 100);\n\n // Polling for a permit succeeds immediately\n assert_ready_ok!(task::spawn(s.acquire(5)).poll());\n assert_eq!(s.available_permits(), 95);\n\n assert_ready_ok!(task::spawn(s.acquire(5)).poll());\n assert_eq!(s.available_permits(), 90);\n}\n\n#[test]\nfn try_acquire_one_available() {\n let s = Semaphore::new(100);\n assert_eq!(s.available_permits(), 100);\n\n assert_ok!(s.try_acquire(1));\n assert_eq!(s.available_permits(), 99);\n\n assert_ok!(s.try_acquire(1));\n assert_eq!(s.available_permits(), 98);\n}\n\n#[test]\nfn try_acquire_many_available() {\n let s = Semaphore::new(100);\n assert_eq!(s.available_permits(), 100);\n\n assert_ok!(s.try_acquire(5));\n assert_eq!(s.available_permits(), 95);\n\n assert_ok!(s.try_acquire(5));\n assert_eq!(s.available_permits(), 90);\n}\n\n#[test]\nfn poll_acquire_one_unavailable() {\n let s = Semaphore::new(1);\n\n // Acquire the first permit\n assert_ready_ok!(task::spawn(s.acquire(1)).poll());\n assert_eq!(s.available_permits(), 0);\n\n let mut acquire_2 = task::spawn(s.acquire(1));\n // Try to acquire the second permit\n assert_pending!(acquire_2.poll());\n assert_eq!(s.available_permits(), 0);\n\n s.release(1);\n\n assert_eq!(s.available_permits(), 0);\n assert!(acquire_2.is_woken());\n assert_ready_ok!(acquire_2.poll());\n assert_eq!(s.available_permits(), 0);\n\n s.release(1);\n assert_eq!(s.available_permits(), 1);\n}\n\n#[test]\nfn poll_acquire_many_unavailable() {\n let s = Semaphore::new(5);\n\n // Acquire the first permit\n assert_ready_ok!(task::spawn(s.acquire(1)).poll());\n assert_eq!(s.available_permits(), 4);\n\n // Try to acquire the second permit\n let mut acquire_2 = task::spawn(s.acquire(5));\n assert_pending!(acquire_2.poll());\n assert_eq!(s.available_permits(), 0);\n\n // Try to acquire the third permit\n let mut acquire_3 = task::spawn(s.acquire(3));\n assert_pending!(acquire_3.poll());\n assert_eq!(s.available_permits(), 0);\n\n s.release(1);\n\n assert_eq!(s.available_permits(), 0);\n assert!(acquire_2.is_woken());\n assert_ready_ok!(acquire_2.poll());\n\n assert!(!acquire_3.is_woken());\n assert_eq!(s.available_permits(), 0);\n\n s.release(1);\n assert!(!acquire_3.is_woken());\n assert_eq!(s.available_permits(), 0);\n\n s.release(2);\n assert!(acquire_3.is_woken());\n\n assert_ready_ok!(acquire_3.poll());\n}\n\n#[test]\nfn try_acquire_one_unavailable() {\n let s = Semaphore::new(1);\n\n // Acquire the first permit\n assert_ok!(s.try_acquire(1));\n assert_eq!(s.available_permits(), 0);\n\n assert_err!(s.try_acquire(1));\n\n s.release(1);\n\n assert_eq!(s.available_permits(), 1);\n assert_ok!(s.try_acquire(1));\n\n s.release(1);\n assert_eq!(s.available_permits(), 1);\n}\n\n#[test]\nfn try_acquire_many_unavailable() {\n let s = Semaphore::new(5);\n\n // Acquire the first permit\n assert_ok!(s.try_acquire(1));\n assert_eq!(s.available_permits(), 4);\n\n assert_err!(s.try_acquire(5));\n\n s.release(1);\n assert_eq!(s.available_permits(), 5);\n\n assert_ok!(s.try_acquire(5));\n\n s.release(1);\n assert_eq!(s.available_permits(), 1);\n\n s.release(1);\n assert_eq!(s.available_permits(), 2);\n}\n\n#[test]\nfn poll_acquire_one_zero_permits() {\n let s = Semaphore::new(0);\n assert_eq!(s.available_permits(), 0);\n\n // Try to acquire the permit\n let mut acquire = task::spawn(s.acquire(1));\n assert_pending!(acquire.poll());\n\n s.release(1);\n\n assert!(acquire.is_woken());\n assert_ready_ok!(acquire.poll());\n}\n\n#[test]\nfn max_permits_doesnt_panic() {\n Semaphore::new(MAX_PERMITS);\n}\n\n#[test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn validates_max_permits() {\n Semaphore::new(MAX_PERMITS + 1);\n}\n\n#[test]\nfn close_semaphore_prevents_acquire() {\n let s = Semaphore::new(5);\n s.close();\n\n assert_eq!(5, s.available_permits());\n\n assert_ready_err!(task::spawn(s.acquire(1)).poll());\n assert_eq!(5, s.available_permits());\n\n assert_ready_err!(task::spawn(s.acquire(1)).poll());\n assert_eq!(5, s.available_permits());\n}\n\n#[test]\nfn close_semaphore_notifies_permit1() {\n let s = Semaphore::new(0);\n let mut acquire = task::spawn(s.acquire(1));\n\n assert_pending!(acquire.poll());\n\n s.close();\n\n assert!(acquire.is_woken());\n assert_ready_err!(acquire.poll());\n}\n\n#[test]\nfn close_semaphore_notifies_permit2() {\n let s = Semaphore::new(2);\n\n // Acquire a couple of permits\n assert_ready_ok!(task::spawn(s.acquire(1)).poll());\n assert_ready_ok!(task::spawn(s.acquire(1)).poll());\n\n let mut acquire3 = task::spawn(s.acquire(1));\n let mut acquire4 = task::spawn(s.acquire(1));\n assert_pending!(acquire3.poll());\n assert_pending!(acquire4.poll());\n\n s.close();\n\n assert!(acquire3.is_woken());\n assert!(acquire4.is_woken());\n\n assert_ready_err!(acquire3.poll());\n assert_ready_err!(acquire4.poll());\n\n assert_eq!(0, s.available_permits());\n\n s.release(1);\n\n assert_eq!(1, s.available_permits());\n\n assert_ready_err!(task::spawn(s.acquire(1)).poll());\n\n s.release(1);\n\n assert_eq!(2, s.available_permits());\n}\n\n#[test]\nfn cancel_acquire_releases_permits() {\n let s = Semaphore::new(10);\n s.try_acquire(4).expect(\"uncontended try_acquire succeeds\");\n assert_eq!(6, s.available_permits());\n\n let mut acquire = task::spawn(s.acquire(8));\n assert_pending!(acquire.poll());\n\n assert_eq!(0, s.available_permits());\n drop(acquire);\n\n assert_eq!(6, s.available_permits());\n assert_ok!(s.try_acquire(6));\n}\n\n#[test]\nfn release_permits_at_drop() {\n use crate::sync::semaphore::*;\n use futures::task::ArcWake;\n use std::future::Future;\n use std::sync::Arc;\n\n let sem = Arc::new(Semaphore::new(1));\n\n struct ReleaseOnDrop(#[allow(dead_code)] Option);\n\n impl ArcWake for ReleaseOnDrop {\n fn wake_by_ref(_arc_self: &Arc) {}\n }\n\n let mut fut = Box::pin(async {\n let _permit = sem.acquire().await.unwrap();\n });\n\n // Second iteration shouldn't deadlock.\n for _ in 0..=1 {\n let waker = futures::task::waker(Arc::new(ReleaseOnDrop(\n sem.clone().try_acquire_owned().ok(),\n )));\n let mut cx = std::task::Context::from_waker(&waker);\n assert!(fut.as_mut().poll(&mut cx).is_pending());\n }\n}\n\n#[test]\nfn forget_permits_basic() {\n let s = Semaphore::new(10);\n assert_eq!(s.forget_permits(4), 4);\n assert_eq!(s.available_permits(), 6);\n assert_eq!(s.forget_permits(10), 6);\n assert_eq!(s.available_permits(), 0);\n}\n\n#[test]\nfn update_permits_many_times() {\n let s = Semaphore::new(5);\n let mut acquire = task::spawn(s.acquire(7));\n assert_pending!(acquire.poll());\n s.release(5);\n assert_ready_ok!(acquire.poll());\n assert_eq!(s.available_permits(), 3);\n assert_eq!(s.forget_permits(3), 3);\n assert_eq!(s.available_permits(), 0);\n}\n" }, { "path": "tokio/src/sync/watch.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(dead_code, unreachable_pub))]\n\n//! A multi-producer, multi-consumer channel that only retains the *last* sent\n//! value.\n//!\n//! This channel is useful for watching for changes to a value from multiple\n//! points in the code base, for example, changes to configuration values.\n//!\n//! # Usage\n//!\n//! [`channel`] returns a [`Sender`] / [`Receiver`] pair. These are the producer\n//! and consumer halves of the channel. The channel is created with an initial\n//! value.\n//!\n//! Each [`Receiver`] independently tracks the last value *seen* by its caller.\n//!\n//! To access the **current** value stored in the channel and mark it as *seen*\n//! by a given [`Receiver`], use [`Receiver::borrow_and_update()`].\n//!\n//! To access the current value **without** marking it as *seen*, use\n//! [`Receiver::borrow()`]. (If the value has already been marked *seen*,\n//! [`Receiver::borrow()`] is equivalent to [`Receiver::borrow_and_update()`].)\n//!\n//! For more information on when to use these methods, see\n//! [here](#borrow_and_update-versus-borrow).\n//!\n//! ## Change notifications\n//!\n//! The [`Receiver`] half provides an asynchronous [`changed`] method. This\n//! method is ready when a new, *unseen* value is sent via the [`Sender`] half.\n//!\n//! * [`Receiver::changed()`] returns:\n//! * `Ok(())` on receiving a new value.\n//! * `Err(`[`RecvError`](error::RecvError)`)` if the\n//! channel has been closed __AND__ the current value is *seen*.\n//! * If the current value is *unseen* when calling [`changed`], then\n//! [`changed`] will return immediately. If the current value is *seen*, then\n//! it will sleep until either a new message is sent via the [`Sender`] half,\n//! or the [`Sender`] is dropped.\n//! * On completion, the [`changed`] method marks the new value as *seen*.\n//! * At creation, the initial value is considered *seen*. In other words,\n//! [`Receiver::changed()`] will not return until a subsequent value is sent.\n//! * New [`Receiver`] instances can be created with [`Sender::subscribe()`].\n//! The current value at the time the [`Receiver`] is created is considered\n//! *seen*.\n//!\n//! ## [`changed`] versus [`has_changed`]\n//!\n//! The [`Receiver`] half provides two methods for checking for changes\n//! in the channel, [`has_changed`] and [`changed`].\n//!\n//! * [`has_changed`] is a *synchronous* method that checks whether the current\n//! value is seen or not and returns a boolean. This method does __not__ mark the\n//! value as seen.\n//!\n//! * [`changed`] is an *asynchronous* method that will return once an unseen\n//! value is in the channel. This method does mark the value as seen.\n//!\n//! Note there are two behavioral differences on when these two methods return\n//! an error.\n//!\n//! - [`has_changed`] errors if and only if the channel is closed.\n//! - [`changed`] errors if the channel has been closed __AND__\n//! the current value is seen.\n//!\n//! See the example below that shows how these methods have different fallibility.\n//!\n//! ## [`borrow_and_update`] versus [`borrow`]\n//!\n//! If the receiver intends to await notifications from [`changed`] in a loop,\n//! [`Receiver::borrow_and_update()`] should be preferred over\n//! [`Receiver::borrow()`]. This avoids a potential race where a new value is\n//! sent between [`changed`] being ready and the value being read. (If\n//! [`Receiver::borrow()`] is used, the loop may run twice with the same value.)\n//!\n//! If the receiver is only interested in the current value, and does not intend\n//! to wait for changes, then [`Receiver::borrow()`] can be used. It may be more\n//! convenient to use [`borrow`](Receiver::borrow) since it's an `&self`\n//! method---[`borrow_and_update`](Receiver::borrow_and_update) requires `&mut\n//! self`.\n//!\n//! # Examples\n//!\n//! The following example prints `hello! world! `.\n//!\n//! ```\n//! use tokio::sync::watch;\n//! use tokio::time::{Duration, sleep};\n//!\n//! # async fn dox() -> Result<(), Box> {\n//! let (tx, mut rx) = watch::channel(\"hello\");\n//!\n//! tokio::spawn(async move {\n//! // Use the equivalent of a \"do-while\" loop so the initial value is\n//! // processed before awaiting the `changed()` future.\n//! loop {\n//! println!(\"{}! \", *rx.borrow_and_update());\n//! if rx.changed().await.is_err() {\n//! break;\n//! }\n//! }\n//! });\n//!\n//! sleep(Duration::from_millis(100)).await;\n//! tx.send(\"world\")?;\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! Difference on fallibility of [`changed`] versus [`has_changed`].\n//! ```\n//! use tokio::sync::watch;\n//!\n//! #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let (tx, mut rx) = watch::channel(\"hello\");\n//! tx.send(\"goodbye\").unwrap();\n//! drop(tx);\n//!\n//! // `has_changed` does not mark the value as seen and errors\n//! // since the channel is closed.\n//! assert!(rx.has_changed().is_err());\n//!\n//! // `changed` returns Ok since the value is not already marked as seen\n//! // even if the channel is closed.\n//! assert!(rx.changed().await.is_ok());\n//!\n//! // The `changed` call above marks the value as seen.\n//! // The next `changed` call now returns an error as the channel is closed\n//! // AND the current value is seen.\n//! assert!(rx.changed().await.is_err());\n//! # }\n//! ```\n//!\n//! # Closing\n//!\n//! [`Sender::is_closed`] and [`Sender::closed`] allow the producer to detect\n//! when all [`Receiver`] handles have been dropped. This indicates that there\n//! is no further interest in the values being produced and work can be stopped.\n//!\n//! The value in the channel will not be dropped until all senders and all\n//! receivers have been dropped.\n//!\n//! # Thread safety\n//!\n//! Both [`Sender`] and [`Receiver`] are thread safe. They can be moved to other\n//! threads and can be used in a concurrent environment. Clones of [`Receiver`]\n//! handles may be moved to separate threads and also used concurrently.\n//!\n//! [`Sender`]: crate::sync::watch::Sender\n//! [`Receiver`]: crate::sync::watch::Receiver\n//! [`changed`]: crate::sync::watch::Receiver::changed\n//! [`has_changed`]: crate::sync::watch::Receiver::has_changed\n//! [`borrow`]: crate::sync::watch::Receiver::borrow\n//! [`borrow_and_update`]: crate::sync::watch::Receiver::borrow_and_update\n//! [`Receiver::changed()`]: crate::sync::watch::Receiver::changed\n//! [`Receiver::borrow()`]: crate::sync::watch::Receiver::borrow\n//! [`Receiver::borrow_and_update()`]:\n//! crate::sync::watch::Receiver::borrow_and_update\n//! [`channel`]: crate::sync::watch::channel\n//! [`Sender::is_closed`]: crate::sync::watch::Sender::is_closed\n//! [`Sender::closed`]: crate::sync::watch::Sender::closed\n//! [`Sender::subscribe()`]: crate::sync::watch::Sender::subscribe\n\nuse crate::sync::notify::Notify;\nuse crate::task::coop::cooperative;\n\nuse crate::loom::sync::atomic::AtomicUsize;\nuse crate::loom::sync::atomic::Ordering::{AcqRel, Relaxed};\nuse crate::loom::sync::{Arc, RwLock, RwLockReadGuard};\nuse std::fmt;\nuse std::mem;\nuse std::ops;\nuse std::panic;\n\n/// Receives values from the associated [`Sender`](struct@Sender).\n///\n/// Instances are created by the [`channel`](fn@channel) function.\n///\n/// To turn this receiver into a `Stream`, you can use the [`WatchStream`]\n/// wrapper.\n///\n/// [`WatchStream`]: https://docs.rs/tokio-stream/0.1/tokio_stream/wrappers/struct.WatchStream.html\n#[derive(Debug)]\npub struct Receiver {\n /// Pointer to the shared state\n shared: Arc>,\n\n /// Last observed version\n version: Version,\n}\n\n/// Sends values to the associated [`Receiver`](struct@Receiver).\n///\n/// Instances are created by the [`channel`](fn@channel) function.\n#[derive(Debug)]\npub struct Sender {\n shared: Arc>,\n}\n\nimpl Clone for Sender {\n fn clone(&self) -> Self {\n self.shared.ref_count_tx.fetch_add(1, Relaxed);\n\n Self {\n shared: self.shared.clone(),\n }\n }\n}\n\nimpl Default for Sender {\n fn default() -> Self {\n Self::new(T::default())\n }\n}\n\n/// Returns a reference to the inner value.\n///\n/// Outstanding borrows hold a read lock on the inner value. This means that\n/// long-lived borrows could cause the producer half to block. It is recommended\n/// to keep the borrow as short-lived as possible. Additionally, if you are\n/// running in an environment that allows `!Send` futures, you must ensure that\n/// the returned `Ref` type is never held alive across an `.await` point,\n/// otherwise, it can lead to a deadlock.\n///\n/// The priority policy of the lock is dependent on the underlying lock\n/// implementation, and this type does not guarantee that any particular policy\n/// will be used. In particular, a producer which is waiting to acquire the lock\n/// in `send` might or might not block concurrent calls to `borrow`, e.g.:\n///\n///
Potential deadlock example\n///\n/// ```text\n/// // Task 1 (on thread A) | // Task 2 (on thread B)\n/// let _ref1 = rx.borrow(); |\n/// | // will block\n/// | let _ = tx.send(());\n/// // may deadlock |\n/// let _ref2 = rx.borrow(); |\n/// ```\n///
\n#[derive(Debug)]\npub struct Ref<'a, T> {\n inner: RwLockReadGuard<'a, T>,\n has_changed: bool,\n}\n\nimpl<'a, T> Ref<'a, T> {\n /// Indicates if the borrowed value is considered as _changed_ since the last\n /// time it has been marked as seen.\n ///\n /// Unlike [`Receiver::has_changed()`], this method does not fail if the channel is closed.\n ///\n /// When borrowed from the [`Sender`] this function will always return `false`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = watch::channel(\"hello\");\n ///\n /// tx.send(\"goodbye\").unwrap();\n /// // The sender does never consider the value as changed.\n /// assert!(!tx.borrow().has_changed());\n ///\n /// // Drop the sender immediately, just for testing purposes.\n /// drop(tx);\n ///\n /// // Even if the sender has already been dropped...\n /// assert!(rx.has_changed().is_err());\n /// // ...the modified value is still readable and detected as changed.\n /// assert_eq!(*rx.borrow(), \"goodbye\");\n /// assert!(rx.borrow().has_changed());\n ///\n /// // Read the changed value and mark it as seen.\n /// {\n /// let received = rx.borrow_and_update();\n /// assert_eq!(*received, \"goodbye\");\n /// assert!(received.has_changed());\n /// // Release the read lock when leaving this scope.\n /// }\n ///\n /// // Now the value has already been marked as seen and could\n /// // never be modified again (after the sender has been dropped).\n /// assert!(!rx.borrow().has_changed());\n /// # }\n /// ```\n pub fn has_changed(&self) -> bool {\n self.has_changed\n }\n}\n\nstruct Shared {\n /// The most recent value.\n value: RwLock,\n\n /// The current version.\n ///\n /// The lowest bit represents a \"closed\" state. The rest of the bits\n /// represent the current version.\n state: AtomicState,\n\n /// Tracks the number of `Receiver` instances.\n ref_count_rx: AtomicUsize,\n\n /// Tracks the number of `Sender` instances.\n ref_count_tx: AtomicUsize,\n\n /// Notifies waiting receivers that the value changed.\n notify_rx: big_notify::BigNotify,\n\n /// Notifies any task listening for `Receiver` dropped events.\n notify_tx: Notify,\n}\n\nimpl fmt::Debug for Shared {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n let state = self.state.load();\n f.debug_struct(\"Shared\")\n .field(\"value\", &self.value)\n .field(\"version\", &state.version())\n .field(\"is_closed\", &state.is_closed())\n .field(\"ref_count_rx\", &self.ref_count_rx)\n .finish()\n }\n}\n\npub mod error {\n //! Watch error types.\n\n use std::error::Error;\n use std::fmt;\n\n /// Error produced when sending a value fails.\n #[derive(PartialEq, Eq, Clone, Copy)]\n pub struct SendError(pub T);\n\n // ===== impl SendError =====\n\n impl fmt::Debug for SendError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"SendError\").finish_non_exhaustive()\n }\n }\n\n impl fmt::Display for SendError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n }\n\n impl Error for SendError {}\n\n /// Error produced when receiving a change notification.\n #[derive(Debug, Clone)]\n pub struct RecvError(pub(super) ());\n\n // ===== impl RecvError =====\n\n impl fmt::Display for RecvError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n }\n\n impl Error for RecvError {}\n}\n\nmod big_notify {\n use super::Notify;\n use crate::sync::notify::Notified;\n\n // To avoid contention on the lock inside the `Notify`, we store multiple\n // copies of it. Then, we use either circular access or randomness to spread\n // out threads over different `Notify` objects.\n //\n // Some simple benchmarks show that randomness performs slightly better than\n // circular access (probably due to contention on `next`), so we prefer to\n // use randomness when Tokio is compiled with a random number generator.\n //\n // When the random number generator is not available, we fall back to\n // circular access.\n\n pub(super) struct BigNotify {\n #[cfg(not(all(not(loom), feature = \"sync\", any(feature = \"rt\", feature = \"macros\"))))]\n next: std::sync::atomic::AtomicUsize,\n inner: [Notify; 8],\n }\n\n impl BigNotify {\n pub(super) fn new() -> Self {\n Self {\n #[cfg(not(all(\n not(loom),\n feature = \"sync\",\n any(feature = \"rt\", feature = \"macros\")\n )))]\n next: std::sync::atomic::AtomicUsize::new(0),\n inner: Default::default(),\n }\n }\n\n pub(super) fn notify_waiters(&self) {\n for notify in &self.inner {\n notify.notify_waiters();\n }\n }\n\n /// This function implements the case where randomness is not available.\n #[cfg(not(all(not(loom), feature = \"sync\", any(feature = \"rt\", feature = \"macros\"))))]\n pub(super) fn notified(&self) -> Notified<'_> {\n let i = self.next.fetch_add(1, std::sync::atomic::Ordering::Relaxed) % 8;\n self.inner[i].notified()\n }\n\n /// This function implements the case where randomness is available.\n #[cfg(all(not(loom), feature = \"sync\", any(feature = \"rt\", feature = \"macros\")))]\n pub(super) fn notified(&self) -> Notified<'_> {\n let i = crate::runtime::context::thread_rng_n(8) as usize;\n self.inner[i].notified()\n }\n }\n}\n\nuse self::state::{AtomicState, Version};\nmod state {\n use crate::loom::sync::atomic::AtomicUsize;\n use crate::loom::sync::atomic::Ordering;\n\n const CLOSED_BIT: usize = 1;\n\n // Using 2 as the step size preserves the `CLOSED_BIT`.\n const STEP_SIZE: usize = 2;\n\n /// The version part of the state. The lowest bit is always zero.\n #[derive(Copy, Clone, Debug, Eq, PartialEq)]\n pub(super) struct Version(usize);\n\n /// Snapshot of the state. The first bit is used as the CLOSED bit.\n /// The remaining bits are used as the version.\n ///\n /// The CLOSED bit tracks whether all senders have been dropped. Dropping all\n /// receivers does not set it.\n #[derive(Copy, Clone, Debug)]\n pub(super) struct StateSnapshot(usize);\n\n /// The state stored in an atomic integer.\n ///\n /// The `Sender` uses `Release` ordering for storing a new state\n /// and the `Receiver`s use `Acquire` ordering for loading the\n /// current state. This ensures that written values are seen by\n /// the `Receiver`s for a proper handover.\n #[derive(Debug)]\n pub(super) struct AtomicState(AtomicUsize);\n\n impl Version {\n /// Decrements the version.\n pub(super) fn decrement(&mut self) {\n // Using a wrapping decrement here is required to ensure that the\n // operation is consistent with `std::sync::atomic::AtomicUsize::fetch_add()`\n // which wraps on overflow.\n self.0 = self.0.wrapping_sub(STEP_SIZE);\n }\n\n pub(super) const INITIAL: Self = Version(0);\n }\n\n impl StateSnapshot {\n /// Extract the version from the state.\n pub(super) fn version(self) -> Version {\n Version(self.0 & !CLOSED_BIT)\n }\n\n /// Is the closed bit set?\n pub(super) fn is_closed(self) -> bool {\n (self.0 & CLOSED_BIT) == CLOSED_BIT\n }\n }\n\n impl AtomicState {\n /// Create a new `AtomicState` that is not closed and which has the\n /// version set to `Version::INITIAL`.\n pub(super) fn new() -> Self {\n AtomicState(AtomicUsize::new(Version::INITIAL.0))\n }\n\n /// Load the current value of the state.\n ///\n /// Only used by the receiver and for debugging purposes.\n ///\n /// The receiver side (read-only) uses `Acquire` ordering for a proper handover\n /// of the shared value with the sender side (single writer). The state is always\n /// updated after modifying and before releasing the (exclusive) lock on the\n /// shared value.\n pub(super) fn load(&self) -> StateSnapshot {\n StateSnapshot(self.0.load(Ordering::Acquire))\n }\n\n /// Increment the version counter.\n pub(super) fn increment_version_while_locked(&self) {\n // Use `Release` ordering to ensure that the shared value\n // has been written before updating the version. The shared\n // value is still protected by an exclusive lock during this\n // method.\n self.0.fetch_add(STEP_SIZE, Ordering::Release);\n }\n\n /// Set the closed bit in the state.\n pub(super) fn set_closed(&self) {\n self.0.fetch_or(CLOSED_BIT, Ordering::Release);\n }\n }\n}\n\n/// Creates a new watch channel, returning the \"send\" and \"receive\" handles.\n///\n/// All values sent by [`Sender`] will become visible to the [`Receiver`] handles.\n/// Only the last value sent is made available to the [`Receiver`] half. All\n/// intermediate values are dropped.\n///\n/// # Examples\n///\n/// The following example prints `hello! world! `.\n///\n/// ```\n/// use tokio::sync::watch;\n/// use tokio::time::{Duration, sleep};\n///\n/// # async fn dox() -> Result<(), Box> {\n/// let (tx, mut rx) = watch::channel(\"hello\");\n///\n/// tokio::spawn(async move {\n/// // Use the equivalent of a \"do-while\" loop so the initial value is\n/// // processed before awaiting the `changed()` future.\n/// loop {\n/// println!(\"{}! \", *rx.borrow_and_update());\n/// if rx.changed().await.is_err() {\n/// break;\n/// }\n/// }\n/// });\n///\n/// sleep(Duration::from_millis(100)).await;\n/// tx.send(\"world\")?;\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`Sender`]: struct@Sender\n/// [`Receiver`]: struct@Receiver\npub fn channel(init: T) -> (Sender, Receiver) {\n let shared = Arc::new(Shared {\n value: RwLock::new(init),\n state: AtomicState::new(),\n ref_count_rx: AtomicUsize::new(1),\n ref_count_tx: AtomicUsize::new(1),\n notify_rx: big_notify::BigNotify::new(),\n notify_tx: Notify::new(),\n });\n\n let tx = Sender {\n shared: shared.clone(),\n };\n\n let rx = Receiver {\n shared,\n version: Version::INITIAL,\n };\n\n (tx, rx)\n}\n\nimpl Receiver {\n fn from_shared(version: Version, shared: Arc>) -> Self {\n // No synchronization necessary as this is only used as a counter and\n // not memory access.\n shared.ref_count_rx.fetch_add(1, Relaxed);\n\n Self { shared, version }\n }\n\n /// Returns a reference to the most recently sent value.\n ///\n /// This method does not mark the returned value as seen, so future calls to\n /// [`changed`] may return immediately even if you have already seen the\n /// value with a call to `borrow`.\n ///\n /// Outstanding borrows hold a read lock on the inner value. This means that\n /// long-lived borrows could cause the producer half to block. It is recommended\n /// to keep the borrow as short-lived as possible. Additionally, if you are\n /// running in an environment that allows `!Send` futures, you must ensure that\n /// the returned `Ref` type is never held alive across an `.await` point,\n /// otherwise, it can lead to a deadlock.\n ///\n /// The priority policy of the lock is dependent on the underlying lock\n /// implementation, and this type does not guarantee that any particular policy\n /// will be used. In particular, a producer which is waiting to acquire the lock\n /// in `send` might or might not block concurrent calls to `borrow`, e.g.:\n ///\n ///
Potential deadlock example\n ///\n /// ```text\n /// // Task 1 (on thread A) | // Task 2 (on thread B)\n /// let _ref1 = rx.borrow(); |\n /// | // will block\n /// | let _ = tx.send(());\n /// // may deadlock |\n /// let _ref2 = rx.borrow(); |\n /// ```\n ///
\n ///\n /// For more information on when to use this method versus\n /// [`borrow_and_update`], see [here](self#borrow_and_update-versus-borrow).\n ///\n /// [`changed`]: Receiver::changed\n /// [`borrow_and_update`]: Receiver::borrow_and_update\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// let (_, rx) = watch::channel(\"hello\");\n /// assert_eq!(*rx.borrow(), \"hello\");\n /// ```\n pub fn borrow(&self) -> Ref<'_, T> {\n let inner = self.shared.value.read();\n\n // After obtaining a read-lock no concurrent writes could occur\n // and the loaded version matches that of the borrowed reference.\n let new_version = self.shared.state.load().version();\n let has_changed = self.version != new_version;\n\n Ref { inner, has_changed }\n }\n\n /// Returns a reference to the most recently sent value and marks that value\n /// as seen.\n ///\n /// This method marks the current value as seen. Subsequent calls to [`changed`]\n /// will not return immediately until the [`Sender`] has modified the shared\n /// value again.\n ///\n /// Outstanding borrows hold a read lock on the inner value. This means that\n /// long-lived borrows could cause the producer half to block. It is recommended\n /// to keep the borrow as short-lived as possible. Additionally, if you are\n /// running in an environment that allows `!Send` futures, you must ensure that\n /// the returned `Ref` type is never held alive across an `.await` point,\n /// otherwise, it can lead to a deadlock.\n ///\n /// The priority policy of the lock is dependent on the underlying lock\n /// implementation, and this type does not guarantee that any particular policy\n /// will be used. In particular, a producer which is waiting to acquire the lock\n /// in `send` might or might not block concurrent calls to `borrow`, e.g.:\n ///\n ///
Potential deadlock example\n ///\n /// ```text\n /// // Task 1 (on thread A) | // Task 2 (on thread B)\n /// let _ref1 = rx1.borrow_and_update(); |\n /// | // will block\n /// | let _ = tx.send(());\n /// // may deadlock |\n /// let _ref2 = rx2.borrow_and_update(); |\n /// ```\n ///
\n ///\n /// For more information on when to use this method versus [`borrow`], see\n /// [here](self#borrow_and_update-versus-borrow).\n ///\n /// [`changed`]: Receiver::changed\n /// [`borrow`]: Receiver::borrow\n pub fn borrow_and_update(&mut self) -> Ref<'_, T> {\n let inner = self.shared.value.read();\n\n // After obtaining a read-lock no concurrent writes could occur\n // and the loaded version matches that of the borrowed reference.\n let new_version = self.shared.state.load().version();\n let has_changed = self.version != new_version;\n\n // Mark the shared value as seen by updating the version\n self.version = new_version;\n\n Ref { inner, has_changed }\n }\n\n /// Checks if this channel contains a message that this receiver has not yet\n /// seen. The current value will not be marked as seen.\n ///\n /// Although this method is called `has_changed`, it does not check\n /// messages for equality, so this call will return true even if the current\n /// message is equal to the previous message.\n ///\n /// # Errors\n ///\n /// Returns a [`RecvError`](error::RecvError) if and only if the channel has been closed.\n ///\n /// # Examples\n ///\n /// ## Basic usage\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = watch::channel(\"hello\");\n ///\n /// tx.send(\"goodbye\").unwrap();\n ///\n /// assert!(rx.has_changed().unwrap());\n /// assert_eq!(*rx.borrow_and_update(), \"goodbye\");\n ///\n /// // The value has been marked as seen\n /// assert!(!rx.has_changed().unwrap());\n /// # }\n /// ```\n ///\n /// ## Closed channel example\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = watch::channel(\"hello\");\n /// tx.send(\"goodbye\").unwrap();\n ///\n /// drop(tx);\n ///\n /// // The channel is closed\n /// assert!(rx.has_changed().is_err());\n /// # }\n /// ```\n pub fn has_changed(&self) -> Result {\n // Load the version from the state\n let state = self.shared.state.load();\n if state.is_closed() {\n // All senders have dropped.\n return Err(error::RecvError(()));\n }\n let new_version = state.version();\n\n Ok(self.version != new_version)\n }\n\n /// Marks the state as changed.\n ///\n /// After invoking this method [`has_changed()`](Self::has_changed)\n /// returns `true` and [`changed()`](Self::changed) returns\n /// immediately, regardless of whether a new value has been sent.\n ///\n /// This is useful for triggering an initial change notification after\n /// subscribing to synchronize new receivers.\n pub fn mark_changed(&mut self) {\n self.version.decrement();\n }\n\n /// Marks the state as unchanged.\n ///\n /// The current value will be considered seen by the receiver.\n ///\n /// This is useful if you are not interested in the current value\n /// visible in the receiver.\n pub fn mark_unchanged(&mut self) {\n let current_version = self.shared.state.load().version();\n self.version = current_version;\n }\n\n /// Waits for a change notification, then marks the current value as seen.\n ///\n /// If the current value in the channel has not yet been marked seen when\n /// this method is called, the method marks that value seen and returns\n /// immediately. If the newest value has already been marked seen, then the\n /// method sleeps until a new message is sent by a [`Sender`] connected to\n /// this `Receiver`, or until all [`Sender`]s are dropped.\n ///\n /// For more information, see\n /// [*Change notifications*](self#change-notifications) in the module-level documentation.\n ///\n /// # Errors\n ///\n /// Returns a [`RecvError`](error::RecvError) if the channel has been closed __AND__\n /// the current value is seen.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that no values have been marked\n /// seen by this call to `changed`.\n ///\n /// [`Sender`]: struct@Sender\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, mut rx) = watch::channel(\"hello\");\n ///\n /// tokio::spawn(async move {\n /// tx.send(\"goodbye\").unwrap();\n /// });\n ///\n /// assert!(rx.changed().await.is_ok());\n /// assert_eq!(*rx.borrow_and_update(), \"goodbye\");\n ///\n /// // The `tx` handle has been dropped\n /// assert!(rx.changed().await.is_err());\n /// # }\n /// ```\n pub async fn changed(&mut self) -> Result<(), error::RecvError> {\n cooperative(changed_impl(&self.shared, &mut self.version)).await\n }\n\n /// Waits for a value that satisfies the provided condition.\n ///\n /// This method will call the provided closure whenever something is sent on\n /// the channel. Once the closure returns `true`, this method will return a\n /// reference to the value that was passed to the closure.\n ///\n /// Before `wait_for` starts waiting for changes, it will call the closure\n /// on the current value. If the closure returns `true` when given the\n /// current value, then `wait_for` will immediately return a reference to\n /// the current value. This is the case even if the current value is already\n /// considered seen.\n ///\n /// The watch channel only keeps track of the most recent value, so if\n /// several messages are sent faster than `wait_for` is able to call the\n /// closure, then it may skip some updates. Whenever the closure is called,\n /// it will be called with the most recent value.\n ///\n /// When this function returns, the value that was passed to the closure\n /// when it returned `true` will be considered seen.\n ///\n /// If the channel is closed, then `wait_for` will return a [`RecvError`].\n /// Once this happens, no more messages can ever be sent on the channel.\n /// When an error is returned, it is guaranteed that the closure has been\n /// called on the last value, and that it returned `false` for that value.\n /// (If the closure returned `true`, then the last value would have been\n /// returned instead of the error.)\n ///\n /// Like the [`borrow`] method, the returned borrow holds a read lock on the\n /// inner value. This means that long-lived borrows could cause the producer\n /// half to block. It is recommended to keep the borrow as short-lived as\n /// possible. See the documentation of `borrow` for more information on\n /// this.\n ///\n /// [`borrow`]: Receiver::borrow\n /// [`RecvError`]: error::RecvError\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If you use it as the event in a\n /// [`tokio::select!`](crate::select) statement and some other branch\n /// completes first, then it is guaranteed that the last seen value `val`\n /// (if any) satisfies `f(val) == false`.\n ///\n /// # Panics\n ///\n /// If and only if the closure `f` panics. In that case, no resource owned\n /// or shared by this [`Receiver`] will be poisoned.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n /// use tokio::time::{sleep, Duration};\n ///\n /// #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// async fn main() {\n /// let (tx, mut rx) = watch::channel(\"hello\");\n ///\n /// tokio::spawn(async move {\n /// sleep(Duration::from_secs(1)).await;\n /// tx.send(\"goodbye\").unwrap();\n /// });\n ///\n /// assert!(rx.wait_for(|val| *val == \"goodbye\").await.is_ok());\n /// assert_eq!(*rx.borrow(), \"goodbye\");\n /// }\n /// ```\n pub async fn wait_for(\n &mut self,\n f: impl FnMut(&T) -> bool,\n ) -> Result, error::RecvError> {\n cooperative(self.wait_for_inner(f)).await\n }\n\n async fn wait_for_inner(\n &mut self,\n mut f: impl FnMut(&T) -> bool,\n ) -> Result, error::RecvError> {\n let mut closed = false;\n loop {\n {\n let inner = self.shared.value.read();\n\n let new_version = self.shared.state.load().version();\n let has_changed = self.version != new_version;\n self.version = new_version;\n\n if !closed || has_changed {\n let result = panic::catch_unwind(panic::AssertUnwindSafe(|| f(&inner)));\n match result {\n Ok(true) => {\n return Ok(Ref { inner, has_changed });\n }\n Ok(false) => {\n // Skip the value.\n }\n Err(panicked) => {\n // Drop the read-lock to avoid poisoning it.\n drop(inner);\n // Forward the panic to the caller.\n panic::resume_unwind(panicked);\n // Unreachable\n }\n };\n }\n }\n\n if closed {\n return Err(error::RecvError(()));\n }\n\n // Wait for the value to change.\n closed = changed_impl(&self.shared, &mut self.version).await.is_err();\n }\n }\n\n /// Returns `true` if receivers belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::watch::channel(true);\n /// let rx2 = rx.clone();\n /// assert!(rx.same_channel(&rx2));\n ///\n /// let (tx3, rx3) = tokio::sync::watch::channel(true);\n /// assert!(!rx3.same_channel(&rx2));\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n Arc::ptr_eq(&self.shared, &other.shared)\n }\n\n cfg_process_driver! {\n pub(crate) fn try_has_changed(&mut self) -> Option> {\n maybe_changed(&self.shared, &mut self.version)\n }\n }\n}\n\nfn maybe_changed(\n shared: &Shared,\n version: &mut Version,\n) -> Option> {\n // Load the version from the state\n let state = shared.state.load();\n let new_version = state.version();\n\n if *version != new_version {\n // Observe the new version and return\n *version = new_version;\n return Some(Ok(()));\n }\n\n if state.is_closed() {\n // All senders have been dropped.\n return Some(Err(error::RecvError(())));\n }\n\n None\n}\n\nasync fn changed_impl(\n shared: &Shared,\n version: &mut Version,\n) -> Result<(), error::RecvError> {\n crate::trace::async_trace_leaf().await;\n\n loop {\n // In order to avoid a race condition, we first request a notification,\n // **then** check the current value's version. If a new version exists,\n // the notification request is dropped.\n let notified = shared.notify_rx.notified();\n\n if let Some(ret) = maybe_changed(shared, version) {\n return ret;\n }\n\n notified.await;\n // loop around again in case the wake-up was spurious\n }\n}\n\nimpl Clone for Receiver {\n fn clone(&self) -> Self {\n let version = self.version;\n let shared = self.shared.clone();\n\n Self::from_shared(version, shared)\n }\n}\n\nimpl Drop for Receiver {\n fn drop(&mut self) {\n // No synchronization necessary as this is only used as a counter and\n // not memory access.\n if 1 == self.shared.ref_count_rx.fetch_sub(1, Relaxed) {\n // This is the last `Receiver` handle, tasks waiting on `Sender::closed()`\n self.shared.notify_tx.notify_waiters();\n }\n }\n}\n\nimpl Sender {\n /// Creates the sending-half of the [`watch`] channel.\n ///\n /// See documentation of [`watch::channel`] for errors when calling this function.\n /// Beware that attempting to send a value when there are no receivers will\n /// return an error.\n ///\n /// [`watch`]: crate::sync::watch\n /// [`watch::channel`]: crate::sync::watch\n ///\n /// # Examples\n /// ```\n /// let sender = tokio::sync::watch::Sender::new(0u8);\n /// assert!(sender.send(3).is_err());\n /// let _rec = sender.subscribe();\n /// assert!(sender.send(4).is_ok());\n /// ```\n pub fn new(init: T) -> Self {\n let (tx, _) = channel(init);\n tx\n }\n\n /// Sends a new value via the channel, notifying all receivers.\n ///\n /// This method fails if the channel is closed, which is the case when\n /// every receiver has been dropped. It is possible to reopen the channel\n /// using the [`subscribe`] method. However, when `send` fails, the value\n /// isn't made available for future receivers (but returned with the\n /// [`SendError`]).\n ///\n /// To always make a new value available for future receivers, even if no\n /// receiver currently exists, one of the other send methods\n /// ([`send_if_modified`], [`send_modify`], or [`send_replace`]) can be\n /// used instead.\n ///\n /// [`subscribe`]: Sender::subscribe\n /// [`SendError`]: error::SendError\n /// [`send_if_modified`]: Sender::send_if_modified\n /// [`send_modify`]: Sender::send_modify\n /// [`send_replace`]: Sender::send_replace\n pub fn send(&self, value: T) -> Result<(), error::SendError> {\n // This is pretty much only useful as a hint anyway, so synchronization isn't critical.\n if 0 == self.receiver_count() {\n return Err(error::SendError(value));\n }\n\n self.send_replace(value);\n Ok(())\n }\n\n /// Modifies the watched value **unconditionally** in-place,\n /// notifying all receivers.\n ///\n /// This can be useful for modifying the watched value, without\n /// having to allocate a new instance. Additionally, this\n /// method permits sending values even when there are no receivers.\n ///\n /// Prefer to use the more versatile function [`Self::send_if_modified()`]\n /// if the value is only modified conditionally during the mutable borrow\n /// to prevent unneeded change notifications for unmodified values.\n ///\n /// # Panics\n ///\n /// This function panics when the invocation of the `modify` closure panics.\n /// No receivers are notified when panicking. All changes of the watched\n /// value applied by the closure before panicking will be visible in\n /// subsequent calls to `borrow`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// struct State {\n /// counter: usize,\n /// }\n /// let (state_tx, state_rx) = watch::channel(State { counter: 0 });\n /// state_tx.send_modify(|state| state.counter += 1);\n /// assert_eq!(state_rx.borrow().counter, 1);\n /// ```\n pub fn send_modify(&self, modify: F)\n where\n F: FnOnce(&mut T),\n {\n self.send_if_modified(|value| {\n modify(value);\n true\n });\n }\n\n /// Modifies the watched value **conditionally** in-place,\n /// notifying all receivers only if modified.\n ///\n /// This can be useful for modifying the watched value, without\n /// having to allocate a new instance. Additionally, this\n /// method permits sending values even when there are no receivers.\n ///\n /// The `modify` closure must return `true` if the value has actually\n /// been modified during the mutable borrow. It should only return `false`\n /// if the value is guaranteed to be unmodified despite the mutable\n /// borrow.\n ///\n /// Receivers are only notified if the closure returned `true`. If the\n /// closure has modified the value but returned `false` this results\n /// in a *silent modification*, i.e. the modified value will be visible\n /// in subsequent calls to `borrow`, but receivers will not receive\n /// a change notification.\n ///\n /// Returns the result of the closure, i.e. `true` if the value has\n /// been modified and `false` otherwise.\n ///\n /// # Panics\n ///\n /// This function panics when the invocation of the `modify` closure panics.\n /// No receivers are notified when panicking. All changes of the watched\n /// value applied by the closure before panicking will be visible in\n /// subsequent calls to `borrow`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// struct State {\n /// counter: usize,\n /// }\n /// let (state_tx, mut state_rx) = watch::channel(State { counter: 1 });\n /// let inc_counter_if_odd = |state: &mut State| {\n /// if state.counter % 2 == 1 {\n /// state.counter += 1;\n /// return true;\n /// }\n /// false\n /// };\n ///\n /// assert_eq!(state_rx.borrow().counter, 1);\n ///\n /// assert!(!state_rx.has_changed().unwrap());\n /// assert!(state_tx.send_if_modified(inc_counter_if_odd));\n /// assert!(state_rx.has_changed().unwrap());\n /// assert_eq!(state_rx.borrow_and_update().counter, 2);\n ///\n /// assert!(!state_rx.has_changed().unwrap());\n /// assert!(!state_tx.send_if_modified(inc_counter_if_odd));\n /// assert!(!state_rx.has_changed().unwrap());\n /// assert_eq!(state_rx.borrow_and_update().counter, 2);\n /// ```\n pub fn send_if_modified(&self, modify: F) -> bool\n where\n F: FnOnce(&mut T) -> bool,\n {\n {\n // Acquire the write lock and update the value.\n let mut lock = self.shared.value.write();\n\n // Update the value and catch possible panic inside func.\n let result = panic::catch_unwind(panic::AssertUnwindSafe(|| modify(&mut lock)));\n match result {\n Ok(modified) => {\n if !modified {\n // Abort, i.e. don't notify receivers if unmodified\n return false;\n }\n // Continue if modified\n }\n Err(panicked) => {\n // Drop the lock to avoid poisoning it.\n drop(lock);\n // Forward the panic to the caller.\n panic::resume_unwind(panicked);\n // Unreachable\n }\n };\n\n self.shared.state.increment_version_while_locked();\n\n // Release the write lock.\n //\n // Incrementing the version counter while holding the lock ensures\n // that receivers are able to figure out the version number of the\n // value they are currently looking at.\n drop(lock);\n }\n\n self.shared.notify_rx.notify_waiters();\n\n true\n }\n\n /// Sends a new value via the channel, notifying all receivers and returning\n /// the previous value in the channel.\n ///\n /// This can be useful for reusing the buffers inside a watched value.\n /// Additionally, this method permits sending values even when there are no\n /// receivers.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// let (tx, _rx) = watch::channel(1);\n /// assert_eq!(tx.send_replace(2), 1);\n /// assert_eq!(tx.send_replace(3), 2);\n /// ```\n pub fn send_replace(&self, mut value: T) -> T {\n // swap old watched value with the new one\n self.send_modify(|old| mem::swap(old, &mut value));\n\n value\n }\n\n /// Returns a reference to the most recently sent value\n ///\n /// Outstanding borrows hold a read lock on the inner value. This means that\n /// long-lived borrows could cause the producer half to block. It is recommended\n /// to keep the borrow as short-lived as possible. Additionally, if you are\n /// running in an environment that allows `!Send` futures, you must ensure that\n /// the returned `Ref` type is never held alive across an `.await` point,\n /// otherwise, it can lead to a deadlock.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// let (tx, _) = watch::channel(\"hello\");\n /// assert_eq!(*tx.borrow(), \"hello\");\n /// ```\n pub fn borrow(&self) -> Ref<'_, T> {\n let inner = self.shared.value.read();\n\n // The sender/producer always sees the current version\n let has_changed = false;\n\n Ref { inner, has_changed }\n }\n\n /// Checks if the channel has been closed. This happens when all receivers\n /// have dropped.\n ///\n /// # Examples\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::watch::channel(());\n /// assert!(!tx.is_closed());\n ///\n /// drop(rx);\n /// assert!(tx.is_closed());\n /// ```\n pub fn is_closed(&self) -> bool {\n self.receiver_count() == 0\n }\n\n /// Completes when all receivers have dropped.\n ///\n /// This allows the producer to get notified when interest in the produced\n /// values is canceled and immediately stop doing work. Once a channel is\n /// closed, the only way to reopen it is to call [`Sender::subscribe`] to\n /// get a new receiver.\n ///\n /// If the channel becomes closed for a brief amount of time (e.g., the last\n /// receiver is dropped and then `subscribe` is called), then this call to\n /// `closed` might return, but it is also possible that it does not \"notice\"\n /// that the channel was closed for a brief amount of time.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx) = watch::channel(\"hello\");\n ///\n /// tokio::spawn(async move {\n /// // use `rx`\n /// drop(rx);\n /// });\n ///\n /// // Waits for `rx` to drop\n /// tx.closed().await;\n /// println!(\"the `rx` handles dropped\")\n /// # }\n /// ```\n pub async fn closed(&self) {\n cooperative(async {\n crate::trace::async_trace_leaf().await;\n\n while self.receiver_count() > 0 {\n let notified = self.shared.notify_tx.notified();\n\n if self.receiver_count() == 0 {\n return;\n }\n\n notified.await;\n // The channel could have been reopened in the meantime by calling\n // `subscribe`, so we loop again.\n }\n })\n .await;\n }\n\n /// Creates a new [`Receiver`] connected to this `Sender`.\n ///\n /// All messages sent before this call to `subscribe` are initially marked\n /// as seen by the new `Receiver`.\n ///\n /// This method can be called even if there are no other receivers. In this\n /// case, the channel is reopened.\n ///\n /// # Examples\n ///\n /// The new channel will receive messages sent on this `Sender`.\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx) = watch::channel(0u64);\n ///\n /// tx.send(5).unwrap();\n ///\n /// let rx = tx.subscribe();\n /// assert_eq!(5, *rx.borrow());\n ///\n /// tx.send(10).unwrap();\n /// assert_eq!(10, *rx.borrow());\n /// # }\n /// ```\n ///\n /// The most recent message is considered seen by the channel, so this test\n /// is guaranteed to pass.\n ///\n /// ```\n /// use tokio::sync::watch;\n /// use tokio::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, _rx) = watch::channel(0u64);\n /// tx.send(5).unwrap();\n /// let mut rx = tx.subscribe();\n ///\n /// tokio::spawn(async move {\n /// // by spawning and sleeping, the message is sent after `main`\n /// // hits the call to `changed`.\n /// # if false {\n /// tokio::time::sleep(Duration::from_millis(10)).await;\n /// # }\n /// tx.send(100).unwrap();\n /// });\n ///\n /// rx.changed().await.unwrap();\n /// assert_eq!(100, *rx.borrow());\n /// # }\n /// ```\n pub fn subscribe(&self) -> Receiver {\n let shared = self.shared.clone();\n let version = shared.state.load().version();\n\n // The CLOSED bit in the state tracks only whether the sender is\n // dropped, so we do not need to unset it if this reopens the channel.\n Receiver::from_shared(version, shared)\n }\n\n /// Returns the number of receivers that currently exist.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx, rx1) = watch::channel(\"hello\");\n ///\n /// assert_eq!(1, tx.receiver_count());\n ///\n /// let mut _rx2 = rx1.clone();\n ///\n /// assert_eq!(2, tx.receiver_count());\n /// # }\n /// ```\n pub fn receiver_count(&self) -> usize {\n self.shared.ref_count_rx.load(Relaxed)\n }\n\n /// Returns the number of senders that currently exist.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::sync::watch;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let (tx1, rx) = watch::channel(\"hello\");\n ///\n /// assert_eq!(1, tx1.sender_count());\n ///\n /// let tx2 = tx1.clone();\n ///\n /// assert_eq!(2, tx1.sender_count());\n /// assert_eq!(2, tx2.sender_count());\n /// # }\n /// ```\n pub fn sender_count(&self) -> usize {\n self.shared.ref_count_tx.load(Relaxed)\n }\n\n /// Returns `true` if senders belong to the same channel.\n ///\n /// # Examples\n ///\n /// ```\n /// let (tx, rx) = tokio::sync::watch::channel(true);\n /// let tx2 = tx.clone();\n /// assert!(tx.same_channel(&tx2));\n ///\n /// let (tx3, rx3) = tokio::sync::watch::channel(true);\n /// assert!(!tx3.same_channel(&tx2));\n /// ```\n pub fn same_channel(&self, other: &Self) -> bool {\n Arc::ptr_eq(&self.shared, &other.shared)\n }\n}\n\nimpl Drop for Sender {\n fn drop(&mut self) {\n if self.shared.ref_count_tx.fetch_sub(1, AcqRel) == 1 {\n self.shared.state.set_closed();\n self.shared.notify_rx.notify_waiters();\n }\n }\n}\n\n// ===== impl Ref =====\n\nimpl ops::Deref for Ref<'_, T> {\n type Target = T;\n\n fn deref(&self) -> &T {\n self.inner.deref()\n }\n}\n\n#[cfg(all(test, loom))]\nmod tests {\n use futures::future::FutureExt;\n use loom::thread;\n\n // test for https://github.com/tokio-rs/tokio/issues/3168\n #[test]\n fn watch_spurious_wakeup() {\n loom::model(|| {\n let (send, mut recv) = crate::sync::watch::channel(0i32);\n\n send.send(1).unwrap();\n\n let send_thread = thread::spawn(move || {\n send.send(2).unwrap();\n send\n });\n\n recv.changed().now_or_never();\n\n let send = send_thread.join().unwrap();\n let recv_thread = thread::spawn(move || {\n recv.changed().now_or_never();\n recv.changed().now_or_never();\n recv\n });\n\n send.send(3).unwrap();\n\n let mut recv = recv_thread.join().unwrap();\n let send_thread = thread::spawn(move || {\n send.send(2).unwrap();\n });\n\n recv.changed().now_or_never();\n\n send_thread.join().unwrap();\n });\n }\n\n #[test]\n fn watch_borrow() {\n loom::model(|| {\n let (send, mut recv) = crate::sync::watch::channel(0i32);\n\n assert!(send.borrow().eq(&0));\n assert!(recv.borrow().eq(&0));\n\n send.send(1).unwrap();\n assert!(send.borrow().eq(&1));\n\n let send_thread = thread::spawn(move || {\n send.send(2).unwrap();\n send\n });\n\n recv.changed().now_or_never();\n\n let send = send_thread.join().unwrap();\n let recv_thread = thread::spawn(move || {\n recv.changed().now_or_never();\n recv.changed().now_or_never();\n recv\n });\n\n send.send(3).unwrap();\n\n let recv = recv_thread.join().unwrap();\n assert!(recv.borrow().eq(&3));\n assert!(send.borrow().eq(&3));\n\n send.send(2).unwrap();\n\n thread::spawn(move || {\n assert!(recv.borrow().eq(&2));\n });\n assert!(send.borrow().eq(&2));\n });\n }\n}\n" }, { "path": "tokio/src/task/blocking.rs", "content": "use crate::task::JoinHandle;\n\ncfg_rt_multi_thread! {\n /// Runs the provided blocking function on the current thread without\n /// blocking the executor.\n ///\n /// In general, issuing a blocking call or performing a lot of compute in a\n /// future without yielding is problematic, as it may prevent the executor\n /// from driving other tasks forward. Calling this function informs the\n /// executor that the currently executing task is about to block the thread,\n /// so the executor is able to hand off any other tasks it has to a new\n /// worker thread before that happens. See the [CPU-bound tasks and blocking\n /// code][blocking] section for more information.\n ///\n /// Be aware that although this function avoids starving other independently\n /// spawned tasks, any other code running concurrently in the same task will\n /// be suspended during the call to `block_in_place`. This can happen e.g.\n /// when using the [`join!`] macro. To avoid this issue, use\n /// [`spawn_blocking`] instead of `block_in_place`.\n ///\n /// Note that this function cannot be used within a [`current_thread`] runtime\n /// because in this case there are no other worker threads to hand off tasks\n /// to. On the other hand, calling the function outside a runtime is\n /// allowed. In this case, `block_in_place` just calls the provided closure\n /// normally.\n ///\n /// Code running behind `block_in_place` cannot be cancelled. When you shut\n /// down the executor, it will wait indefinitely for all blocking operations\n /// to finish. You can use [`shutdown_timeout`] to stop waiting for them\n /// after a certain timeout. Be aware that this will still not cancel the\n /// tasks — they are simply allowed to keep running after the method\n /// returns.\n ///\n /// [blocking]: ../index.html#cpu-bound-tasks-and-blocking-code\n /// [`spawn_blocking`]: fn@crate::task::spawn_blocking\n /// [`join!`]: macro@join\n /// [`thread::spawn`]: fn@std::thread::spawn\n /// [`shutdown_timeout`]: fn@crate::runtime::Runtime::shutdown_timeout\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn docs() {\n /// task::block_in_place(move || {\n /// // do some compute-heavy work or call synchronous code\n /// });\n /// # }\n /// ```\n ///\n /// Code running inside `block_in_place` may use `block_on` to reenter the\n /// async context.\n ///\n /// ```\n /// use tokio::task;\n /// use tokio::runtime::Handle;\n ///\n /// # async fn docs() {\n /// task::block_in_place(move || {\n /// Handle::current().block_on(async move {\n /// // do something async\n /// });\n /// });\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This function panics if called from a [`current_thread`] runtime.\n ///\n /// [`current_thread`]: fn@crate::runtime::Builder::new_current_thread\n #[track_caller]\n pub fn block_in_place(f: F) -> R\n where\n F: FnOnce() -> R,\n {\n crate::runtime::scheduler::block_in_place(f)\n }\n}\n\ncfg_rt! {\n /// Runs the provided closure on a thread where blocking is acceptable.\n ///\n /// In general, issuing a blocking call or performing a lot of compute in a\n /// future without yielding is problematic, as it may prevent the executor from\n /// driving other futures forward. This function runs the provided closure on a\n /// thread dedicated to blocking operations. See the [CPU-bound tasks and\n /// blocking code][blocking] section for more information.\n ///\n /// Tokio will spawn more blocking threads when they are requested through this\n /// function until the upper limit configured on the [`Builder`] is reached.\n /// After reaching the upper limit, the tasks are put in a queue.\n /// The thread limit is very large by default, because `spawn_blocking` is often\n /// used for various kinds of IO operations that cannot be performed\n /// asynchronously. When you run CPU-bound code using `spawn_blocking`, you\n /// should keep this large upper limit in mind. When running many CPU-bound\n /// computations, a semaphore or some other synchronization primitive should be\n /// used to limit the number of computations executed in parallel. Specialized\n /// CPU-bound executors, such as [rayon], may also be a good fit.\n ///\n /// This function is intended for non-async operations that eventually finish on\n /// their own. If you want to spawn an ordinary thread, you should use\n /// [`thread::spawn`] instead.\n ///\n /// Be aware that tasks spawned using `spawn_blocking` cannot be aborted\n /// because they are not async. If you call [`abort`] on a `spawn_blocking`\n /// task, then this *will not have any effect*, and the task will continue\n /// running normally. The exception is if the task has not started running\n /// yet; in that case, calling `abort` may prevent the task from starting.\n ///\n /// When you shut down the executor, it will attempt to `abort` all tasks\n /// including `spawn_blocking` tasks. However, `spawn_blocking` tasks\n /// cannot be aborted once they start running, which means that runtime\n /// shutdown will wait indefinitely for all started `spawn_blocking` to\n /// finish running. You can use [`shutdown_timeout`] to stop waiting for\n /// them after a certain timeout. Be aware that this will still not cancel\n /// the tasks — they are simply allowed to keep running after the method\n /// returns. It is possible for a blocking task to be cancelled if it has\n /// not yet started running, but this is not guaranteed.\n ///\n /// # When to use `spawn_blocking` vs dedicated threads\n ///\n /// `spawn_blocking` is intended for *bounded* blocking work that eventually finishes.\n /// Each call occupies a thread from the runtime's blocking thread pool for the duration\n /// of the task. Long-lived tasks therefore reduce the pool's effective capacity, which\n /// can delay other blocking operations once the pool is saturated and work is queued.\n ///\n /// For workloads that run indefinitely or for extended periods (for example,\n /// background workers or persistent processing loops), prefer a dedicated thread created with\n /// [`thread::spawn`].\n ///\n /// As a rule of thumb:\n /// - Use `spawn_blocking` for short-lived blocking operations\n /// - Use dedicated threads for long-lived or persistent blocking workloads\n ///\n /// Note that if you are using the single threaded runtime, this function will\n /// still spawn additional threads for blocking operations. The current-thread\n /// scheduler's single thread is only used for asynchronous code.\n ///\n /// # Related APIs and patterns for bridging asynchronous and blocking code\n ///\n /// In simple cases, it is sufficient to have the closure accept input\n /// parameters at creation time and return a single value (or struct/tuple, etc.).\n ///\n /// For more complex situations in which it is desirable to stream data to or from\n /// the synchronous context, the [`mpsc channel`] has `blocking_send` and\n /// `blocking_recv` methods for use in non-async code such as the thread created\n /// by `spawn_blocking`.\n ///\n /// Another option is [`SyncIoBridge`] for cases where the synchronous context\n /// is operating on byte streams. For example, you might use an asynchronous\n /// HTTP client such as [hyper] to fetch data, but perform complex parsing\n /// of the payload body using a library written for synchronous I/O.\n ///\n /// Finally, see also [Bridging with sync code][bridgesync] for discussions\n /// around the opposite case of using Tokio as part of a larger synchronous\n /// codebase.\n ///\n /// [`Builder`]: struct@crate::runtime::Builder\n /// [blocking]: ../index.html#cpu-bound-tasks-and-blocking-code\n /// [rayon]: https://docs.rs/rayon\n /// [`mpsc channel`]: crate::sync::mpsc\n /// [`SyncIoBridge`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.SyncIoBridge.html\n /// [hyper]: https://docs.rs/hyper\n /// [`thread::spawn`]: fn@std::thread::spawn\n /// [`shutdown_timeout`]: fn@crate::runtime::Runtime::shutdown_timeout\n /// [bridgesync]: https://tokio.rs/tokio/topics/bridging\n /// [`AtomicBool`]: struct@std::sync::atomic::AtomicBool\n /// [`abort`]: crate::task::JoinHandle::abort\n ///\n /// # Examples\n ///\n /// Pass an input value and receive result of computation:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// # async fn docs() -> Result<(), Box>{\n /// // Initial input\n /// let mut v = \"Hello, \".to_string();\n /// let res = task::spawn_blocking(move || {\n /// // Stand-in for compute-heavy work or using synchronous APIs\n /// v.push_str(\"world\");\n /// // Pass ownership of the value back to the asynchronous context\n /// v\n /// }).await?;\n ///\n /// // `res` is the value returned from the thread\n /// assert_eq!(res.as_str(), \"Hello, world\");\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// Use a channel:\n ///\n /// ```\n /// use tokio::task;\n /// use tokio::sync::mpsc;\n ///\n /// # async fn docs() {\n /// let (tx, mut rx) = mpsc::channel(2);\n /// let start = 5;\n /// let worker = task::spawn_blocking(move || {\n /// for x in 0..10 {\n /// // Stand in for complex computation\n /// tx.blocking_send(start + x).unwrap();\n /// }\n /// });\n ///\n /// let mut acc = 0;\n /// while let Some(v) = rx.recv().await {\n /// acc += v;\n /// }\n /// assert_eq!(acc, 95);\n /// worker.await.unwrap();\n /// # }\n /// ```\n #[track_caller]\n pub fn spawn_blocking(f: F) -> JoinHandle\n where\n F: FnOnce() -> R + Send + 'static,\n R: Send + 'static,\n {\n crate::runtime::spawn_blocking(f)\n }\n}\n" }, { "path": "tokio/src/task/builder.rs", "content": "#![allow(unreachable_pub)]\nuse crate::{\n runtime::{Handle, BOX_FUTURE_THRESHOLD},\n task::{JoinHandle, LocalSet},\n util::trace::SpawnMeta,\n};\nuse std::{future::Future, io, mem};\n\n/// Factory which is used to configure the properties of a new task.\n///\n/// **Note**: This is an [unstable API][unstable]. The public API of this type\n/// may break in 1.x releases. See [the documentation on unstable\n/// features][unstable] for details.\n///\n/// Methods can be chained in order to configure it.\n///\n/// Currently, there is only one configuration option:\n///\n/// - [`name`], which specifies an associated name for\n/// the task\n///\n/// There are three types of task that can be spawned from a Builder:\n/// - [`spawn_local`] for executing not [`Send`] futures\n/// - [`spawn`] for executing [`Send`] futures on the runtime\n/// - [`spawn_blocking`] for executing blocking code in the\n/// blocking thread pool.\n///\n/// ## Example\n///\n/// ```no_run\n/// use tokio::net::{TcpListener, TcpStream};\n///\n/// use std::io;\n///\n/// async fn process(socket: TcpStream) {\n/// // ...\n/// # drop(socket);\n/// }\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n///\n/// loop {\n/// let (socket, _) = listener.accept().await?;\n///\n/// tokio::task::Builder::new()\n/// .name(\"tcp connection handler\")\n/// .spawn(async move {\n/// // Process each socket concurrently.\n/// process(socket).await\n/// })?;\n/// }\n/// }\n/// ```\n/// [unstable]: crate#unstable-features\n/// [`name`]: Builder::name\n/// [`spawn_local`]: Builder::spawn_local\n/// [`spawn`]: Builder::spawn\n/// [`spawn_blocking`]: Builder::spawn_blocking\n#[derive(Default, Debug)]\n#[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\npub struct Builder<'a> {\n name: Option<&'a str>,\n}\n\nimpl<'a> Builder<'a> {\n /// Creates a new task builder.\n pub fn new() -> Self {\n Self::default()\n }\n\n /// Assigns a name to the task which will be spawned.\n pub fn name(&self, name: &'a str) -> Self {\n Self { name: Some(name) }\n }\n\n /// Spawns a task with this builder's settings on the current runtime.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// See [`task::spawn`](crate::task::spawn()) for\n /// more details.\n #[track_caller]\n pub fn spawn(self, future: Fut) -> io::Result>\n where\n Fut: Future + Send + 'static,\n Fut::Output: Send + 'static,\n {\n let fut_size = mem::size_of::();\n Ok(if fut_size > BOX_FUTURE_THRESHOLD {\n super::spawn::spawn_inner(Box::pin(future), SpawnMeta::new(self.name, fut_size))\n } else {\n super::spawn::spawn_inner(future, SpawnMeta::new(self.name, fut_size))\n })\n }\n\n /// Spawn a task with this builder's settings on the provided [runtime\n /// handle].\n ///\n /// See [`Handle::spawn`] for more details.\n ///\n /// [runtime handle]: crate::runtime::Handle\n /// [`Handle::spawn`]: crate::runtime::Handle::spawn\n #[track_caller]\n pub fn spawn_on(self, future: Fut, handle: &Handle) -> io::Result>\n where\n Fut: Future + Send + 'static,\n Fut::Output: Send + 'static,\n {\n let fut_size = mem::size_of::();\n Ok(if fut_size > BOX_FUTURE_THRESHOLD {\n handle.spawn_named(Box::pin(future), SpawnMeta::new(self.name, fut_size))\n } else {\n handle.spawn_named(future, SpawnMeta::new(self.name, fut_size))\n })\n }\n\n /// Spawns a `!Send` task on the current [`LocalSet`] or [`LocalRuntime`] with\n /// this builder's settings.\n ///\n /// The spawned future will be run on the same thread that called `spawn_local`.\n /// This may only be called from the context of a [local task set][`LocalSet`]\n /// or a [`LocalRuntime`].\n ///\n /// # Panics\n ///\n /// This function panics if called outside of a [local task set][`LocalSet`]\n /// or a [`LocalRuntime`].\n ///\n /// See [`task::spawn_local`] for more details.\n ///\n /// [`task::spawn_local`]: crate::task::spawn_local\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`LocalRuntime`]: crate::runtime::LocalRuntime\n #[track_caller]\n pub fn spawn_local(self, future: Fut) -> io::Result>\n where\n Fut: Future + 'static,\n Fut::Output: 'static,\n {\n let fut_size = mem::size_of::();\n Ok(if fut_size > BOX_FUTURE_THRESHOLD {\n super::local::spawn_local_inner(Box::pin(future), SpawnMeta::new(self.name, fut_size))\n } else {\n super::local::spawn_local_inner(future, SpawnMeta::new(self.name, fut_size))\n })\n }\n\n /// Spawns `!Send` a task on the provided [`LocalSet`] with this builder's\n /// settings.\n ///\n /// See [`LocalSet::spawn_local`] for more details.\n ///\n /// [`LocalSet::spawn_local`]: crate::task::LocalSet::spawn_local\n /// [`LocalSet`]: crate::task::LocalSet\n #[track_caller]\n pub fn spawn_local_on(\n self,\n future: Fut,\n local_set: &LocalSet,\n ) -> io::Result>\n where\n Fut: Future + 'static,\n Fut::Output: 'static,\n {\n let fut_size = mem::size_of::();\n Ok(if fut_size > BOX_FUTURE_THRESHOLD {\n local_set.spawn_named(Box::pin(future), SpawnMeta::new(self.name, fut_size))\n } else {\n local_set.spawn_named(future, SpawnMeta::new(self.name, fut_size))\n })\n }\n\n /// Spawns blocking code on the blocking threadpool.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// See [`task::spawn_blocking`](crate::task::spawn_blocking)\n /// for more details.\n #[track_caller]\n pub fn spawn_blocking(\n self,\n function: Function,\n ) -> io::Result>\n where\n Function: FnOnce() -> Output + Send + 'static,\n Output: Send + 'static,\n {\n let handle = Handle::current();\n self.spawn_blocking_on(function, &handle)\n }\n\n /// Spawns blocking code on the provided [runtime handle]'s blocking threadpool.\n ///\n /// See [`Handle::spawn_blocking`] for more details.\n ///\n /// [runtime handle]: crate::runtime::Handle\n /// [`Handle::spawn_blocking`]: crate::runtime::Handle::spawn_blocking\n #[track_caller]\n pub fn spawn_blocking_on(\n self,\n function: Function,\n handle: &Handle,\n ) -> io::Result>\n where\n Function: FnOnce() -> Output + Send + 'static,\n Output: Send + 'static,\n {\n use crate::runtime::Mandatory;\n let fn_size = mem::size_of::();\n let (join_handle, spawn_result) = if fn_size > BOX_FUTURE_THRESHOLD {\n handle.inner.blocking_spawner().spawn_blocking_inner(\n Box::new(function),\n Mandatory::NonMandatory,\n SpawnMeta::new(self.name, fn_size),\n handle,\n )\n } else {\n handle.inner.blocking_spawner().spawn_blocking_inner(\n function,\n Mandatory::NonMandatory,\n SpawnMeta::new(self.name, fn_size),\n handle,\n )\n };\n\n spawn_result?;\n Ok(join_handle)\n }\n}\n" }, { "path": "tokio/src/task/coop/consume_budget.rs", "content": "/// Consumes a unit of budget and returns the execution back to the Tokio\n/// runtime *if* the task's coop budget was exhausted.\n///\n/// The task will only yield if its entire coop budget has been exhausted.\n/// This function can be used in order to insert optional yield points into long\n/// computations that do not use Tokio resources like sockets or semaphores,\n/// without redundantly yielding to the runtime each time.\n///\n/// # Examples\n///\n/// Make sure that a function which returns a sum of (potentially lots of)\n/// iterated values is cooperative.\n///\n/// ```\n/// async fn sum_iterator(input: &mut impl std::iter::Iterator) -> i64 {\n/// let mut sum: i64 = 0;\n/// while let Some(i) = input.next() {\n/// sum += i;\n/// tokio::task::consume_budget().await\n/// }\n/// sum\n/// }\n/// ```\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub async fn consume_budget() {\n let mut status = std::task::Poll::Pending;\n\n std::future::poll_fn(move |cx| {\n std::task::ready!(crate::trace::trace_leaf(cx));\n if status.is_ready() {\n return status;\n }\n status = crate::task::coop::poll_proceed(cx).map(|restore| {\n restore.made_progress();\n });\n status\n })\n .await\n}\n" }, { "path": "tokio/src/task/coop/mod.rs", "content": "#![cfg_attr(not(feature = \"full\"), allow(dead_code))]\n#![cfg_attr(not(feature = \"rt\"), allow(unreachable_pub))]\n\n//! Utilities for improved cooperative scheduling.\n//!\n//! ### Cooperative scheduling\n//!\n//! A single call to [`poll`] on a top-level task may potentially do a lot of\n//! work before it returns `Poll::Pending`. If a task runs for a long period of\n//! time without yielding back to the executor, it can starve other tasks\n//! waiting on that executor to execute them, or drive underlying resources.\n//! Since Rust does not have a runtime, it is difficult to forcibly preempt a\n//! long-running task. Instead, this module provides an opt-in mechanism for\n//! futures to collaborate with the executor to avoid starvation.\n//!\n//! Consider a future like this one:\n//!\n//! ```\n//! # use tokio_stream::{Stream, StreamExt};\n//! async fn drop_all(mut input: I) {\n//! while let Some(_) = input.next().await {}\n//! }\n//! ```\n//!\n//! It may look harmless, but consider what happens under heavy load if the\n//! input stream is _always_ ready. If we spawn `drop_all`, the task will never\n//! yield, and will starve other tasks and resources on the same executor.\n//!\n//! To account for this, Tokio has explicit yield points in a number of library\n//! functions, which force tasks to return to the executor periodically.\n//!\n//!\n//! #### unconstrained\n//!\n//! If necessary, [`task::unconstrained`] lets you opt a future out of Tokio's cooperative\n//! scheduling. When a future is wrapped with `unconstrained`, it will never be forced to yield to\n//! Tokio. For example:\n//!\n//! ```\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! use tokio::{task, sync::mpsc};\n//!\n//! let fut = async {\n//! let (tx, mut rx) = mpsc::unbounded_channel();\n//!\n//! for i in 0..1000 {\n//! let _ = tx.send(());\n//! // This will always be ready. If coop was in effect, this code would be forced to yield\n//! // periodically. However, if left unconstrained, then this code will never yield.\n//! rx.recv().await;\n//! }\n//! };\n//!\n//! task::coop::unconstrained(fut).await;\n//! # }\n//! ```\n//! [`poll`]: method@std::future::Future::poll\n//! [`task::unconstrained`]: crate::task::unconstrained()\n\ncfg_rt! {\n mod consume_budget;\n pub use consume_budget::consume_budget;\n\n mod unconstrained;\n pub use unconstrained::{unconstrained, Unconstrained};\n}\n\n// ```ignore\n// # use tokio_stream::{Stream, StreamExt};\n// async fn drop_all(mut input: I) {\n// while let Some(_) = input.next().await {\n// tokio::coop::proceed().await;\n// }\n// }\n// ```\n//\n// The `proceed` future will coordinate with the executor to make sure that\n// every so often control is yielded back to the executor so it can run other\n// tasks.\n//\n// # Placing yield points\n//\n// Voluntary yield points should be placed _after_ at least some work has been\n// done. If they are not, a future sufficiently deep in the task hierarchy may\n// end up _never_ getting to run because of the number of yield points that\n// inevitably appear before it is reached. In general, you will want yield\n// points to only appear in \"leaf\" futures -- those that do not themselves poll\n// other futures. By doing this, you avoid double-counting each iteration of\n// the outer future against the cooperating budget.\n\nuse crate::runtime::context;\n\n/// Opaque type tracking the amount of \"work\" a task may still do before\n/// yielding back to the scheduler.\n#[derive(Debug, Copy, Clone)]\npub(crate) struct Budget(Option);\n\npub(crate) struct BudgetDecrement {\n success: bool,\n hit_zero: bool,\n}\n\nimpl Budget {\n /// Budget assigned to a task on each poll.\n ///\n /// The value itself is chosen somewhat arbitrarily. It needs to be high\n /// enough to amortize wakeup and scheduling costs, but low enough that we\n /// do not starve other tasks for too long. The value also needs to be high\n /// enough that particularly deep tasks are able to do at least some useful\n /// work at all.\n ///\n /// Note that as more yield points are added in the ecosystem, this value\n /// will probably also have to be raised.\n const fn initial() -> Budget {\n Budget(Some(128))\n }\n\n /// Returns an unconstrained budget. Operations will not be limited.\n pub(crate) const fn unconstrained() -> Budget {\n Budget(None)\n }\n\n fn has_remaining(self) -> bool {\n self.0.map_or(true, |budget| budget > 0)\n }\n}\n\n/// Runs the given closure with a cooperative task budget. When the function\n/// returns, the budget is reset to the value prior to calling the function.\n#[inline(always)]\npub(crate) fn budget(f: impl FnOnce() -> R) -> R {\n with_budget(Budget::initial(), f)\n}\n\n/// Runs the given closure with an unconstrained task budget. When the function returns, the budget\n/// is reset to the value prior to calling the function.\n#[inline(always)]\npub(crate) fn with_unconstrained(f: impl FnOnce() -> R) -> R {\n with_budget(Budget::unconstrained(), f)\n}\n\n#[inline(always)]\nfn with_budget(budget: Budget, f: impl FnOnce() -> R) -> R {\n struct ResetGuard {\n prev: Budget,\n }\n\n impl Drop for ResetGuard {\n fn drop(&mut self) {\n let _ = context::budget(|cell| {\n cell.set(self.prev);\n });\n }\n }\n\n #[allow(unused_variables)]\n let maybe_guard = context::budget(|cell| {\n let prev = cell.get();\n cell.set(budget);\n\n ResetGuard { prev }\n });\n\n // The function is called regardless even if the budget is not successfully\n // set due to the thread-local being destroyed.\n f()\n}\n\n/// Returns `true` if there is still budget left on the task.\n///\n/// # Examples\n///\n/// This example defines a `Timeout` future that requires a given `future` to complete before the\n/// specified duration elapses. If it does, its result is returned; otherwise, an error is returned\n/// and the future is canceled.\n///\n/// Note that the future could exhaust the budget before we evaluate the timeout. Using `has_budget_remaining`,\n/// we can detect this scenario and ensure the timeout is always checked.\n///\n/// ```\n/// # use std::future::Future;\n/// # use std::pin::{pin, Pin};\n/// # use std::task::{ready, Context, Poll};\n/// # use tokio::task::coop;\n/// # use tokio::time::Sleep;\n/// pub struct Timeout {\n/// future: T,\n/// delay: Pin>,\n/// }\n///\n/// impl Future for Timeout\n/// where\n/// T: Future + Unpin,\n/// {\n/// type Output = Result;\n///\n/// fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n/// let this = Pin::into_inner(self);\n/// let future = Pin::new(&mut this.future);\n/// let delay = Pin::new(&mut this.delay);\n///\n/// // check if the future is ready\n/// let had_budget_before = coop::has_budget_remaining();\n/// if let Poll::Ready(v) = future.poll(cx) {\n/// return Poll::Ready(Ok(v));\n/// }\n/// let has_budget_now = coop::has_budget_remaining();\n///\n/// // evaluate the timeout\n/// if let (true, false) = (had_budget_before, has_budget_now) {\n/// // it is the underlying future that exhausted the budget\n/// ready!(pin!(coop::unconstrained(delay)).poll(cx));\n/// } else {\n/// ready!(delay.poll(cx));\n/// }\n/// return Poll::Ready(Err(()));\n/// }\n/// }\n///```\n#[inline(always)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub fn has_budget_remaining() -> bool {\n // If the current budget cannot be accessed due to the thread-local being\n // shutdown, then we assume there is budget remaining.\n context::budget(|cell| cell.get().has_remaining()).unwrap_or(true)\n}\n\ncfg_rt_multi_thread! {\n /// Sets the current task's budget.\n pub(crate) fn set(budget: Budget) {\n let _ = context::budget(|cell| cell.set(budget));\n }\n}\n\ncfg_rt! {\n /// Forcibly removes the budgeting constraints early.\n ///\n /// Returns the remaining budget\n pub(crate) fn stop() -> Budget {\n context::budget(|cell| {\n let prev = cell.get();\n cell.set(Budget::unconstrained());\n prev\n }).unwrap_or(Budget::unconstrained())\n }\n}\n\ncfg_coop! {\n use pin_project_lite::pin_project;\n use std::cell::Cell;\n use std::future::Future;\n use std::marker::PhantomData;\n use std::pin::Pin;\n use std::task::{ready, Context, Poll};\n\n /// Value returned by the [`poll_proceed`] method.\n #[derive(Debug)]\n #[must_use]\n pub struct RestoreOnPending(Cell, PhantomData<*mut ()>);\n\n impl RestoreOnPending {\n fn new(budget: Budget) -> Self {\n RestoreOnPending(\n Cell::new(budget),\n PhantomData,\n )\n }\n\n /// Signals that the task that obtained this `RestoreOnPending` was able to make\n /// progress. This prevents the task budget from being restored to the value\n /// it had prior to obtaining this instance when it is dropped.\n pub fn made_progress(&self) {\n self.0.set(Budget::unconstrained());\n }\n }\n\n impl Drop for RestoreOnPending {\n fn drop(&mut self) {\n // Don't reset if budget was unconstrained or if we made progress.\n // They are both represented as the remembered budget being unconstrained.\n let budget = self.0.get();\n if !budget.is_unconstrained() {\n let _ = context::budget(|cell| {\n cell.set(budget);\n });\n }\n }\n }\n\n /// Decrements the task budget and returns [`Poll::Pending`] if the budget is depleted.\n /// This indicates that the task should yield to the scheduler. Otherwise, returns\n /// [`RestoreOnPending`] which can be used to commit the budget consumption.\n ///\n /// The returned [`RestoreOnPending`] will revert the budget to its former\n /// value when dropped unless [`RestoreOnPending::made_progress`]\n /// is called. It is the caller's responsibility to do so when it _was_ able to\n /// make progress after the call to [`poll_proceed`].\n /// Restoring the budget automatically ensures the task can try to make progress in some other\n /// way.\n ///\n /// Note that [`RestoreOnPending`] restores the budget **as it was before [`poll_proceed`]**.\n /// Therefore, if the budget is _further_ adjusted between when [`poll_proceed`] returns and\n /// [`RestoreOnPending`] is dropped, those adjustments are erased unless the caller indicates\n /// that progress was made.\n ///\n /// # Examples\n ///\n /// This example wraps the `futures::channel::mpsc::UnboundedReceiver` to\n /// cooperate with the Tokio scheduler. Each time a value is received, task budget\n /// is consumed. If no budget is available, the task yields to the scheduler.\n ///\n /// ```\n /// use std::pin::Pin;\n /// use std::task::{ready, Context, Poll};\n /// use tokio::task::coop;\n /// use futures::stream::{Stream, StreamExt};\n /// use futures::channel::mpsc::UnboundedReceiver;\n ///\n /// struct CoopUnboundedReceiver {\n /// receiver: UnboundedReceiver,\n /// }\n ///\n /// impl Stream for CoopUnboundedReceiver {\n /// type Item = T;\n /// fn poll_next(\n /// mut self: Pin<&mut Self>,\n /// cx: &mut Context<'_>\n /// ) -> Poll> {\n /// let coop = ready!(coop::poll_proceed(cx));\n /// match self.receiver.poll_next_unpin(cx) {\n /// Poll::Ready(v) => {\n /// // We received a value, so consume budget.\n /// coop.made_progress();\n /// Poll::Ready(v)\n /// }\n /// Poll::Pending => Poll::Pending,\n /// }\n /// }\n /// }\n /// ```\n #[inline]\n pub fn poll_proceed(cx: &mut Context<'_>) -> Poll {\n context::budget(|cell| {\n let mut budget = cell.get();\n\n let decrement = budget.decrement();\n\n if decrement.success {\n let restore = RestoreOnPending::new(cell.get());\n cell.set(budget);\n\n // avoid double counting\n if decrement.hit_zero {\n inc_budget_forced_yield_count();\n }\n\n Poll::Ready(restore)\n } else {\n register_waker(cx);\n Poll::Pending\n }\n }).unwrap_or(Poll::Ready(RestoreOnPending::new(Budget::unconstrained())))\n }\n\n /// Returns `Poll::Ready` if the current task has budget to consume, and `Poll::Pending` otherwise.\n ///\n /// Note that in contrast to `poll_proceed`, this method does not consume any budget and is used when\n /// polling for budget availability.\n #[inline]\n pub(crate) fn poll_budget_available(cx: &mut Context<'_>) -> Poll<()> {\n if has_budget_remaining() {\n Poll::Ready(())\n } else {\n register_waker(cx);\n\n Poll::Pending\n }\n }\n\n cfg_rt! {\n cfg_unstable_metrics! {\n #[inline(always)]\n fn inc_budget_forced_yield_count() {\n let _ = context::with_current(|handle| {\n handle.scheduler_metrics().inc_budget_forced_yield_count();\n });\n }\n }\n\n cfg_not_unstable_metrics! {\n #[inline(always)]\n fn inc_budget_forced_yield_count() {}\n }\n\n fn register_waker(cx: &mut Context<'_>) {\n context::defer(cx.waker());\n }\n }\n\n cfg_not_rt! {\n #[inline(always)]\n fn inc_budget_forced_yield_count() {}\n\n fn register_waker(cx: &mut Context<'_>) {\n cx.waker().wake_by_ref()\n }\n }\n\n impl Budget {\n /// Decrements the budget. Returns `true` if successful. Decrementing fails\n /// when there is not enough remaining budget.\n fn decrement(&mut self) -> BudgetDecrement {\n if let Some(num) = &mut self.0 {\n if *num > 0 {\n *num -= 1;\n\n let hit_zero = *num == 0;\n\n BudgetDecrement { success: true, hit_zero }\n } else {\n BudgetDecrement { success: false, hit_zero: false }\n }\n } else {\n BudgetDecrement { success: true, hit_zero: false }\n }\n }\n\n fn is_unconstrained(self) -> bool {\n self.0.is_none()\n }\n }\n\n pin_project! {\n /// Future wrapper to ensure cooperative scheduling created by [`cooperative`].\n #[must_use = \"futures do nothing unless polled\"]\n pub struct Coop {\n #[pin]\n pub(crate) fut: F,\n }\n }\n\n impl Future for Coop {\n type Output = F::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let coop = ready!(poll_proceed(cx));\n let me = self.project();\n if let Poll::Ready(ret) = me.fut.poll(cx) {\n coop.made_progress();\n Poll::Ready(ret)\n } else {\n Poll::Pending\n }\n }\n }\n\n /// Creates a wrapper future that makes the inner future cooperate with the Tokio scheduler.\n ///\n /// When polled, the wrapper will first call [`poll_proceed`] to consume task budget, and\n /// immediately yield if the budget has been depleted. If budget was available, the inner future\n /// is polled. The budget consumption will be made final using [`RestoreOnPending::made_progress`]\n /// if the inner future resolves to its final value.\n ///\n /// # Examples\n ///\n /// When you call `recv` on the `Receiver` of a [`tokio::sync::mpsc`](crate::sync::mpsc)\n /// channel, task budget will automatically be consumed when the next value is returned.\n /// This makes tasks that use Tokio mpsc channels automatically cooperative.\n ///\n /// If you're using [`futures::channel::mpsc`](https://docs.rs/futures/latest/futures/channel/mpsc/index.html)\n /// instead, automatic task budget consumption will not happen. This example shows how can use\n /// `cooperative` to make `futures::channel::mpsc` channels cooperate with the scheduler in the\n /// same way Tokio channels do.\n ///\n /// ```\n /// use tokio::task::coop::cooperative;\n /// use futures::channel::mpsc::Receiver;\n /// use futures::stream::StreamExt;\n ///\n /// async fn receive_next(receiver: &mut Receiver) -> Option {\n /// // Use `StreamExt::next` to obtain a `Future` that resolves to the next value\n /// let recv_future = receiver.next();\n /// // Wrap it a cooperative wrapper\n /// let coop_future = cooperative(recv_future);\n /// // And await\n /// coop_future.await\n /// }\n #[inline]\n pub fn cooperative(fut: F) -> Coop {\n Coop { fut }\n }\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\n use wasm_bindgen_test::wasm_bindgen_test as test;\n\n fn get() -> Budget {\n context::budget(|cell| cell.get()).unwrap_or(Budget::unconstrained())\n }\n\n #[test]\n fn budgeting() {\n use std::future::poll_fn;\n use tokio_test::*;\n\n assert!(get().0.is_none());\n\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n\n assert!(get().0.is_none());\n drop(coop);\n assert!(get().0.is_none());\n\n budget(|| {\n assert_eq!(get().0, Budget::initial().0);\n\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 1);\n drop(coop);\n // we didn't make progress\n assert_eq!(get().0, Budget::initial().0);\n\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 1);\n coop.made_progress();\n drop(coop);\n // we _did_ make progress\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 1);\n\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 2);\n coop.made_progress();\n drop(coop);\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 2);\n\n budget(|| {\n assert_eq!(get().0, Budget::initial().0);\n\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 1);\n coop.made_progress();\n drop(coop);\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 1);\n });\n\n assert_eq!(get().0.unwrap(), Budget::initial().0.unwrap() - 2);\n });\n\n assert!(get().0.is_none());\n\n budget(|| {\n let n = get().0.unwrap();\n\n for _ in 0..n {\n let coop = assert_ready!(task::spawn(()).enter(|cx, _| poll_proceed(cx)));\n coop.made_progress();\n }\n\n let mut task = task::spawn(poll_fn(|cx| {\n let coop = std::task::ready!(poll_proceed(cx));\n coop.made_progress();\n Poll::Ready(())\n }));\n\n assert_pending!(task.poll());\n });\n }\n}\n" }, { "path": "tokio/src/task/coop/unconstrained.rs", "content": "use pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// Future for the [`unconstrained`](unconstrained) method.\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n #[must_use = \"Unconstrained does nothing unless polled\"]\n pub struct Unconstrained {\n #[pin]\n inner: F,\n }\n}\n\nimpl Future for Unconstrained\nwhere\n F: Future,\n{\n type Output = ::Output;\n\n cfg_coop! {\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let inner = self.project().inner;\n crate::task::coop::with_unconstrained(|| inner.poll(cx))\n }\n }\n\n cfg_not_coop! {\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let inner = self.project().inner;\n inner.poll(cx)\n }\n }\n}\n\n/// Turn off cooperative scheduling for a future. The future will never be forced to yield by\n/// Tokio. Using this exposes your service to starvation if the unconstrained future never yields\n/// otherwise.\n///\n/// See also the usage example in the [task module](index.html#unconstrained).\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub fn unconstrained(inner: F) -> Unconstrained {\n Unconstrained { inner }\n}\n" }, { "path": "tokio/src/task/join_set.rs", "content": "//! A collection of tasks spawned on a Tokio runtime.\n//!\n//! This module provides the [`JoinSet`] type, a collection which stores a set\n//! of spawned tasks and allows asynchronously awaiting the output of those\n//! tasks as they complete. See the documentation for the [`JoinSet`] type for\n//! details.\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse std::{fmt, panic};\n\nuse crate::runtime::Handle;\nuse crate::task::Id;\nuse crate::task::{unconstrained, AbortHandle, JoinError, JoinHandle, LocalSet};\nuse crate::util::IdleNotifiedSet;\n\n/// A collection of tasks spawned on a Tokio runtime.\n///\n/// A `JoinSet` can be used to await the completion of some or all of the tasks\n/// in the set. The set is not ordered, and the tasks will be returned in the\n/// order they complete.\n///\n/// All of the tasks must have the same return type `T`.\n///\n/// When the `JoinSet` is dropped, all tasks in the `JoinSet` are immediately aborted.\n///\n/// # Examples\n///\n/// Spawn multiple tasks and wait for them.\n///\n/// ```\n/// use tokio::task::JoinSet;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut set = JoinSet::new();\n///\n/// for i in 0..10 {\n/// set.spawn(async move { i });\n/// }\n///\n/// let mut seen = [false; 10];\n/// while let Some(res) = set.join_next().await {\n/// let idx = res.unwrap();\n/// seen[idx] = true;\n/// }\n///\n/// for i in 0..10 {\n/// assert!(seen[i]);\n/// }\n/// # }\n/// ```\n///\n/// # Task ID guarantees\n///\n/// While a task is tracked in a `JoinSet`, that task's ID is unique relative\n/// to all other running tasks in Tokio. For this purpose, tracking a task in a\n/// `JoinSet` is equivalent to holding a [`JoinHandle`] to it. See the [task ID]\n/// documentation for more info.\n///\n/// [`JoinHandle`]: crate::task::JoinHandle\n/// [task ID]: crate::task::Id\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub struct JoinSet {\n inner: IdleNotifiedSet>,\n}\n\n/// A variant of [`task::Builder`] that spawns tasks on a [`JoinSet`] rather\n/// than on the current default runtime.\n///\n/// [`task::Builder`]: crate::task::Builder\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\n#[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\n#[must_use = \"builders do nothing unless used to spawn a task\"]\npub struct Builder<'a, T> {\n joinset: &'a mut JoinSet,\n builder: super::Builder<'a>,\n}\n\nimpl JoinSet {\n /// Create a new `JoinSet`.\n pub fn new() -> Self {\n Self {\n inner: IdleNotifiedSet::new(),\n }\n }\n\n /// Returns the number of tasks currently in the `JoinSet`.\n pub fn len(&self) -> usize {\n self.inner.len()\n }\n\n /// Returns whether the `JoinSet` is empty.\n pub fn is_empty(&self) -> bool {\n self.inner.is_empty()\n }\n}\n\nimpl JoinSet {\n /// Returns a [`Builder`] that can be used to configure a task prior to\n /// spawning it on this `JoinSet`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::task::JoinSet;\n ///\n /// #[tokio::main]\n /// async fn main() -> std::io::Result<()> {\n /// let mut set = JoinSet::new();\n ///\n /// // Use the builder to configure a task's name before spawning it.\n /// set.build_task()\n /// .name(\"my_task\")\n /// .spawn(async { /* ... */ })?;\n ///\n /// Ok(())\n /// }\n /// ```\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n #[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\n pub fn build_task(&mut self) -> Builder<'_, T> {\n Builder {\n builder: super::Builder::new(),\n joinset: self,\n }\n }\n\n /// Spawn the provided task on the `JoinSet`, returning an [`AbortHandle`]\n /// that can be used to remotely cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// `JoinSet`.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn(&mut self, task: F) -> AbortHandle\n where\n F: Future,\n F: Send + 'static,\n T: Send,\n {\n self.insert(crate::spawn(task))\n }\n\n /// Spawn the provided task on the provided runtime and store it in this\n /// `JoinSet` returning an [`AbortHandle`] that can be used to remotely\n /// cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// `JoinSet`.\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_on(&mut self, task: F, handle: &Handle) -> AbortHandle\n where\n F: Future,\n F: Send + 'static,\n T: Send,\n {\n self.insert(handle.spawn(task))\n }\n\n /// Spawn the provided task on the current [`LocalSet`] or [`LocalRuntime`]\n /// and store it in this `JoinSet`, returning an [`AbortHandle`] that can\n /// be used to remotely cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// `JoinSet`.\n ///\n /// # Panics\n ///\n /// This method panics if it is called outside of a `LocalSet` or `LocalRuntime`.\n ///\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`LocalRuntime`]: crate::runtime::LocalRuntime\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_local(&mut self, task: F) -> AbortHandle\n where\n F: Future,\n F: 'static,\n {\n self.insert(crate::task::spawn_local(task))\n }\n\n /// Spawn the provided task on the provided [`LocalSet`] and store it in\n /// this `JoinSet`, returning an [`AbortHandle`] that can be used to\n /// remotely cancel the task.\n ///\n /// Unlike the [`spawn_local`] method, this method may be used to spawn local\n /// tasks on a `LocalSet` that is _not_ currently running. The provided\n /// future will start running whenever the `LocalSet` is next started.\n ///\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`AbortHandle`]: crate::task::AbortHandle\n /// [`spawn_local`]: Self::spawn_local\n #[track_caller]\n pub fn spawn_local_on(&mut self, task: F, local_set: &LocalSet) -> AbortHandle\n where\n F: Future,\n F: 'static,\n {\n self.insert(local_set.spawn_local(task))\n }\n\n /// Spawn the blocking code on the blocking threadpool and store\n /// it in this `JoinSet`, returning an [`AbortHandle`] that can be\n /// used to remotely cancel the task.\n ///\n /// # Examples\n ///\n /// Spawn multiple blocking tasks and wait for them.\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::task::JoinSet;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let mut set = JoinSet::new();\n ///\n /// for i in 0..10 {\n /// set.spawn_blocking(move || { i });\n /// }\n ///\n /// let mut seen = [false; 10];\n /// while let Some(res) = set.join_next().await {\n /// let idx = res.unwrap();\n /// seen[idx] = true;\n /// }\n ///\n /// for i in 0..10 {\n /// assert!(seen[i]);\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking(&mut self, f: F) -> AbortHandle\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send,\n {\n self.insert(crate::runtime::spawn_blocking(f))\n }\n\n /// Spawn the blocking code on the blocking threadpool of the\n /// provided runtime and store it in this `JoinSet`, returning an\n /// [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking_on(&mut self, f: F, handle: &Handle) -> AbortHandle\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send,\n {\n self.insert(handle.spawn_blocking(f))\n }\n\n fn insert(&mut self, jh: JoinHandle) -> AbortHandle {\n let abort = jh.abort_handle();\n let mut entry = self.inner.insert_idle(jh);\n\n // Set the waker that is notified when the task completes.\n entry.with_value_and_context(|jh, ctx| jh.set_join_waker(ctx.waker()));\n abort\n }\n\n /// Waits until one of the tasks in the set completes and returns its output.\n ///\n /// Returns `None` if the set is empty.\n ///\n /// # Cancel Safety\n ///\n /// This method is cancel safe. If `join_next` is used as the event in a `tokio::select!`\n /// statement and some other branch completes first, it is guaranteed that no tasks were\n /// removed from this `JoinSet`.\n pub async fn join_next(&mut self) -> Option> {\n std::future::poll_fn(|cx| self.poll_join_next(cx)).await\n }\n\n /// Waits until one of the tasks in the set completes and returns its\n /// output, along with the [task ID] of the completed task.\n ///\n /// Returns `None` if the set is empty.\n ///\n /// When this method returns an error, then the id of the task that failed can be accessed\n /// using the [`JoinError::id`] method.\n ///\n /// # Cancel Safety\n ///\n /// This method is cancel safe. If `join_next_with_id` is used as the event in a `tokio::select!`\n /// statement and some other branch completes first, it is guaranteed that no tasks were\n /// removed from this `JoinSet`.\n ///\n /// [task ID]: crate::task::Id\n /// [`JoinError::id`]: fn@crate::task::JoinError::id\n pub async fn join_next_with_id(&mut self) -> Option> {\n std::future::poll_fn(|cx| self.poll_join_next_with_id(cx)).await\n }\n\n /// Tries to join one of the tasks in the set that has completed and return its output.\n ///\n /// Returns `None` if there are no completed tasks, or if the set is empty.\n pub fn try_join_next(&mut self) -> Option> {\n // Loop over all notified `JoinHandle`s to find one that's ready, or until none are left.\n loop {\n let mut entry = self.inner.try_pop_notified()?;\n\n let res = entry.with_value_and_context(|jh, ctx| {\n // Since this function is not async and cannot be forced to yield, we should\n // disable budgeting when we want to check for the `JoinHandle` readiness.\n Pin::new(&mut unconstrained(jh)).poll(ctx)\n });\n\n if let Poll::Ready(res) = res {\n let _entry = entry.remove();\n\n return Some(res);\n }\n }\n }\n\n /// Tries to join one of the tasks in the set that has completed and return its output,\n /// along with the [task ID] of the completed task.\n ///\n /// Returns `None` if there are no completed tasks, or if the set is empty.\n ///\n /// When this method returns an error, then the id of the task that failed can be accessed\n /// using the [`JoinError::id`] method.\n ///\n /// [task ID]: crate::task::Id\n /// [`JoinError::id`]: fn@crate::task::JoinError::id\n pub fn try_join_next_with_id(&mut self) -> Option> {\n // Loop over all notified `JoinHandle`s to find one that's ready, or until none are left.\n loop {\n let mut entry = self.inner.try_pop_notified()?;\n\n let res = entry.with_value_and_context(|jh, ctx| {\n // Since this function is not async and cannot be forced to yield, we should\n // disable budgeting when we want to check for the `JoinHandle` readiness.\n Pin::new(&mut unconstrained(jh)).poll(ctx)\n });\n\n if let Poll::Ready(res) = res {\n let entry = entry.remove();\n\n return Some(res.map(|output| (entry.id(), output)));\n }\n }\n }\n\n /// Aborts all tasks and waits for them to finish shutting down.\n ///\n /// Calling this method is equivalent to calling [`abort_all`] and then calling [`join_next`] in\n /// a loop until it returns `None`.\n ///\n /// This method ignores any panics in the tasks shutting down. When this call returns, the\n /// `JoinSet` will be empty.\n ///\n /// [`abort_all`]: fn@Self::abort_all\n /// [`join_next`]: fn@Self::join_next\n pub async fn shutdown(&mut self) {\n self.abort_all();\n while self.join_next().await.is_some() {}\n }\n\n /// Awaits the completion of all tasks in this `JoinSet`, returning a vector of their results.\n ///\n /// The results will be stored in the order they completed not the order they were spawned.\n /// This is a convenience method that is equivalent to calling [`join_next`] in\n /// a loop. If any tasks on the `JoinSet` fail with an [`JoinError`], then this call\n /// to `join_all` will panic and all remaining tasks on the `JoinSet` are\n /// cancelled. To handle errors in any other way, manually call [`join_next`]\n /// in a loop.\n ///\n /// # Examples\n ///\n /// Spawn multiple tasks and `join_all` them.\n ///\n /// ```\n /// use tokio::task::JoinSet;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut set = JoinSet::new();\n ///\n /// for i in 0..3 {\n /// set.spawn(async move {\n /// tokio::time::sleep(Duration::from_secs(3 - i)).await;\n /// i\n /// });\n /// }\n ///\n /// let output = set.join_all().await;\n /// assert_eq!(output, vec![2, 1, 0]);\n /// # }\n /// ```\n ///\n /// Equivalent implementation of `join_all`, using [`join_next`] and loop.\n ///\n /// ```\n /// use tokio::task::JoinSet;\n /// use std::panic;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut set = JoinSet::new();\n ///\n /// for i in 0..3 {\n /// set.spawn(async move {i});\n /// }\n ///\n /// let mut output = Vec::new();\n /// while let Some(res) = set.join_next().await{\n /// match res {\n /// Ok(t) => output.push(t),\n /// Err(err) if err.is_panic() => panic::resume_unwind(err.into_panic()),\n /// Err(err) => panic!(\"{err}\"),\n /// }\n /// }\n /// assert_eq!(output.len(),3);\n /// # }\n /// ```\n /// [`join_next`]: fn@Self::join_next\n /// [`JoinError::id`]: fn@crate::task::JoinError::id\n pub async fn join_all(mut self) -> Vec {\n let mut output = Vec::with_capacity(self.len());\n\n while let Some(res) = self.join_next().await {\n match res {\n Ok(t) => output.push(t),\n Err(err) if err.is_panic() => panic::resume_unwind(err.into_panic()),\n Err(err) => panic!(\"{err}\"),\n }\n }\n output\n }\n\n /// Aborts all tasks on this `JoinSet`.\n ///\n /// This does not remove the tasks from the `JoinSet`. To wait for the tasks to complete\n /// cancellation, you should call `join_next` in a loop until the `JoinSet` is empty.\n pub fn abort_all(&mut self) {\n self.inner.for_each(|jh| jh.abort());\n }\n\n /// Removes all tasks from this `JoinSet` without aborting them.\n ///\n /// The tasks removed by this call will continue to run in the background even if the `JoinSet`\n /// is dropped.\n pub fn detach_all(&mut self) {\n self.inner.drain(drop);\n }\n\n /// Polls for one of the tasks in the set to complete.\n ///\n /// If this returns `Poll::Ready(Some(_))`, then the task that completed is removed from the set.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided `Context` is scheduled\n /// to receive a wakeup when a task in the `JoinSet` completes. Note that on multiple calls to\n /// `poll_join_next`, only the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// # Returns\n ///\n /// This function returns:\n ///\n /// * `Poll::Pending` if the `JoinSet` is not empty but there is no task whose output is\n /// available right now.\n /// * `Poll::Ready(Some(Ok(value)))` if one of the tasks in this `JoinSet` has completed.\n /// The `value` is the return value of one of the tasks that completed.\n /// * `Poll::Ready(Some(Err(err)))` if one of the tasks in this `JoinSet` has panicked or been\n /// aborted. The `err` is the `JoinError` from the panicked/aborted task.\n /// * `Poll::Ready(None)` if the `JoinSet` is empty.\n ///\n /// Note that this method may return `Poll::Pending` even if one of the tasks has completed.\n /// This can happen if the [coop budget] is reached.\n ///\n /// [coop budget]: crate::task::coop#cooperative-scheduling\n pub fn poll_join_next(&mut self, cx: &mut Context<'_>) -> Poll>> {\n // The call to `pop_notified` moves the entry to the `idle` list. It is moved back to\n // the `notified` list if the waker is notified in the `poll` call below.\n let mut entry = match self.inner.pop_notified(cx.waker()) {\n Some(entry) => entry,\n None => {\n if self.is_empty() {\n return Poll::Ready(None);\n } else {\n // The waker was set by `pop_notified`.\n return Poll::Pending;\n }\n }\n };\n\n let res = entry.with_value_and_context(|jh, ctx| Pin::new(jh).poll(ctx));\n\n if let Poll::Ready(res) = res {\n let _entry = entry.remove();\n Poll::Ready(Some(res))\n } else {\n // A JoinHandle generally won't emit a wakeup without being ready unless\n // the coop limit has been reached. We yield to the executor in this\n // case.\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n\n /// Polls for one of the tasks in the set to complete.\n ///\n /// If this returns `Poll::Ready(Some(_))`, then the task that completed is removed from the set.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided `Context` is scheduled\n /// to receive a wakeup when a task in the `JoinSet` completes. Note that on multiple calls to\n /// `poll_join_next`, only the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// # Returns\n ///\n /// This function returns:\n ///\n /// * `Poll::Pending` if the `JoinSet` is not empty but there is no task whose output is\n /// available right now.\n /// * `Poll::Ready(Some(Ok((id, value))))` if one of the tasks in this `JoinSet` has completed.\n /// The `value` is the return value of one of the tasks that completed, and\n /// `id` is the [task ID] of that task.\n /// * `Poll::Ready(Some(Err(err)))` if one of the tasks in this `JoinSet` has panicked or been\n /// aborted. The `err` is the `JoinError` from the panicked/aborted task.\n /// * `Poll::Ready(None)` if the `JoinSet` is empty.\n ///\n /// Note that this method may return `Poll::Pending` even if one of the tasks has completed.\n /// This can happen if the [coop budget] is reached.\n ///\n /// [coop budget]: crate::task::coop#cooperative-scheduling\n /// [task ID]: crate::task::Id\n pub fn poll_join_next_with_id(\n &mut self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n // The call to `pop_notified` moves the entry to the `idle` list. It is moved back to\n // the `notified` list if the waker is notified in the `poll` call below.\n let mut entry = match self.inner.pop_notified(cx.waker()) {\n Some(entry) => entry,\n None => {\n if self.is_empty() {\n return Poll::Ready(None);\n } else {\n // The waker was set by `pop_notified`.\n return Poll::Pending;\n }\n }\n };\n\n let res = entry.with_value_and_context(|jh, ctx| Pin::new(jh).poll(ctx));\n\n if let Poll::Ready(res) = res {\n let entry = entry.remove();\n // If the task succeeded, add the task ID to the output. Otherwise, the\n // `JoinError` will already have the task's ID.\n Poll::Ready(Some(res.map(|output| (entry.id(), output))))\n } else {\n // A JoinHandle generally won't emit a wakeup without being ready unless\n // the coop limit has been reached. We yield to the executor in this\n // case.\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n}\n\nimpl Drop for JoinSet {\n fn drop(&mut self) {\n self.inner.drain(|join_handle| join_handle.abort());\n }\n}\n\nimpl fmt::Debug for JoinSet {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"JoinSet\").field(\"len\", &self.len()).finish()\n }\n}\n\nimpl Default for JoinSet {\n fn default() -> Self {\n Self::new()\n }\n}\n\n/// Collect an iterator of futures into a [`JoinSet`].\n///\n/// This is equivalent to calling [`JoinSet::spawn`] on each element of the iterator.\n///\n/// # Examples\n///\n/// The main example from [`JoinSet`]'s documentation can also be written using [`collect`]:\n///\n/// ```\n/// use tokio::task::JoinSet;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut set: JoinSet<_> = (0..10).map(|i| async move { i }).collect();\n///\n/// let mut seen = [false; 10];\n/// while let Some(res) = set.join_next().await {\n/// let idx = res.unwrap();\n/// seen[idx] = true;\n/// }\n///\n/// for i in 0..10 {\n/// assert!(seen[i]);\n/// }\n/// # }\n/// ```\n///\n/// [`collect`]: std::iter::Iterator::collect\nimpl std::iter::FromIterator for JoinSet\nwhere\n F: Future,\n F: Send + 'static,\n T: Send + 'static,\n{\n fn from_iter>(iter: I) -> Self {\n let mut set = Self::new();\n iter.into_iter().for_each(|task| {\n set.spawn(task);\n });\n set\n }\n}\n\n/// Extend a [`JoinSet`] with futures from an iterator.\n///\n/// This is equivalent to calling [`JoinSet::spawn`] on each element of the iterator.\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use tokio::task::JoinSet;\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let mut set: JoinSet<_> = (0..5).map(|i| async move { i }).collect();\n///\n/// set.extend((5..10).map(|i| async move { i }));\n///\n/// let mut seen = [false; 10];\n/// while let Some(res) = set.join_next().await {\n/// let idx = res.unwrap();\n/// seen[idx] = true;\n/// }\n///\n/// for i in 0..10 {\n/// assert!(seen[i]);\n/// }\n/// }\n/// # }\n/// ```\nimpl std::iter::Extend for JoinSet\nwhere\n F: Future,\n F: Send + 'static,\n T: Send + 'static,\n{\n fn extend(&mut self, iter: I)\n where\n I: IntoIterator,\n {\n iter.into_iter().for_each(|task| {\n self.spawn(task);\n });\n }\n}\n\n// === impl Builder ===\n\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\n#[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\nimpl<'a, T: 'static> Builder<'a, T> {\n /// Assigns a name to the task which will be spawned.\n pub fn name(self, name: &'a str) -> Self {\n let builder = self.builder.name(name);\n Self { builder, ..self }\n }\n\n /// Spawn the provided task with this builder's settings and store it in the\n /// [`JoinSet`], returning an [`AbortHandle`] that can be used to remotely\n /// cancel the task.\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn(self, future: F) -> std::io::Result\n where\n F: Future,\n F: Send + 'static,\n T: Send,\n {\n Ok(self.joinset.insert(self.builder.spawn(future)?))\n }\n\n /// Spawn the provided task on the provided [runtime handle] with this\n /// builder's settings, and store it in the [`JoinSet`].\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n ///\n /// [`AbortHandle`]: crate::task::AbortHandle\n /// [runtime handle]: crate::runtime::Handle\n #[track_caller]\n pub fn spawn_on(self, future: F, handle: &Handle) -> std::io::Result\n where\n F: Future,\n F: Send + 'static,\n T: Send,\n {\n Ok(self.joinset.insert(self.builder.spawn_on(future, handle)?))\n }\n\n /// Spawn the blocking code on the blocking threadpool with this builder's\n /// settings, and store it in the [`JoinSet`].\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`JoinSet`]: crate::task::JoinSet\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking(self, f: F) -> std::io::Result\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send,\n {\n Ok(self.joinset.insert(self.builder.spawn_blocking(f)?))\n }\n\n /// Spawn the blocking code on the blocking threadpool of the provided\n /// runtime handle with this builder's settings, and store it in the\n /// [`JoinSet`].\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// [`JoinSet`]: crate::task::JoinSet\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking_on(self, f: F, handle: &Handle) -> std::io::Result\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send,\n {\n Ok(self\n .joinset\n .insert(self.builder.spawn_blocking_on(f, handle)?))\n }\n\n /// Spawn the provided task on the current [`LocalSet`] or [`LocalRuntime`]\n /// with this builder's settings, and store it in the [`JoinSet`].\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// # Panics\n ///\n /// This method panics if it is called outside of a `LocalSet` or `LocalRuntime`.\n ///\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`LocalRuntime`]: crate::runtime::LocalRuntime\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_local(self, future: F) -> std::io::Result\n where\n F: Future,\n F: 'static,\n {\n Ok(self.joinset.insert(self.builder.spawn_local(future)?))\n }\n\n /// Spawn the provided task on the provided [`LocalSet`] with this builder's\n /// settings, and store it in the [`JoinSet`].\n ///\n /// # Returns\n ///\n /// An [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// [`LocalSet`]: crate::task::LocalSet\n /// [`AbortHandle`]: crate::task::AbortHandle\n #[track_caller]\n pub fn spawn_local_on(self, future: F, local_set: &LocalSet) -> std::io::Result\n where\n F: Future,\n F: 'static,\n {\n Ok(self\n .joinset\n .insert(self.builder.spawn_local_on(future, local_set)?))\n }\n}\n\n// Manual `Debug` impl so that `Builder` is `Debug` regardless of whether `T` is\n// `Debug`.\n#[cfg(all(tokio_unstable, feature = \"tracing\"))]\n#[cfg_attr(docsrs, doc(cfg(all(tokio_unstable, feature = \"tracing\"))))]\nimpl<'a, T> fmt::Debug for Builder<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"join_set::Builder\")\n .field(\"joinset\", &self.joinset)\n .field(\"builder\", &self.builder)\n .finish()\n }\n}\n" }, { "path": "tokio/src/task/local.rs", "content": "//! Runs `!Send` futures on the current thread.\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::{Arc, Mutex};\nuse crate::runtime;\nuse crate::runtime::task::{\n self, JoinHandle, LocalOwnedTasks, SpawnLocation, Task, TaskHarnessScheduleHooks,\n};\nuse crate::runtime::{context, ThreadId, BOX_FUTURE_THRESHOLD};\nuse crate::sync::AtomicWaker;\nuse crate::util::trace::SpawnMeta;\nuse crate::util::RcCell;\n\nuse std::cell::Cell;\nuse std::collections::VecDeque;\nuse std::fmt;\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::mem;\nuse std::pin::Pin;\nuse std::rc::Rc;\nuse std::task::Poll;\n\nuse pin_project_lite::pin_project;\n\ncfg_rt! {\n /// A set of tasks which are executed on the same thread.\n ///\n /// In some cases, it is necessary to run one or more futures that do not\n /// implement [`Send`] and thus are unsafe to send between threads. In these\n /// cases, a [local task set] may be used to schedule one or more `!Send`\n /// futures to run together on the same thread.\n ///\n /// For example, the following code will not compile:\n ///\n /// ```rust,compile_fail\n /// use std::rc::Rc;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// // `Rc` does not implement `Send`, and thus may not be sent between\n /// // threads safely.\n /// let nonsend_data = Rc::new(\"my nonsend data...\");\n ///\n /// let nonsend_data = nonsend_data.clone();\n /// // Because the `async` block here moves `nonsend_data`, the future is `!Send`.\n /// // Since `tokio::spawn` requires the spawned future to implement `Send`, this\n /// // will not compile.\n /// tokio::spawn(async move {\n /// println!(\"{}\", nonsend_data);\n /// // ...\n /// }).await.unwrap();\n /// }\n /// ```\n ///\n /// # Use with `run_until`\n ///\n /// To spawn `!Send` futures, we can use a local task set to schedule them\n /// on the thread calling [`Runtime::block_on`]. When running inside of the\n /// local task set, we can use [`task::spawn_local`], which can spawn\n /// `!Send` futures. For example:\n ///\n /// ```rust\n /// use std::rc::Rc;\n /// use tokio::task;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let nonsend_data = Rc::new(\"my nonsend data...\");\n ///\n /// // Construct a local task set that can run `!Send` futures.\n /// let local = task::LocalSet::new();\n ///\n /// // Run the local task set.\n /// local.run_until(async move {\n /// let nonsend_data = nonsend_data.clone();\n /// // `spawn_local` ensures that the future is spawned on the local\n /// // task set.\n /// task::spawn_local(async move {\n /// println!(\"{}\", nonsend_data);\n /// // ...\n /// }).await.unwrap();\n /// }).await;\n /// # }\n /// ```\n /// **Note:** The `run_until` method can only be used in `#[tokio::main]`,\n /// `#[tokio::test]` or directly inside a call to [`Runtime::block_on`]. It\n /// cannot be used inside a task spawned with `tokio::spawn`.\n ///\n /// ## Awaiting a `LocalSet`\n ///\n /// Additionally, a `LocalSet` itself implements `Future`, completing when\n /// *all* tasks spawned on the `LocalSet` complete. This can be used to run\n /// several futures on a `LocalSet` and drive the whole set until they\n /// complete. For example,\n ///\n /// ```rust\n /// use tokio::{task, time};\n /// use std::rc::Rc;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let nonsend_data = Rc::new(\"world\");\n /// let local = task::LocalSet::new();\n ///\n /// let nonsend_data2 = nonsend_data.clone();\n /// local.spawn_local(async move {\n /// // ...\n /// println!(\"hello {}\", nonsend_data2)\n /// });\n ///\n /// local.spawn_local(async move {\n /// time::sleep(time::Duration::from_millis(100)).await;\n /// println!(\"goodbye {}\", nonsend_data)\n /// });\n ///\n /// // ...\n ///\n /// local.await;\n /// # }\n /// ```\n /// **Note:** Awaiting a `LocalSet` can only be done inside\n /// `#[tokio::main]`, `#[tokio::test]` or directly inside a call to\n /// [`Runtime::block_on`]. It cannot be used inside a task spawned with\n /// `tokio::spawn`.\n ///\n /// ## Use inside `tokio::spawn`\n ///\n /// The two methods mentioned above cannot be used inside `tokio::spawn`, so\n /// to spawn `!Send` futures from inside `tokio::spawn`, we need to do\n /// something else. The solution is to create the `LocalSet` somewhere else,\n /// and communicate with it using an [`mpsc`] channel.\n ///\n /// The following example puts the `LocalSet` inside a new thread.\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Builder;\n /// use tokio::sync::{mpsc, oneshot};\n /// use tokio::task::LocalSet;\n ///\n /// // This struct describes the task you want to spawn. Here we include\n /// // some simple examples. The oneshot channel allows sending a response\n /// // to the spawner.\n /// #[derive(Debug)]\n /// enum Task {\n /// PrintNumber(u32),\n /// AddOne(u32, oneshot::Sender),\n /// }\n ///\n /// #[derive(Clone)]\n /// struct LocalSpawner {\n /// send: mpsc::UnboundedSender,\n /// }\n ///\n /// impl LocalSpawner {\n /// pub fn new() -> Self {\n /// let (send, mut recv) = mpsc::unbounded_channel();\n ///\n /// let rt = Builder::new_current_thread()\n /// .enable_all()\n /// .build()\n /// .unwrap();\n ///\n /// std::thread::spawn(move || {\n /// let local = LocalSet::new();\n ///\n /// local.spawn_local(async move {\n /// while let Some(new_task) = recv.recv().await {\n /// tokio::task::spawn_local(run_task(new_task));\n /// }\n /// // If the while loop returns, then all the LocalSpawner\n /// // objects have been dropped.\n /// });\n ///\n /// // This will return once all senders are dropped and all\n /// // spawned tasks have returned.\n /// rt.block_on(local);\n /// });\n ///\n /// Self {\n /// send,\n /// }\n /// }\n ///\n /// pub fn spawn(&self, task: Task) {\n /// self.send.send(task).expect(\"Thread with LocalSet has shut down.\");\n /// }\n /// }\n ///\n /// // This task may do !Send stuff. We use printing a number as an example,\n /// // but it could be anything.\n /// //\n /// // The Task struct is an enum to support spawning many different kinds\n /// // of operations.\n /// async fn run_task(task: Task) {\n /// match task {\n /// Task::PrintNumber(n) => {\n /// println!(\"{}\", n);\n /// },\n /// Task::AddOne(n, response) => {\n /// // We ignore failures to send the response.\n /// let _ = response.send(n + 1);\n /// },\n /// }\n /// }\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let spawner = LocalSpawner::new();\n ///\n /// let (send, response) = oneshot::channel();\n /// spawner.spawn(Task::AddOne(10, send));\n /// let eleven = response.await.unwrap();\n /// assert_eq!(eleven, 11);\n /// }\n /// # }\n /// ```\n ///\n /// [`Send`]: trait@std::marker::Send\n /// [local task set]: struct@LocalSet\n /// [`Runtime::block_on`]: method@crate::runtime::Runtime::block_on\n /// [`task::spawn_local`]: fn@spawn_local\n /// [`mpsc`]: mod@crate::sync::mpsc\n pub struct LocalSet {\n /// Current scheduler tick.\n tick: Cell,\n\n /// State available from thread-local.\n context: Rc,\n\n /// This type should not be Send.\n _not_send: PhantomData<*const ()>,\n }\n}\n\n/// State available from the thread-local.\nstruct Context {\n /// State shared between threads.\n shared: Arc,\n\n /// True if a task panicked without being handled and the local set is\n /// configured to shutdown on unhandled panic.\n unhandled_panic: Cell,\n}\n\n/// `LocalSet` state shared between threads.\nstruct Shared {\n /// # Safety\n ///\n /// This field must *only* be accessed from the thread that owns the\n /// `LocalSet` (i.e., `Thread::current().id() == owner`).\n local_state: LocalState,\n\n /// Remote run queue sender.\n queue: Mutex>>>>,\n\n /// Wake the `LocalSet` task.\n waker: AtomicWaker,\n\n /// How to respond to unhandled task panics.\n #[cfg(tokio_unstable)]\n pub(crate) unhandled_panic: crate::runtime::UnhandledPanic,\n}\n\n/// Tracks the `LocalSet` state that must only be accessed from the thread that\n/// created the `LocalSet`.\nstruct LocalState {\n /// The `ThreadId` of the thread that owns the `LocalSet`.\n owner: ThreadId,\n\n /// Local run queue sender and receiver.\n local_queue: UnsafeCell>>>,\n\n /// Collection of all active tasks spawned onto this executor.\n owned: LocalOwnedTasks>,\n}\n\npin_project! {\n #[derive(Debug)]\n struct RunUntil<'a, F> {\n local_set: &'a LocalSet,\n #[pin]\n future: F,\n }\n}\n\ntokio_thread_local!(static CURRENT: LocalData = const { LocalData {\n ctx: RcCell::new(),\n wake_on_schedule: Cell::new(false),\n} });\n\nstruct LocalData {\n ctx: RcCell,\n wake_on_schedule: Cell,\n}\n\nimpl LocalData {\n /// Should be called except when we call `LocalSet::enter`.\n /// Especially when we poll a `LocalSet`.\n #[must_use = \"dropping this guard will reset the entered state\"]\n fn enter(&self, ctx: Rc) -> LocalDataEnterGuard<'_> {\n let ctx = self.ctx.replace(Some(ctx));\n let wake_on_schedule = self.wake_on_schedule.replace(false);\n LocalDataEnterGuard {\n local_data_ref: self,\n ctx,\n wake_on_schedule,\n }\n }\n}\n\n/// A guard for `LocalData::enter()`\nstruct LocalDataEnterGuard<'a> {\n local_data_ref: &'a LocalData,\n ctx: Option>,\n wake_on_schedule: bool,\n}\n\nimpl<'a> Drop for LocalDataEnterGuard<'a> {\n fn drop(&mut self) {\n self.local_data_ref.ctx.set(self.ctx.take());\n self.local_data_ref\n .wake_on_schedule\n .set(self.wake_on_schedule)\n }\n}\n\ncfg_rt! {\n /// Spawns a `!Send` future on the current [`LocalSet`] or [`LocalRuntime`].\n ///\n /// This is possible when either using one of these types\n /// explicitly, or (with `tokio_unstable`) by opting to use the\n /// `\"local\"` runtime flavor in `tokio::main`:\n ///\n /// ```ignore\n /// #[tokio::main(flavor = \"local\")]\n /// ```\n ///\n /// The spawned future will run on the same thread that called `spawn_local`.\n ///\n /// The provided future will start running in the background immediately\n /// when `spawn_local` is called, even if you don't await the returned\n /// `JoinHandle`.\n ///\n /// # Panics\n ///\n /// This function panics if called outside of a [`LocalSet`] or [`LocalRuntime`].\n ///\n /// Note that if [`tokio::spawn`] is used from within a `LocalSet`, the\n /// resulting new task will _not_ be inside the `LocalSet`, so you must use\n /// `spawn_local` if you want to stay within the `LocalSet`.\n ///\n /// # Examples\n ///\n /// With `LocalSet`:\n ///\n /// ```rust\n /// use std::rc::Rc;\n /// use tokio::task;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let nonsend_data = Rc::new(\"my nonsend data...\");\n ///\n /// let local = task::LocalSet::new();\n ///\n /// // Run the local task set.\n /// local.run_until(async move {\n /// let nonsend_data = nonsend_data.clone();\n /// task::spawn_local(async move {\n /// println!(\"{}\", nonsend_data);\n /// // ...\n /// }).await.unwrap();\n /// }).await;\n /// # }\n /// ```\n /// With local runtime flavor ([Unstable API][unstable] only).\n ///\n /// ```rust\n /// # #[cfg(tokio_unstable)]\n /// #[tokio::main(flavor = \"local\")]\n /// async fn main() {\n /// let join = tokio::task::spawn_local(async {\n /// println!(\"my nonsend data...\")\n /// });\n ///\n /// join.await.unwrap()\n /// }\n /// # #[cfg(not(tokio_unstable))]\n /// # fn main() {}\n ///\n /// ```\n ///\n /// [`LocalSet`]: struct@crate::task::LocalSet\n /// [`LocalRuntime`]: struct@crate::runtime::LocalRuntime\n /// [`tokio::spawn`]: fn@crate::task::spawn\n /// [unstable]: ../../tokio/index.html#unstable-features\n #[track_caller]\n pub fn spawn_local(future: F) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let fut_size = std::mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n spawn_local_inner(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n spawn_local_inner(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n\n #[track_caller]\n pub(super) fn spawn_local_inner(future: F, meta: SpawnMeta<'_>) -> JoinHandle\n where F: Future + 'static,\n F::Output: 'static\n {\n use crate::runtime::{context, task};\n\n let mut future = Some(future);\n\n let res = context::with_current(|handle| {\n Some(if handle.is_local() {\n if !handle.can_spawn_local_on_local_runtime() {\n return None;\n }\n\n let future = future.take().unwrap();\n\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(\n target_arch = \"aarch64\",\n target_arch = \"x86\",\n target_arch = \"x86_64\"\n )\n ))]\n let future = task::trace::Trace::root(future);\n let id = task::Id::next();\n let task = crate::util::trace::task(future, \"task\", meta, id.as_u64());\n\n // safety: we have verified that this is a `LocalRuntime` owned by the current thread\n unsafe { handle.spawn_local(task, id, meta.spawned_at) }\n } else {\n match CURRENT.with(|LocalData { ctx, .. }| ctx.get()) {\n None => panic!(\"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"),\n Some(cx) => cx.spawn(future.take().unwrap(), meta)\n }\n })\n });\n\n match res {\n Ok(None) => panic!(\"Local tasks can only be spawned on a LocalRuntime from the thread the runtime was created on\"),\n Ok(Some(join_handle)) => join_handle,\n Err(_) => match CURRENT.with(|LocalData { ctx, .. }| ctx.get()) {\n None => panic!(\"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"),\n Some(cx) => cx.spawn(future.unwrap(), meta)\n }\n }\n }\n}\n\n/// Initial queue capacity.\nconst INITIAL_CAPACITY: usize = 64;\n\n/// Max number of tasks to poll per tick.\nconst MAX_TASKS_PER_TICK: usize = 61;\n\n/// How often it check the remote queue first.\nconst REMOTE_FIRST_INTERVAL: u8 = 31;\n\n/// Context guard for `LocalSet`\npub struct LocalEnterGuard {\n ctx: Option>,\n\n /// Distinguishes whether the context was entered or being polled.\n /// When we enter it, the value `wake_on_schedule` is set. In this case\n /// `spawn_local` refers the context, whereas it is not being polled now.\n wake_on_schedule: bool,\n}\n\nimpl Drop for LocalEnterGuard {\n fn drop(&mut self) {\n CURRENT.with(\n |LocalData {\n ctx,\n wake_on_schedule,\n }| {\n ctx.set(self.ctx.take());\n wake_on_schedule.set(self.wake_on_schedule);\n },\n );\n }\n}\n\nimpl fmt::Debug for LocalEnterGuard {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"LocalEnterGuard\").finish()\n }\n}\n\nimpl LocalSet {\n /// Returns a new local task set.\n pub fn new() -> LocalSet {\n let owner = context::thread_id().expect(\"cannot create LocalSet during thread shutdown\");\n\n LocalSet {\n tick: Cell::new(0),\n context: Rc::new(Context {\n shared: Arc::new(Shared {\n local_state: LocalState {\n owner,\n owned: LocalOwnedTasks::new(),\n local_queue: UnsafeCell::new(VecDeque::with_capacity(INITIAL_CAPACITY)),\n },\n queue: Mutex::new(Some(VecDeque::with_capacity(INITIAL_CAPACITY))),\n waker: AtomicWaker::new(),\n #[cfg(tokio_unstable)]\n unhandled_panic: crate::runtime::UnhandledPanic::Ignore,\n }),\n unhandled_panic: Cell::new(false),\n }),\n _not_send: PhantomData,\n }\n }\n\n /// Enters the context of this `LocalSet`.\n ///\n /// The [`spawn_local`] method will spawn tasks on the `LocalSet` whose\n /// context you are inside.\n ///\n /// [`spawn_local`]: fn@crate::task::spawn_local\n pub fn enter(&self) -> LocalEnterGuard {\n CURRENT.with(\n |LocalData {\n ctx,\n wake_on_schedule,\n ..\n }| {\n let ctx = ctx.replace(Some(self.context.clone()));\n let wake_on_schedule = wake_on_schedule.replace(true);\n LocalEnterGuard {\n ctx,\n wake_on_schedule,\n }\n },\n )\n }\n\n /// Spawns a `!Send` task onto the local task set.\n ///\n /// This task is guaranteed to be run on the current thread.\n ///\n /// Unlike the free function [`spawn_local`], this method may be used to\n /// spawn local tasks when the `LocalSet` is _not_ running. The provided\n /// future will start running once the `LocalSet` is next started, even if\n /// you don't await the returned `JoinHandle`.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::task;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let local = task::LocalSet::new();\n ///\n /// // Spawn a future on the local set. This future will be run when\n /// // we call `run_until` to drive the task set.\n /// local.spawn_local(async {\n /// // ...\n /// });\n ///\n /// // Run the local task set.\n /// local.run_until(async move {\n /// // ...\n /// }).await;\n ///\n /// // When `run` finishes, we can spawn _more_ futures, which will\n /// // run in subsequent calls to `run_until`.\n /// local.spawn_local(async {\n /// // ...\n /// });\n ///\n /// local.run_until(async move {\n /// // ...\n /// }).await;\n /// # }\n /// ```\n /// [`spawn_local`]: fn@spawn_local\n #[track_caller]\n pub fn spawn_local(&self, future: F) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let fut_size = mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n self.spawn_named(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n self.spawn_named(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n /// Runs a future to completion on the provided runtime, driving any local\n /// futures spawned on this task set on the current thread.\n ///\n /// This runs the given future on the runtime, blocking until it is\n /// complete, and yielding its resolved result. Any tasks or timers which\n /// the future spawns internally will be executed on the runtime. The future\n /// may also call [`spawn_local`] to `spawn_local` additional local futures on the\n /// current thread.\n ///\n /// This method should not be called from an asynchronous context.\n ///\n /// # Panics\n ///\n /// This function panics if the executor is at capacity, if the provided\n /// future panics, or if called within an asynchronous execution context.\n ///\n /// # Notes\n ///\n /// Since this function internally calls [`Runtime::block_on`], and drives\n /// futures in the local task set inside that call to `block_on`, the local\n /// futures may not use [in-place blocking]. If a blocking call needs to be\n /// issued from a local task, the [`spawn_blocking`] API may be used instead.\n ///\n /// For example, this will panic:\n /// ```should_panic,ignore-wasm\n /// use tokio::runtime::Runtime;\n /// use tokio::task;\n ///\n /// let rt = Runtime::new().unwrap();\n /// let local = task::LocalSet::new();\n /// local.block_on(&rt, async {\n /// let join = task::spawn_local(async {\n /// let blocking_result = task::block_in_place(|| {\n /// // ...\n /// });\n /// // ...\n /// });\n /// join.await.unwrap();\n /// })\n /// ```\n /// This, however, will not panic:\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::runtime::Runtime;\n /// use tokio::task;\n ///\n /// let rt = Runtime::new().unwrap();\n /// let local = task::LocalSet::new();\n /// local.block_on(&rt, async {\n /// let join = task::spawn_local(async {\n /// let blocking_result = task::spawn_blocking(|| {\n /// // ...\n /// }).await;\n /// // ...\n /// });\n /// join.await.unwrap();\n /// })\n /// # }\n /// ```\n ///\n /// [`spawn_local`]: fn@spawn_local\n /// [`Runtime::block_on`]: method@crate::runtime::Runtime::block_on\n /// [in-place blocking]: fn@crate::task::block_in_place\n /// [`spawn_blocking`]: fn@crate::task::spawn_blocking\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn block_on(&self, rt: &crate::runtime::Runtime, future: F) -> F::Output\n where\n F: Future,\n {\n rt.block_on(self.run_until(future))\n }\n\n /// Runs a future to completion on the local set, returning its output.\n ///\n /// This returns a future that runs the given future with a local set,\n /// allowing it to call [`spawn_local`] to spawn additional `!Send` futures.\n /// Any local futures spawned on the local set will be driven in the\n /// background until the future passed to `run_until` completes. When the future\n /// passed to `run_until` finishes, any local futures which have not completed\n /// will remain on the local set, and will be driven on subsequent calls to\n /// `run_until` or when [awaiting the local set] itself.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe when `future` is cancel safe.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::task;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// task::LocalSet::new().run_until(async {\n /// task::spawn_local(async move {\n /// // ...\n /// }).await.unwrap();\n /// // ...\n /// }).await;\n /// # }\n /// ```\n ///\n /// [`spawn_local`]: fn@spawn_local\n /// [awaiting the local set]: #awaiting-a-localset\n pub async fn run_until(&self, future: F) -> F::Output\n where\n F: Future,\n {\n let run_until = RunUntil {\n future,\n local_set: self,\n };\n run_until.await\n }\n\n #[track_caller]\n pub(in crate::task) fn spawn_named(\n &self,\n future: F,\n meta: SpawnMeta<'_>,\n ) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n self.spawn_named_inner(future, meta)\n }\n\n #[track_caller]\n fn spawn_named_inner(&self, future: F, meta: SpawnMeta<'_>) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let handle = self.context.spawn(future, meta);\n\n // Because a task was spawned from *outside* the `LocalSet`, wake the\n // `LocalSet` future to execute the new task, if it hasn't been woken.\n //\n // Spawning via the free fn `spawn` does not require this, as it can\n // only be called from *within* a future executing on the `LocalSet` —\n // in that case, the `LocalSet` must already be awake.\n self.context.shared.waker.wake();\n handle\n }\n\n /// Ticks the scheduler, returning whether the local future needs to be\n /// notified again.\n fn tick(&self) -> bool {\n for _ in 0..MAX_TASKS_PER_TICK {\n // Make sure we didn't hit an unhandled panic\n assert!(!self.context.unhandled_panic.get(), \"a spawned task panicked and the LocalSet is configured to shutdown on unhandled panic\");\n\n match self.next_task() {\n // Run the task\n //\n // Safety: As spawned tasks are `!Send`, `run_unchecked` must be\n // used. We are responsible for maintaining the invariant that\n // `run_unchecked` is only called on threads that spawned the\n // task initially. Because `LocalSet` itself is `!Send`, and\n // `spawn_local` spawns into the `LocalSet` on the current\n // thread, the invariant is maintained.\n Some(task) => crate::task::coop::budget(|| task.run()),\n // We have fully drained the queue of notified tasks, so the\n // local future doesn't need to be notified again — it can wait\n // until something else wakes a task in the local set.\n None => return false,\n }\n }\n\n true\n }\n\n fn next_task(&self) -> Option>> {\n let tick = self.tick.get();\n self.tick.set(tick.wrapping_add(1));\n\n let task = if tick % REMOTE_FIRST_INTERVAL == 0 {\n self.context\n .shared\n .queue\n .lock()\n .as_mut()\n .and_then(|queue| queue.pop_front())\n .or_else(|| self.pop_local())\n } else {\n self.pop_local().or_else(|| {\n self.context\n .shared\n .queue\n .lock()\n .as_mut()\n .and_then(VecDeque::pop_front)\n })\n };\n\n task.map(|task| unsafe {\n // Safety: because the `LocalSet` itself is `!Send`, we know we are\n // on the same thread if we have access to the `LocalSet`, and can\n // therefore access the local run queue.\n self.context.shared.local_state.assert_owner(task)\n })\n }\n\n fn pop_local(&self) -> Option>> {\n unsafe {\n // Safety: because the `LocalSet` itself is `!Send`, we know we are\n // on the same thread if we have access to the `LocalSet`, and can\n // therefore access the local run queue.\n self.context.shared.local_state.task_pop_front()\n }\n }\n\n fn with(&self, f: impl FnOnce() -> T) -> T {\n CURRENT.with(|local_data| {\n let _guard = local_data.enter(self.context.clone());\n f()\n })\n }\n\n /// This method is like `with`, but it just calls `f` without setting the thread-local if that\n /// fails.\n fn with_if_possible(&self, f: impl FnOnce() -> T) -> T {\n let mut f = Some(f);\n\n let res = CURRENT.try_with(|local_data| {\n let _guard = local_data.enter(self.context.clone());\n (f.take().unwrap())()\n });\n\n match res {\n Ok(res) => res,\n Err(_access_error) => (f.take().unwrap())(),\n }\n }\n\n /// Returns the [`Id`] of the current [`LocalSet`] runtime.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::task;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let local_set = task::LocalSet::new();\n /// println!(\"Local set id: {}\", local_set.id());\n /// # }\n /// ```\n ///\n /// [`Id`]: struct@crate::runtime::Id\n pub fn id(&self) -> runtime::Id {\n runtime::Id::new(self.context.shared.local_state.owned.id)\n }\n}\n\ncfg_unstable! {\n impl LocalSet {\n /// Configure how the `LocalSet` responds to an unhandled panic on a\n /// spawned task.\n ///\n /// By default, an unhandled panic (i.e. a panic not caught by\n /// [`std::panic::catch_unwind`]) has no impact on the `LocalSet`'s\n /// execution. The panic is error value is forwarded to the task's\n /// [`JoinHandle`] and all other spawned tasks continue running.\n ///\n /// The `unhandled_panic` option enables configuring this behavior.\n ///\n /// * `UnhandledPanic::Ignore` is the default behavior. Panics on\n /// spawned tasks have no impact on the `LocalSet`'s execution.\n /// * `UnhandledPanic::ShutdownRuntime` will force the `LocalSet` to\n /// shutdown immediately when a spawned task panics even if that\n /// task's `JoinHandle` has not been dropped. All other spawned tasks\n /// will immediately terminate and further calls to\n /// [`LocalSet::block_on`] and [`LocalSet::run_until`] will panic.\n ///\n /// # Panics\n ///\n /// This method panics if called after the `LocalSet` has started\n /// running.\n ///\n /// # Unstable\n ///\n /// This option is currently unstable and its implementation is\n /// incomplete. The API may change or be removed in the future. See\n /// tokio-rs/tokio#4516 for more details.\n ///\n /// # Examples\n ///\n /// The following demonstrates a `LocalSet` configured to shutdown on\n /// panic. The first spawned task panics and results in the `LocalSet`\n /// shutting down. The second spawned task never has a chance to\n /// execute. The call to `run_until` will panic due to the runtime being\n /// forcibly shutdown.\n ///\n /// ```should_panic\n /// use tokio::runtime::UnhandledPanic;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// tokio::task::LocalSet::new()\n /// .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n /// .run_until(async {\n /// tokio::task::spawn_local(async { panic!(\"boom\"); });\n /// tokio::task::spawn_local(async {\n /// // This task never completes\n /// });\n ///\n /// // Do some work, but `run_until` will panic before it completes\n /// # loop { tokio::task::yield_now().await; }\n /// })\n /// .await;\n /// # }\n /// ```\n ///\n /// [`JoinHandle`]: struct@crate::task::JoinHandle\n pub fn unhandled_panic(&mut self, behavior: crate::runtime::UnhandledPanic) -> &mut Self {\n // TODO: This should be set as a builder\n Rc::get_mut(&mut self.context)\n .and_then(|ctx| Arc::get_mut(&mut ctx.shared))\n .expect(\"Unhandled Panic behavior modified after starting LocalSet\")\n .unhandled_panic = behavior;\n self\n }\n }\n}\n\nimpl fmt::Debug for LocalSet {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"LocalSet\").finish()\n }\n}\n\nimpl Future for LocalSet {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll {\n let _no_blocking = crate::runtime::context::disallow_block_in_place();\n\n // Register the waker before starting to work\n self.context.shared.waker.register_by_ref(cx.waker());\n\n if self.with(|| self.tick()) {\n // If `tick` returns true, we need to notify the local future again:\n // there are still tasks remaining in the run queue.\n cx.waker().wake_by_ref();\n Poll::Pending\n\n // Safety: called from the thread that owns `LocalSet`. Because\n // `LocalSet` is `!Send`, this is safe.\n } else if unsafe { self.context.shared.local_state.owned_is_empty() } {\n // If the scheduler has no remaining futures, we're done!\n Poll::Ready(())\n } else {\n // There are still futures in the local set, but we've polled all the\n // futures in the run queue. Therefore, we can just return Pending\n // since the remaining futures will be woken from somewhere else.\n Poll::Pending\n }\n }\n}\n\nimpl Default for LocalSet {\n fn default() -> LocalSet {\n LocalSet::new()\n }\n}\n\nimpl Drop for LocalSet {\n fn drop(&mut self) {\n self.with_if_possible(|| {\n let _no_blocking = crate::runtime::context::disallow_block_in_place();\n\n // Shut down all tasks in the LocalOwnedTasks and close it to\n // prevent new tasks from ever being added.\n unsafe {\n // Safety: called from the thread that owns `LocalSet`\n self.context.shared.local_state.close_and_shutdown_all();\n }\n\n // We already called shutdown on all tasks above, so there is no\n // need to call shutdown.\n\n // Safety: note that this *intentionally* bypasses the unsafe\n // `Shared::local_queue()` method. This is in order to avoid the\n // debug assertion that we are on the thread that owns the\n // `LocalSet`, because on some systems (e.g. at least some macOS\n // versions), attempting to get the current thread ID can panic due\n // to the thread's local data that stores the thread ID being\n // dropped *before* the `LocalSet`.\n //\n // Despite avoiding the assertion here, it is safe for us to access\n // the local queue in `Drop`, because the `LocalSet` itself is\n // `!Send`, so we can reasonably guarantee that it will not be\n // `Drop`ped from another thread.\n let local_queue = unsafe {\n // Safety: called from the thread that owns `LocalSet`\n self.context.shared.local_state.take_local_queue()\n };\n for task in local_queue {\n drop(task);\n }\n\n // Take the queue from the Shared object to prevent pushing\n // notifications to it in the future.\n let queue = self.context.shared.queue.lock().take().unwrap();\n for task in queue {\n drop(task);\n }\n\n // Safety: called from the thread that owns `LocalSet`\n assert!(unsafe { self.context.shared.local_state.owned_is_empty() });\n });\n }\n}\n\n// === impl Context ===\n\nimpl Context {\n #[track_caller]\n fn spawn(&self, future: F, meta: SpawnMeta<'_>) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n let id = crate::runtime::task::Id::next();\n let future = crate::util::trace::task(future, \"local\", meta, id.as_u64());\n\n // Safety: called from the thread that owns the `LocalSet`\n let (handle, notified) = {\n self.shared.local_state.assert_called_from_owner_thread();\n self.shared.local_state.owned.bind(\n future,\n self.shared.clone(),\n id,\n SpawnLocation::capture(),\n )\n };\n\n if let Some(notified) = notified {\n self.shared.schedule(notified);\n }\n\n handle\n }\n}\n\n// === impl LocalFuture ===\n\nimpl Future for RunUntil<'_, T> {\n type Output = T::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll {\n let me = self.project();\n\n me.local_set.with(|| {\n me.local_set\n .context\n .shared\n .waker\n .register_by_ref(cx.waker());\n\n let _no_blocking = crate::runtime::context::disallow_block_in_place();\n let f = me.future;\n\n if let Poll::Ready(output) = f.poll(cx) {\n return Poll::Ready(output);\n }\n\n if me.local_set.tick() {\n // If `tick` returns `true`, we need to notify the local future again:\n // there are still tasks remaining in the run queue.\n cx.waker().wake_by_ref();\n }\n\n Poll::Pending\n })\n }\n}\n\nimpl Shared {\n /// Schedule the provided task on the scheduler.\n fn schedule(&self, task: task::Notified>) {\n CURRENT.with(|localdata| {\n match localdata.ctx.get() {\n // If the current `LocalSet` is being polled, we don't need to wake it.\n // When we `enter` it, then the value `wake_on_schedule` is set to be true.\n // In this case it is not being polled, so we need to wake it.\n Some(cx) if cx.shared.ptr_eq(self) && !localdata.wake_on_schedule.get() => unsafe {\n // Safety: if the current `LocalSet` context points to this\n // `LocalSet`, then we are on the thread that owns it.\n cx.shared.local_state.task_push_back(task);\n },\n\n // We are on the thread that owns the `LocalSet`, so we can\n // wake to the local queue.\n _ if context::thread_id().ok() == Some(self.local_state.owner) => {\n unsafe {\n // Safety: we just checked that the thread ID matches\n // the localset's owner, so this is safe.\n self.local_state.task_push_back(task);\n }\n // We still have to wake the `LocalSet`, because it isn't\n // currently being polled.\n self.waker.wake();\n }\n\n // We are *not* on the thread that owns the `LocalSet`, so we\n // have to wake to the remote queue.\n _ => {\n // First, check whether the queue is still there (if not, the\n // LocalSet is dropped). Then push to it if so, and if not,\n // do nothing.\n let mut lock = self.queue.lock();\n\n if let Some(queue) = lock.as_mut() {\n queue.push_back(task);\n drop(lock);\n self.waker.wake();\n }\n }\n }\n });\n }\n\n fn ptr_eq(&self, other: &Shared) -> bool {\n std::ptr::eq(self, other)\n }\n}\n\n// This is safe because (and only because) we *pinky pwomise* to never touch the\n// local run queue except from the thread that owns the `LocalSet`.\nunsafe impl Sync for Shared {}\n\nimpl task::Schedule for Arc {\n fn release(&self, task: &Task) -> Option> {\n // Safety, this is always called from the thread that owns `LocalSet`\n unsafe { self.local_state.task_remove(task) }\n }\n\n fn schedule(&self, task: task::Notified) {\n Shared::schedule(self, task);\n }\n\n // localset does not currently support task hooks\n fn hooks(&self) -> TaskHarnessScheduleHooks {\n TaskHarnessScheduleHooks {\n task_terminate_callback: None,\n }\n }\n\n cfg_unstable! {\n fn unhandled_panic(&self) {\n use crate::runtime::UnhandledPanic;\n\n match self.unhandled_panic {\n UnhandledPanic::Ignore => {\n // Do nothing\n }\n UnhandledPanic::ShutdownRuntime => {\n // This hook is only called from within the runtime, so\n // `CURRENT` should match with `&self`, i.e. there is no\n // opportunity for a nested scheduler to be called.\n CURRENT.with(|LocalData { ctx, .. }| match ctx.get() {\n Some(cx) if Arc::ptr_eq(self, &cx.shared) => {\n cx.unhandled_panic.set(true);\n // Safety: this is always called from the thread that owns `LocalSet`\n unsafe { cx.shared.local_state.close_and_shutdown_all(); }\n }\n _ => unreachable!(\"runtime core not set in CURRENT thread-local\"),\n })\n }\n }\n }\n }\n}\n\nimpl LocalState {\n /// # Safety\n ///\n /// This method must only be called from the thread who\n /// has the same [`ThreadId`] as [`Self::owner`].\n unsafe fn task_pop_front(&self) -> Option>> {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.local_queue\n .with_mut(|ptr| unsafe { (*ptr).pop_front() })\n }\n\n /// # Safety\n ///\n /// This method must only be called from the thread who\n /// has the same [`ThreadId`] as [`Self::owner`].\n unsafe fn task_push_back(&self, task: task::Notified>) {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.local_queue\n .with_mut(|ptr| unsafe { (*ptr).push_back(task) });\n }\n\n /// # Safety\n ///\n /// This method must only be called from the thread who\n /// has the same [`ThreadId`] as [`Self::owner`].\n unsafe fn take_local_queue(&self) -> VecDeque>> {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.local_queue\n .with_mut(|ptr| std::mem::take(unsafe { &mut (*ptr) }))\n }\n\n unsafe fn task_remove(&self, task: &Task>) -> Option>> {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.owned.remove(task)\n }\n\n /// Returns true if the `LocalSet` does not have any spawned tasks\n unsafe fn owned_is_empty(&self) -> bool {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.owned.is_empty()\n }\n\n unsafe fn assert_owner(\n &self,\n task: task::Notified>,\n ) -> task::LocalNotified> {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.owned.assert_owner(task)\n }\n\n unsafe fn close_and_shutdown_all(&self) {\n // The caller ensures it is called from the same thread that owns\n // the LocalSet.\n self.assert_called_from_owner_thread();\n\n self.owned.close_and_shutdown_all();\n }\n\n #[track_caller]\n fn assert_called_from_owner_thread(&self) {\n // FreeBSD has some weirdness around thread-local destruction.\n // TODO: remove this hack when thread id is cleaned up\n #[cfg(not(any(target_os = \"openbsd\", target_os = \"freebsd\")))]\n debug_assert!(\n // if we couldn't get the thread ID because we're dropping the local\n // data, skip the assertion --- the `Drop` impl is not going to be\n // called from another thread, because `LocalSet` is `!Send`\n context::thread_id()\n .map(|id| id == self.owner)\n .unwrap_or(true),\n \"`LocalSet`'s local run queue must not be accessed by another thread!\"\n );\n }\n}\n\n// This is `Send` because it is stored in `Shared`. It is up to the caller to\n// ensure they are on the same thread that owns the `LocalSet`.\nunsafe impl Send for LocalState {}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use super::*;\n\n // Does a `LocalSet` running on a current-thread runtime...basically work?\n //\n // This duplicates a test in `tests/task_local_set.rs`, but because this is\n // a lib test, it will run under Miri, so this is necessary to catch stacked\n // borrows violations in the `LocalSet` implementation.\n #[test]\n fn local_current_thread_scheduler() {\n let f = async {\n LocalSet::new()\n .run_until(async {\n spawn_local(async {}).await.unwrap();\n })\n .await;\n };\n crate::runtime::Builder::new_current_thread()\n .build()\n .expect(\"rt\")\n .block_on(f)\n }\n\n // Tests that when a task on a `LocalSet` is woken by an io driver on the\n // same thread, the task is woken to the localset's local queue rather than\n // its remote queue.\n //\n // This test has to be defined in the `local.rs` file as a lib test, rather\n // than in `tests/`, because it makes assertions about the local set's\n // internal state.\n #[test]\n fn wakes_to_local_queue() {\n use super::*;\n use crate::sync::Notify;\n let rt = crate::runtime::Builder::new_current_thread()\n .build()\n .expect(\"rt\");\n rt.block_on(async {\n let local = LocalSet::new();\n let notify = Arc::new(Notify::new());\n let task = local.spawn_local({\n let notify = notify.clone();\n async move {\n notify.notified().await;\n }\n });\n let mut run_until = Box::pin(local.run_until(async move {\n task.await.unwrap();\n }));\n\n // poll the run until future once\n std::future::poll_fn(|cx| {\n let _ = run_until.as_mut().poll(cx);\n Poll::Ready(())\n })\n .await;\n\n notify.notify_one();\n let task = unsafe { local.context.shared.local_state.task_pop_front() };\n // TODO(eliza): it would be nice to be able to assert that this is\n // the local task.\n assert!(\n task.is_some(),\n \"task should have been notified to the LocalSet's local queue\"\n );\n })\n }\n}\n" }, { "path": "tokio/src/task/mod.rs", "content": "//! Asynchronous green-threads.\n//!\n//! ## What are Tasks?\n//!\n//! A _task_ is a light weight, non-blocking unit of execution. A task is similar\n//! to an OS thread, but rather than being managed by the OS scheduler, they are\n//! managed by the [Tokio runtime][rt]. Another name for this general pattern is\n//! [green threads]. If you are familiar with [Go's goroutines], [Kotlin's\n//! coroutines], or [Erlang's processes], you can think of Tokio's tasks as\n//! something similar.\n//!\n//! Key points about tasks include:\n//!\n//! * Tasks are **light weight**. Because tasks are scheduled by the Tokio\n//! runtime rather than the operating system, creating new tasks or switching\n//! between tasks does not require a context switch and has fairly low\n//! overhead. Creating, running, and destroying large numbers of tasks is\n//! quite cheap, especially compared to OS threads.\n//!\n//! * Tasks are scheduled **cooperatively**. Most operating systems implement\n//! _preemptive multitasking_. This is a scheduling technique where the\n//! operating system allows each thread to run for a period of time, and then\n//! _preempts_ it, temporarily pausing that thread and switching to another.\n//! Tasks, on the other hand, implement _cooperative multitasking_. In\n//! cooperative multitasking, a task is allowed to run until it _yields_,\n//! indicating to the Tokio runtime's scheduler that it cannot currently\n//! continue executing. When a task yields, the Tokio runtime switches to\n//! executing the next task.\n//!\n//! * Tasks are **non-blocking**. Typically, when an OS thread performs I/O or\n//! must synchronize with another thread, it _blocks_, allowing the OS to\n//! schedule another thread. When a task cannot continue executing, it must\n//! yield instead, allowing the Tokio runtime to schedule another task. Tasks\n//! should generally not perform system calls or other operations that could\n//! block a thread, as this would prevent other tasks running on the same\n//! thread from executing as well. Instead, this module provides APIs for\n//! running blocking operations in an asynchronous context.\n//!\n//! [rt]: crate::runtime\n//! [green threads]: https://en.wikipedia.org/wiki/Green_threads\n//! [Go's goroutines]: https://tour.golang.org/concurrency/1\n//! [Kotlin's coroutines]: https://kotlinlang.org/docs/reference/coroutines-overview.html\n//! [Erlang's processes]: http://erlang.org/doc/getting_started/conc_prog.html#processes\n//!\n//! ## Working with Tasks\n//!\n//! This module provides the following APIs for working with tasks:\n//!\n//! ### Spawning\n//!\n//! Perhaps the most important function in this module is [`task::spawn`]. This\n//! function can be thought of as an async equivalent to the standard library's\n//! [`thread::spawn`][`std::thread::spawn`]. It takes an `async` block or other\n//! [future], and creates a new task to run that work concurrently:\n//!\n//! ```\n//! use tokio::task;\n//!\n//! # async fn doc() {\n//! task::spawn(async {\n//! // perform some work here...\n//! });\n//! # }\n//! ```\n//!\n//! Like [`std::thread::spawn`], `task::spawn` returns a [`JoinHandle`] struct.\n//! A `JoinHandle` is itself a future which may be used to await the output of\n//! the spawned task. For example:\n//!\n//! ```\n//! use tokio::task;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() -> Result<(), Box> {\n//! let join = task::spawn(async {\n//! // ...\n//! \"hello world!\"\n//! });\n//!\n//! // ...\n//!\n//! // Await the result of the spawned task.\n//! let result = join.await?;\n//! assert_eq!(result, \"hello world!\");\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! Again, like `std::thread`'s [`JoinHandle` type][thread_join], if the spawned\n//! task panics, awaiting its `JoinHandle` will return a [`JoinError`]. For\n//! example:\n//!\n//! ```\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::task;\n//!\n//! # #[tokio::main] async fn main() {\n//! let join = task::spawn(async {\n//! panic!(\"something bad happened!\")\n//! });\n//!\n//! // The returned result indicates that the task failed.\n//! assert!(join.await.is_err());\n//! # }\n//! # }\n//! ```\n//!\n//! `spawn`, `JoinHandle`, and `JoinError` are present when the \"rt\"\n//! feature flag is enabled.\n//!\n//! [`task::spawn`]: crate::task::spawn()\n//! [future]: std::future::Future\n//! [`std::thread::spawn`]: std::thread::spawn\n//! [`JoinHandle`]: crate::task::JoinHandle\n//! [thread_join]: std::thread::JoinHandle\n//! [`JoinError`]: crate::task::JoinError\n//!\n//! #### Cancellation\n//!\n//! Spawned tasks may be cancelled using the [`JoinHandle::abort`] or\n//! [`AbortHandle::abort`] methods. When one of these methods are called, the\n//! task is signalled to shut down next time it yields at an `.await` point. If\n//! the task is already idle, then it will be shut down as soon as possible\n//! without running again before being shut down. Additionally, shutting down a\n//! Tokio runtime (e.g. by returning from `#[tokio::main]`) immediately cancels\n//! all tasks on it.\n//!\n//! When tasks are shut down, it will stop running at whichever `.await` it has\n//! yielded at. All local variables are destroyed by running their destructor.\n//! Once shutdown has completed, awaiting the [`JoinHandle`] will fail with a\n//! [cancelled error](crate::task::JoinError::is_cancelled).\n//!\n//! Note that aborting a task does not guarantee that it fails with a cancelled\n//! error, since it may complete normally first. For example, if the task does\n//! not yield to the runtime at any point between the call to `abort` and the\n//! end of the task, then the [`JoinHandle`] will instead report that the task\n//! exited normally.\n//!\n//! Be aware that tasks spawned using [`spawn_blocking`] cannot be aborted\n//! because they are not async. If you call `abort` on a `spawn_blocking`\n//! task, then this *will not have any effect*, and the task will continue\n//! running normally. The exception is if the task has not started running\n//! yet; in that case, calling `abort` may prevent the task from starting.\n//!\n//! Be aware that calls to [`JoinHandle::abort`] just schedule the task for\n//! cancellation, and will return before the cancellation has completed. To wait\n//! for cancellation to complete, wait for the task to finish by awaiting the\n//! [`JoinHandle`]. Similarly, the [`JoinHandle::is_finished`] method does not\n//! return `true` until the cancellation has finished.\n//!\n//! Calling [`JoinHandle::abort`] multiple times has the same effect as calling\n//! it once.\n//!\n//! Tokio also provides an [`AbortHandle`], which is like the [`JoinHandle`],\n//! except that it does not provide a mechanism to wait for the task to finish.\n//! Each task can only have one [`JoinHandle`], but it can have more than one\n//! [`AbortHandle`].\n//!\n//! [`JoinHandle::abort`]: crate::task::JoinHandle::abort\n//! [`AbortHandle::abort`]: crate::task::AbortHandle::abort\n//! [`AbortHandle`]: crate::task::AbortHandle\n//! [`JoinHandle::is_finished`]: crate::task::JoinHandle::is_finished\n//!\n//! ### Blocking and Yielding\n//!\n//! As we discussed above, code running in asynchronous tasks should not perform\n//! operations that can block. A blocking operation performed in a task running\n//! on a thread that is also running other tasks would block the entire thread,\n//! preventing other tasks from running.\n//!\n//! Instead, Tokio provides two APIs for running blocking operations in an\n//! asynchronous context: [`task::spawn_blocking`] and [`task::block_in_place`].\n//!\n//! Be aware that if you call a non-async method from async code, that non-async\n//! method is still inside the asynchronous context, so you should also avoid\n//! blocking operations there. This includes destructors of objects destroyed in\n//! async code.\n//!\n//! #### `spawn_blocking`\n//!\n//! The `task::spawn_blocking` function is similar to the `task::spawn` function\n//! discussed in the previous section, but rather than spawning a\n//! _non-blocking_ future on the Tokio runtime, it instead spawns a\n//! _blocking_ function on a dedicated thread pool for blocking tasks. For\n//! example:\n//!\n//! ```\n//! use tokio::task;\n//!\n//! # async fn docs() {\n//! task::spawn_blocking(|| {\n//! // do some compute-heavy work or call synchronous code\n//! });\n//! # }\n//! ```\n//!\n//! Just like `task::spawn`, `task::spawn_blocking` returns a `JoinHandle`\n//! which we can use to await the result of the blocking operation:\n//!\n//! ```rust\n//! # use tokio::task;\n//! # async fn docs() -> Result<(), Box>{\n//! let join = task::spawn_blocking(|| {\n//! // do some compute-heavy work or call synchronous code\n//! \"blocking completed\"\n//! });\n//!\n//! let result = join.await?;\n//! assert_eq!(result, \"blocking completed\");\n//! # Ok(())\n//! # }\n//! ```\n//!\n//! #### `block_in_place`\n//!\n//! When using the [multi-threaded runtime][rt-multi-thread], the [`task::block_in_place`]\n//! function is also available. Like `task::spawn_blocking`, this function\n//! allows running a blocking operation from an asynchronous context. Unlike\n//! `spawn_blocking`, however, `block_in_place` works by transitioning the\n//! _current_ worker thread to a blocking thread, moving other tasks running on\n//! that thread to another worker thread. This can improve performance by avoiding\n//! context switches.\n//!\n//! For example:\n//!\n//! ```\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::task;\n//!\n//! # async fn docs() {\n//! let result = task::block_in_place(|| {\n//! // do some compute-heavy work or call synchronous code\n//! \"blocking completed\"\n//! });\n//!\n//! assert_eq!(result, \"blocking completed\");\n//! # }\n//! # }\n//! ```\n//!\n//! #### `yield_now`\n//!\n//! In addition, this module provides a [`task::yield_now`] async function\n//! that is analogous to the standard library's [`thread::yield_now`]. Calling\n//! and `await`ing this function will cause the current task to yield to the\n//! Tokio runtime's scheduler, allowing other tasks to be\n//! scheduled. Eventually, the yielding task will be polled again, allowing it\n//! to execute. For example:\n//!\n//! ```rust\n//! use tokio::task;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! async {\n//! task::spawn(async {\n//! // ...\n//! println!(\"spawned task done!\")\n//! });\n//!\n//! // Yield, allowing the newly-spawned task to execute first.\n//! task::yield_now().await;\n//! println!(\"main task done!\");\n//! }\n//! # .await;\n//! # }\n//! ```\n//!\n//! [`task::spawn_blocking`]: crate::task::spawn_blocking\n//! [`task::block_in_place`]: crate::task::block_in_place\n//! [rt-multi-thread]: ../runtime/index.html#threaded-scheduler\n//! [`task::yield_now`]: crate::task::yield_now()\n//! [`thread::yield_now`]: std::thread::yield_now\n\ncfg_rt! {\n pub use crate::runtime::task::{JoinError, JoinHandle};\n\n mod blocking;\n pub use blocking::spawn_blocking;\n\n mod spawn;\n pub use spawn::spawn;\n\n cfg_rt_multi_thread! {\n pub use blocking::block_in_place;\n }\n\n mod yield_now;\n pub use yield_now::yield_now;\n\n pub mod coop;\n #[doc(hidden)]\n #[deprecated = \"Moved to tokio::task::coop::consume_budget\"]\n pub use coop::consume_budget;\n #[doc(hidden)]\n #[deprecated = \"Moved to tokio::task::coop::unconstrained\"]\n pub use coop::unconstrained;\n #[doc(hidden)]\n #[deprecated = \"Moved to tokio::task::coop::Unconstrained\"]\n pub use coop::Unconstrained;\n\n mod local;\n pub use local::{spawn_local, LocalSet, LocalEnterGuard};\n\n mod task_local;\n pub use task_local::LocalKey;\n\n #[doc(inline)]\n pub use join_set::JoinSet;\n pub use crate::runtime::task::AbortHandle;\n\n // Uses #[cfg(...)] instead of macro since the macro adds docsrs annotations.\n #[cfg(not(tokio_unstable))]\n mod join_set;\n #[cfg(tokio_unstable)]\n pub mod join_set;\n\n pub use crate::runtime::task::{Id, id, try_id};\n\n cfg_trace! {\n mod builder;\n pub use builder::Builder;\n }\n\n /// Task-related futures.\n pub mod futures {\n pub use super::task_local::TaskLocalFuture;\n }\n}\n\ncfg_not_rt! {\n pub(crate) mod coop;\n}\n" }, { "path": "tokio/src/task/spawn.rs", "content": "use crate::runtime::BOX_FUTURE_THRESHOLD;\nuse crate::task::JoinHandle;\nuse crate::util::trace::SpawnMeta;\n\nuse std::future::Future;\n\ncfg_rt! {\n /// Spawns a new asynchronous task, returning a\n /// [`JoinHandle`] for it.\n ///\n /// The provided future will start running in the background immediately\n /// when `spawn` is called, even if you don't await the returned\n /// [`JoinHandle`].\n ///\n /// Spawning a task enables the task to execute concurrently to other tasks. The\n /// spawned task may execute on the current thread, or it may be sent to a\n /// different thread to be executed. The specifics depend on the current\n /// [`Runtime`](crate::runtime::Runtime) configuration. In a\n /// [running runtime][running-runtime], the task will start immediately in the\n /// background. On a blocked runtime, the user must drive the runtime forward (for\n /// example, by calling [`Runtime::block_on`](crate::runtime::Runtime::block_on)).\n ///\n /// It is guaranteed that spawn will not synchronously poll the task being spawned.\n /// This means that calling spawn while holding a lock does not pose a risk of\n /// deadlocking with the spawned task.\n ///\n /// There is no guarantee that a spawned task will execute to completion.\n /// When a runtime is shutdown, all outstanding tasks are dropped,\n /// regardless of the lifecycle of that task.\n ///\n /// This function must be called from the context of a Tokio runtime. Tasks running on\n /// the Tokio runtime are always inside its context, but you can also enter the context\n /// using the [`Runtime::enter`](crate::runtime::Runtime::enter()) method.\n ///\n /// [running-runtime]: ../runtime/index.html#driving-the-runtime\n ///\n /// # Examples\n ///\n /// In this example, a server is started and `spawn` is used to start a new task\n /// that processes each received connection.\n ///\n /// ```no_run\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::net::{TcpListener, TcpStream};\n ///\n /// use std::io;\n ///\n /// async fn process(socket: TcpStream) {\n /// // ...\n /// # drop(socket);\n /// }\n ///\n /// #[tokio::main]\n /// async fn main() -> io::Result<()> {\n /// let listener = TcpListener::bind(\"127.0.0.1:8080\").await?;\n ///\n /// loop {\n /// let (socket, _) = listener.accept().await?;\n ///\n /// tokio::spawn(async move {\n /// // Process each socket concurrently.\n /// process(socket).await\n /// });\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// To run multiple tasks in parallel and receive their results, join\n /// handles can be stored in a vector.\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")] async fn main() {\n /// async fn my_background_op(id: i32) -> String {\n /// let s = format!(\"Starting background task {}.\", id);\n /// println!(\"{}\", s);\n /// s\n /// }\n ///\n /// let ops = vec![1, 2, 3];\n /// let mut tasks = Vec::with_capacity(ops.len());\n /// for op in ops {\n /// // This call will make them start running in the background\n /// // immediately.\n /// tasks.push(tokio::spawn(my_background_op(op)));\n /// }\n ///\n /// let mut outputs = Vec::with_capacity(tasks.len());\n /// for task in tasks {\n /// outputs.push(task.await.unwrap());\n /// }\n /// println!(\"{:?}\", outputs);\n /// # }\n /// ```\n /// This example pushes the tasks to `outputs` in the order they were\n /// started in. If you do not care about the ordering of the outputs, then\n /// you can also use a [`JoinSet`].\n ///\n /// [`JoinSet`]: struct@crate::task::JoinSet\n ///\n /// # Panics\n ///\n /// Panics if called from **outside** of the Tokio runtime.\n ///\n /// # Using `!Send` values from a task\n ///\n /// The task supplied to `spawn` must implement `Send`. However, it is\n /// possible to **use** `!Send` values from the task as long as they only\n /// exist between calls to `.await`.\n ///\n /// For example, this will work:\n ///\n /// ```\n /// use tokio::task;\n ///\n /// use std::rc::Rc;\n ///\n /// fn use_rc(rc: Rc<()>) {\n /// // Do stuff w/ rc\n /// # drop(rc);\n /// }\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// tokio::spawn(async {\n /// // Force the `Rc` to stay in a scope with no `.await`\n /// {\n /// let rc = Rc::new(());\n /// use_rc(rc.clone());\n /// }\n ///\n /// task::yield_now().await;\n /// }).await.unwrap();\n /// # }\n /// ```\n ///\n /// This will **not** work:\n ///\n /// ```compile_fail\n /// use tokio::task;\n ///\n /// use std::rc::Rc;\n ///\n /// fn use_rc(rc: Rc<()>) {\n /// // Do stuff w/ rc\n /// # drop(rc);\n /// }\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// tokio::spawn(async {\n /// let rc = Rc::new(());\n ///\n /// task::yield_now().await;\n ///\n /// use_rc(rc.clone());\n /// }).await.unwrap();\n /// }\n /// ```\n ///\n /// Holding on to a `!Send` value across calls to `.await` will result in\n /// an unfriendly compile error message similar to:\n ///\n /// ```text\n /// `[... some type ...]` cannot be sent between threads safely\n /// ```\n ///\n /// or:\n ///\n /// ```text\n /// error[E0391]: cycle detected when processing `main`\n /// ```\n #[track_caller]\n pub fn spawn(future: F) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n let fut_size = std::mem::size_of::();\n if fut_size > BOX_FUTURE_THRESHOLD {\n spawn_inner(Box::pin(future), SpawnMeta::new_unnamed(fut_size))\n } else {\n spawn_inner(future, SpawnMeta::new_unnamed(fut_size))\n }\n }\n\n #[track_caller]\n pub(super) fn spawn_inner(future: T, meta: SpawnMeta<'_>) -> JoinHandle\n where\n T: Future + Send + 'static,\n T::Output: Send + 'static,\n {\n use crate::runtime::{context, task};\n\n #[cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n feature = \"rt\",\n target_os = \"linux\",\n any(\n target_arch = \"aarch64\",\n target_arch = \"x86\",\n target_arch = \"x86_64\"\n )\n ))]\n let future = task::trace::Trace::root(future);\n let id = task::Id::next();\n let task = crate::util::trace::task(future, \"task\", meta, id.as_u64());\n\n match context::with_current(|handle| handle.spawn(task, id, meta.spawned_at)) {\n Ok(join_handle) => join_handle,\n Err(e) => panic!(\"{}\", e),\n }\n }\n}\n" }, { "path": "tokio/src/task/task_local.rs", "content": "use pin_project_lite::pin_project;\nuse std::cell::RefCell;\nuse std::error::Error;\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse std::{fmt, mem, thread};\n\n/// Declares a new task-local key of type [`tokio::task::LocalKey`].\n///\n/// # Syntax\n///\n/// The macro wraps any number of static declarations and makes them local to the current task.\n/// Publicity and attributes for each static is preserved. For example:\n///\n/// # Examples\n///\n/// ```\n/// # use tokio::task_local;\n/// task_local! {\n/// pub static ONE: u32;\n///\n/// #[allow(unused)]\n/// static TWO: f32;\n/// }\n/// # fn main() {}\n/// ```\n///\n/// See [`LocalKey` documentation][`tokio::task::LocalKey`] for more\n/// information.\n///\n/// [`tokio::task::LocalKey`]: struct@crate::task::LocalKey\n#[macro_export]\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\nmacro_rules! task_local {\n // empty (base case for the recursion)\n () => {};\n\n ($(#[$attr:meta])* $vis:vis static $name:ident: $t:ty; $($rest:tt)*) => {\n $crate::__task_local_inner!($(#[$attr])* $vis $name, $t);\n $crate::task_local!($($rest)*);\n };\n\n ($(#[$attr:meta])* $vis:vis static $name:ident: $t:ty) => {\n $crate::__task_local_inner!($(#[$attr])* $vis $name, $t);\n }\n}\n\n#[doc(hidden)]\n#[macro_export]\nmacro_rules! __task_local_inner {\n ($(#[$attr:meta])* $vis:vis $name:ident, $t:ty) => {\n $(#[$attr])*\n $vis static $name: $crate::task::LocalKey<$t> = {\n std::thread_local! {\n static __KEY: std::cell::RefCell> = const { std::cell::RefCell::new(None) };\n }\n\n $crate::task::LocalKey { inner: __KEY }\n };\n };\n}\n\n/// A key for task-local data.\n///\n/// This type is generated by the [`task_local!`] macro.\n///\n/// Unlike [`std::thread::LocalKey`], `tokio::task::LocalKey` will\n/// _not_ lazily initialize the value on first access. Instead, the\n/// value is first initialized when the future containing\n/// the task-local is first polled by a futures executor, like Tokio.\n///\n/// # Examples\n///\n/// ```\n/// # async fn dox() {\n/// tokio::task_local! {\n/// static NUMBER: u32;\n/// }\n///\n/// NUMBER.scope(1, async move {\n/// assert_eq!(NUMBER.get(), 1);\n/// }).await;\n///\n/// NUMBER.scope(2, async move {\n/// assert_eq!(NUMBER.get(), 2);\n///\n/// NUMBER.scope(3, async move {\n/// assert_eq!(NUMBER.get(), 3);\n/// }).await;\n/// }).await;\n/// # }\n/// ```\n///\n/// [`std::thread::LocalKey`]: struct@std::thread::LocalKey\n/// [`task_local!`]: ../macro.task_local.html\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub struct LocalKey {\n #[doc(hidden)]\n pub inner: thread::LocalKey>>,\n}\n\nimpl LocalKey {\n /// Sets a value `T` as the task-local value for the future `F`.\n ///\n /// On completion of `scope`, the task-local will be dropped.\n ///\n /// ### Panics\n ///\n /// If you poll the returned future inside a call to [`with`] or\n /// [`try_with`] on the same `LocalKey`, then the call to `poll` will panic.\n ///\n /// ### Examples\n ///\n /// ```\n /// # async fn dox() {\n /// tokio::task_local! {\n /// static NUMBER: u32;\n /// }\n ///\n /// NUMBER.scope(1, async move {\n /// println!(\"task local value: {}\", NUMBER.get());\n /// }).await;\n /// # }\n /// ```\n ///\n /// [`with`]: fn@Self::with\n /// [`try_with`]: fn@Self::try_with\n pub fn scope(&'static self, value: T, f: F) -> TaskLocalFuture\n where\n F: Future,\n {\n TaskLocalFuture {\n local: self,\n slot: Some(value),\n future: Some(f),\n _pinned: PhantomPinned,\n }\n }\n\n /// Sets a value `T` as the task-local value for the closure `F`.\n ///\n /// On completion of `sync_scope`, the task-local will be dropped.\n ///\n /// ### Panics\n ///\n /// This method panics if called inside a call to [`with`] or [`try_with`]\n /// on the same `LocalKey`.\n ///\n /// ### Examples\n ///\n /// ```\n /// # async fn dox() {\n /// tokio::task_local! {\n /// static NUMBER: u32;\n /// }\n ///\n /// NUMBER.sync_scope(1, || {\n /// println!(\"task local value: {}\", NUMBER.get());\n /// });\n /// # }\n /// ```\n ///\n /// [`with`]: fn@Self::with\n /// [`try_with`]: fn@Self::try_with\n #[track_caller]\n pub fn sync_scope(&'static self, value: T, f: F) -> R\n where\n F: FnOnce() -> R,\n {\n let mut value = Some(value);\n match self.scope_inner(&mut value, f) {\n Ok(res) => res,\n Err(err) => err.panic(),\n }\n }\n\n fn scope_inner(&'static self, slot: &mut Option, f: F) -> Result\n where\n F: FnOnce() -> R,\n {\n struct Guard<'a, T: 'static> {\n local: &'static LocalKey,\n slot: &'a mut Option,\n }\n\n impl<'a, T: 'static> Drop for Guard<'a, T> {\n fn drop(&mut self) {\n // This should not panic.\n //\n // We know that the RefCell was not borrowed before the call to\n // `scope_inner`, so the only way for this to panic is if the\n // closure has created but not destroyed a RefCell guard.\n // However, we never give user-code access to the guards, so\n // there's no way for user-code to forget to destroy a guard.\n //\n // The call to `with` also should not panic, since the\n // thread-local wasn't destroyed when we first called\n // `scope_inner`, and it shouldn't have gotten destroyed since\n // then.\n self.local.inner.with(|inner| {\n let mut ref_mut = inner.borrow_mut();\n mem::swap(self.slot, &mut *ref_mut);\n });\n }\n }\n\n self.inner.try_with(|inner| {\n inner\n .try_borrow_mut()\n .map(|mut ref_mut| mem::swap(slot, &mut *ref_mut))\n })??;\n\n let guard = Guard { local: self, slot };\n\n let res = f();\n\n drop(guard);\n\n Ok(res)\n }\n\n /// Accesses the current task-local and runs the provided closure.\n ///\n /// # Panics\n ///\n /// This function will panic if the task local doesn't have a value set.\n #[track_caller]\n pub fn with(&'static self, f: F) -> R\n where\n F: FnOnce(&T) -> R,\n {\n match self.try_with(f) {\n Ok(res) => res,\n Err(_) => panic!(\"cannot access a task-local storage value without setting it first\"),\n }\n }\n\n /// Accesses the current task-local and runs the provided closure.\n ///\n /// If the task-local with the associated key is not present, this\n /// method will return an `AccessError`. For a panicking variant,\n /// see `with`.\n pub fn try_with(&'static self, f: F) -> Result\n where\n F: FnOnce(&T) -> R,\n {\n // If called after the thread-local storing the task-local is destroyed,\n // then we are outside of a closure where the task-local is set.\n //\n // Therefore, it is correct to return an AccessError if `try_with`\n // returns an error.\n let try_with_res = self.inner.try_with(|v| {\n // This call to `borrow` cannot panic because no user-defined code\n // runs while a `borrow_mut` call is active.\n v.borrow().as_ref().map(f)\n });\n\n match try_with_res {\n Ok(Some(res)) => Ok(res),\n Ok(None) | Err(_) => Err(AccessError { _private: () }),\n }\n }\n}\n\nimpl LocalKey {\n /// Returns a copy of the task-local value\n /// if the task-local value implements `Clone`.\n ///\n /// # Panics\n ///\n /// This function will panic if the task local doesn't have a value set.\n #[track_caller]\n pub fn get(&'static self) -> T {\n self.with(|v| v.clone())\n }\n\n /// Returns a copy of the task-local value\n /// if the task-local value implements `Clone`.\n ///\n /// If the task-local with the associated key is not present, this\n /// method will return an `AccessError`. For a panicking variant,\n /// see `get`.\n pub fn try_get(&'static self) -> Result {\n self.try_with(|v| v.clone())\n }\n}\n\nimpl fmt::Debug for LocalKey {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.pad(\"LocalKey { .. }\")\n }\n}\n\npin_project! {\n /// A future that sets a value `T` of a task local for the future `F` during\n /// its execution.\n ///\n /// The value of the task-local must be `'static` and will be dropped on the\n /// completion of the future.\n ///\n /// Created by the function [`LocalKey::scope`](self::LocalKey::scope).\n ///\n /// ### Examples\n ///\n /// ```\n /// # async fn dox() {\n /// tokio::task_local! {\n /// static NUMBER: u32;\n /// }\n ///\n /// NUMBER.scope(1, async move {\n /// println!(\"task local value: {}\", NUMBER.get());\n /// }).await;\n /// # }\n /// ```\n pub struct TaskLocalFuture\n where\n T: 'static,\n {\n local: &'static LocalKey,\n slot: Option,\n #[pin]\n future: Option,\n #[pin]\n _pinned: PhantomPinned,\n }\n\n impl PinnedDrop for TaskLocalFuture {\n fn drop(this: Pin<&mut Self>) {\n let this = this.project();\n if mem::needs_drop::() && this.future.is_some() {\n // Drop the future while the task-local is set, if possible. Otherwise\n // the future is dropped normally when the `Option` field drops.\n let mut future = this.future;\n let _ = this.local.scope_inner(this.slot, || {\n future.set(None);\n });\n }\n }\n }\n}\n\nimpl TaskLocalFuture\nwhere\n T: 'static,\n{\n /// Returns the value stored in the task local by this `TaskLocalFuture`.\n ///\n /// The function returns:\n ///\n /// * `Some(T)` if the task local value exists.\n /// * `None` if the task local value has already been taken.\n ///\n /// Note that this function attempts to take the task local value even if\n /// the future has not yet completed. In that case, the value will no longer\n /// be available via the task local after the call to `take_value`.\n ///\n /// # Examples\n ///\n /// ```\n /// # async fn dox() {\n /// tokio::task_local! {\n /// static KEY: u32;\n /// }\n ///\n /// let fut = KEY.scope(42, async {\n /// // Do some async work\n /// });\n ///\n /// let mut pinned = Box::pin(fut);\n ///\n /// // Complete the TaskLocalFuture\n /// let _ = pinned.as_mut().await;\n ///\n /// // And here, we can take task local value\n /// let value = pinned.as_mut().take_value();\n ///\n /// assert_eq!(value, Some(42));\n /// # }\n /// ```\n pub fn take_value(self: Pin<&mut Self>) -> Option {\n let this = self.project();\n this.slot.take()\n }\n}\n\nimpl Future for TaskLocalFuture {\n type Output = F::Output;\n\n #[track_caller]\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n let mut future_opt = this.future;\n\n let res = this\n .local\n .scope_inner(this.slot, || match future_opt.as_mut().as_pin_mut() {\n Some(fut) => {\n let res = fut.poll(cx);\n if res.is_ready() {\n future_opt.set(None);\n }\n Some(res)\n }\n None => None,\n });\n\n match res {\n Ok(Some(res)) => res,\n Ok(None) => panic!(\"`TaskLocalFuture` polled after completion\"),\n Err(err) => err.panic(),\n }\n }\n}\n\nimpl fmt::Debug for TaskLocalFuture\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n /// Format the Option without Some.\n struct TransparentOption<'a, T> {\n value: &'a Option,\n }\n impl<'a, T: fmt::Debug> fmt::Debug for TransparentOption<'a, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self.value.as_ref() {\n Some(value) => value.fmt(f),\n // Hitting the None branch should not be possible.\n None => f.pad(\"\"),\n }\n }\n }\n\n f.debug_struct(\"TaskLocalFuture\")\n .field(\"value\", &TransparentOption { value: &self.slot })\n .finish()\n }\n}\n\n/// An error returned by [`LocalKey::try_with`](method@LocalKey::try_with).\n#[derive(Clone, Copy, Eq, PartialEq)]\npub struct AccessError {\n _private: (),\n}\n\nimpl fmt::Debug for AccessError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"AccessError\").finish()\n }\n}\n\nimpl fmt::Display for AccessError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt::Display::fmt(\"task-local value not set\", f)\n }\n}\n\nimpl Error for AccessError {}\n\nenum ScopeInnerErr {\n BorrowError,\n AccessError,\n}\n\nimpl ScopeInnerErr {\n #[track_caller]\n fn panic(&self) -> ! {\n match self {\n Self::BorrowError => panic!(\"cannot enter a task-local scope while the task-local storage is borrowed\"),\n Self::AccessError => panic!(\"cannot enter a task-local scope during or after destruction of the underlying thread-local\"),\n }\n }\n}\n\nimpl From for ScopeInnerErr {\n fn from(_: std::cell::BorrowMutError) -> Self {\n Self::BorrowError\n }\n}\n\nimpl From for ScopeInnerErr {\n fn from(_: std::thread::AccessError) -> Self {\n Self::AccessError\n }\n}\n" }, { "path": "tokio/src/task/yield_now.rs", "content": "use crate::runtime::context;\n\nuse std::future::poll_fn;\nuse std::task::{ready, Poll};\n\n/// Yields execution back to the Tokio runtime.\n///\n/// A task yields by awaiting on `yield_now()`, and may resume when that future\n/// completes (with no output.) The current task will be re-added as a pending\n/// task at the _back_ of the pending queue. Any other pending tasks will be\n/// scheduled. No other waking is required for the task to continue.\n///\n/// See also the usage example in the [task module](index.html#yield_now).\n///\n/// ## Non-guarantees\n///\n/// This function may not yield all the way up to the executor if there are any\n/// special combinators above it in the call stack. For example, if a\n/// [`tokio::select!`] has another branch complete during the same poll as the\n/// `yield_now()`, then the yield is not propagated all the way up to the\n/// runtime.\n///\n/// It is generally not guaranteed that the runtime behaves like you expect it\n/// to when deciding which task to schedule next after a call to `yield_now()`.\n/// In particular, the runtime may choose to poll the task that just ran\n/// `yield_now()` again immediately without polling any other tasks first. For\n/// example, the runtime will not drive the IO driver between every poll of a\n/// task, and this could result in the runtime polling the current task again\n/// immediately even if there is another task that could make progress if that\n/// other task is waiting for a notification from the IO driver.\n///\n/// In general, changes to the order in which the runtime polls tasks is not\n/// considered a breaking change, and your program should be correct no matter\n/// which order the runtime polls your tasks in.\n///\n/// [`tokio::select!`]: macro@crate::select\n#[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\npub async fn yield_now() {\n let mut yielded = false;\n poll_fn(|cx| {\n ready!(crate::trace::trace_leaf(cx));\n\n if yielded {\n return Poll::Ready(());\n }\n\n yielded = true;\n\n context::defer(cx.waker());\n\n Poll::Pending\n })\n .await\n}\n" }, { "path": "tokio/src/time/clock.rs", "content": "#![cfg_attr(not(feature = \"rt\"), allow(dead_code))]\n\n//! Source of time abstraction.\n//!\n//! By default, `std::time::Instant::now()` is used. However, when the\n//! `test-util` feature flag is enabled, the values returned for `now()` are\n//! configurable.\n\ncfg_not_test_util! {\n use crate::time::{Instant};\n\n #[derive(Debug, Clone)]\n pub(crate) struct Clock {}\n\n pub(crate) fn now() -> Instant {\n Instant::from_std(std::time::Instant::now())\n }\n\n impl Clock {\n pub(crate) fn new(_enable_pausing: bool, _start_paused: bool) -> Clock {\n Clock {}\n }\n\n pub(crate) fn now(&self) -> Instant {\n now()\n }\n }\n}\n\ncfg_test_util! {\n use crate::time::{Duration, Instant};\n use crate::loom::sync::Mutex;\n use crate::loom::sync::atomic::Ordering;\n use std::sync::atomic::AtomicBool as StdAtomicBool;\n\n cfg_rt! {\n #[track_caller]\n fn with_clock(f: impl FnOnce(Option<&Clock>) -> Result) -> R {\n use crate::runtime::Handle;\n\n let res = match Handle::try_current() {\n Ok(handle) => f(Some(handle.inner.driver().clock())),\n Err(ref e) if e.is_missing_context() => f(None),\n Err(_) => panic!(\"{}\", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),\n };\n\n match res {\n Ok(ret) => ret,\n Err(msg) => panic!(\"{}\", msg),\n }\n }\n }\n\n cfg_not_rt! {\n #[track_caller]\n fn with_clock(f: impl FnOnce(Option<&Clock>) -> Result) -> R {\n match f(None) {\n Ok(ret) => ret,\n Err(msg) => panic!(\"{}\", msg),\n }\n }\n }\n\n /// A handle to a source of time.\n #[derive(Debug)]\n pub(crate) struct Clock {\n inner: Mutex,\n }\n\n // Used to track if the clock was ever paused. This is an optimization to\n // avoid touching the mutex if `test-util` was accidentally enabled in\n // release mode.\n //\n // A static is used so we can avoid accessing the thread-local as well. The\n // `std` AtomicBool is used directly because loom does not support static\n // atomics.\n static DID_PAUSE_CLOCK: StdAtomicBool = StdAtomicBool::new(false);\n\n #[derive(Debug)]\n struct Inner {\n /// True if the ability to pause time is enabled.\n enable_pausing: bool,\n\n /// Instant to use as the clock's base instant.\n base: std::time::Instant,\n\n /// Instant at which the clock was last unfrozen.\n unfrozen: Option,\n\n /// Number of `inhibit_auto_advance` calls still in effect.\n auto_advance_inhibit_count: usize,\n }\n\n /// Pauses time.\n ///\n /// The current value of `Instant::now()` is saved and all subsequent calls\n /// to `Instant::now()` will return the saved value. The saved value can be\n /// changed by [`advance`] or by the time auto-advancing once the runtime\n /// has no work to do. This only affects the `Instant` type in Tokio, and\n /// the `Instant` in std continues to work as normal.\n ///\n /// Pausing time requires the `current_thread` Tokio runtime. This is the\n /// default runtime used by `#[tokio::test]`. The runtime can be initialized\n /// with time in a paused state using the `Builder::start_paused` method.\n ///\n /// For cases where time is immediately paused, it is better to pause\n /// the time using the `main` or `test` macro:\n /// ```\n /// #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// async fn main() {\n /// println!(\"Hello world\");\n /// }\n /// ```\n ///\n /// # Panics\n ///\n /// Panics if time is already frozen or if called from outside of a\n /// `current_thread` Tokio runtime.\n ///\n /// # Auto-advance\n ///\n /// If time is paused and the runtime has no work to do, the clock is\n /// auto-advanced to the next pending timer. This means that [`Sleep`] or\n /// other timer-backed primitives can cause the runtime to advance the\n /// current time when awaited.\n ///\n /// # Preventing auto-advance\n ///\n /// In some testing scenarios, you may want to keep the clock paused without\n /// auto-advancing, even while waiting for I/O or other asynchronous operations.\n /// This can be achieved by using [`spawn_blocking`] to wrap your I/O operations.\n ///\n /// When a blocking task is running, the clock's auto-advance is temporarily\n /// inhibited. This allows you to wait for I/O to complete while keeping the\n /// paused clock stationary:\n ///\n /// ```ignore\n /// use tokio::time::{Duration, Instant};\n /// use tokio::task;\n ///\n /// #[tokio::test(start_paused = true)]\n /// async fn test_with_io() {\n /// let start = Instant::now();\n ///\n /// // The clock will NOT auto-advance while this blocking task runs\n /// let result = task::spawn_blocking(|| {\n /// // Perform I/O operations here\n /// std::thread::sleep(std::time::Duration::from_millis(10));\n /// 42\n /// }).await.unwrap();\n ///\n /// // Time has not advanced\n /// assert_eq!(start.elapsed(), Duration::ZERO);\n /// }\n /// ```\n ///\n /// [`Sleep`]: crate::time::Sleep\n /// [`advance`]: crate::time::advance\n /// [`spawn_blocking`]: crate::task::spawn_blocking\n #[track_caller]\n pub fn pause() {\n with_clock(|maybe_clock| {\n match maybe_clock {\n Some(clock) => clock.pause(),\n None => Err(\"time cannot be frozen from outside the Tokio runtime\"),\n }\n });\n }\n\n /// Resumes time.\n ///\n /// Clears the saved `Instant::now()` value. Subsequent calls to\n /// `Instant::now()` will return the value returned by the system call.\n ///\n /// # Panics\n ///\n /// Panics if time is not frozen or if called from outside of the Tokio\n /// runtime.\n #[track_caller]\n pub fn resume() {\n with_clock(|maybe_clock| {\n let clock = match maybe_clock {\n Some(clock) => clock,\n None => return Err(\"time cannot be frozen from outside the Tokio runtime\"),\n };\n\n let mut inner = clock.inner.lock();\n\n if inner.unfrozen.is_some() {\n return Err(\"time is not frozen\");\n }\n\n inner.unfrozen = Some(std::time::Instant::now());\n Ok(())\n });\n }\n\n /// Advances time.\n ///\n /// Increments the saved `Instant::now()` value by `duration`. Subsequent\n /// calls to `Instant::now()` will return the result of the increment.\n ///\n /// This function will make the current time jump forward by the given\n /// duration in one jump. This means that all `sleep` calls with a deadline\n /// before the new time will immediately complete \"at the same time\", and\n /// the runtime is free to poll them in any order. Additionally, this\n /// method will not wait for the `sleep` calls it advanced past to complete.\n /// If you want to do that, you should instead call [`sleep`] and rely on\n /// the runtime's auto-advance feature.\n ///\n /// Note that calls to `sleep` are not guaranteed to complete the first time\n /// they are polled after a call to `advance`. For example, this can happen\n /// if the runtime has not yet touched the timer driver after the call to\n /// `advance`. However if they don't, the runtime will poll the task again\n /// shortly.\n ///\n /// # When to use `sleep` instead\n ///\n /// **Important:** `advance` is designed for testing scenarios where you want to\n /// instantly jump forward in time. However, it has limitations that make it\n /// unsuitable for certain use cases:\n ///\n /// - **Forcing timeouts:** If you want to reliably trigger a timeout, prefer\n /// using [`sleep`] with auto-advance rather than `advance`. The `advance`\n /// function jumps time forward but doesn't guarantee that all timers will be\n /// processed before your code continues.\n ///\n /// - **Simulating freezes:** If you're trying to simulate a scenario where the\n /// program freezes and then resumes, the batch behavior of `advance` may not\n /// produce the expected results. All timers that expire during the advance\n /// complete simultaneously.\n ///\n /// For most testing scenarios where you want to wait for a duration to pass\n /// and have all timers fire in order, use [`sleep`] instead:\n ///\n /// ```ignore\n /// use tokio::time::{self, Duration};\n ///\n /// #[tokio::test(start_paused = true)]\n /// async fn test_timeout_reliable() {\n /// // Use sleep with auto-advance for reliable timeout testing\n /// time::sleep(Duration::from_secs(5)).await;\n /// // All timers that were scheduled to fire within 5 seconds\n /// // have now been processed in order\n /// }\n /// ```\n ///\n /// # Panics\n ///\n /// Panics if any of the following conditions are met:\n ///\n /// - The clock is not frozen, which means that you must\n /// call [`pause`] before calling this method.\n /// - If called outside of the Tokio runtime.\n /// - If the input `duration` is too large (such as [`Duration::MAX`])\n /// to be safely added to the current time without causing an overflow.\n ///\n /// # Caveats\n ///\n /// Using a very large `duration` is not recommended,\n /// as it may cause panicking due to overflow.\n ///\n /// # Auto-advance\n ///\n /// If the time is paused and there is no work to do, the runtime advances\n /// time to the next timer. See [`pause`](pause#auto-advance) for more\n /// details.\n ///\n /// [`sleep`]: fn@crate::time::sleep\n pub async fn advance(duration: Duration) {\n with_clock(|maybe_clock| {\n let clock = match maybe_clock {\n Some(clock) => clock,\n None => return Err(\"time cannot be frozen from outside the Tokio runtime\"),\n };\n\n clock.advance(duration)\n });\n\n crate::task::yield_now().await;\n }\n\n /// Returns the current instant, factoring in frozen time.\n pub(crate) fn now() -> Instant {\n if !DID_PAUSE_CLOCK.load(Ordering::Acquire) {\n return Instant::from_std(std::time::Instant::now());\n }\n\n with_clock(|maybe_clock| {\n Ok(if let Some(clock) = maybe_clock {\n clock.now()\n } else {\n Instant::from_std(std::time::Instant::now())\n })\n })\n }\n\n impl Clock {\n /// Returns a new `Clock` instance that uses the current execution context's\n /// source of time.\n pub(crate) fn new(enable_pausing: bool, start_paused: bool) -> Clock {\n let now = std::time::Instant::now();\n\n let clock = Clock {\n inner: Mutex::new(Inner {\n enable_pausing,\n base: now,\n unfrozen: Some(now),\n auto_advance_inhibit_count: 0,\n }),\n };\n\n if start_paused {\n if let Err(msg) = clock.pause() {\n panic!(\"{}\", msg);\n }\n }\n\n clock\n }\n\n pub(crate) fn pause(&self) -> Result<(), &'static str> {\n let mut inner = self.inner.lock();\n\n if !inner.enable_pausing {\n return Err(\"`time::pause()` requires the `current_thread` Tokio runtime. \\\n This is the default Runtime used by `#[tokio::test].\");\n }\n\n // Track that we paused the clock\n DID_PAUSE_CLOCK.store(true, Ordering::Release);\n\n let elapsed = match inner.unfrozen.as_ref() {\n Some(v) => v.elapsed(),\n None => return Err(\"time is already frozen\")\n };\n inner.base += elapsed;\n inner.unfrozen = None;\n\n Ok(())\n }\n\n /// Temporarily stop auto-advancing the clock (see `tokio::time::pause`).\n pub(crate) fn inhibit_auto_advance(&self) {\n let mut inner = self.inner.lock();\n inner.auto_advance_inhibit_count += 1;\n }\n\n pub(crate) fn allow_auto_advance(&self) {\n let mut inner = self.inner.lock();\n inner.auto_advance_inhibit_count -= 1;\n }\n\n pub(crate) fn can_auto_advance(&self) -> bool {\n let inner = self.inner.lock();\n inner.unfrozen.is_none() && inner.auto_advance_inhibit_count == 0\n }\n\n pub(crate) fn advance(&self, duration: Duration) -> Result<(), &'static str> {\n let mut inner = self.inner.lock();\n\n if inner.unfrozen.is_some() {\n return Err(\"time is not frozen\");\n }\n\n inner.base += duration;\n Ok(())\n }\n\n pub(crate) fn now(&self) -> Instant {\n let inner = self.inner.lock();\n\n let mut ret = inner.base;\n\n if let Some(unfrozen) = inner.unfrozen {\n ret += unfrozen.elapsed();\n }\n\n Instant::from_std(ret)\n }\n }\n}\n" }, { "path": "tokio/src/time/error.rs", "content": "//! Time error types.\n\nuse std::error;\nuse std::fmt;\n\n/// Errors encountered by the timer implementation.\n///\n/// Currently, there are two different errors that can occur:\n///\n/// * `shutdown` occurs when a timer operation is attempted, but the timer\n/// instance has been dropped. In this case, the operation will never be able\n/// to complete and the `shutdown` error is returned. This is a permanent\n/// error, i.e., once this error is observed, timer operations will never\n/// succeed in the future.\n///\n/// * `at_capacity` occurs when a timer operation is attempted, but the timer\n/// instance is currently handling its maximum number of outstanding sleep instances.\n/// In this case, the operation is not able to be performed at the current\n/// moment, and `at_capacity` is returned. This is a transient error, i.e., at\n/// some point in the future, if the operation is attempted again, it might\n/// succeed. Callers that observe this error should attempt to [shed load]. One\n/// way to do this would be dropping the future that issued the timer operation.\n///\n/// [shed load]: https://en.wikipedia.org/wiki/Load_Shedding\n#[derive(Debug, Copy, Clone)]\npub struct Error(Kind);\n\n#[derive(Debug, Clone, Copy, Eq, PartialEq)]\n#[repr(u8)]\npub(crate) enum Kind {\n Shutdown = 1,\n AtCapacity = 2,\n Invalid = 3,\n}\n\nimpl From for Error {\n fn from(k: Kind) -> Self {\n Error(k)\n }\n}\n\n/// Errors returned by `Timeout`.\n///\n/// This error is returned when a timeout expires before the function was able\n/// to finish.\n#[derive(Debug, PartialEq, Eq)]\npub struct Elapsed(());\n\n#[derive(Debug)]\npub(crate) enum InsertError {\n Elapsed,\n}\n\n// ===== impl Error =====\n\nimpl Error {\n /// Creates an error representing a shutdown timer.\n pub fn shutdown() -> Error {\n Error(Kind::Shutdown)\n }\n\n /// Returns `true` if the error was caused by the timer being shutdown.\n pub fn is_shutdown(&self) -> bool {\n matches!(self.0, Kind::Shutdown)\n }\n\n /// Creates an error representing a timer at capacity.\n pub fn at_capacity() -> Error {\n Error(Kind::AtCapacity)\n }\n\n /// Returns `true` if the error was caused by the timer being at capacity.\n pub fn is_at_capacity(&self) -> bool {\n matches!(self.0, Kind::AtCapacity)\n }\n\n /// Creates an error representing a misconfigured timer.\n pub fn invalid() -> Error {\n Error(Kind::Invalid)\n }\n\n /// Returns `true` if the error was caused by the timer being misconfigured.\n pub fn is_invalid(&self) -> bool {\n matches!(self.0, Kind::Invalid)\n }\n}\n\nimpl error::Error for Error {}\n\nimpl fmt::Display for Error {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n let descr = match self.0 {\n Kind::Shutdown => {\n \"the timer is shutdown, must be called from the context of Tokio runtime\"\n }\n Kind::AtCapacity => \"timer is at capacity and cannot create a new entry\",\n Kind::Invalid => \"timer duration exceeds maximum duration\",\n };\n write!(fmt, \"{descr}\")\n }\n}\n\n// ===== impl Elapsed =====\n\nimpl Elapsed {\n pub(crate) fn new() -> Self {\n Elapsed(())\n }\n}\n\nimpl fmt::Display for Elapsed {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n \"deadline has elapsed\".fmt(fmt)\n }\n}\n\nimpl std::error::Error for Elapsed {}\n\nimpl From for std::io::Error {\n fn from(_err: Elapsed) -> std::io::Error {\n std::io::ErrorKind::TimedOut.into()\n }\n}\n" }, { "path": "tokio/src/time/instant.rs", "content": "#![allow(clippy::trivially_copy_pass_by_ref)]\n\nuse std::fmt;\nuse std::ops;\nuse std::time::Duration;\n\n/// A measurement of a monotonically nondecreasing clock.\n/// Opaque and useful only with `Duration`.\n///\n/// Instants are always guaranteed to be no less than any previously measured\n/// instant when created, and are often useful for tasks such as measuring\n/// benchmarks or timing how long an operation takes.\n///\n/// Note, however, that instants are not guaranteed to be **steady**. In other\n/// words, each tick of the underlying clock may not be the same length (e.g.\n/// some seconds may be longer than others). An instant may jump forwards or\n/// experience time dilation (slow down or speed up), but it will never go\n/// backwards.\n///\n/// Instants are opaque types that can only be compared to one another. There is\n/// no method to get \"the number of seconds\" from an instant. Instead, it only\n/// allows measuring the duration between two instants (or comparing two\n/// instants).\n///\n/// The size of an `Instant` struct may vary depending on the target operating\n/// system.\n///\n/// # Note\n///\n/// This type wraps the inner `std` variant and is used to align the Tokio\n/// clock for uses of `now()`. This can be useful for testing where you can\n/// take advantage of `time::pause()` and `time::advance()`.\n#[derive(Clone, Copy, Eq, PartialEq, PartialOrd, Ord, Hash)]\npub struct Instant {\n std: std::time::Instant,\n}\n\nimpl Instant {\n /// Returns an instant corresponding to \"now\".\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time::Instant;\n ///\n /// let now = Instant::now();\n /// ```\n pub fn now() -> Instant {\n variant::now()\n }\n\n /// Create a `tokio::time::Instant` from a `std::time::Instant`.\n pub fn from_std(std: std::time::Instant) -> Instant {\n Instant { std }\n }\n\n pub(crate) fn far_future() -> Instant {\n // Roughly 30 years from now.\n // API does not provide a way to obtain max `Instant`\n // or convert specific date in the future to instant.\n // 1000 years overflows on macOS, 100 years overflows on FreeBSD.\n Self::now() + Duration::from_secs(86400 * 365 * 30)\n }\n\n /// Convert the value into a `std::time::Instant`.\n pub fn into_std(self) -> std::time::Instant {\n self.std\n }\n\n /// Returns the amount of time elapsed from another instant to this one, or\n /// zero duration if that instant is later than this one.\n pub fn duration_since(&self, earlier: Instant) -> Duration {\n self.std.saturating_duration_since(earlier.std)\n }\n\n /// Returns the amount of time elapsed from another instant to this one, or\n /// None if that instant is later than this one.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time::{Duration, Instant, sleep};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let now = Instant::now();\n /// sleep(Duration::new(1, 0)).await;\n /// let new_now = Instant::now();\n /// println!(\"{:?}\", new_now.checked_duration_since(now));\n /// println!(\"{:?}\", now.checked_duration_since(new_now)); // None\n /// # }\n /// ```\n pub fn checked_duration_since(&self, earlier: Instant) -> Option {\n self.std.checked_duration_since(earlier.std)\n }\n\n /// Returns the amount of time elapsed from another instant to this one, or\n /// zero duration if that instant is later than this one.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time::{Duration, Instant, sleep};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let now = Instant::now();\n /// sleep(Duration::new(1, 0)).await;\n /// let new_now = Instant::now();\n /// println!(\"{:?}\", new_now.saturating_duration_since(now));\n /// println!(\"{:?}\", now.saturating_duration_since(new_now)); // 0ns\n /// }\n /// ```\n pub fn saturating_duration_since(&self, earlier: Instant) -> Duration {\n self.std.saturating_duration_since(earlier.std)\n }\n\n /// Returns the amount of time elapsed since this instant was created,\n /// or zero duration if this instant is in the future.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time::{Duration, Instant, sleep};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let instant = Instant::now();\n /// let three_secs = Duration::from_secs(3);\n /// sleep(three_secs).await;\n /// assert!(instant.elapsed() >= three_secs);\n /// # }\n /// ```\n pub fn elapsed(&self) -> Duration {\n Instant::now().saturating_duration_since(*self)\n }\n\n /// Returns `Some(t)` where `t` is the time `self + duration` if `t` can be\n /// represented as `Instant` (which means it's inside the bounds of the\n /// underlying data structure), `None` otherwise.\n pub fn checked_add(&self, duration: Duration) -> Option {\n self.std.checked_add(duration).map(Instant::from_std)\n }\n\n /// Returns `Some(t)` where `t` is the time `self - duration` if `t` can be\n /// represented as `Instant` (which means it's inside the bounds of the\n /// underlying data structure), `None` otherwise.\n pub fn checked_sub(&self, duration: Duration) -> Option {\n self.std.checked_sub(duration).map(Instant::from_std)\n }\n}\n\nimpl From for Instant {\n fn from(time: std::time::Instant) -> Instant {\n Instant::from_std(time)\n }\n}\n\nimpl From for std::time::Instant {\n fn from(time: Instant) -> std::time::Instant {\n time.into_std()\n }\n}\n\nimpl ops::Add for Instant {\n type Output = Instant;\n\n fn add(self, other: Duration) -> Instant {\n Instant::from_std(self.std + other)\n }\n}\n\nimpl ops::AddAssign for Instant {\n fn add_assign(&mut self, rhs: Duration) {\n *self = *self + rhs;\n }\n}\n\nimpl ops::Sub for Instant {\n type Output = Duration;\n\n fn sub(self, rhs: Instant) -> Duration {\n self.std.saturating_duration_since(rhs.std)\n }\n}\n\nimpl ops::Sub for Instant {\n type Output = Instant;\n\n fn sub(self, rhs: Duration) -> Instant {\n Instant::from_std(std::time::Instant::sub(self.std, rhs))\n }\n}\n\nimpl ops::SubAssign for Instant {\n fn sub_assign(&mut self, rhs: Duration) {\n *self = *self - rhs;\n }\n}\n\nimpl fmt::Debug for Instant {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n self.std.fmt(fmt)\n }\n}\n\n#[cfg(not(feature = \"test-util\"))]\nmod variant {\n use super::Instant;\n\n pub(super) fn now() -> Instant {\n Instant::from_std(std::time::Instant::now())\n }\n}\n\n#[cfg(feature = \"test-util\")]\nmod variant {\n use super::Instant;\n\n pub(super) fn now() -> Instant {\n crate::time::clock::now()\n }\n}\n" }, { "path": "tokio/src/time/interval.rs", "content": "use crate::time::{sleep_until, Duration, Instant, Sleep};\nuse crate::util::trace;\n\nuse std::future::{poll_fn, Future};\nuse std::panic::Location;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// Creates new [`Interval`] that yields with interval of `period`. The first\n/// tick completes immediately. The default [`MissedTickBehavior`] is\n/// [`Burst`](MissedTickBehavior::Burst), but this can be configured\n/// by calling [`set_missed_tick_behavior`](Interval::set_missed_tick_behavior).\n///\n/// An interval will tick indefinitely. At any time, the [`Interval`] value can\n/// be dropped. This cancels the interval.\n///\n/// This function is equivalent to\n/// [`interval_at(Instant::now(), period)`](interval_at).\n///\n/// # Panics\n///\n/// This function panics if `period` is zero.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::time::{self, Duration};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut interval = time::interval(Duration::from_millis(10));\n///\n/// interval.tick().await; // ticks immediately\n/// interval.tick().await; // ticks after 10ms\n/// interval.tick().await; // ticks after 10ms\n///\n/// // approximately 20ms have elapsed.\n/// # }\n/// ```\n///\n/// A simple example using `interval` to execute a task every two seconds.\n///\n/// The difference between `interval` and [`sleep`] is that an [`Interval`]\n/// measures the time since the last tick, which means that [`.tick().await`]\n/// may wait for a shorter time than the duration specified for the interval\n/// if some time has passed between calls to [`.tick().await`].\n///\n/// If the tick in the example below was replaced with [`sleep`], the task\n/// would only be executed once every three seconds, and not every two\n/// seconds.\n///\n/// ```\n/// use tokio::time;\n///\n/// async fn task_that_takes_a_second() {\n/// println!(\"hello\");\n/// time::sleep(time::Duration::from_secs(1)).await\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut interval = time::interval(time::Duration::from_secs(2));\n/// for _i in 0..5 {\n/// interval.tick().await;\n/// task_that_takes_a_second().await;\n/// }\n/// # }\n/// ```\n///\n/// [`sleep`]: crate::time::sleep()\n/// [`.tick().await`]: Interval::tick\n#[track_caller]\npub fn interval(period: Duration) -> Interval {\n assert!(period > Duration::new(0, 0), \"`period` must be non-zero.\");\n internal_interval_at(Instant::now(), period, trace::caller_location())\n}\n\n/// Creates new [`Interval`] that yields with interval of `period` with the\n/// first tick completing at `start`. The default [`MissedTickBehavior`] is\n/// [`Burst`](MissedTickBehavior::Burst), but this can be configured\n/// by calling [`set_missed_tick_behavior`](Interval::set_missed_tick_behavior).\n///\n/// An interval will tick indefinitely. At any time, the [`Interval`] value can\n/// be dropped. This cancels the interval.\n///\n/// # Panics\n///\n/// This function panics if `period` is zero.\n///\n/// # Examples\n///\n/// ```\n/// use tokio::time::{interval_at, Duration, Instant};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let start = Instant::now() + Duration::from_millis(50);\n/// let mut interval = interval_at(start, Duration::from_millis(10));\n///\n/// interval.tick().await; // ticks after 50ms\n/// interval.tick().await; // ticks after 10ms\n/// interval.tick().await; // ticks after 10ms\n///\n/// // approximately 70ms have elapsed.\n/// # }\n/// ```\n#[track_caller]\npub fn interval_at(start: Instant, period: Duration) -> Interval {\n assert!(period > Duration::new(0, 0), \"`period` must be non-zero.\");\n internal_interval_at(start, period, trace::caller_location())\n}\n\n#[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_variables))]\nfn internal_interval_at(\n start: Instant,\n period: Duration,\n location: Option<&'static Location<'static>>,\n) -> Interval {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = {\n let location = location.expect(\"should have location if tracing\");\n\n tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Interval\",\n kind = \"timer\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n )\n };\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let delay = resource_span.in_scope(|| Box::pin(sleep_until(start)));\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let delay = Box::pin(sleep_until(start));\n\n Interval {\n delay,\n period,\n missed_tick_behavior: MissedTickBehavior::default(),\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span,\n }\n}\n\n/// Defines the behavior of an [`Interval`] when it misses a tick.\n///\n/// Sometimes, an [`Interval`]'s tick is missed. For example, consider the\n/// following:\n///\n/// ```\n/// use tokio::time::{self, Duration};\n/// # async fn task_that_takes_one_to_three_millis() {}\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// // ticks every 2 milliseconds\n/// let mut interval = time::interval(Duration::from_millis(2));\n/// for _ in 0..5 {\n/// interval.tick().await;\n/// // if this takes more than 2 milliseconds, a tick will be delayed\n/// task_that_takes_one_to_three_millis().await;\n/// }\n/// # }\n/// ```\n///\n/// Generally, a tick is missed if too much time is spent without calling\n/// [`Interval::tick()`].\n///\n/// By default, when a tick is missed, [`Interval`] fires ticks as quickly as it\n/// can until it is \"caught up\" in time to where it should be.\n/// `MissedTickBehavior` can be used to specify a different behavior for\n/// [`Interval`] to exhibit. Each variant represents a different strategy.\n///\n/// Note that because the executor cannot guarantee exact precision with timers,\n/// these strategies will only apply when the delay is greater than 5\n/// milliseconds.\n#[derive(Debug, Clone, Copy, PartialEq, Eq)]\npub enum MissedTickBehavior {\n /// Ticks as fast as possible until caught up.\n ///\n /// When this strategy is used, [`Interval`] schedules ticks \"normally\" (the\n /// same as it would have if the ticks hadn't been delayed), which results\n /// in it firing ticks as fast as possible until it is caught up in time to\n /// where it should be. Unlike [`Delay`] and [`Skip`], the ticks yielded\n /// when `Burst` is used (the [`Instant`]s that [`tick`](Interval::tick)\n /// yields) aren't different than they would have been if a tick had not\n /// been missed. Like [`Skip`], and unlike [`Delay`], the ticks may be\n /// shortened.\n ///\n /// This looks something like this:\n /// ```text\n /// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n /// Actual ticks: | work -----| delay | work | work | work -| work -----|\n /// ```\n ///\n /// In code:\n ///\n /// ```\n /// use tokio::time::{interval, Duration};\n /// # async fn task_that_takes_200_millis() {}\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = interval(Duration::from_millis(50));\n ///\n /// // First tick resolves immediately after creation\n /// interval.tick().await;\n ///\n /// task_that_takes_200_millis().await;\n /// // The `Interval` has missed a tick\n ///\n /// // Since we have exceeded our timeout, this will resolve immediately\n /// interval.tick().await;\n ///\n /// // Since we are more than 100ms after the start of `interval`, this will\n /// // also resolve immediately.\n /// interval.tick().await;\n ///\n /// // Also resolves immediately, because it was supposed to resolve at\n /// // 150ms after the start of `interval`\n /// interval.tick().await;\n ///\n /// // Resolves immediately\n /// interval.tick().await;\n ///\n /// // Since we have gotten to 200ms after the start of `interval`, this\n /// // will resolve after 50ms\n /// interval.tick().await;\n /// # }\n /// ```\n ///\n /// This is the default behavior when [`Interval`] is created with\n /// [`interval`] and [`interval_at`].\n ///\n /// [`Delay`]: MissedTickBehavior::Delay\n /// [`Skip`]: MissedTickBehavior::Skip\n Burst,\n\n /// Tick at multiples of `period` from when [`tick`] was called, rather than\n /// from `start`.\n ///\n /// When this strategy is used and [`Interval`] has missed a tick, instead\n /// of scheduling ticks to fire at multiples of `period` from `start` (the\n /// time when the first tick was fired), it schedules all future ticks to\n /// happen at a regular `period` from the point when [`tick`] was called.\n /// Unlike [`Burst`] and [`Skip`], ticks are not shortened, and they aren't\n /// guaranteed to happen at a multiple of `period` from `start` any longer.\n ///\n /// This looks something like this:\n /// ```text\n /// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n /// Actual ticks: | work -----| delay | work -----| work -----| work -----|\n /// ```\n ///\n /// In code:\n ///\n /// ```\n /// use tokio::time::{interval, Duration, MissedTickBehavior};\n /// # async fn task_that_takes_more_than_50_millis() {}\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = interval(Duration::from_millis(50));\n /// interval.set_missed_tick_behavior(MissedTickBehavior::Delay);\n ///\n /// task_that_takes_more_than_50_millis().await;\n /// // The `Interval` has missed a tick\n ///\n /// // Since we have exceeded our timeout, this will resolve immediately\n /// interval.tick().await;\n ///\n /// // But this one, rather than also resolving immediately, as might happen\n /// // with the `Burst` or `Skip` behaviors, will not resolve until\n /// // 50ms after the call to `tick` up above. That is, in `tick`, when we\n /// // recognize that we missed a tick, we schedule the next tick to happen\n /// // 50ms (or whatever the `period` is) from right then, not from when\n /// // were *supposed* to tick\n /// interval.tick().await;\n /// # }\n /// ```\n ///\n /// [`Burst`]: MissedTickBehavior::Burst\n /// [`Skip`]: MissedTickBehavior::Skip\n /// [`tick`]: Interval::tick\n Delay,\n\n /// Skips missed ticks and tick on the next multiple of `period` from\n /// `start`.\n ///\n /// When this strategy is used, [`Interval`] schedules the next tick to fire\n /// at the next-closest tick that is a multiple of `period` away from\n /// `start` (the point where [`Interval`] first ticked). Like [`Burst`], all\n /// ticks remain multiples of `period` away from `start`, but unlike\n /// [`Burst`], the ticks may not be *one* multiple of `period` away from the\n /// last tick. Like [`Delay`], the ticks are no longer the same as they\n /// would have been if ticks had not been missed, but unlike [`Delay`], and\n /// like [`Burst`], the ticks may be shortened to be less than one `period`\n /// away from each other.\n ///\n /// This looks something like this:\n /// ```text\n /// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n /// Actual ticks: | work -----| delay | work ---| work -----| work -----|\n /// ```\n ///\n /// In code:\n ///\n /// ```\n /// use tokio::time::{interval, Duration, MissedTickBehavior};\n /// # async fn task_that_takes_75_millis() {}\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = interval(Duration::from_millis(50));\n /// interval.set_missed_tick_behavior(MissedTickBehavior::Skip);\n ///\n /// task_that_takes_75_millis().await;\n /// // The `Interval` has missed a tick\n ///\n /// // Since we have exceeded our timeout, this will resolve immediately\n /// interval.tick().await;\n ///\n /// // This one will resolve after 25ms, 100ms after the start of\n /// // `interval`, which is the closest multiple of `period` from the start\n /// // of `interval` after the call to `tick` up above.\n /// interval.tick().await;\n /// # }\n /// ```\n ///\n /// [`Burst`]: MissedTickBehavior::Burst\n /// [`Delay`]: MissedTickBehavior::Delay\n Skip,\n}\n\nimpl MissedTickBehavior {\n /// If a tick is missed, this method is called to determine when the next tick should happen.\n fn next_timeout(&self, timeout: Instant, now: Instant, period: Duration) -> Instant {\n match self {\n Self::Burst => timeout + period,\n Self::Delay => now + period,\n Self::Skip => {\n now + period\n - Duration::from_nanos(\n ((now - timeout).as_nanos() % period.as_nanos())\n .try_into()\n // This operation is practically guaranteed not to\n // fail, as in order for it to fail, `period` would\n // have to be longer than `now - timeout`, and both\n // would have to be longer than 584 years.\n //\n // If it did fail, there's not a good way to pass\n // the error along to the user, so we just panic.\n .expect(\n \"too much time has elapsed since the interval was supposed to tick\",\n ),\n )\n }\n }\n }\n}\n\nimpl Default for MissedTickBehavior {\n /// Returns [`MissedTickBehavior::Burst`].\n ///\n /// For most usecases, the [`Burst`] strategy is what is desired.\n /// Additionally, to preserve backwards compatibility, the [`Burst`]\n /// strategy must be the default. For these reasons,\n /// [`MissedTickBehavior::Burst`] is the default for [`MissedTickBehavior`].\n /// See [`Burst`] for more details.\n ///\n /// [`Burst`]: MissedTickBehavior::Burst\n fn default() -> Self {\n Self::Burst\n }\n}\n\n/// Interval returned by [`interval`] and [`interval_at`].\n///\n/// This type allows you to wait on a sequence of instants with a certain\n/// duration between each instant. Unlike calling [`sleep`] in a loop, this lets\n/// you count the time spent between the calls to [`sleep`] as well.\n///\n/// An `Interval` can be turned into a `Stream` with [`IntervalStream`].\n///\n/// [`IntervalStream`]: https://docs.rs/tokio-stream/latest/tokio_stream/wrappers/struct.IntervalStream.html\n/// [`sleep`]: crate::time::sleep()\n#[derive(Debug)]\npub struct Interval {\n /// Future that completes the next time the `Interval` yields a value.\n delay: Pin>,\n\n /// The duration between values yielded by `Interval`.\n period: Duration,\n\n /// The strategy `Interval` should use when a tick is missed.\n missed_tick_behavior: MissedTickBehavior,\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n resource_span: tracing::Span,\n}\n\nimpl Interval {\n /// Completes when the next instant in the interval has been reached.\n ///\n /// # Cancel safety\n ///\n /// This method is cancellation safe. If `tick` is used as the branch in a `tokio::select!` and\n /// another branch completes first, then no tick has been consumed.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time;\n ///\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = time::interval(Duration::from_millis(10));\n ///\n /// interval.tick().await;\n /// // approximately 0ms have elapsed. The first tick completes immediately.\n /// interval.tick().await;\n /// interval.tick().await;\n ///\n /// // approximately 20ms have elapsed.\n /// # }\n /// ```\n pub async fn tick(&mut self) -> Instant {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let resource_span = self.resource_span.clone();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let instant = trace::async_op(\n || poll_fn(|cx| self.poll_tick(cx)),\n resource_span,\n \"Interval::tick\",\n \"poll_tick\",\n false,\n );\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let instant = poll_fn(|cx| self.poll_tick(cx));\n\n instant.await\n }\n\n /// Polls for the next instant in the interval to be reached.\n ///\n /// This method can return the following values:\n ///\n /// * `Poll::Pending` if the next instant has not yet been reached.\n /// * `Poll::Ready(instant)` if the next instant has been reached.\n ///\n /// When this method returns `Poll::Pending`, the current task is scheduled\n /// to receive a wakeup when the instant has elapsed. Note that on multiple\n /// calls to `poll_tick`, only the [`Waker`](std::task::Waker) from the\n /// [`Context`] passed to the most recent call is scheduled to receive a\n /// wakeup.\n pub fn poll_tick(&mut self, cx: &mut Context<'_>) -> Poll {\n // Wait for the delay to be done\n ready!(Pin::new(&mut self.delay).poll(cx));\n\n // Get the time when we were scheduled to tick\n let timeout = self.delay.deadline();\n\n let now = Instant::now();\n\n // If a tick was not missed, and thus we are being called before the\n // next tick is due, just schedule the next tick normally, one `period`\n // after `timeout`\n //\n // However, if a tick took excessively long and we are now behind,\n // schedule the next tick according to how the user specified with\n // `MissedTickBehavior`\n let next = if now > timeout + Duration::from_millis(5) {\n self.missed_tick_behavior\n .next_timeout(timeout, now, self.period)\n } else {\n timeout\n .checked_add(self.period)\n .unwrap_or_else(Instant::far_future)\n };\n\n // When we arrive here, the internal delay returned `Poll::Ready`.\n // Reset the delay but do not register it. It should be registered with\n // the next call to [`poll_tick`].\n self.delay.as_mut().reset_without_reregister(next);\n\n // Return the time when we were scheduled to tick\n Poll::Ready(timeout)\n }\n\n /// Resets the interval to complete one period after the current time.\n ///\n /// This method ignores [`MissedTickBehavior`] strategy.\n ///\n /// This is equivalent to calling `reset_at(Instant::now() + period)`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time;\n ///\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = time::interval(Duration::from_millis(100));\n ///\n /// interval.tick().await;\n ///\n /// time::sleep(Duration::from_millis(50)).await;\n /// interval.reset();\n ///\n /// interval.tick().await;\n /// interval.tick().await;\n ///\n /// // approximately 250ms have elapsed.\n /// # }\n /// ```\n pub fn reset(&mut self) {\n self.delay.as_mut().reset(Instant::now() + self.period);\n }\n\n /// Resets the interval immediately.\n ///\n /// This method ignores [`MissedTickBehavior`] strategy.\n ///\n /// This is equivalent to calling `reset_at(Instant::now())`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time;\n ///\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = time::interval(Duration::from_millis(100));\n ///\n /// interval.tick().await;\n ///\n /// time::sleep(Duration::from_millis(50)).await;\n /// interval.reset_immediately();\n ///\n /// interval.tick().await;\n /// interval.tick().await;\n ///\n /// // approximately 150ms have elapsed.\n /// # }\n /// ```\n pub fn reset_immediately(&mut self) {\n self.delay.as_mut().reset(Instant::now());\n }\n\n /// Resets the interval after the specified [`std::time::Duration`].\n ///\n /// This method ignores [`MissedTickBehavior`] strategy.\n ///\n /// This is equivalent to calling `reset_at(Instant::now() + after)`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time;\n ///\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = time::interval(Duration::from_millis(100));\n /// interval.tick().await;\n ///\n /// time::sleep(Duration::from_millis(50)).await;\n ///\n /// let after = Duration::from_millis(20);\n /// interval.reset_after(after);\n ///\n /// interval.tick().await;\n /// interval.tick().await;\n ///\n /// // approximately 170ms have elapsed.\n /// # }\n /// ```\n pub fn reset_after(&mut self, after: Duration) {\n self.delay.as_mut().reset(Instant::now() + after);\n }\n\n /// Resets the interval to a [`crate::time::Instant`] deadline.\n ///\n /// Sets the next tick to expire at the given instant. If the instant is in\n /// the past, then the [`MissedTickBehavior`] strategy will be used to\n /// catch up. If the instant is in the future, then the next tick will\n /// complete at the given instant, even if that means that it will sleep for\n /// longer than the duration of this [`Interval`]. If the [`Interval`] had\n /// any missed ticks before calling this method, then those are discarded.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio::time::{self, Instant};\n ///\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut interval = time::interval(Duration::from_millis(100));\n /// interval.tick().await;\n ///\n /// time::sleep(Duration::from_millis(50)).await;\n ///\n /// let deadline = Instant::now() + Duration::from_millis(30);\n /// interval.reset_at(deadline);\n ///\n /// interval.tick().await;\n /// interval.tick().await;\n ///\n /// // approximately 180ms have elapsed.\n /// # }\n /// ```\n pub fn reset_at(&mut self, deadline: Instant) {\n self.delay.as_mut().reset(deadline);\n }\n\n /// Returns the [`MissedTickBehavior`] strategy currently being used.\n pub fn missed_tick_behavior(&self) -> MissedTickBehavior {\n self.missed_tick_behavior\n }\n\n /// Sets the [`MissedTickBehavior`] strategy that should be used.\n pub fn set_missed_tick_behavior(&mut self, behavior: MissedTickBehavior) {\n self.missed_tick_behavior = behavior;\n }\n\n /// Returns the period of the interval.\n pub fn period(&self) -> Duration {\n self.period\n }\n}\n" }, { "path": "tokio/src/time/mod.rs", "content": "//! Utilities for tracking time.\n//!\n//! This module provides a number of types for executing code after a set period\n//! of time.\n//!\n//! * [`Sleep`] is a future that does no work and completes at a specific [`Instant`]\n//! in time.\n//!\n//! * [`Interval`] is a stream yielding a value at a fixed period. It is\n//! initialized with a [`Duration`] and repeatedly yields each time the duration\n//! elapses.\n//!\n//! * [`Timeout`]: Wraps a future or stream, setting an upper bound to the amount\n//! of time it is allowed to execute. If the future or stream does not\n//! complete in time, then it is canceled and an error is returned.\n//!\n//! These types are sufficient for handling a large number of scenarios\n//! involving time.\n//!\n//! These types must be used from within the context of the [`Runtime`](crate::runtime::Runtime).\n//!\n//! # Examples\n//!\n//! Wait 100ms and print \"100 ms have elapsed\"\n//!\n//! ```\n//! use std::time::Duration;\n//! use tokio::time::sleep;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! sleep(Duration::from_millis(100)).await;\n//! println!(\"100 ms have elapsed\");\n//! # }\n//! ```\n//!\n//! Require that an operation takes no more than 1s.\n//!\n//! ```\n//! use tokio::time::{timeout, Duration};\n//!\n//! async fn long_future() {\n//! // do work here\n//! }\n//!\n//! # async fn dox() {\n//! let res = timeout(Duration::from_secs(1), long_future()).await;\n//!\n//! if res.is_err() {\n//! println!(\"operation timed out\");\n//! }\n//! # }\n//! ```\n//!\n//! A simple example using [`interval`] to execute a task every two seconds.\n//!\n//! The difference between [`interval`] and [`sleep`] is that an [`interval`]\n//! measures the time since the last tick, which means that `.tick().await` may\n//! wait for a shorter time than the duration specified for the interval\n//! if some time has passed between calls to `.tick().await`.\n//!\n//! If the tick in the example below was replaced with [`sleep`], the task\n//! would only be executed once every three seconds, and not every two\n//! seconds.\n//!\n//! ```\n//! use tokio::time;\n//!\n//! async fn task_that_takes_a_second() {\n//! println!(\"hello\");\n//! time::sleep(time::Duration::from_secs(1)).await\n//! }\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let mut interval = time::interval(time::Duration::from_secs(2));\n//! for _i in 0..5 {\n//! interval.tick().await;\n//! task_that_takes_a_second().await;\n//! }\n//! # }\n//! ```\n//!\n//! [`interval`]: crate::time::interval()\n//! [`sleep`]: sleep()\n\nmod clock;\npub(crate) use self::clock::Clock;\ncfg_test_util! {\n pub use clock::{advance, pause, resume};\n}\n\npub mod error;\n\nmod instant;\npub use self::instant::Instant;\n\nmod interval;\npub use interval::{interval, interval_at, Interval, MissedTickBehavior};\n\nmod sleep;\npub use sleep::{sleep, sleep_until, Sleep};\n\nmod timeout;\n#[doc(inline)]\npub use timeout::{timeout, timeout_at, Timeout};\n\n// Re-export for convenience\n#[doc(no_inline)]\npub use std::time::Duration;\n" }, { "path": "tokio/src/time/sleep.rs", "content": "use crate::runtime::Timer;\nuse crate::time::{error::Error, Duration, Instant};\nuse crate::util::trace;\n\nuse pin_project_lite::pin_project;\nuse std::future::Future;\nuse std::panic::Location;\nuse std::pin::Pin;\nuse std::task::{self, ready, Poll};\n\n/// Waits until `deadline` is reached.\n///\n/// No work is performed while awaiting on the sleep future to complete. `Sleep`\n/// operates at millisecond granularity and should not be used for tasks that\n/// require high-resolution timers.\n///\n/// To run something regularly on a schedule, see [`interval`].\n///\n/// # Cancellation\n///\n/// Canceling a sleep instance is done by dropping the returned future. No additional\n/// cleanup work is required.\n///\n/// # Examples\n///\n/// Wait 100ms and print \"100 ms have elapsed\".\n///\n/// ```\n/// use tokio::time::{sleep_until, Instant, Duration};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// sleep_until(Instant::now() + Duration::from_millis(100)).await;\n/// println!(\"100 ms have elapsed\");\n/// # }\n/// ```\n///\n/// See the documentation for the [`Sleep`] type for more examples.\n///\n/// # Panics\n///\n/// This function panics if there is no current timer set.\n///\n/// It can be triggered when [`Builder::enable_time`] or\n/// [`Builder::enable_all`] are not included in the builder.\n///\n/// It can also panic whenever a timer is created outside of a\n/// Tokio runtime. That is why `rt.block_on(sleep(...))` will panic,\n/// since the function is executed outside of the runtime.\n/// Whereas `rt.block_on(async {sleep(...).await})` doesn't panic.\n/// And this is because wrapping the function on an async makes it lazy,\n/// and so gets executed inside the runtime successfully without\n/// panicking.\n///\n/// [`Sleep`]: struct@crate::time::Sleep\n/// [`interval`]: crate::time::interval()\n/// [`Builder::enable_time`]: crate::runtime::Builder::enable_time\n/// [`Builder::enable_all`]: crate::runtime::Builder::enable_all\n// Alias for old name in 0.x\n#[cfg_attr(docsrs, doc(alias = \"delay_until\"))]\n#[track_caller]\npub fn sleep_until(deadline: Instant) -> Sleep {\n Sleep::new_timeout(deadline, trace::caller_location())\n}\n\n/// Waits until `duration` has elapsed.\n///\n/// Equivalent to `sleep_until(Instant::now() + duration)`. An asynchronous\n/// analog to `std::thread::sleep`.\n///\n/// No work is performed while awaiting on the sleep future to complete. `Sleep`\n/// operates at millisecond granularity and should not be used for tasks that\n/// require high-resolution timers. The implementation is platform specific,\n/// and some platforms (specifically Windows) will provide timers with a\n/// larger resolution than 1 ms.\n///\n/// To run something regularly on a schedule, see [`interval`].\n///\n/// # Cancellation\n///\n/// Canceling a sleep instance is done by dropping the returned future. No additional\n/// cleanup work is required.\n///\n/// # Examples\n///\n/// Wait 100ms and print \"100 ms have elapsed\".\n///\n/// ```\n/// use tokio::time::{sleep, Duration};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// sleep(Duration::from_millis(100)).await;\n/// println!(\"100 ms have elapsed\");\n/// # }\n/// ```\n///\n/// See the documentation for the [`Sleep`] type for more examples.\n///\n/// # Panics\n///\n/// This function panics if there is no current timer set.\n///\n/// It can be triggered when [`Builder::enable_time`] or\n/// [`Builder::enable_all`] are not included in the builder.\n///\n/// It can also panic whenever a timer is created outside of a\n/// Tokio runtime. That is why `rt.block_on(sleep(...))` will panic,\n/// since the function is executed outside of the runtime.\n/// Whereas `rt.block_on(async {sleep(...).await})` doesn't panic.\n/// And this is because wrapping the function on an async makes it lazy,\n/// and so gets executed inside the runtime successfully without\n/// panicking.\n///\n/// [`Sleep`]: struct@crate::time::Sleep\n/// [`interval`]: crate::time::interval()\n/// [`Builder::enable_time`]: crate::runtime::Builder::enable_time\n/// [`Builder::enable_all`]: crate::runtime::Builder::enable_all\n// Alias for old name in 0.x\n#[cfg_attr(docsrs, doc(alias = \"delay_for\"))]\n#[cfg_attr(docsrs, doc(alias = \"wait\"))]\n#[track_caller]\npub fn sleep(duration: Duration) -> Sleep {\n let location = trace::caller_location();\n\n match Instant::now().checked_add(duration) {\n Some(deadline) => Sleep::new_timeout(deadline, location),\n None => Sleep::new_timeout(Instant::far_future(), location),\n }\n}\n\npin_project! {\n /// Future returned by [`sleep`](sleep) and [`sleep_until`](sleep_until).\n ///\n /// This type does not implement the `Unpin` trait, which means that if you\n /// use it with [`select!`] or by calling `poll`, you have to pin it first.\n /// If you use it with `.await`, this does not apply.\n ///\n /// # Examples\n ///\n /// Wait 100ms and print \"100 ms have elapsed\".\n ///\n /// ```\n /// use tokio::time::{sleep, Duration};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// sleep(Duration::from_millis(100)).await;\n /// println!(\"100 ms have elapsed\");\n /// # }\n /// ```\n ///\n /// Use with [`select!`]. Pinning the `Sleep` with [`tokio::pin!`] is\n /// necessary when the same `Sleep` is selected on multiple times.\n /// ```no_run\n /// use tokio::time::{self, Duration, Instant};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let sleep = time::sleep(Duration::from_millis(10));\n /// tokio::pin!(sleep);\n ///\n /// loop {\n /// tokio::select! {\n /// () = &mut sleep => {\n /// println!(\"timer elapsed\");\n /// sleep.as_mut().reset(Instant::now() + Duration::from_millis(50));\n /// },\n /// }\n /// }\n /// # }\n /// ```\n /// Use in a struct with boxing. By pinning the `Sleep` with a `Box`, the\n /// `HasSleep` struct implements `Unpin`, even though `Sleep` does not.\n /// ```\n /// use std::future::Future;\n /// use std::pin::Pin;\n /// use std::task::{Context, Poll};\n /// use tokio::time::Sleep;\n ///\n /// struct HasSleep {\n /// sleep: Pin>,\n /// }\n ///\n /// impl Future for HasSleep {\n /// type Output = ();\n ///\n /// fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n /// self.sleep.as_mut().poll(cx)\n /// }\n /// }\n /// ```\n /// Use in a struct with pin projection. This method avoids the `Box`, but\n /// the `HasSleep` struct will not be `Unpin` as a consequence.\n /// ```\n /// use std::future::Future;\n /// use std::pin::Pin;\n /// use std::task::{Context, Poll};\n /// use tokio::time::Sleep;\n /// use pin_project_lite::pin_project;\n ///\n /// pin_project! {\n /// struct HasSleep {\n /// #[pin]\n /// sleep: Sleep,\n /// }\n /// }\n ///\n /// impl Future for HasSleep {\n /// type Output = ();\n ///\n /// fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n /// self.project().sleep.poll(cx)\n /// }\n /// }\n /// ```\n ///\n /// [`select!`]: ../macro.select.html\n /// [`tokio::pin!`]: ../macro.pin.html\n #[project(!Unpin)]\n // Alias for old name in 0.2\n #[cfg_attr(docsrs, doc(alias = \"Delay\"))]\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Sleep {\n inner: Inner,\n\n // The link between the `Sleep` instance and the timer that drives it.\n #[pin]\n entry: Timer,\n }\n}\n\ncfg_trace! {\n #[derive(Debug)]\n struct Inner {\n ctx: trace::AsyncOpTracingCtx,\n }\n}\n\ncfg_not_trace! {\n #[derive(Debug)]\n struct Inner {\n }\n}\n\nimpl Sleep {\n #[cfg_attr(not(all(tokio_unstable, feature = \"tracing\")), allow(unused_variables))]\n #[track_caller]\n pub(crate) fn new_timeout(\n deadline: Instant,\n location: Option<&'static Location<'static>>,\n ) -> Sleep {\n use crate::runtime::scheduler;\n let handle = scheduler::Handle::current();\n let entry = Timer::new(handle, deadline);\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let inner = {\n let handle = scheduler::Handle::current();\n let clock = handle.driver().clock();\n let handle = &handle.driver().time();\n let time_source = handle.time_source();\n let deadline_tick = time_source.deadline_to_tick(deadline);\n let duration = deadline_tick.saturating_sub(time_source.now(clock));\n\n let location = location.expect(\"should have location if tracing\");\n let resource_span = tracing::trace_span!(\n parent: None,\n \"runtime.resource\",\n concrete_type = \"Sleep\",\n kind = \"timer\",\n loc.file = location.file(),\n loc.line = location.line(),\n loc.col = location.column(),\n );\n\n let async_op_span = resource_span.in_scope(|| {\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n duration = duration,\n duration.unit = \"ms\",\n duration.op = \"override\",\n );\n\n tracing::trace_span!(\"runtime.resource.async_op\", source = \"Sleep::new_timeout\")\n });\n\n let async_op_poll_span =\n async_op_span.in_scope(|| tracing::trace_span!(\"runtime.resource.async_op.poll\"));\n\n let ctx = trace::AsyncOpTracingCtx {\n async_op_span,\n async_op_poll_span,\n resource_span,\n };\n\n Inner { ctx }\n };\n\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let inner = Inner {};\n\n Sleep { inner, entry }\n }\n\n pub(crate) fn far_future(location: Option<&'static Location<'static>>) -> Sleep {\n Self::new_timeout(Instant::far_future(), location)\n }\n\n /// Returns the instant at which the future will complete.\n pub fn deadline(&self) -> Instant {\n self.entry.deadline()\n }\n\n /// Returns `true` if `Sleep` has elapsed.\n ///\n /// A `Sleep` instance is elapsed when the requested duration has elapsed.\n pub fn is_elapsed(&self) -> bool {\n self.entry.is_elapsed()\n }\n\n /// Resets the `Sleep` instance to a new deadline.\n ///\n /// Calling this function allows changing the instant at which the `Sleep`\n /// future completes without having to create new associated state.\n ///\n /// This function can be called both before and after the future has\n /// completed.\n ///\n /// To call this method, you will usually combine the call with\n /// [`Pin::as_mut`], which lets you call the method without consuming the\n /// `Sleep` itself.\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::time::{Duration, Instant};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let sleep = tokio::time::sleep(Duration::from_millis(10));\n /// tokio::pin!(sleep);\n ///\n /// sleep.as_mut().reset(Instant::now() + Duration::from_millis(20));\n /// # }\n /// ```\n ///\n /// See also the top-level examples.\n ///\n /// [`Pin::as_mut`]: fn@std::pin::Pin::as_mut\n pub fn reset(self: Pin<&mut Self>, deadline: Instant) {\n self.reset_inner(deadline);\n }\n\n /// Resets the `Sleep` instance to a new deadline without reregistering it\n /// to be woken up.\n ///\n /// Calling this function allows changing the instant at which the `Sleep`\n /// future completes without having to create new associated state and\n /// without having it registered. This is required in e.g. the\n /// [`crate::time::Interval`] where we want to reset the internal [Sleep]\n /// without having it wake up the last task that polled it.\n pub(crate) fn reset_without_reregister(self: Pin<&mut Self>, deadline: Instant) {\n let mut me = self.project();\n match me.entry.as_ref().flavor() {\n crate::runtime::TimerFlavor::Traditional => {\n me.entry.as_mut().reset(deadline, false);\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n crate::runtime::TimerFlavor::Alternative => {\n let handle = me.entry.as_ref().scheduler_handle().clone();\n me.entry.set(Timer::new(handle, deadline));\n }\n }\n }\n\n fn reset_inner(self: Pin<&mut Self>, deadline: Instant) {\n let mut me = self.project();\n match me.entry.as_ref().flavor() {\n crate::runtime::TimerFlavor::Traditional => {\n me.entry.as_mut().reset(deadline, true);\n }\n #[cfg(all(tokio_unstable, feature = \"rt-multi-thread\"))]\n crate::runtime::TimerFlavor::Alternative => {\n let handle = me.entry.as_ref().scheduler_handle().clone();\n me.entry.set(Timer::new(handle, deadline));\n }\n }\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n {\n let _resource_enter = me.inner.ctx.resource_span.enter();\n me.inner.ctx.async_op_span =\n tracing::trace_span!(\"runtime.resource.async_op\", source = \"Sleep::reset\");\n let _async_op_enter = me.inner.ctx.async_op_span.enter();\n\n me.inner.ctx.async_op_poll_span =\n tracing::trace_span!(\"runtime.resource.async_op.poll\");\n\n let duration = {\n let clock = me.entry.as_ref().clock();\n let time_source = me.entry.as_ref().driver().time_source();\n let now = time_source.now(clock);\n let deadline_tick = time_source.deadline_to_tick(deadline);\n deadline_tick.saturating_sub(now)\n };\n\n tracing::trace!(\n target: \"runtime::resource::state_update\",\n duration = duration,\n duration.unit = \"ms\",\n duration.op = \"override\",\n );\n }\n }\n\n fn poll_elapsed(self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll> {\n let me = self.project();\n\n ready!(crate::trace::trace_leaf(cx));\n\n // Keep track of task budget\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let coop = ready!(trace_poll_op!(\n \"poll_elapsed\",\n crate::task::coop::poll_proceed(cx),\n ));\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n let coop = ready!(crate::task::coop::poll_proceed(cx));\n\n let result = me.entry.poll_elapsed(cx).map(move |r| {\n coop.made_progress();\n r\n });\n\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n return trace_poll_op!(\"poll_elapsed\", result);\n\n #[cfg(any(not(tokio_unstable), not(feature = \"tracing\")))]\n return result;\n }\n}\n\nimpl Future for Sleep {\n type Output = ();\n\n // `poll_elapsed` can return an error in two cases:\n //\n // - AtCapacity: this is a pathological case where far too many\n // sleep instances have been scheduled.\n // - Shutdown: No timer has been setup, which is a misuse error.\n //\n // Both cases are extremely rare, and pretty accurately fit into\n // \"logic errors\", so we just panic in this case. A user couldn't\n // really do much better if we passed the error onwards.\n fn poll(mut self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _res_span = self.inner.ctx.resource_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _ao_span = self.inner.ctx.async_op_span.clone().entered();\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n let _ao_poll_span = self.inner.ctx.async_op_poll_span.clone().entered();\n match ready!(self.as_mut().poll_elapsed(cx)) {\n Ok(()) => Poll::Ready(()),\n Err(e) => panic!(\"timer error: {e}\"),\n }\n }\n}\n" }, { "path": "tokio/src/time/timeout.rs", "content": "//! Allows a future to execute for a maximum amount of time.\n//!\n//! See [`Timeout`] documentation for more details.\n//!\n//! [`Timeout`]: struct@Timeout\n\nuse crate::{\n task::coop,\n time::{error::Elapsed, sleep_until, Duration, Instant, Sleep},\n util::trace,\n};\n\nuse pin_project_lite::pin_project;\nuse std::future::{Future, IntoFuture};\nuse std::pin::Pin;\nuse std::task::{self, Poll};\n\n/// Requires a `Future` to complete before the specified duration has elapsed.\n///\n/// If the future completes before the duration has elapsed, then the completed\n/// value is returned. Otherwise, an error is returned and the future is\n/// canceled.\n///\n/// Note that the timeout is checked before polling the future, so if the future\n/// does not yield during execution then it is possible for the future to complete\n/// and exceed the timeout _without_ returning an error.\n///\n/// This function returns a future whose return type is [`Result`]``, where `T` is the\n/// return type of the provided future.\n///\n/// If the provided future completes immediately, then the future returned from\n/// this function is guaranteed to complete immediately with an [`Ok`] variant\n/// no matter the provided duration.\n///\n/// [`Ok`]: std::result::Result::Ok\n/// [`Result`]: std::result::Result\n/// [`Elapsed`]: crate::time::error::Elapsed\n///\n/// # Cancellation\n///\n/// Cancelling a timeout is done by dropping the future. No additional cleanup\n/// or other work is required.\n///\n/// The original future may be obtained by calling [`Timeout::into_inner`]. This\n/// consumes the `Timeout`.\n///\n/// # Examples\n///\n/// Create a new `Timeout` set to expire in 10 milliseconds.\n///\n/// ```rust\n/// use tokio::time::timeout;\n/// use tokio::sync::oneshot;\n///\n/// use std::time::Duration;\n///\n/// # async fn dox() {\n/// let (tx, rx) = oneshot::channel();\n/// # tx.send(()).unwrap();\n///\n/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.\n/// if let Err(_) = timeout(Duration::from_millis(10), rx).await {\n/// println!(\"did not receive value within 10 ms\");\n/// }\n/// # }\n/// ```\n///\n/// # Panics\n///\n/// This function panics if there is no current timer set.\n///\n/// It can be triggered when [`Builder::enable_time`] or\n/// [`Builder::enable_all`] are not included in the builder.\n///\n/// It can also panic whenever a timer is created outside of a\n/// Tokio runtime. That is why `rt.block_on(sleep(...))` will panic,\n/// since the function is executed outside of the runtime.\n/// Whereas `rt.block_on(async {sleep(...).await})` doesn't panic.\n/// And this is because wrapping the function on an async makes it lazy,\n/// and so gets executed inside the runtime successfully without\n/// panicking.\n///\n/// [`Builder::enable_time`]: crate::runtime::Builder::enable_time\n/// [`Builder::enable_all`]: crate::runtime::Builder::enable_all\n#[track_caller]\npub fn timeout(duration: Duration, future: F) -> Timeout\nwhere\n F: IntoFuture,\n{\n let location = trace::caller_location();\n\n let deadline = Instant::now().checked_add(duration);\n let delay = match deadline {\n Some(deadline) => Sleep::new_timeout(deadline, location),\n None => Sleep::far_future(location),\n };\n Timeout::new_with_delay(future.into_future(), delay)\n}\n\n/// Requires a `Future` to complete before the specified instant in time.\n///\n/// If the future completes before the instant is reached, then the completed\n/// value is returned. Otherwise, an error is returned.\n///\n/// This function returns a future whose return type is [`Result`]``, where `T` is the\n/// return type of the provided future.\n///\n/// If the provided future completes immediately, then the future returned from\n/// this function is guaranteed to complete immediately with an [`Ok`] variant\n/// no matter the provided deadline.\n///\n/// [`Ok`]: std::result::Result::Ok\n/// [`Result`]: std::result::Result\n/// [`Elapsed`]: crate::time::error::Elapsed\n///\n/// # Cancellation\n///\n/// Cancelling a timeout is done by dropping the future. No additional cleanup\n/// or other work is required.\n///\n/// The original future may be obtained by calling [`Timeout::into_inner`]. This\n/// consumes the `Timeout`.\n///\n/// # Examples\n///\n/// Create a new `Timeout` set to expire in 10 milliseconds.\n///\n/// ```rust\n/// use tokio::time::{Instant, timeout_at};\n/// use tokio::sync::oneshot;\n///\n/// use std::time::Duration;\n///\n/// # async fn dox() {\n/// let (tx, rx) = oneshot::channel();\n/// # tx.send(()).unwrap();\n///\n/// // Wrap the future with a `Timeout` set to expire 10 milliseconds into the\n/// // future.\n/// if let Err(_) = timeout_at(Instant::now() + Duration::from_millis(10), rx).await {\n/// println!(\"did not receive value within 10 ms\");\n/// }\n/// # }\n/// ```\npub fn timeout_at(deadline: Instant, future: F) -> Timeout\nwhere\n F: IntoFuture,\n{\n let delay = sleep_until(deadline);\n\n Timeout {\n value: future.into_future(),\n delay,\n }\n}\n\npin_project! {\n /// Future returned by [`timeout`](timeout) and [`timeout_at`](timeout_at).\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n #[derive(Debug)]\n pub struct Timeout {\n #[pin]\n value: T,\n #[pin]\n delay: Sleep,\n }\n}\n\nimpl Timeout {\n pub(crate) fn new_with_delay(value: T, delay: Sleep) -> Timeout {\n Timeout { value, delay }\n }\n\n /// Gets a reference to the underlying value in this timeout.\n pub fn get_ref(&self) -> &T {\n &self.value\n }\n\n /// Gets a mutable reference to the underlying value in this timeout.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.value\n }\n\n /// Consumes this timeout, returning the underlying value.\n pub fn into_inner(self) -> T {\n self.value\n }\n}\n\nimpl Future for Timeout\nwhere\n T: Future,\n{\n type Output = Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll {\n let me = self.project();\n\n let had_budget_before = coop::has_budget_remaining();\n\n // First, try polling the future\n if let Poll::Ready(v) = me.value.poll(cx) {\n return Poll::Ready(Ok(v));\n }\n\n poll_delay(had_budget_before, me.delay, cx).map(Err)\n }\n}\n\n// The T-invariant portion of Timeout::::poll. Pulling this out reduces the\n// amount of code that gets duplicated during monomorphization.\nfn poll_delay(\n had_budget_before: bool,\n delay: Pin<&mut Sleep>,\n cx: &mut task::Context<'_>,\n) -> Poll {\n let delay_poll = || match delay.poll(cx) {\n Poll::Ready(()) => Poll::Ready(Elapsed::new()),\n Poll::Pending => Poll::Pending,\n };\n\n let has_budget_now = coop::has_budget_remaining();\n\n if let (true, false) = (had_budget_before, has_budget_now) {\n // if it is the underlying future that exhausted the budget, we poll\n // the `delay` with an unconstrained one. This prevents pathological\n // cases where the underlying future always exhausts the budget and\n // we never get a chance to evaluate whether the timeout was hit or\n // not.\n coop::with_unconstrained(delay_poll)\n } else {\n delay_poll()\n }\n}\n" }, { "path": "tokio/src/util/as_ref.rs", "content": "use super::typeid;\n\n#[derive(Debug)]\npub(crate) enum OwnedBuf {\n Vec(Vec),\n #[cfg(feature = \"io-util\")]\n Bytes(bytes::Bytes),\n}\n\nimpl AsRef<[u8]> for OwnedBuf {\n fn as_ref(&self) -> &[u8] {\n match self {\n Self::Vec(vec) => vec,\n #[cfg(feature = \"io-util\")]\n Self::Bytes(bytes) => bytes,\n }\n }\n}\n\npub(crate) fn upgrade>(buf: B) -> OwnedBuf {\n let buf = match unsafe { typeid::try_transmute::>(buf) } {\n Ok(vec) => return OwnedBuf::Vec(vec),\n Err(original_buf) => original_buf,\n };\n\n let buf = match unsafe { typeid::try_transmute::(buf) } {\n Ok(string) => return OwnedBuf::Vec(string.into_bytes()),\n Err(original_buf) => original_buf,\n };\n\n #[cfg(feature = \"io-util\")]\n let buf = match unsafe { typeid::try_transmute::(buf) } {\n Ok(bytes) => return OwnedBuf::Bytes(bytes),\n Err(original_buf) => original_buf,\n };\n\n OwnedBuf::Vec(buf.as_ref().to_owned())\n}\n" }, { "path": "tokio/src/util/atomic_cell.rs", "content": "use crate::loom::sync::atomic::AtomicPtr;\n\nuse std::ptr;\nuse std::sync::atomic::Ordering::AcqRel;\n\npub(crate) struct AtomicCell {\n data: AtomicPtr,\n}\n\nunsafe impl Send for AtomicCell {}\nunsafe impl Sync for AtomicCell {}\n\nimpl AtomicCell {\n pub(crate) fn new(data: Option>) -> AtomicCell {\n AtomicCell {\n data: AtomicPtr::new(to_raw(data)),\n }\n }\n\n pub(crate) fn swap(&self, val: Option>) -> Option> {\n let old = self.data.swap(to_raw(val), AcqRel);\n from_raw(old)\n }\n\n pub(crate) fn set(&self, val: Box) {\n let _ = self.swap(Some(val));\n }\n\n pub(crate) fn take(&self) -> Option> {\n self.swap(None)\n }\n}\n\nfn to_raw(data: Option>) -> *mut T {\n data.map_or(ptr::null_mut(), Box::into_raw)\n}\n\nfn from_raw(val: *mut T) -> Option> {\n if val.is_null() {\n None\n } else {\n Some(unsafe { Box::from_raw(val) })\n }\n}\n\nimpl Drop for AtomicCell {\n fn drop(&mut self) {\n // Free any data still held by the cell\n let _ = self.take();\n }\n}\n" }, { "path": "tokio/src/util/bit.rs", "content": "use std::fmt;\n\n#[derive(Clone, Copy, PartialEq)]\npub(crate) struct Pack {\n mask: usize,\n shift: u32,\n}\n\nimpl Pack {\n /// Value is packed in the `width` least-significant bits.\n pub(crate) const fn least_significant(width: u32) -> Pack {\n let mask = mask_for(width);\n\n Pack { mask, shift: 0 }\n }\n\n /// Value is packed in the `width` more-significant bits.\n pub(crate) const fn then(&self, width: u32) -> Pack {\n let shift = usize::BITS - self.mask.leading_zeros();\n let mask = mask_for(width) << shift;\n\n Pack { mask, shift }\n }\n\n /// Width, in bits, dedicated to storing the value.\n pub(crate) const fn width(&self) -> u32 {\n usize::BITS - (self.mask >> self.shift).leading_zeros()\n }\n\n /// Max representable value.\n pub(crate) const fn max_value(&self) -> usize {\n (1 << self.width()) - 1\n }\n\n pub(crate) fn pack(&self, value: usize, base: usize) -> usize {\n assert!(value <= self.max_value());\n (base & !self.mask) | (value << self.shift)\n }\n\n pub(crate) fn unpack(&self, src: usize) -> usize {\n unpack(src, self.mask, self.shift)\n }\n}\n\nimpl fmt::Debug for Pack {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(\n fmt,\n \"Pack {{ mask: {:b}, shift: {} }}\",\n self.mask, self.shift\n )\n }\n}\n\n/// Returns a `usize` with the right-most `n` bits set.\npub(crate) const fn mask_for(n: u32) -> usize {\n let shift = 1usize.wrapping_shl(n - 1);\n shift | (shift - 1)\n}\n\n/// Unpacks a value using a mask & shift.\npub(crate) const fn unpack(src: usize, mask: usize, shift: u32) -> usize {\n (src & mask) >> shift\n}\n" }, { "path": "tokio/src/util/blocking_check.rs", "content": "#[cfg(unix)]\nuse std::os::fd::AsFd;\n\n#[cfg(unix)]\n#[allow(unused_variables)]\n#[track_caller]\npub(crate) fn check_socket_for_blocking(s: &S) -> crate::io::Result<()> {\n #[cfg(not(tokio_allow_from_blocking_fd))]\n {\n let sock = socket2::SockRef::from(s);\n\n debug_assert!(\n sock.nonblocking()?,\n \"Registering a blocking socket with the tokio runtime is unsupported. \\\n If you wish to do anyways, please add `--cfg tokio_allow_from_blocking_fd` to your \\\n RUSTFLAGS. See github.com/tokio-rs/tokio/issues/7172 for details.\"\n );\n }\n\n Ok(())\n}\n\n#[cfg(not(unix))]\n#[allow(unused_variables)]\npub(crate) fn check_socket_for_blocking(s: &S) -> crate::io::Result<()> {\n // we cannot retrieve the nonblocking status on windows\n // and i dont know how to support wasi yet\n Ok(())\n}\n" }, { "path": "tokio/src/util/cacheline.rs", "content": "#![cfg_attr(not(feature = \"sync\"), allow(dead_code, unreachable_pub))]\nuse std::ops::{Deref, DerefMut};\n\n/// Pads and aligns a value to the length of a cache line.\n#[derive(Clone, Copy, Default, Hash, PartialEq, Eq)]\n// Starting from Intel's Sandy Bridge, spatial prefetcher is now pulling pairs of 64-byte cache\n// lines at a time, so we have to align to 128 bytes rather than 64.\n//\n// Sources:\n// - https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-optimization-manual.pdf\n// - https://github.com/facebook/folly/blob/1b5288e6eea6df074758f877c849b6e73bbb9fbb/folly/lang/Align.h#L107\n//\n// ARM's big.LITTLE architecture has asymmetric cores and \"big\" cores have 128-byte cache line size.\n//\n// Sources:\n// - https://www.mono-project.com/news/2016/09/12/arm64-icache/\n//\n// powerpc64 has 128-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_ppc64x.go#L9\n#[cfg_attr(\n any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n ),\n repr(align(128))\n)]\n// arm, mips and mips64 have 32-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_arm.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mipsle.go#L7\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_mips64x.go#L9\n#[cfg_attr(\n any(target_arch = \"arm\", target_arch = \"mips\", target_arch = \"mips64\",),\n repr(align(32))\n)]\n// s390x has 256-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_s390x.go#L7\n#[cfg_attr(target_arch = \"s390x\", repr(align(256)))]\n// x86, riscv and wasm have 64-byte cache line size.\n//\n// Sources:\n// - https://github.com/golang/go/blob/dda2991c2ea0c5914714469c4defc2562a907230/src/internal/cpu/cpu_x86.go#L9\n// - https://github.com/golang/go/blob/3dd58676054223962cd915bb0934d1f9f489d4d2/src/internal/cpu/cpu_wasm.go#L7\n// - https://github.com/torvalds/linux/blob/3516bd729358a2a9b090c1905bd2a3fa926e24c6/arch/riscv/include/asm/cache.h#L10\n//\n// All others are assumed to have 64-byte cache line size.\n#[cfg_attr(\n not(any(\n target_arch = \"x86_64\",\n target_arch = \"aarch64\",\n target_arch = \"powerpc64\",\n target_arch = \"arm\",\n target_arch = \"mips\",\n target_arch = \"mips64\",\n target_arch = \"s390x\",\n )),\n repr(align(64))\n)]\npub(crate) struct CachePadded {\n value: T,\n}\n\nimpl CachePadded {\n /// Pads and aligns a value to the length of a cache line.\n pub(crate) fn new(value: T) -> CachePadded {\n CachePadded:: { value }\n }\n}\n\nimpl Deref for CachePadded {\n type Target = T;\n\n fn deref(&self) -> &T {\n &self.value\n }\n}\n\nimpl DerefMut for CachePadded {\n fn deref_mut(&mut self) -> &mut T {\n &mut self.value\n }\n}\n" }, { "path": "tokio/src/util/error.rs", "content": "// Some combinations of features may not use these constants.\n#![cfg_attr(not(feature = \"full\"), allow(dead_code))]\n\n/// Error string explaining that the Tokio context hasn't been instantiated.\npub(crate) const CONTEXT_MISSING_ERROR: &str =\n \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\";\n\n/// Error string explaining that the Tokio context is shutting down and cannot drive timers.\npub(crate) const RUNTIME_SHUTTING_DOWN_ERROR: &str =\n \"A Tokio 1.x context was found, but it is being shutdown.\";\n\n/// Error string explaining that the Tokio context is not available because the\n/// thread-local storing it has been destroyed. This usually only happens during\n/// destructors of other thread-locals.\npub(crate) const THREAD_LOCAL_DESTROYED_ERROR: &str =\n \"The Tokio context thread-local variable has been destroyed.\";\n" }, { "path": "tokio/src/util/idle_notified_set.rs", "content": "//! This module defines an `IdleNotifiedSet`, which is a collection of elements.\n//! Each element is intended to correspond to a task, and the collection will\n//! keep track of which tasks have had their waker notified, and which have not.\n//!\n//! Each entry in the set holds some user-specified value. The value's type is\n//! specified using the `T` parameter. It will usually be a `JoinHandle` or\n//! similar.\n\nuse std::marker::PhantomPinned;\nuse std::mem::ManuallyDrop;\nuse std::ptr::NonNull;\nuse std::task::{Context, Waker};\n\nuse crate::loom::cell::UnsafeCell;\nuse crate::loom::sync::{Arc, Mutex};\nuse crate::util::linked_list::{self, Link};\nuse crate::util::{waker_ref, Wake};\n\ntype LinkedList =\n linked_list::LinkedList, as linked_list::Link>::Target>;\n\n/// This is the main handle to the collection.\npub(crate) struct IdleNotifiedSet {\n lists: Arc>,\n length: usize,\n}\n\n/// A handle to an entry that is guaranteed to be stored in the idle or notified\n/// list of its `IdleNotifiedSet`. This value borrows the `IdleNotifiedSet`\n/// mutably to prevent the entry from being moved to the `Neither` list, which\n/// only the `IdleNotifiedSet` may do.\n///\n/// The main consequence of being stored in one of the lists is that the `value`\n/// field has not yet been consumed.\n///\n/// Note: This entry can be moved from the idle to the notified list while this\n/// object exists by waking its waker.\npub(crate) struct EntryInOneOfTheLists<'a, T> {\n entry: Arc>,\n set: &'a mut IdleNotifiedSet,\n}\n\ntype Lists = Mutex>;\n\n/// The linked lists hold strong references to the `ListEntry` items, and the\n/// `ListEntry` items also hold a strong reference back to the Lists object, but\n/// the destructor of the `IdleNotifiedSet` will clear the two lists, so once\n/// that object is destroyed, no ref-cycles will remain.\nstruct ListsInner {\n notified: LinkedList,\n idle: LinkedList,\n /// Whenever an element in the `notified` list is woken, this waker will be\n /// notified and consumed, if it exists.\n waker: Option,\n}\n\n/// Which of the two lists in the shared Lists object is this entry stored in?\n///\n/// If the value is `Idle`, then an entry's waker may move it to the notified\n/// list. Otherwise, only the `IdleNotifiedSet` may move it.\n///\n/// If the value is `Neither`, then it is still possible that the entry is in\n/// some third external list (this happens in `drain`).\n#[derive(Copy, Clone, Eq, PartialEq)]\nenum List {\n Notified,\n Idle,\n Neither,\n}\n\n/// An entry in the list.\n///\n/// # Safety\n///\n/// The `my_list` field must only be accessed while holding the mutex in\n/// `parent`. It is an invariant that the value of `my_list` corresponds to\n/// which linked list in the `parent` holds this entry. Once this field takes\n/// the value `Neither`, then it may never be modified again.\n///\n/// If the value of `my_list` is `Notified` or `Idle`, then the `pointers` field\n/// must only be accessed while holding the mutex. If the value of `my_list` is\n/// `Neither`, then the `pointers` field may be accessed by the\n/// `IdleNotifiedSet` (this happens inside `drain`).\n///\n/// The `value` field is owned by the `IdleNotifiedSet` and may only be accessed\n/// by the `IdleNotifiedSet`. The operation that sets the value of `my_list` to\n/// `Neither` assumes ownership of the `value`, and it must either drop it or\n/// move it out from this entry to prevent it from getting leaked. (Since the\n/// two linked lists are emptied in the destructor of `IdleNotifiedSet`, the\n/// value should not be leaked.)\nstruct ListEntry {\n /// The linked list pointers of the list this entry is in.\n pointers: linked_list::Pointers>,\n /// Pointer to the shared `Lists` struct.\n parent: Arc>,\n /// The value stored in this entry.\n value: UnsafeCell>,\n /// Used to remember which list this entry is in.\n my_list: UnsafeCell,\n /// Required by the `linked_list::Pointers` field.\n _pin: PhantomPinned,\n}\n\ngenerate_addr_of_methods! {\n impl ListEntry {\n unsafe fn addr_of_pointers(self: NonNull) -> NonNull>> {\n &self.pointers\n }\n }\n}\n\n// With mutable access to the `IdleNotifiedSet`, you can get mutable access to\n// the values.\nunsafe impl Send for IdleNotifiedSet {}\n// With the current API we strictly speaking don't even need `T: Sync`, but we\n// require it anyway to support adding &self APIs that access the values in the\n// future.\nunsafe impl Sync for IdleNotifiedSet {}\n\n// These impls control when it is safe to create a Waker. Since the waker does\n// not allow access to the value in any way (including its destructor), it is\n// not necessary for `T` to be Send or Sync.\nunsafe impl Send for ListEntry {}\nunsafe impl Sync for ListEntry {}\n\nimpl IdleNotifiedSet {\n /// Create a new `IdleNotifiedSet`.\n pub(crate) fn new() -> Self {\n let lists = Mutex::new(ListsInner {\n notified: LinkedList::new(),\n idle: LinkedList::new(),\n waker: None,\n });\n\n IdleNotifiedSet {\n lists: Arc::new(lists),\n length: 0,\n }\n }\n\n pub(crate) fn len(&self) -> usize {\n self.length\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.length == 0\n }\n\n /// Insert the given value into the `idle` list.\n pub(crate) fn insert_idle(&mut self, value: T) -> EntryInOneOfTheLists<'_, T> {\n self.length += 1;\n\n let entry = Arc::new(ListEntry {\n parent: self.lists.clone(),\n value: UnsafeCell::new(ManuallyDrop::new(value)),\n my_list: UnsafeCell::new(List::Idle),\n pointers: linked_list::Pointers::new(),\n _pin: PhantomPinned,\n });\n\n {\n let mut lock = self.lists.lock();\n lock.idle.push_front(entry.clone());\n }\n\n // Safety: We just put the entry in the idle list, so it is in one of the lists.\n EntryInOneOfTheLists { entry, set: self }\n }\n\n /// Pop an entry from the notified list to poll it. The entry is moved to\n /// the idle list atomically.\n pub(crate) fn pop_notified(&mut self, waker: &Waker) -> Option> {\n // We don't decrement the length because this call moves the entry to\n // the idle list rather than removing it.\n if self.length == 0 {\n // Fast path.\n return None;\n }\n\n let mut lock = self.lists.lock();\n\n let should_update_waker = match lock.waker.as_mut() {\n Some(cur_waker) => !waker.will_wake(cur_waker),\n None => true,\n };\n if should_update_waker {\n lock.waker = Some(waker.clone());\n }\n\n // Pop the entry, returning None if empty.\n let entry = lock.notified.pop_back()?;\n\n lock.idle.push_front(entry.clone());\n\n // Safety: We are holding the lock.\n entry.my_list.with_mut(|ptr| unsafe {\n *ptr = List::Idle;\n });\n\n drop(lock);\n\n // Safety: We just put the entry in the idle list, so it is in one of the lists.\n Some(EntryInOneOfTheLists { entry, set: self })\n }\n\n /// Tries to pop an entry from the notified list to poll it. The entry is moved to\n /// the idle list atomically.\n pub(crate) fn try_pop_notified(&mut self) -> Option> {\n // We don't decrement the length because this call moves the entry to\n // the idle list rather than removing it.\n if self.length == 0 {\n // Fast path.\n return None;\n }\n\n let mut lock = self.lists.lock();\n\n // Pop the entry, returning None if empty.\n let entry = lock.notified.pop_back()?;\n\n lock.idle.push_front(entry.clone());\n\n // Safety: We are holding the lock.\n entry.my_list.with_mut(|ptr| unsafe {\n *ptr = List::Idle;\n });\n\n drop(lock);\n\n // Safety: We just put the entry in the idle list, so it is in one of the lists.\n Some(EntryInOneOfTheLists { entry, set: self })\n }\n\n /// Call a function on every element in this list.\n pub(crate) fn for_each(&mut self, mut func: F) {\n fn get_ptrs(list: &mut LinkedList, ptrs: &mut Vec<*mut T>) {\n let mut node = list.last();\n\n while let Some(entry) = node {\n ptrs.push(entry.value.with_mut(|ptr| {\n let ptr: *mut ManuallyDrop = ptr;\n let ptr: *mut T = ptr.cast();\n ptr\n }));\n\n let prev = entry.pointers.get_prev();\n node = prev.map(|prev| unsafe { &*prev.as_ptr() });\n }\n }\n\n // Atomically get a raw pointer to the value of every entry.\n //\n // Since this only locks the mutex once, it is not possible for a value\n // to get moved from the idle list to the notified list during the\n // operation, which would otherwise result in some value being listed\n // twice.\n let mut ptrs = Vec::with_capacity(self.len());\n {\n let mut lock = self.lists.lock();\n\n get_ptrs(&mut lock.idle, &mut ptrs);\n get_ptrs(&mut lock.notified, &mut ptrs);\n }\n debug_assert_eq!(ptrs.len(), ptrs.capacity());\n\n for ptr in ptrs {\n // Safety: When we grabbed the pointers, the entries were in one of\n // the two lists. This means that their value was valid at the time,\n // and it must still be valid because we are the IdleNotifiedSet,\n // and only we can remove an entry from the two lists. (It's\n // possible that an entry is moved from one list to the other during\n // this loop, but that is ok.)\n func(unsafe { &mut *ptr });\n }\n }\n\n /// Remove all entries in both lists, applying some function to each element.\n ///\n /// The closure is called on all elements even if it panics. Having it panic\n /// twice is a double-panic, and will abort the application.\n pub(crate) fn drain(&mut self, func: F) {\n if self.length == 0 {\n // Fast path.\n return;\n }\n self.length = 0;\n\n // The LinkedList is not cleared on panic, so we use a bomb to clear it.\n //\n // This value has the invariant that any entry in its `all_entries` list\n // has `my_list` set to `Neither` and that the value has not yet been\n // dropped.\n struct AllEntries {\n all_entries: LinkedList,\n func: F,\n }\n\n impl AllEntries {\n fn pop_next(&mut self) -> bool {\n if let Some(entry) = self.all_entries.pop_back() {\n // Safety: We just took this value from the list, so we can\n // destroy the value in the entry.\n entry\n .value\n .with_mut(|ptr| unsafe { (self.func)(ManuallyDrop::take(&mut *ptr)) });\n true\n } else {\n false\n }\n }\n }\n\n impl Drop for AllEntries {\n fn drop(&mut self) {\n while self.pop_next() {}\n }\n }\n\n let mut all_entries = AllEntries {\n all_entries: LinkedList::new(),\n func,\n };\n\n // Atomically move all entries to the new linked list in the AllEntries\n // object.\n {\n let mut lock = self.lists.lock();\n unsafe {\n // Safety: We are holding the lock and `all_entries` is a new\n // LinkedList.\n move_to_new_list(&mut lock.idle, &mut all_entries.all_entries);\n move_to_new_list(&mut lock.notified, &mut all_entries.all_entries);\n }\n }\n\n // Keep destroying entries in the list until it is empty.\n //\n // If the closure panics, then the destructor of the `AllEntries` bomb\n // ensures that we keep running the destructor on the remaining values.\n // A second panic will abort the program.\n while all_entries.pop_next() {}\n }\n}\n\n/// # Safety\n///\n/// The mutex for the entries must be held, and the target list must be such\n/// that setting `my_list` to `Neither` is ok.\nunsafe fn move_to_new_list(from: &mut LinkedList, to: &mut LinkedList) {\n while let Some(entry) = from.pop_back() {\n entry.my_list.with_mut(|ptr| {\n // Safety: pointer is accessed while holding the mutex.\n unsafe {\n *ptr = List::Neither;\n }\n });\n to.push_front(entry);\n }\n}\n\nimpl<'a, T> EntryInOneOfTheLists<'a, T> {\n /// Remove this entry from the list it is in, returning the value associated\n /// with the entry.\n ///\n /// This consumes the value, since it is no longer guaranteed to be in a\n /// list.\n pub(crate) fn remove(self) -> T {\n self.set.length -= 1;\n\n {\n let mut lock = self.set.lists.lock();\n\n // Safety: We are holding the lock so there is no race, and we will\n // remove the entry afterwards to uphold invariants.\n let old_my_list = self.entry.my_list.with_mut(|ptr| unsafe {\n let old_my_list = *ptr;\n *ptr = List::Neither;\n old_my_list\n });\n\n let list = match old_my_list {\n List::Idle => &mut lock.idle,\n List::Notified => &mut lock.notified,\n // An entry in one of the lists is in one of the lists.\n List::Neither => unreachable!(),\n };\n\n unsafe {\n // Safety: We just checked that the entry is in this particular\n // list.\n list.remove(ListEntry::as_raw(&self.entry)).unwrap();\n }\n }\n\n // By setting `my_list` to `Neither`, we have taken ownership of the\n // value. We return it to the caller.\n //\n // Safety: We have a mutable reference to the `IdleNotifiedSet` that\n // owns this entry, so we can use its permission to access the value.\n self.entry\n .value\n .with_mut(|ptr| unsafe { ManuallyDrop::take(&mut *ptr) })\n }\n\n /// Access the value in this entry together with a context for its waker.\n pub(crate) fn with_value_and_context(&mut self, func: F) -> U\n where\n F: FnOnce(&mut T, &mut Context<'_>) -> U,\n T: 'static,\n {\n let waker = waker_ref(&self.entry);\n\n let mut context = Context::from_waker(&waker);\n\n // Safety: We have a mutable reference to the `IdleNotifiedSet` that\n // owns this entry, so we can use its permission to access the value.\n self.entry\n .value\n .with_mut(|ptr| unsafe { func(&mut *ptr, &mut context) })\n }\n}\n\nimpl Drop for IdleNotifiedSet {\n fn drop(&mut self) {\n // Clear both lists.\n self.drain(drop);\n\n #[cfg(debug_assertions)]\n if !std::thread::panicking() {\n let lock = self.lists.lock();\n assert!(lock.idle.is_empty());\n assert!(lock.notified.is_empty());\n }\n }\n}\n\nimpl Wake for ListEntry {\n fn wake_by_ref(me: &Arc) {\n let mut lock = me.parent.lock();\n\n // Safety: We are holding the lock and we will update the lists to\n // maintain invariants.\n let old_my_list = me.my_list.with_mut(|ptr| unsafe {\n let old_my_list = *ptr;\n if old_my_list == List::Idle {\n *ptr = List::Notified;\n }\n old_my_list\n });\n\n if old_my_list == List::Idle {\n // We move ourself to the notified list.\n let me = unsafe {\n // Safety: We just checked that we are in this particular list.\n lock.idle.remove(ListEntry::as_raw(me)).unwrap()\n };\n lock.notified.push_front(me);\n\n if let Some(waker) = lock.waker.take() {\n drop(lock);\n waker.wake();\n }\n }\n }\n\n fn wake(me: Arc) {\n Self::wake_by_ref(&me);\n }\n}\n\n/// # Safety\n///\n/// `ListEntry` is forced to be !Unpin.\nunsafe impl linked_list::Link for ListEntry {\n type Handle = Arc>;\n type Target = ListEntry;\n\n fn as_raw(handle: &Self::Handle) -> NonNull> {\n let ptr: *const ListEntry = Arc::as_ptr(handle);\n // Safety: We can't get a null pointer from `Arc::as_ptr`.\n unsafe { NonNull::new_unchecked(ptr as *mut ListEntry) }\n }\n\n unsafe fn from_raw(ptr: NonNull>) -> Arc> {\n unsafe { Arc::from_raw(ptr.as_ptr()) }\n }\n\n unsafe fn pointers(\n target: NonNull>,\n ) -> NonNull>> {\n unsafe { ListEntry::addr_of_pointers(target) }\n }\n}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use crate::runtime::Builder;\n use crate::task::JoinSet;\n\n // A test that runs under miri.\n //\n // https://github.com/tokio-rs/tokio/pull/5693\n #[test]\n fn join_set_test() {\n let rt = Builder::new_current_thread().build().unwrap();\n\n let mut set = JoinSet::new();\n set.spawn_on(futures::future::ready(()), rt.handle());\n\n rt.block_on(set.join_next()).unwrap().unwrap();\n }\n}\n" }, { "path": "tokio/src/util/linked_list.rs", "content": "#![cfg_attr(not(feature = \"full\"), allow(dead_code))]\n// It doesn't make sense to enforce `unsafe_op_in_unsafe_fn` for this module because\n//\n// * The intrusive linked list naturally relies on unsafe operations.\n// * Excessive `unsafe {}` blocks hurt readability significantly.\n// TODO: replace with `#[expect(unsafe_op_in_unsafe_fn)]` after bumpping\n// the MSRV to 1.81.0.\n#![allow(unsafe_op_in_unsafe_fn)]\n\n//! An intrusive double linked list of data.\n//!\n//! The data structure supports tracking pinned nodes. Most of the data\n//! structure's APIs are `unsafe` as they require the caller to ensure the\n//! specified node is actually contained by the list.\n\nuse core::cell::UnsafeCell;\nuse core::fmt;\nuse core::marker::{PhantomData, PhantomPinned};\nuse core::mem::ManuallyDrop;\nuse core::ptr::{self, NonNull};\n\n/// An intrusive linked list.\n///\n/// Currently, the list is not emptied on drop. It is the caller's\n/// responsibility to ensure the list is empty before dropping it.\npub(crate) struct LinkedList {\n /// Linked list head\n head: Option>,\n\n /// Linked list tail\n tail: Option>,\n\n /// Node type marker.\n _marker: PhantomData<*const L>,\n}\n\nunsafe impl Send for LinkedList where L::Target: Send {}\nunsafe impl Sync for LinkedList where L::Target: Sync {}\n\n/// Defines how a type is tracked within a linked list.\n///\n/// In order to support storing a single type within multiple lists, accessing\n/// the list pointers is decoupled from the entry type.\n///\n/// # Safety\n///\n/// Implementations must guarantee that `Target` types are pinned in memory. In\n/// other words, when a node is inserted, the value will not be moved as long as\n/// it is stored in the list.\npub(crate) unsafe trait Link {\n /// Handle to the list entry.\n ///\n /// This is usually a pointer-ish type.\n type Handle;\n\n /// Node type.\n type Target;\n\n /// Convert the handle to a raw pointer without consuming the handle.\n #[allow(clippy::wrong_self_convention)]\n fn as_raw(handle: &Self::Handle) -> NonNull;\n\n /// Convert the raw pointer to a handle\n unsafe fn from_raw(ptr: NonNull) -> Self::Handle;\n\n /// Return the pointers for a node\n ///\n /// # Safety\n ///\n /// The resulting pointer should have the same tag in the stacked-borrows\n /// stack as the argument. In particular, the method may not create an\n /// intermediate reference in the process of creating the resulting raw\n /// pointer.\n ///\n /// The `target` pointer must be valid.\n unsafe fn pointers(target: NonNull) -> NonNull>;\n}\n\n/// Previous / next pointers.\npub(crate) struct Pointers {\n inner: UnsafeCell>,\n}\n/// We do not want the compiler to put the `noalias` attribute on mutable\n/// references to this type, so the type has been made `!Unpin` with a\n/// `PhantomPinned` field.\n///\n/// Additionally, we never access the `prev` or `next` fields directly, as any\n/// such access would implicitly involve the creation of a reference to the\n/// field, which we want to avoid since the fields are not `!Unpin`, and would\n/// hence be given the `noalias` attribute if we were to do such an access. As\n/// an alternative to accessing the fields directly, the `Pointers` type\n/// provides getters and setters for the two fields, and those are implemented\n/// using `ptr`-specific methods which avoids the creation of intermediate\n/// references.\n///\n/// See this link for more information:\n/// \nstruct PointersInner {\n /// The previous node in the list. null if there is no previous node.\n prev: Option>,\n\n /// The next node in the list. null if there is no previous node.\n next: Option>,\n\n /// This type is !Unpin due to the heuristic from:\n /// \n _pin: PhantomPinned,\n}\n\nunsafe impl Send for Pointers {}\nunsafe impl Sync for Pointers {}\n\n// ===== impl LinkedList =====\n\nimpl LinkedList {\n /// Creates an empty linked list.\n pub(crate) const fn new() -> LinkedList {\n LinkedList {\n head: None,\n tail: None,\n _marker: PhantomData,\n }\n }\n}\n\nimpl LinkedList {\n /// Adds an element first in the list.\n pub(crate) fn push_front(&mut self, val: L::Handle) {\n // The value should not be dropped, it is being inserted into the list\n let val = ManuallyDrop::new(val);\n let ptr = L::as_raw(&val);\n assert_ne!(self.head, Some(ptr));\n unsafe {\n L::pointers(ptr).as_mut().set_next(self.head);\n L::pointers(ptr).as_mut().set_prev(None);\n\n if let Some(head) = self.head {\n L::pointers(head).as_mut().set_prev(Some(ptr));\n }\n\n self.head = Some(ptr);\n\n if self.tail.is_none() {\n self.tail = Some(ptr);\n }\n }\n }\n\n /// Removes the first element from a list and returns it, or None if it is\n /// empty.\n pub(crate) fn pop_front(&mut self) -> Option {\n unsafe {\n let head = self.head?;\n self.head = L::pointers(head).as_ref().get_next();\n\n if let Some(new_head) = L::pointers(head).as_ref().get_next() {\n L::pointers(new_head).as_mut().set_prev(None);\n } else {\n self.tail = None;\n }\n\n L::pointers(head).as_mut().set_prev(None);\n L::pointers(head).as_mut().set_next(None);\n\n Some(L::from_raw(head))\n }\n }\n\n /// Removes the last element from a list and returns it, or None if it is\n /// empty.\n pub(crate) fn pop_back(&mut self) -> Option {\n unsafe {\n let last = self.tail?;\n self.tail = L::pointers(last).as_ref().get_prev();\n\n if let Some(prev) = L::pointers(last).as_ref().get_prev() {\n L::pointers(prev).as_mut().set_next(None);\n } else {\n self.head = None;\n }\n\n L::pointers(last).as_mut().set_prev(None);\n L::pointers(last).as_mut().set_next(None);\n\n Some(L::from_raw(last))\n }\n }\n\n /// Returns whether the linked list does not contain any node\n pub(crate) fn is_empty(&self) -> bool {\n if self.head.is_some() {\n return false;\n }\n\n assert!(self.tail.is_none());\n true\n }\n\n /// Removes the specified node from the list\n ///\n /// # Safety\n ///\n /// The caller **must** ensure that exactly one of the following is true:\n /// - `node` is currently contained by `self`,\n /// - `node` is not contained by any list,\n /// - `node` is currently contained by some other `GuardedLinkedList` **and**\n /// the caller has an exclusive access to that list. This condition is\n /// used by the linked list in `sync::Notify`.\n pub(crate) unsafe fn remove(&mut self, node: NonNull) -> Option {\n if let Some(prev) = L::pointers(node).as_ref().get_prev() {\n debug_assert_eq!(L::pointers(prev).as_ref().get_next(), Some(node));\n L::pointers(prev)\n .as_mut()\n .set_next(L::pointers(node).as_ref().get_next());\n } else {\n if self.head != Some(node) {\n return None;\n }\n\n self.head = L::pointers(node).as_ref().get_next();\n }\n\n if let Some(next) = L::pointers(node).as_ref().get_next() {\n debug_assert_eq!(L::pointers(next).as_ref().get_prev(), Some(node));\n L::pointers(next)\n .as_mut()\n .set_prev(L::pointers(node).as_ref().get_prev());\n } else {\n // This might be the last item in the list\n if self.tail != Some(node) {\n return None;\n }\n\n self.tail = L::pointers(node).as_ref().get_prev();\n }\n\n L::pointers(node).as_mut().set_next(None);\n L::pointers(node).as_mut().set_prev(None);\n\n Some(L::from_raw(node))\n }\n}\n\nimpl fmt::Debug for LinkedList {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"LinkedList\")\n .field(\"head\", &self.head)\n .field(\"tail\", &self.tail)\n .finish()\n }\n}\n\n#[cfg(any(\n feature = \"fs\",\n feature = \"rt\",\n all(unix, feature = \"process\"),\n feature = \"signal\",\n feature = \"sync\",\n))]\nimpl LinkedList {\n pub(crate) fn last(&self) -> Option<&L::Target> {\n let tail = self.tail.as_ref()?;\n unsafe { Some(&*tail.as_ptr()) }\n }\n}\n\nimpl Default for LinkedList {\n fn default() -> Self {\n Self::new()\n }\n}\n\n// ===== impl DrainFilter =====\n\ncfg_io_driver_impl! {\n pub(crate) struct DrainFilter<'a, T: Link, F> {\n list: &'a mut LinkedList,\n filter: F,\n curr: Option>,\n }\n\n impl LinkedList {\n pub(crate) fn drain_filter(&mut self, filter: F) -> DrainFilter<'_, T, F>\n where\n F: FnMut(&T::Target) -> bool,\n {\n let curr = self.head;\n DrainFilter {\n curr,\n filter,\n list: self,\n }\n }\n }\n\n impl<'a, T, F> Iterator for DrainFilter<'a, T, F>\n where\n T: Link,\n F: FnMut(&T::Target) -> bool,\n {\n type Item = T::Handle;\n\n fn next(&mut self) -> Option {\n while let Some(curr) = self.curr {\n // safety: the pointer references data contained by the list\n self.curr = unsafe { T::pointers(curr).as_ref() }.get_next();\n\n // safety: the value is still owned by the linked list.\n if (self.filter)(unsafe { &mut *curr.as_ptr() }) {\n return unsafe { self.list.remove(curr) };\n }\n }\n\n None\n }\n }\n}\n\ncfg_taskdump! {\n impl LinkedList {\n pub(crate) fn for_each(&mut self, mut f: F)\n where\n F: FnMut(&T::Handle),\n {\n let mut next = self.head;\n\n while let Some(curr) = next {\n unsafe {\n let handle = ManuallyDrop::new(T::from_raw(curr));\n f(&handle);\n next = T::pointers(curr).as_ref().get_next();\n }\n }\n }\n }\n}\n\n// ===== impl GuardedLinkedList =====\n\nfeature! {\n #![any(\n feature = \"process\",\n feature = \"sync\",\n feature = \"rt\",\n feature = \"signal\",\n )]\n\n /// An intrusive linked list, but instead of keeping pointers to the head\n /// and tail nodes, it uses a special guard node linked with those nodes.\n /// It means that the list is circular and every pointer of a node from\n /// the list is not `None`, including pointers from the guard node.\n ///\n /// If a list is empty, then both pointers of the guard node are pointing\n /// at the guard node itself.\n pub(crate) struct GuardedLinkedList {\n /// Pointer to the guard node.\n guard: NonNull,\n\n /// Node type marker.\n _marker: PhantomData<*const L>,\n }\n\n impl LinkedList {\n /// Turns a linked list into the guarded version by linking the guard node\n /// with the head and tail nodes. Like with other nodes, you should guarantee\n /// that the guard node is pinned in memory.\n pub(crate) fn into_guarded(self, guard_handle: L::Handle) -> GuardedLinkedList {\n // `guard_handle` is a NonNull pointer, we don't have to care about dropping it.\n let guard = L::as_raw(&guard_handle);\n\n unsafe {\n if let Some(head) = self.head {\n debug_assert!(L::pointers(head).as_ref().get_prev().is_none());\n L::pointers(head).as_mut().set_prev(Some(guard));\n L::pointers(guard).as_mut().set_next(Some(head));\n\n // The list is not empty, so the tail cannot be `None`.\n let tail = self.tail.unwrap();\n debug_assert!(L::pointers(tail).as_ref().get_next().is_none());\n L::pointers(tail).as_mut().set_next(Some(guard));\n L::pointers(guard).as_mut().set_prev(Some(tail));\n } else {\n // The list is empty.\n L::pointers(guard).as_mut().set_prev(Some(guard));\n L::pointers(guard).as_mut().set_next(Some(guard));\n }\n }\n\n GuardedLinkedList { guard, _marker: PhantomData }\n }\n }\n\n impl GuardedLinkedList {\n fn tail(&self) -> Option> {\n let tail_ptr = unsafe {\n L::pointers(self.guard).as_ref().get_prev().unwrap()\n };\n\n // Compare the tail pointer with the address of the guard node itself.\n // If the guard points at itself, then there are no other nodes and\n // the list is considered empty.\n if tail_ptr != self.guard {\n Some(tail_ptr)\n } else {\n None\n }\n }\n\n /// Removes the last element from a list and returns it, or None if it is\n /// empty.\n pub(crate) fn pop_back(&mut self) -> Option {\n unsafe {\n let last = self.tail()?;\n let before_last = L::pointers(last).as_ref().get_prev().unwrap();\n\n L::pointers(self.guard).as_mut().set_prev(Some(before_last));\n L::pointers(before_last).as_mut().set_next(Some(self.guard));\n\n L::pointers(last).as_mut().set_prev(None);\n L::pointers(last).as_mut().set_next(None);\n\n Some(L::from_raw(last))\n }\n }\n }\n}\n\n// ===== impl Pointers =====\n\nimpl Pointers {\n /// Create a new set of empty pointers\n pub(crate) fn new() -> Pointers {\n Pointers {\n inner: UnsafeCell::new(PointersInner {\n prev: None,\n next: None,\n _pin: PhantomPinned,\n }),\n }\n }\n\n pub(crate) fn get_prev(&self) -> Option> {\n // SAFETY: Field is accessed immutably through a reference.\n unsafe { ptr::addr_of!((*self.inner.get()).prev).read() }\n }\n pub(crate) fn get_next(&self) -> Option> {\n // SAFETY: Field is accessed immutably through a reference.\n unsafe { ptr::addr_of!((*self.inner.get()).next).read() }\n }\n\n fn set_prev(&mut self, value: Option>) {\n // SAFETY: Field is accessed mutably through a mutable reference.\n unsafe {\n ptr::addr_of_mut!((*self.inner.get()).prev).write(value);\n }\n }\n fn set_next(&mut self, value: Option>) {\n // SAFETY: Field is accessed mutably through a mutable reference.\n unsafe {\n ptr::addr_of_mut!((*self.inner.get()).next).write(value);\n }\n }\n}\n\nimpl fmt::Debug for Pointers {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n let prev = self.get_prev();\n let next = self.get_next();\n f.debug_struct(\"Pointers\")\n .field(\"prev\", &prev)\n .field(\"next\", &next)\n .finish()\n }\n}\n\n#[cfg(any(test, fuzzing))]\n#[cfg(not(loom))]\npub(crate) mod tests {\n use super::*;\n\n use std::pin::Pin;\n\n #[derive(Debug)]\n #[repr(C)]\n struct Entry {\n pointers: Pointers,\n val: i32,\n }\n\n unsafe impl<'a> Link for &'a Entry {\n type Handle = Pin<&'a Entry>;\n type Target = Entry;\n\n fn as_raw(handle: &Pin<&'_ Entry>) -> NonNull {\n NonNull::from(handle.get_ref())\n }\n\n unsafe fn from_raw(ptr: NonNull) -> Pin<&'a Entry> {\n Pin::new_unchecked(&*ptr.as_ptr())\n }\n\n unsafe fn pointers(target: NonNull) -> NonNull> {\n target.cast()\n }\n }\n\n fn entry(val: i32) -> Pin> {\n Box::pin(Entry {\n pointers: Pointers::new(),\n val,\n })\n }\n\n fn ptr(r: &Pin>) -> NonNull {\n r.as_ref().get_ref().into()\n }\n\n fn collect_list(list: &mut LinkedList<&'_ Entry, <&'_ Entry as Link>::Target>) -> Vec {\n let mut ret = vec![];\n\n while let Some(entry) = list.pop_back() {\n ret.push(entry.val);\n }\n\n ret\n }\n\n fn push_all<'a>(\n list: &mut LinkedList<&'a Entry, <&'_ Entry as Link>::Target>,\n entries: &[Pin<&'a Entry>],\n ) {\n for entry in entries.iter() {\n list.push_front(*entry);\n }\n }\n\n #[cfg(test)]\n macro_rules! assert_clean {\n ($e:ident) => {{\n assert!($e.pointers.get_next().is_none());\n assert!($e.pointers.get_prev().is_none());\n }};\n }\n\n #[cfg(test)]\n macro_rules! assert_ptr_eq {\n ($a:expr, $b:expr) => {{\n // Deal with mapping a Pin<&mut T> -> Option>\n assert_eq!(Some($a.as_ref().get_ref().into()), $b)\n }};\n }\n\n #[test]\n fn const_new() {\n const _: LinkedList<&Entry, <&Entry as Link>::Target> = LinkedList::new();\n }\n\n #[test]\n fn push_and_drain() {\n let a = entry(5);\n let b = entry(7);\n let c = entry(31);\n\n let mut list = LinkedList::new();\n assert!(list.is_empty());\n\n list.push_front(a.as_ref());\n assert!(!list.is_empty());\n list.push_front(b.as_ref());\n list.push_front(c.as_ref());\n\n let items: Vec = collect_list(&mut list);\n assert_eq!([5, 7, 31].to_vec(), items);\n\n assert!(list.is_empty());\n }\n\n #[test]\n fn push_pop_push_pop() {\n let a = entry(5);\n let b = entry(7);\n\n let mut list = LinkedList::<&Entry, <&Entry as Link>::Target>::new();\n\n list.push_front(a.as_ref());\n\n let entry = list.pop_back().unwrap();\n assert_eq!(5, entry.val);\n assert!(list.is_empty());\n\n list.push_front(b.as_ref());\n\n let entry = list.pop_back().unwrap();\n assert_eq!(7, entry.val);\n\n assert!(list.is_empty());\n assert!(list.pop_back().is_none());\n }\n\n #[test]\n fn remove_by_address() {\n let a = entry(5);\n let b = entry(7);\n let c = entry(31);\n\n unsafe {\n // Remove first\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[c.as_ref(), b.as_ref(), a.as_ref()]);\n assert!(list.remove(ptr(&a)).is_some());\n assert_clean!(a);\n // `a` should be no longer there and can't be removed twice\n assert!(list.remove(ptr(&a)).is_none());\n assert!(!list.is_empty());\n\n assert!(list.remove(ptr(&b)).is_some());\n assert_clean!(b);\n // `b` should be no longer there and can't be removed twice\n assert!(list.remove(ptr(&b)).is_none());\n assert!(!list.is_empty());\n\n assert!(list.remove(ptr(&c)).is_some());\n assert_clean!(c);\n // `b` should be no longer there and can't be removed twice\n assert!(list.remove(ptr(&c)).is_none());\n assert!(list.is_empty());\n }\n\n unsafe {\n // Remove middle\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[c.as_ref(), b.as_ref(), a.as_ref()]);\n\n assert!(list.remove(ptr(&a)).is_some());\n assert_clean!(a);\n\n assert_ptr_eq!(b, list.head);\n assert_ptr_eq!(c, b.pointers.get_next());\n assert_ptr_eq!(b, c.pointers.get_prev());\n\n let items = collect_list(&mut list);\n assert_eq!([31, 7].to_vec(), items);\n }\n\n unsafe {\n // Remove middle\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[c.as_ref(), b.as_ref(), a.as_ref()]);\n\n assert!(list.remove(ptr(&b)).is_some());\n assert_clean!(b);\n\n assert_ptr_eq!(c, a.pointers.get_next());\n assert_ptr_eq!(a, c.pointers.get_prev());\n\n let items = collect_list(&mut list);\n assert_eq!([31, 5].to_vec(), items);\n }\n\n unsafe {\n // Remove last\n // Remove middle\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[c.as_ref(), b.as_ref(), a.as_ref()]);\n\n assert!(list.remove(ptr(&c)).is_some());\n assert_clean!(c);\n\n assert!(b.pointers.get_next().is_none());\n assert_ptr_eq!(b, list.tail);\n\n let items = collect_list(&mut list);\n assert_eq!([7, 5].to_vec(), items);\n }\n\n unsafe {\n // Remove first of two\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[b.as_ref(), a.as_ref()]);\n\n assert!(list.remove(ptr(&a)).is_some());\n\n assert_clean!(a);\n\n // a should be no longer there and can't be removed twice\n assert!(list.remove(ptr(&a)).is_none());\n\n assert_ptr_eq!(b, list.head);\n assert_ptr_eq!(b, list.tail);\n\n assert!(b.pointers.get_next().is_none());\n assert!(b.pointers.get_prev().is_none());\n\n let items = collect_list(&mut list);\n assert_eq!([7].to_vec(), items);\n }\n\n unsafe {\n // Remove last of two\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[b.as_ref(), a.as_ref()]);\n\n assert!(list.remove(ptr(&b)).is_some());\n\n assert_clean!(b);\n\n assert_ptr_eq!(a, list.head);\n assert_ptr_eq!(a, list.tail);\n\n assert!(a.pointers.get_next().is_none());\n assert!(a.pointers.get_prev().is_none());\n\n let items = collect_list(&mut list);\n assert_eq!([5].to_vec(), items);\n }\n\n unsafe {\n // Remove last item\n let mut list = LinkedList::new();\n\n push_all(&mut list, &[a.as_ref()]);\n\n assert!(list.remove(ptr(&a)).is_some());\n assert_clean!(a);\n\n assert!(list.head.is_none());\n assert!(list.tail.is_none());\n let items = collect_list(&mut list);\n assert!(items.is_empty());\n }\n\n unsafe {\n // Remove missing\n let mut list = LinkedList::<&Entry, <&Entry as Link>::Target>::new();\n\n list.push_front(b.as_ref());\n list.push_front(a.as_ref());\n\n assert!(list.remove(ptr(&c)).is_none());\n }\n }\n\n /// This is a fuzz test. You run it by entering `cargo fuzz run fuzz_linked_list` in CLI in `/tokio/` module.\n #[cfg(fuzzing)]\n pub fn fuzz_linked_list(ops: &[u8]) {\n enum Op {\n Push,\n Pop,\n Remove(usize),\n }\n use std::collections::VecDeque;\n\n let ops = ops\n .iter()\n .map(|i| match i % 3u8 {\n 0 => Op::Push,\n 1 => Op::Pop,\n 2 => Op::Remove((i / 3u8) as usize),\n _ => unreachable!(),\n })\n .collect::>();\n\n let mut ll = LinkedList::<&Entry, <&Entry as Link>::Target>::new();\n let mut reference = VecDeque::new();\n\n let entries: Vec<_> = (0..ops.len()).map(|i| entry(i as i32)).collect();\n\n for (i, op) in ops.iter().enumerate() {\n match op {\n Op::Push => {\n reference.push_front(i as i32);\n assert_eq!(entries[i].val, i as i32);\n\n ll.push_front(entries[i].as_ref());\n }\n Op::Pop => {\n if reference.is_empty() {\n assert!(ll.is_empty());\n continue;\n }\n\n let v = reference.pop_back();\n assert_eq!(v, ll.pop_back().map(|v| v.val));\n }\n Op::Remove(n) => {\n if reference.is_empty() {\n assert!(ll.is_empty());\n continue;\n }\n\n let idx = n % reference.len();\n let expect = reference.remove(idx).unwrap();\n\n unsafe {\n let entry = ll.remove(ptr(&entries[expect as usize])).unwrap();\n assert_eq!(expect, entry.val);\n }\n }\n }\n }\n }\n}\n" }, { "path": "tokio/src/util/markers.rs", "content": "/// Marker for types that are `Sync` but not `Send`\n#[allow(dead_code)]\npub(crate) struct SyncNotSend(#[allow(dead_code)] *mut ());\n\nunsafe impl Sync for SyncNotSend {}\n\ncfg_rt! {\n pub(crate) struct NotSendOrSync(#[allow(dead_code)] *mut ());\n}\n" }, { "path": "tokio/src/util/memchr.rs", "content": "//! Search for a byte in a byte array using libc.\n//!\n//! When nothing pulls in libc, then just use a trivial implementation. Note\n//! that we only depend on libc on unix.\n\n#[cfg(not(all(unix, feature = \"libc\")))]\nfn memchr_inner(needle: u8, haystack: &[u8]) -> Option {\n haystack.iter().position(|val| needle == *val)\n}\n\n#[cfg(all(unix, feature = \"libc\"))]\nfn memchr_inner(needle: u8, haystack: &[u8]) -> Option {\n let start = haystack.as_ptr();\n\n // SAFETY: `start` is valid for `haystack.len()` bytes.\n let ptr = (unsafe { libc::memchr(start.cast(), needle as _, haystack.len()) })\n .cast::()\n .cast_const();\n\n if ptr.is_null() {\n None\n } else {\n // SAFETY: `ptr` will always be in bounds, since libc guarantees that the ptr will either\n // be to an element inside the array or the ptr will be null\n // since the ptr is in bounds the offset must also always be non null\n // and there can't be more than isize::MAX elements inside an array\n // as rust guarantees that the maximum number of bytes a allocation\n // may occupy is isize::MAX\n unsafe {\n // TODO(MSRV 1.87): When bumping MSRV, switch to `ptr.byte_offset_from_unsigned(start)`.\n Some(usize::try_from(ptr.offset_from(start)).unwrap_unchecked())\n }\n }\n}\n\npub(crate) fn memchr(needle: u8, haystack: &[u8]) -> Option {\n let index = memchr_inner(needle, haystack)?;\n\n // SAFETY: `memchr_inner` returns Some(index) and in that case index must point to an element in haystack\n // or `memchr_inner` None which is guarded by the `?` operator above\n // therefore the index must **always** point to an element in the array\n // and so this indexing operation is safe\n // TODO(MSRV 1.81): When bumping MSRV, switch to `std::hint::assert_unchecked(haystack.get(..=index).is_some());`\n unsafe {\n if haystack.get(..=index).is_none() {\n std::hint::unreachable_unchecked()\n }\n }\n\n Some(index)\n}\n\n#[cfg(test)]\nmod tests {\n use super::memchr;\n\n #[test]\n fn memchr_test() {\n let haystack = b\"123abc456\\0\\xffabc\\n\";\n\n assert_eq!(memchr(b'1', haystack), Some(0));\n assert_eq!(memchr(b'2', haystack), Some(1));\n assert_eq!(memchr(b'3', haystack), Some(2));\n assert_eq!(memchr(b'4', haystack), Some(6));\n assert_eq!(memchr(b'5', haystack), Some(7));\n assert_eq!(memchr(b'6', haystack), Some(8));\n assert_eq!(memchr(b'7', haystack), None);\n assert_eq!(memchr(b'a', haystack), Some(3));\n assert_eq!(memchr(b'b', haystack), Some(4));\n assert_eq!(memchr(b'c', haystack), Some(5));\n assert_eq!(memchr(b'd', haystack), None);\n assert_eq!(memchr(b'A', haystack), None);\n assert_eq!(memchr(0, haystack), Some(9));\n assert_eq!(memchr(0xff, haystack), Some(10));\n assert_eq!(memchr(0xfe, haystack), None);\n assert_eq!(memchr(1, haystack), None);\n assert_eq!(memchr(b'\\n', haystack), Some(14));\n assert_eq!(memchr(b'\\r', haystack), None);\n }\n\n #[test]\n fn memchr_all() {\n let mut arr = Vec::new();\n for b in 0..=255 {\n arr.push(b);\n }\n for b in 0..=255 {\n assert_eq!(memchr(b, &arr), Some(b as usize));\n }\n arr.reverse();\n for b in 0..=255 {\n assert_eq!(memchr(b, &arr), Some(255 - b as usize));\n }\n }\n\n #[test]\n fn memchr_empty() {\n for b in 0..=255 {\n assert_eq!(memchr(b, b\"\"), None);\n }\n }\n}\n" }, { "path": "tokio/src/util/metric_atomics.rs", "content": "use std::sync::atomic::{AtomicUsize, Ordering};\n\ncfg_64bit_metrics! {\n use std::sync::atomic::AtomicU64;\n}\n\n/// `AtomicU64` that is a no-op on platforms without 64-bit atomics\n///\n/// When used on platforms without 64-bit atomics, writes to this are no-ops.\n/// The `load` method is only defined when 64-bit atomics are available.\n#[derive(Debug, Default)]\npub(crate) struct MetricAtomicU64 {\n #[cfg(target_has_atomic = \"64\")]\n value: AtomicU64,\n}\n\n// some of these are currently only used behind cfg_unstable\n#[allow(dead_code)]\nimpl MetricAtomicU64 {\n // Load is only defined when supported\n cfg_64bit_metrics! {\n pub(crate) fn load(&self, ordering: Ordering) -> u64 {\n self.value.load(ordering)\n }\n }\n\n cfg_64bit_metrics! {\n pub(crate) fn store(&self, val: u64, ordering: Ordering) {\n self.value.store(val, ordering)\n }\n\n pub(crate) fn new(value: u64) -> Self {\n Self { value: AtomicU64::new(value) }\n }\n\n pub(crate) fn add(&self, value: u64, ordering: Ordering) {\n self.value.fetch_add(value, ordering);\n }\n }\n\n cfg_no_64bit_metrics! {\n pub(crate) fn store(&self, _val: u64, _ordering: Ordering) { }\n // on platforms without 64-bit atomics, fetch-add returns unit\n pub(crate) fn add(&self, _value: u64, _ordering: Ordering) { }\n pub(crate) fn new(_value: u64) -> Self { Self { } }\n }\n}\n\n#[cfg_attr(not(all(tokio_unstable, feature = \"rt\")), allow(dead_code))]\n/// `AtomicUsize` for use in metrics.\n///\n/// This exposes simplified APIs for use in metrics & uses `std::sync` instead of Loom to avoid polluting loom logs with metric information.\n#[derive(Debug, Default)]\npub(crate) struct MetricAtomicUsize {\n value: AtomicUsize,\n}\n\n#[cfg_attr(not(all(tokio_unstable, feature = \"rt\")), allow(dead_code))]\nimpl MetricAtomicUsize {\n pub(crate) fn new(value: usize) -> Self {\n Self {\n value: AtomicUsize::new(value),\n }\n }\n\n pub(crate) fn load(&self, ordering: Ordering) -> usize {\n self.value.load(ordering)\n }\n\n pub(crate) fn store(&self, val: usize, ordering: Ordering) {\n self.value.store(val, ordering)\n }\n\n pub(crate) fn increment(&self) -> usize {\n self.value.fetch_add(1, Ordering::Relaxed)\n }\n\n pub(crate) fn decrement(&self) -> usize {\n self.value.fetch_sub(1, Ordering::Relaxed)\n }\n}\n" }, { "path": "tokio/src/util/mod.rs", "content": "cfg_io_driver! {\n pub(crate) mod bit;\n}\n\n#[cfg(feature = \"fs\")]\npub(crate) mod as_ref;\n\n#[cfg(feature = \"rt\")]\npub(crate) mod atomic_cell;\n\n#[cfg(feature = \"net\")]\nmod blocking_check;\n#[cfg(feature = \"net\")]\n#[allow(unused_imports)]\npub(crate) use blocking_check::check_socket_for_blocking;\n\npub(crate) mod metric_atomics;\n\n#[cfg(any(\n // io driver uses `WakeList` directly\n feature = \"net\",\n feature = \"process\",\n // `sync` enables `Notify` and `batch_semaphore`, which require `WakeList`.\n feature = \"sync\",\n // `fs` uses `batch_semaphore`, which requires `WakeList`.\n feature = \"fs\",\n // rt and signal use `Notify`, which requires `WakeList`.\n feature = \"rt\",\n feature = \"signal\",\n // time driver uses `WakeList` in `Handle::process_at_time`.\n feature = \"time\",\n))]\nmod wake_list;\n#[cfg(any(\n feature = \"net\",\n feature = \"process\",\n feature = \"sync\",\n feature = \"fs\",\n feature = \"rt\",\n feature = \"signal\",\n feature = \"time\",\n))]\npub(crate) use wake_list::WakeList;\n\n#[cfg(any(\n feature = \"fs\",\n feature = \"net\",\n feature = \"process\",\n feature = \"rt\",\n feature = \"sync\",\n feature = \"signal\",\n feature = \"time\",\n fuzzing,\n))]\npub(crate) mod linked_list;\n\ncfg_rt! {\n pub(crate) mod sharded_list;\n}\n\n#[cfg(any(feature = \"rt\", feature = \"macros\"))]\npub(crate) mod rand;\n\ncfg_rt! {\n mod idle_notified_set;\n pub(crate) use idle_notified_set::IdleNotifiedSet;\n\n pub(crate) use self::rand::RngSeedGenerator;\n\n mod wake;\n pub(crate) use wake::WakerRef;\n pub(crate) use wake::{waker_ref, Wake};\n\n mod sync_wrapper;\n pub(crate) use sync_wrapper::SyncWrapper;\n\n mod rc_cell;\n pub(crate) use rc_cell::RcCell;\n}\n\ncfg_rt_multi_thread! {\n mod try_lock;\n pub(crate) use try_lock::TryLock;\n}\n\npub(crate) mod trace;\n\n#[cfg(feature = \"fs\")]\npub(crate) mod typeid;\n\npub(crate) mod error;\n\n#[cfg(feature = \"io-util\")]\npub(crate) mod memchr;\n\npub(crate) mod markers;\n\npub(crate) mod cacheline;\n\ncfg_io_driver_impl! {\n pub(crate) mod ptr_expose;\n}\n\nuse std::{ops::DerefMut, pin::Pin};\n\n/// Copy of [`std::pin::Pin::as_deref_mut`].\n// TODO: Remove this once we bump the MSRV to 1.84.\npub(crate) fn pin_as_deref_mut(ptr: Pin<&mut Pin

>) -> Pin<&mut P::Target> {\n unsafe { ptr.get_unchecked_mut() }.as_mut()\n}\n" }, { "path": "tokio/src/util/ptr_expose.rs", "content": "//! Utility for helping miri understand our exposed pointers.\n//!\n//! During normal execution, this module is equivalent to pointer casts. However, when running\n//! under miri, pointer casts are replaced with lookups in a hash map. This makes Tokio compatible\n//! with strict provenance when running under miri (which comes with a performance cost).\n\nuse std::marker::PhantomData;\n#[cfg(miri)]\nuse {crate::loom::sync::Mutex, std::collections::BTreeMap};\n\npub(crate) struct PtrExposeDomain {\n #[cfg(miri)]\n map: Mutex>,\n _phantom: PhantomData,\n}\n\n// SAFETY: Actually using the pointers is unsafe, so it's sound to transfer them across threads.\nunsafe impl Sync for PtrExposeDomain {}\n\nimpl PtrExposeDomain {\n pub(crate) const fn new() -> Self {\n Self {\n #[cfg(miri)]\n map: Mutex::const_new(BTreeMap::new()),\n _phantom: PhantomData,\n }\n }\n\n #[inline]\n pub(crate) fn expose_provenance(&self, ptr: *const T) -> usize {\n #[cfg(miri)]\n {\n let addr: usize = ptr.addr();\n self.map.lock().insert(addr, ptr);\n addr\n }\n\n #[cfg(not(miri))]\n {\n ptr as usize\n }\n }\n\n #[inline]\n #[allow(clippy::wrong_self_convention)] // mirrors std name\n pub(crate) fn from_exposed_addr(&self, addr: usize) -> *const T {\n #[cfg(miri)]\n {\n let maybe_ptr = self.map.lock().get(&addr).copied();\n\n // SAFETY: Intentionally trigger a miri failure if the provenance we want is not\n // exposed.\n unsafe { maybe_ptr.unwrap_unchecked() }\n }\n\n #[cfg(not(miri))]\n {\n addr as *const T\n }\n }\n\n #[inline]\n pub(crate) fn unexpose_provenance(&self, _ptr: *const T) {\n #[cfg(miri)]\n {\n let addr: usize = _ptr.addr();\n let maybe_ptr = self.map.lock().remove(&addr);\n\n // SAFETY: Intentionally trigger a miri failure if the provenance we want is not\n // exposed.\n unsafe { maybe_ptr.unwrap_unchecked() };\n }\n }\n}\n" }, { "path": "tokio/src/util/rand/rt.rs", "content": "use super::{FastRand, RngSeed};\n\nuse std::sync::Mutex;\n\n/// A deterministic generator for seeds (and other generators).\n///\n/// Given the same initial seed, the generator will output the same sequence of seeds.\n///\n/// Since the seed generator will be kept in a runtime handle, we need to wrap `FastRand`\n/// in a Mutex to make it thread safe. Different to the `FastRand` that we keep in a\n/// thread local store, the expectation is that seed generation will not need to happen\n/// very frequently, so the cost of the mutex should be minimal.\n#[derive(Debug)]\npub(crate) struct RngSeedGenerator {\n /// Internal state for the seed generator. We keep it in a Mutex so that we can safely\n /// use it across multiple threads.\n state: Mutex,\n}\n\nimpl RngSeedGenerator {\n /// Returns a new generator from the provided seed.\n pub(crate) fn new(seed: RngSeed) -> Self {\n Self {\n state: Mutex::new(FastRand::from_seed(seed)),\n }\n }\n\n /// Returns the next seed in the sequence.\n pub(crate) fn next_seed(&self) -> RngSeed {\n let mut rng = self\n .state\n .lock()\n .expect(\"RNG seed generator is internally corrupt\");\n\n let s = rng.fastrand();\n let r = rng.fastrand();\n\n RngSeed::from_pair(s, r)\n }\n\n /// Directly creates a generator using the next seed.\n pub(crate) fn next_generator(&self) -> Self {\n RngSeedGenerator::new(self.next_seed())\n }\n}\n\nimpl FastRand {\n /// Replaces the state of the random number generator with the provided seed, returning\n /// the seed that represents the previous state of the random number generator.\n ///\n /// The random number generator will become equivalent to one created with\n /// the same seed.\n pub(crate) fn replace_seed(&mut self, seed: RngSeed) -> RngSeed {\n let old_seed = RngSeed::from_pair(self.one, self.two);\n\n self.one = seed.s;\n self.two = seed.r;\n\n old_seed\n }\n}\n" }, { "path": "tokio/src/util/rand/rt_unstable.rs", "content": "use super::RngSeed;\n\nuse std::collections::hash_map::DefaultHasher;\nuse std::hash::Hasher;\n\nimpl RngSeed {\n /// Generates a seed from the provided byte slice.\n ///\n /// # Example\n ///\n /// ```\n /// # use tokio::runtime::RngSeed;\n /// let seed = RngSeed::from_bytes(b\"make me a seed\");\n /// ```\n pub fn from_bytes(bytes: &[u8]) -> Self {\n let mut hasher = DefaultHasher::default();\n hasher.write(bytes);\n Self::from_u64(hasher.finish())\n }\n}\n" }, { "path": "tokio/src/util/rand.rs", "content": "cfg_rt! {\n mod rt;\n pub(crate) use rt::RngSeedGenerator;\n\n cfg_unstable! {\n mod rt_unstable;\n }\n}\n\n/// A seed for random number generation.\n///\n/// In order to make certain functions within a runtime deterministic, a seed\n/// can be specified at the time of creation.\n#[allow(unreachable_pub)]\n#[derive(Clone, Debug)]\npub struct RngSeed {\n s: u32,\n r: u32,\n}\n\n/// Fast random number generate.\n///\n/// Implement `xorshift64+`: 2 32-bit `xorshift` sequences added together.\n/// Shift triplet `[17,7,16]` was calculated as indicated in Marsaglia's\n/// `Xorshift` paper: \n/// This generator passes the SmallCrush suite, part of TestU01 framework:\n/// \n#[derive(Clone, Copy, Debug)]\npub(crate) struct FastRand {\n one: u32,\n two: u32,\n}\n\nimpl RngSeed {\n /// Creates a random seed using loom internally.\n pub(crate) fn new() -> Self {\n Self::from_u64(crate::loom::rand::seed())\n }\n\n fn from_u64(seed: u64) -> Self {\n let one = (seed >> 32) as u32;\n let mut two = seed as u32;\n\n if two == 0 {\n // This value cannot be zero\n two = 1;\n }\n\n Self::from_pair(one, two)\n }\n\n fn from_pair(s: u32, r: u32) -> Self {\n Self { s, r }\n }\n}\n\nimpl FastRand {\n /// Initialize a new fast random number generator using the default source of entropy.\n pub(crate) fn new() -> FastRand {\n FastRand::from_seed(RngSeed::new())\n }\n\n /// Initializes a new, thread-local, fast random number generator.\n pub(crate) fn from_seed(seed: RngSeed) -> FastRand {\n FastRand {\n one: seed.s,\n two: seed.r,\n }\n }\n\n #[cfg(any(\n feature = \"macros\",\n feature = \"rt-multi-thread\",\n all(feature = \"sync\", feature = \"rt\")\n ))]\n pub(crate) fn fastrand_n(&mut self, n: u32) -> u32 {\n // This is similar to fastrand() % n, but faster.\n // See https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/\n let mul = (self.fastrand() as u64).wrapping_mul(n as u64);\n (mul >> 32) as u32\n }\n\n fn fastrand(&mut self) -> u32 {\n let mut s1 = self.one;\n let s0 = self.two;\n\n s1 ^= s1 << 17;\n s1 = s1 ^ s0 ^ s1 >> 7 ^ s0 >> 16;\n\n self.one = s0;\n self.two = s1;\n\n s0.wrapping_add(s1)\n }\n}\n" }, { "path": "tokio/src/util/rc_cell.rs", "content": "use crate::loom::cell::UnsafeCell;\n\nuse std::rc::Rc;\n\n/// This is exactly like `Cell>>`, except that it provides a `get`\n/// method even though `Rc` is not `Copy`.\npub(crate) struct RcCell {\n inner: UnsafeCell>>,\n}\n\nimpl RcCell {\n #[cfg(not(all(loom, test)))]\n pub(crate) const fn new() -> Self {\n Self {\n inner: UnsafeCell::new(None),\n }\n }\n\n // The UnsafeCell in loom does not have a const `new` fn.\n #[cfg(all(loom, test))]\n pub(crate) fn new() -> Self {\n Self {\n inner: UnsafeCell::new(None),\n }\n }\n\n /// Safety: This method may not be called recursively.\n #[inline]\n unsafe fn with_inner(&self, f: F) -> R\n where\n F: FnOnce(&mut Option>) -> R,\n {\n // safety: This type is not Sync, so concurrent calls of this method\n // cannot happen. Furthermore, the caller guarantees that the method is\n // not called recursively. Finally, this is the only place that can\n // create mutable references to the inner Rc. This ensures that any\n // mutable references created here are exclusive.\n self.inner.with_mut(|ptr| f(unsafe { &mut *ptr }))\n }\n\n pub(crate) fn get(&self) -> Option> {\n // safety: The `Rc::clone` method will not call any unknown user-code,\n // so it will not result in a recursive call to `with_inner`.\n unsafe { self.with_inner(|rc| rc.clone()) }\n }\n\n pub(crate) fn replace(&self, val: Option>) -> Option> {\n // safety: No destructors or other unknown user-code will run inside the\n // `with_inner` call, so no recursive call to `with_inner` can happen.\n unsafe { self.with_inner(|rc| std::mem::replace(rc, val)) }\n }\n\n pub(crate) fn set(&self, val: Option>) {\n let old = self.replace(val);\n drop(old);\n }\n}\n" }, { "path": "tokio/src/util/sharded_list.rs", "content": "use std::ptr::NonNull;\nuse std::sync::atomic::Ordering;\n\nuse crate::loom::sync::{Mutex, MutexGuard};\nuse crate::util::metric_atomics::{MetricAtomicU64, MetricAtomicUsize};\n\nuse super::linked_list::{Link, LinkedList};\n\n/// An intrusive linked list supporting highly concurrent updates.\n///\n/// It currently relies on `LinkedList`, so it is the caller's\n/// responsibility to ensure the list is empty before dropping it.\n///\n/// Note: Due to its inner sharded design, the order of nodes cannot be guaranteed.\npub(crate) struct ShardedList {\n lists: Box<[Mutex>]>,\n added: MetricAtomicU64,\n count: MetricAtomicUsize,\n shard_mask: usize,\n}\n\n/// Determines which linked list an item should be stored in.\n///\n/// # Safety\n///\n/// Implementations must guarantee that the id of an item does not change from\n/// call to call.\npub(crate) unsafe trait ShardedListItem: Link {\n /// # Safety\n ///\n /// The provided pointer must point at a valid list item.\n unsafe fn get_shard_id(target: NonNull) -> usize;\n}\n\nimpl ShardedList {\n /// Creates a new and empty sharded linked list with the specified size.\n pub(crate) fn new(sharded_size: usize) -> Self {\n assert!(sharded_size.is_power_of_two());\n\n let shard_mask = sharded_size - 1;\n let mut lists = Vec::with_capacity(sharded_size);\n for _ in 0..sharded_size {\n lists.push(Mutex::new(LinkedList::::new()))\n }\n Self {\n lists: lists.into_boxed_slice(),\n added: MetricAtomicU64::new(0),\n count: MetricAtomicUsize::new(0),\n shard_mask,\n }\n }\n}\n\n/// Used to get the lock of shard.\npub(crate) struct ShardGuard<'a, L, T> {\n lock: MutexGuard<'a, LinkedList>,\n added: &'a MetricAtomicU64,\n count: &'a MetricAtomicUsize,\n id: usize,\n}\n\nimpl ShardedList {\n /// Removes the last element from a list specified by `shard_id` and returns it, or None if it is\n /// empty.\n pub(crate) fn pop_back(&self, shard_id: usize) -> Option {\n let mut lock = self.shard_inner(shard_id);\n let node = lock.pop_back();\n if node.is_some() {\n self.count.decrement();\n }\n node\n }\n\n /// Removes the specified node from the list.\n ///\n /// # Safety\n ///\n /// The caller **must** ensure that exactly one of the following is true:\n /// - `node` is currently contained by `self`,\n /// - `node` is not contained by any list,\n /// - `node` is currently contained by some other `GuardedLinkedList`.\n pub(crate) unsafe fn remove(&self, node: NonNull) -> Option {\n let id = unsafe { L::get_shard_id(node) };\n let mut lock = self.shard_inner(id);\n // SAFETY: Since the shard id cannot change, it's not possible for this node\n // to be in any other list of the same sharded list.\n let node = unsafe { lock.remove(node) };\n if node.is_some() {\n self.count.decrement();\n }\n node\n }\n\n /// Gets the lock of `ShardedList`, makes us have the write permission.\n pub(crate) fn lock_shard(&self, val: &L::Handle) -> ShardGuard<'_, L, L::Target> {\n let id = unsafe { L::get_shard_id(L::as_raw(val)) };\n ShardGuard {\n lock: self.shard_inner(id),\n added: &self.added,\n count: &self.count,\n id,\n }\n }\n\n /// Gets the count of elements in this list.\n pub(crate) fn len(&self) -> usize {\n self.count.load(Ordering::Relaxed)\n }\n\n cfg_unstable_metrics! {\n cfg_64bit_metrics! {\n /// Gets the total number of elements added to this list.\n pub(crate) fn added(&self) -> u64 {\n self.added.load(Ordering::Relaxed)\n }\n }\n }\n\n /// Returns whether the linked list does not contain any node.\n pub(crate) fn is_empty(&self) -> bool {\n self.len() == 0\n }\n\n /// Gets the shard size of this `SharedList`.\n ///\n /// Used to help us to decide the parameter `shard_id` of the `pop_back` method.\n pub(crate) fn shard_size(&self) -> usize {\n self.shard_mask + 1\n }\n\n #[inline]\n fn shard_inner(&self, id: usize) -> MutexGuard<'_, LinkedList::Target>> {\n // Safety: This modulo operation ensures that the index is not out of bounds.\n unsafe { self.lists.get_unchecked(id & self.shard_mask).lock() }\n }\n}\n\nimpl<'a, L: ShardedListItem> ShardGuard<'a, L, L::Target> {\n /// Push a value to this shard.\n pub(crate) fn push(mut self, val: L::Handle) {\n let id = unsafe { L::get_shard_id(L::as_raw(&val)) };\n assert_eq!(id, self.id);\n self.lock.push_front(val);\n self.added.add(1, Ordering::Relaxed);\n self.count.increment();\n }\n}\n\ncfg_taskdump! {\n impl ShardedList {\n pub(crate) fn for_each(&self, mut f: F)\n where\n F: FnMut(&L::Handle),\n {\n let mut guards = Vec::with_capacity(self.lists.len());\n for list in self.lists.iter() {\n guards.push(list.lock());\n }\n for g in &mut guards {\n g.for_each(&mut f);\n }\n }\n }\n}\n" }, { "path": "tokio/src/util/sync_wrapper.rs", "content": "//! This module contains a type that can make `Send + !Sync` types `Sync` by\n//! disallowing all immutable access to the value.\n//!\n//! A similar primitive is provided in the `sync_wrapper` crate.\n\nuse std::any::Any;\n\npub(crate) struct SyncWrapper {\n value: T,\n}\n\n// safety: The SyncWrapper being send allows you to send the inner value across\n// thread boundaries.\nunsafe impl Send for SyncWrapper {}\n\n// safety: An immutable reference to a SyncWrapper is useless, so moving such an\n// immutable reference across threads is safe.\nunsafe impl Sync for SyncWrapper {}\n\nimpl SyncWrapper {\n pub(crate) fn new(value: T) -> Self {\n Self { value }\n }\n\n pub(crate) fn into_inner(self) -> T {\n self.value\n }\n}\n\nimpl SyncWrapper> {\n /// Attempt to downcast using `Any::downcast_ref()` to a type that is known to be `Sync`.\n pub(crate) fn downcast_ref_sync(&self) -> Option<&T> {\n // SAFETY: if the downcast fails, the inner value is not touched,\n // so no thread-safety violation can occur.\n self.value.downcast_ref()\n }\n}\n" }, { "path": "tokio/src/util/trace.rs", "content": "cfg_rt! {\n use std::marker::PhantomData;\n\n #[derive(Copy, Clone)]\n pub(crate) struct SpawnMeta<'a> {\n /// The name of the task\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) name: Option<&'a str>,\n /// The original size of the future or function being spawned\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n pub(crate) original_size: usize,\n /// The source code location where the task was spawned.\n ///\n /// This is wrapped in a type that may be empty when `tokio_unstable` is\n /// not enabled.\n pub(crate) spawned_at: crate::runtime::task::SpawnLocation,\n _pd: PhantomData<&'a ()>,\n }\n\n impl<'a> SpawnMeta<'a> {\n /// Create new spawn meta with a name and original size (before possible auto-boxing)\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n #[track_caller]\n pub(crate) fn new(name: Option<&'a str>, original_size: usize) -> Self {\n Self {\n name,\n original_size,\n spawned_at: crate::runtime::task::SpawnLocation::capture(),\n _pd: PhantomData,\n }\n }\n\n /// Create a new unnamed spawn meta with the original size (before possible auto-boxing)\n #[track_caller]\n pub(crate) fn new_unnamed(original_size: usize) -> Self {\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n let _original_size = original_size;\n\n Self {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n name: None,\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n original_size,\n spawned_at: crate::runtime::task::SpawnLocation::capture(),\n _pd: PhantomData,\n }\n }\n }\n\n cfg_trace! {\n use core::{\n pin::Pin,\n task::{Context, Poll},\n };\n use pin_project_lite::pin_project;\n use std::mem;\n use std::future::Future;\n use tracing::instrument::Instrument;\n pub(crate) use tracing::instrument::Instrumented;\n\n #[inline]\n pub(crate) fn task(task: F, kind: &'static str, meta: SpawnMeta<'_>, id: u64) -> Instrumented {\n fn get_span(kind: &'static str, spawn_meta: SpawnMeta<'_>, id: u64, task_size: usize) -> tracing::Span {\n let original_size = if spawn_meta.original_size != task_size {\n Some(spawn_meta.original_size)\n } else {\n None\n };\n tracing::trace_span!(\n target: \"tokio::task\",\n parent: None,\n \"runtime.spawn\",\n %kind,\n task.name = %spawn_meta.name.unwrap_or_default(),\n task.id = id,\n original_size.bytes = original_size,\n size.bytes = task_size,\n loc.file = spawn_meta.spawned_at.0.file(),\n loc.line = spawn_meta.spawned_at.0.line(),\n loc.col = spawn_meta.spawned_at.0.column(),\n )\n }\n use tracing::instrument::Instrument;\n let span = get_span(kind, meta, id, mem::size_of::());\n task.instrument(span)\n }\n\n #[inline]\n pub(crate) fn blocking_task(task: Fut, spawn_meta: SpawnMeta<'_>, id: u64) -> Instrumented {\n let fn_size = mem::size_of::();\n let original_size = if spawn_meta.original_size != fn_size {\n Some(spawn_meta.original_size)\n } else {\n None\n };\n\n let span = tracing::trace_span!(\n target: \"tokio::task::blocking\",\n \"runtime.spawn\",\n kind = %\"blocking\",\n task.name = %spawn_meta.name.unwrap_or_default(),\n task.id = id,\n \"fn\" = %std::any::type_name::(),\n original_size.bytes = original_size,\n size.bytes = fn_size,\n loc.file = spawn_meta.spawned_at.0.file(),\n loc.line = spawn_meta.spawned_at.0.line(),\n loc.col = spawn_meta.spawned_at.0.column(),\n );\n task.instrument(span)\n\n }\n\n pub(crate) fn async_op(inner: P, resource_span: tracing::Span, source: &str, poll_op_name: &'static str, inherits_child_attrs: bool) -> InstrumentedAsyncOp\n where P: FnOnce() -> F {\n resource_span.in_scope(|| {\n let async_op_span = tracing::trace_span!(\"runtime.resource.async_op\", source = source, inherits_child_attrs = inherits_child_attrs);\n let enter = async_op_span.enter();\n let async_op_poll_span = tracing::trace_span!(\"runtime.resource.async_op.poll\");\n let inner = inner();\n drop(enter);\n let tracing_ctx = AsyncOpTracingCtx {\n async_op_span,\n async_op_poll_span,\n resource_span: resource_span.clone(),\n };\n InstrumentedAsyncOp {\n inner,\n tracing_ctx,\n poll_op_name,\n }\n })\n }\n\n #[derive(Debug, Clone)]\n pub(crate) struct AsyncOpTracingCtx {\n pub(crate) async_op_span: tracing::Span,\n pub(crate) async_op_poll_span: tracing::Span,\n pub(crate) resource_span: tracing::Span,\n }\n\n\n pin_project! {\n #[derive(Debug, Clone)]\n pub(crate) struct InstrumentedAsyncOp {\n #[pin]\n pub(crate) inner: F,\n pub(crate) tracing_ctx: AsyncOpTracingCtx,\n pub(crate) poll_op_name: &'static str\n }\n }\n\n impl Future for InstrumentedAsyncOp {\n type Output = F::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n let poll_op_name = &*this.poll_op_name;\n let _res_enter = this.tracing_ctx.resource_span.enter();\n let _async_op_enter = this.tracing_ctx.async_op_span.enter();\n let _async_op_poll_enter = this.tracing_ctx.async_op_poll_span.enter();\n trace_poll_op!(poll_op_name, this.inner.poll(cx))\n }\n }\n }\n\n cfg_not_trace! {\n #[inline]\n pub(crate) fn task(task: F, _kind: &'static str, _meta: SpawnMeta<'_>, _id: u64) -> F {\n // nop\n task\n }\n\n #[inline]\n pub(crate) fn blocking_task(task: Fut, _spawn_meta: SpawnMeta<'_>, _id: u64) -> Fut {\n let _ = PhantomData::<&Fn>;\n // nop\n task\n }\n }\n}\n\ncfg_time! {\n #[track_caller]\n pub(crate) fn caller_location() -> Option<&'static std::panic::Location<'static>> {\n #[cfg(all(tokio_unstable, feature = \"tracing\"))]\n return Some(std::panic::Location::caller());\n #[cfg(not(all(tokio_unstable, feature = \"tracing\")))]\n None\n }\n}\n" }, { "path": "tokio/src/util/try_lock.rs", "content": "use crate::loom::sync::atomic::AtomicBool;\n\nuse std::cell::UnsafeCell;\nuse std::marker::PhantomData;\nuse std::ops::{Deref, DerefMut};\nuse std::sync::atomic::Ordering::SeqCst;\n\npub(crate) struct TryLock {\n locked: AtomicBool,\n data: UnsafeCell,\n}\n\npub(crate) struct LockGuard<'a, T> {\n lock: &'a TryLock,\n _p: PhantomData>,\n}\n\nunsafe impl Send for TryLock {}\nunsafe impl Sync for TryLock {}\n\nunsafe impl Sync for LockGuard<'_, T> {}\n\nmacro_rules! new {\n ($data:ident) => {\n TryLock {\n locked: AtomicBool::new(false),\n data: UnsafeCell::new($data),\n }\n };\n}\n\nimpl TryLock {\n #[cfg(not(loom))]\n /// Create a new `TryLock`\n pub(crate) const fn new(data: T) -> TryLock {\n new!(data)\n }\n\n #[cfg(loom)]\n /// Create a new `TryLock`\n pub(crate) fn new(data: T) -> TryLock {\n new!(data)\n }\n\n /// Attempt to acquire lock\n pub(crate) fn try_lock(&self) -> Option> {\n if self\n .locked\n .compare_exchange(false, true, SeqCst, SeqCst)\n .is_err()\n {\n return None;\n }\n\n Some(LockGuard {\n lock: self,\n _p: PhantomData,\n })\n }\n}\n\nimpl Deref for LockGuard<'_, T> {\n type Target = T;\n\n fn deref(&self) -> &T {\n unsafe { &*self.lock.data.get() }\n }\n}\n\nimpl DerefMut for LockGuard<'_, T> {\n fn deref_mut(&mut self) -> &mut T {\n unsafe { &mut *self.lock.data.get() }\n }\n}\n\nimpl Drop for LockGuard<'_, T> {\n fn drop(&mut self) {\n self.lock.locked.store(false, SeqCst);\n }\n}\n" }, { "path": "tokio/src/util/typeid.rs", "content": "use std::{\n any::TypeId,\n marker::PhantomData,\n mem::{self, ManuallyDrop},\n};\n\n// SAFETY: this function does not compare lifetimes. Values returned as `Ok`\n// may have their lifetimes extended.\npub(super) unsafe fn try_transmute(x: Src) -> Result {\n if nonstatic_typeid::() == TypeId::of::() {\n let x = ManuallyDrop::new(x);\n // SAFETY: we have checked that the types are the same.\n Ok(unsafe { mem::transmute_copy::(&x) })\n } else {\n Err(x)\n }\n}\n\n// https://github.com/dtolnay/typeid/blob/b06a3c08a0eaccc7df6091ade1ae4e3fb53609d5/src/lib.rs#L197-L222\n#[inline(always)]\nfn nonstatic_typeid() -> TypeId\nwhere\n T: ?Sized,\n{\n trait NonStaticAny {\n fn get_type_id(&self) -> TypeId\n where\n Self: 'static;\n }\n\n impl NonStaticAny for PhantomData {\n #[inline(always)]\n fn get_type_id(&self) -> TypeId\n where\n Self: 'static,\n {\n TypeId::of::()\n }\n }\n\n let phantom_data = PhantomData::;\n NonStaticAny::get_type_id(unsafe {\n mem::transmute::<&dyn NonStaticAny, &(dyn NonStaticAny + 'static)>(&phantom_data)\n })\n}\n" }, { "path": "tokio/src/util/wake.rs", "content": "use crate::loom::sync::Arc;\n\nuse std::marker::PhantomData;\nuse std::mem::ManuallyDrop;\nuse std::ops::Deref;\nuse std::task::{RawWaker, RawWakerVTable, Waker};\n\n/// Simplified waking interface based on Arcs.\npub(crate) trait Wake: Send + Sync + Sized + 'static {\n /// Wake by value.\n fn wake(arc_self: Arc);\n\n /// Wake by reference.\n fn wake_by_ref(arc_self: &Arc);\n}\n\n/// A `Waker` that is only valid for a given lifetime.\n#[derive(Debug)]\npub(crate) struct WakerRef<'a> {\n waker: ManuallyDrop,\n _p: PhantomData<&'a ()>,\n}\n\nimpl Deref for WakerRef<'_> {\n type Target = Waker;\n\n fn deref(&self) -> &Waker {\n &self.waker\n }\n}\n\n/// Creates a reference to a `Waker` from a reference to `Arc`.\npub(crate) fn waker_ref(wake: &Arc) -> WakerRef<'_> {\n let ptr = Arc::as_ptr(wake).cast::<()>();\n\n let waker = unsafe { Waker::from_raw(RawWaker::new(ptr, waker_vtable::())) };\n\n WakerRef {\n waker: ManuallyDrop::new(waker),\n _p: PhantomData,\n }\n}\n\nfn waker_vtable() -> &'static RawWakerVTable {\n &RawWakerVTable::new(\n clone_arc_raw::,\n wake_arc_raw::,\n wake_by_ref_arc_raw::,\n drop_arc_raw::,\n )\n}\n\nunsafe fn clone_arc_raw(data: *const ()) -> RawWaker {\n // Safety: `data` was created from an `Arc::as_ptr` in function `waker_ref`.\n unsafe {\n Arc::::increment_strong_count(data as *const T);\n }\n RawWaker::new(data, waker_vtable::())\n}\n\nunsafe fn wake_arc_raw(data: *const ()) {\n // Safety: `data` was created from an `Arc::as_ptr` in function `waker_ref`.\n let arc: Arc = unsafe { Arc::from_raw(data as *const T) };\n Wake::wake(arc);\n}\n\n// used by `waker_ref`\nunsafe fn wake_by_ref_arc_raw(data: *const ()) {\n // Retain Arc, but don't touch refcount by wrapping in ManuallyDrop\n // Safety: `data` was created from an `Arc::as_ptr` in function `waker_ref`.\n let arc = ManuallyDrop::new(unsafe { Arc::::from_raw(data.cast()) });\n Wake::wake_by_ref(&arc);\n}\n\nunsafe fn drop_arc_raw(data: *const ()) {\n // Safety: `data` was created from an `Arc::as_ptr` in function `waker_ref`.\n drop(unsafe { Arc::::from_raw(data.cast()) });\n}\n" }, { "path": "tokio/src/util/wake_list.rs", "content": "use core::mem::MaybeUninit;\nuse core::ptr;\nuse std::task::Waker;\n\nconst NUM_WAKERS: usize = 32;\n\n/// A list of wakers to be woken.\n///\n/// # Invariants\n///\n/// The first `curr` elements of `inner` are initialized.\npub(crate) struct WakeList {\n inner: [MaybeUninit; NUM_WAKERS],\n curr: usize,\n}\n\nimpl WakeList {\n pub(crate) fn new() -> Self {\n const UNINIT_WAKER: MaybeUninit = MaybeUninit::uninit();\n\n Self {\n inner: [UNINIT_WAKER; NUM_WAKERS],\n curr: 0,\n }\n }\n\n #[inline]\n pub(crate) fn can_push(&self) -> bool {\n self.curr < NUM_WAKERS\n }\n\n pub(crate) fn push(&mut self, val: Waker) {\n debug_assert!(self.can_push());\n\n self.inner[self.curr] = MaybeUninit::new(val);\n self.curr += 1;\n }\n\n pub(crate) fn wake_all(&mut self) {\n struct DropGuard {\n start: *mut Waker,\n end: *mut Waker,\n }\n\n impl Drop for DropGuard {\n fn drop(&mut self) {\n // SAFETY: Both pointers are part of the same object, with `start <= end`.\n let len = unsafe { self.end.offset_from(self.start) } as usize;\n let slice = ptr::slice_from_raw_parts_mut(self.start, len);\n // SAFETY: All elements in `start..len` are initialized, so we can drop them.\n unsafe { ptr::drop_in_place(slice) };\n }\n }\n\n debug_assert!(self.curr <= NUM_WAKERS);\n\n let mut guard = {\n let start = self.inner.as_mut_ptr().cast::();\n // SAFETY: The resulting pointer is in bounds or one after the length of the same object.\n let end = unsafe { start.add(self.curr) };\n // Transfer ownership of the wakers in `inner` to `DropGuard`.\n self.curr = 0;\n DropGuard { start, end }\n };\n while !ptr::eq(guard.start, guard.end) {\n // SAFETY: `start` is always initialized if `start != end`.\n let waker = unsafe { ptr::read(guard.start) };\n // SAFETY: The resulting pointer is in bounds or one after the length of the same object.\n guard.start = unsafe { guard.start.add(1) };\n // If this panics, then `guard` will clean up the remaining wakers.\n waker.wake();\n }\n }\n}\n\nimpl Drop for WakeList {\n fn drop(&mut self) {\n let slice =\n ptr::slice_from_raw_parts_mut(self.inner.as_mut_ptr().cast::(), self.curr);\n // SAFETY: The first `curr` elements are initialized, so we can drop them.\n unsafe { ptr::drop_in_place(slice) };\n }\n}\n" }, { "path": "tokio/tests/_require_full.rs", "content": "#[cfg(not(any(feature = \"full\", target_family = \"wasm\")))]\ncompile_error!(\"run main Tokio tests with `--features full`\");\n\n// CI sets `--cfg tokio_no_parking_lot` when trying to run tests with\n// `parking_lot` disabled. This check prevents \"silent failure\" if `parking_lot`\n// accidentally gets enabled.\n#[cfg(all(tokio_no_parking_lot, feature = \"parking_lot\"))]\ncompile_error!(\"parking_lot feature enabled when it should not be\");\n" }, { "path": "tokio/tests/async_send_sync.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![allow(clippy::type_complexity, clippy::diverging_sub_expression)]\n\nuse std::cell::Cell;\nuse std::future::Future;\nuse std::io::SeekFrom;\nuse std::net::SocketAddr;\nuse std::pin::Pin;\nuse std::rc::Rc;\nuse tokio::net::TcpStream;\nuse tokio::time::{Duration, Instant};\n\n// The names of these structs behaves better when sorted.\n// Send: Yes, Sync: Yes\n#[derive(Clone)]\n#[allow(unused)]\nstruct YY {}\n\n// Send: Yes, Sync: No\n#[derive(Clone)]\n#[allow(unused)]\nstruct YN {\n _value: Cell,\n}\n\n// Send: No, Sync: No\n#[derive(Clone)]\n#[allow(unused)]\nstruct NN {\n _value: Rc,\n}\n\n#[allow(dead_code)]\ntype BoxFutureSync = std::pin::Pin + Send + Sync>>;\n#[allow(dead_code)]\ntype BoxFutureSend = std::pin::Pin + Send>>;\n#[allow(dead_code)]\ntype BoxFuture = std::pin::Pin>>;\n\n#[allow(dead_code)]\ntype BoxAsyncRead = std::pin::Pin>;\n#[allow(dead_code)]\ntype BoxAsyncSeek = std::pin::Pin>;\n#[allow(dead_code)]\ntype BoxAsyncWrite = std::pin::Pin>;\n\n#[allow(dead_code)]\nfn require_send(_t: &T) {}\n#[allow(dead_code)]\nfn require_sync(_t: &T) {}\n#[allow(dead_code)]\nfn require_unpin(_t: &T) {}\n\n#[allow(dead_code)]\nstruct Invalid;\n\n#[allow(unused)]\ntrait AmbiguousIfSend {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfSend<()> for T {}\nimpl AmbiguousIfSend for T {}\n\n#[allow(unused)]\ntrait AmbiguousIfSync {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfSync<()> for T {}\nimpl AmbiguousIfSync for T {}\n\n#[allow(unused)]\ntrait AmbiguousIfUnpin {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfUnpin<()> for T {}\nimpl AmbiguousIfUnpin for T {}\n\nmacro_rules! into_todo {\n ($typ:ty) => {{\n let x: $typ = todo!();\n x\n }};\n}\n\nmacro_rules! async_assert_fn_send {\n (Send & $(!)?Sync & $(!)?Unpin, $value:expr) => {\n require_send(&$value);\n };\n (!Send & $(!)?Sync & $(!)?Unpin, $value:expr) => {\n AmbiguousIfSend::some_item(&$value);\n };\n}\nmacro_rules! async_assert_fn_sync {\n ($(!)?Send & Sync & $(!)?Unpin, $value:expr) => {\n require_sync(&$value);\n };\n ($(!)?Send & !Sync & $(!)?Unpin, $value:expr) => {\n AmbiguousIfSync::some_item(&$value);\n };\n}\nmacro_rules! async_assert_fn_unpin {\n ($(!)?Send & $(!)?Sync & Unpin, $value:expr) => {\n require_unpin(&$value);\n };\n ($(!)?Send & $(!)?Sync & !Unpin, $value:expr) => {\n AmbiguousIfUnpin::some_item(&$value);\n };\n}\n\nmacro_rules! async_assert_fn {\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): $($tok:tt)*) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n async_assert_fn_send!($($tok)*, f);\n async_assert_fn_sync!($($tok)*, f);\n async_assert_fn_unpin!($($tok)*, f);\n };\n };\n}\nmacro_rules! assert_value {\n ($type:ty: $($tok:tt)*) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f: $type = todo!();\n async_assert_fn_send!($($tok)*, f);\n async_assert_fn_sync!($($tok)*, f);\n async_assert_fn_unpin!($($tok)*, f);\n };\n };\n}\n\nmacro_rules! cfg_not_wasi {\n ($($item:item)*) => {\n $(\n #[cfg(not(target_os = \"wasi\"))]\n $item\n )*\n }\n}\n\n// Manually re-implementation of `async_assert_fn` for `poll_fn`. The macro\n// doesn't work for this particular case because constructing the closure\n// is too complicated.\nconst _: fn() = || {\n let pinned = std::marker::PhantomPinned;\n let f = tokio::macros::support::poll_fn(move |_| {\n // Use `pinned` to take ownership of it.\n let _ = &pinned;\n std::task::Poll::Pending::<()>\n });\n require_send(&f);\n require_sync(&f);\n AmbiguousIfUnpin::some_item(&f);\n};\n\ncfg_not_wasi! {\n mod fs {\n use super::*;\n assert_value!(tokio::fs::DirBuilder: Send & Sync & Unpin);\n assert_value!(tokio::fs::DirEntry: Send & Sync & Unpin);\n assert_value!(tokio::fs::File: Send & Sync & Unpin);\n assert_value!(tokio::fs::OpenOptions: Send & Sync & Unpin);\n assert_value!(tokio::fs::ReadDir: Send & Sync & Unpin);\n\n async_assert_fn!(tokio::fs::canonicalize(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::copy(&str, &str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::create_dir(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::create_dir_all(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::hard_link(&str, &str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::metadata(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::read(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::read_dir(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::read_link(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::read_to_string(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::remove_dir(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::remove_dir_all(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::remove_file(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::rename(&str, &str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::set_permissions(&str, std::fs::Permissions): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::symlink_metadata(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::write(&str, Vec): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::ReadDir::next_entry(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::OpenOptions::open(_, &str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::DirBuilder::create(_, &str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::DirEntry::metadata(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::DirEntry::file_type(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::open(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::create(&str): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::sync_all(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::sync_data(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::set_len(_, u64): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::metadata(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::try_clone(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::fs::File::into_std(_): Send & Sync & !Unpin);\n async_assert_fn!(\n tokio::fs::File::set_permissions(_, std::fs::Permissions): Send & Sync & !Unpin\n );\n }\n}\n\ncfg_not_wasi! {\n assert_value!(tokio::net::TcpSocket: Send & Sync & Unpin);\n async_assert_fn!(tokio::net::TcpListener::bind(SocketAddr): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::TcpStream::connect(SocketAddr): Send & Sync & !Unpin);\n}\n\nassert_value!(tokio::net::TcpListener: Send & Sync & Unpin);\nassert_value!(tokio::net::TcpStream: Send & Sync & Unpin);\nassert_value!(tokio::net::tcp::OwnedReadHalf: Send & Sync & Unpin);\nassert_value!(tokio::net::tcp::OwnedWriteHalf: Send & Sync & Unpin);\nassert_value!(tokio::net::tcp::ReadHalf<'_>: Send & Sync & Unpin);\nassert_value!(tokio::net::tcp::ReuniteError: Send & Sync & Unpin);\nassert_value!(tokio::net::tcp::WriteHalf<'_>: Send & Sync & Unpin);\nasync_assert_fn!(tokio::net::TcpListener::accept(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::net::TcpStream::peek(_, &mut [u8]): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::net::TcpStream::readable(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::net::TcpStream::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::net::TcpStream::writable(_): Send & Sync & !Unpin);\n\n// Wasi does not support UDP\ncfg_not_wasi! {\n mod udp_socket {\n use super::*;\n assert_value!(tokio::net::UdpSocket: Send & Sync & Unpin);\n async_assert_fn!(tokio::net::UdpSocket::bind(SocketAddr): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::connect(_, SocketAddr): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::peek_from(_, &mut [u8]): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::recv(_, &mut [u8]): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::recv_from(_, &mut [u8]): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::send(_, &[u8]): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::send_to(_, &[u8], SocketAddr): Send & Sync & !Unpin);\n async_assert_fn!(tokio::net::UdpSocket::writable(_): Send & Sync & !Unpin);\n }\n}\nasync_assert_fn!(tokio::net::lookup_host(SocketAddr): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::net::tcp::ReadHalf::peek(_, &mut [u8]): Send & Sync & !Unpin);\n\n#[cfg(unix)]\nmod unix_datagram {\n use super::*;\n use tokio::net::*;\n assert_value!(UnixDatagram: Send & Sync & Unpin);\n assert_value!(UnixListener: Send & Sync & Unpin);\n assert_value!(UnixStream: Send & Sync & Unpin);\n assert_value!(unix::OwnedReadHalf: Send & Sync & Unpin);\n assert_value!(unix::OwnedWriteHalf: Send & Sync & Unpin);\n assert_value!(unix::ReadHalf<'_>: Send & Sync & Unpin);\n assert_value!(unix::ReuniteError: Send & Sync & Unpin);\n assert_value!(unix::SocketAddr: Send & Sync & Unpin);\n assert_value!(unix::UCred: Send & Sync & Unpin);\n assert_value!(unix::WriteHalf<'_>: Send & Sync & Unpin);\n async_assert_fn!(UnixDatagram::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::recv(_, &mut [u8]): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::recv_from(_, &mut [u8]): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::send(_, &[u8]): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::send_to(_, &[u8], &str): Send & Sync & !Unpin);\n async_assert_fn!(UnixDatagram::writable(_): Send & Sync & !Unpin);\n async_assert_fn!(UnixListener::accept(_): Send & Sync & !Unpin);\n async_assert_fn!(UnixStream::connect(&str): Send & Sync & !Unpin);\n async_assert_fn!(UnixStream::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(UnixStream::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(UnixStream::writable(_): Send & Sync & !Unpin);\n}\n\n#[cfg(unix)]\nmod unix_pipe {\n use super::*;\n use tokio::net::unix::pipe::*;\n assert_value!(OpenOptions: Send & Sync & Unpin);\n assert_value!(Receiver: Send & Sync & Unpin);\n assert_value!(Sender: Send & Sync & Unpin);\n async_assert_fn!(Receiver::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(Receiver::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(Sender::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(Sender::writable(_): Send & Sync & !Unpin);\n}\n\n#[cfg(windows)]\nmod windows_named_pipe {\n use super::*;\n use tokio::net::windows::named_pipe::*;\n assert_value!(ClientOptions: Send & Sync & Unpin);\n assert_value!(NamedPipeClient: Send & Sync & Unpin);\n assert_value!(NamedPipeServer: Send & Sync & Unpin);\n assert_value!(PipeEnd: Send & Sync & Unpin);\n assert_value!(PipeInfo: Send & Sync & Unpin);\n assert_value!(PipeMode: Send & Sync & Unpin);\n assert_value!(ServerOptions: Send & Sync & Unpin);\n async_assert_fn!(NamedPipeClient::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeClient::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeClient::writable(_): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeServer::connect(_): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeServer::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeServer::ready(_, tokio::io::Interest): Send & Sync & !Unpin);\n async_assert_fn!(NamedPipeServer::writable(_): Send & Sync & !Unpin);\n}\n\ncfg_not_wasi! {\n mod test_process {\n use super::*;\n assert_value!(tokio::process::Child: Send & Sync & Unpin);\n assert_value!(tokio::process::ChildStderr: Send & Sync & Unpin);\n assert_value!(tokio::process::ChildStdin: Send & Sync & Unpin);\n assert_value!(tokio::process::ChildStdout: Send & Sync & Unpin);\n assert_value!(tokio::process::Command: Send & Sync & Unpin);\n async_assert_fn!(tokio::process::Child::kill(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::process::Child::wait(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::process::Child::wait_with_output(_): Send & Sync & !Unpin);\n }\n\n async_assert_fn!(tokio::signal::ctrl_c(): Send & Sync & !Unpin);\n}\n\n#[cfg(unix)]\nmod unix_signal {\n use super::*;\n assert_value!(tokio::signal::unix::Signal: Send & Sync & Unpin);\n assert_value!(tokio::signal::unix::SignalKind: Send & Sync & Unpin);\n async_assert_fn!(tokio::signal::unix::Signal::recv(_): Send & Sync & !Unpin);\n}\n#[cfg(windows)]\nmod windows_signal {\n use super::*;\n assert_value!(tokio::signal::windows::CtrlC: Send & Sync & Unpin);\n assert_value!(tokio::signal::windows::CtrlBreak: Send & Sync & Unpin);\n async_assert_fn!(tokio::signal::windows::CtrlC::recv(_): Send & Sync & !Unpin);\n async_assert_fn!(tokio::signal::windows::CtrlBreak::recv(_): Send & Sync & !Unpin);\n}\n\nassert_value!(tokio::sync::AcquireError: Send & Sync & Unpin);\nassert_value!(tokio::sync::Barrier: Send & Sync & Unpin);\nassert_value!(tokio::sync::BarrierWaitResult: Send & Sync & Unpin);\nassert_value!(tokio::sync::MappedMutexGuard<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::MappedMutexGuard<'_, YN>: Send & !Sync & Unpin);\nassert_value!(tokio::sync::MappedMutexGuard<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::Mutex: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::Mutex: Send & Sync & Unpin);\nassert_value!(tokio::sync::Mutex: Send & Sync & Unpin);\nassert_value!(tokio::sync::MutexGuard<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::MutexGuard<'_, YN>: Send & !Sync & Unpin);\nassert_value!(tokio::sync::MutexGuard<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::Notify: Send & Sync & Unpin);\nassert_value!(tokio::sync::OnceCell: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OnceCell: Send & !Sync & Unpin);\nassert_value!(tokio::sync::OnceCell: Send & Sync & Unpin);\nassert_value!(tokio::sync::SetOnce: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::SetOnce: Send & !Sync & Unpin);\nassert_value!(tokio::sync::SetOnce: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMutexGuard: Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMutexGuard: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedMappedMutexGuard: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockMappedWriteGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockMappedWriteGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockMappedWriteGuard: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockReadGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockReadGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockReadGuard: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockWriteGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockWriteGuard: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::OwnedRwLockWriteGuard: Send & Sync & Unpin);\nassert_value!(tokio::sync::OwnedSemaphorePermit: Send & Sync & Unpin);\nassert_value!(tokio::sync::RwLock: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLock: Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLock: Send & Sync & Unpin);\nassert_value!(tokio::sync::RwLockMappedWriteGuard<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockMappedWriteGuard<'_, YN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockMappedWriteGuard<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::RwLockReadGuard<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockReadGuard<'_, YN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockReadGuard<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::RwLockWriteGuard<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockWriteGuard<'_, YN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::RwLockWriteGuard<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::Semaphore: Send & Sync & Unpin);\nassert_value!(tokio::sync::SemaphorePermit<'_>: Send & Sync & Unpin);\nassert_value!(tokio::sync::TryAcquireError: Send & Sync & Unpin);\nassert_value!(tokio::sync::TryLockError: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Receiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Sender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::WeakSender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::broadcast::WeakSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::broadcast::WeakSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::futures::Notified<'_>: Send & Sync & !Unpin);\nassert_value!(tokio::sync::futures::OwnedNotified: Send & Sync & !Unpin);\nassert_value!(tokio::sync::mpsc::OwnedPermit: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::OwnedPermit: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::OwnedPermit: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Permit<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Permit<'_, YN>: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Permit<'_, YY>: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Receiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Sender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedReceiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedReceiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedReceiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedSender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::UnboundedSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakSender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakUnboundedSender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakUnboundedSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::WeakUnboundedSender: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendError: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendError: Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendError: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendTimeoutError: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendTimeoutError: Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::SendTimeoutError: Send & Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::TrySendError: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::TrySendError: Send & !Sync & Unpin);\nassert_value!(tokio::sync::mpsc::error::TrySendError: Send & Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Receiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Sender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::oneshot::Sender: Send & Sync & Unpin);\nassert_value!(tokio::sync::watch::Receiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Receiver: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Receiver: Send & Sync & Unpin);\nassert_value!(tokio::sync::watch::Ref<'_, NN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Ref<'_, YN>: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Ref<'_, YY>: !Send & Sync & Unpin);\nassert_value!(tokio::sync::watch::Sender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Sender: !Send & !Sync & Unpin);\nassert_value!(tokio::sync::watch::Sender: Send & Sync & Unpin);\nassert_value!(tokio::task::JoinError: Send & Sync & Unpin);\nassert_value!(tokio::task::JoinHandle: !Send & !Sync & Unpin);\nassert_value!(tokio::task::JoinHandle: Send & Sync & Unpin);\nassert_value!(tokio::task::JoinHandle: Send & Sync & Unpin);\nassert_value!(tokio::task::JoinSet: !Send & !Sync & Unpin);\nassert_value!(tokio::task::JoinSet: Send & Sync & Unpin);\nassert_value!(tokio::task::JoinSet: Send & Sync & Unpin);\nassert_value!(tokio::task::LocalSet: !Send & !Sync & Unpin);\nassert_value!(tokio::task::coop::RestoreOnPending: !Send & !Sync & Unpin);\nasync_assert_fn!(tokio::sync::Barrier::wait(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock_owned(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock_owned(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Mutex::lock_owned(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Notify::notified(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send + Sync>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send + Sync>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin>>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send + Sync>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send + Sync>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin>>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send + Sync>>): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin + Send>>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_init( _, fn() -> Pin>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send + Sync>>): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin> + Send>>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::OnceCell::get_or_try_init( _, fn() -> Pin>>>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::SetOnce::wait(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::SetOnce::wait(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::SetOnce::wait(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::read(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::write(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::read(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::write(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::read(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::RwLock::write(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Semaphore::acquire(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Semaphore::acquire_many(_, u32): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Semaphore::acquire_many_owned(_, u32): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::Semaphore::acquire_owned(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::broadcast::Receiver::recv(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::broadcast::Receiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::broadcast::Receiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Receiver::recv(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Receiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Receiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::closed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve_owned(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send(_, NN): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send_timeout(_, NN, Duration): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve_owned(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send(_, YN): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send_timeout(_, YN, Duration): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::reserve_owned(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send(_, YY): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::Sender::send_timeout(_, YY, Duration): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedReceiver::recv(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedReceiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedReceiver::recv(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedSender::closed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedSender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::mpsc::UnboundedSender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::oneshot::Sender::closed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::oneshot::Sender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::oneshot::Sender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Receiver::changed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Receiver::changed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Receiver::changed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Sender::closed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Sender::closed(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::sync::watch::Sender::closed(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet>::join_next(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet>::shutdown(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet>::join_next(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet>::shutdown(_): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet::join_next(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::JoinSet::shutdown(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Cell, BoxFuture<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Cell, BoxFutureSend<()>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Cell, BoxFutureSync<()>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Rc, BoxFuture<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Rc, BoxFutureSend<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey>::scope(_, Rc, BoxFutureSync<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey::scope(_, u32, BoxFuture<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey::scope(_, u32, BoxFutureSend<()>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalKey::scope(_, u32, BoxFutureSync<()>): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::task::LocalSet::run_until(_, BoxFutureSync<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::task::unconstrained(BoxFuture<()>): !Send & !Sync & Unpin);\nasync_assert_fn!(tokio::task::unconstrained(BoxFutureSend<()>): Send & !Sync & Unpin);\nasync_assert_fn!(tokio::task::unconstrained(BoxFutureSync<()>): Send & Sync & Unpin);\n\nassert_value!(tokio::runtime::Builder: Send & Sync & Unpin);\nassert_value!(tokio::runtime::EnterGuard<'_>: !Send & Sync & Unpin);\nassert_value!(tokio::runtime::Handle: Send & Sync & Unpin);\nassert_value!(tokio::runtime::Runtime: Send & Sync & Unpin);\n\nassert_value!(tokio::time::Interval: Send & Sync & Unpin);\nassert_value!(tokio::time::Instant: Send & Sync & Unpin);\nassert_value!(tokio::time::Sleep: Send & Sync & !Unpin);\nassert_value!(tokio::time::Timeout>: Send & Sync & !Unpin);\nassert_value!(tokio::time::Timeout>: Send & !Sync & !Unpin);\nassert_value!(tokio::time::Timeout>: !Send & !Sync & !Unpin);\nassert_value!(tokio::time::error::Elapsed: Send & Sync & Unpin);\nassert_value!(tokio::time::error::Error: Send & Sync & Unpin);\nasync_assert_fn!(tokio::time::advance(Duration): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::time::sleep(Duration): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::time::sleep_until(Instant): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout(Duration, BoxFutureSync<()>): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout(Duration, BoxFutureSend<()>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout(Duration, BoxFuture<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout_at(Instant, BoxFutureSync<()>): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout_at(Instant, BoxFutureSend<()>): Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::time::timeout_at(Instant, BoxFuture<()>): !Send & !Sync & !Unpin);\nasync_assert_fn!(tokio::time::Interval::tick(_): Send & Sync & !Unpin);\n\nassert_value!(tokio::io::BufReader: Send & Sync & Unpin);\nassert_value!(tokio::io::BufStream: Send & Sync & Unpin);\nassert_value!(tokio::io::BufWriter: Send & Sync & Unpin);\nassert_value!(tokio::io::DuplexStream: Send & Sync & Unpin);\nassert_value!(tokio::io::Empty: Send & Sync & Unpin);\nassert_value!(tokio::io::Interest: Send & Sync & Unpin);\nassert_value!(tokio::io::Lines: Send & Sync & Unpin);\nassert_value!(tokio::io::ReadBuf<'_>: Send & Sync & Unpin);\nassert_value!(tokio::io::ReadHalf: Send & Sync & Unpin);\nassert_value!(tokio::io::Ready: Send & Sync & Unpin);\nassert_value!(tokio::io::Repeat: Send & Sync & Unpin);\nassert_value!(tokio::io::Sink: Send & Sync & Unpin);\nassert_value!(tokio::io::Split: Send & Sync & Unpin);\nassert_value!(tokio::io::Stderr: Send & Sync & Unpin);\nassert_value!(tokio::io::Stdin: Send & Sync & Unpin);\nassert_value!(tokio::io::Stdout: Send & Sync & Unpin);\nassert_value!(tokio::io::Take: Send & Sync & Unpin);\nassert_value!(tokio::io::WriteHalf: Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::copy(&mut TcpStream, &mut TcpStream): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::copy_bidirectional(&mut TcpStream, &mut TcpStream): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::copy_buf(&mut tokio::io::BufReader, &mut TcpStream): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::empty(): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::repeat(u8): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::sink(): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::split(TcpStream): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::stderr(): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::stdin(): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::stdout(): Send & Sync & Unpin);\nasync_assert_fn!(tokio::io::Split>::next_segment(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::Lines>::next_line(_): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncBufReadExt::read_until(&mut BoxAsyncRead, u8, &mut Vec): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::AsyncBufReadExt::read_line(&mut BoxAsyncRead, &mut String): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::AsyncBufReadExt::fill_buf(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read(&mut BoxAsyncRead, &mut [u8]): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_buf(&mut BoxAsyncRead, &mut Vec): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::AsyncReadExt::read_exact(&mut BoxAsyncRead, &mut [u8]): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u8(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i8(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u16(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i16(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u32(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i32(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u64(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i64(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u128(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i128(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_f32(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_f64(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u16_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i16_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u32_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i32_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u64_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i64_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_u128_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_i128_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_f32_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_f64_le(&mut BoxAsyncRead): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncReadExt::read_to_end(&mut BoxAsyncRead, &mut Vec): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::AsyncReadExt::read_to_string(&mut BoxAsyncRead, &mut String): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::AsyncSeekExt::seek(&mut BoxAsyncSeek, SeekFrom): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncSeekExt::stream_position(&mut BoxAsyncSeek): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncWriteExt::write(&mut BoxAsyncWrite, &[u8]): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_vectored(&mut BoxAsyncWrite, _): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_buf(&mut BoxAsyncWrite, &mut bytes::Bytes): Send\n & Sync\n & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_all_buf(&mut BoxAsyncWrite, &mut bytes::Bytes): Send\n & Sync\n & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_all(&mut BoxAsyncWrite, &[u8]): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::AsyncWriteExt::write_u8(&mut BoxAsyncWrite, u8): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncWriteExt::write_i8(&mut BoxAsyncWrite, i8): Send & Sync & !Unpin);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u16(&mut BoxAsyncWrite, u16): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i16(&mut BoxAsyncWrite, i16): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u32(&mut BoxAsyncWrite, u32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i32(&mut BoxAsyncWrite, i32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u64(&mut BoxAsyncWrite, u64): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i64(&mut BoxAsyncWrite, i64): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u128(&mut BoxAsyncWrite, u128): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i128(&mut BoxAsyncWrite, i128): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_f32(&mut BoxAsyncWrite, f32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_f64(&mut BoxAsyncWrite, f64): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u16_le(&mut BoxAsyncWrite, u16): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i16_le(&mut BoxAsyncWrite, i16): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u32_le(&mut BoxAsyncWrite, u32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i32_le(&mut BoxAsyncWrite, i32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u64_le(&mut BoxAsyncWrite, u64): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i64_le(&mut BoxAsyncWrite, i64): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_u128_le(&mut BoxAsyncWrite, u128): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_i128_le(&mut BoxAsyncWrite, i128): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_f32_le(&mut BoxAsyncWrite, f32): Send & Sync & !Unpin\n);\nasync_assert_fn!(\n tokio::io::AsyncWriteExt::write_f64_le(&mut BoxAsyncWrite, f64): Send & Sync & !Unpin\n);\nasync_assert_fn!(tokio::io::AsyncWriteExt::flush(&mut BoxAsyncWrite): Send & Sync & !Unpin);\nasync_assert_fn!(tokio::io::AsyncWriteExt::shutdown(&mut BoxAsyncWrite): Send & Sync & !Unpin);\n\n#[cfg(unix)]\nmod unix_asyncfd {\n use super::*;\n use tokio::io::unix::*;\n\n #[allow(unused)]\n struct ImplsFd {\n _t: T,\n }\n impl std::os::unix::io::AsRawFd for ImplsFd {\n fn as_raw_fd(&self) -> std::os::unix::io::RawFd {\n unreachable!()\n }\n }\n\n assert_value!(AsyncFd>: Send & Sync & Unpin);\n assert_value!(AsyncFd>: Send & !Sync & Unpin);\n assert_value!(AsyncFd>: !Send & !Sync & Unpin);\n assert_value!(AsyncFdReadyGuard<'_, ImplsFd>: Send & Sync & Unpin);\n assert_value!(AsyncFdReadyGuard<'_, ImplsFd>: !Send & !Sync & Unpin);\n assert_value!(AsyncFdReadyGuard<'_, ImplsFd>: !Send & !Sync & Unpin);\n assert_value!(AsyncFdReadyMutGuard<'_, ImplsFd>: Send & Sync & Unpin);\n assert_value!(AsyncFdReadyMutGuard<'_, ImplsFd>: Send & !Sync & Unpin);\n assert_value!(AsyncFdReadyMutGuard<'_, ImplsFd>: !Send & !Sync & Unpin);\n assert_value!(TryIoError: Send & Sync & Unpin);\n async_assert_fn!(AsyncFd>::readable(_): Send & Sync & !Unpin);\n async_assert_fn!(AsyncFd>::readable_mut(_): Send & Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable(_): Send & Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable_mut(_): Send & Sync & !Unpin);\n async_assert_fn!(AsyncFd>::readable(_): !Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::readable_mut(_): Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable(_): !Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable_mut(_): Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::readable(_): !Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::readable_mut(_): !Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable(_): !Send & !Sync & !Unpin);\n async_assert_fn!(AsyncFd>::writable_mut(_): !Send & !Sync & !Unpin);\n}\n\n#[cfg(tokio_unstable)]\nmod unstable {\n use super::*;\n\n assert_value!(tokio::runtime::LocalRuntime: !Send & !Sync & Unpin);\n assert_value!(tokio::runtime::LocalOptions: !Send & !Sync & Unpin);\n}\n" }, { "path": "tokio/tests/buffered.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support bind()\n\nuse tokio::net::TcpListener;\nuse tokio_test::assert_ok;\n\nuse std::io::prelude::*;\nuse std::net::TcpStream;\nuse std::thread;\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn echo_server() {\n const N: usize = 1024;\n\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n\n let msg = \"foo bar baz\";\n\n let t = thread::spawn(move || {\n let mut s = assert_ok!(TcpStream::connect(addr));\n\n let t2 = thread::spawn(move || {\n let mut s = assert_ok!(TcpStream::connect(addr));\n let mut b = vec![0; msg.len() * N];\n assert_ok!(s.read_exact(&mut b));\n b\n });\n\n let mut expected = Vec::::new();\n for _i in 0..N {\n expected.extend(msg.as_bytes());\n let res = assert_ok!(s.write(msg.as_bytes()));\n assert_eq!(res, msg.len());\n }\n\n (expected, t2)\n });\n\n let (mut a, _) = assert_ok!(srv.accept().await);\n let (mut b, _) = assert_ok!(srv.accept().await);\n\n let n = assert_ok!(tokio::io::copy(&mut a, &mut b).await);\n\n let (expected, t2) = t.join().unwrap();\n let actual = t2.join().unwrap();\n\n assert!(expected == actual);\n assert_eq!(n, msg.len() as u64 * 1024);\n}\n" }, { "path": "tokio/tests/coop_budget.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", target_os = \"linux\"))]\n\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::Arc;\nuse tokio::net::UdpSocket;\nuse tokio::task::coop::{consume_budget, has_budget_remaining};\n\nconst BUDGET: usize = 128;\n\n/// Ensure that UDP sockets have functional budgeting\n///\n/// # Design\n/// Two sockets communicate by spamming packets from one to the other.\n///\n/// In Linux, this packet will be slammed through the entire network stack and into the receiver's buffer during the\n/// send system call because we are using the loopback interface.\n/// This happens because the softirq chain invoked on send when using the loopback interface covers virtually the\n/// entirety of the lifecycle of a packet within the kernel network stack.\n///\n/// As a result, neither socket will ever encounter an EWOULDBLOCK, and the only way for these to yield during the loop\n/// is through budgeting.\n///\n/// A second task runs in the background and increments a counter before yielding, allowing us to know how many times sockets yielded.\n/// Since we are both sending and receiving, that should happen once per 64 packets, because budgets are of size 128\n/// and there are two budget events per packet, a send and a recv.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn coop_budget_udp_send_recv() {\n const N_ITERATIONS: usize = 1024;\n\n const PACKET: &[u8] = b\"Hello, world\";\n const PACKET_LEN: usize = 12;\n\n assert_eq!(\n PACKET_LEN,\n PACKET.len(),\n \"Defect in test, programmer can't do math\"\n );\n\n // bind each socket to a dynamic port, forcing IPv4 addressing on the localhost interface\n let tx = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let rx = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n tx.connect(rx.local_addr().unwrap()).await.unwrap();\n rx.connect(tx.local_addr().unwrap()).await.unwrap();\n\n let tracker = Arc::new(AtomicUsize::default());\n\n let tracker_clone = Arc::clone(&tracker);\n\n tokio::task::yield_now().await;\n\n tokio::spawn(async move {\n loop {\n tracker_clone.fetch_add(1, Ordering::SeqCst);\n\n tokio::task::yield_now().await;\n }\n });\n\n for _ in 0..N_ITERATIONS {\n tx.send(PACKET).await.unwrap();\n\n let mut tmp = [0; PACKET_LEN];\n\n // ensure that we aren't somehow accumulating other\n assert_eq!(\n PACKET_LEN,\n rx.recv(&mut tmp).await.unwrap(),\n \"Defect in test case, received unexpected result from socket\"\n );\n assert_eq!(\n PACKET, &tmp,\n \"Defect in test case, received unexpected result from socket\"\n );\n }\n\n assert_eq!(N_ITERATIONS / (BUDGET / 2), tracker.load(Ordering::SeqCst));\n}\n\n#[tokio::test]\nasync fn test_has_budget_remaining() {\n // At the beginning, budget should be available.\n assert!(has_budget_remaining());\n\n // Deplete the budget\n for _ in 0..BUDGET {\n consume_budget().await;\n }\n\n assert!(!has_budget_remaining());\n}\n" }, { "path": "tokio/tests/dump.rs", "content": "#![cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n))]\n\nuse std::hint::black_box;\nuse tokio::runtime::{self, Handle};\n\n#[inline(never)]\nasync fn a() {\n black_box(b()).await\n}\n\n#[inline(never)]\nasync fn b() {\n black_box(c()).await\n}\n\n#[inline(never)]\nasync fn c() {\n loop {\n black_box(tokio::task::yield_now()).await\n }\n}\n\n#[test]\nfn current_thread() {\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n async fn dump() {\n let handle = Handle::current();\n let dump = handle.dump().await;\n\n let tasks: Vec<_> = dump.tasks().iter().collect();\n\n assert_eq!(tasks.len(), 3);\n\n for task in tasks {\n let id = task.id();\n let trace = task.trace().to_string();\n eprintln!(\"\\n\\n{id}:\\n{trace}\\n\\n\");\n assert!(trace.contains(\"dump::a\"));\n assert!(trace.contains(\"dump::b\"));\n assert!(trace.contains(\"dump::c\"));\n assert!(trace.contains(\"tokio::task::yield_now\"));\n }\n }\n\n rt.block_on(async {\n tokio::select!(\n biased;\n _ = tokio::spawn(a()) => {},\n _ = tokio::spawn(a()) => {},\n _ = tokio::spawn(a()) => {},\n _ = dump() => {},\n );\n });\n}\n\n#[test]\nfn multi_thread() {\n let rt = runtime::Builder::new_multi_thread()\n .enable_all()\n .worker_threads(3)\n .build()\n .unwrap();\n\n async fn dump() {\n let handle = Handle::current();\n let dump = handle.dump().await;\n\n let tasks: Vec<_> = dump.tasks().iter().collect();\n\n assert_eq!(tasks.len(), 3);\n\n for task in tasks {\n let id = task.id();\n let trace = task.trace().to_string();\n eprintln!(\"\\n\\n{id}:\\n{trace}\\n\\n\");\n assert!(trace.contains(\"dump::a\"));\n assert!(trace.contains(\"dump::b\"));\n assert!(trace.contains(\"dump::c\"));\n assert!(trace.contains(\"tokio::task::yield_now\"));\n }\n }\n\n rt.block_on(async {\n tokio::select!(\n biased;\n _ = tokio::spawn(a()) => {},\n _ = tokio::spawn(a()) => {},\n _ = tokio::spawn(a()) => {},\n _ = dump() => {},\n );\n });\n}\n\n/// Regression tests for #6035.\n///\n/// These tests ensure that dumping will not deadlock if a future completes\n/// during a trace.\nmod future_completes_during_trace {\n use super::*;\n\n use core::future::{poll_fn, Future};\n\n /// A future that completes only during a trace.\n fn complete_during_trace() -> impl Future + Send {\n use std::task::Poll;\n poll_fn(|cx| {\n if Handle::is_tracing() {\n Poll::Ready(())\n } else {\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n })\n }\n\n #[test]\n fn current_thread() {\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n async fn dump() {\n let handle = Handle::current();\n let _dump = handle.dump().await;\n }\n\n rt.block_on(async {\n let _ = tokio::join!(tokio::spawn(complete_during_trace()), dump());\n });\n }\n\n #[test]\n fn multi_thread() {\n let rt = runtime::Builder::new_multi_thread()\n .enable_all()\n .build()\n .unwrap();\n\n async fn dump() {\n let handle = Handle::current();\n let _dump = handle.dump().await;\n tokio::task::yield_now().await;\n }\n\n rt.block_on(async {\n let _ = tokio::join!(tokio::spawn(complete_during_trace()), dump());\n });\n }\n}\n\n/// Regression test for #6051.\n///\n/// This test ensures that tasks notified outside of a worker will not be\n/// traced, since doing so will un-set their notified bit prior to them being\n/// run and panic.\n#[test]\nfn notified_during_tracing() {\n let rt = runtime::Builder::new_multi_thread()\n .enable_all()\n .worker_threads(3)\n .build()\n .unwrap();\n\n let timeout = async {\n tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;\n };\n\n let timer = rt.spawn(async {\n loop {\n tokio::time::sleep(tokio::time::Duration::from_nanos(1)).await;\n }\n });\n\n let dump = async {\n loop {\n let handle = Handle::current();\n let _dump = handle.dump().await;\n }\n };\n\n rt.block_on(async {\n tokio::select!(\n biased;\n _ = timeout => {},\n _ = timer => {},\n _ = dump => {},\n );\n });\n}\n" }, { "path": "tokio/tests/duplex_stream.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::io::IoSlice;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\n\nconst HELLO: &[u8] = b\"hello world...\";\n\n#[tokio::test]\nasync fn write_vectored() {\n let (mut client, mut server) = tokio::io::duplex(64);\n\n let ret = client\n .write_vectored(&[IoSlice::new(HELLO), IoSlice::new(HELLO)])\n .await\n .unwrap();\n assert_eq!(ret, HELLO.len() * 2);\n\n client.flush().await.unwrap();\n drop(client);\n\n let mut buf = Vec::with_capacity(HELLO.len() * 2);\n let bytes_read = server.read_to_end(&mut buf).await.unwrap();\n\n assert_eq!(bytes_read, HELLO.len() * 2);\n assert_eq!(buf, [HELLO, HELLO].concat());\n}\n\n#[tokio::test]\nasync fn write_vectored_and_shutdown() {\n let (mut client, mut server) = tokio::io::duplex(64);\n\n let ret = client\n .write_vectored(&[IoSlice::new(HELLO), IoSlice::new(HELLO)])\n .await\n .unwrap();\n assert_eq!(ret, HELLO.len() * 2);\n\n client.shutdown().await.unwrap();\n drop(client);\n\n let mut buf = Vec::with_capacity(HELLO.len() * 2);\n let bytes_read = server.read_to_end(&mut buf).await.unwrap();\n\n assert_eq!(bytes_read, HELLO.len() * 2);\n assert_eq!(buf, [HELLO, HELLO].concat());\n}\n" }, { "path": "tokio/tests/fs.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support file operations\n\nuse tokio::fs;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn path_read_write() {\n let temp = tempdir();\n let dir = temp.path();\n\n assert_ok!(fs::write(dir.join(\"bar\"), b\"bytes\").await);\n let out = assert_ok!(fs::read(dir.join(\"bar\")).await);\n\n assert_eq!(out, b\"bytes\");\n}\n\n#[tokio::test]\nasync fn try_clone_should_preserve_max_buf_size() {\n let buf_size = 128;\n let temp = tempdir();\n let dir = temp.path();\n\n let mut file = fs::File::create(dir.join(\"try_clone_should_preserve_max_buf_size\"))\n .await\n .unwrap();\n file.set_max_buf_size(buf_size);\n\n let cloned = file.try_clone().await.unwrap();\n\n assert_eq!(cloned.max_buf_size(), buf_size);\n}\n\nfn tempdir() -> tempfile::TempDir {\n tempfile::tempdir().unwrap()\n}\n" }, { "path": "tokio/tests/fs_canonicalize_dir.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tokio::fs;\n\n#[tokio::test]\n#[cfg(unix)]\nasync fn canonicalize_root_dir_unix() {\n assert_eq!(fs::canonicalize(\"/.\").await.unwrap().to_str().unwrap(), \"/\");\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn canonicalize_root_dir_windows() {\n // 2-step let bindings due to Rust memory semantics\n let dir_path = fs::canonicalize(\"C:\\\\.\\\\\").await.unwrap();\n\n let dir_name = dir_path.to_str().unwrap();\n\n assert!(dir_name.starts_with(\"\\\\\\\\\"));\n assert!(dir_name.ends_with(\"C:\\\\\"));\n}\n" }, { "path": "tokio/tests/fs_copy.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `fchmod` in miri.\nasync fn copy() {\n let dir = tempdir().unwrap();\n\n let source_path = dir.path().join(\"foo.txt\");\n let dest_path = dir.path().join(\"bar.txt\");\n\n fs::write(&source_path, b\"Hello File!\").await.unwrap();\n fs::copy(&source_path, &dest_path).await.unwrap();\n\n let from = fs::read(&source_path).await.unwrap();\n let to = fs::read(&dest_path).await.unwrap();\n\n assert_eq!(from, to);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `fchmod` in miri.\nasync fn copy_permissions() {\n let dir = tempdir().unwrap();\n let from_path = dir.path().join(\"foo.txt\");\n let to_path = dir.path().join(\"bar.txt\");\n\n let from = tokio::fs::File::create(&from_path).await.unwrap();\n let mut from_perms = from.metadata().await.unwrap().permissions();\n from_perms.set_readonly(true);\n from.set_permissions(from_perms.clone()).await.unwrap();\n\n tokio::fs::copy(from_path, &to_path).await.unwrap();\n\n let to_perms = tokio::fs::metadata(to_path).await.unwrap().permissions();\n\n assert_eq!(from_perms, to_perms);\n}\n" }, { "path": "tokio/tests/fs_dir.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tokio::fs;\nuse tokio_test::{assert_err, assert_ok};\n\nuse std::sync::{Arc, Mutex};\nuse tempfile::tempdir;\n\n#[tokio::test]\nasync fn create_dir() {\n let base_dir = tempdir().unwrap();\n let new_dir = base_dir.path().join(\"foo\");\n let new_dir_2 = new_dir.clone();\n\n assert_ok!(fs::create_dir(new_dir).await);\n\n assert!(new_dir_2.is_dir());\n}\n\n#[tokio::test]\nasync fn create_all() {\n let base_dir = tempdir().unwrap();\n let new_dir = base_dir.path().join(\"foo\").join(\"bar\");\n let new_dir_2 = new_dir.clone();\n\n assert_ok!(fs::create_dir_all(new_dir).await);\n assert!(new_dir_2.is_dir());\n}\n\n#[tokio::test]\nasync fn build_dir() {\n let base_dir = tempdir().unwrap();\n let new_dir = base_dir.path().join(\"foo\").join(\"bar\");\n let new_dir_2 = new_dir.clone();\n\n assert_ok!(fs::DirBuilder::new().recursive(true).create(new_dir).await);\n\n assert!(new_dir_2.is_dir());\n assert_err!(\n fs::DirBuilder::new()\n .recursive(false)\n .create(new_dir_2)\n .await\n );\n}\n\n#[tokio::test]\n#[cfg(unix)]\nasync fn build_dir_mode_read_only() {\n let base_dir = tempdir().unwrap();\n let new_dir = base_dir.path().join(\"abc\");\n\n assert_ok!(\n fs::DirBuilder::new()\n .recursive(true)\n .mode(0o444)\n .create(&new_dir)\n .await\n );\n\n assert!(fs::metadata(new_dir)\n .await\n .expect(\"metadata result\")\n .permissions()\n .readonly());\n}\n\n#[tokio::test]\nasync fn remove() {\n let base_dir = tempdir().unwrap();\n let new_dir = base_dir.path().join(\"foo\");\n let new_dir_2 = new_dir.clone();\n\n std::fs::create_dir(new_dir.clone()).unwrap();\n\n assert_ok!(fs::remove_dir(new_dir).await);\n assert!(!new_dir_2.exists());\n}\n\n#[tokio::test]\nasync fn read_inherent() {\n let base_dir = tempdir().unwrap();\n\n let p = base_dir.path();\n std::fs::create_dir(p.join(\"aa\")).unwrap();\n std::fs::create_dir(p.join(\"bb\")).unwrap();\n std::fs::create_dir(p.join(\"cc\")).unwrap();\n\n let files = Arc::new(Mutex::new(Vec::new()));\n\n let f = files.clone();\n let p = p.to_path_buf();\n\n let mut entries = fs::read_dir(p).await.unwrap();\n\n while let Some(e) = assert_ok!(entries.next_entry().await) {\n let s = e.file_name().to_str().unwrap().to_string();\n f.lock().unwrap().push(s);\n }\n\n let mut files = files.lock().unwrap();\n files.sort(); // because the order is not guaranteed\n assert_eq!(\n *files,\n vec![\"aa\".to_string(), \"bb\".to_string(), \"cc\".to_string()]\n );\n}\n\n#[tokio::test]\nasync fn read_dir_entry_info() {\n let temp_dir = tempdir().unwrap();\n\n let file_path = temp_dir.path().join(\"a.txt\");\n\n fs::write(&file_path, b\"Hello File!\").await.unwrap();\n\n let mut dir = fs::read_dir(temp_dir.path()).await.unwrap();\n\n let first_entry = dir.next_entry().await.unwrap().unwrap();\n\n assert_eq!(first_entry.path(), file_path);\n assert_eq!(first_entry.file_name(), \"a.txt\");\n assert!(first_entry.metadata().await.unwrap().is_file());\n assert!(first_entry.file_type().await.unwrap().is_file());\n}\n" }, { "path": "tokio/tests/fs_file.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse futures::future::FutureExt;\nuse std::io::prelude::*;\nuse std::io::IoSlice;\nuse tempfile::NamedTempFile;\nuse tokio::fs::File;\nuse tokio::io::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt, SeekFrom};\nuse tokio_test::task;\n\nconst HELLO: &[u8] = b\"hello world...\";\n\n#[tokio::test]\nasync fn basic_read() {\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n\n let mut file = File::open(tempfile.path()).await.unwrap();\n\n let mut buf = [0; 1024];\n let n = file.read(&mut buf).await.unwrap();\n\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[tokio::test]\nasync fn basic_write() {\n let tempfile = tempfile();\n\n let mut file = File::create(tempfile.path()).await.unwrap();\n\n file.write_all(HELLO).await.unwrap();\n file.flush().await.unwrap();\n\n let file = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(file, HELLO);\n}\n\n#[tokio::test]\nasync fn basic_write_and_shutdown() {\n let tempfile = tempfile();\n\n let mut file = File::create(tempfile.path()).await.unwrap();\n\n file.write_all(HELLO).await.unwrap();\n file.shutdown().await.unwrap();\n\n let file = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(file, HELLO);\n}\n\n#[tokio::test]\nasync fn write_vectored() {\n let tempfile = tempfile();\n\n let mut file = File::create(tempfile.path()).await.unwrap();\n\n let ret = file\n .write_vectored(&[IoSlice::new(HELLO), IoSlice::new(HELLO)])\n .await\n .unwrap();\n assert_eq!(ret, HELLO.len() * 2);\n file.flush().await.unwrap();\n\n let file = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(file, [HELLO, HELLO].concat());\n}\n\n#[tokio::test]\nasync fn write_vectored_and_shutdown() {\n let tempfile = tempfile();\n\n let mut file = File::create(tempfile.path()).await.unwrap();\n\n let ret = file\n .write_vectored(&[IoSlice::new(HELLO), IoSlice::new(HELLO)])\n .await\n .unwrap();\n assert_eq!(ret, HELLO.len() * 2);\n file.shutdown().await.unwrap();\n\n let file = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(file, [HELLO, HELLO].concat());\n}\n\n#[tokio::test]\nasync fn rewind_seek_position() {\n let tempfile = tempfile();\n\n let mut file = File::create(tempfile.path()).await.unwrap();\n\n file.seek(SeekFrom::Current(10)).await.unwrap();\n\n file.rewind().await.unwrap();\n\n assert_eq!(file.stream_position().await.unwrap(), 0);\n}\n\n#[tokio::test]\nasync fn coop() {\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n\n let mut task = task::spawn(async {\n let mut file = File::open(tempfile.path()).await.unwrap();\n\n let mut buf = [0; 1024];\n\n loop {\n let _ = file.read(&mut buf).await.unwrap();\n file.seek(std::io::SeekFrom::Start(0)).await.unwrap();\n }\n });\n\n for _ in 0..1_000 {\n if task.poll().is_pending() {\n return;\n }\n }\n\n panic!(\"did not yield\");\n}\n\n#[tokio::test]\nasync fn write_to_clone() {\n let tempfile = tempfile();\n\n let file = File::create(tempfile.path()).await.unwrap();\n let mut clone = file.try_clone().await.unwrap();\n\n clone.write_all(HELLO).await.unwrap();\n clone.flush().await.unwrap();\n\n let contents = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(contents, HELLO);\n}\n\n#[tokio::test]\nasync fn write_into_std() {\n let tempfile = tempfile();\n\n let file = File::create(tempfile.path()).await.unwrap();\n let mut std_file = file.into_std().await;\n\n std_file.write_all(HELLO).unwrap();\n\n let contents = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(contents, HELLO);\n}\n\n#[tokio::test]\nasync fn write_into_std_immediate() {\n let tempfile = tempfile();\n\n let file = File::create(tempfile.path()).await.unwrap();\n let mut std_file = file.try_into_std().unwrap();\n\n std_file.write_all(HELLO).unwrap();\n\n let contents = std::fs::read(tempfile.path()).unwrap();\n assert_eq!(contents, HELLO);\n}\n\n#[tokio::test]\nasync fn read_file_from_std() {\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n\n let std_file = std::fs::File::open(tempfile.path()).unwrap();\n let mut file = File::from(std_file);\n\n let mut buf = [0; 1024];\n let n = file.read(&mut buf).await.unwrap();\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[tokio::test]\nasync fn empty_read() {\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n\n let mut file = File::open(tempfile.path()).await.unwrap();\n\n // Perform an empty read and get a length of zero.\n assert!(matches!(file.read(&mut []).now_or_never(), Some(Ok(0))));\n\n // Check that we don't get EOF on the next read.\n let mut buf = [0; 1024];\n let n = file.read(&mut buf).await.unwrap();\n\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\nfn tempfile() -> NamedTempFile {\n NamedTempFile::new().unwrap()\n}\n\n#[tokio::test]\nasync fn set_max_buf_size_read() {\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n let mut file = File::open(tempfile.path()).await.unwrap();\n let mut buf = [0; 1024];\n file.set_max_buf_size(1);\n\n // A single read operation reads a maximum of 1 byte.\n assert_eq!(file.read(&mut buf).await.unwrap(), 1);\n}\n\n#[tokio::test]\nasync fn set_max_buf_size_write() {\n let tempfile = tempfile();\n let mut file = File::create(tempfile.path()).await.unwrap();\n file.set_max_buf_size(1);\n\n // A single write operation writes a maximum of 1 byte.\n assert_eq!(file.write(HELLO).await.unwrap(), 1);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)]\n#[cfg(unix)]\nasync fn file_debug_fmt() {\n let tempfile = tempfile();\n\n let file = File::open(tempfile.path()).await.unwrap();\n\n assert_eq!(\n &format!(\"{file:?}\")[0..33],\n \"tokio::fs::File { std: File { fd:\"\n );\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn file_debug_fmt() {\n let tempfile = tempfile();\n\n let file = File::open(tempfile.path()).await.unwrap();\n\n assert_eq!(\n &format!(\"{:?}\", file)[0..37],\n \"tokio::fs::File { std: File { handle:\"\n );\n}\n\n#[tokio::test]\n#[cfg(unix)]\nasync fn unix_fd_is_valid() {\n use std::os::unix::io::AsRawFd;\n let tempfile = tempfile();\n\n let file = File::create(tempfile.path()).await.unwrap();\n assert!(file.as_raw_fd() as u64 > 0);\n}\n\n#[tokio::test]\n#[cfg(unix)]\nasync fn read_file_from_unix_fd() {\n use std::os::unix::io::{FromRawFd, IntoRawFd};\n\n let mut tempfile = tempfile();\n tempfile.write_all(HELLO).unwrap();\n\n let file1 = File::open(tempfile.path()).await.unwrap();\n let raw_fd = file1.into_std().await.into_raw_fd();\n assert!(raw_fd > 0);\n\n let mut file2 = unsafe { File::from_raw_fd(raw_fd) };\n\n let mut buf = [0; 1024];\n let n = file2.read(&mut buf).await.unwrap();\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn windows_handle() {\n use std::os::windows::io::AsRawHandle;\n let tempfile = tempfile();\n\n let file = File::create(tempfile.path()).await.unwrap();\n assert!(file.as_raw_handle() as u64 > 0);\n}\n" }, { "path": "tokio/tests/fs_link.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tokio::fs;\n\nuse std::io::Write;\nuse tempfile::tempdir;\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `linkat` in miri.\nasync fn test_hard_link() {\n let dir = tempdir().unwrap();\n let src = dir.path().join(\"src.txt\");\n let dst = dir.path().join(\"dst.txt\");\n\n std::fs::File::create(&src)\n .unwrap()\n .write_all(b\"hello\")\n .unwrap();\n\n fs::hard_link(&src, &dst).await.unwrap();\n\n std::fs::File::create(&src)\n .unwrap()\n .write_all(b\"new-data\")\n .unwrap();\n\n let content = fs::read(&dst).await.unwrap();\n assert_eq!(content, b\"new-data\");\n\n // test that this is not a symlink:\n assert!(fs::read_link(&dst).await.is_err());\n}\n\n#[cfg(unix)]\n#[tokio::test]\nasync fn test_symlink() {\n let dir = tempdir().unwrap();\n let src = dir.path().join(\"src.txt\");\n let dst = dir.path().join(\"dst.txt\");\n\n std::fs::File::create(&src)\n .unwrap()\n .write_all(b\"hello\")\n .unwrap();\n\n fs::symlink(&src, &dst).await.unwrap();\n\n std::fs::File::create(&src)\n .unwrap()\n .write_all(b\"new-data\")\n .unwrap();\n\n let content = fs::read(&dst).await.unwrap();\n assert_eq!(content, b\"new-data\");\n\n let read = fs::read_link(dst.clone()).await.unwrap();\n assert!(read == src);\n\n let symlink_meta = fs::symlink_metadata(dst.clone()).await.unwrap();\n assert!(symlink_meta.file_type().is_symlink());\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `linkat` in miri.\nasync fn test_hard_link_error_source_not_found() {\n let dir = tempdir().unwrap();\n let src = dir.path().join(\"nonexistent.txt\");\n let dst = dir.path().join(\"dst.txt\");\n\n let err = fs::hard_link(&src, &dst).await.unwrap_err();\n assert_eq!(err.kind(), std::io::ErrorKind::NotFound);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `linkat` in miri.\nasync fn test_hard_link_error_destination_already_exists() {\n let dir = tempdir().unwrap();\n let src = dir.path().join(\"src.txt\");\n let dst = dir.path().join(\"dst.txt\");\n\n // Create source file\n std::fs::write(&src, b\"source content\").unwrap();\n\n // Create destination file\n std::fs::write(&dst, b\"destination content\").unwrap();\n\n // Attempt to create hard link when destination already exists\n let err = fs::hard_link(&src, &dst).await.unwrap_err();\n assert_eq!(err.kind(), std::io::ErrorKind::AlreadyExists);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `linkat` in miri.\nasync fn test_hard_link_error_source_is_directory() {\n let dir = tempdir().unwrap();\n let src_dir = dir.path().join(\"src_directory\");\n let dst = dir.path().join(\"dst.txt\");\n\n // Create source directory\n fs::create_dir(&src_dir).await.unwrap();\n\n // Attempt to create hard link from a directory\n // On most systems, hard linking directories is not allowed\n let err = fs::hard_link(&src_dir, &dst).await.unwrap_err();\n\n // Different platforms return different error kinds\n #[cfg(unix)]\n assert!(\n err.kind() == std::io::ErrorKind::PermissionDenied\n || err.kind() == std::io::ErrorKind::Other\n );\n\n #[cfg(windows)]\n assert_eq!(err.kind(), std::io::ErrorKind::PermissionDenied);\n}\n" }, { "path": "tokio/tests/fs_open_options.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse std::io::Write;\nuse tempfile::NamedTempFile;\nuse tokio::fs::OpenOptions;\nuse tokio::io::AsyncReadExt;\n\nconst HELLO: &[u8] = b\"hello world...\";\n\n#[tokio::test]\nasync fn open_with_open_options_and_read() {\n let mut tempfile = NamedTempFile::new().unwrap();\n tempfile.write_all(HELLO).unwrap();\n\n let mut file = OpenOptions::new().read(true).open(tempfile).await.unwrap();\n\n let mut buf = [0; 1024];\n let n = file.read(&mut buf).await.unwrap();\n\n assert_eq!(n, HELLO.len());\n assert_eq!(&buf[..n], HELLO);\n}\n\n#[tokio::test]\nasync fn open_options_write() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().write(true)).contains(\"write: true\"));\n}\n\n#[tokio::test]\nasync fn open_options_append() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().append(true)).contains(\"append: true\"));\n}\n\n#[tokio::test]\nasync fn open_options_truncate() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().truncate(true)).contains(\"truncate: true\"));\n}\n\n#[tokio::test]\nasync fn open_options_create() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().create(true)).contains(\"create: true\"));\n}\n\n#[tokio::test]\nasync fn open_options_create_new() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().create_new(true)).contains(\"create_new: true\"));\n}\n\n#[tokio::test]\n#[cfg(unix)]\nasync fn open_options_mode() {\n let mode = format!(\"{:?}\", OpenOptions::new().mode(0o644));\n // TESTING HACK: use Debug output to check the stored data\n assert!(\n mode.contains(\"mode: 420\") || mode.contains(\"mode: 0o000644\"),\n \"mode is: {mode}\"\n );\n}\n\n#[tokio::test]\n#[cfg(target_os = \"linux\")]\nasync fn open_options_custom_flags_linux() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(\n format!(\"{:?}\", OpenOptions::new().custom_flags(libc::O_TRUNC))\n .contains(\"custom_flags: 512\")\n );\n}\n\n#[tokio::test]\n#[cfg(any(target_os = \"freebsd\", target_os = \"macos\"))]\nasync fn open_options_custom_flags_bsd_family() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(\n format!(\"{:?}\", OpenOptions::new().custom_flags(libc::O_NOFOLLOW))\n .contains(\"custom_flags: 256,\")\n );\n}\n" }, { "path": "tokio/tests/fs_open_options_windows.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n#![cfg(windows)]\n\nuse tokio::fs::OpenOptions;\nuse windows_sys::Win32::Storage::FileSystem;\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn open_options_windows_access_mode() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().access_mode(0)).contains(\"access_mode: Some(0)\"));\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn open_options_windows_share_mode() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\"{:?}\", OpenOptions::new().share_mode(0)).contains(\"share_mode: 0,\"));\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn open_options_windows_custom_flags() {\n // TESTING HACK: use Debug output to check the stored data\n assert!(format!(\n \"{:?}\",\n OpenOptions::new().custom_flags(FileSystem::FILE_FLAG_DELETE_ON_CLOSE)\n )\n .contains(\"custom_flags: 67108864,\"));\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn open_options_windows_attributes() {\n assert!(format!(\n \"{:?}\",\n OpenOptions::new().attributes(FileSystem::FILE_ATTRIBUTE_HIDDEN)\n )\n .contains(\"attributes: 2,\"));\n}\n\n#[tokio::test]\n#[cfg(windows)]\nasync fn open_options_windows_security_qos_flags() {\n assert!(format!(\n \"{:?}\",\n OpenOptions::new().security_qos_flags(FileSystem::SECURITY_IDENTIFICATION)\n )\n .contains(\"security_qos_flags: 1114112,\"));\n}\n" }, { "path": "tokio/tests/fs_remove_dir_all.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn remove_dir_all() {\n let temp_dir = tempdir().unwrap();\n\n let test_dir = temp_dir.path().join(\"test\");\n fs::create_dir(&test_dir).await.unwrap();\n\n let file_path = test_dir.as_path().join(\"a.txt\");\n\n fs::write(&file_path, b\"Hello File!\").await.unwrap();\n\n fs::remove_dir_all(test_dir.as_path()).await.unwrap();\n\n // test dir should no longer exist\n match fs::try_exists(test_dir).await {\n Ok(exists) => assert!(!exists),\n Err(_) => println!(\"ignored try_exists error after remove_dir_all\"),\n };\n\n // contents should no longer exist\n match fs::try_exists(file_path).await {\n Ok(exists) => assert!(!exists),\n Err(_) => println!(\"ignored try_exists error after remove_dir_all\"),\n };\n}\n" }, { "path": "tokio/tests/fs_remove_file.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn remove_file() {\n let temp_dir = tempdir().unwrap();\n\n let file_path = temp_dir.path().join(\"a.txt\");\n\n fs::write(&file_path, b\"Hello File!\").await.unwrap();\n\n assert!(fs::try_exists(&file_path).await.unwrap());\n\n fs::remove_file(&file_path).await.unwrap();\n\n // should no longer exist\n match fs::try_exists(file_path).await {\n Ok(exists) => assert!(!exists),\n Err(_) => println!(\"ignored try_exists error after remove_file\"),\n };\n}\n" }, { "path": "tokio/tests/fs_rename.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn rename_file() {\n let temp_dir = tempdir().unwrap();\n\n let file_path = temp_dir.path().join(\"a.txt\");\n\n fs::write(&file_path, b\"Hello File!\").await.unwrap();\n\n assert!(fs::try_exists(&file_path).await.unwrap());\n\n let new_file_path = temp_dir.path().join(\"b.txt\");\n\n fs::rename(&file_path, &new_file_path).await.unwrap();\n\n assert!(fs::try_exists(new_file_path).await.unwrap());\n\n // original file should no longer exist\n match fs::try_exists(file_path).await {\n Ok(exists) => assert!(!exists),\n Err(_) => println!(\"ignored try_exists error after rename\"),\n };\n}\n" }, { "path": "tokio/tests/fs_symlink_dir_windows.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n#![cfg(windows)]\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn symlink_file_windows() {\n const FILE_NAME: &str = \"abc.txt\";\n\n let temp_dir = tempdir().unwrap();\n\n let dir1 = temp_dir.path().join(\"a\");\n fs::create_dir(&dir1).await.unwrap();\n\n let file1 = dir1.as_path().join(FILE_NAME);\n fs::write(&file1, b\"Hello File!\").await.unwrap();\n\n let dir2 = temp_dir.path().join(\"b\");\n fs::symlink_dir(&dir1, &dir2).await.unwrap();\n\n fs::write(&file1, b\"new data!\").await.unwrap();\n\n let file2 = dir2.as_path().join(FILE_NAME);\n\n let from = fs::read(&file1).await.unwrap();\n let to = fs::read(&file2).await.unwrap();\n\n assert_eq!(from, to);\n}\n" }, { "path": "tokio/tests/fs_symlink_file_windows.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n#![cfg(windows)]\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn symlink_file_windows() {\n let dir = tempdir().unwrap();\n\n let source_path = dir.path().join(\"foo.txt\");\n let dest_path = dir.path().join(\"bar.txt\");\n\n fs::write(&source_path, b\"Hello File!\").await.unwrap();\n fs::symlink_file(&source_path, &dest_path).await.unwrap();\n\n fs::write(&source_path, b\"new data!\").await.unwrap();\n\n let from = fs::read(&source_path).await.unwrap();\n let to = fs::read(&dest_path).await.unwrap();\n\n assert_eq!(from, to);\n}\n" }, { "path": "tokio/tests/fs_try_exists.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `chmod` in miri.\nasync fn try_exists() {\n let dir = tempdir().unwrap();\n\n let existing_path = dir.path().join(\"foo.txt\");\n fs::write(&existing_path, b\"Hello File!\").await.unwrap();\n let nonexisting_path = dir.path().join(\"bar.txt\");\n\n assert!(fs::try_exists(existing_path).await.unwrap());\n assert!(!fs::try_exists(nonexisting_path).await.unwrap());\n // FreeBSD root user always has permission to stat.\n #[cfg(all(\n unix,\n not(any(target_os = \"freebsd\", all(tokio_unstable, feature = \"io-uring\")))\n ))]\n {\n use std::os::unix::prelude::PermissionsExt;\n let permission_denied_directory_path = dir.path().join(\"baz\");\n fs::create_dir(&permission_denied_directory_path)\n .await\n .unwrap();\n let permission_denied_file_path = permission_denied_directory_path.join(\"baz.txt\");\n fs::write(&permission_denied_file_path, b\"Hello File!\")\n .await\n .unwrap();\n let mut perms = tokio::fs::metadata(&permission_denied_directory_path)\n .await\n .unwrap()\n .permissions();\n\n perms.set_mode(0o244);\n fs::set_permissions(&permission_denied_directory_path, perms)\n .await\n .unwrap();\n let permission_denied_result = fs::try_exists(permission_denied_file_path).await;\n assert_eq!(\n permission_denied_result.err().unwrap().kind(),\n std::io::ErrorKind::PermissionDenied\n );\n }\n}\n" }, { "path": "tokio/tests/fs_uring.rs", "content": "//! Uring file operations tests.\n\n#![cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n))]\n\nuse futures::future::FutureExt;\nuse std::sync::mpsc;\nuse std::task::Poll;\nuse std::time::Duration;\nuse std::{future::poll_fn, path::PathBuf};\nuse tempfile::NamedTempFile;\nuse tokio::{\n fs::OpenOptions,\n runtime::{Builder, Runtime},\n};\nuse tokio_util::task::TaskTracker;\n\nfn multi_rt(n: usize) -> Box Runtime> {\n Box::new(move || {\n Builder::new_multi_thread()\n .worker_threads(n)\n .enable_all()\n .build()\n .unwrap()\n })\n}\n\nfn current_rt() -> Box Runtime> {\n Box::new(|| Builder::new_current_thread().enable_all().build().unwrap())\n}\n\nfn rt_combinations() -> Vec Runtime>> {\n vec![\n current_rt(),\n multi_rt(1),\n multi_rt(2),\n multi_rt(8),\n multi_rt(64),\n multi_rt(256),\n ]\n}\n\n#[test]\nfn shutdown_runtime_while_performing_io_uring_ops() {\n fn run(rt: Runtime) {\n let (tx, rx) = mpsc::channel();\n let (done_tx, done_rx) = mpsc::channel();\n\n let (_tmp, path) = create_tmp_files(1);\n rt.spawn(async move {\n let path = path[0].clone();\n\n // spawning a bunch of uring operations.\n loop {\n let path = path.clone();\n tokio::spawn(async move {\n let mut opt = OpenOptions::new();\n opt.read(true);\n opt.open(&path).await.unwrap();\n });\n\n // Avoid busy looping.\n tokio::task::yield_now().await;\n }\n });\n\n std::thread::spawn(move || {\n let rt: Runtime = rx.recv().unwrap();\n rt.shutdown_timeout(Duration::from_millis(300));\n done_tx.send(()).unwrap();\n });\n\n tx.send(rt).unwrap();\n done_rx.recv().unwrap();\n }\n\n for rt in rt_combinations() {\n run(rt());\n }\n}\n\n#[test]\nfn open_many_files() {\n fn run(rt: Runtime) {\n const NUM_FILES: usize = 512;\n\n let (_tmp_files, paths): (Vec, Vec) = create_tmp_files(NUM_FILES);\n\n rt.block_on(async move {\n let tracker = TaskTracker::new();\n\n for i in 0..10_000 {\n let path = paths.get(i % NUM_FILES).unwrap().clone();\n tracker.spawn(async move {\n let _file = OpenOptions::new().read(true).open(path).await.unwrap();\n });\n }\n tracker.close();\n tracker.wait().await;\n });\n }\n\n for rt in rt_combinations() {\n run(rt());\n }\n}\n\n#[tokio::test]\nasync fn cancel_op_future() {\n let (_tmp_file, path): (Vec, Vec) = create_tmp_files(1);\n\n let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();\n let handle = tokio::spawn(async move {\n poll_fn(|cx| {\n let opt = {\n let mut opt = tokio::fs::OpenOptions::new();\n opt.read(true);\n opt\n };\n\n let fut = opt.open(&path[0]);\n\n // If io_uring is enabled (and not falling back to the thread pool),\n // the first poll should return Pending.\n let _pending = Box::pin(fut).poll_unpin(cx);\n\n tx.send(()).unwrap();\n\n Poll::<()>::Pending\n })\n .await;\n });\n\n // Wait for the first poll\n rx.recv().await.unwrap();\n\n handle.abort();\n\n let res = handle.await.unwrap_err();\n assert!(res.is_cancelled());\n}\n\nfn create_tmp_files(num_files: usize) -> (Vec, Vec) {\n let mut files = Vec::with_capacity(num_files);\n for _ in 0..num_files {\n let tmp = NamedTempFile::new().unwrap();\n let path = tmp.path().to_path_buf();\n files.push((tmp, path));\n }\n\n files.into_iter().unzip()\n}\n" }, { "path": "tokio/tests/fs_uring_read.rs", "content": "//! Uring file operations tests.\n\n#![cfg(all(\n tokio_unstable,\n feature = \"io-uring\",\n feature = \"rt\",\n feature = \"fs\",\n target_os = \"linux\"\n))]\n\nuse futures::future::Future;\nuse std::future::poll_fn;\nuse std::io::Write;\nuse std::path::PathBuf;\nuse std::sync::mpsc;\nuse std::task::{Context, Poll, Waker};\nuse std::time::Duration;\nuse tempfile::NamedTempFile;\nuse tokio::fs::read;\nuse tokio::runtime::{Builder, Runtime};\nuse tokio_test::assert_pending;\nuse tokio_util::task::TaskTracker;\n\nfn multi_rt(n: usize) -> Box Runtime> {\n Box::new(move || {\n Builder::new_multi_thread()\n .worker_threads(n)\n .enable_all()\n .build()\n .unwrap()\n })\n}\n\nfn current_rt() -> Box Runtime> {\n Box::new(|| Builder::new_current_thread().enable_all().build().unwrap())\n}\n\nfn rt_combinations() -> Vec Runtime>> {\n vec![\n current_rt(),\n multi_rt(1),\n multi_rt(2),\n multi_rt(8),\n multi_rt(64),\n multi_rt(256),\n ]\n}\n\n#[test]\nfn shutdown_runtime_while_performing_io_uring_ops() {\n fn run(rt: Runtime) {\n let (done_tx, done_rx) = mpsc::channel();\n let (_tmp, path) = create_tmp_files(1);\n // keep 100 permits\n const N: i32 = 100;\n rt.spawn(async move {\n let path = path[0].clone();\n\n // spawning a bunch of uring operations.\n let mut futs = vec![];\n\n // spawning a bunch of uring operations.\n for _ in 0..N {\n let path = path.clone();\n let mut fut = Box::pin(read(path));\n\n poll_fn(|cx| {\n assert_pending!(fut.as_mut().poll(cx));\n Poll::<()>::Pending\n })\n .await;\n\n futs.push(fut);\n }\n\n tokio::task::yield_now().await;\n });\n\n std::thread::spawn(move || {\n rt.shutdown_timeout(Duration::from_millis(300));\n done_tx.send(()).unwrap();\n });\n\n done_rx.recv().unwrap();\n }\n\n for rt in rt_combinations() {\n run(rt());\n }\n}\n\n#[test]\nfn read_many_files() {\n fn run(rt: Runtime) {\n const NUM_FILES: usize = 512;\n\n let (_tmp_files, paths): (Vec, Vec) = create_tmp_files(NUM_FILES);\n\n rt.block_on(async move {\n let tracker = TaskTracker::new();\n\n for i in 0..10_000 {\n let path = paths.get(i % NUM_FILES).unwrap().clone();\n tracker.spawn(async move {\n let bytes = read(path).await.unwrap();\n assert_eq!(bytes, vec![20; 1023]);\n });\n }\n tracker.close();\n tracker.wait().await;\n });\n }\n\n for rt in rt_combinations() {\n run(rt());\n }\n}\n\n#[tokio::test]\nasync fn read_small_large_files() {\n let (_tmp, path) = create_large_temp_file();\n\n let bytes = read(path).await.unwrap();\n\n assert_eq!(bytes, create_buf(5000));\n\n let (_tmp, path) = create_small_temp_file();\n\n let bytes = read(path).await.unwrap();\n\n assert_eq!(bytes, create_buf(20));\n}\n\n#[tokio::test]\nasync fn cancel_op_future() {\n let (_tmp_file, path): (Vec, Vec) = create_tmp_files(1);\n let path = path[0].clone();\n\n let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();\n\n let handle = tokio::spawn(async move {\n let fut = read(path.clone());\n tokio::pin!(fut);\n\n poll_fn(move |_| {\n // If io_uring is enabled (and not falling back to the thread pool),\n // the first poll should return Pending.\n assert_pending!(fut.as_mut().poll(&mut Context::from_waker(Waker::noop())));\n tx.send(true).unwrap();\n\n Poll::<()>::Pending\n })\n .await;\n });\n\n // Wait for the first poll\n\n let val = rx.recv().await;\n assert!(val.unwrap());\n\n handle.abort();\n\n let res = handle.await.unwrap_err();\n assert!(res.is_cancelled());\n}\n\nfn create_tmp_files(num_files: usize) -> (Vec, Vec) {\n let mut files = Vec::with_capacity(num_files);\n for _ in 0..num_files {\n let mut tmp = NamedTempFile::new().unwrap();\n let buf = vec![20; 1023];\n tmp.write_all(&buf).unwrap();\n let path = tmp.path().to_path_buf();\n files.push((tmp, path));\n }\n\n files.into_iter().unzip()\n}\n\nfn create_large_temp_file() -> (NamedTempFile, PathBuf) {\n let mut tmp = NamedTempFile::new().unwrap();\n let buf = create_buf(5000);\n\n tmp.write_all(&buf).unwrap();\n let path = tmp.path().to_path_buf();\n\n (tmp, path)\n}\n\nfn create_small_temp_file() -> (NamedTempFile, PathBuf) {\n let mut tmp = NamedTempFile::new().unwrap();\n let buf = create_buf(20);\n\n tmp.write_all(&buf).unwrap();\n let path = tmp.path().to_path_buf();\n\n (tmp, path)\n}\n\nfn create_buf(length: usize) -> Vec {\n (0..length).map(|i| i as u8).collect()\n}\n" }, { "path": "tokio/tests/fs_write.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // WASI does not support all fs operations\n\nuse tempfile::tempdir;\nuse tokio::fs;\n\n#[tokio::test]\nasync fn write() {\n let dir = tempdir().unwrap();\n let path = dir.path().join(\"test.txt\");\n\n fs::write(&path, \"Hello, World!\").await.unwrap();\n\n let contents = fs::read_to_string(&path).await.unwrap();\n assert_eq!(contents, \"Hello, World!\");\n}\n" }, { "path": "tokio/tests/io_async_fd.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(unix, feature = \"full\"))]\n\nuse std::os::unix::io::{AsRawFd, RawFd};\nuse std::sync::{\n atomic::{AtomicBool, Ordering},\n Arc,\n};\nuse std::time::Duration;\nuse std::{\n future::Future,\n io::{self, ErrorKind, Read, Write},\n task::{Context, Waker},\n};\n\nuse nix::unistd::{read, write};\n\nuse futures::poll;\n\nuse tokio::io::unix::{AsyncFd, AsyncFdReadyGuard};\nuse tokio::io::Interest;\nuse tokio_test::{assert_err, assert_pending};\n\nstruct TestWaker {\n inner: Arc,\n waker: Waker,\n}\n\n#[derive(Default)]\nstruct TestWakerInner {\n awoken: AtomicBool,\n}\n\nimpl futures::task::ArcWake for TestWakerInner {\n fn wake_by_ref(arc_self: &Arc) {\n arc_self.awoken.store(true, Ordering::SeqCst);\n }\n}\n\nimpl TestWaker {\n fn new() -> Self {\n let inner: Arc = Default::default();\n\n Self {\n inner: inner.clone(),\n waker: futures::task::waker(inner),\n }\n }\n\n fn awoken(&self) -> bool {\n self.inner.awoken.swap(false, Ordering::SeqCst)\n }\n\n fn context(&self) -> Context<'_> {\n Context::from_waker(&self.waker)\n }\n}\n\n#[derive(Debug)]\nstruct FileDescriptor {\n fd: std::os::fd::OwnedFd,\n}\n\nimpl AsRawFd for FileDescriptor {\n fn as_raw_fd(&self) -> RawFd {\n self.fd.as_raw_fd()\n }\n}\n\nimpl Read for &FileDescriptor {\n fn read(&mut self, buf: &mut [u8]) -> io::Result {\n read(self.fd.as_raw_fd(), buf).map_err(io::Error::from)\n }\n}\n\nimpl Read for FileDescriptor {\n fn read(&mut self, buf: &mut [u8]) -> io::Result {\n (self as &Self).read(buf)\n }\n}\n\nimpl Write for &FileDescriptor {\n fn write(&mut self, buf: &[u8]) -> io::Result {\n write(&self.fd, buf).map_err(io::Error::from)\n }\n\n fn flush(&mut self) -> io::Result<()> {\n Ok(())\n }\n}\n\nimpl Write for FileDescriptor {\n fn write(&mut self, buf: &[u8]) -> io::Result {\n (self as &Self).write(buf)\n }\n\n fn flush(&mut self) -> io::Result<()> {\n (self as &Self).flush()\n }\n}\n\nfn set_nonblocking(fd: RawFd) {\n use nix::fcntl::{OFlag, F_GETFL, F_SETFL};\n\n let flags = nix::fcntl::fcntl(fd, F_GETFL).expect(\"fcntl(F_GETFD)\");\n\n if flags < 0 {\n panic!(\n \"bad return value from fcntl(F_GETFL): {} ({:?})\",\n flags,\n nix::Error::last()\n );\n }\n\n let flags = OFlag::from_bits_truncate(flags) | OFlag::O_NONBLOCK;\n\n nix::fcntl::fcntl(fd, F_SETFL(flags)).expect(\"fcntl(F_SETFD)\");\n}\n\nfn socketpair() -> (FileDescriptor, FileDescriptor) {\n use nix::sys::socket::{self, AddressFamily, SockFlag, SockType};\n\n let (fd_a, fd_b) = socket::socketpair(\n AddressFamily::Unix,\n SockType::Stream,\n None,\n SockFlag::empty(),\n )\n .expect(\"socketpair\");\n let fds = (FileDescriptor { fd: fd_a }, FileDescriptor { fd: fd_b });\n\n set_nonblocking(fds.0.fd.as_raw_fd());\n set_nonblocking(fds.1.fd.as_raw_fd());\n\n fds\n}\n\nfn drain(mut fd: &FileDescriptor, mut amt: usize) {\n let mut buf = [0u8; 512];\n while amt > 0 {\n match fd.read(&mut buf[..]) {\n Err(e) if e.kind() == ErrorKind::WouldBlock => {}\n Ok(0) => panic!(\"unexpected EOF\"),\n Err(e) => panic!(\"unexpected error: {e:?}\"),\n Ok(x) => amt -= x,\n }\n }\n}\n\n#[tokio::test]\nasync fn initially_writable() {\n let (a, b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n let afd_b = AsyncFd::new(b).unwrap();\n\n afd_a.writable().await.unwrap().clear_ready();\n afd_b.writable().await.unwrap().clear_ready();\n\n tokio::select! {\n biased;\n _ = tokio::time::sleep(Duration::from_millis(10)) => {},\n _ = afd_a.readable() => panic!(\"Unexpected readable state\"),\n _ = afd_b.readable() => panic!(\"Unexpected readable state\"),\n }\n}\n\n#[tokio::test]\nasync fn reset_readable() {\n let (a, mut b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n\n let readable = afd_a.readable();\n tokio::pin!(readable);\n\n tokio::select! {\n _ = readable.as_mut() => panic!(),\n _ = tokio::time::sleep(Duration::from_millis(10)) => {}\n }\n\n b.write_all(b\"0\").unwrap();\n\n let mut guard = readable.await.unwrap();\n\n guard\n .try_io(|_| afd_a.get_ref().read(&mut [0]))\n .unwrap()\n .unwrap();\n\n // `a` is not readable, but the reactor still thinks it is\n // (because we have not observed a not-ready error yet)\n afd_a.readable().await.unwrap().retain_ready();\n\n // Explicitly clear the ready state\n guard.clear_ready();\n\n let readable = afd_a.readable();\n tokio::pin!(readable);\n\n tokio::select! {\n _ = readable.as_mut() => panic!(),\n _ = tokio::time::sleep(Duration::from_millis(10)) => {}\n }\n\n b.write_all(b\"0\").unwrap();\n\n // We can observe the new readable event\n afd_a.readable().await.unwrap().clear_ready();\n}\n\n#[tokio::test]\nasync fn reset_writable() {\n let (a, b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n\n let mut guard = afd_a.writable().await.unwrap();\n\n // Write until we get a WouldBlock. This also clears the ready state.\n let mut bytes = 0;\n while let Ok(Ok(amt)) = guard.try_io(|_| afd_a.get_ref().write(&[0; 512][..])) {\n bytes += amt;\n }\n\n // Writable state should be cleared now.\n let writable = afd_a.writable();\n tokio::pin!(writable);\n\n tokio::select! {\n _ = writable.as_mut() => panic!(),\n _ = tokio::time::sleep(Duration::from_millis(10)) => {}\n }\n\n // Read from the other side; we should become writable now.\n drain(&b, bytes);\n\n let _ = writable.await.unwrap();\n}\n\n#[derive(Debug)]\nstruct ArcFd(Arc);\nimpl AsRawFd for ArcFd {\n fn as_raw_fd(&self) -> RawFd {\n self.0.as_raw_fd()\n }\n}\n\n#[tokio::test]\nasync fn drop_closes() {\n let (a, mut b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n\n assert_eq!(\n ErrorKind::WouldBlock,\n b.read(&mut [0]).err().unwrap().kind()\n );\n\n std::mem::drop(afd_a);\n\n assert_eq!(0, b.read(&mut [0]).unwrap());\n\n // into_inner does not close the fd\n\n let (a, mut b) = socketpair();\n let afd_a = AsyncFd::new(a).unwrap();\n let _a: FileDescriptor = afd_a.into_inner();\n\n assert_eq!(\n ErrorKind::WouldBlock,\n b.read(&mut [0]).err().unwrap().kind()\n );\n\n // Drop closure behavior is delegated to the inner object\n let (a, mut b) = socketpair();\n let arc_fd = Arc::new(a);\n let afd_a = AsyncFd::new(ArcFd(arc_fd.clone())).unwrap();\n std::mem::drop(afd_a);\n\n assert_eq!(\n ErrorKind::WouldBlock,\n b.read(&mut [0]).err().unwrap().kind()\n );\n\n std::mem::drop(arc_fd); // suppress unnecessary clone clippy warning\n}\n\n#[tokio::test]\nasync fn reregister() {\n let (a, _b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n let a = afd_a.into_inner();\n AsyncFd::new(a).unwrap();\n}\n\n#[tokio::test]\nasync fn guard_try_io() {\n let (a, mut b) = socketpair();\n\n b.write_all(b\"0\").unwrap();\n\n let afd_a = AsyncFd::new(a).unwrap();\n\n let mut guard = afd_a.readable().await.unwrap();\n\n afd_a.get_ref().read_exact(&mut [0]).unwrap();\n\n // Should not clear the readable state\n let _ = guard.try_io(|_| Ok(()));\n\n // Still readable...\n let _ = afd_a.readable().await.unwrap();\n\n // Should clear the readable state\n let _ = guard.try_io(|_| io::Result::<()>::Err(ErrorKind::WouldBlock.into()));\n\n // Assert not readable\n let readable = afd_a.readable();\n tokio::pin!(readable);\n\n tokio::select! {\n _ = readable.as_mut() => panic!(),\n _ = tokio::time::sleep(Duration::from_millis(10)) => {}\n }\n\n // Write something down b again and make sure we're reawoken\n b.write_all(b\"0\").unwrap();\n let _ = readable.await.unwrap();\n}\n\n#[tokio::test]\nasync fn try_io_readable() {\n let (a, mut b) = socketpair();\n let mut afd_a = AsyncFd::new(a).unwrap();\n\n // Give the runtime some time to update bookkeeping.\n tokio::task::yield_now().await;\n\n {\n let mut called = false;\n let _ = afd_a.try_io_mut(Interest::READABLE, |_| {\n called = true;\n Ok(())\n });\n assert!(\n !called,\n \"closure should not have been called, since socket should not be readable\"\n );\n }\n\n // Make `a` readable by writing to `b`.\n // Give the runtime some time to update bookkeeping.\n b.write_all(&[0]).unwrap();\n tokio::task::yield_now().await;\n\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::READABLE, |_| {\n called = true;\n Ok(())\n });\n assert!(\n called,\n \"closure should have been called, since socket should have data available to read\"\n );\n }\n\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::READABLE, |_| {\n called = true;\n io::Result::<()>::Err(ErrorKind::WouldBlock.into())\n });\n assert!(\n called,\n \"closure should have been called, since socket should have data available to read\"\n );\n }\n\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::READABLE, |_| {\n called = true;\n Ok(())\n });\n assert!(!called, \"closure should not have been called, since socket readable state should have been cleared\");\n }\n}\n\n#[tokio::test]\nasync fn try_io_writable() {\n let (a, _b) = socketpair();\n let afd_a = AsyncFd::new(a).unwrap();\n\n // Give the runtime some time to update bookkeeping.\n tokio::task::yield_now().await;\n\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::WRITABLE, |_| {\n called = true;\n Ok(())\n });\n assert!(\n called,\n \"closure should have been called, since socket should still be marked as writable\"\n );\n }\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::WRITABLE, |_| {\n called = true;\n io::Result::<()>::Err(ErrorKind::WouldBlock.into())\n });\n assert!(\n called,\n \"closure should have been called, since socket should still be marked as writable\"\n );\n }\n\n {\n let mut called = false;\n let _ = afd_a.try_io(Interest::WRITABLE, |_| {\n called = true;\n Ok(())\n });\n assert!(!called, \"closure should not have been called, since socket writable state should have been cleared\");\n }\n}\n\n#[tokio::test]\nasync fn multiple_waiters() {\n let (a, mut b) = socketpair();\n let afd_a = Arc::new(AsyncFd::new(a).unwrap());\n\n let barrier = Arc::new(tokio::sync::Barrier::new(11));\n\n let mut tasks = Vec::new();\n for _ in 0..10 {\n let afd_a = afd_a.clone();\n let barrier = barrier.clone();\n\n let f = async move {\n let notify_barrier = async {\n barrier.wait().await;\n futures::future::pending::<()>().await;\n };\n\n tokio::select! {\n biased;\n guard = afd_a.readable() => {\n tokio::task::yield_now().await;\n guard.unwrap().clear_ready()\n },\n _ = notify_barrier => unreachable!(),\n }\n\n std::mem::drop(afd_a);\n };\n\n tasks.push(tokio::spawn(f));\n }\n\n let mut all_tasks = futures::future::try_join_all(tasks);\n\n tokio::select! {\n r = std::pin::Pin::new(&mut all_tasks) => {\n r.unwrap(); // propagate panic\n panic!(\"Tasks exited unexpectedly\")\n },\n _ = barrier.wait() => {}\n }\n\n b.write_all(b\"0\").unwrap();\n\n all_tasks.await.unwrap();\n}\n\n#[tokio::test]\nasync fn poll_fns() {\n let (a, b) = socketpair();\n let afd_a = Arc::new(AsyncFd::new(a).unwrap());\n let afd_b = Arc::new(AsyncFd::new(b).unwrap());\n\n // Fill up the write side of A\n let mut bytes = 0;\n while let Ok(amt) = afd_a.get_ref().write(&[0; 512]) {\n bytes += amt;\n }\n\n let waker = TestWaker::new();\n\n assert_pending!(afd_a.as_ref().poll_read_ready(&mut waker.context()));\n\n let afd_a_2 = afd_a.clone();\n let r_barrier = Arc::new(tokio::sync::Barrier::new(2));\n let barrier_clone = r_barrier.clone();\n\n let read_fut = tokio::spawn(async move {\n // Move waker onto this task first\n assert_pending!(poll!(std::future::poll_fn(|cx| afd_a_2\n .as_ref()\n .poll_read_ready(cx))));\n barrier_clone.wait().await;\n\n let _ = std::future::poll_fn(|cx| afd_a_2.as_ref().poll_read_ready(cx)).await;\n });\n\n let afd_a_2 = afd_a.clone();\n let w_barrier = Arc::new(tokio::sync::Barrier::new(2));\n let barrier_clone = w_barrier.clone();\n\n let mut write_fut = tokio::spawn(async move {\n // Move waker onto this task first\n assert_pending!(poll!(std::future::poll_fn(|cx| afd_a_2\n .as_ref()\n .poll_write_ready(cx))));\n barrier_clone.wait().await;\n\n let _ = std::future::poll_fn(|cx| afd_a_2.as_ref().poll_write_ready(cx)).await;\n });\n\n r_barrier.wait().await;\n w_barrier.wait().await;\n\n let readable = afd_a.readable();\n tokio::pin!(readable);\n\n tokio::select! {\n _ = &mut readable => unreachable!(),\n _ = tokio::task::yield_now() => {}\n }\n\n // Make A readable. We expect that 'readable' and 'read_fut' will both complete quickly\n afd_b.get_ref().write_all(b\"0\").unwrap();\n\n let _ = tokio::join!(readable, read_fut);\n\n // Our original waker should _not_ be awoken (poll_read_ready retains only the last context)\n assert!(!waker.awoken());\n\n // The writable side should not be awoken\n tokio::select! {\n _ = &mut write_fut => unreachable!(),\n _ = tokio::time::sleep(Duration::from_millis(5)) => {}\n }\n\n // Make it writable now\n drain(afd_b.get_ref(), bytes);\n\n // now we should be writable (ie - the waker for poll_write should still be registered after we wake the read side)\n let _ = write_fut.await;\n}\n\nfn assert_pending>(f: F) -> std::pin::Pin> {\n let mut pinned = Box::pin(f);\n\n assert_pending!(pinned\n .as_mut()\n .poll(&mut Context::from_waker(futures::task::noop_waker_ref())));\n\n pinned\n}\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\n#[test]\nfn driver_shutdown_wakes_currently_pending() {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n let readable = assert_pending(afd_a.readable());\n\n std::mem::drop(rt);\n\n // The future was initialized **before** dropping the rt\n assert_err!(futures::executor::block_on(readable));\n\n // The future is initialized **after** dropping the rt.\n assert_err!(futures::executor::block_on(afd_a.readable()));\n}\n\n#[test]\nfn driver_shutdown_wakes_future_pending() {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n std::mem::drop(rt);\n\n assert_err!(futures::executor::block_on(afd_a.readable()));\n}\n\n#[test]\nfn driver_shutdown_wakes_pending_race() {\n // TODO: make this a loom test\n for _ in 0..100 {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n let _ = std::thread::spawn(move || std::mem::drop(rt));\n\n // This may or may not return an error (but will be awoken)\n let _ = futures::executor::block_on(afd_a.readable());\n\n // However retrying will always return an error\n assert_err!(futures::executor::block_on(afd_a.readable()));\n }\n}\n\nasync fn poll_readable(fd: &AsyncFd) -> std::io::Result> {\n std::future::poll_fn(|cx| fd.poll_read_ready(cx)).await\n}\n\nasync fn poll_writable(fd: &AsyncFd) -> std::io::Result> {\n std::future::poll_fn(|cx| fd.poll_write_ready(cx)).await\n}\n\n#[test]\nfn driver_shutdown_wakes_currently_pending_polls() {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n while afd_a.get_ref().write(&[0; 512]).is_ok() {} // make not writable\n\n let readable = assert_pending(poll_readable(&afd_a));\n let writable = assert_pending(poll_writable(&afd_a));\n\n std::mem::drop(rt);\n\n // Attempting to poll readiness when the rt is dropped is an error\n assert_err!(futures::executor::block_on(readable));\n assert_err!(futures::executor::block_on(writable));\n}\n\n#[test]\nfn driver_shutdown_wakes_poll() {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n std::mem::drop(rt);\n\n assert_err!(futures::executor::block_on(poll_readable(&afd_a)));\n assert_err!(futures::executor::block_on(poll_writable(&afd_a)));\n}\n\n#[test]\nfn driver_shutdown_then_clear_readiness() {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n let mut write_ready = rt.block_on(afd_a.writable()).unwrap();\n\n std::mem::drop(rt);\n\n write_ready.clear_ready();\n}\n\n#[test]\nfn driver_shutdown_wakes_poll_race() {\n // TODO: make this a loom test\n for _ in 0..100 {\n let rt = rt();\n\n let (a, _b) = socketpair();\n let afd_a = {\n let _enter = rt.enter();\n AsyncFd::new(a).unwrap()\n };\n\n while afd_a.get_ref().write(&[0; 512]).is_ok() {} // make not writable\n\n let _ = std::thread::spawn(move || std::mem::drop(rt));\n\n // The poll variants will always return an error in this case\n assert_err!(futures::executor::block_on(poll_readable(&afd_a)));\n assert_err!(futures::executor::block_on(poll_writable(&afd_a)));\n }\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No socket in miri.\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nasync fn priority_event_on_oob_data() {\n use std::net::SocketAddr;\n\n use tokio::io::Interest;\n\n let addr: SocketAddr = \"127.0.0.1:0\".parse().unwrap();\n\n let listener = std::net::TcpListener::bind(addr).unwrap();\n let addr = listener.local_addr().unwrap();\n\n let client = std::net::TcpStream::connect(addr).unwrap();\n let client = AsyncFd::with_interest(client, Interest::PRIORITY).unwrap();\n\n let (stream, _) = listener.accept().unwrap();\n\n // Sending out of band data should trigger priority event.\n send_oob_data(&stream, b\"hello\").unwrap();\n\n let _ = client.ready(Interest::PRIORITY).await.unwrap();\n}\n\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nfn send_oob_data(stream: &S, data: &[u8]) -> io::Result {\n unsafe {\n let res = libc::send(\n stream.as_raw_fd(),\n data.as_ptr().cast(),\n data.len(),\n libc::MSG_OOB,\n );\n if res == -1 {\n Err(io::Error::last_os_error())\n } else {\n Ok(res as usize)\n }\n }\n}\n\n#[tokio::test]\nasync fn clear_ready_matching_clears_ready() {\n use tokio::io::{Interest, Ready};\n\n let (a, mut b) = socketpair();\n\n let afd_a = AsyncFd::new(a).unwrap();\n b.write_all(b\"0\").unwrap();\n\n let mut guard = afd_a\n .ready(Interest::READABLE | Interest::WRITABLE)\n .await\n .unwrap();\n\n assert_eq!(guard.ready(), Ready::READABLE | Ready::WRITABLE);\n\n guard.clear_ready_matching(Ready::READABLE);\n assert_eq!(guard.ready(), Ready::WRITABLE);\n\n guard.clear_ready_matching(Ready::WRITABLE);\n assert_eq!(guard.ready(), Ready::EMPTY);\n}\n\n#[tokio::test]\nasync fn clear_ready_matching_clears_ready_mut() {\n use tokio::io::{Interest, Ready};\n\n let (a, mut b) = socketpair();\n\n let mut afd_a = AsyncFd::new(a).unwrap();\n b.write_all(b\"0\").unwrap();\n\n let mut guard = afd_a\n .ready_mut(Interest::READABLE | Interest::WRITABLE)\n .await\n .unwrap();\n\n assert_eq!(guard.ready(), Ready::READABLE | Ready::WRITABLE);\n\n guard.clear_ready_matching(Ready::READABLE);\n assert_eq!(guard.ready(), Ready::WRITABLE);\n\n guard.clear_ready_matching(Ready::WRITABLE);\n assert_eq!(guard.ready(), Ready::EMPTY);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No socket in miri.\n#[cfg(target_os = \"linux\")]\nasync fn await_error_readiness_timestamping() {\n use std::net::{Ipv4Addr, SocketAddr};\n\n use tokio::io::{Interest, Ready};\n\n let address_a = SocketAddr::from((Ipv4Addr::LOCALHOST, 0));\n let address_b = SocketAddr::from((Ipv4Addr::LOCALHOST, 0));\n\n let socket = std::net::UdpSocket::bind(address_a).unwrap();\n\n socket.set_nonblocking(true).unwrap();\n\n // configure send timestamps\n configure_timestamping_socket(&socket).unwrap();\n\n socket.connect(address_b).unwrap();\n\n let fd = AsyncFd::new(socket).unwrap();\n\n tokio::select! {\n _ = fd.ready(Interest::ERROR) => panic!(),\n _ = tokio::time::sleep(Duration::from_millis(10)) => {}\n }\n\n let buf = b\"hello there\";\n fd.get_ref().send(buf).unwrap();\n\n // the send timestamp should now be in the error queue\n let guard = fd.ready(Interest::ERROR).await.unwrap();\n assert_eq!(guard.ready(), Ready::ERROR);\n}\n\n#[cfg(target_os = \"linux\")]\nfn configure_timestamping_socket(udp_socket: &std::net::UdpSocket) -> std::io::Result {\n // enable software timestamping, and specifically software send timestamping\n let options = libc::SOF_TIMESTAMPING_SOFTWARE | libc::SOF_TIMESTAMPING_TX_SOFTWARE;\n\n let res = unsafe {\n libc::setsockopt(\n udp_socket.as_raw_fd(),\n libc::SOL_SOCKET,\n libc::SO_TIMESTAMP,\n &options as *const _ as *const libc::c_void,\n std::mem::size_of_val(&options) as libc::socklen_t,\n )\n };\n\n if res == -1 {\n Err(std::io::Error::last_os_error())\n } else {\n Ok(res)\n }\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No socket in miri.\n#[cfg(target_os = \"linux\")]\nasync fn await_error_readiness_invalid_address() {\n use std::net::{Ipv4Addr, SocketAddr};\n use tokio::io::{Interest, Ready};\n\n let socket_addr = SocketAddr::from((Ipv4Addr::LOCALHOST, 0));\n let socket = std::net::UdpSocket::bind(socket_addr).unwrap();\n let socket_fd = socket.as_raw_fd();\n\n // Enable IP_RECVERR option to receive error messages\n // https://man7.org/linux/man-pages/man7/ip.7.html has some extra information\n let recv_err: libc::c_int = 1;\n unsafe {\n let res = libc::setsockopt(\n socket.as_raw_fd(),\n libc::SOL_IP,\n libc::IP_RECVERR,\n &recv_err as *const _ as *const libc::c_void,\n std::mem::size_of_val(&recv_err) as libc::socklen_t,\n );\n if res == -1 {\n panic!(\"{:?}\", std::io::Error::last_os_error());\n }\n }\n\n // Spawn a separate thread for sending messages\n tokio::spawn(async move {\n // Set the destination address. This address is invalid in this context. the OS will notice\n // that nobody is listening on port this port. Normally this is ignored (UDP is \"fire and forget\"),\n // but because IP_RECVERR is enabled, the error will actually be reported to the sending socket\n let mut dest_addr =\n unsafe { std::mem::MaybeUninit::::zeroed().assume_init() };\n dest_addr.sin_family = libc::AF_INET as _;\n // based on https://en.wikipedia.org/wiki/Ephemeral_port, we should pick a port number\n // below 1024 to guarantee that other tests don't select this port by accident when they\n // use port 0 to select an ephemeral port.\n dest_addr.sin_port = 512u16.to_be(); // Destination port\n\n // Prepare the message data\n let message = \"Hello, Socket!\";\n\n // Prepare the message structure for sendmsg\n let mut iov = libc::iovec {\n iov_base: message.as_ptr() as *mut libc::c_void,\n iov_len: message.len(),\n };\n\n // Prepare the destination address for the sendmsg call\n let dest_sockaddr: *const libc::sockaddr = &dest_addr as *const _ as *const libc::sockaddr;\n let dest_addrlen: libc::socklen_t = std::mem::size_of_val(&dest_addr) as libc::socklen_t;\n\n let mut msg: libc::msghdr = unsafe { std::mem::MaybeUninit::zeroed().assume_init() };\n msg.msg_name = dest_sockaddr as *mut libc::c_void;\n msg.msg_namelen = dest_addrlen;\n msg.msg_iov = &mut iov;\n msg.msg_iovlen = 1;\n\n if unsafe { libc::sendmsg(socket_fd, &msg, 0) } == -1 {\n panic!(\"{:?}\", std::io::Error::last_os_error())\n }\n });\n\n let fd = AsyncFd::new(socket).unwrap();\n\n let guard = fd.ready(Interest::ERROR).await.unwrap();\n assert_eq!(guard.ready(), Ready::ERROR);\n}\n\n#[derive(Debug, PartialEq, Eq)]\nstruct InvalidSource;\n\nimpl AsRawFd for InvalidSource {\n fn as_raw_fd(&self) -> RawFd {\n -1\n }\n}\n\n#[tokio::test]\nasync fn try_new() {\n let original = Arc::new(InvalidSource);\n\n let error = AsyncFd::try_new(original.clone()).unwrap_err();\n let (returned, _cause) = error.into_parts();\n\n assert!(Arc::ptr_eq(&original, &returned));\n}\n\n#[tokio::test]\nasync fn try_with_interest() {\n let original = Arc::new(InvalidSource);\n\n let error = AsyncFd::try_with_interest(original.clone(), Interest::READABLE).unwrap_err();\n let (returned, _cause) = error.into_parts();\n\n assert!(Arc::ptr_eq(&original, &returned));\n}\n" }, { "path": "tokio/tests/io_async_fd_memory_leak.rs", "content": "//! Regression test for issue #7563 - Memory leak when fd closed before AsyncFd drop\n//!\n//! This test uses a custom global allocator to track actual memory usage,\n//! avoiding false positives from RSS measurements which include freed-but-retained memory.\n\n#![cfg(all(unix, target_os = \"linux\", feature = \"full\"))]\n\nuse std::alloc::{GlobalAlloc, Layout, System};\nuse std::sync::atomic::{AtomicUsize, Ordering};\n\n/// A tracking allocator that counts bytes currently allocated\nstruct TrackingAllocator {\n allocated: AtomicUsize,\n}\n\nimpl TrackingAllocator {\n const fn new() -> Self {\n Self {\n allocated: AtomicUsize::new(0),\n }\n }\n}\n\nunsafe impl GlobalAlloc for TrackingAllocator {\n unsafe fn alloc(&self, layout: Layout) -> *mut u8 {\n let ptr = unsafe { System.alloc(layout) };\n if !ptr.is_null() {\n self.allocated.fetch_add(layout.size(), Ordering::Relaxed);\n }\n ptr\n }\n\n unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {\n unsafe { System.dealloc(ptr, layout) };\n self.allocated.fetch_sub(layout.size(), Ordering::Relaxed);\n }\n\n unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {\n let new_ptr = unsafe { System.realloc(ptr, layout, new_size) };\n if !new_ptr.is_null() {\n // Subtract old size, add new size\n if new_size > layout.size() {\n self.allocated\n .fetch_add(new_size - layout.size(), Ordering::Relaxed);\n } else {\n self.allocated\n .fetch_sub(layout.size() - new_size, Ordering::Relaxed);\n }\n }\n new_ptr\n }\n}\n\n#[global_allocator]\nstatic GLOBAL: TrackingAllocator = TrackingAllocator::new();\n\nfn allocated_bytes() -> usize {\n GLOBAL.allocated.load(Ordering::Relaxed)\n}\n\n#[tokio::test]\nasync fn memory_leak_when_fd_closed_before_drop() {\n use nix::sys::socket::{self, AddressFamily, SockFlag, SockType};\n use std::os::unix::io::{AsRawFd, RawFd};\n use std::sync::Arc;\n use tokio::io::unix::AsyncFd;\n\n struct RawFdWrapper {\n fd: RawFd,\n }\n\n impl AsRawFd for RawFdWrapper {\n fn as_raw_fd(&self) -> RawFd {\n self.fd\n }\n }\n\n struct ArcFd(Arc);\n\n impl AsRawFd for ArcFd {\n fn as_raw_fd(&self) -> RawFd {\n self.0.as_raw_fd()\n }\n }\n\n fn set_nonblocking(fd: RawFd) {\n use nix::fcntl::{OFlag, F_GETFL, F_SETFL};\n\n let flags = nix::fcntl::fcntl(fd, F_GETFL).expect(\"fcntl(F_GETFL)\");\n\n if flags < 0 {\n panic!(\n \"bad return value from fcntl(F_GETFL): {} ({:?})\",\n flags,\n nix::Error::last()\n );\n }\n\n let flags = OFlag::from_bits_truncate(flags) | OFlag::O_NONBLOCK;\n\n nix::fcntl::fcntl(fd, F_SETFL(flags)).expect(\"fcntl(F_SETFL)\");\n }\n\n // Warm up - let runtime and allocator stabilize\n for _ in 0..100 {\n tokio::task::yield_now().await;\n }\n\n const ITERATIONS: usize = 1000;\n\n // Phase 1: Warm up allocations\n for _ in 0..ITERATIONS {\n let (fd_a, _fd_b) = socket::socketpair(\n AddressFamily::Unix,\n SockType::Stream,\n None,\n SockFlag::empty(),\n )\n .unwrap();\n\n let raw_fd = fd_a.as_raw_fd();\n set_nonblocking(raw_fd);\n std::mem::forget(fd_a);\n\n let wrapper = Arc::new(RawFdWrapper { fd: raw_fd });\n let async_fd = AsyncFd::new(ArcFd(wrapper)).unwrap();\n\n // Close fd before dropping AsyncFd - this triggers the bug\n unsafe {\n libc::close(raw_fd);\n }\n\n drop(async_fd);\n }\n\n // Let things settle\n tokio::task::yield_now().await;\n let baseline = allocated_bytes();\n\n // Phase 2: Run more iterations and check for growth\n for _ in 0..ITERATIONS {\n let (fd_a, _fd_b) = socket::socketpair(\n AddressFamily::Unix,\n SockType::Stream,\n None,\n SockFlag::empty(),\n )\n .unwrap();\n\n let raw_fd = fd_a.as_raw_fd();\n set_nonblocking(raw_fd);\n std::mem::forget(fd_a);\n\n let wrapper = Arc::new(RawFdWrapper { fd: raw_fd });\n let async_fd = AsyncFd::new(ArcFd(wrapper)).unwrap();\n\n unsafe {\n libc::close(raw_fd);\n }\n\n drop(async_fd);\n }\n\n tokio::task::yield_now().await;\n let after_phase2 = allocated_bytes();\n\n // Phase 3: Run even more iterations\n for _ in 0..ITERATIONS {\n let (fd_a, _fd_b) = socket::socketpair(\n AddressFamily::Unix,\n SockType::Stream,\n None,\n SockFlag::empty(),\n )\n .unwrap();\n\n let raw_fd = fd_a.as_raw_fd();\n set_nonblocking(raw_fd);\n std::mem::forget(fd_a);\n\n let wrapper = Arc::new(RawFdWrapper { fd: raw_fd });\n let async_fd = AsyncFd::new(ArcFd(wrapper)).unwrap();\n\n unsafe {\n libc::close(raw_fd);\n }\n\n drop(async_fd);\n }\n\n tokio::task::yield_now().await;\n let after_phase3 = allocated_bytes();\n\n let growth_phase2 = after_phase2.saturating_sub(baseline);\n let growth_phase3 = after_phase3.saturating_sub(after_phase2);\n\n // If there's a leak, each phase adds ~250KB (1000 * ~256 bytes per ScheduledIo)\n // If fixed, memory should stabilize (minimal growth between phases)\n // Allow 64KB tolerance for normal allocation variance\n let threshold = 64 * 1024; // 64KB\n\n assert!(\n growth_phase2 < threshold || growth_phase3 < threshold,\n \"Memory leak detected: allocations keep growing without stabilizing. \\\n Phase 1->2: +{growth_phase2} bytes, Phase 2->3: +{growth_phase3} bytes. \\\n (baseline: {baseline} bytes, phase2: {after_phase2} bytes, phase3: {after_phase3} bytes). \\\n Expected at least one phase with <{threshold} bytes growth.\",\n );\n}\n" }, { "path": "tokio/tests/io_async_read.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncRead;\n\n#[test]\nfn assert_obj_safe() {\n fn _assert() {}\n _assert::>();\n}\n" }, { "path": "tokio/tests/io_buf_reader.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\n// https://github.com/rust-lang/futures-rs/blob/1803948ff091b4eabf7f3bf39e16bbbdefca5cc8/futures/tests/io_buf_reader.rs\n\nuse futures::task::{noop_waker_ref, Context, Poll};\nuse std::cmp;\nuse std::io::{self, Cursor};\nuse std::pin::Pin;\nuse tokio::io::{\n AsyncBufRead, AsyncBufReadExt, AsyncRead, AsyncReadExt, AsyncSeek, AsyncSeekExt, AsyncWriteExt,\n BufReader, ReadBuf, SeekFrom,\n};\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\n\nmacro_rules! run_fill_buf {\n ($reader:expr) => {{\n let mut cx = Context::from_waker(noop_waker_ref());\n loop {\n if let Poll::Ready(x) = Pin::new(&mut $reader).poll_fill_buf(&mut cx) {\n break x;\n }\n }\n }};\n}\n\nstruct MaybePending<'a> {\n inner: &'a [u8],\n ready_read: bool,\n ready_fill_buf: bool,\n}\n\nimpl<'a> MaybePending<'a> {\n fn new(inner: &'a [u8]) -> Self {\n Self {\n inner,\n ready_read: false,\n ready_fill_buf: false,\n }\n }\n}\n\nimpl AsyncRead for MaybePending<'_> {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if self.ready_read {\n self.ready_read = false;\n Pin::new(&mut self.inner).poll_read(cx, buf)\n } else {\n self.ready_read = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n}\n\nimpl AsyncBufRead for MaybePending<'_> {\n fn poll_fill_buf(mut self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n if self.ready_fill_buf {\n self.ready_fill_buf = false;\n if self.inner.is_empty() {\n return Poll::Ready(Ok(&[]));\n }\n let len = cmp::min(2, self.inner.len());\n Poll::Ready(Ok(&self.inner[0..len]))\n } else {\n self.ready_fill_buf = true;\n Poll::Pending\n }\n }\n\n fn consume(mut self: Pin<&mut Self>, amt: usize) {\n self.inner = &self.inner[amt..];\n }\n}\n\n#[tokio::test]\nasync fn test_buffered_reader() {\n let inner: &[u8] = &[5, 6, 7, 0, 1, 2, 3, 4];\n let mut reader = BufReader::with_capacity(2, inner);\n\n let mut buf = [0, 0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 3);\n assert_eq!(buf, [5, 6, 7]);\n assert_eq!(reader.buffer(), []);\n\n let mut buf = [0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 2);\n assert_eq!(buf, [0, 1]);\n assert_eq!(reader.buffer(), []);\n\n let mut buf = [0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [2]);\n assert_eq!(reader.buffer(), [3]);\n\n let mut buf = [0, 0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [3, 0, 0]);\n assert_eq!(reader.buffer(), []);\n\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [4, 0, 0]);\n assert_eq!(reader.buffer(), []);\n\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n}\n\n#[tokio::test]\nasync fn test_buffered_reader_seek() {\n let inner: &[u8] = &[5, 6, 7, 0, 1, 2, 3, 4];\n let mut reader = BufReader::with_capacity(2, Cursor::new(inner));\n\n assert_eq!(reader.seek(SeekFrom::Start(3)).await.unwrap(), 3);\n assert_eq!(run_fill_buf!(reader).unwrap(), &[0, 1][..]);\n assert!(reader.seek(SeekFrom::Current(i64::MIN)).await.is_err());\n assert_eq!(run_fill_buf!(reader).unwrap(), &[0, 1][..]);\n assert_eq!(reader.seek(SeekFrom::Current(1)).await.unwrap(), 4);\n assert_eq!(run_fill_buf!(reader).unwrap(), &[1, 2][..]);\n Pin::new(&mut reader).consume(1);\n assert_eq!(reader.seek(SeekFrom::Current(-2)).await.unwrap(), 3);\n}\n\n#[tokio::test]\nasync fn test_buffered_reader_seek_underflow() {\n // gimmick reader that yields its position modulo 256 for each byte\n struct PositionReader {\n pos: u64,\n }\n impl AsyncRead for PositionReader {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let b = buf.initialize_unfilled();\n let len = b.len();\n for x in b {\n *x = self.pos as u8;\n self.pos = self.pos.wrapping_add(1);\n }\n buf.advance(len);\n Poll::Ready(Ok(()))\n }\n }\n impl AsyncSeek for PositionReader {\n fn start_seek(mut self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n match pos {\n SeekFrom::Start(n) => {\n self.pos = n;\n }\n SeekFrom::Current(n) => {\n self.pos = self.pos.wrapping_add(n as u64);\n }\n SeekFrom::End(n) => {\n self.pos = u64::MAX.wrapping_add(n as u64);\n }\n }\n Ok(())\n }\n fn poll_complete(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(self.pos))\n }\n }\n\n let mut reader = BufReader::with_capacity(5, PositionReader { pos: 0 });\n assert_eq!(run_fill_buf!(reader).unwrap(), &[0, 1, 2, 3, 4][..]);\n assert_eq!(reader.seek(SeekFrom::End(-5)).await.unwrap(), u64::MAX - 5);\n assert_eq!(run_fill_buf!(reader).unwrap().len(), 5);\n // the following seek will require two underlying seeks\n let expected = 9_223_372_036_854_775_802;\n assert_eq!(\n reader.seek(SeekFrom::Current(i64::MIN)).await.unwrap(),\n expected\n );\n assert_eq!(run_fill_buf!(reader).unwrap().len(), 5);\n // seeking to 0 should empty the buffer.\n assert_eq!(reader.seek(SeekFrom::Current(0)).await.unwrap(), expected);\n assert_eq!(reader.get_ref().pos, expected);\n}\n\n#[tokio::test]\nasync fn test_short_reads() {\n /// A dummy reader intended at testing short-reads propagation.\n struct ShortReader {\n lengths: Vec,\n }\n\n impl AsyncRead for ShortReader {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if !self.lengths.is_empty() {\n buf.advance(self.lengths.remove(0));\n }\n Poll::Ready(Ok(()))\n }\n }\n\n let inner = ShortReader {\n lengths: vec![0, 1, 2, 0, 1, 0],\n };\n let mut reader = BufReader::new(inner);\n let mut buf = [0, 0];\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 1);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 2);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 1);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n}\n\n#[tokio::test]\nasync fn maybe_pending() {\n let inner: &[u8] = &[5, 6, 7, 0, 1, 2, 3, 4];\n let mut reader = BufReader::with_capacity(2, MaybePending::new(inner));\n\n let mut buf = [0, 0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 3);\n assert_eq!(buf, [5, 6, 7]);\n assert_eq!(reader.buffer(), []);\n\n let mut buf = [0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 2);\n assert_eq!(buf, [0, 1]);\n assert_eq!(reader.buffer(), []);\n\n let mut buf = [0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [2]);\n assert_eq!(reader.buffer(), [3]);\n\n let mut buf = [0, 0, 0];\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [3, 0, 0]);\n assert_eq!(reader.buffer(), []);\n\n let nread = reader.read(&mut buf).await.unwrap();\n assert_eq!(nread, 1);\n assert_eq!(buf, [4, 0, 0]);\n assert_eq!(reader.buffer(), []);\n\n assert_eq!(reader.read(&mut buf).await.unwrap(), 0);\n}\n\n#[tokio::test]\nasync fn maybe_pending_buf_read() {\n let inner = MaybePending::new(&[0, 1, 2, 3, 1, 0]);\n let mut reader = BufReader::with_capacity(2, inner);\n let mut v = Vec::new();\n reader.read_until(3, &mut v).await.unwrap();\n assert_eq!(v, [0, 1, 2, 3]);\n v.clear();\n reader.read_until(1, &mut v).await.unwrap();\n assert_eq!(v, [1]);\n v.clear();\n reader.read_until(8, &mut v).await.unwrap();\n assert_eq!(v, [0]);\n v.clear();\n reader.read_until(9, &mut v).await.unwrap();\n assert_eq!(v, []);\n}\n\n// https://github.com/rust-lang/futures-rs/pull/1573#discussion_r281162309\n#[tokio::test]\nasync fn maybe_pending_seek() {\n struct MaybePendingSeek<'a> {\n inner: Cursor<&'a [u8]>,\n ready: bool,\n seek_res: Option>,\n }\n\n impl<'a> MaybePendingSeek<'a> {\n fn new(inner: &'a [u8]) -> Self {\n Self {\n inner: Cursor::new(inner),\n ready: true,\n seek_res: None,\n }\n }\n }\n\n impl AsyncRead for MaybePendingSeek<'_> {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n Pin::new(&mut self.inner).poll_read(cx, buf)\n }\n }\n\n impl AsyncBufRead for MaybePendingSeek<'_> {\n fn poll_fill_buf(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n let this: *mut Self = &mut *self as *mut _;\n Pin::new(&mut unsafe { &mut *this }.inner).poll_fill_buf(cx)\n }\n\n fn consume(mut self: Pin<&mut Self>, amt: usize) {\n Pin::new(&mut self.inner).consume(amt)\n }\n }\n\n impl AsyncSeek for MaybePendingSeek<'_> {\n fn start_seek(mut self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n self.seek_res = Some(Pin::new(&mut self.inner).start_seek(pos));\n Ok(())\n }\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if self.ready {\n self.ready = false;\n self.seek_res.take().unwrap_or(Ok(()))?;\n Pin::new(&mut self.inner).poll_complete(cx)\n } else {\n self.ready = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n }\n\n let inner: &[u8] = &[5, 6, 7, 0, 1, 2, 3, 4];\n let mut reader = BufReader::with_capacity(2, MaybePendingSeek::new(inner));\n\n assert_eq!(reader.seek(SeekFrom::Current(3)).await.unwrap(), 3);\n assert_eq!(run_fill_buf!(reader).unwrap(), &[0, 1][..]);\n assert!(reader.seek(SeekFrom::Current(i64::MIN)).await.is_err());\n assert_eq!(run_fill_buf!(reader).unwrap(), &[0, 1][..]);\n assert_eq!(reader.seek(SeekFrom::Current(1)).await.unwrap(), 4);\n assert_eq!(run_fill_buf!(reader).unwrap(), &[1, 2][..]);\n Pin::new(&mut reader).consume(1);\n assert_eq!(reader.seek(SeekFrom::Current(-2)).await.unwrap(), 3);\n}\n\n// This tests the AsyncBufReadExt::fill_buf wrapper.\n#[tokio::test]\nasync fn test_fill_buf_wrapper() {\n let (mut write, read) = tokio::io::duplex(16);\n\n let mut read = BufReader::new(read);\n write.write_all(b\"hello world\").await.unwrap();\n\n assert_eq!(read.fill_buf().await.unwrap(), b\"hello world\");\n read.consume(b\"hello \".len());\n assert_eq!(read.fill_buf().await.unwrap(), b\"world\");\n assert_eq!(read.fill_buf().await.unwrap(), b\"world\");\n read.consume(b\"world\".len());\n\n let mut fill = spawn(read.fill_buf());\n assert_pending!(fill.poll());\n\n write.write_all(b\"foo bar\").await.unwrap();\n assert_eq!(assert_ready!(fill.poll()).unwrap(), b\"foo bar\");\n drop(fill);\n\n drop(write);\n assert_eq!(read.fill_buf().await.unwrap(), b\"foo bar\");\n read.consume(b\"foo bar\".len());\n assert_eq!(read.fill_buf().await.unwrap(), b\"\");\n}\n" }, { "path": "tokio/tests/io_buf_writer.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\n// https://github.com/rust-lang/futures-rs/blob/1803948ff091b4eabf7f3bf39e16bbbdefca5cc8/futures/tests/io_buf_writer.rs\n\nuse futures::task::{Context, Poll};\nuse std::io::{self, Cursor};\nuse std::pin::Pin;\nuse tokio::io::{AsyncSeek, AsyncSeekExt, AsyncWrite, AsyncWriteExt, BufWriter, SeekFrom};\n\nuse futures::future;\nuse tokio_test::assert_ok;\n\nuse std::cmp;\nuse std::io::IoSlice;\n\nmod support {\n pub(crate) mod io_vec;\n}\nuse support::io_vec::IoBufs;\n\nstruct MaybePending {\n inner: Vec,\n ready: bool,\n}\n\nimpl MaybePending {\n fn new(inner: Vec) -> Self {\n Self {\n inner,\n ready: false,\n }\n }\n}\n\nimpl AsyncWrite for MaybePending {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n if self.ready {\n self.ready = false;\n Pin::new(&mut self.inner).poll_write(cx, buf)\n } else {\n self.ready = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_shutdown(cx)\n }\n}\n\nasync fn write_vectored(writer: &mut W, bufs: &[IoSlice<'_>]) -> io::Result\nwhere\n W: AsyncWrite + Unpin,\n{\n let mut writer = Pin::new(writer);\n future::poll_fn(|cx| writer.as_mut().poll_write_vectored(cx, bufs)).await\n}\n\n#[tokio::test]\nasync fn buf_writer() {\n let mut writer = BufWriter::with_capacity(2, Vec::new());\n\n assert_eq!(writer.write(&[0, 1]).await.unwrap(), 2);\n assert_eq!(writer.buffer(), []);\n assert_eq!(*writer.get_ref(), [0, 1]);\n\n assert_eq!(writer.write(&[2]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [2]);\n assert_eq!(*writer.get_ref(), [0, 1]);\n\n assert_eq!(writer.write(&[3]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [2, 3]);\n assert_eq!(*writer.get_ref(), [0, 1]);\n\n writer.flush().await.unwrap();\n assert_eq!(writer.buffer(), []);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3]);\n\n assert_eq!(writer.write(&[4]).await.unwrap(), 1);\n assert_eq!(writer.write(&[5]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [4, 5]);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3]);\n\n assert_eq!(writer.write(&[6]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [6]);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3, 4, 5]);\n\n assert_eq!(writer.write(&[7, 8]).await.unwrap(), 2);\n assert_eq!(writer.buffer(), []);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3, 4, 5, 6, 7, 8]);\n\n assert_eq!(writer.write(&[9, 10, 11]).await.unwrap(), 3);\n assert_eq!(writer.buffer(), []);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]);\n\n writer.flush().await.unwrap();\n assert_eq!(writer.buffer(), []);\n assert_eq!(*writer.get_ref(), [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]);\n}\n\n#[tokio::test]\nasync fn buf_writer_inner_flushes() {\n let mut w = BufWriter::with_capacity(3, Vec::new());\n assert_eq!(w.write(&[0, 1]).await.unwrap(), 2);\n assert_eq!(*w.get_ref(), []);\n w.flush().await.unwrap();\n let w = w.into_inner();\n assert_eq!(w, [0, 1]);\n}\n\n#[tokio::test]\nasync fn buf_writer_seek() {\n let mut w = BufWriter::with_capacity(3, Cursor::new(Vec::new()));\n w.write_all(&[0, 1, 2, 3, 4, 5]).await.unwrap();\n w.write_all(&[6, 7]).await.unwrap();\n assert_eq!(w.seek(SeekFrom::Current(0)).await.unwrap(), 8);\n assert_eq!(&w.get_ref().get_ref()[..], &[0, 1, 2, 3, 4, 5, 6, 7][..]);\n assert_eq!(w.seek(SeekFrom::Start(2)).await.unwrap(), 2);\n w.write_all(&[8, 9]).await.unwrap();\n w.flush().await.unwrap();\n assert_eq!(&w.into_inner().into_inner()[..], &[0, 1, 8, 9, 4, 5, 6, 7]);\n}\n\n#[tokio::test]\nasync fn maybe_pending_buf_writer() {\n let mut writer = BufWriter::with_capacity(2, MaybePending::new(Vec::new()));\n\n assert_eq!(writer.write(&[0, 1]).await.unwrap(), 2);\n assert_eq!(writer.buffer(), []);\n assert_eq!(&writer.get_ref().inner, &[0, 1]);\n\n assert_eq!(writer.write(&[2]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [2]);\n assert_eq!(&writer.get_ref().inner, &[0, 1]);\n\n assert_eq!(writer.write(&[3]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [2, 3]);\n assert_eq!(&writer.get_ref().inner, &[0, 1]);\n\n writer.flush().await.unwrap();\n assert_eq!(writer.buffer(), []);\n assert_eq!(&writer.get_ref().inner, &[0, 1, 2, 3]);\n\n assert_eq!(writer.write(&[4]).await.unwrap(), 1);\n assert_eq!(writer.write(&[5]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [4, 5]);\n assert_eq!(&writer.get_ref().inner, &[0, 1, 2, 3]);\n\n assert_eq!(writer.write(&[6]).await.unwrap(), 1);\n assert_eq!(writer.buffer(), [6]);\n assert_eq!(writer.get_ref().inner, &[0, 1, 2, 3, 4, 5]);\n\n assert_eq!(writer.write(&[7, 8]).await.unwrap(), 2);\n assert_eq!(writer.buffer(), []);\n assert_eq!(writer.get_ref().inner, &[0, 1, 2, 3, 4, 5, 6, 7, 8]);\n\n assert_eq!(writer.write(&[9, 10, 11]).await.unwrap(), 3);\n assert_eq!(writer.buffer(), []);\n assert_eq!(\n writer.get_ref().inner,\n &[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n );\n\n writer.flush().await.unwrap();\n assert_eq!(writer.buffer(), []);\n assert_eq!(\n &writer.get_ref().inner,\n &[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n );\n}\n\n#[tokio::test]\nasync fn maybe_pending_buf_writer_inner_flushes() {\n let mut w = BufWriter::with_capacity(3, MaybePending::new(Vec::new()));\n assert_eq!(w.write(&[0, 1]).await.unwrap(), 2);\n assert_eq!(&w.get_ref().inner, &[]);\n w.flush().await.unwrap();\n let w = w.into_inner().inner;\n assert_eq!(w, [0, 1]);\n}\n\n#[tokio::test]\nasync fn maybe_pending_buf_writer_seek() {\n struct MaybePendingSeek {\n inner: Cursor>,\n ready_write: bool,\n ready_seek: bool,\n seek_res: Option>,\n }\n\n impl MaybePendingSeek {\n fn new(inner: Vec) -> Self {\n Self {\n inner: Cursor::new(inner),\n ready_write: false,\n ready_seek: false,\n seek_res: None,\n }\n }\n }\n\n impl AsyncWrite for MaybePendingSeek {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n if self.ready_write {\n self.ready_write = false;\n Pin::new(&mut self.inner).poll_write(cx, buf)\n } else {\n self.ready_write = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.inner).poll_shutdown(cx)\n }\n }\n\n impl AsyncSeek for MaybePendingSeek {\n fn start_seek(mut self: Pin<&mut Self>, pos: SeekFrom) -> io::Result<()> {\n self.seek_res = Some(Pin::new(&mut self.inner).start_seek(pos));\n Ok(())\n }\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if self.ready_seek {\n self.ready_seek = false;\n self.seek_res.take().unwrap_or(Ok(()))?;\n Pin::new(&mut self.inner).poll_complete(cx)\n } else {\n self.ready_seek = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n }\n }\n\n let mut w = BufWriter::with_capacity(3, MaybePendingSeek::new(Vec::new()));\n w.write_all(&[0, 1, 2, 3, 4, 5]).await.unwrap();\n w.write_all(&[6, 7]).await.unwrap();\n assert_eq!(w.seek(SeekFrom::Current(0)).await.unwrap(), 8);\n assert_eq!(\n &w.get_ref().inner.get_ref()[..],\n &[0, 1, 2, 3, 4, 5, 6, 7][..]\n );\n assert_eq!(w.seek(SeekFrom::Start(2)).await.unwrap(), 2);\n w.write_all(&[8, 9]).await.unwrap();\n w.flush().await.unwrap();\n assert_eq!(\n &w.into_inner().inner.into_inner()[..],\n &[0, 1, 8, 9, 4, 5, 6, 7]\n );\n}\n\nstruct MockWriter {\n data: Vec,\n write_len: usize,\n vectored: bool,\n}\n\nimpl MockWriter {\n fn new(write_len: usize) -> Self {\n MockWriter {\n data: Vec::new(),\n write_len,\n vectored: false,\n }\n }\n\n fn vectored(write_len: usize) -> Self {\n MockWriter {\n data: Vec::new(),\n write_len,\n vectored: true,\n }\n }\n\n fn write_up_to(&mut self, buf: &[u8], limit: usize) -> usize {\n let len = cmp::min(buf.len(), limit);\n self.data.extend_from_slice(&buf[..len]);\n len\n }\n}\n\nimpl AsyncWrite for MockWriter {\n fn poll_write(\n self: Pin<&mut Self>,\n _: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n let this = self.get_mut();\n let n = this.write_up_to(buf, this.write_len);\n Ok(n).into()\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n _: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n let this = self.get_mut();\n let mut total_written = 0;\n for buf in bufs {\n let n = this.write_up_to(buf, this.write_len - total_written);\n total_written += n;\n if total_written == this.write_len {\n break;\n }\n }\n Ok(total_written).into()\n }\n\n fn is_write_vectored(&self) -> bool {\n self.vectored\n }\n\n fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n}\n\n#[tokio::test]\nasync fn write_vectored_empty_on_non_vectored() {\n let mut w = BufWriter::new(MockWriter::new(4));\n let n = assert_ok!(write_vectored(&mut w, &[]).await);\n assert_eq!(n, 0);\n\n let io_vec = [IoSlice::new(&[]); 3];\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 0);\n\n assert_ok!(w.flush().await);\n assert!(w.get_ref().data.is_empty());\n}\n\n#[tokio::test]\nasync fn write_vectored_empty_on_vectored() {\n let mut w = BufWriter::new(MockWriter::vectored(4));\n let n = assert_ok!(write_vectored(&mut w, &[]).await);\n assert_eq!(n, 0);\n\n let io_vec = [IoSlice::new(&[]); 3];\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 0);\n\n assert_ok!(w.flush().await);\n assert!(w.get_ref().data.is_empty());\n}\n\n#[tokio::test]\nasync fn write_vectored_basic_on_non_vectored() {\n let msg = b\"foo bar baz\";\n let bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&msg[4..8]),\n IoSlice::new(&msg[8..]),\n ];\n let mut w = BufWriter::new(MockWriter::new(4));\n let n = assert_ok!(write_vectored(&mut w, &bufs).await);\n assert_eq!(n, msg.len());\n assert!(w.buffer() == &msg[..]);\n assert_ok!(w.flush().await);\n assert_eq!(w.get_ref().data, msg);\n}\n\n#[tokio::test]\nasync fn write_vectored_basic_on_vectored() {\n let msg = b\"foo bar baz\";\n let bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&msg[4..8]),\n IoSlice::new(&msg[8..]),\n ];\n let mut w = BufWriter::new(MockWriter::vectored(4));\n let n = assert_ok!(write_vectored(&mut w, &bufs).await);\n assert_eq!(n, msg.len());\n assert!(w.buffer() == &msg[..]);\n assert_ok!(w.flush().await);\n assert_eq!(w.get_ref().data, msg);\n}\n\n#[tokio::test]\nasync fn write_vectored_large_total_on_non_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&msg[4..8]),\n IoSlice::new(&msg[8..]),\n ];\n let io_vec = IoBufs::new(&mut bufs);\n let mut w = BufWriter::with_capacity(8, MockWriter::new(4));\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 8);\n assert!(w.buffer() == &msg[..8]);\n let io_vec = io_vec.advance(n);\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 3);\n assert!(w.get_ref().data.as_slice() == &msg[..8]);\n assert!(w.buffer() == &msg[8..]);\n}\n\n#[tokio::test]\nasync fn write_vectored_large_total_on_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&msg[4..8]),\n IoSlice::new(&msg[8..]),\n ];\n let io_vec = IoBufs::new(&mut bufs);\n let mut w = BufWriter::with_capacity(8, MockWriter::vectored(10));\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 10);\n assert!(w.buffer().is_empty());\n let io_vec = io_vec.advance(n);\n let n = assert_ok!(write_vectored(&mut w, &io_vec).await);\n assert_eq!(n, 1);\n assert!(w.get_ref().data.as_slice() == &msg[..10]);\n assert!(w.buffer() == &msg[10..]);\n}\n\nstruct VectoredWriteHarness {\n writer: BufWriter,\n buf_capacity: usize,\n}\n\nimpl VectoredWriteHarness {\n fn new(buf_capacity: usize) -> Self {\n VectoredWriteHarness {\n writer: BufWriter::with_capacity(buf_capacity, MockWriter::new(4)),\n buf_capacity,\n }\n }\n\n fn with_vectored_backend(buf_capacity: usize) -> Self {\n VectoredWriteHarness {\n writer: BufWriter::with_capacity(buf_capacity, MockWriter::vectored(4)),\n buf_capacity,\n }\n }\n\n async fn write_all<'a, 'b>(&mut self, mut io_vec: IoBufs<'a, 'b>) -> usize {\n let mut total_written = 0;\n while !io_vec.is_empty() {\n let n = assert_ok!(write_vectored(&mut self.writer, &io_vec).await);\n assert!(n != 0);\n assert!(self.writer.buffer().len() <= self.buf_capacity);\n total_written += n;\n io_vec = io_vec.advance(n);\n }\n total_written\n }\n\n async fn flush(&mut self) -> &[u8] {\n assert_ok!(self.writer.flush().await);\n &self.writer.get_ref().data\n }\n}\n\n#[tokio::test]\nasync fn write_vectored_odd_on_non_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&[]),\n IoSlice::new(&msg[4..9]),\n IoSlice::new(&msg[9..]),\n ];\n let mut h = VectoredWriteHarness::new(8);\n let bytes_written = h.write_all(IoBufs::new(&mut bufs)).await;\n assert_eq!(bytes_written, msg.len());\n assert_eq!(h.flush().await, msg);\n}\n\n#[tokio::test]\nasync fn write_vectored_odd_on_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&msg[0..4]),\n IoSlice::new(&[]),\n IoSlice::new(&msg[4..9]),\n IoSlice::new(&msg[9..]),\n ];\n let mut h = VectoredWriteHarness::with_vectored_backend(8);\n let bytes_written = h.write_all(IoBufs::new(&mut bufs)).await;\n assert_eq!(bytes_written, msg.len());\n assert_eq!(h.flush().await, msg);\n}\n\n#[tokio::test]\nasync fn write_vectored_large_slice_on_non_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&[]),\n IoSlice::new(&msg[..9]),\n IoSlice::new(&msg[9..]),\n ];\n let mut h = VectoredWriteHarness::new(8);\n let bytes_written = h.write_all(IoBufs::new(&mut bufs)).await;\n assert_eq!(bytes_written, msg.len());\n assert_eq!(h.flush().await, msg);\n}\n\n#[tokio::test]\nasync fn write_vectored_large_slice_on_vectored() {\n let msg = b\"foo bar baz\";\n let mut bufs = [\n IoSlice::new(&[]),\n IoSlice::new(&msg[..9]),\n IoSlice::new(&msg[9..]),\n ];\n let mut h = VectoredWriteHarness::with_vectored_backend(8);\n let bytes_written = h.write_all(IoBufs::new(&mut bufs)).await;\n assert_eq!(bytes_written, msg.len());\n assert_eq!(h.flush().await, msg);\n}\n" }, { "path": "tokio/tests/io_chain.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncReadExt;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn chain() {\n let mut buf = Vec::new();\n let rd1: &[u8] = b\"hello \";\n let rd2: &[u8] = b\"world\";\n\n let mut rd = rd1.chain(rd2);\n assert_ok!(rd.read_to_end(&mut buf).await);\n assert_eq!(buf, b\"hello world\");\n}\n" }, { "path": "tokio/tests/io_copy.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse bytes::BytesMut;\nuse tokio::io::{self, AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf};\nuse tokio_test::assert_ok;\n\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n#[tokio::test]\nasync fn copy() {\n struct Rd(bool);\n\n impl AsyncRead for Rd {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if self.0 {\n buf.put_slice(b\"hello world\");\n self.0 = false;\n Poll::Ready(Ok(()))\n } else {\n Poll::Ready(Ok(()))\n }\n }\n }\n\n let mut rd = Rd(true);\n let mut wr = Vec::new();\n\n let n = assert_ok!(io::copy(&mut rd, &mut wr).await);\n assert_eq!(n, 11);\n assert_eq!(wr, b\"hello world\");\n}\n\n#[tokio::test]\nasync fn proxy() {\n struct BufferedWd {\n buf: BytesMut,\n writer: io::DuplexStream,\n }\n\n impl AsyncWrite for BufferedWd {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.get_mut().buf.extend_from_slice(buf);\n Poll::Ready(Ok(buf.len()))\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let this = self.get_mut();\n\n while !this.buf.is_empty() {\n let n = ready!(Pin::new(&mut this.writer).poll_write(cx, &this.buf))?;\n let _ = this.buf.split_to(n);\n }\n\n Pin::new(&mut this.writer).poll_flush(cx)\n }\n\n fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.writer).poll_shutdown(cx)\n }\n }\n\n let (rd, wd) = io::duplex(1024);\n let mut rd = rd.take(1024);\n let mut wd = BufferedWd {\n buf: BytesMut::new(),\n writer: wd,\n };\n\n // write start bytes\n assert_ok!(wd.write_all(&[0x42; 512]).await);\n assert_ok!(wd.flush().await);\n\n let n = assert_ok!(io::copy(&mut rd, &mut wd).await);\n\n assert_eq!(n, 1024);\n}\n\n#[tokio::test]\nasync fn copy_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n let mut reader: &[u8] = b\"hello\";\n let mut writer: Vec = vec![];\n let _ = io::copy(&mut reader, &mut writer).await;\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n" }, { "path": "tokio/tests/io_copy_bidirectional.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support bind()\n\nuse std::time::Duration;\nuse tokio::io::{self, copy_bidirectional, AsyncReadExt, AsyncWriteExt};\nuse tokio::net::TcpStream;\nuse tokio::task::JoinHandle;\n\nasync fn make_socketpair() -> (TcpStream, TcpStream) {\n let listener = tokio::net::TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = listener.local_addr().unwrap();\n let connector = TcpStream::connect(addr);\n let acceptor = listener.accept();\n\n let (c1, c2) = tokio::join!(connector, acceptor);\n\n (c1.unwrap(), c2.unwrap().0)\n}\n\nasync fn block_write(s: &mut TcpStream) -> usize {\n static BUF: [u8; 2048] = [0; 2048];\n\n let mut copied = 0;\n loop {\n tokio::select! {\n result = s.write(&BUF) => {\n copied += result.expect(\"write error\")\n },\n _ = tokio::time::sleep(Duration::from_millis(10)) => {\n break;\n }\n }\n }\n\n copied\n}\n\nasync fn symmetric(mut cb: F)\nwhere\n F: FnMut(JoinHandle>, TcpStream, TcpStream) -> Fut,\n Fut: std::future::Future,\n{\n // We run the test twice, with streams passed to copy_bidirectional in\n // different orders, in order to ensure that the two arguments are\n // interchangeable.\n\n let (a, mut a1) = make_socketpair().await;\n let (b, mut b1) = make_socketpair().await;\n\n let handle = tokio::spawn(async move { copy_bidirectional(&mut a1, &mut b1).await });\n cb(handle, a, b).await;\n\n let (a, mut a1) = make_socketpair().await;\n let (b, mut b1) = make_socketpair().await;\n\n let handle = tokio::spawn(async move { copy_bidirectional(&mut b1, &mut a1).await });\n\n cb(handle, b, a).await;\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn test_basic_transfer() {\n symmetric(|_handle, mut a, mut b| async move {\n a.write_all(b\"test\").await.unwrap();\n let mut tmp = [0; 4];\n b.read_exact(&mut tmp).await.unwrap();\n assert_eq!(&tmp[..], b\"test\");\n })\n .await\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn test_transfer_after_close() {\n symmetric(|handle, mut a, mut b| async move {\n AsyncWriteExt::shutdown(&mut a).await.unwrap();\n b.read_to_end(&mut Vec::new()).await.unwrap();\n\n b.write_all(b\"quux\").await.unwrap();\n let mut tmp = [0; 4];\n a.read_exact(&mut tmp).await.unwrap();\n assert_eq!(&tmp[..], b\"quux\");\n\n // Once both are closed, we should have our handle back\n drop(b);\n\n assert_eq!(handle.await.unwrap().unwrap(), (0, 4));\n })\n .await\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn blocking_one_side_does_not_block_other() {\n symmetric(|handle, mut a, mut b| async move {\n block_write(&mut a).await;\n\n b.write_all(b\"quux\").await.unwrap();\n let mut tmp = [0; 4];\n a.read_exact(&mut tmp).await.unwrap();\n assert_eq!(&tmp[..], b\"quux\");\n\n AsyncWriteExt::shutdown(&mut a).await.unwrap();\n\n let mut buf = Vec::new();\n b.read_to_end(&mut buf).await.unwrap();\n\n drop(b);\n\n assert_eq!(handle.await.unwrap().unwrap(), (buf.len() as u64, 4));\n })\n .await\n}\n\n#[tokio::test]\nasync fn immediate_exit_on_write_error() {\n let payload = b\"here, take this\";\n let error = || io::Error::new(io::ErrorKind::Other, \"no thanks!\");\n\n let mut a = tokio_test::io::Builder::new()\n .read(payload)\n .write_error(error())\n .build();\n\n let mut b = tokio_test::io::Builder::new()\n .read(payload)\n .write_error(error())\n .build();\n\n assert!(copy_bidirectional(&mut a, &mut b).await.is_err());\n}\n\n#[tokio::test]\nasync fn immediate_exit_on_read_error() {\n let error = || io::Error::new(io::ErrorKind::Other, \"got nothing!\");\n\n let mut a = tokio_test::io::Builder::new().read_error(error()).build();\n\n let mut b = tokio_test::io::Builder::new().read_error(error()).build();\n\n assert!(copy_bidirectional(&mut a, &mut b).await.is_err());\n}\n\n#[tokio::test]\nasync fn copy_bidirectional_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n let payload = b\"here, take this\";\n\n let mut a = tokio_test::io::Builder::new()\n .read(payload)\n .write(payload)\n .build();\n\n let mut b = tokio_test::io::Builder::new()\n .read(payload)\n .write(payload)\n .build();\n\n let _ = copy_bidirectional(&mut a, &mut b).await;\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n" }, { "path": "tokio/tests/io_driver.rs", "content": "#![warn(rust_2018_idioms)]\n// Wasi does not support panic recovery or threading\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))]\n\nuse tokio::net::TcpListener;\nuse tokio::runtime;\nuse tokio_test::{assert_ok, assert_pending};\n\nuse futures::task::{waker_ref, ArcWake};\nuse std::future::Future;\nuse std::net::TcpStream;\nuse std::pin::Pin;\nuse std::sync::{mpsc, Arc, Mutex};\nuse std::task::Context;\n\nstruct Task {\n future: Mutex>>,\n}\n\nimpl ArcWake for Task {\n fn wake_by_ref(_: &Arc) {\n // Do nothing...\n }\n}\n\nimpl Task {\n fn new(future: T) -> Task {\n Task {\n future: Mutex::new(Box::pin(future)),\n }\n }\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn test_drop_on_notify() {\n // When the reactor receives a kernel notification, it notifies the\n // task that holds the associated socket. If this notification results in\n // the task being dropped, the socket will also be dropped.\n //\n // Previously, there was a deadlock scenario where the reactor, while\n // notifying, held a lock and the task being dropped attempted to acquire\n // that same lock in order to clean up state.\n //\n // To simulate this case, we create a fake executor that does nothing when\n // the task is notified. This simulates an executor in the process of\n // shutting down. Then, when the task handle is dropped, the task itself is\n // dropped.\n\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n let (addr_tx, addr_rx) = mpsc::channel();\n\n // Define a task that just drains the listener\n let task = Arc::new(Task::new(async move {\n // Create a listener\n let listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n\n // Send the address\n let addr = listener.local_addr().unwrap();\n addr_tx.send(addr).unwrap();\n\n loop {\n let _ = listener.accept().await;\n }\n }));\n\n {\n let _enter = rt.enter();\n let waker = waker_ref(&task);\n let mut cx = Context::from_waker(&waker);\n assert_pending!(task.future.lock().unwrap().as_mut().poll(&mut cx));\n }\n\n // Get the address\n let addr = addr_rx.recv().unwrap();\n\n drop(task);\n\n // Establish a connection to the acceptor\n let _s = TcpStream::connect(addr).unwrap();\n\n // Force the reactor to turn\n rt.block_on(async {});\n}\n\n#[test]\n#[should_panic(\n expected = \"A Tokio 1.x context was found, but IO is disabled. Call `enable_io` on the runtime builder to enable IO.\"\n)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn panics_when_io_disabled() {\n let rt = runtime::Builder::new_current_thread().build().unwrap();\n\n rt.block_on(async {\n let listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n listener.set_nonblocking(true).unwrap();\n\n let _ = tokio::net::TcpListener::from_std(listener);\n });\n}\n" }, { "path": "tokio/tests/io_driver_drop.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support bind\n\nuse tokio::net::TcpListener;\nuse tokio::runtime;\nuse tokio_test::{assert_err, assert_pending, assert_ready, task};\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn tcp_doesnt_block() {\n let rt = rt();\n\n let listener = {\n let _enter = rt.enter();\n let listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n listener.set_nonblocking(true).unwrap();\n\n TcpListener::from_std(listener).unwrap()\n };\n\n drop(rt);\n\n let mut task = task::spawn(async move {\n assert_err!(listener.accept().await);\n });\n\n assert_ready!(task.poll());\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn drop_wakes() {\n let rt = rt();\n\n let listener = {\n let _enter = rt.enter();\n let listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n listener.set_nonblocking(true).unwrap();\n\n TcpListener::from_std(listener).unwrap()\n };\n\n let mut task = task::spawn(async move {\n assert_err!(listener.accept().await);\n });\n\n assert_pending!(task.poll());\n\n drop(rt);\n\n assert!(task.is_woken());\n assert_ready!(task.poll());\n}\n\nfn rt() -> runtime::Runtime {\n runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/io_fill_buf.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support file operations\n\nuse tempfile::NamedTempFile;\nuse tokio::fs::File;\nuse tokio::io::{AsyncBufReadExt, BufReader};\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn fill_buf_file() {\n let file = NamedTempFile::new().unwrap();\n\n assert_ok!(std::fs::write(file.path(), b\"hello\"));\n\n let file = assert_ok!(File::open(file.path()).await);\n let mut file = BufReader::new(file);\n\n let mut contents = Vec::new();\n\n loop {\n let consumed = {\n let buffer = assert_ok!(file.fill_buf().await);\n if buffer.is_empty() {\n break;\n }\n contents.extend_from_slice(buffer);\n buffer.len()\n };\n\n file.consume(consumed);\n }\n\n assert_eq!(contents, b\"hello\");\n}\n" }, { "path": "tokio/tests/io_join.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{join, AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, Join, ReadBuf};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nstruct R;\n\nimpl AsyncRead for R {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n buf.put_slice(b\"z\");\n Poll::Ready(Ok(()))\n }\n}\n\nstruct W;\n\nimpl AsyncWrite for W {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n Poll::Ready(Ok(1))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(Ok(2))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n}\n\n#[test]\nfn is_send_and_sync() {\n fn assert_bound() {}\n\n assert_bound::>();\n}\n\n#[test]\nfn method_delegation() {\n let mut rw = join(R, W);\n let mut buf = [0; 1];\n\n tokio_test::block_on(async move {\n assert_eq!(1, rw.read(&mut buf).await.unwrap());\n assert_eq!(b'z', buf[0]);\n\n assert_eq!(1, rw.write(b\"x\").await.unwrap());\n assert_eq!(\n 2,\n rw.write_vectored(&[io::IoSlice::new(b\"x\")]).await.unwrap()\n );\n assert!(rw.is_write_vectored());\n\n assert!(rw.flush().await.is_ok());\n assert!(rw.shutdown().await.is_ok());\n });\n}\n" }, { "path": "tokio/tests/io_lines.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncBufReadExt;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn lines_inherent() {\n let rd: &[u8] = b\"hello\\r\\nworld\\n\\n\";\n let mut st = rd.lines();\n\n let b = assert_ok!(st.next_line().await).unwrap();\n assert_eq!(b, \"hello\");\n let b = assert_ok!(st.next_line().await).unwrap();\n assert_eq!(b, \"world\");\n let b = assert_ok!(st.next_line().await).unwrap();\n assert_eq!(b, \"\");\n assert!(assert_ok!(st.next_line().await).is_none());\n}\n" }, { "path": "tokio/tests/io_mem_stream.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{duplex, AsyncReadExt, AsyncWriteExt};\n\n#[tokio::test]\nasync fn ping_pong() {\n let (mut a, mut b) = duplex(32);\n\n let mut buf = [0u8; 4];\n\n a.write_all(b\"ping\").await.unwrap();\n b.read_exact(&mut buf).await.unwrap();\n assert_eq!(&buf, b\"ping\");\n\n b.write_all(b\"pong\").await.unwrap();\n a.read_exact(&mut buf).await.unwrap();\n assert_eq!(&buf, b\"pong\");\n}\n\n#[tokio::test]\nasync fn across_tasks() {\n let (mut a, mut b) = duplex(32);\n\n let t1 = tokio::spawn(async move {\n a.write_all(b\"ping\").await.unwrap();\n let mut buf = [0u8; 4];\n a.read_exact(&mut buf).await.unwrap();\n assert_eq!(&buf, b\"pong\");\n });\n\n let t2 = tokio::spawn(async move {\n let mut buf = [0u8; 4];\n b.read_exact(&mut buf).await.unwrap();\n assert_eq!(&buf, b\"ping\");\n b.write_all(b\"pong\").await.unwrap();\n });\n\n t1.await.unwrap();\n t2.await.unwrap();\n}\n\n#[tokio::test]\nasync fn disconnect() {\n let (mut a, mut b) = duplex(32);\n\n let t1 = tokio::spawn(async move {\n a.write_all(b\"ping\").await.unwrap();\n // and dropped\n });\n\n let t2 = tokio::spawn(async move {\n let mut buf = [0u8; 32];\n let n = b.read(&mut buf).await.unwrap();\n assert_eq!(&buf[..n], b\"ping\");\n\n let n = b.read(&mut buf).await.unwrap();\n assert_eq!(n, 0);\n });\n\n t1.await.unwrap();\n t2.await.unwrap();\n}\n\n#[tokio::test]\nasync fn disconnect_reader() {\n let (a, mut b) = duplex(2);\n\n let t1 = tokio::spawn(async move {\n // this will block, as not all data fits into duplex\n b.write_all(b\"ping\").await.unwrap_err();\n });\n\n let t2 = tokio::spawn(async move {\n // here we drop the reader side, and we expect the writer in the other\n // task to exit with an error\n drop(a);\n });\n\n t2.await.unwrap();\n t1.await.unwrap();\n}\n\n#[tokio::test]\nasync fn max_write_size() {\n let (mut a, mut b) = duplex(32);\n\n let t1 = tokio::spawn(async move {\n let n = a.write(&[0u8; 64]).await.unwrap();\n assert_eq!(n, 32);\n let n = a.write(&[0u8; 64]).await.unwrap();\n assert_eq!(n, 4);\n });\n\n let mut buf = [0u8; 4];\n b.read_exact(&mut buf).await.unwrap();\n\n t1.await.unwrap();\n\n // drop b only after task t1 finishes writing\n drop(b);\n}\n\n#[tokio::test]\nasync fn duplex_is_cooperative() {\n let (mut tx, mut rx) = tokio::io::duplex(1024 * 8);\n\n tokio::select! {\n biased;\n\n _ = async {\n loop {\n let buf = [3u8; 4096];\n tx.write_all(&buf).await.unwrap();\n let mut buf = [0u8; 4096];\n let _ = rx.read(&mut buf).await.unwrap();\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n" }, { "path": "tokio/tests/io_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n#![cfg(panic = \"unwind\")]\n\nuse std::task::{Context, Poll};\nuse std::{error::Error, pin::Pin};\nuse tokio::io::{self, split, AsyncRead, AsyncWrite, ReadBuf};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\nstruct RW;\n\nimpl AsyncRead for RW {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n buf.put_slice(b\"z\");\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncWrite for RW {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n Poll::Ready(Ok(1))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\n#[cfg(unix)]\nmod unix {\n use std::os::unix::prelude::{AsRawFd, RawFd};\n\n pub struct MockFd;\n\n impl AsRawFd for MockFd {\n fn as_raw_fd(&self) -> RawFd {\n 0\n }\n }\n}\n\n#[test]\nfn read_buf_initialize_unfilled_to_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let mut buffer = Vec::::new();\n let mut read_buf = ReadBuf::new(&mut buffer);\n\n read_buf.initialize_unfilled_to(2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn read_buf_advance_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let mut buffer = Vec::::new();\n let mut read_buf = ReadBuf::new(&mut buffer);\n\n read_buf.advance(2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn read_buf_set_filled_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let mut buffer = Vec::::new();\n let mut read_buf = ReadBuf::new(&mut buffer);\n\n read_buf.set_filled(2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn read_buf_put_slice_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let mut buffer = Vec::::new();\n let mut read_buf = ReadBuf::new(&mut buffer);\n\n let new_slice = [0x40_u8, 0x41_u8];\n\n read_buf.put_slice(&new_slice);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn unsplit_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let (r1, _w1) = split(RW);\n let (_r2, w2) = split(RW);\n r1.unsplit(w2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\nfn async_fd_new_panic_caller() -> Result<(), Box> {\n use tokio::io::unix::AsyncFd;\n use tokio::runtime::Builder;\n\n let panic_location_file = test_panic(|| {\n // Runtime without `enable_io` so it has no IO driver set.\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let fd = unix::MockFd;\n\n let _ = AsyncFd::new(fd);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\nfn async_fd_with_interest_panic_caller() -> Result<(), Box> {\n use tokio::io::unix::AsyncFd;\n use tokio::io::Interest;\n use tokio::runtime::Builder;\n\n let panic_location_file = test_panic(|| {\n // Runtime without `enable_io` so it has no IO driver set.\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let fd = unix::MockFd;\n\n let _ = AsyncFd::with_interest(fd, Interest::READABLE);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\nfn async_fd_try_new_panic_caller() -> Result<(), Box> {\n use tokio::io::unix::AsyncFd;\n use tokio::runtime::Builder;\n\n let panic_location_file = test_panic(|| {\n // Runtime without `enable_io` so it has no IO driver set.\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let fd = unix::MockFd;\n\n let _ = AsyncFd::try_new(fd);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\nfn async_fd_try_with_interest_panic_caller() -> Result<(), Box> {\n use tokio::io::unix::AsyncFd;\n use tokio::io::Interest;\n use tokio::runtime::Builder;\n\n let panic_location_file = test_panic(|| {\n // Runtime without `enable_io` so it has no IO driver set.\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let fd = unix::MockFd;\n\n let _ = AsyncFd::try_with_interest(fd, Interest::READABLE);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n" }, { "path": "tokio/tests/io_poll_aio.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(target_os = \"freebsd\", feature = \"net\"))]\n\nuse mio_aio::{AioFsyncMode, SourceApi};\nuse std::{\n future::Future,\n io, mem,\n os::fd::AsFd,\n os::unix::io::{AsRawFd, RawFd},\n pin::{pin, Pin},\n task::{Context, Poll},\n};\nuse tempfile::tempfile;\nuse tokio::io::bsd::{Aio, AioSource};\nuse tokio_test::assert_pending;\n\nmod aio {\n use super::*;\n\n #[derive(Debug)]\n struct TokioSource<'fd>(mio_aio::Source>);\n\n impl<'fd> AioSource for TokioSource<'fd> {\n fn register(&mut self, kq: RawFd, token: usize) {\n self.0.register_raw(kq, token)\n }\n fn deregister(&mut self) {\n self.0.deregister_raw()\n }\n }\n\n /// A very crude implementation of an AIO-based future\n struct FsyncFut<'fd>(Aio>);\n\n impl<'fd> FsyncFut<'fd> {\n pub fn submit(self: Pin<&mut Self>) -> io::Result<()> {\n let p = unsafe { self.map_unchecked_mut(|s| &mut s.0 .0) };\n match p.submit() {\n Ok(()) => Ok(()),\n Err(e) => Err(io::Error::from_raw_os_error(e as i32)),\n }\n }\n }\n\n impl<'fd> Future for FsyncFut<'fd> {\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let poll_result = self.0.poll_ready(cx);\n match poll_result {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(e)) => Poll::Ready(Err(e)),\n Poll::Ready(Ok(_ev)) => {\n // At this point, we could clear readiness. But there's no\n // point, since we're about to drop the Aio.\n let p = unsafe { self.map_unchecked_mut(|s| &mut s.0 .0) };\n let result = p.aio_return();\n match result {\n Ok(r) => Poll::Ready(Ok(r)),\n Err(e) => Poll::Ready(Err(io::Error::from_raw_os_error(e as i32))),\n }\n }\n }\n }\n }\n\n /// Low-level AIO Source\n ///\n /// An example bypassing mio_aio and Nix to demonstrate how the kevent\n /// registration actually works, under the hood.\n struct LlSource(Pin>);\n\n impl LlSource {\n fn fsync(mut self: Pin<&mut Self>) {\n let r = unsafe {\n let p = self.0.as_mut().get_unchecked_mut();\n libc::aio_fsync(libc::O_SYNC, p)\n };\n assert_eq!(0, r);\n }\n }\n\n impl AioSource for LlSource {\n fn register(&mut self, kq: RawFd, token: usize) {\n let mut sev: libc::sigevent = unsafe { mem::MaybeUninit::zeroed().assume_init() };\n sev.sigev_notify = libc::SIGEV_KEVENT;\n sev.sigev_signo = kq;\n sev.sigev_value = libc::sigval {\n sival_ptr: token as *mut libc::c_void,\n };\n self.0.aio_sigevent = sev;\n }\n\n fn deregister(&mut self) {\n unsafe {\n self.0.aio_sigevent = mem::zeroed();\n }\n }\n }\n\n struct LlFut(Aio);\n\n impl LlFut {\n pub fn fsync(self: Pin<&mut Self>) {\n let p = unsafe { self.map_unchecked_mut(|s| &mut *(s.0)) };\n p.fsync();\n }\n }\n\n impl Future for LlFut {\n type Output = std::io::Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let poll_result = self.0.poll_ready(cx);\n match poll_result {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(e)) => Poll::Ready(Err(e)),\n Poll::Ready(Ok(ev)) => {\n // Clearing readiness makes the future non-idempotent; the\n // caller can't poll it repeatedly after it has already\n // returned Ready. But that's ok; most futures behave this\n // way.\n self.0.clear_ready(ev);\n let r = unsafe { libc::aio_return(self.0 .0.as_mut().get_unchecked_mut()) };\n if r >= 0 {\n Poll::Ready(Ok(r as usize))\n } else {\n Poll::Ready(Err(io::Error::last_os_error()))\n }\n }\n }\n }\n }\n\n #[tokio::test]\n async fn fsync() {\n let f = tempfile().unwrap();\n let fd = f.as_fd();\n let mode = AioFsyncMode::O_SYNC;\n let source = TokioSource(mio_aio::Fsync::fsync(fd, mode, 0));\n let poll_aio = Aio::new_for_aio(source).unwrap();\n let mut fut = pin!(FsyncFut(poll_aio));\n fut.as_mut().submit().unwrap();\n fut.await.unwrap();\n }\n\n #[tokio::test]\n async fn ll_fsync() {\n let f = tempfile().unwrap();\n let fd = f.as_raw_fd();\n let mut aiocb: libc::aiocb = unsafe { mem::MaybeUninit::zeroed().assume_init() };\n aiocb.aio_fildes = fd;\n let source = LlSource(Box::pin(aiocb));\n let mut poll_aio = Aio::new_for_aio(source).unwrap();\n let r = unsafe {\n let p = poll_aio.0.as_mut().get_unchecked_mut();\n libc::aio_fsync(libc::O_SYNC, p)\n };\n assert_eq!(0, r);\n let fut = LlFut(poll_aio);\n fut.await.unwrap();\n }\n\n /// A suitably crafted future type can reuse an Aio object\n #[tokio::test]\n async fn reuse() {\n let f = tempfile().unwrap();\n let fd = f.as_raw_fd();\n let mut aiocb: libc::aiocb = unsafe { mem::MaybeUninit::zeroed().assume_init() };\n aiocb.aio_fildes = fd;\n let source = LlSource(Box::pin(aiocb));\n let poll_aio = Aio::new_for_aio(source).unwrap();\n\n // Send the operation to the kernel the first time\n let mut fut = LlFut(poll_aio);\n {\n let mut pfut = Pin::new(&mut fut);\n pfut.as_mut().fsync();\n pfut.as_mut().await.unwrap();\n }\n\n // Check that readiness was cleared\n let mut ctx = Context::from_waker(futures::task::noop_waker_ref());\n assert_pending!(fut.0.poll_ready(&mut ctx));\n\n // and reuse the future and its Aio object\n {\n let mut pfut = Pin::new(&mut fut);\n pfut.as_mut().fsync();\n pfut.as_mut().await.unwrap();\n }\n }\n}\n\nmod lio {\n use super::*;\n\n /// Low-level source based on lio_listio\n ///\n /// An example demonstrating using AIO with `Interest::Lio`. mio_aio 0.8\n /// doesn't include any bindings for lio_listio, so we've got to go\n /// low-level.\n struct LioSource<'a> {\n aiocb: Pin<&'a mut [&'a mut libc::aiocb; 1]>,\n sev: libc::sigevent,\n }\n\n impl<'a> LioSource<'a> {\n fn new(aiocb: Pin<&'a mut [&'a mut libc::aiocb; 1]>) -> Self {\n LioSource {\n aiocb,\n sev: unsafe { mem::zeroed() },\n }\n }\n\n fn submit(mut self: Pin<&mut Self>) {\n let p: *const *mut libc::aiocb =\n unsafe { self.aiocb.as_mut().get_unchecked_mut() } as *const _ as *const *mut _;\n let r = unsafe { libc::lio_listio(libc::LIO_NOWAIT, p, 1, &mut self.sev) };\n assert_eq!(r, 0);\n }\n }\n\n impl<'a> AioSource for LioSource<'a> {\n fn register(&mut self, kq: RawFd, token: usize) {\n let mut sev: libc::sigevent = unsafe { mem::MaybeUninit::zeroed().assume_init() };\n sev.sigev_notify = libc::SIGEV_KEVENT;\n sev.sigev_signo = kq;\n sev.sigev_value = libc::sigval {\n sival_ptr: token as *mut libc::c_void,\n };\n self.sev = sev;\n }\n\n fn deregister(&mut self) {\n unsafe {\n self.sev = mem::zeroed();\n }\n }\n }\n\n struct LioFut<'a>(Aio>);\n\n impl<'a> LioFut<'a> {\n pub fn submit(self: Pin<&mut Self>) {\n let p = unsafe { self.map_unchecked_mut(|s| &mut *(s.0)) };\n p.submit();\n }\n }\n\n impl<'a> Future for LioFut<'a> {\n type Output = std::io::Result;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let poll_result = self.0.poll_ready(cx);\n match poll_result {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(e)) => Poll::Ready(Err(e)),\n Poll::Ready(Ok(ev)) => {\n // Clearing readiness makes the future non-idempotent; the\n // caller can't poll it repeatedly after it has already\n // returned Ready. But that's ok; most futures behave this\n // way. Clearing readiness is especially useful for\n // lio_listio, because sometimes some operations will be\n // ready but not all.\n self.0.clear_ready(ev);\n let r = unsafe {\n let p1 = self.get_unchecked_mut();\n let p2: &mut [&mut libc::aiocb; 1] =\n p1.0.aiocb.as_mut().get_unchecked_mut();\n let p3: &mut libc::aiocb = p2[0];\n libc::aio_return(p3)\n };\n if r >= 0 {\n Poll::Ready(Ok(r as usize))\n } else {\n Poll::Ready(Err(io::Error::last_os_error()))\n }\n }\n }\n }\n }\n\n /// An lio_listio operation with one fsync element\n #[tokio::test]\n async fn onewrite() {\n const WBUF: &[u8] = b\"abcdef\";\n let f = tempfile().unwrap();\n\n let mut aiocb: libc::aiocb = unsafe { mem::zeroed() };\n aiocb.aio_fildes = f.as_raw_fd();\n aiocb.aio_lio_opcode = libc::LIO_WRITE;\n aiocb.aio_nbytes = WBUF.len();\n aiocb.aio_buf = WBUF.as_ptr() as *mut _;\n let aiocb = pin!([&mut aiocb]);\n let source = LioSource::new(aiocb);\n let poll_aio = Aio::new_for_lio(source).unwrap();\n\n // Send the operation to the kernel\n let mut fut = pin!(LioFut(poll_aio));\n fut.as_mut().submit();\n fut.await.unwrap();\n }\n\n /// A suitably crafted future type can reuse an Aio object\n #[tokio::test]\n async fn reuse() {\n const WBUF: &[u8] = b\"abcdef\";\n let f = tempfile().unwrap();\n\n let mut aiocb: libc::aiocb = unsafe { mem::zeroed() };\n aiocb.aio_fildes = f.as_raw_fd();\n aiocb.aio_lio_opcode = libc::LIO_WRITE;\n aiocb.aio_nbytes = WBUF.len();\n aiocb.aio_buf = WBUF.as_ptr() as *mut _;\n let aiocb = pin!([&mut aiocb]);\n let source = LioSource::new(aiocb);\n let poll_aio = Aio::new_for_lio(source).unwrap();\n\n // Send the operation to the kernel the first time\n let mut fut = LioFut(poll_aio);\n {\n let mut pfut = Pin::new(&mut fut);\n pfut.as_mut().submit();\n pfut.as_mut().await.unwrap();\n }\n\n // Check that readiness was cleared\n let mut ctx = Context::from_waker(futures::task::noop_waker_ref());\n assert_pending!(fut.0.poll_ready(&mut ctx));\n\n // and reuse the future and its Aio object\n {\n let mut pfut = Pin::new(&mut fut);\n pfut.as_mut().submit();\n pfut.as_mut().await.unwrap();\n }\n }\n}\n" }, { "path": "tokio/tests/io_read.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n\nuse tokio::io::{AsyncRead, AsyncReadExt, ReadBuf};\nuse tokio_test::assert_ok;\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nmod support {\n pub(crate) mod leaked_buffers;\n}\nuse support::leaked_buffers::LeakedBuffers;\n\n#[tokio::test]\nasync fn read() {\n #[derive(Default)]\n struct Rd {\n poll_cnt: usize,\n }\n\n impl AsyncRead for Rd {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n assert_eq!(0, self.poll_cnt);\n self.poll_cnt += 1;\n\n buf.put_slice(b\"hello world\");\n Poll::Ready(Ok(()))\n }\n }\n\n let mut buf = Box::new([0; 11]);\n let mut rd = Rd::default();\n\n let n = assert_ok!(rd.read(&mut buf[..]).await);\n assert_eq!(n, 11);\n assert_eq!(buf[..], b\"hello world\"[..]);\n}\n\nstruct BadAsyncRead {\n leaked_buffers: LeakedBuffers,\n}\n\nimpl BadAsyncRead {\n fn new() -> Self {\n Self {\n leaked_buffers: LeakedBuffers::new(),\n }\n }\n}\n\nimpl AsyncRead for BadAsyncRead {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n *buf = ReadBuf::new(unsafe { self.leaked_buffers.create(buf.capacity()) });\n buf.advance(buf.capacity());\n\n Poll::Ready(Ok(()))\n }\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn read_buf_bad_async_read() {\n let mut buf = Vec::with_capacity(10);\n BadAsyncRead::new().read_buf(&mut buf).await.unwrap();\n}\n" }, { "path": "tokio/tests/io_read_buf.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncRead, AsyncReadExt, ReadBuf};\nuse tokio_test::assert_ok;\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn read_buf() {\n struct Rd {\n cnt: usize,\n }\n\n impl AsyncRead for Rd {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.cnt += 1;\n buf.put_slice(b\"hello world\");\n Poll::Ready(Ok(()))\n }\n }\n\n let mut buf = vec![];\n let mut rd = Rd { cnt: 0 };\n\n let n = assert_ok!(rd.read_buf(&mut buf).await);\n assert_eq!(1, rd.cnt);\n assert_eq!(n, 11);\n assert_eq!(buf[..], b\"hello world\"[..]);\n}\n\n#[tokio::test]\n#[cfg(feature = \"io-util\")]\nasync fn issue_5588() {\n use bytes::BufMut;\n\n // steps to zero\n let mut buf = [0; 8];\n let mut read_buf = ReadBuf::new(&mut buf);\n assert_eq!(read_buf.remaining_mut(), 8);\n assert_eq!(read_buf.chunk_mut().len(), 8);\n unsafe {\n read_buf.advance_mut(1);\n }\n assert_eq!(read_buf.remaining_mut(), 7);\n assert_eq!(read_buf.chunk_mut().len(), 7);\n unsafe {\n read_buf.advance_mut(5);\n }\n assert_eq!(read_buf.remaining_mut(), 2);\n assert_eq!(read_buf.chunk_mut().len(), 2);\n unsafe {\n read_buf.advance_mut(2);\n }\n assert_eq!(read_buf.remaining_mut(), 0);\n assert_eq!(read_buf.chunk_mut().len(), 0);\n\n // directly to zero\n let mut buf = [0; 8];\n let mut read_buf = ReadBuf::new(&mut buf);\n assert_eq!(read_buf.remaining_mut(), 8);\n assert_eq!(read_buf.chunk_mut().len(), 8);\n unsafe {\n read_buf.advance_mut(8);\n }\n assert_eq!(read_buf.remaining_mut(), 0);\n assert_eq!(read_buf.chunk_mut().len(), 0);\n\n // uninit\n let mut buf = [std::mem::MaybeUninit::new(1); 8];\n let mut uninit = ReadBuf::uninit(&mut buf);\n assert_eq!(uninit.remaining_mut(), 8);\n assert_eq!(uninit.chunk_mut().len(), 8);\n\n let mut buf = [std::mem::MaybeUninit::uninit(); 8];\n let mut uninit = ReadBuf::uninit(&mut buf);\n unsafe {\n uninit.advance_mut(4);\n }\n assert_eq!(uninit.remaining_mut(), 4);\n assert_eq!(uninit.chunk_mut().len(), 4);\n uninit.put_u8(1);\n assert_eq!(uninit.remaining_mut(), 3);\n assert_eq!(uninit.chunk_mut().len(), 3);\n uninit.put_slice(&[1, 2, 3]);\n assert_eq!(uninit.remaining_mut(), 0);\n assert_eq!(uninit.chunk_mut().len(), 0);\n}\n" }, { "path": "tokio/tests/io_read_exact.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncReadExt;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn read_exact() {\n let mut buf = Box::new([0; 8]);\n let mut rd: &[u8] = b\"hello world\";\n\n let n = assert_ok!(rd.read_exact(&mut buf[..]).await);\n assert_eq!(n, 8);\n assert_eq!(buf[..], b\"hello wo\"[..]);\n}\n" }, { "path": "tokio/tests/io_read_line.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::io::ErrorKind;\nuse tokio::io::{AsyncBufReadExt, BufReader, Error};\nuse tokio_test::{assert_ok, io::Builder};\n\nuse std::io::Cursor;\n\n#[tokio::test]\nasync fn read_line() {\n let mut buf = String::new();\n let mut rd = Cursor::new(b\"hello\\nworld\\n\\n\");\n\n let n = assert_ok!(rd.read_line(&mut buf).await);\n assert_eq!(n, 6);\n assert_eq!(buf, \"hello\\n\");\n buf.clear();\n let n = assert_ok!(rd.read_line(&mut buf).await);\n assert_eq!(n, 6);\n assert_eq!(buf, \"world\\n\");\n buf.clear();\n let n = assert_ok!(rd.read_line(&mut buf).await);\n assert_eq!(n, 1);\n assert_eq!(buf, \"\\n\");\n buf.clear();\n let n = assert_ok!(rd.read_line(&mut buf).await);\n assert_eq!(n, 0);\n assert_eq!(buf, \"\");\n}\n\n#[tokio::test]\nasync fn read_line_not_all_ready() {\n let mock = Builder::new()\n .read(b\"Hello Wor\")\n .read(b\"ld\\nFizzBuz\")\n .read(b\"z\\n1\\n2\")\n .build();\n\n let mut read = BufReader::new(mock);\n\n let mut line = \"We say \".to_string();\n let bytes = read.read_line(&mut line).await.unwrap();\n assert_eq!(bytes, \"Hello World\\n\".len());\n assert_eq!(line.as_str(), \"We say Hello World\\n\");\n\n line = \"I solve \".to_string();\n let bytes = read.read_line(&mut line).await.unwrap();\n assert_eq!(bytes, \"FizzBuzz\\n\".len());\n assert_eq!(line.as_str(), \"I solve FizzBuzz\\n\");\n\n line.clear();\n let bytes = read.read_line(&mut line).await.unwrap();\n assert_eq!(bytes, 2);\n assert_eq!(line.as_str(), \"1\\n\");\n\n line.clear();\n let bytes = read.read_line(&mut line).await.unwrap();\n assert_eq!(bytes, 1);\n assert_eq!(line.as_str(), \"2\");\n}\n\n#[tokio::test]\nasync fn read_line_invalid_utf8() {\n let mock = Builder::new().read(b\"Hello Wor\\xffld.\\n\").build();\n\n let mut read = BufReader::new(mock);\n\n let mut line = \"Foo\".to_string();\n let err = read.read_line(&mut line).await.expect_err(\"Should fail\");\n assert_eq!(err.kind(), ErrorKind::InvalidData);\n assert_eq!(err.to_string(), \"stream did not contain valid UTF-8\");\n assert_eq!(line.as_str(), \"Foo\");\n}\n\n#[tokio::test]\nasync fn read_line_fail() {\n let mock = Builder::new()\n .read(b\"Hello Wor\")\n .read_error(Error::new(ErrorKind::Other, \"The world has no end\"))\n .build();\n\n let mut read = BufReader::new(mock);\n\n let mut line = \"Foo\".to_string();\n let err = read.read_line(&mut line).await.expect_err(\"Should fail\");\n assert_eq!(err.kind(), ErrorKind::Other);\n assert_eq!(err.to_string(), \"The world has no end\");\n assert_eq!(line.as_str(), \"FooHello Wor\");\n}\n\n#[tokio::test]\nasync fn read_line_fail_and_utf8_fail() {\n let mock = Builder::new()\n .read(b\"Hello Wor\")\n .read(b\"\\xff\\xff\\xff\")\n .read_error(Error::new(ErrorKind::Other, \"The world has no end\"))\n .build();\n\n let mut read = BufReader::new(mock);\n\n let mut line = \"Foo\".to_string();\n let err = read.read_line(&mut line).await.expect_err(\"Should fail\");\n assert_eq!(err.kind(), ErrorKind::Other);\n assert_eq!(err.to_string(), \"The world has no end\");\n assert_eq!(line.as_str(), \"Foo\");\n}\n" }, { "path": "tokio/tests/io_read_to_end.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{AsyncRead, AsyncReadExt, ReadBuf};\nuse tokio_test::assert_ok;\nuse tokio_test::io::Builder;\n\n#[tokio::test]\nasync fn read_to_end() {\n let mut buf = vec![];\n let mut rd: &[u8] = b\"hello world\";\n\n let n = assert_ok!(rd.read_to_end(&mut buf).await);\n assert_eq!(n, 11);\n assert_eq!(buf[..], b\"hello world\"[..]);\n}\n\n#[derive(Copy, Clone, Debug)]\nenum State {\n Initializing,\n JustFilling,\n Done,\n}\n\nstruct UninitTest {\n num_init: usize,\n state: State,\n}\n\nimpl AsyncRead for UninitTest {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let me = Pin::into_inner(self);\n let real_num_init = buf.initialized().len() - buf.filled().len();\n assert_eq!(real_num_init, me.num_init, \"{:?}\", me.state);\n\n match me.state {\n State::Initializing => {\n buf.initialize_unfilled_to(me.num_init + 2);\n buf.advance(1);\n me.num_init += 1;\n\n if me.num_init == 24 {\n me.state = State::JustFilling;\n }\n }\n State::JustFilling => {\n buf.advance(1);\n me.num_init -= 1;\n\n if me.num_init == 15 {\n // The buffer is resized on next call.\n me.num_init = 0;\n me.state = State::Done;\n }\n }\n State::Done => { /* .. do nothing .. */ }\n }\n\n Poll::Ready(Ok(()))\n }\n}\n\n#[tokio::test]\nasync fn read_to_end_uninit() {\n let mut buf = Vec::with_capacity(64);\n let mut test = UninitTest {\n num_init: 0,\n state: State::Initializing,\n };\n\n test.read_to_end(&mut buf).await.unwrap();\n assert_eq!(buf.len(), 33);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // too slow with miri\nasync fn read_to_end_doesnt_grow_with_capacity() {\n let arr: Vec = (0..100).collect();\n\n // We only test from 32 since we allocate at least 32 bytes each time\n for len in 32..100 {\n let bytes = &arr[..len];\n for split in 0..len {\n for cap in 0..101 {\n let mut mock = if split == 0 {\n Builder::new().read(bytes).build()\n } else {\n Builder::new()\n .read(&bytes[..split])\n .read(&bytes[split..])\n .build()\n };\n let mut buf = Vec::with_capacity(cap);\n AsyncReadExt::read_to_end(&mut mock, &mut buf)\n .await\n .unwrap();\n // It has the right data.\n assert_eq!(buf.as_slice(), bytes);\n // Unless cap was smaller than length, then we did not reallocate.\n if cap >= len {\n assert_eq!(buf.capacity(), cap);\n }\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn read_to_end_grows_capacity_if_unfit() {\n let bytes = b\"the_vector_startingcap_will_be_smaller\";\n let mut mock = Builder::new().read(bytes).build();\n let initial_capacity = bytes.len() - 4;\n let mut buf = Vec::with_capacity(initial_capacity);\n AsyncReadExt::read_to_end(&mut mock, &mut buf)\n .await\n .unwrap();\n // *4 since it doubles when it doesn't fit and again when reaching EOF\n assert_eq!(buf.capacity(), initial_capacity * 4);\n}\n" }, { "path": "tokio/tests/io_read_to_string.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::io;\nuse tokio::io::AsyncReadExt;\nuse tokio_test::assert_ok;\nuse tokio_test::io::Builder;\n\n#[tokio::test]\nasync fn read_to_string() {\n let mut buf = String::new();\n let mut rd: &[u8] = b\"hello world\";\n\n let n = assert_ok!(rd.read_to_string(&mut buf).await);\n assert_eq!(n, 11);\n assert_eq!(buf[..], \"hello world\"[..]);\n}\n\n#[tokio::test]\nasync fn to_string_does_not_truncate_on_utf8_error() {\n let data = vec![0xff, 0xff, 0xff];\n\n let mut s = \"abc\".to_string();\n\n match AsyncReadExt::read_to_string(&mut data.as_slice(), &mut s).await {\n Ok(len) => panic!(\"Should fail: {len} bytes.\"),\n Err(err) if err.to_string() == \"stream did not contain valid UTF-8\" => {}\n Err(err) => panic!(\"Fail: {err}.\"),\n }\n\n assert_eq!(s, \"abc\");\n}\n\n#[tokio::test]\nasync fn to_string_does_not_truncate_on_io_error() {\n let mut mock = Builder::new()\n .read(b\"def\")\n .read_error(io::Error::new(io::ErrorKind::Other, \"whoops\"))\n .build();\n let mut s = \"abc\".to_string();\n\n match AsyncReadExt::read_to_string(&mut mock, &mut s).await {\n Ok(len) => panic!(\"Should fail: {len} bytes.\"),\n Err(err) if err.to_string() == \"whoops\" => {}\n Err(err) => panic!(\"Fail: {err}.\"),\n }\n\n assert_eq!(s, \"abc\");\n}\n\n#[tokio::test]\nasync fn to_string_appends() {\n let data = b\"def\".to_vec();\n\n let mut s = \"abc\".to_string();\n\n let len = AsyncReadExt::read_to_string(&mut data.as_slice(), &mut s)\n .await\n .unwrap();\n\n assert_eq!(len, 3);\n assert_eq!(s, \"abcdef\");\n}\n" }, { "path": "tokio/tests/io_read_until.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::io::ErrorKind;\nuse tokio::io::{AsyncBufReadExt, BufReader, Error};\nuse tokio_test::{assert_ok, io::Builder};\n\n#[tokio::test]\nasync fn read_until() {\n let mut buf = vec![];\n let mut rd: &[u8] = b\"hello world\";\n\n let n = assert_ok!(rd.read_until(b' ', &mut buf).await);\n assert_eq!(n, 6);\n assert_eq!(buf, b\"hello \");\n buf.clear();\n let n = assert_ok!(rd.read_until(b' ', &mut buf).await);\n assert_eq!(n, 5);\n assert_eq!(buf, b\"world\");\n buf.clear();\n let n = assert_ok!(rd.read_until(b' ', &mut buf).await);\n assert_eq!(n, 0);\n assert_eq!(buf, []);\n}\n\n#[tokio::test]\nasync fn read_until_not_all_ready() {\n let mock = Builder::new()\n .read(b\"Hello Wor\")\n .read(b\"ld#Fizz\\xffBuz\")\n .read(b\"z#1#2\")\n .build();\n\n let mut read = BufReader::new(mock);\n\n let mut chunk = b\"We say \".to_vec();\n let bytes = read.read_until(b'#', &mut chunk).await.unwrap();\n assert_eq!(bytes, b\"Hello World#\".len());\n assert_eq!(chunk, b\"We say Hello World#\");\n\n chunk = b\"I solve \".to_vec();\n let bytes = read.read_until(b'#', &mut chunk).await.unwrap();\n assert_eq!(bytes, b\"Fizz\\xffBuzz\\n\".len());\n assert_eq!(chunk, b\"I solve Fizz\\xffBuzz#\");\n\n chunk.clear();\n let bytes = read.read_until(b'#', &mut chunk).await.unwrap();\n assert_eq!(bytes, 2);\n assert_eq!(chunk, b\"1#\");\n\n chunk.clear();\n let bytes = read.read_until(b'#', &mut chunk).await.unwrap();\n assert_eq!(bytes, 1);\n assert_eq!(chunk, b\"2\");\n}\n\n#[tokio::test]\nasync fn read_until_fail() {\n let mock = Builder::new()\n .read(b\"Hello \\xffWor\")\n .read_error(Error::new(ErrorKind::Other, \"The world has no end\"))\n .build();\n\n let mut read = BufReader::new(mock);\n\n let mut chunk = b\"Foo\".to_vec();\n let err = read\n .read_until(b'#', &mut chunk)\n .await\n .expect_err(\"Should fail\");\n assert_eq!(err.kind(), ErrorKind::Other);\n assert_eq!(err.to_string(), \"The world has no end\");\n assert_eq!(chunk, b\"FooHello \\xffWor\");\n}\n" }, { "path": "tokio/tests/io_repeat.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(miri)))]\n\nuse tokio::io::AsyncReadExt;\n\n#[tokio::test]\nasync fn repeat_poll_read_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n let mut buf = [0u8; 4096];\n tokio::io::repeat(0b101).read_exact(&mut buf).await.unwrap();\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n" }, { "path": "tokio/tests/io_sink.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncWriteExt;\n\n#[tokio::test]\nasync fn sink_poll_write_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n let buf = vec![1, 2, 3];\n tokio::io::sink().write_all(&buf).await.unwrap();\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n\n#[tokio::test]\nasync fn sink_poll_flush_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n tokio::io::sink().flush().await.unwrap();\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n\n#[tokio::test]\nasync fn sink_poll_shutdown_is_cooperative() {\n tokio::select! {\n biased;\n _ = async {\n loop {\n tokio::io::sink().shutdown().await.unwrap();\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n" }, { "path": "tokio/tests/io_split.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n\nuse tokio::io::{\n split, AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf, ReadHalf, WriteHalf,\n};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nstruct RW;\n\nimpl AsyncRead for RW {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n buf.put_slice(b\"z\");\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncWrite for RW {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n Poll::Ready(Ok(1))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n Poll::Ready(Ok(2))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n}\n\n#[test]\nfn is_send_and_sync() {\n fn assert_bound() {}\n\n assert_bound::>();\n assert_bound::>();\n}\n\n#[test]\nfn split_stream_id() {\n let (r1, w1) = split(RW);\n let (r2, w2) = split(RW);\n assert!(r1.is_pair_of(&w1));\n assert!(!r1.is_pair_of(&w2));\n assert!(r2.is_pair_of(&w2));\n assert!(!r2.is_pair_of(&w1));\n}\n\n#[test]\nfn unsplit_ok() {\n let (r, w) = split(RW);\n r.unsplit(w);\n}\n\n#[test]\n#[should_panic]\nfn unsplit_err1() {\n let (r, _) = split(RW);\n let (_, w) = split(RW);\n r.unsplit(w);\n}\n\n#[test]\n#[should_panic]\nfn unsplit_err2() {\n let (_, w) = split(RW);\n let (r, _) = split(RW);\n r.unsplit(w);\n}\n\n#[test]\nfn method_delegation() {\n let (mut r, mut w) = split(RW);\n let mut buf = [0; 1];\n\n tokio_test::block_on(async move {\n assert_eq!(1, r.read(&mut buf).await.unwrap());\n assert_eq!(b'z', buf[0]);\n\n assert_eq!(1, w.write(b\"x\").await.unwrap());\n assert_eq!(\n 2,\n w.write_vectored(&[io::IoSlice::new(b\"x\")]).await.unwrap()\n );\n assert!(w.is_write_vectored());\n\n assert!(w.flush().await.is_ok());\n assert!(w.shutdown().await.is_ok());\n });\n}\n" }, { "path": "tokio/tests/io_take.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{self, AsyncRead, AsyncReadExt, ReadBuf};\nuse tokio_test::assert_ok;\n\nmod support {\n pub(crate) mod leaked_buffers;\n}\nuse support::leaked_buffers::LeakedBuffers;\n\n#[tokio::test]\nasync fn take() {\n let mut buf = [0; 6];\n let rd: &[u8] = b\"hello world\";\n\n let mut rd = rd.take(4);\n let n = assert_ok!(rd.read(&mut buf).await);\n assert_eq!(n, 4);\n assert_eq!(&buf, &b\"hell\\0\\0\"[..]);\n}\n\n#[tokio::test]\nasync fn issue_4435() {\n let mut buf = [0; 8];\n let rd: &[u8] = b\"hello world\";\n\n let rd = rd.take(4);\n tokio::pin!(rd);\n\n let mut read_buf = ReadBuf::new(&mut buf);\n read_buf.put_slice(b\"AB\");\n\n std::future::poll_fn(|cx| rd.as_mut().poll_read(cx, &mut read_buf))\n .await\n .unwrap();\n assert_eq!(&buf, &b\"ABhell\\0\\0\"[..]);\n}\n\nstruct BadReader {\n leaked_buffers: LeakedBuffers,\n}\n\nimpl BadReader {\n fn new() -> Self {\n Self {\n leaked_buffers: LeakedBuffers::new(),\n }\n }\n}\n\nimpl AsyncRead for BadReader {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n read_buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let mut buf = ReadBuf::new(unsafe { self.leaked_buffers.create(10) });\n buf.put_slice(&[123; 10]);\n *read_buf = buf;\n\n Poll::Ready(Ok(()))\n }\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn bad_reader_fails() {\n let mut buf = Vec::with_capacity(10);\n\n BadReader::new().take(10).read_buf(&mut buf).await.unwrap();\n}\n" }, { "path": "tokio/tests/io_util_empty.rs", "content": "#![cfg(feature = \"full\")]\nuse tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncSeekExt};\n\n#[tokio::test]\nasync fn empty_read_is_cooperative() {\n tokio::select! {\n biased;\n\n _ = async {\n loop {\n let mut buf = [0u8; 4096];\n let _ = tokio::io::empty().read(&mut buf).await;\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n\n#[tokio::test]\nasync fn empty_buf_reads_are_cooperative() {\n tokio::select! {\n biased;\n\n _ = async {\n loop {\n let mut buf = String::new();\n let _ = tokio::io::empty().read_line(&mut buf).await;\n }\n } => {},\n _ = tokio::task::yield_now() => {}\n }\n}\n\n#[tokio::test]\nasync fn empty_seek() {\n use std::io::SeekFrom;\n\n let mut empty = tokio::io::empty();\n\n assert!(matches!(empty.seek(SeekFrom::Start(0)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::Start(1)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::Start(u64::MAX)).await, Ok(0)));\n\n assert!(matches!(empty.seek(SeekFrom::End(i64::MIN)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::End(-1)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::End(0)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::End(1)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::End(i64::MAX)).await, Ok(0)));\n\n assert!(matches!(\n empty.seek(SeekFrom::Current(i64::MIN)).await,\n Ok(0)\n ));\n assert!(matches!(empty.seek(SeekFrom::Current(-1)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::Current(0)).await, Ok(0)));\n assert!(matches!(empty.seek(SeekFrom::Current(1)).await, Ok(0)));\n assert!(matches!(\n empty.seek(SeekFrom::Current(i64::MAX)).await,\n Ok(0)\n ));\n}\n" }, { "path": "tokio/tests/io_write.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\nuse tokio_test::assert_ok;\n\nuse bytes::BytesMut;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn write() {\n struct Wr {\n buf: BytesMut,\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n assert_eq!(self.cnt, 0);\n self.buf.extend(&buf[0..4]);\n Ok(4).into()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n cnt: 0,\n };\n\n let n = assert_ok!(wr.write(b\"hello world\").await);\n assert_eq!(n, 4);\n assert_eq!(wr.buf, b\"hell\"[..]);\n}\n\n#[tokio::test]\nasync fn write_cursor() {\n use std::io::Cursor;\n\n let mut wr = Cursor::new(Vec::new());\n\n let n = assert_ok!(wr.write(b\"hello world\").await);\n assert_eq!(n, 11);\n assert_eq!(wr.get_ref().as_slice(), &b\"hello world\"[..]);\n}\n" }, { "path": "tokio/tests/io_write_all.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\nuse tokio_test::assert_ok;\n\nuse bytes::BytesMut;\nuse std::cmp;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn write_all() {\n struct Wr {\n buf: BytesMut,\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n let n = cmp::min(4, buf.len());\n let buf = &buf[0..n];\n\n self.cnt += 1;\n self.buf.extend(buf);\n Ok(buf.len()).into()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n cnt: 0,\n };\n\n assert_ok!(wr.write_all(b\"hello world\").await);\n assert_eq!(wr.buf, b\"hello world\"[..]);\n assert_eq!(wr.cnt, 3);\n}\n" }, { "path": "tokio/tests/io_write_all_buf.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\nuse tokio_test::{assert_err, assert_ok};\n\nuse bytes::{Buf, Bytes, BytesMut};\nuse std::cmp;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn write_all_buf() {\n struct Wr {\n buf: BytesMut,\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n let n = cmp::min(4, buf.len());\n dbg!(buf);\n let buf = &buf[0..n];\n\n self.cnt += 1;\n self.buf.extend(buf);\n Ok(buf.len()).into()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n cnt: 0,\n };\n\n let mut buf = Bytes::from_static(b\"hello\").chain(Bytes::from_static(b\"world\"));\n\n assert_ok!(wr.write_all_buf(&mut buf).await);\n assert_eq!(wr.buf, b\"helloworld\"[..]);\n // expect 4 writes, [hell],[o],[worl],[d]\n assert_eq!(wr.cnt, 4);\n assert!(!buf.has_remaining());\n}\n\n#[tokio::test]\nasync fn write_buf_err() {\n /// Error out after writing the first 4 bytes\n struct Wr {\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n self.cnt += 1;\n if self.cnt == 2 {\n return Poll::Ready(Err(io::Error::new(io::ErrorKind::Other, \"whoops\")));\n }\n Poll::Ready(Ok(4))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr { cnt: 0 };\n\n let mut buf = Bytes::from_static(b\"hello\").chain(Bytes::from_static(b\"world\"));\n\n assert_err!(wr.write_all_buf(&mut buf).await);\n assert_eq!(\n buf.copy_to_bytes(buf.remaining()),\n Bytes::from_static(b\"oworld\")\n );\n}\n\n#[tokio::test]\nasync fn write_all_buf_vectored() {\n struct Wr {\n buf: BytesMut,\n }\n impl AsyncWrite for Wr {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n // When executing `write_all_buf` with this writer,\n // `poll_write` is not called.\n panic!(\"shouldn't be called\")\n }\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n for buf in bufs {\n self.buf.extend_from_slice(buf);\n }\n let n = self.buf.len();\n Ok(n).into()\n }\n fn is_write_vectored(&self) -> bool {\n // Enable vectored write.\n true\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n let mut buf = Bytes::from_static(b\"hello\")\n .chain(Bytes::from_static(b\" \"))\n .chain(Bytes::from_static(b\"world\"));\n\n wr.write_all_buf(&mut buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"hello world\");\n}\n" }, { "path": "tokio/tests/io_write_buf.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\nuse tokio_test::assert_ok;\n\nuse bytes::BytesMut;\nuse std::cmp;\nuse std::io::{self, Cursor};\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn write_all() {\n struct Wr {\n buf: BytesMut,\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n assert_eq!(self.cnt, 0);\n\n let n = cmp::min(4, buf.len());\n let buf = &buf[0..n];\n\n self.cnt += 1;\n self.buf.extend(buf);\n Ok(buf.len()).into()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n cnt: 0,\n };\n\n let mut buf = Cursor::new(&b\"hello world\"[..]);\n\n assert_ok!(wr.write_buf(&mut buf).await);\n assert_eq!(wr.buf, b\"hell\"[..]);\n assert_eq!(wr.cnt, 1);\n assert_eq!(buf.position(), 4);\n}\n\n#[tokio::test]\nasync fn write_buf_vectored() {\n struct Wr {\n buf: BytesMut,\n cnt: usize,\n }\n\n impl AsyncWrite for Wr {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n panic!(\"shouldn't be called\")\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n let mut n = 0;\n for buf in bufs {\n self.buf.extend_from_slice(buf);\n n += buf.len();\n }\n self.cnt += 1;\n Ok(n).into()\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n cnt: 0,\n };\n\n let mut buf = Cursor::new(&b\"hello world\"[..]);\n\n assert_ok!(wr.write_buf(&mut buf).await);\n assert_eq!(wr.buf, b\"hello world\"[..]);\n assert_eq!(wr.cnt, 1);\n assert_eq!(buf.position(), 11);\n}\n" }, { "path": "tokio/tests/io_write_int.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::{AsyncWrite, AsyncWriteExt};\n\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn write_int_should_err_if_write_count_0() {\n struct Wr {}\n\n impl AsyncWrite for Wr {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n Ok(0).into()\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n }\n\n let mut wr = Wr {};\n\n // should be ok just to test these 2, other cases actually expanded by same macro.\n assert!(wr.write_i8(0).await.is_err());\n assert!(wr.write_i32(12).await.is_err());\n}\n" }, { "path": "tokio/tests/join_handle_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support panic recovery\n#![cfg(panic = \"unwind\")]\n\nstruct PanicsOnDrop;\n\nimpl Drop for PanicsOnDrop {\n fn drop(&mut self) {\n panic!(\"I told you so\");\n }\n}\n\n#[tokio::test]\nasync fn test_panics_do_not_propagate_when_dropping_join_handle() {\n let join_handle = tokio::spawn(async move { PanicsOnDrop });\n\n // only drop the JoinHandle when the task has completed\n // (which is difficult to synchronize precisely)\n tokio::time::sleep(std::time::Duration::from_millis(3)).await;\n drop(join_handle);\n}\n" }, { "path": "tokio/tests/macros_join.rs", "content": "#![cfg(feature = \"macros\")]\n#![allow(clippy::disallowed_names)]\nuse std::sync::Arc;\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\n#[cfg(target_pointer_width = \"64\")]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse tokio::sync::{oneshot, Semaphore};\nuse tokio_test::{assert_pending, assert_ready, task};\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_comma() {\n let foo = tokio::join!(async { 1 },);\n assert_eq!(foo, (1,));\n\n let foo = tokio::join!(biased; async { 1 },);\n assert_eq!(foo, (1,));\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_no_comma() {\n let foo = tokio::join!(async { 1 });\n assert_eq!(foo, (1,));\n\n let foo = tokio::join!(biased; async { 1 });\n assert_eq!(foo, (1,));\n}\n\n#[maybe_tokio_test]\nasync fn sync_two_lit_expr_comma() {\n let foo = tokio::join!(async { 1 }, async { 2 },);\n assert_eq!(foo, (1, 2));\n\n let foo = tokio::join!(biased; async { 1 }, async { 2 },);\n assert_eq!(foo, (1, 2));\n}\n\n#[maybe_tokio_test]\nasync fn sync_two_lit_expr_no_comma() {\n let foo = tokio::join!(async { 1 }, async { 2 });\n assert_eq!(foo, (1, 2));\n\n let foo = tokio::join!(biased; async { 1 }, async { 2 });\n assert_eq!(foo, (1, 2));\n}\n\n#[maybe_tokio_test]\nasync fn two_await() {\n let (tx1, rx1) = oneshot::channel::<&str>();\n let (tx2, rx2) = oneshot::channel::();\n\n let mut join = task::spawn(async {\n tokio::join!(async { rx1.await.unwrap() }, async { rx2.await.unwrap() })\n });\n\n assert_pending!(join.poll());\n\n tx2.send(123).unwrap();\n assert!(join.is_woken());\n assert_pending!(join.poll());\n\n tx1.send(\"hello\").unwrap();\n assert!(join.is_woken());\n let res = assert_ready!(join.poll());\n\n assert_eq!((\"hello\", 123), res);\n}\n\n#[test]\n#[cfg(target_pointer_width = \"64\")]\nfn join_size() {\n use futures::future;\n use std::mem;\n\n let fut = async {\n let ready = future::ready(0i32);\n tokio::join!(ready)\n };\n assert_eq!(mem::size_of_val(&fut), 32);\n\n let fut = async {\n let ready1 = future::ready(0i32);\n let ready2 = future::ready(0i32);\n tokio::join!(ready1, ready2)\n };\n assert_eq!(mem::size_of_val(&fut), 48);\n}\n\nasync fn non_cooperative_task(permits: Arc) -> usize {\n let mut exceeded_budget = 0;\n\n for _ in 0..5 {\n // Another task should run after this task uses its whole budget\n for _ in 0..128 {\n let _permit = permits.clone().acquire_owned().await.unwrap();\n }\n\n exceeded_budget += 1;\n }\n\n exceeded_budget\n}\n\nasync fn poor_little_task(permits: Arc) -> usize {\n let mut how_many_times_i_got_to_run = 0;\n\n for _ in 0..5 {\n let _permit = permits.clone().acquire_owned().await.unwrap();\n how_many_times_i_got_to_run += 1;\n }\n\n how_many_times_i_got_to_run\n}\n\n#[tokio::test]\nasync fn join_does_not_allow_tasks_to_starve() {\n let permits = Arc::new(Semaphore::new(1));\n\n // non_cooperative_task should yield after its budget is exceeded and then poor_little_task should run.\n let (non_cooperative_result, little_task_result) = tokio::join!(\n non_cooperative_task(Arc::clone(&permits)),\n poor_little_task(permits)\n );\n\n assert_eq!(5, non_cooperative_result);\n assert_eq!(5, little_task_result);\n}\n\n#[tokio::test]\nasync fn a_different_future_is_polled_first_every_time_poll_fn_is_polled() {\n let poll_order = Arc::new(std::sync::Mutex::new(vec![]));\n\n let fut = |x, poll_order: Arc>>| async move {\n for _ in 0..4 {\n {\n let mut guard = poll_order.lock().unwrap();\n\n guard.push(x);\n }\n\n tokio::task::yield_now().await;\n }\n };\n\n tokio::join!(\n fut(1, Arc::clone(&poll_order)),\n fut(2, Arc::clone(&poll_order)),\n fut(3, Arc::clone(&poll_order)),\n );\n\n // Each time the future created by join! is polled, it should start\n // by polling a different future first.\n assert_eq!(\n vec![1, 2, 3, 2, 3, 1, 3, 1, 2, 1, 2, 3],\n *poll_order.lock().unwrap()\n );\n}\n\n#[tokio::test]\nasync fn futures_are_polled_in_order_in_biased_mode() {\n let poll_order = Arc::new(std::sync::Mutex::new(vec![]));\n\n let fut = |x, poll_order: Arc>>| async move {\n for _ in 0..4 {\n {\n let mut guard = poll_order.lock().unwrap();\n\n guard.push(x);\n }\n\n tokio::task::yield_now().await;\n }\n };\n\n tokio::join!(\n biased;\n fut(1, Arc::clone(&poll_order)),\n fut(2, Arc::clone(&poll_order)),\n fut(3, Arc::clone(&poll_order)),\n );\n\n // Each time the future created by join! is polled, it should start\n // by polling in the order as declared in the macro inputs.\n assert_eq!(\n vec![1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3],\n *poll_order.lock().unwrap()\n );\n}\n\n#[test]\n#[cfg(target_pointer_width = \"64\")]\nfn join_size_biased() {\n use futures::future;\n use std::mem;\n\n let fut = async {\n let ready = future::ready(0i32);\n tokio::join!(biased; ready)\n };\n assert_eq!(mem::size_of_val(&fut), 24);\n\n let fut = async {\n let ready1 = future::ready(0i32);\n let ready2 = future::ready(0i32);\n tokio::join!(biased; ready1, ready2)\n };\n assert_eq!(mem::size_of_val(&fut), 40);\n}\n\n#[tokio::test]\n#[allow(clippy::unit_cmp)]\nasync fn empty_join() {\n assert_eq!(tokio::join!(), ());\n assert_eq!(tokio::join!(biased;), ());\n}\n\n#[tokio::test]\nasync fn join_into_future() {\n struct NotAFuture;\n impl std::future::IntoFuture for NotAFuture {\n type Output = ();\n type IntoFuture = std::future::Ready<()>;\n\n fn into_future(self) -> Self::IntoFuture {\n std::future::ready(())\n }\n }\n\n tokio::join!(NotAFuture);\n}\n\n// Regression test for: https://github.com/tokio-rs/tokio/issues/7637\n// We want to make sure that the `const COUNT: u32` declaration\n// inside the macro body doesn't leak to the caller to cause compiler failures\n// or variable shadowing.\n#[tokio::test]\nasync fn caller_names_const_count() {\n let (tx, rx) = oneshot::channel::();\n\n const COUNT: u32 = 2;\n\n let mut join = task::spawn(async { tokio::join!(async { tx.send(COUNT).unwrap() }) });\n assert_ready!(join.poll());\n\n let res = rx.await.unwrap();\n\n // This passing demonstrates that the const in the macro is\n // not shadowing the caller-specified COUNT value\n assert_eq!(2, res);\n}\n" }, { "path": "tokio/tests/macros_pin.rs", "content": "#![cfg(feature = \"macros\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nasync fn one() {}\nasync fn two() {}\n\n#[maybe_tokio_test]\nasync fn multi_pin() {\n tokio::pin! {\n let f1 = one();\n let f2 = two();\n }\n\n (&mut f1).await;\n (&mut f2).await;\n}\n" }, { "path": "tokio/tests/macros_rename_test.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threading\n\n#[allow(unused_imports)]\nuse std as tokio;\n\nuse ::tokio as tokio1;\n\nmod test {\n pub use ::tokio;\n}\n\nasync fn compute() -> usize {\n let join = tokio1::spawn(async { 1 });\n join.await.unwrap()\n}\n\n#[tokio1::main(crate = \"tokio1\")]\nasync fn compute_main() -> usize {\n compute().await\n}\n\n#[test]\nfn crate_rename_main() {\n assert_eq!(1, compute_main());\n}\n\n#[tokio1::test(crate = \"tokio1\")]\nasync fn crate_rename_test() {\n assert_eq!(1, compute().await);\n}\n\n#[test::tokio::test(crate = \"test::tokio\")]\nasync fn crate_path_test() {\n assert_eq!(1, compute().await);\n}\n" }, { "path": "tokio/tests/macros_select.rs", "content": "#![cfg(feature = \"macros\")]\n#![allow(clippy::disallowed_names)]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse tokio::sync::oneshot;\nuse tokio_test::{assert_ok, assert_pending, assert_ready};\n\nuse std::future::poll_fn;\nuse std::task::Poll::Ready;\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_comma() {\n let foo = tokio::select! {\n foo = async { 1 } => foo,\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn no_branch_else_only() {\n let foo = tokio::select! {\n else => 1,\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn no_branch_else_only_biased() {\n let foo = tokio::select! {\n biased;\n else => 1,\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn nested_one() {\n let foo = tokio::select! {\n foo = async { 1 } => tokio::select! {\n bar = async { foo } => bar,\n },\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_no_comma() {\n let foo = tokio::select! {\n foo = async { 1 } => foo\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_block() {\n let foo = tokio::select! {\n foo = async { 1 } => { foo }\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_await() {\n let foo = tokio::select! {\n foo = one() => foo,\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_ident() {\n let one = one();\n\n let foo = tokio::select! {\n foo = one => foo,\n };\n\n assert_eq!(foo, 1);\n}\n\n#[maybe_tokio_test]\nasync fn sync_two() {\n use std::cell::Cell;\n\n let cnt = Cell::new(0);\n\n let res = tokio::select! {\n foo = async {\n cnt.set(cnt.get() + 1);\n 1\n } => foo,\n bar = async {\n cnt.set(cnt.get() + 1);\n 2\n } => bar,\n };\n\n assert_eq!(1, cnt.get());\n assert!(res == 1 || res == 2);\n}\n\n#[maybe_tokio_test]\nasync fn drop_in_fut() {\n let s = \"hello\".to_string();\n\n let res = tokio::select! {\n foo = async {\n let v = one().await;\n drop(s);\n v\n } => foo\n };\n\n assert_eq!(res, 1);\n}\n\n#[maybe_tokio_test]\n#[cfg(feature = \"full\")]\nasync fn one_ready() {\n let (tx1, rx1) = oneshot::channel::();\n let (_tx2, rx2) = oneshot::channel::();\n\n tx1.send(1).unwrap();\n\n let v = tokio::select! {\n res = rx1 => {\n assert_ok!(res)\n },\n _ = rx2 => unreachable!(),\n };\n\n assert_eq!(1, v);\n}\n\n#[maybe_tokio_test]\n#[cfg(feature = \"full\")]\nasync fn select_streams() {\n use tokio::sync::mpsc;\n\n let (tx1, mut rx1) = mpsc::unbounded_channel::();\n let (tx2, mut rx2) = mpsc::unbounded_channel::();\n\n tokio::spawn(async move {\n assert_ok!(tx2.send(1));\n tokio::task::yield_now().await;\n\n assert_ok!(tx1.send(2));\n tokio::task::yield_now().await;\n\n assert_ok!(tx2.send(3));\n tokio::task::yield_now().await;\n\n drop((tx1, tx2));\n });\n\n let mut rem = true;\n let mut msgs = vec![];\n\n while rem {\n tokio::select! {\n Some(x) = rx1.recv() => {\n msgs.push(x);\n }\n Some(y) = rx2.recv() => {\n msgs.push(y);\n }\n else => {\n rem = false;\n }\n }\n }\n\n msgs.sort_unstable();\n assert_eq!(&msgs[..], &[1, 2, 3]);\n}\n\n#[maybe_tokio_test]\nasync fn move_uncompleted_futures() {\n let (tx1, mut rx1) = oneshot::channel::();\n let (tx2, mut rx2) = oneshot::channel::();\n\n tx1.send(1).unwrap();\n tx2.send(2).unwrap();\n\n let ran;\n\n tokio::select! {\n res = &mut rx1 => {\n assert_eq!(1, assert_ok!(res));\n assert_eq!(2, assert_ok!(rx2.await));\n ran = true;\n },\n res = &mut rx2 => {\n assert_eq!(2, assert_ok!(res));\n assert_eq!(1, assert_ok!(rx1.await));\n ran = true;\n },\n }\n\n assert!(ran);\n}\n\n#[maybe_tokio_test]\nasync fn nested() {\n let res = tokio::select! {\n x = async { 1 } => {\n tokio::select! {\n y = async { 2 } => x + y,\n }\n }\n };\n\n assert_eq!(res, 3);\n}\n\n#[cfg(target_pointer_width = \"64\")]\nmod pointer_64_tests {\n use super::maybe_tokio_test;\n use futures::future;\n use std::mem;\n\n #[maybe_tokio_test]\n async fn struct_size_1() {\n let fut = async {\n let ready = future::ready(0i32);\n\n tokio::select! {\n _ = ready => {},\n }\n };\n\n assert_eq!(mem::size_of_val(&fut), 32);\n }\n\n #[maybe_tokio_test]\n async fn struct_size_2() {\n let fut = async {\n let ready1 = future::ready(0i32);\n let ready2 = future::ready(0i32);\n\n tokio::select! {\n _ = ready1 => {},\n _ = ready2 => {},\n }\n };\n\n assert_eq!(mem::size_of_val(&fut), 40);\n }\n\n #[maybe_tokio_test]\n async fn struct_size_3() {\n let fut = async {\n let ready1 = future::ready(0i32);\n let ready2 = future::ready(0i32);\n let ready3 = future::ready(0i32);\n\n tokio::select! {\n _ = ready1 => {},\n _ = ready2 => {},\n _ = ready3 => {},\n }\n };\n\n assert_eq!(mem::size_of_val(&fut), 48);\n }\n}\n\n#[maybe_tokio_test]\nasync fn mutable_borrowing_future_with_same_borrow_in_block() {\n let mut value = 234;\n\n tokio::select! {\n _ = require_mutable(&mut value) => { },\n _ = async_noop() => {\n value += 5;\n },\n }\n\n assert!(value >= 234);\n}\n\n#[maybe_tokio_test]\nasync fn mutable_borrowing_future_with_same_borrow_in_block_and_else() {\n let mut value = 234;\n\n tokio::select! {\n _ = require_mutable(&mut value) => { },\n _ = async_noop() => {\n value += 5;\n },\n else => {\n value += 27;\n },\n }\n\n assert!(value >= 234);\n}\n\n#[maybe_tokio_test]\nasync fn future_panics_after_poll() {\n use tokio_test::task;\n\n let (tx, rx) = oneshot::channel();\n\n let mut polled = false;\n\n let f = poll_fn(|_| {\n assert!(!polled);\n polled = true;\n Ready(None::<()>)\n });\n\n let mut f = task::spawn(async {\n tokio::select! {\n Some(_) = f => unreachable!(),\n ret = rx => ret.unwrap(),\n }\n });\n\n assert_pending!(f.poll());\n assert_pending!(f.poll());\n\n assert_ok!(tx.send(1));\n\n let res = assert_ready!(f.poll());\n assert_eq!(1, res);\n}\n\n#[maybe_tokio_test]\nasync fn disable_with_if() {\n use tokio_test::task;\n\n let f = poll_fn(|_| panic!());\n let (tx, rx) = oneshot::channel();\n\n let mut f = task::spawn(async {\n tokio::select! {\n _ = f, if false => unreachable!(),\n _ = rx => (),\n }\n });\n\n assert_pending!(f.poll());\n\n assert_ok!(tx.send(()));\n assert!(f.is_woken());\n\n assert_ready!(f.poll());\n}\n\n#[maybe_tokio_test]\nasync fn join_with_select() {\n use tokio_test::task;\n\n let (tx1, mut rx1) = oneshot::channel();\n let (tx2, mut rx2) = oneshot::channel();\n\n let mut f = task::spawn(async {\n let mut a = None;\n let mut b = None;\n\n while a.is_none() || b.is_none() {\n tokio::select! {\n v1 = &mut rx1, if a.is_none() => a = Some(assert_ok!(v1)),\n v2 = &mut rx2, if b.is_none() => b = Some(assert_ok!(v2))\n }\n }\n\n (a.unwrap(), b.unwrap())\n });\n\n assert_pending!(f.poll());\n\n assert_ok!(tx1.send(123));\n assert!(f.is_woken());\n assert_pending!(f.poll());\n\n assert_ok!(tx2.send(456));\n assert!(f.is_woken());\n let (a, b) = assert_ready!(f.poll());\n\n assert_eq!(a, 123);\n assert_eq!(b, 456);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn use_future_in_if_condition() {\n use tokio::time::{self, Duration};\n\n tokio::select! {\n _ = time::sleep(Duration::from_millis(10)), if false => {\n panic!(\"if condition ignored\")\n }\n _ = async { 1u32 } => {\n }\n }\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn use_future_in_if_condition_biased() {\n use tokio::time::{self, Duration};\n\n tokio::select! {\n biased;\n _ = time::sleep(Duration::from_millis(10)), if false => {\n panic!(\"if condition ignored\")\n }\n _ = async { 1u32 } => {\n }\n }\n}\n\n#[maybe_tokio_test]\nasync fn many_branches() {\n let num = tokio::select! {\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n x = async { 1 } => x,\n };\n\n assert_eq!(1, num);\n}\n\n#[maybe_tokio_test]\nasync fn never_branch_no_warnings() {\n let t = tokio::select! {\n _ = async_never() => 0,\n one_async_ready = one() => one_async_ready,\n };\n assert_eq!(t, 1);\n}\n\nasync fn one() -> usize {\n 1\n}\n\nasync fn require_mutable(_: &mut i32) {}\nasync fn async_noop() {}\n\nasync fn async_never() -> ! {\n futures::future::pending().await\n}\n\n// From https://github.com/tokio-rs/tokio/issues/2857\n#[maybe_tokio_test]\nasync fn mut_on_left_hand_side() {\n let v = async move {\n let ok = async { 1 };\n tokio::pin!(ok);\n tokio::select! {\n mut a = &mut ok => {\n a += 1;\n a\n }\n }\n }\n .await;\n assert_eq!(v, 2);\n}\n\n#[maybe_tokio_test]\nasync fn biased_one_not_ready() {\n let (_tx1, rx1) = oneshot::channel::();\n let (tx2, rx2) = oneshot::channel::();\n let (tx3, rx3) = oneshot::channel::();\n\n tx2.send(2).unwrap();\n tx3.send(3).unwrap();\n\n let v = tokio::select! {\n biased;\n\n _ = rx1 => unreachable!(),\n res = rx2 => {\n assert_ok!(res)\n },\n _ = rx3 => {\n panic!(\"This branch should never be activated because `rx2` should be polled before `rx3` due to `biased;`.\")\n }\n };\n\n assert_eq!(2, v);\n}\n\n#[maybe_tokio_test]\n#[cfg(feature = \"full\")]\nasync fn biased_eventually_ready() {\n use tokio::task::yield_now;\n\n let one = async {};\n let two = async { yield_now().await };\n let three = async { yield_now().await };\n\n let mut count = 0u8;\n\n tokio::pin!(one, two, three);\n\n loop {\n tokio::select! {\n biased;\n\n _ = &mut two, if count < 2 => {\n count += 1;\n assert_eq!(count, 2);\n }\n _ = &mut three, if count < 3 => {\n count += 1;\n assert_eq!(count, 3);\n }\n _ = &mut one, if count < 1 => {\n count += 1;\n assert_eq!(count, 1);\n }\n else => break,\n }\n }\n\n assert_eq!(count, 3);\n}\n\n// https://github.com/tokio-rs/tokio/issues/3830\n// https://github.com/rust-lang/rust-clippy/issues/7304\n#[warn(clippy::default_numeric_fallback)]\npub async fn default_numeric_fallback() {\n tokio::select! {\n _ = async {} => (),\n else => (),\n }\n}\n\n// https://github.com/tokio-rs/tokio/issues/4182\n#[maybe_tokio_test]\nasync fn mut_ref_patterns() {\n tokio::select! {\n Some(mut foo) = async { Some(\"1\".to_string()) } => {\n assert_eq!(foo, \"1\");\n foo = \"2\".to_string();\n assert_eq!(foo, \"2\");\n },\n };\n\n tokio::select! {\n Some(ref foo) = async { Some(\"1\".to_string()) } => {\n assert_eq!(*foo, \"1\");\n },\n };\n\n tokio::select! {\n Some(ref mut foo) = async { Some(\"1\".to_string()) } => {\n assert_eq!(*foo, \"1\");\n *foo = \"2\".to_string();\n assert_eq!(*foo, \"2\");\n },\n };\n}\n\n#[cfg(tokio_unstable)]\nmod unstable {\n use tokio::runtime::RngSeed;\n\n #[test]\n fn deterministic_select_current_thread() {\n let seed = b\"bytes used to generate seed\";\n let rt1 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt1_values = rt1.block_on(async { (select_0_to_9().await, select_0_to_9().await) });\n\n let rt2 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt2_values = rt2.block_on(async { (select_0_to_9().await, select_0_to_9().await) });\n\n assert_eq!(rt1_values, rt2_values);\n }\n\n #[test]\n #[cfg(all(feature = \"rt-multi-thread\", not(target_os = \"wasi\")))]\n fn deterministic_select_multi_thread() {\n let seed = b\"bytes used to generate seed\";\n let (tx, rx) = std::sync::mpsc::channel();\n let rt1 = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .on_thread_park(move || tx.send(()).unwrap())\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n\n // This makes sure that `enter_runtime()` from worker thread is called before the one from main thread,\n // ensuring that the RNG state is consistent. See also https://github.com/tokio-rs/tokio/pull/7495.\n rx.recv().unwrap();\n\n let rt1_values = rt1.block_on(async {\n tokio::spawn(async { (select_0_to_9().await, select_0_to_9().await) })\n .await\n .unwrap()\n });\n\n let (tx, rx) = std::sync::mpsc::channel();\n let rt2 = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .on_thread_park(move || tx.send(()).unwrap())\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n\n // This makes sure that `enter_runtime()` from worker thread is called before the one from main thread,\n // ensuring that the RNG state is consistent. See also https://github.com/tokio-rs/tokio/pull/7495.\n rx.recv().unwrap();\n\n let rt2_values = rt2.block_on(async {\n tokio::spawn(async { (select_0_to_9().await, select_0_to_9().await) })\n .await\n .unwrap()\n });\n\n assert_eq!(rt1_values, rt2_values);\n }\n\n async fn select_0_to_9() -> u32 {\n tokio::select!(\n x = async { 0 } => x,\n x = async { 1 } => x,\n x = async { 2 } => x,\n x = async { 3 } => x,\n x = async { 4 } => x,\n x = async { 5 } => x,\n x = async { 6 } => x,\n x = async { 7 } => x,\n x = async { 8 } => x,\n x = async { 9 } => x,\n )\n }\n}\n\n#[tokio::test]\nasync fn select_into_future() {\n struct NotAFuture;\n impl std::future::IntoFuture for NotAFuture {\n type Output = ();\n type IntoFuture = std::future::Ready<()>;\n\n fn into_future(self) -> Self::IntoFuture {\n std::future::ready(())\n }\n }\n\n tokio::select! {\n () = NotAFuture => {},\n }\n}\n\n// regression test for https://github.com/tokio-rs/tokio/issues/6721\n#[tokio::test]\nasync fn temporary_lifetime_extension() {\n tokio::select! {\n () = &mut std::future::ready(()) => {},\n }\n}\n\n#[tokio::test]\nasync fn select_is_budget_aware() {\n const BUDGET: usize = 128;\n\n let task = || {\n Box::pin(async move {\n tokio::select! {\n biased;\n\n () = tokio::task::coop::consume_budget() => {},\n () = std::future::ready(()) => {}\n }\n })\n };\n\n for _ in 0..BUDGET {\n let poll = futures::poll!(&mut task());\n assert!(poll.is_ready());\n }\n\n let poll = futures::poll!(&mut task());\n assert!(poll.is_pending());\n}\n" }, { "path": "tokio/tests/macros_test.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threading\n\nuse tokio::test;\n\n#[test]\nasync fn test_macro_can_be_used_via_use() {\n tokio::spawn(async {}).await.unwrap();\n}\n\n#[tokio::test]\nasync fn test_macro_is_resilient_to_shadowing() {\n tokio::spawn(async {}).await.unwrap();\n}\n\n// https://github.com/tokio-rs/tokio/issues/3403\n#[rustfmt::skip] // this `rustfmt::skip` is necessary because unused_braces does not warn if the block contains newline.\n#[tokio::main]\npub async fn unused_braces_main() { println!(\"hello\") }\n#[rustfmt::skip] // this `rustfmt::skip` is necessary because unused_braces does not warn if the block contains newline.\n#[tokio::test]\nasync fn unused_braces_test() { assert_eq!(1 + 1, 2) }\n\n// https://github.com/tokio-rs/tokio/pull/3766#issuecomment-835508651\n#[std::prelude::v1::test]\nfn trait_method() {\n trait A {\n fn f(self);\n\n fn g(self);\n }\n impl A for () {\n #[tokio::main]\n async fn f(self) {\n self.g()\n }\n\n fn g(self) {}\n }\n ().f()\n}\n\n// https://github.com/tokio-rs/tokio/issues/4175\n#[tokio::main]\npub async fn issue_4175_main_1() -> ! {\n panic!();\n}\n#[tokio::main]\npub async fn issue_4175_main_2() -> std::io::Result<()> {\n panic!();\n}\n#[allow(unreachable_code)]\n#[tokio::test]\npub async fn issue_4175_test() -> std::io::Result<()> {\n return Ok(());\n panic!();\n}\n\n// https://github.com/tokio-rs/tokio/issues/4175\n#[allow(clippy::let_unit_value)]\npub mod clippy_semicolon_if_nothing_returned {\n #![deny(clippy::semicolon_if_nothing_returned)]\n\n #[tokio::main]\n pub async fn local() {\n let _x = ();\n }\n #[tokio::main]\n pub async fn item() {\n fn _f() {}\n }\n #[tokio::main]\n pub async fn semi() {\n panic!();\n }\n #[tokio::main]\n pub async fn empty() {\n // To trigger clippy::semicolon_if_nothing_returned lint, the block needs to contain newline.\n }\n}\n\n// https://github.com/tokio-rs/tokio/issues/5243\npub mod issue_5243 {\n macro_rules! mac {\n (async fn $name:ident() $b:block) => {\n #[::tokio::test]\n async fn $name() {\n $b\n }\n };\n }\n mac!(\n async fn foo() {}\n );\n}\n\n#[cfg(tokio_unstable)]\npub mod macro_rt_arg_unhandled_panic {\n use tokio_test::assert_err;\n\n #[tokio::test(flavor = \"current_thread\", unhandled_panic = \"shutdown_runtime\")]\n #[should_panic]\n async fn unhandled_panic_shutdown_runtime() {\n let _ = tokio::spawn(async {\n panic!(\"This panic should shutdown the runtime.\");\n })\n .await;\n }\n\n #[tokio::test(flavor = \"current_thread\", unhandled_panic = \"ignore\")]\n async fn unhandled_panic_ignore() {\n let rt = tokio::spawn(async {\n panic!(\"This panic should be forwarded to rt as an error.\");\n })\n .await;\n assert_err!(rt);\n }\n}\n" }, { "path": "tokio/tests/macros_try_join.rs", "content": "#![cfg(feature = \"macros\")]\n#![allow(clippy::disallowed_names)]\n\nuse std::{convert::Infallible, sync::Arc};\n\nuse tokio::sync::{oneshot, Semaphore};\nuse tokio_test::{assert_pending, assert_ready, task};\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_comma() {\n let foo = tokio::try_join!(async { ok(1) },);\n assert_eq!(foo, Ok((1,)));\n\n let foo = tokio::try_join!(biased; async { ok(1) },);\n assert_eq!(foo, Ok((1,)));\n}\n\n#[maybe_tokio_test]\nasync fn sync_one_lit_expr_no_comma() {\n let foo = tokio::try_join!(async { ok(1) });\n assert_eq!(foo, Ok((1,)));\n\n let foo = tokio::try_join!(biased; async { ok(1) });\n assert_eq!(foo, Ok((1,)));\n}\n\n#[maybe_tokio_test]\nasync fn sync_two_lit_expr_comma() {\n let foo = tokio::try_join!(async { ok(1) }, async { ok(2) },);\n assert_eq!(foo, Ok((1, 2)));\n\n let foo = tokio::try_join!(biased;async { ok(1) }, async { ok(2) },);\n assert_eq!(foo, Ok((1, 2)));\n}\n\n#[maybe_tokio_test]\nasync fn sync_two_lit_expr_no_comma() {\n let foo = tokio::try_join!(async { ok(1) }, async { ok(2) });\n assert_eq!(foo, Ok((1, 2)));\n\n let foo = tokio::try_join!(biased; async { ok(1) }, async { ok(2) });\n assert_eq!(foo, Ok((1, 2)));\n}\n\n#[maybe_tokio_test]\nasync fn two_await() {\n let (tx1, rx1) = oneshot::channel::<&str>();\n let (tx2, rx2) = oneshot::channel::();\n\n let mut join = task::spawn(async { tokio::try_join!(rx1, rx2) });\n\n assert_pending!(join.poll());\n\n tx2.send(123).unwrap();\n assert!(join.is_woken());\n assert_pending!(join.poll());\n\n tx1.send(\"hello\").unwrap();\n assert!(join.is_woken());\n let res: Result<(&str, u32), _> = assert_ready!(join.poll());\n\n assert_eq!(Ok((\"hello\", 123)), res);\n}\n\n#[maybe_tokio_test]\nasync fn err_abort_early() {\n let (tx1, rx1) = oneshot::channel::<&str>();\n let (tx2, rx2) = oneshot::channel::();\n let (_tx3, rx3) = oneshot::channel::();\n\n let mut join = task::spawn(async { tokio::try_join!(rx1, rx2, rx3) });\n\n assert_pending!(join.poll());\n\n tx2.send(123).unwrap();\n assert!(join.is_woken());\n assert_pending!(join.poll());\n\n drop(tx1);\n assert!(join.is_woken());\n\n let res = assert_ready!(join.poll());\n\n assert!(res.is_err());\n}\n\n#[test]\n#[cfg(target_pointer_width = \"64\")]\nfn try_join_size() {\n use futures::future;\n use std::mem;\n\n let fut = async {\n let ready = future::ready(ok(0i32));\n tokio::try_join!(ready)\n };\n assert_eq!(mem::size_of_val(&fut), 32);\n\n let fut = async {\n let ready1 = future::ready(ok(0i32));\n let ready2 = future::ready(ok(0i32));\n tokio::try_join!(ready1, ready2)\n };\n assert_eq!(mem::size_of_val(&fut), 48);\n}\n\nfn ok(val: T) -> Result {\n Ok(val)\n}\n\nasync fn non_cooperative_task(permits: Arc) -> Result {\n let mut exceeded_budget = 0;\n\n for _ in 0..5 {\n // Another task should run after this task uses its whole budget\n for _ in 0..128 {\n let _permit = permits.clone().acquire_owned().await.unwrap();\n }\n\n exceeded_budget += 1;\n }\n\n Ok(exceeded_budget)\n}\n\nasync fn poor_little_task(permits: Arc) -> Result {\n let mut how_many_times_i_got_to_run = 0;\n\n for _ in 0..5 {\n let _permit = permits.clone().acquire_owned().await.unwrap();\n\n how_many_times_i_got_to_run += 1;\n }\n\n Ok(how_many_times_i_got_to_run)\n}\n\n#[tokio::test]\nasync fn try_join_does_not_allow_tasks_to_starve() {\n let permits = Arc::new(Semaphore::new(10));\n\n // non_cooperative_task should yield after its budget is exceeded and then poor_little_task should run.\n let result = tokio::try_join!(\n non_cooperative_task(Arc::clone(&permits)),\n poor_little_task(permits)\n );\n\n let (non_cooperative_result, little_task_result) = result.unwrap();\n\n assert_eq!(5, non_cooperative_result);\n assert_eq!(5, little_task_result);\n}\n\n#[tokio::test]\nasync fn a_different_future_is_polled_first_every_time_poll_fn_is_polled() {\n let poll_order = Arc::new(std::sync::Mutex::new(vec![]));\n\n let fut = |x, poll_order: Arc>>| async move {\n for _ in 0..4 {\n {\n let mut guard = poll_order.lock().unwrap();\n\n guard.push(x);\n }\n\n tokio::task::yield_now().await;\n }\n Ok::<(), Infallible>(())\n };\n\n tokio::try_join!(\n fut(1, Arc::clone(&poll_order)),\n fut(2, Arc::clone(&poll_order)),\n fut(3, Arc::clone(&poll_order)),\n )\n .unwrap();\n\n // Each time the future created by join! is polled, it should start\n // by polling a different future first.\n assert_eq!(\n vec![1, 2, 3, 2, 3, 1, 3, 1, 2, 1, 2, 3],\n *poll_order.lock().unwrap()\n );\n}\n\n#[tokio::test]\nasync fn futures_are_polled_in_order_in_biased_mode() {\n let poll_order = Arc::new(std::sync::Mutex::new(vec![]));\n\n let fut = |x, poll_order: Arc>>| async move {\n for _ in 0..4 {\n {\n let mut guard = poll_order.lock().unwrap();\n\n guard.push(x);\n }\n\n tokio::task::yield_now().await;\n }\n Ok::<(), Infallible>(())\n };\n\n tokio::try_join!(\n biased;\n fut(1, Arc::clone(&poll_order)),\n fut(2, Arc::clone(&poll_order)),\n fut(3, Arc::clone(&poll_order)),\n )\n .unwrap();\n\n // Each time the future created by join! is polled, it should start\n // by polling in the order as declared in the macro inputs.\n assert_eq!(\n vec![1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3],\n *poll_order.lock().unwrap()\n );\n}\n\n#[test]\n#[cfg(target_pointer_width = \"64\")]\nfn try_join_size_biased() {\n use futures::future;\n use std::mem;\n\n let fut = async {\n let ready = future::ready(ok(0i32));\n tokio::try_join!(biased; ready)\n };\n assert_eq!(mem::size_of_val(&fut), 24);\n\n let fut = async {\n let ready1 = future::ready(ok(0i32));\n let ready2 = future::ready(ok(0i32));\n tokio::try_join!(biased; ready1, ready2)\n };\n assert_eq!(mem::size_of_val(&fut), 40);\n}\n\n#[tokio::test]\nasync fn empty_try_join() {\n assert_eq!(tokio::try_join!() as Result<_, ()>, Ok(()));\n assert_eq!(tokio::try_join!(biased;) as Result<_, ()>, Ok(()));\n}\n\n// Regression test for: https://github.com/tokio-rs/tokio/issues/7637\n// We want to make sure that the `const COUNT: u32` declaration\n// inside the macro body doesn't leak to the caller to cause compiler failures\n// or variable shadowing.\n#[tokio::test]\nasync fn caller_names_const_count() {\n let (tx, rx) = oneshot::channel::();\n\n const COUNT: u32 = 2;\n\n let mut try_join = task::spawn(async { tokio::try_join!(async { tx.send(COUNT) }) });\n assert_ready!(try_join.poll()).unwrap();\n\n let res = rx.await.unwrap();\n\n // This passing demonstrates that the const in the macro is\n // not shadowing the caller-specified COUNT value\n assert_eq!(2, res);\n}\n" }, { "path": "tokio/tests/net_bind_resource.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support panic recovery or bind\n\nuse tokio::net::TcpListener;\n\nuse std::net;\n\n#[test]\n#[should_panic]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn no_runtime_panics_binding_net_tcp_listener() {\n let listener = net::TcpListener::bind(\"127.0.0.1:0\").expect(\"failed to bind listener\");\n let _ = TcpListener::try_from(listener);\n}\n" }, { "path": "tokio/tests/net_lookup_host.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support direct socket operations\n\nuse tokio::net;\nuse tokio_test::assert_ok;\n\nuse std::io;\nuse std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};\n\n#[tokio::test]\nasync fn lookup_socket_addr() {\n let addr: SocketAddr = \"127.0.0.1:8000\".parse().unwrap();\n\n let actual = assert_ok!(net::lookup_host(addr).await).collect::>();\n assert_eq!(vec![addr], actual);\n}\n\n#[tokio::test]\nasync fn lookup_str_socket_addr() {\n let addr: SocketAddr = \"127.0.0.1:8000\".parse().unwrap();\n\n let actual = assert_ok!(net::lookup_host(\"127.0.0.1:8000\").await).collect::>();\n assert_eq!(vec![addr], actual);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `getaddrinfo` in miri.\nasync fn resolve_dns() -> io::Result<()> {\n let mut hosts = net::lookup_host(\"localhost:3000\").await?;\n let host = hosts.next().unwrap();\n\n let expected = if host.is_ipv4() {\n SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 3000)\n } else {\n SocketAddr::new(IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 1)), 3000)\n };\n assert_eq!(host, expected);\n\n Ok(())\n}\n" }, { "path": "tokio/tests/net_named_pipe.rs", "content": "#![cfg(feature = \"full\")]\n#![cfg(windows)]\n\nuse std::io;\nuse std::time::Duration;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::windows::named_pipe::{ClientOptions, PipeMode, ServerOptions};\nuse tokio::time;\nuse windows_sys::Win32::Foundation::{ERROR_NO_DATA, ERROR_PIPE_BUSY};\n\n#[tokio::test]\nasync fn test_named_pipe_client_drop() -> io::Result<()> {\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\test-named-pipe-client-drop\";\n\n let mut server = ServerOptions::new().create(PIPE_NAME)?;\n\n let client = ClientOptions::new().open(PIPE_NAME)?;\n\n server.connect().await?;\n drop(client);\n\n // instance will be broken because client is gone\n match server.write_all(b\"ping\").await {\n Err(e) if e.raw_os_error() == Some(ERROR_NO_DATA as i32) => (),\n x => panic!(\"{:?}\", x),\n }\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_named_pipe_single_client() -> io::Result<()> {\n use tokio::io::{AsyncBufReadExt as _, BufReader};\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\test-named-pipe-single-client\";\n\n let server = ServerOptions::new().create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n // Note: we wait for a client to connect.\n server.connect().await?;\n\n let mut server = BufReader::new(server);\n\n let mut buf = String::new();\n server.read_line(&mut buf).await?;\n server.write_all(b\"pong\\n\").await?;\n Ok::<_, io::Error>(buf)\n });\n\n let client = tokio::spawn(async move {\n let client = ClientOptions::new().open(PIPE_NAME)?;\n\n let mut client = BufReader::new(client);\n\n let mut buf = String::new();\n client.write_all(b\"ping\\n\").await?;\n client.read_line(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n });\n\n let (server, client) = tokio::try_join!(server, client)?;\n\n assert_eq!(server?, \"ping\\n\");\n assert_eq!(client?, \"pong\\n\");\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_named_pipe_multi_client() -> io::Result<()> {\n use tokio::io::{AsyncBufReadExt as _, BufReader};\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\test-named-pipe-multi-client\";\n const N: usize = 10;\n\n // The first server needs to be constructed early so that clients can\n // be correctly connected. Otherwise calling .wait will cause the client to\n // error.\n let mut server = ServerOptions::new().create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n for _ in 0..N {\n // Wait for client to connect.\n server.connect().await?;\n let mut inner = BufReader::new(server);\n\n // Construct the next server to be connected before sending the one\n // we already have of onto a task. This ensures that the server\n // isn't closed (after it's done in the task) before a new one is\n // available. Otherwise the client might error with\n // `io::ErrorKind::NotFound`.\n server = ServerOptions::new().create(PIPE_NAME)?;\n\n tokio::spawn(async move {\n let mut buf = String::new();\n inner.read_line(&mut buf).await?;\n inner.write_all(b\"pong\\n\").await?;\n inner.flush().await?;\n Ok::<_, io::Error>(())\n });\n }\n\n Ok::<_, io::Error>(())\n });\n\n let mut clients = Vec::new();\n\n for _ in 0..N {\n clients.push(tokio::spawn(async move {\n // This showcases a generic connect loop.\n //\n // We immediately try to create a client, if it's not found or the\n // pipe is busy we use the specialized wait function on the client\n // builder.\n let client = loop {\n match ClientOptions::new().open(PIPE_NAME) {\n Ok(client) => break client,\n Err(e) if e.raw_os_error() == Some(ERROR_PIPE_BUSY as i32) => (),\n Err(e) if e.kind() == io::ErrorKind::NotFound => (),\n Err(e) => return Err(e),\n }\n\n // Wait for a named pipe to become available.\n time::sleep(Duration::from_millis(10)).await;\n };\n\n let mut client = BufReader::new(client);\n\n let mut buf = String::new();\n client.write_all(b\"ping\\n\").await?;\n client.flush().await?;\n client.read_line(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n }));\n }\n\n for client in clients {\n let result = client.await?;\n assert_eq!(result?, \"pong\\n\");\n }\n\n server.await??;\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_named_pipe_multi_client_ready() -> io::Result<()> {\n use tokio::io::Interest;\n\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\test-named-pipe-multi-client-ready\";\n const N: usize = 10;\n\n // The first server needs to be constructed early so that clients can\n // be correctly connected. Otherwise calling .wait will cause the client to\n // error.\n let mut server = ServerOptions::new().create(PIPE_NAME)?;\n\n let server = tokio::spawn(async move {\n for _ in 0..N {\n // Wait for client to connect.\n server.connect().await?;\n\n let inner_server = server;\n\n // Construct the next server to be connected before sending the one\n // we already have of onto a task. This ensures that the server\n // isn't closed (after it's done in the task) before a new one is\n // available. Otherwise the client might error with\n // `io::ErrorKind::NotFound`.\n server = ServerOptions::new().create(PIPE_NAME)?;\n\n tokio::spawn(async move {\n let server = inner_server;\n\n {\n let mut read_buf = [0u8; 5];\n let mut read_buf_cursor = 0;\n\n loop {\n server.readable().await?;\n\n let buf = &mut read_buf[read_buf_cursor..];\n\n match server.try_read(buf) {\n Ok(n) => {\n read_buf_cursor += n;\n\n if read_buf_cursor == read_buf.len() {\n break;\n }\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n };\n\n {\n let write_buf = b\"pong\\n\";\n let mut write_buf_cursor = 0;\n\n loop {\n server.writable().await?;\n let buf = &write_buf[write_buf_cursor..];\n\n match server.try_write(buf) {\n Ok(n) => {\n write_buf_cursor += n;\n\n if write_buf_cursor == write_buf.len() {\n break;\n }\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n }\n\n Ok::<_, io::Error>(())\n });\n }\n\n Ok::<_, io::Error>(())\n });\n\n let mut clients = Vec::new();\n\n for _ in 0..N {\n clients.push(tokio::spawn(async move {\n // This showcases a generic connect loop.\n //\n // We immediately try to create a client, if it's not found or the\n // pipe is busy we use the specialized wait function on the client\n // builder.\n let client = loop {\n match ClientOptions::new().open(PIPE_NAME) {\n Ok(client) => break client,\n Err(e) if e.raw_os_error() == Some(ERROR_PIPE_BUSY as i32) => (),\n Err(e) if e.kind() == io::ErrorKind::NotFound => (),\n Err(e) => return Err(e),\n }\n\n // Wait for a named pipe to become available.\n time::sleep(Duration::from_millis(10)).await;\n };\n\n let mut read_buf = [0u8; 5];\n let mut read_buf_cursor = 0;\n let write_buf = b\"ping\\n\";\n let mut write_buf_cursor = 0;\n\n loop {\n let mut interest = Interest::READABLE;\n if write_buf_cursor < write_buf.len() {\n interest |= Interest::WRITABLE;\n }\n\n let ready = client.ready(interest).await?;\n\n if ready.is_readable() {\n let buf = &mut read_buf[read_buf_cursor..];\n\n match client.try_read(buf) {\n Ok(n) => {\n read_buf_cursor += n;\n\n if read_buf_cursor == read_buf.len() {\n break;\n }\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n\n if ready.is_writable() {\n let buf = &write_buf[write_buf_cursor..];\n\n if buf.is_empty() {\n continue;\n }\n\n match client.try_write(buf) {\n Ok(n) => {\n write_buf_cursor += n;\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n continue;\n }\n Err(e) => {\n return Err(e);\n }\n }\n }\n }\n\n let buf = String::from_utf8_lossy(&read_buf).into_owned();\n\n Ok::<_, io::Error>(buf)\n }));\n }\n\n for client in clients {\n let result = client.await?;\n assert_eq!(result?, \"pong\\n\");\n }\n\n server.await??;\n Ok(())\n}\n\n// This tests that message mode works as expected.\n#[tokio::test]\nasync fn test_named_pipe_mode_message() -> io::Result<()> {\n // it's easy to accidentally get a seemingly working test here because byte pipes\n // often return contents at write boundaries. to make sure we're doing the right thing we\n // explicitly test that it doesn't work in byte mode.\n _named_pipe_mode_message(PipeMode::Message).await?;\n _named_pipe_mode_message(PipeMode::Byte).await\n}\n\nasync fn _named_pipe_mode_message(mode: PipeMode) -> io::Result<()> {\n let pipe_name = format!(\n r\"\\\\.\\pipe\\test-named-pipe-mode-message-{}\",\n matches!(mode, PipeMode::Message)\n );\n let mut buf = [0u8; 32];\n\n let mut server = ServerOptions::new()\n .first_pipe_instance(true)\n .pipe_mode(mode)\n .create(&pipe_name)?;\n\n let mut client = ClientOptions::new().pipe_mode(mode).open(&pipe_name)?;\n\n server.connect().await?;\n\n // this needs a few iterations, presumably Windows waits for a few calls before merging buffers\n for _ in 0..10 {\n client.write_all(b\"hello\").await?;\n server.write_all(b\"world\").await?;\n }\n for _ in 0..10 {\n let n = server.read(&mut buf).await?;\n if buf[..n] != b\"hello\"[..] {\n assert!(matches!(mode, PipeMode::Byte));\n return Ok(());\n }\n let n = client.read(&mut buf).await?;\n if buf[..n] != b\"world\"[..] {\n assert!(matches!(mode, PipeMode::Byte));\n return Ok(());\n }\n }\n // byte mode should have errored before.\n assert!(matches!(mode, PipeMode::Message));\n Ok(())\n}\n\n// This tests `NamedPipeServer::connect` with various access settings.\n#[tokio::test]\nasync fn test_named_pipe_access() -> io::Result<()> {\n const PIPE_NAME: &str = r\"\\\\.\\pipe\\test-named-pipe-access\";\n\n for (inb, outb) in [(true, true), (true, false), (false, true)] {\n let (tx, rx) = tokio::sync::oneshot::channel();\n let server = tokio::spawn(async move {\n let s = ServerOptions::new()\n .access_inbound(inb)\n .access_outbound(outb)\n .create(PIPE_NAME)?;\n let mut connect_fut = tokio_test::task::spawn(s.connect());\n assert!(connect_fut.poll().is_pending());\n tx.send(()).unwrap();\n connect_fut.await\n });\n\n // Wait for the server to call connect.\n rx.await.unwrap();\n let _ = ClientOptions::new().read(outb).write(inb).open(PIPE_NAME)?;\n\n server.await??;\n }\n Ok(())\n}\n" }, { "path": "tokio/tests/net_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))]\n#![cfg(panic = \"unwind\")]\n\nuse std::error::Error;\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::runtime::{Builder, Runtime};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn udp_socket_from_std_panic_caller() -> Result<(), Box> {\n use std::net::SocketAddr;\n use tokio::net::UdpSocket;\n\n let addr = \"127.0.0.1:0\".parse::().unwrap();\n let std_sock = std::net::UdpSocket::bind(addr).unwrap();\n std_sock.set_nonblocking(true).unwrap();\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _sock = UdpSocket::from_std(std_sock);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn tcp_listener_from_std_panic_caller() -> Result<(), Box> {\n let std_listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n std_listener.set_nonblocking(true).unwrap();\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = TcpListener::from_std(std_listener);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn tcp_stream_from_std_panic_caller() -> Result<(), Box> {\n let std_listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n let std_stream = std::net::TcpStream::connect(std_listener.local_addr().unwrap()).unwrap();\n std_stream.set_nonblocking(true).unwrap();\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = TcpStream::from_std(std_stream);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn unix_listener_bind_panic_caller() -> Result<(), Box> {\n use tokio::net::UnixListener;\n\n let dir = tempfile::tempdir().unwrap();\n let sock_path = dir.path().join(\"socket\");\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = UnixListener::bind(&sock_path);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn unix_listener_from_std_panic_caller() -> Result<(), Box> {\n use tokio::net::UnixListener;\n\n let dir = tempfile::tempdir().unwrap();\n let sock_path = dir.path().join(\"socket\");\n let std_listener = std::os::unix::net::UnixListener::bind(sock_path).unwrap();\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = UnixListener::from_std(std_listener);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn unix_stream_from_std_panic_caller() -> Result<(), Box> {\n use tokio::net::UnixStream;\n\n let dir = tempfile::tempdir().unwrap();\n let sock_path = dir.path().join(\"socket\");\n let _std_listener = std::os::unix::net::UnixListener::bind(&sock_path).unwrap();\n let std_stream = std::os::unix::net::UnixStream::connect(&sock_path).unwrap();\n\n let panic_location_file = test_panic(|| {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = UnixStream::from_std(std_stream);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(unix)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn unix_datagram_from_std_panic_caller() -> Result<(), Box> {\n use std::os::unix::net::UnixDatagram as StdUDS;\n use tokio::net::UnixDatagram;\n\n let dir = tempfile::tempdir().unwrap();\n let sock_path = dir.path().join(\"socket\");\n\n // Bind the socket to a filesystem path\n // /let socket_path = tmp.path().join(\"socket\");\n let std_socket = StdUDS::bind(sock_path).unwrap();\n std_socket.set_nonblocking(true).unwrap();\n\n let panic_location_file = test_panic(move || {\n let rt = runtime_without_io();\n rt.block_on(async {\n let _ = UnixDatagram::from_std(std_socket);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg(windows)]\nfn server_options_max_instances_panic_caller() -> Result<(), Box> {\n use tokio::net::windows::named_pipe::ServerOptions;\n\n let panic_location_file = test_panic(move || {\n let rt = runtime_without_io();\n rt.block_on(async {\n let mut options = ServerOptions::new();\n options.max_instances(255);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n// Runtime without `enable_io` so it has no IO driver set.\nfn runtime_without_io() -> Runtime {\n Builder::new_current_thread().build().unwrap()\n}\n" }, { "path": "tokio/tests/net_quickack.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(\n feature = \"net\",\n any(\n target_os = \"linux\",\n target_os = \"android\",\n target_os = \"fuchsia\",\n target_os = \"cygwin\",\n )\n))]\n#![cfg(not(miri))] // No `socket` in miri.\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::TcpListener;\nuse tokio::net::TcpStream;\nuse tokio::sync::oneshot;\n\n#[tokio::test]\nasync fn socket_works_with_quickack() {\n const MESSAGE: &str = \"Hello, tokio!\";\n\n let (tx_port, rx_port) = oneshot::channel();\n\n let server = tokio::spawn(async move {\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = listener.local_addr().unwrap();\n\n tx_port.send(addr.port()).unwrap();\n\n let (mut stream, _) = listener.accept().await.unwrap();\n stream.set_quickack(true).unwrap();\n assert!(stream.quickack().unwrap());\n\n stream.write_all(MESSAGE.as_bytes()).await.unwrap();\n\n let mut buf = vec![0; MESSAGE.len()];\n stream.read_exact(&mut buf).await.unwrap();\n assert_eq!(buf, MESSAGE.as_bytes());\n\n // There is nothing special about setting quickack to false\n // at this point, we just want to test the `false` case.\n stream.set_quickack(false).unwrap();\n assert!(!stream.quickack().unwrap());\n\n stream.shutdown().await.unwrap();\n });\n\n let port = rx_port.await.unwrap();\n let client = tokio::spawn(async move {\n let mut stream = TcpStream::connect(format!(\"127.0.0.1:{port}\"))\n .await\n .unwrap();\n stream.set_quickack(true).unwrap();\n assert!(stream.quickack().unwrap());\n\n let mut buf = vec![0; MESSAGE.len()];\n stream.read_exact(&mut buf).await.unwrap();\n assert_eq!(buf, MESSAGE.as_bytes());\n\n stream.write_all(MESSAGE.as_bytes()).await.unwrap();\n\n // There is nothing special about setting quickack to false\n // at this point, we just want to test the `false` case.\n stream.set_quickack(false).unwrap();\n assert!(!stream.quickack().unwrap());\n\n stream.shutdown().await.unwrap();\n });\n\n tokio::try_join!(server, client).unwrap();\n}\n" }, { "path": "tokio/tests/net_unix_pipe.rs", "content": "#![cfg(feature = \"full\")]\n#![cfg(unix)]\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt, Interest};\nuse tokio::net::unix::pipe;\nuse tokio_test::task;\nuse tokio_test::{assert_err, assert_ok, assert_pending, assert_ready_ok};\n\nuse std::fs::File;\nuse std::io;\nuse std::os::unix::fs::OpenOptionsExt;\nuse std::os::unix::io::AsRawFd;\nuse std::path::{Path, PathBuf};\n\n/// Helper struct which will clean up temporary files once dropped.\nstruct TempFifo {\n path: PathBuf,\n _dir: tempfile::TempDir,\n}\n\nimpl TempFifo {\n fn new(name: &str) -> io::Result {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-fifo-tests\")\n .tempdir()?;\n let path = dir.path().join(name);\n nix::unistd::mkfifo(&path, nix::sys::stat::Mode::S_IRWXU)?;\n\n Ok(TempFifo { path, _dir: dir })\n }\n}\n\nimpl AsRef for TempFifo {\n fn as_ref(&self) -> &Path {\n self.path.as_ref()\n }\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn fifo_simple_send() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n let fifo = TempFifo::new(\"simple_send\")?;\n\n // Create a reading task which should wait for data from the pipe.\n let mut reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let mut read_fut = task::spawn(async move {\n let mut buf = vec![0; DATA.len()];\n reader.read_exact(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n });\n assert_pending!(read_fut.poll());\n\n let mut writer = pipe::OpenOptions::new().open_sender(&fifo)?;\n writer.write_all(DATA).await?;\n\n // Let the IO driver poll events for the reader.\n while !read_fut.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Reading task should be ready now.\n let read_data = assert_ready_ok!(read_fut.poll());\n assert_eq!(&read_data, DATA);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn fifo_simple_send_sender_first() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n // Create a new fifo file with *no reading ends open*.\n let fifo = TempFifo::new(\"simple_send_sender_first\")?;\n\n // Simple `open_sender` should fail with ENXIO (no such device or address).\n let err = assert_err!(pipe::OpenOptions::new().open_sender(&fifo));\n assert_eq!(err.raw_os_error(), Some(libc::ENXIO));\n\n // `open_sender` in read-write mode should succeed and the pipe should be ready to write.\n let mut writer = pipe::OpenOptions::new()\n .read_write(true)\n .open_sender(&fifo)?;\n writer.write_all(DATA).await?;\n\n // Read the written data and validate.\n let mut reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let mut read_data = vec![0; DATA.len()];\n reader.read_exact(&mut read_data).await?;\n assert_eq!(&read_data, DATA);\n\n Ok(())\n}\n\n// Opens a FIFO file, write and *close the writer*.\nasync fn write_and_close(path: impl AsRef, msg: &[u8]) -> io::Result<()> {\n let mut writer = pipe::OpenOptions::new().open_sender(path)?;\n writer.write_all(msg).await?;\n drop(writer); // Explicit drop.\n Ok(())\n}\n\n/// Checks EOF behavior with single reader and writers sequentially opening\n/// and closing a FIFO.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn fifo_multiple_writes() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n let fifo = TempFifo::new(\"fifo_multiple_writes\")?;\n\n let mut reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n\n write_and_close(&fifo, DATA).await?;\n let ev = reader.ready(Interest::READABLE).await?;\n assert!(ev.is_readable());\n let mut read_data = vec![0; DATA.len()];\n assert_ok!(reader.read_exact(&mut read_data).await);\n\n // Check that reader hits EOF.\n let err = assert_err!(reader.read_exact(&mut read_data).await);\n assert_eq!(err.kind(), io::ErrorKind::UnexpectedEof);\n\n // Write more data and read again.\n write_and_close(&fifo, DATA).await?;\n assert_ok!(reader.read_exact(&mut read_data).await);\n\n Ok(())\n}\n\n/// Checks behavior of a resilient reader (Receiver in O_RDWR access mode)\n/// with writers sequentially opening and closing a FIFO.\n#[tokio::test]\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn fifo_resilient_reader() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n let fifo = TempFifo::new(\"fifo_resilient_reader\")?;\n\n // Open reader in read-write access mode.\n let mut reader = pipe::OpenOptions::new()\n .read_write(true)\n .open_receiver(&fifo)?;\n\n write_and_close(&fifo, DATA).await?;\n let ev = reader.ready(Interest::READABLE).await?;\n let mut read_data = vec![0; DATA.len()];\n reader.read_exact(&mut read_data).await?;\n\n // Check that reader didn't hit EOF.\n assert!(!ev.is_read_closed());\n\n // Resilient reader can asynchronously wait for the next writer.\n let mut second_read_fut = task::spawn(reader.read_exact(&mut read_data));\n assert_pending!(second_read_fut.poll());\n\n // Write more data and read again.\n write_and_close(&fifo, DATA).await?;\n assert_ok!(second_read_fut.await);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `O_NONBLOCK` for open64 in miri.\nasync fn open_detects_not_a_fifo() -> io::Result<()> {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-fifo-tests\")\n .tempdir()\n .unwrap();\n let path = dir.path().join(\"not_a_fifo\");\n\n // Create an ordinary file.\n File::create(&path)?;\n\n // Check if Sender detects invalid file type.\n let err = assert_err!(pipe::OpenOptions::new().open_sender(&path));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n // Check if Receiver detects invalid file type.\n let err = assert_err!(pipe::OpenOptions::new().open_sender(&path));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn from_file() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n let fifo = TempFifo::new(\"from_file\")?;\n\n // Construct a Receiver from a File.\n let file = std::fs::OpenOptions::new()\n .read(true)\n .custom_flags(libc::O_NONBLOCK)\n .open(&fifo)?;\n let mut reader = pipe::Receiver::from_file(file)?;\n\n // Construct a Sender from a File.\n let file = std::fs::OpenOptions::new()\n .write(true)\n .custom_flags(libc::O_NONBLOCK)\n .open(&fifo)?;\n let mut writer = pipe::Sender::from_file(file)?;\n\n // Write and read some data to test async.\n let mut read_fut = task::spawn(async move {\n let mut buf = vec![0; DATA.len()];\n reader.read_exact(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n });\n assert_pending!(read_fut.poll());\n\n writer.write_all(DATA).await?;\n\n let read_data = assert_ok!(read_fut.await);\n assert_eq!(&read_data, DATA);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `fstat` in miri.\nasync fn from_file_detects_not_a_fifo() -> io::Result<()> {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-fifo-tests\")\n .tempdir()\n .unwrap();\n let path = dir.path().join(\"not_a_fifo\");\n\n // Create an ordinary file.\n File::create(&path)?;\n\n // Check if Sender detects invalid file type.\n let file = std::fs::OpenOptions::new().write(true).open(&path)?;\n let err = assert_err!(pipe::Sender::from_file(file));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n // Check if Receiver detects invalid file type.\n let file = std::fs::OpenOptions::new().read(true).open(&path)?;\n let err = assert_err!(pipe::Receiver::from_file(file));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn from_file_detects_wrong_access_mode() -> io::Result<()> {\n let fifo = TempFifo::new(\"wrong_access_mode\")?;\n\n // Open a read end to open the fifo for writing.\n let _reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n\n // Check if Receiver detects write-only access mode.\n let write_only = std::fs::OpenOptions::new()\n .write(true)\n .custom_flags(libc::O_NONBLOCK)\n .open(&fifo)?;\n let err = assert_err!(pipe::Receiver::from_file(write_only));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n // Check if Sender detects read-only access mode.\n let rdonly = std::fs::OpenOptions::new()\n .read(true)\n .custom_flags(libc::O_NONBLOCK)\n .open(&fifo)?;\n let err = assert_err!(pipe::Sender::from_file(rdonly));\n assert_eq!(err.kind(), io::ErrorKind::InvalidInput);\n\n Ok(())\n}\n\nfn is_nonblocking(fd: &T) -> io::Result {\n let flags = nix::fcntl::fcntl(fd.as_raw_fd(), nix::fcntl::F_GETFL)?;\n Ok((flags & libc::O_NONBLOCK) != 0)\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn from_file_sets_nonblock() -> io::Result<()> {\n let fifo = TempFifo::new(\"sets_nonblock\")?;\n\n // Open read and write ends to let blocking files open.\n let _reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let _writer = pipe::OpenOptions::new().open_sender(&fifo)?;\n\n // Check if Receiver sets the pipe in non-blocking mode.\n let rdonly = std::fs::OpenOptions::new().read(true).open(&fifo)?;\n assert!(!is_nonblocking(&rdonly)?);\n let reader = pipe::Receiver::from_file(rdonly)?;\n assert!(is_nonblocking(&reader)?);\n\n // Check if Sender sets the pipe in non-blocking mode.\n let write_only = std::fs::OpenOptions::new().write(true).open(&fifo)?;\n assert!(!is_nonblocking(&write_only)?);\n let writer = pipe::Sender::from_file(write_only)?;\n assert!(is_nonblocking(&writer)?);\n\n Ok(())\n}\n\nfn writable_by_poll(writer: &pipe::Sender) -> bool {\n task::spawn(writer.writable()).poll().is_ready()\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn try_read_write() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n // Create a pipe pair over a fifo file.\n let fifo = TempFifo::new(\"try_read_write\")?;\n let reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let writer = pipe::OpenOptions::new().open_sender(&fifo)?;\n\n // Fill the pipe buffer with `try_write`.\n let mut write_data = Vec::new();\n while writable_by_poll(&writer) {\n match writer.try_write(DATA) {\n Ok(n) => write_data.extend(&DATA[..n]),\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n\n // Drain the pipe buffer with `try_read`.\n let mut read_data = vec![0; write_data.len()];\n let mut i = 0;\n while i < write_data.len() {\n reader.readable().await?;\n match reader.try_read(&mut read_data[i..]) {\n Ok(n) => i += n,\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n continue;\n }\n }\n }\n\n assert_eq!(read_data, write_data);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn try_read_write_vectored() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n // Create a pipe pair over a fifo file.\n let fifo = TempFifo::new(\"try_read_write_vectored\")?;\n let reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let writer = pipe::OpenOptions::new().open_sender(&fifo)?;\n\n let write_bufs: Vec<_> = DATA.chunks(3).map(io::IoSlice::new).collect();\n\n // Fill the pipe buffer with `try_write_vectored`.\n let mut write_data = Vec::new();\n while writable_by_poll(&writer) {\n match writer.try_write_vectored(&write_bufs) {\n Ok(n) => write_data.extend(&DATA[..n]),\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n\n // Drain the pipe buffer with `try_read_vectored`.\n let mut read_data = vec![0; write_data.len()];\n let mut i = 0;\n while i < write_data.len() {\n reader.readable().await?;\n\n let mut read_bufs: Vec<_> = read_data[i..]\n .chunks_mut(0x10000)\n .map(io::IoSliceMut::new)\n .collect();\n match reader.try_read_vectored(&mut read_bufs) {\n Ok(n) => i += n,\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n continue;\n }\n }\n }\n\n assert_eq!(read_data, write_data);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `mkfifo` in miri.\nasync fn try_read_buf() -> std::io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the fifo\";\n\n // Create a pipe pair over a fifo file.\n let fifo = TempFifo::new(\"try_read_write_vectored\")?;\n let reader = pipe::OpenOptions::new().open_receiver(&fifo)?;\n let writer = pipe::OpenOptions::new().open_sender(&fifo)?;\n\n // Fill the pipe buffer with `try_write`.\n let mut write_data = Vec::new();\n while writable_by_poll(&writer) {\n match writer.try_write(DATA) {\n Ok(n) => write_data.extend(&DATA[..n]),\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n\n // Drain the pipe buffer with `try_read_buf`.\n let mut read_data = vec![0; write_data.len()];\n let mut i = 0;\n while i < write_data.len() {\n reader.readable().await?;\n match reader.try_read_buf(&mut read_data) {\n Ok(n) => i += n,\n Err(e) => {\n assert_eq!(e.kind(), io::ErrorKind::WouldBlock);\n continue;\n }\n }\n }\n\n assert_eq!(read_data, write_data);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn anon_pipe_simple_send() -> io::Result<()> {\n const DATA: &[u8] = b\"this is some data to write to the pipe\";\n\n let (mut writer, mut reader) = pipe::pipe()?;\n\n // Create a reading task which should wait for data from the pipe.\n let mut read_fut = task::spawn(async move {\n let mut buf = vec![0; DATA.len()];\n reader.read_exact(&mut buf).await?;\n Ok::<_, io::Error>(buf)\n });\n assert_pending!(read_fut.poll());\n\n writer.write_all(DATA).await?;\n\n // Let the IO driver poll events for the reader.\n while !read_fut.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Reading task should be ready now.\n let read_data = assert_ready_ok!(read_fut.poll());\n assert_eq!(&read_data, DATA);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `pidfd_spawnp` in miri.\nasync fn anon_pipe_spawn_echo() -> std::io::Result<()> {\n use tokio::process::Command;\n\n const DATA: &str = \"this is some data to write to the pipe\";\n\n let (tx, mut rx) = pipe::pipe()?;\n\n let status = Command::new(\"echo\")\n .arg(\"-n\")\n .arg(DATA)\n .stdout(tx.into_blocking_fd()?)\n .status();\n\n let mut buf = vec![0; DATA.len()];\n rx.read_exact(&mut buf).await?;\n assert_eq!(String::from_utf8(buf).unwrap(), DATA);\n\n let exit_code = status.await?;\n assert!(exit_code.success());\n\n // Check if the pipe is closed.\n buf = Vec::new();\n let total = assert_ok!(rx.try_read(&mut buf));\n assert_eq!(total, 0);\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg(target_os = \"linux\")]\n#[cfg_attr(miri, ignore)] // No `fstat` in miri.\nasync fn anon_pipe_from_owned_fd() -> std::io::Result<()> {\n use nix::fcntl::OFlag;\n\n const DATA: &[u8] = b\"this is some data to write to the pipe\";\n\n let (rx_fd, tx_fd) = nix::unistd::pipe2(OFlag::O_CLOEXEC | OFlag::O_NONBLOCK)?;\n\n let mut rx = pipe::Receiver::from_owned_fd(rx_fd)?;\n let mut tx = pipe::Sender::from_owned_fd(tx_fd)?;\n\n let mut buf = vec![0; DATA.len()];\n tx.write_all(DATA).await?;\n rx.read_exact(&mut buf).await?;\n assert_eq!(buf, DATA);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn anon_pipe_into_nonblocking_fd() -> std::io::Result<()> {\n let (tx, rx) = pipe::pipe()?;\n\n let tx_fd = tx.into_nonblocking_fd()?;\n let rx_fd = rx.into_nonblocking_fd()?;\n\n assert!(is_nonblocking(&tx_fd)?);\n assert!(is_nonblocking(&rx_fd)?);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn anon_pipe_into_blocking_fd() -> std::io::Result<()> {\n let (tx, rx) = pipe::pipe()?;\n\n let tx_fd = tx.into_blocking_fd()?;\n let rx_fd = rx.into_blocking_fd()?;\n\n assert!(!is_nonblocking(&tx_fd)?);\n assert!(!is_nonblocking(&rx_fd)?);\n\n Ok(())\n}\n" }, { "path": "tokio/tests/no_rt.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n\nuse tokio::net::TcpStream;\nuse tokio::sync::oneshot;\nuse tokio::time::{timeout, Duration};\n\nuse futures::executor::block_on;\n\nuse std::net::TcpListener;\n\n#[test]\n#[should_panic(\n expected = \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\"\n)]\nfn timeout_panics_when_no_tokio_context() {\n block_on(timeout_value());\n}\n\n#[test]\n#[should_panic(\n expected = \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\"\n)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn panics_when_no_reactor() {\n let srv = TcpListener::bind(\"127.0.0.1:0\").unwrap();\n let addr = srv.local_addr().unwrap();\n block_on(TcpStream::connect(&addr)).unwrap();\n}\n\nasync fn timeout_value() {\n let (_tx, rx) = oneshot::channel::<()>();\n let dur = Duration::from_millis(10);\n let _ = timeout(dur, rx).await;\n}\n\n#[test]\n#[should_panic(\n expected = \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\"\n)]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn io_panics_when_no_tokio_context() {\n let listener = std::net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n listener.set_nonblocking(true).unwrap();\n\n let _ = tokio::net::TcpListener::from_std(listener);\n}\n" }, { "path": "tokio/tests/process_arg0.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", unix, not(miri)))]\n\nuse tokio::process::Command;\n\n#[tokio::test]\nasync fn arg0() {\n let mut cmd = Command::new(\"sh\");\n cmd.arg0(\"test_string\").arg(\"-c\").arg(\"echo $0\");\n\n let output = cmd.output().await.unwrap();\n assert_eq!(output.stdout, b\"test_string\\n\");\n}\n" }, { "path": "tokio/tests/process_change_of_runtime.rs", "content": "#![cfg(feature = \"process\")]\n#![warn(rust_2018_idioms)]\n// This tests test the behavior of `process::Command::spawn` when it is used\n// outside runtime, and when `process::Child::wait ` is used in a different\n// runtime from which `process::Command::spawn` is used.\n#![cfg(all(unix, not(target_os = \"freebsd\"), not(miri)))] // Miri cannot run system commands\n\nuse std::process::Stdio;\nuse tokio::{process::Command, runtime::Runtime};\n\n#[test]\nfn process_spawned_and_wait_in_different_runtime() {\n let mut child = Runtime::new().unwrap().block_on(async {\n Command::new(\"true\")\n .stdin(Stdio::piped())\n .stdout(Stdio::null())\n .spawn()\n .unwrap()\n });\n Runtime::new().unwrap().block_on(async {\n let _ = child.wait().await.unwrap();\n });\n}\n\n#[test]\n#[should_panic(\n expected = \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\"\n)]\nfn process_spawned_outside_runtime() {\n let _ = Command::new(\"true\")\n .stdin(Stdio::piped())\n .stdout(Stdio::null())\n .spawn();\n}\n" }, { "path": "tokio/tests/process_issue_2174.rs", "content": "#![cfg(feature = \"process\")]\n#![warn(rust_2018_idioms)]\n// This test reveals a difference in behavior of kqueue on FreeBSD. When the\n// reader disconnects, there does not seem to be an `EVFILT_WRITE` filter that\n// is returned.\n//\n// It is expected that `EVFILT_WRITE` would be returned with either the\n// `EV_EOF` or `EV_ERROR` flag set. If either flag is set a write would be\n// attempted, but that does not seem to occur.\n#![cfg(all(unix, not(target_os = \"freebsd\"), not(miri)))]\n\nuse std::process::Stdio;\nuse std::time::Duration;\nuse tokio::io::AsyncWriteExt;\nuse tokio::process::Command;\nuse tokio::time;\nuse tokio_test::assert_err;\n\n#[tokio::test]\n#[cfg_attr(panic = \"abort\", ignore)]\nasync fn issue_2174() {\n let mut child = Command::new(\"sleep\")\n .arg(\"2\")\n .stdin(Stdio::piped())\n .stdout(Stdio::null())\n .spawn()\n .unwrap();\n let mut input = child.stdin.take().unwrap();\n\n // Writes will buffer up to 65_636. This *should* loop at least 8 times\n // and then register interest.\n let handle = tokio::spawn(async move {\n let data = [0u8; 8192];\n loop {\n input.write_all(&data).await.unwrap();\n }\n });\n\n // Sleep enough time so that the child process's stdin's buffer fills.\n time::sleep(Duration::from_secs(1)).await;\n\n // Kill the child process.\n child.kill().await.unwrap();\n\n assert_err!(handle.await);\n}\n" }, { "path": "tokio/tests/process_issue_42.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))]\n\nuse futures::future::join_all;\nuse std::process::Stdio;\nuse tokio::process::Command;\nuse tokio::task;\n\n#[tokio::test]\nasync fn issue_42() {\n // We spawn a many batches of processes which should exit at roughly the\n // same time (modulo OS scheduling delays), to make sure that consuming\n // a readiness event for one process doesn't inadvertently starve another.\n // We then do this many times (in parallel) in an effort to stress test the\n // implementation to ensure there are no race conditions.\n // See alexcrichton/tokio-process#42 for background\n let join_handles = (0..10usize).map(|_| {\n task::spawn(async {\n let processes = (0..10usize).map(|i| {\n let mut child = Command::new(\"echo\")\n .arg(format!(\"I am spawned process #{i}\"))\n .stdin(Stdio::null())\n .stdout(Stdio::null())\n .stderr(Stdio::null())\n .kill_on_drop(true)\n .spawn()\n .unwrap();\n\n async move { child.wait().await }\n });\n\n join_all(processes).await;\n })\n });\n\n join_all(join_handles).await;\n}\n" }, { "path": "tokio/tests/process_issue_7144.rs", "content": "#![cfg(feature = \"process\")]\n#![warn(rust_2018_idioms)]\n#![cfg(target_os = \"linux\")]\n#![cfg(not(miri))]\n\nuse tokio::process::Command;\nuse tokio::time::{sleep, Duration};\n\n#[tokio::test]\nasync fn issue_7144() {\n let mut threads = vec![];\n for _ in 0..20 {\n threads.push(tokio::spawn(test_one()));\n }\n for thread in threads {\n thread.await.unwrap();\n }\n}\n\nasync fn has_strace() -> bool {\n Command::new(\"strace\").arg(\"-V\").output().await.is_ok()\n}\n\nasync fn test_one() {\n if !has_strace().await {\n return;\n }\n\n let mut t = Command::new(\"strace\")\n .args(\"-o /dev/null -D sleep 5\".split(' '))\n .spawn()\n .unwrap();\n sleep(Duration::from_millis(100)).await;\n unsafe { libc::kill(t.id().unwrap() as _, libc::SIGINT) };\n t.wait().await.unwrap();\n}\n" }, { "path": "tokio/tests/process_kill_after_wait.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi/Miri cannot run system commands\n\nuse tokio::process::Command;\n\n#[tokio::test]\nasync fn kill_after_wait() {\n let mut cmd;\n\n if cfg!(windows) {\n cmd = Command::new(\"cmd\");\n cmd.arg(\"/c\");\n } else {\n cmd = Command::new(\"sh\");\n cmd.arg(\"-c\");\n }\n\n let mut child = cmd.arg(\"exit 2\").spawn().unwrap();\n\n child.start_kill().unwrap();\n\n child.wait().await.unwrap();\n\n // Kill after `wait` is fine.\n child.start_kill().unwrap();\n child.kill().await.unwrap();\n}\n" }, { "path": "tokio/tests/process_kill_on_drop.rs", "content": "#![cfg(all(unix, feature = \"process\", not(miri)))]\n#![warn(rust_2018_idioms)]\n\nuse std::io::ErrorKind;\nuse std::process::Stdio;\nuse std::time::Duration;\nuse tokio::io::AsyncReadExt;\nuse tokio::process::Command;\nuse tokio::time::sleep;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn kill_on_drop() {\n let mut cmd = Command::new(\"bash\");\n cmd.args([\n \"-c\",\n \"\n # Fork another child that won't get killed\n sh -c 'sleep 1; echo child ran' &\n disown -a\n\n # Await our death\n sleep 5\n echo hello from beyond the grave\n \",\n ]);\n\n let e = cmd.kill_on_drop(true).stdout(Stdio::piped()).spawn();\n if e.is_err() && e.as_ref().unwrap_err().kind() == ErrorKind::NotFound {\n println!(\"bash not available; skipping test\");\n return;\n }\n let mut child = e.unwrap();\n\n sleep(Duration::from_secs(2)).await;\n\n let mut out = child.stdout.take().unwrap();\n drop(child);\n\n let mut msg = String::new();\n assert_ok!(out.read_to_string(&mut msg).await);\n\n assert_eq!(\"child ran\\n\", msg);\n}\n" }, { "path": "tokio/tests/process_raw_handle.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(windows)]\n#![cfg(not(miri))]\n\nuse tokio::process::Command;\nuse windows_sys::Win32::System::Threading::GetProcessId;\n\n#[tokio::test]\nasync fn obtain_raw_handle() {\n let mut cmd = Command::new(\"cmd\");\n cmd.kill_on_drop(true);\n cmd.arg(\"/c\");\n cmd.arg(\"pause\");\n\n let child = cmd.spawn().unwrap();\n\n let orig_id = child.id().expect(\"missing id\");\n assert!(orig_id > 0);\n\n let handle = child.raw_handle().expect(\"process stopped\");\n let handled_id = unsafe { GetProcessId(handle as _) };\n assert_eq!(handled_id, orig_id);\n}\n" }, { "path": "tokio/tests/process_smoke.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi/Miri cannot run system commands\n\nuse tokio::process::Command;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn simple() {\n let mut cmd;\n\n if cfg!(windows) {\n cmd = Command::new(\"cmd\");\n cmd.arg(\"/c\");\n } else {\n cmd = Command::new(\"sh\");\n cmd.arg(\"-c\");\n }\n\n let mut child = cmd.arg(\"exit 2\").spawn().unwrap();\n\n let id = child.id().expect(\"missing id\");\n assert!(id > 0);\n\n let status = assert_ok!(child.wait().await);\n assert_eq!(status.code(), Some(2));\n\n // test that the `.wait()` method is fused just like the stdlib\n let status = assert_ok!(child.wait().await);\n assert_eq!(status.code(), Some(2));\n\n // Can't get id after process has exited\n assert_eq!(child.id(), None);\n drop(child.kill());\n}\n" }, { "path": "tokio/tests/rt_basic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::runtime::Runtime;\nuse tokio::sync::oneshot;\nuse tokio::time::{timeout, Duration};\nuse tokio_test::{assert_err, assert_ok};\n\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::atomic::{AtomicBool, Ordering};\nuse std::task::{Context, Poll};\nuse std::thread;\n\nmod support {\n pub(crate) mod mpsc_stream;\n}\n\nmacro_rules! cfg_metrics {\n ($($t:tt)*) => {\n #[cfg(all(tokio_unstable, target_has_atomic = \"64\"))]\n {\n $( $t )*\n }\n }\n}\n\n#[test]\nfn spawned_task_does_not_progress_without_block_on() {\n let (tx, mut rx) = oneshot::channel();\n\n let rt = rt();\n\n rt.spawn(async move {\n assert_ok!(tx.send(\"hello\"));\n });\n\n thread::sleep(Duration::from_millis(50));\n\n assert_err!(rx.try_recv());\n\n let out = rt.block_on(async { assert_ok!(rx.await) });\n\n assert_eq!(out, \"hello\");\n}\n\n#[test]\nfn no_extra_poll() {\n use pin_project_lite::pin_project;\n use std::pin::Pin;\n use std::sync::{\n atomic::{AtomicUsize, Ordering::SeqCst},\n Arc,\n };\n use std::task::{Context, Poll};\n use tokio_stream::{Stream, StreamExt};\n\n pin_project! {\n struct TrackPolls {\n npolls: Arc,\n #[pin]\n s: S,\n }\n }\n\n impl Stream for TrackPolls\n where\n S: Stream,\n {\n type Item = S::Item;\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let this = self.project();\n this.npolls.fetch_add(1, SeqCst);\n this.s.poll_next(cx)\n }\n }\n\n let (tx, rx) = support::mpsc_stream::unbounded_channel_stream::<()>();\n let rx = TrackPolls {\n npolls: Arc::new(AtomicUsize::new(0)),\n s: rx,\n };\n let npolls = Arc::clone(&rx.npolls);\n\n let rt = rt();\n\n // TODO: could probably avoid this, but why not.\n let mut rx = Box::pin(rx);\n\n rt.spawn(async move { while rx.next().await.is_some() {} });\n rt.block_on(async {\n tokio::task::yield_now().await;\n });\n\n // should have been polled exactly once: the initial poll\n assert_eq!(npolls.load(SeqCst), 1);\n\n tx.send(()).unwrap();\n rt.block_on(async {\n tokio::task::yield_now().await;\n });\n\n // should have been polled twice more: once to yield Some(), then once to yield Pending\n assert_eq!(npolls.load(SeqCst), 1 + 2);\n\n drop(tx);\n rt.block_on(async {\n tokio::task::yield_now().await;\n });\n\n // should have been polled once more: to yield None\n assert_eq!(npolls.load(SeqCst), 1 + 2 + 1);\n}\n\n#[test]\nfn acquire_mutex_in_drop() {\n use futures::future::pending;\n use tokio::task;\n\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n let rt = rt();\n\n rt.spawn(async move {\n let _ = rx2.await;\n unreachable!();\n });\n\n rt.spawn(async move {\n let _ = rx1.await;\n tx2.send(()).unwrap();\n unreachable!();\n });\n\n // Spawn a task that will never notify\n rt.spawn(async move {\n pending::<()>().await;\n tx1.send(()).unwrap();\n });\n\n // Tick the loop\n rt.block_on(async {\n task::yield_now().await;\n });\n\n // Drop the rt\n drop(rt);\n}\n\n#[test]\nfn drop_tasks_in_context() {\n static SUCCESS: AtomicBool = AtomicBool::new(false);\n\n struct ContextOnDrop;\n\n impl Future for ContextOnDrop {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Pending\n }\n }\n\n impl Drop for ContextOnDrop {\n fn drop(&mut self) {\n if tokio::runtime::Handle::try_current().is_ok() {\n SUCCESS.store(true, Ordering::SeqCst);\n }\n }\n }\n\n let rt = rt();\n rt.spawn(ContextOnDrop);\n drop(rt);\n\n assert!(SUCCESS.load(Ordering::SeqCst));\n}\n\n#[test]\n#[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n#[should_panic(expected = \"boom\")]\nfn wake_in_drop_after_panic() {\n struct WakeOnDrop(Option>);\n\n impl Drop for WakeOnDrop {\n fn drop(&mut self) {\n let _ = self.0.take().unwrap().send(());\n }\n }\n\n let rt = rt();\n\n let (tx1, rx1) = oneshot::channel::<()>();\n let (tx2, rx2) = oneshot::channel::<()>();\n\n // Spawn two tasks. We don't know the order in which they are dropped, so we\n // make both tasks identical. When the first task is dropped, we wake up the\n // second task. This ensures that we trigger a wakeup on a live task while\n // handling the \"boom\" panic, no matter the order in which the tasks are\n // dropped.\n rt.spawn(async move {\n let _wake_on_drop = WakeOnDrop(Some(tx2));\n let _ = rx1.await;\n unreachable!()\n });\n\n rt.spawn(async move {\n let _wake_on_drop = WakeOnDrop(Some(tx1));\n let _ = rx2.await;\n unreachable!()\n });\n\n rt.block_on(async {\n tokio::task::yield_now().await;\n panic!(\"boom\");\n });\n}\n\n#[test]\nfn spawn_two() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n tokio::spawn(async move {\n tx.send(\"ZOMG\").unwrap();\n });\n });\n\n assert_ok!(rx.await)\n });\n\n assert_eq!(out, \"ZOMG\");\n\n cfg_metrics! {\n let metrics = rt.metrics();\n drop(rt);\n assert_eq!(0, metrics.remote_schedule_count());\n\n let mut local = 0;\n for i in 0..metrics.num_workers() {\n local += metrics.worker_local_schedule_count(i);\n }\n\n assert_eq!(2, local);\n }\n}\n\n#[cfg_attr(target_os = \"wasi\", ignore = \"WASI: std::thread::spawn not supported\")]\n#[test]\nfn spawn_remote() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n let handle = tokio::spawn(async move {\n std::thread::spawn(move || {\n std::thread::sleep(Duration::from_millis(10));\n tx.send(\"ZOMG\").unwrap();\n });\n\n rx.await.unwrap()\n });\n\n handle.await.unwrap()\n });\n\n assert_eq!(out, \"ZOMG\");\n\n cfg_metrics! {\n let metrics = rt.metrics();\n drop(rt);\n assert_eq!(1, metrics.remote_schedule_count());\n\n let mut local = 0;\n for i in 0..metrics.num_workers() {\n local += metrics.worker_local_schedule_count(i);\n }\n\n assert_eq!(1, local);\n }\n}\n\n#[test]\n#[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n#[should_panic(\n expected = \"A Tokio 1.x context was found, but timers are disabled. Call `enable_time` on the runtime builder to enable timers.\"\n)]\nfn timeout_panics_when_no_time_handle() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.block_on(async {\n let (_tx, rx) = oneshot::channel::<()>();\n let dur = Duration::from_millis(20);\n let _ = timeout(dur, rx).await;\n });\n}\n\n#[cfg(tokio_unstable)]\nmod unstable {\n use tokio::runtime::{Builder, RngSeed, UnhandledPanic};\n\n #[test]\n #[should_panic(\n expected = \"a spawned task panicked and the runtime is configured to shut down on unhandled panic\"\n )]\n fn shutdown_on_panic() {\n let rt = Builder::new_current_thread()\n .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n .build()\n .unwrap();\n\n rt.block_on(async {\n tokio::spawn(async {\n panic!(\"boom\");\n });\n\n futures::future::pending::<()>().await;\n })\n }\n\n #[test]\n #[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n fn spawns_do_nothing() {\n use std::sync::Arc;\n\n let rt = Builder::new_current_thread()\n .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n .build()\n .unwrap();\n\n let rt1 = Arc::new(rt);\n let rt2 = rt1.clone();\n\n let _ = std::thread::spawn(move || {\n rt2.block_on(async {\n tokio::spawn(async {\n panic!(\"boom\");\n });\n\n futures::future::pending::<()>().await;\n })\n })\n .join();\n\n let task = rt1.spawn(async {});\n let res = futures::executor::block_on(task);\n assert!(res.is_err());\n }\n\n #[test]\n #[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n fn shutdown_all_concurrent_block_on() {\n const N: usize = 2;\n use std::sync::{mpsc, Arc};\n\n let rt = Builder::new_current_thread()\n .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n .build()\n .unwrap();\n\n let rt = Arc::new(rt);\n let mut threads = vec![];\n let (tx, rx) = mpsc::channel();\n\n for _ in 0..N {\n let rt = rt.clone();\n let tx = tx.clone();\n threads.push(std::thread::spawn(move || {\n rt.block_on(async {\n tx.send(()).unwrap();\n futures::future::pending::<()>().await;\n });\n }));\n }\n\n for _ in 0..N {\n rx.recv().unwrap();\n }\n\n rt.spawn(async {\n panic!(\"boom\");\n });\n\n for thread in threads {\n assert!(thread.join().is_err());\n }\n }\n\n #[test]\n fn rng_seed() {\n let seed = b\"bytes used to generate seed\";\n let rt1 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt1_values = rt1.block_on(async {\n let rand_1 = tokio::macros::support::thread_rng_n(100);\n let rand_2 = tokio::macros::support::thread_rng_n(100);\n\n (rand_1, rand_2)\n });\n\n let rt2 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt2_values = rt2.block_on(async {\n let rand_1 = tokio::macros::support::thread_rng_n(100);\n let rand_2 = tokio::macros::support::thread_rng_n(100);\n\n (rand_1, rand_2)\n });\n\n assert_eq!(rt1_values, rt2_values);\n }\n\n #[test]\n fn rng_seed_multi_enter() {\n let seed = b\"bytes used to generate seed\";\n\n fn two_rand_values() -> (u32, u32) {\n let rand_1 = tokio::macros::support::thread_rng_n(100);\n let rand_2 = tokio::macros::support::thread_rng_n(100);\n\n (rand_1, rand_2)\n }\n\n let rt1 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt1_values_1 = rt1.block_on(async { two_rand_values() });\n let rt1_values_2 = rt1.block_on(async { two_rand_values() });\n\n let rt2 = tokio::runtime::Builder::new_current_thread()\n .rng_seed(RngSeed::from_bytes(seed))\n .build()\n .unwrap();\n let rt2_values_1 = rt2.block_on(async { two_rand_values() });\n let rt2_values_2 = rt2.block_on(async { two_rand_values() });\n\n assert_eq!(rt1_values_1, rt2_values_1);\n assert_eq!(rt1_values_2, rt2_values_2);\n }\n}\n\nfn rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\n#[test]\nfn before_park_yields() {\n use futures::task::ArcWake;\n use std::sync::Arc;\n use tokio::runtime::Builder;\n use tokio::sync::Notify;\n\n struct MyWaker(Notify);\n\n impl ArcWake for MyWaker {\n fn wake_by_ref(arc_self: &Arc) {\n arc_self.0.notify_one();\n }\n }\n\n let notify = Arc::new(MyWaker(Notify::new()));\n let notify2 = notify.clone();\n let waker = futures::task::waker(notify2);\n let woken = Arc::new(AtomicBool::new(false));\n let woken2 = woken.clone();\n\n let rt = Builder::new_current_thread()\n .enable_all()\n .on_thread_park(move || {\n if !woken2.swap(true, Ordering::SeqCst) {\n let mut cx = Context::from_waker(&waker);\n // `yield_now` pushes the waker to the defer slot.\n let fut = std::pin::pin!(tokio::task::yield_now());\n let _ = fut.poll(&mut cx);\n }\n })\n .build()\n .unwrap();\n\n rt.block_on(async {\n notify.0.notified().await;\n });\n\n assert!(woken.load(Ordering::SeqCst));\n}\n" }, { "path": "tokio/tests/rt_common.rs", "content": "#![allow(clippy::needless_range_loop)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(miri))]\n\n// Tests to run on both current-thread & multi-thread runtime variants.\n\nmacro_rules! rt_test {\n ($($t:tt)*) => {\n mod current_thread_scheduler {\n $($t)*\n\n #[cfg(not(target_os=\"wasi\"))]\n const NUM_WORKERS: usize = 1;\n\n fn rt() -> Arc {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n .into()\n }\n }\n\n #[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n mod threaded_scheduler_4_threads {\n $($t)*\n\n const NUM_WORKERS: usize = 4;\n\n fn rt() -> Arc {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap()\n .into()\n }\n }\n\n #[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n mod threaded_scheduler_1_thread {\n $($t)*\n\n const NUM_WORKERS: usize = 1;\n\n fn rt() -> Arc {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap()\n .into()\n }\n }\n }\n}\n\n#[test]\nfn send_sync_bound() {\n use tokio::runtime::Runtime;\n fn is_send() {}\n\n is_send::();\n}\n\nrt_test! {\n #[cfg(not(target_os=\"wasi\"))]\n use tokio::net::{TcpListener, TcpStream};\n #[cfg(not(target_os=\"wasi\"))]\n use tokio::io::{AsyncReadExt, AsyncWriteExt};\n\n use tokio::runtime::Runtime;\n use tokio::sync::oneshot;\n use tokio::{task, time};\n\n #[cfg(not(target_os=\"wasi\"))]\n use tokio_test::assert_err;\n use tokio_test::assert_ok;\n\n use std::future::{poll_fn, Future};\n use std::pin::Pin;\n\n #[cfg(not(target_os=\"wasi\"))]\n use std::sync::mpsc;\n\n use std::sync::Arc;\n use std::task::{Context, Poll};\n\n #[cfg(not(target_os=\"wasi\"))]\n use std::thread;\n use std::time::{Duration, Instant};\n\n #[test]\n fn block_on_sync() {\n let rt = rt();\n\n let mut win = false;\n rt.block_on(async {\n win = true;\n });\n\n assert!(win);\n }\n\n\n #[cfg(not(target_os=\"wasi\"))]\n #[test]\n fn block_on_async() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n thread::spawn(move || {\n thread::sleep(Duration::from_millis(50));\n tx.send(\"ZOMG\").unwrap();\n });\n\n assert_ok!(rx.await)\n });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[test]\n fn spawn_one_bg() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n tx.send(\"ZOMG\").unwrap();\n });\n\n assert_ok!(rx.await)\n });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[test]\n fn spawn_one_join() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n let handle = tokio::spawn(async move {\n tx.send(\"ZOMG\").unwrap();\n \"DONE\"\n });\n\n let msg = assert_ok!(rx.await);\n\n let out = assert_ok!(handle.await);\n assert_eq!(out, \"DONE\");\n\n msg\n });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[test]\n fn spawn_two() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n tokio::spawn(async move {\n assert_ok!(tx1.send(\"ZOMG\"));\n });\n\n tokio::spawn(async move {\n let msg = assert_ok!(rx1.await);\n assert_ok!(tx2.send(msg));\n });\n\n assert_ok!(rx2.await)\n });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_many_from_block_on() {\n use tokio::sync::mpsc;\n\n const ITER: usize = 200;\n\n let rt = rt();\n\n let out = rt.block_on(async {\n let (done_tx, mut done_rx) = mpsc::unbounded_channel();\n\n let mut txs = (0..ITER)\n .map(|i| {\n let (tx, rx) = oneshot::channel();\n let done_tx = done_tx.clone();\n\n tokio::spawn(async move {\n let msg = assert_ok!(rx.await);\n assert_eq!(i, msg);\n assert_ok!(done_tx.send(msg));\n });\n\n tx\n })\n .collect::>();\n\n drop(done_tx);\n\n thread::spawn(move || {\n for (i, tx) in txs.drain(..).enumerate() {\n assert_ok!(tx.send(i));\n }\n });\n\n let mut out = vec![];\n while let Some(i) = done_rx.recv().await {\n out.push(i);\n }\n\n out.sort_unstable();\n out\n });\n\n assert_eq!(ITER, out.len());\n\n for i in 0..ITER {\n assert_eq!(i, out[i]);\n }\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_many_from_task() {\n use tokio::sync::mpsc;\n\n const ITER: usize = 500;\n\n let rt = rt();\n\n let out = rt.block_on(async {\n tokio::spawn(async move {\n let (done_tx, mut done_rx) = mpsc::unbounded_channel();\n\n let mut txs = (0..ITER)\n .map(|i| {\n let (tx, rx) = oneshot::channel();\n let done_tx = done_tx.clone();\n\n tokio::spawn(async move {\n let msg = assert_ok!(rx.await);\n assert_eq!(i, msg);\n assert_ok!(done_tx.send(msg));\n });\n\n tx\n })\n .collect::>();\n\n drop(done_tx);\n\n thread::spawn(move || {\n for (i, tx) in txs.drain(..).enumerate() {\n assert_ok!(tx.send(i));\n }\n });\n\n let mut out = vec![];\n while let Some(i) = done_rx.recv().await {\n out.push(i);\n }\n\n out.sort_unstable();\n out\n }).await.unwrap()\n });\n\n assert_eq!(ITER, out.len());\n\n for i in 0..ITER {\n assert_eq!(i, out[i]);\n }\n }\n\n #[test]\n fn spawn_one_from_block_on_called_on_handle() {\n let rt = rt();\n let (tx, rx) = oneshot::channel();\n\n #[allow(clippy::async_yields_async)]\n let handle = rt.handle().block_on(async {\n tokio::spawn(async move {\n tx.send(\"ZOMG\").unwrap();\n \"DONE\"\n })\n });\n\n let out = rt.block_on(async {\n let msg = assert_ok!(rx.await);\n\n let out = assert_ok!(handle.await);\n assert_eq!(out, \"DONE\");\n\n msg\n });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[test]\n fn spawn_await_chain() {\n let rt = rt();\n\n let out = rt.block_on(async {\n assert_ok!(tokio::spawn(async {\n assert_ok!(tokio::spawn(async {\n \"hello\"\n }).await)\n }).await)\n });\n\n assert_eq!(out, \"hello\");\n }\n\n #[test]\n fn outstanding_tasks_dropped() {\n let rt = rt();\n\n let cnt = Arc::new(());\n\n rt.block_on(async {\n let cnt = cnt.clone();\n\n tokio::spawn(poll_fn(move |_| {\n assert_eq!(2, Arc::strong_count(&cnt));\n Poll::<()>::Pending\n }));\n });\n\n assert_eq!(2, Arc::strong_count(&cnt));\n\n drop(rt);\n\n assert_eq!(1, Arc::strong_count(&cnt));\n }\n\n #[test]\n #[should_panic]\n fn nested_rt() {\n let rt1 = rt();\n let rt2 = rt();\n\n rt1.block_on(async { rt2.block_on(async { \"hello\" }) });\n }\n\n #[test]\n fn create_rt_in_block_on() {\n let rt1 = rt();\n let rt2 = rt1.block_on(async { rt() });\n let out = rt2.block_on(async { \"ZOMG\" });\n\n assert_eq!(out, \"ZOMG\");\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn complete_block_on_under_load() {\n let rt = rt();\n\n rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n // Spin hard\n tokio::spawn(async {\n loop {\n yield_once().await;\n }\n });\n\n thread::spawn(move || {\n thread::sleep(Duration::from_millis(50));\n assert_ok!(tx.send(()));\n });\n\n assert_ok!(rx.await);\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn complete_task_under_load() {\n let rt = rt();\n\n rt.block_on(async {\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n // Spin hard\n tokio::spawn(async {\n loop {\n yield_once().await;\n }\n });\n\n thread::spawn(move || {\n thread::sleep(Duration::from_millis(50));\n assert_ok!(tx1.send(()));\n });\n\n tokio::spawn(async move {\n assert_ok!(rx1.await);\n assert_ok!(tx2.send(()));\n });\n\n assert_ok!(rx2.await);\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_from_other_thread_idle() {\n let rt = rt();\n let handle = rt.clone();\n\n let (tx, rx) = oneshot::channel();\n\n thread::spawn(move || {\n thread::sleep(Duration::from_millis(50));\n\n handle.spawn(async move {\n assert_ok!(tx.send(()));\n });\n });\n\n rt.block_on(async move {\n assert_ok!(rx.await);\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_from_other_thread_under_load() {\n let rt = rt();\n let handle = rt.clone();\n\n let (tx, rx) = oneshot::channel();\n\n thread::spawn(move || {\n handle.spawn(async move {\n assert_ok!(tx.send(()));\n });\n });\n\n rt.block_on(async move {\n // Spin hard\n tokio::spawn(async {\n loop {\n yield_once().await;\n }\n });\n\n assert_ok!(rx.await);\n });\n }\n\n #[test]\n fn sleep_at_root() {\n let rt = rt();\n\n let now = Instant::now();\n let dur = Duration::from_millis(50);\n\n rt.block_on(async move {\n time::sleep(dur).await;\n });\n\n assert!(now.elapsed() >= dur);\n }\n\n #[test]\n fn sleep_in_spawn() {\n let rt = rt();\n\n let now = Instant::now();\n let dur = Duration::from_millis(50);\n\n rt.block_on(async move {\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n time::sleep(dur).await;\n assert_ok!(tx.send(()));\n });\n\n assert_ok!(rx.await);\n });\n\n assert!(now.elapsed() >= dur);\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support bind\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn block_on_socket() {\n let rt = rt();\n\n rt.block_on(async move {\n let (tx, rx) = oneshot::channel();\n\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = listener.local_addr().unwrap();\n\n tokio::spawn(async move {\n let _ = listener.accept().await;\n tx.send(()).unwrap();\n });\n\n TcpStream::connect(&addr).await.unwrap();\n rx.await.unwrap();\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_from_blocking() {\n let rt = rt();\n\n let out = rt.block_on(async move {\n let inner = assert_ok!(tokio::task::spawn_blocking(|| {\n tokio::spawn(async move { \"hello\" })\n }).await);\n\n assert_ok!(inner.await)\n });\n\n assert_eq!(out, \"hello\")\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn spawn_blocking_from_blocking() {\n let rt = rt();\n\n let out = rt.block_on(async move {\n let inner = assert_ok!(tokio::task::spawn_blocking(|| {\n tokio::task::spawn_blocking(|| \"hello\")\n }).await);\n\n assert_ok!(inner.await)\n });\n\n assert_eq!(out, \"hello\")\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn sleep_from_blocking() {\n let rt = rt();\n\n rt.block_on(async move {\n assert_ok!(tokio::task::spawn_blocking(|| {\n let now = std::time::Instant::now();\n let dur = Duration::from_millis(1);\n\n // use the futures' block_on fn to make sure we aren't setting\n // any Tokio context\n futures::executor::block_on(async {\n tokio::time::sleep(dur).await;\n });\n\n assert!(now.elapsed() >= dur);\n }).await);\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support bind\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn socket_from_blocking() {\n let rt = rt();\n\n rt.block_on(async move {\n let listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(listener.local_addr());\n\n let peer = tokio::task::spawn_blocking(move || {\n // use the futures' block_on fn to make sure we aren't setting\n // any Tokio context\n futures::executor::block_on(async {\n assert_ok!(TcpStream::connect(addr).await);\n });\n });\n\n // Wait for the client to connect\n let _ = assert_ok!(listener.accept().await);\n\n assert_ok!(peer.await);\n });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn always_active_parker() {\n // This test it to show that we will always have\n // an active parker even if we call block_on concurrently\n\n let rt = rt();\n let rt2 = rt.clone();\n\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n let jh1 = thread::spawn(move || {\n rt.block_on(async move {\n rx2.await.unwrap();\n time::sleep(Duration::from_millis(5)).await;\n tx1.send(()).unwrap();\n });\n });\n\n let jh2 = thread::spawn(move || {\n rt2.block_on(async move {\n tx2.send(()).unwrap();\n time::sleep(Duration::from_millis(5)).await;\n rx1.await.unwrap();\n time::sleep(Duration::from_millis(5)).await;\n });\n });\n\n jh1.join().unwrap();\n jh2.join().unwrap();\n }\n\n #[test]\n // IOCP requires setting the \"max thread\" concurrency value. The sane,\n // default, is to set this to the number of cores. Threads that poll I/O\n // become associated with the IOCP handle. Once those threads sleep for any\n // reason (mutex), they yield their ownership.\n //\n // This test hits an edge case on windows where more threads than cores are\n // created, none of those threads ever yield due to being at capacity, so\n // IOCP gets \"starved\".\n //\n // For now, this is a very edge case that is probably not a real production\n // concern. There also isn't a great/obvious solution to take. For now, the\n // test is disabled.\n #[cfg(not(windows))]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support bind or threads\n fn io_driver_called_when_under_load() {\n let rt = rt();\n\n // Create a lot of constant load. The scheduler will always be busy.\n for _ in 0..100 {\n rt.spawn(async {\n loop {\n // Don't use Tokio's `yield_now()` to avoid special defer\n // logic.\n std::future::poll_fn::<(), _>(|cx| {\n cx.waker().wake_by_ref();\n std::task::Poll::Pending\n }).await;\n }\n });\n }\n\n // Do some I/O work\n rt.block_on(async {\n let listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(listener.local_addr());\n\n let srv = tokio::spawn(async move {\n let (mut stream, _) = assert_ok!(listener.accept().await);\n assert_ok!(stream.write_all(b\"hello world\").await);\n });\n\n let cli = tokio::spawn(async move {\n let mut stream = assert_ok!(TcpStream::connect(addr).await);\n let mut dst = vec![0; 11];\n\n assert_ok!(stream.read_exact(&mut dst).await);\n assert_eq!(dst, b\"hello world\");\n });\n\n assert_ok!(srv.await);\n assert_ok!(cli.await);\n });\n }\n\n /// Tests that yielded tasks are not scheduled until **after** resource\n /// drivers are polled.\n ///\n /// The OS does not guarantee when I/O events are delivered, so there may be\n /// more yields than anticipated. This makes the test slightly flaky. To\n /// help avoid flakiness, we run the test 10 times and only fail it after\n /// 10 failures in a row.\n ///\n /// Note that if the test fails by panicking rather than by returning false,\n /// then we fail it immediately. That kind of failure should not happen\n /// spuriously.\n #[test]\n #[cfg(not(target_os=\"wasi\"))]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n fn yield_defers_until_park() {\n for _ in 0..10 {\n if yield_defers_until_park_inner(false) {\n // test passed\n return;\n }\n\n // Wait a bit and run the test again.\n std::thread::sleep(std::time::Duration::from_secs(2));\n }\n\n panic!(\"yield_defers_until_park is failing consistently\");\n }\n\n /// Same as above, but with cooperative scheduling.\n #[test]\n #[cfg(not(target_os=\"wasi\"))]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n fn coop_yield_defers_until_park() {\n for _ in 0..10 {\n if yield_defers_until_park_inner(true) {\n // test passed\n return;\n }\n\n // Wait a bit and run the test again.\n std::thread::sleep(std::time::Duration::from_secs(2));\n }\n\n panic!(\"yield_defers_until_park is failing consistently\");\n }\n\n /// Implementation of `yield_defers_until_park` test. Returns `true` if the\n /// test passed.\n #[cfg(not(target_os=\"wasi\"))]\n fn yield_defers_until_park_inner(use_coop: bool) -> bool {\n use std::sync::atomic::{AtomicBool, Ordering::SeqCst};\n use std::sync::Barrier;\n\n const BUDGET: usize = 128;\n\n let rt = rt();\n\n let flag = Arc::new(AtomicBool::new(false));\n let barrier = Arc::new(Barrier::new(NUM_WORKERS));\n\n rt.block_on(async {\n // Make sure other workers cannot steal tasks\n #[allow(clippy::reversed_empty_ranges)]\n for _ in 0..(NUM_WORKERS-1) {\n let flag = flag.clone();\n let barrier = barrier.clone();\n\n tokio::spawn(async move {\n barrier.wait();\n\n while !flag.load(SeqCst) {\n std::thread::sleep(std::time::Duration::from_millis(1));\n }\n });\n }\n\n barrier.wait();\n\n let (fail_test, fail_test_recv) = oneshot::channel::<()>();\n let flag_clone = flag.clone();\n let jh = tokio::spawn(async move {\n // Create a TCP listener\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = listener.local_addr().unwrap();\n\n tokio::join!(\n async {\n // Done in a blocking manner intentionally.\n let _socket = std::net::TcpStream::connect(addr).unwrap();\n\n // Yield until connected\n let mut cnt = 0;\n while !flag_clone.load(SeqCst){\n if use_coop {\n // Consume a good chunk of budget, which should\n // force at least one yield.\n for _ in 0..BUDGET {\n tokio::task::consume_budget().await;\n }\n } else {\n tokio::task::yield_now().await;\n }\n cnt += 1;\n\n if cnt >= 10 {\n // yielded too many times; report failure and\n // sleep forever so that the `fail_test` branch\n // of the `select!` below triggers.\n let _ = fail_test.send(());\n futures::future::pending::<()>().await;\n break;\n }\n }\n },\n async {\n let _ = listener.accept().await.unwrap();\n flag_clone.store(true, SeqCst);\n }\n );\n });\n\n // Wait until the spawned task completes or fails. If no message is\n // sent on `fail_test`, then the test succeeds. Otherwise, it fails.\n let success = fail_test_recv.await.is_err();\n\n if success {\n // Setting flag to true ensures that the tasks we spawned at\n // the beginning of the test will exit.\n // If we don't do this, the test will hang since the runtime waits\n // for all spawned tasks to finish when dropping.\n flag.store(true, SeqCst);\n // Check for panics in spawned task.\n jh.abort();\n jh.await.unwrap();\n }\n\n success\n })\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn client_server_block_on() {\n let rt = rt();\n let (tx, rx) = mpsc::channel();\n\n rt.block_on(async move { client_server(tx).await });\n\n assert_ok!(rx.try_recv());\n assert_err!(rx.try_recv());\n }\n\n #[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support threads or panic recovery\")]\n #[cfg(panic = \"unwind\")]\n #[test]\n fn panic_in_task() {\n let rt = rt();\n let (tx, rx) = oneshot::channel();\n\n struct Boom(Option>);\n\n impl Future for Boom {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n panic!();\n }\n }\n\n impl Drop for Boom {\n fn drop(&mut self) {\n assert!(std::thread::panicking());\n self.0.take().unwrap().send(()).unwrap();\n }\n }\n\n rt.spawn(Boom(Some(tx)));\n assert_ok!(rt.block_on(rx));\n }\n\n #[test]\n #[should_panic]\n #[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n fn panic_in_block_on() {\n let rt = rt();\n rt.block_on(async { panic!() });\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n async fn yield_once() {\n let mut yielded = false;\n poll_fn(|cx| {\n if yielded {\n Poll::Ready(())\n } else {\n yielded = true;\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n })\n .await\n }\n\n #[test]\n fn enter_and_spawn() {\n let rt = rt();\n let handle = {\n let _enter = rt.enter();\n tokio::spawn(async {})\n };\n\n assert_ok!(rt.block_on(handle));\n }\n\n #[test]\n fn eagerly_drops_futures_on_shutdown() {\n use std::sync::mpsc;\n\n struct Never {\n drop_tx: mpsc::Sender<()>,\n }\n\n impl Future for Never {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Pending\n }\n }\n\n impl Drop for Never {\n fn drop(&mut self) {\n self.drop_tx.send(()).unwrap();\n }\n }\n\n let rt = rt();\n\n let (drop_tx, drop_rx) = mpsc::channel();\n let (run_tx, run_rx) = oneshot::channel();\n\n rt.block_on(async move {\n tokio::spawn(async move {\n assert_ok!(run_tx.send(()));\n\n Never { drop_tx }.await\n });\n\n assert_ok!(run_rx.await);\n });\n\n drop(rt);\n\n assert_ok!(drop_rx.recv());\n }\n\n #[test]\n fn wake_while_rt_is_dropping() {\n use tokio::sync::Barrier;\n\n struct OnDrop(F);\n\n impl Drop for OnDrop {\n fn drop(&mut self) {\n (self.0)()\n }\n }\n\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n\n let barrier = Arc::new(Barrier::new(3));\n let barrier1 = barrier.clone();\n let barrier2 = barrier.clone();\n\n let rt = rt();\n\n rt.spawn(async move {\n let mut tx2 = Some(tx2);\n let _d = OnDrop(move || {\n let _ = tx2.take().unwrap().send(());\n });\n\n // Ensure a waker gets stored in oneshot 1.\n let _ = tokio::join!(rx1, barrier1.wait());\n });\n\n rt.spawn(async move {\n let mut tx1 = Some(tx1);\n let _d = OnDrop(move || {\n let _ = tx1.take().unwrap().send(());\n });\n\n // Ensure a waker gets stored in oneshot 2.\n let _ = tokio::join!(rx2, barrier2.wait());\n });\n\n // Wait until every oneshot channel has been polled.\n rt.block_on(barrier.wait());\n\n // Drop the rt. Regardless of which task is dropped first, its destructor will wake the\n // other task.\n drop(rt);\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi doesn't support UDP or bind()\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn io_notify_while_shutting_down() {\n use tokio::net::UdpSocket;\n use std::sync::Arc;\n\n for _ in 1..10 {\n let runtime = rt();\n\n runtime.block_on(async {\n let socket = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = socket.local_addr().unwrap();\n let send_half = Arc::new(socket);\n let recv_half = send_half.clone();\n\n tokio::spawn(async move {\n let mut buf = [0];\n loop {\n recv_half.recv_from(&mut buf).await.unwrap();\n std::thread::sleep(Duration::from_millis(2));\n }\n });\n\n tokio::spawn(async move {\n let buf = [0];\n loop {\n send_half.send_to(&buf, &addr).await.unwrap();\n tokio::time::sleep(Duration::from_millis(1)).await;\n }\n });\n\n tokio::time::sleep(Duration::from_millis(5)).await;\n });\n }\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn shutdown_timeout() {\n let (tx, rx) = oneshot::channel();\n let runtime = rt();\n\n runtime.block_on(async move {\n task::spawn_blocking(move || {\n tx.send(()).unwrap();\n thread::sleep(Duration::from_secs(10_000));\n });\n\n rx.await.unwrap();\n });\n\n Arc::try_unwrap(runtime).unwrap().shutdown_timeout(Duration::from_millis(100));\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support threads\n #[test]\n fn shutdown_timeout_0() {\n let runtime = rt();\n\n runtime.block_on(async move {\n task::spawn_blocking(move || {\n thread::sleep(Duration::from_secs(10_000));\n });\n });\n\n let now = Instant::now();\n Arc::try_unwrap(runtime).unwrap().shutdown_timeout(Duration::from_nanos(0));\n assert!(now.elapsed().as_secs() < 1);\n }\n\n #[test]\n fn shutdown_wakeup_time() {\n let runtime = rt();\n\n runtime.block_on(async move {\n tokio::time::sleep(std::time::Duration::from_millis(100)).await;\n });\n\n Arc::try_unwrap(runtime).unwrap().shutdown_timeout(Duration::from_secs(10_000));\n }\n\n // This test is currently ignored on Windows because of a\n // rust-lang issue in thread local storage destructors.\n // See https://github.com/rust-lang/rust/issues/74875\n #[test]\n #[cfg(not(windows))]\n #[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support threads\")]\n fn runtime_in_thread_local() {\n use std::cell::RefCell;\n use std::thread;\n\n thread_local!(\n static R: RefCell> = const { RefCell::new(None) };\n );\n\n thread::spawn(|| {\n R.with(|cell| {\n let rt = rt();\n let rt = Arc::try_unwrap(rt).unwrap();\n *cell.borrow_mut() = Some(rt);\n });\n\n let _rt = rt();\n }).join().unwrap();\n }\n\n #[cfg(not(target_os=\"wasi\"))] // Wasi does not support bind\n async fn client_server(tx: mpsc::Sender<()>) {\n let server = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n\n // Get the assigned address\n let addr = assert_ok!(server.local_addr());\n\n // Spawn the server\n tokio::spawn(async move {\n // Accept a socket\n let (mut socket, _) = server.accept().await.unwrap();\n\n // Write some data\n socket.write_all(b\"hello\").await.unwrap();\n });\n\n let mut client = TcpStream::connect(&addr).await.unwrap();\n\n let mut buf = vec![];\n client.read_to_end(&mut buf).await.unwrap();\n\n assert_eq!(buf, b\"hello\");\n tx.send(()).unwrap();\n }\n\n #[cfg(not(target_os = \"wasi\"))] // Wasi does not support bind\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn local_set_block_on_socket() {\n let rt = rt();\n let local = task::LocalSet::new();\n\n local.block_on(&rt, async move {\n let (tx, rx) = oneshot::channel();\n\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n let addr = listener.local_addr().unwrap();\n\n task::spawn_local(async move {\n let _ = listener.accept().await;\n tx.send(()).unwrap();\n });\n\n TcpStream::connect(&addr).await.unwrap();\n rx.await.unwrap();\n });\n }\n\n #[cfg(not(target_os = \"wasi\"))] // Wasi does not support bind\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn local_set_client_server_block_on() {\n let rt = rt();\n let (tx, rx) = mpsc::channel();\n\n let local = task::LocalSet::new();\n\n local.block_on(&rt, async move { client_server_local(tx).await });\n\n assert_ok!(rx.try_recv());\n assert_err!(rx.try_recv());\n }\n\n #[cfg(not(target_os = \"wasi\"))] // Wasi does not support bind\n async fn client_server_local(tx: mpsc::Sender<()>) {\n let server = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n\n // Get the assigned address\n let addr = assert_ok!(server.local_addr());\n\n // Spawn the server\n task::spawn_local(async move {\n // Accept a socket\n let (mut socket, _) = server.accept().await.unwrap();\n\n // Write some data\n socket.write_all(b\"hello\").await.unwrap();\n });\n\n let mut client = TcpStream::connect(&addr).await.unwrap();\n\n let mut buf = vec![];\n client.read_to_end(&mut buf).await.unwrap();\n\n assert_eq!(buf, b\"hello\");\n tx.send(()).unwrap();\n }\n\n #[test]\n fn coop() {\n use std::task::Poll::Ready;\n use tokio::sync::mpsc;\n\n let rt = rt();\n\n rt.block_on(async {\n let (send, mut recv) = mpsc::unbounded_channel();\n\n // Send a bunch of messages.\n for _ in 0..1_000 {\n send.send(()).unwrap();\n }\n\n poll_fn(|cx| {\n // At least one response should return pending.\n for _ in 0..1_000 {\n if recv.poll_recv(cx).is_pending() {\n return Ready(());\n }\n }\n\n panic!(\"did not yield\");\n }).await;\n });\n }\n\n #[test]\n fn coop_unconstrained() {\n use std::task::Poll::Ready;\n use tokio::sync::mpsc;\n\n let rt = rt();\n\n rt.block_on(async {\n let (send, mut recv) = mpsc::unbounded_channel();\n\n // Send a bunch of messages.\n for _ in 0..1_000 {\n send.send(()).unwrap();\n }\n\n tokio::task::unconstrained(poll_fn(|cx| {\n // All the responses should be ready.\n for _ in 0..1_000 {\n assert_eq!(recv.poll_recv(cx), Poll::Ready(Some(())));\n }\n\n Ready(())\n })).await;\n });\n }\n\n #[cfg(tokio_unstable)]\n #[test]\n fn coop_consume_budget() {\n let rt = rt();\n\n rt.block_on(async {\n poll_fn(|cx| {\n let counter = Arc::new(std::sync::Mutex::new(0));\n let counter_clone = Arc::clone(&counter);\n let mut worker = Box::pin(async move {\n // Consume the budget until a yield happens\n for _ in 0..1000 {\n *counter.lock().unwrap() += 1;\n task::consume_budget().await\n }\n });\n // Assert that the worker was yielded and it didn't manage\n // to finish the whole work (assuming the total budget of 128)\n assert!(Pin::new(&mut worker).poll(cx).is_pending());\n assert!(*counter_clone.lock().unwrap() < 1000);\n std::task::Poll::Ready(())\n }).await;\n });\n }\n\n // Tests that the \"next task\" scheduler optimization is not able to starve\n // other tasks.\n #[test]\n fn ping_pong_saturation() {\n use std::sync::atomic::{Ordering, AtomicBool};\n use tokio::sync::mpsc;\n\n const NUM: usize = 100;\n\n let rt = rt();\n\n let running = Arc::new(AtomicBool::new(true));\n\n rt.block_on(async {\n let (spawned_tx, mut spawned_rx) = mpsc::unbounded_channel();\n\n let mut tasks = vec![];\n // Spawn a bunch of tasks that ping-pong between each other to\n // saturate the runtime.\n for _ in 0..NUM {\n let (tx1, mut rx1) = mpsc::unbounded_channel();\n let (tx2, mut rx2) = mpsc::unbounded_channel();\n let spawned_tx = spawned_tx.clone();\n let running = running.clone();\n tasks.push(task::spawn(async move {\n spawned_tx.send(()).unwrap();\n\n\n while running.load(Ordering::Relaxed) {\n tx1.send(()).unwrap();\n rx2.recv().await.unwrap();\n }\n\n // Close the channel and wait for the other task to exit.\n drop(tx1);\n assert!(rx2.recv().await.is_none());\n }));\n\n tasks.push(task::spawn(async move {\n while rx1.recv().await.is_some() {\n tx2.send(()).unwrap();\n }\n }));\n }\n\n for _ in 0..NUM {\n spawned_rx.recv().await.unwrap();\n }\n\n // spawn another task and wait for it to complete\n let handle = task::spawn(async {\n for _ in 0..5 {\n // Yielding forces it back into the local queue.\n task::yield_now().await;\n }\n });\n handle.await.unwrap();\n running.store(false, Ordering::Relaxed);\n for t in tasks {\n t.await.unwrap();\n }\n });\n }\n\n #[test]\n #[cfg(not(target_os=\"wasi\"))]\n fn shutdown_concurrent_spawn() {\n const NUM_TASKS: usize = 10_000;\n for _ in 0..5 {\n let (tx, rx) = std::sync::mpsc::channel();\n let rt = rt();\n\n let mut txs = vec![];\n\n for _ in 0..NUM_TASKS {\n let (tx, rx) = tokio::sync::oneshot::channel();\n txs.push(tx);\n rt.spawn(async move {\n rx.await.unwrap();\n });\n }\n\n // Prime the tasks\n rt.block_on(async { tokio::task::yield_now().await });\n\n let th = std::thread::spawn(move || {\n tx.send(()).unwrap();\n for tx in txs.drain(..) {\n let _ = tx.send(());\n }\n });\n\n rx.recv().unwrap();\n drop(rt);\n\n th.join().unwrap();\n }\n }\n\n #[test]\n #[cfg_attr(target_family = \"wasm\", ignore)]\n fn wake_by_ref_from_thread_local() {\n wake_from_thread_local(true);\n }\n\n #[test]\n #[cfg_attr(target_family = \"wasm\", ignore)]\n fn wake_by_val_from_thread_local() {\n wake_from_thread_local(false);\n }\n\n fn wake_from_thread_local(by_ref: bool) {\n use std::cell::RefCell;\n use std::sync::mpsc::{channel, Sender};\n use std::task::Waker;\n\n struct TLData {\n by_ref: bool,\n waker: Option,\n done: Sender,\n }\n\n impl Drop for TLData {\n fn drop(&mut self) {\n if self.by_ref {\n self.waker.take().unwrap().wake_by_ref();\n } else {\n self.waker.take().unwrap().wake();\n }\n let _ = self.done.send(true);\n }\n }\n\n std::thread_local! {\n static TL_DATA: RefCell> = const { RefCell::new(None) };\n };\n\n let (send, recv) = channel();\n\n std::thread::spawn(move || {\n let rt = rt();\n rt.block_on(rt.spawn(poll_fn(move |cx| {\n let waker = cx.waker().clone();\n let send = send.clone();\n TL_DATA.with(|tl| {\n tl.replace(Some(TLData {\n by_ref,\n waker: Some(waker),\n done: send,\n }));\n });\n Poll::Ready(())\n })))\n .unwrap();\n })\n .join()\n .unwrap();\n\n assert!(recv.recv().unwrap());\n }\n}\n" }, { "path": "tokio/tests/rt_common_before_park.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n\nuse std::sync::atomic::{AtomicBool, Ordering};\nuse std::sync::Arc;\nuse tokio::runtime::Builder;\nuse tokio::sync::Notify;\n\n#[test]\nfn before_park_wakes_block_on_task() {\n let notify = Arc::new(Notify::new());\n let notify2 = notify.clone();\n let woken = Arc::new(AtomicBool::new(false));\n let woken2 = woken.clone();\n\n let rt = Builder::new_current_thread()\n .enable_all()\n .on_thread_park(move || {\n // Only wake once to avoid busy loop if something goes wrong,\n // though in this test we expect it to unpark immediately.\n if !woken2.swap(true, Ordering::SeqCst) {\n notify2.notify_one();\n }\n })\n .build()\n .unwrap();\n\n rt.block_on(async {\n // This will block until `notify` is notified.\n // `before_park` should run when the runtime is about to park.\n // It will notify `notify`, which should wake this task.\n // The runtime should then see the task is woken and NOT park.\n notify.notified().await;\n });\n\n assert!(woken.load(Ordering::SeqCst));\n}\n\n#[test]\nfn before_park_spawns_task() {\n let notify = Arc::new(Notify::new());\n let notify2 = notify.clone();\n let woken = Arc::new(AtomicBool::new(false));\n let woken2 = woken.clone();\n\n let rt = Builder::new_current_thread()\n .enable_all()\n .on_thread_park(move || {\n if !woken2.swap(true, Ordering::SeqCst) {\n let notify = notify2.clone();\n tokio::spawn(async move {\n notify.notify_one();\n });\n }\n })\n .build()\n .unwrap();\n\n rt.block_on(async {\n // This will block until `notify` is notified.\n // `before_park` should run when the runtime is about to park.\n // It will spawn a task that notifies `notify`.\n // The runtime should see the new task and NOT park.\n // If it parks, it will deadlock.\n notify.notified().await;\n });\n\n assert!(woken.load(Ordering::SeqCst));\n}\n\n#[test]\nfn wake_from_other_thread_block_on() {\n let rt = Builder::new_current_thread().enable_all().build().unwrap();\n let handle = rt.handle().clone();\n let notify = Arc::new(Notify::new());\n let notify2 = notify.clone();\n\n let th = std::thread::spawn(move || {\n // Give the main thread time to park\n std::thread::sleep(std::time::Duration::from_millis(5));\n handle.block_on(async move {\n notify2.notify_one();\n });\n });\n\n rt.block_on(async {\n notify.notified().await;\n });\n\n th.join().unwrap();\n}\n" }, { "path": "tokio/tests/rt_handle.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::sync::Arc;\nuse tokio::runtime::Runtime;\nuse tokio::sync::{mpsc, Barrier};\n\n#[test]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn basic_enter() {\n let rt1 = rt();\n let rt2 = rt();\n\n let enter1 = rt1.enter();\n let enter2 = rt2.enter();\n\n drop(enter2);\n drop(enter1);\n}\n\n#[test]\n#[should_panic]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn interleave_enter_different_rt() {\n let rt1 = rt();\n let rt2 = rt();\n\n let enter1 = rt1.enter();\n let enter2 = rt2.enter();\n\n drop(enter1);\n drop(enter2);\n}\n\n#[test]\n#[should_panic]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn interleave_enter_same_rt() {\n let rt1 = rt();\n\n let _enter1 = rt1.enter();\n let enter2 = rt1.enter();\n let enter3 = rt1.enter();\n\n drop(enter2);\n drop(enter3);\n}\n\n#[test]\n#[cfg(not(target_os = \"wasi\"))]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn interleave_then_enter() {\n let _ = std::panic::catch_unwind(|| {\n let rt1 = rt();\n let rt2 = rt();\n\n let enter1 = rt1.enter();\n let enter2 = rt2.enter();\n\n drop(enter1);\n drop(enter2);\n });\n\n // Can still enter\n let rt3 = rt();\n let _enter = rt3.enter();\n}\n\n// If the cycle causes a leak, then miri will catch it.\n#[test]\nfn drop_tasks_with_reference_cycle() {\n rt().block_on(async {\n let (tx, mut rx) = mpsc::channel(1);\n\n let barrier = Arc::new(Barrier::new(3));\n let barrier_a = barrier.clone();\n let barrier_b = barrier.clone();\n\n let a = tokio::spawn(async move {\n let b = rx.recv().await.unwrap();\n\n // Poll the JoinHandle once. This registers the waker.\n // The other task cannot have finished at this point due to the barrier below.\n futures::future::select(b, std::future::ready(())).await;\n\n barrier_a.wait().await;\n });\n\n let b = tokio::spawn(async move {\n // Poll the JoinHandle once. This registers the waker.\n // The other task cannot have finished at this point due to the barrier below.\n futures::future::select(a, std::future::ready(())).await;\n\n barrier_b.wait().await;\n });\n\n tx.send(b).await.unwrap();\n\n barrier.wait().await;\n });\n}\n\n#[test]\nfn runtime_id_is_same() {\n let rt = rt();\n\n let handle1 = rt.handle();\n let handle2 = rt.handle();\n\n assert_eq!(handle1.id(), handle2.id());\n}\n\n#[test]\nfn runtime_ids_different() {\n let rt1 = rt();\n let rt2 = rt();\n\n assert_ne!(rt1.handle().id(), rt2.handle().id());\n}\n\nfn rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/rt_handle_block_on.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(miri)))]\n\n// All io tests that deal with shutdown is currently ignored because there are known bugs in with\n// shutting down the io driver while concurrently registering new resources. See\n// https://github.com/tokio-rs/tokio/pull/3569#pullrequestreview-612703467 for more details.\n//\n// When this has been fixed we want to re-enable these tests.\n\nuse std::time::Duration;\nuse tokio::runtime::{Handle, Runtime};\nuse tokio::sync::mpsc;\n#[cfg(not(target_os = \"wasi\"))]\nuse tokio::{net, time};\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\nmacro_rules! multi_threaded_rt_test {\n ($($t:tt)*) => {\n mod threaded_scheduler_4_threads_only {\n use super::*;\n\n $($t)*\n\n fn rt() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap()\n }\n }\n\n mod threaded_scheduler_1_thread_only {\n use super::*;\n\n $($t)*\n\n fn rt() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap()\n }\n }\n }\n}\n\n#[cfg(not(target_os = \"wasi\"))]\nmacro_rules! rt_test {\n ($($t:tt)*) => {\n mod current_thread_scheduler {\n use super::*;\n\n $($t)*\n\n fn rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n }\n }\n\n mod threaded_scheduler_4_threads {\n use super::*;\n\n $($t)*\n\n fn rt() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap()\n }\n }\n\n mod threaded_scheduler_1_thread {\n use super::*;\n\n $($t)*\n\n fn rt() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap()\n }\n }\n }\n}\n\n// ==== runtime independent futures ======\n\n#[test]\nfn basic() {\n test_with_runtimes(|| {\n let one = Handle::current().block_on(async { 1 });\n assert_eq!(1, one);\n });\n}\n\n#[test]\nfn bounded_mpsc_channel() {\n test_with_runtimes(|| {\n let (tx, mut rx) = mpsc::channel(1024);\n\n Handle::current().block_on(tx.send(42)).unwrap();\n\n let value = Handle::current().block_on(rx.recv()).unwrap();\n assert_eq!(value, 42);\n });\n}\n\n#[test]\nfn unbounded_mpsc_channel() {\n test_with_runtimes(|| {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n let _ = tx.send(42);\n\n let value = Handle::current().block_on(rx.recv()).unwrap();\n assert_eq!(value, 42);\n })\n}\n\n#[test]\nfn yield_in_block_in_place() {\n test_with_runtimes(|| {\n Handle::current().block_on(async {\n tokio::task::block_in_place(|| {\n Handle::current().block_on(tokio::task::yield_now());\n });\n });\n })\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support file operations or bind\nrt_test! {\n use tokio::fs;\n // ==== spawn blocking futures ======\n\n #[test]\n fn basic_fs() {\n let rt = rt();\n let _enter = rt.enter();\n\n let contents = Handle::current()\n .block_on(fs::read_to_string(\"Cargo.toml\"))\n .unwrap();\n assert!(contents.contains(\"https://tokio.rs\"));\n }\n\n #[test]\n fn fs_shutdown_before_started() {\n let rt = rt();\n let _enter = rt.enter();\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err: std::io::Error = Handle::current()\n .block_on(fs::read_to_string(\"Cargo.toml\"))\n .unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n\n let inner_err = err.get_ref().expect(\"no inner error\");\n assert_eq!(inner_err.to_string(), \"background task failed\");\n }\n\n #[test]\n fn basic_spawn_blocking() {\n use tokio::task::spawn_blocking;\n let rt = rt();\n let _enter = rt.enter();\n\n let answer = Handle::current()\n .block_on(spawn_blocking(|| {\n std::thread::sleep(Duration::from_millis(100));\n 42\n }))\n .unwrap();\n\n assert_eq!(answer, 42);\n }\n\n #[test]\n fn spawn_blocking_after_shutdown_fails() {\n use tokio::task::spawn_blocking;\n let rt = rt();\n let _enter = rt.enter();\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let join_err = Handle::current()\n .block_on(spawn_blocking(|| {\n std::thread::sleep(Duration::from_millis(100));\n 42\n }))\n .unwrap_err();\n\n assert!(join_err.is_cancelled());\n }\n\n #[test]\n fn spawn_blocking_started_before_shutdown_continues() {\n use tokio::task::spawn_blocking;\n let rt = rt();\n let _enter = rt.enter();\n\n let handle = spawn_blocking(|| {\n std::thread::sleep(Duration::from_secs(1));\n 42\n });\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let answer = Handle::current().block_on(handle).unwrap();\n\n assert_eq!(answer, 42);\n }\n\n #[test]\n fn thread_park_ok() {\n use core::task::Poll;\n let rt = rt();\n let mut exit = false;\n rt.handle().block_on(core::future::poll_fn(move |cx| {\n if exit {\n return Poll::Ready(());\n }\n cx.waker().wake_by_ref();\n // Verify that consuming the park token does not result in a hang.\n std::thread::current().unpark();\n std::thread::park();\n exit = true;\n Poll::Pending\n }));\n }\n\n // ==== net ======\n\n #[test]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n fn tcp_listener_bind() {\n let rt = rt();\n let _enter = rt.enter();\n\n Handle::current()\n .block_on(net::TcpListener::bind(\"127.0.0.1:0\"))\n .unwrap();\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[test]\n fn tcp_listener_connect_after_shutdown() {\n let rt = rt();\n let _enter = rt.enter();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err = Handle::current()\n .block_on(net::TcpListener::bind(\"127.0.0.1:0\"))\n .unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(\n err.get_ref().unwrap().to_string(),\n \"A Tokio 1.x context was found, but it is being shutdown.\",\n );\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[test]\n fn tcp_listener_connect_before_shutdown() {\n let rt = rt();\n let _enter = rt.enter();\n\n let bind_future = net::TcpListener::bind(\"127.0.0.1:0\");\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err = Handle::current().block_on(bind_future).unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(\n err.get_ref().unwrap().to_string(),\n \"A Tokio 1.x context was found, but it is being shutdown.\",\n );\n }\n\n #[test]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n fn udp_socket_bind() {\n let rt = rt();\n let _enter = rt.enter();\n\n Handle::current()\n .block_on(net::UdpSocket::bind(\"127.0.0.1:0\"))\n .unwrap();\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[test]\n fn udp_stream_bind_after_shutdown() {\n let rt = rt();\n let _enter = rt.enter();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err = Handle::current()\n .block_on(net::UdpSocket::bind(\"127.0.0.1:0\"))\n .unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(\n err.get_ref().unwrap().to_string(),\n \"A Tokio 1.x context was found, but it is being shutdown.\",\n );\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[test]\n fn udp_stream_bind_before_shutdown() {\n let rt = rt();\n let _enter = rt.enter();\n\n let bind_future = net::UdpSocket::bind(\"127.0.0.1:0\");\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err = Handle::current().block_on(bind_future).unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(\n err.get_ref().unwrap().to_string(),\n \"A Tokio 1.x context was found, but it is being shutdown.\",\n );\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[cfg(unix)]\n #[test]\n fn unix_listener_bind_after_shutdown() {\n let rt = rt();\n let _enter = rt.enter();\n\n let dir = tempfile::tempdir().unwrap();\n let path = dir.path().join(\"socket\");\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n let err = net::UnixListener::bind(path).unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(\n err.get_ref().unwrap().to_string(),\n \"A Tokio 1.x context was found, but it is being shutdown.\",\n );\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[cfg(unix)]\n #[test]\n fn unix_listener_shutdown_after_bind() {\n let rt = rt();\n let _enter = rt.enter();\n\n let dir = tempfile::tempdir().unwrap();\n let path = dir.path().join(\"socket\");\n\n let listener = net::UnixListener::bind(path).unwrap();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n // this should not timeout but fail immediately since the runtime has been shutdown\n let err = Handle::current().block_on(listener.accept()).unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(err.get_ref().unwrap().to_string(), \"reactor gone\");\n }\n\n // All io tests are ignored for now. See above why that is.\n #[ignore]\n #[cfg(unix)]\n #[test]\n fn unix_listener_shutdown_after_accept() {\n let rt = rt();\n let _enter = rt.enter();\n\n let dir = tempfile::tempdir().unwrap();\n let path = dir.path().join(\"socket\");\n\n let listener = net::UnixListener::bind(path).unwrap();\n\n let accept_future = listener.accept();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n // this should not timeout but fail immediately since the runtime has been shutdown\n let err = Handle::current().block_on(accept_future).unwrap_err();\n\n assert_eq!(err.kind(), std::io::ErrorKind::Other);\n assert_eq!(err.get_ref().unwrap().to_string(), \"reactor gone\");\n }\n\n // ==== nesting ======\n\n #[test]\n #[should_panic(\n expected = \"Cannot start a runtime from within a runtime. This happens because a function (like `block_on`) attempted to block the current thread while the thread is being used to drive asynchronous tasks.\"\n )]\n fn nesting() {\n fn some_non_async_function() -> i32 {\n Handle::current().block_on(time::sleep(Duration::from_millis(10)));\n 1\n }\n\n let rt = rt();\n\n rt.block_on(async { some_non_async_function() });\n }\n\n #[test]\n fn spawn_after_runtime_dropped() {\n use futures::future::FutureExt;\n\n let rt = rt();\n\n let handle = rt.block_on(async move {\n Handle::current()\n });\n\n let jh1 = handle.spawn(futures::future::pending::<()>());\n\n drop(rt);\n\n let jh2 = handle.spawn(futures::future::pending::<()>());\n\n let err1 = jh1.now_or_never().unwrap().unwrap_err();\n let err2 = jh2.now_or_never().unwrap().unwrap_err();\n assert!(err1.is_cancelled());\n assert!(err2.is_cancelled());\n }\n}\n\n#[cfg(not(target_os = \"wasi\"))]\nmulti_threaded_rt_test! {\n #[cfg(unix)]\n #[cfg_attr(miri, ignore)] // No `socket` in miri.\n #[test]\n fn unix_listener_bind() {\n let rt = rt();\n let _enter = rt.enter();\n\n let dir = tempfile::tempdir().unwrap();\n let path = dir.path().join(\"socket\");\n\n let listener = net::UnixListener::bind(path).unwrap();\n\n // this should timeout and not fail immediately since the runtime has not been shutdown\n let _: tokio::time::error::Elapsed = Handle::current()\n .block_on(tokio::time::timeout(\n Duration::from_millis(10),\n listener.accept(),\n ))\n .unwrap_err();\n }\n\n // ==== timers ======\n\n // `Handle::block_on` doesn't work with timer futures on a current thread runtime as there is no\n // one to drive the timers so they will just hang forever. Therefore they are not tested.\n\n #[test]\n fn sleep() {\n let rt = rt();\n let _enter = rt.enter();\n\n Handle::current().block_on(time::sleep(Duration::from_millis(100)));\n }\n\n #[test]\n #[should_panic(expected = \"A Tokio 1.x context was found, but it is being shutdown.\")]\n fn sleep_before_shutdown_panics() {\n let rt = rt();\n let _enter = rt.enter();\n\n let f = time::sleep(Duration::from_millis(100));\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n Handle::current().block_on(f);\n }\n\n #[test]\n #[should_panic(expected = \"A Tokio 1.x context was found, but it is being shutdown.\")]\n fn sleep_after_shutdown_panics() {\n let rt = rt();\n let _enter = rt.enter();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n\n Handle::current().block_on(time::sleep(Duration::from_millis(100)));\n }\n}\n\n// ==== utils ======\n\n/// Create a new multi threaded runtime\n#[cfg(not(target_os = \"wasi\"))]\nfn new_multi_thread(n: usize) -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(n)\n .enable_all()\n .build()\n .unwrap()\n}\n\n/// Create a new single threaded runtime\nfn new_current_thread() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\n/// Utility to test things on both kinds of runtimes both before and after shutting it down.\nfn test_with_runtimes(f: F)\nwhere\n F: Fn(),\n{\n {\n let rt = new_current_thread();\n let _enter = rt.enter();\n f();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n f();\n }\n\n #[cfg(not(target_os = \"wasi\"))]\n {\n let rt = new_multi_thread(1);\n let _enter = rt.enter();\n f();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n f();\n }\n\n #[cfg(not(target_os = \"wasi\"))]\n {\n let rt = new_multi_thread(4);\n let _enter = rt.enter();\n f();\n\n rt.shutdown_timeout(Duration::from_secs(1000));\n f();\n }\n}\n" }, { "path": "tokio/tests/rt_local.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", tokio_unstable))]\n\nuse std::panic;\nuse tokio::runtime::LocalOptions;\nuse tokio::task::spawn_local;\nuse tokio::task::LocalSet;\n\n#[test]\nfn test_spawn_local_in_runtime() {\n let rt = rt();\n\n let res = rt.block_on(async move {\n let (tx, rx) = tokio::sync::oneshot::channel();\n\n spawn_local(async {\n tokio::task::yield_now().await;\n tx.send(5).unwrap();\n });\n\n rx.await.unwrap()\n });\n\n assert_eq!(res, 5);\n}\n\n#[test]\nfn test_spawn_from_handle() {\n let rt = rt();\n\n let (tx, rx) = tokio::sync::oneshot::channel();\n\n rt.handle().spawn(async {\n tokio::task::yield_now().await;\n tx.send(5).unwrap();\n });\n\n let res = rt.block_on(async move { rx.await.unwrap() });\n\n assert_eq!(res, 5);\n}\n\n#[test]\nfn test_spawn_local_on_runtime_object() {\n let rt = rt();\n\n let (tx, rx) = tokio::sync::oneshot::channel();\n\n rt.spawn_local(async {\n tokio::task::yield_now().await;\n tx.send(5).unwrap();\n });\n\n let res = rt.block_on(async move { rx.await.unwrap() });\n\n assert_eq!(res, 5);\n}\n\n#[test]\nfn test_spawn_local_from_guard() {\n let rt = rt();\n\n let (tx, rx) = tokio::sync::oneshot::channel();\n\n let _guard = rt.enter();\n\n spawn_local(async {\n tokio::task::yield_now().await;\n tx.send(5).unwrap();\n });\n\n let res = rt.block_on(async move { rx.await.unwrap() });\n\n assert_eq!(res, 5);\n}\n\n#[test]\n#[cfg_attr(target_family = \"wasm\", ignore)] // threads not supported\nfn test_spawn_from_guard_other_thread() {\n let (tx, rx) = std::sync::mpsc::channel();\n\n std::thread::spawn(move || {\n let rt = rt();\n let handle = rt.handle().clone();\n\n tx.send(handle).unwrap();\n });\n\n let handle = rx.recv().unwrap();\n\n let _guard = handle.enter();\n\n tokio::spawn(async {});\n}\n\n#[test]\n#[should_panic = \"Local tasks can only be spawned on a LocalRuntime from the thread the runtime was created on\"]\n#[cfg_attr(target_family = \"wasm\", ignore)] // threads not supported\nfn test_spawn_local_from_guard_other_thread() {\n let (tx, rx) = std::sync::mpsc::channel();\n\n std::thread::spawn(move || {\n let rt = rt();\n let handle = rt.handle().clone();\n\n tx.send(handle).unwrap();\n });\n\n let handle = rx.recv().unwrap();\n\n let _guard = handle.enter();\n\n spawn_local(async {});\n}\n\n// This test guarantees that **`tokio::task::spawn_local` panics** when it is invoked\n// from a thread that is *not* running the `LocalRuntime` / `LocalSet` to which\n// the task would belong.\n// The test creates a `LocalRuntime` and `LocalSet`, drives the `LocalSet` on the `LocalRuntime`'s thread,\n// then spawns a **separate OS thread** and tries to call\n// `tokio::task::spawn_local` there. `std::panic::catch_unwind` is then used\n// to capture the panic and to assert that it indeed occurs.\n#[test]\n#[cfg_attr(target_family = \"wasm\", ignore)] // threads not supported\nfn test_spawn_local_panic() {\n let rt = rt();\n let local = LocalSet::new();\n\n rt.block_on(local.run_until(async {\n let thread_result = std::thread::spawn(|| {\n let panic_result = panic::catch_unwind(|| {\n let _jh = tokio::task::spawn_local(async {\n println!(\"you will never see this line\");\n });\n });\n assert!(panic_result.is_err(), \"Expected panic, but none occurred\");\n })\n .join();\n assert!(thread_result.is_ok(), \"Thread itself panicked unexpectedly\");\n }));\n}\n\n#[test]\n#[should_panic = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"]\nfn test_spawn_local_in_current_thread_runtime() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n rt.block_on(async move {\n spawn_local(async {});\n })\n}\n\n#[test]\n#[should_panic = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"]\nfn test_spawn_local_in_multi_thread_runtime() {\n let rt = tokio::runtime::Builder::new_multi_thread().build().unwrap();\n\n rt.block_on(async move {\n spawn_local(async {});\n })\n}\n\nfn rt() -> tokio::runtime::LocalRuntime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build_local(LocalOptions::default())\n .unwrap()\n}\n" }, { "path": "tokio/tests/rt_metrics.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), target_has_atomic = \"64\"))]\n\nuse std::sync::mpsc;\nuse std::time::Duration;\nuse tokio::runtime::Runtime;\nuse tokio::time;\n\n#[test]\nfn num_workers() {\n let rt = current_thread();\n assert_eq!(1, rt.metrics().num_workers());\n\n let rt = threaded();\n assert_eq!(2, rt.metrics().num_workers());\n}\n\n#[test]\nfn num_alive_tasks() {\n let rt = current_thread();\n let metrics = rt.metrics();\n assert_eq!(0, metrics.num_alive_tasks());\n rt.block_on(rt.spawn(async move {\n assert_eq!(1, metrics.num_alive_tasks());\n }))\n .unwrap();\n\n assert_eq!(0, rt.metrics().num_alive_tasks());\n\n let rt = threaded();\n let metrics = rt.metrics();\n assert_eq!(0, metrics.num_alive_tasks());\n rt.block_on(rt.spawn(async move {\n assert_eq!(1, metrics.num_alive_tasks());\n }))\n .unwrap();\n\n // try for 10 seconds to see if this eventually succeeds.\n // wake_join() is called before the task is released, so in multithreaded\n // code, this means we sometimes exit the block_on before the counter decrements.\n for _ in 0..100 {\n if rt.metrics().num_alive_tasks() == 0 {\n break;\n }\n std::thread::sleep(std::time::Duration::from_millis(100));\n }\n assert_eq!(0, rt.metrics().num_alive_tasks());\n}\n\n#[test]\nfn global_queue_depth_current_thread() {\n use std::thread;\n\n let rt = current_thread();\n let handle = rt.handle().clone();\n let metrics = rt.metrics();\n\n thread::spawn(move || {\n handle.spawn(async {});\n })\n .join()\n .unwrap();\n\n assert_eq!(1, metrics.global_queue_depth());\n}\n\n#[test]\nfn global_queue_depth_multi_thread() {\n for _ in 0..10 {\n let rt = threaded();\n let metrics = rt.metrics();\n\n if let Ok(_blocking_tasks) = try_block_threaded(&rt) {\n for i in 0..10 {\n assert_eq!(i, metrics.global_queue_depth());\n rt.spawn(async {});\n }\n\n return;\n }\n }\n\n panic!(\"exhausted every try to block the runtime\");\n}\n\n#[test]\nfn worker_total_busy_duration() {\n const N: usize = 5;\n\n let zero = Duration::from_millis(0);\n\n let rt = current_thread();\n let metrics = rt.metrics();\n\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(async {\n tokio::task::yield_now().await;\n })\n .await\n .unwrap();\n }\n });\n\n drop(rt);\n\n assert!(zero < metrics.worker_total_busy_duration(0));\n\n let rt = threaded();\n let metrics = rt.metrics();\n\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(async {\n tokio::task::yield_now().await;\n })\n .await\n .unwrap();\n }\n });\n\n drop(rt);\n\n for i in 0..metrics.num_workers() {\n assert!(zero < metrics.worker_total_busy_duration(i));\n }\n}\n\n#[test]\nfn worker_park_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(async {\n time::sleep(Duration::from_millis(1)).await;\n });\n drop(rt);\n assert!(1 <= metrics.worker_park_count(0));\n\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async {\n time::sleep(Duration::from_millis(1)).await;\n });\n drop(rt);\n assert!(1 <= metrics.worker_park_count(0));\n assert!(1 <= metrics.worker_park_count(1));\n}\n\n#[test]\nfn worker_park_unpark_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(rt.spawn(async {})).unwrap();\n drop(rt);\n assert_eq!(0, metrics.worker_park_unpark_count(0) % 2);\n\n let rt = threaded();\n let metrics = rt.metrics();\n\n // Wait for workers to be parked after runtime startup.\n for _ in 0..100 {\n if 1 <= metrics.worker_park_unpark_count(0) && 1 <= metrics.worker_park_unpark_count(1) {\n break;\n }\n std::thread::sleep(std::time::Duration::from_millis(100));\n }\n assert_eq!(1, metrics.worker_park_unpark_count(0));\n assert_eq!(1, metrics.worker_park_unpark_count(1));\n\n // Spawn a task to unpark and then park a worker.\n rt.block_on(rt.spawn(async {})).unwrap();\n for _ in 0..100 {\n if 3 <= metrics.worker_park_unpark_count(0) || 3 <= metrics.worker_park_unpark_count(1) {\n break;\n }\n std::thread::sleep(std::time::Duration::from_millis(100));\n }\n assert!(3 <= metrics.worker_park_unpark_count(0) || 3 <= metrics.worker_park_unpark_count(1));\n\n // Both threads unpark for runtime shutdown.\n drop(rt);\n assert_eq!(0, metrics.worker_park_unpark_count(0) % 2);\n assert_eq!(0, metrics.worker_park_unpark_count(1) % 2);\n assert!(4 <= metrics.worker_park_unpark_count(0) || 4 <= metrics.worker_park_unpark_count(1));\n}\n\nfn try_block_threaded(rt: &Runtime) -> Result>, mpsc::RecvTimeoutError> {\n let (tx, rx) = mpsc::channel();\n\n let blocking_tasks = (0..rt.metrics().num_workers())\n .map(|_| {\n let tx = tx.clone();\n let (task, barrier) = mpsc::channel();\n\n // Spawn a task per runtime worker to block it.\n rt.spawn(async move {\n tx.send(()).ok();\n barrier.recv().ok();\n });\n\n task\n })\n .collect();\n\n // Make sure the previously spawned tasks are blocking the runtime by\n // receiving a message from each blocking task.\n //\n // If this times out we were unsuccessful in blocking the runtime and hit\n // a deadlock instead (which might happen and is expected behaviour).\n for _ in 0..rt.metrics().num_workers() {\n rx.recv_timeout(Duration::from_secs(1))?;\n }\n\n // Return senders of the mpsc channels used for blocking the runtime as a\n // surrogate handle for the tasks. Sending a message or dropping the senders\n // will unblock the runtime.\n Ok(blocking_tasks)\n}\n\nfn current_thread() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\nfn threaded() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/rt_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support panic recovery\n#![cfg(panic = \"unwind\")]\n\nuse futures::future;\nuse std::error::Error;\nuse tokio::runtime::{Builder, Handle, Runtime};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[test]\nfn current_handle_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = Handle::current();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn into_panic_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(move || {\n let rt = current_thread();\n rt.block_on(async {\n let handle = tokio::spawn(future::pending::<()>());\n\n handle.abort();\n\n let err = handle.await.unwrap_err();\n assert!(!&err.is_panic());\n\n let _ = err.into_panic();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn builder_worker_threads_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = Builder::new_multi_thread().worker_threads(0).build();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn builder_max_blocking_threads_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = Builder::new_multi_thread().max_blocking_threads(0).build();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn builder_global_queue_interval_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = Builder::new_multi_thread().global_queue_interval(0).build();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn builder_event_interval_interval_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = Builder::new_multi_thread().event_interval(0).build();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\nfn current_thread() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/rt_poll_callbacks.rs", "content": "#![allow(unknown_lints, unexpected_cfgs)]\n#![cfg(tokio_unstable)]\n\nuse std::sync::{atomic::AtomicUsize, Arc, Mutex};\n\nuse tokio::task::yield_now;\n\n#[cfg(not(target_os = \"wasi\"))]\n#[test]\nfn callbacks_fire_multi_thread() {\n let poll_start_counter = Arc::new(AtomicUsize::new(0));\n let poll_stop_counter = Arc::new(AtomicUsize::new(0));\n let poll_start = poll_start_counter.clone();\n let poll_stop = poll_stop_counter.clone();\n\n let before_task_poll_callback_task_id: Arc>> =\n Arc::new(Mutex::new(None));\n let after_task_poll_callback_task_id: Arc>> =\n Arc::new(Mutex::new(None));\n\n let before_task_poll_callback_task_id_ref = Arc::clone(&before_task_poll_callback_task_id);\n let after_task_poll_callback_task_id_ref = Arc::clone(&after_task_poll_callback_task_id);\n let rt = tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .on_before_task_poll(move |task_meta| {\n before_task_poll_callback_task_id_ref\n .lock()\n .unwrap()\n .replace(task_meta.id());\n poll_start_counter.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n })\n .on_after_task_poll(move |task_meta| {\n after_task_poll_callback_task_id_ref\n .lock()\n .unwrap()\n .replace(task_meta.id());\n poll_stop_counter.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n })\n .build()\n .unwrap();\n let task = rt.spawn(async {\n yield_now().await;\n yield_now().await;\n yield_now().await;\n });\n\n let spawned_task_id = task.id();\n\n rt.block_on(task).expect(\"task should succeed\");\n // We need to drop the runtime to guarantee the workers have exited (and thus called the callback)\n drop(rt);\n\n assert_eq!(\n before_task_poll_callback_task_id.lock().unwrap().unwrap(),\n spawned_task_id\n );\n assert_eq!(\n after_task_poll_callback_task_id.lock().unwrap().unwrap(),\n spawned_task_id\n );\n let actual_count = 4;\n assert_eq!(\n poll_start.load(std::sync::atomic::Ordering::Relaxed),\n actual_count,\n \"unexpected number of poll starts\"\n );\n assert_eq!(\n poll_stop.load(std::sync::atomic::Ordering::Relaxed),\n actual_count,\n \"unexpected number of poll stops\"\n );\n}\n\n#[test]\nfn callbacks_fire_current_thread() {\n let poll_start_counter = Arc::new(AtomicUsize::new(0));\n let poll_stop_counter = Arc::new(AtomicUsize::new(0));\n let poll_start = poll_start_counter.clone();\n let poll_stop = poll_stop_counter.clone();\n\n let before_task_poll_callback_task_id: Arc>> =\n Arc::new(Mutex::new(None));\n let after_task_poll_callback_task_id: Arc>> =\n Arc::new(Mutex::new(None));\n\n let before_task_poll_callback_task_id_ref = Arc::clone(&before_task_poll_callback_task_id);\n let after_task_poll_callback_task_id_ref = Arc::clone(&after_task_poll_callback_task_id);\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .on_before_task_poll(move |task_meta| {\n before_task_poll_callback_task_id_ref\n .lock()\n .unwrap()\n .replace(task_meta.id());\n poll_start_counter.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n })\n .on_after_task_poll(move |task_meta| {\n after_task_poll_callback_task_id_ref\n .lock()\n .unwrap()\n .replace(task_meta.id());\n poll_stop_counter.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n })\n .build()\n .unwrap();\n\n let task = rt.spawn(async {\n yield_now().await;\n yield_now().await;\n yield_now().await;\n });\n\n let spawned_task_id = task.id();\n\n let _ = rt.block_on(task);\n drop(rt);\n\n assert_eq!(\n before_task_poll_callback_task_id.lock().unwrap().unwrap(),\n spawned_task_id\n );\n assert_eq!(\n after_task_poll_callback_task_id.lock().unwrap().unwrap(),\n spawned_task_id\n );\n assert_eq!(poll_start.load(std::sync::atomic::Ordering::Relaxed), 4);\n assert_eq!(poll_stop.load(std::sync::atomic::Ordering::Relaxed), 4);\n}\n" }, { "path": "tokio/tests/rt_shutdown_err.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(miri))] // No socket in miri.\n\nuse std::io;\nuse tokio::net::TcpListener;\nuse tokio::runtime::Builder;\n\nfn rt() -> tokio::runtime::Runtime {\n Builder::new_current_thread().enable_all().build().unwrap()\n}\n\n#[test]\nfn test_is_rt_shutdown_err() {\n let rt1 = rt();\n let rt2 = rt();\n\n let listener = rt1.block_on(async { TcpListener::bind(\"127.0.0.1:0\").await.unwrap() });\n\n drop(rt1);\n\n rt2.block_on(async {\n let res = listener.accept().await;\n assert!(res.is_err());\n let err = res.as_ref().unwrap_err();\n assert!(tokio::runtime::is_rt_shutdown_err(err));\n });\n}\n\n#[test]\nfn test_is_not_rt_shutdown_err() {\n let err = io::Error::new(io::ErrorKind::Other, \"some other error\");\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n\n let err = io::Error::new(io::ErrorKind::NotFound, \"not found\");\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n}\n\n#[test]\n#[cfg_attr(panic = \"abort\", ignore)]\nfn test_join_error_panic() {\n let rt = rt();\n let handle = rt.spawn(async {\n panic!(\"oops\");\n });\n\n let join_err = rt.block_on(handle).unwrap_err();\n let io_err: io::Error = join_err.into();\n assert!(!tokio::runtime::is_rt_shutdown_err(&io_err));\n}\n\n#[test]\nfn test_join_error_cancelled() {\n let rt = rt();\n let handle = rt.spawn(async {\n std::future::pending::<()>().await;\n });\n handle.abort();\n let join_err = rt.block_on(handle).unwrap_err();\n let io_err: io::Error = join_err.into();\n assert!(!tokio::runtime::is_rt_shutdown_err(&io_err));\n}\n\n#[test]\nfn test_other_error_kinds_and_strings() {\n // TimedOut\n let err = io::Error::new(io::ErrorKind::TimedOut, \"timed out\");\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n\n // Interrupted\n let err = io::Error::from(io::ErrorKind::Interrupted);\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n\n // String that contains the shutdown message but has a prefix/suffix\n let msg = \"A Tokio 1.x context was found, but it is being shutdown. (extra info)\";\n let err = io::Error::new(io::ErrorKind::Other, msg);\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n\n let msg = \"Error: A Tokio 1.x context was found, but it is being shutdown.\";\n let err = io::Error::new(io::ErrorKind::Other, msg);\n assert!(!tokio::runtime::is_rt_shutdown_err(&err));\n}\n" }, { "path": "tokio/tests/rt_threaded.rs", "content": "#![warn(rust_2018_idioms)]\n// Too slow on miri.\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))]\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::runtime;\nuse tokio::sync::oneshot;\nuse tokio_test::{assert_err, assert_ok, assert_pending};\n\nuse std::future::{poll_fn, Future};\nuse std::pin::Pin;\nuse std::sync::atomic::Ordering::Relaxed;\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::{mpsc, Arc, Mutex};\nuse std::task::{Context, Poll, Waker};\n\nmacro_rules! cfg_metrics {\n ($($t:tt)*) => {\n #[cfg(all(tokio_unstable, target_has_atomic = \"64\"))]\n {\n $( $t )*\n }\n }\n}\n\n#[test]\nfn single_thread() {\n // No panic when starting a runtime w/ a single thread\n let _ = runtime::Builder::new_multi_thread()\n .enable_all()\n .worker_threads(1)\n .build()\n .unwrap();\n}\n\n#[test]\nfn many_oneshot_futures() {\n // used for notifying the main thread\n const NUM: usize = 1_000;\n\n for _ in 0..5 {\n let (tx, rx) = mpsc::channel();\n\n let rt = rt();\n let cnt = Arc::new(AtomicUsize::new(0));\n\n for _ in 0..NUM {\n let cnt = cnt.clone();\n let tx = tx.clone();\n\n rt.spawn(async move {\n let num = cnt.fetch_add(1, Relaxed) + 1;\n\n if num == NUM {\n tx.send(()).unwrap();\n }\n });\n }\n\n rx.recv().unwrap();\n\n // Wait for the pool to shutdown\n drop(rt);\n }\n}\n\n#[test]\nfn spawn_two() {\n let rt = rt();\n\n let out = rt.block_on(async {\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n tokio::spawn(async move {\n tx.send(\"ZOMG\").unwrap();\n });\n });\n\n assert_ok!(rx.await)\n });\n\n assert_eq!(out, \"ZOMG\");\n\n cfg_metrics! {\n let metrics = rt.metrics();\n drop(rt);\n assert_eq!(1, metrics.remote_schedule_count());\n\n let mut local = 0;\n for i in 0..metrics.num_workers() {\n local += metrics.worker_local_schedule_count(i);\n }\n\n assert_eq!(1, local);\n }\n}\n\n#[test]\nfn many_multishot_futures() {\n const CHAIN: usize = 200;\n const CYCLES: usize = 5;\n const TRACKS: usize = 50;\n\n for _ in 0..50 {\n let rt = rt();\n let mut start_txs = Vec::with_capacity(TRACKS);\n let mut final_rxs = Vec::with_capacity(TRACKS);\n\n for _ in 0..TRACKS {\n let (start_tx, mut chain_rx) = tokio::sync::mpsc::channel(10);\n\n for _ in 0..CHAIN {\n let (next_tx, next_rx) = tokio::sync::mpsc::channel(10);\n\n // Forward all the messages\n rt.spawn(async move {\n while let Some(v) = chain_rx.recv().await {\n next_tx.send(v).await.unwrap();\n }\n });\n\n chain_rx = next_rx;\n }\n\n // This final task cycles if needed\n let (final_tx, final_rx) = tokio::sync::mpsc::channel(10);\n let cycle_tx = start_tx.clone();\n let mut rem = CYCLES;\n\n rt.spawn(async move {\n for _ in 0..CYCLES {\n let msg = chain_rx.recv().await.unwrap();\n\n rem -= 1;\n\n if rem == 0 {\n final_tx.send(msg).await.unwrap();\n } else {\n cycle_tx.send(msg).await.unwrap();\n }\n }\n });\n\n start_txs.push(start_tx);\n final_rxs.push(final_rx);\n }\n\n {\n rt.block_on(async move {\n for start_tx in start_txs {\n start_tx.send(\"ping\").await.unwrap();\n }\n\n for mut final_rx in final_rxs {\n final_rx.recv().await.unwrap();\n }\n });\n }\n }\n}\n\n#[test]\nfn lifo_slot_budget() {\n async fn my_fn() {\n spawn_another();\n }\n\n fn spawn_another() {\n tokio::spawn(my_fn());\n }\n\n let rt = runtime::Builder::new_multi_thread()\n .enable_all()\n .worker_threads(1)\n .build()\n .unwrap();\n\n let (send, recv) = oneshot::channel();\n\n rt.spawn(async move {\n tokio::spawn(my_fn());\n let _ = send.send(());\n });\n\n let _ = rt.block_on(recv);\n}\n\n#[test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nfn spawn_shutdown() {\n let rt = rt();\n let (tx, rx) = mpsc::channel();\n\n rt.block_on(async {\n tokio::spawn(client_server(tx.clone()));\n });\n\n // Use spawner\n rt.spawn(client_server(tx));\n\n assert_ok!(rx.recv());\n assert_ok!(rx.recv());\n\n drop(rt);\n assert_err!(rx.try_recv());\n}\n\nasync fn client_server(tx: mpsc::Sender<()>) {\n let server = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n\n // Get the assigned address\n let addr = assert_ok!(server.local_addr());\n\n // Spawn the server\n tokio::spawn(async move {\n // Accept a socket\n let (mut socket, _) = server.accept().await.unwrap();\n\n // Write some data\n socket.write_all(b\"hello\").await.unwrap();\n });\n\n let mut client = TcpStream::connect(&addr).await.unwrap();\n\n let mut buf = vec![];\n client.read_to_end(&mut buf).await.unwrap();\n\n assert_eq!(buf, b\"hello\");\n tx.send(()).unwrap();\n}\n\n#[test]\nfn drop_threadpool_drops_futures() {\n for _ in 0..1_000 {\n let num_inc = Arc::new(AtomicUsize::new(0));\n let num_dec = Arc::new(AtomicUsize::new(0));\n let num_drop = Arc::new(AtomicUsize::new(0));\n\n struct Never(Arc);\n\n impl Future for Never {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Pending\n }\n }\n\n impl Drop for Never {\n fn drop(&mut self) {\n self.0.fetch_add(1, Relaxed);\n }\n }\n\n let a = num_inc.clone();\n let b = num_dec.clone();\n\n let rt = runtime::Builder::new_multi_thread()\n .enable_all()\n .on_thread_start(move || {\n a.fetch_add(1, Relaxed);\n })\n .on_thread_stop(move || {\n b.fetch_add(1, Relaxed);\n })\n .build()\n .unwrap();\n\n rt.spawn(Never(num_drop.clone()));\n\n // Wait for the pool to shutdown\n drop(rt);\n\n // Assert that only a single thread was spawned.\n let a = num_inc.load(Relaxed);\n assert!(a >= 1);\n\n // Assert that all threads shutdown\n let b = num_dec.load(Relaxed);\n assert_eq!(a, b);\n\n // Assert that the future was dropped\n let c = num_drop.load(Relaxed);\n assert_eq!(c, 1);\n }\n}\n\n#[test]\nfn start_stop_callbacks_called() {\n use std::sync::atomic::{AtomicUsize, Ordering};\n\n let after_start = Arc::new(AtomicUsize::new(0));\n let before_stop = Arc::new(AtomicUsize::new(0));\n\n let after_inner = after_start.clone();\n let before_inner = before_stop.clone();\n let rt = tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .on_thread_start(move || {\n after_inner.clone().fetch_add(1, Ordering::Relaxed);\n })\n .on_thread_stop(move || {\n before_inner.clone().fetch_add(1, Ordering::Relaxed);\n })\n .build()\n .unwrap();\n\n let (tx, rx) = oneshot::channel();\n\n rt.spawn(async move {\n assert_ok!(tx.send(()));\n });\n\n assert_ok!(rt.block_on(rx));\n\n drop(rt);\n\n assert!(after_start.load(Ordering::Relaxed) > 0);\n assert!(before_stop.load(Ordering::Relaxed) > 0);\n}\n\n#[test]\n// too slow on miri\n#[cfg_attr(miri, ignore)]\nfn blocking() {\n // used for notifying the main thread\n const NUM: usize = 1_000;\n\n for _ in 0..10 {\n let (tx, rx) = mpsc::channel();\n\n let rt = rt();\n let cnt = Arc::new(AtomicUsize::new(0));\n\n // there are four workers in the pool\n // so, if we run 4 blocking tasks, we know that handoff must have happened\n let block = Arc::new(std::sync::Barrier::new(5));\n for _ in 0..4 {\n let block = block.clone();\n rt.spawn(async move {\n tokio::task::block_in_place(move || {\n block.wait();\n block.wait();\n })\n });\n }\n block.wait();\n\n for _ in 0..NUM {\n let cnt = cnt.clone();\n let tx = tx.clone();\n\n rt.spawn(async move {\n let num = cnt.fetch_add(1, Relaxed) + 1;\n\n if num == NUM {\n tx.send(()).unwrap();\n }\n });\n }\n\n rx.recv().unwrap();\n\n // Wait for the pool to shutdown\n block.wait();\n }\n}\n\n#[test]\nfn multi_threadpool() {\n use tokio::sync::oneshot;\n\n let rt1 = rt();\n let rt2 = rt();\n\n let (tx, rx) = oneshot::channel();\n let (done_tx, done_rx) = mpsc::channel();\n\n rt2.spawn(async move {\n rx.await.unwrap();\n done_tx.send(()).unwrap();\n });\n\n rt1.spawn(async move {\n tx.send(()).unwrap();\n });\n\n done_rx.recv().unwrap();\n}\n\n// When `block_in_place` returns, it attempts to reclaim the yielded runtime\n// worker. In this case, the remainder of the task is on the runtime worker and\n// must take part in the cooperative task budgeting system.\n//\n// The test ensures that, when this happens, attempting to consume from a\n// channel yields occasionally even if there are values ready to receive.\n#[test]\nfn coop_and_block_in_place() {\n let rt = tokio::runtime::Builder::new_multi_thread()\n // Setting max threads to 1 prevents another thread from claiming the\n // runtime worker yielded as part of `block_in_place` and guarantees the\n // same thread will reclaim the worker at the end of the\n // `block_in_place` call.\n .max_blocking_threads(1)\n .build()\n .unwrap();\n\n rt.block_on(async move {\n let (tx, mut rx) = tokio::sync::mpsc::channel(1024);\n\n // Fill the channel\n for _ in 0..1024 {\n tx.send(()).await.unwrap();\n }\n\n drop(tx);\n\n tokio::spawn(async move {\n // Block in place without doing anything\n tokio::task::block_in_place(|| {});\n\n // Receive all the values, this should trigger a `Pending` as the\n // coop limit will be reached.\n poll_fn(|cx| {\n while let Poll::Ready(v) = {\n tokio::pin! {\n let fut = rx.recv();\n }\n\n Pin::new(&mut fut).poll(cx)\n } {\n if v.is_none() {\n panic!(\"did not yield\");\n }\n }\n\n Poll::Ready(())\n })\n .await\n })\n .await\n .unwrap();\n });\n}\n\n#[test]\nfn yield_after_block_in_place() {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n rt.block_on(async {\n tokio::spawn(async move {\n // Block in place then enter a new runtime\n tokio::task::block_in_place(|| {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n rt.block_on(async {});\n });\n\n // Yield, then complete\n tokio::task::yield_now().await;\n })\n .await\n .unwrap()\n });\n}\n\n// Testing this does not panic\n#[test]\nfn max_blocking_threads() {\n let _rt = tokio::runtime::Builder::new_multi_thread()\n .max_blocking_threads(1)\n .build()\n .unwrap();\n}\n\n#[test]\n#[should_panic]\nfn max_blocking_threads_set_to_zero() {\n let _rt = tokio::runtime::Builder::new_multi_thread()\n .max_blocking_threads(0)\n .build()\n .unwrap();\n}\n\n/// Regression test for #6445.\n///\n/// After #6445, setting `global_queue_interval` to 1 is now technically valid.\n/// This test confirms that there is no regression in `multi_thread_runtime`\n/// when global_queue_interval is set to 1.\n#[test]\nfn global_queue_interval_set_to_one() {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .global_queue_interval(1)\n .build()\n .unwrap();\n\n // Perform a simple work.\n let cnt = Arc::new(AtomicUsize::new(0));\n rt.block_on(async {\n let mut set = tokio::task::JoinSet::new();\n for _ in 0..10 {\n let cnt = cnt.clone();\n set.spawn(async move { cnt.fetch_add(1, Ordering::Relaxed) });\n }\n\n while let Some(res) = set.join_next().await {\n res.unwrap();\n }\n });\n assert_eq!(cnt.load(Relaxed), 10);\n}\n\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 2)]\nasync fn hang_on_shutdown() {\n let (sync_tx, sync_rx) = std::sync::mpsc::channel::<()>();\n tokio::spawn(async move {\n tokio::task::block_in_place(|| sync_rx.recv().ok());\n });\n\n tokio::spawn(async {\n tokio::time::sleep(std::time::Duration::from_secs(2)).await;\n drop(sync_tx);\n });\n tokio::time::sleep(std::time::Duration::from_secs(1)).await;\n}\n\n/// Demonstrates tokio-rs/tokio#3869\n#[test]\nfn wake_during_shutdown() {\n struct Shared {\n waker: Option,\n }\n\n struct MyFuture {\n shared: Arc>,\n put_waker: bool,\n }\n\n impl MyFuture {\n fn new() -> (Self, Self) {\n let shared = Arc::new(Mutex::new(Shared { waker: None }));\n let f1 = MyFuture {\n shared: shared.clone(),\n put_waker: true,\n };\n let f2 = MyFuture {\n shared,\n put_waker: false,\n };\n (f1, f2)\n }\n }\n\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n let me = Pin::into_inner(self);\n let mut lock = me.shared.lock().unwrap();\n if me.put_waker {\n lock.waker = Some(cx.waker().clone());\n }\n Poll::Pending\n }\n }\n\n impl Drop for MyFuture {\n fn drop(&mut self) {\n let mut lock = self.shared.lock().unwrap();\n if !self.put_waker {\n lock.waker.take().unwrap().wake();\n }\n drop(lock);\n }\n }\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n\n let (f1, f2) = MyFuture::new();\n\n rt.spawn(f1);\n rt.spawn(f2);\n\n rt.block_on(async { tokio::time::sleep(tokio::time::Duration::from_millis(20)).await });\n}\n\n#[should_panic]\n#[tokio::test]\nasync fn test_block_in_place1() {\n tokio::task::block_in_place(|| {});\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn test_block_in_place2() {\n tokio::task::block_in_place(|| {});\n}\n\n#[should_panic]\n#[tokio::main(flavor = \"current_thread\")]\n#[test]\nasync fn test_block_in_place3() {\n tokio::task::block_in_place(|| {});\n}\n\n#[tokio::main]\n#[test]\nasync fn test_block_in_place4() {\n tokio::task::block_in_place(|| {});\n}\n\n// Repro for tokio-rs/tokio#5239\n#[test]\nfn test_nested_block_in_place_with_block_on_between() {\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(1)\n // Needs to be more than 0\n .max_blocking_threads(1)\n .build()\n .unwrap();\n\n // Triggered by a race condition, so run a few times to make sure it is OK.\n for _ in 0..100 {\n let h = rt.handle().clone();\n\n rt.block_on(async move {\n tokio::spawn(async move {\n tokio::task::block_in_place(|| {\n h.block_on(async {\n tokio::task::block_in_place(|| {});\n });\n })\n })\n .await\n .unwrap()\n });\n }\n}\n\n#[test]\nfn yield_now_in_block_in_place() {\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n rt.block_on(async {\n tokio::spawn(async {\n tokio::task::block_in_place(|| {\n tokio::runtime::Handle::current().block_on(tokio::task::yield_now());\n })\n })\n .await\n .unwrap()\n })\n}\n\n#[test]\nfn mutex_in_block_in_place() {\n const BUDGET: usize = 128;\n\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n rt.block_on(async {\n let lock = tokio::sync::Mutex::new(0);\n\n tokio::spawn(async move {\n tokio::task::block_in_place(|| {\n tokio::runtime::Handle::current().block_on(async move {\n for i in 0..(BUDGET + 1) {\n let mut guard = lock.lock().await;\n *guard = i;\n }\n });\n })\n })\n .await\n .unwrap();\n })\n}\n\n#[test]\n/// Deferred tasks should be woken before starting the [`tokio::task::block_in_place`]\n// https://github.com/tokio-rs/tokio/issues/7877\nfn wake_deferred_tasks_before_block_in_place() {\n let (tx1, rx1) = oneshot::channel::<()>();\n let (tx2, rx2) = oneshot::channel::<()>();\n\n let deferred_task = tokio_test::task::spawn(tokio::task::yield_now());\n let deffered_task = Arc::new(Mutex::new(deferred_task));\n\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n let jh = {\n let deferred_task = Arc::clone(&deffered_task);\n rt.spawn(async move {\n {\n let mut lock = deferred_task.lock().unwrap();\n assert_pending!(lock.poll());\n }\n tokio::task::block_in_place(|| {\n // signal that the `block_in_place` has started\n tx2.send(()).unwrap();\n // wait for the shutdown signal\n rx1.blocking_recv().unwrap();\n });\n })\n };\n\n // wait for the `block_in_place` to start\n rx2.blocking_recv().unwrap();\n\n // check that the deferred task was woken before the `block_in_place` ends\n let is_woken = {\n let lock = deffered_task.lock().unwrap();\n lock.is_woken()\n };\n\n // signal the `block_in_place` to shutdown\n tx1.send(()).unwrap();\n\n rt.block_on(jh).unwrap();\n\n assert!(is_woken);\n}\n\n// Testing the tuning logic is tricky as it is inherently timing based, and more\n// of a heuristic than an exact behavior. This test checks that the interval\n// changes over time based on load factors. There are no assertions, completion\n// is sufficient. If there is a regression, this test will hang. In theory, we\n// could add limits, but that would be likely to fail on CI.\n#[test]\n#[cfg(not(tokio_no_tuning_tests))]\nfn test_tuning() {\n use std::sync::atomic::AtomicBool;\n use std::time::Duration;\n\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .build()\n .unwrap();\n\n fn iter(flag: Arc, counter: Arc, stall: bool) {\n if flag.load(Relaxed) {\n if stall {\n std::thread::sleep(Duration::from_micros(5));\n }\n\n counter.fetch_add(1, Relaxed);\n tokio::spawn(async move { iter(flag, counter, stall) });\n }\n }\n\n let flag = Arc::new(AtomicBool::new(true));\n let counter = Arc::new(AtomicUsize::new(61));\n let interval = Arc::new(AtomicUsize::new(61));\n\n {\n let flag = flag.clone();\n let counter = counter.clone();\n rt.spawn(async move { iter(flag, counter, true) });\n }\n\n // Now, hammer the injection queue until the interval drops.\n let mut n = 0;\n loop {\n let curr = interval.load(Relaxed);\n\n if curr <= 8 {\n n += 1;\n } else {\n n = 0;\n }\n\n // Make sure we get a few good rounds. Jitter in the tuning could result\n // in one \"good\" value without being representative of reaching a good\n // state.\n if n == 3 {\n break;\n }\n\n if Arc::strong_count(&interval) < 5_000 {\n let counter = counter.clone();\n let interval = interval.clone();\n\n rt.spawn(async move {\n let prev = counter.swap(0, Relaxed);\n interval.store(prev, Relaxed);\n });\n\n std::thread::yield_now();\n }\n }\n\n flag.store(false, Relaxed);\n\n let w = Arc::downgrade(&interval);\n drop(interval);\n\n while w.strong_count() > 0 {\n std::thread::sleep(Duration::from_micros(500));\n }\n\n // Now, run it again with a faster task\n let flag = Arc::new(AtomicBool::new(true));\n // Set it high, we know it shouldn't ever really be this high\n let counter = Arc::new(AtomicUsize::new(10_000));\n let interval = Arc::new(AtomicUsize::new(10_000));\n\n {\n let flag = flag.clone();\n let counter = counter.clone();\n rt.spawn(async move { iter(flag, counter, false) });\n }\n\n // Now, hammer the injection queue until the interval reaches the expected range.\n let mut n = 0;\n loop {\n let curr = interval.load(Relaxed);\n\n if curr <= 1_000 && curr > 32 {\n n += 1;\n } else {\n n = 0;\n }\n\n if n == 3 {\n break;\n }\n\n if Arc::strong_count(&interval) <= 5_000 {\n let counter = counter.clone();\n let interval = interval.clone();\n\n rt.spawn(async move {\n let prev = counter.swap(0, Relaxed);\n interval.store(prev, Relaxed);\n });\n }\n\n std::thread::yield_now();\n }\n\n flag.store(false, Relaxed);\n}\n\nfn rt() -> runtime::Runtime {\n runtime::Runtime::new().unwrap()\n}\n\n#[cfg(tokio_unstable)]\nmod unstable {\n use super::*;\n\n #[test]\n fn test_disable_lifo_slot() {\n use std::sync::mpsc::{channel, RecvTimeoutError};\n\n let rt = runtime::Builder::new_multi_thread()\n .disable_lifo_slot()\n .worker_threads(2)\n .build()\n .unwrap();\n\n // Spawn a background thread to poke the runtime periodically.\n //\n // This is necessary because we may end up triggering the issue in:\n // \n //\n // Spawning a task will wake up the second worker, which will then steal\n // the task. However, the steal will fail if the task is in the LIFO\n // slot, because the LIFO slot cannot be stolen.\n //\n // Note that this only happens rarely. Most of the time, this thread is\n // not necessary.\n let (kill_bg_thread, recv) = channel::<()>();\n let handle = rt.handle().clone();\n let bg_thread = std::thread::spawn(move || {\n let one_sec = std::time::Duration::from_secs(1);\n while recv.recv_timeout(one_sec) == Err(RecvTimeoutError::Timeout) {\n handle.spawn(async {});\n }\n });\n\n rt.block_on(async {\n tokio::spawn(async {\n // Spawn another task and block the thread until completion. If the LIFO slot\n // is used then the test doesn't complete.\n futures::executor::block_on(tokio::spawn(async {})).unwrap();\n })\n .await\n .unwrap();\n });\n\n drop(kill_bg_thread);\n bg_thread.join().unwrap();\n }\n\n #[test]\n fn runtime_id_is_same() {\n let rt = rt();\n\n let handle1 = rt.handle();\n let handle2 = rt.handle();\n\n assert_eq!(handle1.id(), handle2.id());\n }\n\n #[test]\n fn runtime_ids_different() {\n let rt1 = rt();\n let rt2 = rt();\n\n assert_ne!(rt1.handle().id(), rt2.handle().id());\n }\n}\n" }, { "path": "tokio/tests/rt_time_start_paused.rs", "content": "#![cfg(feature = \"full\")]\n\nuse tokio::time::{Duration, Instant};\n\n#[tokio::test(start_paused = true)]\nasync fn test_start_paused() {\n let now = Instant::now();\n\n // Pause a few times w/ std sleep and ensure `now` stays the same\n for _ in 0..5 {\n std::thread::sleep(Duration::from_millis(1));\n assert_eq!(now, Instant::now());\n }\n}\n" }, { "path": "tokio/tests/rt_unstable_metrics.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(\n feature = \"full\",\n tokio_unstable,\n not(target_os = \"wasi\"),\n target_has_atomic = \"64\"\n))]\n\nuse std::future::Future;\nuse std::sync::{mpsc, Arc, Mutex};\nuse std::task::Poll;\nuse std::thread;\nuse tokio::macros::support::poll_fn;\n\nuse tokio::runtime::{HistogramConfiguration, HistogramScale, LogHistogram, Runtime};\nuse tokio::task::consume_budget;\nuse tokio::time::{self, Duration};\n\n#[test]\nfn num_workers() {\n let rt = current_thread();\n assert_eq!(1, rt.metrics().num_workers());\n\n let rt = threaded();\n assert_eq!(2, rt.metrics().num_workers());\n}\n\n#[test]\nfn num_blocking_threads() {\n let rt = current_thread();\n assert_eq!(0, rt.metrics().num_blocking_threads());\n let _ = rt.block_on(rt.spawn_blocking(move || {}));\n assert_eq!(1, rt.metrics().num_blocking_threads());\n\n let rt = threaded();\n assert_eq!(0, rt.metrics().num_blocking_threads());\n let _ = rt.block_on(rt.spawn_blocking(move || {}));\n assert_eq!(1, rt.metrics().num_blocking_threads());\n}\n\n#[test]\nfn num_idle_blocking_threads() {\n let rt = current_thread();\n assert_eq!(0, rt.metrics().num_idle_blocking_threads());\n let _ = rt.block_on(rt.spawn_blocking(move || {}));\n rt.block_on(async {\n time::sleep(Duration::from_millis(5)).await;\n });\n\n // We need to wait until the blocking thread has become idle. Usually 5ms is\n // enough for this to happen, but not always. When it isn't enough, sleep\n // for another second. We don't always wait for a whole second since we want\n // the test suite to finish quickly.\n //\n // Note that the timeout for idle threads to be killed is 10 seconds.\n if 0 == rt.metrics().num_idle_blocking_threads() {\n rt.block_on(async {\n time::sleep(Duration::from_secs(1)).await;\n });\n }\n\n assert_eq!(1, rt.metrics().num_idle_blocking_threads());\n}\n\n#[test]\nfn num_idle_blocking_threads_is_zero_after_shutdown() {\n let rt = current_thread();\n let handle = rt.handle().clone();\n\n // Spawn a blocking task to create a worker thread.\n let _ = rt.block_on(rt.spawn_blocking(move || {}));\n\n // Wait for the thread to become idle.\n rt.block_on(async {\n time::sleep(Duration::from_millis(5)).await;\n });\n if handle.metrics().num_idle_blocking_threads() == 0 {\n rt.block_on(async {\n time::sleep(Duration::from_secs(1)).await;\n });\n }\n assert_eq!(1, handle.metrics().num_idle_blocking_threads());\n\n // Drop the runtime, which triggers shutdown and joins all blocking\n // threads. Before the fix for #6439, the shutdown path incremented\n // num_idle_threads a second time for each idle worker, so this\n // counter stayed at 1 instead of going back to 0.\n drop(rt);\n\n assert_eq!(\n 0,\n handle.metrics().num_idle_blocking_threads(),\n \"num_idle_blocking_threads should be 0 after shutdown (see #6439)\"\n );\n}\n\n#[test]\nfn blocking_queue_depth() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .max_blocking_threads(1)\n .build()\n .unwrap();\n\n assert_eq!(0, rt.metrics().blocking_queue_depth());\n\n let ready = Arc::new(Mutex::new(()));\n let guard = ready.lock().unwrap();\n\n let ready_cloned = ready.clone();\n let wait_until_ready = move || {\n let _unused = ready_cloned.lock().unwrap();\n };\n\n let h1 = rt.spawn_blocking(wait_until_ready.clone());\n let h2 = rt.spawn_blocking(wait_until_ready);\n assert!(rt.metrics().blocking_queue_depth() > 0);\n\n drop(guard);\n\n let _ = rt.block_on(h1);\n let _ = rt.block_on(h2);\n\n assert_eq!(0, rt.metrics().blocking_queue_depth());\n}\n\n#[test]\nfn spawned_tasks_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n assert_eq!(0, metrics.spawned_tasks_count());\n\n rt.block_on(rt.spawn(async move {\n assert_eq!(1, metrics.spawned_tasks_count());\n }))\n .unwrap();\n\n assert_eq!(1, rt.metrics().spawned_tasks_count());\n\n let rt = threaded();\n let metrics = rt.metrics();\n assert_eq!(0, metrics.spawned_tasks_count());\n\n rt.block_on(rt.spawn(async move {\n assert_eq!(1, metrics.spawned_tasks_count());\n }))\n .unwrap();\n\n assert_eq!(1, rt.metrics().spawned_tasks_count());\n}\n\n#[test]\nfn remote_schedule_count() {\n use std::thread;\n\n let rt = current_thread();\n let handle = rt.handle().clone();\n let task = thread::spawn(move || {\n handle.spawn(async {\n // DO nothing\n })\n })\n .join()\n .unwrap();\n\n rt.block_on(task).unwrap();\n\n assert_eq!(1, rt.metrics().remote_schedule_count());\n\n let rt = threaded();\n let handle = rt.handle().clone();\n let task = thread::spawn(move || {\n handle.spawn(async {\n // DO nothing\n })\n })\n .join()\n .unwrap();\n\n rt.block_on(task).unwrap();\n\n assert_eq!(1, rt.metrics().remote_schedule_count());\n}\n\n#[test]\nfn worker_thread_id_current_thread() {\n let rt = current_thread();\n let metrics = rt.metrics();\n\n // Check that runtime is on this thread.\n rt.block_on(async {});\n assert_eq!(Some(thread::current().id()), metrics.worker_thread_id(0));\n\n // Move runtime to another thread.\n let thread_id = std::thread::scope(|scope| {\n let join_handle = scope.spawn(|| {\n rt.block_on(async {});\n });\n join_handle.thread().id()\n });\n assert_eq!(Some(thread_id), metrics.worker_thread_id(0));\n\n // Move runtime back to this thread.\n rt.block_on(async {});\n assert_eq!(Some(thread::current().id()), metrics.worker_thread_id(0));\n}\n\n#[test]\nfn worker_thread_id_threaded() {\n let rt = threaded();\n let metrics = rt.metrics();\n\n rt.block_on(rt.spawn(async move {\n // Check that we are running on a worker thread and determine\n // the index of our worker.\n let thread_id = std::thread::current().id();\n let this_worker = (0..2)\n .position(|w| metrics.worker_thread_id(w) == Some(thread_id))\n .expect(\"task not running on any worker thread\");\n\n // Force worker to another thread.\n let moved_thread_id = tokio::task::block_in_place(|| {\n assert_eq!(thread_id, std::thread::current().id());\n\n // Wait for worker to move to another thread.\n for _ in 0..100 {\n let new_id = metrics.worker_thread_id(this_worker).unwrap();\n if thread_id != new_id {\n return new_id;\n }\n std::thread::sleep(Duration::from_millis(100));\n }\n\n panic!(\"worker did not move to new thread\");\n });\n\n // After blocking task worker either stays on new thread or\n // is moved back to current thread.\n assert!(\n metrics.worker_thread_id(this_worker) == Some(moved_thread_id)\n || metrics.worker_thread_id(this_worker) == Some(thread_id)\n );\n }))\n .unwrap()\n}\n\n#[test]\nfn worker_noop_count() {\n // There isn't really a great way to generate no-op parks as they happen as\n // false-positive events under concurrency.\n\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(async {\n time::sleep(Duration::from_millis(1)).await;\n });\n drop(rt);\n assert!(0 < metrics.worker_noop_count(0));\n\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async {\n time::sleep(Duration::from_millis(1)).await;\n });\n drop(rt);\n assert!(0 < metrics.worker_noop_count(0));\n assert!(0 < metrics.worker_noop_count(1));\n}\n\n#[test]\nfn worker_steal_count() {\n // This metric only applies to the multi-threaded runtime.\n for _ in 0..10 {\n let rt = threaded_no_lifo();\n let metrics = rt.metrics();\n\n let successfully_spawned_stealable_task = rt.block_on(async {\n // The call to `try_spawn_stealable_task` may time out, which means\n // that the sending task couldn't be scheduled due to a deadlock in\n // the runtime.\n // This is expected behaviour, we just retry until we succeed or\n // exhaust all tries, the latter causing this test to fail.\n try_spawn_stealable_task().await.is_ok()\n });\n\n drop(rt);\n\n if successfully_spawned_stealable_task {\n let n: u64 = (0..metrics.num_workers())\n .map(|i| metrics.worker_steal_count(i))\n .sum();\n\n assert_eq!(1, n);\n return;\n }\n }\n\n panic!(\"exhausted every try to schedule the stealable task\");\n}\n\n#[test]\nfn worker_poll_count_and_time() {\n const N: u64 = 5;\n\n async fn task() {\n // Sync sleep\n std::thread::sleep(std::time::Duration::from_micros(10));\n }\n\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(task()).await.unwrap();\n }\n });\n drop(rt);\n assert_eq!(N, metrics.worker_poll_count(0));\n // Not currently supported for current-thread runtime\n assert_eq!(Duration::default(), metrics.worker_mean_poll_time(0));\n\n // Does not populate the histogram\n assert!(!metrics.poll_time_histogram_enabled());\n for i in 0..10 {\n assert_eq!(0, metrics.poll_time_histogram_bucket_count(0, i));\n }\n\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(task()).await.unwrap();\n }\n });\n drop(rt);\n // Account for the `block_on` task\n let n = (0..metrics.num_workers())\n .map(|i| metrics.worker_poll_count(i))\n .sum();\n\n assert_eq!(N, n);\n\n let n: Duration = (0..metrics.num_workers())\n .map(|i| metrics.worker_mean_poll_time(i))\n .sum();\n\n assert!(n > Duration::default());\n\n // Does not populate the histogram\n assert!(!metrics.poll_time_histogram_enabled());\n for n in 0..metrics.num_workers() {\n for i in 0..10 {\n assert_eq!(0, metrics.poll_time_histogram_bucket_count(n, i));\n }\n }\n}\n\n#[test]\nfn log_histogram() {\n const N: u64 = 50;\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::log(\n LogHistogram::builder()\n .max_value(Duration::from_secs(60))\n .min_value(Duration::from_nanos(100))\n .max_error(0.25),\n ))\n .build()\n .unwrap();\n let metrics = rt.metrics();\n let num_buckets = rt.metrics().poll_time_histogram_num_buckets();\n assert_eq!(num_buckets, 119);\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(async {}).await.unwrap();\n }\n });\n drop(rt);\n assert_eq!(\n metrics.poll_time_histogram_bucket_range(0),\n Duration::from_nanos(0)..Duration::from_nanos(96)\n );\n assert_eq!(\n metrics.poll_time_histogram_bucket_range(1),\n Duration::from_nanos(96)..Duration::from_nanos(96 + 2_u64.pow(4))\n );\n assert_eq!(\n metrics.poll_time_histogram_bucket_range(118).end,\n Duration::from_nanos(u64::MAX)\n );\n let n = (0..metrics.num_workers())\n .flat_map(|i| (0..num_buckets).map(move |j| (i, j)))\n .map(|(worker, bucket)| metrics.poll_time_histogram_bucket_count(worker, bucket))\n .sum();\n assert_eq!(N, n);\n}\n\n#[test]\nfn minimal_log_histogram() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::log(\n LogHistogram::builder()\n .max_value(Duration::from_millis(4))\n .min_value(Duration::from_micros(20))\n .precision_exact(0),\n ))\n .build()\n .unwrap();\n let metrics = rt.metrics();\n let num_buckets = rt.metrics().poll_time_histogram_num_buckets();\n for b in 1..num_buckets - 1 {\n let range = metrics.poll_time_histogram_bucket_range(b);\n let size = range.end - range.start;\n // Assert the buckets continue doubling in size\n assert_eq!(\n size,\n Duration::from_nanos((1 << (b - 1)) * 16384),\n \"incorrect range for {b}\"\n );\n }\n assert_eq!(num_buckets, 10);\n}\n\n#[test]\n#[allow(deprecated)]\nfn legacy_log_histogram() {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_count_histogram_scale(HistogramScale::Log)\n .metrics_poll_count_histogram_resolution(Duration::from_micros(50))\n .metrics_poll_count_histogram_buckets(20)\n .build()\n .unwrap();\n let num_buckets = rt.metrics().poll_time_histogram_num_buckets();\n assert_eq!(num_buckets, 20);\n}\n\n#[test]\nfn log_histogram_default_configuration() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::log(\n LogHistogram::default(),\n ))\n .build()\n .unwrap();\n let num_buckets = rt.metrics().poll_time_histogram_num_buckets();\n assert_eq!(num_buckets, 119);\n}\n\n#[test]\nfn worker_poll_count_histogram() {\n const N: u64 = 5;\n\n let rts = [\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::linear(\n Duration::from_millis(50),\n 3,\n ))\n .build()\n .unwrap(),\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::linear(\n Duration::from_millis(50),\n 3,\n ))\n .build()\n .unwrap(),\n ];\n\n for rt in rts {\n let metrics = rt.metrics();\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(async {}).await.unwrap();\n }\n });\n drop(rt);\n\n let num_workers = metrics.num_workers();\n let num_buckets = metrics.poll_time_histogram_num_buckets();\n\n assert!(metrics.poll_time_histogram_enabled());\n assert_eq!(num_buckets, 3);\n\n let n = (0..num_workers)\n .flat_map(|i| (0..num_buckets).map(move |j| (i, j)))\n .map(|(worker, bucket)| metrics.poll_time_histogram_bucket_count(worker, bucket))\n .sum();\n assert_eq!(N, n);\n }\n}\n\n#[test]\nfn worker_poll_count_histogram_range() {\n let max = Duration::from_nanos(u64::MAX);\n\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::linear(us(50), 3))\n .build()\n .unwrap();\n let metrics = rt.metrics();\n\n assert_eq!(metrics.poll_time_histogram_bucket_range(0), us(0)..us(50));\n assert_eq!(metrics.poll_time_histogram_bucket_range(1), us(50)..us(100));\n assert_eq!(metrics.poll_time_histogram_bucket_range(2), us(100)..max);\n\n // ensure the old methods work too\n #[allow(deprecated)]\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .enable_metrics_poll_time_histogram()\n .metrics_poll_count_histogram_scale(tokio::runtime::HistogramScale::Log)\n .metrics_poll_count_histogram_buckets(3)\n .metrics_poll_count_histogram_resolution(us(50))\n .build()\n .unwrap();\n let metrics = rt.metrics();\n\n let a = Duration::from_nanos(50000_u64.next_power_of_two());\n let b = a * 2;\n\n assert_eq!(metrics.poll_time_histogram_bucket_range(0), us(0)..a);\n assert_eq!(metrics.poll_time_histogram_bucket_range(1), a..b);\n assert_eq!(metrics.poll_time_histogram_bucket_range(2), b..max);\n}\n\n#[test]\nfn worker_poll_count_histogram_disabled_without_explicit_enable() {\n let rts = [\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::linear(\n Duration::from_millis(50),\n 3,\n ))\n .build()\n .unwrap(),\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .enable_all()\n .metrics_poll_time_histogram_configuration(HistogramConfiguration::linear(\n Duration::from_millis(50),\n 3,\n ))\n .build()\n .unwrap(),\n ];\n\n for rt in rts {\n let metrics = rt.metrics();\n assert!(!metrics.poll_time_histogram_enabled());\n }\n}\n\n#[test]\nfn worker_local_schedule_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(async {\n tokio::spawn(async {}).await.unwrap();\n });\n drop(rt);\n\n assert_eq!(1, metrics.worker_local_schedule_count(0));\n assert_eq!(0, metrics.remote_schedule_count());\n\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async {\n // Move to the runtime\n tokio::spawn(async {\n tokio::spawn(async {}).await.unwrap();\n })\n .await\n .unwrap();\n });\n drop(rt);\n\n let n: u64 = (0..metrics.num_workers())\n .map(|i| metrics.worker_local_schedule_count(i))\n .sum();\n\n assert_eq!(2, n);\n assert_eq!(1, metrics.remote_schedule_count());\n}\n\n#[test]\nfn worker_overflow_count() {\n // Only applies to the threaded worker\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async {\n // Move to the runtime\n tokio::spawn(async {\n let (tx1, rx1) = std::sync::mpsc::channel();\n let (tx2, rx2) = std::sync::mpsc::channel();\n\n // First, we need to block the other worker until all tasks have\n // been spawned.\n //\n // We spawn from outside the runtime to ensure that the other worker\n // will pick it up:\n // \n tokio::task::spawn_blocking(|| {\n tokio::spawn(async move {\n tx1.send(()).unwrap();\n rx2.recv().unwrap();\n });\n });\n\n rx1.recv().unwrap();\n\n // Spawn many tasks\n for _ in 0..300 {\n tokio::spawn(async {});\n }\n\n tx2.send(()).unwrap();\n })\n .await\n .unwrap();\n });\n drop(rt);\n\n let n: u64 = (0..metrics.num_workers())\n .map(|i| metrics.worker_overflow_count(i))\n .sum();\n\n assert_eq!(1, n);\n}\n\n#[test]\nfn worker_local_queue_depth() {\n const N: usize = 100;\n\n let rt = current_thread();\n let metrics = rt.metrics();\n rt.block_on(async {\n for _ in 0..N {\n tokio::spawn(async {});\n }\n\n assert_eq!(N, metrics.worker_local_queue_depth(0));\n });\n\n let rt = threaded();\n let metrics = rt.metrics();\n rt.block_on(async move {\n // Move to the runtime\n tokio::spawn(async move {\n let (tx1, rx1) = std::sync::mpsc::channel();\n let (tx2, rx2) = std::sync::mpsc::channel();\n\n // First, we need to block the other worker until all tasks have\n // been spawned.\n tokio::spawn(async move {\n tx1.send(()).unwrap();\n rx2.recv().unwrap();\n });\n\n // Bump the next-run spawn\n tokio::spawn(async {});\n\n rx1.recv().unwrap();\n\n // Spawn some tasks\n for _ in 0..100 {\n tokio::spawn(async {});\n }\n\n let n: usize = (0..metrics.num_workers())\n .map(|i| metrics.worker_local_queue_depth(i))\n .sum();\n\n assert_eq!(n, N);\n\n tx2.send(()).unwrap();\n })\n .await\n .unwrap();\n });\n}\n\n#[test]\nfn budget_exhaustion_yield() {\n let rt = current_thread();\n let metrics = rt.metrics();\n\n assert_eq!(0, metrics.budget_forced_yield_count());\n\n let mut did_yield = false;\n\n // block on a task which consumes budget until it yields\n rt.block_on(poll_fn(|cx| loop {\n if did_yield {\n return Poll::Ready(());\n }\n\n let fut = consume_budget();\n tokio::pin!(fut);\n\n if fut.poll(cx).is_pending() {\n did_yield = true;\n return Poll::Pending;\n }\n }));\n\n assert_eq!(1, rt.metrics().budget_forced_yield_count());\n}\n\n#[test]\nfn budget_exhaustion_yield_with_joins() {\n let rt = current_thread();\n let metrics = rt.metrics();\n\n assert_eq!(0, metrics.budget_forced_yield_count());\n\n let mut did_yield_1 = false;\n let mut did_yield_2 = false;\n\n // block on a task which consumes budget until it yields\n rt.block_on(async {\n tokio::join!(\n poll_fn(|cx| loop {\n if did_yield_1 {\n return Poll::Ready(());\n }\n\n let fut = consume_budget();\n tokio::pin!(fut);\n\n if fut.poll(cx).is_pending() {\n did_yield_1 = true;\n return Poll::Pending;\n }\n }),\n poll_fn(|cx| loop {\n if did_yield_2 {\n return Poll::Ready(());\n }\n\n let fut = consume_budget();\n tokio::pin!(fut);\n\n if fut.poll(cx).is_pending() {\n did_yield_2 = true;\n return Poll::Pending;\n }\n })\n )\n });\n\n assert_eq!(1, rt.metrics().budget_forced_yield_count());\n}\n\n#[cfg(any(target_os = \"linux\", target_os = \"macos\"))]\n#[test]\nfn io_driver_fd_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n\n assert_eq!(metrics.io_driver_fd_registered_count(), 0);\n\n let stream = tokio::net::TcpStream::connect(\"google.com:80\");\n let stream = rt.block_on(async move { stream.await.unwrap() });\n\n assert_eq!(metrics.io_driver_fd_registered_count(), 1);\n assert_eq!(metrics.io_driver_fd_deregistered_count(), 0);\n\n drop(stream);\n\n assert_eq!(metrics.io_driver_fd_deregistered_count(), 1);\n assert_eq!(metrics.io_driver_fd_registered_count(), 1);\n}\n\n#[cfg(any(target_os = \"linux\", target_os = \"macos\"))]\n#[test]\nfn io_driver_ready_count() {\n let rt = current_thread();\n let metrics = rt.metrics();\n\n let stream = tokio::net::TcpStream::connect(\"google.com:80\");\n let _stream = rt.block_on(async move { stream.await.unwrap() });\n\n assert_eq!(metrics.io_driver_ready_count(), 1);\n}\n\nasync fn try_spawn_stealable_task() -> Result<(), mpsc::RecvTimeoutError> {\n // We use a blocking channel to synchronize the tasks.\n let (tx, rx) = mpsc::channel();\n\n // Make sure we are in the context of the runtime.\n tokio::spawn(async move {\n // Spawn the task that sends to the channel.\n //\n // Note that the runtime needs to have the lifo slot disabled to make\n // this task stealable.\n tokio::spawn(async move {\n tx.send(()).unwrap();\n });\n\n // Blocking receive on the channel, timing out if the sending task\n // wasn't scheduled in time.\n rx.recv_timeout(Duration::from_secs(1))\n })\n .await\n .unwrap()?;\n\n Ok(())\n}\n\nfn current_thread() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n\nfn threaded() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .enable_all()\n .build()\n .unwrap()\n}\n\nfn threaded_no_lifo() -> Runtime {\n tokio::runtime::Builder::new_multi_thread()\n .worker_threads(2)\n .disable_lifo_slot()\n .enable_all()\n .build()\n .unwrap()\n}\n\nfn us(n: u64) -> Duration {\n Duration::from_micros(n)\n}\n" }, { "path": "tokio/tests/rt_worker_index.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", tokio_unstable, not(target_os = \"wasi\"),))]\n\nuse tokio::runtime::{self, Runtime};\n\n#[test]\nfn worker_index_current_thread() {\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n rt.block_on(async {\n let index = runtime::worker_index();\n assert_eq!(index, Some(0));\n });\n}\n\n#[test]\nfn worker_index_local_runtime() {\n let rt = runtime::LocalRuntime::new().unwrap();\n rt.block_on(async {\n let index = runtime::worker_index();\n assert_eq!(index, Some(0));\n });\n}\n\n#[test]\nfn worker_index_outside_runtime() {\n assert_eq!(runtime::worker_index(), None);\n}\n\n#[cfg(target_has_atomic = \"64\")]\n#[test]\nfn worker_index_matches_metrics_worker_thread_id() {\n let rt = runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap();\n let metrics = rt.metrics();\n\n rt.block_on(async {\n // Spawn a task and verify the worker_index matches the metrics index\n tokio::task::spawn(async move {\n let index = runtime::worker_index().expect(\"should be on worker thread\");\n let current_thread = std::thread::current().id();\n let metrics_thread = metrics.worker_thread_id(index);\n assert_eq!(\n metrics_thread,\n Some(current_thread),\n \"worker_index() returned {index} but metrics.worker_thread_id({index}) \\\n does not match current thread\"\n );\n })\n .await\n .unwrap();\n });\n}\n\n#[test]\nfn worker_index_from_spawn_blocking() {\n let rt = Runtime::new().unwrap();\n rt.block_on(async {\n let index = tokio::task::spawn_blocking(runtime::worker_index)\n .await\n .unwrap();\n assert_eq!(\n index, None,\n \"spawn_blocking should not be on a worker thread\"\n );\n });\n}\n\n#[test]\nfn worker_index_block_on_multi_thread() {\n let rt = Runtime::new().unwrap();\n // block_on runs on the calling thread, not a worker thread\n let index = rt.block_on(async { runtime::worker_index() });\n assert_eq!(\n index, None,\n \"block_on thread is not a worker thread on multi-thread runtime\"\n );\n}\n" }, { "path": "tokio/tests/signal_ctrl_c.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` on Miri\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn ctrl_c() {\n let ctrl_c = signal::ctrl_c();\n\n tokio::spawn(async {\n send_signal(libc::SIGINT);\n });\n\n assert_ok!(ctrl_c.await);\n}\n" }, { "path": "tokio/tests/signal_drop_recv.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` in Miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[tokio::test]\nasync fn drop_then_get_a_signal() {\n let kind = SignalKind::user_defined1();\n let sig = signal(kind).expect(\"failed to create first signal\");\n drop(sig);\n\n send_signal(libc::SIGUSR1);\n let mut sig = signal(kind).expect(\"failed to create second signal\");\n\n let _ = sig.recv().await;\n}\n" }, { "path": "tokio/tests/signal_drop_rt.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` in miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::runtime::Runtime;\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[test]\nfn dropping_loops_does_not_cause_starvation() {\n let kind = SignalKind::user_defined1();\n\n let first_rt = rt();\n let mut first_signal =\n first_rt.block_on(async { signal(kind).expect(\"failed to register first signal\") });\n\n let second_rt = rt();\n let mut second_signal =\n second_rt.block_on(async { signal(kind).expect(\"failed to register second signal\") });\n\n send_signal(libc::SIGUSR1);\n\n first_rt\n .block_on(first_signal.recv())\n .expect(\"failed to await first signal\");\n\n drop(first_rt);\n drop(first_signal);\n\n send_signal(libc::SIGUSR1);\n\n second_rt.block_on(second_signal.recv());\n}\n\nfn rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/signal_drop_signal.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` in miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[tokio::test]\nasync fn dropping_signal_does_not_deregister_any_other_instances() {\n let kind = SignalKind::user_defined1();\n\n // Signals should not starve based on ordering\n let first_duplicate_signal = signal(kind).expect(\"failed to register first duplicate signal\");\n let mut sig = signal(kind).expect(\"failed to register signal\");\n let second_duplicate_signal = signal(kind).expect(\"failed to register second duplicate signal\");\n\n drop(first_duplicate_signal);\n drop(second_duplicate_signal);\n\n send_signal(libc::SIGUSR1);\n let _ = sig.recv().await;\n}\n" }, { "path": "tokio/tests/signal_info.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(any(\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"illumos\"\n))]\n#![cfg(not(miri))] // No `sigaction` on Miri\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal;\nuse tokio::signal::unix::SignalKind;\nuse tokio::time::{timeout, Duration};\n\n#[tokio::test]\nasync fn siginfo() {\n let mut sig = signal::unix::signal(SignalKind::info()).expect(\"installed signal handler\");\n\n tokio::spawn(async {\n send_signal(libc::SIGINFO);\n });\n\n // Add a timeout to ensure the test doesn't hang.\n timeout(Duration::from_secs(5), sig.recv())\n .await\n .expect(\"received SIGINFO signal in time\")\n .expect(\"received SIGINFO signal\");\n}\n" }, { "path": "tokio/tests/signal_multi_rt.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` on Miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::runtime::Runtime;\nuse tokio::signal::unix::{signal, SignalKind};\n\nuse std::sync::mpsc::channel;\nuse std::thread;\n\n#[test]\nfn multi_loop() {\n // An \"ordinary\" (non-future) channel\n let (sender, receiver) = channel();\n // Run multiple times, to make sure there are no race conditions\n for _ in 0..10 {\n // Run multiple event loops, each one in its own thread\n let threads: Vec<_> = (0..4)\n .map(|_| {\n let sender = sender.clone();\n thread::spawn(move || {\n let rt = rt();\n let _ = rt.block_on(async {\n let mut signal = signal(SignalKind::hangup()).unwrap();\n sender.send(()).unwrap();\n signal.recv().await\n });\n })\n })\n .collect();\n // Wait for them to declare they're ready\n for &_ in threads.iter() {\n receiver.recv().unwrap();\n }\n // Send a signal\n send_signal(libc::SIGHUP);\n // Make sure the threads terminated correctly\n for t in threads {\n t.join().unwrap();\n }\n }\n}\n\nfn rt() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/signal_no_rt.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` on Miri.\n\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[cfg_attr(target_os = \"wasi\", ignore = \"Wasi does not support panic recovery\")]\n#[test]\n#[should_panic]\nfn no_runtime_panics_creating_signals() {\n let _ = signal(SignalKind::hangup());\n}\n" }, { "path": "tokio/tests/signal_notify_both.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` on Miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[tokio::test]\nasync fn notify_both() {\n let kind = SignalKind::user_defined2();\n\n let mut signal1 = signal(kind).expect(\"failed to create signal1\");\n let mut signal2 = signal(kind).expect(\"failed to create signal2\");\n\n send_signal(libc::SIGUSR2);\n\n signal1.recv().await;\n signal2.recv().await;\n}\n" }, { "path": "tokio/tests/signal_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(panic = \"unwind\")]\n#![cfg(not(miri))] // No `sigaction` on Miri.\n\nuse std::error::Error;\nuse tokio::runtime::Builder;\nuse tokio::signal::unix::{signal, SignalKind};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[test]\nfn signal_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = Builder::new_current_thread().build().unwrap();\n\n rt.block_on(async {\n let kind = SignalKind::from_raw(-1);\n let _ = signal(kind);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n" }, { "path": "tokio/tests/signal_realtime.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(any(target_os = \"linux\", target_os = \"illumos\"))]\n#![cfg(not(miri))] // No `sigaction` in Miri.\n\nmod support {\n pub mod signal;\n}\n\nuse libc::c_int;\nuse support::signal::send_signal;\n\nuse futures::stream::{FuturesUnordered, StreamExt};\nuse std::collections::HashMap;\nuse tokio::signal::unix::{signal, SignalKind};\nuse tokio::time::{sleep, Duration};\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn signal_realtime() {\n // Attempt to register a real-time signal for everything between SIGRTMIN\n // and SIGRTMAX.\n let signals = (libc::SIGRTMIN()..=sigrt_max())\n .map(|signum| {\n let sig = assert_ok!(\n signal(SignalKind::from_raw(signum)),\n \"failed to create signal for {}\",\n sigrtnum_to_string(signum),\n );\n (signum, sig)\n })\n .collect::>();\n\n eprintln!(\n \"registered {} signals in the range {}..={}\",\n signals.len(),\n libc::SIGRTMIN(),\n libc::SIGRTMAX()\n );\n\n // Now send signals to each of the registered signals.\n for signum in libc::SIGRTMIN()..=sigrt_max() {\n send_signal(signum);\n }\n\n let futures = signals\n .into_iter()\n .map(|(signum, mut sig)| async move {\n let res = sig.recv().await;\n (signum, res)\n })\n .collect::>();\n\n // Ensure that all signals are received in time -- attempt to get whatever\n // we can.\n let sleep = std::pin::pin!(sleep(Duration::from_secs(5)));\n let done = futures.take_until(sleep).collect::>().await;\n\n let mut none = Vec::new();\n let mut missing = Vec::new();\n for signum in libc::SIGRTMIN()..=sigrt_max() {\n match done.get(&signum) {\n Some(Some(())) => {}\n Some(None) => none.push(signum),\n None => missing.push(signum),\n }\n }\n\n if none.is_empty() && missing.is_empty() {\n return;\n }\n\n let mut msg = String::new();\n if !none.is_empty() {\n msg.push_str(\"no signals received for:\\n\");\n for signum in none {\n msg.push_str(&format!(\"- {}\\n\", sigrtnum_to_string(signum)));\n }\n }\n\n if !missing.is_empty() {\n msg.push_str(\"missing signals for:\\n\");\n for signum in missing {\n msg.push_str(&format!(\"- {}\\n\", sigrtnum_to_string(signum)));\n }\n }\n\n panic!(\"{}\", msg);\n}\n\nfn sigrt_max() -> c_int {\n // Generally, you would expect this to be SIGRTMAX. But QEMU only supports\n // 28 real-time signals even though it might report SIGRTMAX to be higher.\n // See https://wiki.qemu.org/ChangeLog/9.2#signals.\n //\n // The goal of this test is to test that real-time signals are supported in\n // general, not necessarily that every single signal is supported (which, as\n // this example suggests, depends on the execution environment). So cap this\n // at SIGRTMIN+27 (i.e. SIGRTMIN..=SIGRTMIN+27, so 28 signals inclusive).\n libc::SIGRTMAX().min(libc::SIGRTMIN() + 27)\n}\n\nfn sigrtnum_to_string(signum: i32) -> String {\n format!(\"SIGRTMIN+{} (signal {})\", signum - libc::SIGRTMIN(), signum)\n}\n" }, { "path": "tokio/tests/signal_twice.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` on Miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal::unix::{signal, SignalKind};\n\n#[tokio::test]\nasync fn twice() {\n let kind = SignalKind::user_defined1();\n let mut sig = signal(kind).expect(\"failed to get signal\");\n\n for _ in 0..2 {\n send_signal(libc::SIGUSR1);\n\n assert!(sig.recv().await.is_some());\n }\n}\n" }, { "path": "tokio/tests/signal_usr1.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `sigaction` in Miri.\n\nmod support {\n pub mod signal;\n}\nuse support::signal::send_signal;\n\nuse tokio::signal::unix::{signal, SignalKind};\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn signal_usr1() {\n let mut signal = assert_ok!(\n signal(SignalKind::user_defined1()),\n \"failed to create signal\"\n );\n\n send_signal(libc::SIGUSR1);\n\n signal.recv().await;\n}\n" }, { "path": "tokio/tests/support/io_vec.rs", "content": "use std::io::IoSlice;\nuse std::ops::Deref;\nuse std::slice;\n\npub struct IoBufs<'a, 'b>(&'b mut [IoSlice<'a>]);\n\nimpl<'a, 'b> IoBufs<'a, 'b> {\n pub fn new(slices: &'b mut [IoSlice<'a>]) -> Self {\n IoBufs(slices)\n }\n\n pub fn is_empty(&self) -> bool {\n self.0.is_empty()\n }\n\n pub fn advance(mut self, n: usize) -> IoBufs<'a, 'b> {\n let mut to_remove = 0;\n let mut remaining_len = n;\n for slice in self.0.iter() {\n if remaining_len < slice.len() {\n break;\n } else {\n remaining_len -= slice.len();\n to_remove += 1;\n }\n }\n self.0 = self.0.split_at_mut(to_remove).1;\n if let Some(slice) = self.0.first_mut() {\n let tail = &slice[remaining_len..];\n // Safety: recasts slice to the original lifetime\n let tail = unsafe { slice::from_raw_parts(tail.as_ptr(), tail.len()) };\n *slice = IoSlice::new(tail);\n } else if remaining_len != 0 {\n panic!(\"advance past the end of the slice vector\");\n }\n self\n }\n}\n\nimpl<'a, 'b> Deref for IoBufs<'a, 'b> {\n type Target = [IoSlice<'a>];\n fn deref(&self) -> &[IoSlice<'a>] {\n self.0\n }\n}\n" }, { "path": "tokio/tests/support/leaked_buffers.rs", "content": "/// Can create buffers of arbitrary lifetime.\n/// Frees created buffers when dropped.\n///\n/// This struct is of course unsafe and the fact that\n/// it must outlive the created slices has to be ensured by\n/// the programmer.\n///\n/// Used at certain test scenarios as a safer version of\n/// Vec::leak, to satisfy the address sanitizer.\npub struct LeakedBuffers {\n leaked_vecs: Vec>,\n}\n\nimpl LeakedBuffers {\n pub fn new() -> Self {\n Self {\n leaked_vecs: vec![],\n }\n }\n pub unsafe fn create<'a>(&mut self, size: usize) -> &'a mut [u8] {\n let new_mem = vec![0u8; size].into_boxed_slice();\n self.leaked_vecs.push(new_mem);\n let new_mem = self.leaked_vecs.last_mut().unwrap();\n std::slice::from_raw_parts_mut(new_mem.as_mut_ptr(), new_mem.len())\n }\n}\n" }, { "path": "tokio/tests/support/mpsc_stream.rs", "content": "#![allow(dead_code)]\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::sync::mpsc::{self, Receiver, Sender, UnboundedReceiver, UnboundedSender};\nuse tokio_stream::Stream;\n\nstruct UnboundedStream {\n recv: UnboundedReceiver,\n}\nimpl Stream for UnboundedStream {\n type Item = T;\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::into_inner(self).recv.poll_recv(cx)\n }\n}\n\npub fn unbounded_channel_stream() -> (UnboundedSender, impl Stream) {\n let (tx, rx) = mpsc::unbounded_channel();\n\n let stream = UnboundedStream { recv: rx };\n\n (tx, stream)\n}\n\nstruct BoundedStream {\n recv: Receiver,\n}\nimpl Stream for BoundedStream {\n type Item = T;\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::into_inner(self).recv.poll_recv(cx)\n }\n}\n\npub fn channel_stream(size: usize) -> (Sender, impl Stream) {\n let (tx, rx) = mpsc::channel(size);\n\n let stream = BoundedStream { recv: rx };\n\n (tx, stream)\n}\n" }, { "path": "tokio/tests/support/panic.rs", "content": "use std::panic;\nuse std::sync::{Arc, Mutex};\n\npub fn test_panic(func: Func) -> Option {\n static PANIC_MUTEX: Mutex<()> = Mutex::new(());\n\n {\n let _guard = PANIC_MUTEX.lock();\n let panic_file: Arc>> = Arc::new(Mutex::new(None));\n\n let prev_hook = panic::take_hook();\n {\n let panic_file = panic_file.clone();\n panic::set_hook(Box::new(move |panic_info| {\n let panic_location = panic_info.location().unwrap();\n panic_file\n .lock()\n .unwrap()\n .clone_from(&Some(panic_location.file().to_string()));\n }));\n }\n\n let result = panic::catch_unwind(func);\n // Return to the previously set panic hook (maybe default) so that we get nice error\n // messages in the tests.\n panic::set_hook(prev_hook);\n\n if result.is_err() {\n panic_file.lock().unwrap().clone()\n } else {\n None\n }\n }\n}\n" }, { "path": "tokio/tests/support/signal.rs", "content": "pub fn send_signal(signal: libc::c_int) {\n use libc::{getpid, kill};\n\n unsafe {\n let pid = getpid();\n assert_eq!(\n kill(pid, signal),\n 0,\n \"kill(pid = {}, {}) failed with error: {}\",\n pid,\n signal,\n std::io::Error::last_os_error(),\n );\n }\n}\n" }, { "path": "tokio/tests/sync_barrier.rs", "content": "#![allow(clippy::unnecessary_operation)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse tokio::sync::Barrier;\n\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\n\nstruct IsSend(T);\n#[test]\nfn barrier_future_is_send() {\n let b = Barrier::new(0);\n IsSend(b.wait());\n}\n\n#[test]\nfn zero_does_not_block() {\n let b = Barrier::new(0);\n\n {\n let mut w = spawn(b.wait());\n let wr = assert_ready!(w.poll());\n assert!(wr.is_leader());\n }\n {\n let mut w = spawn(b.wait());\n let wr = assert_ready!(w.poll());\n assert!(wr.is_leader());\n }\n}\n\n#[test]\nfn single() {\n let b = Barrier::new(1);\n\n {\n let mut w = spawn(b.wait());\n let wr = assert_ready!(w.poll());\n assert!(wr.is_leader());\n }\n {\n let mut w = spawn(b.wait());\n let wr = assert_ready!(w.poll());\n assert!(wr.is_leader());\n }\n {\n let mut w = spawn(b.wait());\n let wr = assert_ready!(w.poll());\n assert!(wr.is_leader());\n }\n}\n\n#[test]\nfn tango() {\n let b = Barrier::new(2);\n\n let mut w1 = spawn(b.wait());\n assert_pending!(w1.poll());\n\n let mut w2 = spawn(b.wait());\n let wr2 = assert_ready!(w2.poll());\n let wr1 = assert_ready!(w1.poll());\n\n assert!(wr1.is_leader() || wr2.is_leader());\n assert!(!(wr1.is_leader() && wr2.is_leader()));\n}\n\n#[test]\nfn lots() {\n let b = Barrier::new(100);\n\n for _ in 0..10 {\n let mut wait = Vec::new();\n for _ in 0..99 {\n let mut w = spawn(b.wait());\n assert_pending!(w.poll());\n wait.push(w);\n }\n for w in &mut wait {\n assert_pending!(w.poll());\n }\n\n // pass the barrier\n let mut w = spawn(b.wait());\n let mut found_leader = assert_ready!(w.poll()).is_leader();\n for mut w in wait {\n let wr = assert_ready!(w.poll());\n if wr.is_leader() {\n assert!(!found_leader);\n found_leader = true;\n }\n }\n assert!(found_leader);\n }\n}\n" }, { "path": "tokio/tests/sync_broadcast.rs", "content": "#![allow(clippy::cognitive_complexity)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse tokio::sync::broadcast;\nuse tokio_test::task;\nuse tokio_test::{\n assert_err, assert_ok, assert_pending, assert_ready, assert_ready_err, assert_ready_ok,\n};\n\nuse std::sync::Arc;\n\nmacro_rules! assert_recv {\n ($e:expr) => {\n match $e.try_recv() {\n Ok(value) => value,\n Err(e) => panic!(\"expected recv; got = {:?}\", e),\n }\n };\n}\n\nmacro_rules! assert_empty {\n ($e:expr) => {\n match $e.try_recv() {\n Ok(value) => panic!(\"expected empty; got = {:?}\", value),\n Err(broadcast::error::TryRecvError::Empty) => {}\n Err(e) => panic!(\"expected empty; got = {:?}\", e),\n }\n };\n}\n\nmacro_rules! assert_lagged {\n ($e:expr, $n:expr) => {\n match assert_err!($e) {\n broadcast::error::TryRecvError::Lagged(n) => {\n assert_eq!(n, $n);\n }\n _ => panic!(\"did not lag\"),\n }\n };\n}\n\nmacro_rules! assert_closed {\n ($e:expr) => {\n match assert_err!($e) {\n broadcast::error::TryRecvError::Closed => {}\n _ => panic!(\"is not closed\"),\n }\n };\n}\n\n#[allow(unused)]\ntrait AssertSend: Send + Sync {}\nimpl AssertSend for broadcast::Sender {}\nimpl AssertSend for broadcast::Receiver {}\nimpl AssertSend for broadcast::WeakSender {}\n\n#[test]\nfn send_try_recv_bounded() {\n let (tx, mut rx) = broadcast::channel(16);\n\n assert_empty!(rx);\n\n let n = assert_ok!(tx.send(\"hello\"));\n assert_eq!(n, 1);\n\n let val = assert_recv!(rx);\n assert_eq!(val, \"hello\");\n\n assert_empty!(rx);\n}\n\n#[test]\nfn send_two_recv() {\n let (tx, mut rx1) = broadcast::channel(16);\n let mut rx2 = tx.subscribe();\n\n assert_empty!(rx1);\n assert_empty!(rx2);\n\n let n = assert_ok!(tx.send(\"hello\"));\n assert_eq!(n, 2);\n\n let val = assert_recv!(rx1);\n assert_eq!(val, \"hello\");\n\n let val = assert_recv!(rx2);\n assert_eq!(val, \"hello\");\n\n assert_empty!(rx1);\n assert_empty!(rx2);\n}\n\n#[test]\nfn send_recv_bounded() {\n let (tx, mut rx) = broadcast::channel(16);\n\n let mut recv = task::spawn(rx.recv());\n\n assert_pending!(recv.poll());\n\n assert_ok!(tx.send(\"hello\"));\n\n assert!(recv.is_woken());\n let val = assert_ready_ok!(recv.poll());\n assert_eq!(val, \"hello\");\n}\n\n#[test]\nfn send_two_recv_bounded() {\n let (tx, mut rx1) = broadcast::channel(16);\n let mut rx2 = tx.subscribe();\n\n let mut recv1 = task::spawn(rx1.recv());\n let mut recv2 = task::spawn(rx2.recv());\n\n assert_pending!(recv1.poll());\n assert_pending!(recv2.poll());\n\n assert_ok!(tx.send(\"hello\"));\n\n assert!(recv1.is_woken());\n assert!(recv2.is_woken());\n\n let val1 = assert_ready_ok!(recv1.poll());\n let val2 = assert_ready_ok!(recv2.poll());\n assert_eq!(val1, \"hello\");\n assert_eq!(val2, \"hello\");\n\n drop((recv1, recv2));\n\n let mut recv1 = task::spawn(rx1.recv());\n let mut recv2 = task::spawn(rx2.recv());\n\n assert_pending!(recv1.poll());\n\n assert_ok!(tx.send(\"world\"));\n\n assert!(recv1.is_woken());\n assert!(!recv2.is_woken());\n\n let val1 = assert_ready_ok!(recv1.poll());\n let val2 = assert_ready_ok!(recv2.poll());\n assert_eq!(val1, \"world\");\n assert_eq!(val2, \"world\");\n}\n\n#[test]\nfn change_tasks() {\n let (tx, mut rx) = broadcast::channel(1);\n\n let mut recv = Box::pin(rx.recv());\n\n let mut task1 = task::spawn(&mut recv);\n assert_pending!(task1.poll());\n\n let mut task2 = task::spawn(&mut recv);\n assert_pending!(task2.poll());\n\n tx.send(\"hello\").unwrap();\n\n assert!(task2.is_woken());\n}\n\n#[test]\nfn send_slow_rx() {\n let (tx, mut rx1) = broadcast::channel(16);\n let mut rx2 = tx.subscribe();\n\n {\n let mut recv2 = task::spawn(rx2.recv());\n\n {\n let mut recv1 = task::spawn(rx1.recv());\n\n assert_pending!(recv1.poll());\n assert_pending!(recv2.poll());\n\n assert_ok!(tx.send(\"one\"));\n\n assert!(recv1.is_woken());\n assert!(recv2.is_woken());\n\n assert_ok!(tx.send(\"two\"));\n\n let val = assert_ready_ok!(recv1.poll());\n assert_eq!(val, \"one\");\n }\n\n let val = assert_ready_ok!(task::spawn(rx1.recv()).poll());\n assert_eq!(val, \"two\");\n\n let mut recv1 = task::spawn(rx1.recv());\n\n assert_pending!(recv1.poll());\n\n assert_ok!(tx.send(\"three\"));\n\n assert!(recv1.is_woken());\n\n let val = assert_ready_ok!(recv1.poll());\n assert_eq!(val, \"three\");\n\n let val = assert_ready_ok!(recv2.poll());\n assert_eq!(val, \"one\");\n }\n\n let val = assert_recv!(rx2);\n assert_eq!(val, \"two\");\n\n let val = assert_recv!(rx2);\n assert_eq!(val, \"three\");\n}\n\n#[test]\nfn drop_rx_while_values_remain() {\n let (tx, mut rx1) = broadcast::channel(16);\n let mut rx2 = tx.subscribe();\n\n assert_ok!(tx.send(\"one\"));\n assert_ok!(tx.send(\"two\"));\n\n assert_recv!(rx1);\n assert_recv!(rx2);\n\n drop(rx2);\n drop(rx1);\n}\n\n#[test]\nfn lagging_rx() {\n let (tx, mut rx1) = broadcast::channel(2);\n let mut rx2 = tx.subscribe();\n\n assert_ok!(tx.send(\"one\"));\n assert_ok!(tx.send(\"two\"));\n\n assert_eq!(\"one\", assert_recv!(rx1));\n\n assert_ok!(tx.send(\"three\"));\n\n // Lagged too far\n let x = dbg!(rx2.try_recv());\n assert_lagged!(x, 1);\n\n // Calling again gets the next value\n assert_eq!(\"two\", assert_recv!(rx2));\n\n assert_eq!(\"two\", assert_recv!(rx1));\n assert_eq!(\"three\", assert_recv!(rx1));\n\n assert_ok!(tx.send(\"four\"));\n assert_ok!(tx.send(\"five\"));\n\n assert_lagged!(rx2.try_recv(), 1);\n\n assert_ok!(tx.send(\"six\"));\n\n assert_lagged!(rx2.try_recv(), 1);\n}\n\n#[test]\nfn send_no_rx() {\n let (tx, _) = broadcast::channel(16);\n\n assert_err!(tx.send(\"hello\"));\n\n let mut rx = tx.subscribe();\n\n assert_ok!(tx.send(\"world\"));\n\n let val = assert_recv!(rx);\n assert_eq!(\"world\", val);\n}\n\n#[test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn zero_capacity() {\n broadcast::channel::<()>(0);\n}\n\n#[test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn capacity_too_big() {\n broadcast::channel::<()>(1 + (usize::MAX >> 1));\n}\n\n#[test]\n#[cfg(panic = \"unwind\")]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn panic_in_clone() {\n use std::panic::{self, AssertUnwindSafe};\n\n #[derive(Eq, PartialEq, Debug)]\n struct MyVal(usize);\n\n impl Clone for MyVal {\n fn clone(&self) -> MyVal {\n assert_ne!(0, self.0);\n MyVal(self.0)\n }\n }\n\n let (tx, mut rx) = broadcast::channel(16);\n\n assert_ok!(tx.send(MyVal(0)));\n assert_ok!(tx.send(MyVal(1)));\n\n let res = panic::catch_unwind(AssertUnwindSafe(|| {\n let _ = rx.try_recv();\n }));\n\n assert_err!(res);\n\n let val = assert_recv!(rx);\n assert_eq!(val, MyVal(1));\n}\n\n#[test]\nfn dropping_tx_notifies_rx() {\n let (tx, mut rx1) = broadcast::channel::<()>(16);\n let mut rx2 = tx.subscribe();\n\n let tx2 = tx.clone();\n\n let mut recv1 = task::spawn(rx1.recv());\n let mut recv2 = task::spawn(rx2.recv());\n\n assert_pending!(recv1.poll());\n assert_pending!(recv2.poll());\n\n drop(tx);\n\n assert_pending!(recv1.poll());\n assert_pending!(recv2.poll());\n\n drop(tx2);\n\n assert!(recv1.is_woken());\n assert!(recv2.is_woken());\n\n let err = assert_ready_err!(recv1.poll());\n assert!(is_closed(err));\n\n let err = assert_ready_err!(recv2.poll());\n assert!(is_closed(err));\n}\n\n#[test]\nfn unconsumed_messages_are_dropped() {\n let (tx, rx) = broadcast::channel(16);\n\n let msg = Arc::new(());\n\n assert_ok!(tx.send(msg.clone()));\n\n assert_eq!(2, Arc::strong_count(&msg));\n\n drop(rx);\n\n assert_eq!(1, Arc::strong_count(&msg));\n}\n\n#[test]\nfn single_capacity_recvs() {\n let (tx, mut rx) = broadcast::channel(1);\n\n assert_ok!(tx.send(1));\n\n assert_eq!(assert_recv!(rx), 1);\n assert_empty!(rx);\n}\n\n#[test]\nfn single_capacity_recvs_after_drop_1() {\n let (tx, mut rx) = broadcast::channel(1);\n\n assert_ok!(tx.send(1));\n drop(tx);\n\n assert_eq!(assert_recv!(rx), 1);\n assert_closed!(rx.try_recv());\n}\n\n#[test]\nfn single_capacity_recvs_after_drop_2() {\n let (tx, mut rx) = broadcast::channel(1);\n\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n drop(tx);\n\n assert_lagged!(rx.try_recv(), 1);\n assert_eq!(assert_recv!(rx), 2);\n assert_closed!(rx.try_recv());\n}\n\n#[test]\nfn dropping_sender_does_not_overwrite() {\n let (tx, mut rx) = broadcast::channel(2);\n\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n drop(tx);\n\n assert_eq!(assert_recv!(rx), 1);\n assert_eq!(assert_recv!(rx), 2);\n assert_closed!(rx.try_recv());\n}\n\n#[test]\nfn lagging_receiver_recovers_after_wrap_closed_1() {\n let (tx, mut rx) = broadcast::channel(2);\n\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n assert_ok!(tx.send(3));\n drop(tx);\n\n assert_lagged!(rx.try_recv(), 1);\n assert_eq!(assert_recv!(rx), 2);\n assert_eq!(assert_recv!(rx), 3);\n assert_closed!(rx.try_recv());\n}\n\n#[test]\nfn lagging_receiver_recovers_after_wrap_closed_2() {\n let (tx, mut rx) = broadcast::channel(2);\n\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n assert_ok!(tx.send(3));\n assert_ok!(tx.send(4));\n drop(tx);\n\n assert_lagged!(rx.try_recv(), 2);\n assert_eq!(assert_recv!(rx), 3);\n assert_eq!(assert_recv!(rx), 4);\n assert_closed!(rx.try_recv());\n}\n\n#[test]\nfn lagging_receiver_recovers_after_wrap_open() {\n let (tx, mut rx) = broadcast::channel(2);\n\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n assert_ok!(tx.send(3));\n\n assert_lagged!(rx.try_recv(), 1);\n assert_eq!(assert_recv!(rx), 2);\n assert_eq!(assert_recv!(rx), 3);\n assert_empty!(rx);\n}\n\n#[test]\nfn receiver_len_with_lagged() {\n let (tx, mut rx) = broadcast::channel(3);\n\n tx.send(10).unwrap();\n tx.send(20).unwrap();\n tx.send(30).unwrap();\n tx.send(40).unwrap();\n\n assert_eq!(rx.len(), 4);\n assert_eq!(assert_recv!(rx), 10);\n\n tx.send(50).unwrap();\n tx.send(60).unwrap();\n\n assert_eq!(rx.len(), 5);\n assert_lagged!(rx.try_recv(), 1);\n}\n\nfn is_closed(err: broadcast::error::RecvError) -> bool {\n matches!(err, broadcast::error::RecvError::Closed)\n}\n\n#[test]\nfn resubscribe_points_to_tail() {\n let (tx, mut rx) = broadcast::channel(3);\n tx.send(1).unwrap();\n\n let mut rx_resub = rx.resubscribe();\n\n // verify we're one behind at the start\n assert_empty!(rx_resub);\n assert_eq!(assert_recv!(rx), 1);\n\n // verify we do not affect rx\n tx.send(2).unwrap();\n assert_eq!(assert_recv!(rx_resub), 2);\n tx.send(3).unwrap();\n assert_eq!(assert_recv!(rx), 2);\n assert_eq!(assert_recv!(rx), 3);\n assert_empty!(rx);\n\n assert_eq!(assert_recv!(rx_resub), 3);\n assert_empty!(rx_resub);\n}\n\n#[test]\nfn resubscribe_lagged() {\n let (tx, mut rx) = broadcast::channel(1);\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n\n let mut rx_resub = rx.resubscribe();\n assert_lagged!(rx.try_recv(), 1);\n assert_empty!(rx_resub);\n\n assert_eq!(assert_recv!(rx), 2);\n assert_empty!(rx);\n assert_empty!(rx_resub);\n}\n\n#[test]\nfn resubscribe_to_closed_channel() {\n let (tx, rx) = tokio::sync::broadcast::channel::(2);\n drop(tx);\n\n let mut rx_resub = rx.resubscribe();\n assert_closed!(rx_resub.try_recv());\n}\n\n#[test]\nfn sender_len() {\n let (tx, mut rx1) = broadcast::channel(4);\n let mut rx2 = tx.subscribe();\n\n assert_eq!(tx.len(), 0);\n assert!(tx.is_empty());\n\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n tx.send(3).unwrap();\n\n assert_eq!(tx.len(), 3);\n assert!(!tx.is_empty());\n\n assert_recv!(rx1);\n assert_recv!(rx1);\n\n assert_eq!(tx.len(), 3);\n assert!(!tx.is_empty());\n\n assert_recv!(rx2);\n\n assert_eq!(tx.len(), 2);\n assert!(!tx.is_empty());\n\n tx.send(4).unwrap();\n tx.send(5).unwrap();\n tx.send(6).unwrap();\n\n assert_eq!(tx.len(), 4);\n assert!(!tx.is_empty());\n}\n\n#[test]\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nfn sender_len_random() {\n let (tx, mut rx1) = broadcast::channel(16);\n let mut rx2 = tx.subscribe();\n\n for _ in 0..1000 {\n match rand::random_range(0..4) {\n 0 => {\n let _ = rx1.try_recv();\n }\n 1 => {\n let _ = rx2.try_recv();\n }\n _ => {\n tx.send(0).unwrap();\n }\n }\n\n let expected_len = usize::min(usize::max(rx1.len(), rx2.len()), 16);\n assert_eq!(tx.len(), expected_len);\n }\n}\n\n#[test]\nfn send_in_waker_drop() {\n use futures::task::ArcWake;\n use std::future::Future;\n use std::task::Context;\n\n struct SendOnDrop(broadcast::Sender<()>);\n\n impl Drop for SendOnDrop {\n fn drop(&mut self) {\n let _ = self.0.send(());\n }\n }\n\n impl ArcWake for SendOnDrop {\n fn wake_by_ref(_arc_self: &Arc) {}\n }\n\n // Test if there is no deadlock when replacing the old waker.\n\n let (tx, mut rx) = broadcast::channel(16);\n\n let mut fut = Box::pin(async {\n let _ = rx.recv().await;\n });\n\n // Store our special waker in the receiving future.\n let waker = futures::task::waker(Arc::new(SendOnDrop(tx)));\n let mut cx = Context::from_waker(&waker);\n assert!(fut.as_mut().poll(&mut cx).is_pending());\n drop(waker);\n\n // Second poll shouldn't deadlock.\n let mut cx = Context::from_waker(futures::task::noop_waker_ref());\n let _ = fut.as_mut().poll(&mut cx);\n\n // Test if there is no deadlock when calling waker.wake().\n\n let (tx, mut rx) = broadcast::channel(16);\n\n let mut fut = Box::pin(async {\n let _ = rx.recv().await;\n });\n\n // Store our special waker in the receiving future.\n let waker = futures::task::waker(Arc::new(SendOnDrop(tx.clone())));\n let mut cx = Context::from_waker(&waker);\n assert!(fut.as_mut().poll(&mut cx).is_pending());\n drop(waker);\n\n // Shouldn't deadlock.\n let _ = tx.send(());\n}\n\n#[tokio::test]\nasync fn receiver_recv_is_cooperative() {\n let (tx, mut rx) = broadcast::channel(8);\n\n tokio::select! {\n biased;\n _ = async {\n loop {\n assert!(tx.send(()).is_ok());\n assert!(rx.recv().await.is_ok());\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[test]\nfn broadcast_sender_closed() {\n let (tx, rx) = broadcast::channel::<()>(1);\n let rx2 = tx.subscribe();\n\n let mut task = task::spawn(tx.closed());\n assert_pending!(task.poll());\n\n drop(rx);\n assert!(!task.is_woken());\n assert_pending!(task.poll());\n\n drop(rx2);\n assert!(task.is_woken());\n assert_ready!(task.poll());\n}\n\n#[test]\nfn broadcast_sender_closed_with_extra_subscribe() {\n let (tx, rx) = broadcast::channel::<()>(1);\n let rx2 = tx.subscribe();\n\n let mut task = task::spawn(tx.closed());\n assert_pending!(task.poll());\n\n drop(rx);\n assert!(!task.is_woken());\n assert_pending!(task.poll());\n\n drop(rx2);\n assert!(task.is_woken());\n\n let rx3 = tx.subscribe();\n assert_pending!(task.poll());\n\n drop(rx3);\n assert!(task.is_woken());\n assert_ready!(task.poll());\n\n let mut task2 = task::spawn(tx.closed());\n assert_ready!(task2.poll());\n\n let rx4 = tx.subscribe();\n let mut task3 = task::spawn(tx.closed());\n assert_pending!(task3.poll());\n\n drop(rx4);\n assert!(task3.is_woken());\n assert_ready!(task3.poll());\n}\n\n#[tokio::test]\nasync fn broadcast_sender_new_must_be_closed() {\n let capacity = 1;\n let tx: broadcast::Sender<()> = broadcast::Sender::new(capacity);\n\n let mut task = task::spawn(tx.closed());\n assert_ready!(task.poll());\n\n let _rx = tx.subscribe();\n\n let mut task2 = task::spawn(tx.closed());\n assert_pending!(task2.poll());\n}\n" }, { "path": "tokio/tests/sync_broadcast_weak.rs", "content": "#![allow(clippy::redundant_clone)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse tokio::sync::broadcast::{self, channel};\n\n#[tokio::test]\nasync fn weak_sender() {\n let (tx, mut rx) = channel(11);\n\n let tx_weak = tokio::spawn(async move {\n let tx_weak = tx.clone().downgrade();\n\n for i in 0..10 {\n if tx.send(i).is_err() {\n return None;\n }\n }\n\n let tx2 = tx_weak\n .upgrade()\n .expect(\"expected to be able to upgrade tx_weak\");\n let _ = tx2.send(20);\n let tx_weak = tx2.downgrade();\n\n Some(tx_weak)\n })\n .await\n .unwrap();\n\n for i in 0..12 {\n let recvd = rx.recv().await;\n\n match recvd {\n Ok(msg) => {\n if i == 10 {\n assert_eq!(msg, 20);\n }\n }\n Err(_) => {\n assert_eq!(i, 11);\n break;\n }\n }\n }\n\n let tx_weak = tx_weak.unwrap();\n let upgraded = tx_weak.upgrade();\n assert!(upgraded.is_none());\n}\n\n// Tests that a `WeakSender` fails to upgrade when no other `Sender` exists.\n#[test]\nfn downgrade_upgrade_sender_failure() {\n let (tx, _rx) = broadcast::channel::(1);\n let weak_tx = tx.downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that a `WeakSender` cannot be upgraded after a `Sender` was dropped,\n// which existed at the time of the `downgrade` call.\n#[test]\nfn downgrade_drop_upgrade() {\n let (tx, _rx) = broadcast::channel::(1);\n\n // the cloned `Tx` is dropped right away\n let weak_tx = tx.clone().downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that `downgrade` does not change the `strong_count` of the channel.\n#[test]\nfn test_tx_count_weak_sender() {\n let (tx, _rx) = broadcast::channel::(1);\n let tx_weak = tx.downgrade();\n let tx_weak2 = tx.downgrade();\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(tx.weak_count(), 2);\n\n drop(tx);\n\n assert!(tx_weak.upgrade().is_none());\n assert!(tx_weak2.upgrade().is_none());\n assert_eq!(tx_weak.strong_count(), 0);\n assert_eq!(tx_weak.weak_count(), 2);\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_dropping_all_senders_except_weak_senders() {\n let (tx, rx) = broadcast::channel::<()>(10);\n let weak_sender = tx.clone().downgrade();\n drop(tx);\n // is_closed should return true after dropping all senders except for a weak sender.\n // The strong count should be 0 while the weak count should remain at 1.\n assert_eq!(weak_sender.strong_count(), 0);\n assert_eq!(weak_sender.weak_count(), 1);\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn sender_strong_count_when_cloned() {\n let (tx, rx) = broadcast::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n}\n\n#[tokio::test]\nasync fn sender_weak_count_when_downgraded() {\n let (tx, _rx) = broadcast::channel::<()>(1);\n\n let weak = tx.downgrade();\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n}\n\n#[tokio::test]\nasync fn sender_strong_count_when_dropped() {\n let (tx, rx) = broadcast::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n drop(tx2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n}\n\n#[tokio::test]\nasync fn sender_weak_count_when_dropped() {\n let (tx, rx) = broadcast::channel::<()>(1);\n\n let weak = tx.downgrade();\n\n drop(weak);\n\n assert_eq!(tx.weak_count(), 0);\n assert_eq!(rx.sender_weak_count(), 0);\n}\n\n#[tokio::test]\nasync fn sender_strong_and_weak_conut() {\n let (tx, rx) = broadcast::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n let weak = tx.downgrade();\n let weak2 = tx2.downgrade();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(weak.strong_count(), 2);\n assert_eq!(weak2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n\n assert_eq!(tx.weak_count(), 2);\n assert_eq!(tx2.weak_count(), 2);\n assert_eq!(weak.weak_count(), 2);\n assert_eq!(weak2.weak_count(), 2);\n assert_eq!(rx.sender_weak_count(), 2);\n\n drop(tx2);\n drop(weak2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(weak.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n assert_eq!(rx.sender_weak_count(), 1);\n}\n" }, { "path": "tokio/tests/sync_errors.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nfn is_error() {}\n\n#[test]\nfn mpsc_error_bound() {\n use tokio::sync::mpsc::error;\n\n is_error::>();\n is_error::>();\n}\n\n#[test]\nfn oneshot_error_bound() {\n use tokio::sync::oneshot::error;\n\n is_error::();\n is_error::();\n}\n\n#[test]\nfn watch_error_bound() {\n use tokio::sync::watch::error;\n\n is_error::>();\n}\n" }, { "path": "tokio/tests/sync_mpsc.rs", "content": "#![allow(clippy::redundant_clone)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse std::fmt;\nuse std::panic;\nuse std::sync::Arc;\nuse tokio::sync::mpsc;\nuse tokio::sync::mpsc::error::{TryRecvError, TrySendError};\nuse tokio_test::*;\n\n#[cfg(not(target_family = \"wasm\"))]\nmod support {\n pub(crate) mod mpsc_stream;\n}\n\n#[allow(unused)]\ntrait AssertRefUnwindSafe: panic::RefUnwindSafe {}\nimpl AssertRefUnwindSafe for mpsc::OwnedPermit {}\nimpl<'a, T> AssertRefUnwindSafe for mpsc::Permit<'a, T> {}\nimpl<'a, T> AssertRefUnwindSafe for mpsc::PermitIterator<'a, T> {}\nimpl AssertRefUnwindSafe for mpsc::Receiver {}\nimpl AssertRefUnwindSafe for mpsc::Sender {}\nimpl AssertRefUnwindSafe for mpsc::UnboundedReceiver {}\nimpl AssertRefUnwindSafe for mpsc::UnboundedSender {}\nimpl AssertRefUnwindSafe for mpsc::WeakSender {}\nimpl AssertRefUnwindSafe for mpsc::WeakUnboundedSender {}\n\n#[allow(unused)]\ntrait AssertUnwindSafe: panic::UnwindSafe {}\nimpl AssertUnwindSafe for mpsc::OwnedPermit {}\nimpl<'a, T> AssertUnwindSafe for mpsc::Permit<'a, T> {}\nimpl<'a, T> AssertUnwindSafe for mpsc::PermitIterator<'a, T> {}\nimpl AssertUnwindSafe for mpsc::Receiver {}\nimpl AssertUnwindSafe for mpsc::Sender {}\nimpl AssertUnwindSafe for mpsc::UnboundedReceiver {}\nimpl AssertUnwindSafe for mpsc::UnboundedSender {}\nimpl AssertUnwindSafe for mpsc::WeakSender {}\nimpl AssertUnwindSafe for mpsc::WeakUnboundedSender {}\n\n#[maybe_tokio_test]\nasync fn send_recv_with_buffer() {\n let (tx, mut rx) = mpsc::channel::(16);\n\n // Using poll_ready / try_send\n // let permit assert_ready_ok!(tx.reserve());\n let permit = tx.reserve().await.unwrap();\n permit.send(1);\n\n // Without poll_ready\n tx.try_send(2).unwrap();\n\n drop(tx);\n\n let val = rx.recv().await;\n assert_eq!(val, Some(1));\n\n let val = rx.recv().await;\n assert_eq!(val, Some(2));\n\n let val = rx.recv().await;\n assert!(val.is_none());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn reserve_disarm() {\n let (tx, mut rx) = mpsc::channel::(2);\n let tx1 = tx.clone();\n let tx2 = tx.clone();\n let tx3 = tx.clone();\n let tx4 = tx;\n\n // We should be able to `poll_ready` two handles without problem\n let permit1 = assert_ok!(tx1.reserve().await);\n let permit2 = assert_ok!(tx2.reserve().await);\n\n // But a third should not be ready\n let mut r3 = tokio_test::task::spawn(tx3.reserve());\n assert_pending!(r3.poll());\n\n let mut r4 = tokio_test::task::spawn(tx4.reserve());\n assert_pending!(r4.poll());\n\n // Using one of the reserved slots should allow a new handle to become ready\n permit1.send(1);\n\n // We also need to receive for the slot to be free\n assert!(!r3.is_woken());\n rx.recv().await.unwrap();\n // Now there's a free slot!\n assert!(r3.is_woken());\n assert!(!r4.is_woken());\n\n // Dropping a permit should also open up a slot\n drop(permit2);\n assert!(r4.is_woken());\n\n let mut r1 = tokio_test::task::spawn(tx1.reserve());\n assert_pending!(r1.poll());\n}\n\n#[tokio::test]\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\nasync fn send_recv_stream_with_buffer() {\n use tokio_stream::StreamExt;\n\n let (tx, rx) = support::mpsc_stream::channel_stream::(16);\n let mut rx = Box::pin(rx);\n\n tokio::spawn(async move {\n assert_ok!(tx.send(1).await);\n assert_ok!(tx.send(2).await);\n });\n\n assert_eq!(Some(1), rx.next().await);\n assert_eq!(Some(2), rx.next().await);\n assert_eq!(None, rx.next().await);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn async_send_recv_with_buffer() {\n let (tx, mut rx) = mpsc::channel(16);\n\n tokio::spawn(async move {\n assert_ok!(tx.send(1).await);\n assert_ok!(tx.send(2).await);\n });\n\n assert_eq!(Some(1), rx.recv().await);\n assert_eq!(Some(2), rx.recv().await);\n assert_eq!(None, rx.recv().await);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn async_send_recv_many_with_buffer() {\n let (tx, mut rx) = mpsc::channel(2);\n let mut buffer = Vec::::with_capacity(3);\n\n // With `limit=0` does not sleep, returns immediately\n assert_eq!(0, rx.recv_many(&mut buffer, 0).await);\n\n let handle = tokio::spawn(async move {\n assert_ok!(tx.send(1).await);\n assert_ok!(tx.send(2).await);\n assert_ok!(tx.send(7).await);\n assert_ok!(tx.send(0).await);\n });\n\n let limit = 3;\n let mut recv_count = 0usize;\n while recv_count < 4 {\n recv_count += rx.recv_many(&mut buffer, limit).await;\n assert_eq!(buffer.len(), recv_count);\n }\n\n assert_eq!(vec![1, 2, 7, 0], buffer);\n assert_eq!(0, rx.recv_many(&mut buffer, limit).await);\n handle.await.unwrap();\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn start_send_past_cap() {\n use std::future::Future;\n\n let mut t1 = tokio_test::task::spawn(());\n\n let (tx1, mut rx) = mpsc::channel(1);\n let tx2 = tx1.clone();\n\n assert_ok!(tx1.try_send(()));\n\n let mut r1 = Box::pin(tx1.reserve());\n t1.enter(|cx, _| assert_pending!(r1.as_mut().poll(cx)));\n\n {\n let mut r2 = tokio_test::task::spawn(tx2.reserve());\n assert_pending!(r2.poll());\n\n drop(r1);\n\n assert!(rx.recv().await.is_some());\n\n assert!(r2.is_woken());\n assert!(!t1.is_woken());\n }\n\n drop(tx1);\n drop(tx2);\n\n assert!(rx.recv().await.is_none());\n}\n\n#[test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn buffer_gteq_one() {\n mpsc::channel::(0);\n}\n\n#[maybe_tokio_test]\nasync fn send_recv_unbounded() {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n // Using `try_send`\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n\n assert_eq!(rx.recv().await, Some(1));\n assert_eq!(rx.recv().await, Some(2));\n\n drop(tx);\n\n assert!(rx.recv().await.is_none());\n}\n\n#[maybe_tokio_test]\nasync fn send_recv_many_unbounded() {\n let (tx, mut rx) = mpsc::unbounded_channel::();\n\n let mut buffer: Vec = Vec::new();\n\n // With `limit=0` does not sleep, returns immediately\n rx.recv_many(&mut buffer, 0).await;\n assert_eq!(0, buffer.len());\n\n assert_ok!(tx.send(7));\n assert_ok!(tx.send(13));\n assert_ok!(tx.send(100));\n assert_ok!(tx.send(1002));\n\n rx.recv_many(&mut buffer, 0).await;\n assert_eq!(0, buffer.len());\n\n let mut count = 0;\n while count < 4 {\n count += rx.recv_many(&mut buffer, 1).await;\n }\n assert_eq!(count, 4);\n assert_eq!(vec![7, 13, 100, 1002], buffer);\n let final_capacity = buffer.capacity();\n assert!(final_capacity > 0);\n\n buffer.clear();\n\n assert_ok!(tx.send(5));\n assert_ok!(tx.send(6));\n assert_ok!(tx.send(7));\n assert_ok!(tx.send(2));\n\n // Re-use existing capacity\n count = rx.recv_many(&mut buffer, 32).await;\n\n assert_eq!(final_capacity, buffer.capacity());\n assert_eq!(count, 4);\n assert_eq!(vec![5, 6, 7, 2], buffer);\n\n drop(tx);\n\n // recv_many will immediately return zero if the channel\n // is closed and no more messages are waiting\n assert_eq!(0, rx.recv_many(&mut buffer, 4).await);\n assert!(rx.recv().await.is_none());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn send_recv_many_bounded_capacity() {\n let mut buffer: Vec = Vec::with_capacity(9);\n let limit = buffer.capacity();\n let (tx, mut rx) = mpsc::channel(100);\n\n let mut expected: Vec = (0..limit)\n .map(|x: usize| format!(\"{x}\"))\n .collect::>();\n for x in expected.clone() {\n tx.send(x).await.unwrap()\n }\n tx.send(\"one more\".to_string()).await.unwrap();\n\n // Here `recv_many` receives all but the last value;\n // the initial capacity is adequate, so the buffer does\n // not increase in side.\n assert_eq!(buffer.capacity(), rx.recv_many(&mut buffer, limit).await);\n assert_eq!(expected, buffer);\n assert_eq!(limit, buffer.capacity());\n\n // Receive up more values:\n assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n assert!(buffer.capacity() > limit);\n expected.push(\"one more\".to_string());\n assert_eq!(expected, buffer);\n\n tokio::spawn(async move {\n tx.send(\"final\".to_string()).await.unwrap();\n });\n\n // 'tx' is dropped, but `recv_many` is guaranteed not\n // to return 0 as the channel has outstanding permits\n assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n expected.push(\"final\".to_string());\n assert_eq!(expected, buffer);\n // The channel is now closed and `recv_many` returns 0.\n assert_eq!(0, rx.recv_many(&mut buffer, limit).await);\n assert_eq!(expected, buffer);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn send_recv_many_unbounded_capacity() {\n let mut buffer: Vec = Vec::with_capacity(9); // capacity >= 9\n let limit = buffer.capacity();\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n let mut expected: Vec = (0..limit)\n .map(|x: usize| format!(\"{x}\"))\n .collect::>();\n for x in expected.clone() {\n tx.send(x).unwrap()\n }\n tx.send(\"one more\".to_string()).unwrap();\n\n // Here `recv_many` receives all but the last value;\n // the initial capacity is adequate, so the buffer does\n // not increase in side.\n assert_eq!(buffer.capacity(), rx.recv_many(&mut buffer, limit).await);\n assert_eq!(expected, buffer);\n assert_eq!(limit, buffer.capacity());\n\n // Receive up more values:\n assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n assert!(buffer.capacity() > limit);\n expected.push(\"one more\".to_string());\n assert_eq!(expected, buffer);\n\n tokio::spawn(async move {\n tx.send(\"final\".to_string()).unwrap();\n });\n\n // 'tx' is dropped, but `recv_many` is guaranteed not\n // to return 0 as the channel has outstanding permits\n assert_eq!(1, rx.recv_many(&mut buffer, limit).await);\n expected.push(\"final\".to_string());\n assert_eq!(expected, buffer);\n // The channel is now closed and `recv_many` returns 0.\n assert_eq!(0, rx.recv_many(&mut buffer, limit).await);\n assert_eq!(expected, buffer);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn async_send_recv_unbounded() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n tokio::spawn(async move {\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n });\n\n assert_eq!(Some(1), rx.recv().await);\n assert_eq!(Some(2), rx.recv().await);\n assert_eq!(None, rx.recv().await);\n}\n\n#[tokio::test]\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\nasync fn send_recv_stream_unbounded() {\n use tokio_stream::StreamExt;\n\n let (tx, rx) = support::mpsc_stream::unbounded_channel_stream::();\n\n let mut rx = Box::pin(rx);\n\n tokio::spawn(async move {\n assert_ok!(tx.send(1));\n assert_ok!(tx.send(2));\n });\n\n assert_eq!(Some(1), rx.next().await);\n assert_eq!(Some(2), rx.next().await);\n assert_eq!(None, rx.next().await);\n}\n\n#[maybe_tokio_test]\nasync fn no_t_bounds_buffer() {\n struct NoImpls;\n\n let (tx, mut rx) = mpsc::channel(100);\n\n // sender should be Debug even though T isn't Debug\n is_debug(&tx);\n // same with Receiver\n is_debug(&rx);\n // and sender should be Clone even though T isn't Clone\n assert!(tx.clone().try_send(NoImpls).is_ok());\n\n assert!(rx.recv().await.is_some());\n}\n\n#[maybe_tokio_test]\nasync fn no_t_bounds_unbounded() {\n struct NoImpls;\n\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n // sender should be Debug even though T isn't Debug\n is_debug(&tx);\n // same with Receiver\n is_debug(&rx);\n // and sender should be Clone even though T isn't Clone\n assert!(tx.clone().send(NoImpls).is_ok());\n\n assert!(rx.recv().await.is_some());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn send_recv_buffer_limited() {\n let (tx, mut rx) = mpsc::channel::(1);\n\n // Reserve capacity\n let p1 = assert_ok!(tx.reserve().await);\n\n // Send first message\n p1.send(1);\n\n // Not ready\n let mut p2 = tokio_test::task::spawn(tx.reserve());\n assert_pending!(p2.poll());\n\n // Take the value\n assert!(rx.recv().await.is_some());\n\n // Notified\n assert!(p2.is_woken());\n\n // Trying to send fails\n assert_err!(tx.try_send(1337));\n\n // Send second\n let permit = assert_ready_ok!(p2.poll());\n permit.send(2);\n\n assert!(rx.recv().await.is_some());\n}\n\n#[maybe_tokio_test]\nasync fn recv_close_gets_none_idle() {\n let (tx, mut rx) = mpsc::channel::(10);\n\n rx.close();\n\n assert!(rx.recv().await.is_none());\n\n assert_err!(tx.send(1).await);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn recv_close_gets_none_reserved() {\n let (tx1, mut rx) = mpsc::channel::(1);\n let tx2 = tx1.clone();\n\n let permit1 = assert_ok!(tx1.reserve().await);\n let mut permit2 = tokio_test::task::spawn(tx2.reserve());\n assert_pending!(permit2.poll());\n\n rx.close();\n\n assert!(permit2.is_woken());\n assert_ready_err!(permit2.poll());\n\n {\n let mut recv = tokio_test::task::spawn(rx.recv());\n assert_pending!(recv.poll());\n\n permit1.send(123);\n assert!(recv.is_woken());\n\n let v = assert_ready!(recv.poll());\n assert_eq!(v, Some(123));\n }\n\n assert!(rx.recv().await.is_none());\n}\n\n#[maybe_tokio_test]\nasync fn tx_close_gets_none() {\n let (_, mut rx) = mpsc::channel::(10);\n assert!(rx.recv().await.is_none());\n}\n\n#[maybe_tokio_test]\nasync fn try_send_fail() {\n let (tx, mut rx) = mpsc::channel(1);\n\n tx.try_send(\"hello\").unwrap();\n\n // This should fail\n match assert_err!(tx.try_send(\"fail\")) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n }\n\n assert_eq!(rx.recv().await, Some(\"hello\"));\n\n assert_ok!(tx.try_send(\"goodbye\"));\n drop(tx);\n\n assert_eq!(rx.recv().await, Some(\"goodbye\"));\n assert!(rx.recv().await.is_none());\n}\n\n#[maybe_tokio_test]\nasync fn try_send_fail_with_try_recv() {\n let (tx, mut rx) = mpsc::channel(1);\n\n tx.try_send(\"hello\").unwrap();\n\n // This should fail\n match assert_err!(tx.try_send(\"fail\")) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n }\n\n assert_eq!(rx.try_recv(), Ok(\"hello\"));\n\n assert_ok!(tx.try_send(\"goodbye\"));\n drop(tx);\n\n assert_eq!(rx.try_recv(), Ok(\"goodbye\"));\n assert_eq!(rx.try_recv(), Err(TryRecvError::Disconnected));\n}\n\n#[maybe_tokio_test]\nasync fn reserve_many_above_cap() {\n const MAX_PERMITS: usize = tokio::sync::Semaphore::MAX_PERMITS;\n let (tx, _rx) = mpsc::channel::<()>(1);\n\n assert_err!(tx.reserve_many(2).await);\n assert_err!(tx.reserve_many(MAX_PERMITS + 1).await);\n assert_err!(tx.reserve_many(usize::MAX).await);\n}\n\n#[test]\nfn try_reserve_many_zero() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n // Succeeds when not closed.\n assert!(assert_ok!(tx.try_reserve_many(0)).next().is_none());\n\n // Even when channel is full.\n tx.try_send(()).unwrap();\n assert!(assert_ok!(tx.try_reserve_many(0)).next().is_none());\n\n drop(rx);\n\n // Closed error when closed.\n assert_eq!(\n assert_err!(tx.try_reserve_many(0)),\n TrySendError::Closed(())\n );\n}\n\n#[maybe_tokio_test]\nasync fn reserve_many_zero() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n // Succeeds when not closed.\n assert!(assert_ok!(tx.reserve_many(0).await).next().is_none());\n\n // Even when channel is full.\n tx.send(()).await.unwrap();\n assert!(assert_ok!(tx.reserve_many(0).await).next().is_none());\n\n drop(rx);\n\n // Closed error when closed.\n assert_err!(tx.reserve_many(0).await);\n}\n\n#[maybe_tokio_test]\nasync fn try_reserve_many_edge_cases() {\n const MAX_PERMITS: usize = tokio::sync::Semaphore::MAX_PERMITS;\n\n let (tx, rx) = mpsc::channel::<()>(1);\n\n let mut permit = assert_ok!(tx.try_reserve_many(0));\n assert!(permit.next().is_none());\n\n let permit = tx.try_reserve_many(MAX_PERMITS + 1);\n match assert_err!(permit) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n }\n\n let permit = tx.try_reserve_many(usize::MAX);\n match assert_err!(permit) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n }\n\n // Dropping the receiver should close the channel\n drop(rx);\n assert_err!(tx.reserve_many(0).await);\n}\n\n#[maybe_tokio_test]\nasync fn try_reserve_fails() {\n let (tx, mut rx) = mpsc::channel(1);\n\n let permit = tx.try_reserve().unwrap();\n\n // This should fail\n match assert_err!(tx.try_reserve()) {\n TrySendError::Full(()) => {}\n _ => panic!(),\n }\n\n permit.send(\"foo\");\n\n assert_eq!(rx.recv().await, Some(\"foo\"));\n\n // Dropping permit releases the slot.\n let permit = tx.try_reserve().unwrap();\n drop(permit);\n\n let _permit = tx.try_reserve().unwrap();\n}\n\n#[maybe_tokio_test]\nasync fn reserve_many_and_send() {\n let (tx, mut rx) = mpsc::channel(100);\n for i in 0..100 {\n for permit in assert_ok!(tx.reserve_many(i).await) {\n permit.send(\"foo\");\n assert_eq!(rx.recv().await, Some(\"foo\"));\n }\n assert_eq!(rx.try_recv(), Err(TryRecvError::Empty));\n }\n}\n#[maybe_tokio_test]\nasync fn try_reserve_many_and_send() {\n let (tx, mut rx) = mpsc::channel(100);\n for i in 0..100 {\n for permit in assert_ok!(tx.try_reserve_many(i)) {\n permit.send(\"foo\");\n assert_eq!(rx.recv().await, Some(\"foo\"));\n }\n assert_eq!(rx.try_recv(), Err(TryRecvError::Empty));\n }\n}\n\n#[maybe_tokio_test]\nasync fn reserve_many_on_closed_channel() {\n let (tx, rx) = mpsc::channel::<()>(100);\n drop(rx);\n assert_err!(tx.reserve_many(10).await);\n}\n\n#[maybe_tokio_test]\nasync fn try_reserve_many_on_closed_channel() {\n let (tx, rx) = mpsc::channel::(100);\n drop(rx);\n match assert_err!(tx.try_reserve_many(10)) {\n TrySendError::Closed(()) => {}\n _ => panic!(),\n };\n}\n\n#[maybe_tokio_test]\n#[cfg_attr(miri, ignore)] // Too slow on miri.\nasync fn try_reserve_many_full() {\n // Reserve n capacity and send k messages\n for n in 1..100 {\n for k in 0..n {\n let (tx, mut rx) = mpsc::channel::(n);\n let permits = assert_ok!(tx.try_reserve_many(n));\n\n assert_eq!(permits.len(), n);\n assert_eq!(tx.capacity(), 0);\n\n match assert_err!(tx.try_reserve_many(1)) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n };\n\n for permit in permits.take(k) {\n permit.send(0);\n }\n // We only used k permits on the n reserved\n assert_eq!(tx.capacity(), n - k);\n\n // We can reserve more permits\n assert_ok!(tx.try_reserve_many(1));\n\n // But not more than the current capacity\n match assert_err!(tx.try_reserve_many(n - k + 1)) {\n TrySendError::Full(..) => {}\n _ => panic!(),\n };\n\n for _i in 0..k {\n assert_eq!(rx.recv().await, Some(0));\n }\n\n // Now that we've received everything, capacity should be back to n\n assert_eq!(tx.capacity(), n);\n }\n }\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn drop_permit_releases_permit() {\n // poll_ready reserves capacity, ensure that the capacity is released if tx\n // is dropped w/o sending a value.\n let (tx1, _rx) = mpsc::channel::(1);\n let tx2 = tx1.clone();\n\n let permit = assert_ok!(tx1.reserve().await);\n\n let mut reserve2 = tokio_test::task::spawn(tx2.reserve());\n assert_pending!(reserve2.poll());\n\n drop(permit);\n\n assert!(reserve2.is_woken());\n assert_ready_ok!(reserve2.poll());\n}\n\n#[maybe_tokio_test]\nasync fn drop_permit_iterator_releases_permits() {\n // poll_ready reserves capacity, ensure that the capacity is released if tx\n // is dropped w/o sending a value.\n for n in 1..100 {\n let (tx1, _rx) = mpsc::channel::(n);\n let tx2 = tx1.clone();\n\n let permits = assert_ok!(tx1.reserve_many(n).await);\n\n let mut reserve2 = tokio_test::task::spawn(tx2.reserve_many(n));\n assert_pending!(reserve2.poll());\n\n drop(permits);\n\n assert!(reserve2.is_woken());\n\n let permits = assert_ready_ok!(reserve2.poll());\n drop(permits);\n\n assert_eq!(tx1.capacity(), n);\n }\n}\n\n#[maybe_tokio_test]\nasync fn dropping_rx_closes_channel() {\n let (tx, rx) = mpsc::channel(100);\n\n let msg = Arc::new(());\n assert_ok!(tx.try_send(msg.clone()));\n\n drop(rx);\n assert_err!(tx.reserve().await);\n assert_err!(tx.reserve_many(10).await);\n assert_eq!(1, Arc::strong_count(&msg));\n}\n\n#[test]\nfn dropping_rx_closes_channel_for_try() {\n let (tx, rx) = mpsc::channel(100);\n\n let msg = Arc::new(());\n tx.try_send(msg.clone()).unwrap();\n\n drop(rx);\n\n assert!(matches!(\n tx.try_send(msg.clone()),\n Err(TrySendError::Closed(_))\n ));\n assert!(matches!(tx.try_reserve(), Err(TrySendError::Closed(_))));\n assert!(matches!(\n tx.try_reserve_owned(),\n Err(TrySendError::Closed(_))\n ));\n\n assert_eq!(1, Arc::strong_count(&msg));\n}\n\n#[test]\nfn unconsumed_messages_are_dropped() {\n let msg = Arc::new(());\n\n let (tx, rx) = mpsc::channel(100);\n\n tx.try_send(msg.clone()).unwrap();\n\n assert_eq!(2, Arc::strong_count(&msg));\n\n drop((tx, rx));\n\n assert_eq!(1, Arc::strong_count(&msg));\n}\n\n#[test]\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\nfn blocking_recv() {\n let (tx, mut rx) = mpsc::channel::(1);\n\n let sync_code = std::thread::spawn(move || {\n assert_eq!(Some(10), rx.blocking_recv());\n });\n\n tokio::runtime::Runtime::new()\n .unwrap()\n .block_on(async move {\n let _ = tx.send(10).await;\n });\n sync_code.join().unwrap()\n}\n\n#[tokio::test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nasync fn blocking_recv_async() {\n let (_tx, mut rx) = mpsc::channel::<()>(1);\n let _ = rx.blocking_recv();\n}\n\n#[test]\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\nfn blocking_send() {\n let (tx, mut rx) = mpsc::channel::(1);\n\n let sync_code = std::thread::spawn(move || {\n tx.blocking_send(10).unwrap();\n });\n\n tokio::runtime::Runtime::new()\n .unwrap()\n .block_on(async move {\n assert_eq!(Some(10), rx.recv().await);\n });\n sync_code.join().unwrap()\n}\n\n#[tokio::test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nasync fn blocking_send_async() {\n let (tx, _rx) = mpsc::channel::<()>(1);\n let _ = tx.blocking_send(());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn ready_close_cancel_bounded() {\n let (tx, mut rx) = mpsc::channel::<()>(100);\n let _tx2 = tx.clone();\n\n let permit = assert_ok!(tx.reserve().await);\n\n rx.close();\n\n let mut recv = tokio_test::task::spawn(rx.recv());\n assert_pending!(recv.poll());\n\n drop(permit);\n\n assert!(recv.is_woken());\n let val = assert_ready!(recv.poll());\n assert!(val.is_none());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn permit_available_not_acquired_close() {\n let (tx1, mut rx) = mpsc::channel::<()>(1);\n let tx2 = tx1.clone();\n\n let permit1 = assert_ok!(tx1.reserve().await);\n\n let mut permit2 = tokio_test::task::spawn(tx2.reserve());\n assert_pending!(permit2.poll());\n\n rx.close();\n\n drop(permit1);\n assert!(permit2.is_woken());\n\n drop(permit2);\n assert!(rx.recv().await.is_none());\n}\n\n#[test]\nfn try_recv_bounded() {\n let (tx, mut rx) = mpsc::channel(5);\n\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n assert!(tx.try_send(\"hello\").is_err());\n\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n assert!(tx.try_send(\"hello\").is_err());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n tx.try_send(\"hello\").unwrap();\n drop(tx);\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Ok(\"hello\"), rx.try_recv());\n assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n}\n\n#[test]\nfn try_recv_unbounded() {\n for num in 0..100 {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n for i in 0..num {\n tx.send(i).unwrap();\n }\n\n for i in 0..num {\n assert_eq!(rx.try_recv(), Ok(i));\n }\n\n assert_eq!(rx.try_recv(), Err(TryRecvError::Empty));\n drop(tx);\n assert_eq!(rx.try_recv(), Err(TryRecvError::Disconnected));\n }\n}\n\n#[test]\nfn try_recv_after_receiver_close() {\n let (_tx, mut rx) = mpsc::channel::<()>(5);\n\n assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n rx.close();\n assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n}\n\n#[test]\nfn try_recv_close_while_empty_bounded() {\n let (tx, mut rx) = mpsc::channel::<()>(5);\n\n assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n drop(tx);\n assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n}\n\n#[test]\nfn try_recv_close_while_empty_unbounded() {\n let (tx, mut rx) = mpsc::unbounded_channel::<()>();\n\n assert_eq!(Err(TryRecvError::Empty), rx.try_recv());\n drop(tx);\n assert_eq!(Err(TryRecvError::Disconnected), rx.try_recv());\n}\n\n#[tokio::test(start_paused = true)]\n#[cfg(feature = \"full\")]\nasync fn recv_timeout() {\n use tokio::sync::mpsc::error::SendTimeoutError::{Closed, Timeout};\n use tokio::time::Duration;\n\n let (tx, rx) = mpsc::channel(5);\n\n assert_eq!(tx.send_timeout(10, Duration::from_secs(1)).await, Ok(()));\n assert_eq!(tx.send_timeout(20, Duration::from_secs(1)).await, Ok(()));\n assert_eq!(tx.send_timeout(30, Duration::from_secs(1)).await, Ok(()));\n assert_eq!(tx.send_timeout(40, Duration::from_secs(1)).await, Ok(()));\n assert_eq!(tx.send_timeout(50, Duration::from_secs(1)).await, Ok(()));\n assert_eq!(\n tx.send_timeout(60, Duration::from_secs(1)).await,\n Err(Timeout(60))\n );\n\n drop(rx);\n assert_eq!(\n tx.send_timeout(70, Duration::from_secs(1)).await,\n Err(Closed(70))\n );\n}\n\n#[test]\n#[should_panic = \"there is no reactor running, must be called from the context of a Tokio 1.x runtime\"]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn recv_timeout_panic() {\n use futures::future::FutureExt;\n use tokio::time::Duration;\n\n let (tx, _rx) = mpsc::channel(5);\n tx.send_timeout(10, Duration::from_secs(1)).now_or_never();\n}\n\n// Tests that channel `capacity` changes and `max_capacity` stays the same\n#[tokio::test]\nasync fn test_tx_capacity() {\n let (tx, _rx) = mpsc::channel::<()>(10);\n // both capacities are same before\n assert_eq!(tx.capacity(), 10);\n assert_eq!(tx.max_capacity(), 10);\n\n let _permit = tx.reserve().await.unwrap();\n // after reserve, only capacity should drop by one\n assert_eq!(tx.capacity(), 9);\n assert_eq!(tx.max_capacity(), 10);\n\n tx.send(()).await.unwrap();\n // after send, capacity should drop by one again\n assert_eq!(tx.capacity(), 8);\n assert_eq!(tx.max_capacity(), 10);\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_calling_close_with_sender() {\n // is_closed should return true after calling close but still has a sender\n let (_tx, mut rx) = mpsc::channel::<()>(10);\n rx.close();\n\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_dropping_all_senders() {\n // is_closed should return true after dropping all senders\n let (tx, rx) = mpsc::channel::<()>(10);\n let another_tx = tx.clone();\n let task = tokio::spawn(async move {\n drop(another_tx);\n });\n\n drop(tx);\n let _ = task.await;\n\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_closed_when_there_are_senders() {\n // is_closed should return false when there is a sender\n let (_tx, rx) = mpsc::channel::<()>(10);\n assert!(!rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_closed_when_there_are_senders_and_buffer_filled() {\n // is_closed should return false when there is a sender, even if enough messages have been sent to fill the channel\n let (tx, rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n assert!(!rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_there_are_no_senders_and_there_are_messages() {\n // is_closed should return true when there are messages in the buffer, but no senders\n let (tx, rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n drop(tx);\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_there_are_messages_and_close_is_called() {\n // is_closed should return true when there are messages in the buffer, and close is called\n let (tx, mut rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n rx.close();\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_closed_when_there_are_permits_but_not_senders() {\n // is_closed should return false when there is a permit (but no senders)\n let (tx, rx) = mpsc::channel::<()>(10);\n let _permit = tx.reserve_owned().await.expect(\"Failed to reserve permit\");\n assert!(!rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_is_empty_when_no_messages_were_sent() {\n let (_tx, rx) = mpsc::channel::<()>(10);\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_empty_when_there_are_messages_in_the_buffer() {\n let (tx, rx) = mpsc::channel::<()>(10);\n assert!(tx.send(()).await.is_ok());\n assert!(!rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_empty_when_the_buffer_is_full() {\n let (tx, rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n assert!(!rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_is_not_empty_when_all_but_one_messages_are_consumed() {\n let (tx, mut rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n\n for _ in 0..9 {\n assert!(rx.recv().await.is_some());\n }\n\n assert!(!rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_is_empty_when_all_messages_are_consumed() {\n let (tx, mut rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n while rx.try_recv().is_ok() {}\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_is_empty_all_senders_are_dropped_and_messages_consumed() {\n let (tx, mut rx) = mpsc::channel(10);\n for i in 0..10 {\n assert!(tx.send(i).await.is_ok());\n }\n drop(tx);\n\n for _ in 0..10 {\n assert!(rx.recv().await.is_some());\n }\n\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_len_on_empty_channel() {\n let (_tx, rx) = mpsc::channel::<()>(100);\n assert_eq!(rx.len(), 0);\n}\n\n#[tokio::test]\nasync fn test_rx_len_on_empty_channel_without_senders() {\n // when all senders are dropped, a \"closed\" value is added to the end of the linked list.\n // here we test that the \"closed\" value does not change the len of the channel.\n\n let (tx, rx) = mpsc::channel::<()>(100);\n drop(tx);\n assert_eq!(rx.len(), 0);\n}\n\n#[tokio::test]\nasync fn test_rx_len_on_filled_channel() {\n let (tx, rx) = mpsc::channel(100);\n\n for i in 0..100 {\n assert!(tx.send(i).await.is_ok());\n }\n assert_eq!(rx.len(), 100);\n}\n\n#[tokio::test]\nasync fn test_rx_len_on_filled_channel_without_senders() {\n let (tx, rx) = mpsc::channel(100);\n\n for i in 0..100 {\n assert!(tx.send(i).await.is_ok());\n }\n drop(tx);\n assert_eq!(rx.len(), 100);\n}\n\n#[tokio::test]\nasync fn test_rx_len_when_consuming_all_messages() {\n let (tx, mut rx) = mpsc::channel(100);\n\n for i in 0..100 {\n assert!(tx.send(i).await.is_ok());\n assert_eq!(rx.len(), i + 1);\n }\n\n drop(tx);\n\n for i in (0..100).rev() {\n assert!(rx.recv().await.is_some());\n assert_eq!(rx.len(), i);\n }\n}\n\n#[tokio::test]\nasync fn test_rx_len_when_close_is_called() {\n let (tx, mut rx) = mpsc::channel(100);\n tx.send(()).await.unwrap();\n rx.close();\n\n assert_eq!(rx.len(), 1);\n}\n\n#[tokio::test]\nasync fn test_rx_len_when_close_is_called_before_dropping_sender() {\n let (tx, mut rx) = mpsc::channel(100);\n tx.send(()).await.unwrap();\n rx.close();\n drop(tx);\n\n assert_eq!(rx.len(), 1);\n}\n\n#[tokio::test]\nasync fn test_rx_len_when_close_is_called_after_dropping_sender() {\n let (tx, mut rx) = mpsc::channel(100);\n tx.send(()).await.unwrap();\n drop(tx);\n rx.close();\n\n assert_eq!(rx.len(), 1);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_closed_when_calling_close_with_sender() {\n // is_closed should return true after calling close but still has a sender\n let (_tx, mut rx) = mpsc::unbounded_channel::<()>();\n rx.close();\n\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_closed_when_dropping_all_senders() {\n // is_closed should return true after dropping all senders\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n let another_tx = tx.clone();\n let task = tokio::spawn(async move {\n drop(another_tx);\n });\n\n drop(tx);\n let _ = task.await;\n\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_not_closed_when_there_are_senders() {\n // is_closed should return false when there is a sender\n let (_tx, rx) = mpsc::unbounded_channel::<()>();\n assert!(!rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_closed_when_there_are_no_senders_and_there_are_messages() {\n // is_closed should return true when there are messages in the buffer, but no senders\n let (tx, rx) = mpsc::unbounded_channel();\n for i in 0..10 {\n assert!(tx.send(i).is_ok());\n }\n drop(tx);\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_closed_when_there_are_messages_and_close_is_called() {\n // is_closed should return true when there are messages in the buffer, and close is called\n let (tx, mut rx) = mpsc::unbounded_channel();\n for i in 0..10 {\n assert!(tx.send(i).is_ok());\n }\n rx.close();\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_empty_when_no_messages_were_sent() {\n let (_tx, rx) = mpsc::unbounded_channel::<()>();\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_not_empty_when_there_are_messages_in_the_buffer() {\n let (tx, rx) = mpsc::unbounded_channel();\n assert!(tx.send(()).is_ok());\n assert!(!rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_not_empty_when_all_but_one_messages_are_consumed() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n for i in 0..10 {\n assert!(tx.send(i).is_ok());\n }\n\n for _ in 0..9 {\n assert!(rx.recv().await.is_some());\n }\n\n assert!(!rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_empty_when_all_messages_are_consumed() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n for i in 0..10 {\n assert!(tx.send(i).is_ok());\n }\n while rx.try_recv().is_ok() {}\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_empty_all_senders_are_dropped_and_messages_consumed() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n for i in 0..10 {\n assert!(tx.send(i).is_ok());\n }\n drop(tx);\n\n for _ in 0..10 {\n assert!(rx.recv().await.is_some());\n }\n\n assert!(rx.is_empty())\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_on_empty_channel() {\n let (_tx, rx) = mpsc::unbounded_channel::<()>();\n assert_eq!(rx.len(), 0);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_on_empty_channel_without_senders() {\n // when all senders are dropped, a \"closed\" value is added to the end of the linked list.\n // here we test that the \"closed\" value does not change the len of the channel.\n\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n drop(tx);\n assert_eq!(rx.len(), 0);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_with_multiple_messages() {\n let (tx, rx) = mpsc::unbounded_channel();\n\n for i in 0..100 {\n assert!(tx.send(i).is_ok());\n }\n assert_eq!(rx.len(), 100);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_with_multiple_messages_and_dropped_senders() {\n let (tx, rx) = mpsc::unbounded_channel();\n\n for i in 0..100 {\n assert!(tx.send(i).is_ok());\n }\n drop(tx);\n assert_eq!(rx.len(), 100);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_when_consuming_all_messages() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n for i in 0..100 {\n assert!(tx.send(i).is_ok());\n assert_eq!(rx.len(), i + 1);\n }\n\n drop(tx);\n\n for i in (0..100).rev() {\n assert!(rx.recv().await.is_some());\n assert_eq!(rx.len(), i);\n }\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_when_close_is_called() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n tx.send(()).unwrap();\n rx.close();\n\n assert_eq!(rx.len(), 1);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_when_close_is_called_before_dropping_sender() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n tx.send(()).unwrap();\n rx.close();\n drop(tx);\n\n assert_eq!(rx.len(), 1);\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_len_when_close_is_called_after_dropping_sender() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n tx.send(()).unwrap();\n drop(tx);\n rx.close();\n\n assert_eq!(rx.len(), 1);\n}\n\n// Regression test for https://github.com/tokio-rs/tokio/issues/6602\n#[tokio::test]\nasync fn test_is_empty_32_msgs() {\n let (sender, mut receiver) = mpsc::channel(33);\n\n for value in 1..257 {\n sender.send(value).await.unwrap();\n receiver.recv().await.unwrap();\n assert!(receiver.is_empty(), \"{value}. len: {}\", receiver.len());\n }\n}\n\n#[test]\n#[cfg(not(panic = \"abort\"))]\nfn drop_all_elements_during_panic() {\n use std::sync::atomic::AtomicUsize;\n use std::sync::atomic::Ordering::Relaxed;\n use tokio::sync::mpsc::UnboundedReceiver;\n use tokio::sync::mpsc::UnboundedSender;\n\n static COUNTER: AtomicUsize = AtomicUsize::new(0);\n\n struct A(bool);\n impl Drop for A {\n // cause a panic when inner value is `true`.\n fn drop(&mut self) {\n COUNTER.fetch_add(1, std::sync::atomic::Ordering::Relaxed);\n if self.0 {\n panic!(\"panic!\")\n }\n }\n }\n\n fn func(tx: UnboundedSender, rx: UnboundedReceiver) {\n tx.send(A(true)).unwrap();\n tx.send(A(false)).unwrap();\n tx.send(A(false)).unwrap();\n\n drop(rx);\n\n // `mpsc::Rx`'s drop is called and gets panicked while dropping the first value,\n // but will keep dropping following elements.\n }\n\n let (tx, rx) = mpsc::unbounded_channel();\n\n let _ = panic::catch_unwind(panic::AssertUnwindSafe(|| {\n func(tx.clone(), rx);\n }));\n\n // all A's destructor should be called at this point, even before `mpsc::Chan`'s\n // drop gets called.\n assert_eq!(COUNTER.load(Relaxed), 3);\n\n drop(tx);\n // `mpsc::Chan`'s drop is called, freeing the `Block` memory allocation.\n}\n\nfn is_debug(_: &T) {}\n" }, { "path": "tokio/tests/sync_mpsc_weak.rs", "content": "#![allow(clippy::redundant_clone)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse std::sync::atomic::AtomicUsize;\nuse std::sync::atomic::Ordering::{Acquire, Release};\nuse tokio::sync::mpsc::{self, channel, unbounded_channel};\nuse tokio::sync::oneshot;\n\n#[tokio::test]\nasync fn weak_sender() {\n let (tx, mut rx) = channel(11);\n\n let tx_weak = tokio::spawn(async move {\n let tx_weak = tx.clone().downgrade();\n\n for i in 0..10 {\n if tx.send(i).await.is_err() {\n return None;\n }\n }\n\n let tx2 = tx_weak\n .upgrade()\n .expect(\"expected to be able to upgrade tx_weak\");\n let _ = tx2.send(20).await;\n let tx_weak = tx2.downgrade();\n\n Some(tx_weak)\n })\n .await\n .unwrap();\n\n for i in 0..12 {\n let recvd = rx.recv().await;\n\n match recvd {\n Some(msg) => {\n if i == 10 {\n assert_eq!(msg, 20);\n }\n }\n None => {\n assert_eq!(i, 11);\n break;\n }\n }\n }\n\n let tx_weak = tx_weak.unwrap();\n let upgraded = tx_weak.upgrade();\n assert!(upgraded.is_none());\n}\n\n#[tokio::test]\nasync fn actor_weak_sender() {\n pub struct MyActor {\n receiver: mpsc::Receiver,\n sender: mpsc::WeakSender,\n next_id: u32,\n pub received_self_msg: bool,\n }\n\n enum ActorMessage {\n GetUniqueId { respond_to: oneshot::Sender },\n SelfMessage {},\n }\n\n impl MyActor {\n fn new(\n receiver: mpsc::Receiver,\n sender: mpsc::WeakSender,\n ) -> Self {\n MyActor {\n receiver,\n sender,\n next_id: 0,\n received_self_msg: false,\n }\n }\n\n fn handle_message(&mut self, msg: ActorMessage) {\n match msg {\n ActorMessage::GetUniqueId { respond_to } => {\n self.next_id += 1;\n\n // The `let _ =` ignores any errors when sending.\n //\n // This can happen if the `select!` macro is used\n // to cancel waiting for the response.\n let _ = respond_to.send(self.next_id);\n }\n ActorMessage::SelfMessage { .. } => {\n self.received_self_msg = true;\n }\n }\n }\n\n async fn send_message_to_self(&mut self) {\n let msg = ActorMessage::SelfMessage {};\n\n let sender = self.sender.clone();\n\n // cannot move self.sender here\n if let Some(sender) = sender.upgrade() {\n let _ = sender.send(msg).await;\n self.sender = sender.downgrade();\n }\n }\n\n async fn run(&mut self) {\n let mut i = 0;\n while let Some(msg) = self.receiver.recv().await {\n self.handle_message(msg);\n\n if i == 0 {\n self.send_message_to_self().await;\n }\n\n i += 1\n }\n\n assert!(self.received_self_msg);\n }\n }\n\n #[derive(Clone)]\n pub struct MyActorHandle {\n sender: mpsc::Sender,\n }\n\n impl MyActorHandle {\n pub fn new() -> (Self, MyActor) {\n let (sender, receiver) = mpsc::channel(8);\n let actor = MyActor::new(receiver, sender.clone().downgrade());\n\n (Self { sender }, actor)\n }\n\n pub async fn get_unique_id(&self) -> u32 {\n let (send, recv) = oneshot::channel();\n let msg = ActorMessage::GetUniqueId { respond_to: send };\n\n // Ignore send errors. If this send fails, so does the\n // recv.await below. There's no reason to check the\n // failure twice.\n let _ = self.sender.send(msg).await;\n recv.await.expect(\"Actor task has been killed\")\n }\n }\n\n let (handle, mut actor) = MyActorHandle::new();\n\n let actor_handle = tokio::spawn(async move { actor.run().await });\n\n let _ = tokio::spawn(async move {\n let _ = handle.get_unique_id().await;\n drop(handle);\n })\n .await;\n\n let _ = actor_handle.await;\n}\n\nstatic NUM_DROPPED: AtomicUsize = AtomicUsize::new(0);\n\n#[derive(Debug)]\nstruct Msg;\n\nimpl Drop for Msg {\n fn drop(&mut self) {\n NUM_DROPPED.fetch_add(1, Release);\n }\n}\n\n// Tests that no pending messages are put onto the channel after `Rx` was\n// dropped.\n//\n// Note: After the introduction of `WeakSender`, which internally\n// used `Arc` and doesn't call a drop of the channel after the last strong\n// `Sender` was dropped while more than one `WeakSender` remains, we want to\n// ensure that no messages are kept in the channel, which were sent after\n// the receiver was dropped.\n#[tokio::test]\nasync fn test_msgs_dropped_on_rx_drop() {\n let (tx, mut rx) = mpsc::channel(3);\n\n tx.send(Msg {}).await.unwrap();\n tx.send(Msg {}).await.unwrap();\n\n // This msg will be pending and should be dropped when `rx` is dropped\n let sent_fut = tx.send(Msg {});\n\n let _ = rx.recv().await.unwrap();\n let _ = rx.recv().await.unwrap();\n\n sent_fut.await.unwrap();\n\n drop(rx);\n\n assert_eq!(NUM_DROPPED.load(Acquire), 3);\n\n // This msg will not be put onto `Tx` list anymore, since `Rx` is closed.\n assert!(tx.send(Msg {}).await.is_err());\n\n assert_eq!(NUM_DROPPED.load(Acquire), 4);\n}\n\n// Tests that a `WeakSender` is upgradeable when other `Sender`s exist.\n#[test]\nfn downgrade_upgrade_sender_success() {\n let (tx, _rx) = mpsc::channel::(1);\n let weak_tx = tx.downgrade();\n assert!(weak_tx.upgrade().is_some());\n}\n\n// Tests that a `WeakSender` fails to upgrade when no other `Sender` exists.\n#[test]\nfn downgrade_upgrade_sender_failure() {\n let (tx, _rx) = mpsc::channel::(1);\n let weak_tx = tx.downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that a `WeakSender` cannot be upgraded after a `Sender` was dropped,\n// which existed at the time of the `downgrade` call.\n#[test]\nfn downgrade_drop_upgrade() {\n let (tx, _rx) = mpsc::channel::(1);\n\n // the cloned `Tx` is dropped right away\n let weak_tx = tx.clone().downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that we can upgrade a weak sender with an outstanding permit\n// but no other strong senders.\n#[tokio::test]\nasync fn downgrade_get_permit_upgrade_no_senders() {\n let (tx, _rx) = mpsc::channel::(1);\n let weak_tx = tx.downgrade();\n let _permit = tx.reserve_owned().await.unwrap();\n assert!(weak_tx.upgrade().is_some());\n}\n\n// Tests that you can downgrade and upgrade a sender with an outstanding permit\n// but no other senders left.\n#[tokio::test]\nasync fn downgrade_upgrade_get_permit_no_senders() {\n let (tx, _rx) = mpsc::channel::(1);\n let tx2 = tx.clone();\n let _permit = tx.reserve_owned().await.unwrap();\n let weak_tx = tx2.downgrade();\n drop(tx2);\n assert!(weak_tx.upgrade().is_some());\n}\n\n// Tests that `downgrade` does not change the `tx_count` of the channel.\n#[test]\nfn test_tx_count_weak_sender() {\n let (tx, _rx) = mpsc::channel::(1);\n let tx_weak = tx.downgrade();\n let tx_weak2 = tx.downgrade();\n drop(tx);\n\n assert!(tx_weak.upgrade().is_none() && tx_weak2.upgrade().is_none());\n}\n\n#[tokio::test]\nasync fn weak_unbounded_sender() {\n let (tx, mut rx) = unbounded_channel();\n\n let tx_weak = tokio::spawn(async move {\n let tx_weak = tx.clone().downgrade();\n\n for i in 0..10 {\n if tx.send(i).is_err() {\n return None;\n }\n }\n\n let tx2 = tx_weak\n .upgrade()\n .expect(\"expected to be able to upgrade tx_weak\");\n let _ = tx2.send(20);\n let tx_weak = tx2.downgrade();\n\n Some(tx_weak)\n })\n .await\n .unwrap();\n\n for i in 0..12 {\n let recvd = rx.recv().await;\n\n match recvd {\n Some(msg) => {\n if i == 10 {\n assert_eq!(msg, 20);\n }\n }\n None => {\n assert_eq!(i, 11);\n break;\n }\n }\n }\n\n let tx_weak = tx_weak.unwrap();\n let upgraded = tx_weak.upgrade();\n assert!(upgraded.is_none());\n}\n\n#[tokio::test]\nasync fn actor_weak_unbounded_sender() {\n pub struct MyActor {\n receiver: mpsc::UnboundedReceiver,\n sender: mpsc::WeakUnboundedSender,\n next_id: u32,\n pub received_self_msg: bool,\n }\n\n enum ActorMessage {\n GetUniqueId { respond_to: oneshot::Sender },\n SelfMessage {},\n }\n\n impl MyActor {\n fn new(\n receiver: mpsc::UnboundedReceiver,\n sender: mpsc::WeakUnboundedSender,\n ) -> Self {\n MyActor {\n receiver,\n sender,\n next_id: 0,\n received_self_msg: false,\n }\n }\n\n fn handle_message(&mut self, msg: ActorMessage) {\n match msg {\n ActorMessage::GetUniqueId { respond_to } => {\n self.next_id += 1;\n\n // The `let _ =` ignores any errors when sending.\n //\n // This can happen if the `select!` macro is used\n // to cancel waiting for the response.\n let _ = respond_to.send(self.next_id);\n }\n ActorMessage::SelfMessage { .. } => {\n self.received_self_msg = true;\n }\n }\n }\n\n async fn send_message_to_self(&mut self) {\n let msg = ActorMessage::SelfMessage {};\n\n let sender = self.sender.clone();\n\n // cannot move self.sender here\n if let Some(sender) = sender.upgrade() {\n let _ = sender.send(msg);\n self.sender = sender.downgrade();\n }\n }\n\n async fn run(&mut self) {\n let mut i = 0;\n while let Some(msg) = self.receiver.recv().await {\n self.handle_message(msg);\n\n if i == 0 {\n self.send_message_to_self().await;\n }\n\n i += 1\n }\n\n assert!(self.received_self_msg);\n }\n }\n\n #[derive(Clone)]\n pub struct MyActorHandle {\n sender: mpsc::UnboundedSender,\n }\n\n impl MyActorHandle {\n pub fn new() -> (Self, MyActor) {\n let (sender, receiver) = mpsc::unbounded_channel();\n let actor = MyActor::new(receiver, sender.clone().downgrade());\n\n (Self { sender }, actor)\n }\n\n pub async fn get_unique_id(&self) -> u32 {\n let (send, recv) = oneshot::channel();\n let msg = ActorMessage::GetUniqueId { respond_to: send };\n\n // Ignore send errors. If this send fails, so does the\n // recv.await below. There's no reason to check the\n // failure twice.\n let _ = self.sender.send(msg);\n recv.await.expect(\"Actor task has been killed\")\n }\n }\n\n let (handle, mut actor) = MyActorHandle::new();\n\n let actor_handle = tokio::spawn(async move { actor.run().await });\n\n let _ = tokio::spawn(async move {\n let _ = handle.get_unique_id().await;\n drop(handle);\n })\n .await;\n\n let _ = actor_handle.await;\n}\n\nstatic NUM_DROPPED_UNBOUNDED: AtomicUsize = AtomicUsize::new(0);\n\n#[derive(Debug)]\nstruct MsgUnbounded;\n\nimpl Drop for MsgUnbounded {\n fn drop(&mut self) {\n NUM_DROPPED_UNBOUNDED.fetch_add(1, Release);\n }\n}\n\n// Tests that no pending messages are put onto the channel after `Rx` was\n// dropped.\n//\n// Note: After the introduction of `UnboundedWeakSender`, which internally\n// used `Arc` and doesn't call a drop of the channel after the last strong\n// `UnboundedSender` was dropped while more than one `UnboundedWeakSender`\n// remains, we want to ensure that no messages are kept in the channel, which\n// were sent after the receiver was dropped.\n#[tokio::test]\nasync fn test_msgs_dropped_on_unbounded_rx_drop() {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n tx.send(MsgUnbounded {}).unwrap();\n tx.send(MsgUnbounded {}).unwrap();\n\n // This msg will be pending and should be dropped when `rx` is dropped\n let sent = tx.send(MsgUnbounded {});\n\n let _ = rx.recv().await.unwrap();\n let _ = rx.recv().await.unwrap();\n\n sent.unwrap();\n\n drop(rx);\n\n assert_eq!(NUM_DROPPED_UNBOUNDED.load(Acquire), 3);\n\n // This msg will not be put onto `Tx` list anymore, since `Rx` is closed.\n assert!(tx.send(MsgUnbounded {}).is_err());\n\n assert_eq!(NUM_DROPPED_UNBOUNDED.load(Acquire), 4);\n}\n\n// Tests that an `WeakUnboundedSender` is upgradeable when other\n// `UnboundedSender`s exist.\n#[test]\nfn downgrade_upgrade_unbounded_sender_success() {\n let (tx, _rx) = mpsc::unbounded_channel::();\n let weak_tx = tx.downgrade();\n assert!(weak_tx.upgrade().is_some());\n}\n\n// Tests that a `WeakUnboundedSender` fails to upgrade when no other\n// `UnboundedSender` exists.\n#[test]\nfn downgrade_upgrade_unbounded_sender_failure() {\n let (tx, _rx) = mpsc::unbounded_channel::();\n let weak_tx = tx.downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that an `WeakUnboundedSender` cannot be upgraded after an\n// `UnboundedSender` was dropped, which existed at the time of the `downgrade` call.\n#[test]\nfn downgrade_drop_upgrade_unbounded() {\n let (tx, _rx) = mpsc::unbounded_channel::();\n\n // the cloned `Tx` is dropped right away\n let weak_tx = tx.clone().downgrade();\n drop(tx);\n assert!(weak_tx.upgrade().is_none());\n}\n\n// Tests that `downgrade` does not change the `tx_count` of the channel.\n#[test]\nfn test_tx_count_weak_unbounded_sender() {\n let (tx, _rx) = mpsc::unbounded_channel::();\n let tx_weak = tx.downgrade();\n let tx_weak2 = tx.downgrade();\n drop(tx);\n\n assert!(tx_weak.upgrade().is_none() && tx_weak2.upgrade().is_none());\n}\n\n#[tokio::test]\nasync fn test_rx_is_closed_when_dropping_all_senders_except_weak_senders() {\n // is_closed should return true after dropping all senders except for a weak sender\n let (tx, rx) = mpsc::channel::<()>(10);\n let _weak_sender = tx.clone().downgrade();\n drop(tx);\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn test_rx_unbounded_is_closed_when_dropping_all_senders_except_weak_senders() {\n // is_closed should return true after dropping all senders except for a weak sender\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n let _weak_sender = tx.clone().downgrade();\n drop(tx);\n assert!(rx.is_closed());\n}\n\n#[tokio::test]\nasync fn sender_strong_count_when_cloned() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n}\n\n#[tokio::test]\nasync fn sender_weak_count_when_downgraded() {\n let (tx, _rx) = mpsc::channel::<()>(1);\n\n let weak = tx.downgrade();\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n}\n\n#[tokio::test]\nasync fn sender_strong_count_when_dropped() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n drop(tx2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n}\n\n#[tokio::test]\nasync fn sender_weak_count_when_dropped() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n let weak = tx.downgrade();\n\n drop(weak);\n\n assert_eq!(tx.weak_count(), 0);\n assert_eq!(rx.sender_weak_count(), 0);\n}\n\n#[tokio::test]\nasync fn sender_strong_and_weak_conut() {\n let (tx, rx) = mpsc::channel::<()>(1);\n\n let tx2 = tx.clone();\n\n let weak = tx.downgrade();\n let weak2 = tx2.downgrade();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(weak.strong_count(), 2);\n assert_eq!(weak2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n\n assert_eq!(tx.weak_count(), 2);\n assert_eq!(tx2.weak_count(), 2);\n assert_eq!(weak.weak_count(), 2);\n assert_eq!(weak2.weak_count(), 2);\n assert_eq!(rx.sender_weak_count(), 2);\n\n drop(tx2);\n drop(weak2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(weak.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n assert_eq!(rx.sender_weak_count(), 1);\n}\n\n#[tokio::test]\nasync fn unbounded_sender_strong_count_when_cloned() {\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n\n let tx2 = tx.clone();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n}\n\n#[tokio::test]\nasync fn unbounded_sender_weak_count_when_downgraded() {\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n\n let weak = tx.downgrade();\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n assert_eq!(rx.sender_weak_count(), 1);\n}\n\n#[tokio::test]\nasync fn unbounded_sender_strong_count_when_dropped() {\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n\n let tx2 = tx.clone();\n\n drop(tx2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n}\n\n#[tokio::test]\nasync fn unbounded_sender_weak_count_when_dropped() {\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n\n let weak = tx.downgrade();\n\n drop(weak);\n\n assert_eq!(tx.weak_count(), 0);\n assert_eq!(rx.sender_weak_count(), 0);\n}\n\n#[tokio::test]\nasync fn unbounded_sender_strong_and_weak_conut() {\n let (tx, rx) = mpsc::unbounded_channel::<()>();\n\n let tx2 = tx.clone();\n\n let weak = tx.downgrade();\n let weak2 = tx2.downgrade();\n\n assert_eq!(tx.strong_count(), 2);\n assert_eq!(tx2.strong_count(), 2);\n assert_eq!(weak.strong_count(), 2);\n assert_eq!(weak2.strong_count(), 2);\n assert_eq!(rx.sender_strong_count(), 2);\n\n assert_eq!(tx.weak_count(), 2);\n assert_eq!(tx2.weak_count(), 2);\n assert_eq!(weak.weak_count(), 2);\n assert_eq!(weak2.weak_count(), 2);\n assert_eq!(rx.sender_weak_count(), 2);\n\n drop(tx2);\n drop(weak2);\n\n assert_eq!(tx.strong_count(), 1);\n assert_eq!(weak.strong_count(), 1);\n assert_eq!(rx.sender_strong_count(), 1);\n\n assert_eq!(tx.weak_count(), 1);\n assert_eq!(weak.weak_count(), 1);\n assert_eq!(rx.sender_weak_count(), 1);\n}\n" }, { "path": "tokio/tests/sync_mutex.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse tokio::sync::Mutex;\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\n\nuse std::sync::Arc;\n\n#[test]\nfn straight_execution() {\n let l = Mutex::new(100);\n\n {\n let mut t = spawn(l.lock());\n let mut g = assert_ready!(t.poll());\n assert_eq!(&*g, &100);\n *g = 99;\n }\n {\n let mut t = spawn(l.lock());\n let mut g = assert_ready!(t.poll());\n assert_eq!(&*g, &99);\n *g = 98;\n }\n {\n let mut t = spawn(l.lock());\n let g = assert_ready!(t.poll());\n assert_eq!(&*g, &98);\n }\n}\n\n#[test]\nfn readiness() {\n let l1 = Arc::new(Mutex::new(100));\n let l2 = Arc::clone(&l1);\n let mut t1 = spawn(l1.lock());\n let mut t2 = spawn(l2.lock());\n\n let g = assert_ready!(t1.poll());\n\n // We can't now acquire the lease since it's already held in g\n assert_pending!(t2.poll());\n\n // But once g unlocks, we can acquire it\n drop(g);\n assert!(t2.is_woken());\n let _t2 = assert_ready!(t2.poll());\n}\n\n/*\n#[test]\n#[ignore]\nfn lock() {\n let mut lock = Mutex::new(false);\n\n let mut lock2 = lock.clone();\n std::thread::spawn(move || {\n let l = lock2.lock();\n pin_mut!(l);\n\n let mut task = MockTask::new();\n let mut g = assert_ready!(task.poll(&mut l));\n std::thread::sleep(std::time::Duration::from_millis(500));\n *g = true;\n drop(g);\n });\n\n std::thread::sleep(std::time::Duration::from_millis(50));\n let mut task = MockTask::new();\n let l = lock.lock();\n pin_mut!(l);\n\n assert_pending!(task.poll(&mut l));\n\n std::thread::sleep(std::time::Duration::from_millis(500));\n assert!(task.is_woken());\n let result = assert_ready!(task.poll(&mut l));\n assert!(*result);\n}\n*/\n\n/// Ensure a mutex is unlocked if a future holding the lock\n/// is aborted prematurely.\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn aborted_future_1() {\n use std::time::Duration;\n use tokio::time::{interval, timeout};\n\n let m1: Arc> = Arc::new(Mutex::new(0));\n {\n let m2 = m1.clone();\n // Try to lock mutex in a future that is aborted prematurely\n timeout(Duration::from_millis(1u64), async move {\n let iv = interval(Duration::from_millis(1000));\n tokio::pin!(iv);\n let _g = m2.lock().await;\n iv.as_mut().tick().await;\n iv.as_mut().tick().await;\n })\n .await\n .unwrap_err();\n }\n // This should succeed as there is no lock left for the mutex.\n timeout(Duration::from_millis(1u64), async move {\n let _g = m1.lock().await;\n })\n .await\n .expect(\"Mutex is locked\");\n}\n\n/// This test is similar to `aborted_future_1` but this time the\n/// aborted future is waiting for the lock.\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn aborted_future_2() {\n use std::time::Duration;\n use tokio::time::timeout;\n\n let m1: Arc> = Arc::new(Mutex::new(0));\n {\n // Lock mutex\n let _lock = m1.lock().await;\n {\n let m2 = m1.clone();\n // Try to lock mutex in a future that is aborted prematurely\n timeout(Duration::from_millis(1u64), async move {\n let _g = m2.lock().await;\n })\n .await\n .unwrap_err();\n }\n }\n // This should succeed as there is no lock left for the mutex.\n timeout(Duration::from_millis(1u64), async move {\n let _g = m1.lock().await;\n })\n .await\n .expect(\"Mutex is locked\");\n}\n\n#[test]\nfn try_lock() {\n let m: Mutex = Mutex::new(0);\n {\n let g1 = m.try_lock();\n assert!(g1.is_ok());\n let g2 = m.try_lock();\n assert!(g2.is_err());\n }\n let g3 = m.try_lock();\n assert!(g3.is_ok());\n}\n\n#[maybe_tokio_test]\nasync fn debug_format() {\n let s = \"debug\";\n let m = Mutex::new(s.to_string());\n assert_eq!(format!(\"{s:?}\"), format!(\"{:?}\", m.lock().await));\n}\n\n#[maybe_tokio_test]\nasync fn mutex_debug() {\n let s = \"data\";\n let m = Mutex::new(s.to_string());\n assert_eq!(format!(\"{m:?}\"), r#\"Mutex { data: \"data\" }\"#);\n let _guard = m.lock().await;\n assert_eq!(format!(\"{m:?}\"), r#\"Mutex { data: }\"#)\n}\n" }, { "path": "tokio/tests/sync_mutex_owned.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse tokio::sync::Mutex;\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\n\nuse std::sync::Arc;\n\n#[test]\nfn straight_execution() {\n let l = Arc::new(Mutex::new(100));\n\n {\n let mut t = spawn(l.clone().lock_owned());\n let mut g = assert_ready!(t.poll());\n assert_eq!(&*g, &100);\n *g = 99;\n }\n {\n let mut t = spawn(l.clone().lock_owned());\n let mut g = assert_ready!(t.poll());\n assert_eq!(&*g, &99);\n *g = 98;\n }\n {\n let mut t = spawn(l.lock_owned());\n let g = assert_ready!(t.poll());\n assert_eq!(&*g, &98);\n }\n}\n\n#[test]\nfn readiness() {\n let l = Arc::new(Mutex::new(100));\n let mut t1 = spawn(l.clone().lock_owned());\n let mut t2 = spawn(l.lock_owned());\n\n let g = assert_ready!(t1.poll());\n\n // We can't now acquire the lease since it's already held in g\n assert_pending!(t2.poll());\n\n // But once g unlocks, we can acquire it\n drop(g);\n assert!(t2.is_woken());\n assert_ready!(t2.poll());\n}\n\n/// Ensure a mutex is unlocked if a future holding the lock\n/// is aborted prematurely.\n#[tokio::test]\n#[cfg(feature = \"full\")]\n#[cfg_attr(miri, ignore)]\nasync fn aborted_future_1() {\n use std::time::Duration;\n use tokio::time::{interval, timeout};\n\n let m1: Arc> = Arc::new(Mutex::new(0));\n {\n let m2 = m1.clone();\n // Try to lock mutex in a future that is aborted prematurely\n timeout(Duration::from_millis(1u64), async move {\n let iv = interval(Duration::from_millis(1000));\n tokio::pin!(iv);\n m2.lock_owned().await;\n iv.as_mut().tick().await;\n iv.as_mut().tick().await;\n })\n .await\n .unwrap_err();\n }\n // This should succeed as there is no lock left for the mutex.\n timeout(Duration::from_millis(1u64), async move {\n m1.lock_owned().await;\n })\n .await\n .expect(\"Mutex is locked\");\n}\n\n/// This test is similar to `aborted_future_1` but this time the\n/// aborted future is waiting for the lock.\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn aborted_future_2() {\n use std::time::Duration;\n use tokio::time::timeout;\n\n let m1: Arc> = Arc::new(Mutex::new(0));\n {\n // Lock mutex\n let _lock = m1.clone().lock_owned().await;\n {\n let m2 = m1.clone();\n // Try to lock mutex in a future that is aborted prematurely\n timeout(Duration::from_millis(1u64), async move {\n m2.lock_owned().await;\n })\n .await\n .unwrap_err();\n }\n }\n // This should succeed as there is no lock left for the mutex.\n timeout(Duration::from_millis(1u64), async move {\n m1.lock_owned().await;\n })\n .await\n .expect(\"Mutex is locked\");\n}\n\n#[test]\nfn try_lock_owned() {\n let m: Arc> = Arc::new(Mutex::new(0));\n {\n let g1 = m.clone().try_lock_owned();\n assert!(g1.is_ok());\n let g2 = m.clone().try_lock_owned();\n assert!(g2.is_err());\n }\n let g3 = m.try_lock_owned();\n assert!(g3.is_ok());\n}\n\n#[maybe_tokio_test]\nasync fn debug_format() {\n let s = \"debug\";\n let m = Arc::new(Mutex::new(s.to_string()));\n assert_eq!(format!(\"{s:?}\"), format!(\"{:?}\", m.lock_owned().await));\n}\n" }, { "path": "tokio/tests/sync_notify.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse tokio::sync::Notify;\nuse tokio_test::task::spawn;\nuse tokio_test::*;\n\n#[allow(unused)]\ntrait AssertSend: Send + Sync {}\nimpl AssertSend for Notify {}\n\n#[test]\nfn notify_notified_one() {\n let notify = Notify::new();\n let mut notified = spawn(async { notify.notified().await });\n\n notify.notify_one();\n assert_ready!(notified.poll());\n}\n\n#[test]\nfn notify_multi_notified_one() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n // add two waiters into the queue\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n // should wakeup the first one\n notify.notify_one();\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n}\n\n#[test]\nfn notify_multi_notified_last() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n // add two waiters into the queue\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n // should wakeup the last one\n notify.notify_last();\n assert_pending!(notified1.poll());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_one_notify() {\n let notify = Notify::new();\n let mut notified = spawn(async { notify.notified().await });\n\n assert_pending!(notified.poll());\n\n notify.notify_one();\n assert!(notified.is_woken());\n assert_ready!(notified.poll());\n}\n\n#[test]\nfn notified_multi_notify() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n assert!(notified1.is_woken());\n assert!(!notified2.is_woken());\n\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n}\n\n#[test]\nfn notify_notified_multi() {\n let notify = Notify::new();\n\n notify.notify_one();\n\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_drop_notified_notify() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n assert_pending!(notified1.poll());\n\n drop(notified1);\n\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_multi_notify_drop_one() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n\n assert!(notified1.is_woken());\n assert!(!notified2.is_woken());\n\n drop(notified1);\n\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_multi_notify_one_drop() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n let mut notified3 = spawn(async { notify.notified().await });\n\n // add waiters by order of poll execution\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n assert_pending!(notified3.poll());\n\n // by default fifo\n notify.notify_one();\n\n drop(notified1);\n\n // next waiter should be the one to be to woken up\n assert_ready!(notified2.poll());\n assert_pending!(notified3.poll());\n}\n\n#[test]\nfn notified_multi_notify_last_drop() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n let mut notified2 = spawn(async { notify.notified().await });\n let mut notified3 = spawn(async { notify.notified().await });\n\n // add waiters by order of poll execution\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n assert_pending!(notified3.poll());\n\n notify.notify_last();\n\n drop(notified3);\n\n // latest waiter added should be the one to woken up\n assert_ready!(notified2.poll());\n assert_pending!(notified1.poll());\n}\n\n#[test]\nfn notify_in_drop_after_wake() {\n use futures::task::ArcWake;\n use std::future::Future;\n use std::sync::Arc;\n\n let notify = Arc::new(Notify::new());\n\n struct NotifyOnDrop(Arc);\n\n impl ArcWake for NotifyOnDrop {\n fn wake_by_ref(_arc_self: &Arc) {}\n }\n\n impl Drop for NotifyOnDrop {\n fn drop(&mut self) {\n self.0.notify_waiters();\n }\n }\n\n let mut fut = Box::pin(async {\n notify.notified().await;\n });\n\n {\n let waker = futures::task::waker(Arc::new(NotifyOnDrop(notify.clone())));\n let mut cx = std::task::Context::from_waker(&waker);\n assert!(fut.as_mut().poll(&mut cx).is_pending());\n }\n\n // Now, notifying **should not** deadlock\n notify.notify_waiters();\n}\n\n#[test]\nfn notify_one_after_dropped_all() {\n let notify = Notify::new();\n let mut notified1 = spawn(async { notify.notified().await });\n\n assert_pending!(notified1.poll());\n\n notify.notify_waiters();\n notify.notify_one();\n\n drop(notified1);\n\n let mut notified2 = spawn(async { notify.notified().await });\n\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn test_notify_one_not_enabled() {\n let notify = Notify::new();\n let mut future = spawn(notify.notified());\n\n notify.notify_one();\n assert_ready!(future.poll());\n}\n\n#[test]\nfn test_notify_one_after_enable() {\n let notify = Notify::new();\n let mut future = spawn(notify.notified());\n\n future.enter(|_, fut| assert!(!fut.enable()));\n\n notify.notify_one();\n assert_ready!(future.poll());\n future.enter(|_, fut| assert!(fut.enable()));\n}\n\n#[test]\nfn test_poll_after_enable() {\n let notify = Notify::new();\n let mut future = spawn(notify.notified());\n\n future.enter(|_, fut| assert!(!fut.enable()));\n assert_pending!(future.poll());\n}\n\n#[test]\nfn test_enable_after_poll() {\n let notify = Notify::new();\n let mut future = spawn(notify.notified());\n\n assert_pending!(future.poll());\n future.enter(|_, fut| assert!(!fut.enable()));\n}\n\n#[test]\nfn test_enable_consumes_permit() {\n let notify = Notify::new();\n\n // Add a permit.\n notify.notify_one();\n\n let mut future1 = spawn(notify.notified());\n future1.enter(|_, fut| assert!(fut.enable()));\n\n let mut future2 = spawn(notify.notified());\n future2.enter(|_, fut| assert!(!fut.enable()));\n}\n\n#[test]\nfn test_waker_update() {\n use futures::task::noop_waker;\n use std::future::Future;\n use std::task::Context;\n\n let notify = Notify::new();\n let mut future = spawn(notify.notified());\n\n let noop = noop_waker();\n future.enter(|_, fut| assert_pending!(fut.poll(&mut Context::from_waker(&noop))));\n\n assert_pending!(future.poll());\n notify.notify_one();\n\n assert!(future.is_woken());\n}\n" }, { "path": "tokio/tests/sync_notify_owned.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse std::sync::Arc;\nuse tokio::sync::Notify;\nuse tokio_test::task::spawn;\nuse tokio_test::*;\n\n#[allow(unused)]\ntrait AssertSend: Send + Sync {}\nimpl AssertSend for Notify {}\n\n#[test]\nfn notify_notified_one() {\n let notify = Arc::new(Notify::new());\n let mut notified = spawn(async { notify.clone().notified_owned().await });\n\n notify.notify_one();\n assert_ready!(notified.poll());\n}\n\n#[test]\nfn notify_multi_notified_one() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n // add two waiters into the queue\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n // should wakeup the first one\n notify.notify_one();\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n}\n\n#[test]\nfn notify_multi_notified_last() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n // add two waiters into the queue\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n // should wakeup the last one\n notify.notify_last();\n assert_pending!(notified1.poll());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_one_notify() {\n let notify = Arc::new(Notify::new());\n let mut notified = spawn(async { notify.clone().notified_owned().await });\n\n assert_pending!(notified.poll());\n\n notify.notify_one();\n assert!(notified.is_woken());\n assert_ready!(notified.poll());\n}\n\n#[test]\nfn notified_multi_notify() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n assert!(notified1.is_woken());\n assert!(!notified2.is_woken());\n\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n}\n\n#[test]\nfn notify_notified_multi() {\n let notify = Arc::new(Notify::new());\n\n notify.notify_one();\n\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n assert_ready!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_drop_notified_notify() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n assert_pending!(notified1.poll());\n\n drop(notified1);\n\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_multi_notify_drop_one() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n\n notify.notify_one();\n\n assert!(notified1.is_woken());\n assert!(!notified2.is_woken());\n\n drop(notified1);\n\n assert!(notified2.is_woken());\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn notified_multi_notify_one_drop() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n let mut notified3 = spawn(async { notify.clone().notified_owned().await });\n\n // add waiters by order of poll execution\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n assert_pending!(notified3.poll());\n\n // by default fifo\n notify.notify_one();\n\n drop(notified1);\n\n // next waiter should be the one to be to woken up\n assert_ready!(notified2.poll());\n assert_pending!(notified3.poll());\n}\n\n#[test]\nfn notified_multi_notify_last_drop() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n let mut notified3 = spawn(async { notify.clone().notified_owned().await });\n\n // add waiters by order of poll execution\n assert_pending!(notified1.poll());\n assert_pending!(notified2.poll());\n assert_pending!(notified3.poll());\n\n notify.notify_last();\n\n drop(notified3);\n\n // latest waiter added should be the one to woken up\n assert_ready!(notified2.poll());\n assert_pending!(notified1.poll());\n}\n\n#[test]\nfn notify_in_drop_after_wake() {\n use futures::task::ArcWake;\n use std::future::Future;\n use std::sync::Arc;\n\n let notify = Arc::new(Notify::new());\n\n struct NotifyOnDrop(Arc);\n\n impl ArcWake for NotifyOnDrop {\n fn wake_by_ref(_arc_self: &Arc) {}\n }\n\n impl Drop for NotifyOnDrop {\n fn drop(&mut self) {\n self.0.notify_waiters();\n }\n }\n\n let mut fut = Box::pin(async {\n notify.clone().notified_owned().await;\n });\n\n {\n let waker = futures::task::waker(Arc::new(NotifyOnDrop(notify.clone())));\n let mut cx = std::task::Context::from_waker(&waker);\n assert!(fut.as_mut().poll(&mut cx).is_pending());\n }\n\n // Now, notifying **should not** deadlock\n notify.notify_waiters();\n}\n\n#[test]\nfn notify_one_after_dropped_all() {\n let notify = Arc::new(Notify::new());\n let mut notified1 = spawn(async { notify.clone().notified_owned().await });\n\n assert_pending!(notified1.poll());\n\n notify.notify_waiters();\n notify.notify_one();\n\n drop(notified1);\n\n let mut notified2 = spawn(async { notify.clone().notified_owned().await });\n\n assert_ready!(notified2.poll());\n}\n\n#[test]\nfn test_notify_one_not_enabled() {\n let notify = Arc::new(Notify::new());\n let mut future = spawn(notify.clone().notified_owned());\n\n notify.notify_one();\n assert_ready!(future.poll());\n}\n\n#[test]\nfn test_notify_one_after_enable() {\n let notify = Arc::new(Notify::new());\n let mut future = spawn(notify.clone().notified_owned());\n\n future.enter(|_, fut| assert!(!fut.enable()));\n\n notify.notify_one();\n assert_ready!(future.poll());\n future.enter(|_, fut| assert!(fut.enable()));\n}\n\n#[test]\nfn test_poll_after_enable() {\n let notify = Arc::new(Notify::new());\n let mut future = spawn(notify.clone().notified_owned());\n\n future.enter(|_, fut| assert!(!fut.enable()));\n assert_pending!(future.poll());\n}\n\n#[test]\nfn test_enable_after_poll() {\n let notify = Arc::new(Notify::new());\n let mut future = spawn(notify.clone().notified_owned());\n\n assert_pending!(future.poll());\n future.enter(|_, fut| assert!(!fut.enable()));\n}\n\n#[test]\nfn test_enable_consumes_permit() {\n let notify = Arc::new(Notify::new());\n\n // Add a permit.\n notify.notify_one();\n\n let mut future1 = spawn(notify.clone().notified_owned());\n future1.enter(|_, fut| assert!(fut.enable()));\n\n let mut future2 = spawn(notify.clone().notified_owned());\n future2.enter(|_, fut| assert!(!fut.enable()));\n}\n\n#[test]\nfn test_waker_update() {\n use futures::task::noop_waker;\n use std::future::Future;\n use std::task::Context;\n\n let notify = Arc::new(Notify::new());\n let mut future = spawn(notify.clone().notified_owned());\n\n let noop = noop_waker();\n future.enter(|_, fut| assert_pending!(fut.poll(&mut Context::from_waker(&noop))));\n\n assert_pending!(future.poll());\n notify.notify_one();\n\n assert!(future.is_woken());\n}\n" }, { "path": "tokio/tests/sync_once_cell.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::mem;\nuse std::sync::atomic::{AtomicU32, Ordering};\nuse std::sync::Arc;\nuse std::time::Duration;\nuse tokio::runtime;\nuse tokio::sync::OnceCell;\nuse tokio::sync::SetError;\nuse tokio::time;\n\nstruct Foo {\n value: Arc,\n}\n\nimpl Drop for Foo {\n fn drop(&mut self) {\n self.value.fetch_add(1, Ordering::Release);\n }\n}\n\nimpl From> for Foo {\n fn from(value: Arc) -> Self {\n Foo { value }\n }\n}\n\n#[test]\nfn drop_cell() {\n let num_drops = Arc::new(AtomicU32::new(0));\n {\n let once_cell = OnceCell::new();\n let prev = once_cell.set(Foo::from(num_drops.clone()));\n assert!(prev.is_ok())\n }\n assert!(num_drops.load(Ordering::Acquire) == 1);\n}\n\n#[test]\nfn drop_cell_new_with() {\n let num_drops = Arc::new(AtomicU32::new(0));\n {\n let once_cell = OnceCell::new_with(Some(Foo::from(num_drops.clone())));\n assert!(once_cell.initialized());\n }\n assert!(num_drops.load(Ordering::Acquire) == 1);\n}\n\n#[test]\nfn drop_into_inner() {\n let num_drops = Arc::new(AtomicU32::new(0));\n\n let once_cell = OnceCell::new();\n assert!(once_cell.set(Foo::from(num_drops.clone())).is_ok());\n let fooer = once_cell.into_inner();\n let count = num_drops.load(Ordering::Acquire);\n assert!(count == 0);\n drop(fooer);\n let count = num_drops.load(Ordering::Acquire);\n assert!(count == 1);\n}\n\n#[test]\nfn drop_into_inner_new_with() {\n let num_drops = Arc::new(AtomicU32::new(0));\n let fooer = Foo::from(num_drops.clone());\n\n let once_cell = OnceCell::new_with(Some(fooer));\n let fooer = once_cell.into_inner();\n let count = num_drops.load(Ordering::Acquire);\n assert!(count == 0);\n mem::drop(fooer);\n let count = num_drops.load(Ordering::Acquire);\n assert!(count == 1);\n}\n\n#[test]\nfn from() {\n let cell = OnceCell::from(2);\n assert_eq!(*cell.get().unwrap(), 2);\n}\n\nasync fn func1() -> u32 {\n 5\n}\n\nasync fn func2() -> u32 {\n time::sleep(Duration::from_millis(1)).await;\n 10\n}\n\nasync fn func_err() -> Result {\n Err(())\n}\n\nasync fn func_ok() -> Result {\n Ok(10)\n}\n\nasync fn func_panic() -> u32 {\n time::sleep(Duration::from_millis(1)).await;\n panic!();\n}\n\nasync fn sleep_and_set() -> u32 {\n // Simulate sleep by pausing time and waiting for another thread to\n // resume clock when calling `set`, then finding the cell being initialized\n // by this call\n time::sleep(Duration::from_millis(2)).await;\n 5\n}\n\nasync fn advance_time_and_set(cell: &'static OnceCell, v: u32) -> Result<(), SetError> {\n time::advance(Duration::from_millis(1)).await;\n cell.set(v)\n}\n\n#[test]\nfn get_or_init() {\n let rt = runtime::Builder::new_current_thread()\n .enable_time()\n .start_paused(true)\n .build()\n .unwrap();\n\n static ONCE: OnceCell = OnceCell::const_new();\n\n rt.block_on(async {\n let handle1 = rt.spawn(async { ONCE.get_or_init(func1).await });\n let handle2 = rt.spawn(async { ONCE.get_or_init(func2).await });\n\n time::advance(Duration::from_millis(1)).await;\n time::resume();\n\n let result1 = handle1.await.unwrap();\n let result2 = handle2.await.unwrap();\n\n assert_eq!(*result1, 5);\n assert_eq!(*result2, 5);\n });\n}\n\n#[test]\nfn get_or_init_panic() {\n let rt = runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap();\n\n static ONCE: OnceCell = OnceCell::const_new();\n\n rt.block_on(async {\n time::pause();\n\n let handle1 = rt.spawn(async { ONCE.get_or_init(func1).await });\n let handle2 = rt.spawn(async { ONCE.get_or_init(func_panic).await });\n\n time::advance(Duration::from_millis(1)).await;\n\n let result1 = handle1.await.unwrap();\n let result2 = handle2.await.unwrap();\n\n assert_eq!(*result1, 5);\n assert_eq!(*result2, 5);\n });\n}\n\n#[test]\nfn set_and_get() {\n let rt = runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap();\n\n static ONCE: OnceCell = OnceCell::const_new();\n\n rt.block_on(async {\n let _ = rt.spawn(async { ONCE.set(5) }).await;\n let value = ONCE.get().unwrap();\n assert_eq!(*value, 5);\n });\n}\n\n#[test]\nfn get_uninit() {\n static ONCE: OnceCell = OnceCell::const_new();\n let uninit = ONCE.get();\n assert!(uninit.is_none());\n}\n\n#[test]\nfn set_twice() {\n static ONCE: OnceCell = OnceCell::const_new();\n\n let first = ONCE.set(5);\n assert_eq!(first, Ok(()));\n let second = ONCE.set(6);\n assert!(second.err().unwrap().is_already_init_err());\n}\n\n#[test]\nfn set_while_initializing() {\n let rt = runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap();\n\n static ONCE: OnceCell = OnceCell::const_new();\n\n rt.block_on(async {\n time::pause();\n\n let handle1 = rt.spawn(async { ONCE.get_or_init(sleep_and_set).await });\n let handle2 = rt.spawn(async { advance_time_and_set(&ONCE, 10).await });\n\n time::advance(Duration::from_millis(2)).await;\n\n let result1 = handle1.await.unwrap();\n let result2 = handle2.await.unwrap();\n\n assert_eq!(*result1, 5);\n assert!(result2.err().unwrap().is_initializing_err());\n });\n}\n\n#[test]\nfn get_or_try_init() {\n let rt = runtime::Builder::new_current_thread()\n .enable_time()\n .start_paused(true)\n .build()\n .unwrap();\n\n static ONCE: OnceCell = OnceCell::const_new();\n\n rt.block_on(async {\n let handle1 = rt.spawn(async { ONCE.get_or_try_init(func_err).await });\n let handle2 = rt.spawn(async { ONCE.get_or_try_init(func_ok).await });\n\n time::advance(Duration::from_millis(1)).await;\n time::resume();\n\n let result1 = handle1.await.unwrap();\n assert!(result1.is_err());\n\n let result2 = handle2.await.unwrap();\n assert_eq!(*result2.unwrap(), 10);\n });\n}\n" }, { "path": "tokio/tests/sync_oneshot.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse tokio::sync::oneshot;\nuse tokio::sync::oneshot::error::TryRecvError;\nuse tokio_test::*;\n\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[allow(unused)]\ntrait AssertSend: Send {}\nimpl AssertSend for oneshot::Sender {}\nimpl AssertSend for oneshot::Receiver {}\n\n#[allow(unused)]\ntrait SenderExt {\n fn poll_closed(&mut self, cx: &mut Context<'_>) -> Poll<()>;\n}\n\nimpl SenderExt for oneshot::Sender {\n fn poll_closed(&mut self, cx: &mut Context<'_>) -> Poll<()> {\n tokio::pin! {\n let fut = self.closed();\n }\n fut.poll(cx)\n }\n}\n\n#[test]\nfn send_recv() {\n let (tx, rx) = oneshot::channel();\n let mut rx = task::spawn(rx);\n\n assert_pending!(rx.poll());\n\n assert_ok!(tx.send(1));\n\n assert!(rx.is_woken());\n\n let val = assert_ready_ok!(rx.poll());\n assert_eq!(val, 1);\n}\n\n#[maybe_tokio_test]\nasync fn async_send_recv() {\n let (tx, rx) = oneshot::channel();\n\n assert_ok!(tx.send(1));\n assert_eq!(1, assert_ok!(rx.await));\n}\n\n#[test]\nfn close_tx() {\n let (tx, rx) = oneshot::channel::();\n let mut rx = task::spawn(rx);\n\n assert_pending!(rx.poll());\n\n drop(tx);\n\n assert!(rx.is_woken());\n assert_ready_err!(rx.poll());\n}\n\n#[test]\nfn close_rx() {\n // First, without checking poll_closed()\n //\n let (tx, _) = oneshot::channel();\n\n assert_err!(tx.send(1));\n\n // Second, via poll_closed();\n\n let (tx, rx) = oneshot::channel();\n let mut tx = task::spawn(tx);\n\n assert_pending!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n drop(rx);\n\n assert!(tx.is_woken());\n assert!(tx.is_closed());\n assert_ready!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n assert_err!(tx.into_inner().send(1));\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn async_rx_closed() {\n let (mut tx, rx) = oneshot::channel::<()>();\n\n tokio::spawn(async move {\n drop(rx);\n });\n\n tx.closed().await;\n}\n\n#[test]\nfn explicit_close_poll() {\n // First, with message sent\n let (tx, rx) = oneshot::channel();\n let mut rx = task::spawn(rx);\n\n assert_ok!(tx.send(1));\n\n rx.close();\n\n let value = assert_ready_ok!(rx.poll());\n assert_eq!(value, 1);\n\n // Second, without the message sent\n let (tx, rx) = oneshot::channel::();\n let mut tx = task::spawn(tx);\n let mut rx = task::spawn(rx);\n\n assert_pending!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n rx.close();\n\n assert!(tx.is_woken());\n assert!(tx.is_closed());\n assert_ready!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n assert_err!(tx.into_inner().send(1));\n assert_ready_err!(rx.poll());\n\n // Again, but without sending the value this time\n let (tx, rx) = oneshot::channel::();\n let mut tx = task::spawn(tx);\n let mut rx = task::spawn(rx);\n\n assert_pending!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n rx.close();\n\n assert!(tx.is_woken());\n assert!(tx.is_closed());\n assert_ready!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n assert_ready_err!(rx.poll());\n}\n\n#[test]\nfn explicit_close_try_recv() {\n // First, with message sent\n let (tx, mut rx) = oneshot::channel();\n\n assert_ok!(tx.send(1));\n\n rx.close();\n\n let val = assert_ok!(rx.try_recv());\n assert_eq!(1, val);\n\n // Second, without the message sent\n let (tx, mut rx) = oneshot::channel::();\n let mut tx = task::spawn(tx);\n\n assert_pending!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n rx.close();\n\n assert!(tx.is_woken());\n assert!(tx.is_closed());\n assert_ready!(tx.enter(|cx, mut tx| tx.poll_closed(cx)));\n\n assert_err!(rx.try_recv());\n}\n\n#[test]\n#[should_panic]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn close_try_recv_poll() {\n let (_tx, rx) = oneshot::channel::();\n let mut rx = task::spawn(rx);\n\n rx.close();\n\n assert_err!(rx.try_recv());\n\n let _ = rx.poll();\n}\n\n#[test]\nfn close_after_recv() {\n let (tx, mut rx) = oneshot::channel::();\n\n tx.send(17).unwrap();\n\n assert_eq!(17, rx.try_recv().unwrap());\n rx.close();\n}\n\n#[test]\nfn try_recv_after_completion() {\n let (tx, mut rx) = oneshot::channel::();\n\n tx.send(17).unwrap();\n\n assert_eq!(17, rx.try_recv().unwrap());\n assert_eq!(Err(TryRecvError::Closed), rx.try_recv());\n rx.close();\n}\n\n#[test]\nfn try_recv_after_completion_await() {\n let (tx, rx) = oneshot::channel::();\n let mut rx = task::spawn(rx);\n\n tx.send(17).unwrap();\n\n assert_eq!(Ok(17), assert_ready!(rx.poll()));\n assert_eq!(Err(TryRecvError::Closed), rx.try_recv());\n rx.close();\n}\n\n#[test]\nfn drops_tasks() {\n let (mut tx, mut rx) = oneshot::channel::();\n let mut tx_task = task::spawn(());\n let mut rx_task = task::spawn(());\n\n assert_pending!(tx_task.enter(|cx, _| tx.poll_closed(cx)));\n assert_pending!(rx_task.enter(|cx, _| Pin::new(&mut rx).poll(cx)));\n\n drop(tx);\n drop(rx);\n\n assert_eq!(1, tx_task.waker_ref_count());\n assert_eq!(1, rx_task.waker_ref_count());\n}\n\n#[test]\nfn receiver_changes_task() {\n let (tx, mut rx) = oneshot::channel();\n\n let mut task1 = task::spawn(());\n let mut task2 = task::spawn(());\n\n assert_pending!(task1.enter(|cx, _| Pin::new(&mut rx).poll(cx)));\n\n assert_eq!(2, task1.waker_ref_count());\n assert_eq!(1, task2.waker_ref_count());\n\n assert_pending!(task2.enter(|cx, _| Pin::new(&mut rx).poll(cx)));\n\n assert_eq!(1, task1.waker_ref_count());\n assert_eq!(2, task2.waker_ref_count());\n\n assert_ok!(tx.send(1));\n\n assert!(!task1.is_woken());\n assert!(task2.is_woken());\n\n assert_ready_ok!(task2.enter(|cx, _| Pin::new(&mut rx).poll(cx)));\n}\n\n#[test]\nfn sender_changes_task() {\n let (mut tx, rx) = oneshot::channel::();\n\n let mut task1 = task::spawn(());\n let mut task2 = task::spawn(());\n\n assert_pending!(task1.enter(|cx, _| tx.poll_closed(cx)));\n\n assert_eq!(2, task1.waker_ref_count());\n assert_eq!(1, task2.waker_ref_count());\n\n assert_pending!(task2.enter(|cx, _| tx.poll_closed(cx)));\n\n assert_eq!(1, task1.waker_ref_count());\n assert_eq!(2, task2.waker_ref_count());\n\n drop(rx);\n\n assert!(!task1.is_woken());\n assert!(task2.is_woken());\n\n assert_ready!(task2.enter(|cx, _| tx.poll_closed(cx)));\n}\n\n#[test]\nfn receiver_is_terminated_send() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated before value is sent\"\n );\n tx.send(17).unwrap();\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated after value is sent\"\n );\n\n let mut task = task::spawn(());\n let poll = task.enter(|cx, _| Pin::new(&mut rx).poll(cx));\n assert_ready_eq!(poll, Ok(17));\n\n assert!(\n rx.is_terminated(),\n \"channel IS terminated after value is read\"\n );\n}\n\n#[test]\nfn receiver_is_terminated_try_recv() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated before value is sent\"\n );\n tx.send(17).unwrap();\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated after value is sent\"\n );\n\n let value = rx.try_recv().expect(\"value is waiting\");\n assert_eq!(value, 17);\n\n assert!(\n rx.is_terminated(),\n \"channel IS terminated after value is read\"\n );\n}\n\n#[test]\nfn receiver_is_terminated_drop() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated before sender is dropped\"\n );\n drop(tx);\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated after sender is dropped\"\n );\n\n let mut task = task::spawn(());\n let poll = task.enter(|cx, _| Pin::new(&mut rx).poll(cx));\n assert_ready_err!(poll);\n\n assert!(\n rx.is_terminated(),\n \"channel IS terminated after value is read\"\n );\n}\n\n#[test]\nfn receiver_is_terminated_rx_close() {\n let (_tx, mut rx) = oneshot::channel::();\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated before closing\"\n );\n rx.close();\n assert!(\n !rx.is_terminated(),\n \"channel is NOT terminated before closing\"\n );\n\n let mut task = task::spawn(());\n let poll = task.enter(|cx, _| Pin::new(&mut rx).poll(cx));\n assert_ready_err!(poll);\n\n assert!(\n rx.is_terminated(),\n \"channel IS terminated after value is read\"\n );\n}\n\n#[test]\nfn receiver_is_empty_send() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(rx.is_empty(), \"channel IS empty before value is sent\");\n tx.send(17).unwrap();\n assert!(!rx.is_empty(), \"channel is NOT empty after value is sent\");\n\n let mut task = task::spawn(());\n let poll = task.enter(|cx, _| Pin::new(&mut rx).poll(cx));\n assert_ready_eq!(poll, Ok(17));\n\n assert!(rx.is_empty(), \"channel IS empty after value is read\");\n}\n\n#[test]\nfn receiver_is_empty_try_recv() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(rx.is_empty(), \"channel IS empty before value is sent\");\n tx.send(17).unwrap();\n assert!(!rx.is_empty(), \"channel is NOT empty after value is sent\");\n\n let value = rx.try_recv().expect(\"value is waiting\");\n assert_eq!(value, 17);\n\n assert!(rx.is_empty(), \"channel IS empty after value is read\");\n}\n\n#[test]\nfn receiver_is_empty_drop() {\n let (tx, mut rx) = oneshot::channel::();\n\n assert!(rx.is_empty(), \"channel IS empty before sender is dropped\");\n drop(tx);\n assert!(rx.is_empty(), \"channel IS empty after sender is dropped\");\n\n let mut task = task::spawn(());\n let poll = task.enter(|cx, _| Pin::new(&mut rx).poll(cx));\n assert_ready_err!(poll);\n\n assert!(rx.is_empty(), \"channel IS empty after value is read\");\n}\n\n#[test]\nfn receiver_is_empty_rx_close() {\n let (_tx, mut rx) = oneshot::channel::();\n assert!(rx.is_empty());\n rx.close();\n assert!(rx.is_empty());\n}\n" }, { "path": "tokio/tests/sync_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))]\n#![cfg(panic = \"unwind\")]\n\nuse std::{error::Error, sync::Arc};\nuse tokio::{\n runtime::{Builder, Runtime},\n sync::{broadcast, mpsc, oneshot, Mutex, RwLock, Semaphore},\n};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[test]\nfn broadcast_channel_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let (_, _) = broadcast::channel::(0);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mutex_blocking_lock_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n rt.block_on(async {\n let mutex = Mutex::new(5_u32);\n let _g = mutex.blocking_lock();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn oneshot_blocking_recv_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n rt.block_on(async {\n let (_tx, rx) = oneshot::channel::();\n let _ = rx.blocking_recv();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn rwlock_with_max_readers_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = RwLock::::with_max_readers(0, (u32::MAX >> 3) + 1);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn rwlock_blocking_read_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n rt.block_on(async {\n let lock = RwLock::::new(0);\n let _ = lock.blocking_read();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn rwlock_blocking_write_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n rt.block_on(async {\n let lock = RwLock::::new(0);\n let _ = lock.blocking_write();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_bounded_channel_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let (_, _) = mpsc::channel::(0);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_bounded_receiver_blocking_recv_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n let (_tx, mut rx) = mpsc::channel::(1);\n rt.block_on(async {\n let _ = rx.blocking_recv();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_bounded_receiver_blocking_recv_many_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n let (_tx, mut rx) = mpsc::channel::(1);\n rt.block_on(async {\n let _ = rx.blocking_recv();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_bounded_sender_blocking_send_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n let (tx, _rx) = mpsc::channel::(1);\n rt.block_on(async {\n let _ = tx.blocking_send(3);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_unbounded_receiver_blocking_recv_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n let (_tx, mut rx) = mpsc::unbounded_channel::();\n rt.block_on(async {\n let _ = rx.blocking_recv();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn mpsc_unbounded_receiver_blocking_recv_many_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n let (_tx, mut rx) = mpsc::unbounded_channel::();\n let mut vec = vec![];\n rt.block_on(async {\n let _ = rx.blocking_recv_many(&mut vec, 1);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn semaphore_merge_unrelated_owned_permits() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let sem1 = Arc::new(Semaphore::new(42));\n let sem2 = Arc::new(Semaphore::new(42));\n let mut p1 = sem1.try_acquire_owned().unwrap();\n let p2 = sem2.try_acquire_owned().unwrap();\n p1.merge(p2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn semaphore_merge_unrelated_permits() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let sem1 = Semaphore::new(42);\n let sem2 = Semaphore::new(42);\n let mut p1 = sem1.try_acquire().unwrap();\n let p2 = sem2.try_acquire().unwrap();\n p1.merge(p2);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\nfn current_thread() -> Runtime {\n Builder::new_current_thread().enable_all().build().unwrap()\n}\n" }, { "path": "tokio/tests/sync_rwlock.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as maybe_tokio_test;\n\n#[cfg(not(all(target_family = \"wasm\", not(target_os = \"wasi\"))))]\nuse tokio::test as maybe_tokio_test;\n\nuse std::task::Poll;\n\nuse futures::future::FutureExt;\n\nuse tokio::sync::{RwLock, RwLockWriteGuard};\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\n\n#[test]\nfn into_inner() {\n let rwlock = RwLock::new(42);\n assert_eq!(rwlock.into_inner(), 42);\n}\n\n// multiple reads should be Ready\n#[test]\nfn read_shared() {\n let rwlock = RwLock::new(100);\n\n let mut t1 = spawn(rwlock.read());\n let _g1 = assert_ready!(t1.poll());\n let mut t2 = spawn(rwlock.read());\n let _g2 = assert_ready!(t2.poll());\n}\n\n// When there is an active shared owner, exclusive access should not be possible\n#[test]\nfn write_shared_pending() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.read());\n\n let _g1 = assert_ready!(t1.poll());\n let mut t2 = spawn(rwlock.write());\n assert_pending!(t2.poll());\n}\n\n// When there is an active exclusive owner, subsequent exclusive access should not be possible\n#[test]\nfn read_exclusive_pending() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.write());\n\n let _g1 = assert_ready!(t1.poll());\n let mut t2 = spawn(rwlock.read());\n assert_pending!(t2.poll());\n}\n\n// If the max shared access is reached and subsequent shared access is pending\n// should be made available when one of the shared accesses is dropped\n#[test]\nfn exhaust_reading() {\n let rwlock = RwLock::with_max_readers(100, 1024);\n let mut reads = Vec::new();\n loop {\n let mut t = spawn(rwlock.read());\n match t.poll() {\n Poll::Ready(guard) => reads.push(guard),\n Poll::Pending => break,\n }\n }\n\n let mut t1 = spawn(rwlock.read());\n assert_pending!(t1.poll());\n let g2 = reads.pop().unwrap();\n drop(g2);\n assert!(t1.is_woken());\n let _g1 = assert_ready!(t1.poll());\n}\n\n// When there is an active exclusive owner, subsequent exclusive access should not be possible\n#[test]\nfn write_exclusive_pending() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.write());\n\n let _g1 = assert_ready!(t1.poll());\n let mut t2 = spawn(rwlock.write());\n assert_pending!(t2.poll());\n}\n\n// When there is an active shared owner, exclusive access should be possible after shared is dropped\n#[test]\nfn write_shared_drop() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.read());\n\n let g1 = assert_ready!(t1.poll());\n let mut t2 = spawn(rwlock.write());\n assert_pending!(t2.poll());\n drop(g1);\n assert!(t2.is_woken());\n let _g2 = assert_ready!(t2.poll());\n}\n\n// when there is an active shared owner, and exclusive access is triggered,\n// subsequent shared access should not be possible as write gathers all the available semaphore permits\n#[test]\nfn write_read_shared_pending() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.read());\n let _g1 = assert_ready!(t1.poll());\n\n let mut t2 = spawn(rwlock.read());\n let _g2 = assert_ready!(t2.poll());\n\n let mut t3 = spawn(rwlock.write());\n assert_pending!(t3.poll());\n\n let mut t4 = spawn(rwlock.read());\n assert_pending!(t4.poll());\n}\n\n// when there is an active shared owner, and exclusive access is triggered,\n// reading should be possible after pending exclusive access is dropped\n#[test]\nfn write_read_shared_drop_pending() {\n let rwlock = RwLock::new(100);\n let mut t1 = spawn(rwlock.read());\n let _g1 = assert_ready!(t1.poll());\n\n let mut t2 = spawn(rwlock.write());\n assert_pending!(t2.poll());\n\n let mut t3 = spawn(rwlock.read());\n assert_pending!(t3.poll());\n drop(t2);\n\n assert!(t3.is_woken());\n let _t3 = assert_ready!(t3.poll());\n}\n\n// Acquire an RwLock nonexclusively by a single task\n#[maybe_tokio_test]\nasync fn read_uncontested() {\n let rwlock = RwLock::new(100);\n let result = *rwlock.read().await;\n\n assert_eq!(result, 100);\n}\n\n// Acquire an uncontested RwLock in exclusive mode\n#[maybe_tokio_test]\nasync fn write_uncontested() {\n let rwlock = RwLock::new(100);\n let mut result = rwlock.write().await;\n *result += 50;\n assert_eq!(*result, 150);\n}\n\n// RwLocks should be acquired in the order that their Futures are waited upon.\n#[maybe_tokio_test]\nasync fn write_order() {\n let rwlock = RwLock::>::new(vec![]);\n let fut2 = rwlock.write().map(|mut guard| guard.push(2));\n let fut1 = rwlock.write().map(|mut guard| guard.push(1));\n fut1.await;\n fut2.await;\n\n let g = rwlock.read().await;\n assert_eq!(*g, vec![1, 2]);\n}\n\n// A single RwLock is contested by tasks in multiple threads\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\n#[cfg_attr(miri, ignore)] // Too slow on miri.\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 8)]\nasync fn multithreaded() {\n use futures::stream::{self, StreamExt};\n use std::sync::Arc;\n use tokio::sync::Barrier;\n\n let barrier = Arc::new(Barrier::new(5));\n let rwlock = Arc::new(RwLock::::new(0));\n let rwclone1 = rwlock.clone();\n let rwclone2 = rwlock.clone();\n let rwclone3 = rwlock.clone();\n let rwclone4 = rwlock.clone();\n\n let b1 = barrier.clone();\n tokio::spawn(async move {\n stream::iter(0..1000)\n .for_each(move |_| {\n let rwlock = rwclone1.clone();\n async move {\n let mut guard = rwlock.write().await;\n *guard += 2;\n }\n })\n .await;\n b1.wait().await;\n });\n\n let b2 = barrier.clone();\n tokio::spawn(async move {\n stream::iter(0..1000)\n .for_each(move |_| {\n let rwlock = rwclone2.clone();\n async move {\n let mut guard = rwlock.write().await;\n *guard += 3;\n }\n })\n .await;\n b2.wait().await;\n });\n\n let b3 = barrier.clone();\n tokio::spawn(async move {\n stream::iter(0..1000)\n .for_each(move |_| {\n let rwlock = rwclone3.clone();\n async move {\n let mut guard = rwlock.write().await;\n *guard += 5;\n }\n })\n .await;\n b3.wait().await;\n });\n\n let b4 = barrier.clone();\n tokio::spawn(async move {\n stream::iter(0..1000)\n .for_each(move |_| {\n let rwlock = rwclone4.clone();\n async move {\n let mut guard = rwlock.write().await;\n *guard += 7;\n }\n })\n .await;\n b4.wait().await;\n });\n\n barrier.wait().await;\n let g = rwlock.read().await;\n assert_eq!(*g, 17_000);\n}\n\n#[maybe_tokio_test]\nasync fn try_write() {\n let lock = RwLock::new(0);\n let read_guard = lock.read().await;\n assert!(lock.try_write().is_err());\n drop(read_guard);\n assert!(lock.try_write().is_ok());\n}\n\n#[test]\nfn try_read_try_write() {\n let lock: RwLock = RwLock::new(15);\n\n {\n let rg1 = lock.try_read().unwrap();\n assert_eq!(*rg1, 15);\n\n assert!(lock.try_write().is_err());\n\n let rg2 = lock.try_read().unwrap();\n assert_eq!(*rg2, 15)\n }\n\n {\n let mut wg = lock.try_write().unwrap();\n *wg = 1515;\n\n assert!(lock.try_read().is_err())\n }\n\n assert_eq!(*lock.try_read().unwrap(), 1515);\n}\n\n#[maybe_tokio_test]\nasync fn downgrade_map() {\n let lock = RwLock::new(0);\n let write_guard = lock.write().await;\n let mut read_t = spawn(lock.read());\n\n // We can't create a read when a write exists\n assert_pending!(read_t.poll());\n\n // During the call to `f`, `read_t` doesn't have access yet.\n let read_guard1 = RwLockWriteGuard::downgrade_map(write_guard, |v| {\n assert_pending!(read_t.poll());\n v\n });\n\n // After the downgrade, `read_t` got the lock\n let read_guard2 = assert_ready!(read_t.poll());\n\n // Ensure they're equal, as we return the original value\n assert_eq!(&*read_guard1 as *const _, &*read_guard2 as *const _);\n}\n\n#[maybe_tokio_test]\nasync fn try_downgrade_map() {\n let lock = RwLock::new(0);\n let write_guard = lock.write().await;\n let mut read_t = spawn(lock.read());\n\n // We can't create a read when a write exists\n assert_pending!(read_t.poll());\n\n // During the call to `f`, `read_t` doesn't have access yet.\n let write_guard = RwLockWriteGuard::try_downgrade_map(write_guard, |_| {\n assert_pending!(read_t.poll());\n None::<&()>\n })\n .expect_err(\"downgrade didn't fail\");\n\n // After `f` returns `None`, `read_t` doesn't have access\n assert_pending!(read_t.poll());\n\n // After `f` returns `Some`, `read_t` does have access\n let read_guard1 = RwLockWriteGuard::try_downgrade_map(write_guard, |v| Some(v))\n .expect(\"downgrade didn't succeed\");\n let read_guard2 = assert_ready!(read_t.poll());\n\n // Ensure they're equal, as we return the original value\n assert_eq!(&*read_guard1 as *const _, &*read_guard2 as *const _);\n}\n" }, { "path": "tokio/tests/sync_semaphore.rs", "content": "#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse std::sync::Arc;\nuse tokio::sync::Semaphore;\n\n#[test]\nfn no_permits() {\n // this should not panic\n Semaphore::new(0);\n}\n\n#[test]\nfn try_acquire() {\n let sem = Semaphore::new(1);\n {\n let p1 = sem.try_acquire();\n assert!(p1.is_ok());\n let p2 = sem.try_acquire();\n assert!(p2.is_err());\n }\n let p3 = sem.try_acquire();\n assert!(p3.is_ok());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn acquire() {\n let sem = Arc::new(Semaphore::new(1));\n let p1 = sem.try_acquire().unwrap();\n let sem_clone = sem.clone();\n let j = tokio::spawn(async move {\n let _p2 = sem_clone.acquire().await;\n });\n drop(p1);\n j.await.unwrap();\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn add_permits() {\n let sem = Arc::new(Semaphore::new(0));\n let sem_clone = sem.clone();\n let j = tokio::spawn(async move {\n let _p2 = sem_clone.acquire().await;\n });\n sem.add_permits(1);\n j.await.unwrap();\n}\n\n#[test]\nfn forget() {\n let sem = Arc::new(Semaphore::new(1));\n {\n let p = sem.try_acquire().unwrap();\n assert_eq!(sem.available_permits(), 0);\n p.forget();\n assert_eq!(sem.available_permits(), 0);\n }\n assert_eq!(sem.available_permits(), 0);\n assert!(sem.try_acquire().is_err());\n}\n\n#[test]\nfn merge() {\n let sem = Arc::new(Semaphore::new(3));\n {\n let mut p1 = sem.try_acquire().unwrap();\n assert_eq!(sem.available_permits(), 2);\n let p2 = sem.try_acquire_many(2).unwrap();\n assert_eq!(sem.available_permits(), 0);\n p1.merge(p2);\n assert_eq!(sem.available_permits(), 0);\n }\n assert_eq!(sem.available_permits(), 3);\n}\n\n#[test]\n#[cfg(not(target_family = \"wasm\"))] // No stack unwinding on wasm targets\n#[should_panic]\nfn merge_unrelated_permits() {\n let sem1 = Arc::new(Semaphore::new(3));\n let sem2 = Arc::new(Semaphore::new(3));\n let mut p1 = sem1.try_acquire().unwrap();\n let p2 = sem2.try_acquire().unwrap();\n p1.merge(p2);\n}\n\n#[test]\nfn split() {\n let sem = Semaphore::new(5);\n let mut p1 = sem.try_acquire_many(3).unwrap();\n assert_eq!(sem.available_permits(), 2);\n assert_eq!(p1.num_permits(), 3);\n let mut p2 = p1.split(1).unwrap();\n assert_eq!(sem.available_permits(), 2);\n assert_eq!(p1.num_permits(), 2);\n assert_eq!(p2.num_permits(), 1);\n let p3 = p1.split(0).unwrap();\n assert_eq!(p3.num_permits(), 0);\n drop(p1);\n assert_eq!(sem.available_permits(), 4);\n let p4 = p2.split(1).unwrap();\n assert_eq!(p2.num_permits(), 0);\n assert_eq!(p4.num_permits(), 1);\n assert!(p2.split(1).is_none());\n drop(p2);\n assert_eq!(sem.available_permits(), 4);\n drop(p3);\n assert_eq!(sem.available_permits(), 4);\n drop(p4);\n assert_eq!(sem.available_permits(), 5);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn stress_test() {\n let sem = Arc::new(Semaphore::new(5));\n let mut join_handles = Vec::new();\n for _ in 0..1000 {\n let sem_clone = sem.clone();\n join_handles.push(tokio::spawn(async move {\n let _p = sem_clone.acquire().await;\n }));\n }\n for j in join_handles {\n j.await.unwrap();\n }\n // there should be exactly 5 semaphores available now\n let _p1 = sem.try_acquire().unwrap();\n let _p2 = sem.try_acquire().unwrap();\n let _p3 = sem.try_acquire().unwrap();\n let _p4 = sem.try_acquire().unwrap();\n let _p5 = sem.try_acquire().unwrap();\n assert!(sem.try_acquire().is_err());\n}\n\n#[test]\nfn add_max_amount_permits() {\n let s = tokio::sync::Semaphore::new(0);\n s.add_permits(tokio::sync::Semaphore::MAX_PERMITS);\n assert_eq!(s.available_permits(), tokio::sync::Semaphore::MAX_PERMITS);\n}\n\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\n#[test]\n#[should_panic]\nfn add_more_than_max_amount_permits1() {\n let s = tokio::sync::Semaphore::new(1);\n s.add_permits(tokio::sync::Semaphore::MAX_PERMITS);\n}\n\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\n#[test]\n#[should_panic]\nfn add_more_than_max_amount_permits2() {\n let s = Semaphore::new(Semaphore::MAX_PERMITS - 1);\n s.add_permits(1);\n s.add_permits(1);\n}\n\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\n#[test]\n#[should_panic]\nfn panic_when_exceeds_maxpermits() {\n let _ = Semaphore::new(Semaphore::MAX_PERMITS + 1);\n}\n\n#[test]\nfn no_panic_at_maxpermits() {\n let _ = Semaphore::new(Semaphore::MAX_PERMITS);\n let s = Semaphore::new(Semaphore::MAX_PERMITS - 1);\n s.add_permits(1);\n}\n" }, { "path": "tokio/tests/sync_semaphore_owned.rs", "content": "#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse std::sync::Arc;\nuse tokio::sync::Semaphore;\n\n#[test]\nfn try_acquire() {\n let sem = Arc::new(Semaphore::new(1));\n {\n let p1 = sem.clone().try_acquire_owned();\n assert!(p1.is_ok());\n let p2 = sem.clone().try_acquire_owned();\n assert!(p2.is_err());\n }\n let p3 = sem.try_acquire_owned();\n assert!(p3.is_ok());\n}\n\n#[test]\nfn try_acquire_many() {\n let sem = Arc::new(Semaphore::new(42));\n {\n let p1 = sem.clone().try_acquire_many_owned(42);\n assert!(p1.is_ok());\n let p2 = sem.clone().try_acquire_owned();\n assert!(p2.is_err());\n }\n let p3 = sem.clone().try_acquire_many_owned(32);\n assert!(p3.is_ok());\n let p4 = sem.clone().try_acquire_many_owned(10);\n assert!(p4.is_ok());\n assert!(sem.try_acquire_owned().is_err());\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn acquire() {\n let sem = Arc::new(Semaphore::new(1));\n let p1 = sem.clone().try_acquire_owned().unwrap();\n let sem_clone = sem.clone();\n let j = tokio::spawn(async move {\n let _p2 = sem_clone.acquire_owned().await;\n });\n drop(p1);\n j.await.unwrap();\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn acquire_many() {\n let semaphore = Arc::new(Semaphore::new(42));\n let permit32 = semaphore.clone().try_acquire_many_owned(32).unwrap();\n let (sender, receiver) = tokio::sync::oneshot::channel();\n let join_handle = tokio::spawn(async move {\n let _permit10 = semaphore.clone().acquire_many_owned(10).await.unwrap();\n sender.send(()).unwrap();\n let _permit32 = semaphore.acquire_many_owned(32).await.unwrap();\n });\n receiver.await.unwrap();\n drop(permit32);\n join_handle.await.unwrap();\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn add_permits() {\n let sem = Arc::new(Semaphore::new(0));\n let sem_clone = sem.clone();\n let j = tokio::spawn(async move {\n let _p2 = sem_clone.acquire_owned().await;\n });\n sem.add_permits(1);\n j.await.unwrap();\n}\n\n#[test]\nfn forget() {\n let sem = Arc::new(Semaphore::new(1));\n {\n let p = sem.clone().try_acquire_owned().unwrap();\n assert_eq!(sem.available_permits(), 0);\n p.forget();\n assert_eq!(sem.available_permits(), 0);\n }\n assert_eq!(sem.available_permits(), 0);\n assert!(sem.try_acquire_owned().is_err());\n}\n\n#[test]\nfn merge() {\n let sem = Arc::new(Semaphore::new(3));\n {\n let mut p1 = sem.clone().try_acquire_owned().unwrap();\n assert_eq!(sem.available_permits(), 2);\n let p2 = sem.clone().try_acquire_many_owned(2).unwrap();\n assert_eq!(sem.available_permits(), 0);\n p1.merge(p2);\n assert_eq!(sem.available_permits(), 0);\n }\n assert_eq!(sem.available_permits(), 3);\n}\n\n#[test]\n#[cfg(not(target_family = \"wasm\"))] // No stack unwinding on wasm targets\n#[should_panic]\nfn merge_unrelated_permits() {\n let sem1 = Arc::new(Semaphore::new(3));\n let sem2 = Arc::new(Semaphore::new(3));\n let mut p1 = sem1.try_acquire_owned().unwrap();\n let p2 = sem2.try_acquire_owned().unwrap();\n p1.merge(p2)\n}\n\n#[test]\nfn split() {\n let sem = Arc::new(Semaphore::new(5));\n let mut p1 = sem.clone().try_acquire_many_owned(3).unwrap();\n assert_eq!(sem.available_permits(), 2);\n assert_eq!(p1.num_permits(), 3);\n let mut p2 = p1.split(1).unwrap();\n assert_eq!(sem.available_permits(), 2);\n assert_eq!(p1.num_permits(), 2);\n assert_eq!(p2.num_permits(), 1);\n let p3 = p1.split(0).unwrap();\n assert_eq!(p3.num_permits(), 0);\n drop(p1);\n assert_eq!(sem.available_permits(), 4);\n let p4 = p2.split(1).unwrap();\n assert_eq!(p2.num_permits(), 0);\n assert_eq!(p4.num_permits(), 1);\n assert!(p2.split(1).is_none());\n drop(p2);\n assert_eq!(sem.available_permits(), 4);\n drop(p3);\n assert_eq!(sem.available_permits(), 4);\n drop(p4);\n assert_eq!(sem.available_permits(), 5);\n}\n\n#[tokio::test]\n#[cfg(feature = \"full\")]\nasync fn stress_test() {\n let sem = Arc::new(Semaphore::new(5));\n let mut join_handles = Vec::new();\n for _ in 0..1000 {\n let sem_clone = sem.clone();\n join_handles.push(tokio::spawn(async move {\n let _p = sem_clone.acquire_owned().await;\n }));\n }\n for j in join_handles {\n j.await.unwrap();\n }\n // there should be exactly 5 semaphores available now\n let _p1 = sem.clone().try_acquire_owned().unwrap();\n let _p2 = sem.clone().try_acquire_owned().unwrap();\n let _p3 = sem.clone().try_acquire_owned().unwrap();\n let _p4 = sem.clone().try_acquire_owned().unwrap();\n let _p5 = sem.clone().try_acquire_owned().unwrap();\n assert!(sem.try_acquire_owned().is_err());\n}\n" }, { "path": "tokio/tests/sync_set_once.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::sync::{\n atomic::{AtomicU32, Ordering},\n Arc,\n};\nuse tokio::sync::SetOnce;\n\n#[derive(Clone)]\nstruct DropCounter {\n drops: Arc,\n}\n\nimpl DropCounter {\n fn new() -> Self {\n DropCounter {\n drops: Arc::new(AtomicU32::new(0)),\n }\n }\n\n fn assert_num_drops(&self, value: u32) {\n assert_eq!(value, self.drops.load(Ordering::Relaxed));\n }\n}\n\nimpl Drop for DropCounter {\n fn drop(&mut self) {\n self.drops.fetch_add(1, Ordering::Relaxed);\n }\n}\n\n#[test]\nfn drop_cell() {\n let fooer = DropCounter::new();\n let fooer_cl = fooer.clone();\n\n {\n let once_cell = SetOnce::new();\n let prev = once_cell.set(fooer_cl);\n assert!(prev.is_ok())\n }\n\n fooer.assert_num_drops(1);\n}\n\n#[test]\nfn drop_cell_new_with() {\n let fooer = DropCounter::new();\n\n {\n let once_cell = SetOnce::new_with(Some(fooer.clone()));\n assert!(once_cell.initialized());\n }\n\n fooer.assert_num_drops(1);\n}\n\n#[test]\nfn drop_into_inner() {\n let fooer = DropCounter::new();\n\n let once_cell = SetOnce::new();\n assert!(once_cell.set(fooer.clone()).is_ok());\n let val = once_cell.into_inner();\n fooer.assert_num_drops(0);\n drop(val);\n fooer.assert_num_drops(1);\n}\n\n#[test]\nfn drop_into_inner_new_with() {\n let fooer = DropCounter::new();\n\n let once_cell = SetOnce::new_with(Some(fooer.clone()));\n let val = once_cell.into_inner();\n fooer.assert_num_drops(0);\n drop(val);\n fooer.assert_num_drops(1);\n}\n\n#[test]\nfn from() {\n let cell = SetOnce::from(2);\n assert_eq!(*cell.get().unwrap(), 2);\n}\n\n#[test]\nfn set_and_get() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n ONCE.set(5).unwrap();\n let value = ONCE.get().unwrap();\n assert_eq!(*value, 5);\n}\n\n#[tokio::test]\nasync fn set_and_wait() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n tokio::spawn(async { ONCE.set(5) });\n\n let value = ONCE.wait().await;\n assert_eq!(*value, 5);\n}\n\n#[test]\n#[cfg_attr(target_family = \"wasm\", ignore)]\nfn set_and_wait_multiple_threads() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n let res1 = std::thread::spawn(|| ONCE.set(4));\n\n let res2 = std::thread::spawn(|| ONCE.set(3));\n\n let result_first = res1.join().unwrap().is_err();\n let result_two = res2.join().unwrap().is_err();\n\n assert!(result_first != result_two);\n}\n\n#[tokio::test]\n#[cfg_attr(target_family = \"wasm\", ignore)]\nasync fn set_and_wait_threads() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n let thread = std::thread::spawn(|| {\n ONCE.set(4).unwrap();\n });\n\n let value = ONCE.wait().await;\n thread.join().unwrap();\n assert_eq!(*value, 4);\n}\n\n#[test]\nfn get_uninit() {\n static ONCE: SetOnce = SetOnce::const_new();\n let uninit = ONCE.get();\n assert!(uninit.is_none());\n}\n\n#[test]\nfn set_twice() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n let first = ONCE.set(5);\n assert_eq!(first, Ok(()));\n let second = ONCE.set(6);\n assert!(second.is_err());\n}\n\n#[test]\nfn is_none_initializing() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n assert_eq!(ONCE.get(), None);\n\n ONCE.set(20).unwrap();\n\n assert!(ONCE.set(10).is_err());\n}\n\n#[tokio::test]\nasync fn is_some_initializing() {\n static ONCE: SetOnce = SetOnce::const_new();\n\n tokio::spawn(async { ONCE.set(20) });\n\n assert_eq!(*ONCE.wait().await, 20);\n}\n\n#[test]\nfn into_inner_int_empty_setonce() {\n let once = SetOnce::::new();\n\n let val = once.into_inner();\n\n assert!(val.is_none());\n}\n" }, { "path": "tokio/tests/sync_watch.rs", "content": "#![allow(clippy::cognitive_complexity)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"sync\")]\n\n#[cfg(all(target_family = \"wasm\", not(target_os = \"wasi\")))]\nuse wasm_bindgen_test::wasm_bindgen_test as test;\n\nuse tokio::sync::watch;\nuse tokio_test::task::spawn;\nuse tokio_test::{\n assert_pending, assert_ready, assert_ready_eq, assert_ready_err, assert_ready_ok,\n};\n\n#[test]\nfn single_rx_recv() {\n let (tx, mut rx) = watch::channel(\"one\");\n\n {\n // Not initially notified\n let mut t = spawn(rx.changed());\n assert_pending!(t.poll());\n }\n assert_eq!(*rx.borrow(), \"one\");\n\n {\n let mut t = spawn(rx.changed());\n assert_pending!(t.poll());\n\n tx.send(\"two\").unwrap();\n\n assert!(t.is_woken());\n\n assert_ready_ok!(t.poll());\n }\n assert_eq!(*rx.borrow(), \"two\");\n\n {\n let mut t = spawn(rx.changed());\n assert_pending!(t.poll());\n\n drop(tx);\n\n assert!(t.is_woken());\n assert_ready_err!(t.poll());\n }\n assert_eq!(*rx.borrow(), \"two\");\n}\n\n#[test]\nfn rx_version_underflow() {\n let (_tx, mut rx) = watch::channel(\"one\");\n\n // Version starts at 2, validate we do not underflow\n rx.mark_changed();\n rx.mark_changed();\n}\n\n#[test]\nfn rx_mark_changed() {\n let (tx, mut rx) = watch::channel(\"one\");\n\n let mut rx2 = rx.clone();\n let mut rx3 = rx.clone();\n let mut rx4 = rx.clone();\n {\n rx.mark_changed();\n assert!(rx.has_changed().unwrap());\n\n let mut t = spawn(rx.changed());\n assert_ready_ok!(t.poll());\n }\n\n {\n assert!(!rx2.has_changed().unwrap());\n\n let mut t = spawn(rx2.changed());\n assert_pending!(t.poll());\n }\n\n {\n rx3.mark_changed();\n assert_eq!(*rx3.borrow(), \"one\");\n\n assert!(rx3.has_changed().unwrap());\n\n assert_eq!(*rx3.borrow_and_update(), \"one\");\n\n assert!(!rx3.has_changed().unwrap());\n\n let mut t = spawn(rx3.changed());\n assert_pending!(t.poll());\n }\n\n {\n tx.send(\"two\").unwrap();\n assert!(rx4.has_changed().unwrap());\n assert_eq!(*rx4.borrow_and_update(), \"two\");\n\n rx4.mark_changed();\n assert!(rx4.has_changed().unwrap());\n assert_eq!(*rx4.borrow_and_update(), \"two\")\n }\n\n assert_eq!(*rx.borrow(), \"two\");\n}\n\n#[test]\nfn rx_mark_unchanged() {\n let (tx, mut rx) = watch::channel(\"one\");\n\n let mut rx2 = rx.clone();\n\n {\n assert!(!rx.has_changed().unwrap());\n\n rx.mark_changed();\n assert!(rx.has_changed().unwrap());\n\n rx.mark_unchanged();\n assert!(!rx.has_changed().unwrap());\n\n let mut t = spawn(rx.changed());\n assert_pending!(t.poll());\n }\n\n {\n assert!(!rx2.has_changed().unwrap());\n\n tx.send(\"two\").unwrap();\n assert!(rx2.has_changed().unwrap());\n\n rx2.mark_unchanged();\n assert!(!rx2.has_changed().unwrap());\n assert_eq!(*rx2.borrow_and_update(), \"two\");\n }\n\n assert_eq!(*rx.borrow(), \"two\");\n}\n\n#[test]\nfn multi_rx() {\n let (tx, mut rx1) = watch::channel(\"one\");\n let mut rx2 = rx1.clone();\n\n {\n let mut t1 = spawn(rx1.changed());\n let mut t2 = spawn(rx2.changed());\n\n assert_pending!(t1.poll());\n assert_pending!(t2.poll());\n }\n assert_eq!(*rx1.borrow(), \"one\");\n assert_eq!(*rx2.borrow(), \"one\");\n\n let mut t2 = spawn(rx2.changed());\n\n {\n let mut t1 = spawn(rx1.changed());\n\n assert_pending!(t1.poll());\n assert_pending!(t2.poll());\n\n tx.send(\"two\").unwrap();\n\n assert!(t1.is_woken());\n assert!(t2.is_woken());\n\n assert_ready_ok!(t1.poll());\n }\n assert_eq!(*rx1.borrow(), \"two\");\n\n {\n let mut t1 = spawn(rx1.changed());\n\n assert_pending!(t1.poll());\n\n tx.send(\"three\").unwrap();\n\n assert!(t1.is_woken());\n assert!(t2.is_woken());\n\n assert_ready_ok!(t1.poll());\n assert_ready_ok!(t2.poll());\n }\n assert_eq!(*rx1.borrow(), \"three\");\n\n drop(t2);\n\n assert_eq!(*rx2.borrow(), \"three\");\n\n {\n let mut t1 = spawn(rx1.changed());\n let mut t2 = spawn(rx2.changed());\n\n assert_pending!(t1.poll());\n assert_pending!(t2.poll());\n\n tx.send(\"four\").unwrap();\n\n assert_ready_ok!(t1.poll());\n assert_ready_ok!(t2.poll());\n }\n assert_eq!(*rx1.borrow(), \"four\");\n assert_eq!(*rx2.borrow(), \"four\");\n}\n\n#[test]\nfn rx_observes_final_value() {\n // Initial value\n\n let (tx, mut rx) = watch::channel(\"one\");\n drop(tx);\n\n {\n let mut t1 = spawn(rx.changed());\n assert_ready_err!(t1.poll());\n }\n assert_eq!(*rx.borrow(), \"one\");\n\n // Sending a value\n\n let (tx, mut rx) = watch::channel(\"one\");\n\n tx.send(\"two\").unwrap();\n\n {\n let mut t1 = spawn(rx.changed());\n assert_ready_ok!(t1.poll());\n }\n assert_eq!(*rx.borrow(), \"two\");\n\n {\n let mut t1 = spawn(rx.changed());\n assert_pending!(t1.poll());\n\n tx.send(\"three\").unwrap();\n drop(tx);\n\n assert!(t1.is_woken());\n\n assert_ready_ok!(t1.poll());\n }\n assert_eq!(*rx.borrow(), \"three\");\n\n {\n let mut t1 = spawn(rx.changed());\n assert_ready_err!(t1.poll());\n }\n assert_eq!(*rx.borrow(), \"three\");\n}\n\n#[test]\nfn poll_close() {\n let (tx, rx) = watch::channel(\"one\");\n\n {\n let mut t = spawn(tx.closed());\n assert_pending!(t.poll());\n\n drop(rx);\n\n assert!(t.is_woken());\n assert_ready!(t.poll());\n }\n\n assert!(tx.send(\"two\").is_err());\n}\n\n#[test]\nfn borrow_and_update() {\n let (tx, mut rx) = watch::channel(\"one\");\n\n assert!(!rx.has_changed().unwrap());\n\n tx.send(\"two\").unwrap();\n assert!(rx.has_changed().unwrap());\n assert_ready!(spawn(rx.changed()).poll()).unwrap();\n assert_pending!(spawn(rx.changed()).poll());\n assert!(!rx.has_changed().unwrap());\n\n tx.send(\"three\").unwrap();\n assert!(rx.has_changed().unwrap());\n assert_eq!(*rx.borrow_and_update(), \"three\");\n assert_pending!(spawn(rx.changed()).poll());\n assert!(!rx.has_changed().unwrap());\n\n drop(tx);\n assert_eq!(*rx.borrow_and_update(), \"three\");\n assert_ready!(spawn(rx.changed()).poll()).unwrap_err();\n assert!(rx.has_changed().is_err());\n}\n\n#[test]\nfn reopened_after_subscribe() {\n let (tx, rx) = watch::channel(\"one\");\n assert!(!tx.is_closed());\n\n drop(rx);\n assert!(tx.is_closed());\n\n let rx = tx.subscribe();\n assert!(!tx.is_closed());\n\n drop(rx);\n assert!(tx.is_closed());\n}\n\n#[test]\n#[cfg(panic = \"unwind\")]\n#[cfg(not(target_family = \"wasm\"))] // wasm currently doesn't support unwinding\nfn send_modify_panic() {\n let (tx, mut rx) = watch::channel(\"one\");\n\n tx.send_modify(|old| *old = \"two\");\n assert_eq!(*rx.borrow_and_update(), \"two\");\n\n let mut rx2 = rx.clone();\n assert_eq!(*rx2.borrow_and_update(), \"two\");\n\n let mut task = spawn(rx2.changed());\n\n let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n tx.send_modify(|old| {\n *old = \"panicked\";\n panic!();\n })\n }));\n assert!(result.is_err());\n\n assert_pending!(task.poll());\n assert_eq!(*rx.borrow(), \"panicked\");\n\n tx.send_modify(|old| *old = \"three\");\n assert_ready_ok!(task.poll());\n assert_eq!(*rx.borrow_and_update(), \"three\");\n}\n\n#[tokio::test]\nasync fn multiple_sender() {\n let (tx1, mut rx) = watch::channel(0);\n let tx2 = tx1.clone();\n\n let mut t = spawn(async {\n rx.changed().await.unwrap();\n let v1 = *rx.borrow_and_update();\n rx.changed().await.unwrap();\n let v2 = *rx.borrow_and_update();\n (v1, v2)\n });\n\n tx1.send(1).unwrap();\n assert_pending!(t.poll());\n tx2.send(2).unwrap();\n assert_ready_eq!(t.poll(), (1, 2));\n}\n\n#[tokio::test]\nasync fn receiver_is_notified_when_last_sender_is_dropped() {\n let (tx1, mut rx) = watch::channel(0);\n let tx2 = tx1.clone();\n\n let mut t = spawn(rx.changed());\n assert_pending!(t.poll());\n\n drop(tx1);\n assert!(!t.is_woken());\n drop(tx2);\n\n assert!(t.is_woken());\n}\n\n#[tokio::test]\nasync fn receiver_changed_is_cooperative() {\n let (tx, mut rx) = watch::channel(());\n\n drop(tx);\n\n tokio::select! {\n biased;\n _ = async {\n loop {\n assert!(rx.changed().await.is_err());\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[tokio::test]\nasync fn receiver_changed_is_cooperative_ok() {\n let (tx, mut rx) = watch::channel(());\n\n tokio::select! {\n biased;\n _ = async {\n loop {\n assert!(tx.send(()).is_ok());\n assert!(rx.changed().await.is_ok());\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[tokio::test]\nasync fn receiver_wait_for_is_cooperative() {\n let (tx, mut rx) = watch::channel(0);\n\n drop(tx);\n\n tokio::select! {\n biased;\n _ = async {\n loop {\n assert!(rx.wait_for(|val| *val == 1).await.is_err());\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[tokio::test]\nasync fn receiver_wait_for_is_cooperative_ok() {\n let (tx, mut rx) = watch::channel(0);\n\n tokio::select! {\n biased;\n _ = async {\n loop {\n assert!(tx.send(1).is_ok());\n assert!(rx.wait_for(|val| *val == 1).await.is_ok());\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[tokio::test]\nasync fn sender_closed_is_cooperative() {\n let (tx, rx) = watch::channel(());\n\n drop(rx);\n\n tokio::select! {\n _ = async {\n loop {\n tx.closed().await;\n }\n } => {},\n _ = tokio::task::yield_now() => {},\n }\n}\n\n#[tokio::test]\nasync fn changed_succeeds_on_closed_channel_with_unseen_value() {\n let (tx, mut rx) = watch::channel(\"A\");\n tx.send(\"B\").unwrap();\n\n drop(tx);\n\n rx.changed()\n .await\n .expect(\"should not return error as long as the current value is not seen\");\n}\n\n#[tokio::test]\nasync fn changed_errors_on_closed_channel_with_seen_value() {\n let (tx, mut rx) = watch::channel(\"A\");\n drop(tx);\n\n rx.changed()\n .await\n .expect_err(\"should return error if the tx is closed and the current value is seen\");\n}\n\n#[test]\nfn has_changed_errors_on_closed_channel_with_unseen_value() {\n let (tx, rx) = watch::channel(\"A\");\n tx.send(\"B\").unwrap();\n\n drop(tx);\n\n rx.has_changed()\n .expect_err(\"`has_changed` returns an error if and only if channel is closed. Even if the current value is not seen.\");\n}\n\n#[test]\nfn has_changed_errors_on_closed_channel_with_seen_value() {\n let (tx, rx) = watch::channel(\"A\");\n drop(tx);\n\n rx.has_changed()\n .expect_err(\"`has_changed` returns an error if and only if channel is closed.\");\n}\n\n#[tokio::test]\nasync fn wait_for_errors_on_closed_channel_true_predicate() {\n let (tx, mut rx) = watch::channel(\"A\");\n tx.send(\"B\").unwrap();\n drop(tx);\n\n rx.wait_for(|_| true).await.expect(\n \"`wait_for` call does not return error even if channel is closed when predicate is true for last value.\",\n );\n}\n" }, { "path": "tokio/tests/task_abort.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support panic recovery\n\nuse std::sync::Arc;\nuse std::thread::sleep;\nuse tokio::time::Duration;\n\nuse tokio::runtime::Builder;\n\n#[cfg(panic = \"unwind\")]\nstruct PanicOnDrop;\n\n#[cfg(panic = \"unwind\")]\nimpl Drop for PanicOnDrop {\n fn drop(&mut self) {\n panic!(\"Well what did you expect would happen...\");\n }\n}\n\n/// Checks that a suspended task can be aborted without panicking as reported in\n/// issue #3157: .\n#[test]\nfn test_abort_without_panic_3157() {\n let rt = Builder::new_multi_thread()\n .enable_time()\n .worker_threads(1)\n .build()\n .unwrap();\n\n rt.block_on(async move {\n let handle = tokio::spawn(async move { tokio::time::sleep(Duration::new(100, 0)).await });\n\n // wait for task to sleep.\n tokio::time::sleep(Duration::from_millis(10)).await;\n\n handle.abort();\n let _ = handle.await;\n });\n}\n\n/// Checks that a suspended task can be aborted inside of a current_thread\n/// executor without panicking as reported in issue #3662:\n/// .\n#[test]\nfn test_abort_without_panic_3662() {\n use std::sync::atomic::{AtomicBool, Ordering};\n use std::sync::Arc;\n\n struct DropCheck(Arc);\n\n impl Drop for DropCheck {\n fn drop(&mut self) {\n self.0.store(true, Ordering::SeqCst);\n }\n }\n\n let rt = Builder::new_current_thread().build().unwrap();\n\n rt.block_on(async move {\n let drop_flag = Arc::new(AtomicBool::new(false));\n let drop_check = DropCheck(drop_flag.clone());\n\n let j = tokio::spawn(async move {\n // NB: just grab the drop check here so that it becomes part of the\n // task.\n let _drop_check = drop_check;\n futures::future::pending::<()>().await;\n });\n\n let drop_flag2 = drop_flag.clone();\n\n let task = std::thread::spawn(move || {\n // This runs in a separate thread so it doesn't have immediate\n // thread-local access to the executor. It does however transition\n // the underlying task to be completed, which will cause it to be\n // dropped (but not in this thread).\n assert!(!drop_flag2.load(Ordering::SeqCst));\n j.abort();\n j\n })\n .join()\n .unwrap();\n\n let result = task.await;\n assert!(drop_flag.load(Ordering::SeqCst));\n assert!(result.unwrap_err().is_cancelled());\n\n // Note: We do the following to trigger a deferred task cleanup.\n //\n // The relevant piece of code you want to look at is in:\n // `Inner::block_on` of `scheduler/current_thread.rs`.\n //\n // We cause the cleanup to happen by having a poll return Pending once\n // so that the scheduler can go into the \"auxiliary tasks\" mode, at\n // which point the task is removed from the scheduler.\n let i = tokio::spawn(async move {\n tokio::task::yield_now().await;\n });\n\n i.await.unwrap();\n });\n}\n\n/// Checks that a suspended LocalSet task can be aborted from a remote thread\n/// without panicking and without running the tasks destructor on the wrong thread.\n/// \n#[test]\nfn remote_abort_local_set_3929() {\n struct DropCheck {\n created_on: std::thread::ThreadId,\n not_send: std::marker::PhantomData<*const ()>,\n }\n\n impl DropCheck {\n fn new() -> Self {\n Self {\n created_on: std::thread::current().id(),\n not_send: std::marker::PhantomData,\n }\n }\n }\n impl Drop for DropCheck {\n fn drop(&mut self) {\n if std::thread::current().id() != self.created_on {\n panic!(\"non-Send value dropped in another thread!\");\n }\n }\n }\n\n let rt = Builder::new_current_thread().build().unwrap();\n let local = tokio::task::LocalSet::new();\n\n let check = DropCheck::new();\n let jh = local.spawn_local(async move {\n futures::future::pending::<()>().await;\n drop(check);\n });\n\n let jh2 = std::thread::spawn(move || {\n sleep(Duration::from_millis(10));\n jh.abort();\n });\n\n rt.block_on(local);\n jh2.join().unwrap();\n}\n\n/// Checks that a suspended task can be aborted even if the `JoinHandle` is immediately dropped.\n/// issue #3964: .\n#[test]\nfn test_abort_wakes_task_3964() {\n let rt = Builder::new_current_thread().enable_time().build().unwrap();\n\n rt.block_on(async move {\n let notify_dropped = Arc::new(());\n let weak_notify_dropped = Arc::downgrade(¬ify_dropped);\n\n let handle = tokio::spawn(async move {\n // Make sure the Arc is moved into the task\n let _notify_dropped = notify_dropped;\n tokio::time::sleep(Duration::new(100, 0)).await\n });\n\n // wait for task to sleep.\n tokio::time::sleep(Duration::from_millis(10)).await;\n\n handle.abort();\n drop(handle);\n\n // wait for task to abort.\n tokio::time::sleep(Duration::from_millis(10)).await;\n\n // Check that the Arc has been dropped.\n assert!(weak_notify_dropped.upgrade().is_none());\n });\n}\n\n/// Checks that aborting a task whose destructor panics does not allow the\n/// panic to escape the task.\n#[test]\n#[cfg(panic = \"unwind\")]\nfn test_abort_task_that_panics_on_drop_contained() {\n let rt = Builder::new_current_thread().enable_time().build().unwrap();\n\n rt.block_on(async move {\n let handle = tokio::spawn(async move {\n // Make sure the Arc is moved into the task\n let _panic_dropped = PanicOnDrop;\n tokio::time::sleep(Duration::new(100, 0)).await\n });\n\n // wait for task to sleep.\n tokio::time::sleep(Duration::from_millis(10)).await;\n\n handle.abort();\n drop(handle);\n\n // wait for task to abort.\n tokio::time::sleep(Duration::from_millis(10)).await;\n });\n}\n\n/// Checks that aborting a task whose destructor panics has the expected result.\n#[test]\n#[cfg(panic = \"unwind\")]\nfn test_abort_task_that_panics_on_drop_returned() {\n let rt = Builder::new_current_thread().enable_time().build().unwrap();\n\n rt.block_on(async move {\n let handle = tokio::spawn(async move {\n // Make sure the Arc is moved into the task\n let _panic_dropped = PanicOnDrop;\n tokio::time::sleep(Duration::new(100, 0)).await\n });\n\n // wait for task to sleep.\n tokio::time::sleep(Duration::from_millis(10)).await;\n\n handle.abort();\n assert!(handle.await.unwrap_err().is_panic());\n });\n}\n\n// It's not clear where these tests belong. This was the place suggested by @Darksonn:\n// https://github.com/tokio-rs/tokio/pull/6753#issuecomment-2271434176\n/// Checks that a `JoinError` with a panic payload prints the expected text.\n#[test]\n#[cfg(panic = \"unwind\")]\nfn test_join_error_display() {\n let rt = Builder::new_current_thread().build().unwrap();\n\n rt.block_on(async move {\n // `String` payload\n let join_err = tokio::spawn(async move {\n let value = 1234;\n panic!(\"Format-args payload: {value}\")\n })\n .await\n .unwrap_err();\n\n // We can't assert the full output because the task ID can change.\n let join_err_str = join_err.to_string();\n\n assert!(\n join_err_str.starts_with(\"task \")\n && join_err_str.ends_with(\" panicked with message \\\"Format-args payload: 1234\\\"\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n\n // `&'static str` payload\n let join_err = tokio::spawn(async move { panic!(\"Const payload\") })\n .await\n .unwrap_err();\n\n let join_err_str = join_err.to_string();\n\n assert!(\n join_err_str.starts_with(\"task \")\n && join_err_str.ends_with(\" panicked with message \\\"Const payload\\\"\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n\n // Non-string payload\n let join_err = tokio::spawn(async move { std::panic::panic_any(1234i32) })\n .await\n .unwrap_err();\n\n let join_err_str = join_err.to_string();\n\n assert!(\n join_err_str.starts_with(\"task \") && join_err_str.ends_with(\" panicked\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n });\n}\n\n/// Checks that a `JoinError` with a panic payload prints the expected text from `Debug`.\n#[test]\n#[cfg(panic = \"unwind\")]\nfn test_join_error_debug() {\n let rt = Builder::new_current_thread().build().unwrap();\n\n rt.block_on(async move {\n // `String` payload\n let join_err = tokio::spawn(async move {\n let value = 1234;\n panic!(\"Format-args payload: {value}\")\n })\n .await\n .unwrap_err();\n\n // We can't assert the full output because the task ID can change.\n let join_err_str = format!(\"{join_err:?}\");\n\n assert!(\n join_err_str.starts_with(\"JoinError::Panic(Id(\")\n && join_err_str.ends_with(\"), \\\"Format-args payload: 1234\\\", ...)\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n\n // `&'static str` payload\n let join_err = tokio::spawn(async move { panic!(\"Const payload\") })\n .await\n .unwrap_err();\n\n let join_err_str = format!(\"{join_err:?}\");\n\n assert!(\n join_err_str.starts_with(\"JoinError::Panic(Id(\")\n && join_err_str.ends_with(\"), \\\"Const payload\\\", ...)\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n\n // Non-string payload\n let join_err = tokio::spawn(async move { std::panic::panic_any(1234i32) })\n .await\n .unwrap_err();\n\n let join_err_str = format!(\"{join_err:?}\");\n\n assert!(\n join_err_str.starts_with(\"JoinError::Panic(Id(\") && join_err_str.ends_with(\"), ...)\"),\n \"Unexpected join_err_str {join_err_str:?}\"\n );\n });\n}\n" }, { "path": "tokio/tests/task_blocking.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support threads\n\nuse tokio::{runtime, task, time};\nuse tokio_test::assert_ok;\n\nuse std::thread;\nuse std::time::Duration;\n\nmod support {\n pub(crate) mod mpsc_stream;\n}\n\n#[tokio::test]\nasync fn basic_blocking() {\n // Run a few times\n for _ in 0..100 {\n let out = assert_ok!(\n tokio::spawn(async {\n assert_ok!(\n task::spawn_blocking(|| {\n thread::sleep(Duration::from_millis(5));\n \"hello\"\n })\n .await\n )\n })\n .await\n );\n\n assert_eq!(out, \"hello\");\n }\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn block_in_blocking() {\n // Run a few times\n for _ in 0..100 {\n let out = assert_ok!(\n tokio::spawn(async {\n assert_ok!(\n task::spawn_blocking(|| {\n task::block_in_place(|| {\n thread::sleep(Duration::from_millis(5));\n });\n \"hello\"\n })\n .await\n )\n })\n .await\n );\n\n assert_eq!(out, \"hello\");\n }\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn block_in_block() {\n // Run a few times\n for _ in 0..100 {\n let out = assert_ok!(\n tokio::spawn(async {\n task::block_in_place(|| {\n task::block_in_place(|| {\n thread::sleep(Duration::from_millis(5));\n });\n \"hello\"\n })\n })\n .await\n );\n\n assert_eq!(out, \"hello\");\n }\n}\n\n#[tokio::test(flavor = \"current_thread\")]\n#[should_panic]\nasync fn no_block_in_current_thread_scheduler() {\n task::block_in_place(|| {});\n}\n\n#[test]\nfn yes_block_in_threaded_block_on() {\n let rt = runtime::Runtime::new().unwrap();\n rt.block_on(async {\n task::block_in_place(|| {});\n });\n}\n\n#[test]\n#[should_panic]\nfn no_block_in_current_thread_block_on() {\n let rt = runtime::Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n task::block_in_place(|| {});\n });\n}\n\n#[test]\nfn can_enter_current_thread_rt_from_within_block_in_place() {\n let outer = tokio::runtime::Runtime::new().unwrap();\n\n outer.block_on(async {\n tokio::task::block_in_place(|| {\n let inner = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n\n inner.block_on(async {})\n })\n });\n}\n\n#[test]\n#[cfg(panic = \"unwind\")]\nfn useful_panic_message_when_dropping_rt_in_rt() {\n use std::panic::{catch_unwind, AssertUnwindSafe};\n\n let outer = tokio::runtime::Runtime::new().unwrap();\n\n let result = catch_unwind(AssertUnwindSafe(|| {\n outer.block_on(async {\n let _ = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n });\n }));\n\n assert!(result.is_err());\n let err = result.unwrap_err();\n let err: &'static str = err.downcast_ref::<&'static str>().unwrap();\n\n assert!(\n err.contains(\"Cannot drop a runtime\"),\n \"Wrong panic message: {err:?}\"\n );\n}\n\n#[test]\nfn can_shutdown_with_zero_timeout_in_runtime() {\n let outer = tokio::runtime::Runtime::new().unwrap();\n\n outer.block_on(async {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.shutdown_timeout(Duration::from_nanos(0));\n });\n}\n\n#[test]\nfn can_shutdown_now_in_runtime() {\n let outer = tokio::runtime::Runtime::new().unwrap();\n\n outer.block_on(async {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.shutdown_background();\n });\n}\n\n#[test]\nfn coop_disabled_in_block_in_place() {\n let outer = tokio::runtime::Builder::new_multi_thread()\n .enable_time()\n .build()\n .unwrap();\n\n let (tx, rx) = support::mpsc_stream::unbounded_channel_stream();\n\n for i in 0..200 {\n tx.send(i).unwrap();\n }\n drop(tx);\n\n outer.block_on(async move {\n let jh = tokio::spawn(async move {\n tokio::task::block_in_place(move || {\n futures::executor::block_on(async move {\n use tokio_stream::StreamExt;\n assert_eq!(rx.fold(0, |n, _| n + 1).await, 200);\n })\n })\n });\n\n tokio::time::timeout(Duration::from_secs(1), jh)\n .await\n .expect(\"timed out (probably hanging)\")\n .unwrap()\n });\n}\n\n#[test]\nfn coop_disabled_in_block_in_place_in_block_on() {\n let (done_tx, done_rx) = std::sync::mpsc::channel();\n let done = done_tx.clone();\n thread::spawn(move || {\n let outer = tokio::runtime::Runtime::new().unwrap();\n\n let (tx, rx) = support::mpsc_stream::unbounded_channel_stream();\n\n for i in 0..200 {\n tx.send(i).unwrap();\n }\n drop(tx);\n\n outer.block_on(async move {\n tokio::task::block_in_place(move || {\n futures::executor::block_on(async move {\n use tokio_stream::StreamExt;\n assert_eq!(rx.fold(0, |n, _| n + 1).await, 200);\n })\n })\n });\n\n let _ = done.send(Ok(()));\n });\n\n thread::spawn(move || {\n thread::sleep(Duration::from_secs(1));\n let _ = done_tx.send(Err(\"timed out (probably hanging)\"));\n });\n\n done_rx.recv().unwrap().unwrap();\n}\n\n#[cfg(feature = \"test-util\")]\n#[tokio::test(start_paused = true)]\nasync fn blocking_when_paused() {\n // Do not auto-advance time when we have started a blocking task that has\n // not yet finished.\n time::timeout(\n Duration::from_secs(3),\n task::spawn_blocking(|| thread::sleep(Duration::from_millis(1))),\n )\n .await\n .expect(\"timeout should not trigger\")\n .expect(\"blocking task should finish\");\n\n // Really: Do not auto-advance time, even if the timeout is short and the\n // blocking task runs for longer than that. It doesn't matter: Tokio time\n // is paused; system time is not.\n time::timeout(\n Duration::from_millis(1),\n task::spawn_blocking(|| thread::sleep(Duration::from_millis(50))),\n )\n .await\n .expect(\"timeout should not trigger\")\n .expect(\"blocking task should finish\");\n}\n\n#[cfg(feature = \"test-util\")]\n#[tokio::test(start_paused = true)]\nasync fn blocking_task_wakes_paused_runtime() {\n let t0 = std::time::Instant::now();\n time::timeout(\n Duration::from_secs(15),\n task::spawn_blocking(|| thread::sleep(Duration::from_millis(1))),\n )\n .await\n .expect(\"timeout should not trigger\")\n .expect(\"blocking task should finish\");\n assert!(\n t0.elapsed() < Duration::from_secs(10),\n \"completing a spawn_blocking should wake the scheduler if it's parked while time is paused\"\n );\n}\n\n#[cfg(feature = \"test-util\")]\n#[tokio::test(start_paused = true)]\nasync fn unawaited_blocking_task_wakes_paused_runtime() {\n let t0 = std::time::Instant::now();\n\n // When this task finishes, time should auto-advance, even though the\n // JoinHandle has not been awaited yet.\n let a = task::spawn_blocking(|| {\n thread::sleep(Duration::from_millis(1));\n });\n\n crate::time::sleep(Duration::from_secs(15)).await;\n a.await.expect(\"blocking task should finish\");\n assert!(\n t0.elapsed() < Duration::from_secs(10),\n \"completing a spawn_blocking should wake the scheduler if it's parked while time is paused\"\n );\n}\n\n#[cfg(panic = \"unwind\")]\n#[cfg(feature = \"test-util\")]\n#[tokio::test(start_paused = true)]\nasync fn panicking_blocking_task_wakes_paused_runtime() {\n let t0 = std::time::Instant::now();\n let result = time::timeout(\n Duration::from_secs(15),\n task::spawn_blocking(|| {\n thread::sleep(Duration::from_millis(1));\n panic!(\"blocking task panicked\");\n }),\n )\n .await\n .expect(\"timeout should not trigger\");\n assert!(result.is_err(), \"blocking task should have panicked\");\n assert!(\n t0.elapsed() < Duration::from_secs(10),\n \"completing a spawn_blocking should wake the scheduler if it's parked while time is paused\"\n );\n}\n" }, { "path": "tokio/tests/task_builder.rs", "content": "#![cfg(all(tokio_unstable, feature = \"tracing\"))]\n\nuse std::rc::Rc;\nuse tokio::task::{Builder, LocalSet};\n\n#[tokio::test]\nasync fn spawn_with_name() {\n let result = Builder::new()\n .name(\"name\")\n .spawn(async { \"task executed\" })\n .unwrap()\n .await;\n\n assert_eq!(result.unwrap(), \"task executed\");\n}\n\n#[tokio::test(flavor = \"local\")]\nasync fn spawn_local_on_local_runtime() {\n let result = Builder::new()\n .spawn_local(async { \"task executed\" })\n .unwrap()\n .await;\n\n assert_eq!(result.unwrap(), \"task executed\");\n}\n\n#[tokio::test]\n#[should_panic = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"]\nasync fn spawn_local_panics_outside_local_set_or_local_runtime() {\n let _ = Builder::new()\n .spawn_local(async { \"task executed\" })\n .unwrap()\n .await;\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\n#[should_panic = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"]\nasync fn spawn_local_panics_in_multi_thread_runtime() {\n let _ = Builder::new()\n .spawn_local(async { \"task executed\" })\n .unwrap()\n .await;\n}\n\n#[tokio::test]\nasync fn spawn_blocking_with_name() {\n let result = Builder::new()\n .name(\"name\")\n .spawn_blocking(|| \"task executed\")\n .unwrap()\n .await;\n\n assert_eq!(result.unwrap(), \"task executed\");\n}\n\n#[tokio::test]\nasync fn spawn_local_with_name() {\n let unsend_data = Rc::new(\"task executed\");\n let result = LocalSet::new()\n .run_until(async move {\n Builder::new()\n .name(\"name\")\n .spawn_local(async move { unsend_data })\n .unwrap()\n .await\n })\n .await;\n\n assert_eq!(*result.unwrap(), \"task executed\");\n}\n\n#[tokio::test]\nasync fn spawn_without_name() {\n let result = Builder::new()\n .spawn(async { \"task executed\" })\n .unwrap()\n .await;\n\n assert_eq!(result.unwrap(), \"task executed\");\n}\n\n#[tokio::test]\nasync fn spawn_blocking_without_name() {\n let result = Builder::new()\n .spawn_blocking(|| \"task executed\")\n .unwrap()\n .await;\n\n assert_eq!(result.unwrap(), \"task executed\");\n}\n\n#[tokio::test]\nasync fn spawn_local_without_name() {\n let unsend_data = Rc::new(\"task executed\");\n let result = LocalSet::new()\n .run_until(async move {\n Builder::new()\n .spawn_local(async move { unsend_data })\n .unwrap()\n .await\n })\n .await;\n\n assert_eq!(*result.unwrap(), \"task executed\");\n}\n" }, { "path": "tokio/tests/task_hooks.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", tokio_unstable, target_has_atomic = \"64\"))]\n\nuse std::collections::HashSet;\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::{Arc, Mutex};\n\nuse tokio::runtime::Builder;\n\nconst TASKS: usize = 8;\nconst ITERATIONS: usize = 64;\n/// Assert that the spawn task hook always fires when set.\n#[test]\nfn spawn_task_hook_fires() {\n let count = Arc::new(AtomicUsize::new(0));\n let count2 = Arc::clone(&count);\n\n let ids = Arc::new(Mutex::new(HashSet::new()));\n let ids2 = Arc::clone(&ids);\n\n let runtime = Builder::new_current_thread()\n .on_task_spawn(move |data| {\n ids2.lock().unwrap().insert(data.id());\n\n count2.fetch_add(1, Ordering::SeqCst);\n })\n .build()\n .unwrap();\n\n for _ in 0..TASKS {\n runtime.spawn(std::future::pending::<()>());\n }\n\n let count_realized = count.load(Ordering::SeqCst);\n assert_eq!(\n TASKS, count_realized,\n \"Total number of spawned task hook invocations was incorrect, expected {TASKS}, got {count_realized}\"\n );\n\n let count_ids_realized = ids.lock().unwrap().len();\n\n assert_eq!(\n TASKS, count_ids_realized,\n \"Total number of spawned task hook invocations was incorrect, expected {TASKS}, got {count_realized}\"\n );\n}\n\n/// Assert that the terminate task hook always fires when set.\n#[test]\nfn terminate_task_hook_fires() {\n let count = Arc::new(AtomicUsize::new(0));\n let count2 = Arc::clone(&count);\n\n let runtime = Builder::new_current_thread()\n .on_task_terminate(move |_data| {\n count2.fetch_add(1, Ordering::SeqCst);\n })\n .build()\n .unwrap();\n\n for _ in 0..TASKS {\n runtime.spawn(std::future::ready(()));\n }\n\n runtime.block_on(async {\n // tick the runtime a bunch to close out tasks\n for _ in 0..ITERATIONS {\n tokio::task::yield_now().await;\n }\n });\n\n assert_eq!(TASKS, count.load(Ordering::SeqCst));\n}\n\n/// Test that the correct spawn location is provided to the task hooks on a\n/// current thread runtime.\n#[test]\nfn task_hook_spawn_location_current_thread() {\n let spawns = Arc::new(AtomicUsize::new(0));\n let poll_starts = Arc::new(AtomicUsize::new(0));\n let poll_ends = Arc::new(AtomicUsize::new(0));\n\n let runtime = Builder::new_current_thread()\n .on_task_spawn(mk_spawn_location_hook(\n \"(current_thread) on_task_spawn\",\n &spawns,\n ))\n .on_before_task_poll(mk_spawn_location_hook(\n \"(current_thread) on_before_task_poll\",\n &poll_starts,\n ))\n .on_after_task_poll(mk_spawn_location_hook(\n \"(current_thread) on_after_task_poll\",\n &poll_ends,\n ))\n .build()\n .unwrap();\n\n let task = runtime.spawn(async move { tokio::task::yield_now().await });\n runtime.block_on(async move {\n // Spawn tasks using both `runtime.spawn(...)` and `tokio::spawn(...)`\n // to ensure the correct location is captured in both code paths.\n task.await.unwrap();\n tokio::spawn(async move {}).await.unwrap();\n\n // tick the runtime a bunch to close out tasks\n for _ in 0..ITERATIONS {\n tokio::task::yield_now().await;\n }\n });\n\n assert_eq!(spawns.load(Ordering::SeqCst), 2);\n let poll_starts = poll_starts.load(Ordering::SeqCst);\n assert!(poll_starts > 2);\n assert_eq!(poll_starts, poll_ends.load(Ordering::SeqCst));\n}\n\n/// Test that the correct spawn location is provided to the task hooks on a\n/// multi-thread runtime.\n///\n/// Testing this separately is necessary as the spawn code paths are different\n/// and we should ensure that `#[track_caller]` is passed through correctly\n/// for both runtimes.\n#[cfg_attr(\n target_os = \"wasi\",\n ignore = \"WASI does not support multi-threaded runtime\"\n)]\n#[test]\nfn task_hook_spawn_location_multi_thread() {\n let spawns = Arc::new(AtomicUsize::new(0));\n let poll_starts = Arc::new(AtomicUsize::new(0));\n let poll_ends = Arc::new(AtomicUsize::new(0));\n\n let runtime = Builder::new_multi_thread()\n .on_task_spawn(mk_spawn_location_hook(\n \"(multi_thread) on_task_spawn\",\n &spawns,\n ))\n .on_before_task_poll(mk_spawn_location_hook(\n \"(multi_thread) on_before_task_poll\",\n &poll_starts,\n ))\n .on_after_task_poll(mk_spawn_location_hook(\n \"(multi_thread) on_after_task_poll\",\n &poll_ends,\n ))\n .build()\n .unwrap();\n\n let task = runtime.spawn(async move { tokio::task::yield_now().await });\n runtime.block_on(async move {\n // Spawn tasks using both `runtime.spawn(...)` and `tokio::spawn(...)`\n // to ensure the correct location is captured in both code paths.\n task.await.unwrap();\n tokio::spawn(async move {}).await.unwrap();\n\n // tick the runtime a bunch to close out tasks\n for _ in 0..ITERATIONS {\n tokio::task::yield_now().await;\n }\n });\n\n // Give the runtime to shut down so that we see all the expected calls to\n // the task hooks.\n runtime.shutdown_timeout(std::time::Duration::from_secs(60));\n\n // Note: we \"read\" the counters using `fetch_add(0, SeqCst)` rather than\n // `load(SeqCst)` because read-write-modify operations are guaranteed to\n // observe the latest value, while the load is not.\n // This avoids a race that may cause test flakiness.\n assert_eq!(spawns.fetch_add(0, Ordering::SeqCst), 2);\n let poll_starts = poll_starts.fetch_add(0, Ordering::SeqCst);\n assert!(poll_starts > 2);\n assert_eq!(poll_starts, poll_ends.fetch_add(0, Ordering::SeqCst));\n}\n\nfn mk_spawn_location_hook(\n event: &'static str,\n count: &Arc,\n) -> impl Fn(&tokio::runtime::TaskMeta<'_>) {\n let count = Arc::clone(count);\n move |data| {\n eprintln!(\"{event} ({:?}): {:?}\", data.id(), data.spawned_at());\n // Assert that the spawn location is in this file.\n // Don't make assertions about line number/column here, as these\n // may change as new code is added to the test file...\n assert_eq!(\n data.spawned_at().file(),\n file!(),\n \"incorrect spawn location in {event} hook\",\n );\n count.fetch_add(1, Ordering::SeqCst);\n }\n}\n" }, { "path": "tokio/tests/task_id.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::error::Error;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::runtime::Runtime;\nuse tokio::sync::oneshot;\nuse tokio::task::{self, Id, LocalSet};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_spawn() {\n tokio::spawn(async { println!(\"task id: {}\", task::id()) })\n .await\n .unwrap();\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_spawn_blocking() {\n task::spawn_blocking(|| println!(\"task id: {}\", task::id()))\n .await\n .unwrap();\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_collision_current_thread() {\n let handle1 = tokio::spawn(async { task::id() });\n let handle2 = tokio::spawn(async { task::id() });\n\n let (id1, id2) = tokio::join!(handle1, handle2);\n assert_ne!(id1.unwrap(), id2.unwrap());\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn task_id_collision_multi_thread() {\n let handle1 = tokio::spawn(async { task::id() });\n let handle2 = tokio::spawn(async { task::id() });\n\n let (id1, id2) = tokio::join!(handle1, handle2);\n assert_ne!(id1.unwrap(), id2.unwrap());\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_ids_match_current_thread() {\n let (tx, rx) = oneshot::channel();\n let handle = tokio::spawn(async {\n let id = rx.await.unwrap();\n assert_eq!(id, task::id());\n });\n tx.send(handle.id()).unwrap();\n handle.await.unwrap();\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn task_ids_match_multi_thread() {\n let (tx, rx) = oneshot::channel();\n let handle = tokio::spawn(async {\n let id = rx.await.unwrap();\n assert_eq!(id, task::id());\n });\n tx.send(handle.id()).unwrap();\n handle.await.unwrap();\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn task_id_future_destructor_completion() {\n struct MyFuture {\n tx: Option>,\n }\n\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Ready(())\n }\n }\n\n impl Drop for MyFuture {\n fn drop(&mut self) {\n let _ = self.tx.take().unwrap().send(task::id());\n }\n }\n\n let (tx, rx) = oneshot::channel();\n let handle = tokio::spawn(MyFuture { tx: Some(tx) });\n let id = handle.id();\n handle.await.unwrap();\n assert_eq!(rx.await.unwrap(), id);\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn task_id_future_destructor_abort() {\n struct MyFuture {\n tx: Option>,\n }\n\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Pending\n }\n }\n impl Drop for MyFuture {\n fn drop(&mut self) {\n let _ = self.tx.take().unwrap().send(task::id());\n }\n }\n\n let (tx, rx) = oneshot::channel();\n let handle = tokio::spawn(MyFuture { tx: Some(tx) });\n let id = handle.id();\n handle.abort();\n assert!(handle.await.unwrap_err().is_cancelled());\n assert_eq!(rx.await.unwrap(), id);\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_output_destructor_handle_dropped_before_completion() {\n struct MyOutput {\n tx: Option>,\n }\n\n impl Drop for MyOutput {\n fn drop(&mut self) {\n let _ = self.tx.take().unwrap().send(task::id());\n }\n }\n\n struct MyFuture {\n tx: Option>,\n }\n\n impl Future for MyFuture {\n type Output = MyOutput;\n\n fn poll(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n Poll::Ready(MyOutput { tx: self.tx.take() })\n }\n }\n\n let (tx, mut rx) = oneshot::channel();\n let handle = tokio::spawn(MyFuture { tx: Some(tx) });\n let id = handle.id();\n drop(handle);\n assert!(rx.try_recv().is_err());\n assert_eq!(rx.await.unwrap(), id);\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_output_destructor_handle_dropped_after_completion() {\n struct MyOutput {\n tx: Option>,\n }\n\n impl Drop for MyOutput {\n fn drop(&mut self) {\n let _ = self.tx.take().unwrap().send(task::id());\n }\n }\n\n struct MyFuture {\n tx_output: Option>,\n tx_future: Option>,\n }\n\n impl Future for MyFuture {\n type Output = MyOutput;\n\n fn poll(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n let _ = self.tx_future.take().unwrap().send(());\n Poll::Ready(MyOutput {\n tx: self.tx_output.take(),\n })\n }\n }\n\n let (tx_output, mut rx_output) = oneshot::channel();\n let (tx_future, rx_future) = oneshot::channel();\n let handle = tokio::spawn(MyFuture {\n tx_output: Some(tx_output),\n tx_future: Some(tx_future),\n });\n let id = handle.id();\n rx_future.await.unwrap();\n assert!(rx_output.try_recv().is_err());\n drop(handle);\n assert_eq!(rx_output.await.unwrap(), id);\n}\n\n#[test]\nfn task_try_id_outside_task() {\n assert_eq!(None, task::try_id());\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[test]\nfn task_try_id_inside_block_on() {\n let rt = Runtime::new().unwrap();\n rt.block_on(async {\n assert_eq!(None, task::try_id());\n });\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_spawn_local() {\n LocalSet::new()\n .run_until(async {\n task::spawn_local(async { println!(\"task id: {}\", task::id()) })\n .await\n .unwrap();\n })\n .await\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn task_id_nested_spawn_local() {\n LocalSet::new()\n .run_until(async {\n task::spawn_local(async {\n let parent_id = task::id();\n LocalSet::new()\n .run_until(async {\n task::spawn_local(async move {\n assert_ne!(parent_id, task::id());\n })\n .await\n .unwrap();\n })\n .await;\n assert_eq!(parent_id, task::id());\n })\n .await\n .unwrap();\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))]\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn task_id_block_in_place_block_on_spawn() {\n use tokio::runtime::Builder;\n\n task::spawn(async {\n let parent_id = task::id();\n\n task::block_in_place(move || {\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(rt.spawn(async move {\n assert_ne!(parent_id, task::id());\n }))\n .unwrap();\n });\n\n assert_eq!(parent_id, task::id());\n })\n .await\n .unwrap();\n}\n\n#[test]\n#[cfg_attr(not(panic = \"unwind\"), ignore)]\nfn task_id_outside_task_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = task::id();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\n#[cfg_attr(not(panic = \"unwind\"), ignore)]\nfn task_id_inside_block_on_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = Runtime::new().unwrap();\n rt.block_on(async {\n task::id();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n" }, { "path": "tokio/tests/task_join_set.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse futures::future::{pending, FutureExt};\nuse std::panic;\nuse tokio::sync::oneshot;\nuse tokio::task::{JoinSet, LocalSet};\nuse tokio::time::Duration;\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n}\n\n// Spawn `N` tasks that return their index (`i`).\nfn spawn_index_tasks(set: &mut JoinSet, n: usize, on: Option<&LocalSet>) {\n for i in 0..n {\n let rc = std::rc::Rc::new(i);\n match on {\n None => set.spawn_local(async move { *rc }),\n Some(local) => set.spawn_local_on(async move { *rc }, local),\n };\n }\n}\n\n// Spawn `N` “pending” tasks that own a `oneshot::Sender`.\n// When the task is aborted the sender is dropped, which is observed\n// via the returned `Receiver`s.\nfn spawn_pending_tasks(\n set: &mut JoinSet<()>,\n receivers: &mut Vec>,\n n: usize,\n on: Option<&LocalSet>,\n) {\n for _ in 0..n {\n let (tx, rx) = oneshot::channel::<()>();\n receivers.push(rx);\n\n let fut = async move {\n pending::<()>().await;\n drop(tx);\n };\n\n match on {\n None => set.spawn_local(fut),\n Some(local) => set.spawn_local_on(fut, local),\n };\n }\n}\n\n// Await every task in a JoinSet and assert every task returns its own index.\nasync fn drain_joinset_and_assert(mut set: JoinSet, n: usize) {\n let mut seen = vec![false; n];\n while let Some(res) = set.join_next().await {\n let idx = res.expect(\"task panicked\");\n seen[idx] = true;\n }\n assert!(seen.into_iter().all(|b| b));\n assert!(set.is_empty());\n}\n\n// Await every receiver and assert they all return `Err` because the\n// corresponding sender (inside an aborted task) was dropped.\nasync fn await_receivers_and_assert(receivers: Vec>) {\n for rx in receivers {\n assert!(\n rx.await.is_err(),\n \"the task should have been aborted and the sender dropped\"\n );\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn test_with_sleep() {\n let mut set = JoinSet::new();\n\n for i in 0..10 {\n set.spawn(async move { i });\n assert_eq!(set.len(), 1 + i);\n }\n set.detach_all();\n assert_eq!(set.len(), 0);\n\n assert!(set.join_next().await.is_none());\n\n for i in 0..10 {\n set.spawn(async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n i\n });\n assert_eq!(set.len(), 1 + i);\n }\n\n let mut seen = [false; 10];\n while let Some(res) = set.join_next().await.transpose().unwrap() {\n seen[res] = true;\n }\n\n for was_seen in &seen {\n assert!(was_seen);\n }\n assert!(set.join_next().await.is_none());\n\n // Do it again.\n for i in 0..10 {\n set.spawn(async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n i\n });\n }\n\n let mut seen = [false; 10];\n while let Some(res) = set.join_next().await.transpose().unwrap() {\n seen[res] = true;\n }\n\n for was_seen in &seen {\n assert!(was_seen);\n }\n assert!(set.join_next().await.is_none());\n}\n\n#[tokio::test]\nasync fn test_abort_on_drop() {\n let mut set = JoinSet::new();\n\n let mut recvs = Vec::new();\n\n for _ in 0..16 {\n let (send, recv) = oneshot::channel::<()>();\n recvs.push(recv);\n\n set.spawn(async {\n // This task will never complete on its own.\n futures::future::pending::<()>().await;\n drop(send);\n });\n }\n\n drop(set);\n\n for recv in recvs {\n // The task is aborted soon and we will receive an error.\n assert!(recv.await.is_err());\n }\n}\n\n#[tokio::test]\nasync fn alternating() {\n let mut set = JoinSet::new();\n\n assert_eq!(set.len(), 0);\n set.spawn(async {});\n assert_eq!(set.len(), 1);\n set.spawn(async {});\n assert_eq!(set.len(), 2);\n\n for _ in 0..16 {\n let () = set.join_next().await.unwrap().unwrap();\n assert_eq!(set.len(), 1);\n set.spawn(async {});\n assert_eq!(set.len(), 2);\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn abort_tasks() {\n let mut set = JoinSet::new();\n let mut num_canceled = 0;\n let mut num_completed = 0;\n for i in 0..16 {\n let abort = set.spawn(async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n i\n });\n\n if i % 2 != 0 {\n // abort odd-numbered tasks.\n abort.abort();\n }\n }\n loop {\n match set.join_next().await {\n Some(Ok(res)) => {\n num_completed += 1;\n assert_eq!(res % 2, 0);\n }\n Some(Err(e)) => {\n assert!(e.is_cancelled());\n num_canceled += 1;\n }\n None => break,\n }\n }\n\n assert_eq!(num_canceled, 8);\n assert_eq!(num_completed, 8);\n}\n\n#[test]\nfn runtime_gone() {\n let mut set = JoinSet::new();\n {\n let rt = rt();\n set.spawn_on(async { 1 }, rt.handle());\n drop(rt);\n }\n\n assert!(rt()\n .block_on(set.join_next())\n .unwrap()\n .unwrap_err()\n .is_cancelled());\n}\n\n#[tokio::test]\nasync fn join_all() {\n let mut set: JoinSet = JoinSet::new();\n\n for _ in 0..5 {\n set.spawn(async { 1 });\n }\n let res: Vec = set.join_all().await;\n\n assert_eq!(res.len(), 5);\n for itm in res.into_iter() {\n assert_eq!(itm, 1)\n }\n}\n\n#[cfg(panic = \"unwind\")]\n#[tokio::test(start_paused = true)]\nasync fn task_panics() {\n let mut set: JoinSet<()> = JoinSet::new();\n\n let (tx, mut rx) = oneshot::channel();\n assert_eq!(set.len(), 0);\n\n set.spawn(async move {\n tokio::time::sleep(Duration::from_secs(2)).await;\n tx.send(()).unwrap();\n });\n assert_eq!(set.len(), 1);\n\n set.spawn(async {\n tokio::time::sleep(Duration::from_secs(1)).await;\n panic!();\n });\n assert_eq!(set.len(), 2);\n\n let panic = tokio::spawn(set.join_all()).await.unwrap_err();\n assert!(rx.try_recv().is_err());\n assert!(panic.is_panic());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn abort_all() {\n let mut set: JoinSet<()> = JoinSet::new();\n\n for _ in 0..5 {\n set.spawn(futures::future::pending());\n }\n for _ in 0..5 {\n set.spawn(async {\n tokio::time::sleep(Duration::from_secs(1)).await;\n });\n }\n\n // The join set will now have 5 pending tasks and 5 ready tasks.\n tokio::time::sleep(Duration::from_secs(2)).await;\n\n set.abort_all();\n assert_eq!(set.len(), 10);\n\n let mut count = 0;\n while let Some(res) = set.join_next().await {\n if let Err(err) = res {\n assert!(err.is_cancelled());\n }\n count += 1;\n }\n assert_eq!(count, 10);\n assert_eq!(set.len(), 0);\n}\n\n// This ensures that `join_next` works correctly when the coop budget is\n// exhausted.\n#[tokio::test(flavor = \"current_thread\")]\nasync fn join_set_coop() {\n // Large enough to trigger coop.\n const TASK_NUM: u32 = 1000;\n\n static SEM: tokio::sync::Semaphore = tokio::sync::Semaphore::const_new(0);\n\n let mut set = JoinSet::new();\n\n for _ in 0..TASK_NUM {\n set.spawn(async {\n SEM.add_permits(1);\n });\n }\n\n // Wait for all tasks to complete.\n //\n // Since this is a `current_thread` runtime, there's no race condition\n // between the last permit being added and the task completing.\n let _ = SEM.acquire_many(TASK_NUM).await.unwrap();\n\n let mut count = 0;\n let mut coop_count = 0;\n loop {\n match set.join_next().now_or_never() {\n Some(Some(Ok(()))) => {}\n Some(Some(Err(err))) => panic!(\"failed: {err}\"),\n None => {\n coop_count += 1;\n tokio::task::yield_now().await;\n continue;\n }\n Some(None) => break,\n }\n\n count += 1;\n }\n assert!(coop_count >= 1);\n assert_eq!(count, TASK_NUM);\n}\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn try_join_next() {\n const TASK_NUM: u32 = 1000;\n\n let (send, recv) = tokio::sync::watch::channel(());\n\n let mut set = JoinSet::new();\n\n for _ in 0..TASK_NUM {\n let mut recv = recv.clone();\n set.spawn(async move { recv.changed().await.unwrap() });\n }\n drop(recv);\n\n assert!(set.try_join_next().is_none());\n\n send.send_replace(());\n send.closed().await;\n\n let mut count = 0;\n loop {\n match set.try_join_next() {\n Some(Ok(())) => {\n count += 1;\n }\n Some(Err(err)) => panic!(\"failed: {err}\"),\n None => {\n break;\n }\n }\n }\n\n assert_eq!(count, TASK_NUM);\n}\n\n#[cfg(tokio_unstable)]\n#[tokio::test(flavor = \"current_thread\")]\nasync fn try_join_next_with_id() {\n const TASK_NUM: u32 = 1000;\n\n let (send, recv) = tokio::sync::watch::channel(());\n\n let mut set = JoinSet::new();\n let mut spawned = std::collections::HashSet::with_capacity(TASK_NUM as usize);\n\n for _ in 0..TASK_NUM {\n let mut recv = recv.clone();\n let handle = set.spawn(async move { recv.changed().await.unwrap() });\n\n spawned.insert(handle.id());\n }\n drop(recv);\n\n assert!(set.try_join_next_with_id().is_none());\n\n send.send_replace(());\n send.closed().await;\n\n let mut count = 0;\n let mut joined = std::collections::HashSet::with_capacity(TASK_NUM as usize);\n loop {\n match set.try_join_next_with_id() {\n Some(Ok((id, ()))) => {\n count += 1;\n joined.insert(id);\n }\n Some(Err(err)) => panic!(\"failed: {err}\"),\n None => {\n break;\n }\n }\n }\n\n assert_eq!(count, TASK_NUM);\n assert_eq!(joined, spawned);\n}\n\n#[tokio::test]\nasync fn extend() {\n let mut set: JoinSet<_> = (0..5).map(|i| async move { i }).collect();\n\n set.extend((5..10).map(|i| async move { i }));\n\n let mut seen = [false; 10];\n while let Some(res) = set.join_next().await {\n let idx = res.unwrap();\n seen[idx] = true;\n }\n\n for s in &seen {\n assert!(s);\n }\n}\n\nmod spawn_local {\n use super::*;\n\n #[test]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n fn panic_outside_any_runtime() {\n let mut set = JoinSet::new();\n set.spawn_local(async {});\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n async fn panic_in_multi_thread_runtime() {\n let mut set = JoinSet::new();\n set.spawn_local(async {});\n }\n\n #[cfg(tokio_unstable)]\n mod local_runtime {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n\n let mut set = JoinSet::new();\n spawn_index_tasks(&mut set, N, None);\n\n assert!(set.try_join_next().is_none());\n drain_joinset_and_assert(set, N).await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinSet`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, None);\n\n assert!(set.try_join_next().is_none());\n set.shutdown().await;\n assert!(set.is_empty());\n\n await_receivers_and_assert(receivers).await;\n }\n\n /// Spawn several pending-forever tasks, and then drop the [`JoinSet`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, None);\n\n assert!(set.try_join_next().is_none());\n drop(set);\n\n await_receivers_and_assert(receivers).await;\n }\n }\n\n mod local_set {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async move {\n let mut set = JoinSet::new();\n spawn_index_tasks(&mut set, N, None);\n drain_joinset_and_assert(set, N).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinSet`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async {\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, None);\n assert!(set.try_join_next().is_none());\n\n set.shutdown().await;\n assert!(set.is_empty());\n\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then drop the [`JoinSet`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async {\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, None);\n assert!(set.try_join_next().is_none());\n\n drop(set);\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n }\n}\n\nmod spawn_local_on {\n use super::*;\n\n #[cfg(tokio_unstable)]\n mod local_runtime {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n\n let local = LocalSet::new();\n let mut set = JoinSet::new();\n\n spawn_index_tasks(&mut set, N, Some(&local));\n assert!(set.try_join_next().is_none());\n\n local\n .run_until(async move {\n drain_joinset_and_assert(set, N).await;\n })\n .await;\n }\n }\n\n mod local_set {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n let local = LocalSet::new();\n let mut pending_set = JoinSet::new();\n\n spawn_index_tasks(&mut pending_set, N, Some(&local));\n assert!(pending_set.try_join_next().is_none());\n\n local\n .run_until(async move {\n drain_joinset_and_assert(pending_set, N).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinSet`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n let local = LocalSet::new();\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, Some(&local));\n assert!(set.try_join_next().is_none());\n\n local\n .run_until(async move {\n set.shutdown().await;\n assert!(set.is_empty());\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks and then drop the [`JoinSet`]\n /// before the `LocalSet` is driven and while the `LocalSet` is already driven.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n\n {\n let local = LocalSet::new();\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, Some(&local));\n assert!(set.try_join_next().is_none());\n\n drop(set);\n\n local\n .run_until(async move {\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n\n {\n let local = LocalSet::new();\n let mut set = JoinSet::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut set, &mut receivers, N, Some(&local));\n assert!(set.try_join_next().is_none());\n\n local\n .run_until(async move {\n drop(set);\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n }\n }\n}\n" }, { "path": "tokio/tests/task_local.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\n\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::sync::oneshot;\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn local() {\n tokio::task_local! {\n static REQ_ID: u32;\n pub static FOO: bool;\n }\n\n let j1 = tokio::spawn(REQ_ID.scope(1, async move {\n assert_eq!(REQ_ID.get(), 1);\n assert_eq!(REQ_ID.get(), 1);\n }));\n\n let j2 = tokio::spawn(REQ_ID.scope(2, async move {\n REQ_ID.with(|v| {\n assert_eq!(REQ_ID.get(), 2);\n assert_eq!(*v, 2);\n });\n\n tokio::time::sleep(std::time::Duration::from_millis(10)).await;\n\n assert_eq!(REQ_ID.get(), 2);\n }));\n\n let j3 = tokio::spawn(FOO.scope(true, async move {\n assert!(FOO.get());\n }));\n\n j1.await.unwrap();\n j2.await.unwrap();\n j3.await.unwrap();\n}\n\n#[tokio::test]\nasync fn task_local_available_on_abort() {\n tokio::task_local! {\n static KEY: u32;\n }\n\n struct MyFuture {\n tx_poll: Option>,\n tx_drop: Option>,\n }\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n if let Some(tx_poll) = self.tx_poll.take() {\n let _ = tx_poll.send(());\n }\n Poll::Pending\n }\n }\n impl Drop for MyFuture {\n fn drop(&mut self) {\n let _ = self.tx_drop.take().unwrap().send(KEY.get());\n }\n }\n\n let (tx_drop, rx_drop) = oneshot::channel();\n let (tx_poll, rx_poll) = oneshot::channel();\n\n let h = tokio::spawn(KEY.scope(\n 42,\n MyFuture {\n tx_poll: Some(tx_poll),\n tx_drop: Some(tx_drop),\n },\n ));\n\n rx_poll.await.unwrap();\n h.abort();\n assert_eq!(rx_drop.await.unwrap(), 42);\n\n let err = h.await.unwrap_err();\n if !err.is_cancelled() {\n if let Ok(panic) = err.try_into_panic() {\n std::panic::resume_unwind(panic);\n } else {\n panic!();\n }\n }\n}\n\n#[tokio::test]\nasync fn task_local_available_on_completion_drop() {\n tokio::task_local! {\n static KEY: u32;\n }\n\n struct MyFuture {\n tx: Option>,\n }\n impl Future for MyFuture {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {\n Poll::Ready(())\n }\n }\n impl Drop for MyFuture {\n fn drop(&mut self) {\n let _ = self.tx.take().unwrap().send(KEY.get());\n }\n }\n\n let (tx, rx) = oneshot::channel();\n\n let h = tokio::spawn(KEY.scope(42, MyFuture { tx: Some(tx) }));\n\n assert_eq!(rx.await.unwrap(), 42);\n h.await.unwrap();\n}\n\n#[tokio::test]\nasync fn take_value() {\n tokio::task_local! {\n static KEY: u32\n }\n let fut = KEY.scope(1, async {});\n let mut pinned = Box::pin(fut);\n assert_eq!(pinned.as_mut().take_value(), Some(1));\n assert_eq!(pinned.as_mut().take_value(), None);\n}\n\n#[tokio::test]\nasync fn poll_after_take_value_should_fail() {\n tokio::task_local! {\n static KEY: u32\n }\n let fut = KEY.scope(1, async {\n let result = KEY.try_with(|_| {});\n // The task local value no longer exists.\n assert!(result.is_err());\n });\n let mut fut = Box::pin(fut);\n fut.as_mut().take_value();\n\n // Poll the future after `take_value` has been called\n fut.await;\n}\n\n#[tokio::test]\nasync fn get_value() {\n tokio::task_local! {\n static KEY: u32\n }\n\n KEY.scope(1, async {\n assert_eq!(KEY.get(), 1);\n assert_eq!(KEY.try_get().unwrap(), 1);\n })\n .await;\n\n let fut = KEY.scope(1, async {\n let result = KEY.try_get();\n // The task local value no longer exists.\n assert!(result.is_err());\n });\n let mut fut = Box::pin(fut);\n fut.as_mut().take_value();\n\n // Poll the future after `take_value` has been called\n fut.await;\n}\n" }, { "path": "tokio/tests/task_local_set.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse futures::{\n future::{pending, ready},\n FutureExt,\n};\n\nuse tokio::runtime;\nuse tokio::sync::{mpsc, oneshot};\nuse tokio::task::{self, LocalSet};\nuse tokio::time;\n\n#[cfg(not(target_os = \"wasi\"))]\nuse std::cell::Cell;\nuse std::sync::atomic::AtomicBool;\n#[cfg(not(target_os = \"wasi\"))]\nuse std::sync::atomic::AtomicUsize;\nuse std::sync::atomic::Ordering;\n#[cfg(not(target_os = \"wasi\"))]\nuse std::sync::atomic::Ordering::SeqCst;\nuse std::time::Duration;\n\n#[tokio::test(flavor = \"current_thread\")]\nasync fn local_current_thread_scheduler() {\n LocalSet::new()\n .run_until(async {\n task::spawn_local(async {}).await.unwrap();\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn local_threadpool() {\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n LocalSet::new()\n .run_until(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n })\n .await\n .unwrap();\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn localset_future_threadpool() {\n thread_local! {\n static ON_LOCAL_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_LOCAL_THREAD.with(|cell| cell.set(true));\n\n let local = LocalSet::new();\n local.spawn_local(async move {\n assert!(ON_LOCAL_THREAD.with(|cell| cell.get()));\n });\n local.await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn localset_future_timers() {\n static RAN1: AtomicBool = AtomicBool::new(false);\n static RAN2: AtomicBool = AtomicBool::new(false);\n\n let local = LocalSet::new();\n local.spawn_local(async move {\n time::sleep(Duration::from_millis(5)).await;\n RAN1.store(true, Ordering::SeqCst);\n });\n local.spawn_local(async move {\n time::sleep(Duration::from_millis(10)).await;\n RAN2.store(true, Ordering::SeqCst);\n });\n local.await;\n assert!(RAN1.load(Ordering::SeqCst));\n assert!(RAN2.load(Ordering::SeqCst));\n}\n\n#[tokio::test]\nasync fn localset_future_drives_all_local_futs() {\n static RAN1: AtomicBool = AtomicBool::new(false);\n static RAN2: AtomicBool = AtomicBool::new(false);\n static RAN3: AtomicBool = AtomicBool::new(false);\n\n let local = LocalSet::new();\n local.spawn_local(async move {\n task::spawn_local(async {\n task::yield_now().await;\n RAN3.store(true, Ordering::SeqCst);\n });\n task::yield_now().await;\n RAN1.store(true, Ordering::SeqCst);\n });\n local.spawn_local(async move {\n task::yield_now().await;\n RAN2.store(true, Ordering::SeqCst);\n });\n local.await;\n assert!(RAN1.load(Ordering::SeqCst));\n assert!(RAN2.load(Ordering::SeqCst));\n assert!(RAN3.load(Ordering::SeqCst));\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn local_threadpool_timer() {\n // This test ensures that runtime services like the timer are properly\n // set for the local task set.\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n LocalSet::new()\n .run_until(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n let join = task::spawn_local(async move {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n time::sleep(Duration::from_millis(10)).await;\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n });\n join.await.unwrap();\n })\n .await;\n}\n#[test]\nfn enter_guard_spawn() {\n let local = LocalSet::new();\n let _guard = local.enter();\n // Run the local task set.\n\n let join = task::spawn_local(async { true });\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n local.block_on(&rt, async move {\n assert!(join.await.unwrap());\n });\n}\n\n#[cfg(not(target_os = \"wasi\"))]\nmod block_in_place_cases {\n use super::*;\n use std::future::Future;\n use std::pin::Pin;\n use std::task::{Context, Poll};\n\n struct BlockInPlaceOnDrop;\n impl Future for BlockInPlaceOnDrop {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n Poll::Pending\n }\n }\n impl Drop for BlockInPlaceOnDrop {\n fn drop(&mut self) {\n tokio::task::block_in_place(|| {});\n }\n }\n\n async fn complete(jh: tokio::task::JoinHandle<()>) {\n match jh.await {\n Ok(()) => {}\n Err(err) if err.is_panic() => std::panic::resume_unwind(err.into_panic()),\n Err(err) if err.is_cancelled() => panic!(\"task cancelled\"),\n Err(err) => panic!(\"{err:?}\"),\n }\n }\n\n #[test]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n fn local_threadpool_blocking_in_place() {\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n LocalSet::new().block_on(&rt, async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n let join = task::spawn_local(async move {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::block_in_place(|| {});\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n });\n complete(join).await;\n });\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_run_until_mt() {\n let local_set = LocalSet::new();\n local_set\n .run_until(async {\n tokio::task::block_in_place(|| {});\n })\n .await;\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_spawn_local_mt() {\n let local_set = LocalSet::new();\n let jh = local_set.spawn_local(async {\n tokio::task::block_in_place(|| {});\n });\n local_set.await;\n complete(jh).await;\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_spawn_local_drop_mt() {\n let local_set = LocalSet::new();\n let jh = local_set.spawn_local(BlockInPlaceOnDrop);\n local_set.run_until(tokio::task::yield_now()).await;\n drop(local_set);\n complete(jh).await;\n }\n\n #[tokio::test(flavor = \"current_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_run_until_ct() {\n let local_set = LocalSet::new();\n local_set\n .run_until(async {\n tokio::task::block_in_place(|| {});\n })\n .await;\n }\n\n #[tokio::test(flavor = \"current_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_spawn_local_ct() {\n let local_set = LocalSet::new();\n let jh = local_set.spawn_local(async {\n tokio::task::block_in_place(|| {});\n });\n local_set.await;\n complete(jh).await;\n }\n\n #[tokio::test(flavor = \"current_thread\")]\n #[should_panic = \"can call blocking only when running on the multi-threaded runtime\"]\n async fn block_in_place_in_spawn_local_drop_ct() {\n let local_set = LocalSet::new();\n let jh = local_set.spawn_local(BlockInPlaceOnDrop);\n local_set.run_until(tokio::task::yield_now()).await;\n drop(local_set);\n complete(jh).await;\n }\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn local_threadpool_blocking_run() {\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n LocalSet::new()\n .run_until(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n let join = task::spawn_local(async move {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_blocking(|| {\n assert!(\n !ON_RT_THREAD.with(|cell| cell.get()),\n \"blocking must not run on the local task set's thread\"\n );\n })\n .await\n .unwrap();\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n });\n join.await.unwrap();\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn all_spawns_are_local() {\n use futures::future;\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n LocalSet::new()\n .run_until(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n let handles = (0..128)\n .map(|_| {\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n })\n })\n .collect::>();\n for joined in future::join_all(handles).await {\n joined.unwrap();\n }\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn nested_spawn_is_local() {\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n LocalSet::new()\n .run_until(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n task::spawn_local(async {\n assert!(ON_RT_THREAD.with(|cell| cell.get()));\n })\n .await\n .unwrap();\n })\n .await\n .unwrap();\n })\n .await\n .unwrap();\n })\n .await\n .unwrap();\n })\n .await;\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[test]\nfn join_local_future_elsewhere() {\n thread_local! {\n static ON_RT_THREAD: Cell = const { Cell::new(false) };\n }\n\n ON_RT_THREAD.with(|cell| cell.set(true));\n\n let rt = runtime::Runtime::new().unwrap();\n let local = LocalSet::new();\n local.block_on(&rt, async move {\n let (tx, rx) = oneshot::channel();\n let join = task::spawn_local(async move {\n assert!(\n ON_RT_THREAD.with(|cell| cell.get()),\n \"local task must run on local thread, no matter where it is awaited\"\n );\n rx.await.unwrap();\n\n \"hello world\"\n });\n let join2 = task::spawn(async move {\n assert!(\n !ON_RT_THREAD.with(|cell| cell.get()),\n \"spawned task should be on a worker\"\n );\n\n tx.send(()).expect(\"task shouldn't have ended yet\");\n\n join.await.expect(\"task should complete successfully\");\n });\n join2.await.unwrap()\n });\n}\n\n// Tests for \n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn localset_in_thread_local() {\n thread_local! {\n static LOCAL_SET: LocalSet = LocalSet::new();\n }\n\n // holds runtime thread until end of main fn.\n let (_tx, rx) = oneshot::channel::<()>();\n let handle = tokio::runtime::Handle::current();\n\n std::thread::spawn(move || {\n LOCAL_SET.with(|local_set| {\n handle.block_on(local_set.run_until(async move {\n let _ = rx.await;\n }))\n });\n });\n}\n\n#[test]\nfn drop_cancels_tasks() {\n use std::rc::Rc;\n\n // This test reproduces issue #1842\n let rt = rt();\n let rc1 = Rc::new(());\n let rc2 = rc1.clone();\n\n let (started_tx, started_rx) = oneshot::channel();\n\n let local = LocalSet::new();\n local.spawn_local(async move {\n // Move this in\n let _rc2 = rc2;\n\n started_tx.send(()).unwrap();\n futures::future::pending::<()>().await;\n });\n\n local.block_on(&rt, async {\n started_rx.await.unwrap();\n });\n drop(local);\n drop(rt);\n\n assert_eq!(1, Rc::strong_count(&rc1));\n}\n\n/// Runs a test function in a separate thread, and panics if the test does not\n/// complete within the specified timeout, or if the test function panics.\n///\n/// This is intended for running tests whose failure mode is a hang or infinite\n/// loop that cannot be detected otherwise.\nfn with_timeout(timeout: Duration, f: impl FnOnce() + Send + 'static) {\n use std::sync::mpsc::RecvTimeoutError;\n\n let (done_tx, done_rx) = std::sync::mpsc::channel();\n let thread = std::thread::spawn(move || {\n f();\n\n // Send a message on the channel so that the test thread can\n // determine if we have entered an infinite loop:\n done_tx.send(()).unwrap();\n });\n\n // Since the failure mode of this test is an infinite loop, rather than\n // something we can easily make assertions about, we'll run it in a\n // thread. When the test thread finishes, it will send a message on a\n // channel to this thread. We'll wait for that message with a fairly\n // generous timeout, and if we don't receive it, we assume the test\n // thread has hung.\n //\n // Note that it should definitely complete in under a minute, but just\n // in case CI is slow, we'll give it a long timeout.\n match done_rx.recv_timeout(timeout) {\n Err(RecvTimeoutError::Timeout) => panic!(\n \"test did not complete within {timeout:?} seconds, \\\n we have (probably) entered an infinite loop!\",\n ),\n // Did the test thread panic? We'll find out for sure when we `join`\n // with it.\n Err(RecvTimeoutError::Disconnected) => {}\n // Test completed successfully!\n Ok(()) => {}\n }\n\n thread.join().expect(\"test thread should not panic!\")\n}\n\n#[cfg_attr(\n target_os = \"wasi\",\n ignore = \"`unwrap()` in `with_timeout()` panics on Wasi\"\n)]\n#[test]\nfn drop_cancels_remote_tasks() {\n // This test reproduces issue #1885.\n with_timeout(Duration::from_secs(60), || {\n let (tx, mut rx) = mpsc::channel::<()>(1024);\n\n let rt = rt();\n\n let local = LocalSet::new();\n local.spawn_local(async move { while rx.recv().await.is_some() {} });\n local.block_on(&rt, async {\n time::sleep(Duration::from_millis(1)).await;\n });\n\n drop(tx);\n\n // This enters an infinite loop if the remote notified tasks are not\n // properly cancelled.\n drop(local);\n });\n}\n\n#[cfg_attr(\n target_os = \"wasi\",\n ignore = \"FIXME: `task::spawn_local().await.unwrap()` panics on Wasi\"\n)]\n#[test]\nfn local_tasks_wake_join_all() {\n // This test reproduces issue #2460.\n with_timeout(Duration::from_secs(60), || {\n use futures::future::join_all;\n use tokio::task::LocalSet;\n\n let rt = rt();\n let set = LocalSet::new();\n let mut handles = Vec::new();\n\n for _ in 1..=128 {\n handles.push(set.spawn_local(async move {\n tokio::task::spawn_local(async move {}).await.unwrap();\n }));\n }\n\n rt.block_on(set.run_until(join_all(handles)));\n });\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support panic recovery\n#[test]\nfn local_tasks_are_polled_after_tick() {\n // This test depends on timing, so we run it up to five times.\n for _ in 0..4 {\n let res = std::panic::catch_unwind(local_tasks_are_polled_after_tick_inner);\n if res.is_ok() {\n // success\n return;\n }\n }\n\n // Test failed 4 times. Try one more time without catching panics. If it\n // fails again, the test fails.\n local_tasks_are_polled_after_tick_inner();\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support panic recovery\n#[tokio::main(flavor = \"current_thread\")]\nasync fn local_tasks_are_polled_after_tick_inner() {\n // Reproduces issues #1899 and #1900\n\n static RX1: AtomicUsize = AtomicUsize::new(0);\n static RX2: AtomicUsize = AtomicUsize::new(0);\n const EXPECTED: usize = 500;\n\n RX1.store(0, SeqCst);\n RX2.store(0, SeqCst);\n\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n let local = LocalSet::new();\n\n local\n .run_until(async {\n let task2 = task::spawn(async move {\n // Wait a bit\n time::sleep(Duration::from_millis(10)).await;\n\n let mut oneshots = Vec::with_capacity(EXPECTED);\n\n // Send values\n for _ in 0..EXPECTED {\n let (oneshot_tx, oneshot_rx) = oneshot::channel();\n oneshots.push(oneshot_tx);\n tx.send(oneshot_rx).unwrap();\n }\n\n time::sleep(Duration::from_millis(10)).await;\n\n for tx in oneshots.drain(..) {\n tx.send(()).unwrap();\n }\n\n loop {\n time::sleep(Duration::from_millis(20)).await;\n let rx1 = RX1.load(SeqCst);\n let rx2 = RX2.load(SeqCst);\n\n if rx1 == EXPECTED && rx2 == EXPECTED {\n break;\n }\n }\n });\n\n while let Some(oneshot) = rx.recv().await {\n RX1.fetch_add(1, SeqCst);\n\n task::spawn_local(async move {\n oneshot.await.unwrap();\n RX2.fetch_add(1, SeqCst);\n });\n }\n\n task2.await.unwrap();\n })\n .await;\n}\n\n#[tokio::test]\nasync fn acquire_mutex_in_drop() {\n use futures::future::pending;\n\n let (tx1, rx1) = oneshot::channel();\n let (tx2, rx2) = oneshot::channel();\n let local = LocalSet::new();\n\n local.spawn_local(async move {\n let _ = rx2.await;\n unreachable!();\n });\n\n local.spawn_local(async move {\n let _ = rx1.await;\n tx2.send(()).unwrap();\n unreachable!();\n });\n\n // Spawn a task that will never notify\n local.spawn_local(async move {\n pending::<()>().await;\n tx1.send(()).unwrap();\n });\n\n // Tick the loop\n local\n .run_until(async {\n task::yield_now().await;\n })\n .await;\n\n // Drop the LocalSet\n drop(local);\n}\n\n#[tokio::test]\nasync fn spawn_wakes_localset() {\n let local = LocalSet::new();\n futures::select! {\n _ = local.run_until(pending::<()>()).fuse() => unreachable!(),\n ret = async { local.spawn_local(ready(())).await.unwrap()}.fuse() => ret\n }\n}\n\n/// Checks that the task wakes up with `enter`.\n/// Reproduces .\n#[tokio::test]\nasync fn sleep_with_local_enter_guard() {\n let local = LocalSet::new();\n let _guard = local.enter();\n\n let (tx, rx) = oneshot::channel();\n\n local\n .run_until(async move {\n tokio::task::spawn_local(async move {\n time::sleep(Duration::ZERO).await;\n\n tx.send(()).expect(\"failed to send\");\n });\n assert_eq!(rx.await, Ok(()));\n })\n .await;\n}\n\n#[test]\nfn store_local_set_in_thread_local_with_runtime() {\n use tokio::runtime::Runtime;\n\n thread_local! {\n static CURRENT: RtAndLocalSet = RtAndLocalSet::new();\n }\n\n struct RtAndLocalSet {\n rt: Runtime,\n local: LocalSet,\n }\n\n impl RtAndLocalSet {\n fn new() -> RtAndLocalSet {\n RtAndLocalSet {\n rt: tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap(),\n local: LocalSet::new(),\n }\n }\n\n async fn inner_method(&self) {\n self.local\n .run_until(async move {\n tokio::task::spawn_local(async {});\n })\n .await\n }\n\n fn method(&self) {\n self.rt.block_on(self.inner_method());\n }\n }\n\n CURRENT.with(|f| {\n f.method();\n });\n}\n\n#[cfg(tokio_unstable)]\nmod unstable {\n use tokio::runtime::UnhandledPanic;\n use tokio::task::LocalSet;\n\n #[tokio::test]\n #[should_panic(\n expected = \"a spawned task panicked and the LocalSet is configured to shutdown on unhandled panic\"\n )]\n async fn shutdown_on_panic() {\n LocalSet::new()\n .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n .run_until(async {\n tokio::task::spawn_local(async {\n panic!(\"boom\");\n });\n\n futures::future::pending::<()>().await;\n })\n .await;\n }\n\n // This test compares that, when the task driving `run_until` has already\n // consumed budget, the `run_until` future has less budget than a \"spawned\"\n // task.\n //\n // \"Budget\" is a fuzzy metric as the Tokio runtime is able to change values\n // internally. This is why the test uses indirection to test this.\n #[tokio::test]\n async fn run_until_does_not_get_own_budget() {\n // Consume some budget\n tokio::task::consume_budget().await;\n\n LocalSet::new()\n .run_until(async {\n let spawned = tokio::spawn(async {\n let mut spawned_n = 0;\n\n {\n let mut spawned = tokio_test::task::spawn(async {\n loop {\n spawned_n += 1;\n tokio::task::consume_budget().await;\n }\n });\n // Poll once\n assert!(!spawned.poll().is_ready());\n }\n\n spawned_n\n });\n\n let mut run_until_n = 0;\n {\n let mut run_until = tokio_test::task::spawn(async {\n loop {\n run_until_n += 1;\n tokio::task::consume_budget().await;\n }\n });\n // Poll once\n assert!(!run_until.poll().is_ready());\n }\n\n let spawned_n = spawned.await.unwrap();\n assert_ne!(spawned_n, 0);\n assert_ne!(run_until_n, 0);\n assert!(spawned_n > run_until_n);\n })\n .await\n }\n}\n\nfn rt() -> runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/task_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))]\n#![cfg(panic = \"unwind\")]\n\nuse futures::future;\nuse std::error::Error;\nuse tokio::runtime::Builder;\nuse tokio::task::{self, block_in_place};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\n#[test]\nfn block_in_place_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = Builder::new_current_thread().enable_all().build().unwrap();\n rt.block_on(async {\n block_in_place(|| {});\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_set_spawn_local_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _local = task::LocalSet::new();\n\n task::spawn_local(async {});\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_set_block_on_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = Builder::new_current_thread().enable_all().build().unwrap();\n let local = task::LocalSet::new();\n\n rt.block_on(async {\n local.block_on(&rt, future::pending::<()>());\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn spawn_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n tokio::spawn(future::pending::<()>());\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_key_sync_scope_panic_caller() -> Result<(), Box> {\n tokio::task_local! {\n static NUMBER: u32;\n }\n\n let panic_location_file = test_panic(|| {\n NUMBER.sync_scope(1, || {\n NUMBER.with(|_| {\n NUMBER.sync_scope(1, || {});\n });\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_key_with_panic_caller() -> Result<(), Box> {\n tokio::task_local! {\n static NUMBER: u32;\n }\n\n let panic_location_file = test_panic(|| {\n NUMBER.with(|_| {});\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_key_get_panic_caller() -> Result<(), Box> {\n tokio::task_local! {\n static NUMBER: u32;\n }\n\n let panic_location_file = test_panic(|| {\n NUMBER.get();\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n" }, { "path": "tokio/tests/task_trace_self.rs", "content": "#![allow(unknown_lints, unexpected_cfgs)]\n#![cfg(all(\n tokio_unstable,\n feature = \"taskdump\",\n target_os = \"linux\",\n any(target_arch = \"aarch64\", target_arch = \"x86\", target_arch = \"x86_64\")\n))]\n\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::{Arc, Mutex};\nuse std::task::{Context, Poll};\nuse std::time::{Duration, Instant};\n\nuse tokio::runtime::dump::{Root, Trace};\n\npin_project_lite::pin_project! {\n pub struct PrettyFuture {\n #[pin]\n f: Root,\n t_last: State,\n logs: Arc>>,\n }\n}\n\nenum State {\n NotStarted,\n Running { since: Instant },\n Alerted,\n}\n\nimpl PrettyFuture {\n pub fn pretty(f: F, logs: Arc>>) -> Self {\n PrettyFuture {\n f: Trace::root(f),\n t_last: State::NotStarted,\n logs,\n }\n }\n}\n\nimpl Future for PrettyFuture {\n type Output = F::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let mut this = self.project();\n let now = Instant::now();\n let t_last = match this.t_last {\n State::Running { since } => Some(*since),\n State::NotStarted => {\n *this.t_last = State::Running { since: now };\n None\n }\n State::Alerted => {\n // don't double-alert for the same future\n None\n }\n };\n if t_last.is_some_and(|t_last| now.duration_since(t_last) > Duration::from_millis(500)) {\n let (res, trace) = tokio::runtime::dump::Trace::capture(|| this.f.as_mut().poll(cx));\n this.logs.lock().unwrap().push(trace);\n *this.t_last = State::Alerted;\n return res;\n }\n this.f.poll(cx)\n }\n}\n\n#[tokio::test]\nasync fn task_trace_self() {\n let log = Arc::new(Mutex::new(vec![]));\n let log2 = Arc::new(Mutex::new(vec![]));\n let mut good_line = vec![];\n let mut bad_line = vec![];\n PrettyFuture::pretty(\n PrettyFuture::pretty(\n async {\n bad_line.push(line!() + 1);\n tokio::task::yield_now().await;\n bad_line.push(line!() + 1);\n tokio::time::sleep(Duration::from_millis(1)).await;\n for _ in 0..100 {\n good_line.push(line!() + 1);\n tokio::time::sleep(Duration::from_millis(10)).await;\n }\n },\n log.clone(),\n ),\n log2.clone(),\n )\n .await;\n for line in good_line {\n let s = format!(\"{}:{}:\", file!(), line);\n assert!(log.lock().unwrap().iter().any(|x| {\n eprintln!(\"{x}\");\n format!(\"{x}\").contains(&s)\n }));\n }\n for line in bad_line {\n let s = format!(\"{}:{}:\", file!(), line);\n assert!(!log\n .lock()\n .unwrap()\n .iter()\n .any(|x| format!(\"{x}\").contains(&s)));\n }\n}\n" }, { "path": "tokio/tests/task_yield_now.rs", "content": "#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), tokio_unstable))]\n\nuse tokio::task;\nuse tokio_test::task::spawn;\n\n// `yield_now` is tested within the runtime in `rt_common`.\n#[test]\nfn yield_now_outside_of_runtime() {\n let mut task = spawn(async {\n task::yield_now().await;\n });\n\n assert!(task.poll().is_pending());\n assert!(task.is_woken());\n assert!(task.poll().is_ready());\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\nasync fn yield_now_external_executor_and_block_in_place() {\n let j = tokio::spawn(async {\n task::block_in_place(|| futures::executor::block_on(task::yield_now()));\n });\n j.await.unwrap();\n}\n" }, { "path": "tokio/tests/tcp_accept.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::sync::{mpsc, oneshot};\nuse tokio_test::assert_ok;\n\nuse std::io;\nuse std::net::{IpAddr, SocketAddr};\n\nmacro_rules! test_accept {\n ($(($ident:ident, $target:expr),)*) => {\n $(\n #[tokio::test]\n async fn $ident() {\n let listener = assert_ok!(TcpListener::bind($target).await);\n let addr = listener.local_addr().unwrap();\n\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n let (socket, _) = assert_ok!(listener.accept().await);\n assert_ok!(tx.send(socket));\n });\n\n let cli = assert_ok!(TcpStream::connect(&addr).await);\n let srv = assert_ok!(rx.await);\n\n assert_eq!(cli.local_addr().unwrap(), srv.peer_addr().unwrap());\n }\n )*\n }\n}\n\ntest_accept! {\n (ip_str, \"127.0.0.1:0\"),\n (host_str, \"localhost:0\"),\n (socket_addr, \"127.0.0.1:0\".parse::().unwrap()),\n (str_port_tuple, (\"127.0.0.1\", 0)),\n (ip_port_tuple, (\"127.0.0.1\".parse::().unwrap(), 0)),\n}\n\nuse std::pin::Pin;\nuse std::sync::{\n atomic::{AtomicUsize, Ordering::SeqCst},\n Arc,\n};\nuse std::task::{Context, Poll};\nuse tokio_stream::{Stream, StreamExt};\n\nstruct TrackPolls<'a> {\n npolls: Arc,\n listener: &'a mut TcpListener,\n}\n\nimpl<'a> Stream for TrackPolls<'a> {\n type Item = io::Result<(TcpStream, SocketAddr)>;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.npolls.fetch_add(1, SeqCst);\n self.listener.poll_accept(cx).map(Some)\n }\n}\n\n#[tokio::test]\nasync fn no_extra_poll() {\n let mut listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = listener.local_addr().unwrap();\n\n let (tx, rx) = oneshot::channel();\n let (accepted_tx, mut accepted_rx) = mpsc::unbounded_channel();\n\n tokio::spawn(async move {\n let mut incoming = TrackPolls {\n npolls: Arc::new(AtomicUsize::new(0)),\n listener: &mut listener,\n };\n assert_ok!(tx.send(Arc::clone(&incoming.npolls)));\n while incoming.next().await.is_some() {\n accepted_tx.send(()).unwrap();\n }\n });\n\n let npolls = assert_ok!(rx.await);\n tokio::task::yield_now().await;\n\n // should have been polled exactly once: the initial poll\n assert_eq!(npolls.load(SeqCst), 1);\n\n let _ = assert_ok!(TcpStream::connect(&addr).await);\n accepted_rx.recv().await.unwrap();\n\n // should have been polled twice more: once to yield Some(), then once to yield Pending\n assert_eq!(npolls.load(SeqCst), 1 + 2);\n}\n\n#[tokio::test]\nasync fn accept_many() {\n use std::future::{poll_fn, Future};\n use std::sync::atomic::AtomicBool;\n\n const N: usize = 50;\n\n let listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let listener = Arc::new(listener);\n let addr = listener.local_addr().unwrap();\n let connected = Arc::new(AtomicBool::new(false));\n\n let (pending_tx, mut pending_rx) = mpsc::unbounded_channel();\n let (notified_tx, mut notified_rx) = mpsc::unbounded_channel();\n\n for _ in 0..N {\n let listener = listener.clone();\n let connected = connected.clone();\n let pending_tx = pending_tx.clone();\n let notified_tx = notified_tx.clone();\n\n tokio::spawn(async move {\n let accept = listener.accept();\n tokio::pin!(accept);\n\n let mut polled = false;\n\n poll_fn(|cx| {\n if !polled {\n polled = true;\n assert!(Pin::new(&mut accept).poll(cx).is_pending());\n pending_tx.send(()).unwrap();\n Poll::Pending\n } else if connected.load(SeqCst) {\n notified_tx.send(()).unwrap();\n Poll::Ready(())\n } else {\n Poll::Pending\n }\n })\n .await;\n\n pending_tx.send(()).unwrap();\n });\n }\n\n // Wait for all tasks to have polled at least once\n for _ in 0..N {\n pending_rx.recv().await.unwrap();\n }\n\n // Establish a TCP connection\n connected.store(true, SeqCst);\n let _sock = TcpStream::connect(addr).await.unwrap();\n\n // Wait for all notifications\n for _ in 0..N {\n notified_rx.recv().await.unwrap();\n }\n}\n" }, { "path": "tokio/tests/tcp_connect.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::sync::oneshot;\nuse tokio_test::assert_ok;\n\nuse futures::join;\n\n#[tokio::test]\nasync fn connect_v4() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n assert!(addr.is_ipv4());\n\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n let (socket, addr) = assert_ok!(srv.accept().await);\n assert_eq!(addr, assert_ok!(socket.peer_addr()));\n assert_ok!(tx.send(socket));\n });\n\n let mine = assert_ok!(TcpStream::connect(&addr).await);\n let theirs = assert_ok!(rx.await);\n\n assert_eq!(\n assert_ok!(mine.local_addr()),\n assert_ok!(theirs.peer_addr())\n );\n assert_eq!(\n assert_ok!(theirs.local_addr()),\n assert_ok!(mine.peer_addr())\n );\n}\n\n#[tokio::test]\nasync fn connect_v6() {\n let srv = assert_ok!(TcpListener::bind(\"[::1]:0\").await);\n let addr = assert_ok!(srv.local_addr());\n assert!(addr.is_ipv6());\n\n let (tx, rx) = oneshot::channel();\n\n tokio::spawn(async move {\n let (socket, addr) = assert_ok!(srv.accept().await);\n assert_eq!(addr, assert_ok!(socket.peer_addr()));\n assert_ok!(tx.send(socket));\n });\n\n let mine = assert_ok!(TcpStream::connect(&addr).await);\n let theirs = assert_ok!(rx.await);\n\n assert_eq!(\n assert_ok!(mine.local_addr()),\n assert_ok!(theirs.peer_addr())\n );\n assert_eq!(\n assert_ok!(theirs.local_addr()),\n assert_ok!(mine.peer_addr())\n );\n}\n\n#[tokio::test]\nasync fn connect_addr_ip_string() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = format!(\"127.0.0.1:{}\", addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(addr).await);\n };\n\n join!(server, client);\n}\n\n#[tokio::test]\nasync fn connect_addr_ip_str_slice() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = format!(\"127.0.0.1:{}\", addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(&addr[..]).await);\n };\n\n join!(server, client);\n}\n\n#[tokio::test]\nasync fn connect_addr_host_string() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = format!(\"localhost:{}\", addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(addr).await);\n };\n\n join!(server, client);\n}\n\n#[tokio::test]\nasync fn connect_addr_ip_port_tuple() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = (addr.ip(), addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(&addr).await);\n };\n\n join!(server, client);\n}\n\n#[tokio::test]\nasync fn connect_addr_ip_str_port_tuple() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = (\"127.0.0.1\", addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(&addr).await);\n };\n\n join!(server, client);\n}\n\n#[tokio::test]\nasync fn connect_addr_host_str_port_tuple() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n let addr = (\"localhost\", addr.port());\n\n let server = async {\n assert_ok!(srv.accept().await);\n };\n\n let client = async {\n assert_ok!(TcpStream::connect(&addr).await);\n };\n\n join!(server, client);\n}\n\n/*\n * TODO: bring this back once TCP exposes HUP again\n *\n#[cfg(target_os = \"linux\")]\nmod linux {\n use tokio::net::{TcpListener, TcpStream};\n use tokio::io::{AsyncReadExt, AsyncWriteExt};\n use tokio_test::assert_ok;\n\n use mio::unix::UnixReady;\n\n use futures_util::future::poll_fn;\n use std::io::Write;\n use std::time::Duration;\n use std::{net, thread};\n\n #[tokio::test]\n fn poll_hup() {\n let addr = assert_ok!(\"127.0.0.1:0\".parse());\n let mut srv = assert_ok!(TcpListener::bind(&addr));\n let addr = assert_ok!(srv.local_addr());\n\n tokio::spawn(async move {\n let (mut client, _) = assert_ok!(srv.accept().await);\n assert_ok!(client.set_zero_linger());\n assert_ok!(client.write_all(b\"hello world\").await);\n\n // TODO: Drop?\n });\n\n /*\n let t = thread::spawn(move || {\n let mut client = assert_ok!(srv.accept()).0;\n client.set_zero_linger().unwrap();\n client.write(b\"hello world\").unwrap();\n thread::sleep(Duration::from_millis(200));\n });\n */\n\n let mut stream = assert_ok!(TcpStream::connect(&addr).await);\n\n // Poll for HUP before reading.\n future::poll_fn(|| stream.poll_read_ready(UnixReady::hup().into()))\n .wait()\n .unwrap();\n\n // Same for write half\n future::poll_fn(|| stream.poll_write_ready())\n .wait()\n .unwrap();\n\n let mut buf = vec![0; 11];\n\n // Read the data\n future::poll_fn(|| stream.poll_read(&mut buf))\n .wait()\n .unwrap();\n\n assert_eq!(b\"hello world\", &buf[..]);\n\n t.join().unwrap();\n }\n}\n*/\n" }, { "path": "tokio/tests/tcp_echo.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No socket on miri.\n\nuse tokio::io::{self, AsyncReadExt, AsyncWriteExt};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::sync::oneshot;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn echo_server() {\n const ITER: usize = 1024;\n\n let (tx, rx) = oneshot::channel();\n\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n\n let msg = \"foo bar baz\";\n tokio::spawn(async move {\n let mut stream = assert_ok!(TcpStream::connect(&addr).await);\n\n for _ in 0..ITER {\n // write\n assert_ok!(stream.write_all(msg.as_bytes()).await);\n\n // read\n let mut buf = [0; 11];\n assert_ok!(stream.read_exact(&mut buf).await);\n assert_eq!(&buf[..], msg.as_bytes());\n }\n\n assert_ok!(tx.send(()));\n });\n\n let (mut stream, _) = assert_ok!(srv.accept().await);\n let (mut rd, mut wr) = stream.split();\n\n let n = assert_ok!(io::copy(&mut rd, &mut wr).await);\n assert_eq!(n, (ITER * msg.len()) as u64);\n\n assert_ok!(rx.await);\n}\n" }, { "path": "tokio/tests/tcp_into_split.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse std::io::{Error, ErrorKind, Result};\nuse std::io::{Read, Write};\nuse std::{net, thread};\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::try_join;\n\n#[tokio::test]\nasync fn split() -> Result<()> {\n const MSG: &[u8] = b\"split\";\n\n let listener = TcpListener::bind(\"127.0.0.1:0\").await?;\n let addr = listener.local_addr()?;\n\n let (stream1, (mut stream2, _)) = try_join! {\n TcpStream::connect(&addr),\n listener.accept(),\n }?;\n let (mut read_half, mut write_half) = stream1.into_split();\n\n let ((), (), ()) = try_join! {\n async {\n let len = stream2.write(MSG).await?;\n assert_eq!(len, MSG.len());\n\n let mut read_buf = vec![0u8; 32];\n let read_len = stream2.read(&mut read_buf).await?;\n assert_eq!(&read_buf[..read_len], MSG);\n Result::Ok(())\n },\n async {\n let len = write_half.write(MSG).await?;\n assert_eq!(len, MSG.len());\n Ok(())\n },\n async {\n let mut read_buf = [0u8; 32];\n let peek_len1 = read_half.peek(&mut read_buf[..]).await?;\n let peek_len2 = read_half.peek(&mut read_buf[..]).await?;\n assert_eq!(peek_len1, peek_len2);\n\n let read_len = read_half.read(&mut read_buf[..]).await?;\n assert_eq!(peek_len1, read_len);\n assert_eq!(&read_buf[..read_len], MSG);\n Ok(())\n },\n }?;\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn reunite() -> Result<()> {\n let listener = net::TcpListener::bind(\"127.0.0.1:0\")?;\n let addr = listener.local_addr()?;\n\n let handle = thread::spawn(move || {\n drop(listener.accept().unwrap());\n drop(listener.accept().unwrap());\n });\n\n let stream1 = TcpStream::connect(&addr).await?;\n let (read1, write1) = stream1.into_split();\n\n let stream2 = TcpStream::connect(&addr).await?;\n let (_, write2) = stream2.into_split();\n\n let read1 = match read1.reunite(write2) {\n Ok(_) => panic!(\"Reunite should not succeed\"),\n Err(err) => err.0,\n };\n\n read1.reunite(write1).expect(\"Reunite should succeed\");\n\n handle.join().unwrap();\n Ok(())\n}\n\n/// Test that dropping the write half actually closes the stream.\n#[tokio::test]\nasync fn drop_write() -> Result<()> {\n const MSG: &[u8] = b\"split\";\n\n let listener = net::TcpListener::bind(\"127.0.0.1:0\")?;\n let addr = listener.local_addr()?;\n\n let handle = thread::spawn(move || {\n let (mut stream, _) = listener.accept().unwrap();\n stream.write_all(MSG).unwrap();\n\n let mut read_buf = [0u8; 32];\n let res = match stream.read(&mut read_buf) {\n Ok(0) => Ok(()),\n Ok(len) => Err(Error::new(\n ErrorKind::Other,\n format!(\"Unexpected read: {len} bytes.\"),\n )),\n Err(err) => Err(err),\n };\n\n drop(stream);\n\n res\n });\n\n let stream = TcpStream::connect(&addr).await?;\n let (mut read_half, write_half) = stream.into_split();\n\n let mut read_buf = [0u8; 32];\n let read_len = read_half.read(&mut read_buf[..]).await?;\n assert_eq!(&read_buf[..read_len], MSG);\n\n // drop it while the read is in progress\n std::thread::spawn(move || {\n thread::sleep(std::time::Duration::from_millis(10));\n drop(write_half);\n });\n\n match read_half.read(&mut read_buf[..]).await {\n Ok(0) => {}\n Ok(len) => panic!(\"Unexpected read: {len} bytes.\"),\n Err(err) => panic!(\"Unexpected error: {err}.\"),\n }\n\n handle.join().unwrap().unwrap();\n Ok(())\n}\n" }, { "path": "tokio/tests/tcp_into_std.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No socket on `miri`.\n\nuse std::io::Read;\nuse std::io::Result;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::TcpListener;\nuse tokio::net::TcpStream;\n\n#[tokio::test]\nasync fn tcp_into_std() -> Result<()> {\n let mut data = [0u8; 12];\n let listener = TcpListener::bind(\"127.0.0.1:0\").await?;\n let addr = listener.local_addr().unwrap().to_string();\n\n let handle = tokio::spawn(async {\n let stream: TcpStream = TcpStream::connect(addr).await.unwrap();\n stream\n });\n\n let (tokio_tcp_stream, _) = listener.accept().await?;\n let mut std_tcp_stream = tokio_tcp_stream.into_std()?;\n std_tcp_stream\n .set_nonblocking(false)\n .expect(\"set_nonblocking call failed\");\n\n let mut client = handle.await.expect(\"The task being joined has panicked\");\n client.write_all(b\"Hello world!\").await?;\n\n std_tcp_stream\n .read_exact(&mut data)\n .expect(\"std TcpStream read failed!\");\n assert_eq!(b\"Hello world!\", &data);\n\n // test back to tokio stream\n std_tcp_stream\n .set_nonblocking(true)\n .expect(\"set_nonblocking call failed\");\n let mut tokio_tcp_stream = TcpStream::from_std(std_tcp_stream)?;\n client.write_all(b\"Hello tokio!\").await?;\n let _size = tokio_tcp_stream.read_exact(&mut data).await?;\n assert_eq!(b\"Hello tokio!\", &data);\n\n Ok(())\n}\n" }, { "path": "tokio/tests/tcp_peek.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse tokio::io::AsyncReadExt;\nuse tokio::net::TcpStream;\n\nuse tokio_test::assert_ok;\n\nuse std::thread;\nuse std::{io::Write, net};\n\n#[tokio::test]\nasync fn peek() {\n let listener = net::TcpListener::bind(\"127.0.0.1:0\").unwrap();\n\n let addr = listener.local_addr().unwrap();\n let t = thread::spawn(move || assert_ok!(listener.accept()).0);\n\n let left = net::TcpStream::connect(addr).unwrap();\n\n left.set_nonblocking(true).unwrap();\n\n let mut right = t.join().unwrap();\n\n right.set_nonblocking(true).unwrap();\n\n let _ = right.write(&[1, 2, 3, 4]).unwrap();\n\n let mut left: TcpStream = left.try_into().unwrap();\n let mut buf = [0u8; 16];\n let n = assert_ok!(left.peek(&mut buf).await);\n assert_eq!([1, 2, 3, 4], buf[..n]);\n\n let n = assert_ok!(left.read(&mut buf).await);\n assert_eq!([1, 2, 3, 4], buf[..n]);\n}\n" }, { "path": "tokio/tests/tcp_shutdown.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse tokio::io::{self, AsyncReadExt, AsyncWriteExt};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::sync::oneshot::channel;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn shutdown() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n\n let handle = tokio::spawn(async move {\n let mut stream = assert_ok!(TcpStream::connect(&addr).await);\n\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n\n let mut buf = [0u8; 1];\n let n = assert_ok!(stream.read(&mut buf).await);\n assert_eq!(n, 0);\n });\n\n let (mut stream, _) = assert_ok!(srv.accept().await);\n let (mut rd, mut wr) = stream.split();\n\n let n = assert_ok!(io::copy(&mut rd, &mut wr).await);\n assert_eq!(n, 0);\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n handle.await.unwrap()\n}\n\n#[tokio::test]\nasync fn shutdown_after_tcp_reset() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n\n let (connected_tx, connected_rx) = channel();\n let (dropped_tx, dropped_rx) = channel();\n\n let handle = tokio::spawn(async move {\n let mut stream = assert_ok!(TcpStream::connect(&addr).await);\n connected_tx.send(()).unwrap();\n\n dropped_rx.await.unwrap();\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n });\n\n let (stream, _) = assert_ok!(srv.accept().await);\n // By setting linger to 0 we will trigger a TCP reset\n stream.set_zero_linger().unwrap();\n connected_rx.await.unwrap();\n\n drop(stream);\n dropped_tx.send(()).unwrap();\n\n handle.await.unwrap();\n}\n\n#[tokio::test]\nasync fn shutdown_multiple_calls() {\n let srv = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(srv.local_addr());\n\n let (connected_tx, connected_rx) = channel();\n\n let handle = tokio::spawn(async move {\n let mut stream = assert_ok!(TcpStream::connect(&addr).await);\n connected_tx.send(()).unwrap();\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n });\n\n let (mut stream, _) = assert_ok!(srv.accept().await);\n connected_rx.await.unwrap();\n\n assert_ok!(AsyncWriteExt::shutdown(&mut stream).await);\n handle.await.unwrap();\n}\n" }, { "path": "tokio/tests/tcp_socket.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse std::time::Duration;\nuse tokio::net::TcpSocket;\nuse tokio_test::assert_ok;\n\n#[tokio::test]\nasync fn basic_usage_v4() {\n // Create server\n let addr = assert_ok!(\"127.0.0.1:0\".parse());\n let srv = assert_ok!(TcpSocket::new_v4());\n assert_ok!(srv.bind(addr));\n\n let srv = assert_ok!(srv.listen(128));\n\n // Create client & connect\n let addr = srv.local_addr().unwrap();\n let cli = assert_ok!(TcpSocket::new_v4());\n let _cli = assert_ok!(cli.connect(addr).await);\n\n // Accept\n let _ = assert_ok!(srv.accept().await);\n}\n\n#[tokio::test]\nasync fn basic_usage_v6() {\n // Create server\n let addr = assert_ok!(\"[::1]:0\".parse());\n let srv = assert_ok!(TcpSocket::new_v6());\n assert_ok!(srv.bind(addr));\n\n let srv = assert_ok!(srv.listen(128));\n\n // Create client & connect\n let addr = srv.local_addr().unwrap();\n let cli = assert_ok!(TcpSocket::new_v6());\n let _cli = assert_ok!(cli.connect(addr).await);\n\n // Accept\n let _ = assert_ok!(srv.accept().await);\n}\n\n#[tokio::test]\nasync fn bind_before_connect() {\n // Create server\n let any_addr = assert_ok!(\"127.0.0.1:0\".parse());\n let srv = assert_ok!(TcpSocket::new_v4());\n assert_ok!(srv.bind(any_addr));\n\n let srv = assert_ok!(srv.listen(128));\n\n // Create client & connect\n let addr = srv.local_addr().unwrap();\n let cli = assert_ok!(TcpSocket::new_v4());\n assert_ok!(cli.bind(any_addr));\n let _cli = assert_ok!(cli.connect(addr).await);\n\n // Accept\n let _ = assert_ok!(srv.accept().await);\n}\n\n#[tokio::test]\nasync fn basic_linger() {\n // Create server\n let addr = assert_ok!(\"127.0.0.1:0\".parse());\n let srv = assert_ok!(TcpSocket::new_v4());\n assert_ok!(srv.bind(addr));\n\n assert!(srv.linger().unwrap().is_none());\n\n srv.set_zero_linger().unwrap();\n assert_eq!(srv.linger().unwrap(), Some(Duration::new(0, 0)));\n}\n\n/// Macro to create a simple test to set and get a socket option.\nmacro_rules! test {\n // Test using the `arg`ument as expected return value.\n ($( #[ $attr: meta ] )* $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n test!($( #[$attr] )* $get_fn, $set_fn($arg), $arg);\n };\n ($( #[ $attr: meta ] )* $get_fn: ident, $set_fn: ident ( $arg: expr ), $expected: expr ) => {\n #[test]\n $( #[$attr] )*\n fn $get_fn() {\n test!(__ new_v4, $get_fn, $set_fn($arg), $expected);\n #[cfg(not(target_os = \"vita\"))]\n test!(__ new_v6, $get_fn, $set_fn($arg), $expected);\n }\n };\n // Only test using a IPv4 socket.\n (IPv4 $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n #[test]\n fn $get_fn() {\n test!(__ new_v4, $get_fn, $set_fn($arg), $arg);\n }\n };\n // Only test using a IPv6 socket.\n (IPv6 $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n #[test]\n fn $get_fn() {\n test!(__ new_v6, $get_fn, $set_fn($arg), $arg);\n }\n };\n\n // Internal to this macro.\n (__ $constructor: ident, $get_fn: ident, $set_fn: ident ( $arg: expr ), $expected: expr ) => {\n let socket = TcpSocket::$constructor().expect(\"failed to create `TcpSocket`\");\n\n let initial = socket.$get_fn().expect(\"failed to get initial value\");\n let arg = $arg;\n assert_ne!(initial, arg, \"initial value and argument are the same\");\n\n socket.$set_fn(arg).expect(\"failed to set option\");\n let got = socket.$get_fn().expect(\"failed to get value\");\n let expected = $expected;\n assert_eq!(got, expected, \"set and get values differ\");\n };\n}\n\nconst SET_BUF_SIZE: u32 = 4096;\n// Linux doubles the buffer size for kernel usage, and exposes that when\n// retrieving the buffer size.\n\n#[cfg(not(any(target_os = \"android\", target_os = \"linux\")))]\nconst GET_BUF_SIZE: u32 = SET_BUF_SIZE;\n\n#[cfg(any(target_os = \"android\", target_os = \"linux\"))]\nconst GET_BUF_SIZE: u32 = 2 * SET_BUF_SIZE;\n\ntest!(keepalive, set_keepalive(true));\n\ntest!(reuseaddr, set_reuseaddr(true));\n\n#[cfg(all(\n unix,\n not(target_os = \"solaris\"),\n not(target_os = \"illumos\"),\n not(target_os = \"cygwin\"),\n))]\ntest!(reuseport, set_reuseport(true));\n\ntest!(\n send_buffer_size,\n set_send_buffer_size(SET_BUF_SIZE),\n GET_BUF_SIZE\n);\n\ntest!(\n recv_buffer_size,\n set_recv_buffer_size(SET_BUF_SIZE),\n GET_BUF_SIZE\n);\n\ntest!(\n #[expect(deprecated, reason = \"set_linger is deprecated\")]\n linger,\n set_linger(Some(Duration::from_secs(10)))\n);\n\ntest!(nodelay, set_nodelay(true));\n\n#[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n))]\ntest!(IPv6 tclass_v6, set_tclass_v6(96));\n\n#[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n)))]\ntest!(IPv4 tos_v4, set_tos_v4(96));\n" }, { "path": "tokio/tests/tcp_split.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi doesn't support bind\n // No `socket` on miri.\n\nuse std::io::Result;\nuse std::io::{Read, Write};\nuse std::{net, thread};\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::net::TcpStream;\n\n#[tokio::test]\nasync fn split() -> Result<()> {\n const MSG: &[u8] = b\"split\";\n\n let listener = net::TcpListener::bind(\"127.0.0.1:0\")?;\n let addr = listener.local_addr()?;\n\n let handle = thread::spawn(move || {\n let (mut stream, _) = listener.accept().unwrap();\n stream.write_all(MSG).unwrap();\n\n let mut read_buf = [0u8; 32];\n let read_len = stream.read(&mut read_buf).unwrap();\n assert_eq!(&read_buf[..read_len], MSG);\n });\n\n let mut stream = TcpStream::connect(&addr).await?;\n let (mut read_half, mut write_half) = stream.split();\n\n let mut read_buf = [0u8; 32];\n let peek_len1 = read_half.peek(&mut read_buf[..]).await?;\n let peek_len2 = read_half.peek(&mut read_buf[..]).await?;\n assert_eq!(peek_len1, peek_len2);\n\n let read_len = read_half.read(&mut read_buf[..]).await?;\n assert_eq!(peek_len1, read_len);\n assert_eq!(&read_buf[..read_len], MSG);\n\n assert_eq!(write_half.write(MSG).await?, MSG.len());\n handle.join().unwrap();\n Ok(())\n}\n" }, { "path": "tokio/tests/tcp_stream.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support bind\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt, Interest};\nuse tokio::net::{TcpListener, TcpStream};\nuse tokio::try_join;\nuse tokio_test::task;\nuse tokio_test::{assert_ok, assert_pending, assert_ready_ok};\n\nuse std::future::poll_fn;\nuse std::io;\nuse std::task::Poll;\nuse std::time::Duration;\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\n#[expect(deprecated)] // set_linger is deprecated\nasync fn set_linger() {\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n\n let stream = TcpStream::connect(listener.local_addr().unwrap())\n .await\n .unwrap();\n\n assert_ok!(stream.set_linger(Some(Duration::from_secs(1))));\n assert_eq!(stream.linger().unwrap().unwrap().as_secs(), 1);\n\n assert_ok!(stream.set_zero_linger());\n assert_eq!(stream.linger().unwrap().unwrap().as_secs(), 0);\n\n assert_ok!(stream.set_linger(None));\n assert!(stream.linger().unwrap().is_none());\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn try_read_write() {\n const DATA: &[u8] = b\"this is some data to write to the socket\";\n\n // Create listener\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Create socket pair\n let client = TcpStream::connect(listener.local_addr().unwrap())\n .await\n .unwrap();\n let (server, _) = listener.accept().await.unwrap();\n let mut written = DATA.to_vec();\n\n // Track the server receiving data\n let mut readable = task::spawn(server.readable());\n assert_pending!(readable.poll());\n\n // Write data.\n client.writable().await.unwrap();\n assert_eq!(DATA.len(), client.try_write(DATA).unwrap());\n\n // The task should be notified\n while !readable.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Fill the write buffer using non-vectored I/O\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write(DATA) {\n Ok(n) => written.extend(&DATA[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end using non-vectored I/O\n let mut read = vec![0; written.len()];\n let mut i = 0;\n\n while i < read.len() {\n server.readable().await.unwrap();\n\n match server.try_read(&mut read[i..]) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n written.clear();\n client.writable().await.unwrap();\n\n // Fill the write buffer using vectored I/O\n let data_bufs: Vec<_> = DATA.chunks(10).map(io::IoSlice::new).collect();\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write_vectored(&data_bufs) {\n Ok(n) => written.extend(&DATA[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end using vectored I/O\n let mut read = vec![0; written.len()];\n let mut i = 0;\n\n while i < read.len() {\n server.readable().await.unwrap();\n\n let mut bufs: Vec<_> = read[i..]\n .chunks_mut(0x10000)\n .map(io::IoSliceMut::new)\n .collect();\n match server.try_read_vectored(&mut bufs) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n // Now, we listen for shutdown\n drop(client);\n\n loop {\n let ready = server.ready(Interest::READABLE).await.unwrap();\n\n if ready.is_read_closed() {\n return;\n } else {\n tokio::task::yield_now().await;\n }\n }\n}\n\n#[test]\nfn buffer_not_included_in_future() {\n use std::mem;\n\n const N: usize = 4096;\n\n let fut = async {\n let stream = TcpStream::connect(\"127.0.0.1:8080\").await.unwrap();\n\n loop {\n stream.readable().await.unwrap();\n\n let mut buf = [0; N];\n let n = stream.try_read(&mut buf[..]).unwrap();\n\n if n == 0 {\n break;\n }\n }\n };\n\n let n = mem::size_of_val(&fut);\n assert!(n < 1000);\n}\n\nmacro_rules! assert_readable_by_polling {\n ($stream:expr) => {\n assert_ok!(poll_fn(|cx| $stream.poll_read_ready(cx)).await);\n };\n}\n\nmacro_rules! assert_not_readable_by_polling {\n ($stream:expr) => {\n poll_fn(|cx| {\n assert_pending!($stream.poll_read_ready(cx));\n Poll::Ready(())\n })\n .await;\n };\n}\n\nmacro_rules! assert_writable_by_polling {\n ($stream:expr) => {\n assert_ok!(poll_fn(|cx| $stream.poll_write_ready(cx)).await);\n };\n}\n\nmacro_rules! assert_not_writable_by_polling {\n ($stream:expr) => {\n poll_fn(|cx| {\n assert_pending!($stream.poll_write_ready(cx));\n Poll::Ready(())\n })\n .await;\n };\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn poll_read_ready() {\n let (mut client, mut server) = create_pair().await;\n\n // Initial state - not readable.\n assert_not_readable_by_polling!(server);\n\n // There is data in the buffer - readable.\n assert_ok!(client.write_all(b\"ping\").await);\n assert_readable_by_polling!(server);\n\n // Readable until calls to `poll_read` return `Poll::Pending`.\n let mut buf = [0u8; 4];\n assert_ok!(server.read_exact(&mut buf).await);\n assert_readable_by_polling!(server);\n read_until_pending(&mut server);\n assert_not_readable_by_polling!(server);\n\n // Detect the client disconnect.\n drop(client);\n assert_readable_by_polling!(server);\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn poll_write_ready() {\n let (mut client, server) = create_pair().await;\n\n // Initial state - writable.\n assert_writable_by_polling!(client);\n\n // No space to write - not writable.\n write_until_pending(&mut client);\n assert_not_writable_by_polling!(client);\n\n // Detect the server disconnect.\n drop(server);\n assert_writable_by_polling!(client);\n}\n\nasync fn create_pair() -> (TcpStream, TcpStream) {\n let listener = assert_ok!(TcpListener::bind(\"127.0.0.1:0\").await);\n let addr = assert_ok!(listener.local_addr());\n let (client, (server, _)) = assert_ok!(try_join!(TcpStream::connect(&addr), listener.accept()));\n (client, server)\n}\n\nfn read_until_pending(stream: &mut TcpStream) -> usize {\n let mut buf = vec![0u8; 1024 * 1024];\n let mut total = 0;\n loop {\n match stream.try_read(&mut buf) {\n Ok(n) => total += n,\n Err(err) => {\n assert_eq!(err.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n total\n}\n\nfn write_until_pending(stream: &mut TcpStream) -> usize {\n let buf = vec![0u8; 1024 * 1024];\n let mut total = 0;\n loop {\n match stream.try_write(&buf) {\n Ok(n) => total += n,\n Err(err) => {\n assert_eq!(err.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n total\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn try_read_buf() {\n const DATA: &[u8] = b\"this is some data to write to the socket\";\n\n // Create listener\n let listener = TcpListener::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Create socket pair\n let client = TcpStream::connect(listener.local_addr().unwrap())\n .await\n .unwrap();\n let (server, _) = listener.accept().await.unwrap();\n let mut written = DATA.to_vec();\n\n // Track the server receiving data\n let mut readable = task::spawn(server.readable());\n assert_pending!(readable.poll());\n\n // Write data.\n client.writable().await.unwrap();\n assert_eq!(DATA.len(), client.try_write(DATA).unwrap());\n\n // The task should be notified\n while !readable.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Fill the write buffer\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write(DATA) {\n Ok(n) => written.extend(&DATA[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end\n let mut read = Vec::with_capacity(written.len());\n let mut i = 0;\n\n while i < read.capacity() {\n server.readable().await.unwrap();\n\n match server.try_read_buf(&mut read) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n // Now, we listen for shutdown\n drop(client);\n\n loop {\n let ready = server.ready(Interest::READABLE).await.unwrap();\n\n if ready.is_read_closed() {\n return;\n } else {\n tokio::task::yield_now().await;\n }\n }\n}\n\n// read_closed is a best effort event, so test only for no false positives.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn read_closed() {\n let (client, mut server) = create_pair().await;\n\n let mut ready_fut = task::spawn(client.ready(Interest::READABLE));\n assert_pending!(ready_fut.poll());\n\n assert_ok!(server.write_all(b\"ping\").await);\n\n let ready_event = assert_ok!(ready_fut.await);\n\n assert!(!ready_event.is_read_closed());\n}\n\n// write_closed is a best effort event, so test only for no false positives.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn write_closed() {\n let (mut client, mut server) = create_pair().await;\n\n // Fill the write buffer.\n let write_size = write_until_pending(&mut client);\n let mut ready_fut = task::spawn(client.ready(Interest::WRITABLE));\n assert_pending!(ready_fut.poll());\n\n // Drain the socket to make client writable.\n let mut read_size = 0;\n while read_size < write_size {\n server.readable().await.unwrap();\n read_size += read_until_pending(&mut server);\n }\n\n let ready_event = assert_ok!(ready_fut.await);\n\n assert!(!ready_event.is_write_closed());\n}\n" }, { "path": "tokio/tests/test_clock.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::time::{self, Duration, Instant};\n\n#[tokio::test]\nasync fn resume_lets_time_move_forward_instead_of_resetting_it() {\n let start = Instant::now();\n time::pause();\n time::advance(Duration::from_secs(10)).await;\n let advanced_by_ten_secs = Instant::now();\n assert!(advanced_by_ten_secs - start > Duration::from_secs(10));\n assert!(advanced_by_ten_secs - start < Duration::from_secs(11));\n time::resume();\n assert!(advanced_by_ten_secs < Instant::now());\n assert!(Instant::now() - advanced_by_ten_secs < Duration::from_secs(1));\n}\n\n#[tokio::test]\nasync fn can_pause_after_resume() {\n let start = Instant::now();\n time::pause();\n time::advance(Duration::from_secs(10)).await;\n time::resume();\n time::pause();\n time::advance(Duration::from_secs(10)).await;\n assert!(Instant::now() - start > Duration::from_secs(20));\n assert!(Instant::now() - start < Duration::from_secs(21));\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn freezing_time_while_frozen_panics() {\n time::pause();\n time::pause();\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn advancing_time_when_time_is_not_frozen_panics() {\n time::advance(Duration::from_secs(1)).await;\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn resuming_time_when_not_frozen_panics() {\n time::pause();\n time::resume();\n time::resume();\n}\n" }, { "path": "tokio/tests/time_alt.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(tokio_unstable, feature = \"time\", feature = \"rt-multi-thread\"))]\n\nuse tokio::runtime::Runtime;\nuse tokio::time::*;\n\nfn rt_combinations() -> Vec {\n let mut rts = vec![];\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n #[cfg(tokio_unstable)]\n {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n }\n\n rts\n}\n\n#[test]\nfn sleep() {\n const N: u32 = 512;\n\n for rt in rt_combinations() {\n rt.block_on(async {\n let mut jhs = vec![];\n\n // sleep outside of the worker threads\n let now = Instant::now();\n tokio::time::sleep(Duration::from_millis(10)).await;\n assert!(now.elapsed() >= Duration::from_millis(10));\n\n for _ in 0..N {\n let jh = tokio::spawn(async move {\n // sleep inside of the worker threads\n let now = Instant::now();\n tokio::time::sleep(Duration::from_millis(10)).await;\n assert!(now.elapsed() >= Duration::from_millis(10));\n });\n jhs.push(jh);\n }\n\n for jh in jhs {\n jh.await.unwrap();\n }\n });\n }\n}\n\n#[test]\nfn timeout() {\n const N: u32 = 512;\n\n for rt in rt_combinations() {\n rt.block_on(async {\n let mut jhs = vec![];\n\n // timeout outside of the worker threads\n let now = Instant::now();\n tokio::time::timeout(Duration::from_millis(10), std::future::pending::<()>())\n .await\n .expect_err(\"timeout should occur\");\n assert!(now.elapsed() >= Duration::from_millis(10));\n\n for _ in 0..N {\n let jh = tokio::spawn(async move {\n let now = Instant::now();\n // timeout inside of the worker threads\n tokio::time::timeout(Duration::from_millis(10), std::future::pending::<()>())\n .await\n .expect_err(\"timeout should occur\");\n assert!(now.elapsed() >= Duration::from_millis(10));\n });\n jhs.push(jh);\n }\n\n for jh in jhs {\n jh.await.unwrap();\n }\n });\n }\n}\n" }, { "path": "tokio/tests/time_interval.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nuse futures::{Stream, StreamExt};\nuse tokio::time::{self, Duration, Instant, Interval, MissedTickBehavior};\nuse tokio_test::{assert_pending, assert_ready, assert_ready_eq, task};\n\n// Takes the `Interval` task, `start` variable, and optional time deltas\n// For each time delta, it polls the `Interval` and asserts that the result is\n// equal to `start` + the specific time delta. Then it asserts that the\n// `Interval` is pending.\nmacro_rules! check_interval_poll {\n ($i:ident, $start:ident, $($delta:expr),*$(,)?) => {\n $(\n assert_ready_eq!(poll_next(&mut $i), $start + ms($delta));\n )*\n assert_pending!(poll_next(&mut $i));\n };\n ($i:ident, $start:ident) => {\n check_interval_poll!($i, $start,);\n };\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn interval_zero_duration() {\n let _ = time::interval_at(Instant::now(), ms(0));\n}\n\n// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n// Actual ticks: | work -----| delay | work | work | work -| work -----|\n// Poll behavior: | | | | | | | |\n// | | | | | | | |\n// Ready(s) | | Ready(s + 2p) | | | |\n// Pending | Ready(s + 3p) | | |\n// Ready(s + p) Ready(s + 4p) | |\n// Ready(s + 5p) |\n// Ready(s + 6p)\n#[tokio::test(start_paused = true)]\nasync fn burst() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(650)).await;\n check_interval_poll!(i, start, 600, 900);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start, 1200);\n\n time::advance(ms(250)).await;\n check_interval_poll!(i, start, 1500);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1800);\n}\n\n// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n// Actual ticks: | work -----| delay | work -----| work -----| work -----|\n// Poll behavior: | | | | | | | |\n// | | | | | | | |\n// Ready(s) | | Ready(s + 2p) | | | |\n// Pending | Pending | | |\n// Ready(s + p) Ready(s + 2p + d) | |\n// Ready(s + 3p + d) |\n// Ready(s + 4p + d)\n#[tokio::test(start_paused = true)]\nasync fn delay() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n i.set_missed_tick_behavior(MissedTickBehavior::Delay);\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(650)).await;\n check_interval_poll!(i, start, 600);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n // We have to add one here for the same reason as is above.\n // Because `Interval` has reset its timer according to `Instant::now()`,\n // we have to go forward 1 more millisecond than is expected so that the\n // runtime realizes that it's time to resolve the timer.\n time::advance(ms(201)).await;\n // We add one because when using the `Delay` behavior, `Interval`\n // adds the `period` from `Instant::now()`, which will always be off by one\n // because we have to advance time by 1 (see above).\n check_interval_poll!(i, start, 1251);\n\n time::advance(ms(300)).await;\n // Again, we add one.\n check_interval_poll!(i, start, 1551);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1851);\n}\n\n// Expected ticks: | 1 | 2 | 3 | 4 | 5 | 6 |\n// Actual ticks: | work -----| delay | work ---| work -----| work -----|\n// Poll behavior: | | | | | | |\n// | | | | | | |\n// Ready(s) | | Ready(s + 2p) | | |\n// Pending | Ready(s + 4p) | |\n// Ready(s + p) Ready(s + 5p) |\n// Ready(s + 6p)\n#[tokio::test(start_paused = true)]\nasync fn skip() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n i.set_missed_tick_behavior(MissedTickBehavior::Skip);\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(650)).await;\n check_interval_poll!(i, start, 600);\n\n time::advance(ms(250)).await;\n check_interval_poll!(i, start, 1200);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1500);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1800);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn reset() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n i.reset();\n\n time::advance(ms(250)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(50)).await;\n // We add one because when using `reset` method, `Interval` adds the\n // `period` from `Instant::now()`, which will always be off by one\n check_interval_poll!(i, start, 701);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1001);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn reset_immediately() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n i.reset_immediately();\n\n // We add one because when using `reset` method, `Interval` adds the\n // `period` from `Instant::now()`, which will always be off by one\n check_interval_poll!(i, start, 401);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 701);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn reset_after() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n i.reset_after(Duration::from_millis(20));\n\n // We add one because when using `reset` method, `Interval` adds the\n // `period` from `Instant::now()`, which will always be off by one\n time::advance(ms(20)).await;\n check_interval_poll!(i, start, 421);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 721);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn reset_at() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n i.reset_at(Instant::now() + Duration::from_millis(40));\n\n // We add one because when using `reset` method, `Interval` adds the\n // `period` from `Instant::now()`, which will always be off by one\n time::advance(ms(40)).await;\n check_interval_poll!(i, start, 441);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 741);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn reset_at_bigger_than_interval() {\n let start = Instant::now();\n\n // This is necessary because the timer is only so granular, and in order for\n // all our ticks to resolve, the time needs to be 1ms ahead of what we\n // expect, so that the runtime will see that it is time to resolve the timer\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n check_interval_poll!(i, start, 0);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n time::advance(ms(200)).await;\n check_interval_poll!(i, start, 300);\n\n time::advance(ms(100)).await;\n check_interval_poll!(i, start);\n\n i.reset_at(Instant::now() + Duration::from_millis(1000));\n\n // Validate the interval does not tick until 1000ms have passed\n time::advance(ms(300)).await;\n check_interval_poll!(i, start);\n time::advance(ms(300)).await;\n check_interval_poll!(i, start);\n time::advance(ms(300)).await;\n check_interval_poll!(i, start);\n\n // We add one because when using `reset` method, `Interval` adds the\n // `period` from `Instant::now()`, which will always be off by one\n time::advance(ms(100)).await;\n check_interval_poll!(i, start, 1401);\n\n time::advance(ms(300)).await;\n check_interval_poll!(i, start, 1701);\n}\n\nfn poll_next(interval: &mut task::Spawn) -> Poll {\n interval.enter(|cx, mut interval| interval.poll_tick(cx))\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n\n/// Helper struct to test the [tokio::time::Interval::poll_tick()] method.\n///\n/// `poll_tick()` should register the waker in the context only if it returns\n/// `Poll::Pending`, not when returning `Poll::Ready`. This struct contains an\n/// interval timer and counts up on every tick when used as stream. When the\n/// counter is a multiple of four, it yields the current counter value.\n/// Depending on the value for `wake_on_pending`, it will reschedule itself when\n/// it returns `Poll::Pending` or not. When used with `wake_on_pending=false`,\n/// we expect that the stream stalls because the timer will **not** reschedule\n/// the next wake-up itself once it returned `Poll::Ready`.\nstruct IntervalStreamer {\n counter: u32,\n timer: Interval,\n wake_on_pending: bool,\n}\n\nimpl Stream for IntervalStreamer {\n type Item = u32;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let this = Pin::into_inner(self);\n\n if this.counter > 12 {\n return Poll::Ready(None);\n }\n\n match this.timer.poll_tick(cx) {\n Poll::Pending => Poll::Pending,\n Poll::Ready(_) => {\n this.counter += 1;\n if this.counter % 4 == 0 {\n Poll::Ready(Some(this.counter))\n } else {\n if this.wake_on_pending {\n // Schedule this task for wake-up\n cx.waker().wake_by_ref();\n }\n Poll::Pending\n }\n }\n }\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn stream_with_interval_poll_tick_self_waking() {\n let stream = IntervalStreamer {\n counter: 0,\n timer: tokio::time::interval(tokio::time::Duration::from_millis(10)),\n wake_on_pending: true,\n };\n\n let (res_tx, mut res_rx) = tokio::sync::mpsc::channel(12);\n\n // Wrap task in timeout so that it will finish eventually even if the stream\n // stalls.\n tokio::spawn(tokio::time::timeout(\n tokio::time::Duration::from_millis(150),\n async move {\n tokio::pin!(stream);\n\n while let Some(item) = stream.next().await {\n res_tx.send(item).await.ok();\n }\n },\n ));\n\n let mut items = Vec::with_capacity(3);\n while let Some(result) = res_rx.recv().await {\n items.push(result);\n }\n\n // We expect the stream to yield normally and thus three items.\n assert_eq!(items, vec![4, 8, 12]);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn stream_with_interval_poll_tick_no_waking() {\n let stream = IntervalStreamer {\n counter: 0,\n timer: tokio::time::interval(tokio::time::Duration::from_millis(10)),\n wake_on_pending: false,\n };\n\n let (res_tx, mut res_rx) = tokio::sync::mpsc::channel(12);\n\n // Wrap task in timeout so that it will finish eventually even if the stream\n // stalls.\n tokio::spawn(tokio::time::timeout(\n tokio::time::Duration::from_millis(150),\n async move {\n tokio::pin!(stream);\n\n while let Some(item) = stream.next().await {\n res_tx.send(item).await.ok();\n }\n },\n ));\n\n let mut items = Vec::with_capacity(0);\n while let Some(result) = res_rx.recv().await {\n items.push(result);\n }\n\n // We expect the stream to stall because it does not reschedule itself on\n // `Poll::Pending` and neither does [tokio::time::Interval] reschedule the\n // task when returning `Poll::Ready`.\n assert_eq!(items, vec![]);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn interval_doesnt_panic_max_duration_when_polling() {\n let mut timer = task::spawn(time::interval(Duration::MAX));\n assert_ready!(timer.enter(|cx, mut timer| timer.poll_tick(cx)));\n}\n" }, { "path": "tokio/tests/time_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support panic recovery\n#![cfg(panic = \"unwind\")]\n\nuse futures::future;\nuse std::error::Error;\nuse std::time::Duration;\nuse tokio::runtime::{Builder, Runtime};\nuse tokio::time::{self, interval, interval_at, timeout, Instant};\n\nmod support {\n pub mod panic;\n}\nuse support::panic::test_panic;\n\nfn rt_combinations() -> Vec {\n let mut rts = vec![];\n\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n #[cfg(tokio_unstable)]\n {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n }\n\n rts\n}\n\n#[test]\nfn pause_panic_caller() -> Result<(), Box> {\n for rt in rt_combinations() {\n let panic_location_file = test_panic(|| {\n rt.block_on(async {\n time::pause();\n time::pause();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n }\n\n Ok(())\n}\n\n#[test]\nfn resume_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = current_thread();\n\n rt.block_on(async {\n time::resume();\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn interval_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = interval(Duration::from_millis(0));\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn interval_at_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = interval_at(Instant::now(), Duration::from_millis(0));\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn timeout_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n // Runtime without `enable_time` so it has no current timer set.\n let rt = Builder::new_current_thread().build().unwrap();\n rt.block_on(async {\n let _timeout = timeout(Duration::from_millis(5), future::pending::<()>());\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\nfn current_thread() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio/tests/time_pause.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(miri))] // Too slow on miri.\n\nuse rand::SeedableRng;\nuse rand::{rngs::StdRng, Rng};\nuse tokio::time::{self, Duration, Instant, Sleep};\nuse tokio_test::{assert_elapsed, assert_pending, assert_ready, assert_ready_eq, task};\n\n#[cfg(not(target_os = \"wasi\"))]\nuse tokio_test::assert_err;\n\nuse std::{\n future::Future,\n pin::Pin,\n task::{Context, Poll},\n};\n\n#[tokio::test]\nasync fn pause_time_in_main() {\n tokio::time::pause();\n}\n\n#[tokio::test]\nasync fn pause_time_in_task() {\n let t = tokio::spawn(async {\n tokio::time::pause();\n });\n\n t.await.unwrap();\n}\n\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 1)]\n#[should_panic]\nasync fn pause_time_in_main_threads() {\n tokio::time::pause();\n}\n\n#[cfg_attr(panic = \"abort\", ignore)]\n#[cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\n#[tokio::test(flavor = \"multi_thread\", worker_threads = 1)]\nasync fn pause_time_in_spawn_threads() {\n let t = tokio::spawn(async {\n tokio::time::pause();\n });\n\n assert_err!(t.await);\n}\n\n#[test]\nfn paused_time_is_deterministic() {\n let run_1 = paused_time_stress_run();\n let run_2 = paused_time_stress_run();\n\n assert_eq!(run_1, run_2);\n}\n\n#[tokio::main(flavor = \"current_thread\", start_paused = true)]\nasync fn paused_time_stress_run() -> Vec {\n let mut rng = StdRng::seed_from_u64(1);\n\n let mut times = vec![];\n let start = Instant::now();\n for _ in 0..10_000 {\n let sleep = rng.random_range(Duration::from_secs(0)..Duration::from_secs(1));\n time::sleep(sleep).await;\n times.push(start.elapsed());\n }\n\n times\n}\n\n#[tokio::test(start_paused = true)]\nasync fn advance_after_poll() {\n time::sleep(ms(1)).await;\n\n let start = Instant::now();\n\n let mut sleep = task::spawn(time::sleep_until(start + ms(300)));\n\n assert_pending!(sleep.poll());\n\n let before = Instant::now();\n time::advance(ms(100)).await;\n assert_elapsed!(before, ms(100));\n\n assert_pending!(sleep.poll());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn sleep_no_poll() {\n let start = Instant::now();\n\n // TODO: Skip this\n time::advance(ms(1)).await;\n\n let mut sleep = task::spawn(time::sleep_until(start + ms(300)));\n\n let before = Instant::now();\n time::advance(ms(100)).await;\n assert_elapsed!(before, ms(100));\n\n assert_pending!(sleep.poll());\n}\n\nenum State {\n Begin,\n AwaitingAdvance(Pin>>),\n AfterAdvance,\n}\n\nstruct Tester {\n sleep: Pin>,\n state: State,\n before: Option,\n poll: bool,\n}\n\nimpl Future for Tester {\n type Output = ();\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n match &mut self.state {\n State::Begin => {\n if self.poll {\n assert_pending!(self.sleep.as_mut().poll(cx));\n }\n self.before = Some(Instant::now());\n let advance_fut = Box::pin(time::advance(ms(100)));\n self.state = State::AwaitingAdvance(advance_fut);\n self.poll(cx)\n }\n State::AwaitingAdvance(ref mut advance_fut) => match advance_fut.as_mut().poll(cx) {\n Poll::Pending => Poll::Pending,\n Poll::Ready(()) => {\n self.state = State::AfterAdvance;\n self.poll(cx)\n }\n },\n State::AfterAdvance => {\n assert_elapsed!(self.before.unwrap(), ms(100));\n\n assert_pending!(self.sleep.as_mut().poll(cx));\n\n Poll::Ready(())\n }\n }\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn sleep_same_task() {\n let start = Instant::now();\n\n // TODO: Skip this\n time::advance(ms(1)).await;\n\n let sleep = Box::pin(time::sleep_until(start + ms(300)));\n\n Tester {\n sleep,\n state: State::Begin,\n before: None,\n poll: true,\n }\n .await;\n}\n\n#[tokio::test(start_paused = true)]\nasync fn sleep_same_task_no_poll() {\n let start = Instant::now();\n\n // TODO: Skip this\n time::advance(ms(1)).await;\n\n let sleep = Box::pin(time::sleep_until(start + ms(300)));\n\n Tester {\n sleep,\n state: State::Begin,\n before: None,\n poll: false,\n }\n .await;\n}\n\n#[tokio::test(start_paused = true)]\nasync fn interval() {\n let start = Instant::now();\n\n // TODO: Skip this\n time::advance(ms(1)).await;\n\n let mut i = task::spawn(time::interval_at(start, ms(300)));\n\n assert_ready_eq!(poll_next(&mut i), start);\n assert_pending!(poll_next(&mut i));\n\n let before = Instant::now();\n time::advance(ms(100)).await;\n assert_elapsed!(before, ms(100));\n assert_pending!(poll_next(&mut i));\n\n let before = Instant::now();\n time::advance(ms(200)).await;\n assert_elapsed!(before, ms(200));\n assert_ready_eq!(poll_next(&mut i), start + ms(300));\n assert_pending!(poll_next(&mut i));\n\n let before = Instant::now();\n time::advance(ms(400)).await;\n assert_elapsed!(before, ms(400));\n assert_ready_eq!(poll_next(&mut i), start + ms(600));\n assert_pending!(poll_next(&mut i));\n\n let before = Instant::now();\n time::advance(ms(500)).await;\n assert_elapsed!(before, ms(500));\n assert_ready_eq!(poll_next(&mut i), start + ms(900));\n assert_ready_eq!(poll_next(&mut i), start + ms(1200));\n assert_pending!(poll_next(&mut i));\n}\n\n#[tokio::test(start_paused = true)]\nasync fn test_time_advance_sub_ms() {\n let now = Instant::now();\n\n let dur = Duration::from_micros(51_592);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n\n let now = Instant::now();\n let dur = Duration::from_micros(1);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn test_time_advance_3ms_and_change() {\n let now = Instant::now();\n\n let dur = Duration::from_micros(3_141_592);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n\n let now = Instant::now();\n let dur = Duration::from_micros(3_123_456);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn regression_3710_with_submillis_advance() {\n let start = Instant::now();\n\n time::advance(Duration::from_millis(1)).await;\n\n let mut sleep = task::spawn(time::sleep_until(start + Duration::from_secs(60)));\n\n assert_pending!(sleep.poll());\n\n let before = Instant::now();\n let dur = Duration::from_micros(51_592);\n time::advance(dur).await;\n assert_eq!(before.elapsed(), dur);\n\n assert_pending!(sleep.poll());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn exact_1ms_advance() {\n let now = Instant::now();\n\n let dur = Duration::from_millis(1);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n\n let now = Instant::now();\n let dur = Duration::from_millis(1);\n time::advance(dur).await;\n\n assert_eq!(now.elapsed(), dur);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn advance_once_with_timer() {\n let mut sleep = task::spawn(time::sleep(Duration::from_millis(1)));\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(250)).await;\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(1500)).await;\n\n assert!(sleep.is_woken());\n assert_ready!(sleep.poll());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn advance_multi_with_timer() {\n // Round to the nearest ms\n // time::sleep(Duration::from_millis(1)).await;\n\n let mut sleep = task::spawn(time::sleep(Duration::from_millis(1)));\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(250)).await;\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(250)).await;\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(250)).await;\n assert_pending!(sleep.poll());\n\n time::advance(Duration::from_micros(250)).await;\n assert!(sleep.is_woken());\n assert_ready!(sleep.poll());\n}\n\nfn poll_next(interval: &mut task::Spawn) -> Poll {\n interval.enter(|cx, mut interval| interval.poll_tick(cx))\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n" }, { "path": "tokio/tests/time_rt.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::runtime::Runtime;\nuse tokio::time::*;\n\nuse std::sync::mpsc;\n\nfn rt_combinations() -> Vec {\n let mut rts = vec![];\n\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n #[cfg(tokio_unstable)]\n {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(1)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n\n let rt = tokio::runtime::Builder::new_multi_thread()\n .worker_threads(4)\n .enable_alt_timer()\n .enable_all()\n .build()\n .unwrap();\n rts.push(rt);\n }\n\n rts\n}\n\n#[cfg(all(feature = \"rt-multi-thread\", not(target_os = \"wasi\")))] // Wasi doesn't support threads\n#[test]\nfn timer_with_threaded_runtime() {\n use tokio::runtime::Runtime;\n\n {\n let rt = Runtime::new().unwrap();\n let (tx, rx) = mpsc::channel();\n\n rt.spawn(async move {\n let when = Instant::now() + Duration::from_millis(10);\n\n sleep_until(when).await;\n assert!(Instant::now() >= when);\n\n tx.send(()).unwrap();\n });\n\n rx.recv().unwrap();\n }\n\n #[cfg(tokio_unstable)]\n {\n let rt = tokio::runtime::Builder::new_multi_thread()\n .enable_alt_timer()\n .build()\n .unwrap();\n let (tx, rx) = mpsc::channel();\n\n rt.block_on(async move {\n let when = Instant::now() + Duration::from_millis(10);\n\n sleep_until(when).await;\n assert!(Instant::now() >= when);\n\n tx.send(()).unwrap();\n });\n\n rx.recv().unwrap();\n }\n}\n\n#[test]\nfn timer_with_current_thread_scheduler() {\n use tokio::runtime::Builder;\n\n let rt = Builder::new_current_thread().enable_all().build().unwrap();\n let (tx, rx) = mpsc::channel();\n\n rt.block_on(async move {\n let when = Instant::now() + Duration::from_millis(10);\n\n sleep_until(when).await;\n assert!(Instant::now() >= when);\n\n tx.send(()).unwrap();\n });\n\n rx.recv().unwrap();\n}\n\n#[test]\nfn starving() {\n use std::future::Future;\n use std::pin::Pin;\n use std::task::{Context, Poll};\n\n struct Starve + Unpin>(T, u64);\n\n impl + Unpin> Future for Starve {\n type Output = u64;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n if Pin::new(&mut self.0).poll(cx).is_ready() {\n return Poll::Ready(self.1);\n }\n\n self.1 += 1;\n\n cx.waker().wake_by_ref();\n\n Poll::Pending\n }\n }\n\n for rt in rt_combinations() {\n rt.block_on(async {\n let when = Instant::now() + Duration::from_millis(10);\n let starve = Starve(Box::pin(sleep_until(when)), 0);\n\n starve.await;\n assert!(Instant::now() >= when);\n });\n }\n}\n\n#[test]\nfn timeout_value() {\n use tokio::sync::oneshot;\n\n for rt in rt_combinations() {\n rt.block_on(async {\n let (_tx, rx) = oneshot::channel::<()>();\n\n let now = Instant::now();\n let dur = Duration::from_millis(10);\n\n let res = timeout(dur, rx).await;\n assert!(res.is_err());\n assert!(Instant::now() >= now + dur);\n });\n }\n}\n" }, { "path": "tokio/tests/time_sleep.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(not(miri))] // Too slow on Miri.\n\nuse std::future::Future;\nuse std::task::Context;\n\nuse futures::task::noop_waker_ref;\n\nuse tokio::time::{self, Duration, Instant};\nuse tokio_test::{assert_elapsed, assert_pending, assert_ready, task};\n\n#[tokio::test]\nasync fn immediate_sleep() {\n time::pause();\n\n let now = Instant::now();\n\n // Ready!\n time::sleep_until(now).await;\n assert_elapsed!(now, ms(1));\n}\n\n#[tokio::test]\nasync fn is_elapsed() {\n time::pause();\n\n let sleep = time::sleep(Duration::from_millis(10));\n\n tokio::pin!(sleep);\n\n assert!(!sleep.is_elapsed());\n\n assert!(futures::poll!(sleep.as_mut()).is_pending());\n\n assert!(!sleep.is_elapsed());\n\n sleep.as_mut().await;\n\n assert!(sleep.is_elapsed());\n}\n\n#[tokio::test]\nasync fn delayed_sleep_level_0() {\n time::pause();\n\n for &i in &[1, 10, 60] {\n let now = Instant::now();\n let dur = ms(i);\n\n time::sleep_until(now + dur).await;\n\n assert_elapsed!(now, dur);\n }\n}\n\n#[tokio::test]\nasync fn sub_ms_delayed_sleep() {\n time::pause();\n\n for _ in 0..5 {\n let now = Instant::now();\n let deadline = now + ms(1) + Duration::new(0, 1);\n\n time::sleep_until(deadline).await;\n\n assert_elapsed!(now, ms(1));\n }\n}\n\n#[tokio::test]\nasync fn delayed_sleep_wrapping_level_0() {\n time::pause();\n\n time::sleep(ms(5)).await;\n\n let now = Instant::now();\n time::sleep_until(now + ms(60)).await;\n\n assert_elapsed!(now, ms(60));\n}\n\n#[tokio::test]\nasync fn reset_future_sleep_before_fire() {\n time::pause();\n\n let now = Instant::now();\n\n let mut sleep = task::spawn(Box::pin(time::sleep_until(now + ms(100))));\n assert_pending!(sleep.poll());\n\n let mut sleep = sleep.into_inner();\n\n sleep.as_mut().reset(Instant::now() + ms(200));\n sleep.await;\n\n assert_elapsed!(now, ms(200));\n}\n\n#[tokio::test]\nasync fn reset_past_sleep_before_turn() {\n time::pause();\n\n let now = Instant::now();\n\n let mut sleep = task::spawn(Box::pin(time::sleep_until(now + ms(100))));\n assert_pending!(sleep.poll());\n\n let mut sleep = sleep.into_inner();\n\n sleep.as_mut().reset(now + ms(80));\n sleep.await;\n\n assert_elapsed!(now, ms(80));\n}\n\n#[tokio::test]\nasync fn reset_past_sleep_before_fire() {\n time::pause();\n\n let now = Instant::now();\n\n let mut sleep = task::spawn(Box::pin(time::sleep_until(now + ms(100))));\n assert_pending!(sleep.poll());\n\n let mut sleep = sleep.into_inner();\n\n time::sleep(ms(10)).await;\n\n sleep.as_mut().reset(now + ms(80));\n sleep.await;\n\n assert_elapsed!(now, ms(80));\n}\n\n#[tokio::test]\nasync fn reset_future_sleep_after_fire() {\n time::pause();\n\n let now = Instant::now();\n let mut sleep = Box::pin(time::sleep_until(now + ms(100)));\n\n sleep.as_mut().await;\n assert_elapsed!(now, ms(100));\n\n sleep.as_mut().reset(now + ms(110));\n sleep.await;\n assert_elapsed!(now, ms(110));\n}\n\n#[tokio::test]\nasync fn reset_sleep_to_past() {\n time::pause();\n\n let now = Instant::now();\n\n let mut sleep = task::spawn(Box::pin(time::sleep_until(now + ms(100))));\n assert_pending!(sleep.poll());\n\n time::sleep(ms(50)).await;\n\n assert!(!sleep.is_woken());\n\n sleep.as_mut().reset(now + ms(40));\n\n // TODO: is this required?\n //assert!(sleep.is_woken());\n\n assert_ready!(sleep.poll());\n}\n\n#[cfg(not(target_os = \"wasi\"))] // Wasi doesn't support panic recovery\n#[test]\n#[should_panic]\nfn creating_sleep_outside_of_context() {\n let now = Instant::now();\n\n // This creates a delay outside of the context of a mock timer. This tests\n // that it will panic.\n let _fut = time::sleep_until(now + ms(500));\n}\n\n#[tokio::test]\nasync fn greater_than_max() {\n const YR_5: u64 = 5 * 365 * 24 * 60 * 60 * 1000;\n\n time::pause();\n time::sleep_until(Instant::now() + ms(YR_5)).await;\n}\n\n#[tokio::test]\nasync fn short_sleeps() {\n for _ in 0..1000 {\n tokio::time::sleep(std::time::Duration::from_millis(0)).await;\n }\n}\n\n#[tokio::test]\nasync fn multi_long_sleeps() {\n tokio::time::pause();\n\n for _ in 0..5u32 {\n tokio::time::sleep(Duration::from_secs(\n // about a year\n 365 * 24 * 3600,\n ))\n .await;\n }\n\n let deadline = tokio::time::Instant::now()\n + Duration::from_secs(\n // about 10 years\n 10 * 365 * 24 * 3600,\n );\n\n tokio::time::sleep_until(deadline).await;\n\n assert!(tokio::time::Instant::now() >= deadline);\n}\n\n#[tokio::test]\nasync fn long_sleeps() {\n tokio::time::pause();\n\n let deadline = tokio::time::Instant::now()\n + Duration::from_secs(\n // about 10 years\n 10 * 365 * 24 * 3600,\n );\n\n tokio::time::sleep_until(deadline).await;\n\n assert!(tokio::time::Instant::now() >= deadline);\n assert!(tokio::time::Instant::now() <= deadline + Duration::from_millis(1));\n}\n\n#[tokio::test]\nasync fn reset_after_firing() {\n let timer = tokio::time::sleep(std::time::Duration::from_millis(1));\n tokio::pin!(timer);\n\n let deadline = timer.deadline();\n\n timer.as_mut().await;\n assert_ready!(timer\n .as_mut()\n .poll(&mut Context::from_waker(noop_waker_ref())));\n timer\n .as_mut()\n .reset(tokio::time::Instant::now() + std::time::Duration::from_secs(600));\n\n assert_ne!(deadline, timer.deadline());\n\n assert_pending!(timer\n .as_mut()\n .poll(&mut Context::from_waker(noop_waker_ref())));\n assert_pending!(timer\n .as_mut()\n .poll(&mut Context::from_waker(noop_waker_ref())));\n}\n\n#[tokio::test]\nasync fn exactly_max() {\n time::pause();\n time::sleep(Duration::MAX).await;\n}\n\n#[tokio::test]\nasync fn issue_5183() {\n time::pause();\n\n let big = std::time::Duration::from_secs(u64::MAX / 10);\n // This is a workaround since awaiting sleep(big) will never finish.\n #[rustfmt::skip]\n tokio::select! {\n\tbiased;\n _ = tokio::time::sleep(big) => {}\n _ = tokio::time::sleep(std::time::Duration::from_nanos(1)) => {}\n }\n}\n\n#[tokio::test]\nasync fn no_out_of_bounds_close_to_max() {\n time::pause();\n time::sleep(Duration::MAX - Duration::from_millis(1)).await;\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n\n#[tokio::test]\nasync fn drop_after_reschedule_at_new_scheduled_time() {\n use futures::poll;\n\n tokio::time::pause();\n\n let start = tokio::time::Instant::now();\n\n let mut a = Box::pin(tokio::time::sleep(Duration::from_millis(5)));\n let mut b = Box::pin(tokio::time::sleep(Duration::from_millis(5)));\n let mut c = Box::pin(tokio::time::sleep(Duration::from_millis(10)));\n\n let _ = poll!(&mut a);\n let _ = poll!(&mut b);\n let _ = poll!(&mut c);\n\n b.as_mut().reset(start + Duration::from_millis(10));\n a.await;\n\n drop(b);\n}\n\n#[tokio::test]\nasync fn drop_from_wake() {\n use std::future::Future;\n use std::pin::Pin;\n use std::sync::atomic::{AtomicBool, Ordering};\n use std::sync::{Arc, Mutex};\n use std::task::Context;\n\n let panicked = Arc::new(AtomicBool::new(false));\n let list: Arc>>>> = Arc::new(Mutex::new(Vec::new()));\n\n let arc_wake = Arc::new(DropWaker(panicked.clone(), list.clone()));\n let arc_wake = futures::task::waker(arc_wake);\n\n tokio::time::pause();\n\n {\n let mut lock = list.lock().unwrap();\n\n for _ in 0..100 {\n let mut timer = Box::pin(tokio::time::sleep(Duration::from_millis(10)));\n\n let _ = timer.as_mut().poll(&mut Context::from_waker(&arc_wake));\n\n lock.push(timer);\n }\n }\n\n tokio::time::sleep(Duration::from_millis(11)).await;\n\n assert!(\n !panicked.load(Ordering::SeqCst),\n \"panicked when dropping timers\"\n );\n\n #[derive(Clone)]\n struct DropWaker(\n Arc,\n Arc>>>>,\n );\n\n impl futures::task::ArcWake for DropWaker {\n fn wake_by_ref(arc_self: &Arc) {\n let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n *arc_self.1.lock().expect(\"panic in lock\") = Vec::new()\n }));\n\n if result.is_err() {\n arc_self.0.store(true, Ordering::SeqCst);\n }\n }\n }\n}\n" }, { "path": "tokio/tests/time_timeout.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::sync::oneshot;\nuse tokio::time::{self, timeout, timeout_at, Instant};\nuse tokio_test::*;\n\nuse futures::future::pending;\nuse std::time::Duration;\n\n#[tokio::test]\nasync fn simultaneous_deadline_future_completion() {\n // Create a future that is immediately ready\n let mut fut = task::spawn(timeout_at(Instant::now(), async {}));\n\n // Ready!\n assert_ready_ok!(fut.poll());\n}\n\n#[cfg_attr(target_os = \"wasi\", ignore = \"FIXME: `fut.poll()` panics on Wasi\")]\n#[tokio::test]\nasync fn completed_future_past_deadline() {\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout_at(Instant::now() - ms(1000), async {}));\n\n // Ready!\n assert_ready_ok!(fut.poll());\n}\n\n#[tokio::test]\nasync fn future_and_deadline_in_future() {\n time::pause();\n\n // Not yet complete\n let (tx, rx) = oneshot::channel();\n\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout_at(Instant::now() + ms(100), rx));\n\n assert_pending!(fut.poll());\n\n // Turn the timer, it runs for the elapsed time\n time::advance(ms(90)).await;\n\n assert_pending!(fut.poll());\n\n // Complete the future\n tx.send(()).unwrap();\n assert!(fut.is_woken());\n\n assert_ready_ok!(fut.poll()).unwrap();\n}\n\n#[tokio::test]\nasync fn future_and_timeout_in_future() {\n time::pause();\n\n // Not yet complete\n let (tx, rx) = oneshot::channel();\n\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout(ms(100), rx));\n\n // Ready!\n assert_pending!(fut.poll());\n\n // Turn the timer, it runs for the elapsed time\n time::advance(ms(90)).await;\n\n assert_pending!(fut.poll());\n\n // Complete the future\n tx.send(()).unwrap();\n\n assert_ready_ok!(fut.poll()).unwrap();\n}\n\n#[tokio::test]\nasync fn very_large_timeout() {\n time::pause();\n\n // Not yet complete\n let (tx, rx) = oneshot::channel();\n\n // copy-paste unstable `Duration::MAX`\n let duration_max = Duration::from_secs(u64::MAX) + Duration::from_nanos(999_999_999);\n\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout(duration_max, rx));\n\n // Ready!\n assert_pending!(fut.poll());\n\n // Turn the timer, it runs for the elapsed time\n time::advance(Duration::from_secs(86400 * 365 * 10)).await;\n\n assert_pending!(fut.poll());\n\n // Complete the future\n tx.send(()).unwrap();\n\n assert_ready_ok!(fut.poll()).unwrap();\n}\n\n#[tokio::test]\nasync fn deadline_now_elapses() {\n use futures::future::pending;\n\n time::pause();\n\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout_at(Instant::now(), pending::<()>()));\n\n // Factor in jitter\n // TODO: don't require this\n time::advance(ms(1)).await;\n\n assert_ready_err!(fut.poll());\n}\n\n#[tokio::test]\nasync fn deadline_future_elapses() {\n time::pause();\n\n // Wrap it with a deadline\n let mut fut = task::spawn(timeout_at(Instant::now() + ms(300), pending::<()>()));\n\n assert_pending!(fut.poll());\n\n time::advance(ms(301)).await;\n\n assert!(fut.is_woken());\n assert_ready_err!(fut.poll());\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n\n#[tokio::test]\nasync fn timeout_is_not_exhausted_by_future() {\n let fut = timeout(ms(1), async {\n let mut buffer = [0u8; 1];\n loop {\n use tokio::io::AsyncReadExt;\n let _ = tokio::io::empty().read(&mut buffer).await;\n }\n });\n\n assert!(fut.await.is_err());\n}\n" }, { "path": "tokio/tests/time_wasm.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(target_arch = \"wasm32\", not(target_os = \"wasi\")))]\n\nuse wasm_bindgen_test::wasm_bindgen_test;\n\n#[wasm_bindgen_test]\n#[should_panic]\nfn instant_now_panics() {\n let _ = tokio::time::Instant::now();\n}\n\n#[cfg(all(feature = \"rt\", not(feature = \"time\")))]\n#[wasm_bindgen_test]\nfn runtime_without_time_does_not_panic() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.block_on(async {});\n}\n\n#[cfg(all(feature = \"rt\", feature = \"time\"))]\n#[wasm_bindgen_test]\n#[should_panic] // should remove this once time is supported\nfn runtime_with_time_does_not_panic() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap();\n rt.block_on(async {});\n}\n\n#[cfg(all(feature = \"rt\", feature = \"time\"))]\n#[wasm_bindgen_test]\n#[should_panic]\nfn sleep_panics_on_unknown_unknown() {\n let rt = tokio::runtime::Builder::new_current_thread()\n .enable_time()\n .build()\n .unwrap();\n rt.block_on(async { tokio::time::sleep(core::time::Duration::from_millis(1)).await });\n}\n" }, { "path": "tokio/tests/tracing_sync.rs", "content": "//! Tests for sync instrumentation.\n//!\n//! These tests ensure that the instrumentation for tokio\n//! synchronization primitives is correct.\n#![warn(rust_2018_idioms)]\n#![cfg(all(tokio_unstable, feature = \"tracing\", target_has_atomic = \"64\"))]\n\nuse tokio::sync;\nuse tracing_mock::{expect, subscriber};\n\n#[tokio::test]\nasync fn test_barrier_creates_span() {\n let barrier_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::barrier\");\n\n let size_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"size\").with_value(&1_u64));\n\n let arrived_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"arrived\").with_value(&0_i64));\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(\n barrier_span\n .clone()\n .with_ancestry(expect::is_explicit_root()),\n )\n .enter(&barrier_span)\n .event(size_event)\n .event(arrived_event)\n .exit(&barrier_span)\n .drop_span(&barrier_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n let _ = sync::Barrier::new(1);\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn test_mutex_creates_span() {\n let mutex_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::mutex\");\n\n let locked_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"locked\").with_value(&false));\n\n let batch_semaphore_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::batch_semaphore\");\n\n let batch_semaphore_permits_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"permits\").with_value(&1u64))\n .with_fields(expect::field(\"permits.op\").with_value(&\"override\"));\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(mutex_span.clone().with_ancestry(expect::is_explicit_root()))\n .enter(&mutex_span)\n .event(locked_event)\n .new_span(\n batch_semaphore_span\n .clone()\n .with_ancestry(expect::is_explicit_root()),\n )\n .enter(&batch_semaphore_span)\n .event(batch_semaphore_permits_event)\n .exit(&batch_semaphore_span)\n .exit(&mutex_span)\n .drop_span(&mutex_span)\n .drop_span(&batch_semaphore_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n let _ = sync::Mutex::new(true);\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn test_oneshot_creates_span() {\n let oneshot_span_id = expect::id();\n let oneshot_span = expect::span()\n .with_id(oneshot_span_id.clone())\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::oneshot\");\n\n let initial_tx_dropped_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"tx_dropped\").with_value(&false))\n .with_fields(expect::field(\"tx_dropped.op\").with_value(&\"override\"));\n\n let final_tx_dropped_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"tx_dropped\").with_value(&true))\n .with_fields(expect::field(\"tx_dropped.op\").with_value(&\"override\"));\n\n let initial_rx_dropped_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"rx_dropped\").with_value(&false))\n .with_fields(expect::field(\"rx_dropped.op\").with_value(&\"override\"));\n\n let final_rx_dropped_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"rx_dropped\").with_value(&true))\n .with_fields(expect::field(\"rx_dropped.op\").with_value(&\"override\"));\n\n let value_sent_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"value_sent\").with_value(&false))\n .with_fields(expect::field(\"value_sent.op\").with_value(&\"override\"));\n\n let value_received_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"value_received\").with_value(&false))\n .with_fields(expect::field(\"value_received.op\").with_value(&\"override\"));\n\n let async_op_span_id = expect::id();\n let async_op_span = expect::span()\n .with_id(async_op_span_id.clone())\n .named(\"runtime.resource.async_op\")\n .with_target(\"tokio::sync::oneshot\");\n\n let async_op_poll_span = expect::span()\n .named(\"runtime.resource.async_op.poll\")\n .with_target(\"tokio::sync::oneshot\");\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(\n oneshot_span\n .clone()\n .with_ancestry(expect::is_explicit_root()),\n )\n .enter(&oneshot_span)\n .event(initial_tx_dropped_event)\n .exit(&oneshot_span)\n .enter(&oneshot_span)\n .event(initial_rx_dropped_event)\n .exit(&oneshot_span)\n .enter(&oneshot_span)\n .event(value_sent_event)\n .exit(&oneshot_span)\n .enter(&oneshot_span)\n .event(value_received_event)\n .exit(&oneshot_span)\n .enter(&oneshot_span)\n .new_span(\n async_op_span\n .clone()\n .with_ancestry(expect::has_contextual_parent(&oneshot_span_id)),\n )\n .exit(&oneshot_span)\n .enter(&async_op_span)\n .new_span(\n async_op_poll_span\n .clone()\n .with_ancestry(expect::has_contextual_parent(&async_op_span_id)),\n )\n .exit(&async_op_span)\n .enter(&oneshot_span)\n .event(final_tx_dropped_event)\n .exit(&oneshot_span)\n .enter(&oneshot_span)\n .event(final_rx_dropped_event)\n .exit(&oneshot_span)\n .drop_span(oneshot_span)\n .drop_span(async_op_span)\n .drop_span(&async_op_poll_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n let _ = sync::oneshot::channel::();\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn test_rwlock_creates_span() {\n let rwlock_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::rwlock\");\n\n let max_readers_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"max_readers\").with_value(&0x1FFFFFFF_u64));\n\n let write_locked_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"write_locked\").with_value(&false));\n\n let current_readers_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"current_readers\").with_value(&0_i64));\n\n let batch_semaphore_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::batch_semaphore\");\n\n let batch_semaphore_permits_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"permits\").with_value(&1u64))\n .with_fields(expect::field(\"permits.op\").with_value(&\"override\"));\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(\n rwlock_span\n .clone()\n .with_ancestry(expect::is_explicit_root()),\n )\n .enter(rwlock_span.clone())\n .event(max_readers_event)\n .event(write_locked_event)\n .event(current_readers_event)\n .exit(rwlock_span.clone())\n .enter(rwlock_span.clone())\n .new_span(batch_semaphore_span.clone())\n .enter(batch_semaphore_span.clone())\n .event(batch_semaphore_permits_event)\n .exit(batch_semaphore_span.clone())\n .exit(rwlock_span.clone())\n .drop_span(rwlock_span)\n .drop_span(batch_semaphore_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n let _ = sync::RwLock::new(true);\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn test_semaphore_creates_span() {\n let semaphore_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::semaphore\");\n\n let batch_semaphore_span = expect::span()\n .named(\"runtime.resource\")\n .with_target(\"tokio::sync::batch_semaphore\");\n\n let batch_semaphore_permits_event = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(expect::field(\"permits\").with_value(&1u64))\n .with_fields(expect::field(\"permits.op\").with_value(&\"override\"));\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(\n semaphore_span\n .clone()\n .with_ancestry(expect::is_explicit_root()),\n )\n .enter(semaphore_span.clone())\n .new_span(batch_semaphore_span.clone())\n .enter(batch_semaphore_span.clone())\n .event(batch_semaphore_permits_event)\n .exit(batch_semaphore_span.clone())\n .exit(semaphore_span.clone())\n .drop_span(semaphore_span)\n .drop_span(batch_semaphore_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n let _ = sync::Semaphore::new(1);\n }\n\n handle.assert_finished();\n}\n" }, { "path": "tokio/tests/tracing_task.rs", "content": "//! Tests for task instrumentation.\n//!\n//! These tests ensure that the instrumentation for task spawning and task\n//! lifecycles is correct.\n#![warn(rust_2018_idioms)]\n#![cfg(all(tokio_unstable, feature = \"tracing\", target_has_atomic = \"64\"))]\n\nuse std::{mem, time::Duration};\n\nuse tokio::task;\nuse tracing_mock::{expect, span::NewSpan, subscriber};\n\n#[tokio::test]\nasync fn task_spawn_creates_span() {\n let task_span = expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\");\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(&task_span)\n .enter(&task_span)\n .exit(&task_span)\n // The task span is entered once more when it gets dropped\n .enter(&task_span)\n .exit(&task_span)\n .drop_span(task_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n tokio::spawn(futures::future::ready(()))\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn task_spawn_loc_file_recorded() {\n let task_span = expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\")\n .with_fields(expect::field(\"loc.file\").with_value(&file!()));\n\n let (subscriber, handle) = subscriber::mock().new_span(task_span).run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n\n tokio::spawn(futures::future::ready(()))\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn task_builder_name_recorded() {\n let task_span = expect_task_named(\"test-task\");\n\n let (subscriber, handle) = subscriber::mock().new_span(task_span).run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n task::Builder::new()\n .name(\"test-task\")\n .spawn(futures::future::ready(()))\n .unwrap()\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn task_builder_loc_file_recorded() {\n let task_span = expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\")\n .with_fields(expect::field(\"loc.file\").with_value(&file!()));\n\n let (subscriber, handle) = subscriber::mock().new_span(task_span).run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n\n task::Builder::new()\n .spawn(futures::future::ready(()))\n .unwrap()\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn task_spawn_sizes_recorded() {\n let future = futures::future::ready(());\n let size = mem::size_of_val(&future) as u64;\n\n let task_span = expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\")\n // TODO(hds): check that original_size.bytes is NOT recorded when this can be done in\n // tracing-mock without listing every other field.\n .with_fields(expect::field(\"size.bytes\").with_value(&size));\n\n let (subscriber, handle) = subscriber::mock().new_span(task_span).run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n\n task::Builder::new()\n .spawn(future)\n .unwrap()\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n#[tokio::test]\nasync fn task_big_spawn_sizes_recorded() {\n let future = {\n async fn big() {\n let mut a = [0_u8; N];\n for (idx, item) in a.iter_mut().enumerate() {\n *item = (idx % 256) as u8;\n }\n tokio::time::sleep(Duration::from_millis(10)).await;\n for (idx, item) in a.iter_mut().enumerate() {\n assert_eq!(*item, (idx % 256) as u8);\n }\n }\n\n // This is larger than the release auto-boxing threshold\n big::<20_000>()\n };\n\n fn boxed_size(_: &T) -> usize {\n mem::size_of::>()\n }\n let size = mem::size_of_val(&future) as u64;\n let boxed_size = boxed_size(&future);\n\n let task_span = expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\")\n .with_fields(\n expect::field(\"size.bytes\")\n .with_value(&boxed_size)\n .and(expect::field(\"original_size.bytes\").with_value(&size)),\n );\n\n let (subscriber, handle) = subscriber::mock().new_span(task_span).run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n\n task::Builder::new()\n .spawn(future)\n .unwrap()\n .await\n .expect(\"failed to await join handle\");\n }\n\n handle.assert_finished();\n}\n\n/// Expect a task with name\n///\n/// This is a convenience function to create the expectation for a new task\n/// with the `task.name` field set to the provided name.\nfn expect_task_named(name: &str) -> NewSpan {\n expect::span()\n .named(\"runtime.spawn\")\n .with_target(\"tokio::task\")\n .with_fields(\n expect::field(\"task.name\").with_value(&tracing::field::debug(format_args!(\"{name}\"))),\n )\n}\n" }, { "path": "tokio/tests/tracing_time.rs", "content": "//! Tests for time resource instrumentation.\n//!\n//! These tests ensure that the instrumentation for tokio\n//! synchronization primitives is correct.\n#![warn(rust_2018_idioms)]\n#![cfg(all(tokio_unstable, feature = \"tracing\", target_has_atomic = \"64\"))]\n\nuse std::time::Duration;\n\nuse tracing_mock::{expect, subscriber};\n\n#[tokio::test]\nasync fn test_sleep_creates_span() {\n let sleep_span_id = expect::id();\n let sleep_span = expect::span()\n .with_id(sleep_span_id.clone())\n .named(\"runtime.resource\")\n .with_target(\"tokio::time::sleep\");\n\n let state_update = expect::event()\n .with_target(\"runtime::resource::state_update\")\n .with_fields(\n expect::field(\"duration\")\n // FIXME(hds): This value isn't stable and doesn't seem to make sense. We're not\n // going to test on it until the resource instrumentation has been\n // refactored and we know that we're getting a stable value here.\n //.with_value(&(7_u64 + 1))\n .and(expect::field(\"duration.op\").with_value(&\"override\")),\n );\n\n let async_op_span_id = expect::id();\n let async_op_span = expect::span()\n .with_id(async_op_span_id.clone())\n .named(\"runtime.resource.async_op\")\n .with_target(\"tokio::time::sleep\");\n\n let async_op_poll_span = expect::span()\n .named(\"runtime.resource.async_op.poll\")\n .with_target(\"tokio::time::sleep\");\n\n let (subscriber, handle) = subscriber::mock()\n .new_span(sleep_span.clone().with_ancestry(expect::is_explicit_root()))\n .enter(sleep_span.clone())\n .event(state_update)\n .new_span(\n async_op_span\n .clone()\n .with_ancestry(expect::has_contextual_parent(&sleep_span_id))\n .with_fields(expect::field(\"source\").with_value(&\"Sleep::new_timeout\")),\n )\n .exit(sleep_span.clone())\n .enter(async_op_span.clone())\n .new_span(\n async_op_poll_span\n .clone()\n .with_ancestry(expect::has_contextual_parent(&async_op_span_id)),\n )\n .exit(async_op_span.clone())\n .drop_span(async_op_span)\n .drop_span(async_op_poll_span)\n .drop_span(sleep_span)\n .run_with_handle();\n\n {\n let _guard = tracing::subscriber::set_default(subscriber);\n\n _ = tokio::time::sleep(Duration::from_millis(7));\n }\n\n handle.assert_finished();\n}\n" }, { "path": "tokio/tests/udp.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\"), not(miri)))] // Wasi does not support bind or UDP\n // No `socket` on miri.\n\nuse std::future::poll_fn;\nuse std::io;\nuse std::sync::Arc;\nuse tokio::{io::ReadBuf, net::UdpSocket};\nuse tokio_test::assert_ok;\n\nconst MSG: &[u8] = b\"hello\";\nconst MSG_LEN: usize = MSG.len();\n\n#[tokio::test]\nasync fn send_recv() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n sender.connect(receiver.local_addr()?).await?;\n receiver.connect(sender.local_addr()?).await?;\n\n sender.send(MSG).await?;\n\n let mut recv_buf = [0u8; 32];\n let len = receiver.recv(&mut recv_buf[..]).await?;\n\n assert_eq!(&recv_buf[..len], MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_recv_poll() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n sender.connect(receiver.local_addr()?).await?;\n receiver.connect(sender.local_addr()?).await?;\n\n poll_fn(|cx| sender.poll_send(cx, MSG)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n poll_fn(|cx| receiver.poll_recv(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_to_recv_from() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let receiver_addr = receiver.local_addr()?;\n sender.send_to(MSG, &receiver_addr).await?;\n\n let mut recv_buf = [0u8; 32];\n let (len, addr) = receiver.recv_from(&mut recv_buf[..]).await?;\n\n assert_eq!(&recv_buf[..len], MSG);\n assert_eq!(addr, sender.local_addr()?);\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_to_recv_from_poll() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let receiver_addr = receiver.local_addr()?;\n poll_fn(|cx| sender.poll_send_to(cx, MSG, receiver_addr)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n let addr = poll_fn(|cx| receiver.poll_recv_from(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), MSG);\n assert_eq!(addr, sender.local_addr()?);\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_to_peek_from() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let receiver_addr = receiver.local_addr()?;\n poll_fn(|cx| sender.poll_send_to(cx, MSG, receiver_addr)).await?;\n\n // peek\n let mut recv_buf = [0u8; 32];\n let (n, addr) = receiver.peek_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n // peek\n let mut recv_buf = [0u8; 32];\n let (n, addr) = receiver.peek_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n let mut recv_buf = [0u8; 32];\n let (n, addr) = receiver.recv_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_to_try_peek_from() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let receiver_addr = receiver.local_addr()?;\n poll_fn(|cx| sender.poll_send_to(cx, MSG, receiver_addr)).await?;\n\n // peek\n let mut recv_buf = [0u8; 32];\n\n loop {\n match receiver.try_peek_from(&mut recv_buf) {\n Ok((n, addr)) => {\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n break;\n }\n Err(e) if e.kind() == io::ErrorKind::WouldBlock => {\n receiver.readable().await?;\n }\n Err(e) => return Err(e),\n }\n }\n\n // peek\n let mut recv_buf = [0u8; 32];\n let (n, addr) = receiver.peek_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n let mut recv_buf = [0u8; 32];\n let (n, addr) = receiver.recv_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..n], MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn send_to_peek_from_poll() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let receiver_addr = receiver.local_addr()?;\n poll_fn(|cx| sender.poll_send_to(cx, MSG, receiver_addr)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n let addr = poll_fn(|cx| receiver.poll_peek_from(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), MSG);\n assert_eq!(addr, sender.local_addr()?);\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n poll_fn(|cx| receiver.poll_peek_from(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), MSG);\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n\n poll_fn(|cx| receiver.poll_recv_from(cx, &mut read)).await?;\n assert_eq!(read.filled(), MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn peek_sender() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let sender_addr = sender.local_addr()?;\n let receiver_addr = receiver.local_addr()?;\n\n let msg = b\"Hello, world!\";\n sender.send_to(msg, receiver_addr).await?;\n\n let peeked_sender = receiver.peek_sender().await?;\n assert_eq!(peeked_sender, sender_addr);\n\n // Assert that `peek_sender()` returns the right sender but\n // doesn't remove from the receive queue.\n let mut recv_buf = [0u8; 32];\n let (read, received_sender) = receiver.recv_from(&mut recv_buf).await?;\n\n assert_eq!(&recv_buf[..read], msg);\n assert_eq!(received_sender, peeked_sender);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn poll_peek_sender() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let sender_addr = sender.local_addr()?;\n let receiver_addr = receiver.local_addr()?;\n\n let msg = b\"Hello, world!\";\n poll_fn(|cx| sender.poll_send_to(cx, msg, receiver_addr)).await?;\n\n let peeked_sender = poll_fn(|cx| receiver.poll_peek_sender(cx)).await?;\n assert_eq!(peeked_sender, sender_addr);\n\n // Assert that `poll_peek_sender()` returns the right sender but\n // doesn't remove from the receive queue.\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n let received_sender = poll_fn(|cx| receiver.poll_recv_from(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), msg);\n assert_eq!(received_sender, peeked_sender);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn try_peek_sender() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let sender_addr = sender.local_addr()?;\n let receiver_addr = receiver.local_addr()?;\n\n let msg = b\"Hello, world!\";\n sender.send_to(msg, receiver_addr).await?;\n\n let peeked_sender = loop {\n match receiver.try_peek_sender() {\n Ok(peeked_sender) => break peeked_sender,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n receiver.readable().await?;\n }\n Err(e) => return Err(e),\n }\n };\n\n assert_eq!(peeked_sender, sender_addr);\n\n // Assert that `try_peek_sender()` returns the right sender but\n // didn't remove from the receive queue.\n let mut recv_buf = [0u8; 32];\n // We already peeked the sender so there must be data in the receive queue.\n let (read, received_sender) = receiver.try_recv_from(&mut recv_buf).unwrap();\n\n assert_eq!(&recv_buf[..read], msg);\n assert_eq!(received_sender, peeked_sender);\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn split() -> std::io::Result<()> {\n let socket = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let s = Arc::new(socket);\n let r = s.clone();\n\n let addr = s.local_addr()?;\n tokio::spawn(async move {\n s.send_to(MSG, &addr).await.unwrap();\n });\n let mut recv_buf = [0u8; 32];\n let (len, _) = r.recv_from(&mut recv_buf[..]).await?;\n assert_eq!(&recv_buf[..len], MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn split_chan() -> std::io::Result<()> {\n // setup UdpSocket that will echo all sent items\n let socket = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let addr = socket.local_addr().unwrap();\n let s = Arc::new(socket);\n let r = s.clone();\n\n let (tx, mut rx) = tokio::sync::mpsc::channel::<(Vec, std::net::SocketAddr)>(1_000);\n tokio::spawn(async move {\n while let Some((bytes, addr)) = rx.recv().await {\n s.send_to(&bytes, &addr).await.unwrap();\n }\n });\n\n tokio::spawn(async move {\n let mut buf = [0u8; 32];\n loop {\n let (len, addr) = r.recv_from(&mut buf).await.unwrap();\n tx.send((buf[..len].to_vec(), addr)).await.unwrap();\n }\n });\n\n // test that we can send a value and get back some response\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n sender.send_to(MSG, addr).await?;\n let mut recv_buf = [0u8; 32];\n let (len, _) = sender.recv_from(&mut recv_buf).await?;\n assert_eq!(&recv_buf[..len], MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn split_chan_poll() -> std::io::Result<()> {\n // setup UdpSocket that will echo all sent items\n let socket = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let addr = socket.local_addr().unwrap();\n let s = Arc::new(socket);\n let r = s.clone();\n\n let (tx, mut rx) = tokio::sync::mpsc::channel::<(Vec, std::net::SocketAddr)>(1_000);\n tokio::spawn(async move {\n while let Some((bytes, addr)) = rx.recv().await {\n poll_fn(|cx| s.poll_send_to(cx, &bytes, addr))\n .await\n .unwrap();\n }\n });\n\n tokio::spawn(async move {\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n loop {\n let addr = poll_fn(|cx| r.poll_recv_from(cx, &mut read)).await.unwrap();\n tx.send((read.filled().to_vec(), addr)).await.unwrap();\n }\n });\n\n // test that we can send a value and get back some response\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n poll_fn(|cx| sender.poll_send_to(cx, MSG, addr)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n let _ = poll_fn(|cx| sender.poll_recv_from(cx, &mut read)).await?;\n assert_eq!(read.filled(), MSG);\n Ok(())\n}\n\n// # Note\n//\n// This test is purposely written such that each time `sender` sends data on\n// the socket, `receiver` awaits the data. On Unix, it would be okay waiting\n// until the end of the test to receive all the data. On Windows, this would\n// **not** be okay because it's resources are completion based (via IOCP).\n// If data is sent and not yet received, attempting to send more data will\n// result in `ErrorKind::WouldBlock` until the first operation completes.\n#[tokio::test]\nasync fn try_send_spawn() {\n const MSG2: &[u8] = b\"world!\";\n const MSG2_LEN: usize = MSG2.len();\n\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n receiver\n .connect(sender.local_addr().unwrap())\n .await\n .unwrap();\n\n sender.writable().await.unwrap();\n\n let sent = &sender\n .try_send_to(MSG, receiver.local_addr().unwrap())\n .unwrap();\n assert_eq!(sent, &MSG_LEN);\n let mut buf = [0u8; 32];\n let mut received = receiver.recv(&mut buf[..]).await.unwrap();\n\n sender\n .connect(receiver.local_addr().unwrap())\n .await\n .unwrap();\n let sent = &sender.try_send(MSG2).unwrap();\n assert_eq!(sent, &MSG2_LEN);\n received += receiver.recv(&mut buf[..]).await.unwrap();\n\n std::thread::spawn(move || {\n let sent = &sender.try_send(MSG).unwrap();\n assert_eq!(sent, &MSG_LEN);\n })\n .join()\n .unwrap();\n received += receiver.recv(&mut buf[..]).await.unwrap();\n\n assert_eq!(received, MSG_LEN * 2 + MSG2_LEN);\n}\n\n#[tokio::test]\nasync fn try_send_recv() {\n // Create listener\n let server = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Create socket pair\n let client = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Connect the two\n client.connect(server.local_addr().unwrap()).await.unwrap();\n server.connect(client.local_addr().unwrap()).await.unwrap();\n\n for _ in 0..5 {\n loop {\n client.writable().await.unwrap();\n\n match client.try_send(b\"hello world\") {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await.unwrap();\n\n let mut buf = [0; 512];\n\n match server.try_recv(&mut buf) {\n Ok(n) => {\n assert_eq!(n, 11);\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn try_send_to_recv_from() {\n // Create listener\n let server = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let saddr = server.local_addr().unwrap();\n\n // Create socket pair\n let client = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let caddr = client.local_addr().unwrap();\n\n for _ in 0..5 {\n loop {\n client.writable().await.unwrap();\n\n match client.try_send_to(b\"hello world\", saddr) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await.unwrap();\n\n let mut buf = [0; 512];\n\n match server.try_recv_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr, caddr);\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn try_recv_buf() {\n // Create listener\n let server = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Create socket pair\n let client = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n\n // Connect the two\n client.connect(server.local_addr().unwrap()).await.unwrap();\n server.connect(client.local_addr().unwrap()).await.unwrap();\n\n for _ in 0..5 {\n loop {\n client.writable().await.unwrap();\n\n match client.try_send(b\"hello world\") {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await.unwrap();\n\n let mut buf = Vec::with_capacity(512);\n\n match server.try_recv_buf(&mut buf) {\n Ok(n) => {\n assert_eq!(n, 11);\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn recv_buf() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n sender.connect(receiver.local_addr()?).await?;\n receiver.connect(sender.local_addr()?).await?;\n\n sender.send(MSG).await?;\n let mut recv_buf = Vec::with_capacity(32);\n let len = receiver.recv_buf(&mut recv_buf).await?;\n\n assert_eq!(len, MSG_LEN);\n assert_eq!(&recv_buf[..len], MSG);\n Ok(())\n}\n\n#[tokio::test]\nasync fn try_recv_buf_from() {\n // Create listener\n let server = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let saddr = server.local_addr().unwrap();\n\n // Create socket pair\n let client = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let caddr = client.local_addr().unwrap();\n\n for _ in 0..5 {\n loop {\n client.writable().await.unwrap();\n\n match client.try_send_to(b\"hello world\", saddr) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await.unwrap();\n\n let mut buf = Vec::with_capacity(512);\n\n match server.try_recv_buf_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr, caddr);\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn recv_buf_from() -> std::io::Result<()> {\n let sender = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let receiver = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n sender.connect(receiver.local_addr()?).await?;\n\n sender.send(MSG).await?;\n let mut recv_buf = Vec::with_capacity(32);\n let (len, caddr) = receiver.recv_buf_from(&mut recv_buf).await?;\n\n assert_eq!(len, MSG_LEN);\n assert_eq!(&recv_buf[..len], MSG);\n assert_eq!(caddr, sender.local_addr()?);\n Ok(())\n}\n\n#[tokio::test]\nasync fn poll_ready() {\n // Create listener\n let server = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let saddr = server.local_addr().unwrap();\n\n // Create socket pair\n let client = UdpSocket::bind(\"127.0.0.1:0\").await.unwrap();\n let caddr = client.local_addr().unwrap();\n\n for _ in 0..5 {\n loop {\n assert_ok!(poll_fn(|cx| client.poll_send_ready(cx)).await);\n\n match client.try_send_to(b\"hello world\", saddr) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n assert_ok!(poll_fn(|cx| server.poll_recv_ready(cx)).await);\n\n let mut buf = Vec::with_capacity(512);\n\n match server.try_recv_buf_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr, caddr);\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n}\n\n/// Macro to create a simple test to set and get a socket option.\nmacro_rules! test {\n // Test using the `arg`ument as expected return value.\n ($( #[ $attr: meta ] )* $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n test!($( #[$attr] )* $get_fn, $set_fn($arg), $arg);\n };\n ($( #[ $attr: meta ] )* $get_fn: ident, $set_fn: ident ( $arg: expr ), $expected: expr ) => {\n #[tokio::test]\n $( #[$attr] )*\n async fn $get_fn() {\n test!(__ \"127.0.0.1:0\", $get_fn, $set_fn($arg), $expected);\n #[cfg(not(target_os = \"vita\"))]\n test!(__ \"[::1]:0\", $get_fn, $set_fn($arg), $expected);\n }\n };\n // Only test using a IPv4 socket.\n (IPv4 $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n #[tokio::test]\n async fn $get_fn() {\n test!(__ \"127.0.0.1:0\", $get_fn, $set_fn($arg), $arg);\n }\n };\n // Only test using a IPv6 socket.\n (IPv6 $get_fn: ident, $set_fn: ident ( $arg: expr ) ) => {\n #[tokio::test]\n async fn $get_fn() {\n test!(__ \"[::1]:0\", $get_fn, $set_fn($arg), $arg);\n }\n };\n\n // Internal to this macro.\n (__ $addr: literal, $get_fn: ident, $set_fn: ident ( $arg: expr ), $expected: expr ) => {\n let socket = UdpSocket::bind($addr).await.expect(\"failed to create `UdpSocket`\");\n\n let initial = socket.$get_fn().expect(\"failed to get initial value\");\n let arg = $arg;\n assert_ne!(initial, arg, \"initial value and argument are the same\");\n\n socket.$set_fn(arg).expect(\"failed to set option\");\n let got = socket.$get_fn().expect(\"failed to get value\");\n let expected = $expected;\n assert_eq!(got, expected, \"set and get values differ\");\n };\n}\n\ntest!(broadcast, set_broadcast(true));\n\ntest!(IPv4 multicast_loop_v4, set_multicast_loop_v4(false));\n\n#[cfg(target_os = \"linux\")] // broken on non-Linux platforms https://github.com/rust-lang/socket2/pull/630\ntest!(multicast_ttl_v4, set_multicast_ttl_v4(40));\n\ntest!(IPv6 multicast_loop_v6, set_multicast_loop_v6(false));\n\n#[cfg(any(\n target_os = \"android\",\n target_os = \"dragonfly\",\n target_os = \"freebsd\",\n target_os = \"fuchsia\",\n target_os = \"linux\",\n target_os = \"macos\",\n target_os = \"netbsd\",\n target_os = \"openbsd\",\n target_os = \"cygwin\",\n))]\ntest!(IPv6 tclass_v6, set_tclass_v6(96));\n\ntest!(IPv4 ttl, set_ttl(40));\n\n#[cfg(not(any(\n target_os = \"fuchsia\",\n target_os = \"redox\",\n target_os = \"solaris\",\n target_os = \"illumos\",\n target_os = \"haiku\"\n)))]\ntest!(IPv4 tos_v4, set_tos_v4(96));\n" }, { "path": "tokio/tests/uds_cred.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(all(unix, not(target_os = \"dragonfly\"), not(miri)))] // No `getsockopt` on miri.\n\nuse tokio::net::UnixStream;\n\nuse libc::getegid;\nuse libc::geteuid;\n\n#[tokio::test]\n#[cfg_attr(\n target_os = \"netbsd\",\n ignore = \"NetBSD does not support getpeereid() for sockets created by socketpair()\"\n)]\nasync fn test_socket_pair() {\n let (a, b) = UnixStream::pair().unwrap();\n let cred_a = a.peer_cred().unwrap();\n let cred_b = b.peer_cred().unwrap();\n assert_eq!(cred_a, cred_b);\n\n let uid = unsafe { geteuid() };\n let gid = unsafe { getegid() };\n\n assert_eq!(cred_a.uid(), uid);\n assert_eq!(cred_a.gid(), gid);\n}\n" }, { "path": "tokio/tests/uds_datagram.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n\nuse tokio::io::ReadBuf;\nuse tokio::net::UnixDatagram;\nuse tokio::try_join;\n\nuse std::future::poll_fn;\nuse std::io;\nuse std::sync::Arc;\n\nasync fn echo_server(socket: UnixDatagram) -> io::Result<()> {\n let mut recv_buf = vec![0u8; 1024];\n loop {\n let (len, peer_addr) = socket.recv_from(&mut recv_buf[..]).await?;\n if let Some(path) = peer_addr.as_pathname() {\n socket.send_to(&recv_buf[..len], path).await?;\n }\n }\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn echo() -> io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n let server_socket = UnixDatagram::bind(server_path.clone())?;\n\n tokio::spawn(async move {\n let _ = echo_server(server_socket).await;\n });\n\n {\n let socket = UnixDatagram::bind(&client_path).unwrap();\n socket.connect(server_path)?;\n socket.send(b\"ECHO\").await?;\n let mut recv_buf = [0u8; 16];\n let len = socket.recv(&mut recv_buf[..]).await?;\n assert_eq!(&recv_buf[..len], b\"ECHO\");\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn echo_from() -> io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n let server_socket = UnixDatagram::bind(server_path.clone())?;\n\n tokio::spawn(async move {\n let _ = echo_server(server_socket).await;\n });\n\n {\n let socket = UnixDatagram::bind(&client_path).unwrap();\n socket.connect(&server_path)?;\n socket.send(b\"ECHO\").await?;\n let mut recv_buf = [0u8; 16];\n let (len, addr) = socket.recv_from(&mut recv_buf[..]).await?;\n assert_eq!(&recv_buf[..len], b\"ECHO\");\n assert_eq!(addr.as_pathname(), Some(server_path.as_path()));\n }\n\n Ok(())\n}\n\n// Even though we use sync non-blocking io we still need a reactor.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No SOCK_DGRAM for `socketpair` in miri.\nasync fn try_send_recv_never_block() -> io::Result<()> {\n let mut recv_buf = [0u8; 16];\n let payload = b\"PAYLOAD\";\n let mut count = 0;\n\n let (dgram1, dgram2) = UnixDatagram::pair()?;\n\n // Send until we hit the OS `net.unix.max_dgram_qlen`.\n loop {\n dgram1.writable().await.unwrap();\n\n match dgram1.try_send(payload) {\n Err(err) => match (err.kind(), err.raw_os_error()) {\n (io::ErrorKind::WouldBlock, _) => break,\n (_, Some(libc::ENOBUFS)) => break,\n _ => {\n panic!(\"unexpected error {err:?}\");\n }\n },\n Ok(len) => {\n assert_eq!(len, payload.len());\n }\n }\n count += 1;\n }\n\n // Read every dgram we sent.\n while count > 0 {\n dgram2.readable().await.unwrap();\n let len = dgram2.try_recv(&mut recv_buf[..])?;\n assert_eq!(len, payload.len());\n assert_eq!(payload, &recv_buf[..len]);\n count -= 1;\n }\n\n let err = dgram2.try_recv(&mut recv_buf[..]).unwrap_err();\n match err.kind() {\n io::ErrorKind::WouldBlock => (),\n _ => unreachable!(\"unexpected error {:?}\", err),\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn split() -> std::io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let path = dir.path().join(\"split.sock\");\n let s = Arc::new(UnixDatagram::bind(path.clone())?);\n let r = s.clone();\n\n let msg = b\"hello\";\n let ((), ()) = try_join! {\n async {\n s.send_to(msg, path).await?;\n io::Result::Ok(())\n },\n async {\n let mut recv_buf = [0u8; 32];\n let (len, _) = r.recv_from(&mut recv_buf[..]).await?;\n assert_eq!(&recv_buf[..len], msg);\n Ok(())\n },\n }?;\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn send_to_recv_from_poll() -> std::io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let sender_path = dir.path().join(\"sender.sock\");\n let receiver_path = dir.path().join(\"receiver.sock\");\n\n let sender = UnixDatagram::bind(&sender_path)?;\n let receiver = UnixDatagram::bind(&receiver_path)?;\n\n let msg = b\"hello\";\n poll_fn(|cx| sender.poll_send_to(cx, msg, &receiver_path)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n let addr = poll_fn(|cx| receiver.poll_recv_from(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), msg);\n assert_eq!(addr.as_pathname(), Some(sender_path.as_ref()));\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn send_recv_poll() -> std::io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let sender_path = dir.path().join(\"sender.sock\");\n let receiver_path = dir.path().join(\"receiver.sock\");\n\n let sender = UnixDatagram::bind(&sender_path)?;\n let receiver = UnixDatagram::bind(&receiver_path)?;\n\n sender.connect(&receiver_path)?;\n receiver.connect(&sender_path)?;\n\n let msg = b\"hello\";\n poll_fn(|cx| sender.poll_send(cx, msg)).await?;\n\n let mut recv_buf = [0u8; 32];\n let mut read = ReadBuf::new(&mut recv_buf);\n poll_fn(|cx| receiver.poll_recv(cx, &mut read)).await?;\n\n assert_eq!(read.filled(), msg);\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn try_send_to_recv_from() -> std::io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n // Create listener\n let server = UnixDatagram::bind(&server_path)?;\n\n // Create socket pair\n let client = UnixDatagram::bind(&client_path)?;\n\n for _ in 0..5 {\n loop {\n client.writable().await?;\n\n match client.try_send_to(b\"hello world\", &server_path) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await?;\n\n let mut buf = [0; 512];\n\n match server.try_recv_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr.as_pathname(), Some(client_path.as_ref()));\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn try_recv_buf_from() -> std::io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n // Create listener\n let server = UnixDatagram::bind(&server_path)?;\n\n // Create socket pair\n let client = UnixDatagram::bind(&client_path)?;\n\n for _ in 0..5 {\n loop {\n client.writable().await?;\n\n match client.try_send_to(b\"hello world\", &server_path) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n server.readable().await?;\n\n let mut buf = Vec::with_capacity(512);\n\n match server.try_recv_buf_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr.as_pathname(), Some(client_path.as_ref()));\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn recv_buf_from() -> std::io::Result<()> {\n let tmp = tempfile::tempdir()?;\n\n // Bind each socket to a filesystem path\n let tx_path = tmp.path().join(\"tx\");\n let tx = UnixDatagram::bind(&tx_path)?;\n let rx_path = tmp.path().join(\"rx\");\n let rx = UnixDatagram::bind(&rx_path)?;\n\n let bytes = b\"hello world\";\n tx.send_to(bytes, &rx_path).await?;\n\n let mut buf = Vec::with_capacity(24);\n let (size, addr) = rx.recv_buf_from(&mut buf).await?;\n\n let dgram = &buf[..size];\n assert_eq!(dgram, bytes);\n assert_eq!(addr.as_pathname().unwrap(), &tx_path);\n Ok(())\n}\n\n// Even though we use sync non-blocking io we still need a reactor.\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No SOCK_DGRAM for `socketpair` in miri.\nasync fn try_recv_buf_never_block() -> io::Result<()> {\n let payload = b\"PAYLOAD\";\n let mut count = 0;\n\n let (dgram1, dgram2) = UnixDatagram::pair()?;\n\n // Send until we hit the OS `net.unix.max_dgram_qlen`.\n loop {\n dgram1.writable().await.unwrap();\n\n match dgram1.try_send(payload) {\n Err(err) => match (err.kind(), err.raw_os_error()) {\n (io::ErrorKind::WouldBlock, _) => break,\n (_, Some(libc::ENOBUFS)) => break,\n _ => {\n panic!(\"unexpected error {err:?}\");\n }\n },\n Ok(len) => {\n assert_eq!(len, payload.len());\n }\n }\n count += 1;\n }\n\n // Read every dgram we sent.\n while count > 0 {\n let mut recv_buf = Vec::with_capacity(16);\n\n dgram2.readable().await.unwrap();\n let len = dgram2.try_recv_buf(&mut recv_buf)?;\n assert_eq!(len, payload.len());\n assert_eq!(payload, &recv_buf[..len]);\n count -= 1;\n }\n\n let mut recv_buf = vec![0; 16];\n let err = dgram2.try_recv_from(&mut recv_buf).unwrap_err();\n match err.kind() {\n io::ErrorKind::WouldBlock => (),\n _ => unreachable!(\"unexpected error {:?}\", err),\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No SOCK_DGRAM for `socketpair` in miri.\nasync fn recv_buf() -> std::io::Result<()> {\n // Create the pair of sockets\n let (sock1, sock2) = UnixDatagram::pair()?;\n\n // Since the sockets are paired, the paired send/recv\n // functions can be used\n let bytes = b\"hello world\";\n sock1.send(bytes).await?;\n\n let mut buff = Vec::with_capacity(24);\n let size = sock2.recv_buf(&mut buff).await?;\n\n let dgram = &buff[..size];\n assert_eq!(dgram, bytes);\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` on miri.\nasync fn poll_ready() -> io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n // Create listener\n let server = UnixDatagram::bind(&server_path)?;\n\n // Create socket pair\n let client = UnixDatagram::bind(&client_path)?;\n\n for _ in 0..5 {\n loop {\n poll_fn(|cx| client.poll_send_ready(cx)).await?;\n\n match client.try_send_to(b\"hello world\", &server_path) {\n Ok(n) => {\n assert_eq!(n, 11);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n\n loop {\n poll_fn(|cx| server.poll_recv_ready(cx)).await?;\n\n let mut buf = Vec::with_capacity(512);\n\n match server.try_recv_buf_from(&mut buf) {\n Ok((n, addr)) => {\n assert_eq!(n, 11);\n assert_eq!(addr.as_pathname(), Some(client_path.as_ref()));\n assert_eq!(&buf[0..11], &b\"hello world\"[..]);\n break;\n }\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"{e:?}\"),\n }\n }\n }\n\n Ok(())\n}\n" }, { "path": "tokio/tests/uds_socket.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n\nuse futures::future::try_join;\nuse std::io;\nuse tokio::{\n io::{AsyncReadExt, AsyncWriteExt},\n net::UnixSocket,\n};\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn datagram_echo_server() -> io::Result<()> {\n let dir = tempfile::tempdir().unwrap();\n let server_path = dir.path().join(\"server.sock\");\n let client_path = dir.path().join(\"client.sock\");\n\n let server_socket = {\n let socket = UnixSocket::new_datagram()?;\n socket.bind(&server_path)?;\n socket.datagram()?\n };\n\n tokio::spawn(async move {\n let mut recv_buf = vec![0u8; 1024];\n loop {\n let (len, peer_addr) = server_socket.recv_from(&mut recv_buf[..]).await?;\n if let Some(path) = peer_addr.as_pathname() {\n server_socket.send_to(&recv_buf[..len], path).await?;\n }\n }\n\n #[allow(unreachable_code)]\n Ok::<(), io::Error>(())\n });\n\n {\n let socket = UnixSocket::new_datagram()?;\n socket.bind(&client_path).unwrap();\n let socket = socket.datagram()?;\n\n socket.connect(server_path)?;\n socket.send(b\"ECHO\").await?;\n\n let mut recv_buf = [0u8; 16];\n let len = socket.recv(&mut recv_buf[..]).await?;\n assert_eq!(&recv_buf[..len], b\"ECHO\");\n }\n\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn listen_and_stream() -> std::io::Result<()> {\n let dir = tempfile::Builder::new().tempdir().unwrap();\n let sock_path = dir.path().join(\"connect.sock\");\n let peer_path = dir.path().join(\"peer.sock\");\n\n let listener = {\n let sock = UnixSocket::new_stream()?;\n sock.bind(&sock_path)?;\n sock.listen(1024)?\n };\n\n let accept = listener.accept();\n let connect = {\n let sock = UnixSocket::new_stream()?;\n sock.bind(&peer_path)?;\n sock.connect(&sock_path)\n };\n\n let ((mut server, _), mut client) = try_join(accept, connect).await?;\n\n assert_eq!(\n server.peer_addr().unwrap().as_pathname().unwrap(),\n &peer_path\n );\n\n // Write to the client.\n client.write_all(b\"hello\").await?;\n drop(client);\n\n // Read from the server.\n let mut buf = vec![];\n server.read_to_end(&mut buf).await?;\n assert_eq!(&buf, b\"hello\");\n let len = server.read(&mut buf).await?;\n assert_eq!(len, 0);\n Ok(())\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // No `socket` in miri.\nasync fn assert_usage() -> std::io::Result<()> {\n let datagram_socket = UnixSocket::new_datagram()?;\n let result = datagram_socket\n .connect(std::path::PathBuf::new().join(\"invalid.sock\"))\n .await;\n assert_eq!(\n result.unwrap_err().to_string(),\n \"connect cannot be called on a datagram socket\"\n );\n\n let datagram_socket = UnixSocket::new_datagram()?;\n let result = datagram_socket.listen(1024);\n assert_eq!(\n result.unwrap_err().to_string(),\n \"listen cannot be called on a datagram socket\"\n );\n\n let stream_socket = UnixSocket::new_stream()?;\n let result = stream_socket.datagram();\n assert_eq!(\n result.unwrap_err().to_string(),\n \"datagram cannot be called on a stream socket\"\n );\n\n Ok(())\n}\n" }, { "path": "tokio/tests/uds_split.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n#![cfg(unix)]\n#![cfg(not(miri))] // No `socket` in miri.\n\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};\nuse tokio::net::UnixStream;\n\n/// Checks that `UnixStream` can be split into a read half and a write half using\n/// `UnixStream::split` and `UnixStream::split_mut`.\n///\n/// Verifies that the implementation of `AsyncWrite::poll_shutdown` shutdowns the stream for\n/// writing by reading to the end of stream on the other side of the connection.\n#[tokio::test]\nasync fn split() -> std::io::Result<()> {\n let (mut a, mut b) = UnixStream::pair()?;\n\n let (mut a_read, mut a_write) = a.split();\n let (mut b_read, mut b_write) = b.split();\n\n let (a_response, b_response) = futures::future::try_join(\n send_recv_all(&mut a_read, &mut a_write, b\"A\"),\n send_recv_all(&mut b_read, &mut b_write, b\"B\"),\n )\n .await?;\n\n assert_eq!(a_response, b\"B\");\n assert_eq!(b_response, b\"A\");\n\n Ok(())\n}\n\nasync fn send_recv_all(\n read: &mut (dyn AsyncRead + Unpin),\n write: &mut (dyn AsyncWrite + Unpin),\n input: &[u8],\n) -> std::io::Result> {\n write.write_all(input).await?;\n write.shutdown().await?;\n\n let mut output = Vec::new();\n read.read_to_end(&mut output).await?;\n Ok(output)\n}\n" }, { "path": "tokio/tests/uds_stream.rs", "content": "#![cfg(feature = \"full\")]\n#![warn(rust_2018_idioms)]\n#![cfg(unix)]\n#![cfg(not(miri))] // No socket in miri.\n\nuse std::io;\n#[cfg(target_os = \"android\")]\nuse std::os::android::net::SocketAddrExt;\n#[cfg(target_os = \"linux\")]\nuse std::os::linux::net::SocketAddrExt;\nuse std::task::Poll;\n\nuse tokio::io::{AsyncReadExt, AsyncWriteExt, Interest};\nuse tokio::net::{UnixListener, UnixStream};\nuse tokio_test::{assert_ok, assert_pending, assert_ready_ok, task};\n\nuse futures::future::{poll_fn, try_join};\n\n#[tokio::test]\nasync fn accept_read_write() -> std::io::Result<()> {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-uds-tests\")\n .tempdir()\n .unwrap();\n let sock_path = dir.path().join(\"connect.sock\");\n\n let listener = UnixListener::bind(&sock_path)?;\n\n let accept = listener.accept();\n let connect = UnixStream::connect(&sock_path);\n let ((mut server, _), mut client) = try_join(accept, connect).await?;\n\n // Write to the client.\n client.write_all(b\"hello\").await?;\n drop(client);\n\n // Read from the server.\n let mut buf = vec![];\n server.read_to_end(&mut buf).await?;\n assert_eq!(&buf, b\"hello\");\n let len = server.read(&mut buf).await?;\n assert_eq!(len, 0);\n Ok(())\n}\n\n#[tokio::test]\nasync fn shutdown() -> std::io::Result<()> {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-uds-tests\")\n .tempdir()\n .unwrap();\n let sock_path = dir.path().join(\"connect.sock\");\n\n let listener = UnixListener::bind(&sock_path)?;\n\n let accept = listener.accept();\n let connect = UnixStream::connect(&sock_path);\n let ((mut server, _), mut client) = try_join(accept, connect).await?;\n\n // Shut down the client\n AsyncWriteExt::shutdown(&mut client).await?;\n // Read from the server should return 0 to indicate the channel has been closed.\n let mut buf = [0u8; 1];\n let n = server.read(&mut buf).await?;\n assert_eq!(n, 0);\n Ok(())\n}\n\n#[tokio::test]\nasync fn try_read_write() -> std::io::Result<()> {\n let msg = b\"hello world\";\n\n let dir = tempfile::tempdir()?;\n let bind_path = dir.path().join(\"bind.sock\");\n\n // Create listener\n let listener = UnixListener::bind(&bind_path)?;\n\n // Create socket pair\n let client = UnixStream::connect(&bind_path).await?;\n\n let (server, _) = listener.accept().await?;\n let mut written = msg.to_vec();\n\n // Track the server receiving data\n let mut readable = task::spawn(server.readable());\n assert_pending!(readable.poll());\n\n // Write data.\n client.writable().await?;\n assert_eq!(msg.len(), client.try_write(msg)?);\n\n // The task should be notified\n while !readable.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Fill the write buffer using non-vectored I/O\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write(msg) {\n Ok(n) => written.extend(&msg[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end using non-vectored I/O\n let mut read = vec![0; written.len()];\n let mut i = 0;\n\n while i < read.len() {\n server.readable().await?;\n\n match server.try_read(&mut read[i..]) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n written.clear();\n client.writable().await.unwrap();\n\n // Fill the write buffer using vectored I/O\n let msg_bufs: Vec<_> = msg.chunks(3).map(io::IoSlice::new).collect();\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write_vectored(&msg_bufs) {\n Ok(n) => written.extend(&msg[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end using vectored I/O\n let mut read = vec![0; written.len()];\n let mut i = 0;\n\n while i < read.len() {\n server.readable().await?;\n\n let mut bufs: Vec<_> = read[i..]\n .chunks_mut(0x10000)\n .map(io::IoSliceMut::new)\n .collect();\n match server.try_read_vectored(&mut bufs) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n // Now, we listen for shutdown\n drop(client);\n\n loop {\n let ready = server.ready(Interest::READABLE).await?;\n\n if ready.is_read_closed() {\n break;\n } else {\n tokio::task::yield_now().await;\n }\n }\n\n Ok(())\n}\n\nasync fn create_pair() -> (UnixStream, UnixStream) {\n let dir = assert_ok!(tempfile::tempdir());\n let bind_path = dir.path().join(\"bind.sock\");\n\n let listener = assert_ok!(UnixListener::bind(&bind_path));\n\n let accept = listener.accept();\n let connect = UnixStream::connect(&bind_path);\n let ((server, _), client) = assert_ok!(try_join(accept, connect).await);\n\n (client, server)\n}\n\nmacro_rules! assert_readable_by_polling {\n ($stream:expr) => {\n assert_ok!(poll_fn(|cx| $stream.poll_read_ready(cx)).await);\n };\n}\n\nmacro_rules! assert_not_readable_by_polling {\n ($stream:expr) => {\n poll_fn(|cx| {\n assert_pending!($stream.poll_read_ready(cx));\n Poll::Ready(())\n })\n .await;\n };\n}\n\nmacro_rules! assert_writable_by_polling {\n ($stream:expr) => {\n assert_ok!(poll_fn(|cx| $stream.poll_write_ready(cx)).await);\n };\n}\n\nmacro_rules! assert_not_writable_by_polling {\n ($stream:expr) => {\n poll_fn(|cx| {\n assert_pending!($stream.poll_write_ready(cx));\n Poll::Ready(())\n })\n .await;\n };\n}\n\n#[tokio::test]\nasync fn poll_read_ready() {\n let (mut client, mut server) = create_pair().await;\n\n // Initial state - not readable.\n assert_not_readable_by_polling!(server);\n\n // There is data in the buffer - readable.\n assert_ok!(client.write_all(b\"ping\").await);\n assert_readable_by_polling!(server);\n\n // Readable until calls to `poll_read` return `Poll::Pending`.\n let mut buf = [0u8; 4];\n assert_ok!(server.read_exact(&mut buf).await);\n assert_readable_by_polling!(server);\n read_until_pending(&mut server);\n assert_not_readable_by_polling!(server);\n\n // Detect the client disconnect.\n drop(client);\n assert_readable_by_polling!(server);\n}\n\n#[tokio::test]\nasync fn poll_write_ready() {\n let (mut client, server) = create_pair().await;\n\n // Initial state - writable.\n assert_writable_by_polling!(client);\n\n // No space to write - not writable.\n write_until_pending(&mut client);\n assert_not_writable_by_polling!(client);\n\n // Detect the server disconnect.\n drop(server);\n assert_writable_by_polling!(client);\n}\n\nfn read_until_pending(stream: &mut UnixStream) {\n let mut buf = vec![0u8; 1024 * 1024];\n loop {\n match stream.try_read(&mut buf) {\n Ok(_) => (),\n Err(err) => {\n assert_eq!(err.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n}\n\nfn write_until_pending(stream: &mut UnixStream) {\n let buf = vec![0u8; 1024 * 1024];\n loop {\n match stream.try_write(&buf) {\n Ok(_) => (),\n Err(err) => {\n assert_eq!(err.kind(), io::ErrorKind::WouldBlock);\n break;\n }\n }\n }\n}\n\n#[tokio::test]\nasync fn try_read_buf() -> std::io::Result<()> {\n let msg = b\"hello world\";\n\n let dir = tempfile::tempdir()?;\n let bind_path = dir.path().join(\"bind.sock\");\n\n // Create listener\n let listener = UnixListener::bind(&bind_path)?;\n\n // Create socket pair\n let client = UnixStream::connect(&bind_path).await?;\n\n let (server, _) = listener.accept().await?;\n let mut written = msg.to_vec();\n\n // Track the server receiving data\n let mut readable = task::spawn(server.readable());\n assert_pending!(readable.poll());\n\n // Write data.\n client.writable().await?;\n assert_eq!(msg.len(), client.try_write(msg)?);\n\n // The task should be notified\n while !readable.is_woken() {\n tokio::task::yield_now().await;\n }\n\n // Fill the write buffer\n loop {\n // Still ready\n let mut writable = task::spawn(client.writable());\n assert_ready_ok!(writable.poll());\n\n match client.try_write(msg) {\n Ok(n) => written.extend(&msg[..n]),\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n break;\n }\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n {\n // Write buffer full\n let mut writable = task::spawn(client.writable());\n assert_pending!(writable.poll());\n\n // Drain the socket from the server end\n let mut read = Vec::with_capacity(written.len());\n let mut i = 0;\n\n while i < read.capacity() {\n server.readable().await?;\n\n match server.try_read_buf(&mut read) {\n Ok(n) => i += n,\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => continue,\n Err(e) => panic!(\"error = {e:?}\"),\n }\n }\n\n assert_eq!(read, written);\n }\n\n // Now, we listen for shutdown\n drop(client);\n\n loop {\n let ready = server.ready(Interest::READABLE).await?;\n\n if ready.is_read_closed() {\n break;\n } else {\n tokio::task::yield_now().await;\n }\n }\n\n Ok(())\n}\n\n// https://github.com/tokio-rs/tokio/issues/3879\n#[tokio::test]\n#[cfg(not(target_os = \"macos\"))]\nasync fn epollhup() -> io::Result<()> {\n let dir = tempfile::Builder::new()\n .prefix(\"tokio-uds-tests\")\n .tempdir()\n .unwrap();\n let sock_path = dir.path().join(\"connect.sock\");\n\n let listener = UnixListener::bind(&sock_path)?;\n let connect = UnixStream::connect(&sock_path);\n tokio::pin!(connect);\n\n // Poll `connect` once.\n poll_fn(|cx| {\n use std::future::Future;\n\n assert_pending!(connect.as_mut().poll(cx));\n Poll::Ready(())\n })\n .await;\n\n drop(listener);\n\n let err = connect.await.unwrap_err();\n let errno = err.kind();\n assert!(\n // As far as I can tell, whether we see ECONNREFUSED or ECONNRESET here\n // seems relatively inconsistent, at least on non-Linux operating\n // systems. The difference in meaning between these errnos is not\n // particularly well-defined, so let's just accept either.\n matches!(\n errno,\n io::ErrorKind::ConnectionRefused | io::ErrorKind::ConnectionReset\n ),\n \"unexpected error kind: {errno:?} (expected ConnectionRefused or ConnectionReset)\"\n );\n Ok(())\n}\n\n// test for https://github.com/tokio-rs/tokio/issues/6767\n#[tokio::test]\n#[cfg(any(target_os = \"linux\", target_os = \"android\"))]\nasync fn abstract_socket_name() {\n let socket_path = \"\\0aaa\";\n let listener = UnixListener::bind(socket_path).unwrap();\n\n let accept = listener.accept();\n let connect = UnixStream::connect(&socket_path);\n\n let ((stream, _), _) = try_join(accept, connect).await.unwrap();\n\n let local_addr = stream.into_std().unwrap().local_addr().unwrap();\n let abstract_path_name = local_addr.as_abstract_name().unwrap();\n\n // `as_abstract_name` removes leading zero bytes\n assert_eq!(abstract_path_name, b\"aaa\");\n}\n" }, { "path": "tokio/tests/unwindsafe.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n\nuse std::panic::{RefUnwindSafe, UnwindSafe};\n\n#[test]\nfn notify_is_unwind_safe() {\n is_unwind_safe::();\n}\n\n#[test]\nfn join_handle_is_unwind_safe() {\n is_unwind_safe::>();\n}\n\n#[test]\nfn net_types_are_unwind_safe() {\n is_unwind_safe::();\n is_unwind_safe::();\n is_unwind_safe::();\n is_unwind_safe::();\n}\n\n#[test]\n#[cfg(unix)]\nfn unix_net_types_are_unwind_safe() {\n is_unwind_safe::();\n is_unwind_safe::();\n is_unwind_safe::();\n}\n\n#[test]\n#[cfg(windows)]\nfn windows_net_types_are_unwind_safe() {\n use tokio::net::windows::named_pipe::NamedPipeClient;\n use tokio::net::windows::named_pipe::NamedPipeServer;\n\n is_unwind_safe::();\n is_unwind_safe::();\n}\n\nfn is_unwind_safe() {}\n" }, { "path": "tokio-macros/CHANGELOG.md", "content": "# 2.6.1 (Mar 2nd, 2026)\n\n- macros: improve error message for return type mismatch in #[tokio::main] ([#7856])\n- macros: use call_site hygiene to avoid unused qualification ([#7866])\n\n[#7856]: https://github.com/tokio-rs/tokio/pull/7856\n[#7866]: https://github.com/tokio-rs/tokio/pull/7866\n\n# 2.6.0 (Oct 14th, 2025)\n\nThe MSRV is raised to 1.71.\n\n- msrv: increase MSRV to 1.71 ([#7658])\n- macros: add `local` runtime flavor ([#7375], [#7597])\n- macros: suppress `clippy::unwrap_in_result` in `#[tokio::main]` ([#7651])\n\n[#7375]: https://github.com/tokio-rs/tokio/pull/7375\n[#7597]: https://github.com/tokio-rs/tokio/pull/7597\n[#7651]: https://github.com/tokio-rs/tokio/pull/7651\n[#7658]: https://github.com/tokio-rs/tokio/pull/7658\n\n# 2.5.0 (Jan 8th, 2025)\n\n- macros: suppress `clippy::needless_return` in `#[tokio::main]` ([#6874])\n\n[#6874]: https://github.com/tokio-rs/tokio/pull/6874\n\n# 2.4.0 (July 22nd, 2024)\n\n- msrv: increase MSRV to 1.70 ([#6645])\n- macros: allow `unhandled_panic` behavior for `#[tokio::main]` and `#[tokio::test]` ([#6593])\n\n[#6593]: https://github.com/tokio-rs/tokio/pull/6593\n[#6645]: https://github.com/tokio-rs/tokio/pull/6645\n\n# 2.3.0 (May 30th, 2024)\n\n- macros: make `#[tokio::test]` append `#[test]` at the end of the attribute list ([#6497])\n\n[#6497]: https://github.com/tokio-rs/tokio/pull/6497\n\n# 2.2.0 (November 19th, 2023)\n\n### Changed\n\n- use `::core` qualified imports instead of `::std` inside `tokio::test` macro ([#5973])\n\n[#5973]: https://github.com/tokio-rs/tokio/pull/5973\n\n# 2.1.0 (April 25th, 2023)\n\n- macros: fix typo in `#[tokio::test]` docs ([#5636])\n- macros: make entrypoints more efficient ([#5621])\n\n[#5621]: https://github.com/tokio-rs/tokio/pull/5621\n[#5636]: https://github.com/tokio-rs/tokio/pull/5636\n\n# 2.0.0 (March 24th, 2023)\n\nThis major release updates the dependency on the syn crate to 2.0.0, and\nincreases the MSRV to 1.56.\n\nAs part of this release, we are adopting a policy of depending on a specific minor\nrelease of tokio-macros. This prevents Tokio from being able to pull in many different\nversions of tokio-macros.\n\n- macros: update `syn` ([#5572])\n- macros: accept path as crate rename ([#5557])\n\n[#5572]: https://github.com/tokio-rs/tokio/pull/5572\n[#5557]: https://github.com/tokio-rs/tokio/pull/5557\n\n# 1.8.2 (November 30th, 2022)\n\n- fix a regression introduced in 1.8.1 ([#5244])\n\n[#5244]: https://github.com/tokio-rs/tokio/pull/5244\n\n# 1.8.1 (November 29th, 2022)\n\n(yanked)\n\n- macros: Pin Futures in `#[tokio::test]` to stack ([#5205])\n- macros: Reduce usage of last statement spans in proc-macros ([#5092])\n- macros: Improve the documentation for `#[tokio::test]` ([#4761])\n\n[#5205]: https://github.com/tokio-rs/tokio/pull/5205\n[#5092]: https://github.com/tokio-rs/tokio/pull/5092\n[#4761]: https://github.com/tokio-rs/tokio/pull/4761\n\n# 1.8.0 (June 4th, 2022)\n\n- macros: always emit return statement ([#4636])\n- macros: support setting a custom crate name for `#[tokio::main]` and `#[tokio::test]` ([#4613])\n\n[#4613]: https://github.com/tokio-rs/tokio/pull/4613\n[#4636]: https://github.com/tokio-rs/tokio/pull/4636\n\n# 1.7.0 (December 15th, 2021)\n\n- macros: address remaining `clippy::semicolon_if_nothing_returned` warning ([#4252])\n\n[#4252]: https://github.com/tokio-rs/tokio/pull/4252\n\n# 1.6.0 (November 16th, 2021)\n\n- macros: fix mut patterns in `select!` macro ([#4211])\n\n[#4211]: https://github.com/tokio-rs/tokio/pull/4211\n\n# 1.5.1 (October 29th, 2021)\n\n- macros: fix type resolution error in `#[tokio::main]` ([#4176])\n\n[#4176]: https://github.com/tokio-rs/tokio/pull/4176\n\n# 1.5.0 (October 13th, 2021)\n\n- macros: make tokio-macros attributes more IDE friendly ([#4162])\n\n[#4162]: https://github.com/tokio-rs/tokio/pull/4162\n\n# 1.4.1 (September 30th, 2021)\n\nReverted: run `current_thread` inside `LocalSet` ([#4027])\n\n# 1.4.0 (September 29th, 2021)\n\n(yanked)\n\n### Changed\n\n- macros: run `current_thread` inside `LocalSet` ([#4027])\n- macros: explicitly relaxed clippy lint for `.expect()` in runtime entry macro ([#4030])\n\n### Fixed\n\n- macros: fix invalid error messages in functions wrapped with `#[main]` or `#[test]` ([#4067])\n\n[#4027]: https://github.com/tokio-rs/tokio/pull/4027\n[#4030]: https://github.com/tokio-rs/tokio/pull/4030\n[#4067]: https://github.com/tokio-rs/tokio/pull/4067\n\n# 1.3.0 (July 7, 2021)\n\n- macros: don't trigger `clippy::unwrap_used` ([#3926])\n\n[#3926]: https://github.com/tokio-rs/tokio/pull/3926\n\n# 1.2.0 (May 14, 2021)\n\n- macros: forward input arguments in `#[tokio::test]` ([#3691])\n- macros: improve diagnostics on type mismatch ([#3766])\n- macros: various error message improvements ([#3677])\n\n[#3677]: https://github.com/tokio-rs/tokio/pull/3677\n[#3691]: https://github.com/tokio-rs/tokio/pull/3691\n[#3766]: https://github.com/tokio-rs/tokio/pull/3766\n\n# 1.1.0 (February 5, 2021)\n\n- add `start_paused` option to macros ([#3492])\n\n# 1.0.0 (December 23, 2020)\n\n- track `tokio` 1.0 release.\n\n# 0.3.1 (October 25, 2020)\n\n### Fixed\n\n- fix incorrect docs regarding `max_threads` option ([#3038])\n\n# 0.3.0 (October 15, 2020)\n\n- Track `tokio` 0.3 release.\n\n### Changed\n- options are renamed to track `tokio` runtime builder fn names.\n- `#[tokio::main]` macro requires `rt-multi-thread` when no `flavor` is specified.\n\n# 0.2.5 (February 27, 2019)\n\n### Fixed\n- doc improvements ([#2225]).\n\n# 0.2.4 (January 27, 2019)\n\n### Fixed\n- generics on `#[tokio::main]` function ([#2177]).\n\n### Added\n- support for `tokio::select!` ([#2152]).\n\n# 0.2.3 (January 7, 2019)\n\n### Fixed\n- Revert breaking change.\n\n# 0.2.2 (January 7, 2019)\n\n### Added\n- General refactoring and inclusion of additional runtime options ([#2022] and [#2038])\n\n# 0.2.1 (December 18, 2019)\n\n### Fixes\n- inherit visibility when wrapping async fn ([#1954]).\n\n# 0.2.0 (November 26, 2019)\n\n- Initial release\n\n[#1954]: https://github.com/tokio-rs/tokio/pull/1954\n[#2022]: https://github.com/tokio-rs/tokio/pull/2022\n[#2038]: https://github.com/tokio-rs/tokio/pull/2038\n[#2152]: https://github.com/tokio-rs/tokio/pull/2152\n[#2177]: https://github.com/tokio-rs/tokio/pull/2177\n[#2225]: https://github.com/tokio-rs/tokio/pull/2225\n[#3038]: https://github.com/tokio-rs/tokio/pull/3038\n[#3492]: https://github.com/tokio-rs/tokio/pull/3492\n" }, { "path": "tokio-macros/Cargo.toml", "content": "[package]\nname = \"tokio-macros\"\n# When releasing to crates.io:\n# - Remove path dependencies (if any)\n# - Update CHANGELOG.md.\n# - Create \"tokio-macros-x.y.z\" git tag.\nversion = \"2.6.1\"\nedition = \"2021\"\nrust-version = \"1.71\"\nauthors = [\"Tokio Contributors \"]\nlicense = \"MIT\"\nrepository = \"https://github.com/tokio-rs/tokio\"\nhomepage = \"https://tokio.rs\"\ndescription = \"\"\"\nTokio's proc macros.\n\"\"\"\ncategories = [\"asynchronous\"]\n\n[lib]\nproc-macro = true\n\n[features]\n\n[dependencies]\nproc-macro2 = \"1.0.60\"\nquote = \"1\"\nsyn = { version = \"2.0\", features = [\"full\"] }\n\n[dev-dependencies]\ntokio = { version = \"1.0.0\", features = [\"full\", \"test-util\"] }\n\n[package.metadata.docs.rs]\nall-features = true\n\n[lints]\nworkspace = true\n" }, { "path": "tokio-macros/LICENSE", "content": "MIT License\n\nCopyright (c) 2019 Yoshua Wuyts\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "tokio-macros/README.md", "content": "# Tokio Macros\n\nProcedural macros for use with Tokio\n\n## License\n\nThis project is licensed under the [MIT license](LICENSE).\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you, shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "tokio-macros/src/entry.rs", "content": "use proc_macro2::{Span, TokenStream, TokenTree};\nuse quote::{quote, quote_spanned, ToTokens};\nuse syn::parse::{Parse, ParseStream, Parser};\nuse syn::{braced, Attribute, Ident, Path, Signature, Visibility};\n\n// syn::AttributeArgs does not implement syn::Parse\ntype AttributeArgs = syn::punctuated::Punctuated;\n\n#[derive(Clone, Copy, PartialEq)]\nenum RuntimeFlavor {\n CurrentThread,\n Threaded,\n Local,\n}\n\nimpl RuntimeFlavor {\n fn from_str(s: &str) -> Result {\n match s {\n \"current_thread\" => Ok(RuntimeFlavor::CurrentThread),\n \"multi_thread\" => Ok(RuntimeFlavor::Threaded),\n \"local\" => Ok(RuntimeFlavor::Local),\n \"single_thread\" => Err(\"The single threaded runtime flavor is called `current_thread`.\".to_string()),\n \"basic_scheduler\" => Err(\"The `basic_scheduler` runtime flavor has been renamed to `current_thread`.\".to_string()),\n \"threaded_scheduler\" => Err(\"The `threaded_scheduler` runtime flavor has been renamed to `multi_thread`.\".to_string()),\n _ => Err(format!(\"No such runtime flavor `{s}`. The runtime flavors are `current_thread`, `local`, and `multi_thread`.\")),\n }\n }\n}\n\n#[derive(Clone, Copy, PartialEq)]\nenum UnhandledPanic {\n Ignore,\n ShutdownRuntime,\n}\n\nimpl UnhandledPanic {\n fn from_str(s: &str) -> Result {\n match s {\n \"ignore\" => Ok(UnhandledPanic::Ignore),\n \"shutdown_runtime\" => Ok(UnhandledPanic::ShutdownRuntime),\n _ => Err(format!(\"No such unhandled panic behavior `{s}`. The unhandled panic behaviors are `ignore` and `shutdown_runtime`.\")),\n }\n }\n\n fn into_tokens(self, crate_path: &TokenStream) -> TokenStream {\n match self {\n UnhandledPanic::Ignore => quote! { #crate_path::runtime::UnhandledPanic::Ignore },\n UnhandledPanic::ShutdownRuntime => {\n quote! { #crate_path::runtime::UnhandledPanic::ShutdownRuntime }\n }\n }\n }\n}\n\nstruct FinalConfig {\n flavor: RuntimeFlavor,\n worker_threads: Option,\n start_paused: Option,\n crate_name: Option,\n unhandled_panic: Option,\n}\n\n/// Config used in case of the attribute not being able to build a valid config\nconst DEFAULT_ERROR_CONFIG: FinalConfig = FinalConfig {\n flavor: RuntimeFlavor::CurrentThread,\n worker_threads: None,\n start_paused: None,\n crate_name: None,\n unhandled_panic: None,\n};\n\nstruct Configuration {\n rt_multi_thread_available: bool,\n default_flavor: RuntimeFlavor,\n flavor: Option,\n worker_threads: Option<(usize, Span)>,\n start_paused: Option<(bool, Span)>,\n is_test: bool,\n crate_name: Option,\n unhandled_panic: Option<(UnhandledPanic, Span)>,\n}\n\nimpl Configuration {\n fn new(is_test: bool, rt_multi_thread: bool) -> Self {\n Configuration {\n rt_multi_thread_available: rt_multi_thread,\n default_flavor: match is_test {\n true => RuntimeFlavor::CurrentThread,\n false => RuntimeFlavor::Threaded,\n },\n flavor: None,\n worker_threads: None,\n start_paused: None,\n is_test,\n crate_name: None,\n unhandled_panic: None,\n }\n }\n\n fn set_flavor(&mut self, runtime: syn::Lit, span: Span) -> Result<(), syn::Error> {\n if self.flavor.is_some() {\n return Err(syn::Error::new(span, \"`flavor` set multiple times.\"));\n }\n\n let runtime_str = parse_string(runtime, span, \"flavor\")?;\n let runtime =\n RuntimeFlavor::from_str(&runtime_str).map_err(|err| syn::Error::new(span, err))?;\n self.flavor = Some(runtime);\n Ok(())\n }\n\n fn set_worker_threads(\n &mut self,\n worker_threads: syn::Lit,\n span: Span,\n ) -> Result<(), syn::Error> {\n if self.worker_threads.is_some() {\n return Err(syn::Error::new(\n span,\n \"`worker_threads` set multiple times.\",\n ));\n }\n\n let worker_threads = parse_int(worker_threads, span, \"worker_threads\")?;\n if worker_threads == 0 {\n return Err(syn::Error::new(span, \"`worker_threads` may not be 0.\"));\n }\n self.worker_threads = Some((worker_threads, span));\n Ok(())\n }\n\n fn set_start_paused(&mut self, start_paused: syn::Lit, span: Span) -> Result<(), syn::Error> {\n if self.start_paused.is_some() {\n return Err(syn::Error::new(span, \"`start_paused` set multiple times.\"));\n }\n\n let start_paused = parse_bool(start_paused, span, \"start_paused\")?;\n self.start_paused = Some((start_paused, span));\n Ok(())\n }\n\n fn set_crate_name(&mut self, name: syn::Lit, span: Span) -> Result<(), syn::Error> {\n if self.crate_name.is_some() {\n return Err(syn::Error::new(span, \"`crate` set multiple times.\"));\n }\n let name_path = parse_path(name, span, \"crate\")?;\n self.crate_name = Some(name_path);\n Ok(())\n }\n\n fn set_unhandled_panic(\n &mut self,\n unhandled_panic: syn::Lit,\n span: Span,\n ) -> Result<(), syn::Error> {\n if self.unhandled_panic.is_some() {\n return Err(syn::Error::new(\n span,\n \"`unhandled_panic` set multiple times.\",\n ));\n }\n\n let unhandled_panic = parse_string(unhandled_panic, span, \"unhandled_panic\")?;\n let unhandled_panic =\n UnhandledPanic::from_str(&unhandled_panic).map_err(|err| syn::Error::new(span, err))?;\n self.unhandled_panic = Some((unhandled_panic, span));\n Ok(())\n }\n\n fn macro_name(&self) -> &'static str {\n if self.is_test {\n \"tokio::test\"\n } else {\n \"tokio::main\"\n }\n }\n\n fn build(&self) -> Result {\n use RuntimeFlavor as F;\n\n let flavor = self.flavor.unwrap_or(self.default_flavor);\n\n let worker_threads = match (flavor, self.worker_threads) {\n (F::CurrentThread | F::Local, Some((_, worker_threads_span))) => {\n let msg = format!(\n \"The `worker_threads` option requires the `multi_thread` runtime flavor. Use `#[{}(flavor = \\\"multi_thread\\\")]`\",\n self.macro_name(),\n );\n return Err(syn::Error::new(worker_threads_span, msg));\n }\n (F::CurrentThread | F::Local, None) => None,\n (F::Threaded, worker_threads) if self.rt_multi_thread_available => {\n worker_threads.map(|(val, _span)| val)\n }\n (F::Threaded, _) => {\n let msg = if self.flavor.is_none() {\n \"The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled.\"\n } else {\n \"The runtime flavor `multi_thread` requires the `rt-multi-thread` feature.\"\n };\n return Err(syn::Error::new(Span::call_site(), msg));\n }\n };\n\n let start_paused = match (flavor, self.start_paused) {\n (F::Threaded, Some((_, start_paused_span))) => {\n let msg = format!(\n \"The `start_paused` option requires the `current_thread` runtime flavor. Use `#[{}(flavor = \\\"current_thread\\\")]`\",\n self.macro_name(),\n );\n return Err(syn::Error::new(start_paused_span, msg));\n }\n (F::CurrentThread | F::Local, Some((start_paused, _))) => Some(start_paused),\n (_, None) => None,\n };\n\n let unhandled_panic = match (flavor, self.unhandled_panic) {\n (F::Threaded, Some((_, unhandled_panic_span))) => {\n let msg = format!(\n \"The `unhandled_panic` option requires the `current_thread` runtime flavor. Use `#[{}(flavor = \\\"current_thread\\\")]`\",\n self.macro_name(),\n );\n return Err(syn::Error::new(unhandled_panic_span, msg));\n }\n (F::CurrentThread | F::Local, Some((unhandled_panic, _))) => Some(unhandled_panic),\n (_, None) => None,\n };\n\n Ok(FinalConfig {\n crate_name: self.crate_name.clone(),\n flavor,\n worker_threads,\n start_paused,\n unhandled_panic,\n })\n }\n}\n\nfn parse_int(int: syn::Lit, span: Span, field: &str) -> Result {\n match int {\n syn::Lit::Int(lit) => match lit.base10_parse::() {\n Ok(value) => Ok(value),\n Err(e) => Err(syn::Error::new(\n span,\n format!(\"Failed to parse value of `{field}` as integer: {e}\"),\n )),\n },\n _ => Err(syn::Error::new(\n span,\n format!(\"Failed to parse value of `{field}` as integer.\"),\n )),\n }\n}\n\nfn parse_string(int: syn::Lit, span: Span, field: &str) -> Result {\n match int {\n syn::Lit::Str(s) => Ok(s.value()),\n syn::Lit::Verbatim(s) => Ok(s.to_string()),\n _ => Err(syn::Error::new(\n span,\n format!(\"Failed to parse value of `{field}` as string.\"),\n )),\n }\n}\n\nfn parse_path(lit: syn::Lit, span: Span, field: &str) -> Result {\n match lit {\n syn::Lit::Str(s) => {\n let err = syn::Error::new(\n span,\n format!(\n \"Failed to parse value of `{}` as path: \\\"{}\\\"\",\n field,\n s.value()\n ),\n );\n s.parse::().map_err(|_| err.clone())\n }\n _ => Err(syn::Error::new(\n span,\n format!(\"Failed to parse value of `{field}` as path.\"),\n )),\n }\n}\n\nfn parse_bool(bool: syn::Lit, span: Span, field: &str) -> Result {\n match bool {\n syn::Lit::Bool(b) => Ok(b.value),\n _ => Err(syn::Error::new(\n span,\n format!(\"Failed to parse value of `{field}` as bool.\"),\n )),\n }\n}\n\nfn contains_impl_trait(ty: &syn::Type) -> bool {\n match ty {\n syn::Type::ImplTrait(_) => true,\n syn::Type::Array(t) => contains_impl_trait(&t.elem),\n syn::Type::Ptr(t) => contains_impl_trait(&t.elem),\n syn::Type::Reference(t) => contains_impl_trait(&t.elem),\n syn::Type::Slice(t) => contains_impl_trait(&t.elem),\n syn::Type::Tuple(t) => t.elems.iter().any(contains_impl_trait),\n syn::Type::Paren(t) => contains_impl_trait(&t.elem),\n syn::Type::Group(t) => contains_impl_trait(&t.elem),\n syn::Type::Path(t) => match t.path.segments.last() {\n Some(segment) => match &segment.arguments {\n syn::PathArguments::AngleBracketed(args) => args.args.iter().any(|arg| match arg {\n syn::GenericArgument::Type(t) => contains_impl_trait(t),\n syn::GenericArgument::AssocType(t) => contains_impl_trait(&t.ty),\n _ => false,\n }),\n syn::PathArguments::Parenthesized(args) => {\n args.inputs.iter().any(contains_impl_trait)\n || matches!(&args.output, syn::ReturnType::Type(_, t) if contains_impl_trait(t))\n }\n syn::PathArguments::None => false,\n },\n None => false,\n },\n _ => false,\n }\n}\n\nfn build_config(\n input: &ItemFn,\n args: AttributeArgs,\n is_test: bool,\n rt_multi_thread: bool,\n) -> Result {\n if input.sig.asyncness.is_none() {\n let msg = \"the `async` keyword is missing from the function declaration\";\n return Err(syn::Error::new_spanned(input.sig.fn_token, msg));\n }\n\n let mut config = Configuration::new(is_test, rt_multi_thread);\n let macro_name = config.macro_name();\n\n for arg in args {\n match arg {\n syn::Meta::NameValue(namevalue) => {\n let ident = namevalue\n .path\n .get_ident()\n .ok_or_else(|| {\n syn::Error::new_spanned(&namevalue, \"Must have specified ident\")\n })?\n .to_string()\n .to_lowercase();\n let lit = match &namevalue.value {\n syn::Expr::Lit(syn::ExprLit { lit, .. }) => lit,\n expr => return Err(syn::Error::new_spanned(expr, \"Must be a literal\")),\n };\n match ident.as_str() {\n \"worker_threads\" => {\n config.set_worker_threads(lit.clone(), syn::spanned::Spanned::span(lit))?;\n }\n \"flavor\" => {\n config.set_flavor(lit.clone(), syn::spanned::Spanned::span(lit))?;\n }\n \"start_paused\" => {\n config.set_start_paused(lit.clone(), syn::spanned::Spanned::span(lit))?;\n }\n \"core_threads\" => {\n let msg = \"Attribute `core_threads` is renamed to `worker_threads`\";\n return Err(syn::Error::new_spanned(namevalue, msg));\n }\n \"crate\" => {\n config.set_crate_name(lit.clone(), syn::spanned::Spanned::span(lit))?;\n }\n \"unhandled_panic\" => {\n config\n .set_unhandled_panic(lit.clone(), syn::spanned::Spanned::span(lit))?;\n }\n name => {\n let msg = format!(\n \"Unknown attribute {name} is specified; expected one of: `flavor`, `worker_threads`, `start_paused`, `crate`, `unhandled_panic`\",\n );\n return Err(syn::Error::new_spanned(namevalue, msg));\n }\n }\n }\n syn::Meta::Path(path) => {\n let name = path\n .get_ident()\n .ok_or_else(|| syn::Error::new_spanned(&path, \"Must have specified ident\"))?\n .to_string()\n .to_lowercase();\n let msg = match name.as_str() {\n \"threaded_scheduler\" | \"multi_thread\" => {\n format!(\n \"Set the runtime flavor with #[{macro_name}(flavor = \\\"multi_thread\\\")].\"\n )\n }\n \"basic_scheduler\" | \"current_thread\" | \"single_threaded\" => {\n format!(\n \"Set the runtime flavor with #[{macro_name}(flavor = \\\"current_thread\\\")].\"\n )\n }\n \"flavor\" | \"worker_threads\" | \"start_paused\" | \"crate\" | \"unhandled_panic\" => {\n format!(\"The `{name}` attribute requires an argument.\")\n }\n name => {\n format!(\"Unknown attribute {name} is specified; expected one of: `flavor`, `worker_threads`, `start_paused`, `crate`, `unhandled_panic`.\")\n }\n };\n return Err(syn::Error::new_spanned(path, msg));\n }\n other => {\n return Err(syn::Error::new_spanned(\n other,\n \"Unknown attribute inside the macro\",\n ));\n }\n }\n }\n\n config.build()\n}\n\nfn parse_knobs(mut input: ItemFn, is_test: bool, config: FinalConfig) -> TokenStream {\n input.sig.asyncness = None;\n\n // If type mismatch occurs, the current rustc points to the last statement.\n let (last_stmt_start_span, last_stmt_end_span) = {\n let mut last_stmt = input.stmts.last().cloned().unwrap_or_default().into_iter();\n\n // `Span` on stable Rust has a limitation that only points to the first\n // token, not the whole tokens. We can work around this limitation by\n // using the first/last span of the tokens like\n // `syn::Error::new_spanned` does.\n let start = last_stmt.next().map_or_else(Span::call_site, |t| t.span());\n let end = last_stmt.last().map_or(start, |t| t.span());\n (start, end)\n };\n\n let crate_path = config\n .crate_name\n .map(ToTokens::into_token_stream)\n .unwrap_or_else(|| {\n Ident::new(\"tokio\", Span::call_site().located_at(last_stmt_start_span))\n .into_token_stream()\n });\n\n let use_builder = quote_spanned! {Span::call_site().located_at(last_stmt_start_span)=>\n use #crate_path::runtime::Builder;\n };\n\n let mut rt = match config.flavor {\n RuntimeFlavor::CurrentThread | RuntimeFlavor::Local => {\n quote_spanned! {last_stmt_start_span=>\n Builder::new_current_thread()\n }\n }\n RuntimeFlavor::Threaded => quote_spanned! {last_stmt_start_span=>\n Builder::new_multi_thread()\n },\n };\n\n let mut checks = vec![];\n let mut errors = vec![];\n\n let build = if let RuntimeFlavor::Local = config.flavor {\n checks.push(quote! { tokio_unstable });\n errors.push(\"The local runtime flavor is only available when `tokio_unstable` is set.\");\n quote_spanned! {last_stmt_start_span=> build_local(Default::default())}\n } else {\n quote_spanned! {last_stmt_start_span=> build()}\n };\n\n if let Some(v) = config.worker_threads {\n rt = quote_spanned! {last_stmt_start_span=> #rt.worker_threads(#v) };\n }\n if let Some(v) = config.start_paused {\n rt = quote_spanned! {last_stmt_start_span=> #rt.start_paused(#v) };\n }\n if let Some(v) = config.unhandled_panic {\n let unhandled_panic = v.into_tokens(&crate_path);\n rt = quote_spanned! {last_stmt_start_span=> #rt.unhandled_panic(#unhandled_panic) };\n }\n\n let generated_attrs = if is_test {\n quote! {\n #[::core::prelude::v1::test]\n }\n } else {\n quote! {}\n };\n\n let do_checks: TokenStream = checks\n .iter()\n .zip(&errors)\n .map(|(check, error)| {\n quote! {\n #[cfg(not(#check))]\n compile_error!(#error);\n }\n })\n .collect();\n\n let body_ident = quote! { body };\n // This explicit `return` is intentional. See tokio-rs/tokio#4636\n let last_block = quote_spanned! {last_stmt_end_span=>\n #do_checks\n\n #[cfg(all(#(#checks),*))]\n #[allow(clippy::expect_used, clippy::diverging_sub_expression, clippy::needless_return, clippy::unwrap_in_result)]\n {\n #use_builder\n\n return #rt\n .enable_all()\n .#build\n .expect(\"Failed building the Runtime\")\n .block_on(#body_ident);\n }\n\n #[cfg(not(all(#(#checks),*)))]\n {\n panic!(\"fell through checks\")\n }\n };\n\n let body = input.body();\n\n // For test functions pin the body to the stack and use `Pin<&mut dyn\n // Future>` to reduce the amount of `Runtime::block_on` (and related\n // functions) copies we generate during compilation due to the generic\n // parameter `F` (the future to block on). This could have an impact on\n // performance, but because it's only for testing it's unlikely to be very\n // large.\n //\n // We don't do this for the main function as it should only be used once so\n // there will be no benefit.\n let output_type = match &input.sig.output {\n // For functions with no return value syn doesn't print anything,\n // but that doesn't work as `Output` for our boxed `Future`, so\n // default to `()` (the same type as the function output).\n syn::ReturnType::Default => quote! { () },\n syn::ReturnType::Type(_, ret_type) => quote! { #ret_type },\n };\n\n let body = if is_test {\n quote! {\n let body = async #body;\n #crate_path::pin!(body);\n let body: ::core::pin::Pin<&mut dyn ::core::future::Future> = body;\n }\n } else {\n // force typecheck without runtime overhead\n let check_block = match &input.sig.output {\n syn::ReturnType::Type(_, t)\n if matches!(**t, syn::Type::Never(_)) || contains_impl_trait(t) =>\n {\n quote! {}\n }\n _ => quote! {\n if false {\n let _: &dyn ::core::future::Future = &body;\n }\n },\n };\n\n quote! {\n let body = async #body;\n // Compile-time assertion that the future's output matches the return type.\n let body = {\n #check_block\n body\n };\n }\n };\n\n input.into_tokens(generated_attrs, body, last_block)\n}\n\nfn token_stream_with_error(mut tokens: TokenStream, error: syn::Error) -> TokenStream {\n tokens.extend(error.into_compile_error());\n tokens\n}\n\npub(crate) fn main(args: TokenStream, item: TokenStream, rt_multi_thread: bool) -> TokenStream {\n // If any of the steps for this macro fail, we still want to expand to an item that is as close\n // to the expected output as possible. This helps out IDEs such that completions and other\n // related features keep working.\n let input: ItemFn = match syn::parse2(item.clone()) {\n Ok(it) => it,\n Err(e) => return token_stream_with_error(item, e),\n };\n\n let config = if input.sig.ident == \"main\" && !input.sig.inputs.is_empty() {\n let msg = \"the main function cannot accept arguments\";\n Err(syn::Error::new_spanned(&input.sig.ident, msg))\n } else {\n AttributeArgs::parse_terminated\n .parse2(args)\n .and_then(|args| build_config(&input, args, false, rt_multi_thread))\n };\n\n match config {\n Ok(config) => parse_knobs(input, false, config),\n Err(e) => token_stream_with_error(parse_knobs(input, false, DEFAULT_ERROR_CONFIG), e),\n }\n}\n\n// Check whether given attribute is a test attribute of forms:\n// * `#[test]`\n// * `#[core::prelude::*::test]` or `#[::core::prelude::*::test]`\n// * `#[std::prelude::*::test]` or `#[::std::prelude::*::test]`\nfn is_test_attribute(attr: &Attribute) -> bool {\n let path = match &attr.meta {\n syn::Meta::Path(path) => path,\n _ => return false,\n };\n let candidates = [\n [\"core\", \"prelude\", \"*\", \"test\"],\n [\"std\", \"prelude\", \"*\", \"test\"],\n ];\n if path.leading_colon.is_none()\n && path.segments.len() == 1\n && path.segments[0].arguments.is_none()\n && path.segments[0].ident == \"test\"\n {\n return true;\n } else if path.segments.len() != candidates[0].len() {\n return false;\n }\n candidates.into_iter().any(|segments| {\n path.segments.iter().zip(segments).all(|(segment, path)| {\n segment.arguments.is_none() && (path == \"*\" || segment.ident == path)\n })\n })\n}\n\npub(crate) fn test(args: TokenStream, item: TokenStream, rt_multi_thread: bool) -> TokenStream {\n // If any of the steps for this macro fail, we still want to expand to an item that is as close\n // to the expected output as possible. This helps out IDEs such that completions and other\n // related features keep working.\n let input: ItemFn = match syn::parse2(item.clone()) {\n Ok(it) => it,\n Err(e) => return token_stream_with_error(item, e),\n };\n let config = if let Some(attr) = input.attrs().find(|attr| is_test_attribute(attr)) {\n let msg = \"second test attribute is supplied, consider removing or changing the order of your test attributes\";\n Err(syn::Error::new_spanned(attr, msg))\n } else {\n AttributeArgs::parse_terminated\n .parse2(args)\n .and_then(|args| build_config(&input, args, true, rt_multi_thread))\n };\n\n match config {\n Ok(config) => parse_knobs(input, true, config),\n Err(e) => token_stream_with_error(parse_knobs(input, true, DEFAULT_ERROR_CONFIG), e),\n }\n}\n\nstruct ItemFn {\n outer_attrs: Vec,\n vis: Visibility,\n sig: Signature,\n brace_token: syn::token::Brace,\n inner_attrs: Vec,\n stmts: Vec,\n}\n\nimpl ItemFn {\n /// Access all attributes of the function item.\n fn attrs(&self) -> impl Iterator {\n self.outer_attrs.iter().chain(self.inner_attrs.iter())\n }\n\n /// Get the body of the function item in a manner so that it can be\n /// conveniently used with the `quote!` macro.\n fn body(&self) -> Body<'_> {\n Body {\n brace_token: self.brace_token,\n stmts: &self.stmts,\n }\n }\n\n /// Convert our local function item into a token stream.\n fn into_tokens(\n self,\n generated_attrs: proc_macro2::TokenStream,\n body: proc_macro2::TokenStream,\n last_block: proc_macro2::TokenStream,\n ) -> TokenStream {\n let mut tokens = proc_macro2::TokenStream::new();\n // Outer attributes are simply streamed as-is.\n for attr in self.outer_attrs {\n attr.to_tokens(&mut tokens);\n }\n\n // Inner attributes require extra care, since they're not supported on\n // blocks (which is what we're expanded into) we instead lift them\n // outside of the function. This matches the behavior of `syn`.\n for mut attr in self.inner_attrs {\n attr.style = syn::AttrStyle::Outer;\n attr.to_tokens(&mut tokens);\n }\n\n // Add generated macros at the end, so macros processed later are aware of them.\n generated_attrs.to_tokens(&mut tokens);\n\n self.vis.to_tokens(&mut tokens);\n self.sig.to_tokens(&mut tokens);\n\n self.brace_token.surround(&mut tokens, |tokens| {\n body.to_tokens(tokens);\n last_block.to_tokens(tokens);\n });\n\n tokens\n }\n}\n\nimpl Parse for ItemFn {\n #[inline]\n fn parse(input: ParseStream<'_>) -> syn::Result {\n // This parse implementation has been largely lifted from `syn`, with\n // the exception of:\n // * We don't have access to the plumbing necessary to parse inner\n // attributes in-place.\n // * We do our own statements parsing to avoid recursively parsing\n // entire statements and only look for the parts we're interested in.\n\n let outer_attrs = input.call(Attribute::parse_outer)?;\n let vis: Visibility = input.parse()?;\n let sig: Signature = input.parse()?;\n\n let content;\n let brace_token = braced!(content in input);\n let inner_attrs = Attribute::parse_inner(&content)?;\n\n let mut buf = proc_macro2::TokenStream::new();\n let mut stmts = Vec::new();\n\n while !content.is_empty() {\n if let Some(semi) = content.parse::>()? {\n semi.to_tokens(&mut buf);\n stmts.push(buf);\n buf = proc_macro2::TokenStream::new();\n continue;\n }\n\n // Parse a single token tree and extend our current buffer with it.\n // This avoids parsing the entire content of the sub-tree.\n buf.extend([content.parse::()?]);\n }\n\n if !buf.is_empty() {\n stmts.push(buf);\n }\n\n Ok(Self {\n outer_attrs,\n vis,\n sig,\n brace_token,\n inner_attrs,\n stmts,\n })\n }\n}\n\nstruct Body<'a> {\n brace_token: syn::token::Brace,\n // Statements, with terminating `;`.\n stmts: &'a [TokenStream],\n}\n\nimpl ToTokens for Body<'_> {\n fn to_tokens(&self, tokens: &mut proc_macro2::TokenStream) {\n self.brace_token.surround(tokens, |tokens| {\n for stmt in self.stmts {\n stmt.to_tokens(tokens);\n }\n });\n }\n}\n" }, { "path": "tokio-macros/src/lib.rs", "content": "#![allow(clippy::needless_doctest_main)]\n#![warn(\n missing_debug_implementations,\n missing_docs,\n rust_2018_idioms,\n unreachable_pub\n)]\n#![doc(test(\n no_crate_inject,\n attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))\n))]\n\n//! Macros for use with Tokio\n\nmod entry;\nmod select;\n\nuse proc_macro::TokenStream;\n\n/// Marks async function to be executed by the selected runtime. This macro\n/// helps set up a `Runtime` without requiring the user to use\n/// [Runtime](../tokio/runtime/struct.Runtime.html) or\n/// [Builder](../tokio/runtime/struct.Builder.html) directly.\n///\n/// Note: This macro is designed to be simplistic and targets applications that\n/// do not require a complex setup. If the provided functionality is not\n/// sufficient, you may be interested in using\n/// [Builder](../tokio/runtime/struct.Builder.html), which provides a more\n/// powerful interface.\n///\n/// Note: This macro can be used on any function and not just the `main`\n/// function. Using it on a non-main function makes the function behave as if it\n/// was synchronous by starting a new runtime each time it is called. If the\n/// function is called often, it is preferable to create the runtime using the\n/// runtime builder so the runtime can be reused across calls.\n///\n/// # Non-worker async function\n///\n/// Note that the async function marked with this macro does not run as a\n/// worker. The expectation is that other tasks are spawned by the function here.\n/// Awaiting on other futures from the function provided here will not\n/// perform as fast as those spawned as workers.\n///\n/// # Runtime flavors\n///\n/// The macro can be configured with a `flavor` parameter to select\n/// different runtime configurations.\n///\n/// ## Multi-threaded\n///\n/// To use the multi-threaded runtime, the macro can be configured using\n///\n/// ```\n/// #[tokio::main(flavor = \"multi_thread\", worker_threads = 10)]\n/// # async fn main() {}\n/// ```\n///\n/// The `worker_threads` option configures the number of worker threads, and\n/// defaults to the number of cpus on the system. This is the default flavor.\n///\n/// Note: The multi-threaded runtime requires the `rt-multi-thread` feature\n/// flag.\n///\n/// ## Current-thread\n///\n/// To use the single-threaded runtime known as the `current_thread` runtime,\n/// the macro can be configured using\n///\n/// ```rust\n/// #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {}\n/// ```\n///\n/// ## Local\n///\n/// [Unstable API][unstable] only.\n///\n/// To use the [local runtime], the macro can be configured using\n///\n/// ```rust\n/// # #[cfg(tokio_unstable)]\n/// #[tokio::main(flavor = \"local\")]\n/// # async fn main() {}\n/// # #[cfg(not(tokio_unstable))]\n/// # fn main() {}\n/// ```\n///\n/// # Function arguments\n///\n/// Arguments are allowed for any functions, aside from `main` which is special.\n///\n/// # Usage\n///\n/// ## Using the multi-threaded runtime\n///\n/// ```rust\n/// #[tokio::main]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// fn main() {\n/// tokio::runtime::Builder::new_multi_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// ## Using the current-thread runtime\n///\n/// The basic scheduler is single-threaded.\n///\n/// ```rust\n/// #[tokio::main(flavor = \"current_thread\")]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// fn main() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// ## Using the local runtime\n///\n/// Available in the [unstable API][unstable] only.\n///\n/// The [local runtime] is similar to the current-thread runtime but\n/// supports [`task::spawn_local`](../tokio/task/fn.spawn_local.html).\n///\n/// ```rust\n/// # #[cfg(tokio_unstable)]\n/// #[tokio::main(flavor = \"local\")]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// # #[cfg(not(tokio_unstable))]\n/// # fn main() {}\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// # #[cfg(tokio_unstable)]\n/// fn main() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .build_local(tokio::runtime::LocalOptions::default())\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// # #[cfg(not(tokio_unstable))]\n/// # fn main() {}\n/// ```\n///\n///\n/// ## Set number of worker threads\n///\n/// ```rust\n/// #[tokio::main(worker_threads = 2)]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// fn main() {\n/// tokio::runtime::Builder::new_multi_thread()\n/// .worker_threads(2)\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// ## Configure the runtime to start with time paused\n///\n/// ```rust\n/// #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// fn main() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .start_paused(true)\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// Note that `start_paused` requires the `test-util` feature to be enabled.\n///\n/// ## Rename package\n///\n/// ```rust\n/// use tokio as tokio1;\n///\n/// #[tokio1::main(crate = \"tokio1\")]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// use tokio as tokio1;\n///\n/// fn main() {\n/// tokio1::runtime::Builder::new_multi_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// ## Configure unhandled panic behavior\n///\n/// Available options are `shutdown_runtime` and `ignore`. For more details, see\n/// [`Builder::unhandled_panic`].\n///\n/// This option is only compatible with the `current_thread` runtime.\n///\n/// ```no_run\n/// #[cfg(tokio_unstable)]\n/// #[tokio::main(flavor = \"current_thread\", unhandled_panic = \"shutdown_runtime\")]\n/// async fn main() {\n/// let _ = tokio::spawn(async {\n/// panic!(\"This panic will shutdown the runtime.\");\n/// }).await;\n/// }\n/// # #[cfg(not(tokio_unstable))]\n/// # fn main() { }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```no_run\n/// #[cfg(tokio_unstable)]\n/// fn main() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .unhandled_panic(tokio::runtime::UnhandledPanic::ShutdownRuntime)\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// let _ = tokio::spawn(async {\n/// panic!(\"This panic will shutdown the runtime.\");\n/// }).await;\n/// })\n/// }\n/// # #[cfg(not(tokio_unstable))]\n/// # fn main() { }\n/// ```\n///\n/// **Note**: This option depends on Tokio's [unstable API][unstable]. See [the\n/// documentation on unstable features][unstable] for details on how to enable\n/// Tokio's unstable features.\n///\n/// [`Builder::unhandled_panic`]: ../tokio/runtime/struct.Builder.html#method.unhandled_panic\n/// [unstable]: ../tokio/index.html#unstable-features\n/// [local runtime]: ../tokio/runtime/struct.LocalRuntime.html\n#[proc_macro_attribute]\npub fn main(args: TokenStream, item: TokenStream) -> TokenStream {\n entry::main(args.into(), item.into(), true).into()\n}\n\n/// Marks async function to be executed by selected runtime. This macro helps set up a `Runtime`\n/// without requiring the user to use [Runtime](../tokio/runtime/struct.Runtime.html) or\n/// [Builder](../tokio/runtime/struct.Builder.html) directly.\n///\n/// ## Function arguments:\n///\n/// Arguments are allowed for any functions aside from `main` which is special\n///\n/// ## Usage\n///\n/// ### Using default\n///\n/// ```rust\n/// #[tokio::main(flavor = \"current_thread\")]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// fn main() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n///\n/// ### Rename package\n///\n/// ```rust\n/// use tokio as tokio1;\n///\n/// #[tokio1::main(crate = \"tokio1\")]\n/// async fn main() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::main]`\n///\n/// ```rust\n/// use tokio as tokio1;\n///\n/// fn main() {\n/// tokio1::runtime::Builder::new_multi_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// println!(\"Hello world\");\n/// })\n/// }\n/// ```\n#[proc_macro_attribute]\npub fn main_rt(args: TokenStream, item: TokenStream) -> TokenStream {\n entry::main(args.into(), item.into(), false).into()\n}\n\n/// Marks async function to be executed by runtime, suitable to test environment.\n/// This macro helps set up a `Runtime` without requiring the user to use\n/// [Runtime](../tokio/runtime/struct.Runtime.html) or\n/// [Builder](../tokio/runtime/struct.Builder.html) directly.\n///\n/// Note: This macro is designed to be simplistic and targets applications that\n/// do not require a complex setup. If the provided functionality is not\n/// sufficient, you may be interested in using\n/// [Builder](../tokio/runtime/struct.Builder.html), which provides a more\n/// powerful interface.\n///\n/// # Multi-threaded runtime\n///\n/// To use the multi-threaded runtime, the macro can be configured using\n///\n/// ```no_run\n/// #[tokio::test(flavor = \"multi_thread\", worker_threads = 1)]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// The `worker_threads` option configures the number of worker threads, and\n/// defaults to the number of cpus on the system.\n///\n/// Note: The multi-threaded runtime requires the `rt-multi-thread` feature\n/// flag.\n///\n/// # Current thread runtime\n///\n/// The default test runtime is single-threaded. Each test gets a\n/// separate current-thread runtime.\n///\n/// ```no_run\n/// #[tokio::test]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// ## Usage\n///\n/// ### Using the multi-thread runtime\n///\n/// ```no_run\n/// #[tokio::test(flavor = \"multi_thread\")]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::test]`\n///\n/// ```no_run\n/// #[test]\n/// fn my_test() {\n/// tokio::runtime::Builder::new_multi_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// assert!(true);\n/// })\n/// }\n/// ```\n///\n/// ### Using current thread runtime\n///\n/// ```no_run\n/// #[tokio::test]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::test]`\n///\n/// ```no_run\n/// #[test]\n/// fn my_test() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// assert!(true);\n/// })\n/// }\n/// ```\n///\n/// ### Set number of worker threads\n///\n/// ```no_run\n/// #[tokio::test(flavor = \"multi_thread\", worker_threads = 2)]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::test]`\n///\n/// ```no_run\n/// #[test]\n/// fn my_test() {\n/// tokio::runtime::Builder::new_multi_thread()\n/// .worker_threads(2)\n/// .enable_all()\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// assert!(true);\n/// })\n/// }\n/// ```\n///\n/// ### Configure the runtime to start with time paused\n///\n/// ```no_run\n/// #[tokio::test(start_paused = true)]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n///\n/// Equivalent code not using `#[tokio::test]`\n///\n/// ```no_run\n/// #[test]\n/// fn my_test() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .start_paused(true)\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// assert!(true);\n/// })\n/// }\n/// ```\n///\n/// Note that `start_paused` requires the `test-util` feature to be enabled.\n///\n/// ### Rename package\n///\n/// ```rust\n/// use tokio as tokio1;\n///\n/// #[tokio1::test(crate = \"tokio1\")]\n/// async fn my_test() {\n/// println!(\"Hello world\");\n/// }\n/// ```\n///\n/// ### Configure unhandled panic behavior\n///\n/// Available options are `shutdown_runtime` and `ignore`. For more details, see\n/// [`Builder::unhandled_panic`].\n///\n/// This option is only compatible with the `current_thread` runtime.\n///\n/// ```no_run\n/// #[cfg(tokio_unstable)]\n/// #[tokio::test(flavor = \"current_thread\", unhandled_panic = \"shutdown_runtime\")]\n/// async fn my_test() {\n/// let _ = tokio::spawn(async {\n/// panic!(\"This panic will shutdown the runtime.\");\n/// }).await;\n/// }\n///\n/// # fn main() { }\n/// ```\n///\n/// Equivalent code not using `#[tokio::test]`\n///\n/// ```no_run\n/// #[cfg(tokio_unstable)]\n/// #[test]\n/// fn my_test() {\n/// tokio::runtime::Builder::new_current_thread()\n/// .enable_all()\n/// .unhandled_panic(UnhandledPanic::ShutdownRuntime)\n/// .build()\n/// .unwrap()\n/// .block_on(async {\n/// let _ = tokio::spawn(async {\n/// panic!(\"This panic will shutdown the runtime.\");\n/// }).await;\n/// })\n/// }\n///\n/// # fn main() { }\n/// ```\n///\n/// **Note**: This option depends on Tokio's [unstable API][unstable]. See [the\n/// documentation on unstable features][unstable] for details on how to enable\n/// Tokio's unstable features.\n///\n/// [`Builder::unhandled_panic`]: ../tokio/runtime/struct.Builder.html#method.unhandled_panic\n/// [unstable]: ../tokio/index.html#unstable-features\n#[proc_macro_attribute]\npub fn test(args: TokenStream, item: TokenStream) -> TokenStream {\n entry::test(args.into(), item.into(), true).into()\n}\n\n/// Marks async function to be executed by runtime, suitable to test environment\n///\n/// ## Usage\n///\n/// ```no_run\n/// #[tokio::test]\n/// async fn my_test() {\n/// assert!(true);\n/// }\n/// ```\n#[proc_macro_attribute]\npub fn test_rt(args: TokenStream, item: TokenStream) -> TokenStream {\n entry::test(args.into(), item.into(), false).into()\n}\n\n/// Always fails with the error message below.\n/// ```text\n/// The #[tokio::main] macro requires rt or rt-multi-thread.\n/// ```\n#[proc_macro_attribute]\npub fn main_fail(_args: TokenStream, _item: TokenStream) -> TokenStream {\n syn::Error::new(\n proc_macro2::Span::call_site(),\n \"The #[tokio::main] macro requires rt or rt-multi-thread.\",\n )\n .to_compile_error()\n .into()\n}\n\n/// Always fails with the error message below.\n/// ```text\n/// The #[tokio::test] macro requires rt or rt-multi-thread.\n/// ```\n#[proc_macro_attribute]\npub fn test_fail(_args: TokenStream, _item: TokenStream) -> TokenStream {\n syn::Error::new(\n proc_macro2::Span::call_site(),\n \"The #[tokio::test] macro requires rt or rt-multi-thread.\",\n )\n .to_compile_error()\n .into()\n}\n\n/// Implementation detail of the `select!` macro. This macro is **not** intended\n/// to be used as part of the public API and is permitted to change.\n#[proc_macro]\n#[doc(hidden)]\npub fn select_priv_declare_output_enum(input: TokenStream) -> TokenStream {\n select::declare_output_enum(input)\n}\n\n/// Implementation detail of the `select!` macro. This macro is **not** intended\n/// to be used as part of the public API and is permitted to change.\n#[proc_macro]\n#[doc(hidden)]\npub fn select_priv_clean_pattern(input: TokenStream) -> TokenStream {\n select::clean_pattern_macro(input)\n}\n" }, { "path": "tokio-macros/src/select.rs", "content": "use proc_macro::{TokenStream, TokenTree};\nuse proc_macro2::Span;\nuse quote::quote;\nuse syn::{parse::Parser, Ident};\n\npub(crate) fn declare_output_enum(input: TokenStream) -> TokenStream {\n // passed in is: `(_ _ _)` with one `_` per branch\n let branches = match input.into_iter().next() {\n Some(TokenTree::Group(group)) => group.stream().into_iter().count(),\n _ => panic!(\"unexpected macro input\"),\n };\n\n let variants = (0..branches)\n .map(|num| Ident::new(&format!(\"_{num}\"), Span::call_site()))\n .collect::>();\n\n // Use a bitfield to track which futures completed\n let mask = Ident::new(\n if branches <= 8 {\n \"u8\"\n } else if branches <= 16 {\n \"u16\"\n } else if branches <= 32 {\n \"u32\"\n } else if branches <= 64 {\n \"u64\"\n } else {\n panic!(\"up to 64 branches supported\");\n },\n Span::call_site(),\n );\n\n TokenStream::from(quote! {\n pub(super) enum Out<#( #variants ),*> {\n #( #variants(#variants), )*\n // Include a `Disabled` variant signifying that all select branches\n // failed to resolve.\n Disabled,\n }\n\n pub(super) type Mask = #mask;\n })\n}\n\npub(crate) fn clean_pattern_macro(input: TokenStream) -> TokenStream {\n // If this isn't a pattern, we return the token stream as-is. The select!\n // macro is using it in a location requiring a pattern, so an error will be\n // emitted there.\n let mut input: syn::Pat = match syn::Pat::parse_single.parse(input.clone()) {\n Ok(it) => it,\n Err(_) => return input,\n };\n\n clean_pattern(&mut input);\n quote::ToTokens::into_token_stream(input).into()\n}\n\n// Removes any occurrences of ref or mut in the provided pattern.\nfn clean_pattern(pat: &mut syn::Pat) {\n match pat {\n syn::Pat::Lit(_literal) => {}\n syn::Pat::Macro(_macro) => {}\n syn::Pat::Path(_path) => {}\n syn::Pat::Range(_range) => {}\n syn::Pat::Rest(_rest) => {}\n syn::Pat::Verbatim(_tokens) => {}\n syn::Pat::Wild(_underscore) => {}\n syn::Pat::Ident(ident) => {\n ident.by_ref = None;\n ident.mutability = None;\n if let Some((_at, pat)) = &mut ident.subpat {\n clean_pattern(&mut *pat);\n }\n }\n syn::Pat::Or(or) => {\n for case in &mut or.cases {\n clean_pattern(case);\n }\n }\n syn::Pat::Slice(slice) => {\n for elem in &mut slice.elems {\n clean_pattern(elem);\n }\n }\n syn::Pat::Struct(struct_pat) => {\n for field in &mut struct_pat.fields {\n clean_pattern(&mut field.pat);\n }\n }\n syn::Pat::Tuple(tuple) => {\n for elem in &mut tuple.elems {\n clean_pattern(elem);\n }\n }\n syn::Pat::TupleStruct(tuple) => {\n for elem in &mut tuple.elems {\n clean_pattern(elem);\n }\n }\n syn::Pat::Reference(reference) => {\n reference.mutability = None;\n clean_pattern(&mut reference.pat);\n }\n syn::Pat::Type(type_pat) => {\n clean_pattern(&mut type_pat.pat);\n }\n _ => {}\n }\n}\n" }, { "path": "tokio-stream/CHANGELOG.md", "content": "# 0.1.18 (January 4th, 2026)\n\n### Added\n\n- stream: add `ChunksTimeout::into_remainder` ([#7715])\n- stream: add examples to wrapper types ([#7024])\n- sync: implement `Stream::size_hint` for `ReceiverStream` and `UnboundedReceiverStream` ([#7492])\n\n### Fixed\n\n- stream: work around the rustc bug in `StreamExt::collect` ([#7754])\n\n### Documented\n\n- stream: improve the the docs of `TcpListenerStream` ([#7578])\n\n[#7024]: https://github.com/tokio-rs/tokio/pull/7024\n[#7492]: https://github.com/tokio-rs/tokio/pull/7492\n[#7578]: https://github.com/tokio-rs/tokio/pull/7578\n[#7715]: https://github.com/tokio-rs/tokio/pull/7715\n[#7754]: https://github.com/tokio-rs/tokio/pull/7754\n\n# 0.1.17 (December 6th, 2024)\n\n- deps: fix dev-dependency on tokio-test ([#6931], [#7019])\n- stream: fix link on `Peekable` ([#6861])\n- sync: fix `Stream` link in broadcast docs ([#6873])\n\n[#6861]: https://github.com/tokio-rs/tokio/pull/6861\n[#6873]: https://github.com/tokio-rs/tokio/pull/6873\n[#6931]: https://github.com/tokio-rs/tokio/pull/6931\n[#7019]: https://github.com/tokio-rs/tokio/pull/7019\n\n# 0.1.16 (September 5th, 2024)\n\nThis release bumps the MSRV of tokio-stream to 1.70.\n\n- stream: add `next_many` and `poll_next_many` to `StreamMap` ([#6409])\n- stream: make stream adapters public ([#6658])\n- readme: add readme for tokio-stream ([#6456])\n\n[#6409]: https://github.com/tokio-rs/tokio/pull/6409\n[#6658]: https://github.com/tokio-rs/tokio/pull/6658\n[#6456]: https://github.com/tokio-rs/tokio/pull/6456\n\n# 0.1.15 (March 14th, 2024)\n\nThis release bumps the MSRV of tokio-stream to 1.63.\n\n- docs: fix typo in argument name ([#6389])\n- docs: fix typo in peekable docs ([#6130])\n- docs: link to latest version of tokio-util docs ([#5694])\n- docs: typographic improvements ([#6262])\n- stream: add `StreamExt::peekable` ([#6095])\n\n[#5694]: https://github.com/tokio-rs/tokio/pull/5694\n[#6095]: https://github.com/tokio-rs/tokio/pull/6095\n[#6130]: https://github.com/tokio-rs/tokio/pull/6130\n[#6262]: https://github.com/tokio-rs/tokio/pull/6262\n[#6389]: https://github.com/tokio-rs/tokio/pull/6389\n\n# 0.1.14 (April 26th, 2023)\n\nThis bugfix release bumps the minimum version of Tokio to 1.15, which is\nnecessary for `timeout_repeating` to compile. ([#5657])\n\n[#5657]: https://github.com/tokio-rs/tokio/pull/5657\n\n# 0.1.13 (April 25th, 2023)\n\nThis release bumps the MSRV of tokio-stream to 1.56.\n\n- stream: add \"full\" feature flag ([#5639])\n- stream: add `StreamExt::timeout_repeating` ([#5577])\n- stream: add `StreamNotifyClose` ([#4851])\n\n[#4851]: https://github.com/tokio-rs/tokio/pull/4851\n[#5577]: https://github.com/tokio-rs/tokio/pull/5577\n[#5639]: https://github.com/tokio-rs/tokio/pull/5639\n\n# 0.1.12 (January 20, 2023)\n\n- time: remove `Unpin` bound on `Throttle` methods ([#5105])\n- time: document that `throttle` operates on ms granularity ([#5101])\n- sync: add `WatchStream::from_changes` ([#5432])\n\n[#5105]: https://github.com/tokio-rs/tokio/pull/5105\n[#5101]: https://github.com/tokio-rs/tokio/pull/5101\n[#5432]: https://github.com/tokio-rs/tokio/pull/5432\n\n# 0.1.11 (October 11, 2022)\n\n- time: allow `StreamExt::chunks_timeout` outside of a runtime ([#5036])\n\n[#5036]: https://github.com/tokio-rs/tokio/pull/5036\n\n# 0.1.10 (Sept 18, 2022)\n\n- time: add `StreamExt::chunks_timeout` ([#4695])\n- stream: add track_caller to public APIs ([#4786])\n\n[#4695]: https://github.com/tokio-rs/tokio/pull/4695\n[#4786]: https://github.com/tokio-rs/tokio/pull/4786\n\n# 0.1.9 (June 4, 2022)\n\n- deps: upgrade `tokio-util` dependency to `0.7.x` ([#3762])\n- stream: add `StreamExt::map_while` ([#4351])\n- stream: add `StreamExt::then` ([#4355])\n- stream: add cancel-safety docs to `StreamExt::next` and `try_next` ([#4715])\n- stream: expose `Elapsed` error ([#4502])\n- stream: expose `Timeout` ([#4601])\n- stream: implement `Extend` for `StreamMap` ([#4272])\n- sync: add `Clone` to `RecvError` types ([#4560])\n\n[#3762]: https://github.com/tokio-rs/tokio/pull/3762\n[#4272]: https://github.com/tokio-rs/tokio/pull/4272\n[#4351]: https://github.com/tokio-rs/tokio/pull/4351\n[#4355]: https://github.com/tokio-rs/tokio/pull/4355\n[#4502]: https://github.com/tokio-rs/tokio/pull/4502\n[#4560]: https://github.com/tokio-rs/tokio/pull/4560\n[#4601]: https://github.com/tokio-rs/tokio/pull/4601\n[#4715]: https://github.com/tokio-rs/tokio/pull/4715\n\n# 0.1.8 (October 29, 2021)\n\n- stream: add `From>` impl for receiver streams ([#4080])\n- stream: impl `FromIterator` for `StreamMap` ([#4052])\n- signal: make windows docs for signal module show up on unix builds ([#3770])\n\n[#3770]: https://github.com/tokio-rs/tokio/pull/3770\n[#4052]: https://github.com/tokio-rs/tokio/pull/4052\n[#4080]: https://github.com/tokio-rs/tokio/pull/4080\n\n# 0.1.7 (July 7, 2021)\n\n### Fixed\n\n- sync: fix watch wrapper ([#3914])\n- time: fix `Timeout::size_hint` ([#3902])\n\n[#3902]: https://github.com/tokio-rs/tokio/pull/3902\n[#3914]: https://github.com/tokio-rs/tokio/pull/3914\n\n# 0.1.6 (May 14, 2021)\n\n### Added\n\n- stream: implement `Error` and `Display` for `BroadcastStreamRecvError` ([#3745])\n\n### Fixed\n\n- stream: avoid yielding in `AllFuture` and `AnyFuture` ([#3625])\n\n[#3745]: https://github.com/tokio-rs/tokio/pull/3745\n[#3625]: https://github.com/tokio-rs/tokio/pull/3625\n\n# 0.1.5 (March 20, 2021)\n\n### Fixed\n\n- stream: documentation note for throttle `Unpin` ([#3600])\n\n[#3600]: https://github.com/tokio-rs/tokio/pull/3600\n\n# 0.1.4 (March 9, 2021)\n\nAdded\n\n- signal: add `Signal` wrapper ([#3510])\n\nFixed\n\n- stream: remove duplicate `doc_cfg` declaration ([#3561])\n- sync: yield initial value in `WatchStream` ([#3576])\n\n[#3510]: https://github.com/tokio-rs/tokio/pull/3510\n[#3561]: https://github.com/tokio-rs/tokio/pull/3561\n[#3576]: https://github.com/tokio-rs/tokio/pull/3576\n\n# 0.1.3 (February 5, 2021)\n\nAdded\n\n - sync: add wrapper for broadcast and watch ([#3384], [#3504])\n\n[#3384]: https://github.com/tokio-rs/tokio/pull/3384\n[#3504]: https://github.com/tokio-rs/tokio/pull/3504\n\n# 0.1.2 (January 12, 2021)\n\nFixed\n\n - docs: fix some wrappers missing in documentation ([#3378])\n\n[#3378]: https://github.com/tokio-rs/tokio/pull/3378\n\n# 0.1.1 (January 4, 2021)\n\nAdded\n\n - add `Stream` wrappers ([#3343])\n\nFixed\n\n - move `async-stream` to `dev-dependencies` ([#3366])\n\n[#3366]: https://github.com/tokio-rs/tokio/pull/3366\n[#3343]: https://github.com/tokio-rs/tokio/pull/3343\n\n# 0.1.0 (December 23, 2020)\n\n - Initial release\n" }, { "path": "tokio-stream/Cargo.toml", "content": "[package]\nname = \"tokio-stream\"\n# When releasing to crates.io:\n# - Remove path dependencies (if any)\n# - Update CHANGELOG.md.\n# - Create \"tokio-stream-0.1.x\" git tag.\nversion = \"0.1.18\"\nedition = \"2021\"\nrust-version = \"1.71\"\nauthors = [\"Tokio Contributors \"]\nlicense = \"MIT\"\nrepository = \"https://github.com/tokio-rs/tokio\"\nhomepage = \"https://tokio.rs\"\ndescription = \"\"\"\nUtilities to work with `Stream` and `tokio`.\n\"\"\"\ncategories = [\"asynchronous\"]\n\n[features]\ndefault = [\"time\"]\n\nfull = [\n \"time\",\n \"net\",\n \"io-util\",\n \"fs\",\n \"sync\",\n \"signal\"\n]\n\ntime = [\"tokio/time\"]\nnet = [\"tokio/net\"]\nio-util = [\"tokio/io-util\"]\nfs = [\"tokio/fs\"]\nsync = [\"tokio/sync\", \"tokio-util\"]\nsignal = [\"tokio/signal\"]\n\n[dependencies]\nfutures-core = { version = \"0.3.0\" }\npin-project-lite = \"0.2.11\"\ntokio = { version = \"1.38.0\", features = [\"sync\"] }\ntokio-util = { version = \"0.7.0\", optional = true }\n\n[dev-dependencies]\ntokio = { version = \"1.38.0\", features = [\"full\", \"test-util\"] }\nasync-stream = \"0.3\"\nparking_lot = \"0.12.0\"\ntokio-test = \"0.4\"\nfutures = { version = \"0.3\", default-features = false }\n\n[package.metadata.docs.rs]\nall-features = true\nrustdoc-args = [\"--cfg\", \"docsrs\"]\n# Issue #3770\n#\n# This should allow `docsrs` to be read across projects, so that `tokio-stream`\n# can pick up stubbed types exported by `tokio`.\nrustc-args = [\"--cfg\", \"docsrs\"]\n\n[lints]\nworkspace = true\n" }, { "path": "tokio-stream/LICENSE", "content": "MIT License\n\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "tokio-stream/README.md", "content": "# tokio-stream\n\nUtilities to work with `Stream` and `tokio`.\n\n## License\n\nThis project is licensed under the [MIT license](LICENSE).\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you, shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "tokio-stream/fuzz/.gitignore", "content": "target\ncorpus\nartifacts\ncoverage\n" }, { "path": "tokio-stream/fuzz/Cargo.toml", "content": "[package]\nname = \"tokio-stream-fuzz\"\nversion = \"0.0.0\"\npublish = false\nedition = \"2021\"\n\n[package.metadata]\ncargo-fuzz = true\n\n[dependencies]\nlibfuzzer-sys = \"0.4\"\ntokio-test = { path = \"../../tokio-test\" }\n\n[dependencies.tokio-stream]\npath = \"..\"\n\n\n# Prevent this from interfering with workspaces\n[workspace]\nmembers = [\".\"]\n\n[profile.release]\ndebug = 1\n\n[[bin]]\nname = \"fuzz_stream_map\"\npath = \"fuzz_targets/fuzz_stream_map.rs\"\ntest = false\ndoc = false\n" }, { "path": "tokio-stream/fuzz/fuzz_targets/fuzz_stream_map.rs", "content": "#![no_main]\n\nuse libfuzzer_sys::fuzz_target;\nuse std::pin::Pin;\n\nuse tokio_stream::{self as stream, Stream, StreamMap};\nuse tokio_test::{assert_pending, assert_ready, task};\n\nmacro_rules! assert_ready_none {\n ($($t:tt)*) => {\n match assert_ready!($($t)*) {\n None => {}\n Some(v) => panic!(\"expected `None`, got `Some({:?})`\", v),\n }\n };\n}\n\nfn pin_box + 'static, U>(s: T) -> Pin>> {\n Box::pin(s)\n}\n\nfuzz_target!(|data: [bool; 64]| {\n use std::task::{Context, Poll};\n\n struct DidPoll {\n did_poll: bool,\n inner: T,\n }\n\n impl Stream for DidPoll {\n type Item = T::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.did_poll = true;\n Pin::new(&mut self.inner).poll_next(cx)\n }\n }\n\n // Try the test with each possible length.\n for len in 0..data.len() {\n let mut map = task::spawn(StreamMap::new());\n let mut expect = 0;\n\n for (i, is_empty) in data[..len].iter().copied().enumerate() {\n let inner = if is_empty {\n pin_box(stream::empty::<()>())\n } else {\n expect += 1;\n pin_box(stream::pending::<()>())\n };\n\n let stream = DidPoll {\n did_poll: false,\n inner,\n };\n\n map.insert(i, stream);\n }\n\n if expect == 0 {\n assert_ready_none!(map.poll_next());\n } else {\n assert_pending!(map.poll_next());\n\n assert_eq!(expect, map.values().count());\n\n for stream in map.values() {\n assert!(stream.did_poll);\n }\n }\n }\n});\n" }, { "path": "tokio-stream/src/empty.rs", "content": "use crate::Stream;\n\nuse core::marker::PhantomData;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\n/// Stream for the [`empty`](fn@empty) function.\n#[derive(Debug)]\n#[must_use = \"streams do nothing unless polled\"]\npub struct Empty(PhantomData);\n\nimpl Unpin for Empty {}\nunsafe impl Send for Empty {}\nunsafe impl Sync for Empty {}\n\n/// Creates a stream that yields nothing.\n///\n/// The returned stream is immediately ready and returns `None`. Use\n/// [`stream::pending()`](super::pending()) to obtain a stream that is never\n/// ready.\n///\n/// # Examples\n///\n/// Basic usage:\n///\n/// ```\n/// use tokio_stream::{self as stream, StreamExt};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut none = stream::empty::();\n///\n/// assert_eq!(None, none.next().await);\n/// # }\n/// ```\npub const fn empty() -> Empty {\n Empty(PhantomData)\n}\n\nimpl Stream for Empty {\n type Item = T;\n\n fn poll_next(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Ready(None)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (0, Some(0))\n }\n}\n" }, { "path": "tokio-stream/src/iter.rs", "content": "use crate::Stream;\n\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\n/// Stream for the [`iter`](fn@iter) function.\n#[derive(Debug)]\n#[must_use = \"streams do nothing unless polled\"]\npub struct Iter {\n iter: I,\n yield_amt: usize,\n}\n\nimpl Unpin for Iter {}\n\n/// Converts an `Iterator` into a `Stream` which is always ready\n/// to yield the next value.\n///\n/// Iterators in Rust don't express the ability to block, so this adapter\n/// simply always calls `iter.next()` and returns that.\n///\n/// ```\n/// # async fn dox() {\n/// use tokio_stream::{self as stream, StreamExt};\n///\n/// let mut stream = stream::iter(vec![17, 19]);\n///\n/// assert_eq!(stream.next().await, Some(17));\n/// assert_eq!(stream.next().await, Some(19));\n/// assert_eq!(stream.next().await, None);\n/// # }\n/// ```\npub fn iter(i: I) -> Iter\nwhere\n I: IntoIterator,\n{\n Iter {\n iter: i.into_iter(),\n yield_amt: 0,\n }\n}\n\nimpl Stream for Iter\nwhere\n I: Iterator,\n{\n type Item = I::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n // TODO: add coop back\n if self.yield_amt >= 32 {\n self.yield_amt = 0;\n\n cx.waker().wake_by_ref();\n\n Poll::Pending\n } else {\n self.yield_amt += 1;\n\n Poll::Ready(self.iter.next())\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n self.iter.size_hint()\n }\n}\n" }, { "path": "tokio-stream/src/lib.rs", "content": "#![allow(\n clippy::cognitive_complexity,\n clippy::large_enum_variant,\n clippy::needless_doctest_main\n)]\n#![warn(\n missing_debug_implementations,\n missing_docs,\n rust_2018_idioms,\n unreachable_pub\n)]\n#![cfg_attr(docsrs, feature(doc_cfg))]\n#![doc(test(\n no_crate_inject,\n attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))\n))]\n\n//! Stream utilities for Tokio.\n//!\n//! A `Stream` is an asynchronous sequence of values. It can be thought of as\n//! an asynchronous version of the standard library's `Iterator` trait.\n//!\n//! This crate provides helpers to work with them. For examples of usage and a more in-depth\n//! description of streams you can also refer to the [streams\n//! tutorial](https://tokio.rs/tokio/tutorial/streams) on the tokio website.\n//!\n//! # Iterating over a Stream\n//!\n//! Due to similarities with the standard library's `Iterator` trait, some new\n//! users may assume that they can use `for in` syntax to iterate over a\n//! `Stream`, but this is unfortunately not possible. Instead, you can use a\n//! `while let` loop as follows:\n//!\n//! ```rust\n//! use tokio_stream::{self as stream, StreamExt};\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let mut stream = stream::iter(vec![0, 1, 2]);\n//!\n//! while let Some(value) = stream.next().await {\n//! println!(\"Got {}\", value);\n//! }\n//! # }\n//! ```\n//!\n//! # Returning a Stream from a function\n//!\n//! A common way to stream values from a function is to pass in the sender\n//! half of a channel and use the receiver as the stream. This requires awaiting\n//! both futures to ensure progress is made. Another alternative is the\n//! [async-stream] crate, which contains macros that provide a `yield` keyword\n//! and allow you to return an `impl Stream`.\n//!\n//! [async-stream]: https://docs.rs/async-stream\n//!\n//! # Conversion to and from `AsyncRead`/`AsyncWrite`\n//!\n//! It is often desirable to convert a `Stream` into an [`AsyncRead`],\n//! especially when dealing with plaintext formats streamed over the network.\n//! The opposite conversion from an [`AsyncRead`] into a `Stream` is also\n//! another commonly required feature. To enable these conversions,\n//! [`tokio-util`] provides the [`StreamReader`] and [`ReaderStream`]\n//! types when the io feature is enabled.\n//!\n//! [`tokio-util`]: https://docs.rs/tokio-util/latest/tokio_util/codec/index.html\n//! [`tokio::io`]: https://docs.rs/tokio/latest/tokio/io/index.html\n//! [`AsyncRead`]: https://docs.rs/tokio/latest/tokio/io/trait.AsyncRead.html\n//! [`AsyncWrite`]: https://docs.rs/tokio/latest/tokio/io/trait.AsyncWrite.html\n//! [`ReaderStream`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.ReaderStream.html\n//! [`StreamReader`]: https://docs.rs/tokio-util/latest/tokio_util/io/struct.StreamReader.html\n\n#[macro_use]\nmod macros;\n\npub mod wrappers;\n\nmod stream_ext;\npub use stream_ext::{collect::FromStream, StreamExt};\n/// Adapters for [`Stream`]s created by methods in [`StreamExt`].\npub mod adapters {\n pub use crate::stream_ext::{\n Chain, Filter, FilterMap, Fuse, Map, MapWhile, Merge, Peekable, Skip, SkipWhile, Take,\n TakeWhile, Then,\n };\n cfg_time! {\n pub use crate::stream_ext::{ChunksTimeout, Timeout, TimeoutRepeating};\n }\n}\n\ncfg_time! {\n #[deprecated = \"Import those symbols from adapters instead\"]\n #[doc(hidden)]\n pub use stream_ext::timeout::Timeout;\n pub use stream_ext::timeout::Elapsed;\n}\n\nmod empty;\npub use empty::{empty, Empty};\n\nmod iter;\npub use iter::{iter, Iter};\n\nmod once;\npub use once::{once, Once};\n\nmod pending;\npub use pending::{pending, Pending};\n\nmod stream_map;\npub use stream_map::StreamMap;\n\nmod stream_close;\npub use stream_close::StreamNotifyClose;\n\n#[doc(no_inline)]\npub use futures_core::Stream;\n" }, { "path": "tokio-stream/src/macros.rs", "content": "macro_rules! cfg_fs {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"fs\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"fs\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io_util {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"io-util\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"net\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"net\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_time {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_sync {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"sync\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"sync\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_signal {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"signal\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"signal\")))]\n $item\n )*\n }\n}\n" }, { "path": "tokio-stream/src/once.rs", "content": "use crate::{Iter, Stream};\n\nuse core::option;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\n/// Stream for the [`once`](fn@once) function.\n#[derive(Debug)]\n#[must_use = \"streams do nothing unless polled\"]\npub struct Once {\n iter: Iter>,\n}\n\nimpl Unpin for Once {}\n\n/// Creates a stream that emits an element exactly once.\n///\n/// The returned stream is immediately ready and emits the provided value once.\n///\n/// # Examples\n///\n/// ```\n/// use tokio_stream::{self as stream, StreamExt};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// // one is the loneliest number\n/// let mut one = stream::once(1);\n///\n/// assert_eq!(Some(1), one.next().await);\n///\n/// // just one, that's all we get\n/// assert_eq!(None, one.next().await);\n/// # }\n/// ```\npub fn once(value: T) -> Once {\n Once {\n iter: crate::iter(Some(value)),\n }\n}\n\nimpl Stream for Once {\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::new(&mut self.iter).poll_next(cx)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n self.iter.size_hint()\n }\n}\n" }, { "path": "tokio-stream/src/pending.rs", "content": "use crate::Stream;\n\nuse core::marker::PhantomData;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\n/// Stream for the [`pending`](fn@pending) function.\n#[derive(Debug)]\n#[must_use = \"streams do nothing unless polled\"]\npub struct Pending(PhantomData);\n\nimpl Unpin for Pending {}\nunsafe impl Send for Pending {}\nunsafe impl Sync for Pending {}\n\n/// Creates a stream that is never ready\n///\n/// The returned stream is never ready. Attempting to call\n/// [`next()`](crate::StreamExt::next) will never complete. Use\n/// [`stream::empty()`](super::empty()) to obtain a stream that is\n/// immediately empty but returns no values.\n///\n/// # Examples\n///\n/// Basic usage:\n///\n/// ```no_run\n/// use tokio_stream::{self as stream, StreamExt};\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let mut never = stream::pending::();\n///\n/// // This will never complete\n/// never.next().await;\n///\n/// unreachable!();\n/// }\n/// ```\npub const fn pending() -> Pending {\n Pending(PhantomData)\n}\n\nimpl Stream for Pending {\n type Item = T;\n\n fn poll_next(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll> {\n Poll::Pending\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (0, None)\n }\n}\n" }, { "path": "tokio-stream/src/stream_close.rs", "content": "use crate::Stream;\nuse pin_project_lite::pin_project;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A `Stream` that wraps the values in an `Option`.\n ///\n /// Whenever the wrapped stream yields an item, this stream yields that item\n /// wrapped in `Some`. When the inner stream ends, then this stream first\n /// yields a `None` item, and then this stream will also end.\n ///\n /// # Example\n ///\n /// Using `StreamNotifyClose` to handle closed streams with `StreamMap`.\n ///\n /// ```\n /// use tokio_stream::{StreamExt, StreamMap, StreamNotifyClose};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut map = StreamMap::new();\n /// let stream = StreamNotifyClose::new(tokio_stream::iter(vec![0, 1]));\n /// let stream2 = StreamNotifyClose::new(tokio_stream::iter(vec![0, 1]));\n /// map.insert(0, stream);\n /// map.insert(1, stream2);\n /// while let Some((key, val)) = map.next().await {\n /// match val {\n /// Some(val) => println!(\"got {val:?} from stream {key:?}\"),\n /// None => println!(\"stream {key:?} closed\"),\n /// }\n /// }\n /// # }\n /// ```\n #[must_use = \"streams do nothing unless polled\"]\n pub struct StreamNotifyClose {\n #[pin]\n inner: Option,\n }\n}\n\nimpl StreamNotifyClose {\n /// Create a new `StreamNotifyClose`.\n pub fn new(stream: S) -> Self {\n Self {\n inner: Some(stream),\n }\n }\n\n /// Get back the inner `Stream`.\n ///\n /// Returns `None` if the stream has reached its end.\n pub fn into_inner(self) -> Option {\n self.inner\n }\n}\n\nimpl Stream for StreamNotifyClose\nwhere\n S: Stream,\n{\n type Item = Option;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n // We can't invoke poll_next after it ended, so we unset the inner stream as a marker.\n match self\n .as_mut()\n .project()\n .inner\n .as_pin_mut()\n .map(|stream| S::poll_next(stream, cx))\n {\n Some(Poll::Ready(Some(item))) => Poll::Ready(Some(Some(item))),\n Some(Poll::Ready(None)) => {\n self.project().inner.set(None);\n Poll::Ready(Some(None))\n }\n Some(Poll::Pending) => Poll::Pending,\n None => Poll::Ready(None),\n }\n }\n\n #[inline]\n fn size_hint(&self) -> (usize, Option) {\n if let Some(inner) = &self.inner {\n // We always return +1 because when there's a stream there's at least one more item.\n let (l, u) = inner.size_hint();\n (l.saturating_add(1), u.and_then(|u| u.checked_add(1)))\n } else {\n (0, Some(0))\n }\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/all.rs", "content": "use crate::Stream;\n\nuse core::future::Future;\nuse core::marker::PhantomPinned;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Future for the [`all`](super::StreamExt::all) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct AllFuture<'a, St: ?Sized, F> {\n stream: &'a mut St,\n f: F,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl<'a, St: ?Sized, F> AllFuture<'a, St, F> {\n pub(super) fn new(stream: &'a mut St, f: F) -> Self {\n Self {\n stream,\n f,\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl Future for AllFuture<'_, St, F>\nwhere\n St: ?Sized + Stream + Unpin,\n F: FnMut(St::Item) -> bool,\n{\n type Output = bool;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n let mut stream = Pin::new(me.stream);\n\n // Take a maximum of 32 items from the stream before yielding.\n for _ in 0..32 {\n match ready!(stream.as_mut().poll_next(cx)) {\n Some(v) => {\n if !(me.f)(v) {\n return Poll::Ready(false);\n }\n }\n None => return Poll::Ready(true),\n }\n }\n\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/any.rs", "content": "use crate::Stream;\n\nuse core::future::Future;\nuse core::marker::PhantomPinned;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Future for the [`any`](super::StreamExt::any) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct AnyFuture<'a, St: ?Sized, F> {\n stream: &'a mut St,\n f: F,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl<'a, St: ?Sized, F> AnyFuture<'a, St, F> {\n pub(super) fn new(stream: &'a mut St, f: F) -> Self {\n Self {\n stream,\n f,\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl Future for AnyFuture<'_, St, F>\nwhere\n St: ?Sized + Stream + Unpin,\n F: FnMut(St::Item) -> bool,\n{\n type Output = bool;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n let mut stream = Pin::new(me.stream);\n\n // Take a maximum of 32 items from the stream before yielding.\n for _ in 0..32 {\n match ready!(stream.as_mut().poll_next(cx)) {\n Some(v) => {\n if (me.f)(v) {\n return Poll::Ready(true);\n }\n }\n None => return Poll::Ready(false),\n }\n }\n\n cx.waker().wake_by_ref();\n Poll::Pending\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/chain.rs", "content": "use crate::stream_ext::Fuse;\nuse crate::Stream;\n\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream returned by the [`chain`](super::StreamExt::chain) method.\n pub struct Chain {\n #[pin]\n a: Fuse,\n #[pin]\n b: U,\n }\n}\n\nimpl Chain {\n pub(super) fn new(a: T, b: U) -> Chain\n where\n T: Stream,\n U: Stream,\n {\n Chain { a: Fuse::new(a), b }\n }\n}\n\nimpl Stream for Chain\nwhere\n T: Stream,\n U: Stream,\n{\n type Item = T::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n use Poll::Ready;\n\n let me = self.project();\n\n if let Some(v) = ready!(me.a.poll_next(cx)) {\n return Ready(Some(v));\n }\n\n me.b.poll_next(cx)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n super::merge_size_hints(self.a.size_hint(), self.b.size_hint())\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/chunks_timeout.rs", "content": "use crate::stream_ext::Fuse;\nuse crate::Stream;\nuse tokio::time::{sleep, Sleep};\n\nuse core::future::Future;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\nuse std::time::Duration;\n\npin_project! {\n /// Stream returned by the [`chunks_timeout`](super::StreamExt::chunks_timeout) method.\n #[must_use = \"streams do nothing unless polled\"]\n #[derive(Debug)]\n pub struct ChunksTimeout {\n #[pin]\n stream: Fuse,\n #[pin]\n deadline: Option,\n duration: Duration,\n items: Vec,\n cap: usize, // https://github.com/rust-lang/futures-rs/issues/1475\n }\n}\n\nimpl ChunksTimeout {\n pub(super) fn new(stream: S, max_size: usize, duration: Duration) -> Self {\n ChunksTimeout {\n stream: Fuse::new(stream),\n deadline: None,\n duration,\n items: Vec::with_capacity(max_size),\n cap: max_size,\n }\n }\n\n /// Consumes the [`ChunksTimeout`] and then returns all buffered items.\n pub fn into_remainder(mut self: Pin<&mut Self>) -> Vec {\n let me = self.as_mut().project();\n std::mem::take(me.items)\n }\n}\n\nimpl Stream for ChunksTimeout {\n type Item = Vec;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let mut me = self.as_mut().project();\n loop {\n match me.stream.as_mut().poll_next(cx) {\n Poll::Pending => break,\n Poll::Ready(Some(item)) => {\n if me.items.is_empty() {\n me.deadline.set(Some(sleep(*me.duration)));\n me.items.reserve_exact(*me.cap);\n }\n me.items.push(item);\n if me.items.len() >= *me.cap {\n return Poll::Ready(Some(std::mem::take(me.items)));\n }\n }\n Poll::Ready(None) => {\n // Returning Some here is only correct because we fuse the inner stream.\n let last = if me.items.is_empty() {\n None\n } else {\n Some(std::mem::take(me.items))\n };\n\n return Poll::Ready(last);\n }\n }\n }\n\n if !me.items.is_empty() {\n if let Some(deadline) = me.deadline.as_pin_mut() {\n ready!(deadline.poll(cx));\n }\n return Poll::Ready(Some(std::mem::take(me.items)));\n }\n\n Poll::Pending\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let chunk_len = if self.items.is_empty() { 0 } else { 1 };\n let (lower, upper) = self.stream.size_hint();\n let lower = (lower / self.cap).saturating_add(chunk_len);\n let upper = upper.and_then(|x| x.checked_add(chunk_len));\n (lower, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/collect.rs", "content": "use crate::Stream;\n\nuse core::future::Future;\nuse core::marker::{PhantomData, PhantomPinned};\nuse core::mem;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\nuse std::collections::{BTreeMap, BTreeSet, BinaryHeap, HashMap, HashSet, LinkedList, VecDeque};\nuse std::hash::Hash;\n\n// Do not export this struct until `FromStream` can be unsealed.\npin_project! {\n /// Future returned by the [`collect`](super::StreamExt::collect) method.\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n #[derive(Debug)]\n pub struct Collect\n {\n #[pin]\n stream: T,\n collection: C,\n _output: PhantomData,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\n/// Convert from a [`Stream`].\n///\n/// This trait is not intended to be used directly. Instead, call\n/// [`StreamExt::collect()`](super::StreamExt::collect).\n///\n/// # Implementing\n///\n/// Currently, this trait may not be implemented by third parties. The trait is\n/// sealed in order to make changes in the future. Stabilization is pending\n/// enhancements to the Rust language.\npub trait FromStream: sealed::FromStreamPriv {}\n\nimpl Collect\nwhere\n T: Stream,\n U: FromStream,\n{\n pub(super) fn new(stream: T) -> Collect {\n let (lower, upper) = stream.size_hint();\n let collection = U::initialize(sealed::Internal, lower, upper);\n\n Collect {\n stream,\n collection,\n _output: PhantomData,\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl Future for Collect\nwhere\n T: Stream,\n U: FromStream,\n{\n type Output = U;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n use Poll::Ready;\n\n loop {\n let me = self.as_mut().project();\n\n let item = match ready!(me.stream.poll_next(cx)) {\n Some(item) => item,\n None => {\n return Ready(U::finalize(sealed::Internal, me.collection));\n }\n };\n\n if !U::extend(sealed::Internal, me.collection, item) {\n return Ready(U::finalize(sealed::Internal, me.collection));\n }\n }\n }\n}\n\n// ===== FromStream implementations\n\nimpl FromStream<()> for () {}\n\nimpl sealed::FromStreamPriv<()> for () {\n type InternalCollection = ();\n\n fn initialize(_: sealed::Internal, _lower: usize, _upper: Option) {}\n\n fn extend(_: sealed::Internal, _collection: &mut (), _item: ()) -> bool {\n true\n }\n\n fn finalize(_: sealed::Internal, _collection: &mut ()) {}\n}\n\nimpl> FromStream for String {}\n\nimpl> sealed::FromStreamPriv for String {\n type InternalCollection = String;\n\n fn initialize(_: sealed::Internal, _lower: usize, _upper: Option) -> String {\n String::new()\n }\n\n fn extend(_: sealed::Internal, collection: &mut String, item: T) -> bool {\n collection.push_str(item.as_ref());\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut String) -> String {\n mem::take(collection)\n }\n}\n\nimpl FromStream for Vec {}\n\nimpl sealed::FromStreamPriv for Vec {\n type InternalCollection = Vec;\n\n fn initialize(_: sealed::Internal, lower: usize, _upper: Option) -> Vec {\n Vec::with_capacity(lower)\n }\n\n fn extend(_: sealed::Internal, collection: &mut Vec, item: T) -> bool {\n collection.push(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut Vec) -> Vec {\n mem::take(collection)\n }\n}\n\nimpl FromStream for VecDeque {}\n\nimpl sealed::FromStreamPriv for VecDeque {\n type InternalCollection = VecDeque;\n\n fn initialize(_: sealed::Internal, lower: usize, _upper: Option) -> VecDeque {\n VecDeque::with_capacity(lower)\n }\n\n fn extend(_: sealed::Internal, collection: &mut VecDeque, item: T) -> bool {\n collection.push_back(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut VecDeque) -> VecDeque {\n mem::take(collection)\n }\n}\n\nimpl FromStream for LinkedList {}\n\nimpl sealed::FromStreamPriv for LinkedList {\n type InternalCollection = LinkedList;\n\n fn initialize(_: sealed::Internal, _lower: usize, _upper: Option) -> LinkedList {\n LinkedList::new()\n }\n\n fn extend(_: sealed::Internal, collection: &mut LinkedList, item: T) -> bool {\n collection.push_back(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut LinkedList) -> LinkedList {\n mem::take(collection)\n }\n}\n\nimpl FromStream for BTreeSet {}\n\nimpl sealed::FromStreamPriv for BTreeSet {\n type InternalCollection = BTreeSet;\n\n fn initialize(_: sealed::Internal, _lower: usize, _upper: Option) -> BTreeSet {\n BTreeSet::new()\n }\n\n fn extend(_: sealed::Internal, collection: &mut BTreeSet, item: T) -> bool {\n collection.insert(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut BTreeSet) -> BTreeSet {\n mem::take(collection)\n }\n}\n\nimpl FromStream<(K, V)> for BTreeMap {}\n\nimpl sealed::FromStreamPriv<(K, V)> for BTreeMap {\n type InternalCollection = BTreeMap;\n\n fn initialize(_: sealed::Internal, _lower: usize, _upper: Option) -> BTreeMap {\n BTreeMap::new()\n }\n\n fn extend(_: sealed::Internal, collection: &mut BTreeMap, (key, value): (K, V)) -> bool {\n collection.insert(key, value);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut BTreeMap) -> BTreeMap {\n mem::take(collection)\n }\n}\n\nimpl FromStream for HashSet {}\n\nimpl sealed::FromStreamPriv for HashSet {\n type InternalCollection = HashSet;\n\n fn initialize(_: sealed::Internal, lower: usize, _upper: Option) -> HashSet {\n HashSet::with_capacity(lower)\n }\n\n fn extend(_: sealed::Internal, collection: &mut HashSet, item: T) -> bool {\n collection.insert(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut HashSet) -> HashSet {\n mem::take(collection)\n }\n}\n\nimpl FromStream<(K, V)> for HashMap {}\n\nimpl sealed::FromStreamPriv<(K, V)> for HashMap {\n type InternalCollection = HashMap;\n\n fn initialize(_: sealed::Internal, lower: usize, _upper: Option) -> HashMap {\n HashMap::with_capacity(lower)\n }\n\n fn extend(_: sealed::Internal, collection: &mut HashMap, (key, value): (K, V)) -> bool {\n collection.insert(key, value);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut HashMap) -> HashMap {\n mem::take(collection)\n }\n}\n\nimpl FromStream for BinaryHeap {}\n\nimpl sealed::FromStreamPriv for BinaryHeap {\n type InternalCollection = BinaryHeap;\n\n fn initialize(_: sealed::Internal, lower: usize, _upper: Option) -> BinaryHeap {\n BinaryHeap::with_capacity(lower)\n }\n\n fn extend(_: sealed::Internal, collection: &mut BinaryHeap, item: T) -> bool {\n collection.push(item);\n true\n }\n\n fn finalize(_: sealed::Internal, collection: &mut BinaryHeap) -> BinaryHeap {\n mem::take(collection)\n }\n}\n\nimpl FromStream for Box<[T]> {}\n\nimpl sealed::FromStreamPriv for Box<[T]> {\n type InternalCollection = Vec;\n\n fn initialize(_: sealed::Internal, lower: usize, upper: Option) -> Vec {\n as sealed::FromStreamPriv>::initialize(sealed::Internal, lower, upper)\n }\n\n fn extend(_: sealed::Internal, collection: &mut Vec, item: T) -> bool {\n as sealed::FromStreamPriv>::extend(sealed::Internal, collection, item)\n }\n\n fn finalize(_: sealed::Internal, collection: &mut Vec) -> Box<[T]> {\n as sealed::FromStreamPriv>::finalize(sealed::Internal, collection)\n .into_boxed_slice()\n }\n}\n\nimpl FromStream> for Result where U: FromStream {}\n\nimpl sealed::FromStreamPriv> for Result\nwhere\n U: FromStream,\n{\n type InternalCollection = Result;\n\n fn initialize(\n _: sealed::Internal,\n lower: usize,\n upper: Option,\n ) -> Result {\n Ok(U::initialize(sealed::Internal, lower, upper))\n }\n\n fn extend(\n _: sealed::Internal,\n collection: &mut Self::InternalCollection,\n item: Result,\n ) -> bool {\n assert!(collection.is_ok());\n match item {\n Ok(item) => {\n let collection = collection.as_mut().ok().expect(\"invalid state\");\n U::extend(sealed::Internal, collection, item)\n }\n Err(err) => {\n *collection = Err(err);\n false\n }\n }\n }\n\n fn finalize(_: sealed::Internal, collection: &mut Self::InternalCollection) -> Result {\n if let Ok(collection) = collection.as_mut() {\n Ok(U::finalize(sealed::Internal, collection))\n } else {\n let res = mem::replace(collection, Ok(U::initialize(sealed::Internal, 0, Some(0))));\n\n Err(res.map(drop).unwrap_err())\n }\n }\n}\n\npub(crate) mod sealed {\n #[doc(hidden)]\n pub trait FromStreamPriv {\n /// Intermediate type used during collection process\n ///\n /// The name of this type is internal and cannot be relied upon.\n type InternalCollection;\n\n /// Initialize the collection\n fn initialize(\n internal: Internal,\n lower: usize,\n upper: Option,\n ) -> Self::InternalCollection;\n\n /// Extend the collection with the received item\n ///\n /// Return `true` to continue streaming, `false` complete collection.\n fn extend(internal: Internal, collection: &mut Self::InternalCollection, item: T) -> bool;\n\n /// Finalize collection into target type.\n fn finalize(internal: Internal, collection: &mut Self::InternalCollection) -> Self;\n }\n\n #[allow(missing_debug_implementations)]\n pub struct Internal;\n}\n" }, { "path": "tokio-stream/src/stream_ext/filter.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream returned by the [`filter`](super::StreamExt::filter) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Filter {\n #[pin]\n stream: St,\n f: F,\n }\n}\n\nimpl fmt::Debug for Filter\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Filter\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl Filter {\n pub(super) fn new(stream: St, f: F) -> Self {\n Self { stream, f }\n }\n}\n\nimpl Stream for Filter\nwhere\n St: Stream,\n F: FnMut(&St::Item) -> bool,\n{\n type Item = St::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n loop {\n match ready!(self.as_mut().project().stream.poll_next(cx)) {\n Some(e) => {\n if (self.as_mut().project().f)(&e) {\n return Poll::Ready(Some(e));\n }\n }\n None => return Poll::Ready(None),\n }\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (0, self.stream.size_hint().1) // can't know a lower bound, due to the predicate\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/filter_map.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream returned by the [`filter_map`](super::StreamExt::filter_map) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct FilterMap {\n #[pin]\n stream: St,\n f: F,\n }\n}\n\nimpl fmt::Debug for FilterMap\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"FilterMap\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl FilterMap {\n pub(super) fn new(stream: St, f: F) -> Self {\n Self { stream, f }\n }\n}\n\nimpl Stream for FilterMap\nwhere\n St: Stream,\n F: FnMut(St::Item) -> Option,\n{\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n loop {\n match ready!(self.as_mut().project().stream.poll_next(cx)) {\n Some(e) => {\n if let Some(e) = (self.as_mut().project().f)(e) {\n return Poll::Ready(Some(e));\n }\n }\n None => return Poll::Ready(None),\n }\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (0, self.stream.size_hint().1) // can't know a lower bound, due to the predicate\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/fold.rs", "content": "use crate::Stream;\n\nuse core::future::Future;\nuse core::marker::PhantomPinned;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Future returned by the [`fold`](super::StreamExt::fold) method.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct FoldFuture {\n #[pin]\n stream: St,\n acc: Option,\n f: F,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl FoldFuture {\n pub(super) fn new(stream: St, init: B, f: F) -> Self {\n Self {\n stream,\n acc: Some(init),\n f,\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl Future for FoldFuture\nwhere\n St: Stream,\n F: FnMut(B, St::Item) -> B,\n{\n type Output = B;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let mut me = self.project();\n loop {\n let next = ready!(me.stream.as_mut().poll_next(cx));\n\n match next {\n Some(v) => {\n let old = me.acc.take().unwrap();\n let new = (me.f)(old, v);\n *me.acc = Some(new);\n }\n None => return Poll::Ready(me.acc.take().unwrap()),\n }\n }\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/fuse.rs", "content": "use crate::Stream;\n\nuse pin_project_lite::pin_project;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// Stream returned by [`fuse()`][super::StreamExt::fuse].\n #[derive(Debug)]\n pub struct Fuse {\n #[pin]\n stream: Option,\n }\n}\n\nimpl Fuse\nwhere\n T: Stream,\n{\n pub(crate) fn new(stream: T) -> Fuse {\n Fuse {\n stream: Some(stream),\n }\n }\n}\n\nimpl Stream for Fuse\nwhere\n T: Stream,\n{\n type Item = T::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let res = match Option::as_pin_mut(self.as_mut().project().stream) {\n Some(stream) => ready!(stream.poll_next(cx)),\n None => return Poll::Ready(None),\n };\n\n if res.is_none() {\n // Do not poll the stream anymore\n self.as_mut().project().stream.set(None);\n }\n\n Poll::Ready(res)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n match self.stream {\n Some(ref stream) => stream.size_hint(),\n None => (0, Some(0)),\n }\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/map.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`map`](super::StreamExt::map) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Map {\n #[pin]\n stream: St,\n f: F,\n }\n}\n\nimpl fmt::Debug for Map\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Map\").field(\"stream\", &self.stream).finish()\n }\n}\n\nimpl Map {\n pub(super) fn new(stream: St, f: F) -> Self {\n Map { stream, f }\n }\n}\n\nimpl Stream for Map\nwhere\n St: Stream,\n F: FnMut(St::Item) -> T,\n{\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.as_mut()\n .project()\n .stream\n .poll_next(cx)\n .map(|opt| opt.map(|x| (self.as_mut().project().f)(x)))\n }\n\n fn size_hint(&self) -> (usize, Option) {\n self.stream.size_hint()\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/map_while.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`map_while`](super::StreamExt::map_while) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct MapWhile {\n #[pin]\n stream: St,\n f: F,\n }\n}\n\nimpl fmt::Debug for MapWhile\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"MapWhile\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl MapWhile {\n pub(super) fn new(stream: St, f: F) -> Self {\n MapWhile { stream, f }\n }\n}\n\nimpl Stream for MapWhile\nwhere\n St: Stream,\n F: FnMut(St::Item) -> Option,\n{\n type Item = T;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n let f = me.f;\n me.stream.poll_next(cx).map(|opt| opt.and_then(f))\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let (_, upper) = self.stream.size_hint();\n (0, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/merge.rs", "content": "use crate::stream_ext::Fuse;\nuse crate::Stream;\n\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream returned by the [`merge`](super::StreamExt::merge) method.\n pub struct Merge {\n #[pin]\n a: Fuse,\n #[pin]\n b: Fuse,\n // When `true`, poll `a` first, otherwise, `poll` b`.\n a_first: bool,\n }\n}\n\nimpl Merge {\n pub(super) fn new(a: T, b: U) -> Merge\n where\n T: Stream,\n U: Stream,\n {\n Merge {\n a: Fuse::new(a),\n b: Fuse::new(b),\n a_first: true,\n }\n }\n}\n\nimpl Stream for Merge\nwhere\n T: Stream,\n U: Stream,\n{\n type Item = T::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n let a_first = *me.a_first;\n\n // Toggle the flag\n *me.a_first = !a_first;\n\n if a_first {\n poll_next(me.a, me.b, cx)\n } else {\n poll_next(me.b, me.a, cx)\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n super::merge_size_hints(self.a.size_hint(), self.b.size_hint())\n }\n}\n\nfn poll_next(\n first: Pin<&mut T>,\n second: Pin<&mut U>,\n cx: &mut Context<'_>,\n) -> Poll>\nwhere\n T: Stream,\n U: Stream,\n{\n let mut done = true;\n\n match first.poll_next(cx) {\n Poll::Ready(Some(val)) => return Poll::Ready(Some(val)),\n Poll::Ready(None) => {}\n Poll::Pending => done = false,\n }\n\n match second.poll_next(cx) {\n Poll::Ready(Some(val)) => return Poll::Ready(Some(val)),\n Poll::Ready(None) => {}\n Poll::Pending => done = false,\n }\n\n if done {\n Poll::Ready(None)\n } else {\n Poll::Pending\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/next.rs", "content": "use crate::Stream;\n\nuse core::future::Future;\nuse core::marker::PhantomPinned;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Future for the [`next`](super::StreamExt::next) method.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. It only\n /// holds onto a reference to the underlying stream,\n /// so dropping it will never lose a value.\n ///\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct Next<'a, St: ?Sized> {\n stream: &'a mut St,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl<'a, St: ?Sized> Next<'a, St> {\n pub(super) fn new(stream: &'a mut St) -> Self {\n Next {\n stream,\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl Future for Next<'_, St> {\n type Output = Option;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n Pin::new(me.stream).poll_next(cx)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/peekable.rs", "content": "use std::pin::Pin;\nuse std::task::{Context, Poll};\n\nuse futures_core::Stream;\nuse pin_project_lite::pin_project;\n\nuse crate::stream_ext::Fuse;\nuse crate::StreamExt;\n\npin_project! {\n /// Stream returned by the [`peekable`](super::StreamExt::peekable) method.\n pub struct Peekable {\n peek: Option,\n #[pin]\n stream: Fuse,\n }\n}\n\nimpl Peekable {\n pub(crate) fn new(stream: T) -> Self {\n let stream = stream.fuse();\n Self { peek: None, stream }\n }\n\n /// Peek at the next item in the stream.\n pub async fn peek(&mut self) -> Option<&T::Item>\n where\n T: Unpin,\n {\n if let Some(ref it) = self.peek {\n Some(it)\n } else {\n self.peek = self.next().await;\n self.peek.as_ref()\n }\n }\n}\n\nimpl Stream for Peekable {\n type Item = T::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let this = self.project();\n if let Some(it) = this.peek.take() {\n Poll::Ready(Some(it))\n } else {\n this.stream.poll_next(cx)\n }\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/skip.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`skip`](super::StreamExt::skip) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Skip {\n #[pin]\n stream: St,\n remaining: usize,\n }\n}\n\nimpl fmt::Debug for Skip\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Skip\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl Skip {\n pub(super) fn new(stream: St, remaining: usize) -> Self {\n Self { stream, remaining }\n }\n}\n\nimpl Stream for Skip\nwhere\n St: Stream,\n{\n type Item = St::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n loop {\n match ready!(self.as_mut().project().stream.poll_next(cx)) {\n Some(e) => {\n if self.remaining == 0 {\n return Poll::Ready(Some(e));\n }\n *self.as_mut().project().remaining -= 1;\n }\n None => return Poll::Ready(None),\n }\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let (lower, upper) = self.stream.size_hint();\n\n let lower = lower.saturating_sub(self.remaining);\n let upper = upper.map(|x| x.saturating_sub(self.remaining));\n\n (lower, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/skip_while.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`skip_while`](super::StreamExt::skip_while) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct SkipWhile {\n #[pin]\n stream: St,\n predicate: Option,\n }\n}\n\nimpl fmt::Debug for SkipWhile\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"SkipWhile\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl SkipWhile {\n pub(super) fn new(stream: St, predicate: F) -> Self {\n Self {\n stream,\n predicate: Some(predicate),\n }\n }\n}\n\nimpl Stream for SkipWhile\nwhere\n St: Stream,\n F: FnMut(&St::Item) -> bool,\n{\n type Item = St::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let mut this = self.project();\n if let Some(predicate) = this.predicate {\n loop {\n match ready!(this.stream.as_mut().poll_next(cx)) {\n Some(item) => {\n if !(predicate)(&item) {\n *this.predicate = None;\n return Poll::Ready(Some(item));\n }\n }\n None => return Poll::Ready(None),\n }\n }\n } else {\n this.stream.poll_next(cx)\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let (lower, upper) = self.stream.size_hint();\n\n if self.predicate.is_some() {\n return (0, upper);\n }\n\n (lower, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/take.rs", "content": "use crate::Stream;\n\nuse core::cmp;\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`take`](super::StreamExt::take) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Take {\n #[pin]\n stream: St,\n remaining: usize,\n }\n}\n\nimpl fmt::Debug for Take\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Take\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl Take {\n pub(super) fn new(stream: St, remaining: usize) -> Self {\n Self { stream, remaining }\n }\n}\n\nimpl Stream for Take\nwhere\n St: Stream,\n{\n type Item = St::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if *self.as_mut().project().remaining > 0 {\n self.as_mut().project().stream.poll_next(cx).map(|ready| {\n match &ready {\n Some(_) => {\n *self.as_mut().project().remaining -= 1;\n }\n None => {\n *self.as_mut().project().remaining = 0;\n }\n }\n ready\n })\n } else {\n Poll::Ready(None)\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n if self.remaining == 0 {\n return (0, Some(0));\n }\n\n let (lower, upper) = self.stream.size_hint();\n\n let lower = cmp::min(lower, self.remaining);\n\n let upper = match upper {\n Some(x) if x < self.remaining => Some(x),\n _ => Some(self.remaining),\n };\n\n (lower, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/take_while.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`take_while`](super::StreamExt::take_while) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct TakeWhile {\n #[pin]\n stream: St,\n predicate: F,\n done: bool,\n }\n}\n\nimpl fmt::Debug for TakeWhile\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"TakeWhile\")\n .field(\"stream\", &self.stream)\n .field(\"done\", &self.done)\n .finish()\n }\n}\n\nimpl TakeWhile {\n pub(super) fn new(stream: St, predicate: F) -> Self {\n Self {\n stream,\n predicate,\n done: false,\n }\n }\n}\n\nimpl Stream for TakeWhile\nwhere\n St: Stream,\n F: FnMut(&St::Item) -> bool,\n{\n type Item = St::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if !*self.as_mut().project().done {\n self.as_mut().project().stream.poll_next(cx).map(|ready| {\n let ready = ready.and_then(|item| {\n if !(self.as_mut().project().predicate)(&item) {\n None\n } else {\n Some(item)\n }\n });\n\n if ready.is_none() {\n *self.as_mut().project().done = true;\n }\n\n ready\n })\n } else {\n Poll::Ready(None)\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n if self.done {\n return (0, Some(0));\n }\n\n let (_, upper) = self.stream.size_hint();\n\n (0, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/then.rs", "content": "use crate::Stream;\n\nuse core::fmt;\nuse core::future::Future;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream for the [`then`](super::StreamExt::then) method.\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Then {\n #[pin]\n stream: St,\n #[pin]\n future: Option,\n f: F,\n }\n}\n\nimpl fmt::Debug for Then\nwhere\n St: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Then\")\n .field(\"stream\", &self.stream)\n .finish()\n }\n}\n\nimpl Then {\n pub(super) fn new(stream: St, f: F) -> Self {\n Then {\n stream,\n future: None,\n f,\n }\n }\n}\n\nimpl Stream for Then\nwhere\n St: Stream,\n Fut: Future,\n F: FnMut(St::Item) -> Fut,\n{\n type Item = Fut::Output;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let mut me = self.project();\n\n loop {\n if let Some(future) = me.future.as_mut().as_pin_mut() {\n match future.poll(cx) {\n Poll::Ready(item) => {\n me.future.set(None);\n return Poll::Ready(Some(item));\n }\n Poll::Pending => return Poll::Pending,\n }\n }\n\n match me.stream.as_mut().poll_next(cx) {\n Poll::Ready(Some(item)) => {\n me.future.set(Some((me.f)(item)));\n }\n Poll::Ready(None) => return Poll::Ready(None),\n Poll::Pending => return Poll::Pending,\n }\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let future_len = usize::from(self.future.is_some());\n let (lower, upper) = self.stream.size_hint();\n\n let lower = lower.saturating_add(future_len);\n let upper = upper.and_then(|upper| upper.checked_add(future_len));\n\n (lower, upper)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/throttle.rs", "content": "//! Slow down a stream by enforcing a delay between items.\n\nuse crate::Stream;\nuse tokio::time::{Duration, Instant, Sleep};\n\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::task::{self, ready, Poll};\n\nuse pin_project_lite::pin_project;\n\npub(super) fn throttle(duration: Duration, stream: T) -> Throttle\nwhere\n T: Stream,\n{\n Throttle {\n delay: tokio::time::sleep_until(Instant::now() + duration),\n duration,\n has_delayed: true,\n stream,\n }\n}\n\npin_project! {\n /// Stream for the [`throttle`](throttle) function. This object is `!Unpin`. If you need it to\n /// implement `Unpin` you can pin your throttle like this: `Box::pin(your_throttle)`.\n #[derive(Debug)]\n #[must_use = \"streams do nothing unless polled\"]\n pub struct Throttle {\n #[pin]\n delay: Sleep,\n duration: Duration,\n\n // Set to true when `delay` has returned ready, but `stream` hasn't.\n has_delayed: bool,\n\n // The stream to throttle\n #[pin]\n stream: T,\n }\n}\n\nimpl Throttle {\n /// Acquires a reference to the underlying stream that this combinator is\n /// pulling from.\n pub fn get_ref(&self) -> &T {\n &self.stream\n }\n\n /// Acquires a mutable reference to the underlying stream that this combinator\n /// is pulling from.\n ///\n /// Note that care must be taken to avoid tampering with the state of the stream\n /// which may otherwise confuse this combinator.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.stream\n }\n\n /// Consumes this combinator, returning the underlying stream.\n ///\n /// Note that this may discard intermediate state of this combinator, so care\n /// should be taken to avoid losing resources when this is called.\n pub fn into_inner(self) -> T {\n self.stream\n }\n}\n\nimpl Stream for Throttle {\n type Item = T::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll> {\n let mut me = self.project();\n let dur = *me.duration;\n\n if !*me.has_delayed && !is_zero(dur) {\n ready!(me.delay.as_mut().poll(cx));\n *me.has_delayed = true;\n }\n\n let value = ready!(me.stream.poll_next(cx));\n\n if value.is_some() {\n if !is_zero(dur) {\n me.delay.reset(Instant::now() + dur);\n }\n\n *me.has_delayed = false;\n }\n\n Poll::Ready(value)\n }\n}\n\nfn is_zero(dur: Duration) -> bool {\n dur == Duration::from_millis(0)\n}\n" }, { "path": "tokio-stream/src/stream_ext/timeout.rs", "content": "use crate::stream_ext::Fuse;\nuse crate::Stream;\nuse tokio::time::{Instant, Sleep};\n\nuse core::future::Future;\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::time::Duration;\n\npin_project! {\n /// Stream returned by the [`timeout`](super::StreamExt::timeout) method.\n #[must_use = \"streams do nothing unless polled\"]\n #[derive(Debug)]\n pub struct Timeout {\n #[pin]\n stream: Fuse,\n #[pin]\n deadline: Sleep,\n duration: Duration,\n poll_deadline: bool,\n }\n}\n\n/// Error returned by `Timeout` and `TimeoutRepeating`.\n#[derive(Debug, PartialEq, Eq)]\npub struct Elapsed(());\n\nimpl Timeout {\n pub(super) fn new(stream: S, duration: Duration) -> Self {\n let next = Instant::now() + duration;\n let deadline = tokio::time::sleep_until(next);\n\n Timeout {\n stream: Fuse::new(stream),\n deadline,\n duration,\n poll_deadline: true,\n }\n }\n}\n\nimpl Stream for Timeout {\n type Item = Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n\n match me.stream.poll_next(cx) {\n Poll::Ready(v) => {\n if v.is_some() {\n let next = Instant::now() + *me.duration;\n me.deadline.reset(next);\n *me.poll_deadline = true;\n }\n return Poll::Ready(v.map(Ok));\n }\n Poll::Pending => {}\n };\n\n if *me.poll_deadline {\n ready!(me.deadline.poll(cx));\n *me.poll_deadline = false;\n return Poll::Ready(Some(Err(Elapsed::new())));\n }\n\n Poll::Pending\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let (lower, upper) = self.stream.size_hint();\n\n // The timeout stream may insert an error before and after each message\n // from the underlying stream, but no more than one error between each\n // message. Hence the upper bound is computed as 2x+1.\n\n // Using a helper function to enable use of question mark operator.\n fn twice_plus_one(value: Option) -> Option {\n value?.checked_mul(2)?.checked_add(1)\n }\n\n (lower, twice_plus_one(upper))\n }\n}\n\n// ===== impl Elapsed =====\n\nimpl Elapsed {\n pub(crate) fn new() -> Self {\n Elapsed(())\n }\n}\n\nimpl fmt::Display for Elapsed {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n \"deadline has elapsed\".fmt(fmt)\n }\n}\n\nimpl std::error::Error for Elapsed {}\n\nimpl From for std::io::Error {\n fn from(_err: Elapsed) -> std::io::Error {\n std::io::ErrorKind::TimedOut.into()\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/timeout_repeating.rs", "content": "use crate::stream_ext::Fuse;\nuse crate::{Elapsed, Stream};\nuse tokio::time::Interval;\n\nuse core::pin::Pin;\nuse core::task::{ready, Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Stream returned by the [`timeout_repeating`](super::StreamExt::timeout_repeating) method.\n #[must_use = \"streams do nothing unless polled\"]\n #[derive(Debug)]\n pub struct TimeoutRepeating {\n #[pin]\n stream: Fuse,\n #[pin]\n interval: Interval,\n }\n}\n\nimpl TimeoutRepeating {\n pub(super) fn new(stream: S, interval: Interval) -> Self {\n TimeoutRepeating {\n stream: Fuse::new(stream),\n interval,\n }\n }\n}\n\nimpl Stream for TimeoutRepeating {\n type Item = Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let mut me = self.project();\n\n match me.stream.poll_next(cx) {\n Poll::Ready(v) => {\n if v.is_some() {\n me.interval.reset();\n }\n return Poll::Ready(v.map(Ok));\n }\n Poll::Pending => {}\n };\n\n ready!(me.interval.poll_tick(cx));\n Poll::Ready(Some(Err(Elapsed::new())))\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let (lower, _) = self.stream.size_hint();\n\n // The timeout stream may insert an error an infinite number of times.\n (lower, None)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext/try_next.rs", "content": "use crate::stream_ext::Next;\nuse crate::Stream;\n\nuse core::future::Future;\nuse core::marker::PhantomPinned;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\nuse pin_project_lite::pin_project;\n\npin_project! {\n /// Future for the [`try_next`](super::StreamExt::try_next) method.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. It only\n /// holds onto a reference to the underlying stream,\n /// so dropping it will never lose a value.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct TryNext<'a, St: ?Sized> {\n #[pin]\n inner: Next<'a, St>,\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n\nimpl<'a, St: ?Sized> TryNext<'a, St> {\n pub(super) fn new(stream: &'a mut St) -> Self {\n Self {\n inner: Next::new(stream),\n _pin: PhantomPinned,\n }\n }\n}\n\nimpl> + Unpin> Future for TryNext<'_, St> {\n type Output = Result, E>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n me.inner.poll(cx).map(Option::transpose)\n }\n}\n" }, { "path": "tokio-stream/src/stream_ext.rs", "content": "use core::future::Future;\nuse futures_core::Stream;\n\nmod all;\nuse all::AllFuture;\n\nmod any;\nuse any::AnyFuture;\n\nmod chain;\npub use chain::Chain;\n\npub(crate) mod collect;\nuse collect::{Collect, FromStream};\n\nmod filter;\npub use filter::Filter;\n\nmod filter_map;\npub use filter_map::FilterMap;\n\nmod fold;\nuse fold::FoldFuture;\n\nmod fuse;\npub use fuse::Fuse;\n\nmod map;\npub use map::Map;\n\nmod map_while;\npub use map_while::MapWhile;\n\nmod merge;\npub use merge::Merge;\n\nmod next;\nuse next::Next;\n\nmod skip;\npub use skip::Skip;\n\nmod skip_while;\npub use skip_while::SkipWhile;\n\nmod take;\npub use take::Take;\n\nmod take_while;\npub use take_while::TakeWhile;\n\nmod then;\npub use then::Then;\n\nmod try_next;\nuse try_next::TryNext;\n\nmod peekable;\npub use peekable::Peekable;\n\ncfg_time! {\n pub(crate) mod timeout;\n pub(crate) mod timeout_repeating;\n pub use timeout::Timeout;\n pub use timeout_repeating::TimeoutRepeating;\n use tokio::time::{Duration, Interval};\n mod throttle;\n use throttle::{throttle, Throttle};\n mod chunks_timeout;\n pub use chunks_timeout::ChunksTimeout;\n}\n\n/// An extension trait for the [`Stream`] trait that provides a variety of\n/// convenient combinator functions.\n///\n/// Be aware that the `Stream` trait in Tokio is a re-export of the trait found\n/// in the [futures] crate, however both Tokio and futures provide separate\n/// `StreamExt` utility traits, and some utilities are only available on one of\n/// these traits. Click [here][futures-StreamExt] to see the other `StreamExt`\n/// trait in the futures crate.\n///\n/// If you need utilities from both `StreamExt` traits, you should prefer to\n/// import one of them, and use the other through the fully qualified call\n/// syntax. For example:\n/// ```\n/// // import one of the traits:\n/// use futures::stream::StreamExt;\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n///\n/// let a = tokio_stream::iter(vec![1, 3, 5]);\n/// let b = tokio_stream::iter(vec![2, 4, 6]);\n///\n/// // use the fully qualified call syntax for the other trait:\n/// let merged = tokio_stream::StreamExt::merge(a, b);\n///\n/// // use normal call notation for futures::stream::StreamExt::collect\n/// let output: Vec<_> = merged.collect().await;\n/// assert_eq!(output, vec![1, 2, 3, 4, 5, 6]);\n/// # }\n/// ```\n///\n/// [`Stream`]: crate::Stream\n/// [futures]: https://docs.rs/futures\n/// [futures-StreamExt]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html\npub trait StreamExt: Stream {\n /// Consumes and returns the next value in the stream or `None` if the\n /// stream is finished.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn next(&mut self) -> Option;\n /// ```\n ///\n /// Note that because `next` doesn't take ownership over the stream,\n /// the [`Stream`] type must be [`Unpin`]. If you want to use `next` with a\n /// [`!Unpin`](Unpin) stream, you'll first have to pin the stream. This can\n /// be done by boxing the stream using [`Box::pin`] or\n /// pinning it to the stack using the `pin_mut!` macro from the `pin_utils`\n /// crate.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. The returned future only\n /// holds onto a reference to the underlying stream,\n /// so dropping it will never lose a value.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let mut stream = stream::iter(1..=3);\n ///\n /// assert_eq!(stream.next().await, Some(1));\n /// assert_eq!(stream.next().await, Some(2));\n /// assert_eq!(stream.next().await, Some(3));\n /// assert_eq!(stream.next().await, None);\n /// # }\n /// ```\n fn next(&mut self) -> Next<'_, Self>\n where\n Self: Unpin,\n {\n Next::new(self)\n }\n\n /// Consumes and returns the next item in the stream. If an error is\n /// encountered before the next item, the error is returned instead.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn try_next(&mut self) -> Result, E>;\n /// ```\n ///\n /// This is similar to the [`next`](StreamExt::next) combinator,\n /// but returns a [`Result, E>`](Result) rather than\n /// an [`Option>`](Option), making for easy use\n /// with the [`?`](std::ops::Try) operator.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. The returned future only\n /// holds onto a reference to the underlying stream,\n /// so dropping it will never lose a value.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n ///\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let mut stream = stream::iter(vec![Ok(1), Ok(2), Err(\"nope\")]);\n ///\n /// assert_eq!(stream.try_next().await, Ok(Some(1)));\n /// assert_eq!(stream.try_next().await, Ok(Some(2)));\n /// assert_eq!(stream.try_next().await, Err(\"nope\"));\n /// # }\n /// ```\n fn try_next(&mut self) -> TryNext<'_, Self>\n where\n Self: Stream> + Unpin,\n {\n TryNext::new(self)\n }\n\n /// Maps this stream's items to a different type, returning a new stream of\n /// the resulting type.\n ///\n /// The provided closure is executed over all elements of this stream as\n /// they are made available. It is executed inline with calls to\n /// [`poll_next`](Stream::poll_next).\n ///\n /// Note that this function consumes the stream passed into it and returns a\n /// wrapped version of it, similar to the existing `map` methods in the\n /// standard library.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let stream = stream::iter(1..=3);\n /// let mut stream = stream.map(|x| x + 3);\n ///\n /// assert_eq!(stream.next().await, Some(4));\n /// assert_eq!(stream.next().await, Some(5));\n /// assert_eq!(stream.next().await, Some(6));\n /// # }\n /// ```\n fn map(self, f: F) -> Map\n where\n F: FnMut(Self::Item) -> T,\n Self: Sized,\n {\n Map::new(self, f)\n }\n\n /// Map this stream's items to a different type for as long as determined by\n /// the provided closure. A stream of the target type will be returned,\n /// which will yield elements until the closure returns `None`.\n ///\n /// The provided closure is executed over all elements of this stream as\n /// they are made available, until it returns `None`. It is executed inline\n /// with calls to [`poll_next`](Stream::poll_next). Once `None` is returned,\n /// the underlying stream will not be polled again.\n ///\n /// Note that this function consumes the stream passed into it and returns a\n /// wrapped version of it, similar to the [`Iterator::map_while`] method in the\n /// standard library.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let stream = stream::iter(1..=10);\n /// let mut stream = stream.map_while(|x| {\n /// if x < 4 {\n /// Some(x + 3)\n /// } else {\n /// None\n /// }\n /// });\n /// assert_eq!(stream.next().await, Some(4));\n /// assert_eq!(stream.next().await, Some(5));\n /// assert_eq!(stream.next().await, Some(6));\n /// assert_eq!(stream.next().await, None);\n /// # }\n /// ```\n fn map_while(self, f: F) -> MapWhile\n where\n F: FnMut(Self::Item) -> Option,\n Self: Sized,\n {\n MapWhile::new(self, f)\n }\n\n /// Maps this stream's items asynchronously to a different type, returning a\n /// new stream of the resulting type.\n ///\n /// The provided closure is executed over all elements of this stream as\n /// they are made available, and the returned future is executed. Only one\n /// future is executed at the time.\n ///\n /// Note that this function consumes the stream passed into it and returns a\n /// wrapped version of it, similar to the existing `then` methods in the\n /// standard library.\n ///\n /// Be aware that if the future is not `Unpin`, then neither is the `Stream`\n /// returned by this method. To handle this, you can use `tokio::pin!` as in\n /// the example below or put the stream in a `Box` with `Box::pin(stream)`.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// async fn do_async_work(value: i32) -> i32 {\n /// value + 3\n /// }\n ///\n /// let stream = stream::iter(1..=3);\n /// let stream = stream.then(do_async_work);\n ///\n /// tokio::pin!(stream);\n ///\n /// assert_eq!(stream.next().await, Some(4));\n /// assert_eq!(stream.next().await, Some(5));\n /// assert_eq!(stream.next().await, Some(6));\n /// # }\n /// ```\n fn then(self, f: F) -> Then\n where\n F: FnMut(Self::Item) -> Fut,\n Fut: Future,\n Self: Sized,\n {\n Then::new(self, f)\n }\n\n /// Combine two streams into one by interleaving the output of both as it\n /// is produced.\n ///\n /// Values are produced from the merged stream in the order they arrive from\n /// the two source streams. If both source streams provide values\n /// simultaneously, the merge stream alternates between them. This provides\n /// some level of fairness. You should not chain calls to `merge`, as this\n /// will break the fairness of the merging.\n ///\n /// The merged stream completes once **both** source streams complete. When\n /// one source stream completes before the other, the merge stream\n /// exclusively polls the remaining stream.\n ///\n /// For merging multiple streams, consider using [`StreamMap`] instead.\n ///\n /// [`StreamMap`]: crate::StreamMap\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamExt, Stream};\n /// use tokio::sync::mpsc;\n /// use tokio::time;\n ///\n /// use std::time::Duration;\n /// use std::pin::Pin;\n ///\n /// # /*\n /// #[tokio::main]\n /// # */\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// async fn main() {\n /// # time::pause();\n /// let (tx1, mut rx1) = mpsc::channel::(10);\n /// let (tx2, mut rx2) = mpsc::channel::(10);\n ///\n /// // Convert the channels to a `Stream`.\n /// let rx1 = Box::pin(async_stream::stream! {\n /// while let Some(item) = rx1.recv().await {\n /// yield item;\n /// }\n /// }) as Pin + Send>>;\n ///\n /// let rx2 = Box::pin(async_stream::stream! {\n /// while let Some(item) = rx2.recv().await {\n /// yield item;\n /// }\n /// }) as Pin + Send>>;\n ///\n /// let mut rx = rx1.merge(rx2);\n ///\n /// tokio::spawn(async move {\n /// // Send some values immediately\n /// tx1.send(1).await.unwrap();\n /// tx1.send(2).await.unwrap();\n ///\n /// // Let the other task send values\n /// time::sleep(Duration::from_millis(20)).await;\n ///\n /// tx1.send(4).await.unwrap();\n /// });\n ///\n /// tokio::spawn(async move {\n /// // Wait for the first task to send values\n /// time::sleep(Duration::from_millis(5)).await;\n ///\n /// tx2.send(3).await.unwrap();\n ///\n /// time::sleep(Duration::from_millis(25)).await;\n ///\n /// // Send the final value\n /// tx2.send(5).await.unwrap();\n /// });\n ///\n /// assert_eq!(1, rx.next().await.unwrap());\n /// assert_eq!(2, rx.next().await.unwrap());\n /// assert_eq!(3, rx.next().await.unwrap());\n /// assert_eq!(4, rx.next().await.unwrap());\n /// assert_eq!(5, rx.next().await.unwrap());\n ///\n /// // The merged stream is consumed\n /// assert!(rx.next().await.is_none());\n /// }\n /// ```\n fn merge(self, other: U) -> Merge\n where\n U: Stream,\n Self: Sized,\n {\n Merge::new(self, other)\n }\n\n /// Filters the values produced by this stream according to the provided\n /// predicate.\n ///\n /// As values of this stream are made available, the provided predicate `f`\n /// will be run against them. If the predicate\n /// resolves to `true`, then the stream will yield the value, but if the\n /// predicate resolves to `false`, then the value\n /// will be discarded and the next value will be produced.\n ///\n /// Note that this function consumes the stream passed into it and returns a\n /// wrapped version of it, similar to [`Iterator::filter`] method in the\n /// standard library.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let stream = stream::iter(1..=8);\n /// let mut evens = stream.filter(|x| x % 2 == 0);\n ///\n /// assert_eq!(Some(2), evens.next().await);\n /// assert_eq!(Some(4), evens.next().await);\n /// assert_eq!(Some(6), evens.next().await);\n /// assert_eq!(Some(8), evens.next().await);\n /// assert_eq!(None, evens.next().await);\n /// # }\n /// ```\n fn filter(self, f: F) -> Filter\n where\n F: FnMut(&Self::Item) -> bool,\n Self: Sized,\n {\n Filter::new(self, f)\n }\n\n /// Filters the values produced by this stream while simultaneously mapping\n /// them to a different type according to the provided closure.\n ///\n /// As values of this stream are made available, the provided function will\n /// be run on them. If the predicate `f` resolves to\n /// [`Some(item)`](Some) then the stream will yield the value `item`, but if\n /// it resolves to [`None`], then the value will be skipped.\n ///\n /// Note that this function consumes the stream passed into it and returns a\n /// wrapped version of it, similar to [`Iterator::filter_map`] method in the\n /// standard library.\n ///\n /// # Examples\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let stream = stream::iter(1..=8);\n /// let mut evens = stream.filter_map(|x| {\n /// if x % 2 == 0 { Some(x + 1) } else { None }\n /// });\n ///\n /// assert_eq!(Some(3), evens.next().await);\n /// assert_eq!(Some(5), evens.next().await);\n /// assert_eq!(Some(7), evens.next().await);\n /// assert_eq!(Some(9), evens.next().await);\n /// assert_eq!(None, evens.next().await);\n /// # }\n /// ```\n fn filter_map(self, f: F) -> FilterMap\n where\n F: FnMut(Self::Item) -> Option,\n Self: Sized,\n {\n FilterMap::new(self, f)\n }\n\n /// Creates a stream which ends after the first `None`.\n ///\n /// After a stream returns `None`, behavior is undefined. Future calls to\n /// `poll_next` may or may not return `Some(T)` again or they may panic.\n /// `fuse()` adapts a stream, ensuring that after `None` is given, it will\n /// return `None` forever.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{Stream, StreamExt};\n ///\n /// use std::pin::Pin;\n /// use std::task::{Context, Poll};\n ///\n /// // a stream which alternates between Some and None\n /// struct Alternate {\n /// state: i32,\n /// }\n ///\n /// impl Stream for Alternate {\n /// type Item = i32;\n ///\n /// fn poll_next(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n /// let val = self.state;\n /// self.state = self.state + 1;\n ///\n /// // if it's even, Some(i32), else None\n /// if val % 2 == 0 {\n /// Poll::Ready(Some(val))\n /// } else {\n /// Poll::Ready(None)\n /// }\n /// }\n /// }\n ///\n /// # /*\n /// #[tokio::main]\n /// # */\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// async fn main() {\n /// let mut stream = Alternate { state: 0 };\n ///\n /// // the stream goes back and forth\n /// assert_eq!(stream.next().await, Some(0));\n /// assert_eq!(stream.next().await, None);\n /// assert_eq!(stream.next().await, Some(2));\n /// assert_eq!(stream.next().await, None);\n ///\n /// // however, once it is fused\n /// let mut stream = stream.fuse();\n ///\n /// assert_eq!(stream.next().await, Some(4));\n /// assert_eq!(stream.next().await, None);\n ///\n /// // it will always return `None` after the first time.\n /// assert_eq!(stream.next().await, None);\n /// assert_eq!(stream.next().await, None);\n /// assert_eq!(stream.next().await, None);\n /// }\n /// ```\n fn fuse(self) -> Fuse\n where\n Self: Sized,\n {\n Fuse::new(self)\n }\n\n /// Creates a new stream of at most `n` items of the underlying stream.\n ///\n /// Once `n` items have been yielded from this stream then it will always\n /// return that the stream is done.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let mut stream = stream::iter(1..=10).take(3);\n ///\n /// assert_eq!(Some(1), stream.next().await);\n /// assert_eq!(Some(2), stream.next().await);\n /// assert_eq!(Some(3), stream.next().await);\n /// assert_eq!(None, stream.next().await);\n /// # }\n /// ```\n fn take(self, n: usize) -> Take\n where\n Self: Sized,\n {\n Take::new(self, n)\n }\n\n /// Take elements from this stream while the provided predicate\n /// resolves to `true`.\n ///\n /// This function, like `Iterator::take_while`, will take elements from the\n /// stream until the predicate `f` resolves to `false`. Once one element\n /// returns false it will always return that the stream is done.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let mut stream = stream::iter(1..=10).take_while(|x| *x <= 3);\n ///\n /// assert_eq!(Some(1), stream.next().await);\n /// assert_eq!(Some(2), stream.next().await);\n /// assert_eq!(Some(3), stream.next().await);\n /// assert_eq!(None, stream.next().await);\n /// # }\n /// ```\n fn take_while(self, f: F) -> TakeWhile\n where\n F: FnMut(&Self::Item) -> bool,\n Self: Sized,\n {\n TakeWhile::new(self, f)\n }\n\n /// Creates a new stream that will skip the `n` first items of the\n /// underlying stream.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let mut stream = stream::iter(1..=10).skip(7);\n ///\n /// assert_eq!(Some(8), stream.next().await);\n /// assert_eq!(Some(9), stream.next().await);\n /// assert_eq!(Some(10), stream.next().await);\n /// assert_eq!(None, stream.next().await);\n /// # }\n /// ```\n fn skip(self, n: usize) -> Skip\n where\n Self: Sized,\n {\n Skip::new(self, n)\n }\n\n /// Skip elements from the underlying stream while the provided predicate\n /// resolves to `true`.\n ///\n /// This function, like [`Iterator::skip_while`], will ignore elements from the\n /// stream until the predicate `f` resolves to `false`. Once one element\n /// returns false, the rest of the elements will be yielded.\n ///\n /// [`Iterator::skip_while`]: std::iter::Iterator::skip_while()\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n /// let mut stream = stream::iter(vec![1,2,3,4,1]).skip_while(|x| *x < 3);\n ///\n /// assert_eq!(Some(3), stream.next().await);\n /// assert_eq!(Some(4), stream.next().await);\n /// assert_eq!(Some(1), stream.next().await);\n /// assert_eq!(None, stream.next().await);\n /// # }\n /// ```\n fn skip_while(self, f: F) -> SkipWhile\n where\n F: FnMut(&Self::Item) -> bool,\n Self: Sized,\n {\n SkipWhile::new(self, f)\n }\n\n /// Tests if every element of the stream matches a predicate.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn all(&mut self, f: F) -> bool;\n /// ```\n ///\n /// `all()` takes a closure that returns `true` or `false`. It applies\n /// this closure to each element of the stream, and if they all return\n /// `true`, then so does `all`. If any of them return `false`, it\n /// returns `false`. An empty stream returns `true`.\n ///\n /// `all()` is short-circuiting; in other words, it will stop processing\n /// as soon as it finds a `false`, given that no matter what else happens,\n /// the result will also be `false`.\n ///\n /// An empty stream returns `true`.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let a = [1, 2, 3];\n ///\n /// assert!(stream::iter(&a).all(|&x| x > 0).await);\n ///\n /// assert!(!stream::iter(&a).all(|&x| x > 2).await);\n /// # }\n /// ```\n ///\n /// Stopping at the first `false`:\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let a = [1, 2, 3];\n ///\n /// let mut iter = stream::iter(&a);\n ///\n /// assert!(!iter.all(|&x| x != 2).await);\n ///\n /// // we can still use `iter`, as there are more elements.\n /// assert_eq!(iter.next().await, Some(&3));\n /// # }\n /// ```\n fn all(&mut self, f: F) -> AllFuture<'_, Self, F>\n where\n Self: Unpin,\n F: FnMut(Self::Item) -> bool,\n {\n AllFuture::new(self, f)\n }\n\n /// Tests if any element of the stream matches a predicate.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn any(&mut self, f: F) -> bool;\n /// ```\n ///\n /// `any()` takes a closure that returns `true` or `false`. It applies\n /// this closure to each element of the stream, and if any of them return\n /// `true`, then so does `any()`. If they all return `false`, it\n /// returns `false`.\n ///\n /// `any()` is short-circuiting; in other words, it will stop processing\n /// as soon as it finds a `true`, given that no matter what else happens,\n /// the result will also be `true`.\n ///\n /// An empty stream returns `false`.\n ///\n /// Basic usage:\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let a = [1, 2, 3];\n ///\n /// assert!(stream::iter(&a).any(|&x| x > 0).await);\n ///\n /// assert!(!stream::iter(&a).any(|&x| x > 5).await);\n /// # }\n /// ```\n ///\n /// Stopping at the first `true`:\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// let a = [1, 2, 3];\n ///\n /// let mut iter = stream::iter(&a);\n ///\n /// assert!(iter.any(|&x| x != 2).await);\n ///\n /// // we can still use `iter`, as there are more elements.\n /// assert_eq!(iter.next().await, Some(&2));\n /// # }\n /// ```\n fn any(&mut self, f: F) -> AnyFuture<'_, Self, F>\n where\n Self: Unpin,\n F: FnMut(Self::Item) -> bool,\n {\n AnyFuture::new(self, f)\n }\n\n /// Combine two streams into one by first returning all values from the\n /// first stream then all values from the second stream.\n ///\n /// As long as `self` still has values to emit, no values from `other` are\n /// emitted, even if some are ready.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let one = stream::iter(vec![1, 2, 3]);\n /// let two = stream::iter(vec![4, 5, 6]);\n ///\n /// let mut stream = one.chain(two);\n ///\n /// assert_eq!(stream.next().await, Some(1));\n /// assert_eq!(stream.next().await, Some(2));\n /// assert_eq!(stream.next().await, Some(3));\n /// assert_eq!(stream.next().await, Some(4));\n /// assert_eq!(stream.next().await, Some(5));\n /// assert_eq!(stream.next().await, Some(6));\n /// assert_eq!(stream.next().await, None);\n /// # }\n /// ```\n fn chain(self, other: U) -> Chain\n where\n U: Stream,\n Self: Sized,\n {\n Chain::new(self, other)\n }\n\n /// A combinator that applies a function to every element in a stream\n /// producing a single, final value.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn fold(self, init: B, f: F) -> B;\n /// ```\n ///\n /// # Examples\n /// Basic usage:\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, *};\n ///\n /// let s = stream::iter(vec![1u8, 2, 3]);\n /// let sum = s.fold(0, |acc, x| acc + x).await;\n ///\n /// assert_eq!(sum, 6);\n /// # }\n /// ```\n fn fold(self, init: B, f: F) -> FoldFuture\n where\n Self: Sized,\n F: FnMut(B, Self::Item) -> B,\n {\n FoldFuture::new(self, init, f)\n }\n\n /// Drain stream pushing all emitted values into a collection.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn collect(self) -> T;\n /// ```\n ///\n /// `collect` streams all values, awaiting as needed. Values are pushed into\n /// a collection. A number of different target collection types are\n /// supported, including [`Vec`], [`String`], and [`Bytes`].\n ///\n /// [`Bytes`]: https://docs.rs/bytes/0.6.0/bytes/struct.Bytes.html\n ///\n /// # `Result`\n ///\n /// `collect()` can also be used with streams of type `Result` where\n /// `T: FromStream<_>`. In this case, `collect()` will stream as long as\n /// values yielded from the stream are `Ok(_)`. If `Err(_)` is encountered,\n /// streaming is terminated and `collect()` returns the `Err`.\n ///\n /// # Notes\n ///\n /// `FromStream` is currently a sealed trait. Stabilization is pending\n /// enhancements to the Rust language.\n ///\n /// # Examples\n ///\n /// Basic usage:\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let doubled: Vec =\n /// stream::iter(vec![1, 2, 3])\n /// .map(|x| x * 2)\n /// .collect()\n /// .await;\n ///\n /// assert_eq!(vec![2, 4, 6], doubled);\n /// # }\n /// ```\n ///\n /// Collecting a stream of `Result` values\n ///\n /// ```\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// // A stream containing only `Ok` values will be collected\n /// let values: Result, &str> =\n /// stream::iter(vec![Ok(1), Ok(2), Ok(3)])\n /// .collect()\n /// .await;\n ///\n /// assert_eq!(Ok(vec![1, 2, 3]), values);\n ///\n /// // A stream containing `Err` values will return the first error.\n /// let results = vec![Ok(1), Err(\"no\"), Ok(2), Ok(3), Err(\"nein\")];\n ///\n /// let values: Result, &str> =\n /// stream::iter(results)\n /// .collect()\n /// .await;\n ///\n /// assert_eq!(Err(\"no\"), values);\n /// # }\n /// ```\n fn collect(self) -> Collect\n where\n T: FromStream,\n Self: Sized,\n {\n Collect::new(self)\n }\n\n /// Applies a per-item timeout to the passed stream.\n ///\n /// `timeout()` takes a `Duration` that represents the maximum amount of\n /// time each element of the stream has to complete before timing out.\n ///\n /// If the wrapped stream yields a value before the deadline is reached, the\n /// value is returned. Otherwise, an error is returned. The caller may decide\n /// to continue consuming the stream and will eventually get the next source\n /// stream value once it becomes available. See\n /// [`timeout_repeating`](StreamExt::timeout_repeating) for an alternative\n /// where the timeouts will repeat.\n ///\n /// # Notes\n ///\n /// This function consumes the stream passed into it and returns a\n /// wrapped version of it.\n ///\n /// Polling the returned stream will continue to poll the inner stream even\n /// if one or more items time out.\n ///\n /// # Examples\n ///\n /// Suppose we have a stream `int_stream` that yields 3 numbers (1, 2, 3):\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n /// use std::time::Duration;\n /// # let int_stream = stream::iter(1..=3);\n ///\n /// let int_stream = int_stream.timeout(Duration::from_secs(1));\n /// tokio::pin!(int_stream);\n ///\n /// // When no items time out, we get the 3 elements in succession:\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(2)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(3)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n ///\n /// // If the second item times out, we get an error and continue polling the stream:\n /// # let mut int_stream = stream::iter(vec![Ok(1), Err(()), Ok(2), Ok(3)]);\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert!(int_stream.try_next().await.is_err());\n /// assert_eq!(int_stream.try_next().await, Ok(Some(2)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(3)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n ///\n /// // If we want to stop consuming the source stream the first time an\n /// // element times out, we can use the `take_while` operator:\n /// # let int_stream = stream::iter(vec![Ok(1), Err(()), Ok(2), Ok(3)]);\n /// let mut int_stream = int_stream.take_while(Result::is_ok);\n ///\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n /// # }\n /// ```\n ///\n /// Once a timeout error is received, no further events will be received\n /// unless the wrapped stream yields a value (timeouts do not repeat).\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// # async fn main() {\n /// use tokio_stream::{StreamExt, wrappers::IntervalStream};\n /// use std::time::Duration;\n /// let interval_stream = IntervalStream::new(tokio::time::interval(Duration::from_millis(100)));\n /// let timeout_stream = interval_stream.timeout(Duration::from_millis(10));\n /// tokio::pin!(timeout_stream);\n ///\n /// // Only one timeout will be received between values in the source stream.\n /// assert!(timeout_stream.try_next().await.is_ok());\n /// assert!(timeout_stream.try_next().await.is_err(), \"expected one timeout\");\n /// assert!(timeout_stream.try_next().await.is_ok(), \"expected no more timeouts\");\n /// # }\n /// ```\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n fn timeout(self, duration: Duration) -> Timeout\n where\n Self: Sized,\n {\n Timeout::new(self, duration)\n }\n\n /// Applies a per-item timeout to the passed stream.\n ///\n /// `timeout_repeating()` takes an [`Interval`] that controls the time each\n /// element of the stream has to complete before timing out.\n ///\n /// If the wrapped stream yields a value before the deadline is reached, the\n /// value is returned. Otherwise, an error is returned. The caller may decide\n /// to continue consuming the stream and will eventually get the next source\n /// stream value once it becomes available. Unlike `timeout()`, if no value\n /// becomes available before the deadline is reached, additional errors are\n /// returned at the specified interval. See [`timeout`](StreamExt::timeout)\n /// for an alternative where the timeouts do not repeat.\n ///\n /// # Notes\n ///\n /// This function consumes the stream passed into it and returns a\n /// wrapped version of it.\n ///\n /// Polling the returned stream will continue to poll the inner stream even\n /// if one or more items time out.\n ///\n /// # Examples\n ///\n /// Suppose we have a stream `int_stream` that yields 3 numbers (1, 2, 3):\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_stream::{self as stream, StreamExt};\n /// use std::time::Duration;\n /// # let int_stream = stream::iter(1..=3);\n ///\n /// let int_stream = int_stream.timeout_repeating(tokio::time::interval(Duration::from_secs(1)));\n /// tokio::pin!(int_stream);\n ///\n /// // When no items time out, we get the 3 elements in succession:\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(2)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(3)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n ///\n /// // If the second item times out, we get an error and continue polling the stream:\n /// # let mut int_stream = stream::iter(vec![Ok(1), Err(()), Ok(2), Ok(3)]);\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert!(int_stream.try_next().await.is_err());\n /// assert_eq!(int_stream.try_next().await, Ok(Some(2)));\n /// assert_eq!(int_stream.try_next().await, Ok(Some(3)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n ///\n /// // If we want to stop consuming the source stream the first time an\n /// // element times out, we can use the `take_while` operator:\n /// # let int_stream = stream::iter(vec![Ok(1), Err(()), Ok(2), Ok(3)]);\n /// let mut int_stream = int_stream.take_while(Result::is_ok);\n ///\n /// assert_eq!(int_stream.try_next().await, Ok(Some(1)));\n /// assert_eq!(int_stream.try_next().await, Ok(None));\n /// # }\n /// ```\n ///\n /// Timeout errors will be continuously produced at the specified interval\n /// until the wrapped stream yields a value.\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// # async fn main() {\n /// use tokio_stream::{StreamExt, wrappers::IntervalStream};\n /// use std::time::Duration;\n /// let interval_stream = IntervalStream::new(tokio::time::interval(Duration::from_millis(23)));\n /// let timeout_stream = interval_stream.timeout_repeating(tokio::time::interval(Duration::from_millis(9)));\n /// tokio::pin!(timeout_stream);\n ///\n /// // Multiple timeouts will be received between values in the source stream.\n /// assert!(timeout_stream.try_next().await.is_ok());\n /// assert!(timeout_stream.try_next().await.is_err(), \"expected one timeout\");\n /// assert!(timeout_stream.try_next().await.is_err(), \"expected a second timeout\");\n /// // Will eventually receive another value from the source stream...\n /// assert!(timeout_stream.try_next().await.is_ok(), \"expected non-timeout\");\n /// # }\n /// ```\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n fn timeout_repeating(self, interval: Interval) -> TimeoutRepeating\n where\n Self: Sized,\n {\n TimeoutRepeating::new(self, interval)\n }\n\n /// Slows down a stream by enforcing a delay between items.\n ///\n /// The underlying timer behind this utility has a granularity of one millisecond.\n ///\n /// # Example\n ///\n /// Create a throttled stream.\n /// ```rust,no_run\n /// use std::time::Duration;\n /// use tokio_stream::StreamExt;\n ///\n /// # async fn dox() {\n /// let item_stream = futures::stream::repeat(\"one\").throttle(Duration::from_secs(2));\n /// tokio::pin!(item_stream);\n ///\n /// loop {\n /// // The string will be produced at most every 2 seconds\n /// println!(\"{:?}\", item_stream.next().await);\n /// }\n /// # }\n /// ```\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n fn throttle(self, duration: Duration) -> Throttle\n where\n Self: Sized,\n {\n throttle(duration, self)\n }\n\n /// Batches the items in the given stream using a maximum duration and size for each batch.\n ///\n /// This stream returns the next batch of items in the following situations:\n /// 1. The inner stream has returned at least `max_size` many items since the last batch.\n /// 2. The time since the first item of a batch is greater than the given duration.\n /// 3. The end of the stream is reached.\n ///\n /// The length of the returned vector is never empty or greater than the maximum size. Empty batches\n /// will not be emitted if no items are received upstream.\n ///\n /// # Panics\n ///\n /// This function panics if `max_size` is zero\n ///\n /// # Example\n ///\n /// ```rust\n /// use std::time::Duration;\n /// use tokio::time;\n /// use tokio_stream::{self as stream, StreamExt};\n /// use futures::FutureExt;\n ///\n /// #[tokio::main]\n /// # async fn _unused() {}\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// async fn main() {\n /// let iter = vec![1, 2, 3, 4].into_iter();\n /// let stream0 = stream::iter(iter);\n ///\n /// let iter = vec![5].into_iter();\n /// let stream1 = stream::iter(iter)\n /// .then(move |n| time::sleep(Duration::from_secs(5)).map(move |_| n));\n ///\n /// let chunk_stream = stream0\n /// .chain(stream1)\n /// .chunks_timeout(3, Duration::from_secs(2));\n /// tokio::pin!(chunk_stream);\n ///\n /// // a full batch was received\n /// assert_eq!(chunk_stream.next().await, Some(vec![1,2,3]));\n /// // deadline was reached before max_size was reached\n /// assert_eq!(chunk_stream.next().await, Some(vec![4]));\n /// // last element in the stream\n /// assert_eq!(chunk_stream.next().await, Some(vec![5]));\n /// }\n /// ```\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n #[track_caller]\n fn chunks_timeout(self, max_size: usize, duration: Duration) -> ChunksTimeout\n where\n Self: Sized,\n {\n assert!(max_size > 0, \"`max_size` must be non-zero.\");\n ChunksTimeout::new(self, max_size, duration)\n }\n\n /// Turns the stream into a peekable stream, whose next element can be peeked at without being\n /// consumed.\n /// ```rust\n /// use tokio_stream::{self as stream, StreamExt};\n ///\n /// #[tokio::main]\n /// # async fn _unused() {}\n /// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n /// async fn main() {\n /// let iter = vec![1, 2, 3, 4].into_iter();\n /// let mut stream = stream::iter(iter).peekable();\n ///\n /// assert_eq!(*stream.peek().await.unwrap(), 1);\n /// assert_eq!(*stream.peek().await.unwrap(), 1);\n /// assert_eq!(stream.next().await.unwrap(), 1);\n /// assert_eq!(*stream.peek().await.unwrap(), 2);\n /// }\n /// ```\n fn peekable(self) -> Peekable\n where\n Self: Sized,\n {\n Peekable::new(self)\n }\n}\n\nimpl StreamExt for St where St: Stream {}\n\n/// Merge the size hints from two streams.\nfn merge_size_hints(\n (left_low, left_high): (usize, Option),\n (right_low, right_high): (usize, Option),\n) -> (usize, Option) {\n let low = left_low.saturating_add(right_low);\n let high = match (left_high, right_high) {\n (Some(h1), Some(h2)) => h1.checked_add(h2),\n _ => None,\n };\n (low, high)\n}\n" }, { "path": "tokio-stream/src/stream_map.rs", "content": "use crate::Stream;\n\nuse std::borrow::Borrow;\nuse std::future::poll_fn;\nuse std::hash::Hash;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// Combine many streams into one, indexing each source stream with a unique\n/// key.\n///\n/// `StreamMap` is similar to [`StreamExt::merge`] in that it combines source\n/// streams into a single merged stream that yields values in the order that\n/// they arrive from the source streams. However, `StreamMap` has a lot more\n/// flexibility in usage patterns.\n///\n/// `StreamMap` can:\n///\n/// * Merge an arbitrary number of streams.\n/// * Track which source stream the value was received from.\n/// * Handle inserting and removing streams from the set of managed streams at\n/// any point during iteration.\n///\n/// All source streams held by `StreamMap` are indexed using a key. This key is\n/// included with the value when a source stream yields a value. The key is also\n/// used to remove the stream from the `StreamMap` before the stream has\n/// completed streaming.\n///\n/// # `Unpin`\n///\n/// Because the `StreamMap` API moves streams during runtime, both streams and\n/// keys must be `Unpin`. In order to insert a `!Unpin` stream into a\n/// `StreamMap`, use [`pin!`] to pin the stream to the stack or [`Box::pin`] to\n/// pin the stream in the heap.\n///\n/// # Implementation\n///\n/// `StreamMap` is backed by a `Vec<(K, V)>`. There is no guarantee that this\n/// internal implementation detail will persist in future versions, but it is\n/// important to know the runtime implications. In general, `StreamMap` works\n/// best with a \"smallish\" number of streams as all entries are scanned on\n/// insert, remove, and polling. In cases where a large number of streams need\n/// to be merged, it may be advisable to use tasks sending values on a shared\n/// [`mpsc`] channel.\n///\n/// # Notes\n///\n/// `StreamMap` removes finished streams automatically, without alerting the user.\n/// In some scenarios, the caller would want to know on closed streams.\n/// To do this, use [`StreamNotifyClose`] as a wrapper to your stream.\n/// It will return None when the stream is closed.\n///\n/// [`StreamExt::merge`]: crate::StreamExt::merge\n/// [`mpsc`]: https://docs.rs/tokio/1.0/tokio/sync/mpsc/index.html\n/// [`pin!`]: https://docs.rs/tokio/1.0/tokio/macro.pin.html\n/// [`Box::pin`]: std::boxed::Box::pin\n/// [`StreamNotifyClose`]: crate::StreamNotifyClose\n///\n/// # Examples\n///\n/// Merging two streams, then remove them after receiving the first value\n///\n/// ```\n/// use tokio_stream::{StreamExt, StreamMap, Stream};\n/// use tokio::sync::mpsc;\n/// use std::pin::Pin;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let (tx1, mut rx1) = mpsc::channel::(10);\n/// let (tx2, mut rx2) = mpsc::channel::(10);\n///\n/// // Convert the channels to a `Stream`.\n/// let rx1 = Box::pin(async_stream::stream! {\n/// while let Some(item) = rx1.recv().await {\n/// yield item;\n/// }\n/// }) as Pin + Send>>;\n///\n/// let rx2 = Box::pin(async_stream::stream! {\n/// while let Some(item) = rx2.recv().await {\n/// yield item;\n/// }\n/// }) as Pin + Send>>;\n///\n/// tokio::spawn(async move {\n/// tx1.send(1).await.unwrap();\n///\n/// // This value will never be received. The send may or may not return\n/// // `Err` depending on if the remote end closed first or not.\n/// let _ = tx1.send(2).await;\n/// });\n///\n/// tokio::spawn(async move {\n/// tx2.send(3).await.unwrap();\n/// let _ = tx2.send(4).await;\n/// });\n///\n/// let mut map = StreamMap::new();\n///\n/// // Insert both streams\n/// map.insert(\"one\", rx1);\n/// map.insert(\"two\", rx2);\n///\n/// // Read twice\n/// for _ in 0..2 {\n/// let (key, val) = map.next().await.unwrap();\n///\n/// if key == \"one\" {\n/// assert_eq!(val, 1);\n/// } else {\n/// assert_eq!(val, 3);\n/// }\n///\n/// // Remove the stream to prevent reading the next value\n/// map.remove(key);\n/// }\n/// # }\n/// ```\n///\n/// This example models a read-only client to a chat system with channels. The\n/// client sends commands to join and leave channels. `StreamMap` is used to\n/// manage active channel subscriptions.\n///\n/// For simplicity, messages are displayed with `println!`, but they could be\n/// sent to the client over a socket.\n///\n/// ```no_run\n/// use tokio_stream::{Stream, StreamExt, StreamMap};\n///\n/// enum Command {\n/// Join(String),\n/// Leave(String),\n/// }\n///\n/// fn commands() -> impl Stream {\n/// // Streams in user commands by parsing `stdin`.\n/// # tokio_stream::pending()\n/// }\n///\n/// // Join a channel, returns a stream of messages received on the channel.\n/// fn join(channel: &str) -> impl Stream + Unpin {\n/// // left as an exercise to the reader\n/// # tokio_stream::pending()\n/// }\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let mut channels = StreamMap::new();\n///\n/// // Input commands (join / leave channels).\n/// let cmds = commands();\n/// tokio::pin!(cmds);\n///\n/// loop {\n/// tokio::select! {\n/// Some(cmd) = cmds.next() => {\n/// match cmd {\n/// Command::Join(chan) => {\n/// // Join the channel and add it to the `channels`\n/// // stream map\n/// let msgs = join(&chan);\n/// channels.insert(chan, msgs);\n/// }\n/// Command::Leave(chan) => {\n/// channels.remove(&chan);\n/// }\n/// }\n/// }\n/// Some((chan, msg)) = channels.next() => {\n/// // Received a message, display it on stdout with the channel\n/// // it originated from.\n/// println!(\"{}: {}\", chan, msg);\n/// }\n/// // Both the `commands` stream and the `channels` stream are\n/// // complete. There is no more work to do, so leave the loop.\n/// else => break,\n/// }\n/// }\n/// }\n/// ```\n///\n/// Using `StreamNotifyClose` to handle closed streams with `StreamMap`.\n///\n/// ```\n/// use tokio_stream::{StreamExt, StreamMap, StreamNotifyClose};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut map = StreamMap::new();\n/// let stream = StreamNotifyClose::new(tokio_stream::iter(vec![0, 1]));\n/// let stream2 = StreamNotifyClose::new(tokio_stream::iter(vec![0, 1]));\n/// map.insert(0, stream);\n/// map.insert(1, stream2);\n/// while let Some((key, val)) = map.next().await {\n/// match val {\n/// Some(val) => println!(\"got {val:?} from stream {key:?}\"),\n/// None => println!(\"stream {key:?} closed\"),\n/// }\n/// }\n/// # }\n/// ```\n\n#[derive(Debug)]\npub struct StreamMap {\n /// Streams stored in the map\n entries: Vec<(K, V)>,\n}\n\nimpl StreamMap {\n /// An iterator visiting all key-value pairs in arbitrary order.\n ///\n /// The iterator element type is `&'a (K, V)`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// map.insert(\"a\", pending::());\n /// map.insert(\"b\", pending());\n /// map.insert(\"c\", pending());\n ///\n /// for (key, stream) in map.iter() {\n /// println!(\"({}, {:?})\", key, stream);\n /// }\n /// ```\n pub fn iter(&self) -> impl Iterator {\n self.entries.iter()\n }\n\n /// An iterator visiting all key-value pairs mutably in arbitrary order.\n ///\n /// The iterator element type is `&'a mut (K, V)`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// map.insert(\"a\", pending::());\n /// map.insert(\"b\", pending());\n /// map.insert(\"c\", pending());\n ///\n /// for (key, stream) in map.iter_mut() {\n /// println!(\"({}, {:?})\", key, stream);\n /// }\n /// ```\n pub fn iter_mut(&mut self) -> impl Iterator {\n self.entries.iter_mut()\n }\n\n /// Creates an empty `StreamMap`.\n ///\n /// The stream map is initially created with a capacity of `0`, so it will\n /// not allocate until it is first inserted into.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, Pending};\n ///\n /// let map: StreamMap<&str, Pending<()>> = StreamMap::new();\n /// ```\n pub fn new() -> StreamMap {\n StreamMap { entries: vec![] }\n }\n\n /// Creates an empty `StreamMap` with the specified capacity.\n ///\n /// The stream map will be able to hold at least `capacity` elements without\n /// reallocating. If `capacity` is 0, the stream map will not allocate.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, Pending};\n ///\n /// let map: StreamMap<&str, Pending<()>> = StreamMap::with_capacity(10);\n /// ```\n pub fn with_capacity(capacity: usize) -> StreamMap {\n StreamMap {\n entries: Vec::with_capacity(capacity),\n }\n }\n\n /// Returns an iterator visiting all keys in arbitrary order.\n ///\n /// The iterator element type is `&'a K`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// map.insert(\"a\", pending::());\n /// map.insert(\"b\", pending());\n /// map.insert(\"c\", pending());\n ///\n /// for key in map.keys() {\n /// println!(\"{}\", key);\n /// }\n /// ```\n pub fn keys(&self) -> impl Iterator {\n self.iter().map(|(k, _)| k)\n }\n\n /// An iterator visiting all values in arbitrary order.\n ///\n /// The iterator element type is `&'a V`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// map.insert(\"a\", pending::());\n /// map.insert(\"b\", pending());\n /// map.insert(\"c\", pending());\n ///\n /// for stream in map.values() {\n /// println!(\"{:?}\", stream);\n /// }\n /// ```\n pub fn values(&self) -> impl Iterator {\n self.iter().map(|(_, v)| v)\n }\n\n /// An iterator visiting all values mutably in arbitrary order.\n ///\n /// The iterator element type is `&'a mut V`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// map.insert(\"a\", pending::());\n /// map.insert(\"b\", pending());\n /// map.insert(\"c\", pending());\n ///\n /// for stream in map.values_mut() {\n /// println!(\"{:?}\", stream);\n /// }\n /// ```\n pub fn values_mut(&mut self) -> impl Iterator {\n self.iter_mut().map(|(_, v)| v)\n }\n\n /// Returns the number of streams the map can hold without reallocating.\n ///\n /// This number is a lower bound; the `StreamMap` might be able to hold\n /// more, but is guaranteed to be able to hold at least this many.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, Pending};\n ///\n /// let map: StreamMap> = StreamMap::with_capacity(100);\n /// assert!(map.capacity() >= 100);\n /// ```\n pub fn capacity(&self) -> usize {\n self.entries.capacity()\n }\n\n /// Returns the number of streams in the map.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut a = StreamMap::new();\n /// assert_eq!(a.len(), 0);\n /// a.insert(1, pending::());\n /// assert_eq!(a.len(), 1);\n /// ```\n pub fn len(&self) -> usize {\n self.entries.len()\n }\n\n /// Returns `true` if the map contains no elements.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut a = StreamMap::new();\n /// assert!(a.is_empty());\n /// a.insert(1, pending::());\n /// assert!(!a.is_empty());\n /// ```\n pub fn is_empty(&self) -> bool {\n self.entries.is_empty()\n }\n\n /// Clears the map, removing all key-stream pairs. Keeps the allocated\n /// memory for reuse.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut a = StreamMap::new();\n /// a.insert(1, pending::());\n /// a.clear();\n /// assert!(a.is_empty());\n /// ```\n pub fn clear(&mut self) {\n self.entries.clear();\n }\n\n /// Insert a key-stream pair into the map.\n ///\n /// If the map did not have this key present, `None` is returned.\n ///\n /// If the map did have this key present, the new `stream` replaces the old\n /// one and the old stream is returned.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n ///\n /// assert!(map.insert(37, pending::()).is_none());\n /// assert!(!map.is_empty());\n ///\n /// map.insert(37, pending());\n /// assert!(map.insert(37, pending()).is_some());\n /// ```\n pub fn insert(&mut self, k: K, stream: V) -> Option\n where\n K: Hash + Eq,\n {\n let ret = self.remove(&k);\n self.entries.push((k, stream));\n\n ret\n }\n\n /// Removes a key from the map, returning the stream at the key if the key was previously in the map.\n ///\n /// The key may be any borrowed form of the map's key type, but `Hash` and\n /// `Eq` on the borrowed form must match those for the key type.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n /// map.insert(1, pending::());\n /// assert!(map.remove(&1).is_some());\n /// assert!(map.remove(&1).is_none());\n /// ```\n pub fn remove(&mut self, k: &Q) -> Option\n where\n K: Borrow,\n Q: Hash + Eq + ?Sized,\n {\n for i in 0..self.entries.len() {\n if self.entries[i].0.borrow() == k {\n return Some(self.entries.swap_remove(i).1);\n }\n }\n\n None\n }\n\n /// Returns `true` if the map contains a stream for the specified key.\n ///\n /// The key may be any borrowed form of the map's key type, but `Hash` and\n /// `Eq` on the borrowed form must match those for the key type.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_stream::{StreamMap, pending};\n ///\n /// let mut map = StreamMap::new();\n /// map.insert(1, pending::());\n /// assert_eq!(map.contains_key(&1), true);\n /// assert_eq!(map.contains_key(&2), false);\n /// ```\n pub fn contains_key(&self, k: &Q) -> bool\n where\n K: Borrow,\n Q: Hash + Eq + ?Sized,\n {\n for i in 0..self.entries.len() {\n if self.entries[i].0.borrow() == k {\n return true;\n }\n }\n\n false\n }\n}\n\nimpl StreamMap\nwhere\n K: Unpin,\n V: Stream + Unpin,\n{\n /// Polls the next value, includes the vec entry index\n fn poll_next_entry(&mut self, cx: &mut Context<'_>) -> Poll> {\n let start = self::rand::thread_rng_n(self.entries.len() as u32) as usize;\n let mut idx = start;\n\n for _ in 0..self.entries.len() {\n let (_, stream) = &mut self.entries[idx];\n\n match Pin::new(stream).poll_next(cx) {\n Poll::Ready(Some(val)) => return Poll::Ready(Some((idx, val))),\n Poll::Ready(None) => {\n // Remove the entry\n self.entries.swap_remove(idx);\n\n // Check if this was the last entry, if so the cursor needs\n // to wrap\n if idx == self.entries.len() {\n idx = 0;\n } else if idx < start && start <= self.entries.len() {\n // The stream being swapped into the current index has\n // already been polled, so skip it.\n idx = idx.wrapping_add(1) % self.entries.len();\n }\n }\n Poll::Pending => {\n idx = idx.wrapping_add(1) % self.entries.len();\n }\n }\n }\n\n // If the map is empty, then the stream is complete.\n if self.entries.is_empty() {\n Poll::Ready(None)\n } else {\n Poll::Pending\n }\n }\n}\n\nimpl Default for StreamMap {\n fn default() -> Self {\n Self::new()\n }\n}\n\nimpl StreamMap\nwhere\n K: Clone + Unpin,\n V: Stream + Unpin,\n{\n /// Receives multiple items on this [`StreamMap`], extending the provided `buffer`.\n ///\n /// This method returns the number of items that is appended to the `buffer`.\n ///\n /// Note that this method does not guarantee that exactly `limit` items\n /// are received. Rather, if at least one item is available, it returns\n /// as many items as it can up to the given limit. This method returns\n /// zero only if the `StreamMap` is empty (or if `limit` is zero).\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe. If `next_many` is used as the event in a\n /// [`tokio::select!`] statement and some other branch completes first,\n /// it is guaranteed that no items were received on any of the underlying\n /// streams.\n ///\n /// [`tokio::select!`]: https://docs.rs/tokio/latest/tokio/macro.select.html\n pub async fn next_many(&mut self, buffer: &mut Vec<(K, V::Item)>, limit: usize) -> usize {\n poll_fn(|cx| self.poll_next_many(cx, buffer, limit)).await\n }\n\n /// Polls to receive multiple items on this `StreamMap`, extending the provided `buffer`.\n ///\n /// This method returns:\n /// * `Poll::Pending` if no items are available but the `StreamMap` is not empty.\n /// * `Poll::Ready(count)` where `count` is the number of items successfully received and\n /// stored in `buffer`. This can be less than, or equal to, `limit`.\n /// * `Poll::Ready(0)` if `limit` is set to zero or when the `StreamMap` is empty.\n ///\n /// Note that this method does not guarantee that exactly `limit` items\n /// are received. Rather, if at least one item is available, it returns\n /// as many items as it can up to the given limit. This method returns\n /// zero only if the `StreamMap` is empty (or if `limit` is zero).\n pub fn poll_next_many(\n &mut self,\n cx: &mut Context<'_>,\n buffer: &mut Vec<(K, V::Item)>,\n limit: usize,\n ) -> Poll {\n if limit == 0 || self.entries.is_empty() {\n return Poll::Ready(0);\n }\n\n let mut added = 0;\n\n let start = self::rand::thread_rng_n(self.entries.len() as u32) as usize;\n let mut idx = start;\n\n while added < limit {\n // Indicates whether at least one stream returned a value when polled or not\n let mut should_loop = false;\n\n for _ in 0..self.entries.len() {\n let (_, stream) = &mut self.entries[idx];\n\n match Pin::new(stream).poll_next(cx) {\n Poll::Ready(Some(val)) => {\n added += 1;\n\n let key = self.entries[idx].0.clone();\n buffer.push((key, val));\n\n should_loop = true;\n\n idx = idx.wrapping_add(1) % self.entries.len();\n }\n Poll::Ready(None) => {\n // Remove the entry\n self.entries.swap_remove(idx);\n\n // Check if this was the last entry, if so the cursor needs\n // to wrap\n if idx == self.entries.len() {\n idx = 0;\n } else if idx < start && start <= self.entries.len() {\n // The stream being swapped into the current index has\n // already been polled, so skip it.\n idx = idx.wrapping_add(1) % self.entries.len();\n }\n }\n Poll::Pending => {\n idx = idx.wrapping_add(1) % self.entries.len();\n }\n }\n }\n\n if !should_loop {\n break;\n }\n }\n\n if added > 0 {\n Poll::Ready(added)\n } else if self.entries.is_empty() {\n Poll::Ready(0)\n } else {\n Poll::Pending\n }\n }\n}\n\nimpl Stream for StreamMap\nwhere\n K: Clone + Unpin,\n V: Stream + Unpin,\n{\n type Item = (K, V::Item);\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if let Some((idx, val)) = ready!(self.poll_next_entry(cx)) {\n let key = self.entries[idx].0.clone();\n Poll::Ready(Some((key, val)))\n } else {\n Poll::Ready(None)\n }\n }\n\n fn size_hint(&self) -> (usize, Option) {\n let mut ret = (0, Some(0));\n\n for (_, stream) in &self.entries {\n let hint = stream.size_hint();\n\n ret.0 += hint.0;\n\n match (ret.1, hint.1) {\n (Some(a), Some(b)) => ret.1 = Some(a + b),\n (Some(_), None) => ret.1 = None,\n _ => {}\n }\n }\n\n ret\n }\n}\n\nimpl FromIterator<(K, V)> for StreamMap\nwhere\n K: Hash + Eq,\n{\n fn from_iter>(iter: T) -> Self {\n let iterator = iter.into_iter();\n let (lower_bound, _) = iterator.size_hint();\n let mut stream_map = Self::with_capacity(lower_bound);\n\n for (key, value) in iterator {\n stream_map.insert(key, value);\n }\n\n stream_map\n }\n}\n\nimpl Extend<(K, V)> for StreamMap {\n fn extend(&mut self, iter: T)\n where\n T: IntoIterator,\n {\n self.entries.extend(iter);\n }\n}\n\nmod rand {\n use std::cell::Cell;\n\n mod loom {\n #[cfg(not(loom))]\n pub(crate) mod rand {\n use std::collections::hash_map::RandomState;\n use std::hash::BuildHasher;\n use std::sync::atomic::AtomicU32;\n use std::sync::atomic::Ordering::Relaxed;\n\n static COUNTER: AtomicU32 = AtomicU32::new(1);\n\n pub(crate) fn seed() -> u64 {\n // Hash some unique-ish data to generate some new state\n RandomState::new().hash_one(COUNTER.fetch_add(1, Relaxed))\n }\n }\n\n #[cfg(loom)]\n pub(crate) mod rand {\n pub(crate) fn seed() -> u64 {\n 1\n }\n }\n }\n\n /// Fast random number generate\n ///\n /// Implement `xorshift64+`: 2 32-bit `xorshift` sequences added together.\n /// Shift triplet `[17,7,16]` was calculated as indicated in Marsaglia's\n /// `Xorshift` paper: \n /// This generator passes the SmallCrush suite, part of TestU01 framework:\n /// \n #[derive(Debug)]\n pub(crate) struct FastRand {\n one: Cell,\n two: Cell,\n }\n\n impl FastRand {\n /// Initialize a new, thread-local, fast random number generator.\n pub(crate) fn new(seed: u64) -> FastRand {\n let one = (seed >> 32) as u32;\n let mut two = seed as u32;\n\n if two == 0 {\n // This value cannot be zero\n two = 1;\n }\n\n FastRand {\n one: Cell::new(one),\n two: Cell::new(two),\n }\n }\n\n pub(crate) fn fastrand_n(&self, n: u32) -> u32 {\n // This is similar to fastrand() % n, but faster.\n // See https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/\n let mul = (self.fastrand() as u64).wrapping_mul(n as u64);\n (mul >> 32) as u32\n }\n\n fn fastrand(&self) -> u32 {\n let mut s1 = self.one.get();\n let s0 = self.two.get();\n\n s1 ^= s1 << 17;\n s1 = s1 ^ s0 ^ s1 >> 7 ^ s0 >> 16;\n\n self.one.set(s0);\n self.two.set(s1);\n\n s0.wrapping_add(s1)\n }\n }\n\n // Used by `StreamMap`\n pub(crate) fn thread_rng_n(n: u32) -> u32 {\n thread_local! {\n static THREAD_RNG: FastRand = FastRand::new(loom::rand::seed());\n }\n\n THREAD_RNG.with(|rng| rng.fastrand_n(n))\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/broadcast.rs", "content": "use std::pin::Pin;\nuse tokio::sync::broadcast::error::RecvError;\nuse tokio::sync::broadcast::Receiver;\n\nuse futures_core::Stream;\nuse tokio_util::sync::ReusableBoxFuture;\n\nuse std::fmt;\nuse std::task::{ready, Context, Poll};\n\n/// A wrapper around [`tokio::sync::broadcast::Receiver`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```\n/// use tokio::sync::broadcast;\n/// use tokio_stream::wrappers::BroadcastStream;\n/// use tokio_stream::StreamExt;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), tokio::sync::broadcast::error::SendError> {\n/// let (tx, rx) = broadcast::channel(16);\n/// tx.send(10)?;\n/// tx.send(20)?;\n/// # // prevent the doc test from hanging\n/// drop(tx);\n///\n/// let mut stream = BroadcastStream::new(rx);\n/// assert_eq!(stream.next().await, Some(Ok(10)));\n/// assert_eq!(stream.next().await, Some(Ok(20)));\n/// assert_eq!(stream.next().await, None);\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`tokio::sync::broadcast::Receiver`]: struct@tokio::sync::broadcast::Receiver\n/// [`Stream`]: trait@futures_core::Stream\n#[cfg_attr(docsrs, doc(cfg(feature = \"sync\")))]\npub struct BroadcastStream {\n inner: ReusableBoxFuture<'static, (Result, Receiver)>,\n}\n\n/// An error returned from the inner stream of a [`BroadcastStream`].\n#[derive(Debug, PartialEq, Eq, Clone)]\npub enum BroadcastStreamRecvError {\n /// The receiver lagged too far behind. Attempting to receive again will\n /// return the oldest message still retained by the channel.\n ///\n /// Includes the number of skipped messages.\n Lagged(u64),\n}\n\nimpl fmt::Display for BroadcastStreamRecvError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n BroadcastStreamRecvError::Lagged(amt) => write!(f, \"channel lagged by {amt}\"),\n }\n }\n}\n\nimpl std::error::Error for BroadcastStreamRecvError {}\n\nasync fn make_future(mut rx: Receiver) -> (Result, Receiver) {\n let result = rx.recv().await;\n (result, rx)\n}\n\nimpl BroadcastStream {\n /// Create a new `BroadcastStream`.\n pub fn new(rx: Receiver) -> Self {\n Self {\n inner: ReusableBoxFuture::new(make_future(rx)),\n }\n }\n}\n\nimpl Stream for BroadcastStream {\n type Item = Result;\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let (result, rx) = ready!(self.inner.poll(cx));\n self.inner.set(make_future(rx));\n match result {\n Ok(item) => Poll::Ready(Some(Ok(item))),\n Err(RecvError::Closed) => Poll::Ready(None),\n Err(RecvError::Lagged(n)) => {\n Poll::Ready(Some(Err(BroadcastStreamRecvError::Lagged(n))))\n }\n }\n }\n}\n\nimpl fmt::Debug for BroadcastStream {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"BroadcastStream\").finish()\n }\n}\n\nimpl From> for BroadcastStream {\n fn from(recv: Receiver) -> Self {\n Self::new(recv)\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/interval.rs", "content": "use crate::Stream;\nuse futures_core::stream::FusedStream;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::time::{Instant, Interval};\n\n/// A wrapper around [`Interval`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```\n/// use tokio::time::{Duration, Instant, interval};\n/// use tokio_stream::wrappers::IntervalStream;\n/// use tokio_stream::StreamExt;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let start = Instant::now();\n/// let interval = interval(Duration::from_millis(10));\n/// let mut stream = IntervalStream::new(interval);\n/// for _ in 0..3 {\n/// if let Some(instant) = stream.next().await {\n/// println!(\"elapsed: {:.1?}\", instant.duration_since(start));\n/// }\n/// }\n/// # }\n/// ```\n///\n/// [`Interval`]: struct@tokio::time::Interval\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\npub struct IntervalStream {\n inner: Interval,\n}\n\nimpl IntervalStream {\n /// Create a new `IntervalStream`.\n pub fn new(interval: Interval) -> Self {\n Self { inner: interval }\n }\n\n /// Get back the inner `Interval`.\n pub fn into_inner(self) -> Interval {\n self.inner\n }\n}\n\nimpl Stream for IntervalStream {\n type Item = Instant;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_tick(cx).map(Some)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (usize::MAX, None)\n }\n}\n\nimpl FusedStream for IntervalStream {\n fn is_terminated(&self) -> bool {\n false\n }\n}\n\nimpl AsRef for IntervalStream {\n fn as_ref(&self) -> &Interval {\n &self.inner\n }\n}\n\nimpl AsMut for IntervalStream {\n fn as_mut(&mut self) -> &mut Interval {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/lines.rs", "content": "use crate::Stream;\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{AsyncBufRead, Lines};\n\npin_project! {\n /// A wrapper around [`tokio::io::Lines`] that implements [`Stream`].\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::io::AsyncBufReadExt;\n /// use tokio_stream::wrappers::LinesStream;\n /// use tokio_stream::StreamExt;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> std::io::Result<()> {\n /// let input = b\"Hello\\nWorld\\n\";\n /// let mut stream = LinesStream::new(input.lines());\n /// while let Some(line) = stream.next().await {\n /// println!(\"{}\", line?);\n /// }\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`tokio::io::Lines`]: struct@tokio::io::Lines\n /// [`Stream`]: trait@crate::Stream\n #[derive(Debug)]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct LinesStream {\n #[pin]\n inner: Lines,\n }\n}\n\nimpl LinesStream {\n /// Create a new `LinesStream`.\n pub fn new(lines: Lines) -> Self {\n Self { inner: lines }\n }\n\n /// Get back the inner `Lines`.\n pub fn into_inner(self) -> Lines {\n self.inner\n }\n\n /// Obtain a pinned reference to the inner `Lines`.\n pub fn as_pin_mut(self: Pin<&mut Self>) -> Pin<&mut Lines> {\n self.project().inner\n }\n}\n\nimpl Stream for LinesStream {\n type Item = io::Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project()\n .inner\n .poll_next_line(cx)\n .map(Result::transpose)\n }\n}\n\nimpl AsRef> for LinesStream {\n fn as_ref(&self) -> &Lines {\n &self.inner\n }\n}\n\nimpl AsMut> for LinesStream {\n fn as_mut(&mut self) -> &mut Lines {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/mpsc_bounded.rs", "content": "use crate::Stream;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::sync::mpsc::Receiver;\n\n/// A wrapper around [`tokio::sync::mpsc::Receiver`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```\n/// use tokio::sync::mpsc;\n/// use tokio_stream::wrappers::ReceiverStream;\n/// use tokio_stream::StreamExt;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), tokio::sync::mpsc::error::SendError> {\n/// let (tx, rx) = mpsc::channel(2);\n/// tx.send(10).await?;\n/// tx.send(20).await?;\n/// # // prevent the doc test from hanging\n/// drop(tx);\n///\n/// let mut stream = ReceiverStream::new(rx);\n/// assert_eq!(stream.next().await, Some(10));\n/// assert_eq!(stream.next().await, Some(20));\n/// assert_eq!(stream.next().await, None);\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`tokio::sync::mpsc::Receiver`]: struct@tokio::sync::mpsc::Receiver\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\npub struct ReceiverStream {\n inner: Receiver,\n}\n\nimpl ReceiverStream {\n /// Create a new `ReceiverStream`.\n pub fn new(recv: Receiver) -> Self {\n Self { inner: recv }\n }\n\n /// Get back the inner `Receiver`.\n pub fn into_inner(self) -> Receiver {\n self.inner\n }\n\n /// Closes the receiving half of a channel without dropping it.\n ///\n /// This prevents any further messages from being sent on the channel while\n /// still enabling the receiver to drain messages that are buffered. Any\n /// outstanding [`Permit`] values will still be able to send messages.\n ///\n /// To guarantee no messages are dropped, after calling `close()`, you must\n /// receive all items from the stream until `None` is returned.\n ///\n /// [`Permit`]: struct@tokio::sync::mpsc::Permit\n pub fn close(&mut self) {\n self.inner.close();\n }\n}\n\nimpl Stream for ReceiverStream {\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx)\n }\n\n /// Returns the bounds of the stream based on the underlying receiver.\n ///\n /// For open channels, it returns `(receiver.len(), None)`.\n ///\n /// For closed channels, it returns `(receiver.len(), Some(used_capacity))`\n /// where `used_capacity` is calculated as `receiver.max_capacity() -\n /// receiver.capacity()`. This accounts for any [`Permit`] that is still\n /// able to send a message.\n ///\n /// [`Permit`]: struct@tokio::sync::mpsc::Permit\n fn size_hint(&self) -> (usize, Option) {\n if self.inner.is_closed() {\n let used_capacity = self.inner.max_capacity() - self.inner.capacity();\n (self.inner.len(), Some(used_capacity))\n } else {\n (self.inner.len(), None)\n }\n }\n}\n\nimpl AsRef> for ReceiverStream {\n fn as_ref(&self) -> &Receiver {\n &self.inner\n }\n}\n\nimpl AsMut> for ReceiverStream {\n fn as_mut(&mut self) -> &mut Receiver {\n &mut self.inner\n }\n}\n\nimpl From> for ReceiverStream {\n fn from(recv: Receiver) -> Self {\n Self::new(recv)\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/mpsc_unbounded.rs", "content": "use crate::Stream;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::sync::mpsc::UnboundedReceiver;\n\n/// A wrapper around [`tokio::sync::mpsc::UnboundedReceiver`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```\n/// use tokio::sync::mpsc;\n/// use tokio_stream::wrappers::UnboundedReceiverStream;\n/// use tokio_stream::StreamExt;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), tokio::sync::mpsc::error::SendError> {\n/// let (tx, rx) = mpsc::unbounded_channel();\n/// tx.send(10)?;\n/// tx.send(20)?;\n/// # // prevent the doc test from hanging\n/// drop(tx);\n///\n/// let mut stream = UnboundedReceiverStream::new(rx);\n/// assert_eq!(stream.next().await, Some(10));\n/// assert_eq!(stream.next().await, Some(20));\n/// assert_eq!(stream.next().await, None);\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`tokio::sync::mpsc::UnboundedReceiver`]: struct@tokio::sync::mpsc::UnboundedReceiver\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\npub struct UnboundedReceiverStream {\n inner: UnboundedReceiver,\n}\n\nimpl UnboundedReceiverStream {\n /// Create a new `UnboundedReceiverStream`.\n pub fn new(recv: UnboundedReceiver) -> Self {\n Self { inner: recv }\n }\n\n /// Get back the inner `UnboundedReceiver`.\n pub fn into_inner(self) -> UnboundedReceiver {\n self.inner\n }\n\n /// Closes the receiving half of a channel without dropping it.\n ///\n /// This prevents any further messages from being sent on the channel while\n /// still enabling the receiver to drain messages that are buffered.\n pub fn close(&mut self) {\n self.inner.close();\n }\n}\n\nimpl Stream for UnboundedReceiverStream {\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx)\n }\n\n /// Returns the bounds of the stream based on the underlying receiver.\n ///\n /// For open channels, it returns `(receiver.len(), None)`.\n ///\n /// For closed channels, it returns `(receiver.len(), receiver.len())`.\n fn size_hint(&self) -> (usize, Option) {\n if self.inner.is_closed() {\n let len = self.inner.len();\n (len, Some(len))\n } else {\n (self.inner.len(), None)\n }\n }\n}\n\nimpl AsRef> for UnboundedReceiverStream {\n fn as_ref(&self) -> &UnboundedReceiver {\n &self.inner\n }\n}\n\nimpl AsMut> for UnboundedReceiverStream {\n fn as_mut(&mut self) -> &mut UnboundedReceiver {\n &mut self.inner\n }\n}\n\nimpl From> for UnboundedReceiverStream {\n fn from(recv: UnboundedReceiver) -> Self {\n Self::new(recv)\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/read_dir.rs", "content": "use crate::Stream;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::fs::{DirEntry, ReadDir};\n\n/// A wrapper around [`tokio::fs::ReadDir`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```\n/// use tokio::fs::read_dir;\n/// use tokio_stream::{StreamExt, wrappers::ReadDirStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let dirs = read_dir(\".\").await?;\n/// let mut dirs = ReadDirStream::new(dirs);\n/// while let Some(dir) = dirs.next().await {\n/// let dir = dir?;\n/// println!(\"{}\", dir.path().display());\n/// }\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`tokio::fs::ReadDir`]: struct@tokio::fs::ReadDir\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"fs\")))]\npub struct ReadDirStream {\n inner: ReadDir,\n}\n\nimpl ReadDirStream {\n /// Create a new `ReadDirStream`.\n pub fn new(read_dir: ReadDir) -> Self {\n Self { inner: read_dir }\n }\n\n /// Get back the inner `ReadDir`.\n pub fn into_inner(self) -> ReadDir {\n self.inner\n }\n}\n\nimpl Stream for ReadDirStream {\n type Item = io::Result;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_next_entry(cx).map(Result::transpose)\n }\n}\n\nimpl AsRef for ReadDirStream {\n fn as_ref(&self) -> &ReadDir {\n &self.inner\n }\n}\n\nimpl AsMut for ReadDirStream {\n fn as_mut(&mut self) -> &mut ReadDir {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/signal_unix.rs", "content": "use crate::Stream;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::signal::unix::Signal;\n\n/// A wrapper around [`Signal`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```no_run\n/// use tokio::signal::unix::{signal, SignalKind};\n/// use tokio_stream::{StreamExt, wrappers::SignalStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let signals = signal(SignalKind::hangup())?;\n/// let mut stream = SignalStream::new(signals);\n/// while stream.next().await.is_some() {\n/// println!(\"hangup signal received\");\n/// }\n/// # Ok(())\n/// # }\n/// ```\n/// [`Signal`]: struct@tokio::signal::unix::Signal\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(all(unix, feature = \"signal\"))))]\npub struct SignalStream {\n inner: Signal,\n}\n\nimpl SignalStream {\n /// Create a new `SignalStream`.\n pub fn new(signal: Signal) -> Self {\n Self { inner: signal }\n }\n\n /// Get back the inner `Signal`.\n pub fn into_inner(self) -> Signal {\n self.inner\n }\n}\n\nimpl Stream for SignalStream {\n type Item = ();\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx)\n }\n}\n\nimpl AsRef for SignalStream {\n fn as_ref(&self) -> &Signal {\n &self.inner\n }\n}\n\nimpl AsMut for SignalStream {\n fn as_mut(&mut self) -> &mut Signal {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/signal_windows.rs", "content": "use crate::Stream;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::signal::windows::{CtrlBreak, CtrlC};\n\n/// A wrapper around [`CtrlC`] that implements [`Stream`].\n///\n/// [`CtrlC`]: struct@tokio::signal::windows::CtrlC\n/// [`Stream`]: trait@crate::Stream\n///\n/// # Example\n///\n/// ```no_run\n/// use tokio::signal::windows::ctrl_c;\n/// use tokio_stream::{StreamExt, wrappers::CtrlCStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let signals = ctrl_c()?;\n/// let mut stream = CtrlCStream::new(signals);\n/// while stream.next().await.is_some() {\n/// println!(\"ctrl-c received\");\n/// }\n/// # Ok(())\n/// # }\n/// ```\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(all(windows, feature = \"signal\"))))]\npub struct CtrlCStream {\n inner: CtrlC,\n}\n\nimpl CtrlCStream {\n /// Create a new `CtrlCStream`.\n pub fn new(interval: CtrlC) -> Self {\n Self { inner: interval }\n }\n\n /// Get back the inner `CtrlC`.\n pub fn into_inner(self) -> CtrlC {\n self.inner\n }\n}\n\nimpl Stream for CtrlCStream {\n type Item = ();\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx)\n }\n}\n\nimpl AsRef for CtrlCStream {\n fn as_ref(&self) -> &CtrlC {\n &self.inner\n }\n}\n\nimpl AsMut for CtrlCStream {\n fn as_mut(&mut self) -> &mut CtrlC {\n &mut self.inner\n }\n}\n\n/// A wrapper around [`CtrlBreak`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```no_run\n/// use tokio::signal::windows::ctrl_break;\n/// use tokio_stream::{StreamExt, wrappers::CtrlBreakStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let signals = ctrl_break()?;\n/// let mut stream = CtrlBreakStream::new(signals);\n/// while stream.next().await.is_some() {\n/// println!(\"ctrl-break received\");\n/// }\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`CtrlBreak`]: struct@tokio::signal::windows::CtrlBreak\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(all(windows, feature = \"signal\"))))]\npub struct CtrlBreakStream {\n inner: CtrlBreak,\n}\n\nimpl CtrlBreakStream {\n /// Create a new `CtrlBreakStream`.\n pub fn new(interval: CtrlBreak) -> Self {\n Self { inner: interval }\n }\n\n /// Get back the inner `CtrlBreak`.\n pub fn into_inner(self) -> CtrlBreak {\n self.inner\n }\n}\n\nimpl Stream for CtrlBreakStream {\n type Item = ();\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.inner.poll_recv(cx)\n }\n}\n\nimpl AsRef for CtrlBreakStream {\n fn as_ref(&self) -> &CtrlBreak {\n &self.inner\n }\n}\n\nimpl AsMut for CtrlBreakStream {\n fn as_mut(&mut self) -> &mut CtrlBreak {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/split.rs", "content": "use crate::Stream;\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{AsyncBufRead, Split};\n\npin_project! {\n /// A wrapper around [`tokio::io::Split`] that implements [`Stream`].\n ///\n /// # Example\n ///\n /// ```\n /// use tokio::io::AsyncBufReadExt;\n /// use tokio_stream::{StreamExt, wrappers::SplitStream};\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> std::io::Result<()> {\n /// let input = \"Hello\\nWorld\\n\".as_bytes();\n /// let lines = AsyncBufReadExt::split(input, b'\\n');\n ///\n /// let mut stream = SplitStream::new(lines);\n /// while let Some(line) = stream.next().await {\n /// println!(\"length = {}\", line?.len())\n /// }\n /// # Ok(())\n /// # }\n /// ```\n /// [`tokio::io::Split`]: struct@tokio::io::Split\n /// [`Stream`]: trait@crate::Stream\n #[derive(Debug)]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n pub struct SplitStream {\n #[pin]\n inner: Split,\n }\n}\n\nimpl SplitStream {\n /// Create a new `SplitStream`.\n pub fn new(split: Split) -> Self {\n Self { inner: split }\n }\n\n /// Get back the inner `Split`.\n pub fn into_inner(self) -> Split {\n self.inner\n }\n\n /// Obtain a pinned reference to the inner `Split`.\n pub fn as_pin_mut(self: Pin<&mut Self>) -> Pin<&mut Split> {\n self.project().inner\n }\n}\n\nimpl Stream for SplitStream {\n type Item = io::Result>;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project()\n .inner\n .poll_next_segment(cx)\n .map(Result::transpose)\n }\n}\n\nimpl AsRef> for SplitStream {\n fn as_ref(&self) -> &Split {\n &self.inner\n }\n}\n\nimpl AsMut> for SplitStream {\n fn as_mut(&mut self) -> &mut Split {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/tcp_listener.rs", "content": "use crate::Stream;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::net::{TcpListener, TcpStream};\n\n/// A wrapper around [`TcpListener`] that implements [`Stream`].\n///\n/// # Example\n///\n/// Accept connections from both IPv4 and IPv6 listeners in the same loop:\n///\n/// ```no_run\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use std::net::{Ipv4Addr, Ipv6Addr};\n///\n/// use tokio::net::TcpListener;\n/// use tokio_stream::{StreamExt, wrappers::TcpListenerStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let ipv4_listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 8080)).await?;\n/// let ipv6_listener = TcpListener::bind((Ipv6Addr::LOCALHOST, 8080)).await?;\n/// let ipv4_connections = TcpListenerStream::new(ipv4_listener);\n/// let ipv6_connections = TcpListenerStream::new(ipv6_listener);\n///\n/// let mut connections = ipv4_connections.merge(ipv6_connections);\n/// while let Some(tcp_stream) = connections.next().await {\n/// let stream = tcp_stream?;\n/// let peer_addr = stream.peer_addr()?;\n/// println!(\"accepted connection; peer address = {peer_addr}\");\n/// }\n/// # Ok(())\n/// # }\n/// # }\n/// ```\n///\n/// [`TcpListener`]: struct@tokio::net::TcpListener\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(feature = \"net\")))]\npub struct TcpListenerStream {\n inner: TcpListener,\n}\n\nimpl TcpListenerStream {\n /// Create a new `TcpListenerStream`.\n pub fn new(listener: TcpListener) -> Self {\n Self { inner: listener }\n }\n\n /// Get back the inner `TcpListener`.\n pub fn into_inner(self) -> TcpListener {\n self.inner\n }\n}\n\nimpl Stream for TcpListenerStream {\n type Item = io::Result;\n\n fn poll_next(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n match self.inner.poll_accept(cx) {\n Poll::Ready(Ok((stream, _))) => Poll::Ready(Some(Ok(stream))),\n Poll::Ready(Err(err)) => Poll::Ready(Some(Err(err))),\n Poll::Pending => Poll::Pending,\n }\n }\n}\n\nimpl AsRef for TcpListenerStream {\n fn as_ref(&self) -> &TcpListener {\n &self.inner\n }\n}\n\nimpl AsMut for TcpListenerStream {\n fn as_mut(&mut self) -> &mut TcpListener {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/unix_listener.rs", "content": "use crate::Stream;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::net::{UnixListener, UnixStream};\n\n/// A wrapper around [`UnixListener`] that implements [`Stream`].\n///\n/// # Example\n///\n/// ```no_run\n/// use tokio::net::UnixListener;\n/// use tokio_stream::{StreamExt, wrappers::UnixListenerStream};\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// let listener = UnixListener::bind(\"/tmp/sock\")?;\n/// let mut incoming = UnixListenerStream::new(listener);\n///\n/// while let Some(stream) = incoming.next().await {\n/// let stream = stream?;\n/// let peer_addr = stream.peer_addr()?;\n/// println!(\"Accepted connection from: {peer_addr:?}\");\n/// }\n/// # Ok(())\n/// # }\n/// ```\n/// [`UnixListener`]: struct@tokio::net::UnixListener\n/// [`Stream`]: trait@crate::Stream\n#[derive(Debug)]\n#[cfg_attr(docsrs, doc(cfg(all(unix, feature = \"net\"))))]\npub struct UnixListenerStream {\n inner: UnixListener,\n}\n\nimpl UnixListenerStream {\n /// Create a new `UnixListenerStream`.\n pub fn new(listener: UnixListener) -> Self {\n Self { inner: listener }\n }\n\n /// Get back the inner `UnixListener`.\n pub fn into_inner(self) -> UnixListener {\n self.inner\n }\n}\n\nimpl Stream for UnixListenerStream {\n type Item = io::Result;\n\n fn poll_next(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n match self.inner.poll_accept(cx) {\n Poll::Ready(Ok((stream, _))) => Poll::Ready(Some(Ok(stream))),\n Poll::Ready(Err(err)) => Poll::Ready(Some(Err(err))),\n Poll::Pending => Poll::Pending,\n }\n }\n}\n\nimpl AsRef for UnixListenerStream {\n fn as_ref(&self) -> &UnixListener {\n &self.inner\n }\n}\n\nimpl AsMut for UnixListenerStream {\n fn as_mut(&mut self) -> &mut UnixListener {\n &mut self.inner\n }\n}\n" }, { "path": "tokio-stream/src/wrappers/watch.rs", "content": "use std::pin::Pin;\nuse tokio::sync::watch::Receiver;\n\nuse futures_core::Stream;\nuse tokio_util::sync::ReusableBoxFuture;\n\nuse std::fmt;\nuse std::task::{ready, Context, Poll};\nuse tokio::sync::watch::error::RecvError;\n\n/// A wrapper around [`tokio::sync::watch::Receiver`] that implements [`Stream`].\n///\n/// This stream will start by yielding the current value when the `WatchStream` is polled,\n/// regardless of whether it was the initial value or sent afterwards,\n/// unless you use [`WatchStream::from_changes`].\n///\n/// # Examples\n///\n/// ```\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// use tokio_stream::{StreamExt, wrappers::WatchStream};\n/// use tokio::sync::watch;\n///\n/// let (tx, rx) = watch::channel(\"hello\");\n/// let mut rx = WatchStream::new(rx);\n///\n/// assert_eq!(rx.next().await, Some(\"hello\"));\n///\n/// tx.send(\"goodbye\").unwrap();\n/// assert_eq!(rx.next().await, Some(\"goodbye\"));\n/// # }\n/// ```\n///\n/// ```\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// use tokio_stream::{StreamExt, wrappers::WatchStream};\n/// use tokio::sync::watch;\n///\n/// let (tx, rx) = watch::channel(\"hello\");\n/// let mut rx = WatchStream::new(rx);\n///\n/// // existing rx output with \"hello\" is ignored here\n///\n/// tx.send(\"goodbye\").unwrap();\n/// assert_eq!(rx.next().await, Some(\"goodbye\"));\n/// # }\n/// ```\n///\n/// Example with [`WatchStream::from_changes`]:\n///\n/// ```\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// use futures::future::FutureExt;\n/// use tokio::sync::watch;\n/// use tokio_stream::{StreamExt, wrappers::WatchStream};\n///\n/// let (tx, rx) = watch::channel(\"hello\");\n/// let mut rx = WatchStream::from_changes(rx);\n///\n/// // no output from rx is available at this point - let's check this:\n/// assert!(rx.next().now_or_never().is_none());\n///\n/// tx.send(\"goodbye\").unwrap();\n/// assert_eq!(rx.next().await, Some(\"goodbye\"));\n/// # }\n/// ```\n///\n/// [`tokio::sync::watch::Receiver`]: struct@tokio::sync::watch::Receiver\n/// [`Stream`]: trait@crate::Stream\n#[cfg_attr(docsrs, doc(cfg(feature = \"sync\")))]\npub struct WatchStream {\n inner: ReusableBoxFuture<'static, (Result<(), RecvError>, Receiver)>,\n}\n\nasync fn make_future(\n mut rx: Receiver,\n) -> (Result<(), RecvError>, Receiver) {\n let result = rx.changed().await;\n (result, rx)\n}\n\nimpl WatchStream {\n /// Create a new `WatchStream`.\n pub fn new(rx: Receiver) -> Self {\n Self {\n inner: ReusableBoxFuture::new(async move { (Ok(()), rx) }),\n }\n }\n\n /// Create a new `WatchStream` that waits for the value to be changed.\n pub fn from_changes(rx: Receiver) -> Self {\n Self {\n inner: ReusableBoxFuture::new(make_future(rx)),\n }\n }\n}\n\nimpl Stream for WatchStream {\n type Item = T;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let (result, mut rx) = ready!(self.inner.poll(cx));\n match result {\n Ok(_) => {\n let received = (*rx.borrow_and_update()).clone();\n self.inner.set(make_future(rx));\n Poll::Ready(Some(received))\n }\n Err(_) => {\n self.inner.set(make_future(rx));\n Poll::Ready(None)\n }\n }\n }\n}\n\nimpl Unpin for WatchStream {}\n\nimpl fmt::Debug for WatchStream {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"WatchStream\").finish()\n }\n}\n\nimpl From> for WatchStream {\n fn from(recv: Receiver) -> Self {\n Self::new(recv)\n }\n}\n" }, { "path": "tokio-stream/src/wrappers.rs", "content": "//! Wrappers for Tokio types that implement `Stream`.\n\n/// Error types for the wrappers.\npub mod errors {\n cfg_sync! {\n pub use crate::wrappers::broadcast::BroadcastStreamRecvError;\n }\n}\n\nmod mpsc_bounded;\npub use mpsc_bounded::ReceiverStream;\n\nmod mpsc_unbounded;\npub use mpsc_unbounded::UnboundedReceiverStream;\n\ncfg_sync! {\n mod broadcast;\n pub use broadcast::BroadcastStream;\n\n mod watch;\n pub use watch::WatchStream;\n}\n\ncfg_signal! {\n #[cfg(all(unix, not(loom)))]\n mod signal_unix;\n #[cfg(all(unix, not(loom)))]\n pub use signal_unix::SignalStream;\n\n #[cfg(any(windows, docsrs))]\n mod signal_windows;\n #[cfg(any(windows, docsrs))]\n pub use signal_windows::{CtrlCStream, CtrlBreakStream};\n}\n\ncfg_time! {\n mod interval;\n pub use interval::IntervalStream;\n}\n\ncfg_net! {\n #[cfg(not(loom))]\n mod tcp_listener;\n #[cfg(not(loom))]\n pub use tcp_listener::TcpListenerStream;\n\n #[cfg(all(unix, not(loom)))]\n mod unix_listener;\n #[cfg(all(unix, not(loom)))]\n pub use unix_listener::UnixListenerStream;\n}\n\ncfg_io_util! {\n mod split;\n pub use split::SplitStream;\n\n mod lines;\n pub use lines::LinesStream;\n}\n\ncfg_fs! {\n #[cfg(not(loom))]\n mod read_dir;\n #[cfg(not(loom))]\n pub use read_dir::ReadDirStream;\n}\n" }, { "path": "tokio-stream/tests/async_send_sync.rs", "content": "#![allow(clippy::diverging_sub_expression)]\n\nuse std::rc::Rc;\n\n#[allow(dead_code)]\ntype BoxStream = std::pin::Pin>>;\n\n#[allow(dead_code)]\nfn require_send(_t: &T) {}\n#[allow(dead_code)]\nfn require_sync(_t: &T) {}\n#[allow(dead_code)]\nfn require_unpin(_t: &T) {}\n\n#[allow(dead_code)]\nstruct Invalid;\n\n#[allow(unused)]\ntrait AmbiguousIfSend {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfSend<()> for T {}\nimpl AmbiguousIfSend for T {}\n\n#[allow(unused)]\ntrait AmbiguousIfSync {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfSync<()> for T {}\nimpl AmbiguousIfSync for T {}\n\n#[allow(unused)]\ntrait AmbiguousIfUnpin {\n fn some_item(&self) {}\n}\nimpl AmbiguousIfUnpin<()> for T {}\nimpl AmbiguousIfUnpin for T {}\n\nmacro_rules! into_todo {\n ($typ:ty) => {{\n let x: $typ = todo!();\n x\n }};\n}\n\nmacro_rules! async_assert_fn {\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): Send & Sync) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n require_send(&f);\n require_sync(&f);\n };\n };\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): Send & !Sync) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n require_send(&f);\n AmbiguousIfSync::some_item(&f);\n };\n };\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): !Send & Sync) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n AmbiguousIfSend::some_item(&f);\n require_sync(&f);\n };\n };\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): !Send & !Sync) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n AmbiguousIfSend::some_item(&f);\n AmbiguousIfSync::some_item(&f);\n };\n };\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): !Unpin) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n AmbiguousIfUnpin::some_item(&f);\n };\n };\n ($($f:ident $(< $($generic:ty),* > )? )::+($($arg:ty),*): Unpin) => {\n #[allow(unreachable_code)]\n #[allow(unused_variables)]\n const _: fn() = || {\n let f = $($f $(::<$($generic),*>)? )::+( $( into_todo!($arg) ),* );\n require_unpin(&f);\n };\n };\n}\n\nasync_assert_fn!(tokio_stream::empty>(): Send & Sync);\nasync_assert_fn!(tokio_stream::pending>(): Send & Sync);\nasync_assert_fn!(tokio_stream::iter(std::vec::IntoIter): Send & Sync);\n\nasync_assert_fn!(tokio_stream::StreamExt::next(&mut BoxStream<()>): !Unpin);\nasync_assert_fn!(tokio_stream::StreamExt::try_next(&mut BoxStream>): !Unpin);\nasync_assert_fn!(tokio_stream::StreamExt::all(&mut BoxStream<()>, fn(())->bool): !Unpin);\nasync_assert_fn!(tokio_stream::StreamExt::any(&mut BoxStream<()>, fn(())->bool): !Unpin);\nasync_assert_fn!(tokio_stream::StreamExt::fold(&mut BoxStream<()>, (), fn((), ())->()): !Unpin);\nasync_assert_fn!(tokio_stream::StreamExt::collect>(&mut BoxStream<()>): !Unpin);\n" }, { "path": "tokio-stream/tests/chunks_timeout.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"time\", feature = \"sync\", feature = \"io-util\"))]\n\nuse tokio::time;\nuse tokio_stream::{self as stream, StreamExt};\nuse tokio_test::assert_pending;\nuse tokio_test::task;\n\nuse futures::FutureExt;\nuse std::time::Duration;\n\n#[tokio::test(start_paused = true)]\nasync fn usage() {\n let iter = vec![1, 2, 3].into_iter();\n let stream0 = stream::iter(iter);\n\n let iter = vec![4].into_iter();\n let stream1 =\n stream::iter(iter).then(move |n| time::sleep(Duration::from_secs(3)).map(move |_| n));\n\n let chunk_stream = stream0\n .chain(stream1)\n .chunks_timeout(4, Duration::from_secs(2));\n\n let mut chunk_stream = task::spawn(chunk_stream);\n\n assert_pending!(chunk_stream.poll_next());\n time::advance(Duration::from_secs(2)).await;\n assert_eq!(chunk_stream.next().await, Some(vec![1, 2, 3]));\n\n assert_pending!(chunk_stream.poll_next());\n time::advance(Duration::from_secs(2)).await;\n assert_eq!(chunk_stream.next().await, Some(vec![4]));\n}\n\n#[tokio::test(start_paused = true)]\nasync fn full_chunk_with_timeout() {\n let iter = vec![1, 2].into_iter();\n let stream0 = stream::iter(iter);\n\n let iter = vec![3].into_iter();\n let stream1 =\n stream::iter(iter).then(move |n| time::sleep(Duration::from_secs(1)).map(move |_| n));\n\n let iter = vec![4].into_iter();\n let stream2 =\n stream::iter(iter).then(move |n| time::sleep(Duration::from_secs(3)).map(move |_| n));\n\n let chunk_stream = stream0\n .chain(stream1)\n .chain(stream2)\n .chunks_timeout(3, Duration::from_secs(2));\n\n let mut chunk_stream = task::spawn(chunk_stream);\n\n assert_pending!(chunk_stream.poll_next());\n time::advance(Duration::from_secs(2)).await;\n assert_eq!(chunk_stream.next().await, Some(vec![1, 2, 3]));\n\n assert_pending!(chunk_stream.poll_next());\n time::advance(Duration::from_secs(2)).await;\n assert_eq!(chunk_stream.next().await, Some(vec![4]));\n}\n\n#[tokio::test]\n#[ignore]\nasync fn real_time() {\n let iter = vec![1, 2, 3, 4].into_iter();\n let stream0 = stream::iter(iter);\n\n let iter = vec![5].into_iter();\n let stream1 =\n stream::iter(iter).then(move |n| time::sleep(Duration::from_secs(5)).map(move |_| n));\n\n let chunk_stream = stream0\n .chain(stream1)\n .chunks_timeout(3, Duration::from_secs(2));\n\n let mut chunk_stream = task::spawn(chunk_stream);\n\n assert_eq!(chunk_stream.next().await, Some(vec![1, 2, 3]));\n assert_eq!(chunk_stream.next().await, Some(vec![4]));\n assert_eq!(chunk_stream.next().await, Some(vec![5]));\n}\n" }, { "path": "tokio-stream/tests/mpsc_bounded_stream.rs", "content": "use futures::{Stream, StreamExt};\nuse tokio::sync::mpsc;\nuse tokio_stream::wrappers::ReceiverStream;\n\n#[tokio::test]\nasync fn size_hint_stream_open() {\n let (tx, rx) = mpsc::channel(4);\n\n tx.send(1).await.unwrap();\n tx.send(2).await.unwrap();\n\n let mut stream = ReceiverStream::new(rx);\n\n assert_eq!(stream.size_hint(), (2, None));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, None));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, None));\n}\n\n#[tokio::test]\nasync fn size_hint_stream_closed() {\n let (tx, rx) = mpsc::channel(4);\n\n tx.send(1).await.unwrap();\n tx.send(2).await.unwrap();\n\n let mut stream = ReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (2, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, Some(1)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[tokio::test]\nasync fn size_hint_sender_dropped() {\n let (tx, rx) = mpsc::channel(4);\n\n tx.send(1).await.unwrap();\n tx.send(2).await.unwrap();\n\n let mut stream = ReceiverStream::new(rx);\n drop(tx);\n\n assert_eq!(stream.size_hint(), (2, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, Some(1)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[test]\nfn size_hint_stream_instantly_closed() {\n let (_tx, rx) = mpsc::channel::(4);\n\n let mut stream = ReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[tokio::test]\nasync fn size_hint_stream_closed_permits_send() {\n let (tx, rx) = mpsc::channel(4);\n\n tx.send(1).await.unwrap();\n let permit1 = tx.reserve().await.unwrap();\n let permit2 = tx.reserve().await.unwrap();\n\n let mut stream = ReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (1, Some(3)));\n permit1.send(2);\n assert_eq!(stream.size_hint(), (2, Some(3)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(1)));\n permit2.send(3);\n assert_eq!(stream.size_hint(), (1, Some(1)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(stream.next().await, None);\n}\n\n#[tokio::test]\nasync fn size_hint_stream_closed_permits_drop() {\n let (tx, rx) = mpsc::channel(4);\n\n tx.send(1).await.unwrap();\n let permit1 = tx.reserve().await.unwrap();\n let permit2 = tx.reserve().await.unwrap();\n\n let mut stream = ReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (1, Some(3)));\n drop(permit1);\n assert_eq!(stream.size_hint(), (1, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(1)));\n drop(permit2);\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(stream.next().await, None);\n}\n" }, { "path": "tokio-stream/tests/mpsc_unbounded_stream.rs", "content": "use futures::{Stream, StreamExt};\nuse tokio::sync::mpsc;\nuse tokio_stream::wrappers::UnboundedReceiverStream;\n\n#[tokio::test]\nasync fn size_hint_stream_open() {\n let (tx, rx) = mpsc::unbounded_channel();\n\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n\n let mut stream = UnboundedReceiverStream::new(rx);\n\n assert_eq!(stream.size_hint(), (2, None));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, None));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, None));\n}\n\n#[tokio::test]\nasync fn size_hint_stream_closed() {\n let (tx, rx) = mpsc::unbounded_channel();\n\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n\n let mut stream = UnboundedReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (2, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, Some(1)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[tokio::test]\nasync fn size_hint_sender_dropped() {\n let (tx, rx) = mpsc::unbounded_channel();\n\n tx.send(1).unwrap();\n tx.send(2).unwrap();\n\n let mut stream = UnboundedReceiverStream::new(rx);\n drop(tx);\n\n assert_eq!(stream.size_hint(), (2, Some(2)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (1, Some(1)));\n stream.next().await;\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[test]\nfn size_hint_stream_instantly_closed() {\n let (_tx, rx) = mpsc::unbounded_channel::();\n\n let mut stream = UnboundedReceiverStream::new(rx);\n stream.close();\n\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n" }, { "path": "tokio-stream/tests/stream_chain.rs", "content": "use tokio_stream::{self as stream, Stream, StreamExt};\nuse tokio_test::{assert_pending, assert_ready, task};\n\nmod support {\n pub(crate) mod mpsc;\n}\n\nuse support::mpsc;\nuse tokio_stream::adapters::Chain;\n\n#[tokio::test]\nasync fn basic_usage() {\n let one = stream::iter(vec![1, 2, 3]);\n let two = stream::iter(vec![4, 5, 6]);\n\n let mut stream = visibility_test(one, two);\n\n assert_eq!(stream.size_hint(), (6, Some(6)));\n assert_eq!(stream.next().await, Some(1));\n\n assert_eq!(stream.size_hint(), (5, Some(5)));\n assert_eq!(stream.next().await, Some(2));\n\n assert_eq!(stream.size_hint(), (4, Some(4)));\n assert_eq!(stream.next().await, Some(3));\n\n assert_eq!(stream.size_hint(), (3, Some(3)));\n assert_eq!(stream.next().await, Some(4));\n\n assert_eq!(stream.size_hint(), (2, Some(2)));\n assert_eq!(stream.next().await, Some(5));\n\n assert_eq!(stream.size_hint(), (1, Some(1)));\n assert_eq!(stream.next().await, Some(6));\n\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(stream.next().await, None);\n\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(stream.next().await, None);\n}\n\nfn visibility_test(s1: S1, s2: S2) -> Chain\nwhere\n S1: Stream,\n S2: Stream,\n{\n s1.chain(s2)\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // Block on https://github.com/tokio-rs/tokio/issues/6860\nasync fn pending_first() {\n let (tx1, rx1) = mpsc::unbounded_channel_stream();\n let (tx2, rx2) = mpsc::unbounded_channel_stream();\n\n let mut stream = task::spawn(rx1.chain(rx2));\n assert_eq!(stream.size_hint(), (0, None));\n\n assert_pending!(stream.poll_next());\n\n tx2.send(2).unwrap();\n assert!(!stream.is_woken());\n\n assert_pending!(stream.poll_next());\n\n tx1.send(1).unwrap();\n assert!(stream.is_woken());\n assert_eq!(Some(1), assert_ready!(stream.poll_next()));\n\n assert_pending!(stream.poll_next());\n\n drop(tx1);\n\n assert_eq!(stream.size_hint(), (0, None));\n\n assert!(stream.is_woken());\n assert_eq!(Some(2), assert_ready!(stream.poll_next()));\n\n assert_eq!(stream.size_hint(), (0, None));\n\n drop(tx2);\n\n assert_eq!(stream.size_hint(), (0, None));\n assert_eq!(None, assert_ready!(stream.poll_next()));\n}\n\n#[test]\nfn size_overflow() {\n struct Monster;\n\n impl tokio_stream::Stream for Monster {\n type Item = ();\n fn poll_next(\n self: std::pin::Pin<&mut Self>,\n _cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n panic!()\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (usize::MAX, Some(usize::MAX))\n }\n }\n\n let m1 = Monster;\n let m2 = Monster;\n let m = m1.chain(m2);\n assert_eq!(m.size_hint(), (usize::MAX, None));\n}\n" }, { "path": "tokio-stream/tests/stream_chunks_timeout.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse futures::FutureExt;\nuse std::error::Error;\nuse tokio::time;\nuse tokio::time::Duration;\nuse tokio_stream::{self as stream, StreamExt};\nuse tokio_test::assert_pending;\nuse tokio_test::task;\n\n#[tokio::test(start_paused = true)]\nasync fn stream_chunks_remainder() -> Result<(), Box> {\n let stream1 =\n stream::iter([5]).then(move |n| time::sleep(Duration::from_secs(1)).map(move |_| n));\n\n let inner = stream::iter([1, 2, 3, 4]).chain(stream1);\n tokio::pin!(inner);\n\n let chunked = (&mut inner).chunks_timeout(10, Duration::from_millis(20));\n\n let mut chunked = task::spawn(chunked);\n assert_pending!(chunked.poll_next());\n\n let remainder = chunked.enter(|_, stream| stream.into_remainder());\n\n assert_eq!(remainder, vec![1, 2, 3, 4]);\n time::advance(Duration::from_secs(2)).await;\n assert_eq!(inner.next().await, Some(5));\n Ok(())\n}\n" }, { "path": "tokio-stream/tests/stream_close.rs", "content": "use tokio_stream::{StreamExt, StreamNotifyClose};\n\n#[tokio::test]\nasync fn basic_usage() {\n let mut stream = StreamNotifyClose::new(tokio_stream::iter(vec![0, 1]));\n\n assert_eq!(stream.next().await, Some(Some(0)));\n assert_eq!(stream.next().await, Some(Some(1)));\n assert_eq!(stream.next().await, Some(None));\n assert_eq!(stream.next().await, None);\n}\n" }, { "path": "tokio-stream/tests/stream_collect.rs", "content": "use std::collections::{BTreeMap, BTreeSet, BinaryHeap, HashMap, HashSet, LinkedList, VecDeque};\n\nuse tokio_stream::{self as stream, StreamExt};\nuse tokio_test::{assert_pending, assert_ready, assert_ready_err, assert_ready_ok, task};\n\nmod support {\n pub(crate) mod mpsc;\n}\n\nuse support::mpsc;\n\n#[allow(clippy::let_unit_value)]\n#[tokio::test]\nasync fn empty_unit() {\n // Drains the stream.\n let mut iter = vec![(), (), ()].into_iter();\n let _: () = stream::iter(&mut iter).collect().await;\n assert!(iter.next().is_none());\n}\n\n#[tokio::test]\nasync fn empty_vec() {\n let coll: Vec = stream::empty().collect().await;\n assert!(coll.is_empty());\n}\n\n#[tokio::test]\nasync fn empty_box_slice() {\n let coll: Box<[u32]> = stream::empty().collect().await;\n assert!(coll.is_empty());\n}\n\n#[tokio::test]\nasync fn empty_string() {\n let coll: String = stream::empty::<&str>().collect().await;\n assert!(coll.is_empty());\n}\n\n#[tokio::test]\nasync fn empty_result() {\n let coll: Result, &str> = stream::empty().collect().await;\n assert_eq!(Ok(vec![]), coll);\n}\n\n#[tokio::test]\nasync fn collect_vec_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(1).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(2).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(vec![1, 2], coll);\n}\n\n#[tokio::test]\nasync fn collect_vecdeque_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n let xs = [1, 2, 42, 3];\n for x in xs {\n tx.send(x).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n }\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n\n assert_eq!(coll, VecDeque::from(xs));\n assert_eq!(coll.into_iter().collect::>(), xs);\n}\n\n#[tokio::test]\nasync fn collect_linkedlist_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n let xs = [1, 2, 42, 3];\n for x in xs {\n tx.send(x).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n }\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n\n assert_eq!(coll, LinkedList::from(xs));\n assert_eq!(coll.into_iter().collect::>(), xs);\n}\n\n#[tokio::test]\nasync fn collect_btreeset_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(2).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(1).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(BTreeSet::from([1, 2]), coll);\n}\n\n#[tokio::test]\nasync fn collect_btreemap_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send((3, 4)).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send((1, 2)).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(BTreeMap::from([(1, 2), (3, 4)]), coll);\n}\n\n#[tokio::test]\nasync fn collect_hashset_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(1).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(2).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(HashSet::from([1, 2]), coll);\n}\n\n#[tokio::test]\nasync fn collect_hashmap_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send((1, 2)).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send((3, 4)).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(HashMap::from([(1, 2), (3, 4)]), coll);\n}\n\n#[tokio::test]\nasync fn collect_binaryheap_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(2).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(1).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(vec![1, 2], coll.into_sorted_vec());\n}\n\n#[tokio::test]\nasync fn collect_string_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n\n let mut fut = task::spawn(rx.collect::());\n\n assert_pending!(fut.poll());\n\n tx.send(\"hello \".to_string()).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(\"world\".to_string()).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(\"hello world\", coll);\n}\n\n#[tokio::test]\nasync fn collect_str_items() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n\n let mut fut = task::spawn(rx.collect::());\n\n assert_pending!(fut.poll());\n\n tx.send(\"hello \").unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(\"world\").unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready!(fut.poll());\n assert_eq!(\"hello world\", coll);\n}\n\n#[tokio::test]\nasync fn collect_results_ok() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(Ok(\"hello \")).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(Ok(\"world\")).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n drop(tx);\n assert!(fut.is_woken());\n let coll = assert_ready_ok!(fut.poll());\n assert_eq!(\"hello world\", coll);\n}\n\n#[tokio::test]\nasync fn collect_results_err() {\n let (tx, rx) = mpsc::unbounded_channel_stream();\n\n let mut fut = task::spawn(rx.collect::>());\n\n assert_pending!(fut.poll());\n\n tx.send(Ok(\"hello \")).unwrap();\n assert!(fut.is_woken());\n assert_pending!(fut.poll());\n\n tx.send(Err(\"oh no\")).unwrap();\n assert!(fut.is_woken());\n let err = assert_ready_err!(fut.poll());\n assert_eq!(\"oh no\", err);\n}\n" }, { "path": "tokio-stream/tests/stream_empty.rs", "content": "use tokio_stream::{self as stream, Stream, StreamExt};\n\n#[tokio::test]\nasync fn basic_usage() {\n let mut stream = stream::empty::();\n\n for _ in 0..2 {\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(None, stream.next().await);\n }\n}\n" }, { "path": "tokio-stream/tests/stream_fuse.rs", "content": "use tokio_stream::{Stream, StreamExt};\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n// a stream which alternates between Some and None\nstruct Alternate {\n state: i32,\n}\n\nimpl Stream for Alternate {\n type Item = i32;\n\n fn poll_next(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n let val = self.state;\n self.state += 1;\n\n // if it's even, Some(i32), else None\n if val % 2 == 0 {\n Poll::Ready(Some(val))\n } else {\n Poll::Ready(None)\n }\n }\n}\n\n#[tokio::test]\nasync fn basic_usage() {\n let mut stream = Alternate { state: 0 };\n\n // the stream goes back and forth\n assert_eq!(stream.next().await, Some(0));\n assert_eq!(stream.next().await, None);\n assert_eq!(stream.next().await, Some(2));\n assert_eq!(stream.next().await, None);\n\n // however, once it is fused\n let mut stream = stream.fuse();\n\n assert_eq!(stream.size_hint(), (0, None));\n assert_eq!(stream.next().await, Some(4));\n\n assert_eq!(stream.size_hint(), (0, None));\n assert_eq!(stream.next().await, None);\n\n // it will always return `None` after the first time.\n assert_eq!(stream.size_hint(), (0, Some(0)));\n assert_eq!(stream.next().await, None);\n assert_eq!(stream.size_hint(), (0, Some(0)));\n}\n\n#[tokio::test]\n#[cfg(feature = \"time\")]\nasync fn interval_stream_is_never_terminated() {\n use futures_core::stream::FusedStream;\n use tokio_stream::wrappers::IntervalStream;\n\n let interval = tokio::time::interval(std::time::Duration::from_millis(1));\n let stream = IntervalStream::new(interval);\n\n assert!(!stream.is_terminated());\n}\n" }, { "path": "tokio-stream/tests/stream_iter.rs", "content": "use tokio_stream as stream;\nuse tokio_test::task;\n\nuse std::iter;\n\n#[tokio::test]\nasync fn coop() {\n let mut stream = task::spawn(stream::iter(iter::repeat(1)));\n\n for _ in 0..10_000 {\n if stream.poll_next().is_pending() {\n assert!(stream.is_woken());\n return;\n }\n }\n\n panic!(\"did not yield\");\n}\n" }, { "path": "tokio-stream/tests/stream_merge.rs", "content": "use tokio_stream::{self as stream, Stream, StreamExt};\nuse tokio_test::task;\nuse tokio_test::{assert_pending, assert_ready};\n\nmod support {\n pub(crate) mod mpsc;\n}\n\nuse support::mpsc;\n\n#[tokio::test]\nasync fn merge_sync_streams() {\n let mut s = stream::iter(vec![0, 2, 4, 6]).merge(stream::iter(vec![1, 3, 5]));\n\n for i in 0..7 {\n let rem = 7 - i;\n assert_eq!(s.size_hint(), (rem, Some(rem)));\n assert_eq!(Some(i), s.next().await);\n }\n\n assert!(s.next().await.is_none());\n}\n\n#[tokio::test]\nasync fn merge_async_streams() {\n let (tx1, rx1) = mpsc::unbounded_channel_stream();\n let (tx2, rx2) = mpsc::unbounded_channel_stream();\n\n let mut rx = task::spawn(rx1.merge(rx2));\n\n assert_eq!(rx.size_hint(), (0, None));\n\n assert_pending!(rx.poll_next());\n\n tx1.send(1).unwrap();\n\n assert!(rx.is_woken());\n assert_eq!(Some(1), assert_ready!(rx.poll_next()));\n\n assert_pending!(rx.poll_next());\n tx2.send(2).unwrap();\n\n assert!(rx.is_woken());\n assert_eq!(Some(2), assert_ready!(rx.poll_next()));\n assert_pending!(rx.poll_next());\n\n drop(tx1);\n assert!(rx.is_woken());\n assert_pending!(rx.poll_next());\n\n tx2.send(3).unwrap();\n assert!(rx.is_woken());\n assert_eq!(Some(3), assert_ready!(rx.poll_next()));\n assert_pending!(rx.poll_next());\n\n drop(tx2);\n assert!(rx.is_woken());\n assert_eq!(None, assert_ready!(rx.poll_next()));\n}\n\n#[test]\nfn size_overflow() {\n struct Monster;\n\n impl tokio_stream::Stream for Monster {\n type Item = ();\n fn poll_next(\n self: std::pin::Pin<&mut Self>,\n _cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n panic!()\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (usize::MAX, Some(usize::MAX))\n }\n }\n\n let m1 = Monster;\n let m2 = Monster;\n let m = m1.merge(m2);\n assert_eq!(m.size_hint(), (usize::MAX, None));\n}\n" }, { "path": "tokio-stream/tests/stream_once.rs", "content": "use tokio_stream::{self as stream, Stream, StreamExt};\n\n#[tokio::test]\nasync fn basic_usage() {\n let mut one = stream::once(1);\n\n assert_eq!(one.size_hint(), (1, Some(1)));\n assert_eq!(Some(1), one.next().await);\n\n assert_eq!(one.size_hint(), (0, Some(0)));\n assert_eq!(None, one.next().await);\n}\n" }, { "path": "tokio-stream/tests/stream_panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"time\", not(target_os = \"wasi\")))] // Wasi does not support panic recovery\n#![cfg(panic = \"unwind\")]\n\nuse parking_lot::{const_mutex, Mutex};\nuse std::error::Error;\nuse std::panic;\nuse std::sync::Arc;\nuse tokio::time::Duration;\nuse tokio_stream::{self as stream, StreamExt};\n\nfn test_panic(func: Func) -> Option {\n static PANIC_MUTEX: Mutex<()> = const_mutex(());\n\n {\n let _guard = PANIC_MUTEX.lock();\n let panic_file: Arc>> = Arc::new(Mutex::new(None));\n\n let prev_hook = panic::take_hook();\n {\n let panic_file = panic_file.clone();\n panic::set_hook(Box::new(move |panic_info| {\n let panic_location = panic_info.location().unwrap();\n panic_file\n .lock()\n .clone_from(&Some(panic_location.file().to_string()));\n }));\n }\n\n let result = panic::catch_unwind(func);\n // Return to the previously set panic hook (maybe default) so that we get nice error\n // messages in the tests.\n panic::set_hook(prev_hook);\n\n if result.is_err() {\n panic_file.lock().clone()\n } else {\n None\n }\n }\n}\n\n#[test]\nfn stream_chunks_timeout_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let iter = vec![1, 2, 3].into_iter();\n let stream0 = stream::iter(iter);\n\n let _chunk_stream = stream0.chunks_timeout(0, Duration::from_secs(2));\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n" }, { "path": "tokio-stream/tests/stream_pending.rs", "content": "use tokio_stream::{self as stream, Stream, StreamExt};\nuse tokio_test::{assert_pending, task};\n\n#[tokio::test]\nasync fn basic_usage() {\n let mut stream = stream::pending::();\n\n for _ in 0..2 {\n assert_eq!(stream.size_hint(), (0, None));\n\n let mut next = task::spawn(async { stream.next().await });\n assert_pending!(next.poll());\n }\n}\n" }, { "path": "tokio-stream/tests/stream_stream_map.rs", "content": "use futures::stream::iter;\nuse tokio_stream::{self as stream, pending, Stream, StreamExt, StreamMap};\nuse tokio_test::{assert_ok, assert_pending, assert_ready, task};\n\nuse std::future::{poll_fn, Future};\nuse std::pin::{pin, Pin};\nuse std::task::Poll;\n\nmod support {\n pub(crate) mod mpsc;\n}\n\nuse support::mpsc;\n\nmacro_rules! assert_ready_some {\n ($($t:tt)*) => {\n match assert_ready!($($t)*) {\n Some(v) => v,\n None => panic!(\"expected `Some`, got `None`\"),\n }\n };\n}\n\nmacro_rules! assert_ready_none {\n ($($t:tt)*) => {\n match assert_ready!($($t)*) {\n None => {}\n Some(v) => panic!(\"expected `None`, got `Some({:?})`\", v),\n }\n };\n}\n\n#[tokio::test]\nasync fn empty() {\n let mut map = StreamMap::<&str, stream::Pending<()>>::new();\n\n assert_eq!(map.len(), 0);\n assert!(map.is_empty());\n\n assert!(map.next().await.is_none());\n assert!(map.next().await.is_none());\n\n assert!(map.remove(\"foo\").is_none());\n}\n\n#[tokio::test]\nasync fn single_entry() {\n let mut map = task::spawn(StreamMap::new());\n let (tx, rx) = mpsc::unbounded_channel_stream();\n let rx = Box::pin(rx);\n\n assert_ready_none!(map.poll_next());\n\n assert!(map.insert(\"foo\", rx).is_none());\n assert!(map.contains_key(\"foo\"));\n assert!(!map.contains_key(\"bar\"));\n\n assert_eq!(map.len(), 1);\n assert!(!map.is_empty());\n\n assert_pending!(map.poll_next());\n\n assert_ok!(tx.send(1));\n\n assert!(map.is_woken());\n let (k, v) = assert_ready_some!(map.poll_next());\n assert_eq!(k, \"foo\");\n assert_eq!(v, 1);\n\n assert_pending!(map.poll_next());\n\n assert_ok!(tx.send(2));\n\n assert!(map.is_woken());\n let (k, v) = assert_ready_some!(map.poll_next());\n assert_eq!(k, \"foo\");\n assert_eq!(v, 2);\n\n assert_pending!(map.poll_next());\n drop(tx);\n assert!(map.is_woken());\n assert_ready_none!(map.poll_next());\n}\n\n#[tokio::test]\nasync fn multiple_entries() {\n let mut map = task::spawn(StreamMap::new());\n let (tx1, rx1) = mpsc::unbounded_channel_stream();\n let (tx2, rx2) = mpsc::unbounded_channel_stream();\n\n let rx1 = Box::pin(rx1);\n let rx2 = Box::pin(rx2);\n\n map.insert(\"foo\", rx1);\n map.insert(\"bar\", rx2);\n\n assert_pending!(map.poll_next());\n\n assert_ok!(tx1.send(1));\n\n assert!(map.is_woken());\n let (k, v) = assert_ready_some!(map.poll_next());\n assert_eq!(k, \"foo\");\n assert_eq!(v, 1);\n\n assert_pending!(map.poll_next());\n\n assert_ok!(tx2.send(2));\n\n assert!(map.is_woken());\n let (k, v) = assert_ready_some!(map.poll_next());\n assert_eq!(k, \"bar\");\n assert_eq!(v, 2);\n\n assert_pending!(map.poll_next());\n\n assert_ok!(tx1.send(3));\n assert_ok!(tx2.send(4));\n\n assert!(map.is_woken());\n\n // Given the randomization, there is no guarantee what order the values will\n // be received in.\n let mut v = (0..2)\n .map(|_| assert_ready_some!(map.poll_next()))\n .collect::>();\n\n assert_pending!(map.poll_next());\n\n v.sort_unstable();\n assert_eq!(v[0].0, \"bar\");\n assert_eq!(v[0].1, 4);\n assert_eq!(v[1].0, \"foo\");\n assert_eq!(v[1].1, 3);\n\n drop(tx1);\n assert!(map.is_woken());\n assert_pending!(map.poll_next());\n drop(tx2);\n\n assert_ready_none!(map.poll_next());\n}\n\n#[tokio::test]\nasync fn insert_remove() {\n let mut map = task::spawn(StreamMap::new());\n let (tx, rx) = mpsc::unbounded_channel_stream();\n\n let rx = Box::pin(rx);\n\n assert_ready_none!(map.poll_next());\n\n assert!(map.insert(\"foo\", rx).is_none());\n let rx = map.remove(\"foo\").unwrap();\n\n assert_ok!(tx.send(1));\n\n assert!(!map.is_woken());\n assert_ready_none!(map.poll_next());\n\n assert!(map.insert(\"bar\", rx).is_none());\n\n let v = assert_ready_some!(map.poll_next());\n assert_eq!(v.0, \"bar\");\n assert_eq!(v.1, 1);\n\n assert!(map.remove(\"bar\").is_some());\n assert_ready_none!(map.poll_next());\n\n assert!(map.is_empty());\n assert_eq!(0, map.len());\n}\n\n#[tokio::test]\nasync fn replace() {\n let mut map = task::spawn(StreamMap::new());\n let (tx1, rx1) = mpsc::unbounded_channel_stream();\n let (tx2, rx2) = mpsc::unbounded_channel_stream();\n\n let rx1 = Box::pin(rx1);\n let rx2 = Box::pin(rx2);\n\n assert!(map.insert(\"foo\", rx1).is_none());\n\n assert_pending!(map.poll_next());\n\n let _rx1 = map.insert(\"foo\", rx2).unwrap();\n\n assert_pending!(map.poll_next());\n\n tx1.send(1).unwrap();\n assert_pending!(map.poll_next());\n\n tx2.send(2).unwrap();\n assert!(map.is_woken());\n let v = assert_ready_some!(map.poll_next());\n assert_eq!(v.0, \"foo\");\n assert_eq!(v.1, 2);\n}\n\n#[test]\nfn size_hint_with_upper() {\n let mut map = StreamMap::new();\n\n map.insert(\"a\", stream::iter(vec![1]));\n map.insert(\"b\", stream::iter(vec![1, 2]));\n map.insert(\"c\", stream::iter(vec![1, 2, 3]));\n\n assert_eq!(3, map.len());\n assert!(!map.is_empty());\n\n let size_hint = map.size_hint();\n assert_eq!(size_hint, (6, Some(6)));\n}\n\n#[test]\nfn size_hint_without_upper() {\n let mut map = StreamMap::new();\n\n map.insert(\"a\", pin_box(stream::iter(vec![1])));\n map.insert(\"b\", pin_box(stream::iter(vec![1, 2])));\n map.insert(\"c\", pin_box(pending()));\n\n let size_hint = map.size_hint();\n assert_eq!(size_hint, (3, None));\n}\n\n#[test]\nfn new_capacity_zero() {\n let map = StreamMap::<&str, stream::Pending<()>>::new();\n assert_eq!(0, map.capacity());\n\n assert!(map.keys().next().is_none());\n}\n\n#[test]\nfn with_capacity() {\n let map = StreamMap::<&str, stream::Pending<()>>::with_capacity(10);\n assert!(10 <= map.capacity());\n\n assert!(map.keys().next().is_none());\n}\n\n#[test]\nfn iter_keys() {\n let mut map = StreamMap::new();\n\n map.insert(\"a\", pending::());\n map.insert(\"b\", pending());\n map.insert(\"c\", pending());\n\n let mut keys = map.keys().collect::>();\n keys.sort_unstable();\n\n assert_eq!(&keys[..], &[&\"a\", &\"b\", &\"c\"]);\n}\n\n#[test]\nfn iter_values() {\n let mut map = StreamMap::new();\n\n map.insert(\"a\", stream::iter(vec![1]));\n map.insert(\"b\", stream::iter(vec![1, 2]));\n map.insert(\"c\", stream::iter(vec![1, 2, 3]));\n\n let mut size_hints = map.values().map(|s| s.size_hint().0).collect::>();\n\n size_hints.sort_unstable();\n\n assert_eq!(&size_hints[..], &[1, 2, 3]);\n}\n\n#[test]\nfn iter_values_mut() {\n let mut map = StreamMap::new();\n\n map.insert(\"a\", stream::iter(vec![1]));\n map.insert(\"b\", stream::iter(vec![1, 2]));\n map.insert(\"c\", stream::iter(vec![1, 2, 3]));\n\n let mut size_hints = map\n .values_mut()\n .map(|s: &mut _| s.size_hint().0)\n .collect::>();\n\n size_hints.sort_unstable();\n\n assert_eq!(&size_hints[..], &[1, 2, 3]);\n}\n\n#[test]\nfn clear() {\n let mut map = task::spawn(StreamMap::new());\n\n map.insert(\"a\", stream::iter(vec![1]));\n map.insert(\"b\", stream::iter(vec![1, 2]));\n map.insert(\"c\", stream::iter(vec![1, 2, 3]));\n\n assert_ready_some!(map.poll_next());\n\n map.clear();\n\n assert_ready_none!(map.poll_next());\n assert!(map.is_empty());\n}\n\n#[test]\nfn contains_key_borrow() {\n let mut map = StreamMap::new();\n map.insert(\"foo\".to_string(), pending::<()>());\n\n assert!(map.contains_key(\"foo\"));\n}\n\n#[test]\nfn one_ready_many_none() {\n // Run a few times because of randomness\n for _ in 0..100 {\n let mut map = task::spawn(StreamMap::new());\n\n map.insert(0, pin_box(stream::empty()));\n map.insert(1, pin_box(stream::empty()));\n map.insert(2, pin_box(stream::once(\"hello\")));\n map.insert(3, pin_box(stream::pending()));\n\n let v = assert_ready_some!(map.poll_next());\n assert_eq!(v, (2, \"hello\"));\n }\n}\n\nfn pin_box + 'static, U>(s: T) -> Pin>> {\n Box::pin(s)\n}\n\ntype UsizeStream = Pin + Send>>;\n\n#[tokio::test]\nasync fn poll_next_many_zero() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(pending()) as UsizeStream);\n\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut vec![], 0)).await;\n\n assert_eq!(n, 0);\n}\n\n#[tokio::test]\nasync fn poll_next_many_empty() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut vec![], 1)).await;\n\n assert_eq!(n, 0);\n}\n\n#[tokio::test]\nasync fn poll_next_many_pending() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(pending()) as UsizeStream);\n\n let mut is_pending = false;\n poll_fn(|cx| {\n let poll = stream_map.poll_next_many(cx, &mut vec![], 1);\n\n is_pending = poll.is_pending();\n\n Poll::Ready(())\n })\n .await;\n\n assert!(is_pending);\n}\n\n#[tokio::test]\nasync fn poll_next_many_not_enough() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 3)).await;\n\n assert_eq!(n, 2);\n assert_eq!(buffer.len(), 2);\n assert!(buffer.contains(&(0, 0)));\n assert!(buffer.contains(&(1, 1)));\n}\n\n#[tokio::test]\nasync fn poll_next_many_enough() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 2)).await;\n\n assert_eq!(n, 2);\n assert_eq!(buffer.len(), 2);\n assert!(buffer.contains(&(0, 0)));\n assert!(buffer.contains(&(1, 1)));\n}\n\n#[tokio::test]\nasync fn poll_next_many_correctly_loops_around() {\n for _ in 0..10 {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([0usize, 1].into_iter())) as UsizeStream);\n stream_map.insert(2, Box::pin(iter([0usize, 1, 2].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 3)).await;\n assert_eq!(n, 3);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![0, 0, 0]\n );\n\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 2)).await;\n assert_eq!(n, 2);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![1, 1]\n );\n\n let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 1)).await;\n assert_eq!(n, 1);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![2]\n );\n }\n}\n\n#[tokio::test]\nasync fn next_many_zero() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(pending()) as UsizeStream);\n\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut vec![], 0)).poll(cx)).await;\n\n assert_eq!(n, 0);\n}\n\n#[tokio::test]\nasync fn next_many_empty() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n let n = stream_map.next_many(&mut vec![], 1).await;\n\n assert_eq!(n, 0);\n}\n\n#[tokio::test]\nasync fn next_many_pending() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(pending()) as UsizeStream);\n\n let mut is_pending = false;\n poll_fn(|cx| {\n let poll = pin!(stream_map.next_many(&mut vec![], 1)).poll(cx);\n\n is_pending = poll.is_pending();\n\n Poll::Ready(())\n })\n .await;\n\n assert!(is_pending);\n}\n\n#[tokio::test]\nasync fn next_many_not_enough() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 3)).poll(cx)).await;\n\n assert_eq!(n, 2);\n assert_eq!(buffer.len(), 2);\n assert!(buffer.contains(&(0, 0)));\n assert!(buffer.contains(&(1, 1)));\n}\n\n#[tokio::test]\nasync fn next_many_enough() {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 2)).poll(cx)).await;\n\n assert_eq!(n, 2);\n assert_eq!(buffer.len(), 2);\n assert!(buffer.contains(&(0, 0)));\n assert!(buffer.contains(&(1, 1)));\n}\n\n#[tokio::test]\nasync fn next_many_correctly_loops_around() {\n for _ in 0..10 {\n let mut stream_map: StreamMap = StreamMap::new();\n\n stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);\n stream_map.insert(1, Box::pin(iter([0usize, 1].into_iter())) as UsizeStream);\n stream_map.insert(2, Box::pin(iter([0usize, 1, 2].into_iter())) as UsizeStream);\n\n let mut buffer = vec![];\n\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 3)).poll(cx)).await;\n assert_eq!(n, 3);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![0, 0, 0]\n );\n\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 2)).poll(cx)).await;\n assert_eq!(n, 2);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![1, 1]\n );\n\n let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 1)).poll(cx)).await;\n assert_eq!(n, 1);\n assert_eq!(\n std::mem::take(&mut buffer)\n .into_iter()\n .map(|(_, v)| v)\n .collect::>(),\n vec![2]\n );\n }\n}\n" }, { "path": "tokio-stream/tests/stream_timeout.rs", "content": "#![cfg(all(feature = \"time\", feature = \"sync\", feature = \"io-util\"))]\n\nuse tokio::time::{self, sleep, Duration};\nuse tokio_stream::StreamExt;\nuse tokio_test::*;\n\nuse futures::stream;\n\nasync fn maybe_sleep(idx: i32) -> i32 {\n if idx % 2 == 0 {\n sleep(ms(200)).await;\n }\n idx\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n\n#[tokio::test]\nasync fn basic_usage() {\n time::pause();\n\n // Items 2 and 4 time out. If we run the stream until it completes,\n // we end up with the following items:\n //\n // [Ok(1), Err(Elapsed), Ok(2), Ok(3), Err(Elapsed), Ok(4)]\n\n let stream = stream::iter(1..=4).then(maybe_sleep).timeout(ms(100));\n let mut stream = task::spawn(stream);\n\n // First item completes immediately\n assert_ready_eq!(stream.poll_next(), Some(Ok(1)));\n\n // Second item is delayed 200ms, times out after 100ms\n assert_pending!(stream.poll_next());\n\n time::advance(ms(150)).await;\n let v = assert_ready!(stream.poll_next());\n assert!(v.unwrap().is_err());\n\n assert_pending!(stream.poll_next());\n\n time::advance(ms(100)).await;\n assert_ready_eq!(stream.poll_next(), Some(Ok(2)));\n\n // Third item is ready immediately\n assert_ready_eq!(stream.poll_next(), Some(Ok(3)));\n\n // Fourth item is delayed 200ms, times out after 100ms\n assert_pending!(stream.poll_next());\n\n time::advance(ms(60)).await;\n assert_pending!(stream.poll_next()); // nothing ready yet\n\n time::advance(ms(60)).await;\n let v = assert_ready!(stream.poll_next());\n assert!(v.unwrap().is_err()); // timeout!\n\n time::advance(ms(120)).await;\n assert_ready_eq!(stream.poll_next(), Some(Ok(4)));\n\n // Done.\n assert_ready_eq!(stream.poll_next(), None);\n}\n\n#[tokio::test]\nasync fn return_elapsed_errors_only_once() {\n time::pause();\n\n let stream = stream::iter(1..=3).then(maybe_sleep).timeout(ms(50));\n let mut stream = task::spawn(stream);\n\n // First item completes immediately\n assert_ready_eq!(stream.poll_next(), Some(Ok(1)));\n\n // Second item is delayed 200ms, times out after 50ms. Only one `Elapsed`\n // error is returned.\n assert_pending!(stream.poll_next());\n //\n time::advance(ms(51)).await;\n let v = assert_ready!(stream.poll_next());\n assert!(v.unwrap().is_err()); // timeout!\n\n // deadline elapses again, but no error is returned\n time::advance(ms(50)).await;\n assert_pending!(stream.poll_next());\n\n time::advance(ms(100)).await;\n assert_ready_eq!(stream.poll_next(), Some(Ok(2)));\n assert_ready_eq!(stream.poll_next(), Some(Ok(3)));\n\n // Done\n assert_ready_eq!(stream.poll_next(), None);\n}\n\n#[tokio::test]\nasync fn no_timeouts() {\n let stream = stream::iter(vec![1, 3, 5])\n .then(maybe_sleep)\n .timeout(ms(100));\n\n let mut stream = task::spawn(stream);\n\n assert_ready_eq!(stream.poll_next(), Some(Ok(1)));\n assert_ready_eq!(stream.poll_next(), Some(Ok(3)));\n assert_ready_eq!(stream.poll_next(), Some(Ok(5)));\n assert_ready_eq!(stream.poll_next(), None);\n}\n" }, { "path": "tokio-stream/tests/support/mpsc.rs", "content": "use async_stream::stream;\nuse tokio::sync::mpsc::{self, UnboundedSender};\nuse tokio_stream::Stream;\n\npub fn unbounded_channel_stream() -> (UnboundedSender, impl Stream) {\n let (tx, mut rx) = mpsc::unbounded_channel();\n\n let stream = stream! {\n while let Some(item) = rx.recv().await {\n yield item;\n }\n };\n\n (tx, stream)\n}\n" }, { "path": "tokio-stream/tests/time_throttle.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"time\", feature = \"sync\", feature = \"io-util\"))]\n\nuse tokio::time;\nuse tokio_stream::StreamExt;\nuse tokio_test::*;\n\nuse std::time::Duration;\n\n#[tokio::test]\nasync fn usage() {\n time::pause();\n\n let mut stream = task::spawn(futures::stream::repeat(()).throttle(Duration::from_millis(100)));\n\n assert_ready!(stream.poll_next());\n assert_pending!(stream.poll_next());\n\n time::advance(Duration::from_millis(90)).await;\n\n assert_pending!(stream.poll_next());\n\n time::advance(Duration::from_millis(101)).await;\n\n assert!(stream.is_woken());\n\n assert_ready!(stream.poll_next());\n}\n" }, { "path": "tokio-stream/tests/watch.rs", "content": "#![cfg(feature = \"sync\")]\n\nuse tokio::sync::watch;\nuse tokio_stream::wrappers::WatchStream;\nuse tokio_stream::StreamExt;\nuse tokio_test::assert_pending;\nuse tokio_test::task::spawn;\n\n#[tokio::test]\nasync fn watch_stream_message_not_twice() {\n let (tx, rx) = watch::channel(\"hello\");\n\n let mut counter = 0;\n let mut stream = WatchStream::new(rx).map(move |payload| {\n println!(\"{payload}\");\n if payload == \"goodbye\" {\n counter += 1;\n }\n if counter >= 2 {\n panic!(\"too many goodbyes\");\n }\n });\n\n let task = tokio::spawn(async move { while stream.next().await.is_some() {} });\n\n // Send goodbye just once\n tx.send(\"goodbye\").unwrap();\n\n drop(tx);\n task.await.unwrap();\n}\n\n#[tokio::test]\nasync fn watch_stream_from_rx() {\n let (tx, rx) = watch::channel(\"hello\");\n\n let mut stream = WatchStream::from(rx);\n\n assert_eq!(stream.next().await.unwrap(), \"hello\");\n\n tx.send(\"bye\").unwrap();\n\n assert_eq!(stream.next().await.unwrap(), \"bye\");\n}\n\n#[tokio::test]\nasync fn watch_stream_from_changes() {\n let (tx, rx) = watch::channel(\"hello\");\n\n let mut stream = WatchStream::from_changes(rx);\n\n assert_pending!(spawn(&mut stream).poll_next());\n\n tx.send(\"bye\").unwrap();\n\n assert_eq!(stream.next().await.unwrap(), \"bye\");\n}\n" }, { "path": "tokio-test/CHANGELOG.md", "content": "# 0.4.5 (January 4th, 2026)\n\n### Added\n\n- test: add `io::Builder::name` for better panic messages ([#7212])\n\n### Fixed\n\n- test: make `Spawn` forward `size_hint` ([#6607])\n\n### Changed\n\n- test: remove unused `async-stream` and `bytes` dependencies ([#7214])\n\n[#6607]: https://github.com/tokio-rs/tokio/pull/6607\n[#7212]: https://github.com/tokio-rs/tokio/pull/7212\n[#7214]: https://github.com/tokio-rs/tokio/pull/7214\n\n# 0.4.4 (March 14, 2024)\n\n- task: mark `Spawn` as `#[must_use]` ([#6371])\n- test: increase MSRV to 1.63 ([#6126])\n- test: update category slug ([#5953])\n\n[#5953]: https://github.com/tokio-rs/tokio/pull/5953\n[#6126]: https://github.com/tokio-rs/tokio/pull/6126\n[#6371]: https://github.com/tokio-rs/tokio/pull/6371\n\n# 0.4.3 (August 23, 2023)\n\n- deps: fix minimum required version of `async-stream` ([#5347])\n- deps: fix minimum required version of `tokio-stream` ([#4376])\n- docs: improve `tokio_test::task` docs ([#5132])\n- io: fetch actions from mock handle before write ([#5814])\n- io: fix wait operation on mock ([#5554])\n\n[#4376]: https://github.com/tokio-rs/tokio/pull/4376\n[#5132]: https://github.com/tokio-rs/tokio/pull/5132\n[#5347]: https://github.com/tokio-rs/tokio/pull/5347\n[#5554]: https://github.com/tokio-rs/tokio/pull/5554\n[#5814]: https://github.com/tokio-rs/tokio/pull/5814\n\n# 0.4.2 (May 14, 2021)\n\n- test: add `assert_elapsed!` macro ([#3728])\n\n[#3728]: https://github.com/tokio-rs/tokio/pull/3728\n\n# 0.4.1 (March 10, 2021)\n\n- Fix `io::Mock` to be `Send` and `Sync` ([#3594])\n\n[#3594]: https://github.com/tokio-rs/tokio/pull/3594\n\n# 0.4.0 (December 23, 2020)\n\n- Track `tokio` 1.0 release.\n\n# 0.3.0 (October 15, 2020)\n\n- Track `tokio` 0.3 release.\n\n# 0.2.1 (April 17, 2020)\n\n- Add `Future` and `Stream` implementations for `task::Spawn`.\n\n# 0.2.0 (November 25, 2019)\n\n- Initial release\n" }, { "path": "tokio-test/Cargo.toml", "content": "[package]\nname = \"tokio-test\"\n# When releasing to crates.io:\n# - Remove path dependencies (if any)\n# - Update CHANGELOG.md.\n# - Create \"tokio-test-0.4.x\" git tag.\nversion = \"0.4.5\"\nedition = \"2021\"\nrust-version = \"1.71\"\nauthors = [\"Tokio Contributors \"]\nlicense = \"MIT\"\nrepository = \"https://github.com/tokio-rs/tokio\"\nhomepage = \"https://tokio.rs\"\ndescription = \"\"\"\nTesting utilities for Tokio- and futures-based code\n\"\"\"\ncategories = [\"asynchronous\", \"development-tools::testing\"]\n\n[dependencies]\ntokio = { version = \"1.2.0\", features = [\"rt\", \"sync\", \"time\", \"test-util\"] }\ntokio-stream = \"0.1.1\"\nfutures-core = \"0.3.0\"\n\n[dev-dependencies]\ntokio = { version = \"1.2.0\", features = [\"full\"] }\nfutures-util = \"0.3.0\"\n\n[package.metadata.docs.rs]\nall-features = true\n\n[lints]\nworkspace = true\n" }, { "path": "tokio-test/LICENSE", "content": "MIT License\n\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "tokio-test/README.md", "content": "# tokio-test\n\nTokio and Futures based testing utilities\n\n## License\n\nThis project is licensed under the [MIT license](LICENSE).\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you, shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "tokio-test/src/io.rs", "content": "#![cfg(not(loom))]\n\n//! A mock type implementing [`AsyncRead`] and [`AsyncWrite`].\n//!\n//!\n//! # Overview\n//!\n//! Provides a type that implements [`AsyncRead`] + [`AsyncWrite`] that can be configured\n//! to handle an arbitrary sequence of read and write operations. This is useful\n//! for writing unit tests for networking services as using an actual network\n//! type is fairly non deterministic.\n//!\n//! # Usage\n//!\n//! Attempting to write data that the mock isn't expecting will result in a\n//! panic.\n//!\n//! [`AsyncRead`]: tokio::io::AsyncRead\n//! [`AsyncWrite`]: tokio::io::AsyncWrite\n\nuse tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\nuse tokio::sync::mpsc;\nuse tokio::time::{self, Duration, Instant, Sleep};\nuse tokio_stream::wrappers::UnboundedReceiverStream;\n\nuse futures_core::Stream;\nuse std::collections::VecDeque;\nuse std::fmt;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{self, ready, Poll, Waker};\nuse std::{cmp, io};\n\n/// An I/O object that follows a predefined script.\n///\n/// This value is created by `Builder` and implements `AsyncRead` + `AsyncWrite`. It\n/// follows the scenario described by the builder and panics otherwise.\n#[derive(Debug)]\npub struct Mock {\n inner: Inner,\n}\n\n/// A handle to send additional actions to the related `Mock`.\n#[derive(Debug)]\npub struct Handle {\n tx: mpsc::UnboundedSender,\n}\n\n/// Builds `Mock` instances.\n#[derive(Debug, Clone, Default)]\npub struct Builder {\n // Sequence of actions for the Mock to take\n actions: VecDeque,\n name: String,\n}\n\n#[derive(Debug, Clone)]\nenum Action {\n Read(Vec),\n Write(Vec),\n Wait(Duration),\n // Wrapped in Arc so that Builder can be cloned and Send.\n // Mock is not cloned as does not need to check Rc for ref counts.\n ReadError(Option>),\n WriteError(Option>),\n}\n\nstruct Inner {\n actions: VecDeque,\n waiting: Option,\n sleep: Option>>,\n read_wait: Option,\n rx: UnboundedReceiverStream,\n name: String,\n}\n\nimpl Builder {\n /// Return a new, empty `Builder`.\n pub fn new() -> Self {\n Self::default()\n }\n\n /// Sequence a `read` operation.\n ///\n /// The next operation in the mock's script will be to expect a `read` call\n /// and return `buf`.\n pub fn read(&mut self, buf: &[u8]) -> &mut Self {\n self.actions.push_back(Action::Read(buf.into()));\n self\n }\n\n /// Sequence a `read` operation that produces an error.\n ///\n /// The next operation in the mock's script will be to expect a `read` call\n /// and return `error`.\n pub fn read_error(&mut self, error: io::Error) -> &mut Self {\n let error = Some(error.into());\n self.actions.push_back(Action::ReadError(error));\n self\n }\n\n /// Sequence a `write` operation.\n ///\n /// The next operation in the mock's script will be to expect a `write`\n /// call.\n pub fn write(&mut self, buf: &[u8]) -> &mut Self {\n self.actions.push_back(Action::Write(buf.into()));\n self\n }\n\n /// Sequence a `write` operation that produces an error.\n ///\n /// The next operation in the mock's script will be to expect a `write`\n /// call that provides `error`.\n pub fn write_error(&mut self, error: io::Error) -> &mut Self {\n let error = Some(error.into());\n self.actions.push_back(Action::WriteError(error));\n self\n }\n\n /// Sequence a wait.\n ///\n /// The next operation in the mock's script will be to wait without doing so\n /// for `duration` amount of time.\n pub fn wait(&mut self, duration: Duration) -> &mut Self {\n let duration = cmp::max(duration, Duration::from_millis(1));\n self.actions.push_back(Action::Wait(duration));\n self\n }\n\n /// Set name of the mock IO object to include in panic messages and debug output\n pub fn name(&mut self, name: impl Into) -> &mut Self {\n self.name = name.into();\n self\n }\n\n /// Build a `Mock` value according to the defined script.\n pub fn build(&mut self) -> Mock {\n let (mock, _) = self.build_with_handle();\n mock\n }\n\n /// Build a `Mock` value paired with a handle\n pub fn build_with_handle(&mut self) -> (Mock, Handle) {\n let (inner, handle) = Inner::new(self.actions.clone(), self.name.clone());\n\n let mock = Mock { inner };\n\n (mock, handle)\n }\n}\n\nimpl Handle {\n /// Sequence a `read` operation.\n ///\n /// The next operation in the mock's script will be to expect a `read` call\n /// and return `buf`.\n pub fn read(&mut self, buf: &[u8]) -> &mut Self {\n self.tx.send(Action::Read(buf.into())).unwrap();\n self\n }\n\n /// Sequence a `read` operation error.\n ///\n /// The next operation in the mock's script will be to expect a `read` call\n /// and return `error`.\n pub fn read_error(&mut self, error: io::Error) -> &mut Self {\n let error = Some(error.into());\n self.tx.send(Action::ReadError(error)).unwrap();\n self\n }\n\n /// Sequence a `write` operation.\n ///\n /// The next operation in the mock's script will be to expect a `write`\n /// call.\n pub fn write(&mut self, buf: &[u8]) -> &mut Self {\n self.tx.send(Action::Write(buf.into())).unwrap();\n self\n }\n\n /// Sequence a `write` operation error.\n ///\n /// The next operation in the mock's script will be to expect a `write`\n /// call error.\n pub fn write_error(&mut self, error: io::Error) -> &mut Self {\n let error = Some(error.into());\n self.tx.send(Action::WriteError(error)).unwrap();\n self\n }\n}\n\nimpl Inner {\n fn new(actions: VecDeque, name: String) -> (Inner, Handle) {\n let (tx, rx) = mpsc::unbounded_channel();\n\n let rx = UnboundedReceiverStream::new(rx);\n\n let inner = Inner {\n actions,\n sleep: None,\n read_wait: None,\n rx,\n waiting: None,\n name,\n };\n\n let handle = Handle { tx };\n\n (inner, handle)\n }\n\n fn poll_action(&mut self, cx: &mut task::Context<'_>) -> Poll> {\n Pin::new(&mut self.rx).poll_next(cx)\n }\n\n fn read(&mut self, dst: &mut ReadBuf<'_>) -> io::Result<()> {\n match self.action() {\n Some(&mut Action::Read(ref mut data)) => {\n // Figure out how much to copy\n let n = cmp::min(dst.remaining(), data.len());\n\n // Copy the data into the `dst` slice\n dst.put_slice(&data[..n]);\n\n // Drain the data from the source\n data.drain(..n);\n\n Ok(())\n }\n Some(&mut Action::ReadError(ref mut err)) => {\n // As the\n let err = err.take().expect(\"Should have been removed from actions.\");\n let err = Arc::try_unwrap(err).expect(\"There are no other references.\");\n Err(err)\n }\n Some(_) => {\n // Either waiting or expecting a write\n Err(io::ErrorKind::WouldBlock.into())\n }\n None => Ok(()),\n }\n }\n\n fn write(&mut self, mut src: &[u8]) -> io::Result {\n let mut ret = 0;\n\n if self.actions.is_empty() {\n return Err(io::ErrorKind::BrokenPipe.into());\n }\n\n if let Some(&mut Action::Wait(..)) = self.action() {\n return Err(io::ErrorKind::WouldBlock.into());\n }\n\n if let Some(&mut Action::WriteError(ref mut err)) = self.action() {\n let err = err.take().expect(\"Should have been removed from actions.\");\n let err = Arc::try_unwrap(err).expect(\"There are no other references.\");\n return Err(err);\n }\n\n for i in 0..self.actions.len() {\n match self.actions[i] {\n Action::Write(ref mut expect) => {\n let n = cmp::min(src.len(), expect.len());\n\n assert_eq!(&src[..n], &expect[..n], \"name={} i={}\", self.name, i);\n\n // Drop data that was matched\n expect.drain(..n);\n src = &src[n..];\n\n ret += n;\n\n if src.is_empty() {\n return Ok(ret);\n }\n }\n Action::Wait(..) | Action::WriteError(..) => {\n break;\n }\n _ => {}\n }\n\n // TODO: remove write\n }\n\n Ok(ret)\n }\n\n fn remaining_wait(&mut self) -> Option {\n match self.action() {\n Some(&mut Action::Wait(dur)) => Some(dur),\n _ => None,\n }\n }\n\n fn action(&mut self) -> Option<&mut Action> {\n loop {\n if self.actions.is_empty() {\n return None;\n }\n\n match self.actions[0] {\n Action::Read(ref mut data) => {\n if !data.is_empty() {\n break;\n }\n }\n Action::Write(ref mut data) => {\n if !data.is_empty() {\n break;\n }\n }\n Action::Wait(ref mut dur) => {\n if let Some(until) = self.waiting {\n let now = Instant::now();\n\n if now < until {\n break;\n } else {\n self.waiting = None;\n }\n } else {\n self.waiting = Some(Instant::now() + *dur);\n break;\n }\n }\n Action::ReadError(ref mut error) | Action::WriteError(ref mut error) => {\n if error.is_some() {\n break;\n }\n }\n }\n\n let _action = self.actions.pop_front();\n }\n\n self.actions.front_mut()\n }\n}\n\n// ===== impl Inner =====\n\nimpl Mock {\n fn maybe_wakeup_reader(&mut self) {\n match self.inner.action() {\n Some(&mut Action::Read(_)) | Some(&mut Action::ReadError(_)) | None => {\n if let Some(waker) = self.inner.read_wait.take() {\n waker.wake();\n }\n }\n _ => {}\n }\n }\n}\n\nimpl AsyncRead for Mock {\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n loop {\n if let Some(ref mut sleep) = self.inner.sleep {\n ready!(Pin::new(sleep).poll(cx));\n }\n\n // If a sleep is set, it has already fired\n self.inner.sleep = None;\n\n // Capture 'filled' to monitor if it changed\n let filled = buf.filled().len();\n\n match self.inner.read(buf) {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n if let Some(rem) = self.inner.remaining_wait() {\n let until = Instant::now() + rem;\n self.inner.sleep = Some(Box::pin(time::sleep_until(until)));\n } else {\n self.inner.read_wait = Some(cx.waker().clone());\n return Poll::Pending;\n }\n }\n Ok(()) => {\n if buf.filled().len() == filled {\n match ready!(self.inner.poll_action(cx)) {\n Some(action) => {\n self.inner.actions.push_back(action);\n continue;\n }\n None => {\n return Poll::Ready(Ok(()));\n }\n }\n } else {\n return Poll::Ready(Ok(()));\n }\n }\n Err(e) => return Poll::Ready(Err(e)),\n }\n }\n }\n}\n\nimpl AsyncWrite for Mock {\n fn poll_write(\n mut self: Pin<&mut Self>,\n cx: &mut task::Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n loop {\n if let Some(ref mut sleep) = self.inner.sleep {\n ready!(Pin::new(sleep).poll(cx));\n }\n\n // If a sleep is set, it has already fired\n self.inner.sleep = None;\n\n if self.inner.actions.is_empty() {\n match self.inner.poll_action(cx) {\n Poll::Pending => {\n // do not propagate pending\n }\n Poll::Ready(Some(action)) => {\n self.inner.actions.push_back(action);\n }\n Poll::Ready(None) => {\n panic!(\"unexpected write {}\", self.pmsg());\n }\n }\n }\n\n match self.inner.write(buf) {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {\n if let Some(rem) = self.inner.remaining_wait() {\n let until = Instant::now() + rem;\n self.inner.sleep = Some(Box::pin(time::sleep_until(until)));\n } else {\n // A race condition (TOCTOU) can occur if the\n // timer expires between the `write()` call\n // and `remaining_wait()` due to preemption or other\n // delays. In this case, the `Wait` action is already popped by\n // `action()`, so we continue to the next one.\n //\n // Consider the following sequence:\n //\n // poll_write Inner action()\n // |--write()--->| |\n // | |--action()---->| (returns Wait)\n // |<-WouldBlk---| |\n // | | |\n // | <--- TIMEOUT! ---> |\n // | (due to preemption, etc.) |\n // | | |\n // |-rem_wait()->| |\n // | |--action()---->| (time's up, pop Wait)\n // |<--None------| |\n // | | |\n // |---continue->| (process next action)\n //\n // See .\n continue;\n }\n }\n Ok(0) => {\n // TODO: Is this correct?\n if !self.inner.actions.is_empty() {\n return Poll::Pending;\n }\n\n // TODO: Extract\n match ready!(self.inner.poll_action(cx)) {\n Some(action) => {\n self.inner.actions.push_back(action);\n continue;\n }\n None => {\n panic!(\"unexpected write {}\", self.pmsg());\n }\n }\n }\n ret => {\n self.maybe_wakeup_reader();\n return Poll::Ready(ret);\n }\n }\n }\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut task::Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut task::Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\n/// Ensures that Mock isn't dropped with data \"inside\".\nimpl Drop for Mock {\n fn drop(&mut self) {\n // Avoid double panicking, since makes debugging much harder.\n if std::thread::panicking() {\n return;\n }\n\n self.inner.actions.iter().for_each(|a| match a {\n Action::Read(data) => assert!(\n data.is_empty(),\n \"There is still data left to read. {}\",\n self.pmsg()\n ),\n Action::Write(data) => assert!(\n data.is_empty(),\n \"There is still data left to write. {}\",\n self.pmsg()\n ),\n _ => (),\n });\n }\n}\n/*\n/// Returns `true` if called from the context of a futures-rs Task\nfn is_task_ctx() -> bool {\n use std::panic;\n\n // Save the existing panic hook\n let h = panic::take_hook();\n\n // Install a new one that does nothing\n panic::set_hook(Box::new(|_| {}));\n\n // Attempt to call the fn\n let r = panic::catch_unwind(|| task::current()).is_ok();\n\n // Re-install the old one\n panic::set_hook(h);\n\n // Return the result\n r\n}\n*/\n\nimpl fmt::Debug for Inner {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n if self.name.is_empty() {\n write!(f, \"Inner {{...}}\")\n } else {\n write!(f, \"Inner {{name={}, ...}}\", self.name)\n }\n }\n}\n\nstruct PanicMsgSnippet<'a>(&'a Inner);\n\nimpl<'a> fmt::Display for PanicMsgSnippet<'a> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n if self.0.name.is_empty() {\n write!(f, \"({} actions remain)\", self.0.actions.len())\n } else {\n write!(\n f,\n \"(name {}, {} actions remain)\",\n self.0.name,\n self.0.actions.len()\n )\n }\n }\n}\n\nimpl Mock {\n fn pmsg(&self) -> PanicMsgSnippet<'_> {\n PanicMsgSnippet(&self.inner)\n }\n}\n" }, { "path": "tokio-test/src/lib.rs", "content": "#![warn(\n missing_debug_implementations,\n missing_docs,\n rust_2018_idioms,\n unreachable_pub\n)]\n#![doc(test(\n no_crate_inject,\n attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))\n))]\n\n//! Tokio and Futures based testing utilities\n\npub mod io;\npub mod stream_mock;\n\nmod macros;\npub mod task;\n\n/// Runs the provided future, blocking the current thread until the\n/// future completes.\n///\n/// For more information, see the documentation for\n/// [`tokio::runtime::Runtime::block_on`][runtime-block-on].\n///\n/// [runtime-block-on]: https://docs.rs/tokio/1.3.0/tokio/runtime/struct.Runtime.html#method.block_on\npub fn block_on(future: F) -> F::Output {\n use tokio::runtime;\n\n let rt = runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap();\n\n rt.block_on(future)\n}\n" }, { "path": "tokio-test/src/macros.rs", "content": "//! A collection of useful macros for testing futures and tokio based code\n\n/// Asserts a `Poll` is ready, returning the value.\n///\n/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use futures_util::future;\n/// use tokio_test::{assert_ready, task};\n///\n/// let mut fut = task::spawn(future::ready(()));\n/// assert_ready!(fut.poll());\n/// ```\n#[macro_export]\nmacro_rules! assert_ready {\n ($e:expr) => {{\n use core::task::Poll;\n match $e {\n Poll::Ready(v) => v,\n Poll::Pending => panic!(\"pending\"),\n }\n }};\n ($e:expr, $($msg:tt)+) => {{\n use core::task::Poll;\n match $e {\n Poll::Ready(v) => v,\n Poll::Pending => {\n panic!(\"pending; {}\", format_args!($($msg)+))\n }\n }\n }};\n}\n\n/// Asserts a `Poll>` is ready and `Ok`, returning the value.\n///\n/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready(Ok(..))` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use futures_util::future;\n/// use tokio_test::{assert_ready_ok, task};\n///\n/// let mut fut = task::spawn(future::ok::<_, ()>(()));\n/// assert_ready_ok!(fut.poll());\n/// ```\n#[macro_export]\nmacro_rules! assert_ready_ok {\n ($e:expr) => {{\n use tokio_test::{assert_ready, assert_ok};\n let val = assert_ready!($e);\n assert_ok!(val)\n }};\n ($e:expr, $($msg:tt)+) => {{\n use tokio_test::{assert_ready, assert_ok};\n let val = assert_ready!($e, $($msg)*);\n assert_ok!(val, $($msg)*)\n }};\n}\n\n/// Asserts a `Poll>` is ready and `Err`, returning the error.\n///\n/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready(Err(..))` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use futures_util::future;\n/// use tokio_test::{assert_ready_err, task};\n///\n/// let mut fut = task::spawn(future::err::<(), _>(()));\n/// assert_ready_err!(fut.poll());\n/// ```\n#[macro_export]\nmacro_rules! assert_ready_err {\n ($e:expr) => {{\n use tokio_test::{assert_ready, assert_err};\n let val = assert_ready!($e);\n assert_err!(val)\n }};\n ($e:expr, $($msg:tt)+) => {{\n use tokio_test::{assert_ready, assert_err};\n let val = assert_ready!($e, $($msg)*);\n assert_err!(val, $($msg)*)\n }};\n}\n\n/// Asserts a `Poll` is pending.\n///\n/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Pending` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use futures_util::future;\n/// use tokio_test::{assert_pending, task};\n///\n/// let mut fut = task::spawn(future::pending::<()>());\n/// assert_pending!(fut.poll());\n/// ```\n#[macro_export]\nmacro_rules! assert_pending {\n ($e:expr) => {{\n use core::task::Poll;\n match $e {\n Poll::Pending => {}\n Poll::Ready(v) => panic!(\"ready; value = {:?}\", v),\n }\n }};\n ($e:expr, $($msg:tt)+) => {{\n use core::task::Poll;\n match $e {\n Poll::Pending => {}\n Poll::Ready(v) => {\n panic!(\"ready; value = {:?}; {}\", v, format_args!($($msg)+))\n }\n }\n }};\n}\n\n/// Asserts if a poll is ready and check for equality on the value\n///\n/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready` at\n/// runtime and the value produced does not partially equal the expected value.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use futures_util::future;\n/// use tokio_test::{assert_ready_eq, task};\n///\n/// let mut fut = task::spawn(future::ready(42));\n/// assert_ready_eq!(fut.poll(), 42);\n/// ```\n#[macro_export]\nmacro_rules! assert_ready_eq {\n ($e:expr, $expect:expr) => {\n let val = $crate::assert_ready!($e);\n assert_eq!(val, $expect)\n };\n\n ($e:expr, $expect:expr, $($msg:tt)+) => {\n let val = $crate::assert_ready!($e, $($msg)*);\n assert_eq!(val, $expect, $($msg)*)\n };\n}\n\n/// Asserts that the expression evaluates to `Ok` and returns the value.\n///\n/// This will invoke the `panic!` macro if the provided expression does not evaluate to `Ok` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use tokio_test::assert_ok;\n///\n/// let n: u32 = assert_ok!(\"123\".parse());\n///\n/// let s = \"123\";\n/// let n: u32 = assert_ok!(s.parse(), \"testing parsing {:?} as a u32\", s);\n/// ```\n#[macro_export]\nmacro_rules! assert_ok {\n ($e:expr) => {\n assert_ok!($e,)\n };\n ($e:expr,) => {{\n use std::result::Result::*;\n match $e {\n Ok(v) => v,\n Err(e) => panic!(\"assertion failed: Err({:?})\", e),\n }\n }};\n ($e:expr, $($arg:tt)+) => {{\n use std::result::Result::*;\n match $e {\n Ok(v) => v,\n Err(e) => panic!(\"assertion failed: Err({:?}): {}\", e, format_args!($($arg)+)),\n }\n }};\n}\n\n/// Asserts that the expression evaluates to `Err` and returns the error.\n///\n/// This will invoke the `panic!` macro if the provided expression does not evaluate to `Err` at\n/// runtime.\n///\n/// # Custom Messages\n///\n/// This macro has a second form, where a custom panic message can be provided with or without\n/// arguments for formatting.\n///\n/// # Examples\n///\n/// ```\n/// use tokio_test::assert_err;\n/// use std::str::FromStr;\n///\n///\n/// let err = assert_err!(u32::from_str(\"fail\"));\n///\n/// let msg = \"fail\";\n/// let err = assert_err!(u32::from_str(msg), \"testing parsing {:?} as u32\", msg);\n/// ```\n#[macro_export]\nmacro_rules! assert_err {\n ($e:expr) => {\n assert_err!($e,);\n };\n ($e:expr,) => {{\n use std::result::Result::*;\n match $e {\n Ok(v) => panic!(\"assertion failed: Ok({:?})\", v),\n Err(e) => e,\n }\n }};\n ($e:expr, $($arg:tt)+) => {{\n use std::result::Result::*;\n match $e {\n Ok(v) => panic!(\"assertion failed: Ok({:?}): {}\", v, format_args!($($arg)+)),\n Err(e) => e,\n }\n }};\n}\n\n/// Asserts that an exact duration has elapsed since the start instant ±1ms.\n///\n/// ```rust\n/// use tokio::time::{self, Instant};\n/// use std::time::Duration;\n/// use tokio_test::assert_elapsed;\n/// # async fn test_time_passed() {\n///\n/// let start = Instant::now();\n/// let dur = Duration::from_millis(50);\n/// time::sleep(dur).await;\n/// assert_elapsed!(start, dur);\n/// # }\n/// ```\n///\n/// This 1ms buffer is required because Tokio's hashed-wheel timer has finite time resolution and\n/// will not always sleep for the exact interval.\n#[macro_export]\nmacro_rules! assert_elapsed {\n ($start:expr, $dur:expr) => {{\n let elapsed = $start.elapsed();\n // type ascription improves compiler error when wrong type is passed\n let lower: std::time::Duration = $dur;\n\n // Handles ms rounding\n assert!(\n elapsed >= lower && elapsed <= lower + std::time::Duration::from_millis(1),\n \"actual = {:?}, expected = {:?}\",\n elapsed,\n lower\n );\n }};\n}\n" }, { "path": "tokio-test/src/stream_mock.rs", "content": "#![cfg(not(loom))]\n\n//! A mock stream implementing [`Stream`].\n//!\n//! # Overview\n//! This crate provides a `StreamMock` that can be used to test code that interacts with streams.\n//! It allows you to mock the behavior of a stream and control the items it yields and the waiting\n//! intervals between items.\n//!\n//! # Usage\n//! To use the `StreamMock`, you need to create a builder using [`StreamMockBuilder`]. The builder\n//! allows you to enqueue actions such as returning items or waiting for a certain duration.\n//!\n//! # Example\n//! ```rust\n//!\n//! use futures_util::StreamExt;\n//! use std::time::Duration;\n//! use tokio_test::stream_mock::StreamMockBuilder;\n//!\n//! async fn test_stream_mock_wait() {\n//! let mut stream_mock = StreamMockBuilder::new()\n//! .next(1)\n//! .wait(Duration::from_millis(300))\n//! .next(2)\n//! .build();\n//!\n//! assert_eq!(stream_mock.next().await, Some(1));\n//! let start = std::time::Instant::now();\n//! assert_eq!(stream_mock.next().await, Some(2));\n//! let elapsed = start.elapsed();\n//! assert!(elapsed >= Duration::from_millis(300));\n//! assert_eq!(stream_mock.next().await, None);\n//! }\n//! ```\n\nuse std::collections::VecDeque;\nuse std::pin::Pin;\nuse std::task::{ready, Poll};\nuse std::time::Duration;\n\nuse futures_core::Stream;\nuse std::future::Future;\nuse tokio::time::{sleep_until, Instant, Sleep};\n\n#[derive(Debug, Clone)]\nenum Action {\n Next(T),\n Wait(Duration),\n}\n\n/// A builder for [`StreamMock`]\n#[derive(Debug, Clone)]\npub struct StreamMockBuilder {\n actions: VecDeque>,\n}\n\nimpl StreamMockBuilder {\n /// Create a new empty [`StreamMockBuilder`]\n pub fn new() -> Self {\n StreamMockBuilder::default()\n }\n\n /// Queue an item to be returned by the stream\n pub fn next(mut self, value: T) -> Self {\n self.actions.push_back(Action::Next(value));\n self\n }\n\n // Queue an item to be consumed by the sink,\n // commented out until Sink is implemented.\n //\n // pub fn consume(mut self, value: T) -> Self {\n // self.actions.push_back(Action::Consume(value));\n // self\n // }\n\n /// Queue the stream to wait for a duration\n pub fn wait(mut self, duration: Duration) -> Self {\n self.actions.push_back(Action::Wait(duration));\n self\n }\n\n /// Build the [`StreamMock`]\n pub fn build(self) -> StreamMock {\n StreamMock {\n actions: self.actions,\n sleep: None,\n }\n }\n}\n\nimpl Default for StreamMockBuilder {\n fn default() -> Self {\n StreamMockBuilder {\n actions: VecDeque::new(),\n }\n }\n}\n\n/// A mock stream implementing [`Stream`]\n///\n/// See [`StreamMockBuilder`] for more information.\n#[derive(Debug)]\npub struct StreamMock {\n actions: VecDeque>,\n sleep: Option>>,\n}\n\nimpl StreamMock {\n fn next_action(&mut self) -> Option> {\n self.actions.pop_front()\n }\n}\n\nimpl Stream for StreamMock {\n type Item = T;\n\n fn poll_next(\n mut self: std::pin::Pin<&mut Self>,\n cx: &mut std::task::Context<'_>,\n ) -> std::task::Poll> {\n // Try polling the sleep future first\n if let Some(ref mut sleep) = self.sleep {\n ready!(Pin::new(sleep).poll(cx));\n // Since we're ready, discard the sleep future\n self.sleep.take();\n }\n\n match self.next_action() {\n Some(action) => match action {\n Action::Next(item) => Poll::Ready(Some(item)),\n Action::Wait(duration) => {\n // Set up a sleep future and schedule this future to be polled again for it.\n self.sleep = Some(Box::pin(sleep_until(Instant::now() + duration)));\n cx.waker().wake_by_ref();\n\n Poll::Pending\n }\n },\n None => Poll::Ready(None),\n }\n }\n}\n\nimpl Drop for StreamMock {\n fn drop(&mut self) {\n // Avoid double panicking to make debugging easier.\n if std::thread::panicking() {\n return;\n }\n\n let undropped_count = self\n .actions\n .iter()\n .filter(|action| match action {\n Action::Next(_) => true,\n Action::Wait(_) => false,\n })\n .count();\n\n assert!(\n undropped_count == 0,\n \"StreamMock was dropped before all actions were consumed, {undropped_count} actions were not consumed\"\n );\n }\n}\n" }, { "path": "tokio-test/src/task.rs", "content": "//! Futures task based helpers to easily test futures and manually written futures.\n//!\n//! The [`Spawn`] type is used as a mock task harness that allows you to poll futures\n//! without needing to setup pinning or context. Any future can be polled but if the\n//! future requires the tokio async context you will need to ensure that you poll the\n//! [`Spawn`] within a tokio context, this means that as long as you are inside the\n//! runtime it will work and you can poll it via [`Spawn`].\n//!\n//! [`Spawn`] also supports [`Stream`] to call `poll_next` without pinning\n//! or context.\n//!\n//! In addition to circumventing the need for pinning and context, [`Spawn`] also tracks\n//! the amount of times the future/task was woken. This can be useful to track if some\n//! leaf future notified the root task correctly.\n//!\n//! # Example\n//!\n//! ```\n//! use tokio_test::task;\n//!\n//! let fut = async {};\n//!\n//! let mut task = task::spawn(fut);\n//!\n//! assert!(task.poll().is_ready(), \"Task was not ready!\");\n//! ```\n\nuse std::future::Future;\nuse std::mem;\nuse std::ops;\nuse std::pin::Pin;\nuse std::sync::{Arc, Condvar, Mutex};\nuse std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};\n\nuse tokio_stream::Stream;\n\n/// Spawn a future into a [`Spawn`] which wraps the future in a mocked executor.\n///\n/// This can be used to spawn a [`Future`] or a [`Stream`].\n///\n/// For more information, check the module docs.\npub fn spawn(task: T) -> Spawn {\n Spawn {\n task: MockTask::new(),\n future: Box::pin(task),\n }\n}\n\n/// Future spawned on a mock task that can be used to poll the future or stream\n/// without needing pinning or context types.\n#[derive(Debug)]\n#[must_use = \"futures do nothing unless you `.await` or poll them\"]\npub struct Spawn {\n task: MockTask,\n future: Pin>,\n}\n\n#[derive(Debug, Clone)]\nstruct MockTask {\n waker: Arc,\n}\n\n#[derive(Debug)]\nstruct ThreadWaker {\n state: Mutex,\n condvar: Condvar,\n}\n\nconst IDLE: usize = 0;\nconst WAKE: usize = 1;\nconst SLEEP: usize = 2;\n\nimpl Spawn {\n /// Consumes `self` returning the inner value\n pub fn into_inner(self) -> T\n where\n T: Unpin,\n {\n *Pin::into_inner(self.future)\n }\n\n /// Returns `true` if the inner future has received a wake notification\n /// since the last call to `enter`.\n pub fn is_woken(&self) -> bool {\n self.task.is_woken()\n }\n\n /// Returns the number of references to the task waker\n ///\n /// The task itself holds a reference. The return value will never be zero.\n pub fn waker_ref_count(&self) -> usize {\n self.task.waker_ref_count()\n }\n\n /// Enter the task context\n pub fn enter(&mut self, f: F) -> R\n where\n F: FnOnce(&mut Context<'_>, Pin<&mut T>) -> R,\n {\n let fut = self.future.as_mut();\n self.task.enter(|cx| f(cx, fut))\n }\n}\n\nimpl ops::Deref for Spawn {\n type Target = T;\n\n fn deref(&self) -> &T {\n &self.future\n }\n}\n\nimpl ops::DerefMut for Spawn {\n fn deref_mut(&mut self) -> &mut T {\n &mut self.future\n }\n}\n\nimpl Spawn {\n /// If `T` is a [`Future`] then poll it. This will handle pinning and the context\n /// type for the future.\n pub fn poll(&mut self) -> Poll {\n let fut = self.future.as_mut();\n self.task.enter(|cx| fut.poll(cx))\n }\n}\n\nimpl Spawn {\n /// If `T` is a [`Stream`] then `poll_next` it. This will handle pinning and the context\n /// type for the stream.\n pub fn poll_next(&mut self) -> Poll> {\n let stream = self.future.as_mut();\n self.task.enter(|cx| stream.poll_next(cx))\n }\n}\n\nimpl Future for Spawn {\n type Output = T::Output;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n self.future.as_mut().poll(cx)\n }\n}\n\nimpl Stream for Spawn {\n type Item = T::Item;\n\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.future.as_mut().poll_next(cx)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n self.future.size_hint()\n }\n}\n\nimpl MockTask {\n /// Creates new mock task\n fn new() -> Self {\n MockTask {\n waker: Arc::new(ThreadWaker::new()),\n }\n }\n\n /// Runs a closure from the context of the task.\n ///\n /// Any wake notifications resulting from the execution of the closure are\n /// tracked.\n fn enter(&mut self, f: F) -> R\n where\n F: FnOnce(&mut Context<'_>) -> R,\n {\n self.waker.clear();\n let waker = self.waker();\n let mut cx = Context::from_waker(&waker);\n\n f(&mut cx)\n }\n\n /// Returns `true` if the inner future has received a wake notification\n /// since the last call to `enter`.\n fn is_woken(&self) -> bool {\n self.waker.is_woken()\n }\n\n /// Returns the number of references to the task waker\n ///\n /// The task itself holds a reference. The return value will never be zero.\n fn waker_ref_count(&self) -> usize {\n Arc::strong_count(&self.waker)\n }\n\n fn waker(&self) -> Waker {\n unsafe {\n let raw = to_raw(self.waker.clone());\n Waker::from_raw(raw)\n }\n }\n}\n\nimpl Default for MockTask {\n fn default() -> Self {\n Self::new()\n }\n}\n\nimpl ThreadWaker {\n fn new() -> Self {\n ThreadWaker {\n state: Mutex::new(IDLE),\n condvar: Condvar::new(),\n }\n }\n\n /// Clears any previously received wakes, avoiding potential spurious\n /// wake notifications. This should only be called immediately before running the\n /// task.\n fn clear(&self) {\n *self.state.lock().unwrap() = IDLE;\n }\n\n fn is_woken(&self) -> bool {\n match *self.state.lock().unwrap() {\n IDLE => false,\n WAKE => true,\n _ => unreachable!(),\n }\n }\n\n fn wake(&self) {\n // First, try transitioning from IDLE -> NOTIFY, this does not require a lock.\n let mut state = self.state.lock().unwrap();\n let prev = *state;\n\n if prev == WAKE {\n return;\n }\n\n *state = WAKE;\n\n if prev == IDLE {\n return;\n }\n\n // The other half is sleeping, so we wake it up.\n assert_eq!(prev, SLEEP);\n self.condvar.notify_one();\n }\n}\n\nstatic VTABLE: RawWakerVTable = RawWakerVTable::new(clone, wake, wake_by_ref, drop_waker);\n\nunsafe fn to_raw(waker: Arc) -> RawWaker {\n RawWaker::new(Arc::into_raw(waker) as *const (), &VTABLE)\n}\n\nunsafe fn from_raw(raw: *const ()) -> Arc {\n Arc::from_raw(raw as *const ThreadWaker)\n}\n\nunsafe fn clone(raw: *const ()) -> RawWaker {\n let waker = from_raw(raw);\n\n // Increment the ref count\n mem::forget(waker.clone());\n\n to_raw(waker)\n}\n\nunsafe fn wake(raw: *const ()) {\n let waker = from_raw(raw);\n waker.wake();\n}\n\nunsafe fn wake_by_ref(raw: *const ()) {\n let waker = from_raw(raw);\n waker.wake();\n\n // We don't actually own a reference to the unparker\n mem::forget(waker);\n}\n\nunsafe fn drop_waker(raw: *const ()) {\n let _ = from_raw(raw);\n}\n" }, { "path": "tokio-test/tests/block_on.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::time::{sleep_until, Duration, Instant};\nuse tokio_test::block_on;\n\n#[test]\nfn async_block() {\n assert_eq!(4, block_on(async { 4 }));\n}\n\nasync fn five() -> u8 {\n 5\n}\n\n#[test]\nfn async_fn() {\n assert_eq!(5, block_on(five()));\n}\n\n#[test]\nfn test_sleep() {\n let deadline = Instant::now() + Duration::from_millis(100);\n\n block_on(async {\n sleep_until(deadline).await;\n });\n}\n" }, { "path": "tokio-test/tests/io.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse std::io;\nuse tokio::io::{AsyncReadExt, AsyncWriteExt};\nuse tokio::time::{Duration, Instant};\nuse tokio_test::io::Builder;\n\n#[tokio::test]\nasync fn read() {\n let mut mock = Builder::new().read(b\"hello \").read(b\"world!\").build();\n\n let mut buf = [0; 256];\n\n let n = mock.read(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..n], b\"hello \");\n\n let n = mock.read(&mut buf).await.expect(\"read 2\");\n assert_eq!(&buf[..n], b\"world!\");\n}\n\n#[tokio::test]\nasync fn read_error() {\n let error = io::Error::new(io::ErrorKind::Other, \"cruel\");\n let mut mock = Builder::new()\n .read(b\"hello \")\n .read_error(error)\n .read(b\"world!\")\n .build();\n let mut buf = [0; 256];\n\n let n = mock.read(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..n], b\"hello \");\n\n match mock.read(&mut buf).await {\n Err(error) => {\n assert_eq!(error.kind(), io::ErrorKind::Other);\n assert_eq!(\"cruel\", format!(\"{error}\"));\n }\n Ok(_) => panic!(\"error not received\"),\n }\n\n let n = mock.read(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..n], b\"world!\");\n}\n\n#[tokio::test]\nasync fn write() {\n let mut mock = Builder::new().write(b\"hello \").write(b\"world!\").build();\n\n mock.write_all(b\"hello \").await.expect(\"write 1\");\n mock.write_all(b\"world!\").await.expect(\"write 2\");\n}\n\n#[tokio::test]\nasync fn write_with_handle() {\n let (mut mock, mut handle) = Builder::new().build_with_handle();\n handle.write(b\"hello \");\n handle.write(b\"world!\");\n\n mock.write_all(b\"hello \").await.expect(\"write 1\");\n mock.write_all(b\"world!\").await.expect(\"write 2\");\n}\n\n#[tokio::test]\nasync fn read_with_handle() {\n let (mut mock, mut handle) = Builder::new().build_with_handle();\n handle.read(b\"hello \");\n handle.read(b\"world!\");\n\n let mut buf = vec![0; 6];\n mock.read_exact(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..], b\"hello \");\n mock.read_exact(&mut buf).await.expect(\"read 2\");\n assert_eq!(&buf[..], b\"world!\");\n}\n\n#[tokio::test]\nasync fn write_error() {\n let error = io::Error::new(io::ErrorKind::Other, \"cruel\");\n let mut mock = Builder::new()\n .write(b\"hello \")\n .write_error(error)\n .write(b\"world!\")\n .build();\n mock.write_all(b\"hello \").await.expect(\"write 1\");\n\n match mock.write_all(b\"whoa\").await {\n Err(error) => {\n assert_eq!(error.kind(), io::ErrorKind::Other);\n assert_eq!(\"cruel\", format!(\"{error}\"));\n }\n Ok(_) => panic!(\"error not received\"),\n }\n\n mock.write_all(b\"world!\").await.expect(\"write 2\");\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn mock_panics_read_data_left() {\n use tokio_test::io::Builder;\n Builder::new().read(b\"read\").build();\n}\n\n#[tokio::test]\n#[should_panic]\nasync fn mock_panics_write_data_left() {\n use tokio_test::io::Builder;\n Builder::new().write(b\"write\").build();\n}\n\n#[tokio::test(start_paused = true)]\nasync fn wait() {\n const FIRST_WAIT: Duration = Duration::from_secs(1);\n\n let mut mock = Builder::new()\n .wait(FIRST_WAIT)\n .read(b\"hello \")\n .read(b\"world!\")\n .build();\n\n let mut buf = [0; 256];\n\n let start = Instant::now(); // record the time the read call takes\n //\n let n = mock.read(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..n], b\"hello \");\n println!(\"time elapsed after first read {:?}\", start.elapsed());\n\n let n = mock.read(&mut buf).await.expect(\"read 2\");\n assert_eq!(&buf[..n], b\"world!\");\n println!(\"time elapsed after second read {:?}\", start.elapsed());\n\n // make sure the .wait() instruction worked\n assert!(\n start.elapsed() >= FIRST_WAIT,\n \"consuming the whole mock only took {}ms\",\n start.elapsed().as_millis()\n );\n}\n\n#[tokio::test(start_paused = true)]\nasync fn multiple_wait() {\n const FIRST_WAIT: Duration = Duration::from_secs(1);\n const SECOND_WAIT: Duration = Duration::from_secs(1);\n\n let mut mock = Builder::new()\n .wait(FIRST_WAIT)\n .read(b\"hello \")\n .wait(SECOND_WAIT)\n .read(b\"world!\")\n .build();\n\n let mut buf = [0; 256];\n\n let start = Instant::now(); // record the time it takes to consume the mock\n\n let n = mock.read(&mut buf).await.expect(\"read 1\");\n assert_eq!(&buf[..n], b\"hello \");\n println!(\"time elapsed after first read {:?}\", start.elapsed());\n\n let n = mock.read(&mut buf).await.expect(\"read 2\");\n assert_eq!(&buf[..n], b\"world!\");\n println!(\"time elapsed after second read {:?}\", start.elapsed());\n\n // make sure the .wait() instruction worked\n assert!(\n start.elapsed() >= FIRST_WAIT + SECOND_WAIT,\n \"consuming the whole mock only took {}ms\",\n start.elapsed().as_millis()\n );\n}\n" }, { "path": "tokio-test/tests/macros.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse std::task::Poll;\nuse tokio_test::{\n assert_pending, assert_ready, assert_ready_eq, assert_ready_err, assert_ready_ok,\n};\n\nfn ready() -> Poll<()> {\n Poll::Ready(())\n}\n\nfn ready_ok() -> Poll> {\n Poll::Ready(Ok(()))\n}\n\nfn ready_err() -> Poll> {\n Poll::Ready(Err(()))\n}\n\nfn pending() -> Poll<()> {\n Poll::Pending\n}\n\n#[derive(Debug)]\nenum Test {\n Data,\n}\n\n#[test]\nfn assert_ready() {\n let poll = ready();\n assert_ready!(poll);\n assert_ready!(poll, \"some message\");\n assert_ready!(poll, \"{:?}\", ());\n assert_ready!(poll, \"{:?}\", Test::Data);\n}\n\n#[test]\n#[should_panic]\nfn assert_ready_on_pending() {\n let poll = pending();\n assert_ready!(poll);\n}\n\n#[test]\nfn assert_pending() {\n let poll = pending();\n assert_pending!(poll);\n assert_pending!(poll, \"some message\");\n assert_pending!(poll, \"{:?}\", ());\n assert_pending!(poll, \"{:?}\", Test::Data);\n}\n\n#[test]\n#[should_panic]\nfn assert_pending_on_ready() {\n let poll = ready();\n assert_pending!(poll);\n}\n\n#[test]\nfn assert_ready_ok() {\n let poll = ready_ok();\n assert_ready_ok!(poll);\n assert_ready_ok!(poll, \"some message\");\n assert_ready_ok!(poll, \"{:?}\", ());\n assert_ready_ok!(poll, \"{:?}\", Test::Data);\n}\n\n#[test]\n#[should_panic]\nfn assert_ok_on_err() {\n let poll = ready_err();\n assert_ready_ok!(poll);\n}\n\n#[test]\nfn assert_ready_err() {\n let poll = ready_err();\n assert_ready_err!(poll);\n assert_ready_err!(poll, \"some message\");\n assert_ready_err!(poll, \"{:?}\", ());\n assert_ready_err!(poll, \"{:?}\", Test::Data);\n}\n\n#[test]\n#[should_panic]\nfn assert_err_on_ok() {\n let poll = ready_ok();\n assert_ready_err!(poll);\n}\n\n#[test]\nfn assert_ready_eq() {\n let poll = ready();\n assert_ready_eq!(poll, ());\n assert_ready_eq!(poll, (), \"some message\");\n assert_ready_eq!(poll, (), \"{:?}\", ());\n assert_ready_eq!(poll, (), \"{:?}\", Test::Data);\n}\n\n#[test]\n#[should_panic]\nfn assert_eq_on_not_eq() {\n let poll = ready_err();\n assert_ready_eq!(poll, Ok(()));\n}\n" }, { "path": "tokio-test/tests/stream_mock.rs", "content": "use futures_util::StreamExt;\nuse std::time::Duration;\nuse tokio_test::stream_mock::StreamMockBuilder;\n\n#[tokio::test]\nasync fn test_stream_mock_empty() {\n let mut stream_mock = StreamMockBuilder::::new().build();\n\n assert_eq!(stream_mock.next().await, None);\n assert_eq!(stream_mock.next().await, None);\n}\n\n#[tokio::test]\nasync fn test_stream_mock_items() {\n let mut stream_mock = StreamMockBuilder::new().next(1).next(2).build();\n\n assert_eq!(stream_mock.next().await, Some(1));\n assert_eq!(stream_mock.next().await, Some(2));\n assert_eq!(stream_mock.next().await, None);\n}\n\n#[tokio::test]\nasync fn test_stream_mock_wait() {\n let mut stream_mock = StreamMockBuilder::new()\n .next(1)\n .wait(Duration::from_millis(300))\n .next(2)\n .build();\n\n assert_eq!(stream_mock.next().await, Some(1));\n let start = std::time::Instant::now();\n assert_eq!(stream_mock.next().await, Some(2));\n let elapsed = start.elapsed();\n assert!(elapsed >= Duration::from_millis(300));\n assert_eq!(stream_mock.next().await, None);\n}\n\n#[tokio::test]\n#[should_panic(expected = \"StreamMock was dropped before all actions were consumed\")]\nasync fn test_stream_mock_drop_without_consuming_all() {\n let stream_mock = StreamMockBuilder::new().next(1).next(2).build();\n drop(stream_mock);\n}\n\n#[tokio::test]\n#[should_panic(expected = \"test panic was not masked\")]\nasync fn test_stream_mock_drop_during_panic_doesnt_mask_panic() {\n let _stream_mock = StreamMockBuilder::new().next(1).next(2).build();\n panic!(\"test panic was not masked\");\n}\n" }, { "path": "tokio-test/tests/task.rs", "content": "use std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio_stream::Stream;\nuse tokio_test::task;\n\n/// A [`Stream`] that has a stub size hint.\nstruct SizedStream;\n\nimpl Stream for SizedStream {\n type Item = ();\n\n fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Pending\n }\n\n fn size_hint(&self) -> (usize, Option) {\n (100, Some(200))\n }\n}\n\n#[test]\nfn test_spawn_stream_size_hint() {\n let spawn = task::spawn(SizedStream);\n assert_eq!(spawn.size_hint(), (100, Some(200)));\n}\n" }, { "path": "tokio-util/CHANGELOG.md", "content": "# 0.7.18 (January 4th, 2026)\n\n### Added\n\n- io: add `tokio_util::io::simplex` ([#7565])\n\n### Changed\n\n- task: remove unnecessary trait bounds on the `Debug` implementation for `JoinQueue` and `AbortOnDropHandle` ([#7720])\n\n### Fixed\n\n- deps: bump `tokio` to `1.44.0` ([#7733])\n\n### Documented\n\n- io: document the default capacity of the `ReaderStream` ([#7147])\n- sync: fix a typo in the docs of `PollSender::is_closed` ([#7737])\n\n[#7147]: https://github.com/tokio-rs/tokio/pull/7147\n[#7565]: https://github.com/tokio-rs/tokio/pull/7565\n[#7720]: https://github.com/tokio-rs/tokio/pull/7720\n[#7733]: https://github.com/tokio-rs/tokio/pull/7733\n[#7737]: https://github.com/tokio-rs/tokio/pull/7737\n\n# 0.7.17 (November 2nd, 2025)\n\nThe MSRV is increased to 1.71.\n\n### Added\n\n- codec: add `{FramedRead,FramedWrite}::into_parts()` ([#7566])\n- time: add `#[track_caller]` to `FutureExt::timeout` ([#7588])\n- task: add `tokio_util::task::JoinQueue` ([#7590])\n\n### Changed\n\n- codec: remove unnecessary trait bounds on all Framed constructors ([#7716])\n\n### Documented\n\n- time: clarify the cancellation safety of the `DelayQueue` ([#7564])\n- docs: fix some docs links ([#7654])\n- task: simplify the example of `TaskTracker` ([#7657])\n- task: clarify the behavior of several `spawn_local` methods ([#7669])\n\n[#7564]: https://github.com/tokio-rs/tokio/pull/7564\n[#7566]: https://github.com/tokio-rs/tokio/pull/7566\n[#7588]: https://github.com/tokio-rs/tokio/pull/7588\n[#7590]: https://github.com/tokio-rs/tokio/pull/7590\n[#7654]: https://github.com/tokio-rs/tokio/pull/7654\n[#7657]: https://github.com/tokio-rs/tokio/pull/7657\n[#7669]: https://github.com/tokio-rs/tokio/pull/7669\n[#7716]: https://github.com/tokio-rs/tokio/pull/7716\n\n# 0.7.16 (August 3rd, 2025)\n\n### Added\n\n- codec: add `FramedWrite::with_capacity` ([#7493])\n- future: add adapters of `CancellationToken` for `FutureExt` ([#7475])\n- sync: add `DropGuardRef` for `CancellationToken` ([#7407])\n- task: add `AbortOnDropHandle::detach` ([#7400])\n- task: stabilise `JoinMap` ([#7075])\n\n### Changed\n\n- codec: also apply capacity to read buffer in `Framed::with_capacity` ([#7500])\n- sync: make `CancellationToken::run_until_cancelled` biased towards the token ([#7462])\n- task: remove raw-entry feature from hashbrown dep ([#7252])\n\n### Documented\n\n- compat: add more documentation to `tokio_util::compat` ([#7279])\n- sync: improve docs of `tokio_util::sync::CancellationToken` ([#7408])\n\n[#7075]: https://github.com/tokio-rs/tokio/pull/7075\n[#7252]: https://github.com/tokio-rs/tokio/pull/7252\n[#7279]: https://github.com/tokio-rs/tokio/pull/7279\n[#7400]: https://github.com/tokio-rs/tokio/pull/7400\n[#7407]: https://github.com/tokio-rs/tokio/pull/7407\n[#7408]: https://github.com/tokio-rs/tokio/pull/7408\n[#7462]: https://github.com/tokio-rs/tokio/pull/7462\n[#7475]: https://github.com/tokio-rs/tokio/pull/7475\n[#7493]: https://github.com/tokio-rs/tokio/pull/7493\n[#7500]: https://github.com/tokio-rs/tokio/pull/7500\n\n# 0.7.15 (April 23rd, 2025)\n\n### Fixed\n\n- task: properly handle removed entries in `JoinMap` ([#7264])\n\n### Updated\n\n- deps: update hashbrown to 0.15 ([#7219])\n\n### Documented\n\n- task: explicitly state that `TaskTracker` does not abort tasks on Drop ([#7223])\n\n[#7219]: https://github.com/tokio-rs/tokio/pull/7219\n[#7223]: https://github.com/tokio-rs/tokio/pull/7223\n[#7264]: https://github.com/tokio-rs/tokio/pull/7264\n\n# 0.7.14 (March 12th, 2025)\n\n### Added\n\n- io: add `get_ref` and `get_mut` for `SyncIoBridge` ([#7128])\n- io: add `read_exact_arc` ([#7165])\n- sync: add `CancellationToken::run_until_cancelled_owned` ([#7081])\n\n### Changed\n\n- codec: optimize buffer reserve for `AnyDelimiterCodec::encode` ([#7188])\n- either: enable `Either` to use underlying `AsyncWrite` implementation ([#7025])\n\n### Fixed\n\n- codec: fix typo in API docs ([#7044])\n- util: fix example in `StreamReader` docs ([#7167])\n\n### Documented\n\n- io: add docs for `SyncIoBridge` with examples and alternatives ([#6815])\n\n### Internal\n\n- io: clean up buffer casts ([#7142])\n- task: run `spawn_pinned` tests with miri ([#7023])\n\n[#6815]: https://github.com/tokio-rs/tokio/pull/6815\n[#7023]: https://github.com/tokio-rs/tokio/pull/7023\n[#7025]: https://github.com/tokio-rs/tokio/pull/7025\n[#7044]: https://github.com/tokio-rs/tokio/pull/7044\n[#7081]: https://github.com/tokio-rs/tokio/pull/7081\n[#7128]: https://github.com/tokio-rs/tokio/pull/7128\n[#7142]: https://github.com/tokio-rs/tokio/pull/7142\n[#7165]: https://github.com/tokio-rs/tokio/pull/7165\n[#7167]: https://github.com/tokio-rs/tokio/pull/7167\n[#7188]: https://github.com/tokio-rs/tokio/pull/7188\n\n# 0.7.13 (December 4th, 2024)\n\n### Fixed\n\n- codec: fix incorrect handling of invalid utf-8 in `LinesCodec::decode_eof` ([#7011])\n\n[#7011]: https://github.com/tokio-rs/tokio/pull/7011\n\n# 0.7.12 (September 5th, 2024)\n\nThis release bumps the MSRV to 1.70. ([#6645])\n\n### Added\n- sync: Add `run_until_cancelled` to `tokio_util::sync::CancellationToken` ([#6618])\n- task: add `AbortOnDropHandle` type ([#6786])\n\n### Changed\n- deps: no default features for hashbrown ([#6541])\n- time: wake `DelayQueue` when removing last item ([#6752])\n- deps: enable the full feature when compiled for the playground ([#6818])\n\n### Documented\n- task: fix typo in `TaskTracker` docs ([#6792])\n\n[#6645]: https://github.com/tokio-rs/tokio/pull/6645\n[#6541]: https://github.com/tokio-rs/tokio/pull/6541\n[#6618]: https://github.com/tokio-rs/tokio/pull/6618\n[#6752]: https://github.com/tokio-rs/tokio/pull/6752\n[#6786]: https://github.com/tokio-rs/tokio/pull/6786\n[#6792]: https://github.com/tokio-rs/tokio/pull/6792\n[#6818]: https://github.com/tokio-rs/tokio/pull/6818\n\n# 0.7.11 (May 4th, 2024)\n\nThis release updates the MSRV to 1.63. ([#6126])\n\n### Added\n\n- either: implement `Sink` for `Either` ([#6239])\n- time: add `DelayQueue::deadline` ([#6163])\n- time: add `FutureExt::timeout` ([#6276])\n\n### Changed\n\n- codec: assert compatibility between `LengthDelimitedCodec` options ([#6414])\n- codec: make tracing feature optional for codecs ([#6434])\n- io: add `T: ?Sized` to `tokio_util::io::poll_read_buf` ([#6441])\n- sync: remove `'static` bound on `impl Sink for PollSender` ([#6397])\n\n### Documented\n\n- codec: add examples for `FramedRead` and `FramedWrite` ([#6310])\n- codec: document cancel safety of `SinkExt::send` and `StreamExt::next` ([#6417])\n\n[#6126]: https://github.com/tokio-rs/tokio/pull/6126\n[#6163]: https://github.com/tokio-rs/tokio/pull/6163\n[#6239]: https://github.com/tokio-rs/tokio/pull/6239\n[#6276]: https://github.com/tokio-rs/tokio/pull/6276\n[#6310]: https://github.com/tokio-rs/tokio/pull/6310\n[#6397]: https://github.com/tokio-rs/tokio/pull/6397\n[#6414]: https://github.com/tokio-rs/tokio/pull/6414\n[#6417]: https://github.com/tokio-rs/tokio/pull/6417\n[#6434]: https://github.com/tokio-rs/tokio/pull/6434\n[#6441]: https://github.com/tokio-rs/tokio/pull/6441\n\n# 0.7.10 (October 24th, 2023)\n\n### Added\n\n- task: add `TaskTracker` ([#6033])\n- task: add `JoinMap::keys` ([#6046])\n- io: implement `Seek` for `SyncIoBridge` ([#6058])\n\n### Changed\n\n- deps: update hashbrown to 0.14 ([#6102])\n\n[#6033]: https://github.com/tokio-rs/tokio/pull/6033\n[#6046]: https://github.com/tokio-rs/tokio/pull/6046\n[#6058]: https://github.com/tokio-rs/tokio/pull/6058\n[#6102]: https://github.com/tokio-rs/tokio/pull/6102\n\n# 0.7.9 (September 20th, 2023)\n\n### Added\n\n- io: add passthrough `AsyncRead`/`AsyncWrite` to `InspectWriter`/`InspectReader` ([#5739])\n- task: add spawn blocking methods to `JoinMap` ([#5797])\n- io: pass through traits for `StreamReader` and `SinkWriter` ([#5941])\n- io: add `SyncIoBridge::into_inner` ([#5971])\n\n### Fixed\n\n- sync: handle possibly dangling reference safely ([#5812])\n- util: fix broken intra-doc link ([#5849])\n- compat: fix clippy warnings ([#5891])\n\n### Documented\n\n- codec: Specify the line ending of `LinesCodec` ([#5982])\n\n[#5739]: https://github.com/tokio-rs/tokio/pull/5739\n[#5797]: https://github.com/tokio-rs/tokio/pull/5797\n[#5941]: https://github.com/tokio-rs/tokio/pull/5941\n[#5971]: https://github.com/tokio-rs/tokio/pull/5971\n[#5812]: https://github.com/tokio-rs/tokio/pull/5812\n[#5849]: https://github.com/tokio-rs/tokio/pull/5849\n[#5891]: https://github.com/tokio-rs/tokio/pull/5891\n[#5982]: https://github.com/tokio-rs/tokio/pull/5982\n\n# 0.7.8 (April 25th, 2023)\n\nThis release bumps the MSRV of tokio-util to 1.56.\n\n### Added\n\n- time: add `DelayQueue::peek` ([#5569])\n\n### Changed\n\nThis release contains one performance improvement:\n\n- sync: try to lock the parent first in `CancellationToken` ([#5561])\n\n### Fixed\n\n- time: fix panic in `DelayQueue` ([#5630])\n\n### Documented\n\n- sync: improve `CancellationToken` doc on child tokens ([#5632])\n\n[#5561]: https://github.com/tokio-rs/tokio/pull/5561\n[#5569]: https://github.com/tokio-rs/tokio/pull/5569\n[#5630]: https://github.com/tokio-rs/tokio/pull/5630\n[#5632]: https://github.com/tokio-rs/tokio/pull/5632\n\n# 0.7.7 (February 12th, 2023)\n\nThis release reverts the removal of the `Encoder` bound on the `FramedParts`\nconstructor from [#5280] since it turned out to be a breaking change. ([#5450])\n\n[#5450]: https://github.com/tokio-rs/tokio/pull/5450\n\n# 0.7.6 (February 10, 2023)\n\nThis release fixes a compilation failure in 0.7.5 when it is used together with\nTokio version 1.21 and unstable features are enabled. ([#5445])\n\n[#5445]: https://github.com/tokio-rs/tokio/pull/5445\n\n# 0.7.5 (February 9, 2023)\n\nThis release fixes an accidental breaking change where `UnwindSafe` was\naccidentally removed from `CancellationToken`.\n\n### Added\n- codec: add `Framed::backpressure_boundary` ([#5124])\n- io: add `InspectReader` and `InspectWriter` ([#5033])\n- io: add `tokio_util::io::{CopyToBytes, SinkWriter}` ([#5070], [#5436])\n- io: impl `std::io::BufRead` on `SyncIoBridge` ([#5265])\n- sync: add `PollSemaphore::poll_acquire_many` ([#5137])\n- sync: add owned future for `CancellationToken` ([#5153])\n- time: add `DelayQueue::try_remove` ([#5052])\n\n### Fixed\n- codec: fix `LengthDelimitedCodec` buffer over-reservation ([#4997])\n- sync: impl `UnwindSafe` on `CancellationToken` ([#5438])\n- util: remove `Encoder` bound on `FramedParts` constructor ([#5280])\n\n### Documented\n- io: add lines example for `StreamReader` ([#5145])\n\n[#4997]: https://github.com/tokio-rs/tokio/pull/4997\n[#5033]: https://github.com/tokio-rs/tokio/pull/5033\n[#5052]: https://github.com/tokio-rs/tokio/pull/5052\n[#5070]: https://github.com/tokio-rs/tokio/pull/5070\n[#5124]: https://github.com/tokio-rs/tokio/pull/5124\n[#5137]: https://github.com/tokio-rs/tokio/pull/5137\n[#5145]: https://github.com/tokio-rs/tokio/pull/5145\n[#5153]: https://github.com/tokio-rs/tokio/pull/5153\n[#5265]: https://github.com/tokio-rs/tokio/pull/5265\n[#5280]: https://github.com/tokio-rs/tokio/pull/5280\n[#5436]: https://github.com/tokio-rs/tokio/pull/5436\n[#5438]: https://github.com/tokio-rs/tokio/pull/5438\n\n# 0.7.4 (September 8, 2022)\n\n### Added\n\n- io: add `SyncIoBridge::shutdown()` ([#4938])\n- task: improve `LocalPoolHandle` ([#4680])\n\n### Fixed\n\n- util: add `track_caller` to public APIs ([#4785])\n\n### Unstable\n\n- task: fix compilation errors in `JoinMap` with Tokio v1.21.0 ([#4755])\n- task: remove the unstable, deprecated `JoinMap::join_one` ([#4920])\n\n[#4680]: https://github.com/tokio-rs/tokio/pull/4680\n[#4755]: https://github.com/tokio-rs/tokio/pull/4755\n[#4785]: https://github.com/tokio-rs/tokio/pull/4785\n[#4920]: https://github.com/tokio-rs/tokio/pull/4920\n[#4938]: https://github.com/tokio-rs/tokio/pull/4938\n\n# 0.7.3 (June 4, 2022)\n\n### Changed\n\n- tracing: don't require default tracing features ([#4592])\n- util: simplify implementation of `ReusableBoxFuture` ([#4675])\n\n### Added (unstable)\n\n- task: add `JoinMap` ([#4640], [#4697])\n\n[#4592]: https://github.com/tokio-rs/tokio/pull/4592\n[#4640]: https://github.com/tokio-rs/tokio/pull/4640\n[#4675]: https://github.com/tokio-rs/tokio/pull/4675\n[#4697]: https://github.com/tokio-rs/tokio/pull/4697\n\n# 0.7.2 (May 14, 2022)\n\nThis release contains a rewrite of `CancellationToken` that fixes a memory leak. ([#4652])\n\n[#4652]: https://github.com/tokio-rs/tokio/pull/4652\n\n# 0.7.1 (February 21, 2022)\n\n### Added\n\n- codec: add `length_field_type` to `LengthDelimitedCodec` builder ([#4508])\n- io: add `StreamReader::into_inner_with_chunk()` ([#4559])\n\n### Changed\n\n- switch from log to tracing ([#4539])\n\n### Fixed\n\n- sync: fix waker update condition in `CancellationToken` ([#4497])\n- bumped tokio dependency to 1.6 to satisfy minimum requirements ([#4490])\n\n[#4490]: https://github.com/tokio-rs/tokio/pull/4490\n[#4497]: https://github.com/tokio-rs/tokio/pull/4497\n[#4508]: https://github.com/tokio-rs/tokio/pull/4508\n[#4539]: https://github.com/tokio-rs/tokio/pull/4539\n[#4559]: https://github.com/tokio-rs/tokio/pull/4559\n\n# 0.7.0 (February 9, 2022)\n\n### Added\n\n- task: add `spawn_pinned` ([#3370])\n- time: add `shrink_to_fit` and `compact` methods to `DelayQueue` ([#4170])\n- codec: improve `Builder::max_frame_length` docs ([#4352])\n- codec: add mutable reference getters for codecs to pinned `Framed` ([#4372])\n- net: add generic trait to combine `UnixListener` and `TcpListener` ([#4385])\n- codec: implement `Framed::map_codec` ([#4427])\n- codec: implement `Encoder` for `BytesCodec` ([#4465])\n\n### Changed\n\n- sync: add lifetime parameter to `ReusableBoxFuture` ([#3762])\n- sync: refactored `PollSender` to fix a subtly broken `Sink` implementation ([#4214])\n- time: remove error case from the infallible `DelayQueue::poll_elapsed` ([#4241])\n\n[#3370]: https://github.com/tokio-rs/tokio/pull/3370\n[#4170]: https://github.com/tokio-rs/tokio/pull/4170\n[#4352]: https://github.com/tokio-rs/tokio/pull/4352\n[#4372]: https://github.com/tokio-rs/tokio/pull/4372\n[#4385]: https://github.com/tokio-rs/tokio/pull/4385\n[#4427]: https://github.com/tokio-rs/tokio/pull/4427\n[#4465]: https://github.com/tokio-rs/tokio/pull/4465\n[#3762]: https://github.com/tokio-rs/tokio/pull/3762\n[#4214]: https://github.com/tokio-rs/tokio/pull/4214\n[#4241]: https://github.com/tokio-rs/tokio/pull/4241\n\n# 0.6.10 (May 14, 2021)\n\nThis is a backport for the memory leak in `CancellationToken` that was originally fixed in 0.7.2. ([#4652])\n\n[#4652]: https://github.com/tokio-rs/tokio/pull/4652\n\n# 0.6.9 (October 29, 2021)\n\n### Added\n\n- codec: implement `Clone` for `LengthDelimitedCodec` ([#4089])\n- io: add `SyncIoBridge` ([#4146])\n\n### Fixed\n\n- time: update deadline on removal in `DelayQueue` ([#4178])\n- codec: Update stream impl for Framed to return None after Err ([#4166])\n\n[#4089]: https://github.com/tokio-rs/tokio/pull/4089\n[#4146]: https://github.com/tokio-rs/tokio/pull/4146\n[#4166]: https://github.com/tokio-rs/tokio/pull/4166\n[#4178]: https://github.com/tokio-rs/tokio/pull/4178\n\n# 0.6.8 (September 3, 2021)\n\n### Added\n\n- sync: add drop guard for `CancellationToken` ([#3839])\n- compact: added `AsyncSeek` compat ([#4078])\n- time: expose `Key` used in `DelayQueue`'s `Expired` ([#4081])\n- io: add `with_capacity` to `ReaderStream` ([#4086])\n\n### Fixed\n\n- codec: remove unnecessary `doc(cfg(...))` ([#3989])\n\n[#3839]: https://github.com/tokio-rs/tokio/pull/3839\n[#4078]: https://github.com/tokio-rs/tokio/pull/4078\n[#4081]: https://github.com/tokio-rs/tokio/pull/4081\n[#4086]: https://github.com/tokio-rs/tokio/pull/4086\n[#3989]: https://github.com/tokio-rs/tokio/pull/3989\n\n# 0.6.7 (May 14, 2021)\n\n### Added\n\n- udp: make `UdpFramed` take `Borrow` ([#3451])\n- compat: implement `AsRawFd`/`AsRawHandle` for `Compat` ([#3765])\n\n[#3451]: https://github.com/tokio-rs/tokio/pull/3451\n[#3765]: https://github.com/tokio-rs/tokio/pull/3765\n\n# 0.6.6 (April 12, 2021)\n\n### Added\n\n- util: makes `Framed` and `FramedStream` resumable after eof ([#3272])\n- util: add `PollSemaphore::{add_permits, available_permits}` ([#3683])\n\n### Fixed\n\n- chore: avoid allocation if `PollSemaphore` is unused ([#3634])\n\n[#3272]: https://github.com/tokio-rs/tokio/pull/3272\n[#3634]: https://github.com/tokio-rs/tokio/pull/3634\n[#3683]: https://github.com/tokio-rs/tokio/pull/3683\n\n# 0.6.5 (March 20, 2021)\n\n### Fixed\n\n- util: annotate time module as requiring `time` feature ([#3606])\n\n[#3606]: https://github.com/tokio-rs/tokio/pull/3606\n\n# 0.6.4 (March 9, 2021)\n\n### Added\n\n- codec: `AnyDelimiter` codec ([#3406])\n- sync: add pollable `mpsc::Sender` ([#3490])\n\n### Fixed\n\n- codec: `LinesCodec` should only return `MaxLineLengthExceeded` once per line ([#3556])\n- sync: fuse PollSemaphore ([#3578])\n\n[#3406]: https://github.com/tokio-rs/tokio/pull/3406\n[#3490]: https://github.com/tokio-rs/tokio/pull/3490\n[#3556]: https://github.com/tokio-rs/tokio/pull/3556\n[#3578]: https://github.com/tokio-rs/tokio/pull/3578\n\n# 0.6.3 (January 31, 2021)\n\n### Added\n\n- sync: add `ReusableBoxFuture` utility ([#3464])\n\n### Changed\n\n- sync: use `ReusableBoxFuture` for `PollSemaphore` ([#3463])\n- deps: remove `async-stream` dependency ([#3463])\n- deps: remove `tokio-stream` dependency ([#3487])\n\n# 0.6.2 (January 21, 2021)\n\n### Added\n\n- sync: add pollable `Semaphore` ([#3444])\n\n### Fixed\n\n- time: fix panics on updating `DelayQueue` entries ([#3270])\n\n# 0.6.1 (January 12, 2021)\n\n### Added\n\n- codec: `get_ref()`, `get_mut()`, `get_pin_mut()` and `into_inner()` for\n `Framed`, `FramedRead`, `FramedWrite` and `StreamReader` ([#3364]).\n- codec: `write_buffer()` and `write_buffer_mut()` for `Framed` and\n `FramedWrite` ([#3387]).\n\n# 0.6.0 (December 23, 2020)\n\n### Changed\n- depend on `tokio` 1.0.\n\n### Added\n- rt: add constructors to `TokioContext` (#3221).\n\n# 0.5.1 (December 3, 2020)\n\n### Added\n- io: `poll_read_buf` util fn (#2972).\n- io: `poll_write_buf` util fn with vectored write support (#3156).\n\n# 0.5.0 (October 30, 2020)\n\n### Changed\n- io: update `bytes` to 0.6 (#3071).\n\n# 0.4.0 (October 15, 2020)\n\n### Added\n- sync: `CancellationToken` for coordinating task cancellation (#2747).\n- rt: `TokioContext` sets the Tokio runtime for the duration of a future (#2791)\n- io: `StreamReader`/`ReaderStream` map between `AsyncRead` values and `Stream`\n of bytes (#2788).\n- time: `DelayQueue` to manage many delays (#2897).\n\n# 0.3.1 (March 18, 2020)\n\n### Fixed\n\n- Adjust minimum-supported Tokio version to v0.2.5 to account for an internal\n dependency on features in that version of Tokio. ([#2326])\n\n# 0.3.0 (March 4, 2020)\n\n### Changed\n\n- **Breaking Change**: Change `Encoder` trait to take a generic `Item` parameter, which allows\n codec writers to pass references into `Framed` and `FramedWrite` types. ([#1746])\n\n### Added\n\n- Add futures-io/tokio::io compatibility layer. ([#2117])\n- Add `Framed::with_capacity`. ([#2215])\n\n### Fixed\n\n- Use advance over split_to when data is not needed. ([#2198])\n\n# 0.2.0 (November 26, 2019)\n\n- Initial release\n\n[#3487]: https://github.com/tokio-rs/tokio/pull/3487\n[#3464]: https://github.com/tokio-rs/tokio/pull/3464\n[#3463]: https://github.com/tokio-rs/tokio/pull/3463\n[#3444]: https://github.com/tokio-rs/tokio/pull/3444\n[#3387]: https://github.com/tokio-rs/tokio/pull/3387\n[#3364]: https://github.com/tokio-rs/tokio/pull/3364\n[#3270]: https://github.com/tokio-rs/tokio/pull/3270\n[#2326]: https://github.com/tokio-rs/tokio/pull/2326\n[#2215]: https://github.com/tokio-rs/tokio/pull/2215\n[#2198]: https://github.com/tokio-rs/tokio/pull/2198\n[#2117]: https://github.com/tokio-rs/tokio/pull/2117\n[#1746]: https://github.com/tokio-rs/tokio/pull/1746\n" }, { "path": "tokio-util/Cargo.toml", "content": "[package]\nname = \"tokio-util\"\n# When releasing to crates.io:\n# - Remove path dependencies (if any)\n# - Update CHANGELOG.md.\n# - Create \"tokio-util-0.7.x\" git tag.\nversion = \"0.7.18\"\nedition = \"2021\"\nrust-version = \"1.71\"\nauthors = [\"Tokio Contributors \"]\nlicense = \"MIT\"\nrepository = \"https://github.com/tokio-rs/tokio\"\nhomepage = \"https://tokio.rs\"\ndescription = \"\"\"\nAdditional utilities for working with Tokio.\n\"\"\"\ncategories = [\"asynchronous\"]\n\n[features]\n# No features on by default\ndefault = []\n\n# Shorthand for enabling everything\nfull = [\"codec\", \"compat\", \"io-util\", \"time\", \"net\", \"rt\", \"join-map\"]\n\nnet = [\"tokio/net\"]\ncompat = [\"futures-io\"]\ncodec = []\ntime = [\"tokio/time\", \"slab\"]\nio = []\nio-util = [\"io\", \"tokio/rt\", \"tokio/io-util\"]\nrt = [\"tokio/rt\", \"tokio/sync\", \"futures-util\"]\njoin-map = [\"rt\", \"hashbrown\"]\n\n__docs_rs = [\"futures-util\"]\n\n[dependencies]\ntokio = { version = \"1.47.0\", features = [\"sync\"] }\nbytes = \"1.5.0\"\nfutures-core = \"0.3.0\"\nfutures-sink = \"0.3.0\"\nfutures-io = { version = \"0.3.0\", optional = true }\nfutures-util = { version = \"0.3.0\", optional = true }\npin-project-lite = \"0.2.11\"\nslab = { version = \"0.4.4\", optional = true } # Backs `DelayQueue`\ntracing = { version = \"0.1.29\", default-features = false, features = [\"std\"], optional = true }\nhashbrown = { version = \"0.15.0\", default-features = false, optional = true }\n\n[dev-dependencies]\ntokio = { version = \"1.0.0\", features = [\"full\"] }\ntokio-test = \"0.4.0\"\ntokio-stream = \"0.1\"\n\nasync-stream = \"0.3.0\"\nfutures = \"0.3.0\"\nfutures-test = \"0.3.5\"\nparking_lot = \"0.12.0\"\ntempfile = \"3.1.0\"\n\n[target.'cfg(loom)'.dev-dependencies]\nloom = { version = \"0.7\", features = [\"futures\", \"checkpoint\"] }\n\n[package.metadata.docs.rs]\nall-features = true\n# enable unstable features in the documentation\nrustdoc-args = [\"--cfg\", \"docsrs\", \"--cfg\", \"tokio_unstable\"]\n# it's necessary to _also_ pass `--cfg tokio_unstable` to rustc, or else\n# dependencies will not be enabled, and the docs build will fail.\nrustc-args = [\"--cfg\", \"docsrs\", \"--cfg\", \"tokio_unstable\"]\n\n[package.metadata.playground]\nfeatures = [\"full\"]\n\n[lints]\nworkspace = true\n" }, { "path": "tokio-util/LICENSE", "content": "MIT License\n\nCopyright (c) Tokio Contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "tokio-util/README.md", "content": "# tokio-util\n\nUtilities for working with Tokio.\n\n## License\n\nThis project is licensed under the [MIT license](LICENSE).\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in Tokio by you, shall be licensed as MIT, without any additional\nterms or conditions.\n" }, { "path": "tokio-util/src/cfg.rs", "content": "macro_rules! cfg_codec {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"codec\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"codec\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_compat {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"compat\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"compat\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_net {\n ($($item:item)*) => {\n $(\n #[cfg(all(feature = \"net\", feature = \"codec\"))]\n #[cfg_attr(docsrs, doc(cfg(all(feature = \"net\", feature = \"codec\"))))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_io {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"io\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io\")))]\n $item\n )*\n }\n}\n\ncfg_io! {\n macro_rules! cfg_io_util {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"io-util\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"io-util\")))]\n $item\n )*\n }\n }\n}\n\nmacro_rules! cfg_rt {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_not_rt {\n ($($item:item)*) => {\n $(\n #[cfg(not(feature = \"rt\"))]\n $item\n )*\n }\n}\n\nmacro_rules! cfg_time {\n ($($item:item)*) => {\n $(\n #[cfg(feature = \"time\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"time\")))]\n $item\n )*\n }\n}\n" }, { "path": "tokio-util/src/codec/any_delimiter_codec.rs", "content": "use crate::codec::decoder::Decoder;\nuse crate::codec::encoder::Encoder;\n\nuse bytes::{Buf, BufMut, Bytes, BytesMut};\nuse std::{cmp, fmt, io, str};\n\nconst DEFAULT_SEEK_DELIMITERS: &[u8] = b\",;\\n\\r\";\nconst DEFAULT_SEQUENCE_WRITER: &[u8] = b\",\";\n/// A simple [`Decoder`] and [`Encoder`] implementation that splits up data into chunks based on any character in the given delimiter string.\n///\n/// [`Decoder`]: crate::codec::Decoder\n/// [`Encoder`]: crate::codec::Encoder\n///\n/// # Example\n/// Decode string of bytes containing various different delimiters.\n///\n/// [`BytesMut`]: bytes::BytesMut\n/// [`Error`]: std::io::Error\n///\n/// ```\n/// use tokio_util::codec::{AnyDelimiterCodec, Decoder};\n/// use bytes::{BufMut, BytesMut};\n///\n/// #\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), std::io::Error> {\n/// let mut codec = AnyDelimiterCodec::new(b\",;\\r\\n\".to_vec(),b\";\".to_vec());\n/// let buf = &mut BytesMut::new();\n/// buf.reserve(200);\n/// buf.put_slice(b\"chunk 1,chunk 2;chunk 3\\n\\r\");\n/// assert_eq!(\"chunk 1\", codec.decode(buf).unwrap().unwrap());\n/// assert_eq!(\"chunk 2\", codec.decode(buf).unwrap().unwrap());\n/// assert_eq!(\"chunk 3\", codec.decode(buf).unwrap().unwrap());\n/// assert_eq!(\"\", codec.decode(buf).unwrap().unwrap());\n/// assert_eq!(None, codec.decode(buf).unwrap());\n/// # Ok(())\n/// # }\n/// ```\n///\n#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]\npub struct AnyDelimiterCodec {\n // Stored index of the next index to examine for the delimiter character.\n // This is used to optimize searching.\n // For example, if `decode` was called with `abc` and the delimiter is '{}', it would hold `3`,\n // because that is the next index to examine.\n // The next time `decode` is called with `abcde}`, the method will\n // only look at `de}` before returning.\n next_index: usize,\n\n /// The maximum length for a given chunk. If `usize::MAX`, chunks will be\n /// read until a delimiter character is reached.\n max_length: usize,\n\n /// Are we currently discarding the remainder of a chunk which was over\n /// the length limit?\n is_discarding: bool,\n\n /// The bytes that are using for search during decode\n seek_delimiters: Vec,\n\n /// The bytes that are using for encoding\n sequence_writer: Vec,\n}\n\nimpl AnyDelimiterCodec {\n /// Returns a `AnyDelimiterCodec` for splitting up data into chunks.\n ///\n /// # Note\n ///\n /// The returned `AnyDelimiterCodec` will not have an upper bound on the length\n /// of a buffered chunk. See the documentation for [`new_with_max_length`]\n /// for information on why this could be a potential security risk.\n ///\n /// [`new_with_max_length`]: crate::codec::AnyDelimiterCodec::new_with_max_length()\n pub fn new(seek_delimiters: Vec, sequence_writer: Vec) -> AnyDelimiterCodec {\n AnyDelimiterCodec {\n next_index: 0,\n max_length: usize::MAX,\n is_discarding: false,\n seek_delimiters,\n sequence_writer,\n }\n }\n\n /// Returns a `AnyDelimiterCodec` with a maximum chunk length limit.\n ///\n /// If this is set, calls to `AnyDelimiterCodec::decode` will return a\n /// [`AnyDelimiterCodecError`] when a chunk exceeds the length limit. Subsequent calls\n /// will discard up to `limit` bytes from that chunk until a delimiter\n /// character is reached, returning `None` until the delimiter over the limit\n /// has been fully discarded. After that point, calls to `decode` will\n /// function as normal.\n ///\n /// # Note\n ///\n /// Setting a length limit is highly recommended for any `AnyDelimiterCodec` which\n /// will be exposed to untrusted input. Otherwise, the size of the buffer\n /// that holds the chunk currently being read is unbounded. An attacker could\n /// exploit this unbounded buffer by sending an unbounded amount of input\n /// without any delimiter characters, causing unbounded memory consumption.\n ///\n /// [`AnyDelimiterCodecError`]: crate::codec::AnyDelimiterCodecError\n pub fn new_with_max_length(\n seek_delimiters: Vec,\n sequence_writer: Vec,\n max_length: usize,\n ) -> Self {\n AnyDelimiterCodec {\n max_length,\n ..AnyDelimiterCodec::new(seek_delimiters, sequence_writer)\n }\n }\n\n /// Returns the maximum chunk length when decoding.\n ///\n /// ```\n /// use std::usize;\n /// use tokio_util::codec::AnyDelimiterCodec;\n ///\n /// let codec = AnyDelimiterCodec::new(b\",;\\n\".to_vec(), b\";\".to_vec());\n /// assert_eq!(codec.max_length(), usize::MAX);\n /// ```\n /// ```\n /// use tokio_util::codec::AnyDelimiterCodec;\n ///\n /// let codec = AnyDelimiterCodec::new_with_max_length(b\",;\\n\".to_vec(), b\";\".to_vec(), 256);\n /// assert_eq!(codec.max_length(), 256);\n /// ```\n pub fn max_length(&self) -> usize {\n self.max_length\n }\n}\n\nimpl Decoder for AnyDelimiterCodec {\n type Item = Bytes;\n type Error = AnyDelimiterCodecError;\n\n fn decode(&mut self, buf: &mut BytesMut) -> Result, AnyDelimiterCodecError> {\n loop {\n // Determine how far into the buffer we'll search for a delimiter. If\n // there's no max_length set, we'll read to the end of the buffer.\n let read_to = cmp::min(self.max_length.saturating_add(1), buf.len());\n\n let new_chunk_offset = buf[self.next_index..read_to]\n .iter()\n .position(|b| self.seek_delimiters.contains(b));\n\n match (self.is_discarding, new_chunk_offset) {\n (true, Some(offset)) => {\n // If we found a new chunk, discard up to that offset and\n // then stop discarding. On the next iteration, we'll try\n // to read a chunk normally.\n buf.advance(offset + self.next_index + 1);\n self.is_discarding = false;\n self.next_index = 0;\n }\n (true, None) => {\n // Otherwise, we didn't find a new chunk, so we'll discard\n // everything we read. On the next iteration, we'll continue\n // discarding up to max_len bytes unless we find a new chunk.\n buf.advance(read_to);\n self.next_index = 0;\n if buf.is_empty() {\n return Ok(None);\n }\n }\n (false, Some(offset)) => {\n // Found a chunk!\n let new_chunk_index = offset + self.next_index;\n self.next_index = 0;\n let mut chunk = buf.split_to(new_chunk_index + 1);\n chunk.truncate(chunk.len() - 1);\n let chunk = chunk.freeze();\n return Ok(Some(chunk));\n }\n (false, None) if buf.len() > self.max_length => {\n // Reached the maximum length without finding a\n // new chunk, return an error and start discarding on the\n // next call.\n self.is_discarding = true;\n return Err(AnyDelimiterCodecError::MaxChunkLengthExceeded);\n }\n (false, None) => {\n // We didn't find a chunk or reach the length limit, so the next\n // call will resume searching at the current offset.\n self.next_index = read_to;\n return Ok(None);\n }\n }\n }\n }\n\n fn decode_eof(&mut self, buf: &mut BytesMut) -> Result, AnyDelimiterCodecError> {\n Ok(match self.decode(buf)? {\n Some(frame) => Some(frame),\n None => {\n // return remaining data, if any\n if buf.is_empty() {\n None\n } else {\n let chunk = buf.split_to(buf.len());\n self.next_index = 0;\n Some(chunk.freeze())\n }\n }\n })\n }\n}\n\nimpl Encoder for AnyDelimiterCodec\nwhere\n T: AsRef,\n{\n type Error = AnyDelimiterCodecError;\n\n fn encode(&mut self, chunk: T, buf: &mut BytesMut) -> Result<(), AnyDelimiterCodecError> {\n let chunk = chunk.as_ref();\n buf.reserve(chunk.len() + self.sequence_writer.len());\n buf.put(chunk.as_bytes());\n buf.put(self.sequence_writer.as_ref());\n\n Ok(())\n }\n}\n\nimpl Default for AnyDelimiterCodec {\n fn default() -> Self {\n Self::new(\n DEFAULT_SEEK_DELIMITERS.to_vec(),\n DEFAULT_SEQUENCE_WRITER.to_vec(),\n )\n }\n}\n\n/// An error occurred while encoding or decoding a chunk.\n#[derive(Debug)]\npub enum AnyDelimiterCodecError {\n /// The maximum chunk length was exceeded.\n MaxChunkLengthExceeded,\n /// An IO error occurred.\n Io(io::Error),\n}\n\nimpl fmt::Display for AnyDelimiterCodecError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n AnyDelimiterCodecError::MaxChunkLengthExceeded => {\n write!(f, \"max chunk length exceeded\")\n }\n AnyDelimiterCodecError::Io(e) => write!(f, \"{e}\"),\n }\n }\n}\n\nimpl From for AnyDelimiterCodecError {\n fn from(e: io::Error) -> AnyDelimiterCodecError {\n AnyDelimiterCodecError::Io(e)\n }\n}\n\nimpl std::error::Error for AnyDelimiterCodecError {}\n" }, { "path": "tokio-util/src/codec/bytes_codec.rs", "content": "use crate::codec::decoder::Decoder;\nuse crate::codec::encoder::Encoder;\n\nuse bytes::{BufMut, Bytes, BytesMut};\nuse std::io;\n\n/// A simple [`Decoder`] and [`Encoder`] implementation that just ships bytes around.\n///\n/// [`Decoder`]: crate::codec::Decoder\n/// [`Encoder`]: crate::codec::Encoder\n///\n/// # Example\n///\n/// Turn an [`AsyncRead`] into a stream of `Result<`[`BytesMut`]`, `[`Error`]`>`.\n///\n/// [`AsyncRead`]: tokio::io::AsyncRead\n/// [`BytesMut`]: bytes::BytesMut\n/// [`Error`]: std::io::Error\n///\n/// ```\n/// # mod hidden {\n/// # #[allow(unused_imports)]\n/// use tokio::fs::File;\n/// # }\n/// use tokio::io::AsyncRead;\n/// use tokio_util::codec::{FramedRead, BytesCodec};\n///\n/// # enum File {}\n/// # impl File {\n/// # async fn open(_name: &str) -> Result {\n/// # use std::io::Cursor;\n/// # Ok(Cursor::new(vec![0, 1, 2, 3, 4, 5]))\n/// # }\n/// # }\n/// #\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), std::io::Error> {\n/// let my_async_read = File::open(\"filename.txt\").await?;\n/// let my_stream_of_bytes = FramedRead::new(my_async_read, BytesCodec::new());\n/// # Ok(())\n/// # }\n/// ```\n///\n#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash, Default)]\npub struct BytesCodec(());\n\nimpl BytesCodec {\n /// Creates a new `BytesCodec` for shipping around raw bytes.\n pub fn new() -> BytesCodec {\n BytesCodec(())\n }\n}\n\nimpl Decoder for BytesCodec {\n type Item = BytesMut;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> Result, io::Error> {\n if !buf.is_empty() {\n let len = buf.len();\n Ok(Some(buf.split_to(len)))\n } else {\n Ok(None)\n }\n }\n}\n\nimpl Encoder for BytesCodec {\n type Error = io::Error;\n\n fn encode(&mut self, data: Bytes, buf: &mut BytesMut) -> Result<(), io::Error> {\n buf.reserve(data.len());\n buf.put(data);\n Ok(())\n }\n}\n\nimpl Encoder for BytesCodec {\n type Error = io::Error;\n\n fn encode(&mut self, data: BytesMut, buf: &mut BytesMut) -> Result<(), io::Error> {\n buf.reserve(data.len());\n buf.put(data);\n Ok(())\n }\n}\n" }, { "path": "tokio-util/src/codec/decoder.rs", "content": "use crate::codec::Framed;\n\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse bytes::BytesMut;\nuse std::io;\n\n/// Decoding of frames via buffers.\n///\n/// This trait is used when constructing an instance of [`Framed`] or\n/// [`FramedRead`]. An implementation of `Decoder` takes a byte stream that has\n/// already been buffered in `src` and decodes the data into a stream of\n/// `Self::Item` frames.\n///\n/// Implementations are able to track state on `self`, which enables\n/// implementing stateful streaming parsers. In many cases, though, this type\n/// will simply be a unit struct (e.g. `struct HttpDecoder`).\n///\n/// For some underlying data-sources, namely files and FIFOs,\n/// it's possible to temporarily read 0 bytes by reaching EOF.\n///\n/// In these cases `decode_eof` will be called until it signals\n/// fulfillment of all closing frames by returning `Ok(None)`.\n/// After that, repeated attempts to read from the [`Framed`] or [`FramedRead`]\n/// will not invoke `decode` or `decode_eof` again, until data can be read\n/// during a retry.\n///\n/// It is up to the Decoder to keep track of a restart after an EOF,\n/// and to decide how to handle such an event by, for example,\n/// allowing frames to cross EOF boundaries, re-emitting opening frames, or\n/// resetting the entire internal state.\n///\n/// [`Framed`]: crate::codec::Framed\n/// [`FramedRead`]: crate::codec::FramedRead\npub trait Decoder {\n /// The type of decoded frames.\n type Item;\n\n /// The type of unrecoverable frame decoding errors.\n ///\n /// If an individual message is ill-formed but can be ignored without\n /// interfering with the processing of future messages, it may be more\n /// useful to report the failure as an `Item`.\n ///\n /// `From` is required in the interest of making `Error` suitable\n /// for returning directly from a [`FramedRead`], and to enable the default\n /// implementation of `decode_eof` to yield an `io::Error` when the decoder\n /// fails to consume all available data.\n ///\n /// Note that implementors of this trait can simply indicate `type Error =\n /// io::Error` to use I/O errors as this type.\n ///\n /// [`FramedRead`]: crate::codec::FramedRead\n type Error: From;\n\n /// Attempts to decode a frame from the provided buffer of bytes.\n ///\n /// This method is called by [`FramedRead`] whenever bytes are ready to be\n /// parsed. The provided buffer of bytes is what's been read so far, and\n /// this instance of `Decode` can determine whether an entire frame is in\n /// the buffer and is ready to be returned.\n ///\n /// If an entire frame is available, then this instance will remove those\n /// bytes from the buffer provided and return them as a decoded\n /// frame. Note that removing bytes from the provided buffer doesn't always\n /// necessarily copy the bytes, so this should be an efficient operation in\n /// most circumstances.\n ///\n /// If the bytes look valid, but a frame isn't fully available yet, then\n /// `Ok(None)` is returned. This indicates to the [`Framed`] instance that\n /// it needs to read some more bytes before calling this method again.\n ///\n /// Note that the bytes provided may be empty. If a previous call to\n /// `decode` consumed all the bytes in the buffer then `decode` will be\n /// called again until it returns `Ok(None)`, indicating that more bytes need to\n /// be read.\n ///\n /// Finally, if the bytes in the buffer are malformed then an error is\n /// returned indicating why. This informs [`Framed`] that the stream is now\n /// corrupt and should be terminated.\n ///\n /// [`Framed`]: crate::codec::Framed\n /// [`FramedRead`]: crate::codec::FramedRead\n ///\n /// # Buffer management\n ///\n /// Before returning from the function, implementations should ensure that\n /// the buffer has appropriate capacity in anticipation of future calls to\n /// `decode`. Failing to do so leads to inefficiency.\n ///\n /// For example, if frames have a fixed length, or if the length of the\n /// current frame is known from a header, a possible buffer management\n /// strategy is:\n ///\n /// ```no_run\n /// # use std::io;\n /// #\n /// # use bytes::BytesMut;\n /// # use tokio_util::codec::Decoder;\n /// #\n /// # struct MyCodec;\n /// #\n /// impl Decoder for MyCodec {\n /// // ...\n /// # type Item = BytesMut;\n /// # type Error = io::Error;\n ///\n /// fn decode(&mut self, src: &mut BytesMut) -> Result, Self::Error> {\n /// // ...\n ///\n /// // Reserve enough to complete decoding of the current frame.\n /// let current_frame_len: usize = 1000; // Example.\n /// // And to start decoding the next frame.\n /// let next_frame_header_len: usize = 10; // Example.\n /// src.reserve(current_frame_len + next_frame_header_len);\n ///\n /// return Ok(None);\n /// }\n /// }\n /// ```\n ///\n /// An optimal buffer management strategy minimizes reallocations and\n /// over-allocations.\n fn decode(&mut self, src: &mut BytesMut) -> Result, Self::Error>;\n\n /// A default method available to be called when there are no more bytes\n /// available to be read from the underlying I/O.\n ///\n /// This method defaults to calling `decode` and returns an error if\n /// `Ok(None)` is returned while there is unconsumed data in `buf`.\n /// Typically this doesn't need to be implemented unless the framing\n /// protocol differs near the end of the stream, or if you need to construct\n /// frames _across_ eof boundaries on sources that can be resumed.\n ///\n /// Note that the `buf` argument may be empty. If a previous call to\n /// `decode_eof` consumed all the bytes in the buffer, `decode_eof` will be\n /// called again until it returns `None`, indicating that there are no more\n /// frames to yield. This behavior enables returning finalization frames\n /// that may not be based on inbound data.\n ///\n /// Once `None` has been returned, `decode_eof` won't be called again until\n /// an attempt to resume the stream has been made, where the underlying stream\n /// actually returned more data.\n fn decode_eof(&mut self, buf: &mut BytesMut) -> Result, Self::Error> {\n match self.decode(buf)? {\n Some(frame) => Ok(Some(frame)),\n None => {\n if buf.is_empty() {\n Ok(None)\n } else {\n Err(io::Error::new(io::ErrorKind::Other, \"bytes remaining on stream\").into())\n }\n }\n }\n }\n\n /// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this\n /// `Io` object, using `Decode` and `Encode` to read and write the raw data.\n ///\n /// Raw I/O objects work with byte sequences, but higher-level code usually\n /// wants to batch these into meaningful chunks, called \"frames\". This\n /// method layers framing on top of an I/O object, by using the `Codec`\n /// traits to handle encoding and decoding of messages frames. Note that\n /// the incoming and outgoing frame types may be distinct.\n ///\n /// This function returns a *single* object that is both `Stream` and\n /// `Sink`; grouping this into a single object is often useful for layering\n /// things like gzip or TLS, which require both read and write access to the\n /// underlying object.\n ///\n /// If you want to work more directly with the streams and sink, consider\n /// calling `split` on the [`Framed`] returned by this method, which will\n /// break them into separate objects, allowing them to interact more easily.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`Sink`]: futures_sink::Sink\n /// [`Framed`]: crate::codec::Framed\n fn framed(self, io: T) -> Framed\n where\n Self: Sized,\n {\n Framed::new(io, self)\n }\n}\n" }, { "path": "tokio-util/src/codec/encoder.rs", "content": "use bytes::BytesMut;\nuse std::io;\n\n/// Trait of helper objects to write out messages as bytes, for use with\n/// [`FramedWrite`].\n///\n/// [`FramedWrite`]: crate::codec::FramedWrite\npub trait Encoder {\n /// The type of encoding errors.\n ///\n /// [`FramedWrite`] requires `Encoder`s errors to implement `From`\n /// in the interest of letting it return `Error`s directly.\n ///\n /// [`FramedWrite`]: crate::codec::FramedWrite\n type Error: From;\n\n /// Encodes a frame into the buffer provided.\n ///\n /// This method will encode `item` into the byte buffer provided by `dst`.\n /// The `dst` provided is an internal buffer of the [`FramedWrite`] instance and\n /// will be written out when possible.\n ///\n /// [`FramedWrite`]: crate::codec::FramedWrite\n fn encode(&mut self, item: Item, dst: &mut BytesMut) -> Result<(), Self::Error>;\n}\n" }, { "path": "tokio-util/src/codec/framed.rs", "content": "use crate::codec::decoder::Decoder;\nuse crate::codec::encoder::Encoder;\nuse crate::codec::framed_impl::{FramedImpl, RWFrames, ReadFrame, WriteFrame};\n\nuse futures_core::Stream;\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse bytes::BytesMut;\nuse futures_sink::Sink;\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A unified [`Stream`] and [`Sink`] interface to an underlying I/O object, using\n /// the `Encoder` and `Decoder` traits to encode and decode frames.\n ///\n /// You can create a `Framed` instance by using the [`Decoder::framed`] adapter, or\n /// by using the `new` function seen below.\n ///\n /// # Cancellation safety\n ///\n /// * [`futures_util::sink::SinkExt::send`]: if send is used as the event in a\n /// `tokio::select!` statement and some other branch completes first, then it is\n /// guaranteed that the message was not sent, but the message itself is lost.\n /// * [`tokio_stream::StreamExt::next`]: This method is cancel safe. The returned\n /// future only holds onto a reference to the underlying stream, so dropping it will\n /// never lose a value.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`Sink`]: futures_sink::Sink\n /// [`AsyncRead`]: tokio::io::AsyncRead\n /// [`Decoder::framed`]: crate::codec::Decoder::framed()\n /// [`futures_util::sink::SinkExt::send`]: futures_util::sink::SinkExt::send\n /// [`tokio_stream::StreamExt::next`]: https://docs.rs/tokio-stream/latest/tokio_stream/trait.StreamExt.html#method.next\n pub struct Framed {\n #[pin]\n inner: FramedImpl\n }\n}\n\nimpl Framed {\n /// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this\n /// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data.\n ///\n /// Raw I/O objects work with byte sequences, but higher-level code usually\n /// wants to batch these into meaningful chunks, called \"frames\". This\n /// method layers framing on top of an I/O object, by using the codec\n /// traits to handle encoding and decoding of messages frames. Note that\n /// the incoming and outgoing frame types may be distinct.\n ///\n /// This function returns a *single* object that is both [`Stream`] and\n /// [`Sink`]; grouping this into a single object is often useful for layering\n /// things like gzip or TLS, which require both read and write access to the\n /// underlying object.\n ///\n /// If you want to work more directly with the streams and sink, consider\n /// calling [`split`] on the `Framed` returned by this method, which will\n /// break them into separate objects, allowing them to interact more easily.\n ///\n /// Note that, for some byte sources, the stream can be resumed after an EOF\n /// by reading from it, even after it has returned `None`. Repeated attempts\n /// to do so, without new data available, continue to return `None` without\n /// creating more (closing) frames.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`Sink`]: futures_sink::Sink\n /// [`Decode`]: crate::codec::Decoder\n /// [`Encoder`]: crate::codec::Encoder\n /// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split\n pub fn new(inner: T, codec: U) -> Framed {\n Framed {\n inner: FramedImpl {\n inner,\n codec,\n state: Default::default(),\n },\n }\n }\n\n /// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this\n /// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data,\n /// with a specific read buffer initial capacity.\n ///\n /// Raw I/O objects work with byte sequences, but higher-level code usually\n /// wants to batch these into meaningful chunks, called \"frames\". This\n /// method layers framing on top of an I/O object, by using the codec\n /// traits to handle encoding and decoding of messages frames. Note that\n /// the incoming and outgoing frame types may be distinct.\n ///\n /// This function returns a *single* object that is both [`Stream`] and\n /// [`Sink`]; grouping this into a single object is often useful for layering\n /// things like gzip or TLS, which require both read and write access to the\n /// underlying object.\n ///\n /// If you want to work more directly with the streams and sink, consider\n /// calling [`split`] on the `Framed` returned by this method, which will\n /// break them into separate objects, allowing them to interact more easily.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`Sink`]: futures_sink::Sink\n /// [`Decode`]: crate::codec::Decoder\n /// [`Encoder`]: crate::codec::Encoder\n /// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split\n pub fn with_capacity(inner: T, codec: U, capacity: usize) -> Framed {\n Framed {\n inner: FramedImpl {\n inner,\n codec,\n state: RWFrames {\n read: ReadFrame {\n eof: false,\n is_readable: false,\n buffer: BytesMut::with_capacity(capacity),\n has_errored: false,\n },\n write: WriteFrame {\n buffer: BytesMut::with_capacity(capacity),\n backpressure_boundary: capacity,\n },\n },\n },\n }\n }\n\n /// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this\n /// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data.\n ///\n /// Raw I/O objects work with byte sequences, but higher-level code usually\n /// wants to batch these into meaningful chunks, called \"frames\". This\n /// method layers framing on top of an I/O object, by using the `Codec`\n /// traits to handle encoding and decoding of messages frames. Note that\n /// the incoming and outgoing frame types may be distinct.\n ///\n /// This function returns a *single* object that is both [`Stream`] and\n /// [`Sink`]; grouping this into a single object is often useful for layering\n /// things like gzip or TLS, which require both read and write access to the\n /// underlying object.\n ///\n /// This objects takes a stream and a `readbuffer` and a `writebuffer`. These field\n /// can be obtained from an existing `Framed` with the [`into_parts`] method.\n ///\n /// If you want to work more directly with the streams and sink, consider\n /// calling [`split`] on the `Framed` returned by this method, which will\n /// break them into separate objects, allowing them to interact more easily.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`Sink`]: futures_sink::Sink\n /// [`Decoder`]: crate::codec::Decoder\n /// [`Encoder`]: crate::codec::Encoder\n /// [`into_parts`]: crate::codec::Framed::into_parts()\n /// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split\n pub fn from_parts(parts: FramedParts) -> Framed {\n Framed {\n inner: FramedImpl {\n inner: parts.io,\n codec: parts.codec,\n state: RWFrames {\n read: parts.read_buf.into(),\n write: parts.write_buf.into(),\n },\n },\n }\n }\n\n /// Returns a reference to the underlying I/O stream wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_ref(&self) -> &T {\n &self.inner.inner\n }\n\n /// Returns a mutable reference to the underlying I/O stream wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.inner.inner\n }\n\n /// Returns a pinned mutable reference to the underlying I/O stream wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut T> {\n self.project().inner.project().inner\n }\n\n /// Returns a reference to the underlying codec wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn codec(&self) -> &U {\n &self.inner.codec\n }\n\n /// Returns a mutable reference to the underlying codec wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn codec_mut(&mut self) -> &mut U {\n &mut self.inner.codec\n }\n\n /// Maps the codec `U` to `C`, preserving the read and write buffers\n /// wrapped by `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn map_codec(self, map: F) -> Framed\n where\n F: FnOnce(U) -> C,\n {\n // This could be potentially simplified once rust-lang/rust#86555 hits stable\n let parts = self.into_parts();\n Framed::from_parts(FramedParts {\n io: parts.io,\n codec: map(parts.codec),\n read_buf: parts.read_buf,\n write_buf: parts.write_buf,\n _priv: (),\n })\n }\n\n /// Returns a mutable reference to the underlying codec wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn codec_pin_mut(self: Pin<&mut Self>) -> &mut U {\n self.project().inner.project().codec\n }\n\n /// Returns a reference to the read buffer.\n pub fn read_buffer(&self) -> &BytesMut {\n &self.inner.state.read.buffer\n }\n\n /// Returns a mutable reference to the read buffer.\n pub fn read_buffer_mut(&mut self) -> &mut BytesMut {\n &mut self.inner.state.read.buffer\n }\n\n /// Returns a reference to the write buffer.\n pub fn write_buffer(&self) -> &BytesMut {\n &self.inner.state.write.buffer\n }\n\n /// Returns a mutable reference to the write buffer.\n pub fn write_buffer_mut(&mut self) -> &mut BytesMut {\n &mut self.inner.state.write.buffer\n }\n\n /// Returns backpressure boundary\n pub fn backpressure_boundary(&self) -> usize {\n self.inner.state.write.backpressure_boundary\n }\n\n /// Updates backpressure boundary\n pub fn set_backpressure_boundary(&mut self, boundary: usize) {\n self.inner.state.write.backpressure_boundary = boundary;\n }\n\n /// Consumes the `Framed`, returning its underlying I/O stream.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn into_inner(self) -> T {\n self.inner.inner\n }\n\n /// Consumes the `Framed`, returning its underlying I/O stream, the buffer\n /// with unprocessed data, and the codec.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn into_parts(self) -> FramedParts {\n FramedParts {\n io: self.inner.inner,\n codec: self.inner.codec,\n read_buf: self.inner.state.read.buffer,\n write_buf: self.inner.state.write.buffer,\n _priv: (),\n }\n }\n}\n\n// This impl just defers to the underlying FramedImpl\nimpl Stream for Framed\nwhere\n T: AsyncRead,\n U: Decoder,\n{\n type Item = Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_next(cx)\n }\n}\n\n// This impl just defers to the underlying FramedImpl\nimpl Sink for Framed\nwhere\n T: AsyncWrite,\n U: Encoder,\n U::Error: From,\n{\n type Error = U::Error;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_ready(cx)\n }\n\n fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {\n self.project().inner.start_send(item)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_close(cx)\n }\n}\n\nimpl fmt::Debug for Framed\nwhere\n T: fmt::Debug,\n U: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"Framed\")\n .field(\"io\", self.get_ref())\n .field(\"codec\", self.codec())\n .finish()\n }\n}\n\n/// `FramedParts` contains an export of the data of a Framed transport.\n/// It can be used to construct a new [`Framed`] with a different codec.\n/// It contains all current buffers and the inner transport.\n///\n/// [`Framed`]: crate::codec::Framed\n#[derive(Debug)]\n#[allow(clippy::manual_non_exhaustive)]\npub struct FramedParts {\n /// The inner transport used to read bytes to and write bytes to\n pub io: T,\n\n /// The codec\n pub codec: U,\n\n /// The buffer with read but unprocessed data.\n pub read_buf: BytesMut,\n\n /// A buffer with unprocessed data which are not written yet.\n pub write_buf: BytesMut,\n\n /// This private field allows us to add additional fields in the future in a\n /// backwards compatible way.\n pub(crate) _priv: (),\n}\n\nimpl FramedParts {\n /// Create a new, default, `FramedParts`\n pub fn new(io: T, codec: U) -> FramedParts\n where\n U: Encoder,\n {\n FramedParts {\n io,\n codec,\n read_buf: BytesMut::new(),\n write_buf: BytesMut::new(),\n _priv: (),\n }\n }\n}\n" }, { "path": "tokio-util/src/codec/framed_impl.rs", "content": "use crate::codec::decoder::Decoder;\nuse crate::codec::encoder::Encoder;\n\nuse futures_core::Stream;\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse bytes::BytesMut;\nuse futures_sink::Sink;\nuse pin_project_lite::pin_project;\nuse std::borrow::{Borrow, BorrowMut};\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n #[derive(Debug)]\n pub(crate) struct FramedImpl {\n #[pin]\n pub(crate) inner: T,\n pub(crate) state: State,\n pub(crate) codec: U,\n }\n}\n\nconst INITIAL_CAPACITY: usize = 8 * 1024;\n\n#[derive(Debug)]\npub(crate) struct ReadFrame {\n pub(crate) eof: bool,\n pub(crate) is_readable: bool,\n pub(crate) buffer: BytesMut,\n pub(crate) has_errored: bool,\n}\n\npub(crate) struct WriteFrame {\n pub(crate) buffer: BytesMut,\n pub(crate) backpressure_boundary: usize,\n}\n\n#[derive(Default)]\npub(crate) struct RWFrames {\n pub(crate) read: ReadFrame,\n pub(crate) write: WriteFrame,\n}\n\nimpl Default for ReadFrame {\n fn default() -> Self {\n Self {\n eof: false,\n is_readable: false,\n buffer: BytesMut::with_capacity(INITIAL_CAPACITY),\n has_errored: false,\n }\n }\n}\n\nimpl Default for WriteFrame {\n fn default() -> Self {\n Self {\n buffer: BytesMut::with_capacity(INITIAL_CAPACITY),\n backpressure_boundary: INITIAL_CAPACITY,\n }\n }\n}\n\nimpl From for ReadFrame {\n fn from(mut buffer: BytesMut) -> Self {\n let is_readable = !buffer.is_empty();\n let size = buffer.capacity();\n if size < INITIAL_CAPACITY {\n buffer.reserve(INITIAL_CAPACITY - size);\n }\n\n Self {\n buffer,\n is_readable,\n eof: false,\n has_errored: false,\n }\n }\n}\n\nimpl From for WriteFrame {\n fn from(mut buffer: BytesMut) -> Self {\n let size = buffer.capacity();\n if size < INITIAL_CAPACITY {\n buffer.reserve(INITIAL_CAPACITY - size);\n }\n\n Self {\n buffer,\n backpressure_boundary: INITIAL_CAPACITY,\n }\n }\n}\n\nimpl Borrow for RWFrames {\n fn borrow(&self) -> &ReadFrame {\n &self.read\n }\n}\nimpl BorrowMut for RWFrames {\n fn borrow_mut(&mut self) -> &mut ReadFrame {\n &mut self.read\n }\n}\nimpl Borrow for RWFrames {\n fn borrow(&self) -> &WriteFrame {\n &self.write\n }\n}\nimpl BorrowMut for RWFrames {\n fn borrow_mut(&mut self) -> &mut WriteFrame {\n &mut self.write\n }\n}\nimpl Stream for FramedImpl\nwhere\n T: AsyncRead,\n U: Decoder,\n R: BorrowMut,\n{\n type Item = Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n use crate::util::poll_read_buf;\n\n let mut pinned = self.project();\n let state: &mut ReadFrame = pinned.state.borrow_mut();\n // The following loops implements a state machine with each state corresponding\n // to a combination of the `is_readable` and `eof` flags. States persist across\n // loop entries and most state transitions occur with a return.\n //\n // The initial state is `reading`.\n //\n // | state | eof | is_readable | has_errored |\n // |---------|-------|-------------|-------------|\n // | reading | false | false | false |\n // | framing | false | true | false |\n // | pausing | true | true | false |\n // | paused | true | false | false |\n // | errored | | | true |\n // `decode_eof` returns Err\n // ┌────────────────────────────────────────────────────────┐\n // `decode_eof` returns │ │\n // `Ok(Some)` │ │\n // ┌─────┐ │ `decode_eof` returns After returning │\n // Read 0 bytes ├─────▼──┴┐ `Ok(None)` ┌────────┐ ◄───┐ `None` ┌───▼─────┐\n // ┌────────────────►│ Pausing ├───────────────────────►│ Paused ├─┐ └───────────┤ Errored │\n // │ └─────────┘ └─┬──▲───┘ │ └───▲───▲─┘\n // Pending read │ │ │ │ │ │\n // ┌──────┐ │ `decode` returns `Some` │ └─────┘ │ │\n // │ │ │ ┌──────┐ │ Pending │ │\n // │ ┌────▼──┴─┐ Read n>0 bytes ┌┴──────▼─┐ read n>0 bytes │ read │ │\n // └─┤ Reading ├───────────────►│ Framing │◄────────────────────────┘ │ │\n // └──┬─▲────┘ └─────┬──┬┘ │ │\n // │ │ │ │ `decode` returns Err │ │\n // │ └───decode` returns `None`──┘ └───────────────────────────────────────────────────────┘ │\n // │ read returns Err │\n // └────────────────────────────────────────────────────────────────────────────────────────────┘\n loop {\n // Return `None` if we have encountered an error from the underlying decoder\n // See: https://github.com/tokio-rs/tokio/issues/3976\n if state.has_errored {\n // preparing has_errored -> paused\n trace!(\"Returning None and setting paused\");\n state.is_readable = false;\n state.has_errored = false;\n return Poll::Ready(None);\n }\n\n // Repeatedly call `decode` or `decode_eof` while the buffer is \"readable\",\n // i.e. it _might_ contain data consumable as a frame or closing frame.\n // Both signal that there is no such data by returning `None`.\n //\n // If `decode` couldn't read a frame and the upstream source has returned eof,\n // `decode_eof` will attempt to decode the remaining bytes as closing frames.\n //\n // If the underlying AsyncRead is resumable, we may continue after an EOF,\n // but must finish emitting all of it's associated `decode_eof` frames.\n // Furthermore, we don't want to emit any `decode_eof` frames on retried\n // reads after an EOF unless we've actually read more data.\n if state.is_readable {\n // pausing or framing\n if state.eof {\n // pausing\n let frame = pinned.codec.decode_eof(&mut state.buffer).map_err(|err| {\n trace!(\"Got an error, going to errored state\");\n state.has_errored = true;\n err\n })?;\n if frame.is_none() {\n state.is_readable = false; // prepare pausing -> paused\n }\n // implicit pausing -> pausing or pausing -> paused\n return Poll::Ready(frame.map(Ok));\n }\n\n // framing\n trace!(\"attempting to decode a frame\");\n\n if let Some(frame) = pinned.codec.decode(&mut state.buffer).map_err(|op| {\n trace!(\"Got an error, going to errored state\");\n state.has_errored = true;\n op\n })? {\n trace!(\"frame decoded from buffer\");\n // implicit framing -> framing\n return Poll::Ready(Some(Ok(frame)));\n }\n\n // framing -> reading\n state.is_readable = false;\n }\n // reading or paused\n // If we can't build a frame yet, try to read more data and try again.\n // Make sure we've got room for at least one byte to read to ensure\n // that we don't get a spurious 0 that looks like EOF.\n state.buffer.reserve(1);\n #[allow(clippy::blocks_in_conditions)]\n let bytect = match poll_read_buf(pinned.inner.as_mut(), cx, &mut state.buffer).map_err(\n |err| {\n trace!(\"Got an error, going to errored state\");\n state.has_errored = true;\n err\n },\n )? {\n Poll::Ready(ct) => ct,\n // implicit reading -> reading or implicit paused -> paused\n Poll::Pending => return Poll::Pending,\n };\n if bytect == 0 {\n if state.eof {\n // We're already at an EOF, and since we've reached this path\n // we're also not readable. This implies that we've already finished\n // our `decode_eof` handling, so we can simply return `None`.\n // implicit paused -> paused\n return Poll::Ready(None);\n }\n // prepare reading -> paused\n state.eof = true;\n } else {\n // prepare paused -> framing or noop reading -> framing\n state.eof = false;\n }\n\n // paused -> framing or reading -> framing or reading -> pausing\n state.is_readable = true;\n }\n }\n}\n\nimpl Sink for FramedImpl\nwhere\n T: AsyncWrite,\n U: Encoder,\n U::Error: From,\n W: BorrowMut,\n{\n type Error = U::Error;\n\n fn poll_ready(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if self.state.borrow().buffer.len() >= self.state.borrow().backpressure_boundary {\n self.as_mut().poll_flush(cx)\n } else {\n Poll::Ready(Ok(()))\n }\n }\n\n fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {\n let pinned = self.project();\n pinned\n .codec\n .encode(item, &mut pinned.state.borrow_mut().buffer)?;\n Ok(())\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n use crate::util::poll_write_buf;\n trace!(\"flushing framed transport\");\n let mut pinned = self.project();\n\n while !pinned.state.borrow_mut().buffer.is_empty() {\n let WriteFrame { buffer, .. } = pinned.state.borrow_mut();\n trace!(remaining = buffer.len(), \"writing;\");\n\n let n = ready!(poll_write_buf(pinned.inner.as_mut(), cx, buffer))?;\n\n if n == 0 {\n return Poll::Ready(Err(io::Error::new(\n io::ErrorKind::WriteZero,\n \"failed to \\\n write frame to transport\",\n )\n .into()));\n }\n }\n\n // Try flushing the underlying IO\n ready!(pinned.inner.poll_flush(cx))?;\n\n trace!(\"framed transport flushed\");\n Poll::Ready(Ok(()))\n }\n\n fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(self.as_mut().poll_flush(cx))?;\n ready!(self.project().inner.poll_shutdown(cx))?;\n\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio-util/src/codec/framed_read.rs", "content": "use crate::codec::framed_impl::{FramedImpl, ReadFrame};\nuse crate::codec::Decoder;\n\nuse futures_core::Stream;\nuse tokio::io::AsyncRead;\n\nuse bytes::BytesMut;\nuse futures_sink::Sink;\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nuse super::FramedParts;\n\npin_project! {\n /// A [`Stream`] of messages decoded from an [`AsyncRead`].\n ///\n /// For examples of how to use `FramedRead` with a codec, see the\n /// examples on the [`codec`] module.\n ///\n /// # Cancellation safety\n /// * [`tokio_stream::StreamExt::next`]: This method is cancel safe. The returned\n /// future only holds onto a reference to the underlying stream, so dropping it will\n /// never lose a value.\n ///\n /// [`Stream`]: futures_core::Stream\n /// [`AsyncRead`]: tokio::io::AsyncRead\n /// [`codec`]: crate::codec\n /// [`tokio_stream::StreamExt::next`]: https://docs.rs/tokio-stream/latest/tokio_stream/trait.StreamExt.html#method.next\n pub struct FramedRead {\n #[pin]\n inner: FramedImpl,\n }\n}\n\n// ===== impl FramedRead =====\n\nimpl FramedRead {\n /// Creates a new `FramedRead` with the given `decoder`.\n pub fn new(inner: T, decoder: D) -> FramedRead {\n FramedRead {\n inner: FramedImpl {\n inner,\n codec: decoder,\n state: Default::default(),\n },\n }\n }\n\n /// Creates a new `FramedRead` with the given `decoder` and a buffer of `capacity`\n /// initial size.\n pub fn with_capacity(inner: T, decoder: D, capacity: usize) -> FramedRead {\n FramedRead {\n inner: FramedImpl {\n inner,\n codec: decoder,\n state: ReadFrame {\n eof: false,\n is_readable: false,\n buffer: BytesMut::with_capacity(capacity),\n has_errored: false,\n },\n },\n }\n }\n\n /// Returns a reference to the underlying I/O stream wrapped by\n /// `FramedRead`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_ref(&self) -> &T {\n &self.inner.inner\n }\n\n /// Returns a mutable reference to the underlying I/O stream wrapped by\n /// `FramedRead`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.inner.inner\n }\n\n /// Returns a pinned mutable reference to the underlying I/O stream wrapped by\n /// `FramedRead`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut T> {\n self.project().inner.project().inner\n }\n\n /// Consumes the `FramedRead`, returning its underlying I/O stream.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn into_inner(self) -> T {\n self.inner.inner\n }\n\n /// Returns a reference to the underlying decoder.\n pub fn decoder(&self) -> &D {\n &self.inner.codec\n }\n\n /// Returns a mutable reference to the underlying decoder.\n pub fn decoder_mut(&mut self) -> &mut D {\n &mut self.inner.codec\n }\n\n /// Maps the decoder `D` to `C`, preserving the read buffer\n /// wrapped by `Framed`.\n pub fn map_decoder(self, map: F) -> FramedRead\n where\n F: FnOnce(D) -> C,\n {\n // This could be potentially simplified once rust-lang/rust#86555 hits stable\n let FramedImpl {\n inner,\n state,\n codec,\n } = self.inner;\n FramedRead {\n inner: FramedImpl {\n inner,\n state,\n codec: map(codec),\n },\n }\n }\n\n /// Returns a mutable reference to the underlying decoder.\n pub fn decoder_pin_mut(self: Pin<&mut Self>) -> &mut D {\n self.project().inner.project().codec\n }\n\n /// Returns a reference to the read buffer.\n pub fn read_buffer(&self) -> &BytesMut {\n &self.inner.state.buffer\n }\n\n /// Returns a mutable reference to the read buffer.\n pub fn read_buffer_mut(&mut self) -> &mut BytesMut {\n &mut self.inner.state.buffer\n }\n\n /// Consumes the `FramedRead`, returning its underlying I/O stream, the buffer\n /// with unprocessed data, and the codec.\n pub fn into_parts(self) -> FramedParts {\n FramedParts {\n io: self.inner.inner,\n codec: self.inner.codec,\n read_buf: self.inner.state.buffer,\n write_buf: BytesMut::new(),\n _priv: (),\n }\n }\n}\n\n// This impl just defers to the underlying FramedImpl\nimpl Stream for FramedRead\nwhere\n T: AsyncRead,\n D: Decoder,\n{\n type Item = Result;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_next(cx)\n }\n}\n\n// This impl just defers to the underlying T: Sink\nimpl Sink for FramedRead\nwhere\n T: Sink,\n{\n type Error = T::Error;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.project().inner.poll_ready(cx)\n }\n\n fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {\n self.project().inner.project().inner.start_send(item)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.project().inner.poll_flush(cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.project().inner.poll_close(cx)\n }\n}\n\nimpl fmt::Debug for FramedRead\nwhere\n T: fmt::Debug,\n D: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"FramedRead\")\n .field(\"inner\", &self.get_ref())\n .field(\"decoder\", &self.decoder())\n .field(\"eof\", &self.inner.state.eof)\n .field(\"is_readable\", &self.inner.state.is_readable)\n .field(\"buffer\", &self.read_buffer())\n .finish()\n }\n}\n" }, { "path": "tokio-util/src/codec/framed_write.rs", "content": "use crate::codec::encoder::Encoder;\nuse crate::codec::framed_impl::{FramedImpl, WriteFrame};\n\nuse futures_core::Stream;\nuse tokio::io::AsyncWrite;\n\nuse bytes::BytesMut;\nuse futures_sink::Sink;\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nuse super::FramedParts;\n\npin_project! {\n /// A [`Sink`] of frames encoded to an `AsyncWrite`.\n ///\n /// For examples of how to use `FramedWrite` with a codec, see the\n /// examples on the [`codec`] module.\n ///\n /// # Cancellation safety\n ///\n /// * [`futures_util::sink::SinkExt::send`]: if send is used as the event in a\n /// `tokio::select!` statement and some other branch completes first, then it is\n /// guaranteed that the message was not sent, but the message itself is lost.\n ///\n /// [`Sink`]: futures_sink::Sink\n /// [`codec`]: crate::codec\n /// [`futures_util::sink::SinkExt::send`]: futures_util::sink::SinkExt::send\n pub struct FramedWrite {\n #[pin]\n inner: FramedImpl,\n }\n}\n\nimpl FramedWrite {\n /// Creates a new `FramedWrite` with the given `encoder`.\n pub fn new(inner: T, encoder: E) -> FramedWrite {\n FramedWrite {\n inner: FramedImpl {\n inner,\n codec: encoder,\n state: WriteFrame::default(),\n },\n }\n }\n\n /// Creates a new `FramedWrite` with the given `encoder` and a buffer of `capacity`\n /// initial size.\n pub fn with_capacity(inner: T, encoder: E, capacity: usize) -> FramedWrite {\n FramedWrite {\n inner: FramedImpl {\n inner,\n codec: encoder,\n state: WriteFrame {\n buffer: BytesMut::with_capacity(capacity),\n backpressure_boundary: capacity,\n },\n },\n }\n }\n\n /// Returns a reference to the underlying I/O stream wrapped by\n /// `FramedWrite`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_ref(&self) -> &T {\n &self.inner.inner\n }\n\n /// Returns a mutable reference to the underlying I/O stream wrapped by\n /// `FramedWrite`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.inner.inner\n }\n\n /// Returns a pinned mutable reference to the underlying I/O stream wrapped by\n /// `FramedWrite`.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut T> {\n self.project().inner.project().inner\n }\n\n /// Consumes the `FramedWrite`, returning its underlying I/O stream.\n ///\n /// Note that care should be taken to not tamper with the underlying stream\n /// of data coming in as it may corrupt the stream of frames otherwise\n /// being worked with.\n pub fn into_inner(self) -> T {\n self.inner.inner\n }\n\n /// Returns a reference to the underlying encoder.\n pub fn encoder(&self) -> &E {\n &self.inner.codec\n }\n\n /// Returns a mutable reference to the underlying encoder.\n pub fn encoder_mut(&mut self) -> &mut E {\n &mut self.inner.codec\n }\n\n /// Maps the encoder `E` to `C`, preserving the write buffer\n /// wrapped by `Framed`.\n pub fn map_encoder(self, map: F) -> FramedWrite\n where\n F: FnOnce(E) -> C,\n {\n // This could be potentially simplified once rust-lang/rust#86555 hits stable\n let FramedImpl {\n inner,\n state,\n codec,\n } = self.inner;\n FramedWrite {\n inner: FramedImpl {\n inner,\n state,\n codec: map(codec),\n },\n }\n }\n\n /// Returns a mutable reference to the underlying encoder.\n pub fn encoder_pin_mut(self: Pin<&mut Self>) -> &mut E {\n self.project().inner.project().codec\n }\n\n /// Returns a reference to the write buffer.\n pub fn write_buffer(&self) -> &BytesMut {\n &self.inner.state.buffer\n }\n\n /// Returns a mutable reference to the write buffer.\n pub fn write_buffer_mut(&mut self) -> &mut BytesMut {\n &mut self.inner.state.buffer\n }\n\n /// Returns backpressure boundary\n pub fn backpressure_boundary(&self) -> usize {\n self.inner.state.backpressure_boundary\n }\n\n /// Updates backpressure boundary\n pub fn set_backpressure_boundary(&mut self, boundary: usize) {\n self.inner.state.backpressure_boundary = boundary;\n }\n\n /// Consumes the `FramedWrite`, returning its underlying I/O stream, the buffer\n /// with unprocessed data, and the codec.\n pub fn into_parts(self) -> FramedParts {\n FramedParts {\n io: self.inner.inner,\n codec: self.inner.codec,\n read_buf: BytesMut::new(),\n write_buf: self.inner.state.buffer,\n _priv: (),\n }\n }\n}\n\n// This impl just defers to the underlying FramedImpl\nimpl Sink for FramedWrite\nwhere\n T: AsyncWrite,\n E: Encoder,\n E::Error: From,\n{\n type Error = E::Error;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_ready(cx)\n }\n\n fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {\n self.project().inner.start_send(item)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_close(cx)\n }\n}\n\n// This impl just defers to the underlying T: Stream\nimpl Stream for FramedWrite\nwhere\n T: Stream,\n{\n type Item = T::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.project().inner.poll_next(cx)\n }\n}\n\nimpl fmt::Debug for FramedWrite\nwhere\n T: fmt::Debug,\n U: fmt::Debug,\n{\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"FramedWrite\")\n .field(\"inner\", &self.get_ref())\n .field(\"encoder\", &self.encoder())\n .field(\"buffer\", &self.inner.state.buffer)\n .finish()\n }\n}\n" }, { "path": "tokio-util/src/codec/length_delimited.rs", "content": "//! Frame a stream of bytes based on a length prefix\n//!\n//! Many protocols delimit their frames by prefacing frame data with a\n//! frame head that specifies the length of the frame. The\n//! `length_delimited` module provides utilities for handling the length\n//! based framing. This allows the consumer to work with entire frames\n//! without having to worry about buffering or other framing logic.\n//!\n//! # Getting started\n//!\n//! If implementing a protocol from scratch, using length delimited framing\n//! is an easy way to get started. [`LengthDelimitedCodec::new()`] will\n//! return a length delimited codec using default configuration values.\n//! This can then be used to construct a framer to adapt a full-duplex\n//! byte stream into a stream of frames.\n//!\n//! ```\n//! use tokio::io::{AsyncRead, AsyncWrite};\n//! use tokio_util::codec::{Framed, LengthDelimitedCodec};\n//!\n//! fn bind_transport(io: T)\n//! -> Framed\n//! {\n//! Framed::new(io, LengthDelimitedCodec::new())\n//! }\n//! # pub fn main() {}\n//! ```\n//!\n//! The returned transport implements `Sink + Stream` for `BytesMut`. It\n//! encodes the frame with a big-endian `u32` header denoting the frame\n//! payload length:\n//!\n//! ```text\n//! +----------+--------------------------------+\n//! | len: u32 | frame payload |\n//! +----------+--------------------------------+\n//! ```\n//!\n//! Specifically, given the following:\n//!\n//! ```\n//! use tokio::io::{AsyncRead, AsyncWrite};\n//! use tokio_util::codec::{Framed, LengthDelimitedCodec};\n//!\n//! use futures::SinkExt;\n//! use bytes::Bytes;\n//!\n//! async fn write_frame(io: T) -> Result<(), Box>\n//! where\n//! T: AsyncRead + AsyncWrite + Unpin,\n//! {\n//! let mut transport = Framed::new(io, LengthDelimitedCodec::new());\n//! let frame = Bytes::from(\"hello world\");\n//!\n//! transport.send(frame).await?;\n//! Ok(())\n//! }\n//! ```\n//!\n//! The encoded frame will look like this:\n//!\n//! ```text\n//! +---- len: u32 ----+---- data ----+\n//! | \\x00\\x00\\x00\\x0b | hello world |\n//! +------------------+--------------+\n//! ```\n//!\n//! # Decoding\n//!\n//! [`FramedRead`] adapts an [`AsyncRead`] into a `Stream` of [`BytesMut`],\n//! such that each yielded [`BytesMut`] value contains the contents of an\n//! entire frame. There are many configuration parameters enabling\n//! [`FramedRead`] to handle a wide range of protocols. Here are some\n//! examples that will cover the various options at a high level.\n//!\n//! ## Example 1\n//!\n//! The following will parse a `u16` length field at offset 0, omitting the\n//! frame head in the yielded `BytesMut`.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\x00\\x0BHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(0) // default value\n//! .length_field_type::()\n//! .length_adjustment(0) // default value\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"Hello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT DECODED\n//! +-- len ---+--- Payload ---+ +--- Payload ---+\n//! | \\x00\\x0B | Hello world | --> | Hello world |\n//! +----------+---------------+ +---------------+\n//! ```\n//!\n//! The value of the length field is 11 (`\\x0B`) which represents the length\n//! of the payload, `hello world`. By default, [`FramedRead`] assumes that\n//! the length field represents the number of bytes that **follows** the\n//! length field. Thus, the entire frame has a length of 13: 2 bytes for the\n//! frame head + 11 bytes for the payload.\n//!\n//! ## Example 2\n//!\n//! The following will parse a `u16` length field at offset 0, including the\n//! frame head in the yielded `BytesMut`.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\x00\\x0BHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(0) // default value\n//! .length_field_type::()\n//! .length_adjustment(2) // Add head size to length\n//! .num_skip(0) // Do NOT skip the head\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"\\x00\\x0BHello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT DECODED\n//! +-- len ---+--- Payload ---+ +-- len ---+--- Payload ---+\n//! | \\x00\\x0B | Hello world | --> | \\x00\\x0B | Hello world |\n//! +----------+---------------+ +----------+---------------+\n//! ```\n//!\n//! This is similar to the first example, the only difference is that the\n//! frame head is **included** in the yielded `BytesMut` value. To achieve\n//! this, we need to add the header size to the length with `length_adjustment`,\n//! and set `num_skip` to `0` to prevent skipping the head.\n//!\n//! ## Example 3\n//!\n//! The following will parse a `u16` length field at offset 0, omitting the\n//! frame head in the yielded `BytesMut`. In this case, the length field\n//! **includes** the frame head length.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\x00\\x0DHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(0) // default value\n//! .length_field_type::()\n//! .length_adjustment(-2) // size of head\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"Hello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT DECODED\n//! +-- len ---+--- Payload ---+ +--- Payload ---+\n//! | \\x00\\x0D | Hello world | --> | Hello world |\n//! +----------+---------------+ +---------------+\n//! ```\n//!\n//! In most cases, the length field represents the length of the payload\n//! only, as shown in the previous examples. However, in some protocols the\n//! length field represents the length of the whole frame, including the\n//! head. In such cases, we specify a negative `length_adjustment` to adjust\n//! the value provided in the frame head to represent the payload length.\n//!\n//! ## Example 4\n//!\n//! The following will parse a 3 byte length field at offset 0 in a 5 byte\n//! frame head, including the frame head in the yielded `BytesMut`.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\x00\\x00\\x0B\\xCA\\xFEHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(0) // default value\n//! .length_field_length(3)\n//! .length_adjustment(3 + 2) // len field and remaining head\n//! .num_skip(0)\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"\\x00\\x00\\x0B\\xCA\\xFEHello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT\n//! +---- len -----+- head -+--- Payload ---+\n//! | \\x00\\x00\\x0B | \\xCAFE | Hello world |\n//! +--------------+--------+---------------+\n//!\n//! DECODED\n//! +---- len -----+- head -+--- Payload ---+\n//! | \\x00\\x00\\x0B | \\xCAFE | Hello world |\n//! +--------------+--------+---------------+\n//! ```\n//!\n//! A more advanced example that shows a case where there is extra frame\n//! head data between the length field and the payload. In such cases, it is\n//! usually desirable to include the frame head as part of the yielded\n//! `BytesMut`. This lets consumers of the length delimited framer to\n//! process the frame head as needed.\n//!\n//! The positive `length_adjustment` value lets `FramedRead` factor in the\n//! additional head into the frame length calculation.\n//!\n//! ## Example 5\n//!\n//! The following will parse a `u16` length field at offset 1 of a 4 byte\n//! frame head. The first byte and the length field will be omitted from the\n//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be\n//! included.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\xCA\\x00\\x0B\\xFEHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(1) // length of hdr1\n//! .length_field_type::()\n//! .length_adjustment(1) // length of hdr2\n//! .num_skip(3) // length of hdr1 + LEN\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"\\xFEHello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT\n//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+\n//! | \\xCA | \\x00\\x0B | \\xFE | Hello world |\n//! +--------+----------+--------+---------------+\n//!\n//! DECODED\n//! +- hdr2 -+--- Payload ---+\n//! | \\xFE | Hello world |\n//! +--------+---------------+\n//! ```\n//!\n//! The length field is situated in the middle of the frame head. In this\n//! case, the first byte in the frame head could be a version or some other\n//! identifier that is not needed for processing. On the other hand, the\n//! second half of the head is needed.\n//!\n//! `length_field_offset` indicates how many bytes to skip before starting\n//! to read the length field. `length_adjustment` is the number of bytes to\n//! skip starting at the end of the length field. In this case, it is the\n//! second half of the head.\n//!\n//! ## Example 6\n//!\n//! The following will parse a `u16` length field at offset 1 of a 4 byte\n//! frame head. The first byte and the length field will be omitted from the\n//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be\n//! included. In this case, the length field **includes** the frame head\n//! length.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\xCA\\x00\\x0F\\xFEHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(1) // length of hdr1\n//! .length_field_type::()\n//! .length_adjustment(-3) // length of hdr1 + LEN, negative\n//! .num_skip(3)\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"\\xFEHello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT\n//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+\n//! | \\xCA | \\x00\\x0F | \\xFE | Hello world |\n//! +--------+----------+--------+---------------+\n//!\n//! DECODED\n//! +- hdr2 -+--- Payload ---+\n//! | \\xFE | Hello world |\n//! +--------+---------------+\n//! ```\n//!\n//! Similar to the example above, the difference is that the length field\n//! represents the length of the entire frame instead of just the payload.\n//! The length of `hdr1` and `len` must be counted in `length_adjustment`.\n//! Note that the length of `hdr2` does **not** need to be explicitly set\n//! anywhere because it already is factored into the total frame length that\n//! is read from the byte stream.\n//!\n//! ## Example 7\n//!\n//! The following will parse a 3 byte length field at offset 0 in a 4 byte\n//! frame head, excluding the 4th byte from the yielded `BytesMut`.\n//!\n//! ```\n//! # use tokio_stream::StreamExt;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! # let io: &[u8] = b\"\\x00\\x00\\x0B\\xFFHello world\";\n//! let mut reader = LengthDelimitedCodec::builder()\n//! .length_field_offset(0) // default value\n//! .length_field_length(3)\n//! .length_adjustment(0) // default value\n//! .num_skip(4) // skip the first 4 bytes\n//! .new_read(io);\n//! # let res = reader.next().await.unwrap().unwrap().to_vec();\n//! # assert_eq!(res, b\"Hello world\");\n//! # }\n//! ```\n//!\n//! The following frame will be decoded as such:\n//!\n//! ```text\n//! INPUT DECODED\n//! +------- len ------+--- Payload ---+ +--- Payload ---+\n//! | \\x00\\x00\\x0B\\xFF | Hello world | => | Hello world |\n//! +------------------+---------------+ +---------------+\n//! ```\n//!\n//! A simple example where there are unused bytes between the length field\n//! and the payload.\n//!\n//! # Encoding\n//!\n//! [`FramedWrite`] adapts an [`AsyncWrite`] into a `Sink` of [`BytesMut`],\n//! such that each submitted [`BytesMut`] is prefaced by a length field.\n//! There are fewer configuration options than [`FramedRead`]. Given\n//! protocols that have more complex frame heads, an encoder should probably\n//! be written by hand using [`Encoder`].\n//!\n//! Here is a simple example, given a `FramedWrite` with the following\n//! configuration:\n//!\n//! ```\n//! # use tokio::io::AsyncWrite;\n//! # use tokio_util::codec::LengthDelimitedCodec;\n//! # fn write_frame(io: T) {\n//! # let _ =\n//! LengthDelimitedCodec::builder()\n//! .length_field_type::()\n//! .new_write(io);\n//! # }\n//! # pub fn main() {}\n//! ```\n//!\n//! A payload of `hello world` will be encoded as:\n//!\n//! ```text\n//! +- len: u16 -+---- data ----+\n//! | \\x00\\x0b | hello world |\n//! +------------+--------------+\n//! ```\n//!\n//! [`LengthDelimitedCodec::new()`]: method@LengthDelimitedCodec::new\n//! [`FramedRead`]: struct@FramedRead\n//! [`FramedWrite`]: struct@FramedWrite\n//! [`AsyncRead`]: trait@tokio::io::AsyncRead\n//! [`AsyncWrite`]: trait@tokio::io::AsyncWrite\n//! [`Encoder`]: trait@Encoder\n//! [`BytesMut`]: bytes::BytesMut\n\nuse crate::codec::{Decoder, Encoder, Framed, FramedRead, FramedWrite};\n\nuse tokio::io::{AsyncRead, AsyncWrite};\n\nuse bytes::{Buf, BufMut, Bytes, BytesMut};\nuse std::error::Error as StdError;\nuse std::io::{self, Cursor};\nuse std::{cmp, fmt, mem};\n\n/// Configure length delimited `LengthDelimitedCodec`s.\n///\n/// `Builder` enables constructing configured length delimited codecs. Note\n/// that not all configuration settings apply to both encoding and decoding. See\n/// the documentation for specific methods for more detail.\n///\n/// Note that the if the value of [`Builder::max_frame_length`] becomes larger than\n/// what can actually fit in [`Builder::length_field_length`], it will be clipped to\n/// the maximum value that can fit.\n#[derive(Debug, Clone, Copy)]\npub struct Builder {\n // Maximum frame length\n max_frame_len: usize,\n\n // Number of bytes representing the field length\n length_field_len: usize,\n\n // Number of bytes in the header before the length field\n length_field_offset: usize,\n\n // Adjust the length specified in the header field by this amount\n length_adjustment: isize,\n\n // Total number of bytes to skip before reading the payload, if not set,\n // `length_field_len + length_field_offset`\n num_skip: Option,\n\n // Length field byte order (little or big endian)\n length_field_is_big_endian: bool,\n}\n\n/// An error when the number of bytes read is more than max frame length.\npub struct LengthDelimitedCodecError {\n _priv: (),\n}\n\n/// A codec for frames delimited by a frame head specifying their lengths.\n///\n/// This allows the consumer to work with entire frames without having to worry\n/// about buffering or other framing logic.\n///\n/// See [module level] documentation for more detail.\n///\n/// [module level]: index.html\n#[derive(Debug, Clone)]\npub struct LengthDelimitedCodec {\n // Configuration values\n builder: Builder,\n\n // Read state\n state: DecodeState,\n}\n\n#[derive(Debug, Clone, Copy)]\nenum DecodeState {\n Head,\n Data(usize),\n}\n\n// ===== impl LengthDelimitedCodec ======\n\nimpl LengthDelimitedCodec {\n /// Creates a new `LengthDelimitedCodec` with the default configuration values.\n pub fn new() -> Self {\n Self {\n builder: Builder::new(),\n state: DecodeState::Head,\n }\n }\n\n /// Creates a new length delimited codec builder with default configuration\n /// values.\n pub fn builder() -> Builder {\n Builder::new()\n }\n\n /// Returns the current max frame setting\n ///\n /// This is the largest size this codec will accept from the wire. Larger\n /// frames will be rejected.\n pub fn max_frame_length(&self) -> usize {\n self.builder.max_frame_len\n }\n\n /// Updates the max frame setting.\n ///\n /// The change takes effect the next time a frame is decoded. In other\n /// words, if a frame is currently in process of being decoded with a frame\n /// size greater than `val` but less than the max frame length in effect\n /// before calling this function, then the frame will be allowed.\n pub fn set_max_frame_length(&mut self, val: usize) {\n self.builder.max_frame_length(val);\n }\n\n fn decode_head(&mut self, src: &mut BytesMut) -> io::Result> {\n let head_len = self.builder.num_head_bytes();\n let field_len = self.builder.length_field_len;\n\n if src.len() < head_len {\n // Not enough data\n return Ok(None);\n }\n\n let n = {\n let mut src = Cursor::new(&mut *src);\n\n // Skip the required bytes\n src.advance(self.builder.length_field_offset);\n\n // match endianness\n let n = if self.builder.length_field_is_big_endian {\n src.get_uint(field_len)\n } else {\n src.get_uint_le(field_len)\n };\n\n if n > self.builder.max_frame_len as u64 {\n return Err(io::Error::new(\n io::ErrorKind::InvalidData,\n LengthDelimitedCodecError { _priv: () },\n ));\n }\n\n // The check above ensures there is no overflow\n let n = n as usize;\n\n // Adjust `n` with bounds checking\n let n = if self.builder.length_adjustment < 0 {\n n.checked_sub(-self.builder.length_adjustment as usize)\n } else {\n n.checked_add(self.builder.length_adjustment as usize)\n };\n\n // Error handling\n match n {\n Some(n) => n,\n None => {\n return Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n \"provided length would overflow after adjustment\",\n ));\n }\n }\n };\n\n src.advance(self.builder.get_num_skip());\n\n // Ensure that the buffer has enough space to read the incoming\n // payload\n src.reserve(n.saturating_sub(src.len()));\n\n Ok(Some(n))\n }\n\n fn decode_data(&self, n: usize, src: &mut BytesMut) -> Option {\n // At this point, the buffer has already had the required capacity\n // reserved. All there is to do is read.\n if src.len() < n {\n return None;\n }\n\n Some(src.split_to(n))\n }\n}\n\nimpl Decoder for LengthDelimitedCodec {\n type Item = BytesMut;\n type Error = io::Error;\n\n fn decode(&mut self, src: &mut BytesMut) -> io::Result> {\n let n = match self.state {\n DecodeState::Head => match self.decode_head(src)? {\n Some(n) => {\n self.state = DecodeState::Data(n);\n n\n }\n None => return Ok(None),\n },\n DecodeState::Data(n) => n,\n };\n\n match self.decode_data(n, src) {\n Some(data) => {\n // Update the decode state\n self.state = DecodeState::Head;\n\n // Make sure the buffer has enough space to read the next head\n src.reserve(self.builder.num_head_bytes().saturating_sub(src.len()));\n\n Ok(Some(data))\n }\n None => Ok(None),\n }\n }\n}\n\nimpl Encoder for LengthDelimitedCodec {\n type Error = io::Error;\n\n fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), io::Error> {\n let n = data.len();\n\n if n > self.builder.max_frame_len {\n return Err(io::Error::new(\n io::ErrorKind::InvalidInput,\n LengthDelimitedCodecError { _priv: () },\n ));\n }\n\n // Adjust `n` with bounds checking\n let n = if self.builder.length_adjustment < 0 {\n n.checked_add(-self.builder.length_adjustment as usize)\n } else {\n n.checked_sub(self.builder.length_adjustment as usize)\n };\n\n let n = n.ok_or_else(|| {\n io::Error::new(\n io::ErrorKind::InvalidInput,\n \"provided length would overflow after adjustment\",\n )\n })?;\n\n // Reserve capacity in the destination buffer to fit the frame and\n // length field (plus adjustment).\n dst.reserve(self.builder.length_field_len + n);\n\n if self.builder.length_field_is_big_endian {\n dst.put_uint(n as u64, self.builder.length_field_len);\n } else {\n dst.put_uint_le(n as u64, self.builder.length_field_len);\n }\n\n // Write the frame to the buffer\n dst.extend_from_slice(&data[..]);\n\n Ok(())\n }\n}\n\nimpl Default for LengthDelimitedCodec {\n fn default() -> Self {\n Self::new()\n }\n}\n\n// ===== impl Builder =====\n\nmod builder {\n /// Types that can be used with `Builder::length_field_type`.\n pub trait LengthFieldType {}\n\n impl LengthFieldType for u8 {}\n impl LengthFieldType for u16 {}\n impl LengthFieldType for u32 {}\n impl LengthFieldType for u64 {}\n\n #[cfg(any(\n target_pointer_width = \"16\",\n target_pointer_width = \"32\",\n target_pointer_width = \"64\",\n ))]\n impl LengthFieldType for usize {}\n}\n\nimpl Builder {\n /// Creates a new length delimited codec builder with default configuration\n /// values.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_offset(0)\n /// .length_field_type::()\n /// .length_adjustment(0)\n /// .num_skip(0)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn new() -> Builder {\n Builder {\n // Default max frame length of 8MB\n max_frame_len: 8 * 1_024 * 1_024,\n\n // Default byte length of 4\n length_field_len: 4,\n\n // Default to the header field being at the start of the header.\n length_field_offset: 0,\n\n length_adjustment: 0,\n\n // Total number of bytes to skip before reading the payload, if not set,\n // `length_field_len + length_field_offset`\n num_skip: None,\n\n // Default to reading the length field in network (big) endian.\n length_field_is_big_endian: true,\n }\n }\n\n /// Read the length field as a big endian integer\n ///\n /// This is the default setting.\n ///\n /// This configuration option applies to both encoding and decoding.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .big_endian()\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn big_endian(&mut self) -> &mut Self {\n self.length_field_is_big_endian = true;\n self\n }\n\n /// Read the length field as a little endian integer\n ///\n /// The default setting is big endian.\n ///\n /// This configuration option applies to both encoding and decoding.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .little_endian()\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn little_endian(&mut self) -> &mut Self {\n self.length_field_is_big_endian = false;\n self\n }\n\n /// Read the length field as a native endian integer\n ///\n /// The default setting is big endian.\n ///\n /// This configuration option applies to both encoding and decoding.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .native_endian()\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn native_endian(&mut self) -> &mut Self {\n if cfg!(target_endian = \"big\") {\n self.big_endian()\n } else {\n self.little_endian()\n }\n }\n\n /// Sets the max frame length in bytes\n ///\n /// This configuration option applies to both encoding and decoding. The\n /// default value is 8MB.\n ///\n /// When decoding, the length field read from the byte stream is checked\n /// against this setting **before** any adjustments are applied. When\n /// encoding, the length of the submitted payload is checked against this\n /// setting.\n ///\n /// When frames exceed the max length, an `io::Error` with the custom value\n /// of the `LengthDelimitedCodecError` type will be returned.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .max_frame_length(8 * 1024 * 1024)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn max_frame_length(&mut self, val: usize) -> &mut Self {\n self.max_frame_len = val;\n self\n }\n\n /// Sets the unsigned integer type used to represent the length field.\n ///\n /// The default type is [`u32`]. The max type is [`u64`] (or [`usize`] on\n /// 64-bit targets).\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_type::()\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n ///\n /// Unlike [`Builder::length_field_length`], this does not fail at runtime\n /// and instead produces a compile error:\n ///\n /// ```compile_fail\n /// # use tokio::io::AsyncRead;\n /// # use tokio_util::codec::LengthDelimitedCodec;\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_type::()\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn length_field_type(&mut self) -> &mut Self {\n self.length_field_length(mem::size_of::())\n }\n\n /// Sets the number of bytes used to represent the length field\n ///\n /// The default value is `4`. The max value is `8`.\n ///\n /// This configuration option applies to both encoding and decoding.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_length(4)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn length_field_length(&mut self, val: usize) -> &mut Self {\n assert!(val > 0 && val <= 8, \"invalid length field length\");\n self.length_field_len = val;\n self\n }\n\n /// Sets the number of bytes in the header before the length field\n ///\n /// This configuration option only applies to decoding.\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_offset(1)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn length_field_offset(&mut self, val: usize) -> &mut Self {\n self.length_field_offset = val;\n self\n }\n\n /// Delta between the payload length specified in the header and the real\n /// payload length\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_adjustment(-2)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn length_adjustment(&mut self, val: isize) -> &mut Self {\n self.length_adjustment = val;\n self\n }\n\n /// Sets the number of bytes to skip before reading the payload\n ///\n /// Default value is `length_field_len + length_field_offset`\n ///\n /// This configuration option only applies to decoding\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .num_skip(4)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn num_skip(&mut self, val: usize) -> &mut Self {\n self.num_skip = Some(val);\n self\n }\n\n /// Create a configured length delimited `LengthDelimitedCodec`\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::codec::LengthDelimitedCodec;\n /// # pub fn main() {\n /// LengthDelimitedCodec::builder()\n /// .length_field_offset(0)\n /// .length_field_type::()\n /// .length_adjustment(0)\n /// .num_skip(0)\n /// .new_codec();\n /// # }\n /// ```\n pub fn new_codec(&self) -> LengthDelimitedCodec {\n let mut builder = *self;\n\n builder.adjust_max_frame_len();\n\n LengthDelimitedCodec {\n builder,\n state: DecodeState::Head,\n }\n }\n\n /// Create a configured length delimited `FramedRead`\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncRead;\n /// use tokio_util::codec::LengthDelimitedCodec;\n ///\n /// # fn bind_read(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_offset(0)\n /// .length_field_type::()\n /// .length_adjustment(0)\n /// .num_skip(0)\n /// .new_read(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn new_read(&self, upstream: T) -> FramedRead\n where\n T: AsyncRead,\n {\n FramedRead::new(upstream, self.new_codec())\n }\n\n /// Create a configured length delimited `FramedWrite`\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::AsyncWrite;\n /// # use tokio_util::codec::LengthDelimitedCodec;\n /// # fn write_frame(io: T) {\n /// LengthDelimitedCodec::builder()\n /// .length_field_type::()\n /// .new_write(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn new_write(&self, inner: T) -> FramedWrite\n where\n T: AsyncWrite,\n {\n FramedWrite::new(inner, self.new_codec())\n }\n\n /// Create a configured length delimited `Framed`\n ///\n /// # Examples\n ///\n /// ```\n /// # use tokio::io::{AsyncRead, AsyncWrite};\n /// # use tokio_util::codec::LengthDelimitedCodec;\n /// # fn write_frame(io: T) {\n /// # let _ =\n /// LengthDelimitedCodec::builder()\n /// .length_field_type::()\n /// .new_framed(io);\n /// # }\n /// # pub fn main() {}\n /// ```\n pub fn new_framed(&self, inner: T) -> Framed\n where\n T: AsyncRead + AsyncWrite,\n {\n Framed::new(inner, self.new_codec())\n }\n\n fn num_head_bytes(&self) -> usize {\n let num = self.length_field_offset + self.length_field_len;\n cmp::max(num, self.num_skip.unwrap_or(0))\n }\n\n fn get_num_skip(&self) -> usize {\n self.num_skip\n .unwrap_or(self.length_field_offset + self.length_field_len)\n }\n\n fn adjust_max_frame_len(&mut self) {\n // Calculate the maximum number that can be represented using `length_field_len` bytes.\n let max_number = match 1u64.checked_shl((8 * self.length_field_len) as u32) {\n Some(shl) => shl - 1,\n None => u64::MAX,\n };\n\n let max_allowed_len = max_number.saturating_add_signed(self.length_adjustment as i64);\n\n if self.max_frame_len as u64 > max_allowed_len {\n self.max_frame_len = usize::try_from(max_allowed_len).unwrap_or(usize::MAX);\n }\n }\n}\n\nimpl Default for Builder {\n fn default() -> Self {\n Self::new()\n }\n}\n\n// ===== impl LengthDelimitedCodecError =====\n\nimpl fmt::Debug for LengthDelimitedCodecError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"LengthDelimitedCodecError\").finish()\n }\n}\n\nimpl fmt::Display for LengthDelimitedCodecError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.write_str(\"frame size too big\")\n }\n}\n\nimpl StdError for LengthDelimitedCodecError {}\n" }, { "path": "tokio-util/src/codec/lines_codec.rs", "content": "use crate::codec::decoder::Decoder;\nuse crate::codec::encoder::Encoder;\n\nuse bytes::{Buf, BufMut, BytesMut};\nuse std::{cmp, fmt, io, str};\n\n/// A simple [`Decoder`] and [`Encoder`] implementation that splits up data into lines.\n///\n/// This uses the `\\n` character as the line ending on all platforms.\n///\n/// [`Decoder`]: crate::codec::Decoder\n/// [`Encoder`]: crate::codec::Encoder\n#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]\npub struct LinesCodec {\n // Stored index of the next index to examine for a `\\n` character.\n // This is used to optimize searching.\n // For example, if `decode` was called with `abc`, it would hold `3`,\n // because that is the next index to examine.\n // The next time `decode` is called with `abcde\\n`, the method will\n // only look at `de\\n` before returning.\n next_index: usize,\n\n /// The maximum length for a given line. If `usize::MAX`, lines will be\n /// read until a `\\n` character is reached.\n max_length: usize,\n\n /// Are we currently discarding the remainder of a line which was over\n /// the length limit?\n is_discarding: bool,\n}\n\nimpl LinesCodec {\n /// Returns a `LinesCodec` for splitting up data into lines.\n ///\n /// # Note\n ///\n /// The returned `LinesCodec` will not have an upper bound on the length\n /// of a buffered line. See the documentation for [`new_with_max_length`]\n /// for information on why this could be a potential security risk.\n ///\n /// [`new_with_max_length`]: crate::codec::LinesCodec::new_with_max_length()\n pub fn new() -> LinesCodec {\n LinesCodec {\n next_index: 0,\n max_length: usize::MAX,\n is_discarding: false,\n }\n }\n\n /// Returns a `LinesCodec` with a maximum line length limit.\n ///\n /// If this is set, calls to `LinesCodec::decode` will return a\n /// [`LinesCodecError`] when a line exceeds the length limit. Subsequent calls\n /// will discard up to `limit` bytes from that line until a newline\n /// character is reached, returning `None` until the line over the limit\n /// has been fully discarded. After that point, calls to `decode` will\n /// function as normal.\n ///\n /// # Note\n ///\n /// Setting a length limit is highly recommended for any `LinesCodec` which\n /// will be exposed to untrusted input. Otherwise, the size of the buffer\n /// that holds the line currently being read is unbounded. An attacker could\n /// exploit this unbounded buffer by sending an unbounded amount of input\n /// without any `\\n` characters, causing unbounded memory consumption.\n ///\n /// [`LinesCodecError`]: crate::codec::LinesCodecError\n pub fn new_with_max_length(max_length: usize) -> Self {\n LinesCodec {\n max_length,\n ..LinesCodec::new()\n }\n }\n\n /// Returns the maximum line length when decoding.\n ///\n /// ```\n /// use std::usize;\n /// use tokio_util::codec::LinesCodec;\n ///\n /// let codec = LinesCodec::new();\n /// assert_eq!(codec.max_length(), usize::MAX);\n /// ```\n /// ```\n /// use tokio_util::codec::LinesCodec;\n ///\n /// let codec = LinesCodec::new_with_max_length(256);\n /// assert_eq!(codec.max_length(), 256);\n /// ```\n pub fn max_length(&self) -> usize {\n self.max_length\n }\n}\n\nfn utf8(buf: &[u8]) -> Result<&str, io::Error> {\n str::from_utf8(buf)\n .map_err(|_| io::Error::new(io::ErrorKind::InvalidData, \"Unable to decode input as UTF8\"))\n}\n\nfn without_carriage_return(s: &[u8]) -> &[u8] {\n if let Some(&b'\\r') = s.last() {\n &s[..s.len() - 1]\n } else {\n s\n }\n}\n\nimpl Decoder for LinesCodec {\n type Item = String;\n type Error = LinesCodecError;\n\n fn decode(&mut self, buf: &mut BytesMut) -> Result, LinesCodecError> {\n loop {\n // Determine how far into the buffer we'll search for a newline. If\n // there's no max_length set, we'll read to the end of the buffer.\n let read_to = cmp::min(self.max_length.saturating_add(1), buf.len());\n\n let newline_offset = buf[self.next_index..read_to]\n .iter()\n .position(|b| *b == b'\\n');\n\n match (self.is_discarding, newline_offset) {\n (true, Some(offset)) => {\n // If we found a newline, discard up to that offset and\n // then stop discarding. On the next iteration, we'll try\n // to read a line normally.\n buf.advance(offset + self.next_index + 1);\n self.is_discarding = false;\n self.next_index = 0;\n }\n (true, None) => {\n // Otherwise, we didn't find a newline, so we'll discard\n // everything we read. On the next iteration, we'll continue\n // discarding up to max_len bytes unless we find a newline.\n buf.advance(read_to);\n self.next_index = 0;\n if buf.is_empty() {\n return Ok(None);\n }\n }\n (false, Some(offset)) => {\n // Found a line!\n let newline_index = offset + self.next_index;\n self.next_index = 0;\n let line = buf.split_to(newline_index + 1);\n let line = &line[..line.len() - 1];\n let line = without_carriage_return(line);\n let line = utf8(line)?;\n return Ok(Some(line.to_string()));\n }\n (false, None) if buf.len() > self.max_length => {\n // Reached the maximum length without finding a\n // newline, return an error and start discarding on the\n // next call.\n self.is_discarding = true;\n return Err(LinesCodecError::MaxLineLengthExceeded);\n }\n (false, None) => {\n // We didn't find a line or reach the length limit, so the next\n // call will resume searching at the current offset.\n self.next_index = read_to;\n return Ok(None);\n }\n }\n }\n }\n\n fn decode_eof(&mut self, buf: &mut BytesMut) -> Result, LinesCodecError> {\n Ok(match self.decode(buf)? {\n Some(frame) => Some(frame),\n None => {\n self.next_index = 0;\n // No terminating newline - return remaining data, if any\n if buf.is_empty() || buf == &b\"\\r\"[..] {\n None\n } else {\n let line = buf.split_to(buf.len());\n let line = without_carriage_return(&line);\n let line = utf8(line)?;\n Some(line.to_string())\n }\n }\n })\n }\n}\n\nimpl Encoder for LinesCodec\nwhere\n T: AsRef,\n{\n type Error = LinesCodecError;\n\n fn encode(&mut self, line: T, buf: &mut BytesMut) -> Result<(), LinesCodecError> {\n let line = line.as_ref();\n buf.reserve(line.len() + 1);\n buf.put(line.as_bytes());\n buf.put_u8(b'\\n');\n Ok(())\n }\n}\n\nimpl Default for LinesCodec {\n fn default() -> Self {\n Self::new()\n }\n}\n\n/// An error occurred while encoding or decoding a line.\n#[derive(Debug)]\npub enum LinesCodecError {\n /// The maximum line length was exceeded.\n MaxLineLengthExceeded,\n /// An IO error occurred.\n Io(io::Error),\n}\n\nimpl fmt::Display for LinesCodecError {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n LinesCodecError::MaxLineLengthExceeded => write!(f, \"max line length exceeded\"),\n LinesCodecError::Io(e) => write!(f, \"{e}\"),\n }\n }\n}\n\nimpl From for LinesCodecError {\n fn from(e: io::Error) -> LinesCodecError {\n LinesCodecError::Io(e)\n }\n}\n\nimpl std::error::Error for LinesCodecError {}\n" }, { "path": "tokio-util/src/codec/mod.rs", "content": "//! Adaptors from `AsyncRead`/`AsyncWrite` to Stream/Sink\n//!\n//! Raw I/O objects work with byte sequences, but higher-level code usually\n//! wants to batch these into meaningful chunks, called \"frames\".\n//!\n//! This module contains adapters to go from streams of bytes, [`AsyncRead`] and\n//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].\n//! Framed streams are also known as transports.\n//!\n//! # Example encoding using `LinesCodec`\n//!\n//! The following example demonstrates how to use a codec such as [`LinesCodec`] to\n//! write framed data. [`FramedWrite`] can be used to achieve this. Data sent to\n//! [`FramedWrite`] are first framed according to a specific codec, and then sent to\n//! an implementor of [`AsyncWrite`].\n//!\n//! ```\n//! use futures::sink::SinkExt;\n//! use tokio_util::codec::LinesCodec;\n//! use tokio_util::codec::FramedWrite;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let buffer = Vec::new();\n//! let messages = vec![\"Hello\", \"World\"];\n//! let encoder = LinesCodec::new();\n//!\n//! // FramedWrite is a sink which means you can send values into it\n//! // asynchronously.\n//! let mut writer = FramedWrite::new(buffer, encoder);\n//!\n//! // To be able to send values into a FramedWrite, you need to bring the\n//! // `SinkExt` trait into scope.\n//! writer.send(messages[0]).await.unwrap();\n//! writer.send(messages[1]).await.unwrap();\n//!\n//! let buffer = writer.get_ref();\n//!\n//! assert_eq!(buffer.as_slice(), \"Hello\\nWorld\\n\".as_bytes());\n//! # }\n//!```\n//!\n//! # Example decoding using `LinesCodec`\n//! The following example demonstrates how to use a codec such as [`LinesCodec`] to\n//! read a stream of framed data. [`FramedRead`] can be used to achieve this. [`FramedRead`]\n//! will keep reading from an [`AsyncRead`] implementor until a whole frame, according to a codec,\n//! can be parsed.\n//!\n//!```\n//! use tokio_stream::StreamExt;\n//! use tokio_util::codec::LinesCodec;\n//! use tokio_util::codec::FramedRead;\n//!\n//! # #[tokio::main(flavor = \"current_thread\")]\n//! # async fn main() {\n//! let message = \"Hello\\nWorld\".as_bytes();\n//! let decoder = LinesCodec::new();\n//!\n//! // FramedRead can be used to read a stream of values that are framed according to\n//! // a codec. FramedRead will read from its input (here `buffer`) until a whole frame\n//! // can be parsed.\n//! let mut reader = FramedRead::new(message, decoder);\n//!\n//! // To read values from a FramedRead, you need to bring the\n//! // `StreamExt` trait into scope.\n//! let frame1 = reader.next().await.unwrap().unwrap();\n//! let frame2 = reader.next().await.unwrap().unwrap();\n//!\n//! assert!(reader.next().await.is_none());\n//! assert_eq!(frame1, \"Hello\");\n//! assert_eq!(frame2, \"World\");\n//! # }\n//! ```\n//!\n//! # The Decoder trait\n//!\n//! A [`Decoder`] is used together with [`FramedRead`] or [`Framed`] to turn an\n//! [`AsyncRead`] into a [`Stream`]. The job of the decoder trait is to specify\n//! how sequences of bytes are turned into a sequence of frames, and to\n//! determine where the boundaries between frames are. The job of the\n//! `FramedRead` is to repeatedly switch between reading more data from the IO\n//! resource, and asking the decoder whether we have received enough data to\n//! decode another frame of data.\n//!\n//! The main method on the `Decoder` trait is the [`decode`] method. This method\n//! takes as argument the data that has been read so far, and when it is called,\n//! it will be in one of the following situations:\n//!\n//! 1. The buffer contains less than a full frame.\n//! 2. The buffer contains exactly a full frame.\n//! 3. The buffer contains more than a full frame.\n//!\n//! In the first situation, the decoder should return `Ok(None)`.\n//!\n//! In the second situation, the decoder should clear the provided buffer and\n//! return `Ok(Some(the_decoded_frame))`.\n//!\n//! In the third situation, the decoder should use a method such as [`split_to`]\n//! or [`advance`] to modify the buffer such that the frame is removed from the\n//! buffer, but any data in the buffer after that frame should still remain in\n//! the buffer. The decoder should also return `Ok(Some(the_decoded_frame))` in\n//! this case.\n//!\n//! Finally the decoder may return an error if the data is invalid in some way.\n//! The decoder should _not_ return an error just because it has yet to receive\n//! a full frame.\n//!\n//! It is guaranteed that, from one call to `decode` to another, the provided\n//! buffer will contain the exact same data as before, except that if more data\n//! has arrived through the IO resource, that data will have been appended to\n//! the buffer. This means that reading frames from a `FramedRead` is\n//! essentially equivalent to the following loop:\n//!\n//! ```no_run\n//! use tokio::io::AsyncReadExt;\n//! # // This uses async_stream to create an example that compiles.\n//! # fn foo() -> impl futures_core::Stream> { async_stream::try_stream! {\n//! # use tokio_util::codec::Decoder;\n//! # let mut decoder = tokio_util::codec::BytesCodec::new();\n//! # let io_resource = &mut &[0u8, 1, 2, 3][..];\n//!\n//! let mut buf = bytes::BytesMut::new();\n//! loop {\n//! // The read_buf call will append to buf rather than overwrite existing data.\n//! let len = io_resource.read_buf(&mut buf).await?;\n//!\n//! if len == 0 {\n//! while let Some(frame) = decoder.decode_eof(&mut buf)? {\n//! yield frame;\n//! }\n//! break;\n//! }\n//!\n//! while let Some(frame) = decoder.decode(&mut buf)? {\n//! yield frame;\n//! }\n//! }\n//! # }}\n//! ```\n//! The example above uses `yield` whenever the `Stream` produces an item.\n//!\n//! ## Example decoder\n//!\n//! As an example, consider a protocol that can be used to send strings where\n//! each frame is a four byte integer that contains the length of the frame,\n//! followed by that many bytes of string data. The decoder fails with an error\n//! if the string data is not valid utf-8 or too long.\n//!\n//! Such a decoder can be written like this:\n//! ```\n//! use tokio_util::codec::Decoder;\n//! use bytes::{BytesMut, Buf};\n//!\n//! struct MyStringDecoder {}\n//!\n//! const MAX: usize = 8 * 1024 * 1024;\n//!\n//! impl Decoder for MyStringDecoder {\n//! type Item = String;\n//! type Error = std::io::Error;\n//!\n//! fn decode(\n//! &mut self,\n//! src: &mut BytesMut\n//! ) -> Result, Self::Error> {\n//! if src.len() < 4 {\n//! // Not enough data to read length marker.\n//! return Ok(None);\n//! }\n//!\n//! // Read length marker.\n//! let mut length_bytes = [0u8; 4];\n//! length_bytes.copy_from_slice(&src[..4]);\n//! let length = u32::from_le_bytes(length_bytes) as usize;\n//!\n//! // Check that the length is not too large to avoid a denial of\n//! // service attack where the server runs out of memory.\n//! if length > MAX {\n//! return Err(std::io::Error::new(\n//! std::io::ErrorKind::InvalidData,\n//! format!(\"Frame of length {} is too large.\", length)\n//! ));\n//! }\n//!\n//! if src.len() < 4 + length {\n//! // The full string has not yet arrived.\n//! //\n//! // We reserve more space in the buffer. This is not strictly\n//! // necessary, but is a good idea performance-wise.\n//! src.reserve(4 + length - src.len());\n//!\n//! // We inform the Framed that we need more bytes to form the next\n//! // frame.\n//! return Ok(None);\n//! }\n//!\n//! // Use advance to modify src such that it no longer contains\n//! // this frame.\n//! let data = src[4..4 + length].to_vec();\n//! src.advance(4 + length);\n//!\n//! // Convert the data to a string, or fail if it is not valid utf-8.\n//! match String::from_utf8(data) {\n//! Ok(string) => Ok(Some(string)),\n//! Err(utf8_error) => {\n//! Err(std::io::Error::new(\n//! std::io::ErrorKind::InvalidData,\n//! utf8_error.utf8_error(),\n//! ))\n//! },\n//! }\n//! }\n//! }\n//! ```\n//!\n//! # The Encoder trait\n//!\n//! An [`Encoder`] is used together with [`FramedWrite`] or [`Framed`] to turn\n//! an [`AsyncWrite`] into a [`Sink`]. The job of the encoder trait is to\n//! specify how frames are turned into a sequences of bytes. The job of the\n//! `FramedWrite` is to take the resulting sequence of bytes and write it to the\n//! IO resource.\n//!\n//! The main method on the `Encoder` trait is the [`encode`] method. This method\n//! takes an item that is being written, and a buffer to write the item to. The\n//! buffer may already contain data, and in this case, the encoder should append\n//! the new frame to the buffer rather than overwrite the existing data.\n//!\n//! It is guaranteed that, from one call to `encode` to another, the provided\n//! buffer will contain the exact same data as before, except that some of the\n//! data may have been removed from the front of the buffer. Writing to a\n//! `FramedWrite` is essentially equivalent to the following loop:\n//!\n//! ```no_run\n//! use tokio::io::AsyncWriteExt;\n//! use bytes::Buf; // for advance\n//! # use tokio_util::codec::Encoder;\n//! # async fn next_frame() -> bytes::Bytes { bytes::Bytes::new() }\n//! # async fn no_more_frames() { }\n//! # #[tokio::main] async fn main() -> std::io::Result<()> {\n//! # let mut io_resource = tokio::io::sink();\n//! # let mut encoder = tokio_util::codec::BytesCodec::new();\n//!\n//! const MAX: usize = 8192;\n//!\n//! let mut buf = bytes::BytesMut::new();\n//! loop {\n//! tokio::select! {\n//! num_written = io_resource.write(&buf), if !buf.is_empty() => {\n//! buf.advance(num_written?);\n//! },\n//! frame = next_frame(), if buf.len() < MAX => {\n//! encoder.encode(frame, &mut buf)?;\n//! },\n//! _ = no_more_frames() => {\n//! io_resource.write_all(&buf).await?;\n//! io_resource.shutdown().await?;\n//! return Ok(());\n//! },\n//! }\n//! }\n//! # }\n//! ```\n//! Here the `next_frame` method corresponds to any frames you write to the\n//! `FramedWrite`. The `no_more_frames` method corresponds to closing the\n//! `FramedWrite` with [`SinkExt::close`].\n//!\n//! ## Example encoder\n//!\n//! As an example, consider a protocol that can be used to send strings where\n//! each frame is a four byte integer that contains the length of the frame,\n//! followed by that many bytes of string data. The encoder will fail if the\n//! string is too long.\n//!\n//! Such an encoder can be written like this:\n//! ```\n//! use tokio_util::codec::Encoder;\n//! use bytes::BytesMut;\n//!\n//! struct MyStringEncoder {}\n//!\n//! const MAX: usize = 8 * 1024 * 1024;\n//!\n//! impl Encoder for MyStringEncoder {\n//! type Error = std::io::Error;\n//!\n//! fn encode(&mut self, item: String, dst: &mut BytesMut) -> Result<(), Self::Error> {\n//! // Don't send a string if it is longer than the other end will\n//! // accept.\n//! if item.len() > MAX {\n//! return Err(std::io::Error::new(\n//! std::io::ErrorKind::InvalidData,\n//! format!(\"Frame of length {} is too large.\", item.len())\n//! ));\n//! }\n//!\n//! // Convert the length into a byte array.\n//! // The cast to u32 cannot overflow due to the length check above.\n//! let len_slice = u32::to_le_bytes(item.len() as u32);\n//!\n//! // Reserve space in the buffer.\n//! dst.reserve(4 + item.len());\n//!\n//! // Write the length and string to the buffer.\n//! dst.extend_from_slice(&len_slice);\n//! dst.extend_from_slice(item.as_bytes());\n//! Ok(())\n//! }\n//! }\n//! ```\n//!\n//! [`AsyncRead`]: tokio::io::AsyncRead\n//! [`AsyncWrite`]: tokio::io::AsyncWrite\n//! [`Stream`]: futures_core::Stream\n//! [`Sink`]: futures_sink::Sink\n//! [`SinkExt`]: https://docs.rs/futures/0.3/futures/sink/trait.SinkExt.html\n//! [`SinkExt::close`]: https://docs.rs/futures/0.3/futures/sink/trait.SinkExt.html#method.close\n//! [`FramedRead`]: struct@crate::codec::FramedRead\n//! [`FramedWrite`]: struct@crate::codec::FramedWrite\n//! [`Framed`]: struct@crate::codec::Framed\n//! [`Decoder`]: trait@crate::codec::Decoder\n//! [`decode`]: fn@crate::codec::Decoder::decode\n//! [`encode`]: fn@crate::codec::Encoder::encode\n//! [`split_to`]: fn@bytes::BytesMut::split_to\n//! [`advance`]: fn@bytes::Buf::advance\n\nmod bytes_codec;\npub use self::bytes_codec::BytesCodec;\n\nmod decoder;\npub use self::decoder::Decoder;\n\nmod encoder;\npub use self::encoder::Encoder;\n\nmod framed_impl;\n#[allow(unused_imports)]\npub(crate) use self::framed_impl::{FramedImpl, RWFrames, ReadFrame, WriteFrame};\n\nmod framed;\npub use self::framed::{Framed, FramedParts};\n\nmod framed_read;\npub use self::framed_read::FramedRead;\n\nmod framed_write;\npub use self::framed_write::FramedWrite;\n\npub mod length_delimited;\npub use self::length_delimited::{LengthDelimitedCodec, LengthDelimitedCodecError};\n\nmod lines_codec;\npub use self::lines_codec::{LinesCodec, LinesCodecError};\n\nmod any_delimiter_codec;\npub use self::any_delimiter_codec::{AnyDelimiterCodec, AnyDelimiterCodecError};\n" }, { "path": "tokio-util/src/compat.rs", "content": "//! Compatibility between the `tokio::io` and `futures-io` versions of the\n//! `AsyncRead` and `AsyncWrite` traits.\n//!\n//! ## Bridging Tokio and Futures I/O with `compat()`\n//!\n//! The [`compat()`] function provides a compatibility layer that allows types implementing\n//! [`tokio::io::AsyncRead`] or [`tokio::io::AsyncWrite`] to be used as their\n//! [`futures::io::AsyncRead`] or [`futures::io::AsyncWrite`] counterparts — and vice versa.\n//!\n//! This is especially useful when working with libraries that expect I/O types from one ecosystem\n//! (usually `futures`) but you are using types from the other (usually `tokio`).\n//!\n//! ## Compatibility Overview\n//!\n//! | Inner Type Implements... | `Compat` Implements... |\n//! |-----------------------------|-----------------------------|\n//! | [`tokio::io::AsyncRead`] | [`futures::io::AsyncRead`] |\n//! | [`futures::io::AsyncRead`] | [`tokio::io::AsyncRead`] |\n//! | [`tokio::io::AsyncWrite`] | [`futures::io::AsyncWrite`] |\n//! | [`futures::io::AsyncWrite`] | [`tokio::io::AsyncWrite`] |\n//!\n//! ## Feature Flag\n//!\n//! This functionality is available through the `compat` feature flag:\n//!\n//! ```toml\n//! tokio-util = { version = \"...\", features = [\"compat\"] }\n//! ```\n//!\n//! ## Example 1: Tokio -> Futures (`AsyncRead`)\n//!\n//! This example demonstrates sending data over a [`tokio::net::TcpStream`] and using\n//! [`futures::io::AsyncReadExt::read`] from the `futures` crate to read it after adapting the\n//! stream via [`compat()`].\n//!\n//! ```no_run\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use tokio::net::{TcpListener, TcpStream};\n//! use tokio::io::AsyncWriteExt;\n//! use tokio_util::compat::TokioAsyncReadCompatExt;\n//! use futures::io::AsyncReadExt;\n//!\n//! #[tokio::main]\n//! async fn main() -> std::io::Result<()> {\n//! let listener = TcpListener::bind(\"127.0.0.1:8081\").await?;\n//!\n//! tokio::spawn(async {\n//! let mut client = TcpStream::connect(\"127.0.0.1:8081\").await.unwrap();\n//! client.write_all(b\"Hello World\").await.unwrap();\n//! });\n//!\n//! let (stream, _) = listener.accept().await?;\n//!\n//! // Adapt `tokio::TcpStream` to be used with `futures::io::AsyncReadExt`\n//! let mut compat_stream = stream.compat();\n//! let mut buffer = [0; 20];\n//! let n = compat_stream.read(&mut buffer).await?;\n//! println!(\"Received: {}\", String::from_utf8_lossy(&buffer[..n]));\n//!\n//! Ok(())\n//! }\n//! # }\n//! ```\n//!\n//! ## Example 2: Futures -> Tokio (`AsyncRead`)\n//!\n//! The reverse is also possible: you can take a [`futures::io::AsyncRead`] (e.g. a cursor) and\n//! adapt it to be used with [`tokio::io::AsyncReadExt::read_to_end`]\n//!\n//! ```\n//! # #[cfg(not(target_family = \"wasm\"))]\n//! # {\n//! use futures::io::Cursor;\n//! use tokio_util::compat::FuturesAsyncReadCompatExt;\n//! use tokio::io::AsyncReadExt;\n//!\n//! fn main() {\n//! let future = async {\n//! let reader = Cursor::new(b\"Hello from futures\");\n//! let mut compat_reader = reader.compat();\n//! let mut buf = Vec::new();\n//! compat_reader.read_to_end(&mut buf).await.unwrap();\n//! assert_eq!(&buf, b\"Hello from futures\");\n//! };\n//!\n//! // Run the future inside a Tokio runtime\n//! tokio::runtime::Runtime::new().unwrap().block_on(future);\n//! }\n//! # }\n//! ```\n//!\n//! ## Common Use Cases\n//!\n//! - Using `tokio` sockets with `async-tungstenite`, `async-compression`, or `futures-rs`-based\n//! libraries.\n//! - Bridging I/O interfaces between mixed-ecosystem libraries.\n//! - Avoiding rewrites or duplication of I/O code in async environments.\n//!\n//! ## See Also\n//!\n//! - [`Compat`] type\n//! - [`TokioAsyncReadCompatExt`]\n//! - [`FuturesAsyncReadCompatExt`]\n//! - [`tokio::io`]\n//! - [`futures::io`]\n//!\n//! [`futures::io`]: https://docs.rs/futures/latest/futures/io/\n//! [`futures::io::AsyncRead`]: https://docs.rs/futures/latest/futures/io/trait.AsyncRead.html\n//! [`futures::io::AsyncWrite`]: https://docs.rs/futures/latest/futures/io/trait.AsyncWrite.html\n//! [`futures::io::AsyncReadExt::read`]: https://docs.rs/futures/latest/futures/io/trait.AsyncReadExt.html#method.read\n//! [`compat()`]: TokioAsyncReadCompatExt::compat\n\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\npin_project! {\n /// A compatibility layer that allows conversion between the\n /// `tokio::io` and `futures-io` `AsyncRead` and `AsyncWrite` traits.\n #[derive(Copy, Clone, Debug)]\n pub struct Compat {\n #[pin]\n inner: T,\n seek_pos: Option,\n }\n}\n\n/// Extension trait that allows converting a type implementing\n/// `futures_io::AsyncRead` to implement `tokio::io::AsyncRead`.\npub trait FuturesAsyncReadCompatExt: futures_io::AsyncRead {\n /// Wraps `self` with a compatibility layer that implements\n /// `tokio_io::AsyncRead`.\n fn compat(self) -> Compat\n where\n Self: Sized,\n {\n Compat::new(self)\n }\n}\n\nimpl FuturesAsyncReadCompatExt for T {}\n\n/// Extension trait that allows converting a type implementing\n/// `futures_io::AsyncWrite` to implement `tokio::io::AsyncWrite`.\npub trait FuturesAsyncWriteCompatExt: futures_io::AsyncWrite {\n /// Wraps `self` with a compatibility layer that implements\n /// `tokio::io::AsyncWrite`.\n fn compat_write(self) -> Compat\n where\n Self: Sized,\n {\n Compat::new(self)\n }\n}\n\nimpl FuturesAsyncWriteCompatExt for T {}\n\n/// Extension trait that allows converting a type implementing\n/// `tokio::io::AsyncRead` to implement `futures_io::AsyncRead`.\npub trait TokioAsyncReadCompatExt: tokio::io::AsyncRead {\n /// Wraps `self` with a compatibility layer that implements\n /// `futures_io::AsyncRead`.\n fn compat(self) -> Compat\n where\n Self: Sized,\n {\n Compat::new(self)\n }\n}\n\nimpl TokioAsyncReadCompatExt for T {}\n\n/// Extension trait that allows converting a type implementing\n/// `tokio::io::AsyncWrite` to implement `futures_io::AsyncWrite`.\npub trait TokioAsyncWriteCompatExt: tokio::io::AsyncWrite {\n /// Wraps `self` with a compatibility layer that implements\n /// `futures_io::AsyncWrite`.\n fn compat_write(self) -> Compat\n where\n Self: Sized,\n {\n Compat::new(self)\n }\n}\n\nimpl TokioAsyncWriteCompatExt for T {}\n\n// === impl Compat ===\n\nimpl Compat {\n fn new(inner: T) -> Self {\n Self {\n inner,\n seek_pos: None,\n }\n }\n\n /// Get a reference to the `Future`, `Stream`, `AsyncRead`, or `AsyncWrite` object\n /// contained within.\n pub fn get_ref(&self) -> &T {\n &self.inner\n }\n\n /// Get a mutable reference to the `Future`, `Stream`, `AsyncRead`, or `AsyncWrite` object\n /// contained within.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.inner\n }\n\n /// Returns the wrapped item.\n pub fn into_inner(self) -> T {\n self.inner\n }\n}\n\nimpl tokio::io::AsyncRead for Compat\nwhere\n T: futures_io::AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut tokio::io::ReadBuf<'_>,\n ) -> Poll> {\n // We can't trust the inner type to not peak at the bytes,\n // so we must defensively initialize the buffer.\n let slice = buf.initialize_unfilled();\n let n = ready!(futures_io::AsyncRead::poll_read(\n self.project().inner,\n cx,\n slice\n ))?;\n buf.advance(n);\n Poll::Ready(Ok(()))\n }\n}\n\nimpl futures_io::AsyncRead for Compat\nwhere\n T: tokio::io::AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n slice: &mut [u8],\n ) -> Poll> {\n let mut buf = tokio::io::ReadBuf::new(slice);\n ready!(tokio::io::AsyncRead::poll_read(\n self.project().inner,\n cx,\n &mut buf\n ))?;\n Poll::Ready(Ok(buf.filled().len()))\n }\n}\n\nimpl tokio::io::AsyncBufRead for Compat\nwhere\n T: futures_io::AsyncBufRead,\n{\n fn poll_fill_buf<'a>(\n self: Pin<&'a mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n futures_io::AsyncBufRead::poll_fill_buf(self.project().inner, cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n futures_io::AsyncBufRead::consume(self.project().inner, amt)\n }\n}\n\nimpl futures_io::AsyncBufRead for Compat\nwhere\n T: tokio::io::AsyncBufRead,\n{\n fn poll_fill_buf<'a>(\n self: Pin<&'a mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n tokio::io::AsyncBufRead::poll_fill_buf(self.project().inner, cx)\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n tokio::io::AsyncBufRead::consume(self.project().inner, amt)\n }\n}\n\nimpl tokio::io::AsyncWrite for Compat\nwhere\n T: futures_io::AsyncWrite,\n{\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n futures_io::AsyncWrite::poll_write(self.project().inner, cx, buf)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n futures_io::AsyncWrite::poll_flush(self.project().inner, cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n futures_io::AsyncWrite::poll_close(self.project().inner, cx)\n }\n}\n\nimpl futures_io::AsyncWrite for Compat\nwhere\n T: tokio::io::AsyncWrite,\n{\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n tokio::io::AsyncWrite::poll_write(self.project().inner, cx, buf)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n tokio::io::AsyncWrite::poll_flush(self.project().inner, cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n tokio::io::AsyncWrite::poll_shutdown(self.project().inner, cx)\n }\n}\n\nimpl futures_io::AsyncSeek for Compat {\n fn poll_seek(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n pos: io::SeekFrom,\n ) -> Poll> {\n if self.seek_pos != Some(pos) {\n // Ensure previous seeks have finished before starting a new one\n ready!(self.as_mut().project().inner.poll_complete(cx))?;\n self.as_mut().project().inner.start_seek(pos)?;\n *self.as_mut().project().seek_pos = Some(pos);\n }\n let res = ready!(self.as_mut().project().inner.poll_complete(cx));\n *self.as_mut().project().seek_pos = None;\n Poll::Ready(res)\n }\n}\n\nimpl tokio::io::AsyncSeek for Compat {\n fn start_seek(mut self: Pin<&mut Self>, pos: io::SeekFrom) -> io::Result<()> {\n *self.as_mut().project().seek_pos = Some(pos);\n Ok(())\n }\n\n fn poll_complete(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let pos = match self.seek_pos {\n None => {\n // tokio 1.x AsyncSeek recommends calling poll_complete before start_seek.\n // We don't have to guarantee that the value returned by\n // poll_complete called without start_seek is correct,\n // so we'll return 0.\n return Poll::Ready(Ok(0));\n }\n Some(pos) => pos,\n };\n let res = ready!(self.as_mut().project().inner.poll_seek(cx, pos));\n *self.as_mut().project().seek_pos = None;\n Poll::Ready(res)\n }\n}\n\n#[cfg(unix)]\nimpl std::os::unix::io::AsRawFd for Compat {\n fn as_raw_fd(&self) -> std::os::unix::io::RawFd {\n self.inner.as_raw_fd()\n }\n}\n\n#[cfg(windows)]\nimpl std::os::windows::io::AsRawHandle for Compat {\n fn as_raw_handle(&self) -> std::os::windows::io::RawHandle {\n self.inner.as_raw_handle()\n }\n}\n" }, { "path": "tokio-util/src/context.rs", "content": "//! Tokio context aware futures utilities.\n//!\n//! This module includes utilities around integrating tokio with other runtimes\n//! by allowing the context to be attached to futures. This allows spawning\n//! futures on other executors while still using tokio to drive them. This\n//! can be useful if you need to use a tokio based library in an executor/runtime\n//! that does not provide a tokio context.\n\nuse pin_project_lite::pin_project;\nuse std::{\n future::Future,\n pin::Pin,\n task::{Context, Poll},\n};\nuse tokio::runtime::{Handle, Runtime};\n\npin_project! {\n /// `TokioContext` allows running futures that must be inside Tokio's\n /// context on a non-Tokio runtime.\n ///\n /// It contains a [`Handle`] to the runtime. A handle to the runtime can be\n /// obtain by calling the [`Runtime::handle()`] method.\n ///\n /// Note that the `TokioContext` wrapper only works if the `Runtime` it is\n /// connected to has not yet been destroyed. You must keep the `Runtime`\n /// alive until the future has finished executing.\n ///\n /// **Warning:** If `TokioContext` is used together with a [current thread]\n /// runtime, that runtime must be inside a call to `block_on` for the\n /// wrapped future to work. For this reason, it is recommended to use a\n /// [multi thread] runtime, even if you configure it to only spawn one\n /// worker thread.\n ///\n /// # Examples\n ///\n /// This example creates two runtimes, but only [enables time] on one of\n /// them. It then uses the context of the runtime with the timer enabled to\n /// execute a [`sleep`] future on the runtime with timing disabled.\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::time::{sleep, Duration};\n /// use tokio_util::context::RuntimeExt;\n ///\n /// // This runtime has timers enabled.\n /// let rt = tokio::runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .build()\n /// .unwrap();\n ///\n /// // This runtime has timers disabled.\n /// let rt2 = tokio::runtime::Builder::new_multi_thread()\n /// .build()\n /// .unwrap();\n ///\n /// // Wrap the sleep future in the context of rt.\n /// let fut = rt.wrap(async { sleep(Duration::from_millis(2)).await });\n ///\n /// // Execute the future on rt2.\n /// rt2.block_on(fut);\n /// # }\n /// ```\n ///\n /// [`Handle`]: struct@tokio::runtime::Handle\n /// [`Runtime::handle()`]: fn@tokio::runtime::Runtime::handle\n /// [`RuntimeExt`]: trait@crate::context::RuntimeExt\n /// [`new_static`]: fn@Self::new_static\n /// [`sleep`]: fn@tokio::time::sleep\n /// [current thread]: fn@tokio::runtime::Builder::new_current_thread\n /// [enables time]: fn@tokio::runtime::Builder::enable_time\n /// [multi thread]: fn@tokio::runtime::Builder::new_multi_thread\n pub struct TokioContext {\n #[pin]\n inner: F,\n handle: Handle,\n }\n}\n\nimpl TokioContext {\n /// Associate the provided future with the context of the runtime behind\n /// the provided `Handle`.\n ///\n /// This constructor uses a `'static` lifetime to opt-out of checking that\n /// the runtime still exists.\n ///\n /// # Examples\n ///\n /// This is the same as the example above, but uses the `new` constructor\n /// rather than [`RuntimeExt::wrap`].\n ///\n /// [`RuntimeExt::wrap`]: fn@RuntimeExt::wrap\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::time::{sleep, Duration};\n /// use tokio_util::context::TokioContext;\n ///\n /// // This runtime has timers enabled.\n /// let rt = tokio::runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .build()\n /// .unwrap();\n ///\n /// // This runtime has timers disabled.\n /// let rt2 = tokio::runtime::Builder::new_multi_thread()\n /// .build()\n /// .unwrap();\n ///\n /// let fut = TokioContext::new(\n /// async { sleep(Duration::from_millis(2)).await },\n /// rt.handle().clone(),\n /// );\n ///\n /// // Execute the future on rt2.\n /// rt2.block_on(fut);\n /// # }\n /// ```\n pub fn new(future: F, handle: Handle) -> TokioContext {\n TokioContext {\n inner: future,\n handle,\n }\n }\n\n /// Obtain a reference to the handle inside this `TokioContext`.\n pub fn handle(&self) -> &Handle {\n &self.handle\n }\n\n /// Remove the association between the Tokio runtime and the wrapped future.\n pub fn into_inner(self) -> F {\n self.inner\n }\n}\n\nimpl Future for TokioContext {\n type Output = F::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let me = self.project();\n let handle = me.handle;\n let fut = me.inner;\n\n let _enter = handle.enter();\n fut.poll(cx)\n }\n}\n\n/// Extension trait that simplifies bundling a `Handle` with a `Future`.\npub trait RuntimeExt {\n /// Create a [`TokioContext`] that wraps the provided future and runs it in\n /// this runtime's context.\n ///\n /// # Examples\n ///\n /// This example creates two runtimes, but only [enables time] on one of\n /// them. It then uses the context of the runtime with the timer enabled to\n /// execute a [`sleep`] future on the runtime with timing disabled.\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio::time::{sleep, Duration};\n /// use tokio_util::context::RuntimeExt;\n ///\n /// // This runtime has timers enabled.\n /// let rt = tokio::runtime::Builder::new_multi_thread()\n /// .enable_all()\n /// .build()\n /// .unwrap();\n ///\n /// // This runtime has timers disabled.\n /// let rt2 = tokio::runtime::Builder::new_multi_thread()\n /// .build()\n /// .unwrap();\n ///\n /// // Wrap the sleep future in the context of rt.\n /// let fut = rt.wrap(async { sleep(Duration::from_millis(2)).await });\n ///\n /// // Execute the future on rt2.\n /// rt2.block_on(fut);\n /// # }\n /// ```\n ///\n /// [`TokioContext`]: struct@crate::context::TokioContext\n /// [`sleep`]: fn@tokio::time::sleep\n /// [enables time]: fn@tokio::runtime::Builder::enable_time\n fn wrap(&self, fut: F) -> TokioContext;\n}\n\nimpl RuntimeExt for Runtime {\n fn wrap(&self, fut: F) -> TokioContext {\n TokioContext {\n inner: fut,\n handle: self.handle().clone(),\n }\n }\n}\n" }, { "path": "tokio-util/src/either.rs", "content": "//! Module defining an Either type.\nuse std::{\n future::Future,\n io::SeekFrom,\n pin::Pin,\n task::{Context, Poll},\n};\nuse tokio::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf, Result};\n\n/// Combines two different futures, streams, or sinks having the same associated types into a single type.\n///\n/// This type implements common asynchronous traits such as [`Future`] and those in Tokio.\n///\n/// [`Future`]: std::future::Future\n///\n/// # Example\n///\n/// The following code will not work:\n///\n/// ```compile_fail\n/// # fn some_condition() -> bool { true }\n/// # async fn some_async_function() -> u32 { 10 }\n/// # async fn other_async_function() -> u32 { 20 }\n/// #[tokio::main]\n/// async fn main() {\n/// let result = if some_condition() {\n/// some_async_function()\n/// } else {\n/// other_async_function() // <- Will print: \"`if` and `else` have incompatible types\"\n/// };\n///\n/// println!(\"Result is {}\", result.await);\n/// }\n/// ```\n///\n// This is because although the output types for both futures is the same, the exact future\n// types are different, but the compiler must be able to choose a single type for the\n// `result` variable.\n///\n/// When the output type is the same, we can wrap each future in `Either` to avoid the\n/// issue:\n///\n/// ```\n/// use tokio_util::either::Either;\n/// # fn some_condition() -> bool { true }\n/// # async fn some_async_function() -> u32 { 10 }\n/// # async fn other_async_function() -> u32 { 20 }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let result = if some_condition() {\n/// Either::Left(some_async_function())\n/// } else {\n/// Either::Right(other_async_function())\n/// };\n///\n/// let value = result.await;\n/// println!(\"Result is {}\", value);\n/// # assert_eq!(value, 10);\n/// # }\n/// ```\n#[allow(missing_docs)] // Doc-comments for variants in this particular case don't make much sense.\n#[derive(Debug, Clone)]\npub enum Either {\n Left(L),\n Right(R),\n}\n\n/// A small helper macro which reduces amount of boilerplate in the actual trait method implementation.\n/// It takes an invocation of method as an argument (e.g. `self.poll(cx)`), and redirects it to either\n/// enum variant held in `self`.\nmacro_rules! delegate_call {\n ($self:ident.$method:ident($($args:ident),+)) => {\n unsafe {\n match $self.get_unchecked_mut() {\n Self::Left(l) => Pin::new_unchecked(l).$method($($args),+),\n Self::Right(r) => Pin::new_unchecked(r).$method($($args),+),\n }\n }\n }\n}\n\nimpl Future for Either\nwhere\n L: Future,\n R: Future,\n{\n type Output = O;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n delegate_call!(self.poll(cx))\n }\n}\n\nimpl AsyncRead for Either\nwhere\n L: AsyncRead,\n R: AsyncRead,\n{\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n delegate_call!(self.poll_read(cx, buf))\n }\n}\n\nimpl AsyncBufRead for Either\nwhere\n L: AsyncBufRead,\n R: AsyncBufRead,\n{\n fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n delegate_call!(self.poll_fill_buf(cx))\n }\n\n fn consume(self: Pin<&mut Self>, amt: usize) {\n delegate_call!(self.consume(amt));\n }\n}\n\nimpl AsyncSeek for Either\nwhere\n L: AsyncSeek,\n R: AsyncSeek,\n{\n fn start_seek(self: Pin<&mut Self>, position: SeekFrom) -> Result<()> {\n delegate_call!(self.start_seek(position))\n }\n\n fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n delegate_call!(self.poll_complete(cx))\n }\n}\n\nimpl AsyncWrite for Either\nwhere\n L: AsyncWrite,\n R: AsyncWrite,\n{\n fn poll_write(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll> {\n delegate_call!(self.poll_write(cx, buf))\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n delegate_call!(self.poll_flush(cx))\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n delegate_call!(self.poll_shutdown(cx))\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[std::io::IoSlice<'_>],\n ) -> Poll> {\n delegate_call!(self.poll_write_vectored(cx, bufs))\n }\n\n fn is_write_vectored(&self) -> bool {\n match self {\n Self::Left(l) => l.is_write_vectored(),\n Self::Right(r) => r.is_write_vectored(),\n }\n }\n}\n\nimpl futures_core::stream::Stream for Either\nwhere\n L: futures_core::stream::Stream,\n R: futures_core::stream::Stream,\n{\n type Item = L::Item;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n delegate_call!(self.poll_next(cx))\n }\n}\n\nimpl futures_sink::Sink for Either\nwhere\n L: futures_sink::Sink,\n R: futures_sink::Sink,\n{\n type Error = Error;\n\n fn poll_ready(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n delegate_call!(self.poll_ready(cx))\n }\n\n fn start_send(self: Pin<&mut Self>, item: Item) -> std::result::Result<(), Self::Error> {\n delegate_call!(self.start_send(item))\n }\n\n fn poll_flush(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n delegate_call!(self.poll_flush(cx))\n }\n\n fn poll_close(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n delegate_call!(self.poll_close(cx))\n }\n}\n\n#[cfg(all(test, not(loom)))]\nmod tests {\n use super::*;\n use tokio::io::{repeat, AsyncReadExt, Repeat};\n use tokio_stream::{once, Once, StreamExt};\n\n #[tokio::test]\n async fn either_is_stream() {\n let mut either: Either, Once> = Either::Left(once(1));\n\n assert_eq!(Some(1u32), either.next().await);\n }\n\n #[tokio::test]\n async fn either_is_async_read() {\n let mut buffer = [0; 3];\n let mut either: Either = Either::Right(repeat(0b101));\n\n either.read_exact(&mut buffer).await.unwrap();\n assert_eq!(buffer, [0b101, 0b101, 0b101]);\n }\n}\n" }, { "path": "tokio-util/src/future/with_cancellation_token.rs", "content": "use std::{\n future::Future,\n pin::Pin,\n task::{Context, Poll},\n};\n\nuse pin_project_lite::pin_project;\n\nuse crate::sync::{CancellationToken, RunUntilCancelledFuture, RunUntilCancelledFutureOwned};\n\npin_project! {\n /// A [`Future`] that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled or a given [`Future`] gets resolved.\n ///\n /// This future is immediately resolved if the corresponding [`CancellationToken`]\n /// is already cancelled, otherwise, in case of concurrent completion and\n /// cancellation, this is biased towards the future completion.\n #[must_use = \"futures do nothing unless polled\"]\n pub struct WithCancellationTokenFuture<'a, F: Future> {\n #[pin]\n run_until_cancelled: Option>\n }\n}\n\nimpl<'a, F: Future> WithCancellationTokenFuture<'a, F> {\n pub(crate) fn new(cancellation_token: &'a CancellationToken, future: F) -> Self {\n Self {\n run_until_cancelled: (!cancellation_token.is_cancelled())\n .then(|| RunUntilCancelledFuture::new(cancellation_token, future)),\n }\n }\n}\n\nimpl<'a, F: Future> Future for WithCancellationTokenFuture<'a, F> {\n type Output = Option;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n match this.run_until_cancelled.as_pin_mut() {\n Some(fut) => fut.poll(cx),\n None => Poll::Ready(None),\n }\n }\n}\n\npin_project! {\n /// A [`Future`] that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled or a given [`Future`] gets resolved.\n ///\n /// This future is immediately resolved if the corresponding [`CancellationToken`]\n /// is already cancelled, otherwise, in case of concurrent completion and\n /// cancellation, this is biased towards the future completion.\n #[must_use = \"futures do nothing unless polled\"]\n pub struct WithCancellationTokenFutureOwned {\n #[pin]\n run_until_cancelled: Option>\n }\n}\n\nimpl WithCancellationTokenFutureOwned {\n pub(crate) fn new(cancellation_token: CancellationToken, future: F) -> Self {\n Self {\n run_until_cancelled: (!cancellation_token.is_cancelled())\n .then(|| RunUntilCancelledFutureOwned::new(cancellation_token, future)),\n }\n }\n}\n\nimpl Future for WithCancellationTokenFutureOwned {\n type Output = Option;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n match this.run_until_cancelled.as_pin_mut() {\n Some(fut) => fut.poll(cx),\n None => Poll::Ready(None),\n }\n }\n}\n" }, { "path": "tokio-util/src/future.rs", "content": "//! An extension trait for Futures that provides a variety of convenient adapters.\n\nmod with_cancellation_token;\nuse with_cancellation_token::{WithCancellationTokenFuture, WithCancellationTokenFutureOwned};\n\nuse std::future::Future;\n\nuse crate::sync::CancellationToken;\n\n/// A trait which contains a variety of convenient adapters and utilities for `Future`s.\npub trait FutureExt: Future {\n cfg_time! {\n /// A wrapper around [`tokio::time::timeout`], with the advantage that it is easier to write\n /// fluent call chains.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::{sync::oneshot, time::Duration};\n /// use tokio_util::future::FutureExt;\n ///\n /// # async fn dox() {\n /// let (_tx, rx) = oneshot::channel::<()>();\n ///\n /// let res = rx.timeout(Duration::from_millis(10)).await;\n /// assert!(res.is_err());\n /// # }\n /// ```\n #[track_caller]\n fn timeout(self, timeout: std::time::Duration) -> tokio::time::Timeout\n where\n Self: Sized,\n {\n tokio::time::timeout(timeout, self)\n }\n\n /// A wrapper around [`tokio::time::timeout_at`], with the advantage that it is easier to write\n /// fluent call chains.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::{sync::oneshot, time::{Duration, Instant}};\n /// use tokio_util::future::FutureExt;\n ///\n /// # async fn dox() {\n /// let (_tx, rx) = oneshot::channel::<()>();\n /// let deadline = Instant::now() + Duration::from_millis(10);\n ///\n /// let res = rx.timeout_at(deadline).await;\n /// assert!(res.is_err());\n /// # }\n /// ```\n fn timeout_at(self, deadline: tokio::time::Instant) -> tokio::time::Timeout\n where\n Self: Sized,\n {\n tokio::time::timeout_at(deadline, self)\n }\n }\n\n /// Similar to [`CancellationToken::run_until_cancelled`],\n /// but with the advantage that it is easier to write fluent call chains.\n ///\n /// # Fairness\n ///\n /// Calling this on an already-cancelled token directly returns `None`.\n /// For all subsequent polls, in case of concurrent completion and\n /// cancellation, this is biased towards the `self` future completion.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::sync::oneshot;\n /// use tokio_util::future::FutureExt;\n /// use tokio_util::sync::CancellationToken;\n ///\n /// # async fn dox() {\n /// let (_tx, rx) = oneshot::channel::<()>();\n /// let token = CancellationToken::new();\n /// let token_clone = token.clone();\n /// tokio::spawn(async move {\n /// tokio::time::sleep(std::time::Duration::from_millis(10)).await;\n /// token.cancel();\n /// });\n /// assert!(rx.with_cancellation_token(&token_clone).await.is_none())\n /// # }\n /// ```\n fn with_cancellation_token(\n self,\n cancellation_token: &CancellationToken,\n ) -> WithCancellationTokenFuture<'_, Self>\n where\n Self: Sized,\n {\n WithCancellationTokenFuture::new(cancellation_token, self)\n }\n\n /// Similar to [`CancellationToken::run_until_cancelled_owned`],\n /// but with the advantage that it is easier to write fluent call chains.\n ///\n /// # Fairness\n ///\n /// Calling this on an already-cancelled token directly returns `None`.\n /// For all subsequent polls, in case of concurrent completion and\n /// cancellation, this is biased towards the `self` future completion.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio::sync::oneshot;\n /// use tokio_util::future::FutureExt;\n /// use tokio_util::sync::CancellationToken;\n ///\n /// # async fn dox() {\n /// let (_tx, rx) = oneshot::channel::<()>();\n /// let token = CancellationToken::new();\n /// let token_clone = token.clone();\n /// tokio::spawn(async move {\n /// tokio::time::sleep(std::time::Duration::from_millis(10)).await;\n /// token.cancel();\n /// });\n /// assert!(rx.with_cancellation_token_owned(token_clone).await.is_none())\n /// # }\n /// ```\n fn with_cancellation_token_owned(\n self,\n cancellation_token: CancellationToken,\n ) -> WithCancellationTokenFutureOwned\n where\n Self: Sized,\n {\n WithCancellationTokenFutureOwned::new(cancellation_token, self)\n }\n}\n\nimpl FutureExt for T {}\n" }, { "path": "tokio-util/src/io/copy_to_bytes.rs", "content": "use bytes::Bytes;\nuse futures_core::stream::Stream;\nuse futures_sink::Sink;\nuse pin_project_lite::pin_project;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\npin_project! {\n /// A helper that wraps a [`Sink`]`<`[`Bytes`]`>` and converts it into a\n /// [`Sink`]`<&'a [u8]>` by copying each byte slice into an owned [`Bytes`].\n ///\n /// See the documentation for [`SinkWriter`] for an example.\n ///\n /// [`Bytes`]: bytes::Bytes\n /// [`SinkWriter`]: crate::io::SinkWriter\n /// [`Sink`]: futures_sink::Sink\n #[derive(Debug)]\n pub struct CopyToBytes {\n #[pin]\n inner: S,\n }\n}\n\nimpl CopyToBytes {\n /// Creates a new [`CopyToBytes`].\n pub fn new(inner: S) -> Self {\n Self { inner }\n }\n\n /// Gets a reference to the underlying sink.\n pub fn get_ref(&self) -> &S {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying sink.\n pub fn get_mut(&mut self) -> &mut S {\n &mut self.inner\n }\n\n /// Consumes this [`CopyToBytes`], returning the underlying sink.\n pub fn into_inner(self) -> S {\n self.inner\n }\n}\n\nimpl<'a, S> Sink<&'a [u8]> for CopyToBytes\nwhere\n S: Sink,\n{\n type Error = S::Error;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_ready(cx)\n }\n\n fn start_send(self: Pin<&mut Self>, item: &'a [u8]) -> Result<(), Self::Error> {\n self.project()\n .inner\n .start_send(Bytes::copy_from_slice(item))\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_close(cx)\n }\n}\n\nimpl Stream for CopyToBytes {\n type Item = S::Item;\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_next(cx)\n }\n}\n" }, { "path": "tokio-util/src/io/inspect.rs", "content": "use pin_project_lite::pin_project;\nuse std::io::{IoSlice, Result};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\nuse tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\n\npin_project! {\n /// An adapter that lets you inspect the data that's being read.\n ///\n /// This is useful for things like hashing data as it's read in.\n pub struct InspectReader {\n #[pin]\n reader: R,\n f: F,\n }\n}\n\nimpl InspectReader {\n /// Create a new `InspectReader`, wrapping `reader` and calling `f` for the\n /// new data supplied by each read call.\n ///\n /// The closure will only be called with an empty slice if the inner reader\n /// returns without reading data into the buffer. This happens at EOF, or if\n /// `poll_read` is called with a zero-size buffer.\n pub fn new(reader: R, f: F) -> InspectReader\n where\n R: AsyncRead,\n F: FnMut(&[u8]),\n {\n InspectReader { reader, f }\n }\n\n /// Consumes the `InspectReader`, returning the wrapped reader\n pub fn into_inner(self) -> R {\n self.reader\n }\n}\n\nimpl AsyncRead for InspectReader {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let me = self.project();\n let filled_length = buf.filled().len();\n ready!(me.reader.poll_read(cx, buf))?;\n (me.f)(&buf.filled()[filled_length..]);\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncWrite for InspectReader {\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n self.project().reader.poll_write(cx, buf)\n }\n\n fn poll_flush(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n self.project().reader.poll_flush(cx)\n }\n\n fn poll_shutdown(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n ) -> Poll> {\n self.project().reader.poll_shutdown(cx)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n self.project().reader.poll_write_vectored(cx, bufs)\n }\n\n fn is_write_vectored(&self) -> bool {\n self.reader.is_write_vectored()\n }\n}\n\npin_project! {\n /// An adapter that lets you inspect the data that's being written.\n ///\n /// This is useful for things like hashing data as it's written out.\n pub struct InspectWriter {\n #[pin]\n writer: W,\n f: F,\n }\n}\n\nimpl InspectWriter {\n /// Create a new `InspectWriter`, wrapping `write` and calling `f` for the\n /// data successfully written by each write call.\n ///\n /// The closure `f` will never be called with an empty slice. A vectored\n /// write can result in multiple calls to `f` - at most one call to `f` per\n /// buffer supplied to `poll_write_vectored`.\n pub fn new(writer: W, f: F) -> InspectWriter\n where\n W: AsyncWrite,\n F: FnMut(&[u8]),\n {\n InspectWriter { writer, f }\n }\n\n /// Consumes the `InspectWriter`, returning the wrapped writer\n pub fn into_inner(self) -> W {\n self.writer\n }\n}\n\nimpl AsyncWrite for InspectWriter {\n fn poll_write(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll> {\n let me = self.project();\n let res = me.writer.poll_write(cx, buf);\n if let Poll::Ready(Ok(count)) = res {\n if count != 0 {\n (me.f)(&buf[..count]);\n }\n }\n res\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n me.writer.poll_flush(cx)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n me.writer.poll_shutdown(cx)\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n let me = self.project();\n let res = me.writer.poll_write_vectored(cx, bufs);\n if let Poll::Ready(Ok(mut count)) = res {\n for buf in bufs {\n if count == 0 {\n break;\n }\n let size = count.min(buf.len());\n if size != 0 {\n (me.f)(&buf[..size]);\n count -= size;\n }\n }\n }\n res\n }\n\n fn is_write_vectored(&self) -> bool {\n self.writer.is_write_vectored()\n }\n}\n\nimpl AsyncRead for InspectWriter {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n self.project().writer.poll_read(cx, buf)\n }\n}\n" }, { "path": "tokio-util/src/io/mod.rs", "content": "//! Helpers for IO related tasks.\n//!\n//! The stream types are often used in combination with hyper or reqwest, as they\n//! allow converting between a hyper [`Body`] and [`AsyncRead`].\n//!\n//! The [`SyncIoBridge`] type converts from the world of async I/O\n//! to synchronous I/O; this may often come up when using synchronous APIs\n//! inside [`tokio::task::spawn_blocking`].\n//!\n//! [`Body`]: https://docs.rs/hyper/0.13/hyper/struct.Body.html\n//! [`AsyncRead`]: tokio::io::AsyncRead\n\nmod copy_to_bytes;\nmod inspect;\nmod read_buf;\nmod reader_stream;\npub mod simplex;\nmod sink_writer;\nmod stream_reader;\nmod write_all_vectored;\n\ncfg_io_util! {\n mod read_arc;\n pub use self::read_arc::read_exact_arc;\n\n mod sync_bridge;\n pub use self::sync_bridge::SyncIoBridge;\n}\n\npub use self::copy_to_bytes::CopyToBytes;\npub use self::inspect::{InspectReader, InspectWriter};\npub use self::read_buf::read_buf;\npub use self::reader_stream::ReaderStream;\npub use self::sink_writer::SinkWriter;\npub use self::stream_reader::StreamReader;\npub use self::write_all_vectored::{write_all_vectored, WriteAllVectored};\npub use crate::util::{poll_read_buf, poll_write_buf};\n" }, { "path": "tokio-util/src/io/read_arc.rs", "content": "use std::io;\nuse std::mem::MaybeUninit;\nuse std::sync::Arc;\nuse tokio::io::{AsyncRead, AsyncReadExt};\n\n/// Read data from an `AsyncRead` into an `Arc`.\n///\n/// This uses `Arc::new_uninit_slice` and reads into the resulting uninitialized `Arc`.\n///\n/// # Example\n///\n/// ```\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n/// use tokio_util::io::read_exact_arc;\n///\n/// let read = tokio::io::repeat(42);\n///\n/// let arc = read_exact_arc(read, 4).await?;\n///\n/// assert_eq!(&arc[..], &[42; 4]);\n/// # Ok(())\n/// # }\n/// ```\npub async fn read_exact_arc(read: R, len: usize) -> io::Result> {\n tokio::pin!(read);\n // TODO(MSRV 1.82): When bumping MSRV, switch to `Arc::new_uninit_slice(len)`. The following is\n // equivalent, and generates the same assembly, but works without requiring MSRV 1.82.\n let arc: Arc<[MaybeUninit]> = (0..len).map(|_| MaybeUninit::uninit()).collect();\n // TODO(MSRV future): Use `Arc::get_mut_unchecked` once it's stabilized.\n // SAFETY: We're the only owner of the `Arc`, and we keep the `Arc` valid throughout this loop\n // as we write through this reference.\n let mut buf = unsafe { &mut *(Arc::as_ptr(&arc) as *mut [MaybeUninit]) };\n while !buf.is_empty() {\n if read.read_buf(&mut buf).await? == 0 {\n return Err(io::Error::new(io::ErrorKind::UnexpectedEof, \"early eof\"));\n }\n }\n // TODO(MSRV 1.82): When bumping MSRV, switch to `arc.assume_init()`. The following is\n // equivalent, and generates the same assembly, but works without requiring MSRV 1.82.\n // SAFETY: This changes `[MaybeUninit]` to `[u8]`, and we've initialized all the bytes in\n // the loop above.\n Ok(unsafe { Arc::from_raw(Arc::into_raw(arc) as *const [u8]) })\n}\n" }, { "path": "tokio-util/src/io/read_buf.rs", "content": "use bytes::BufMut;\nuse std::future::poll_fn;\nuse std::io;\nuse std::pin::Pin;\nuse tokio::io::AsyncRead;\n\n/// Read data from an `AsyncRead` into an implementer of the [`BufMut`] trait.\n///\n/// [`BufMut`]: bytes::BufMut\n///\n/// # Example\n///\n/// ```\n/// use bytes::{Bytes, BytesMut};\n/// use tokio_stream as stream;\n/// use tokio::io::Result;\n/// use tokio_util::io::{StreamReader, read_buf};\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n///\n/// // Create a reader from an iterator. This particular reader will always be\n/// // ready.\n/// let mut read = StreamReader::new(stream::iter(vec![Result::Ok(Bytes::from_static(&[0, 1, 2, 3]))]));\n///\n/// let mut buf = BytesMut::new();\n/// let mut reads = 0;\n///\n/// loop {\n/// reads += 1;\n/// let n = read_buf(&mut read, &mut buf).await?;\n///\n/// if n == 0 {\n/// break;\n/// }\n/// }\n///\n/// // one or more reads might be necessary.\n/// assert!(reads >= 1);\n/// assert_eq!(&buf[..], &[0, 1, 2, 3]);\n/// # Ok(())\n/// # }\n/// ```\npub async fn read_buf(read: &mut R, buf: &mut B) -> io::Result\nwhere\n R: AsyncRead + Unpin,\n B: BufMut,\n{\n poll_fn(|cx| crate::util::poll_read_buf(Pin::new(read), cx, buf)).await\n}\n" }, { "path": "tokio-util/src/io/reader_stream.rs", "content": "use bytes::{Bytes, BytesMut};\nuse futures_core::stream::Stream;\nuse pin_project_lite::pin_project;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::AsyncRead;\n\nconst DEFAULT_CAPACITY: usize = 4096;\n\npin_project! {\n /// Convert an [`AsyncRead`] into a [`Stream`] of byte chunks.\n ///\n /// This stream is fused. It performs the inverse operation of\n /// [`StreamReader`].\n ///\n /// # Example\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> std::io::Result<()> {\n /// use tokio_stream::StreamExt;\n /// use tokio_util::io::ReaderStream;\n ///\n /// // Create a stream of data.\n /// let data = b\"hello, world!\";\n /// let mut stream = ReaderStream::new(&data[..]);\n ///\n /// // Read all of the chunks into a vector.\n /// let mut stream_contents = Vec::new();\n /// while let Some(chunk) = stream.next().await {\n /// stream_contents.extend_from_slice(&chunk?);\n /// }\n ///\n /// // Once the chunks are concatenated, we should have the\n /// // original data.\n /// assert_eq!(stream_contents, data);\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`AsyncRead`]: tokio::io::AsyncRead\n /// [`StreamReader`]: crate::io::StreamReader\n /// [`Stream`]: futures_core::Stream\n #[derive(Debug)]\n pub struct ReaderStream {\n // Reader itself.\n //\n // This value is `None` if the stream has terminated.\n #[pin]\n reader: Option,\n // Working buffer, used to optimize allocations.\n buf: BytesMut,\n capacity: usize,\n }\n}\n\nimpl ReaderStream {\n /// Convert an [`AsyncRead`] into a [`Stream`] with item type\n /// `Result`.\n ///\n /// Currently, the default capacity 4096 bytes (4 KiB).\n /// This capacity is not part of the semver contract\n /// and may be tweaked in future releases without\n /// requiring a major version bump.\n ///\n /// [`AsyncRead`]: tokio::io::AsyncRead\n /// [`Stream`]: futures_core::Stream\n pub fn new(reader: R) -> Self {\n ReaderStream {\n reader: Some(reader),\n buf: BytesMut::new(),\n capacity: DEFAULT_CAPACITY,\n }\n }\n\n /// Convert an [`AsyncRead`] into a [`Stream`] with item type\n /// `Result`,\n /// with a specific read buffer initial capacity.\n ///\n /// [`AsyncRead`]: tokio::io::AsyncRead\n /// [`Stream`]: futures_core::Stream\n pub fn with_capacity(reader: R, capacity: usize) -> Self {\n ReaderStream {\n reader: Some(reader),\n buf: BytesMut::with_capacity(capacity),\n capacity,\n }\n }\n}\n\nimpl Stream for ReaderStream {\n type Item = std::io::Result;\n fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n use crate::util::poll_read_buf;\n\n let mut this = self.as_mut().project();\n\n let reader = match this.reader.as_pin_mut() {\n Some(r) => r,\n None => return Poll::Ready(None),\n };\n\n if this.buf.capacity() == 0 {\n this.buf.reserve(*this.capacity);\n }\n\n match poll_read_buf(reader, cx, &mut this.buf) {\n Poll::Pending => Poll::Pending,\n Poll::Ready(Err(err)) => {\n self.project().reader.set(None);\n Poll::Ready(Some(Err(err)))\n }\n Poll::Ready(Ok(0)) => {\n self.project().reader.set(None);\n Poll::Ready(None)\n }\n Poll::Ready(Ok(_)) => {\n let chunk = this.buf.split();\n Poll::Ready(Some(Ok(chunk.freeze())))\n }\n }\n }\n}\n" }, { "path": "tokio-util/src/io/simplex.rs", "content": "//! Unidirectional byte-oriented channel.\n\nuse crate::util::poll_proceed;\n\nuse bytes::Buf;\nuse bytes::BytesMut;\nuse futures_core::ready;\nuse std::io::Error as IoError;\nuse std::io::ErrorKind as IoErrorKind;\nuse std::io::IoSlice;\nuse std::pin::Pin;\nuse std::sync::{Arc, Mutex};\nuse std::task::{Context, Poll, Waker};\nuse tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\n\ntype IoResult = Result;\n\nconst CLOSED_ERROR_MSG: &str = \"simplex has been closed\";\n\n#[derive(Debug)]\nstruct Inner {\n /// `poll_write` will return [`Poll::Pending`] if the backpressure boundary is reached\n backpressure_boundary: usize,\n\n /// either [`Sender`] or [`Receiver`] is closed\n is_closed: bool,\n\n /// Waker used to wake the [`Receiver`]\n receiver_waker: Option,\n\n /// Waker used to wake the [`Sender`]\n sender_waker: Option,\n\n /// Buffer used to read and write data\n buf: BytesMut,\n}\n\nimpl Inner {\n fn with_capacity(capacity: usize) -> Self {\n Self {\n backpressure_boundary: capacity,\n is_closed: false,\n receiver_waker: None,\n sender_waker: None,\n buf: BytesMut::with_capacity(capacity),\n }\n }\n\n fn register_receiver_waker(&mut self, waker: &Waker) -> Option {\n match self.receiver_waker.as_mut() {\n Some(old) if old.will_wake(waker) => None,\n _ => self.receiver_waker.replace(waker.clone()),\n }\n }\n\n fn register_sender_waker(&mut self, waker: &Waker) -> Option {\n match self.sender_waker.as_mut() {\n Some(old) if old.will_wake(waker) => None,\n _ => self.sender_waker.replace(waker.clone()),\n }\n }\n\n fn take_receiver_waker(&mut self) -> Option {\n self.receiver_waker.take()\n }\n\n fn take_sender_waker(&mut self) -> Option {\n self.sender_waker.take()\n }\n\n fn is_closed(&self) -> bool {\n self.is_closed\n }\n\n fn close_receiver(&mut self) -> Option {\n self.is_closed = true;\n self.take_sender_waker()\n }\n\n fn close_sender(&mut self) -> Option {\n self.is_closed = true;\n self.take_receiver_waker()\n }\n}\n\n/// Receiver of the simplex channel.\n///\n/// # Cancellation safety\n///\n/// The `Receiver` is cancel safe. If it is used as the event in a\n/// [`tokio::select!`] statement and some other branch completes\n/// first, it is guaranteed that no bytes were received on this\n/// channel.\n///\n/// You can still read the remaining data from the buffer\n/// even if the write half has been dropped.\n/// See [`Sender::poll_shutdown`] and [`Sender::drop`] for more details.\n///\n/// [`tokio::select!`]: https://docs.rs/tokio/latest/tokio/macro.select.html\n#[derive(Debug)]\npub struct Receiver {\n inner: Arc>,\n}\n\nimpl Drop for Receiver {\n /// This also wakes up the [`Sender`].\n fn drop(&mut self) {\n let maybe_waker = {\n let mut inner = self.inner.lock().unwrap();\n inner.close_receiver()\n };\n\n if let Some(waker) = maybe_waker {\n waker.wake();\n }\n }\n}\n\nimpl AsyncRead for Receiver {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let coop = ready!(poll_proceed(cx));\n\n let mut inner = self.inner.lock().unwrap();\n\n let to_read = buf.remaining().min(inner.buf.remaining());\n if to_read == 0 {\n if inner.is_closed() || buf.remaining() == 0 {\n return Poll::Ready(Ok(()));\n }\n\n let old_waker = inner.register_receiver_waker(cx.waker());\n let maybe_waker = inner.take_sender_waker();\n\n // unlock before waking up and dropping old waker\n drop(inner);\n drop(old_waker);\n if let Some(waker) = maybe_waker {\n waker.wake();\n }\n return Poll::Pending;\n }\n\n // this is to avoid starving other tasks\n coop.made_progress();\n\n buf.put_slice(&inner.buf[..to_read]);\n inner.buf.advance(to_read);\n\n let waker = inner.take_sender_waker();\n drop(inner); // unlock before waking up\n if let Some(waker) = waker {\n waker.wake();\n }\n\n Poll::Ready(Ok(()))\n }\n}\n\n/// Sender of the simplex channel.\n///\n/// # Cancellation safety\n///\n/// The `Sender` is cancel safe. If it is used as the event in a\n/// [`tokio::select!`] statement and some other branch completes\n/// first, it is guaranteed that no bytes were sent on this channel.\n///\n/// # Shutdown\n///\n/// See [`Sender::poll_shutdown`].\n///\n/// [`tokio::select!`]: https://docs.rs/tokio/latest/tokio/macro.select.html\n#[derive(Debug)]\npub struct Sender {\n inner: Arc>,\n}\n\nimpl Drop for Sender {\n /// This also wakes up the [`Receiver`].\n fn drop(&mut self) {\n let maybe_waker = {\n let mut inner = self.inner.lock().unwrap();\n inner.close_sender()\n };\n\n if let Some(waker) = maybe_waker {\n waker.wake();\n }\n }\n}\n\nimpl AsyncWrite for Sender {\n /// # Errors\n ///\n /// This method will return [`IoErrorKind::BrokenPipe`]\n /// if the channel has been closed.\n fn poll_write(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll> {\n let coop = ready!(poll_proceed(cx));\n\n let mut inner = self.inner.lock().unwrap();\n\n if inner.is_closed() {\n return Poll::Ready(Err(IoError::new(IoErrorKind::BrokenPipe, CLOSED_ERROR_MSG)));\n }\n\n let free = inner\n .backpressure_boundary\n .checked_sub(inner.buf.len())\n .expect(\"backpressure boundary overflow\");\n let to_write = buf.len().min(free);\n if to_write == 0 {\n if buf.is_empty() {\n return Poll::Ready(Ok(0));\n }\n\n let old_waker = inner.register_sender_waker(cx.waker());\n let waker = inner.take_receiver_waker();\n\n // unlock before waking up and dropping old waker\n drop(inner);\n drop(old_waker);\n if let Some(waker) = waker {\n waker.wake();\n }\n\n return Poll::Pending;\n }\n\n // this is to avoid starving other tasks\n coop.made_progress();\n\n inner.buf.extend_from_slice(&buf[..to_write]);\n\n let waker = inner.take_receiver_waker();\n drop(inner); // unlock before waking up\n if let Some(waker) = waker {\n waker.wake();\n }\n\n Poll::Ready(Ok(to_write))\n }\n\n /// # Errors\n ///\n /// This method will return [`IoErrorKind::BrokenPipe`]\n /// if the channel has been closed.\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n let inner = self.inner.lock().unwrap();\n if inner.is_closed() {\n Poll::Ready(Err(IoError::new(IoErrorKind::BrokenPipe, CLOSED_ERROR_MSG)))\n } else {\n Poll::Ready(Ok(()))\n }\n }\n\n /// After returns [`Poll::Ready`], all the following call to\n /// [`Sender::poll_write`] and [`Sender::poll_flush`]\n /// will return error.\n ///\n /// The [`Receiver`] can still be used to read remaining data\n /// until all bytes have been consumed.\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n let maybe_waker = {\n let mut inner = self.inner.lock().unwrap();\n inner.close_sender()\n };\n\n if let Some(waker) = maybe_waker {\n waker.wake();\n }\n\n Poll::Ready(Ok(()))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n\n fn poll_write_vectored(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n let coop = ready!(poll_proceed(cx));\n\n let mut inner = self.inner.lock().unwrap();\n if inner.is_closed() {\n return Poll::Ready(Err(IoError::new(IoErrorKind::BrokenPipe, CLOSED_ERROR_MSG)));\n }\n\n let free = inner\n .backpressure_boundary\n .checked_sub(inner.buf.len())\n .expect(\"backpressure boundary overflow\");\n if free == 0 {\n let old_waker = inner.register_sender_waker(cx.waker());\n let maybe_waker = inner.take_receiver_waker();\n\n // unlock before waking up and dropping old waker\n drop(inner);\n drop(old_waker);\n if let Some(waker) = maybe_waker {\n waker.wake();\n }\n\n return Poll::Pending;\n }\n\n // this is to avoid starving other tasks\n coop.made_progress();\n\n let mut rem = free;\n for buf in bufs {\n if rem == 0 {\n break;\n }\n\n let to_write = buf.len().min(rem);\n if to_write == 0 {\n assert_ne!(rem, 0);\n assert_eq!(buf.len(), 0);\n continue;\n }\n\n inner.buf.extend_from_slice(&buf[..to_write]);\n rem -= to_write;\n }\n\n let waker = inner.take_receiver_waker();\n drop(inner); // unlock before waking up\n if let Some(waker) = waker {\n waker.wake();\n }\n\n Poll::Ready(Ok(free - rem))\n }\n}\n\n/// Create a simplex channel.\n///\n/// The `capacity` parameter specifies the maximum number of bytes that can be\n/// stored in the channel without making the [`Sender::poll_write`]\n/// return [`Poll::Pending`].\n///\n/// # Panics\n///\n/// This function will panic if `capacity` is zero.\npub fn new(capacity: usize) -> (Sender, Receiver) {\n assert_ne!(capacity, 0, \"capacity must be greater than zero\");\n\n let inner = Arc::new(Mutex::new(Inner::with_capacity(capacity)));\n let tx = Sender {\n inner: Arc::clone(&inner),\n };\n let rx = Receiver { inner };\n (tx, rx)\n}\n" }, { "path": "tokio-util/src/io/sink_writer.rs", "content": "use futures_sink::Sink;\n\nuse futures_core::stream::Stream;\nuse pin_project_lite::pin_project;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse tokio::io::{AsyncRead, AsyncWrite};\n\npin_project! {\n /// Convert a [`Sink`] of byte chunks into an [`AsyncWrite`].\n ///\n /// Whenever you write to this [`SinkWriter`], the supplied bytes are\n /// forwarded to the inner [`Sink`]. When `shutdown` is called on this\n /// [`SinkWriter`], the inner sink is closed.\n ///\n /// This adapter takes a `Sink<&[u8]>` and provides an [`AsyncWrite`] impl\n /// for it. Because of the lifetime, this trait is relatively rarely\n /// implemented. The main ways to get a `Sink<&[u8]>` that you can use with\n /// this type are:\n ///\n /// * With the codec module by implementing the [`Encoder`]`<&[u8]>` trait.\n /// * By wrapping a `Sink` in a [`CopyToBytes`].\n /// * Manually implementing `Sink<&[u8]>` directly.\n ///\n /// The opposite conversion of implementing `Sink<_>` for an [`AsyncWrite`]\n /// is done using the [`codec`] module.\n ///\n /// # Example\n ///\n /// ```\n /// use bytes::Bytes;\n /// use futures_util::SinkExt;\n /// use std::io::{Error, ErrorKind};\n /// use tokio::io::AsyncWriteExt;\n /// use tokio_util::io::{SinkWriter, CopyToBytes};\n /// use tokio_util::sync::PollSender;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() -> Result<(), Error> {\n /// // We use an mpsc channel as an example of a `Sink`.\n /// let (tx, mut rx) = tokio::sync::mpsc::channel::(1);\n /// let sink = PollSender::new(tx).sink_map_err(|_| Error::from(ErrorKind::BrokenPipe));\n ///\n /// // Wrap it in `CopyToBytes` to get a `Sink<&[u8]>`.\n /// let mut writer = SinkWriter::new(CopyToBytes::new(sink));\n ///\n /// // Write data to our interface...\n /// let data: [u8; 4] = [1, 2, 3, 4];\n /// let _ = writer.write(&data).await?;\n ///\n /// // ... and receive it.\n /// assert_eq!(data.as_slice(), &*rx.recv().await.unwrap());\n /// # Ok(())\n /// # }\n /// ```\n ///\n /// [`AsyncWrite`]: tokio::io::AsyncWrite\n /// [`CopyToBytes`]: crate::io::CopyToBytes\n /// [`Encoder`]: crate::codec::Encoder\n /// [`Sink`]: futures_sink::Sink\n /// [`codec`]: crate::codec\n #[derive(Debug)]\n pub struct SinkWriter {\n #[pin]\n inner: S,\n }\n}\n\nimpl SinkWriter {\n /// Creates a new [`SinkWriter`].\n pub fn new(sink: S) -> Self {\n Self { inner: sink }\n }\n\n /// Gets a reference to the underlying sink.\n pub fn get_ref(&self) -> &S {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying sink.\n pub fn get_mut(&mut self) -> &mut S {\n &mut self.inner\n }\n\n /// Consumes this [`SinkWriter`], returning the underlying sink.\n pub fn into_inner(self) -> S {\n self.inner\n }\n}\nimpl AsyncWrite for SinkWriter\nwhere\n for<'a> S: Sink<&'a [u8], Error = E>,\n E: Into,\n{\n fn poll_write(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n let mut this = self.project();\n\n ready!(this.inner.as_mut().poll_ready(cx).map_err(Into::into))?;\n match this.inner.as_mut().start_send(buf) {\n Ok(()) => Poll::Ready(Ok(buf.len())),\n Err(e) => Poll::Ready(Err(e.into())),\n }\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx).map_err(Into::into)\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_close(cx).map_err(Into::into)\n }\n}\n\nimpl Stream for SinkWriter {\n type Item = S::Item;\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_next(cx)\n }\n}\n\nimpl AsyncRead for SinkWriter {\n fn poll_read(\n self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut tokio::io::ReadBuf<'_>,\n ) -> Poll> {\n self.project().inner.poll_read(cx, buf)\n }\n}\n" }, { "path": "tokio-util/src/io/stream_reader.rs", "content": "use bytes::Buf;\nuse futures_core::stream::Stream;\nuse futures_sink::Sink;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{AsyncBufRead, AsyncRead, ReadBuf};\n\n/// Convert a [`Stream`] of byte chunks into an [`AsyncRead`].\n///\n/// This type performs the inverse operation of [`ReaderStream`].\n///\n/// This type also implements the [`AsyncBufRead`] trait, so you can use it\n/// to read a `Stream` of byte chunks line-by-line. See the examples below.\n///\n/// # Example\n///\n/// ```\n/// use bytes::Bytes;\n/// use tokio::io::{AsyncReadExt, Result};\n/// use tokio_util::io::StreamReader;\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n///\n/// // Create a stream from an iterator.\n/// let stream = tokio_stream::iter(vec![\n/// Result::Ok(Bytes::from_static(&[0, 1, 2, 3])),\n/// Result::Ok(Bytes::from_static(&[4, 5, 6, 7])),\n/// Result::Ok(Bytes::from_static(&[8, 9, 10, 11])),\n/// ]);\n///\n/// // Convert it to an AsyncRead.\n/// let mut read = StreamReader::new(stream);\n///\n/// // Read five bytes from the stream.\n/// let mut buf = [0; 5];\n/// read.read_exact(&mut buf).await?;\n/// assert_eq!(buf, [0, 1, 2, 3, 4]);\n///\n/// // Read the rest of the current chunk.\n/// assert_eq!(read.read(&mut buf).await?, 3);\n/// assert_eq!(&buf[..3], [5, 6, 7]);\n///\n/// // Read the next chunk.\n/// assert_eq!(read.read(&mut buf).await?, 4);\n/// assert_eq!(&buf[..4], [8, 9, 10, 11]);\n///\n/// // We have now reached the end.\n/// assert_eq!(read.read(&mut buf).await?, 0);\n///\n/// # Ok(())\n/// # }\n/// ```\n///\n/// If the stream produces errors which are not [`std::io::Error`],\n/// the errors can be converted using [`StreamExt`] to map each\n/// element.\n///\n/// ```\n/// use bytes::Bytes;\n/// use tokio::io::AsyncReadExt;\n/// use tokio_util::io::StreamReader;\n/// use tokio_stream::StreamExt;\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n///\n/// // Create a stream from an iterator, including an error.\n/// let stream = tokio_stream::iter(vec![\n/// Result::Ok(Bytes::from_static(&[0, 1, 2, 3])),\n/// Result::Ok(Bytes::from_static(&[4, 5, 6, 7])),\n/// Result::Err(\"Something bad happened!\")\n/// ]);\n///\n/// // Use StreamExt to map the stream and error to a std::io::Error\n/// let stream = stream.map(|result| result.map_err(|err| {\n/// std::io::Error::new(std::io::ErrorKind::Other, err)\n/// }));\n///\n/// // Convert it to an AsyncRead.\n/// let mut read = StreamReader::new(stream);\n///\n/// // Read five bytes from the stream.\n/// let mut buf = [0; 5];\n/// read.read_exact(&mut buf).await?;\n/// assert_eq!(buf, [0, 1, 2, 3, 4]);\n///\n/// // Read the rest of the current chunk.\n/// assert_eq!(read.read(&mut buf).await?, 3);\n/// assert_eq!(&buf[..3], [5, 6, 7]);\n///\n/// // Reading the next chunk will produce an error\n/// let error = read.read(&mut buf).await.unwrap_err();\n/// assert_eq!(error.kind(), std::io::ErrorKind::Other);\n/// assert_eq!(error.into_inner().unwrap().to_string(), \"Something bad happened!\");\n///\n/// // We have now reached the end.\n/// assert_eq!(read.read(&mut buf).await?, 0);\n///\n/// # Ok(())\n/// # }\n/// ```\n///\n/// Using the [`AsyncBufRead`] impl, you can read a `Stream` of byte chunks\n/// line-by-line. Note that you will usually also need to convert the error\n/// type when doing this. See the second example for an explanation of how\n/// to do this.\n///\n/// ```\n/// use tokio::io::{Result, AsyncBufReadExt};\n/// use tokio_util::io::StreamReader;\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n///\n/// // Create a stream of byte chunks.\n/// let stream = tokio_stream::iter(vec![\n/// Result::Ok(b\"The first line.\\n\".as_slice()),\n/// Result::Ok(b\"The second line.\".as_slice()),\n/// Result::Ok(b\"\\nThe third\".as_slice()),\n/// Result::Ok(b\" line.\\nThe fourth line.\\nThe fifth line.\\n\".as_slice()),\n/// ]);\n///\n/// // Convert it to an AsyncRead.\n/// let mut read = StreamReader::new(stream);\n///\n/// // Loop through the lines from the `StreamReader`.\n/// let mut line = String::new();\n/// let mut lines = Vec::new();\n/// loop {\n/// line.clear();\n/// let len = read.read_line(&mut line).await?;\n/// if len == 0 { break; }\n/// lines.push(line.clone());\n/// }\n///\n/// // Verify that we got the lines we expected.\n/// assert_eq!(\n/// lines,\n/// vec![\n/// \"The first line.\\n\",\n/// \"The second line.\\n\",\n/// \"The third line.\\n\",\n/// \"The fourth line.\\n\",\n/// \"The fifth line.\\n\",\n/// ]\n/// );\n/// # Ok(())\n/// # }\n/// ```\n///\n/// [`AsyncRead`]: tokio::io::AsyncRead\n/// [`AsyncBufRead`]: tokio::io::AsyncBufRead\n/// [`Stream`]: futures_core::Stream\n/// [`ReaderStream`]: crate::io::ReaderStream\n/// [`StreamExt`]: https://docs.rs/tokio-stream/latest/tokio_stream/trait.StreamExt.html\n#[derive(Debug)]\npub struct StreamReader {\n // This field is pinned.\n inner: S,\n // This field is not pinned.\n chunk: Option,\n}\n\nimpl StreamReader\nwhere\n S: Stream>,\n B: Buf,\n E: Into,\n{\n /// Convert a stream of byte chunks into an [`AsyncRead`].\n ///\n /// The item should be a [`Result`] with the ok variant being something that\n /// implements the [`Buf`] trait (e.g. `Cursor>` or `Bytes`). The error\n /// should be convertible into an [io error].\n ///\n /// [`Result`]: std::result::Result\n /// [`Buf`]: bytes::Buf\n /// [io error]: std::io::Error\n pub fn new(stream: S) -> Self {\n Self {\n inner: stream,\n chunk: None,\n }\n }\n\n /// Do we have a chunk and is it non-empty?\n fn has_chunk(&self) -> bool {\n if let Some(ref chunk) = self.chunk {\n chunk.remaining() > 0\n } else {\n false\n }\n }\n\n /// Consumes this `StreamReader`, returning a Tuple consisting\n /// of the underlying stream and an Option of the internal buffer,\n /// which is Some in case the buffer contains elements.\n pub fn into_inner_with_chunk(self) -> (S, Option) {\n if self.has_chunk() {\n (self.inner, self.chunk)\n } else {\n (self.inner, None)\n }\n }\n}\n\nimpl StreamReader {\n /// Gets a reference to the underlying stream.\n ///\n /// It is inadvisable to directly read from the underlying stream.\n pub fn get_ref(&self) -> &S {\n &self.inner\n }\n\n /// Gets a mutable reference to the underlying stream.\n ///\n /// It is inadvisable to directly read from the underlying stream.\n pub fn get_mut(&mut self) -> &mut S {\n &mut self.inner\n }\n\n /// Gets a pinned mutable reference to the underlying stream.\n ///\n /// It is inadvisable to directly read from the underlying stream.\n pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut S> {\n self.project().inner\n }\n\n /// Consumes this `BufWriter`, returning the underlying stream.\n ///\n /// Note that any leftover data in the internal buffer is lost.\n /// If you additionally want access to the internal buffer use\n /// [`into_inner_with_chunk`].\n ///\n /// [`into_inner_with_chunk`]: crate::io::StreamReader::into_inner_with_chunk\n pub fn into_inner(self) -> S {\n self.inner\n }\n}\n\nimpl AsyncRead for StreamReader\nwhere\n S: Stream>,\n B: Buf,\n E: Into,\n{\n fn poll_read(\n mut self: Pin<&mut Self>,\n cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if buf.remaining() == 0 {\n return Poll::Ready(Ok(()));\n }\n\n let inner_buf = match self.as_mut().poll_fill_buf(cx) {\n Poll::Ready(Ok(buf)) => buf,\n Poll::Ready(Err(err)) => return Poll::Ready(Err(err)),\n Poll::Pending => return Poll::Pending,\n };\n let len = std::cmp::min(inner_buf.len(), buf.remaining());\n buf.put_slice(&inner_buf[..len]);\n\n self.consume(len);\n Poll::Ready(Ok(()))\n }\n}\n\nimpl AsyncBufRead for StreamReader\nwhere\n S: Stream>,\n B: Buf,\n E: Into,\n{\n fn poll_fill_buf(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n loop {\n if self.as_mut().has_chunk() {\n // This unwrap is very sad, but it can't be avoided.\n let buf = self.project().chunk.as_ref().unwrap().chunk();\n return Poll::Ready(Ok(buf));\n } else {\n match self.as_mut().project().inner.poll_next(cx) {\n Poll::Ready(Some(Ok(chunk))) => {\n // Go around the loop in case the chunk is empty.\n *self.as_mut().project().chunk = Some(chunk);\n }\n Poll::Ready(Some(Err(err))) => return Poll::Ready(Err(err.into())),\n Poll::Ready(None) => return Poll::Ready(Ok(&[])),\n Poll::Pending => return Poll::Pending,\n }\n }\n }\n }\n fn consume(self: Pin<&mut Self>, amt: usize) {\n if amt > 0 {\n self.project()\n .chunk\n .as_mut()\n .expect(\"No chunk present\")\n .advance(amt);\n }\n }\n}\n\n// The code below is a manual expansion of the code that pin-project-lite would\n// generate. This is done because pin-project-lite fails by hitting the recursion\n// limit on this struct. (Every line of documentation is handled recursively by\n// the macro.)\n\nimpl Unpin for StreamReader {}\n\nstruct StreamReaderProject<'a, S, B> {\n inner: Pin<&'a mut S>,\n chunk: &'a mut Option,\n}\n\nimpl StreamReader {\n #[inline]\n fn project(self: Pin<&mut Self>) -> StreamReaderProject<'_, S, B> {\n // SAFETY: We define that only `inner` should be pinned when `Self` is\n // and have an appropriate `impl Unpin` for this.\n let me = unsafe { Pin::into_inner_unchecked(self) };\n StreamReaderProject {\n inner: unsafe { Pin::new_unchecked(&mut me.inner) },\n chunk: &mut me.chunk,\n }\n }\n}\n\nimpl, B, E, T> Sink for StreamReader {\n type Error = E;\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_ready(cx)\n }\n\n fn start_send(self: Pin<&mut Self>, item: T) -> Result<(), Self::Error> {\n self.project().inner.start_send(item)\n }\n\n fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_flush(cx)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n self.project().inner.poll_close(cx)\n }\n}\n" }, { "path": "tokio-util/src/io/sync_bridge.rs", "content": "use std::io::{BufRead, Read, Seek, Write};\nuse tokio::io::{\n AsyncBufRead, AsyncBufReadExt, AsyncRead, AsyncReadExt, AsyncSeek, AsyncSeekExt, AsyncWrite,\n AsyncWriteExt,\n};\n\n/// Use a [`tokio::io::AsyncRead`] synchronously as a [`std::io::Read`] or\n/// a [`tokio::io::AsyncWrite`] synchronously as a [`std::io::Write`].\n///\n/// # Alternatives\n///\n/// In many cases, there are better alternatives to using `SyncIoBridge`, especially\n/// if you want to avoid blocking the async runtime. Consider the following scenarios:\n///\n/// When hashing data, using `SyncIoBridge` can lead to suboptimal performance and\n/// might not fully leverage the async capabilities of the system.\n///\n/// ### Why It Matters:\n///\n/// `SyncIoBridge` allows you to use asynchronous I/O operations in an synchronous\n/// context by blocking the current thread. However, this can be inefficient because:\n/// - **Inefficient Resource Usage**: `SyncIoBridge` takes up an entire OS thread,\n/// which is inefficient compared to asynchronous code that can multiplex many\n/// tasks on a single thread.\n/// - **Thread Pool Saturation**: Excessive use of `SyncIoBridge` can exhaust the\n/// async runtime's thread pool, reducing the number of threads available for\n/// other tasks and impacting overall performance.\n/// - **Missed Concurrency Benefits**: By using synchronous operations with\n/// `SyncIoBridge`, you lose the ability to interleave tasks efficiently,\n/// which is a key advantage of asynchronous programming.\n///\n/// ## Example 1: Hashing Data\n///\n/// The use of `SyncIoBridge` is unnecessary when hashing data. Instead, you can\n/// process the data asynchronously by reading it into memory, which avoids blocking\n/// the async runtime.\n///\n/// There are two strategies for avoiding `SyncIoBridge` when hashing data. When\n/// the data fits into memory, the easiest is to read the data into a `Vec`\n/// and hash it:\n///\n/// Explanation: This example demonstrates how to asynchronously read data from a\n/// reader into memory and hash it using a synchronous hashing function. The\n/// `SyncIoBridge` is avoided, ensuring that the async runtime is not blocked.\n/// ```rust\n/// use tokio::io::AsyncReadExt;\n/// use tokio::io::AsyncRead;\n/// use std::io::Cursor;\n/// # mod blake3 { pub fn hash(_: &[u8]) {} }\n///\n/// async fn hash_contents(mut reader: impl AsyncRead + Unpin) -> Result<(), std::io::Error> {\n/// // Read all data from the reader into a Vec.\n/// let mut data = Vec::new();\n/// reader.read_to_end(&mut data).await?;\n///\n/// // Hash the data using the blake3 hashing function.\n/// let hash = blake3::hash(&data);\n///\n/// Ok(hash)\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), std::io::Error> {\n/// // Example: In-memory data.\n/// let data = b\"Hello, world!\"; // A byte slice.\n/// let reader = Cursor::new(data); // Create an in-memory AsyncRead.\n/// hash_contents(reader).await\n/// # }\n/// ```\n///\n/// When the data doesn't fit into memory, the hashing library will usually\n/// provide a `hasher` that you can repeatedly call `update` on to hash the data\n/// one chunk at the time.\n///\n/// Explanation: This example demonstrates how to asynchronously stream data in\n/// chunks for hashing. Each chunk is read asynchronously, and the hash is updated\n/// incrementally. This avoids blocking and improves performance over using\n/// `SyncIoBridge`.\n///\n/// ```rust\n/// use tokio::io::AsyncReadExt;\n/// use tokio::io::AsyncRead;\n/// use std::io::Cursor;\n/// # struct Hasher;\n/// # impl Hasher { pub fn update(&mut self, _: &[u8]) {} pub fn finalize(&self) {} }\n///\n/// /// Asynchronously streams data from an async reader, processes it in chunks,\n/// /// and hashes the data incrementally.\n/// async fn hash_stream(mut reader: impl AsyncRead + Unpin, mut hasher: Hasher) -> Result<(), std::io::Error> {\n/// // Create a buffer to read data into, sized for performance.\n/// let mut data = vec![0; 16 * 1024];\n/// loop {\n/// // Read data from the reader into the buffer.\n/// let len = reader.read(&mut data).await?;\n/// if len == 0 { break; } // Exit loop if no more data.\n///\n/// // Update the hash with the data read.\n/// hasher.update(&data[..len]);\n/// }\n///\n/// // Finalize the hash after all data has been processed.\n/// let hash = hasher.finalize();\n///\n/// Ok(hash)\n/// }\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> Result<(), std::io::Error> {\n/// // Example: In-memory data.\n/// let data = b\"Hello, world!\"; // A byte slice.\n/// let reader = Cursor::new(data); // Create an in-memory AsyncRead.\n/// let hasher = Hasher;\n/// hash_stream(reader, hasher).await\n/// # }\n/// ```\n///\n///\n/// ## Example 2: Compressing Data\n///\n/// When compressing data, the use of `SyncIoBridge` is unnecessary as it introduces\n/// blocking and inefficient code. Instead, you can utilize an async compression library\n/// such as the [`async-compression`](https://docs.rs/async-compression/latest/async_compression/)\n/// crate, which is built to handle asynchronous data streams efficiently.\n///\n/// Explanation: This example shows how to asynchronously compress data using an\n/// async compression library. By reading and writing asynchronously, it avoids\n/// blocking and is more efficient than using `SyncIoBridge` with a non-async\n/// compression library.\n///\n/// ```ignore\n/// use async_compression::tokio::write::GzipEncoder;\n/// use std::io::Cursor;\n/// use tokio::io::AsyncRead;\n///\n/// /// Asynchronously compresses data from an async reader using Gzip and an async encoder.\n/// async fn compress_data(mut reader: impl AsyncRead + Unpin) -> Result<(), std::io::Error> {\n/// let writer = tokio::io::sink();\n///\n/// // Create a Gzip encoder that wraps the writer.\n/// let mut encoder = GzipEncoder::new(writer);\n///\n/// // Copy data from the reader to the encoder, compressing it.\n/// tokio::io::copy(&mut reader, &mut encoder).await?;\n///\n/// Ok(())\n///}\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), std::io::Error> {\n/// // Example: In-memory data.\n/// let data = b\"Hello, world!\"; // A byte slice.\n/// let reader = Cursor::new(data); // Create an in-memory AsyncRead.\n/// compress_data(reader).await?;\n///\n/// Ok(())\n/// }\n/// ```\n///\n///\n/// ## Example 3: Parsing Data Formats\n///\n///\n/// `SyncIoBridge` is not ideal when parsing data formats such as `JSON`, as it\n/// blocks async operations. A more efficient approach is to read data asynchronously\n/// into memory and then `deserialize` it, avoiding unnecessary synchronization overhead.\n///\n/// Explanation: This example shows how to asynchronously read data into memory\n/// and then parse it as `JSON`. By avoiding `SyncIoBridge`, the asynchronous runtime\n/// remains unblocked, leading to better performance when working with asynchronous\n/// I/O streams.\n///\n/// ```rust,no_run\n/// use tokio::io::AsyncRead;\n/// use tokio::io::AsyncReadExt;\n/// use std::io::Cursor;\n/// # mod serde {\n/// # pub trait DeserializeOwned: 'static {}\n/// # impl DeserializeOwned for T {}\n/// # }\n/// # mod serde_json {\n/// # use super::serde::DeserializeOwned;\n/// # pub fn from_slice(_: &[u8]) -> Result {\n/// # unimplemented!()\n/// # }\n/// # }\n/// # #[derive(Debug)] struct MyStruct;\n///\n///\n/// async fn parse_json(mut reader: impl AsyncRead + Unpin) -> Result {\n/// // Read all data from the reader into a Vec.\n/// let mut data = Vec::new();\n/// reader.read_to_end(&mut data).await?;\n///\n/// // Deserialize the data from the Vec into a MyStruct instance.\n/// let value: MyStruct = serde_json::from_slice(&data)?;\n///\n/// Ok(value)\n///}\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), std::io::Error> {\n/// // Example: In-memory data.\n/// let data = b\"Hello, world!\"; // A byte slice.\n/// let reader = Cursor::new(data); // Create an in-memory AsyncRead.\n/// parse_json(reader).await?;\n/// Ok(())\n/// }\n/// ```\n///\n/// ## Correct Usage of `SyncIoBridge` inside `spawn_blocking`\n///\n/// `SyncIoBridge` is mainly useful when you need to interface with synchronous\n/// libraries from an asynchronous context.\n///\n/// Explanation: This example shows how to use `SyncIoBridge` inside a `spawn_blocking`\n/// task to safely perform synchronous I/O without blocking the async runtime. The\n/// `spawn_blocking` ensures that the synchronous code is offloaded to a dedicated\n/// thread pool, preventing it from interfering with the async tasks.\n///\n/// ```rust\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use tokio::task::spawn_blocking;\n/// use tokio_util::io::SyncIoBridge;\n/// use tokio::io::AsyncRead;\n/// use std::marker::Unpin;\n/// use std::io::Cursor;\n///\n/// /// Wraps an async reader with `SyncIoBridge` and performs synchronous I/O operations in a blocking task.\n/// async fn process_sync_io(reader: impl AsyncRead + Unpin + Send + 'static) -> Result, std::io::Error> {\n/// // Wrap the async reader with `SyncIoBridge` to allow synchronous reading.\n/// let mut sync_reader = SyncIoBridge::new(reader);\n///\n/// // Spawn a blocking task to perform synchronous I/O operations.\n/// let result = spawn_blocking(move || {\n/// // Create an in-memory buffer to hold the copied data.\n/// let mut buffer = Vec::new();\n/// // Copy data from the sync_reader to the buffer.\n/// std::io::copy(&mut sync_reader, &mut buffer)?;\n/// // Return the buffer containing the copied data.\n/// Ok::<_, std::io::Error>(buffer)\n/// })\n/// .await??;\n///\n/// // Return the result from the blocking task.\n/// Ok(result)\n///}\n///\n/// #[tokio::main]\n/// async fn main() -> Result<(), std::io::Error> {\n/// // Example: In-memory data.\n/// let data = b\"Hello, world!\"; // A byte slice.\n/// let reader = Cursor::new(data); // Create an in-memory AsyncRead.\n/// let result = process_sync_io(reader).await?;\n///\n/// // You can use `result` here as needed.\n///\n/// Ok(())\n/// }\n/// # }\n/// ```\n///\n#[derive(Debug)]\npub struct SyncIoBridge {\n src: T,\n rt: tokio::runtime::Handle,\n}\n\nimpl BufRead for SyncIoBridge {\n fn fill_buf(&mut self) -> std::io::Result<&[u8]> {\n let src = &mut self.src;\n self.rt.block_on(AsyncBufReadExt::fill_buf(src))\n }\n\n fn consume(&mut self, amt: usize) {\n let src = &mut self.src;\n AsyncBufReadExt::consume(src, amt)\n }\n\n fn read_until(&mut self, byte: u8, buf: &mut Vec) -> std::io::Result {\n let src = &mut self.src;\n self.rt\n .block_on(AsyncBufReadExt::read_until(src, byte, buf))\n }\n fn read_line(&mut self, buf: &mut String) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(AsyncBufReadExt::read_line(src, buf))\n }\n}\n\nimpl Read for SyncIoBridge {\n fn read(&mut self, buf: &mut [u8]) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(AsyncReadExt::read(src, buf))\n }\n\n fn read_to_end(&mut self, buf: &mut Vec) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(src.read_to_end(buf))\n }\n\n fn read_to_string(&mut self, buf: &mut String) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(src.read_to_string(buf))\n }\n\n fn read_exact(&mut self, buf: &mut [u8]) -> std::io::Result<()> {\n let src = &mut self.src;\n // The AsyncRead trait returns the count, synchronous doesn't.\n let _n = self.rt.block_on(src.read_exact(buf))?;\n Ok(())\n }\n}\n\nimpl Write for SyncIoBridge {\n fn write(&mut self, buf: &[u8]) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(src.write(buf))\n }\n\n fn flush(&mut self) -> std::io::Result<()> {\n let src = &mut self.src;\n self.rt.block_on(src.flush())\n }\n\n fn write_all(&mut self, buf: &[u8]) -> std::io::Result<()> {\n let src = &mut self.src;\n self.rt.block_on(src.write_all(buf))\n }\n\n fn write_vectored(&mut self, bufs: &[std::io::IoSlice<'_>]) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(src.write_vectored(bufs))\n }\n}\n\nimpl Seek for SyncIoBridge {\n fn seek(&mut self, pos: std::io::SeekFrom) -> std::io::Result {\n let src = &mut self.src;\n self.rt.block_on(AsyncSeekExt::seek(src, pos))\n }\n}\n\n// Because https://doc.rust-lang.org/std/io/trait.Write.html#method.is_write_vectored is at the time\n// of this writing still unstable, we expose this as part of a standalone method.\nimpl SyncIoBridge {\n /// Determines if the underlying [`tokio::io::AsyncWrite`] target supports efficient vectored writes.\n ///\n /// See [`tokio::io::AsyncWrite::is_write_vectored`].\n pub fn is_write_vectored(&self) -> bool {\n self.src.is_write_vectored()\n }\n}\n\nimpl SyncIoBridge {\n /// Shutdown this writer. This method provides a way to call the [`AsyncWriteExt::shutdown`]\n /// function of the inner [`tokio::io::AsyncWrite`] instance.\n ///\n /// # Errors\n ///\n /// This method returns the same errors as [`AsyncWriteExt::shutdown`].\n ///\n /// [`AsyncWriteExt::shutdown`]: tokio::io::AsyncWriteExt::shutdown\n pub fn shutdown(&mut self) -> std::io::Result<()> {\n let src = &mut self.src;\n self.rt.block_on(src.shutdown())\n }\n}\n\nimpl SyncIoBridge {\n /// Use a [`tokio::io::AsyncRead`] synchronously as a [`std::io::Read`] or\n /// a [`tokio::io::AsyncWrite`] as a [`std::io::Write`].\n ///\n /// When this struct is created, it captures a handle to the current thread's runtime with [`tokio::runtime::Handle::current`].\n /// It is hence OK to move this struct into a separate thread outside the runtime, as created\n /// by e.g. [`tokio::task::spawn_blocking`].\n ///\n /// Stated even more strongly: to make use of this bridge, you *must* move\n /// it into a separate thread outside the runtime. The synchronous I/O will use the\n /// underlying handle to block on the backing asynchronous source, via\n /// [`tokio::runtime::Handle::block_on`]. As noted in the documentation for that\n /// function, an attempt to `block_on` from an asynchronous execution context\n /// will panic.\n ///\n /// # Wrapping `!Unpin` types\n ///\n /// Use e.g. `SyncIoBridge::new(Box::pin(src))`.\n ///\n /// # Panics\n ///\n /// This will panic if called outside the context of a Tokio runtime.\n #[track_caller]\n pub fn new(src: T) -> Self {\n Self::new_with_handle(src, tokio::runtime::Handle::current())\n }\n\n /// Use a [`tokio::io::AsyncRead`] synchronously as a [`std::io::Read`] or\n /// a [`tokio::io::AsyncWrite`] as a [`std::io::Write`].\n ///\n /// This is the same as [`SyncIoBridge::new`], but allows passing an arbitrary handle and hence may\n /// be initially invoked outside of an asynchronous context.\n pub fn new_with_handle(src: T, rt: tokio::runtime::Handle) -> Self {\n Self { src, rt }\n }\n\n /// Consume this bridge, returning the underlying stream.\n pub fn into_inner(self) -> T {\n self.src\n }\n}\n\nimpl AsMut for SyncIoBridge {\n fn as_mut(&mut self) -> &mut T {\n &mut self.src\n }\n}\n\nimpl AsRef for SyncIoBridge {\n fn as_ref(&self) -> &T {\n &self.src\n }\n}\n" }, { "path": "tokio-util/src/io/write_all_vectored.rs", "content": "use tokio::io::AsyncWrite;\n\nuse pin_project_lite::pin_project;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse std::{future::Future, io::IoSlice};\nuse std::{io, mem};\n\npin_project! {\n /// A future that writes all data from multiple buffers to a writer.\n #[derive(Debug)]\n #[must_use = \"futures do nothing unless you `.await` or poll them\"]\n pub struct WriteAllVectored<'a, 'b, W: ?Sized> {\n writer: &'a mut W,\n bufs: &'a mut [IoSlice<'b>],\n // Make this future `!Unpin` for compatibility with async trait methods.\n #[pin]\n _pin: PhantomPinned,\n }\n}\n/// Like [`write_all`] but writes all data from multiple buffers into this writer.\n///\n/// This function writes multiple (possibly non-contiguous) buffers into the writer,\n/// using the `writev` syscall to potentially write in a single system call.\n///\n/// Equivalent to:\n///\n/// ```ignore\n/// async fn write_all_vectored(\n/// writer: &mut W,\n/// mut bufs: &mut [IoSlice<'_>]\n/// ) -> io::Result<()> {\n/// while !bufs.is_empty() {\n/// let n = write_vectored(writer, bufs).await?;\n/// if n == 0 {\n/// return Err(io::ErrorKind::WriteZero.into());\n/// }\n/// IoSlice::advance_slices(&mut bufs, n);\n/// }\n/// Ok(())\n/// }\n/// ```\n///\n/// # Cancel safety\n///\n/// This method is not cancellation safe. If it is used as the event\n/// in a `tokio::select!` statement and some other\n/// branch completes first, then the provided buffer may have been\n/// partially written, but future calls to `write_all_vectored` will\n/// have lost its place in the buffer.\n///\n/// # Examples\n///\n/// ```rust\n/// use tokio_util::io::write_all_vectored;\n/// use std::io::IoSlice;\n///\n/// #[tokio::main(flavor = \"current_thread\")]\n/// async fn main() -> std::io::Result<()> {\n///\n/// let mut writer = Vec::new();\n/// let bufs = &mut [\n/// IoSlice::new(&[1]),\n/// IoSlice::new(&[2, 3]),\n/// IoSlice::new(&[4, 5, 6]),\n/// ];\n///\n/// write_all_vectored(&mut writer, bufs).await?;\n///\n/// // Note: `bufs` has been modified by `IoSlice::advance_slices` and should not be reused.\n/// assert_eq!(writer, &[1, 2, 3, 4, 5, 6]);\n/// Ok(())\n/// }\n/// ```\n///\n/// # Notes\n///\n/// See the documentation for [`Write::write_all_vectored`] from std.\n/// After calling this function, the buffer slices may have\n/// been advanced and should not be reused.\n///\n/// [`Write::write_all_vectored`]: std::io::Write::write_all_vectored\n/// [`write_all`]: tokio::io::AsyncWriteExt::write_all\n/// [`writev`]: https://man7.org/linux/man-pages/man3/writev.3p.html\npub fn write_all_vectored<'a, 'b, W>(\n writer: &'a mut W,\n bufs: &'a mut [IoSlice<'b>],\n) -> WriteAllVectored<'a, 'b, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n WriteAllVectored {\n writer,\n bufs,\n _pin: PhantomPinned,\n }\n}\n\nimpl Future for WriteAllVectored<'_, '_, W>\nwhere\n W: AsyncWrite + Unpin + ?Sized,\n{\n type Output = io::Result<()>;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let me = self.project();\n while !me.bufs.is_empty() {\n // advance to first non-empty buffer\n let non_empty = match me.bufs.iter().position(|b| !b.is_empty()) {\n Some(pos) => pos,\n None => return Poll::Ready(Ok(())),\n };\n\n // drop empty buffers at the start\n *me.bufs = &mut mem::take(me.bufs)[non_empty..];\n\n let n = ready!(Pin::new(&mut *me.writer).poll_write_vectored(cx, me.bufs))?;\n if n == 0 {\n return Poll::Ready(Err(io::ErrorKind::WriteZero.into()));\n }\n self::advance_slices(me.bufs, n);\n }\n\n Poll::Ready(Ok(()))\n }\n}\n\n// copied from `std::IoSlice::advance_slices`\n// replace with method when MSRV is 1.81.0\nfn advance_slices<'a>(bufs: &mut &mut [IoSlice<'a>], n: usize) {\n // Number of buffers to remove.\n let mut remove = 0;\n // Remaining length before reaching n. This prevents overflow\n // that could happen if the length of slices in `bufs` were instead\n // accumulated. Those slice may be aliased and, if they are large\n // enough, their added length may overflow a `usize`.\n let mut left = n;\n for buf in bufs.iter() {\n if let Some(remainder) = left.checked_sub(buf.len()) {\n left = remainder;\n remove += 1;\n } else {\n break;\n }\n }\n\n *bufs = &mut std::mem::take(bufs)[remove..];\n if let Some(first) = bufs.first_mut() {\n let buf = &first[..left];\n // necessary due to limitating in the borrow checker,\n // when tokio MSRV reaches 1.81.0 this entire function\n // can be replaced with `IoSlice::advance_slices`\n //\n // SAFETY: transmute a sub-slice of an IoSlice<'a> back to\n // the lifetime `'a`. This is safe because the underlying memory\n // is guaranteed to live for 'a, we have shared access, and no\n // underlying data is reinterpreted to a different type.\n unsafe {\n *first = IoSlice::new(std::mem::transmute::<&[u8], &'a [u8]>(buf));\n }\n } else {\n assert!(left == 0, \"advancing io slices beyond their length\");\n }\n}\n" }, { "path": "tokio-util/src/lib.rs", "content": "#![allow(clippy::needless_doctest_main)]\n#![warn(\n missing_debug_implementations,\n missing_docs,\n rust_2018_idioms,\n unreachable_pub\n)]\n#![doc(test(\n no_crate_inject,\n attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))\n))]\n#![cfg_attr(docsrs, feature(doc_cfg))]\n\n//! Utilities for working with Tokio.\n//!\n//! This crate is not versioned in lockstep with the core\n//! [`tokio`] crate. However, `tokio-util` _will_ respect Rust's\n//! semantic versioning policy, especially with regard to breaking changes.\n\n#[macro_use]\nmod cfg;\n\nmod loom;\n\ncfg_codec! {\n #[macro_use]\n mod tracing;\n\n pub mod codec;\n}\n\ncfg_net! {\n #[cfg(not(target_arch = \"wasm32\"))]\n pub mod udp;\n pub mod net;\n}\n\ncfg_compat! {\n pub mod compat;\n}\n\ncfg_io! {\n pub mod io;\n}\n\ncfg_rt! {\n pub mod context;\n}\n\n#[cfg(feature = \"rt\")]\npub mod task;\n\ncfg_time! {\n pub mod time;\n}\n\npub mod sync;\n\npub mod either;\n\npub use bytes;\n\nmod util;\n\npub mod future;\n" }, { "path": "tokio-util/src/loom.rs", "content": "//! This module abstracts over `loom` and `std::sync` types depending on whether we\n//! are running loom tests or not.\n\npub(crate) mod sync {\n #[cfg(all(test, loom))]\n pub(crate) use loom::sync::{Arc, Mutex, MutexGuard};\n #[cfg(not(all(test, loom)))]\n pub(crate) use std::sync::{Arc, Mutex, MutexGuard};\n}\n" }, { "path": "tokio-util/src/net/mod.rs", "content": "#![cfg(not(loom))]\n\n//! TCP/UDP/Unix helpers for tokio.\n\nuse crate::either::Either;\nuse std::future::Future;\nuse std::io::Result;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[cfg(unix)]\npub mod unix;\n\n/// A trait for a listener: `TcpListener` and `UnixListener`.\npub trait Listener {\n /// The stream's type of this listener.\n type Io: tokio::io::AsyncRead + tokio::io::AsyncWrite;\n /// The socket address type of this listener.\n type Addr;\n\n /// Polls to accept a new incoming connection to this listener.\n fn poll_accept(&mut self, cx: &mut Context<'_>) -> Poll>;\n\n /// Accepts a new incoming connection from this listener.\n fn accept(&mut self) -> ListenerAcceptFut<'_, Self>\n where\n Self: Sized,\n {\n ListenerAcceptFut { listener: self }\n }\n\n /// Returns the local address that this listener is bound to.\n fn local_addr(&self) -> Result;\n}\n\nimpl Listener for tokio::net::TcpListener {\n type Io = tokio::net::TcpStream;\n type Addr = std::net::SocketAddr;\n\n fn poll_accept(&mut self, cx: &mut Context<'_>) -> Poll> {\n Self::poll_accept(self, cx)\n }\n\n fn local_addr(&self) -> Result {\n self.local_addr()\n }\n}\n\n/// Future for accepting a new connection from a listener.\n#[derive(Debug)]\n#[must_use = \"futures do nothing unless you `.await` or poll them\"]\npub struct ListenerAcceptFut<'a, L> {\n listener: &'a mut L,\n}\n\nimpl<'a, L> Future for ListenerAcceptFut<'a, L>\nwhere\n L: Listener,\n{\n type Output = Result<(L::Io, L::Addr)>;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n self.listener.poll_accept(cx)\n }\n}\n\nimpl Either\nwhere\n L: Listener,\n R: Listener,\n{\n /// Accepts a new incoming connection from this listener.\n pub async fn accept(&mut self) -> Result> {\n match self {\n Either::Left(listener) => {\n let (stream, addr) = listener.accept().await?;\n Ok(Either::Left((stream, addr)))\n }\n Either::Right(listener) => {\n let (stream, addr) = listener.accept().await?;\n Ok(Either::Right((stream, addr)))\n }\n }\n }\n\n /// Returns the local address that this listener is bound to.\n pub fn local_addr(&self) -> Result> {\n match self {\n Either::Left(listener) => {\n let addr = listener.local_addr()?;\n Ok(Either::Left(addr))\n }\n Either::Right(listener) => {\n let addr = listener.local_addr()?;\n Ok(Either::Right(addr))\n }\n }\n }\n}\n" }, { "path": "tokio-util/src/net/unix/mod.rs", "content": "//! Unix domain socket helpers.\n\nuse super::Listener;\nuse std::io::Result;\nuse std::task::{Context, Poll};\n\nimpl Listener for tokio::net::UnixListener {\n type Io = tokio::net::UnixStream;\n type Addr = tokio::net::unix::SocketAddr;\n\n fn poll_accept(&mut self, cx: &mut Context<'_>) -> Poll> {\n Self::poll_accept(self, cx)\n }\n\n fn local_addr(&self) -> Result {\n self.local_addr()\n }\n}\n" }, { "path": "tokio-util/src/sync/cancellation_token/guard.rs", "content": "use crate::sync::CancellationToken;\n\n/// A wrapper for cancellation token which automatically cancels\n/// it on drop. It is created using [`drop_guard`] method on the [`CancellationToken`].\n///\n/// [`drop_guard`]: CancellationToken::drop_guard\n#[derive(Debug)]\npub struct DropGuard {\n pub(super) inner: Option,\n}\n\nimpl DropGuard {\n /// Returns stored cancellation token and removes this drop guard instance\n /// (i.e. it will no longer cancel token). Other guards for this token\n /// are not affected.\n pub fn disarm(mut self) -> CancellationToken {\n self.inner\n .take()\n .expect(\"`inner` can be only None in a destructor\")\n }\n}\n\nimpl Drop for DropGuard {\n fn drop(&mut self) {\n if let Some(inner) = &self.inner {\n inner.cancel();\n }\n }\n}\n" }, { "path": "tokio-util/src/sync/cancellation_token/guard_ref.rs", "content": "use crate::sync::CancellationToken;\n\n/// A wrapper for cancellation token which automatically cancels\n/// it on drop. It is created using [`drop_guard_ref`] method on the [`CancellationToken`].\n///\n/// This is a borrowed version of [`DropGuard`].\n///\n/// [`drop_guard_ref`]: CancellationToken::drop_guard_ref\n/// [`DropGuard`]: super::DropGuard\n#[derive(Debug)]\npub struct DropGuardRef<'a> {\n pub(super) inner: Option<&'a CancellationToken>,\n}\n\nimpl<'a> DropGuardRef<'a> {\n /// Returns stored cancellation token and removes this drop guard instance\n /// (i.e. it will no longer cancel token). Other guards for this token\n /// are not affected.\n pub fn disarm(mut self) -> &'a CancellationToken {\n self.inner\n .take()\n .expect(\"`inner` can be only None in a destructor\")\n }\n}\n\nimpl Drop for DropGuardRef<'_> {\n fn drop(&mut self) {\n if let Some(inner) = self.inner {\n inner.cancel();\n }\n }\n}\n" }, { "path": "tokio-util/src/sync/cancellation_token/tree_node.rs", "content": "//! This mod provides the logic for the inner tree structure of the `CancellationToken`.\n//!\n//! `CancellationTokens` are only light handles with references to [`TreeNode`].\n//! All the logic is actually implemented in the [`TreeNode`].\n//!\n//! A [`TreeNode`] is part of the cancellation tree and may have one parent and an arbitrary number of\n//! children.\n//!\n//! A [`TreeNode`] can receive the request to perform a cancellation through a `CancellationToken`.\n//! This cancellation request will cancel the node and all of its descendants.\n//!\n//! As soon as a node cannot get cancelled any more (because it was already cancelled or it has no\n//! more `CancellationTokens` pointing to it any more), it gets removed from the tree, to keep the\n//! tree as small as possible.\n//!\n//! # Invariants\n//!\n//! Those invariants shall be true at any time.\n//!\n//! 1. A node that has no parents and no handles can no longer be cancelled.\n//! This is important during both cancellation and refcounting.\n//!\n//! 2. If node B *is* or *was* a child of node A, then node B was created *after* node A.\n//! This is important for deadlock safety, as it is used for lock order.\n//! Node B can only become the child of node A in two ways:\n//! - being created with `child_node()`, in which case it is trivially true that\n//! node A already existed when node B was created\n//! - being moved A->C->B to A->B because node C was removed in `decrease_handle_refcount()`\n//! or `cancel()`. In this case the invariant still holds, as B was younger than C, and C\n//! was younger than A, therefore B is also younger than A.\n//!\n//! 3. If two nodes are both unlocked and node A is the parent of node B, then node B is a child of\n//! node A. It is important to always restore that invariant before dropping the lock of a node.\n//!\n//! # Deadlock safety\n//!\n//! We always lock in the order of creation time. We can prove this through invariant #2.\n//! Specifically, through invariant #2, we know that we always have to lock a parent\n//! before its child.\n//!\nuse crate::loom::sync::{Arc, Mutex, MutexGuard};\n\n/// A node of the cancellation tree structure\n///\n/// The actual data it holds is wrapped inside a mutex for synchronization.\npub(crate) struct TreeNode {\n inner: Mutex,\n waker: tokio::sync::Notify,\n}\nimpl TreeNode {\n pub(crate) fn new() -> Self {\n Self {\n inner: Mutex::new(Inner {\n parent: None,\n parent_idx: 0,\n children: vec![],\n is_cancelled: false,\n num_handles: 1,\n }),\n waker: tokio::sync::Notify::new(),\n }\n }\n\n pub(crate) fn notified(&self) -> tokio::sync::futures::Notified<'_> {\n self.waker.notified()\n }\n}\n\n/// The data contained inside a `TreeNode`.\n///\n/// This struct exists so that the data of the node can be wrapped\n/// in a Mutex.\nstruct Inner {\n parent: Option>,\n parent_idx: usize,\n children: Vec>,\n is_cancelled: bool,\n num_handles: usize,\n}\n\n/// Returns whether or not the node is cancelled\npub(crate) fn is_cancelled(node: &Arc) -> bool {\n node.inner.lock().unwrap().is_cancelled\n}\n\n/// Creates a child node\npub(crate) fn child_node(parent: &Arc) -> Arc {\n let mut locked_parent = parent.inner.lock().unwrap();\n\n // Do not register as child if we are already cancelled.\n // Cancelled trees can never be uncancelled and therefore\n // need no connection to parents or children any more.\n if locked_parent.is_cancelled {\n return Arc::new(TreeNode {\n inner: Mutex::new(Inner {\n parent: None,\n parent_idx: 0,\n children: vec![],\n is_cancelled: true,\n num_handles: 1,\n }),\n waker: tokio::sync::Notify::new(),\n });\n }\n\n let child = Arc::new(TreeNode {\n inner: Mutex::new(Inner {\n parent: Some(parent.clone()),\n parent_idx: locked_parent.children.len(),\n children: vec![],\n is_cancelled: false,\n num_handles: 1,\n }),\n waker: tokio::sync::Notify::new(),\n });\n\n locked_parent.children.push(child.clone());\n\n child\n}\n\n/// Disconnects the given parent from all of its children.\n///\n/// Takes a reference to [Inner] to make sure the parent is already locked.\nfn disconnect_children(node: &mut Inner) {\n for child in std::mem::take(&mut node.children) {\n let mut locked_child = child.inner.lock().unwrap();\n locked_child.parent_idx = 0;\n locked_child.parent = None;\n }\n}\n\n/// Figures out the parent of the node and locks the node and its parent atomically.\n///\n/// The basic principle of preventing deadlocks in the tree is\n/// that we always lock the parent first, and then the child.\n/// For more info look at *deadlock safety* and *invariant #2*.\n///\n/// Sadly, it's impossible to figure out the parent of a node without\n/// locking it. To then achieve locking order consistency, the node\n/// has to be unlocked before the parent gets locked.\n/// This leaves a small window where we already assume that we know the parent,\n/// but neither the parent nor the node is locked. Therefore, the parent could change.\n///\n/// To prevent that this problem leaks into the rest of the code, it is abstracted\n/// in this function.\n///\n/// The locked child and optionally its locked parent, if a parent exists, get passed\n/// to the `func` argument via (node, None) or (node, Some(parent)).\nfn with_locked_node_and_parent(node: &Arc, func: F) -> Ret\nwhere\n F: FnOnce(MutexGuard<'_, Inner>, Option>) -> Ret,\n{\n use std::sync::TryLockError;\n\n let mut locked_node = node.inner.lock().unwrap();\n\n // Every time this fails, the number of ancestors of the node decreases,\n // so the loop must succeed after a finite number of iterations.\n loop {\n // Look up the parent of the currently locked node.\n let potential_parent = match locked_node.parent.as_ref() {\n Some(potential_parent) => potential_parent.clone(),\n None => return func(locked_node, None),\n };\n\n // Lock the parent. This may require unlocking the child first.\n let locked_parent = match potential_parent.inner.try_lock() {\n Ok(locked_parent) => locked_parent,\n Err(TryLockError::WouldBlock) => {\n drop(locked_node);\n // Deadlock safety:\n //\n // Due to invariant #2, the potential parent must come before\n // the child in the creation order. Therefore, we can safely\n // lock the child while holding the parent lock.\n let locked_parent = potential_parent.inner.lock().unwrap();\n locked_node = node.inner.lock().unwrap();\n locked_parent\n }\n // https://github.com/tokio-rs/tokio/pull/6273#discussion_r1443752911\n #[allow(clippy::unnecessary_literal_unwrap)]\n Err(TryLockError::Poisoned(err)) => Err(err).unwrap(),\n };\n\n // If we unlocked the child, then the parent may have changed. Check\n // that we still have the right parent.\n if let Some(actual_parent) = locked_node.parent.as_ref() {\n if Arc::ptr_eq(actual_parent, &potential_parent) {\n return func(locked_node, Some(locked_parent));\n }\n }\n }\n}\n\n/// Moves all children from `node` to `parent`.\n///\n/// `parent` MUST have been a parent of the node when they both got locked,\n/// otherwise there is a potential for a deadlock as invariant #2 would be violated.\n///\n/// To acquire the locks for node and parent, use [`with_locked_node_and_parent`].\nfn move_children_to_parent(node: &mut Inner, parent: &mut Inner) {\n // Pre-allocate in the parent, for performance\n parent.children.reserve(node.children.len());\n\n for child in std::mem::take(&mut node.children) {\n {\n let mut child_locked = child.inner.lock().unwrap();\n child_locked.parent.clone_from(&node.parent);\n child_locked.parent_idx = parent.children.len();\n }\n parent.children.push(child);\n }\n}\n\n/// Removes a child from the parent.\n///\n/// `parent` MUST be the parent of `node`.\n/// To acquire the locks for node and parent, use [`with_locked_node_and_parent`].\nfn remove_child(parent: &mut Inner, mut node: MutexGuard<'_, Inner>) {\n // Query the position from where to remove a node\n let pos = node.parent_idx;\n node.parent = None;\n node.parent_idx = 0;\n\n // Unlock node, so that only one child at a time is locked.\n // Otherwise we would violate the lock order (see 'deadlock safety') as we\n // don't know the creation order of the child nodes\n drop(node);\n\n // If `node` is the last element in the list, we don't need any swapping\n if parent.children.len() == pos + 1 {\n parent.children.pop().unwrap();\n } else {\n // If `node` is not the last element in the list, we need to\n // replace it with the last element\n let replacement_child = parent.children.pop().unwrap();\n replacement_child.inner.lock().unwrap().parent_idx = pos;\n parent.children[pos] = replacement_child;\n }\n\n let len = parent.children.len();\n if 4 * len <= parent.children.capacity() {\n parent.children.shrink_to(2 * len);\n }\n}\n\n/// Increases the reference count of handles.\npub(crate) fn increase_handle_refcount(node: &Arc) {\n let mut locked_node = node.inner.lock().unwrap();\n\n // Once no handles are left over, the node gets detached from the tree.\n // There should never be a new handle once all handles are dropped.\n assert!(locked_node.num_handles > 0);\n\n locked_node.num_handles += 1;\n}\n\n/// Decreases the reference count of handles.\n///\n/// Once no handle is left, we can remove the node from the\n/// tree and connect its parent directly to its children.\npub(crate) fn decrease_handle_refcount(node: &Arc) {\n let num_handles = {\n let mut locked_node = node.inner.lock().unwrap();\n locked_node.num_handles -= 1;\n locked_node.num_handles\n };\n\n if num_handles == 0 {\n with_locked_node_and_parent(node, |mut node, parent| {\n // Remove the node from the tree\n match parent {\n Some(mut parent) => {\n // As we want to remove ourselves from the tree,\n // we have to move the children to the parent, so that\n // they still receive the cancellation event without us.\n // Moving them does not violate invariant #1.\n move_children_to_parent(&mut node, &mut parent);\n\n // Remove the node from the parent\n remove_child(&mut parent, node);\n }\n None => {\n // Due to invariant #1, we can assume that our\n // children can no longer be cancelled through us.\n // (as we now have neither a parent nor handles)\n // Therefore we can disconnect them.\n disconnect_children(&mut node);\n }\n }\n });\n }\n}\n\n/// Cancels a node and its children.\npub(crate) fn cancel(node: &Arc) {\n let mut locked_node = node.inner.lock().unwrap();\n\n if locked_node.is_cancelled {\n return;\n }\n\n // One by one, adopt grandchildren and then cancel and detach the child\n while let Some(child) = locked_node.children.pop() {\n // This can't deadlock because the mutex we are already\n // holding is the parent of child.\n let mut locked_child = child.inner.lock().unwrap();\n\n // Detach the child from node\n // No need to modify node.children, as the child already got removed with `.pop`\n locked_child.parent = None;\n locked_child.parent_idx = 0;\n\n // If child is already cancelled, detaching is enough\n if locked_child.is_cancelled {\n continue;\n }\n\n // Cancel or adopt grandchildren\n while let Some(grandchild) = locked_child.children.pop() {\n // This can't deadlock because the two mutexes we are already\n // holding is the parent and grandparent of grandchild.\n let mut locked_grandchild = grandchild.inner.lock().unwrap();\n\n // Detach the grandchild\n locked_grandchild.parent = None;\n locked_grandchild.parent_idx = 0;\n\n // If grandchild is already cancelled, detaching is enough\n if locked_grandchild.is_cancelled {\n continue;\n }\n\n // For performance reasons, only adopt grandchildren that have children.\n // Otherwise, just cancel them right away, no need for another iteration.\n if locked_grandchild.children.is_empty() {\n // Cancel the grandchild\n locked_grandchild.is_cancelled = true;\n locked_grandchild.children = Vec::new();\n drop(locked_grandchild);\n grandchild.waker.notify_waiters();\n } else {\n // Otherwise, adopt grandchild\n locked_grandchild.parent = Some(node.clone());\n locked_grandchild.parent_idx = locked_node.children.len();\n drop(locked_grandchild);\n locked_node.children.push(grandchild);\n }\n }\n\n // Cancel the child\n locked_child.is_cancelled = true;\n locked_child.children = Vec::new();\n drop(locked_child);\n child.waker.notify_waiters();\n\n // Now the child is cancelled and detached and all its children are adopted.\n // Just continue until all (including adopted) children are cancelled and detached.\n }\n\n // Cancel the node itself.\n locked_node.is_cancelled = true;\n locked_node.children = Vec::new();\n drop(locked_node);\n node.waker.notify_waiters();\n}\n" }, { "path": "tokio-util/src/sync/cancellation_token.rs", "content": "//! An asynchronously awaitable [`CancellationToken`].\n//! The token allows to signal a cancellation request to one or more tasks.\npub(crate) mod guard;\npub(crate) mod guard_ref;\nmod tree_node;\n\nuse crate::loom::sync::Arc;\nuse crate::util::MaybeDangling;\nuse core::future::Future;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\nuse guard::DropGuard;\nuse guard_ref::DropGuardRef;\nuse pin_project_lite::pin_project;\n\n/// A token which can be used to signal a cancellation request to one or more\n/// tasks.\n///\n/// Tasks can call [`CancellationToken::cancelled()`] in order to\n/// obtain a Future which will be resolved when cancellation is requested.\n///\n/// Cancellation can be requested through the [`CancellationToken::cancel`] method.\n///\n/// # Examples\n///\n/// ```no_run\n/// use tokio::select;\n/// use tokio_util::sync::CancellationToken;\n///\n/// #[tokio::main]\n/// async fn main() {\n/// let token = CancellationToken::new();\n/// let cloned_token = token.clone();\n///\n/// let join_handle = tokio::spawn(async move {\n/// // Wait for either cancellation or a very long time\n/// select! {\n/// _ = cloned_token.cancelled() => {\n/// // The token was cancelled\n/// 5\n/// }\n/// _ = tokio::time::sleep(std::time::Duration::from_secs(9999)) => {\n/// 99\n/// }\n/// }\n/// });\n///\n/// tokio::spawn(async move {\n/// tokio::time::sleep(std::time::Duration::from_millis(10)).await;\n/// token.cancel();\n/// });\n///\n/// assert_eq!(5, join_handle.await.unwrap());\n/// }\n/// ```\npub struct CancellationToken {\n inner: Arc,\n}\n\nimpl std::panic::UnwindSafe for CancellationToken {}\nimpl std::panic::RefUnwindSafe for CancellationToken {}\n\npin_project! {\n /// A Future that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled.\n #[must_use = \"futures do nothing unless polled\"]\n pub struct WaitForCancellationFuture<'a> {\n cancellation_token: &'a CancellationToken,\n #[pin]\n future: tokio::sync::futures::Notified<'a>,\n }\n}\n\npin_project! {\n /// A Future that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled.\n ///\n /// This is the counterpart to [`WaitForCancellationFuture`] that takes\n /// [`CancellationToken`] by value instead of using a reference.\n #[must_use = \"futures do nothing unless polled\"]\n pub struct WaitForCancellationFutureOwned {\n // This field internally has a reference to the cancellation token, but camouflages\n // the relationship with `'static`. To avoid Undefined Behavior, we must ensure\n // that the reference is only used while the cancellation token is still alive. To\n // do that, we ensure that the future is the first field, so that it is dropped\n // before the cancellation token.\n //\n // We use `MaybeDanglingFuture` here because without it, the compiler could assert\n // the reference inside `future` to be valid even after the destructor of that\n // field runs. (Specifically, when the `WaitForCancellationFutureOwned` is passed\n // as an argument to a function, the reference can be asserted to be valid for the\n // rest of that function.) To avoid that, we use `MaybeDangling` which tells the\n // compiler that the reference stored inside it might not be valid.\n //\n // See \n // for more info.\n #[pin]\n future: MaybeDangling>,\n cancellation_token: CancellationToken,\n }\n}\n\n// ===== impl CancellationToken =====\n\nimpl core::fmt::Debug for CancellationToken {\n fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {\n f.debug_struct(\"CancellationToken\")\n .field(\"is_cancelled\", &self.is_cancelled())\n .finish()\n }\n}\n\nimpl Clone for CancellationToken {\n /// Creates a clone of the [`CancellationToken`] which will get cancelled\n /// whenever the current token gets cancelled, and vice versa.\n fn clone(&self) -> Self {\n tree_node::increase_handle_refcount(&self.inner);\n CancellationToken {\n inner: self.inner.clone(),\n }\n }\n}\n\nimpl Drop for CancellationToken {\n fn drop(&mut self) {\n tree_node::decrease_handle_refcount(&self.inner);\n }\n}\n\nimpl Default for CancellationToken {\n fn default() -> CancellationToken {\n CancellationToken::new()\n }\n}\n\nimpl CancellationToken {\n /// Creates a new [`CancellationToken`] in the non-cancelled state.\n pub fn new() -> CancellationToken {\n CancellationToken {\n inner: Arc::new(tree_node::TreeNode::new()),\n }\n }\n\n /// Creates a [`CancellationToken`] which will get cancelled whenever the\n /// current token gets cancelled. Unlike a cloned [`CancellationToken`],\n /// cancelling a child token does not cancel the parent token.\n ///\n /// If the current token is already cancelled, the child token will get\n /// returned in cancelled state.\n ///\n /// # Examples\n ///\n /// ```no_run\n /// use tokio::select;\n /// use tokio_util::sync::CancellationToken;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// let token = CancellationToken::new();\n /// let child_token = token.child_token();\n ///\n /// let join_handle = tokio::spawn(async move {\n /// // Wait for either cancellation or a very long time\n /// select! {\n /// _ = child_token.cancelled() => {\n /// // The token was cancelled\n /// 5\n /// }\n /// _ = tokio::time::sleep(std::time::Duration::from_secs(9999)) => {\n /// 99\n /// }\n /// }\n /// });\n ///\n /// tokio::spawn(async move {\n /// tokio::time::sleep(std::time::Duration::from_millis(10)).await;\n /// token.cancel();\n /// });\n ///\n /// assert_eq!(5, join_handle.await.unwrap());\n /// }\n /// ```\n pub fn child_token(&self) -> CancellationToken {\n CancellationToken {\n inner: tree_node::child_node(&self.inner),\n }\n }\n\n /// Cancel the [`CancellationToken`] and all child tokens which had been\n /// derived from it.\n ///\n /// This will wake up all tasks which are waiting for cancellation.\n ///\n /// Be aware that cancellation is not an atomic operation. It is possible\n /// for another thread running in parallel with a call to `cancel` to first\n /// receive `true` from `is_cancelled` on one child node, and then receive\n /// `false` from `is_cancelled` on another child node. However, once the\n /// call to `cancel` returns, all child nodes have been fully cancelled.\n pub fn cancel(&self) {\n tree_node::cancel(&self.inner);\n }\n\n /// Returns `true` if the `CancellationToken` is cancelled.\n pub fn is_cancelled(&self) -> bool {\n tree_node::is_cancelled(&self.inner)\n }\n\n /// Returns a [`Future`] that gets fulfilled when cancellation is requested.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn cancelled(&self);\n /// ```\n ///\n /// The future will complete immediately if the token is already cancelled\n /// when this method is called.\n ///\n /// # Cancellation safety\n ///\n /// This method is cancel safe.\n pub fn cancelled(&self) -> WaitForCancellationFuture<'_> {\n WaitForCancellationFuture {\n cancellation_token: self,\n future: self.inner.notified(),\n }\n }\n\n /// Returns a [`Future`] that gets fulfilled when cancellation is requested.\n ///\n /// Equivalent to:\n ///\n /// ```ignore\n /// async fn cancelled_owned(self);\n /// ```\n ///\n /// The future will complete immediately if the token is already cancelled\n /// when this method is called.\n ///\n /// The function takes self by value and returns a future that owns the\n /// token.\n ///\n /// # Cancellation safety\n ///\n /// This method is cancel safe.\n pub fn cancelled_owned(self) -> WaitForCancellationFutureOwned {\n WaitForCancellationFutureOwned::new(self)\n }\n\n /// Creates a [`DropGuard`] for this token.\n ///\n /// Returned guard will cancel this token (and all its children) on drop\n /// unless disarmed.\n pub fn drop_guard(self) -> DropGuard {\n DropGuard { inner: Some(self) }\n }\n\n /// Creates a [`DropGuardRef`] for this token.\n ///\n /// Returned guard will cancel this token (and all its children) on drop\n /// unless disarmed.\n pub fn drop_guard_ref(&self) -> DropGuardRef<'_> {\n DropGuardRef { inner: Some(self) }\n }\n\n /// Runs a future to completion and returns its result wrapped inside of an `Option`\n /// unless the [`CancellationToken`] is cancelled. In that case the function returns\n /// `None` and the future gets dropped.\n ///\n /// # Fairness\n ///\n /// Calling this on an already-cancelled token directly returns `None`.\n /// For all subsequent polls, in case of concurrent completion and\n /// cancellation, this is biased towards the future completion.\n ///\n /// # Cancellation safety\n ///\n /// This method is only cancel safe if `fut` is cancel safe.\n pub async fn run_until_cancelled(&self, fut: F) -> Option\n where\n F: Future,\n {\n if self.is_cancelled() {\n None\n } else {\n RunUntilCancelledFuture {\n cancellation: self.cancelled(),\n future: fut,\n }\n .await\n }\n }\n\n /// Runs a future to completion and returns its result wrapped inside of an `Option`\n /// unless the [`CancellationToken`] is cancelled. In that case the function returns\n /// `None` and the future gets dropped.\n ///\n /// The function takes self by value and returns a future that owns the token.\n ///\n /// # Fairness\n ///\n /// Calling this on an already-cancelled token directly returns `None`.\n /// For all subsequent polls, in case of concurrent completion and\n /// cancellation, this is biased towards the future completion.\n ///\n /// # Cancellation safety\n ///\n /// This method is only cancel safe if `fut` is cancel safe.\n pub async fn run_until_cancelled_owned(self, fut: F) -> Option\n where\n F: Future,\n {\n self.run_until_cancelled(fut).await\n }\n}\n\n// ===== impl WaitForCancellationFuture =====\n\nimpl<'a> core::fmt::Debug for WaitForCancellationFuture<'a> {\n fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {\n f.debug_struct(\"WaitForCancellationFuture\").finish()\n }\n}\n\nimpl<'a> Future for WaitForCancellationFuture<'a> {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n let mut this = self.project();\n loop {\n if this.cancellation_token.is_cancelled() {\n return Poll::Ready(());\n }\n\n // No wakeups can be lost here because there is always a call to\n // `is_cancelled` between the creation of the future and the call to\n // `poll`, and the code that sets the cancelled flag does so before\n // waking the `Notified`.\n if this.future.as_mut().poll(cx).is_pending() {\n return Poll::Pending;\n }\n\n this.future.set(this.cancellation_token.inner.notified());\n }\n }\n}\n\n// ===== impl WaitForCancellationFutureOwned =====\n\nimpl core::fmt::Debug for WaitForCancellationFutureOwned {\n fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {\n f.debug_struct(\"WaitForCancellationFutureOwned\").finish()\n }\n}\n\nimpl WaitForCancellationFutureOwned {\n fn new(cancellation_token: CancellationToken) -> Self {\n WaitForCancellationFutureOwned {\n // cancellation_token holds a heap allocation and is guaranteed to have a\n // stable deref, thus it would be ok to move the cancellation_token while\n // the future holds a reference to it.\n //\n // # Safety\n //\n // cancellation_token is dropped after future due to the field ordering.\n future: MaybeDangling::new(unsafe { Self::new_future(&cancellation_token) }),\n cancellation_token,\n }\n }\n\n /// # Safety\n /// The returned future must be destroyed before the cancellation token is\n /// destroyed.\n unsafe fn new_future(\n cancellation_token: &CancellationToken,\n ) -> tokio::sync::futures::Notified<'static> {\n let inner_ptr = Arc::as_ptr(&cancellation_token.inner);\n // SAFETY: The `Arc::as_ptr` method guarantees that `inner_ptr` remains\n // valid until the strong count of the Arc drops to zero, and the caller\n // guarantees that they will drop the future before that happens.\n (*inner_ptr).notified()\n }\n}\n\nimpl Future for WaitForCancellationFutureOwned {\n type Output = ();\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n let mut this = self.project();\n\n loop {\n if this.cancellation_token.is_cancelled() {\n return Poll::Ready(());\n }\n\n // No wakeups can be lost here because there is always a call to\n // `is_cancelled` between the creation of the future and the call to\n // `poll`, and the code that sets the cancelled flag does so before\n // waking the `Notified`.\n if this.future.as_mut().poll(cx).is_pending() {\n return Poll::Pending;\n }\n\n // # Safety\n //\n // cancellation_token is dropped after future due to the field ordering.\n this.future.set(MaybeDangling::new(unsafe {\n Self::new_future(this.cancellation_token)\n }));\n }\n }\n}\n\npin_project! {\n /// A Future that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled or a given Future gets resolved. It is biased towards the\n /// Future completion.\n #[must_use = \"futures do nothing unless polled\"]\n pub(crate) struct RunUntilCancelledFuture<'a, F: Future> {\n #[pin]\n cancellation: WaitForCancellationFuture<'a>,\n #[pin]\n future: F,\n }\n}\n\nimpl<'a, F: Future> RunUntilCancelledFuture<'a, F> {\n pub(crate) fn new(cancellation_token: &'a CancellationToken, future: F) -> Self {\n Self {\n cancellation: cancellation_token.cancelled(),\n future,\n }\n }\n}\n\nimpl<'a, F: Future> Future for RunUntilCancelledFuture<'a, F> {\n type Output = Option;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n if let Poll::Ready(res) = this.future.poll(cx) {\n Poll::Ready(Some(res))\n } else if this.cancellation.poll(cx).is_ready() {\n Poll::Ready(None)\n } else {\n Poll::Pending\n }\n }\n}\n\npin_project! {\n /// A Future that is resolved once the corresponding [`CancellationToken`]\n /// is cancelled or a given Future gets resolved. It is biased towards the\n /// Future completion.\n #[must_use = \"futures do nothing unless polled\"]\n pub(crate) struct RunUntilCancelledFutureOwned {\n #[pin]\n cancellation: WaitForCancellationFutureOwned,\n #[pin]\n future: F,\n }\n}\n\nimpl Future for RunUntilCancelledFutureOwned {\n type Output = Option;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n let this = self.project();\n if let Poll::Ready(res) = this.future.poll(cx) {\n Poll::Ready(Some(res))\n } else if this.cancellation.poll(cx).is_ready() {\n Poll::Ready(None)\n } else {\n Poll::Pending\n }\n }\n}\n\nimpl RunUntilCancelledFutureOwned {\n pub(crate) fn new(cancellation_token: CancellationToken, future: F) -> Self {\n Self {\n cancellation: cancellation_token.cancelled_owned(),\n future,\n }\n }\n}\n" }, { "path": "tokio-util/src/sync/mod.rs", "content": "//! Synchronization primitives\n\nmod cancellation_token;\npub use cancellation_token::{\n guard::DropGuard, guard_ref::DropGuardRef, CancellationToken, WaitForCancellationFuture,\n WaitForCancellationFutureOwned,\n};\npub(crate) use cancellation_token::{RunUntilCancelledFuture, RunUntilCancelledFutureOwned};\n\nmod mpsc;\npub use mpsc::{PollSendError, PollSender};\n\nmod poll_semaphore;\npub use poll_semaphore::PollSemaphore;\n\nmod reusable_box;\npub use reusable_box::ReusableBoxFuture;\n\n#[cfg(test)]\nmod tests;\n" }, { "path": "tokio-util/src/sync/mpsc.rs", "content": "use futures_sink::Sink;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse std::{fmt, mem};\nuse tokio::sync::mpsc::OwnedPermit;\nuse tokio::sync::mpsc::Sender;\n\nuse super::ReusableBoxFuture;\n\n/// Error returned by the `PollSender` when the channel is closed.\n#[derive(Debug)]\npub struct PollSendError(Option);\n\nimpl PollSendError {\n /// Consumes the stored value, if any.\n ///\n /// If this error was encountered when calling `start_send`/`send_item`, this will be the item\n /// that the caller attempted to send. Otherwise, it will be `None`.\n pub fn into_inner(self) -> Option {\n self.0\n }\n}\n\nimpl fmt::Display for PollSendError {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n write!(fmt, \"channel closed\")\n }\n}\n\nimpl std::error::Error for PollSendError {}\n\n#[derive(Debug)]\nenum State {\n Idle(Sender),\n Acquiring,\n ReadyToSend(OwnedPermit),\n Closed,\n}\n\n/// A wrapper around [`mpsc::Sender`] that can be polled.\n///\n/// [`mpsc::Sender`]: tokio::sync::mpsc::Sender\n#[derive(Debug)]\npub struct PollSender {\n sender: Option>,\n state: State,\n acquire: PollSenderFuture,\n}\n\n// Creates a future for acquiring a permit from the underlying channel. This is used to ensure\n// there's capacity for a send to complete.\n//\n// By reusing the same async fn for both `Some` and `None`, we make sure every future passed to\n// ReusableBoxFuture has the same underlying type, and hence the same size and alignment.\nasync fn make_acquire_future(\n data: Option>,\n) -> Result, PollSendError> {\n match data {\n Some(sender) => sender\n .reserve_owned()\n .await\n .map_err(|_| PollSendError(None)),\n None => unreachable!(\"this future should not be pollable in this state\"),\n }\n}\n\ntype InnerFuture<'a, T> = ReusableBoxFuture<'a, Result, PollSendError>>;\n\n#[derive(Debug)]\n// TODO: This should be replace with a type_alias_impl_trait to eliminate `'static` and all the transmutes\nstruct PollSenderFuture(InnerFuture<'static, T>);\n\nimpl PollSenderFuture {\n /// Create with an empty inner future with no `Send` bound.\n fn empty() -> Self {\n // We don't use `make_acquire_future` here because our relaxed bounds on `T` are not\n // compatible with the transitive bounds required by `Sender`.\n Self(ReusableBoxFuture::new(async { unreachable!() }))\n }\n}\n\nimpl PollSenderFuture {\n /// Create with an empty inner future.\n fn new() -> Self {\n let v = InnerFuture::new(make_acquire_future(None));\n // This is safe because `make_acquire_future(None)` is actually `'static`\n Self(unsafe { mem::transmute::, InnerFuture<'static, T>>(v) })\n }\n\n /// Poll the inner future.\n fn poll(&mut self, cx: &mut Context<'_>) -> Poll, PollSendError>> {\n self.0.poll(cx)\n }\n\n /// Replace the inner future.\n fn set(&mut self, sender: Option>) {\n let inner: *mut InnerFuture<'static, T> = &mut self.0;\n let inner: *mut InnerFuture<'_, T> = inner.cast();\n // SAFETY: The `make_acquire_future(sender)` future must not exist after the type `T`\n // becomes invalid, and this casts away the type-level lifetime check for that. However, the\n // inner future is never moved out of this `PollSenderFuture`, so the future will not\n // live longer than the `PollSenderFuture` lives. A `PollSenderFuture` is guaranteed\n // to not exist after the type `T` becomes invalid, because it is annotated with a `T`, so\n // this is ok.\n let inner = unsafe { &mut *inner };\n inner.set(make_acquire_future(sender));\n }\n}\n\nimpl PollSender {\n /// Creates a new `PollSender`.\n pub fn new(sender: Sender) -> Self {\n Self {\n sender: Some(sender.clone()),\n state: State::Idle(sender),\n acquire: PollSenderFuture::new(),\n }\n }\n\n fn take_state(&mut self) -> State {\n mem::replace(&mut self.state, State::Closed)\n }\n\n /// Attempts to prepare the sender to receive a value.\n ///\n /// This method must be called and return `Poll::Ready(Ok(()))` prior to each call to\n /// `send_item`.\n ///\n /// This method returns `Poll::Ready` once the underlying channel is ready to receive a value,\n /// by reserving a slot in the channel for the item to be sent. If this method returns\n /// `Poll::Pending`, the current task is registered to be notified (via\n /// `cx.waker().wake_by_ref()`) when `poll_reserve` should be called again.\n ///\n /// # Errors\n ///\n /// If the channel is closed, an error will be returned. This is a permanent state.\n pub fn poll_reserve(&mut self, cx: &mut Context<'_>) -> Poll>> {\n loop {\n let (result, next_state) = match self.take_state() {\n State::Idle(sender) => {\n // Start trying to acquire a permit to reserve a slot for our send, and\n // immediately loop back around to poll it the first time.\n self.acquire.set(Some(sender));\n (None, State::Acquiring)\n }\n State::Acquiring => match self.acquire.poll(cx) {\n // Channel has capacity.\n Poll::Ready(Ok(permit)) => {\n (Some(Poll::Ready(Ok(()))), State::ReadyToSend(permit))\n }\n // Channel is closed.\n Poll::Ready(Err(e)) => (Some(Poll::Ready(Err(e))), State::Closed),\n // Channel doesn't have capacity yet, so we need to wait.\n Poll::Pending => (Some(Poll::Pending), State::Acquiring),\n },\n // We're closed, either by choice or because the underlying sender was closed.\n s @ State::Closed => (Some(Poll::Ready(Err(PollSendError(None)))), s),\n // We're already ready to send an item.\n s @ State::ReadyToSend(_) => (Some(Poll::Ready(Ok(()))), s),\n };\n\n self.state = next_state;\n if let Some(result) = result {\n return result;\n }\n }\n }\n\n /// Sends an item to the channel.\n ///\n /// Before calling `send_item`, `poll_reserve` must be called with a successful return\n /// value of `Poll::Ready(Ok(()))`.\n ///\n /// # Errors\n ///\n /// If the channel is closed, an error will be returned. This is a permanent state.\n ///\n /// # Panics\n ///\n /// If `poll_reserve` was not successfully called prior to calling `send_item`, then this method\n /// will panic.\n #[track_caller]\n pub fn send_item(&mut self, value: T) -> Result<(), PollSendError> {\n let (result, next_state) = match self.take_state() {\n State::Idle(_) | State::Acquiring => {\n panic!(\"`send_item` called without first calling `poll_reserve`\")\n }\n // We have a permit to send our item, so go ahead, which gets us our sender back.\n State::ReadyToSend(permit) => (Ok(()), State::Idle(permit.send(value))),\n // We're closed, either by choice or because the underlying sender was closed.\n State::Closed => (Err(PollSendError(Some(value))), State::Closed),\n };\n\n // Handle deferred closing if `close` was called between `poll_reserve` and `send_item`.\n self.state = if self.sender.is_some() {\n next_state\n } else {\n State::Closed\n };\n result\n }\n\n /// Checks whether this sender is closed.\n ///\n /// The underlying channel that this sender was wrapping may still be open.\n pub fn is_closed(&self) -> bool {\n matches!(self.state, State::Closed) || self.sender.is_none()\n }\n\n /// Gets a reference to the `Sender` of the underlying channel.\n ///\n /// If `PollSender` has been closed, `None` is returned. The underlying channel that this sender\n /// was wrapping may still be open.\n pub fn get_ref(&self) -> Option<&Sender> {\n self.sender.as_ref()\n }\n\n /// Closes this sender.\n ///\n /// No more messages will be able to be sent from this sender, but the underlying channel will\n /// remain open until all senders have dropped, or until the [`Receiver`] closes the channel.\n ///\n /// If a slot was previously reserved by calling `poll_reserve`, then a final call can be made\n /// to `send_item` in order to consume the reserved slot. After that, no further sends will be\n /// possible. If you do not intend to send another item, you can release the reserved slot back\n /// to the underlying sender by calling [`abort_send`].\n ///\n /// [`abort_send`]: crate::sync::PollSender::abort_send\n /// [`Receiver`]: tokio::sync::mpsc::Receiver\n pub fn close(&mut self) {\n // Mark ourselves officially closed by dropping our main sender.\n self.sender = None;\n\n // If we're already idle, closed, or we haven't yet reserved a slot, we can quickly\n // transition to the closed state. Otherwise, leave the existing permit in place for the\n // caller if they want to complete the send.\n match self.state {\n State::Idle(_) => self.state = State::Closed,\n State::Acquiring => {\n self.acquire.set(None);\n self.state = State::Closed;\n }\n _ => {}\n }\n }\n\n /// Aborts the current in-progress send, if any.\n ///\n /// Returns `true` if a send was aborted. If the sender was closed prior to calling\n /// `abort_send`, then the sender will remain in the closed state, otherwise the sender will be\n /// ready to attempt another send.\n pub fn abort_send(&mut self) -> bool {\n // We may have been closed in the meantime, after a call to `poll_reserve` already\n // succeeded. We'll check if `self.sender` is `None` to see if we should transition to the\n // closed state when we actually abort a send, rather than resetting ourselves back to idle.\n\n let (result, next_state) = match self.take_state() {\n // We're currently trying to reserve a slot to send into.\n State::Acquiring => {\n // Replacing the future drops the in-flight one.\n self.acquire.set(None);\n\n // If we haven't closed yet, we have to clone our stored sender since we have no way\n // to get it back from the acquire future we just dropped.\n let state = match self.sender.clone() {\n Some(sender) => State::Idle(sender),\n None => State::Closed,\n };\n (true, state)\n }\n // We got the permit. If we haven't closed yet, get the sender back.\n State::ReadyToSend(permit) => {\n let state = if self.sender.is_some() {\n State::Idle(permit.release())\n } else {\n State::Closed\n };\n (true, state)\n }\n s => (false, s),\n };\n\n self.state = next_state;\n result\n }\n}\n\nimpl Clone for PollSender {\n /// Clones this `PollSender`.\n ///\n /// The resulting `PollSender` will have an initial state identical to calling `PollSender::new`.\n fn clone(&self) -> PollSender {\n let (sender, state) = match self.sender.clone() {\n Some(sender) => (Some(sender.clone()), State::Idle(sender)),\n None => (None, State::Closed),\n };\n\n Self {\n sender,\n state,\n acquire: PollSenderFuture::empty(),\n }\n }\n}\n\nimpl Sink for PollSender {\n type Error = PollSendError;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::into_inner(self).poll_reserve(cx)\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn start_send(self: Pin<&mut Self>, item: T) -> Result<(), Self::Error> {\n Pin::into_inner(self).send_item(item)\n }\n\n fn poll_close(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Pin::into_inner(self).close();\n Poll::Ready(Ok(()))\n }\n}\n" }, { "path": "tokio-util/src/sync/poll_semaphore.rs", "content": "use futures_core::Stream;\nuse std::fmt;\nuse std::pin::Pin;\nuse std::sync::Arc;\nuse std::task::{ready, Context, Poll};\nuse tokio::sync::{AcquireError, OwnedSemaphorePermit, Semaphore, TryAcquireError};\n\nuse super::ReusableBoxFuture;\n\n/// A wrapper around [`Semaphore`] that provides a `poll_acquire` method.\n///\n/// [`Semaphore`]: tokio::sync::Semaphore\npub struct PollSemaphore {\n semaphore: Arc,\n permit_fut: Option<(\n u32, // The number of permits requested.\n ReusableBoxFuture<'static, Result>,\n )>,\n}\n\nimpl PollSemaphore {\n /// Create a new `PollSemaphore`.\n pub fn new(semaphore: Arc) -> Self {\n Self {\n semaphore,\n permit_fut: None,\n }\n }\n\n /// Closes the semaphore.\n pub fn close(&self) {\n self.semaphore.close();\n }\n\n /// Obtain a clone of the inner semaphore.\n pub fn clone_inner(&self) -> Arc {\n self.semaphore.clone()\n }\n\n /// Get back the inner semaphore.\n pub fn into_inner(self) -> Arc {\n self.semaphore\n }\n\n /// Poll to acquire a permit from the semaphore.\n ///\n /// This can return the following values:\n ///\n /// - `Poll::Pending` if a permit is not currently available.\n /// - `Poll::Ready(Some(permit))` if a permit was acquired.\n /// - `Poll::Ready(None)` if the semaphore has been closed.\n ///\n /// When this method returns `Poll::Pending`, the current task is scheduled\n /// to receive a wakeup when a permit becomes available, or when the\n /// semaphore is closed. Note that on multiple calls to `poll_acquire`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n pub fn poll_acquire(&mut self, cx: &mut Context<'_>) -> Poll> {\n self.poll_acquire_many(cx, 1)\n }\n\n /// Poll to acquire many permits from the semaphore.\n ///\n /// This can return the following values:\n ///\n /// - `Poll::Pending` if a permit is not currently available.\n /// - `Poll::Ready(Some(permit))` if a permit was acquired.\n /// - `Poll::Ready(None)` if the semaphore has been closed.\n ///\n /// When this method returns `Poll::Pending`, the current task is scheduled\n /// to receive a wakeup when the permits become available, or when the\n /// semaphore is closed. Note that on multiple calls to `poll_acquire`, only\n /// the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n pub fn poll_acquire_many(\n &mut self,\n cx: &mut Context<'_>,\n permits: u32,\n ) -> Poll> {\n let permit_future = match self.permit_fut.as_mut() {\n Some((prev_permits, fut)) if *prev_permits == permits => fut,\n Some((old_permits, fut_box)) => {\n // We're requesting a different number of permits, so replace the future\n // and record the new amount.\n let fut = Arc::clone(&self.semaphore).acquire_many_owned(permits);\n fut_box.set(fut);\n *old_permits = permits;\n fut_box\n }\n None => {\n // avoid allocations completely if we can grab a permit immediately\n match Arc::clone(&self.semaphore).try_acquire_many_owned(permits) {\n Ok(permit) => return Poll::Ready(Some(permit)),\n Err(TryAcquireError::Closed) => return Poll::Ready(None),\n Err(TryAcquireError::NoPermits) => {}\n }\n\n let next_fut = Arc::clone(&self.semaphore).acquire_many_owned(permits);\n &mut self\n .permit_fut\n .get_or_insert((permits, ReusableBoxFuture::new(next_fut)))\n .1\n }\n };\n\n let result = ready!(permit_future.poll(cx));\n\n // Assume we'll request the same amount of permits in a subsequent call.\n let next_fut = Arc::clone(&self.semaphore).acquire_many_owned(permits);\n permit_future.set(next_fut);\n\n match result {\n Ok(permit) => Poll::Ready(Some(permit)),\n Err(_closed) => {\n self.permit_fut = None;\n Poll::Ready(None)\n }\n }\n }\n\n /// Returns the current number of available permits.\n ///\n /// This is equivalent to the [`Semaphore::available_permits`] method on the\n /// `tokio::sync::Semaphore` type.\n ///\n /// [`Semaphore::available_permits`]: tokio::sync::Semaphore::available_permits\n pub fn available_permits(&self) -> usize {\n self.semaphore.available_permits()\n }\n\n /// Adds `n` new permits to the semaphore.\n ///\n /// The maximum number of permits is [`Semaphore::MAX_PERMITS`], and this function\n /// will panic if the limit is exceeded.\n ///\n /// This is equivalent to the [`Semaphore::add_permits`] method on the\n /// `tokio::sync::Semaphore` type.\n ///\n /// [`Semaphore::add_permits`]: tokio::sync::Semaphore::add_permits\n pub fn add_permits(&self, n: usize) {\n self.semaphore.add_permits(n);\n }\n}\n\nimpl Stream for PollSemaphore {\n type Item = OwnedSemaphorePermit;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n Pin::into_inner(self).poll_acquire(cx)\n }\n}\n\nimpl Clone for PollSemaphore {\n fn clone(&self) -> PollSemaphore {\n PollSemaphore::new(self.clone_inner())\n }\n}\n\nimpl fmt::Debug for PollSemaphore {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"PollSemaphore\")\n .field(\"semaphore\", &self.semaphore)\n .finish()\n }\n}\n\nimpl AsRef for PollSemaphore {\n fn as_ref(&self) -> &Semaphore {\n &self.semaphore\n }\n}\n" }, { "path": "tokio-util/src/sync/reusable_box.rs", "content": "use std::alloc::Layout;\nuse std::fmt;\nuse std::future::{self, Future};\nuse std::mem::{self, ManuallyDrop};\nuse std::pin::Pin;\nuse std::ptr;\nuse std::task::{Context, Poll};\n\n/// A reusable `Pin + Send + 'a>>`.\n///\n/// This type lets you replace the future stored in the box without\n/// reallocating when the size and alignment permits this.\npub struct ReusableBoxFuture<'a, T> {\n boxed: Pin + Send + 'a>>,\n}\n\nimpl<'a, T> ReusableBoxFuture<'a, T> {\n /// Create a new `ReusableBoxFuture` containing the provided future.\n pub fn new(future: F) -> Self\n where\n F: Future + Send + 'a,\n {\n Self {\n boxed: Box::pin(future),\n }\n }\n\n /// Replace the future currently stored in this box.\n ///\n /// This reallocates if and only if the layout of the provided future is\n /// different from the layout of the currently stored future.\n pub fn set(&mut self, future: F)\n where\n F: Future + Send + 'a,\n {\n if let Err(future) = self.try_set(future) {\n *self = Self::new(future);\n }\n }\n\n /// Replace the future currently stored in this box.\n ///\n /// This function never reallocates, but returns an error if the provided\n /// future has a different size or alignment from the currently stored\n /// future.\n pub fn try_set(&mut self, future: F) -> Result<(), F>\n where\n F: Future + Send + 'a,\n {\n // If we try to inline the contents of this function, the type checker complains because\n // the bound `T: 'a` is not satisfied in the call to `pending()`. But by putting it in an\n // inner function that doesn't have `T` as a generic parameter, we implicitly get the bound\n // `F::Output: 'a` transitively through `F: 'a`, allowing us to call `pending()`.\n #[inline(always)]\n fn real_try_set<'a, F>(\n this: &mut ReusableBoxFuture<'a, F::Output>,\n future: F,\n ) -> Result<(), F>\n where\n F: Future + Send + 'a,\n {\n // future::Pending is a ZST so this never allocates.\n let boxed = mem::replace(&mut this.boxed, Box::pin(future::pending()));\n reuse_pin_box(boxed, future, |boxed| this.boxed = Pin::from(boxed))\n }\n\n real_try_set(self, future)\n }\n\n /// Get a pinned reference to the underlying future.\n pub fn get_pin(&mut self) -> Pin<&mut (dyn Future + Send)> {\n self.boxed.as_mut()\n }\n\n /// Poll the future stored inside this box.\n pub fn poll(&mut self, cx: &mut Context<'_>) -> Poll {\n self.get_pin().poll(cx)\n }\n}\n\nimpl Future for ReusableBoxFuture<'_, T> {\n type Output = T;\n\n /// Poll the future stored inside this box.\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n Pin::into_inner(self).get_pin().poll(cx)\n }\n}\n\n// The only method called on self.boxed is poll, which takes &mut self, so this\n// struct being Sync does not permit any invalid access to the Future, even if\n// the future is not Sync.\nunsafe impl Sync for ReusableBoxFuture<'_, T> {}\n\nimpl fmt::Debug for ReusableBoxFuture<'_, T> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"ReusableBoxFuture\").finish()\n }\n}\n\nfn reuse_pin_box(boxed: Pin>, new_value: U, callback: F) -> Result\nwhere\n F: FnOnce(Box) -> O,\n{\n let layout = Layout::for_value::(&*boxed);\n if layout != Layout::new::() {\n return Err(new_value);\n }\n\n // SAFETY: We don't ever construct a non-pinned reference to the old `T` from now on, and we\n // always drop the `T`.\n let raw: *mut T = Box::into_raw(unsafe { Pin::into_inner_unchecked(boxed) });\n\n // When dropping the old value panics, we still want to call `callback` — so move the rest of\n // the code into a guard type.\n let guard = CallOnDrop::new(|| {\n let raw: *mut U = raw.cast::();\n unsafe { raw.write(new_value) };\n\n // SAFETY:\n // - `T` and `U` have the same layout.\n // - `raw` comes from a `Box` that uses the same allocator as this one.\n // - `raw` points to a valid instance of `U` (we just wrote it in).\n let boxed = unsafe { Box::from_raw(raw) };\n\n callback(boxed)\n });\n\n // Drop the old value.\n unsafe { ptr::drop_in_place(raw) };\n\n // Run the rest of the code.\n Ok(guard.call())\n}\n\nstruct CallOnDrop O> {\n f: ManuallyDrop,\n}\n\nimpl O> CallOnDrop {\n fn new(f: F) -> Self {\n let f = ManuallyDrop::new(f);\n Self { f }\n }\n fn call(self) -> O {\n let mut this = ManuallyDrop::new(self);\n let f = unsafe { ManuallyDrop::take(&mut this.f) };\n f()\n }\n}\n\nimpl O> Drop for CallOnDrop {\n fn drop(&mut self) {\n let f = unsafe { ManuallyDrop::take(&mut self.f) };\n f();\n }\n}\n" }, { "path": "tokio-util/src/sync/tests/loom_cancellation_token.rs", "content": "use crate::sync::CancellationToken;\n\nuse loom::{future::block_on, thread};\nuse tokio_test::assert_ok;\n\n#[test]\nfn cancel_token() {\n loom::model(|| {\n let token = CancellationToken::new();\n let token1 = token.clone();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n token1.cancelled().await;\n });\n });\n\n let th2 = thread::spawn(move || {\n token.cancel();\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n#[test]\nfn cancel_token_owned() {\n loom::model(|| {\n let token = CancellationToken::new();\n let token1 = token.clone();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n token1.cancelled_owned().await;\n });\n });\n\n let th2 = thread::spawn(move || {\n token.cancel();\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n });\n}\n\n#[test]\nfn cancel_with_child() {\n loom::model(|| {\n let token = CancellationToken::new();\n let token1 = token.clone();\n let token2 = token.clone();\n let child_token = token.child_token();\n\n let th1 = thread::spawn(move || {\n block_on(async {\n token1.cancelled().await;\n });\n });\n\n let th2 = thread::spawn(move || {\n token2.cancel();\n });\n\n let th3 = thread::spawn(move || {\n block_on(async {\n child_token.cancelled().await;\n });\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n\n#[test]\nfn drop_token_no_child() {\n loom::model(|| {\n let token = CancellationToken::new();\n let token1 = token.clone();\n let token2 = token.clone();\n\n let th1 = thread::spawn(move || {\n drop(token1);\n });\n\n let th2 = thread::spawn(move || {\n drop(token2);\n });\n\n let th3 = thread::spawn(move || {\n drop(token);\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n\n// Temporarily disabled due to a false positive in loom -\n// see https://github.com/tokio-rs/tokio/pull/7644#issuecomment-3328381344\n#[ignore]\n#[test]\nfn drop_token_with_children() {\n loom::model(|| {\n let token1 = CancellationToken::new();\n let child_token1 = token1.child_token();\n let child_token2 = token1.child_token();\n\n let th1 = thread::spawn(move || {\n drop(token1);\n });\n\n let th2 = thread::spawn(move || {\n drop(child_token1);\n });\n\n let th3 = thread::spawn(move || {\n drop(child_token2);\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n\n// Temporarily disabled due to a false positive in loom -\n// see https://github.com/tokio-rs/tokio/pull/7644#issuecomment-3328381344\n#[ignore]\n#[test]\nfn drop_and_cancel_token() {\n loom::model(|| {\n let token1 = CancellationToken::new();\n let token2 = token1.clone();\n let child_token = token1.child_token();\n\n let th1 = thread::spawn(move || {\n drop(token1);\n });\n\n let th2 = thread::spawn(move || {\n token2.cancel();\n });\n\n let th3 = thread::spawn(move || {\n drop(child_token);\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n\n// Temporarily disabled due to a false positive in loom -\n// see https://github.com/tokio-rs/tokio/pull/7644#issuecomment-3328381344\n#[ignore]\n#[test]\nfn cancel_parent_and_child() {\n loom::model(|| {\n let token1 = CancellationToken::new();\n let token2 = token1.clone();\n let child_token = token1.child_token();\n\n let th1 = thread::spawn(move || {\n drop(token1);\n });\n\n let th2 = thread::spawn(move || {\n token2.cancel();\n });\n\n let th3 = thread::spawn(move || {\n child_token.cancel();\n });\n\n assert_ok!(th1.join());\n assert_ok!(th2.join());\n assert_ok!(th3.join());\n });\n}\n" }, { "path": "tokio-util/src/sync/tests/mod.rs", "content": "#[cfg(loom)]\nmod loom_cancellation_token;\n" }, { "path": "tokio-util/src/task/abort_on_drop.rs", "content": "//! An [`AbortOnDropHandle`] is like a [`JoinHandle`], except that it\n//! will abort the task as soon as it is dropped.\n//!\n//! Correspondingly, an [`AbortOnDrop`] is like a [`AbortHandle`] that will abort\n//! the task as soon as it is dropped.\n\nuse tokio::task::{AbortHandle, JoinError, JoinHandle};\n\nuse std::{\n future::Future,\n mem::ManuallyDrop,\n pin::Pin,\n task::{Context, Poll},\n};\n\n/// A wrapper around a [`tokio::task::JoinHandle`],\n/// which [aborts] the task when it is dropped.\n///\n/// [aborts]: tokio::task::JoinHandle::abort\n#[must_use = \"Dropping the handle aborts the task immediately\"]\npub struct AbortOnDropHandle(JoinHandle);\n\nimpl Drop for AbortOnDropHandle {\n fn drop(&mut self) {\n self.abort()\n }\n}\n\nimpl AbortOnDropHandle {\n /// Create an [`AbortOnDropHandle`] from a [`JoinHandle`].\n pub fn new(handle: JoinHandle) -> Self {\n Self(handle)\n }\n\n /// Abort the task associated with this handle,\n /// equivalent to [`JoinHandle::abort`].\n #[inline]\n pub fn abort(&self) {\n self.0.abort()\n }\n\n /// Checks if the task associated with this handle is finished,\n /// equivalent to [`JoinHandle::is_finished`].\n #[inline]\n pub fn is_finished(&self) -> bool {\n self.0.is_finished()\n }\n\n /// Returns a new [`AbortHandle`] that can be used to remotely abort this task,\n /// equivalent to [`JoinHandle::abort_handle`].\n pub fn abort_handle(&self) -> AbortHandle {\n self.0.abort_handle()\n }\n\n /// Cancels aborting on drop and returns the original [`JoinHandle`].\n pub fn detach(self) -> JoinHandle {\n // Avoid invoking `AbortOnDropHandle`'s `Drop` impl\n let this = ManuallyDrop::new(self);\n // SAFETY: `&this.0` is a reference, so it is certainly initialized, and\n // it won't be double-dropped because it's in a `ManuallyDrop`\n unsafe { std::ptr::read(&this.0) }\n }\n}\n\nimpl std::fmt::Debug for AbortOnDropHandle {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"AbortOnDropHandle\")\n .field(\"id\", &self.0.id())\n .finish()\n }\n}\n\nimpl Future for AbortOnDropHandle {\n type Output = Result;\n\n fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n Pin::new(&mut self.0).poll(cx)\n }\n}\n\nimpl AsRef> for AbortOnDropHandle {\n fn as_ref(&self) -> &JoinHandle {\n &self.0\n }\n}\n\n/// A wrapper around a [`tokio::task::AbortHandle`],\n/// which [aborts] the task when it is dropped.\n///\n/// Unlike [`AbortOnDropHandle`], [`AbortOnDrop`] cannot be `.await`ed for a result.\n///\n/// It has no generic parameter, making it suitable when you only need to keep\n/// a task handle in a struct and do not care about the output.\n///\n/// [aborts]: tokio::task::AbortHandle::abort\n#[must_use = \"Dropping the handle aborts the task immediately\"]\npub struct AbortOnDrop(AbortHandle);\n\nimpl Drop for AbortOnDrop {\n fn drop(&mut self) {\n self.abort()\n }\n}\n\nimpl AbortOnDrop {\n /// Create an [`AbortOnDrop`] from a [`AbortHandle`].\n pub fn new(handle: AbortHandle) -> Self {\n Self(handle)\n }\n\n /// Abort the task associated with this handle,\n /// equivalent to [`AbortHandle::abort`].\n #[inline]\n pub fn abort(&self) {\n self.0.abort()\n }\n\n /// Checks if the task associated with this handle is finished,\n /// equivalent to [`AbortHandle::is_finished`].\n #[inline]\n pub fn is_finished(&self) -> bool {\n self.0.is_finished()\n }\n\n /// Cancels aborting on drop and returns the original [`AbortHandle`].\n pub fn detach(self) -> AbortHandle {\n // Avoid invoking `AbortOnDrop`'s `Drop` impl\n let this = ManuallyDrop::new(self);\n // SAFETY: `&this.0` is a reference, so it is certainly initialized, and\n // it won't be double-dropped because it's in a `ManuallyDrop`\n unsafe { std::ptr::read(&this.0) }\n }\n}\n\nimpl std::fmt::Debug for AbortOnDrop {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_struct(\"AbortOnDrop\")\n .field(\"id\", &self.0.id())\n .finish()\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n /// A simple type that does not implement [`std::fmt::Debug`].\n struct NotDebug;\n\n fn is_debug() {}\n\n #[test]\n fn assert_debug() {\n is_debug::();\n is_debug::>();\n }\n}\n" }, { "path": "tokio-util/src/task/join_map.rs", "content": "use hashbrown::hash_table::Entry;\nuse hashbrown::{HashMap, HashTable};\nuse std::borrow::Borrow;\nuse std::collections::hash_map::RandomState;\nuse std::fmt;\nuse std::future::Future;\nuse std::hash::{BuildHasher, Hash};\nuse std::marker::PhantomData;\nuse tokio::runtime::Handle;\nuse tokio::task::{AbortHandle, Id, JoinError, JoinSet, LocalSet};\n\n/// A collection of tasks spawned on a Tokio runtime, associated with hash map\n/// keys.\n///\n/// This type is very similar to the [`JoinSet`] type in `tokio::task`, with the\n/// addition of a set of keys associated with each task. These keys allow\n/// [cancelling a task][abort] or [multiple tasks][abort_matching] in the\n/// `JoinMap` based on their keys, or [test whether a task corresponding to a\n/// given key exists][contains] in the `JoinMap`.\n///\n/// In addition, when tasks in the `JoinMap` complete, they will return the\n/// associated key along with the value returned by the task, if any.\n///\n/// A `JoinMap` can be used to await the completion of some or all of the tasks\n/// in the map. The map is not ordered, and the tasks will be returned in the\n/// order they complete.\n///\n/// All of the tasks must have the same return type `V`.\n///\n/// When the `JoinMap` is dropped, all tasks in the `JoinMap` are immediately aborted.\n///\n/// # Examples\n///\n/// Spawn multiple tasks and wait for them:\n///\n/// ```\n/// use tokio_util::task::JoinMap;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut map = JoinMap::new();\n///\n/// for i in 0..10 {\n/// // Spawn a task on the `JoinMap` with `i` as its key.\n/// map.spawn(i, async move { /* ... */ });\n/// }\n///\n/// let mut seen = [false; 10];\n///\n/// // When a task completes, `join_next` returns the task's key along\n/// // with its output.\n/// while let Some((key, res)) = map.join_next().await {\n/// seen[key] = true;\n/// assert!(res.is_ok(), \"task {} completed successfully!\", key);\n/// }\n///\n/// for i in 0..10 {\n/// assert!(seen[i]);\n/// }\n/// # }\n/// ```\n///\n/// Cancel tasks based on their keys:\n///\n/// ```\n/// use tokio_util::task::JoinMap;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let mut map = JoinMap::new();\n///\n/// map.spawn(\"hello world\", std::future::ready(1));\n/// map.spawn(\"goodbye world\", std::future::pending());\n///\n/// // Look up the \"goodbye world\" task in the map and abort it.\n/// let aborted = map.abort(\"goodbye world\");\n///\n/// // `JoinMap::abort` returns `true` if a task existed for the\n/// // provided key.\n/// assert!(aborted);\n///\n/// while let Some((key, res)) = map.join_next().await {\n/// if key == \"goodbye world\" {\n/// // The aborted task should complete with a cancelled `JoinError`.\n/// assert!(res.unwrap_err().is_cancelled());\n/// } else {\n/// // Other tasks should complete normally.\n/// assert_eq!(res.unwrap(), 1);\n/// }\n/// }\n/// # }\n/// ```\n///\n/// [`JoinSet`]: tokio::task::JoinSet\n/// [abort]: fn@Self::abort\n/// [abort_matching]: fn@Self::abort_matching\n/// [contains]: fn@Self::contains_key\npub struct JoinMap {\n /// A map of the [`AbortHandle`]s of the tasks spawned on this `JoinMap`,\n /// indexed by their keys.\n tasks_by_key: HashTable<(K, AbortHandle)>,\n\n /// A map from task IDs to the hash of the key associated with that task.\n ///\n /// This map is used to perform reverse lookups of tasks in the\n /// `tasks_by_key` map based on their task IDs. When a task terminates, the\n /// ID is provided to us by the `JoinSet`, so we can look up the hash value\n /// of that task's key, and then remove it from the `tasks_by_key` map using\n /// the raw hash code, resolving collisions by comparing task IDs.\n hashes_by_task: HashMap,\n\n /// The [`JoinSet`] that awaits the completion of tasks spawned on this\n /// `JoinMap`.\n tasks: JoinSet,\n}\n\nimpl JoinMap {\n /// Creates a new empty `JoinMap`.\n ///\n /// The `JoinMap` is initially created with a capacity of 0, so it will not\n /// allocate until a task is first spawned on it.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::JoinMap;\n /// let map: JoinMap<&str, i32> = JoinMap::new();\n /// ```\n #[inline]\n #[must_use]\n pub fn new() -> Self {\n Self::with_hasher(RandomState::new())\n }\n\n /// Creates an empty `JoinMap` with the specified capacity.\n ///\n /// The `JoinMap` will be able to hold at least `capacity` tasks without\n /// reallocating.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::JoinMap;\n /// let map: JoinMap<&str, i32> = JoinMap::with_capacity(10);\n /// ```\n #[inline]\n #[must_use]\n pub fn with_capacity(capacity: usize) -> Self {\n JoinMap::with_capacity_and_hasher(capacity, Default::default())\n }\n}\n\nimpl JoinMap {\n /// Creates an empty `JoinMap` which will use the given hash builder to hash\n /// keys.\n ///\n /// The created map has the default initial capacity.\n ///\n /// Warning: `hash_builder` is normally randomly generated, and\n /// is designed to allow `JoinMap` to be resistant to attacks that\n /// cause many collisions and very poor performance. Setting it\n /// manually using this function can expose a DoS attack vector.\n ///\n /// The `hash_builder` passed should implement the [`BuildHasher`] trait for\n /// the `JoinMap` to be useful, see its documentation for details.\n #[inline]\n #[must_use]\n pub fn with_hasher(hash_builder: S) -> Self {\n Self::with_capacity_and_hasher(0, hash_builder)\n }\n\n /// Creates an empty `JoinMap` with the specified capacity, using `hash_builder`\n /// to hash the keys.\n ///\n /// The `JoinMap` will be able to hold at least `capacity` elements without\n /// reallocating. If `capacity` is 0, the `JoinMap` will not allocate.\n ///\n /// Warning: `hash_builder` is normally randomly generated, and\n /// is designed to allow HashMaps to be resistant to attacks that\n /// cause many collisions and very poor performance. Setting it\n /// manually using this function can expose a DoS attack vector.\n ///\n /// The `hash_builder` passed should implement the [`BuildHasher`] trait for\n /// the `JoinMap`to be useful, see its documentation for details.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_util::task::JoinMap;\n /// use std::collections::hash_map::RandomState;\n ///\n /// let s = RandomState::new();\n /// let mut map = JoinMap::with_capacity_and_hasher(10, s);\n /// map.spawn(1, async move { \"hello world!\" });\n /// # }\n /// ```\n #[inline]\n #[must_use]\n pub fn with_capacity_and_hasher(capacity: usize, hash_builder: S) -> Self {\n Self {\n tasks_by_key: HashTable::with_capacity(capacity),\n hashes_by_task: HashMap::with_capacity_and_hasher(capacity, hash_builder),\n tasks: JoinSet::new(),\n }\n }\n\n /// Returns the number of tasks currently in the `JoinMap`.\n pub fn len(&self) -> usize {\n let len = self.tasks_by_key.len();\n debug_assert_eq!(len, self.hashes_by_task.len());\n len\n }\n\n /// Returns whether the `JoinMap` is empty.\n pub fn is_empty(&self) -> bool {\n let empty = self.tasks_by_key.is_empty();\n debug_assert_eq!(empty, self.hashes_by_task.is_empty());\n empty\n }\n\n /// Returns the number of tasks the map can hold without reallocating.\n ///\n /// This number is a lower bound; the `JoinMap` might be able to hold\n /// more, but is guaranteed to be able to hold at least this many.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::JoinMap;\n ///\n /// let map: JoinMap = JoinMap::with_capacity(100);\n /// assert!(map.capacity() >= 100);\n /// ```\n #[inline]\n pub fn capacity(&self) -> usize {\n let capacity = self.tasks_by_key.capacity();\n debug_assert_eq!(capacity, self.hashes_by_task.capacity());\n capacity\n }\n}\n\nimpl JoinMap\nwhere\n K: Hash + Eq,\n V: 'static,\n S: BuildHasher,\n{\n /// Spawn the provided task and store it in this `JoinMap` with the provided\n /// key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn(&mut self, key: K, task: F)\n where\n F: Future,\n F: Send + 'static,\n V: Send,\n {\n let task = self.tasks.spawn(task);\n self.insert(key, task)\n }\n\n /// Spawn the provided task on the provided runtime and store it in this\n /// `JoinMap` with the provided key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn_on(&mut self, key: K, task: F, handle: &Handle)\n where\n F: Future,\n F: Send + 'static,\n V: Send,\n {\n let task = self.tasks.spawn_on(task, handle);\n self.insert(key, task);\n }\n\n /// Spawn the blocking code on the blocking threadpool and store it in this `JoinMap` with the provided\n /// key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// Note that blocking tasks cannot be cancelled after execution starts.\n /// Replaced blocking tasks will still run to completion if the task has begun\n /// to execute when it is replaced. A blocking task which is replaced before\n /// it has been scheduled on a blocking worker thread will be cancelled.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn_blocking(&mut self, key: K, f: F)\n where\n F: FnOnce() -> V,\n F: Send + 'static,\n V: Send,\n {\n let task = self.tasks.spawn_blocking(f);\n self.insert(key, task)\n }\n\n /// Spawn the blocking code on the blocking threadpool of the provided runtime and store it in this\n /// `JoinMap` with the provided key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// Note that blocking tasks cannot be cancelled after execution starts.\n /// Replaced blocking tasks will still run to completion if the task has begun\n /// to execute when it is replaced. A blocking task which is replaced before\n /// it has been scheduled on a blocking worker thread will be cancelled.\n ///\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn_blocking_on(&mut self, key: K, f: F, handle: &Handle)\n where\n F: FnOnce() -> V,\n F: Send + 'static,\n V: Send,\n {\n let task = self.tasks.spawn_blocking_on(f, handle);\n self.insert(key, task);\n }\n\n /// Spawn the provided task on the current [`LocalSet`] or [`LocalRuntime`]\n /// and store it in this `JoinMap` with the provided key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// # Panics\n ///\n /// This method panics if it is called outside of a `LocalSet` or `LocalRuntime`.\n ///\n /// [`LocalSet`]: tokio::task::LocalSet\n /// [`LocalRuntime`]: tokio::runtime::LocalRuntime\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn_local(&mut self, key: K, task: F)\n where\n F: Future,\n F: 'static,\n {\n let task = self.tasks.spawn_local(task);\n self.insert(key, task);\n }\n\n /// Spawn the provided task on the provided [`LocalSet`] and store it in\n /// this `JoinMap` with the provided key.\n ///\n /// If a task previously existed in the `JoinMap` for this key, that task\n /// will be cancelled and replaced with the new one. The previous task will\n /// be removed from the `JoinMap`; a subsequent call to [`join_next`] will\n /// *not* return a cancelled [`JoinError`] for that task.\n ///\n /// [`LocalSet`]: tokio::task::LocalSet\n /// [`join_next`]: Self::join_next\n #[track_caller]\n pub fn spawn_local_on(&mut self, key: K, task: F, local_set: &LocalSet)\n where\n F: Future,\n F: 'static,\n {\n let task = self.tasks.spawn_local_on(task, local_set);\n self.insert(key, task)\n }\n\n fn insert(&mut self, mut key: K, mut abort: AbortHandle) {\n let hash_builder = self.hashes_by_task.hasher();\n let hash = hash_builder.hash_one(&key);\n let id = abort.id();\n\n // Insert the new key into the map of tasks by keys.\n let entry =\n self.tasks_by_key\n .entry(hash, |(k, _)| *k == key, |(k, _)| hash_builder.hash_one(k));\n match entry {\n Entry::Occupied(occ) => {\n // There was a previous task spawned with the same key! Cancel\n // that task, and remove its ID from the map of hashes by task IDs.\n (key, abort) = std::mem::replace(occ.into_mut(), (key, abort));\n\n // Remove the old task ID.\n let _prev_hash = self.hashes_by_task.remove(&abort.id());\n debug_assert_eq!(Some(hash), _prev_hash);\n\n // Associate the key's hash with the new task's ID, for looking up tasks by ID.\n let _prev = self.hashes_by_task.insert(id, hash);\n debug_assert!(_prev.is_none(), \"no prior task should have had the same ID\");\n\n // Note: it's important to drop `key` and abort the task here.\n // This defends against any panics during drop handling for causing inconsistent state.\n abort.abort();\n drop(key);\n }\n Entry::Vacant(vac) => {\n vac.insert((key, abort));\n\n // Associate the key's hash with this task's ID, for looking up tasks by ID.\n let _prev = self.hashes_by_task.insert(id, hash);\n debug_assert!(_prev.is_none(), \"no prior task should have had the same ID\");\n }\n };\n }\n\n /// Waits until one of the tasks in the map completes and returns its\n /// output, along with the key corresponding to that task.\n ///\n /// Returns `None` if the map is empty.\n ///\n /// # Cancel Safety\n ///\n /// This method is cancel safe. If `join_next` is used as the event in a [`tokio::select!`]\n /// statement and some other branch completes first, it is guaranteed that no tasks were\n /// removed from this `JoinMap`.\n ///\n /// # Returns\n ///\n /// This function returns:\n ///\n /// * `Some((key, Ok(value)))` if one of the tasks in this `JoinMap` has\n /// completed. The `value` is the return value of that ask, and `key` is\n /// the key associated with the task.\n /// * `Some((key, Err(err))` if one of the tasks in this `JoinMap` has\n /// panicked or been aborted. `key` is the key associated with the task\n /// that panicked or was aborted.\n /// * `None` if the `JoinMap` is empty.\n ///\n /// [`tokio::select!`]: https://docs.rs/tokio/latest/tokio/macro.select.html\n pub async fn join_next(&mut self) -> Option<(K, Result)> {\n loop {\n let (res, id) = match self.tasks.join_next_with_id().await {\n Some(Ok((id, output))) => (Ok(output), id),\n Some(Err(e)) => {\n let id = e.id();\n (Err(e), id)\n }\n None => return None,\n };\n if let Some(key) = self.remove_by_id(id) {\n break Some((key, res));\n }\n }\n }\n\n /// Aborts all tasks and waits for them to finish shutting down.\n ///\n /// Calling this method is equivalent to calling [`abort_all`] and then calling [`join_next`] in\n /// a loop until it returns `None`.\n ///\n /// This method ignores any panics in the tasks shutting down. When this call returns, the\n /// `JoinMap` will be empty.\n ///\n /// [`abort_all`]: fn@Self::abort_all\n /// [`join_next`]: fn@Self::join_next\n pub async fn shutdown(&mut self) {\n self.abort_all();\n while self.join_next().await.is_some() {}\n }\n\n /// Abort the task corresponding to the provided `key`.\n ///\n /// If this `JoinMap` contains a task corresponding to `key`, this method\n /// will abort that task and return `true`. Otherwise, if no task exists for\n /// `key`, this method returns `false`.\n ///\n /// # Examples\n ///\n /// Aborting a task by key:\n ///\n /// ```\n /// use tokio_util::task::JoinMap;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut map = JoinMap::new();\n ///\n /// map.spawn(\"hello world\", std::future::ready(1));\n /// map.spawn(\"goodbye world\", std::future::pending());\n ///\n /// // Look up the \"goodbye world\" task in the map and abort it.\n /// map.abort(\"goodbye world\");\n ///\n /// while let Some((key, res)) = map.join_next().await {\n /// if key == \"goodbye world\" {\n /// // The aborted task should complete with a cancelled `JoinError`.\n /// assert!(res.unwrap_err().is_cancelled());\n /// } else {\n /// // Other tasks should complete normally.\n /// assert_eq!(res.unwrap(), 1);\n /// }\n /// }\n /// # }\n /// ```\n ///\n /// `abort` returns `true` if a task was aborted:\n /// ```\n /// use tokio_util::task::JoinMap;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut map = JoinMap::new();\n ///\n /// map.spawn(\"hello world\", async move { /* ... */ });\n /// map.spawn(\"goodbye world\", async move { /* ... */});\n ///\n /// // A task for the key \"goodbye world\" should exist in the map:\n /// assert!(map.abort(\"goodbye world\"));\n ///\n /// // Aborting a key that does not exist will return `false`:\n /// assert!(!map.abort(\"goodbye universe\"));\n /// # }\n /// ```\n pub fn abort(&mut self, key: &Q) -> bool\n where\n Q: ?Sized + Hash + Eq,\n K: Borrow,\n {\n match self.get_by_key(key) {\n Some((_, handle)) => {\n handle.abort();\n true\n }\n None => false,\n }\n }\n\n /// Aborts all tasks with keys matching `predicate`.\n ///\n /// `predicate` is a function called with a reference to each key in the\n /// map. If it returns `true` for a given key, the corresponding task will\n /// be cancelled.\n ///\n /// # Examples\n /// ```\n /// use tokio_util::task::JoinMap;\n ///\n /// # // use the current thread rt so that spawned tasks don't\n /// # // complete in the background before they can be aborted.\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut map = JoinMap::new();\n ///\n /// map.spawn(\"hello world\", async move {\n /// // ...\n /// # tokio::task::yield_now().await; // don't complete immediately, get aborted!\n /// });\n /// map.spawn(\"goodbye world\", async move {\n /// // ...\n /// # tokio::task::yield_now().await; // don't complete immediately, get aborted!\n /// });\n /// map.spawn(\"hello san francisco\", async move {\n /// // ...\n /// # tokio::task::yield_now().await; // don't complete immediately, get aborted!\n /// });\n /// map.spawn(\"goodbye universe\", async move {\n /// // ...\n /// # tokio::task::yield_now().await; // don't complete immediately, get aborted!\n /// });\n ///\n /// // Abort all tasks whose keys begin with \"goodbye\"\n /// map.abort_matching(|key| key.starts_with(\"goodbye\"));\n ///\n /// let mut seen = 0;\n /// while let Some((key, res)) = map.join_next().await {\n /// seen += 1;\n /// if key.starts_with(\"goodbye\") {\n /// // The aborted task should complete with a cancelled `JoinError`.\n /// assert!(res.unwrap_err().is_cancelled());\n /// } else {\n /// // Other tasks should complete normally.\n /// assert!(key.starts_with(\"hello\"));\n /// assert!(res.is_ok());\n /// }\n /// }\n ///\n /// // All spawned tasks should have completed.\n /// assert_eq!(seen, 4);\n /// # }\n /// ```\n pub fn abort_matching(&mut self, mut predicate: impl FnMut(&K) -> bool) {\n // Note: this method iterates over the tasks and keys *without* removing\n // any entries, so that the keys from aborted tasks can still be\n // returned when calling `join_next` in the future.\n for (key, task) in &self.tasks_by_key {\n if predicate(key) {\n task.abort();\n }\n }\n }\n\n /// Returns an iterator visiting all keys in this `JoinMap` in arbitrary order.\n ///\n /// If a task has completed, but its output hasn't yet been consumed by a\n /// call to [`join_next`], this method will still return its key.\n ///\n /// [`join_next`]: fn@Self::join_next\n pub fn keys(&self) -> JoinMapKeys<'_, K, V> {\n JoinMapKeys {\n iter: self.tasks_by_key.iter(),\n _value: PhantomData,\n }\n }\n\n /// Returns `true` if this `JoinMap` contains a task for the provided key.\n ///\n /// If the task has completed, but its output hasn't yet been consumed by a\n /// call to [`join_next`], this method will still return `true`.\n ///\n /// [`join_next`]: fn@Self::join_next\n pub fn contains_key(&self, key: &Q) -> bool\n where\n Q: ?Sized + Hash + Eq,\n K: Borrow,\n {\n self.get_by_key(key).is_some()\n }\n\n /// Returns `true` if this `JoinMap` contains a task with the provided\n /// [task ID].\n ///\n /// If the task has completed, but its output hasn't yet been consumed by a\n /// call to [`join_next`], this method will still return `true`.\n ///\n /// [`join_next`]: fn@Self::join_next\n /// [task ID]: tokio::task::Id\n pub fn contains_task(&self, task: &Id) -> bool {\n self.hashes_by_task.contains_key(task)\n }\n\n /// Reserves capacity for at least `additional` more tasks to be spawned\n /// on this `JoinMap` without reallocating for the map of task keys. The\n /// collection may reserve more space to avoid frequent reallocations.\n ///\n /// Note that spawning a task will still cause an allocation for the task\n /// itself.\n ///\n /// # Panics\n ///\n /// Panics if the new allocation size overflows [`usize`].\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::JoinMap;\n ///\n /// let mut map: JoinMap<&str, i32> = JoinMap::new();\n /// map.reserve(10);\n /// ```\n #[inline]\n pub fn reserve(&mut self, additional: usize) {\n self.tasks_by_key.reserve(additional, |(k, _)| {\n self.hashes_by_task.hasher().hash_one(k)\n });\n self.hashes_by_task.reserve(additional);\n }\n\n /// Shrinks the capacity of the `JoinMap` as much as possible. It will drop\n /// down as much as possible while maintaining the internal rules\n /// and possibly leaving some space in accordance with the resize policy.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_util::task::JoinMap;\n ///\n /// let mut map: JoinMap = JoinMap::with_capacity(100);\n /// map.spawn(1, async move { 2 });\n /// map.spawn(3, async move { 4 });\n /// assert!(map.capacity() >= 100);\n /// map.shrink_to_fit();\n /// assert!(map.capacity() >= 2);\n /// # }\n /// ```\n #[inline]\n pub fn shrink_to_fit(&mut self) {\n self.hashes_by_task.shrink_to_fit();\n self.tasks_by_key\n .shrink_to_fit(|(k, _)| self.hashes_by_task.hasher().hash_one(k));\n }\n\n /// Shrinks the capacity of the map with a lower limit. It will drop\n /// down no lower than the supplied limit while maintaining the internal rules\n /// and possibly leaving some space in accordance with the resize policy.\n ///\n /// If the current capacity is less than the lower limit, this is a no-op.\n ///\n /// # Examples\n ///\n /// ```\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// use tokio_util::task::JoinMap;\n ///\n /// let mut map: JoinMap = JoinMap::with_capacity(100);\n /// map.spawn(1, async move { 2 });\n /// map.spawn(3, async move { 4 });\n /// assert!(map.capacity() >= 100);\n /// map.shrink_to(10);\n /// assert!(map.capacity() >= 10);\n /// map.shrink_to(0);\n /// assert!(map.capacity() >= 2);\n /// # }\n /// ```\n #[inline]\n pub fn shrink_to(&mut self, min_capacity: usize) {\n self.hashes_by_task.shrink_to(min_capacity);\n self.tasks_by_key.shrink_to(min_capacity, |(k, _)| {\n self.hashes_by_task.hasher().hash_one(k)\n })\n }\n\n /// Look up a task in the map by its key, returning the key and abort handle.\n fn get_by_key<'map, Q>(&'map self, key: &Q) -> Option<&'map (K, AbortHandle)>\n where\n Q: ?Sized + Hash + Eq,\n K: Borrow,\n {\n let hash = self.hashes_by_task.hasher().hash_one(key);\n self.tasks_by_key.find(hash, |(k, _)| k.borrow() == key)\n }\n\n /// Remove a task from the map by ID, returning the key for that task.\n fn remove_by_id(&mut self, id: Id) -> Option {\n // Get the hash for the given ID.\n let hash = self.hashes_by_task.remove(&id)?;\n\n // Remove the entry for that hash.\n let entry = self\n .tasks_by_key\n .find_entry(hash, |(_, abort)| abort.id() == id);\n let (key, _) = match entry {\n Ok(entry) => entry.remove().0,\n _ => return None,\n };\n Some(key)\n }\n}\n\nimpl JoinMap\nwhere\n V: 'static,\n{\n /// Aborts all tasks on this `JoinMap`.\n ///\n /// This does not remove the tasks from the `JoinMap`. To wait for the tasks to complete\n /// cancellation, you should call `join_next` in a loop until the `JoinMap` is empty.\n pub fn abort_all(&mut self) {\n self.tasks.abort_all()\n }\n\n /// Removes all tasks from this `JoinMap` without aborting them.\n ///\n /// The tasks removed by this call will continue to run in the background even if the `JoinMap`\n /// is dropped. They may still be aborted by key.\n pub fn detach_all(&mut self) {\n self.tasks.detach_all();\n self.tasks_by_key.clear();\n self.hashes_by_task.clear();\n }\n}\n\n// Hand-written `fmt::Debug` implementation in order to avoid requiring `V:\n// Debug`, since no value is ever actually stored in the map.\nimpl fmt::Debug for JoinMap {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n // format the task keys and abort handles a little nicer by just\n // printing the key and task ID pairs, without format the `Key` struct\n // itself or the `AbortHandle`, which would just format the task's ID\n // again.\n struct KeySet<'a, K: fmt::Debug>(&'a HashTable<(K, AbortHandle)>);\n impl fmt::Debug for KeySet<'_, K> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_map()\n .entries(self.0.iter().map(|(key, abort)| (key, abort.id())))\n .finish()\n }\n }\n\n f.debug_struct(\"JoinMap\")\n // The `tasks_by_key` map is the only one that contains information\n // that's really worth formatting for the user, since it contains\n // the tasks' keys and IDs. The other fields are basically\n // implementation details.\n .field(\"tasks\", &KeySet(&self.tasks_by_key))\n .finish()\n }\n}\n\nimpl Default for JoinMap {\n fn default() -> Self {\n Self::new()\n }\n}\n\n/// An iterator over the keys of a [`JoinMap`].\n#[derive(Debug, Clone)]\npub struct JoinMapKeys<'a, K, V> {\n iter: hashbrown::hash_table::Iter<'a, (K, AbortHandle)>,\n /// To make it easier to change `JoinMap` in the future, keep V as a generic\n /// parameter.\n _value: PhantomData<&'a V>,\n}\n\nimpl<'a, K, V> Iterator for JoinMapKeys<'a, K, V> {\n type Item = &'a K;\n\n fn next(&mut self) -> Option<&'a K> {\n self.iter.next().map(|(key, _)| key)\n }\n\n fn size_hint(&self) -> (usize, Option) {\n self.iter.size_hint()\n }\n}\n\nimpl<'a, K, V> ExactSizeIterator for JoinMapKeys<'a, K, V> {\n fn len(&self) -> usize {\n self.iter.len()\n }\n}\n\nimpl<'a, K, V> std::iter::FusedIterator for JoinMapKeys<'a, K, V> {}\n" }, { "path": "tokio-util/src/task/join_queue.rs", "content": "use super::AbortOnDropHandle;\nuse std::{\n collections::VecDeque,\n future::Future,\n pin::Pin,\n task::{Context, Poll},\n};\nuse tokio::{\n runtime::Handle,\n task::{AbortHandle, Id, JoinError, JoinHandle},\n};\n\n/// A FIFO queue for of tasks spawned on a Tokio runtime.\n///\n/// A [`JoinQueue`] can be used to await the completion of the tasks in FIFO\n/// order. That is, if tasks are spawned in the order A, B, C, then\n/// awaiting the next completed task will always return A first, then B,\n/// then C, regardless of the order in which the tasks actually complete.\n///\n/// All of the tasks must have the same return type `T`.\n///\n/// When the [`JoinQueue`] is dropped, all tasks in the [`JoinQueue`] are\n/// immediately aborted.\npub struct JoinQueue(VecDeque>);\n\nimpl JoinQueue {\n /// Create a new empty [`JoinQueue`].\n pub const fn new() -> Self {\n Self(VecDeque::new())\n }\n\n /// Creates an empty [`JoinQueue`] with space for at least `capacity` tasks.\n pub fn with_capacity(capacity: usize) -> Self {\n Self(VecDeque::with_capacity(capacity))\n }\n\n /// Returns the number of tasks currently in the [`JoinQueue`].\n ///\n /// This includes both tasks that are currently running and tasks that have\n /// completed but not yet been removed from the queue because outputting of\n /// them waits for FIFO order.\n pub fn len(&self) -> usize {\n self.0.len()\n }\n\n /// Returns whether the [`JoinQueue`] is empty.\n pub fn is_empty(&self) -> bool {\n self.0.is_empty()\n }\n\n /// Spawn the provided task on the [`JoinQueue`], returning an [`AbortHandle`]\n /// that can be used to remotely cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// [`JoinQueue`].\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`AbortHandle`]: tokio::task::AbortHandle\n #[track_caller]\n pub fn spawn(&mut self, task: F) -> AbortHandle\n where\n F: Future + Send + 'static,\n T: Send + 'static,\n {\n self.push_back(tokio::spawn(task))\n }\n\n /// Spawn the provided task on the provided runtime and store it in this\n /// [`JoinQueue`] returning an [`AbortHandle`] that can be used to remotely\n /// cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// [`JoinQueue`].\n ///\n /// [`AbortHandle`]: tokio::task::AbortHandle\n #[track_caller]\n pub fn spawn_on(&mut self, task: F, handle: &Handle) -> AbortHandle\n where\n F: Future + Send + 'static,\n T: Send + 'static,\n {\n self.push_back(handle.spawn(task))\n }\n\n /// Spawn the provided task on the current [`LocalSet`] or [`LocalRuntime`]\n /// and store it in this [`JoinQueue`], returning an [`AbortHandle`] that\n /// can be used to remotely cancel the task.\n ///\n /// The provided future will start running in the background immediately\n /// when this method is called, even if you don't await anything on this\n /// [`JoinQueue`].\n ///\n /// # Panics\n ///\n /// This method panics if it is called outside of a `LocalSet` or `LocalRuntime`.\n ///\n /// [`LocalSet`]: tokio::task::LocalSet\n /// [`LocalRuntime`]: tokio::runtime::LocalRuntime\n /// [`AbortHandle`]: tokio::task::AbortHandle\n #[track_caller]\n pub fn spawn_local(&mut self, task: F) -> AbortHandle\n where\n F: Future + 'static,\n T: 'static,\n {\n self.push_back(tokio::task::spawn_local(task))\n }\n\n /// Spawn the blocking code on the blocking threadpool and store\n /// it in this [`JoinQueue`], returning an [`AbortHandle`] that can be\n /// used to remotely cancel the task.\n ///\n /// # Panics\n ///\n /// This method panics if called outside of a Tokio runtime.\n ///\n /// [`AbortHandle`]: tokio::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking(&mut self, f: F) -> AbortHandle\n where\n F: FnOnce() -> T + Send + 'static,\n T: Send + 'static,\n {\n self.push_back(tokio::task::spawn_blocking(f))\n }\n\n /// Spawn the blocking code on the blocking threadpool of the\n /// provided runtime and store it in this [`JoinQueue`], returning an\n /// [`AbortHandle`] that can be used to remotely cancel the task.\n ///\n /// [`AbortHandle`]: tokio::task::AbortHandle\n #[track_caller]\n pub fn spawn_blocking_on(&mut self, f: F, handle: &Handle) -> AbortHandle\n where\n F: FnOnce() -> T + Send + 'static,\n T: Send + 'static,\n {\n self.push_back(handle.spawn_blocking(f))\n }\n\n fn push_back(&mut self, jh: JoinHandle) -> AbortHandle {\n let jh = AbortOnDropHandle::new(jh);\n let abort_handle = jh.abort_handle();\n self.0.push_back(jh);\n abort_handle\n }\n\n /// Waits until the next task in FIFO order completes and returns its output.\n ///\n /// Returns `None` if the queue is empty.\n ///\n /// # Cancel Safety\n ///\n /// This method is cancel safe. If `join_next` is used as the event in a `tokio::select!`\n /// statement and some other branch completes first, it is guaranteed that no tasks were\n /// removed from this [`JoinQueue`].\n pub async fn join_next(&mut self) -> Option> {\n std::future::poll_fn(|cx| self.poll_join_next(cx)).await\n }\n\n /// Waits until the next task in FIFO order completes and returns its output,\n /// along with the [task ID] of the completed task.\n ///\n /// Returns `None` if the queue is empty.\n ///\n /// When this method returns an error, then the id of the task that failed can be accessed\n /// using the [`JoinError::id`] method.\n ///\n /// # Cancel Safety\n ///\n /// This method is cancel safe. If `join_next_with_id` is used as the event in a `tokio::select!`\n /// statement and some other branch completes first, it is guaranteed that no tasks were\n /// removed from this [`JoinQueue`].\n ///\n /// [task ID]: tokio::task::Id\n /// [`JoinError::id`]: fn@tokio::task::JoinError::id\n pub async fn join_next_with_id(&mut self) -> Option> {\n std::future::poll_fn(|cx| self.poll_join_next_with_id(cx)).await\n }\n\n /// Tries to poll an `AbortOnDropHandle` without blocking or yielding.\n ///\n /// Note that on success the handle will panic on subsequent polls\n /// since it becomes consumed.\n fn try_poll_handle(jh: &mut AbortOnDropHandle) -> Option> {\n let waker = futures_util::task::noop_waker();\n let mut cx = Context::from_waker(&waker);\n\n // Since this function is not async and cannot be forced to yield, we should\n // disable budgeting when we want to check for the `JoinHandle` readiness.\n let jh = std::pin::pin!(tokio::task::coop::unconstrained(jh));\n if let Poll::Ready(res) = jh.poll(&mut cx) {\n Some(res)\n } else {\n None\n }\n }\n\n /// Tries to join the next task in FIFO order if it has completed.\n ///\n /// Returns `None` if the queue is empty or if the next task is not yet ready.\n pub fn try_join_next(&mut self) -> Option> {\n let jh = self.0.front_mut()?;\n let res = Self::try_poll_handle(jh)?;\n // Use `detach` to avoid calling `abort` on a task that has already completed.\n // Dropping `AbortOnDropHandle` would abort the task, but since it is finished,\n // we only need to drop the `JoinHandle` for cleanup.\n drop(self.0.pop_front().unwrap().detach());\n Some(res)\n }\n\n /// Tries to join the next task in FIFO order if it has completed and return its output,\n /// along with its [task ID].\n ///\n /// Returns `None` if the queue is empty or if the next task is not yet ready.\n ///\n /// When this method returns an error, then the id of the task that failed can be accessed\n /// using the [`JoinError::id`] method.\n ///\n /// [task ID]: tokio::task::Id\n /// [`JoinError::id`]: fn@tokio::task::JoinError::id\n pub fn try_join_next_with_id(&mut self) -> Option> {\n let jh = self.0.front_mut()?;\n let res = Self::try_poll_handle(jh)?;\n // Use `detach` to avoid calling `abort` on a task that has already completed.\n // Dropping `AbortOnDropHandle` would abort the task, but since it is finished,\n // we only need to drop the `JoinHandle` for cleanup.\n let jh = self.0.pop_front().unwrap().detach();\n let id = jh.id();\n drop(jh);\n Some(res.map(|output| (id, output)))\n }\n\n /// Aborts all tasks and waits for them to finish shutting down.\n ///\n /// Calling this method is equivalent to calling [`abort_all`] and then calling [`join_next`] in\n /// a loop until it returns `None`.\n ///\n /// This method ignores any panics in the tasks shutting down. When this call returns, the\n /// [`JoinQueue`] will be empty.\n ///\n /// [`abort_all`]: fn@Self::abort_all\n /// [`join_next`]: fn@Self::join_next\n pub async fn shutdown(&mut self) {\n self.abort_all();\n while self.join_next().await.is_some() {}\n }\n\n /// Awaits the completion of all tasks in this [`JoinQueue`], returning a vector of their results.\n ///\n /// The results will be stored in the order they were spawned, not the order they completed.\n /// This is a convenience method that is equivalent to calling [`join_next`] in\n /// a loop. If any tasks on the [`JoinQueue`] fail with an [`JoinError`], then this call\n /// to `join_all` will panic and all remaining tasks on the [`JoinQueue`] are\n /// cancelled. To handle errors in any other way, manually call [`join_next`]\n /// in a loop.\n ///\n /// # Cancel Safety\n ///\n /// This method is not cancel safe as it calls `join_next` in a loop. If you need\n /// cancel safety, manually call `join_next` in a loop with `Vec` accumulator.\n ///\n /// [`join_next`]: fn@Self::join_next\n /// [`JoinError::id`]: fn@tokio::task::JoinError::id\n pub async fn join_all(mut self) -> Vec {\n let mut output = Vec::with_capacity(self.len());\n\n while let Some(res) = self.join_next().await {\n match res {\n Ok(t) => output.push(t),\n Err(err) if err.is_panic() => std::panic::resume_unwind(err.into_panic()),\n Err(err) => panic!(\"{err}\"),\n }\n }\n output\n }\n\n /// Aborts all tasks on this [`JoinQueue`].\n ///\n /// This does not remove the tasks from the [`JoinQueue`]. To wait for the tasks to complete\n /// cancellation, you should call `join_next` in a loop until the [`JoinQueue`] is empty.\n pub fn abort_all(&mut self) {\n self.0.iter().for_each(|jh| jh.abort());\n }\n\n /// Removes all tasks from this [`JoinQueue`] without aborting them.\n ///\n /// The tasks removed by this call will continue to run in the background even if the [`JoinQueue`]\n /// is dropped.\n pub fn detach_all(&mut self) {\n self.0.drain(..).for_each(|jh| drop(jh.detach()));\n }\n\n /// Polls for the next task in [`JoinQueue`] to complete.\n ///\n /// If this returns `Poll::Ready(Some(_))`, then the task that completed is removed from the queue.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided `Context` is scheduled\n /// to receive a wakeup when a task in the [`JoinQueue`] completes. Note that on multiple calls to\n /// `poll_join_next`, only the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// # Returns\n ///\n /// This function returns:\n ///\n /// * `Poll::Pending` if the [`JoinQueue`] is not empty but there is no task whose output is\n /// available right now.\n /// * `Poll::Ready(Some(Ok(value)))` if the next task in this [`JoinQueue`] has completed.\n /// The `value` is the return value that task.\n /// * `Poll::Ready(Some(Err(err)))` if the next task in this [`JoinQueue`] has panicked or been\n /// aborted. The `err` is the `JoinError` from the panicked/aborted task.\n /// * `Poll::Ready(None)` if the [`JoinQueue`] is empty.\n pub fn poll_join_next(&mut self, cx: &mut Context<'_>) -> Poll>> {\n let jh = match self.0.front_mut() {\n None => return Poll::Ready(None),\n Some(jh) => jh,\n };\n if let Poll::Ready(res) = Pin::new(jh).poll(cx) {\n // Use `detach` to avoid calling `abort` on a task that has already completed.\n // Dropping `AbortOnDropHandle` would abort the task, but since it is finished,\n // we only need to drop the `JoinHandle` for cleanup.\n drop(self.0.pop_front().unwrap().detach());\n Poll::Ready(Some(res))\n } else {\n Poll::Pending\n }\n }\n\n /// Polls for the next task in [`JoinQueue`] to complete.\n ///\n /// If this returns `Poll::Ready(Some(_))`, then the task that completed is removed from the queue.\n ///\n /// When the method returns `Poll::Pending`, the `Waker` in the provided `Context` is scheduled\n /// to receive a wakeup when a task in the [`JoinQueue`] completes. Note that on multiple calls to\n /// `poll_join_next`, only the `Waker` from the `Context` passed to the most recent call is\n /// scheduled to receive a wakeup.\n ///\n /// # Returns\n ///\n /// This function returns:\n ///\n /// * `Poll::Pending` if the [`JoinQueue`] is not empty but there is no task whose output is\n /// available right now.\n /// * `Poll::Ready(Some(Ok((id, value))))` if the next task in this [`JoinQueue`] has completed.\n /// The `value` is the return value that task, and `id` is its [task ID].\n /// * `Poll::Ready(Some(Err(err)))` if the next task in this [`JoinQueue`] has panicked or been\n /// aborted. The `err` is the `JoinError` from the panicked/aborted task.\n /// * `Poll::Ready(None)` if the [`JoinQueue`] is empty.\n ///\n /// [task ID]: tokio::task::Id\n pub fn poll_join_next_with_id(\n &mut self,\n cx: &mut Context<'_>,\n ) -> Poll>> {\n let jh = match self.0.front_mut() {\n None => return Poll::Ready(None),\n Some(jh) => jh,\n };\n if let Poll::Ready(res) = Pin::new(jh).poll(cx) {\n // Use `detach` to avoid calling `abort` on a task that has already completed.\n // Dropping `AbortOnDropHandle` would abort the task, but since it is finished,\n // we only need to drop the `JoinHandle` for cleanup.\n let jh = self.0.pop_front().unwrap().detach();\n let id = jh.id();\n drop(jh);\n // If the task succeeded, add the task ID to the output. Otherwise, the\n // `JoinError` will already have the task's ID.\n Poll::Ready(Some(res.map(|output| (id, output))))\n } else {\n Poll::Pending\n }\n }\n}\n\nimpl std::fmt::Debug for JoinQueue {\n fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n f.debug_list()\n .entries(self.0.iter().map(|jh| JoinHandle::id(jh.as_ref())))\n .finish()\n }\n}\n\nimpl Default for JoinQueue {\n fn default() -> Self {\n Self::new()\n }\n}\n\n/// Collect an iterator of futures into a [`JoinQueue`].\n///\n/// This is equivalent to calling [`JoinQueue::spawn`] on each element of the iterator.\nimpl std::iter::FromIterator for JoinQueue\nwhere\n F: Future + Send + 'static,\n T: Send + 'static,\n{\n fn from_iter>(iter: I) -> Self {\n let mut set = Self::new();\n iter.into_iter().for_each(|task| {\n set.spawn(task);\n });\n set\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n /// A simple type that does not implement [`std::fmt::Debug`].\n struct NotDebug;\n\n fn is_debug() {}\n\n #[test]\n fn assert_debug() {\n is_debug::>();\n }\n}\n" }, { "path": "tokio-util/src/task/mod.rs", "content": "//! Extra utilities for spawning tasks\n//!\n//! This module is only available when the `rt` feature is enabled. Note that enabling the\n//! `join-map` feature will automatically also enable the `rt` feature.\n\ncfg_rt! {\n mod spawn_pinned;\n pub use spawn_pinned::LocalPoolHandle;\n\n pub mod task_tracker;\n #[doc(inline)]\n pub use task_tracker::TaskTracker;\n\n mod abort_on_drop;\n pub use abort_on_drop::{AbortOnDrop, AbortOnDropHandle};\n\n mod join_queue;\n pub use join_queue::JoinQueue;\n}\n\n#[cfg(feature = \"join-map\")]\nmod join_map;\n#[cfg(feature = \"join-map\")]\n#[cfg_attr(docsrs, doc(cfg(feature = \"join-map\")))]\npub use join_map::{JoinMap, JoinMapKeys};\n" }, { "path": "tokio-util/src/task/spawn_pinned.rs", "content": "use futures_util::future::{AbortHandle, Abortable};\nuse std::fmt;\nuse std::fmt::{Debug, Formatter};\nuse std::future::Future;\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::Arc;\nuse tokio::runtime::Builder;\nuse tokio::sync::mpsc::{unbounded_channel, UnboundedReceiver, UnboundedSender};\nuse tokio::sync::oneshot;\nuse tokio::task::{spawn_local, JoinHandle, LocalSet};\n\n/// A cloneable handle to a local pool, used for spawning `!Send` tasks.\n///\n/// Internally the local pool uses a [`tokio::task::LocalSet`] for each worker thread\n/// in the pool. Consequently you can also use [`tokio::task::spawn_local`] (which will\n/// execute on the same thread) inside the Future you supply to the various spawn methods\n/// of `LocalPoolHandle`.\n///\n/// [`tokio::task::LocalSet`]: tokio::task::LocalSet\n/// [`tokio::task::spawn_local`]: tokio::task::spawn_local\n///\n/// # Examples\n///\n/// ```\n/// # #[cfg(not(target_family = \"wasm\"))]\n/// # {\n/// use std::rc::Rc;\n/// use tokio::task;\n/// use tokio_util::task::LocalPoolHandle;\n///\n/// #[tokio::main(flavor = \"current_thread\")]\n/// async fn main() {\n/// let pool = LocalPoolHandle::new(5);\n///\n/// let output = pool.spawn_pinned(|| {\n/// // `data` is !Send + !Sync\n/// let data = Rc::new(\"local data\");\n/// let data_clone = data.clone();\n///\n/// async move {\n/// task::spawn_local(async move {\n/// println!(\"{}\", data_clone);\n/// });\n///\n/// data.to_string()\n/// }\n/// }).await.unwrap();\n/// println!(\"output: {}\", output);\n/// }\n/// # }\n/// ```\n///\n#[derive(Clone)]\npub struct LocalPoolHandle {\n pool: Arc,\n}\n\nimpl LocalPoolHandle {\n /// Create a new pool of threads to handle `!Send` tasks. Spawn tasks onto this\n /// pool via [`LocalPoolHandle::spawn_pinned`].\n ///\n /// # Panics\n ///\n /// Panics if the pool size is less than one.\n #[track_caller]\n pub fn new(pool_size: usize) -> LocalPoolHandle {\n assert!(pool_size > 0);\n\n let workers = (0..pool_size)\n .map(|_| LocalWorkerHandle::new_worker())\n .collect();\n\n let pool = Arc::new(LocalPool { workers });\n\n LocalPoolHandle { pool }\n }\n\n /// Returns the number of threads of the Pool.\n #[inline]\n pub fn num_threads(&self) -> usize {\n self.pool.workers.len()\n }\n\n /// Returns the number of tasks scheduled on each worker. The indices of the\n /// worker threads correspond to the indices of the returned `Vec`.\n pub fn get_task_loads_for_each_worker(&self) -> Vec {\n self.pool\n .workers\n .iter()\n .map(|worker| worker.task_count.load(Ordering::SeqCst))\n .collect::>()\n }\n\n /// Spawn a task onto a worker thread and pin it there so it can't be moved\n /// off of the thread. Note that the future is not [`Send`], but the\n /// [`FnOnce`] which creates it is.\n ///\n /// # Examples\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use std::rc::Rc;\n /// use tokio_util::task::LocalPoolHandle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// // Create the local pool\n /// let pool = LocalPoolHandle::new(1);\n ///\n /// // Spawn a !Send future onto the pool and await it\n /// let output = pool\n /// .spawn_pinned(|| {\n /// // Rc is !Send + !Sync\n /// let local_data = Rc::new(\"test\");\n ///\n /// // This future holds an Rc, so it is !Send\n /// async move { local_data.to_string() }\n /// })\n /// .await\n /// .unwrap();\n ///\n /// assert_eq!(output, \"test\");\n /// }\n /// # }\n /// ```\n pub fn spawn_pinned(&self, create_task: F) -> JoinHandle\n where\n F: FnOnce() -> Fut,\n F: Send + 'static,\n Fut: Future + 'static,\n Fut::Output: Send + 'static,\n {\n self.pool\n .spawn_pinned(create_task, WorkerChoice::LeastBurdened)\n }\n\n /// Differs from `spawn_pinned` only in that you can choose a specific worker thread\n /// of the pool, whereas `spawn_pinned` chooses the worker with the smallest\n /// number of tasks scheduled.\n ///\n /// A worker thread is chosen by index. Indices are 0 based and the largest index\n /// is given by `num_threads() - 1`\n ///\n /// # Panics\n ///\n /// This method panics if the index is out of bounds.\n ///\n /// # Examples\n ///\n /// This method can be used to spawn a task on all worker threads of the pool:\n ///\n /// ```\n /// # #[cfg(not(target_family = \"wasm\"))]\n /// # {\n /// use tokio_util::task::LocalPoolHandle;\n ///\n /// #[tokio::main]\n /// async fn main() {\n /// const NUM_WORKERS: usize = 3;\n /// let pool = LocalPoolHandle::new(NUM_WORKERS);\n /// let handles = (0..pool.num_threads())\n /// .map(|worker_idx| {\n /// pool.spawn_pinned_by_idx(\n /// || {\n /// async {\n /// \"test\"\n /// }\n /// },\n /// worker_idx,\n /// )\n /// })\n /// .collect::>();\n ///\n /// for handle in handles {\n /// handle.await.unwrap();\n /// }\n /// }\n /// # }\n /// ```\n ///\n #[track_caller]\n pub fn spawn_pinned_by_idx(&self, create_task: F, idx: usize) -> JoinHandle\n where\n F: FnOnce() -> Fut,\n F: Send + 'static,\n Fut: Future + 'static,\n Fut::Output: Send + 'static,\n {\n self.pool\n .spawn_pinned(create_task, WorkerChoice::ByIdx(idx))\n }\n}\n\nimpl Debug for LocalPoolHandle {\n fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {\n f.write_str(\"LocalPoolHandle\")\n }\n}\n\nenum WorkerChoice {\n LeastBurdened,\n ByIdx(usize),\n}\n\nstruct LocalPool {\n workers: Box<[LocalWorkerHandle]>,\n}\n\nimpl LocalPool {\n /// Spawn a `?Send` future onto a worker\n #[track_caller]\n fn spawn_pinned(\n &self,\n create_task: F,\n worker_choice: WorkerChoice,\n ) -> JoinHandle\n where\n F: FnOnce() -> Fut,\n F: Send + 'static,\n Fut: Future + 'static,\n Fut::Output: Send + 'static,\n {\n let (sender, receiver) = oneshot::channel();\n let (worker, job_guard) = match worker_choice {\n WorkerChoice::LeastBurdened => self.find_and_incr_least_burdened_worker(),\n WorkerChoice::ByIdx(idx) => self.find_worker_by_idx(idx),\n };\n let worker_spawner = worker.spawner.clone();\n\n // Spawn a future onto the worker's runtime so we can immediately return\n // a join handle.\n worker.runtime_handle.spawn(async move {\n // Move the job guard into the task\n let _job_guard = job_guard;\n\n // Propagate aborts via Abortable/AbortHandle\n let (abort_handle, abort_registration) = AbortHandle::new_pair();\n let _abort_guard = AbortGuard(abort_handle);\n\n // Inside the future we can't run spawn_local yet because we're not\n // in the context of a LocalSet. We need to send create_task to the\n // LocalSet task for spawning.\n let spawn_task = Box::new(move || {\n // Once we're in the LocalSet context we can call spawn_local\n let join_handle =\n spawn_local(\n async move { Abortable::new(create_task(), abort_registration).await },\n );\n\n // Send the join handle back to the spawner. If sending fails,\n // we assume the parent task was canceled, so cancel this task\n // as well.\n if let Err(join_handle) = sender.send(join_handle) {\n join_handle.abort()\n }\n });\n\n // Send the callback to the LocalSet task\n if let Err(e) = worker_spawner.send(spawn_task) {\n // Propagate the error as a panic in the join handle.\n panic!(\"Failed to send job to worker: {e}\");\n }\n\n // Wait for the task's join handle\n let join_handle = match receiver.await {\n Ok(handle) => handle,\n Err(e) => {\n // We sent the task successfully, but failed to get its\n // join handle... We assume something happened to the worker\n // and the task was not spawned. Propagate the error as a\n // panic in the join handle.\n panic!(\"Worker failed to send join handle: {e}\");\n }\n };\n\n // Wait for the task to complete\n let join_result = join_handle.await;\n\n match join_result {\n Ok(Ok(output)) => output,\n Ok(Err(_)) => {\n // Pinned task was aborted. But that only happens if this\n // task is aborted. So this is an impossible branch.\n unreachable!(\n \"Reaching this branch means this task was previously \\\n aborted but it continued running anyways\"\n )\n }\n Err(e) => {\n if e.is_panic() {\n std::panic::resume_unwind(e.into_panic());\n } else if e.is_cancelled() {\n // No one else should have the join handle, so this is\n // unexpected. Forward this error as a panic in the join\n // handle.\n panic!(\"spawn_pinned task was canceled: {e}\");\n } else {\n // Something unknown happened (not a panic or\n // cancellation). Forward this error as a panic in the\n // join handle.\n panic!(\"spawn_pinned task failed: {e}\");\n }\n }\n }\n })\n }\n\n /// Find the worker with the least number of tasks, increment its task\n /// count, and return its handle. Make sure to actually spawn a task on\n /// the worker so the task count is kept consistent with load.\n ///\n /// A job count guard is also returned to ensure the task count gets\n /// decremented when the job is done.\n fn find_and_incr_least_burdened_worker(&self) -> (&LocalWorkerHandle, JobCountGuard) {\n loop {\n let (worker, task_count) = self\n .workers\n .iter()\n .map(|worker| (worker, worker.task_count.load(Ordering::SeqCst)))\n .min_by_key(|&(_, count)| count)\n .expect(\"There must be more than one worker\");\n\n // Make sure the task count hasn't changed since when we choose this\n // worker. Otherwise, restart the search.\n if worker\n .task_count\n .compare_exchange(\n task_count,\n task_count + 1,\n Ordering::SeqCst,\n Ordering::Relaxed,\n )\n .is_ok()\n {\n return (worker, JobCountGuard(Arc::clone(&worker.task_count)));\n }\n }\n }\n\n #[track_caller]\n fn find_worker_by_idx(&self, idx: usize) -> (&LocalWorkerHandle, JobCountGuard) {\n let worker = &self.workers[idx];\n worker.task_count.fetch_add(1, Ordering::SeqCst);\n\n (worker, JobCountGuard(Arc::clone(&worker.task_count)))\n }\n}\n\n/// Automatically decrements a worker's job count when a job finishes (when\n/// this gets dropped).\nstruct JobCountGuard(Arc);\n\nimpl Drop for JobCountGuard {\n fn drop(&mut self) {\n // Decrement the job count\n let previous_value = self.0.fetch_sub(1, Ordering::SeqCst);\n debug_assert!(previous_value >= 1);\n }\n}\n\n/// Calls abort on the handle when dropped.\nstruct AbortGuard(AbortHandle);\n\nimpl Drop for AbortGuard {\n fn drop(&mut self) {\n self.0.abort();\n }\n}\n\ntype PinnedFutureSpawner = Box;\n\nstruct LocalWorkerHandle {\n runtime_handle: tokio::runtime::Handle,\n spawner: UnboundedSender,\n task_count: Arc,\n}\n\nimpl LocalWorkerHandle {\n /// Create a new worker for executing pinned tasks\n fn new_worker() -> LocalWorkerHandle {\n let (sender, receiver) = unbounded_channel();\n let runtime = Builder::new_current_thread()\n .enable_all()\n .build()\n .expect(\"Failed to start a pinned worker thread runtime\");\n let runtime_handle = runtime.handle().clone();\n let task_count = Arc::new(AtomicUsize::new(0));\n let task_count_clone = Arc::clone(&task_count);\n\n std::thread::spawn(|| Self::run(runtime, receiver, task_count_clone));\n\n LocalWorkerHandle {\n runtime_handle,\n spawner: sender,\n task_count,\n }\n }\n\n fn run(\n runtime: tokio::runtime::Runtime,\n mut task_receiver: UnboundedReceiver,\n task_count: Arc,\n ) {\n let local_set = LocalSet::new();\n local_set.block_on(&runtime, async {\n while let Some(spawn_task) = task_receiver.recv().await {\n // Calls spawn_local(future)\n (spawn_task)();\n }\n });\n\n // If there are any tasks on the runtime associated with a LocalSet task\n // that has already completed, but whose output has not yet been\n // reported, let that task complete.\n //\n // Since the task_count is decremented when the runtime task exits,\n // reading that counter lets us know if any such tasks completed during\n // the call to `block_on`.\n //\n // Tasks on the LocalSet can't complete during this loop since they're\n // stored on the LocalSet and we aren't accessing it.\n let mut previous_task_count = task_count.load(Ordering::SeqCst);\n loop {\n // This call will also run tasks spawned on the runtime.\n runtime.block_on(tokio::task::yield_now());\n let new_task_count = task_count.load(Ordering::SeqCst);\n if new_task_count == previous_task_count {\n break;\n } else {\n previous_task_count = new_task_count;\n }\n }\n\n // It's now no longer possible for a task on the runtime to be\n // associated with a LocalSet task that has completed. Drop both the\n // LocalSet and runtime to let tasks on the runtime be cancelled if and\n // only if they are still on the LocalSet.\n //\n // Drop the LocalSet task first so that anyone awaiting the runtime\n // JoinHandle will see the cancelled error after the LocalSet task\n // destructor has completed.\n drop(local_set);\n drop(runtime);\n }\n}\n" }, { "path": "tokio-util/src/task/task_tracker.rs", "content": "//! Types related to the [`TaskTracker`] collection.\n//!\n//! See the documentation of [`TaskTracker`] for more information.\n\nuse pin_project_lite::pin_project;\nuse std::fmt;\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::sync::atomic::{AtomicUsize, Ordering};\nuse std::sync::Arc;\nuse std::task::{Context, Poll};\nuse tokio::sync::{futures::Notified, Notify};\n\n#[cfg(feature = \"rt\")]\nuse tokio::{\n runtime::Handle,\n task::{JoinHandle, LocalSet},\n};\n\n/// A task tracker used for waiting until tasks exit.\n///\n/// This is usually used together with [`CancellationToken`] to implement [graceful shutdown]. The\n/// `CancellationToken` is used to signal to tasks that they should shut down, and the\n/// `TaskTracker` is used to wait for them to finish shutting down.\n///\n/// The `TaskTracker` will also keep track of a `closed` boolean. This is used to handle the case\n/// where the `TaskTracker` is empty, but we don't want to shut down yet. This means that the\n/// [`wait`] method will wait until *both* of the following happen at the same time:\n///\n/// * The `TaskTracker` must be closed using the [`close`] method.\n/// * The `TaskTracker` must be empty, that is, all tasks that it is tracking must have exited.\n///\n/// When a call to [`wait`] returns, it is guaranteed that all tracked tasks have exited and that\n/// the destructor of the future has finished running. However, there might be a short amount of\n/// time where [`JoinHandle::is_finished`] returns false.\n///\n/// # Comparison to `JoinSet`\n///\n/// The main Tokio crate has a similar collection known as [`JoinSet`]. The `JoinSet` type has a\n/// lot more features than `TaskTracker`, so `TaskTracker` should only be used when one of its\n/// unique features is required:\n///\n/// 1. When tasks exit, a `TaskTracker` will allow the task to immediately free its memory.\n/// 2. By not closing the `TaskTracker`, [`wait`] will be prevented from returning even if\n/// the `TaskTracker` is empty.\n/// 3. A `TaskTracker` does not require mutable access to insert tasks.\n/// 4. A `TaskTracker` can be cloned to share it with many tasks.\n///\n/// The first point is the most important one. A [`JoinSet`] keeps track of the return value of\n/// every inserted task. This means that if the caller keeps inserting tasks and never calls\n/// [`join_next`], then their return values will keep building up and consuming memory, _even if_\n/// most of the tasks have already exited. This can cause the process to run out of memory. With a\n/// `TaskTracker`, this does not happen. Once tasks exit, they are immediately removed from the\n/// `TaskTracker`.\n///\n/// Note that unlike [`JoinSet`], dropping a `TaskTracker` does not abort the tasks.\n///\n/// # Examples\n///\n/// For more examples, please see the topic page on [graceful shutdown].\n///\n/// ## Spawn tasks and wait for them to exit\n///\n/// This is a simple example. For this case, [`JoinSet`] should probably be used instead.\n///\n/// ```\n/// use tokio_util::task::TaskTracker;\n///\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() {\n/// let tracker = TaskTracker::new();\n///\n/// for i in 0..10 {\n/// tracker.spawn(async move {\n/// println!(\"Task {} is running!\", i);\n/// });\n/// }\n/// // Once we spawned everything, we close the tracker.\n/// tracker.close();\n///\n/// // Wait for everything to finish.\n/// tracker.wait().await;\n///\n/// println!(\"This is printed after all of the tasks.\");\n/// # }\n/// ```\n///\n/// ## Wait for tasks to exit\n///\n/// This example shows the intended use-case of `TaskTracker`. It is used together with\n/// [`CancellationToken`] to implement graceful shutdown.\n/// ```\n/// use tokio_util::sync::CancellationToken;\n/// use tokio_util::task::TaskTracker;\n/// use tokio_util::time::FutureExt;\n///\n/// use tokio::time::{self, Duration};\n///\n/// async fn background_task(num: u64) {\n/// for i in 0..10 {\n/// time::sleep(Duration::from_millis(100*num)).await;\n/// println!(\"Background task {} in iteration {}.\", num, i);\n/// }\n/// }\n///\n/// #[tokio::main]\n/// # async fn _hidden() {}\n/// # #[tokio::main(flavor = \"current_thread\", start_paused = true)]\n/// async fn main() {\n/// let tracker = TaskTracker::new();\n/// let token = CancellationToken::new();\n///\n/// for i in 0..10 {\n/// let token = token.clone();\n/// tracker.spawn(async move {\n/// // Use a `with_cancellation_token_owned` to kill the background task\n/// // if the token is cancelled.\n/// match background_task(i)\n/// .with_cancellation_token_owned(token)\n/// .await\n/// {\n/// Some(()) => println!(\"Task {} exiting normally.\", i),\n/// None => {\n/// // Do some cleanup before we really exit.\n/// time::sleep(Duration::from_millis(50)).await;\n/// println!(\"Task {} finished cleanup.\", i);\n/// }\n/// }\n/// });\n/// }\n///\n/// // Spawn a background task that will send the shutdown signal.\n/// {\n/// let tracker = tracker.clone();\n/// tokio::spawn(async move {\n/// // Normally you would use something like ctrl-c instead of\n/// // sleeping.\n/// time::sleep(Duration::from_secs(2)).await;\n/// tracker.close();\n/// token.cancel();\n/// });\n/// }\n///\n/// // Wait for all tasks to exit.\n/// tracker.wait().await;\n///\n/// println!(\"All tasks have exited now.\");\n/// }\n/// ```\n///\n/// [`CancellationToken`]: crate::sync::CancellationToken\n/// [`JoinHandle::is_finished`]: tokio::task::JoinHandle::is_finished\n/// [`JoinSet`]: tokio::task::JoinSet\n/// [`close`]: Self::close\n/// [`join_next`]: tokio::task::JoinSet::join_next\n/// [`wait`]: Self::wait\n/// [graceful shutdown]: https://tokio.rs/tokio/topics/shutdown\npub struct TaskTracker {\n inner: Arc,\n}\n\n/// Represents a task tracked by a [`TaskTracker`].\n#[must_use]\n#[derive(Debug)]\npub struct TaskTrackerToken {\n task_tracker: TaskTracker,\n}\n\nstruct TaskTrackerInner {\n /// Keeps track of the state.\n ///\n /// The lowest bit is whether the task tracker is closed.\n ///\n /// The rest of the bits count the number of tracked tasks.\n state: AtomicUsize,\n /// Used to notify when the last task exits.\n on_last_exit: Notify,\n}\n\npin_project! {\n /// A future that is tracked as a task by a [`TaskTracker`].\n ///\n /// The associated [`TaskTracker`] cannot complete until this future is dropped.\n ///\n /// This future is returned by [`TaskTracker::track_future`].\n #[must_use = \"futures do nothing unless polled\"]\n pub struct TrackedFuture {\n #[pin]\n future: F,\n token: TaskTrackerToken,\n }\n}\n\npin_project! {\n /// A future that completes when the [`TaskTracker`] is empty and closed.\n ///\n /// This future is returned by [`TaskTracker::wait`].\n #[must_use = \"futures do nothing unless polled\"]\n pub struct TaskTrackerWaitFuture<'a> {\n #[pin]\n future: Notified<'a>,\n inner: Option<&'a TaskTrackerInner>,\n }\n}\n\nimpl TaskTrackerInner {\n #[inline]\n fn new() -> Self {\n Self {\n state: AtomicUsize::new(0),\n on_last_exit: Notify::new(),\n }\n }\n\n #[inline]\n fn is_closed_and_empty(&self) -> bool {\n // If empty and closed bit set, then we are done.\n //\n // The acquire load will synchronize with the release store of any previous call to\n // `set_closed` and `drop_task`.\n self.state.load(Ordering::Acquire) == 1\n }\n\n #[inline]\n fn set_closed(&self) -> bool {\n // The AcqRel ordering makes the closed bit behave like a `Mutex` for synchronization\n // purposes. We do this because it makes the return value of `TaskTracker::{close,reopen}`\n // more meaningful for the user. Without these orderings, this assert could fail:\n // ```\n // // thread 1\n // some_other_atomic.store(true, Relaxed);\n // tracker.close();\n //\n // // thread 2\n // if tracker.reopen() {\n // assert!(some_other_atomic.load(Relaxed));\n // }\n // ```\n // However, with the AcqRel ordering, we establish a happens-before relationship from the\n // call to `close` and the later call to `reopen` that returned true.\n let state = self.state.fetch_or(1, Ordering::AcqRel);\n\n // If there are no tasks, and if it was not already closed:\n if state == 0 {\n self.notify_now();\n }\n\n (state & 1) == 0\n }\n\n #[inline]\n fn set_open(&self) -> bool {\n // See `set_closed` regarding the AcqRel ordering.\n let state = self.state.fetch_and(!1, Ordering::AcqRel);\n (state & 1) == 1\n }\n\n #[inline]\n fn add_task(&self) {\n self.state.fetch_add(2, Ordering::Relaxed);\n }\n\n #[inline]\n fn drop_task(&self) {\n let state = self.state.fetch_sub(2, Ordering::Release);\n\n // If this was the last task and we are closed:\n if state == 3 {\n self.notify_now();\n }\n }\n\n #[cold]\n fn notify_now(&self) {\n // Insert an acquire fence. This matters for `drop_task` but doesn't matter for\n // `set_closed` since it already uses AcqRel.\n //\n // This synchronizes with the release store of any other call to `drop_task`, and with the\n // release store in the call to `set_closed`. That ensures that everything that happened\n // before those other calls to `drop_task` or `set_closed` will be visible after this load,\n // and those things will also be visible to anything woken by the call to `notify_waiters`.\n self.state.load(Ordering::Acquire);\n\n self.on_last_exit.notify_waiters();\n }\n}\n\nimpl TaskTracker {\n /// Creates a new `TaskTracker`.\n ///\n /// The `TaskTracker` will start out as open.\n #[must_use]\n pub fn new() -> Self {\n Self {\n inner: Arc::new(TaskTrackerInner::new()),\n }\n }\n\n /// Waits until this `TaskTracker` is both closed and empty.\n ///\n /// If the `TaskTracker` is already closed and empty when this method is called, then it\n /// returns immediately.\n ///\n /// The `wait` future is resistant against [ABA problems][aba]. That is, if the `TaskTracker`\n /// becomes both closed and empty for a short amount of time, then it is guarantee that all\n /// `wait` futures that were created before the short time interval will trigger, even if they\n /// are not polled during that short time interval.\n ///\n /// # Cancel safety\n ///\n /// This method is cancel safe.\n ///\n /// However, the resistance against [ABA problems][aba] is lost when using `wait` as the\n /// condition in a `tokio::select!` loop.\n ///\n /// [aba]: https://en.wikipedia.org/wiki/ABA_problem\n #[inline]\n pub fn wait(&self) -> TaskTrackerWaitFuture<'_> {\n TaskTrackerWaitFuture {\n future: self.inner.on_last_exit.notified(),\n inner: if self.inner.is_closed_and_empty() {\n None\n } else {\n Some(&self.inner)\n },\n }\n }\n\n /// Close this `TaskTracker`.\n ///\n /// This allows [`wait`] futures to complete. It does not prevent you from spawning new tasks.\n ///\n /// Returns `true` if this closed the `TaskTracker`, or `false` if it was already closed.\n ///\n /// [`wait`]: Self::wait\n #[inline]\n pub fn close(&self) -> bool {\n self.inner.set_closed()\n }\n\n /// Reopen this `TaskTracker`.\n ///\n /// This prevents [`wait`] futures from completing even if the `TaskTracker` is empty.\n ///\n /// Returns `true` if this reopened the `TaskTracker`, or `false` if it was already open.\n ///\n /// [`wait`]: Self::wait\n #[inline]\n pub fn reopen(&self) -> bool {\n self.inner.set_open()\n }\n\n /// Returns `true` if this `TaskTracker` is [closed](Self::close).\n #[inline]\n #[must_use]\n pub fn is_closed(&self) -> bool {\n (self.inner.state.load(Ordering::Acquire) & 1) != 0\n }\n\n /// Returns the number of tasks tracked by this `TaskTracker`.\n #[inline]\n #[must_use]\n pub fn len(&self) -> usize {\n self.inner.state.load(Ordering::Acquire) >> 1\n }\n\n /// Returns `true` if there are no tasks in this `TaskTracker`.\n #[inline]\n #[must_use]\n pub fn is_empty(&self) -> bool {\n self.inner.state.load(Ordering::Acquire) <= 1\n }\n\n /// Spawn the provided future on the current Tokio runtime, and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `tokio::spawn(tracker.track_future(task))`.\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn(&self, task: F) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n tokio::task::spawn(self.track_future(task))\n }\n\n /// Spawn the provided future on the provided Tokio runtime, and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `handle.spawn(tracker.track_future(task))`.\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn_on(&self, task: F, handle: &Handle) -> JoinHandle\n where\n F: Future + Send + 'static,\n F::Output: Send + 'static,\n {\n handle.spawn(self.track_future(task))\n }\n\n /// Spawn the provided future on the current [`LocalSet`] or [`LocalRuntime`]\n /// and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `tokio::task::spawn_local(tracker.track_future(task))`.\n ///\n /// # Panics\n ///\n /// This method panics if it is called outside of a `LocalSet` or `LocalRuntime`.\n ///\n /// [`LocalSet`]: tokio::task::LocalSet\n /// [`LocalRuntime`]: tokio::runtime::LocalRuntime\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn_local(&self, task: F) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n tokio::task::spawn_local(self.track_future(task))\n }\n\n /// Spawn the provided future on the provided [`LocalSet`], and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `local_set.spawn_local(tracker.track_future(task))`.\n ///\n /// [`LocalSet`]: tokio::task::LocalSet\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn_local_on(&self, task: F, local_set: &LocalSet) -> JoinHandle\n where\n F: Future + 'static,\n F::Output: 'static,\n {\n local_set.spawn_local(self.track_future(task))\n }\n\n /// Spawn the provided blocking task on the current Tokio runtime, and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `tokio::task::spawn_blocking(tracker.track_future(task))`.\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg(not(target_family = \"wasm\"))]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn_blocking(&self, task: F) -> JoinHandle\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send + 'static,\n {\n let token = self.token();\n tokio::task::spawn_blocking(move || {\n let res = task();\n drop(token);\n res\n })\n }\n\n /// Spawn the provided blocking task on the provided Tokio runtime, and track it in this `TaskTracker`.\n ///\n /// This is equivalent to `handle.spawn_blocking(tracker.track_future(task))`.\n #[inline]\n #[track_caller]\n #[cfg(feature = \"rt\")]\n #[cfg(not(target_family = \"wasm\"))]\n #[cfg_attr(docsrs, doc(cfg(feature = \"rt\")))]\n pub fn spawn_blocking_on(&self, task: F, handle: &Handle) -> JoinHandle\n where\n F: FnOnce() -> T,\n F: Send + 'static,\n T: Send + 'static,\n {\n let token = self.token();\n handle.spawn_blocking(move || {\n let res = task();\n drop(token);\n res\n })\n }\n\n /// Track the provided future.\n ///\n /// The returned [`TrackedFuture`] will count as a task tracked by this collection, and will\n /// prevent calls to [`wait`] from returning until the task is dropped.\n ///\n /// The task is removed from the collection when it is dropped, not when [`poll`] returns\n /// [`Poll::Ready`].\n ///\n /// # Examples\n ///\n /// Track a future spawned with [`tokio::spawn`].\n ///\n /// ```\n /// # async fn my_async_fn() {}\n /// use tokio_util::task::TaskTracker;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let tracker = TaskTracker::new();\n ///\n /// tokio::spawn(tracker.track_future(my_async_fn()));\n /// # }\n /// ```\n ///\n /// Track a future spawned on a [`JoinSet`].\n /// ```\n /// # async fn my_async_fn() {}\n /// use tokio::task::JoinSet;\n /// use tokio_util::task::TaskTracker;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let tracker = TaskTracker::new();\n /// let mut join_set = JoinSet::new();\n ///\n /// join_set.spawn(tracker.track_future(my_async_fn()));\n /// # }\n /// ```\n ///\n /// [`JoinSet`]: tokio::task::JoinSet\n /// [`Poll::Pending`]: std::task::Poll::Pending\n /// [`poll`]: std::future::Future::poll\n /// [`wait`]: Self::wait\n #[inline]\n pub fn track_future(&self, future: F) -> TrackedFuture {\n TrackedFuture {\n future,\n token: self.token(),\n }\n }\n\n /// Creates a [`TaskTrackerToken`] representing a task tracked by this `TaskTracker`.\n ///\n /// This token is a lower-level utility than the spawn methods. Each token is considered to\n /// correspond to a task. As long as the token exists, the `TaskTracker` cannot complete.\n /// Furthermore, the count returned by the [`len`] method will include the tokens in the count.\n ///\n /// Dropping the token indicates to the `TaskTracker` that the task has exited.\n ///\n /// [`len`]: TaskTracker::len\n #[inline]\n pub fn token(&self) -> TaskTrackerToken {\n self.inner.add_task();\n TaskTrackerToken {\n task_tracker: self.clone(),\n }\n }\n\n /// Returns `true` if both task trackers correspond to the same set of tasks.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::TaskTracker;\n ///\n /// let tracker_1 = TaskTracker::new();\n /// let tracker_2 = TaskTracker::new();\n /// let tracker_1_clone = tracker_1.clone();\n ///\n /// assert!(TaskTracker::ptr_eq(&tracker_1, &tracker_1_clone));\n /// assert!(!TaskTracker::ptr_eq(&tracker_1, &tracker_2));\n /// ```\n #[inline]\n #[must_use]\n pub fn ptr_eq(left: &TaskTracker, right: &TaskTracker) -> bool {\n Arc::ptr_eq(&left.inner, &right.inner)\n }\n}\n\nimpl Default for TaskTracker {\n /// Creates a new `TaskTracker`.\n ///\n /// The `TaskTracker` will start out as open.\n #[inline]\n fn default() -> TaskTracker {\n TaskTracker::new()\n }\n}\n\nimpl Clone for TaskTracker {\n /// Returns a new `TaskTracker` that tracks the same set of tasks.\n ///\n /// Since the new `TaskTracker` shares the same set of tasks, changes to one set are visible in\n /// all other clones.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::task::TaskTracker;\n ///\n /// #[tokio::main]\n /// # async fn _hidden() {}\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// async fn main() {\n /// let tracker = TaskTracker::new();\n /// let cloned = tracker.clone();\n ///\n /// // Spawns on `tracker` are visible in `cloned`.\n /// tracker.spawn(std::future::pending::<()>());\n /// assert_eq!(cloned.len(), 1);\n ///\n /// // Spawns on `cloned` are visible in `tracker`.\n /// cloned.spawn(std::future::pending::<()>());\n /// assert_eq!(tracker.len(), 2);\n ///\n /// // Calling `close` is visible to `cloned`.\n /// tracker.close();\n /// assert!(cloned.is_closed());\n ///\n /// // Calling `reopen` is visible to `tracker`.\n /// cloned.reopen();\n /// assert!(!tracker.is_closed());\n /// }\n /// ```\n #[inline]\n fn clone(&self) -> TaskTracker {\n Self {\n inner: self.inner.clone(),\n }\n }\n}\n\nfn debug_inner(inner: &TaskTrackerInner, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n let state = inner.state.load(Ordering::Acquire);\n let is_closed = (state & 1) != 0;\n let len = state >> 1;\n\n f.debug_struct(\"TaskTracker\")\n .field(\"len\", &len)\n .field(\"is_closed\", &is_closed)\n .field(\"inner\", &(inner as *const TaskTrackerInner))\n .finish()\n}\n\nimpl fmt::Debug for TaskTracker {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n debug_inner(&self.inner, f)\n }\n}\n\nimpl TaskTrackerToken {\n /// Returns the [`TaskTracker`] that this token is associated with.\n #[inline]\n #[must_use]\n pub fn task_tracker(&self) -> &TaskTracker {\n &self.task_tracker\n }\n}\n\nimpl Clone for TaskTrackerToken {\n /// Returns a new `TaskTrackerToken` associated with the same [`TaskTracker`].\n ///\n /// This is equivalent to `token.task_tracker().token()`.\n #[inline]\n fn clone(&self) -> TaskTrackerToken {\n self.task_tracker.token()\n }\n}\n\nimpl Drop for TaskTrackerToken {\n /// Dropping the token indicates to the [`TaskTracker`] that the task has exited.\n #[inline]\n fn drop(&mut self) {\n self.task_tracker.inner.drop_task();\n }\n}\n\nimpl Future for TrackedFuture {\n type Output = F::Output;\n\n #[inline]\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n self.project().future.poll(cx)\n }\n}\n\nimpl fmt::Debug for TrackedFuture {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n f.debug_struct(\"TrackedFuture\")\n .field(\"future\", &self.future)\n .field(\"task_tracker\", self.token.task_tracker())\n .finish()\n }\n}\n\nimpl<'a> Future for TaskTrackerWaitFuture<'a> {\n type Output = ();\n\n #[inline]\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {\n let me = self.project();\n\n let inner = match me.inner.as_ref() {\n None => return Poll::Ready(()),\n Some(inner) => inner,\n };\n\n let ready = inner.is_closed_and_empty() || me.future.poll(cx).is_ready();\n if ready {\n *me.inner = None;\n Poll::Ready(())\n } else {\n Poll::Pending\n }\n }\n}\n\nimpl<'a> fmt::Debug for TaskTrackerWaitFuture<'a> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n struct Helper<'a>(&'a TaskTrackerInner);\n\n impl fmt::Debug for Helper<'_> {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n debug_inner(self.0, f)\n }\n }\n\n f.debug_struct(\"TaskTrackerWaitFuture\")\n .field(\"future\", &self.future)\n .field(\"task_tracker\", &self.inner.map(Helper))\n .finish()\n }\n}\n" }, { "path": "tokio-util/src/time/delay_queue.rs", "content": "//! A queue of delayed elements.\n//!\n//! See [`DelayQueue`] for more details.\n//!\n//! [`DelayQueue`]: struct@DelayQueue\n\nuse crate::time::wheel::{self, Wheel};\n\nuse tokio::time::{sleep_until, Duration, Instant, Sleep};\n\nuse core::ops::{Index, IndexMut};\nuse slab::Slab;\nuse std::cmp;\nuse std::collections::HashMap;\nuse std::convert::From;\nuse std::fmt;\nuse std::fmt::Debug;\nuse std::future::Future;\nuse std::marker::PhantomData;\nuse std::pin::Pin;\nuse std::task::{self, ready, Poll, Waker};\n\n/// A queue of delayed elements.\n///\n/// Once an element is inserted into the `DelayQueue`, it is yielded once the\n/// specified deadline has been reached.\n///\n/// # Usage\n///\n/// Elements are inserted into `DelayQueue` using the [`insert`] or\n/// [`insert_at`] methods. A deadline is provided with the item and a [`Key`] is\n/// returned. The key is used to remove the entry or to change the deadline at\n/// which it should be yielded back.\n///\n/// Once delays have been configured, the `DelayQueue` is used via its\n/// [`Stream`] implementation. [`poll_expired`] is called. If an entry has reached its\n/// deadline, it is returned. If not, `Poll::Pending` is returned indicating that the\n/// current task will be notified once the deadline has been reached.\n///\n/// # `Stream` implementation\n///\n/// Items are retrieved from the queue via [`DelayQueue::poll_expired`]. If no delays have\n/// expired, no items are returned. In this case, [`Poll::Pending`] is returned and the\n/// current task is registered to be notified once the next item's delay has\n/// expired.\n///\n/// If no items are in the queue, i.e. `is_empty()` returns `true`, then `poll`\n/// returns `Poll::Ready(None)`. This indicates that the stream has reached an end.\n/// However, if a new item is inserted *after*, `poll` will once again start\n/// returning items or `Poll::Pending`.\n///\n/// Items are returned ordered by their expirations. Items that are configured\n/// to expire first will be returned first. There are no ordering guarantees\n/// for items configured to expire at the same instant. Also note that delays are\n/// rounded to the closest millisecond.\n///\n/// # Implementation\n///\n/// The [`DelayQueue`] is backed by a separate instance of a timer wheel similar to that used internally\n/// by Tokio's standalone timer utilities such as [`sleep`]. Because of this, it offers the same\n/// performance and scalability benefits.\n///\n/// State associated with each entry is stored in a [`slab`]. This amortizes the cost of allocation,\n/// and allows reuse of the memory allocated for expired entries.\n///\n/// Capacity can be checked using [`capacity`] and allocated preemptively by using\n/// the [`reserve`] method.\n///\n/// # Cancellation safety\n///\n/// [`DelayQueue`]'s implementation of [`StreamExt::next`] is cancellation safe.\n///\n/// # Usage\n///\n/// Using [`DelayQueue`] to manage cache entries.\n///\n/// ```rust,no_run\n/// use tokio_util::time::{DelayQueue, delay_queue};\n///\n/// use std::collections::HashMap;\n/// use std::task::{ready, Context, Poll};\n/// use std::time::Duration;\n/// # type CacheKey = String;\n/// # type Value = String;\n///\n/// struct Cache {\n/// entries: HashMap,\n/// expirations: DelayQueue,\n/// }\n///\n/// const TTL_SECS: u64 = 30;\n///\n/// impl Cache {\n/// fn insert(&mut self, key: CacheKey, value: Value) {\n/// let delay = self.expirations\n/// .insert(key.clone(), Duration::from_secs(TTL_SECS));\n///\n/// self.entries.insert(key, (value, delay));\n/// }\n///\n/// fn get(&self, key: &CacheKey) -> Option<&Value> {\n/// self.entries.get(key)\n/// .map(|&(ref v, _)| v)\n/// }\n///\n/// fn remove(&mut self, key: &CacheKey) {\n/// if let Some((_, cache_key)) = self.entries.remove(key) {\n/// self.expirations.remove(&cache_key);\n/// }\n/// }\n///\n/// fn poll_purge(&mut self, cx: &mut Context<'_>) -> Poll<()> {\n/// while let Some(entry) = ready!(self.expirations.poll_expired(cx)) {\n/// self.entries.remove(entry.get_ref());\n/// }\n///\n/// Poll::Ready(())\n/// }\n/// }\n/// ```\n///\n/// [`insert`]: method@Self::insert\n/// [`insert_at`]: method@Self::insert_at\n/// [`Key`]: struct@Key\n/// [`Stream`]: https://docs.rs/futures/0.3.31/futures/stream/trait.Stream.html\n/// [`StreamExt::next`]: https://docs.rs/tokio-stream/0.1.17/tokio_stream/trait.StreamExt.html#method.next\n/// [`poll_expired`]: method@Self::poll_expired\n/// [`Stream::poll_expired`]: method@Self::poll_expired\n/// [`DelayQueue`]: struct@DelayQueue\n/// [`sleep`]: fn@tokio::time::sleep\n/// [`slab`]: slab\n/// [`capacity`]: method@Self::capacity\n/// [`reserve`]: method@Self::reserve\n#[derive(Debug)]\npub struct DelayQueue {\n /// Stores data associated with entries\n slab: SlabStorage,\n\n /// Lookup structure tracking all delays in the queue\n wheel: Wheel>,\n\n /// Delays that were inserted when already expired. These cannot be stored\n /// in the wheel\n expired: Stack,\n\n /// Delay expiring when the *first* item in the queue expires\n delay: Option>>,\n\n /// Wheel polling state\n wheel_now: u64,\n\n /// Instant at which the timer starts\n start: Instant,\n\n /// Waker that is invoked when we potentially need to reset the timer.\n /// Because we lazily create the timer when the first entry is created, we\n /// need to awaken any poller that polled us before that point.\n waker: Option,\n}\n\n#[derive(Default)]\nstruct SlabStorage {\n inner: Slab>,\n\n // A `compact` call requires a re-mapping of the `Key`s that were changed\n // during the `compact` call of the `slab`. Since the keys that were given out\n // cannot be changed retroactively we need to keep track of these re-mappings.\n // The keys of `key_map` correspond to the old keys that were given out and\n // the values to the `Key`s that were re-mapped by the `compact` call.\n key_map: HashMap,\n\n // Index used to create new keys to hand out.\n next_key_index: usize,\n\n // Whether `compact` has been called, necessary in order to decide whether\n // to include keys in `key_map`.\n compact_called: bool,\n}\n\nimpl SlabStorage {\n pub(crate) fn with_capacity(capacity: usize) -> SlabStorage {\n SlabStorage {\n inner: Slab::with_capacity(capacity),\n key_map: HashMap::new(),\n next_key_index: 0,\n compact_called: false,\n }\n }\n\n // Inserts data into the inner slab and re-maps keys if necessary\n pub(crate) fn insert(&mut self, val: Data) -> Key {\n let mut key = KeyInternal::new(self.inner.insert(val));\n let key_contained = self.key_map.contains_key(&key.into());\n\n if key_contained {\n // It's possible that a `compact` call creates capacity in `self.inner` in\n // such a way that a `self.inner.insert` call creates a `key` which was\n // previously given out during an `insert` call prior to the `compact` call.\n // If `key` is contained in `self.key_map`, we have encountered this exact situation,\n // We need to create a new key `key_to_give_out` and include the relation\n // `key_to_give_out` -> `key` in `self.key_map`.\n let key_to_give_out = self.create_new_key();\n assert!(!self.key_map.contains_key(&key_to_give_out.into()));\n self.key_map.insert(key_to_give_out.into(), key);\n key = key_to_give_out;\n } else if self.compact_called {\n // Include an identity mapping in `self.key_map` in order to allow us to\n // panic if a key that was handed out is removed more than once.\n self.key_map.insert(key.into(), key);\n }\n\n key.into()\n }\n\n // Re-map the key in case compact was previously called.\n // Note: Since we include identity mappings in key_map after compact was called,\n // we have information about all keys that were handed out. In the case in which\n // compact was called and we try to remove a Key that was previously removed\n // we can detect invalid keys if no key is found in `key_map`. This is necessary\n // in order to prevent situations in which a previously removed key\n // corresponds to a re-mapped key internally and which would then be incorrectly\n // removed from the slab.\n //\n // Example to illuminate this problem:\n //\n // Let's assume our `key_map` is {1 -> 2, 2 -> 1} and we call remove(1). If we\n // were to remove 1 again, we would not find it inside `key_map` anymore.\n // If we were to imply from this that no re-mapping was necessary, we would\n // incorrectly remove 1 from `self.slab.inner`, which corresponds to the\n // handed-out key 2.\n pub(crate) fn remove(&mut self, key: &Key) -> Data {\n let remapped_key = if self.compact_called {\n match self.key_map.remove(key) {\n Some(key_internal) => key_internal,\n None => panic!(\"invalid key\"),\n }\n } else {\n (*key).into()\n };\n\n self.inner.remove(remapped_key.index)\n }\n\n pub(crate) fn shrink_to_fit(&mut self) {\n self.inner.shrink_to_fit();\n self.key_map.shrink_to_fit();\n }\n\n pub(crate) fn compact(&mut self) {\n if !self.compact_called {\n for (key, _) in self.inner.iter() {\n self.key_map.insert(Key::new(key), KeyInternal::new(key));\n }\n }\n\n let mut remapping = HashMap::new();\n self.inner.compact(|_, from, to| {\n remapping.insert(from, to);\n true\n });\n\n // At this point `key_map` contains a mapping for every element.\n for internal_key in self.key_map.values_mut() {\n if let Some(new_internal_key) = remapping.get(&internal_key.index) {\n *internal_key = KeyInternal::new(*new_internal_key);\n }\n }\n\n if self.key_map.capacity() > 2 * self.key_map.len() {\n self.key_map.shrink_to_fit();\n }\n\n self.compact_called = true;\n }\n\n // Tries to re-map a `Key` that was given out to the user to its\n // corresponding internal key.\n fn remap_key(&self, key: &Key) -> Option {\n let key_map = &self.key_map;\n if self.compact_called {\n key_map.get(key).copied()\n } else {\n Some((*key).into())\n }\n }\n\n fn create_new_key(&mut self) -> KeyInternal {\n while self.key_map.contains_key(&Key::new(self.next_key_index)) {\n self.next_key_index = self.next_key_index.wrapping_add(1);\n }\n\n KeyInternal::new(self.next_key_index)\n }\n\n pub(crate) fn len(&self) -> usize {\n self.inner.len()\n }\n\n pub(crate) fn capacity(&self) -> usize {\n self.inner.capacity()\n }\n\n pub(crate) fn clear(&mut self) {\n self.inner.clear();\n self.key_map.clear();\n self.compact_called = false;\n }\n\n pub(crate) fn reserve(&mut self, additional: usize) {\n self.inner.reserve(additional);\n\n if self.compact_called {\n self.key_map.reserve(additional);\n }\n }\n\n pub(crate) fn is_empty(&self) -> bool {\n self.inner.is_empty()\n }\n\n pub(crate) fn contains(&self, key: &Key) -> bool {\n let remapped_key = self.remap_key(key);\n\n match remapped_key {\n Some(internal_key) => self.inner.contains(internal_key.index),\n None => false,\n }\n }\n}\n\nimpl fmt::Debug for SlabStorage\nwhere\n T: fmt::Debug,\n{\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n if fmt.alternate() {\n fmt.debug_map().entries(self.inner.iter()).finish()\n } else {\n fmt.debug_struct(\"Slab\")\n .field(\"len\", &self.len())\n .field(\"cap\", &self.capacity())\n .finish()\n }\n }\n}\n\nimpl Index for SlabStorage {\n type Output = Data;\n\n fn index(&self, key: Key) -> &Self::Output {\n let remapped_key = self.remap_key(&key);\n\n match remapped_key {\n Some(internal_key) => &self.inner[internal_key.index],\n None => panic!(\"Invalid index {}\", key.index),\n }\n }\n}\n\nimpl IndexMut for SlabStorage {\n fn index_mut(&mut self, key: Key) -> &mut Data {\n let remapped_key = self.remap_key(&key);\n\n match remapped_key {\n Some(internal_key) => &mut self.inner[internal_key.index],\n None => panic!(\"Invalid index {}\", key.index),\n }\n }\n}\n\n/// An entry in `DelayQueue` that has expired and been removed.\n///\n/// Values are returned by [`DelayQueue::poll_expired`].\n///\n/// [`DelayQueue::poll_expired`]: method@DelayQueue::poll_expired\n#[derive(Debug)]\npub struct Expired {\n /// The data stored in the queue\n data: T,\n\n /// The expiration time\n deadline: Instant,\n\n /// The key associated with the entry\n key: Key,\n}\n\n/// Token to a value stored in a `DelayQueue`.\n///\n/// Instances of `Key` are returned by [`DelayQueue::insert`]. See [`DelayQueue`]\n/// documentation for more details.\n///\n/// [`DelayQueue`]: struct@DelayQueue\n/// [`DelayQueue::insert`]: method@DelayQueue::insert\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\npub struct Key {\n index: usize,\n}\n\n// Whereas `Key` is given out to users that use `DelayQueue`, internally we use\n// `KeyInternal` as the key type in order to make the logic of mapping between keys\n// as a result of `compact` calls clearer.\n#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]\nstruct KeyInternal {\n index: usize,\n}\n\n#[derive(Debug)]\nstruct Stack {\n /// Head of the stack\n head: Option,\n _p: PhantomData T>,\n}\n\n#[derive(Debug)]\nstruct Data {\n /// The data being stored in the queue and will be returned at the requested\n /// instant.\n inner: T,\n\n /// The instant at which the item is returned.\n when: u64,\n\n /// Set to true when stored in the `expired` queue\n expired: bool,\n\n /// Next entry in the stack\n next: Option,\n\n /// Previous entry in the stack\n prev: Option,\n}\n\n/// Maximum number of entries the queue can handle\nconst MAX_ENTRIES: usize = (1 << 30) - 1;\n\nimpl DelayQueue {\n /// Creates a new, empty, `DelayQueue`.\n ///\n /// The queue will not allocate storage until items are inserted into it.\n ///\n /// # Examples\n ///\n /// ```rust\n /// # use tokio_util::time::DelayQueue;\n /// let delay_queue: DelayQueue = DelayQueue::new();\n /// ```\n pub fn new() -> DelayQueue {\n DelayQueue::with_capacity(0)\n }\n\n /// Creates a new, empty, `DelayQueue` with the specified capacity.\n ///\n /// The queue will be able to hold at least `capacity` elements without\n /// reallocating. If `capacity` is 0, the queue will not allocate for\n /// storage.\n ///\n /// # Examples\n ///\n /// ```rust\n /// # use tokio_util::time::DelayQueue;\n /// # use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::with_capacity(10);\n ///\n /// // These insertions are done without further allocation\n /// for i in 0..10 {\n /// delay_queue.insert(i, Duration::from_secs(i));\n /// }\n ///\n /// // This will make the queue allocate additional storage\n /// delay_queue.insert(11, Duration::from_secs(11));\n /// # }\n /// ```\n pub fn with_capacity(capacity: usize) -> DelayQueue {\n DelayQueue {\n wheel: Wheel::new(),\n slab: SlabStorage::with_capacity(capacity),\n expired: Stack::default(),\n delay: None,\n wheel_now: 0,\n start: Instant::now(),\n waker: None,\n }\n }\n\n /// Inserts `value` into the queue set to expire at a specific instant in\n /// time.\n ///\n /// This function is identical to `insert`, but takes an `Instant` instead\n /// of a `Duration`.\n ///\n /// `value` is stored in the queue until `when` is reached. At which point,\n /// `value` will be returned from [`poll_expired`]. If `when` has already been\n /// reached, then `value` is immediately made available to poll.\n ///\n /// The return value represents the insertion and is used as an argument to\n /// [`remove`] and [`reset`]. Note that [`Key`] is a token and is reused once\n /// `value` is removed from the queue either by calling [`poll_expired`] after\n /// `when` is reached or by calling [`remove`]. At this point, the caller\n /// must take care to not use the returned [`Key`] again as it may reference\n /// a different item in the queue.\n ///\n /// See [type] level documentation for more details.\n ///\n /// # Panics\n ///\n /// This function panics if `when` is too far in the future.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio::time::{Duration, Instant};\n /// use tokio_util::time::DelayQueue;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert_at(\n /// \"foo\", Instant::now() + Duration::from_secs(5));\n ///\n /// // Remove the entry\n /// let item = delay_queue.remove(&key);\n /// assert_eq!(*item.get_ref(), \"foo\");\n /// # }\n /// ```\n ///\n /// [`poll_expired`]: method@Self::poll_expired\n /// [`remove`]: method@Self::remove\n /// [`reset`]: method@Self::reset\n /// [`Key`]: struct@Key\n /// [type]: #\n #[track_caller]\n pub fn insert_at(&mut self, value: T, when: Instant) -> Key {\n assert!(self.slab.len() < MAX_ENTRIES, \"max entries exceeded\");\n\n // Normalize the deadline. Values cannot be set to expire in the past.\n let when = self.normalize_deadline(when);\n\n // Insert the value in the store\n let key = self.slab.insert(Data {\n inner: value,\n when,\n expired: false,\n next: None,\n prev: None,\n });\n\n self.insert_idx(when, key);\n\n // Set a new delay if the current's deadline is later than the one of the new item\n let should_set_delay = if let Some(ref delay) = self.delay {\n let current_exp = self.normalize_deadline(delay.deadline());\n current_exp > when\n } else {\n true\n };\n\n if should_set_delay {\n if let Some(waker) = self.waker.take() {\n waker.wake();\n }\n\n let delay_time = self.start + Duration::from_millis(when);\n if let Some(ref mut delay) = &mut self.delay {\n delay.as_mut().reset(delay_time);\n } else {\n self.delay = Some(Box::pin(sleep_until(delay_time)));\n }\n }\n\n key\n }\n\n /// Attempts to pull out the next value of the delay queue, registering the\n /// current task for wakeup if the value is not yet available, and returning\n /// `None` if the queue is exhausted.\n pub fn poll_expired(&mut self, cx: &mut task::Context<'_>) -> Poll>> {\n if !self\n .waker\n .as_ref()\n .map(|w| w.will_wake(cx.waker()))\n .unwrap_or(false)\n {\n self.waker = Some(cx.waker().clone());\n }\n\n let item = ready!(self.poll_idx(cx));\n Poll::Ready(item.map(|key| {\n let data = self.slab.remove(&key);\n debug_assert!(data.next.is_none());\n debug_assert!(data.prev.is_none());\n\n Expired {\n key,\n data: data.inner,\n deadline: self.start + Duration::from_millis(data.when),\n }\n }))\n }\n\n /// Inserts `value` into the queue set to expire after the requested duration\n /// elapses.\n ///\n /// This function is identical to `insert_at`, but takes a `Duration`\n /// instead of an `Instant`.\n ///\n /// `value` is stored in the queue until `timeout` duration has\n /// elapsed after `insert` was called. At that point, `value` will\n /// be returned from [`poll_expired`]. If `timeout` is a `Duration` of\n /// zero, then `value` is immediately made available to poll.\n ///\n /// The return value represents the insertion and is used as an\n /// argument to [`remove`] and [`reset`]. Note that [`Key`] is a\n /// token and is reused once `value` is removed from the queue\n /// either by calling [`poll_expired`] after `timeout` has elapsed\n /// or by calling [`remove`]. At this point, the caller must not\n /// use the returned [`Key`] again as it may reference a different\n /// item in the queue.\n ///\n /// See [type] level documentation for more details.\n ///\n /// # Panics\n ///\n /// This function panics if `timeout` is greater than the maximum\n /// duration supported by the timer in the current `Runtime`.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// // Remove the entry\n /// let item = delay_queue.remove(&key);\n /// assert_eq!(*item.get_ref(), \"foo\");\n /// # }\n /// ```\n ///\n /// [`poll_expired`]: method@Self::poll_expired\n /// [`remove`]: method@Self::remove\n /// [`reset`]: method@Self::reset\n /// [`Key`]: struct@Key\n /// [type]: #\n #[track_caller]\n pub fn insert(&mut self, value: T, timeout: Duration) -> Key {\n self.insert_at(value, Instant::now() + timeout)\n }\n\n #[track_caller]\n fn insert_idx(&mut self, when: u64, key: Key) {\n use self::wheel::{InsertError, Stack};\n\n // Register the deadline with the timer wheel\n match self.wheel.insert(when, key, &mut self.slab) {\n Ok(_) => {}\n Err((_, InsertError::Elapsed)) => {\n self.slab[key].expired = true;\n // The delay is already expired, store it in the expired queue\n self.expired.push(key, &mut self.slab);\n }\n Err((_, err)) => panic!(\"invalid deadline; err={err:?}\"),\n }\n }\n\n /// Returns the deadline of the item associated with `key`.\n ///\n /// Since the queue operates at millisecond granularity, the returned\n /// deadline may not exactly match the value that was given when initially\n /// inserting the item into the queue.\n ///\n /// # Panics\n ///\n /// This function panics if `key` is not contained by the queue.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n ///\n /// let key1 = delay_queue.insert(\"foo\", Duration::from_secs(5));\n /// let key2 = delay_queue.insert(\"bar\", Duration::from_secs(10));\n ///\n /// assert!(delay_queue.deadline(&key1) < delay_queue.deadline(&key2));\n /// # }\n /// ```\n #[track_caller]\n pub fn deadline(&self, key: &Key) -> Instant {\n self.start + Duration::from_millis(self.slab[*key].when)\n }\n\n /// Removes the key from the expired queue or the timer wheel\n /// depending on its expiration status.\n ///\n /// # Panics\n ///\n /// Panics if the key is not contained in the expired queue or the wheel.\n #[track_caller]\n fn remove_key(&mut self, key: &Key) {\n use crate::time::wheel::Stack;\n\n // Special case the `expired` queue\n if self.slab[*key].expired {\n self.expired.remove(key, &mut self.slab);\n } else {\n self.wheel.remove(key, &mut self.slab);\n }\n }\n\n /// Removes the item associated with `key` from the queue.\n ///\n /// There must be an item associated with `key`. The function returns the\n /// removed item as well as the `Instant` at which it will the delay will\n /// have expired.\n ///\n /// # Panics\n ///\n /// The function panics if `key` is not contained by the queue.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// // Remove the entry\n /// let item = delay_queue.remove(&key);\n /// assert_eq!(*item.get_ref(), \"foo\");\n /// # }\n /// ```\n #[track_caller]\n pub fn remove(&mut self, key: &Key) -> Expired {\n let prev_deadline = self.next_deadline();\n\n self.remove_key(key);\n let data = self.slab.remove(key);\n\n let next_deadline = self.next_deadline();\n if prev_deadline != next_deadline {\n match (next_deadline, &mut self.delay) {\n (None, _) => self.delay = None,\n (Some(deadline), Some(delay)) => delay.as_mut().reset(deadline),\n (Some(deadline), None) => self.delay = Some(Box::pin(sleep_until(deadline))),\n }\n }\n\n if self.slab.is_empty() {\n if let Some(waker) = self.waker.take() {\n waker.wake();\n }\n }\n\n Expired {\n key: Key::new(key.index),\n data: data.inner,\n deadline: self.start + Duration::from_millis(data.when),\n }\n }\n\n /// Attempts to remove the item associated with `key` from the queue.\n ///\n /// Removes the item associated with `key`, and returns it along with the\n /// `Instant` at which it would have expired, if it exists.\n ///\n /// Returns `None` if `key` is not in the queue.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// // The item is in the queue, `try_remove` returns `Some(Expired(\"foo\"))`.\n /// let item = delay_queue.try_remove(&key);\n /// assert_eq!(item.unwrap().into_inner(), \"foo\");\n ///\n /// // The item is not in the queue anymore, `try_remove` returns `None`.\n /// let item = delay_queue.try_remove(&key);\n /// assert!(item.is_none());\n /// # }\n /// ```\n pub fn try_remove(&mut self, key: &Key) -> Option> {\n if self.slab.contains(key) {\n Some(self.remove(key))\n } else {\n None\n }\n }\n\n /// Sets the delay of the item associated with `key` to expire at `when`.\n ///\n /// This function is identical to `reset` but takes an `Instant` instead of\n /// a `Duration`.\n ///\n /// The item remains in the queue but the delay is set to expire at `when`.\n /// If `when` is in the past, then the item is immediately made available to\n /// the caller.\n ///\n /// # Panics\n ///\n /// This function panics if `when` is too far in the future or if `key` is\n /// not contained by the queue.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio::time::{Duration, Instant};\n /// use tokio_util::time::DelayQueue;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// // \"foo\" is scheduled to be returned in 5 seconds\n ///\n /// delay_queue.reset_at(&key, Instant::now() + Duration::from_secs(10));\n ///\n /// // \"foo\" is now scheduled to be returned in 10 seconds\n /// # }\n /// ```\n #[track_caller]\n pub fn reset_at(&mut self, key: &Key, when: Instant) {\n self.remove_key(key);\n\n // Normalize the deadline. Values cannot be set to expire in the past.\n let when = self.normalize_deadline(when);\n\n self.slab[*key].when = when;\n self.slab[*key].expired = false;\n\n self.insert_idx(when, *key);\n\n let next_deadline = self.next_deadline();\n if let (Some(ref mut delay), Some(deadline)) = (&mut self.delay, next_deadline) {\n // This should awaken us if necessary (ie, if already expired)\n delay.as_mut().reset(deadline);\n }\n }\n\n /// Shrink the capacity of the slab, which `DelayQueue` uses internally for storage allocation.\n /// This function is not guaranteed to, and in most cases, won't decrease the capacity of the slab\n /// to the number of elements still contained in it, because elements cannot be moved to a different\n /// index. To decrease the capacity to the size of the slab use [`compact`].\n ///\n /// This function can take O(n) time even when the capacity cannot be reduced or the allocation is\n /// shrunk in place. Repeated calls run in O(1) though.\n ///\n /// [`compact`]: method@Self::compact\n pub fn shrink_to_fit(&mut self) {\n self.slab.shrink_to_fit();\n }\n\n /// Shrink the capacity of the slab, which `DelayQueue` uses internally for storage allocation,\n /// to the number of elements that are contained in it.\n ///\n /// This methods runs in O(n).\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::with_capacity(10);\n ///\n /// let key1 = delay_queue.insert(5, Duration::from_secs(5));\n /// let key2 = delay_queue.insert(10, Duration::from_secs(10));\n /// let key3 = delay_queue.insert(15, Duration::from_secs(15));\n ///\n /// delay_queue.remove(&key2);\n ///\n /// delay_queue.compact();\n /// assert_eq!(delay_queue.capacity(), 2);\n /// # }\n /// ```\n pub fn compact(&mut self) {\n self.slab.compact();\n }\n\n /// Gets the [`Key`] that [`poll_expired`] will pull out of the queue next, without\n /// pulling it out or waiting for the deadline to expire.\n ///\n /// Entries that have already expired may be returned in any order, but it is\n /// guaranteed that this method returns them in the same order as when items\n /// are popped from the `DelayQueue`.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n ///\n /// let key1 = delay_queue.insert(\"foo\", Duration::from_secs(10));\n /// let key2 = delay_queue.insert(\"bar\", Duration::from_secs(5));\n /// let key3 = delay_queue.insert(\"baz\", Duration::from_secs(15));\n ///\n /// assert_eq!(delay_queue.peek().unwrap(), key2);\n /// # }\n /// ```\n ///\n /// [`Key`]: struct@Key\n /// [`poll_expired`]: method@Self::poll_expired\n pub fn peek(&self) -> Option {\n use self::wheel::Stack;\n\n self.expired.peek().or_else(|| self.wheel.peek())\n }\n\n /// Returns the next time to poll as determined by the wheel.\n ///\n /// Note that this does not include deadlines in the `expired` queue.\n fn next_deadline(&self) -> Option {\n self.wheel\n .poll_at()\n .map(|poll_at| self.start + Duration::from_millis(poll_at))\n }\n\n /// Sets the delay of the item associated with `key` to expire after\n /// `timeout`.\n ///\n /// This function is identical to `reset_at` but takes a `Duration` instead\n /// of an `Instant`.\n ///\n /// The item remains in the queue but the delay is set to expire after\n /// `timeout`. If `timeout` is zero, then the item is immediately made\n /// available to the caller.\n ///\n /// # Panics\n ///\n /// This function panics if `timeout` is greater than the maximum supported\n /// duration or if `key` is not contained by the queue.\n ///\n /// # Examples\n ///\n /// Basic usage\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// let key = delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// // \"foo\" is scheduled to be returned in 5 seconds\n ///\n /// delay_queue.reset(&key, Duration::from_secs(10));\n ///\n /// // \"foo\"is now scheduled to be returned in 10 seconds\n /// # }\n /// ```\n #[track_caller]\n pub fn reset(&mut self, key: &Key, timeout: Duration) {\n self.reset_at(key, Instant::now() + timeout);\n }\n\n /// Clears the queue, removing all items.\n ///\n /// After calling `clear`, [`poll_expired`] will return `Ok(Ready(None))`.\n ///\n /// Note that this method has no effect on the allocated capacity.\n ///\n /// [`poll_expired`]: method@Self::poll_expired\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n ///\n /// delay_queue.insert(\"foo\", Duration::from_secs(5));\n ///\n /// assert!(!delay_queue.is_empty());\n ///\n /// delay_queue.clear();\n ///\n /// assert!(delay_queue.is_empty());\n /// # }\n /// ```\n pub fn clear(&mut self) {\n self.slab.clear();\n self.expired = Stack::default();\n self.wheel = Wheel::new();\n self.delay = None;\n }\n\n /// Returns the number of elements the queue can hold without reallocating.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n ///\n /// let delay_queue: DelayQueue = DelayQueue::with_capacity(10);\n /// assert_eq!(delay_queue.capacity(), 10);\n /// ```\n pub fn capacity(&self) -> usize {\n self.slab.capacity()\n }\n\n /// Returns the number of elements currently in the queue.\n ///\n /// # Examples\n ///\n /// ```rust\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue: DelayQueue = DelayQueue::with_capacity(10);\n /// assert_eq!(delay_queue.len(), 0);\n /// delay_queue.insert(3, Duration::from_secs(5));\n /// assert_eq!(delay_queue.len(), 1);\n /// # }\n /// ```\n pub fn len(&self) -> usize {\n self.slab.len()\n }\n\n /// Reserves capacity for at least `additional` more items to be queued\n /// without allocating.\n ///\n /// `reserve` does nothing if the queue already has sufficient capacity for\n /// `additional` more values. If more capacity is required, a new segment of\n /// memory will be allocated and all existing values will be copied into it.\n /// As such, if the queue is already very large, a call to `reserve` can end\n /// up being expensive.\n ///\n /// The queue may reserve more than `additional` extra space in order to\n /// avoid frequent reallocations.\n ///\n /// # Panics\n ///\n /// Panics if the new capacity exceeds the maximum number of entries the\n /// queue can contain.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n ///\n /// delay_queue.insert(\"hello\", Duration::from_secs(10));\n /// delay_queue.reserve(10);\n ///\n /// assert!(delay_queue.capacity() >= 11);\n /// # }\n /// ```\n #[track_caller]\n pub fn reserve(&mut self, additional: usize) {\n assert!(\n self.slab.capacity() + additional <= MAX_ENTRIES,\n \"max queue capacity exceeded\"\n );\n self.slab.reserve(additional);\n }\n\n /// Returns `true` if there are no items in the queue.\n ///\n /// Note that this function returns `false` even if all items have not yet\n /// expired and a call to `poll` will return `Poll::Pending`.\n ///\n /// # Examples\n ///\n /// ```\n /// use tokio_util::time::DelayQueue;\n /// use std::time::Duration;\n ///\n /// # #[tokio::main(flavor = \"current_thread\")]\n /// # async fn main() {\n /// let mut delay_queue = DelayQueue::new();\n /// assert!(delay_queue.is_empty());\n ///\n /// delay_queue.insert(\"hello\", Duration::from_secs(5));\n /// assert!(!delay_queue.is_empty());\n /// # }\n /// ```\n pub fn is_empty(&self) -> bool {\n self.slab.is_empty()\n }\n\n /// Polls the queue, returning the index of the next slot in the slab that\n /// should be returned.\n ///\n /// A slot should be returned when the associated deadline has been reached.\n fn poll_idx(&mut self, cx: &mut task::Context<'_>) -> Poll> {\n use self::wheel::Stack;\n\n let expired = self.expired.pop(&mut self.slab);\n\n if expired.is_some() {\n return Poll::Ready(expired);\n }\n\n loop {\n if let Some(ref mut delay) = self.delay {\n if !delay.is_elapsed() {\n ready!(Pin::new(&mut *delay).poll(cx));\n }\n\n let now = crate::time::ms(delay.deadline() - self.start, crate::time::Round::Down);\n\n self.wheel_now = now;\n }\n\n // We poll the wheel to get the next value out before finding the next deadline.\n let wheel_idx = self.wheel.poll(self.wheel_now, &mut self.slab);\n\n self.delay = self.next_deadline().map(|when| Box::pin(sleep_until(when)));\n\n if let Some(idx) = wheel_idx {\n return Poll::Ready(Some(idx));\n }\n\n if self.delay.is_none() {\n return Poll::Ready(None);\n }\n }\n }\n\n fn normalize_deadline(&self, when: Instant) -> u64 {\n let when = if when < self.start {\n 0\n } else {\n crate::time::ms(when - self.start, crate::time::Round::Up)\n };\n\n cmp::max(when, self.wheel.elapsed())\n }\n}\n\n// We never put `T` in a `Pin`...\nimpl Unpin for DelayQueue {}\n\nimpl Default for DelayQueue {\n fn default() -> DelayQueue {\n DelayQueue::new()\n }\n}\n\nimpl futures_core::Stream for DelayQueue {\n // DelayQueue seems much more specific, where a user may care that it\n // has reached capacity, so return those errors instead of panicking.\n type Item = Expired;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll> {\n DelayQueue::poll_expired(self.get_mut(), cx)\n }\n}\n\nimpl wheel::Stack for Stack {\n type Owned = Key;\n type Borrowed = Key;\n type Store = SlabStorage;\n\n fn is_empty(&self) -> bool {\n self.head.is_none()\n }\n\n fn push(&mut self, item: Self::Owned, store: &mut Self::Store) {\n // Ensure the entry is not already in a stack.\n debug_assert!(store[item].next.is_none());\n debug_assert!(store[item].prev.is_none());\n\n // Remove the old head entry\n let old = self.head.take();\n\n if let Some(idx) = old {\n store[idx].prev = Some(item);\n }\n\n store[item].next = old;\n self.head = Some(item);\n }\n\n fn pop(&mut self, store: &mut Self::Store) -> Option {\n if let Some(key) = self.head {\n self.head = store[key].next;\n\n if let Some(idx) = self.head {\n store[idx].prev = None;\n }\n\n store[key].next = None;\n debug_assert!(store[key].prev.is_none());\n\n Some(key)\n } else {\n None\n }\n }\n\n fn peek(&self) -> Option {\n self.head\n }\n\n #[track_caller]\n fn remove(&mut self, item: &Self::Borrowed, store: &mut Self::Store) {\n let key = *item;\n assert!(store.contains(item));\n\n // Ensure that the entry is in fact contained by the stack\n debug_assert!({\n // This walks the full linked list even if an entry is found.\n let mut next = self.head;\n let mut contains = false;\n\n while let Some(idx) = next {\n let data = &store[idx];\n\n if idx == *item {\n debug_assert!(!contains);\n contains = true;\n }\n\n next = data.next;\n }\n\n contains\n });\n\n if let Some(next) = store[key].next {\n store[next].prev = store[key].prev;\n }\n\n if let Some(prev) = store[key].prev {\n store[prev].next = store[key].next;\n } else {\n self.head = store[key].next;\n }\n\n store[key].next = None;\n store[key].prev = None;\n }\n\n fn when(item: &Self::Borrowed, store: &Self::Store) -> u64 {\n store[*item].when\n }\n}\n\nimpl Default for Stack {\n fn default() -> Stack {\n Stack {\n head: None,\n _p: PhantomData,\n }\n }\n}\n\nimpl Key {\n pub(crate) fn new(index: usize) -> Key {\n Key { index }\n }\n}\n\nimpl KeyInternal {\n pub(crate) fn new(index: usize) -> KeyInternal {\n KeyInternal { index }\n }\n}\n\nimpl From for KeyInternal {\n fn from(item: Key) -> Self {\n KeyInternal::new(item.index)\n }\n}\n\nimpl From for Key {\n fn from(item: KeyInternal) -> Self {\n Key::new(item.index)\n }\n}\n\nimpl Expired {\n /// Returns a reference to the inner value.\n pub fn get_ref(&self) -> &T {\n &self.data\n }\n\n /// Returns a mutable reference to the inner value.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.data\n }\n\n /// Consumes `self` and returns the inner value.\n pub fn into_inner(self) -> T {\n self.data\n }\n\n /// Returns the deadline that the expiration was set to.\n pub fn deadline(&self) -> Instant {\n self.deadline\n }\n\n /// Returns the key that the expiration is indexed by.\n pub fn key(&self) -> Key {\n self.key\n }\n}\n" }, { "path": "tokio-util/src/time/mod.rs", "content": "//! Additional utilities for tracking time.\n//!\n//! This module provides additional utilities for executing code after a set period\n//! of time. Currently there is only one:\n//!\n//! * `DelayQueue`: A queue where items are returned once the requested delay\n//! has expired.\n//!\n//! This type must be used from within the context of the `Runtime`.\n\nuse std::time::Duration;\n\nmod wheel;\n\npub mod delay_queue;\n\n// re-export `FutureExt` to avoid breaking change\n#[doc(inline)]\npub use crate::future::FutureExt;\n\n#[doc(inline)]\npub use delay_queue::DelayQueue;\n\n// ===== Internal utils =====\n\nenum Round {\n Up,\n Down,\n}\n\n/// Convert a `Duration` to milliseconds, rounding up and saturating at\n/// `u64::MAX`.\n///\n/// The saturating is fine because `u64::MAX` milliseconds are still many\n/// million years.\n#[inline]\nfn ms(duration: Duration, round: Round) -> u64 {\n const NANOS_PER_MILLI: u32 = 1_000_000;\n const MILLIS_PER_SEC: u64 = 1_000;\n\n // Round up.\n let millis = match round {\n Round::Up => (duration.subsec_nanos() + NANOS_PER_MILLI - 1) / NANOS_PER_MILLI,\n Round::Down => duration.subsec_millis(),\n };\n\n duration\n .as_secs()\n .saturating_mul(MILLIS_PER_SEC)\n .saturating_add(u64::from(millis))\n}\n" }, { "path": "tokio-util/src/time/wheel/level.rs", "content": "use crate::time::wheel::Stack;\n\nuse std::fmt;\n\n/// Wheel for a single level in the timer. This wheel contains 64 slots.\npub(crate) struct Level {\n level: usize,\n\n /// Bit field tracking which slots currently contain entries.\n ///\n /// Using a bit field to track slots that contain entries allows avoiding a\n /// scan to find entries. This field is updated when entries are added or\n /// removed from a slot.\n ///\n /// The least-significant bit represents slot zero.\n occupied: u64,\n\n /// Slots\n slot: [T; LEVEL_MULT],\n}\n\n/// Indicates when a slot must be processed next.\n#[derive(Debug)]\npub(crate) struct Expiration {\n /// The level containing the slot.\n pub(crate) level: usize,\n\n /// The slot index.\n pub(crate) slot: usize,\n\n /// The instant at which the slot needs to be processed.\n pub(crate) deadline: u64,\n}\n\n/// Level multiplier.\n///\n/// Being a power of 2 is very important.\nconst LEVEL_MULT: usize = 64;\n\nimpl Level {\n pub(crate) fn new(level: usize) -> Level {\n Level {\n level,\n occupied: 0,\n slot: std::array::from_fn(|_| T::default()),\n }\n }\n\n /// Finds the slot that needs to be processed next and returns the slot and\n /// `Instant` at which this slot must be processed.\n pub(crate) fn next_expiration(&self, now: u64) -> Option {\n // Use the `occupied` bit field to get the index of the next slot that\n // needs to be processed.\n let slot = self.next_occupied_slot(now)?;\n\n // From the slot index, calculate the `Instant` at which it needs to be\n // processed. This value *must* be in the future with respect to `now`.\n\n let level_range = level_range(self.level);\n let slot_range = slot_range(self.level);\n\n // TODO: This can probably be simplified w/ power of 2 math\n let level_start = now - (now % level_range);\n let mut deadline = level_start + slot as u64 * slot_range;\n if deadline < now {\n // A timer is in a slot \"prior\" to the current time. This can occur\n // because we do not have an infinite hierarchy of timer levels, and\n // eventually a timer scheduled for a very distant time might end up\n // being placed in a slot that is beyond the end of all of the\n // arrays.\n //\n // To deal with this, we first limit timers to being scheduled no\n // more than MAX_DURATION ticks in the future; that is, they're at\n // most one rotation of the top level away. Then, we force timers\n // that logically would go into the top+1 level, to instead go into\n // the top level's slots.\n //\n // What this means is that the top level's slots act as a\n // pseudo-ring buffer, and we rotate around them indefinitely. If we\n // compute a deadline before now, and it's the top level, it\n // therefore means we're actually looking at a slot in the future.\n debug_assert_eq!(self.level, super::NUM_LEVELS - 1);\n\n deadline += level_range;\n }\n debug_assert!(\n deadline >= now,\n \"deadline={:016X}; now={:016X}; level={}; slot={}; occupied={:b}\",\n deadline,\n now,\n self.level,\n slot,\n self.occupied\n );\n\n Some(Expiration {\n level: self.level,\n slot,\n deadline,\n })\n }\n\n fn next_occupied_slot(&self, now: u64) -> Option {\n if self.occupied == 0 {\n return None;\n }\n\n // Get the slot for now using Maths\n let now_slot = (now / slot_range(self.level)) as usize;\n let occupied = self.occupied.rotate_right(now_slot as u32);\n let zeros = occupied.trailing_zeros() as usize;\n let slot = (zeros + now_slot) % 64;\n\n Some(slot)\n }\n\n pub(crate) fn add_entry(&mut self, when: u64, item: T::Owned, store: &mut T::Store) {\n let slot = slot_for(when, self.level);\n\n self.slot[slot].push(item, store);\n self.occupied |= occupied_bit(slot);\n }\n\n pub(crate) fn remove_entry(&mut self, when: u64, item: &T::Borrowed, store: &mut T::Store) {\n let slot = slot_for(when, self.level);\n\n self.slot[slot].remove(item, store);\n\n if self.slot[slot].is_empty() {\n // The bit is currently set\n debug_assert!(self.occupied & occupied_bit(slot) != 0);\n\n // Unset the bit\n self.occupied ^= occupied_bit(slot);\n }\n }\n\n pub(crate) fn pop_entry_slot(&mut self, slot: usize, store: &mut T::Store) -> Option {\n let ret = self.slot[slot].pop(store);\n\n if ret.is_some() && self.slot[slot].is_empty() {\n // The bit is currently set\n debug_assert!(self.occupied & occupied_bit(slot) != 0);\n\n self.occupied ^= occupied_bit(slot);\n }\n\n ret\n }\n\n pub(crate) fn peek_entry_slot(&self, slot: usize) -> Option {\n self.slot[slot].peek()\n }\n}\n\nimpl fmt::Debug for Level {\n fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {\n fmt.debug_struct(\"Level\")\n .field(\"occupied\", &self.occupied)\n .finish()\n }\n}\n\nfn occupied_bit(slot: usize) -> u64 {\n 1 << slot\n}\n\nfn slot_range(level: usize) -> u64 {\n LEVEL_MULT.pow(level as u32) as u64\n}\n\nfn level_range(level: usize) -> u64 {\n LEVEL_MULT as u64 * slot_range(level)\n}\n\n/// Convert a duration (milliseconds) and a level to a slot position\nfn slot_for(duration: u64, level: usize) -> usize {\n ((duration >> (level * 6)) % LEVEL_MULT as u64) as usize\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_slot_for() {\n for pos in 0..64 {\n assert_eq!(pos as usize, slot_for(pos, 0));\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(pos, slot_for(a as u64, level));\n }\n }\n }\n}\n" }, { "path": "tokio-util/src/time/wheel/mod.rs", "content": "mod level;\npub(crate) use self::level::Expiration;\nuse self::level::Level;\n\nmod stack;\npub(crate) use self::stack::Stack;\n\nuse std::borrow::Borrow;\nuse std::fmt::Debug;\n\n/// Timing wheel implementation.\n///\n/// This type provides the hashed timing wheel implementation that backs\n/// [`DelayQueue`].\n///\n/// The structure is generic over `T: Stack`. This allows handling timeout data\n/// being stored on the heap or in a slab. In order to support the latter case,\n/// the slab must be passed into each function allowing the implementation to\n/// lookup timer entries.\n///\n/// See `Driver` documentation for some implementation notes.\n///\n/// [`DelayQueue`]: crate::time::DelayQueue\n#[derive(Debug)]\npub(crate) struct Wheel {\n /// The number of milliseconds elapsed since the wheel started.\n elapsed: u64,\n\n /// Timer wheel.\n ///\n /// Levels:\n ///\n /// * 1 ms slots / 64 ms range\n /// * 64 ms slots / ~ 4 sec range\n /// * ~ 4 sec slots / ~ 4 min range\n /// * ~ 4 min slots / ~ 4 hr range\n /// * ~ 4 hr slots / ~ 12 day range\n /// * ~ 12 day slots / ~ 2 yr range\n levels: Box<[Level]>,\n}\n\n/// Number of levels. Each level has 64 slots. By using 6 levels with 64 slots\n/// each, the timer is able to track time up to 2 years into the future with a\n/// precision of 1 millisecond.\nconst NUM_LEVELS: usize = 6;\n\n/// The maximum duration of a delay\nconst MAX_DURATION: u64 = (1 << (6 * NUM_LEVELS)) - 1;\n\n#[derive(Debug)]\npub(crate) enum InsertError {\n Elapsed,\n Invalid,\n}\n\nimpl Wheel\nwhere\n T: Stack,\n{\n /// Create a new timing wheel\n pub(crate) fn new() -> Wheel {\n let levels = (0..NUM_LEVELS).map(Level::new).collect();\n\n Wheel { elapsed: 0, levels }\n }\n\n /// Return the number of milliseconds that have elapsed since the timing\n /// wheel's creation.\n pub(crate) fn elapsed(&self) -> u64 {\n self.elapsed\n }\n\n /// Insert an entry into the timing wheel.\n ///\n /// # Arguments\n ///\n /// * `when`: is the instant at which the entry should be fired. It is\n /// represented as the number of milliseconds since the creation\n /// of the timing wheel.\n ///\n /// * `item`: The item to insert into the wheel.\n ///\n /// * `store`: The slab or `()` when using heap storage.\n ///\n /// # Return\n ///\n /// Returns `Ok` when the item is successfully inserted, `Err` otherwise.\n ///\n /// `Err(Elapsed)` indicates that `when` represents an instant that has\n /// already passed. In this case, the caller should fire the timeout\n /// immediately.\n ///\n /// `Err(Invalid)` indicates an invalid `when` argument as been supplied.\n pub(crate) fn insert(\n &mut self,\n when: u64,\n item: T::Owned,\n store: &mut T::Store,\n ) -> Result<(), (T::Owned, InsertError)> {\n if when <= self.elapsed {\n return Err((item, InsertError::Elapsed));\n } else if when - self.elapsed > MAX_DURATION {\n return Err((item, InsertError::Invalid));\n }\n\n // Get the level at which the entry should be stored\n let level = self.level_for(when);\n\n self.levels[level].add_entry(when, item, store);\n\n debug_assert!({\n self.levels[level]\n .next_expiration(self.elapsed)\n .map(|e| e.deadline >= self.elapsed)\n .unwrap_or(true)\n });\n\n Ok(())\n }\n\n /// Remove `item` from the timing wheel.\n #[track_caller]\n pub(crate) fn remove(&mut self, item: &T::Borrowed, store: &mut T::Store) {\n let when = T::when(item, store);\n\n assert!(\n self.elapsed <= when,\n \"elapsed={}; when={}\",\n self.elapsed,\n when\n );\n\n let level = self.level_for(when);\n\n self.levels[level].remove_entry(when, item, store);\n }\n\n /// Instant at which to poll\n pub(crate) fn poll_at(&self) -> Option {\n self.next_expiration().map(|expiration| expiration.deadline)\n }\n\n /// Next key that will expire\n pub(crate) fn peek(&self) -> Option {\n self.next_expiration()\n .and_then(|expiration| self.peek_entry(&expiration))\n }\n\n /// Advances the timer up to the instant represented by `now`.\n pub(crate) fn poll(&mut self, now: u64, store: &mut T::Store) -> Option {\n loop {\n let expiration = self.next_expiration().and_then(|expiration| {\n if expiration.deadline > now {\n None\n } else {\n Some(expiration)\n }\n });\n\n match expiration {\n Some(ref expiration) => {\n if let Some(item) = self.poll_expiration(expiration, store) {\n return Some(item);\n }\n\n self.set_elapsed(expiration.deadline);\n }\n None => {\n // in this case the poll did not indicate an expiration\n // _and_ we were not able to find a next expiration in\n // the current list of timers. advance to the poll's\n // current time and do nothing else.\n self.set_elapsed(now);\n return None;\n }\n }\n }\n }\n\n /// Returns the instant at which the next timeout expires.\n fn next_expiration(&self) -> Option {\n // Check all levels\n for level in 0..NUM_LEVELS {\n if let Some(expiration) = self.levels[level].next_expiration(self.elapsed) {\n // There cannot be any expirations at a higher level that happen\n // before this one.\n debug_assert!(self.no_expirations_before(level + 1, expiration.deadline));\n\n return Some(expiration);\n }\n }\n\n None\n }\n\n /// Used for debug assertions\n fn no_expirations_before(&self, start_level: usize, before: u64) -> bool {\n let mut res = true;\n\n for l2 in start_level..NUM_LEVELS {\n if let Some(e2) = self.levels[l2].next_expiration(self.elapsed) {\n if e2.deadline < before {\n res = false;\n }\n }\n }\n\n res\n }\n\n /// iteratively find entries that are between the wheel's current\n /// time and the expiration time. for each in that population either\n /// return it for notification (in the case of the last level) or tier\n /// it down to the next level (in all other cases).\n pub(crate) fn poll_expiration(\n &mut self,\n expiration: &Expiration,\n store: &mut T::Store,\n ) -> Option {\n while let Some(item) = self.pop_entry(expiration, store) {\n if expiration.level == 0 {\n debug_assert_eq!(T::when(item.borrow(), store), expiration.deadline);\n\n return Some(item);\n } else {\n let when = T::when(item.borrow(), store);\n\n let next_level = expiration.level - 1;\n\n self.levels[next_level].add_entry(when, item, store);\n }\n }\n\n None\n }\n\n fn set_elapsed(&mut self, when: u64) {\n assert!(\n self.elapsed <= when,\n \"elapsed={:?}; when={:?}\",\n self.elapsed,\n when\n );\n\n if when > self.elapsed {\n self.elapsed = when;\n }\n }\n\n fn pop_entry(&mut self, expiration: &Expiration, store: &mut T::Store) -> Option {\n self.levels[expiration.level].pop_entry_slot(expiration.slot, store)\n }\n\n fn peek_entry(&self, expiration: &Expiration) -> Option {\n self.levels[expiration.level].peek_entry_slot(expiration.slot)\n }\n\n fn level_for(&self, when: u64) -> usize {\n level_for(self.elapsed, when)\n }\n}\n\nfn level_for(elapsed: u64, when: u64) -> usize {\n const SLOT_MASK: u64 = (1 << 6) - 1;\n\n // Mask in the trailing bits ignored by the level calculation in order to cap\n // the possible leading zeros\n let mut masked = elapsed ^ when | SLOT_MASK;\n if masked >= MAX_DURATION {\n // Fudge the timer into the top level\n masked = MAX_DURATION - 1;\n }\n let leading_zeros = masked.leading_zeros() as usize;\n let significant = 63 - leading_zeros;\n significant / 6\n}\n\n#[cfg(all(test, not(loom)))]\nmod test {\n use super::*;\n\n #[test]\n fn test_level_for() {\n for pos in 0..64 {\n assert_eq!(0, level_for(0, pos), \"level_for({pos}) -- binary = {pos:b}\");\n }\n\n for level in 1..5 {\n for pos in level..64 {\n let a = pos * 64_usize.pow(level as u32);\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n\n if pos > level {\n let a = a - 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n\n if pos < 64 {\n let a = a + 1;\n assert_eq!(\n level,\n level_for(0, a as u64),\n \"level_for({a}) -- binary = {a:b}\"\n );\n }\n }\n }\n }\n}\n" }, { "path": "tokio-util/src/time/wheel/stack.rs", "content": "use std::borrow::Borrow;\nuse std::cmp::Eq;\nuse std::hash::Hash;\n\n/// Abstracts the stack operations needed to track timeouts.\npub(crate) trait Stack: Default {\n /// Type of the item stored in the stack\n type Owned: Borrow;\n\n /// Borrowed item\n type Borrowed: Eq + Hash;\n\n /// Item storage, this allows a slab to be used instead of just the heap\n type Store;\n\n /// Returns `true` if the stack is empty\n fn is_empty(&self) -> bool;\n\n /// Push an item onto the stack\n fn push(&mut self, item: Self::Owned, store: &mut Self::Store);\n\n /// Pop an item from the stack\n fn pop(&mut self, store: &mut Self::Store) -> Option;\n\n /// Peek into the stack.\n fn peek(&self) -> Option;\n\n fn remove(&mut self, item: &Self::Borrowed, store: &mut Self::Store);\n\n fn when(item: &Self::Borrowed, store: &Self::Store) -> u64;\n}\n" }, { "path": "tokio-util/src/tracing.rs", "content": "macro_rules! trace {\n ($($arg:tt)*) => {\n #[cfg(feature = \"tracing\")]\n tracing::trace!($($arg)*);\n };\n}\n" }, { "path": "tokio-util/src/udp/frame.rs", "content": "use crate::codec::{Decoder, Encoder};\n\nuse futures_core::Stream;\nuse tokio::{io::ReadBuf, net::UdpSocket};\n\nuse bytes::{BufMut, BytesMut};\nuse futures_sink::Sink;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\nuse std::{\n borrow::Borrow,\n net::{Ipv4Addr, SocketAddr, SocketAddrV4},\n};\n\n/// A unified [`Stream`] and [`Sink`] interface to an underlying `UdpSocket`, using\n/// the `Encoder` and `Decoder` traits to encode and decode frames.\n///\n/// Raw UDP sockets work with datagrams, but higher-level code usually wants to\n/// batch these into meaningful chunks, called \"frames\". This method layers\n/// framing on top of this socket by using the `Encoder` and `Decoder` traits to\n/// handle encoding and decoding of messages frames. Note that the incoming and\n/// outgoing frame types may be distinct.\n///\n/// This function returns a *single* object that is both [`Stream`] and [`Sink`];\n/// grouping this into a single object is often useful for layering things which\n/// require both read and write access to the underlying object.\n///\n/// If you want to work more directly with the streams and sink, consider\n/// calling [`split`] on the `UdpFramed` returned by this method, which will break\n/// them into separate objects, allowing them to interact more easily.\n///\n/// [`Stream`]: futures_core::Stream\n/// [`Sink`]: futures_sink::Sink\n/// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split\n#[must_use = \"sinks do nothing unless polled\"]\n#[derive(Debug)]\npub struct UdpFramed {\n socket: T,\n codec: C,\n rd: BytesMut,\n wr: BytesMut,\n out_addr: SocketAddr,\n flushed: bool,\n is_readable: bool,\n current_addr: Option,\n}\n\nconst INITIAL_RD_CAPACITY: usize = 64 * 1024;\nconst INITIAL_WR_CAPACITY: usize = 8 * 1024;\n\nimpl Unpin for UdpFramed {}\n\nimpl Stream for UdpFramed\nwhere\n T: Borrow,\n C: Decoder,\n{\n type Item = Result<(C::Item, SocketAddr), C::Error>;\n\n fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n let pin = self.get_mut();\n\n pin.rd.reserve(INITIAL_RD_CAPACITY);\n\n loop {\n // Are there still bytes left in the read buffer to decode?\n if pin.is_readable {\n if let Some(frame) = pin.codec.decode_eof(&mut pin.rd)? {\n let current_addr = pin\n .current_addr\n .expect(\"will always be set before this line is called\");\n\n return Poll::Ready(Some(Ok((frame, current_addr))));\n }\n\n // if this line has been reached then decode has returned `None`.\n pin.is_readable = false;\n pin.rd.clear();\n }\n\n // We're out of data. Try and fetch more data to decode\n let addr = {\n // Safety: `chunk_mut()` returns a `&mut UninitSlice`, and `UninitSlice` is a\n // transparent wrapper around `[MaybeUninit]`.\n let buf = unsafe { pin.rd.chunk_mut().as_uninit_slice_mut() };\n let mut read = ReadBuf::uninit(buf);\n let ptr = read.filled().as_ptr();\n let res = ready!(pin.socket.borrow().poll_recv_from(cx, &mut read));\n\n assert_eq!(ptr, read.filled().as_ptr());\n let addr = res?;\n\n let filled = read.filled().len();\n // Safety: This is guaranteed to be the number of initialized (and read) bytes due\n // to the invariants provided by `ReadBuf::filled`.\n unsafe { pin.rd.advance_mut(filled) };\n\n addr\n };\n\n pin.current_addr = Some(addr);\n pin.is_readable = true;\n }\n }\n}\n\nimpl Sink<(I, SocketAddr)> for UdpFramed\nwhere\n T: Borrow,\n C: Encoder,\n{\n type Error = C::Error;\n\n fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if !self.flushed {\n match self.poll_flush(cx)? {\n Poll::Ready(()) => {}\n Poll::Pending => return Poll::Pending,\n }\n }\n\n Poll::Ready(Ok(()))\n }\n\n fn start_send(self: Pin<&mut Self>, item: (I, SocketAddr)) -> Result<(), Self::Error> {\n let (frame, out_addr) = item;\n\n let pin = self.get_mut();\n\n pin.codec.encode(frame, &mut pin.wr)?;\n pin.out_addr = out_addr;\n pin.flushed = false;\n\n Ok(())\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n if self.flushed {\n return Poll::Ready(Ok(()));\n }\n\n let Self {\n ref socket,\n ref mut out_addr,\n ref mut wr,\n ..\n } = *self;\n\n let n = ready!(socket.borrow().poll_send_to(cx, wr, *out_addr))?;\n\n let wrote_all = n == self.wr.len();\n self.wr.clear();\n self.flushed = true;\n\n let res = if wrote_all {\n Ok(())\n } else {\n Err(io::Error::new(\n io::ErrorKind::Other,\n \"failed to write entire datagram to socket\",\n )\n .into())\n };\n\n Poll::Ready(res)\n }\n\n fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> {\n ready!(self.poll_flush(cx))?;\n Poll::Ready(Ok(()))\n }\n}\n\nimpl UdpFramed\nwhere\n T: Borrow,\n{\n /// Create a new `UdpFramed` backed by the given socket and codec.\n ///\n /// See struct level documentation for more details.\n pub fn new(socket: T, codec: C) -> UdpFramed {\n Self {\n socket,\n codec,\n out_addr: SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(0, 0, 0, 0), 0)),\n rd: BytesMut::with_capacity(INITIAL_RD_CAPACITY),\n wr: BytesMut::with_capacity(INITIAL_WR_CAPACITY),\n flushed: true,\n is_readable: false,\n current_addr: None,\n }\n }\n\n /// Returns a reference to the underlying I/O stream wrapped by `Framed`.\n ///\n /// # Note\n ///\n /// Care should be taken to not tamper with the underlying stream of data\n /// coming in as it may corrupt the stream of frames otherwise being worked\n /// with.\n pub fn get_ref(&self) -> &T {\n &self.socket\n }\n\n /// Returns a mutable reference to the underlying I/O stream wrapped by `Framed`.\n ///\n /// # Note\n ///\n /// Care should be taken to not tamper with the underlying stream of data\n /// coming in as it may corrupt the stream of frames otherwise being worked\n /// with.\n pub fn get_mut(&mut self) -> &mut T {\n &mut self.socket\n }\n\n /// Returns a reference to the underlying codec wrapped by\n /// `Framed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn codec(&self) -> &C {\n &self.codec\n }\n\n /// Returns a mutable reference to the underlying codec wrapped by\n /// `UdpFramed`.\n ///\n /// Note that care should be taken to not tamper with the underlying codec\n /// as it may corrupt the stream of frames otherwise being worked with.\n pub fn codec_mut(&mut self) -> &mut C {\n &mut self.codec\n }\n\n /// Returns a reference to the read buffer.\n pub fn read_buffer(&self) -> &BytesMut {\n &self.rd\n }\n\n /// Returns a mutable reference to the read buffer.\n pub fn read_buffer_mut(&mut self) -> &mut BytesMut {\n &mut self.rd\n }\n\n /// Consumes the `Framed`, returning its underlying I/O stream.\n pub fn into_inner(self) -> T {\n self.socket\n }\n}\n" }, { "path": "tokio-util/src/udp/mod.rs", "content": "#![cfg(not(loom))]\n\n//! UDP framing\n\nmod frame;\npub use frame::UdpFramed;\n" }, { "path": "tokio-util/src/util/maybe_dangling.rs", "content": "use core::future::Future;\nuse core::mem::MaybeUninit;\nuse core::pin::Pin;\nuse core::task::{Context, Poll};\n\n/// A wrapper type that tells the compiler that the contents might not be valid.\n///\n/// This is necessary mainly when `T` contains a reference. In that case, the\n/// compiler will sometimes assume that the reference is always valid; in some\n/// cases it will assume this even after the destructor of `T` runs. For\n/// example, when a reference is used as a function argument, then the compiler\n/// will assume that the reference is valid until the function returns, even if\n/// the reference is destroyed during the function. When the reference is used\n/// as part of a self-referential struct, that assumption can be false. Wrapping\n/// the reference in this type prevents the compiler from making that\n/// assumption.\n///\n/// # Invariants\n///\n/// The `MaybeUninit` will always contain a valid value until the destructor runs.\n//\n// Reference\n// See \n//\n// TODO: replace this with an official solution once RFC #3336 or similar is available.\n// \n#[repr(transparent)]\npub(crate) struct MaybeDangling(MaybeUninit);\n\nimpl Drop for MaybeDangling {\n fn drop(&mut self) {\n // Safety: `0` is always initialized.\n unsafe { core::ptr::drop_in_place(self.0.as_mut_ptr()) };\n }\n}\n\nimpl MaybeDangling {\n pub(crate) fn new(inner: T) -> Self {\n Self(MaybeUninit::new(inner))\n }\n}\n\nimpl Future for MaybeDangling {\n type Output = F::Output;\n\n fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll {\n // Safety: `0` is always initialized.\n let fut = unsafe { self.map_unchecked_mut(|this| this.0.assume_init_mut()) };\n fut.poll(cx)\n }\n}\n\n#[test]\nfn maybedangling_runs_drop() {\n struct SetOnDrop<'a>(&'a mut bool);\n\n impl Drop for SetOnDrop<'_> {\n fn drop(&mut self) {\n *self.0 = true;\n }\n }\n\n let mut success = false;\n\n drop(MaybeDangling::new(SetOnDrop(&mut success)));\n assert!(success);\n}\n" }, { "path": "tokio-util/src/util/mod.rs", "content": "mod maybe_dangling;\n#[cfg(any(feature = \"io\", feature = \"codec\"))]\nmod poll_buf;\n\npub(crate) use maybe_dangling::MaybeDangling;\n#[cfg(any(feature = \"io\", feature = \"codec\"))]\n#[cfg_attr(not(feature = \"io\"), allow(unreachable_pub))]\npub use poll_buf::{poll_read_buf, poll_write_buf};\n\ncfg_rt! {\n #[cfg_attr(not(feature = \"io\"), allow(unused))]\n pub(crate) use tokio::task::coop::poll_proceed;\n}\n\ncfg_not_rt! {\n #[cfg_attr(not(feature = \"io\"), allow(unused))]\n use std::task::{Context, Poll};\n\n #[cfg_attr(not(feature = \"io\"), allow(unused))]\n pub(crate) struct RestoreOnPending;\n\n #[cfg_attr(not(feature = \"io\"), allow(unused))]\n impl RestoreOnPending {\n pub(crate) fn made_progress(&self) {}\n }\n\n #[cfg_attr(not(feature = \"io\"), allow(unused))]\n pub(crate) fn poll_proceed(_cx: &mut Context<'_>) -> Poll {\n Poll::Ready(RestoreOnPending)\n }\n}\n" }, { "path": "tokio-util/src/util/poll_buf.rs", "content": "use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\n\nuse bytes::{Buf, BufMut};\nuse std::io::{self, IoSlice};\nuse std::pin::Pin;\nuse std::task::{ready, Context, Poll};\n\n/// Try to read data from an `AsyncRead` into an implementer of the [`BufMut`] trait.\n///\n/// [`BufMut`]: bytes::Buf\n///\n/// # Example\n///\n/// ```\n/// use bytes::{Bytes, BytesMut};\n/// use tokio_stream as stream;\n/// use tokio::io::Result;\n/// use tokio_util::io::{StreamReader, poll_read_buf};\n/// use std::future::poll_fn;\n/// use std::pin::Pin;\n/// # #[tokio::main(flavor = \"current_thread\")]\n/// # async fn main() -> std::io::Result<()> {\n///\n/// // Create a reader from an iterator. This particular reader will always be\n/// // ready.\n/// let mut read = StreamReader::new(stream::iter(vec![Result::Ok(Bytes::from_static(&[0, 1, 2, 3]))]));\n///\n/// let mut buf = BytesMut::new();\n/// let mut reads = 0;\n///\n/// loop {\n/// reads += 1;\n/// let n = poll_fn(|cx| poll_read_buf(Pin::new(&mut read), cx, &mut buf)).await?;\n///\n/// if n == 0 {\n/// break;\n/// }\n/// }\n///\n/// // one or more reads might be necessary.\n/// assert!(reads >= 1);\n/// assert_eq!(&buf[..], &[0, 1, 2, 3]);\n/// # Ok(())\n/// # }\n/// ```\n#[cfg_attr(not(feature = \"io\"), allow(unreachable_pub))]\npub fn poll_read_buf(\n io: Pin<&mut T>,\n cx: &mut Context<'_>,\n buf: &mut B,\n) -> Poll> {\n if !buf.has_remaining_mut() {\n return Poll::Ready(Ok(0));\n }\n\n let n = {\n let dst = buf.chunk_mut();\n\n // Safety: `chunk_mut()` returns a `&mut UninitSlice`, and `UninitSlice` is a\n // transparent wrapper around `[MaybeUninit]`.\n let dst = unsafe { dst.as_uninit_slice_mut() };\n let mut buf = ReadBuf::uninit(dst);\n let ptr = buf.filled().as_ptr();\n ready!(io.poll_read(cx, &mut buf)?);\n\n // Ensure the pointer does not change from under us\n assert_eq!(ptr, buf.filled().as_ptr());\n buf.filled().len()\n };\n\n // Safety: This is guaranteed to be the number of initialized (and read)\n // bytes due to the invariants provided by `ReadBuf::filled`.\n unsafe {\n buf.advance_mut(n);\n }\n\n Poll::Ready(Ok(n))\n}\n\n/// Try to write data from an implementer of the [`Buf`] trait to an\n/// [`AsyncWrite`], advancing the buffer's internal cursor.\n///\n/// This function will use [vectored writes] when the [`AsyncWrite`] supports\n/// vectored writes.\n///\n/// # Examples\n///\n/// [`File`] implements [`AsyncWrite`] and [`Cursor<&[u8]>`] implements\n/// [`Buf`]:\n///\n/// ```no_run\n/// use tokio_util::io::poll_write_buf;\n/// use tokio::io;\n/// use tokio::fs::File;\n///\n/// use bytes::Buf;\n/// use std::future::poll_fn;\n/// use std::io::Cursor;\n/// use std::pin::Pin;\n///\n/// #[tokio::main]\n/// async fn main() -> io::Result<()> {\n/// let mut file = File::create(\"foo.txt\").await?;\n/// let mut buf = Cursor::new(b\"data to write\");\n///\n/// // Loop until the entire contents of the buffer are written to\n/// // the file.\n/// while buf.has_remaining() {\n/// poll_fn(|cx| poll_write_buf(Pin::new(&mut file), cx, &mut buf)).await?;\n/// }\n///\n/// Ok(())\n/// }\n/// ```\n///\n/// [`Buf`]: bytes::Buf\n/// [`AsyncWrite`]: tokio::io::AsyncWrite\n/// [`File`]: tokio::fs::File\n/// [vectored writes]: tokio::io::AsyncWrite::poll_write_vectored\n#[cfg_attr(not(feature = \"io\"), allow(unreachable_pub))]\npub fn poll_write_buf(\n io: Pin<&mut T>,\n cx: &mut Context<'_>,\n buf: &mut B,\n) -> Poll> {\n const MAX_BUFS: usize = 64;\n\n if !buf.has_remaining() {\n return Poll::Ready(Ok(0));\n }\n\n let n = if io.is_write_vectored() {\n let mut slices = [IoSlice::new(&[]); MAX_BUFS];\n let cnt = buf.chunks_vectored(&mut slices);\n ready!(io.poll_write_vectored(cx, &slices[..cnt]))?\n } else {\n ready!(io.poll_write(cx, buf.chunk()))?\n };\n\n buf.advance(n);\n\n Poll::Ready(Ok(n))\n}\n" }, { "path": "tokio-util/tests/_require_full.rs", "content": "#![cfg(not(feature = \"full\"))]\ncompile_error!(\"run tokio-util tests with `--features full`\");\n" }, { "path": "tokio-util/tests/abort_on_drop.rs", "content": "use tokio::{sync::oneshot, task::yield_now};\nuse tokio_util::task::{AbortOnDrop, AbortOnDropHandle};\n\n#[tokio::test]\nasync fn aborts_task_on_drop() {\n let (mut tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDropHandle::new(handle);\n drop(handle);\n tx.closed().await;\n assert!(tx.is_closed());\n}\n\n#[tokio::test]\nasync fn aborts_task_directly() {\n let (mut tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDropHandle::new(handle);\n handle.abort();\n tx.closed().await;\n assert!(tx.is_closed());\n assert!(handle.is_finished());\n}\n\n#[tokio::test]\nasync fn does_not_abort_after_detach() {\n let (tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDropHandle::new(handle);\n handle.detach(); // returns and drops the original join handle\n yield_now().await;\n assert!(!tx.is_closed()); // the task is still alive\n}\n\n#[tokio::test]\nasync fn handle_aborts_task_on_drop() {\n let (mut tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDrop::new(handle.abort_handle());\n drop(handle);\n tx.closed().await;\n assert!(tx.is_closed());\n}\n\n#[tokio::test]\nasync fn handle_aborts_task_directly() {\n let (mut tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDrop::new(handle.abort_handle());\n handle.abort();\n tx.closed().await;\n assert!(tx.is_closed());\n assert!(handle.is_finished());\n}\n\n#[tokio::test]\nasync fn handle_does_not_abort_after_detach() {\n let (tx, rx) = oneshot::channel::();\n let handle = tokio::spawn(async move {\n let _ = rx.await;\n });\n let handle = AbortOnDrop::new(handle.abort_handle());\n handle.detach(); // returns and drops the original abort handle\n yield_now().await;\n assert!(!tx.is_closed()); // the task is still alive\n}\n" }, { "path": "tokio-util/tests/codecs.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio_util::codec::{AnyDelimiterCodec, BytesCodec, Decoder, Encoder, LinesCodec};\n\nuse bytes::{BufMut, Bytes, BytesMut};\n\n#[test]\nfn bytes_decoder() {\n let mut codec = BytesCodec::new();\n let buf = &mut BytesMut::new();\n buf.put_slice(b\"abc\");\n assert_eq!(\"abc\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"a\");\n assert_eq!(\"a\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn bytes_encoder() {\n let mut codec = BytesCodec::new();\n\n // Default capacity of BytesMut\n #[cfg(target_pointer_width = \"64\")]\n const INLINE_CAP: usize = 4 * 8 - 1;\n #[cfg(target_pointer_width = \"32\")]\n const INLINE_CAP: usize = 4 * 4 - 1;\n\n let mut buf = BytesMut::new();\n codec\n .encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf)\n .unwrap();\n\n // Default capacity of Framed Read\n const INITIAL_CAPACITY: usize = 8 * 1024;\n\n let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);\n codec\n .encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf)\n .unwrap();\n codec\n .encode(BytesMut::from(&b\"hello\"[..]), &mut buf)\n .unwrap();\n}\n\n#[test]\nfn lines_decoder() {\n let mut codec = LinesCodec::new();\n let buf = &mut BytesMut::new();\n buf.reserve(200);\n buf.put_slice(b\"line 1\\nline 2\\r\\nline 3\\n\\r\\n\\r\");\n assert_eq!(\"line 1\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"line 2\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"line 3\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n buf.put_slice(b\"k\");\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(\"\\rk\", codec.decode_eof(buf).unwrap().unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n}\n\n#[test]\nfn lines_decoder_invalid_utf8() {\n let mut codec = LinesCodec::new();\n let buf = &mut BytesMut::new();\n buf.reserve(200);\n buf.put_slice(b\"line 1\\xc3\\x28\");\n assert_eq!(None, codec.decode(buf).unwrap());\n assert!(codec.decode_eof(buf).is_err());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n buf.put_slice(b\"line 22222222222222\\n\");\n assert_eq!(\"line 22222222222222\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn lines_decoder_max_length() {\n const MAX_LENGTH: usize = 6;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"line 1 is too long\\nline 2\\nline 3\\r\\nline 4\\n\\r\\n\\r\");\n\n assert!(codec.decode(buf).is_err());\n\n let line = codec.decode(buf).unwrap().unwrap();\n assert!(line.len() <= MAX_LENGTH, \"{line:?}.len() <= {MAX_LENGTH:?}\");\n assert_eq!(\"line 2\", line);\n\n assert!(codec.decode(buf).is_err());\n\n let line = codec.decode(buf).unwrap().unwrap();\n assert!(line.len() <= MAX_LENGTH, \"{line:?}.len() <= {MAX_LENGTH:?}\");\n assert_eq!(\"line 4\", line);\n\n let line = codec.decode(buf).unwrap().unwrap();\n assert!(line.len() <= MAX_LENGTH, \"{line:?}.len() <= {MAX_LENGTH:?}\");\n assert_eq!(\"\", line);\n\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n buf.put_slice(b\"k\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n let line = codec.decode_eof(buf).unwrap().unwrap();\n assert!(line.len() <= MAX_LENGTH, \"{line:?}.len() <= {MAX_LENGTH:?}\");\n assert_eq!(\"\\rk\", line);\n\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n\n // Line that's one character too long. This could cause an out of bounds\n // error if we peek at the next characters using slice indexing.\n buf.put_slice(b\"aaabbbc\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn lines_decoder_max_length_underrun() {\n const MAX_LENGTH: usize = 6;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"line \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too l\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"ong\\n\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n buf.put_slice(b\"line 2\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"\\n\");\n assert_eq!(\"line 2\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn lines_decoder_max_length_bursts() {\n const MAX_LENGTH: usize = 10;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"line \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too l\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"ong\\n\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn lines_decoder_max_length_big_burst() {\n const MAX_LENGTH: usize = 10;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"line \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too long!\\n\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn lines_decoder_max_length_newline_between_decodes() {\n const MAX_LENGTH: usize = 5;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"hello\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n buf.put_slice(b\"\\nworld\");\n assert_eq!(\"hello\", codec.decode(buf).unwrap().unwrap());\n}\n\n// Regression test for [infinite loop bug](https://github.com/tokio-rs/tokio/issues/1483)\n#[test]\nfn lines_decoder_discard_repeat() {\n const MAX_LENGTH: usize = 1;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"aa\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"a\");\n assert_eq!(None, codec.decode(buf).unwrap());\n}\n\n// Regression test for [subsequent calls to LinesCodec decode does not return the desired results bug](https://github.com/tokio-rs/tokio/issues/3555)\n#[test]\nfn lines_decoder_max_length_underrun_twice() {\n const MAX_LENGTH: usize = 11;\n\n let mut codec = LinesCodec::new_with_max_length(MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"line \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too very l\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"aaaaaaaaaaaaaaaaaaaaaaa\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"ong\\nshort\\n\");\n assert_eq!(\"short\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn lines_encoder() {\n let mut codec = LinesCodec::new();\n let mut buf = BytesMut::new();\n\n codec.encode(\"line 1\", &mut buf).unwrap();\n assert_eq!(\"line 1\\n\", buf);\n\n codec.encode(\"line 2\", &mut buf).unwrap();\n assert_eq!(\"line 1\\nline 2\\n\", buf);\n}\n\n#[test]\nfn any_delimiters_decoder_any_character() {\n let mut codec = AnyDelimiterCodec::new(b\",;\\n\\r\".to_vec(), b\",\".to_vec());\n let buf = &mut BytesMut::new();\n buf.reserve(200);\n buf.put_slice(b\"chunk 1,chunk 2;chunk 3\\n\\r\");\n assert_eq!(\"chunk 1\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"chunk 2\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"chunk 3\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(\"\", codec.decode(buf).unwrap().unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n buf.put_slice(b\"k\");\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(\"k\", codec.decode_eof(buf).unwrap().unwrap());\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n}\n\n#[test]\nfn any_delimiters_decoder_max_length() {\n const MAX_LENGTH: usize = 7;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"chunk 1 is too long\\nchunk 2\\nchunk 3\\r\\nchunk 4\\n\\r\\n\");\n\n assert!(codec.decode(buf).is_err());\n\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"chunk 2\", chunk);\n\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"chunk 3\", chunk);\n\n // \\r\\n cause empty chunk\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"\", chunk);\n\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"chunk 4\", chunk);\n\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"\", chunk);\n\n let chunk = codec.decode(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"\", chunk);\n\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n buf.put_slice(b\"k\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n let chunk = codec.decode_eof(buf).unwrap().unwrap();\n assert!(\n chunk.len() <= MAX_LENGTH,\n \"{chunk:?}.len() <= {MAX_LENGTH:?}\"\n );\n assert_eq!(\"k\", chunk);\n\n assert_eq!(None, codec.decode(buf).unwrap());\n assert_eq!(None, codec.decode_eof(buf).unwrap());\n\n // Delimiter that's one character too long. This could cause an out of bounds\n // error if we peek at the next characters using slice indexing.\n buf.put_slice(b\"aaabbbcc\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn any_delimiter_decoder_max_length_underrun() {\n const MAX_LENGTH: usize = 7;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"chunk \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too l\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"ong\\n\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n buf.put_slice(b\"chunk 2\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\",\");\n assert_eq!(\"chunk 2\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn any_delimiter_decoder_max_length_underrun_twice() {\n const MAX_LENGTH: usize = 11;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"chunk \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too very l\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"aaaaaaaaaaaaaaaaaaaaaaa\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"ong\\nshort\\n\");\n assert_eq!(\"short\", codec.decode(buf).unwrap().unwrap());\n}\n#[test]\nfn any_delimiter_decoder_max_length_bursts() {\n const MAX_LENGTH: usize = 11;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"chunk \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too l\");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"ong\\n\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn any_delimiter_decoder_max_length_big_burst() {\n const MAX_LENGTH: usize = 11;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"chunk \");\n assert_eq!(None, codec.decode(buf).unwrap());\n buf.put_slice(b\"too long!\\n\");\n assert!(codec.decode(buf).is_err());\n}\n\n#[test]\nfn any_delimiter_decoder_max_length_delimiter_between_decodes() {\n const MAX_LENGTH: usize = 5;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"hello\");\n assert_eq!(None, codec.decode(buf).unwrap());\n\n buf.put_slice(b\",world\");\n assert_eq!(\"hello\", codec.decode(buf).unwrap().unwrap());\n}\n\n#[test]\nfn any_delimiter_decoder_discard_repeat() {\n const MAX_LENGTH: usize = 1;\n\n let mut codec =\n AnyDelimiterCodec::new_with_max_length(b\",;\\n\\r\".to_vec(), b\",\".to_vec(), MAX_LENGTH);\n let buf = &mut BytesMut::new();\n\n buf.reserve(200);\n buf.put_slice(b\"aa\");\n assert!(codec.decode(buf).is_err());\n buf.put_slice(b\"a\");\n assert_eq!(None, codec.decode(buf).unwrap());\n}\n\n#[test]\nfn any_delimiter_encoder() {\n let mut codec = AnyDelimiterCodec::new(b\",\".to_vec(), b\";--;\".to_vec());\n let mut buf = BytesMut::new();\n\n codec.encode(\"chunk 1\", &mut buf).unwrap();\n assert_eq!(\"chunk 1;--;\", buf);\n\n codec.encode(\"chunk 2\", &mut buf).unwrap();\n assert_eq!(\"chunk 1;--;chunk 2;--;\", buf);\n}\n" }, { "path": "tokio-util/tests/compat.rs", "content": "#![cfg(feature = \"compat\")]\n#![cfg(not(target_os = \"wasi\"))] // WASI does not support all fs operations\n#![warn(rust_2018_idioms)]\n\nuse futures_io::SeekFrom;\nuse futures_util::{AsyncReadExt, AsyncSeekExt, AsyncWriteExt};\nuse tempfile::NamedTempFile;\nuse tokio::fs::OpenOptions;\nuse tokio_util::compat::TokioAsyncWriteCompatExt;\n\n#[tokio::test]\nasync fn compat_file_seek() -> futures_util::io::Result<()> {\n let temp_file = NamedTempFile::new()?;\n let mut file = OpenOptions::new()\n .read(true)\n .write(true)\n .create(true)\n .truncate(true)\n .open(temp_file)\n .await?\n .compat_write();\n\n file.write_all(&[0, 1, 2, 3, 4, 5]).await?;\n file.write_all(&[6, 7]).await?;\n\n assert_eq!(file.stream_position().await?, 8);\n\n // Modify elements at position 2.\n assert_eq!(file.seek(SeekFrom::Start(2)).await?, 2);\n file.write_all(&[8, 9]).await?;\n\n file.flush().await?;\n\n // Verify we still have 8 elements.\n assert_eq!(file.seek(SeekFrom::End(0)).await?, 8);\n // Seek back to the start of the file to read and verify contents.\n file.seek(SeekFrom::Start(0)).await?;\n\n let mut buf = Vec::new();\n let num_bytes = file.read_to_end(&mut buf).await?;\n assert_eq!(&buf[..num_bytes], &[0, 1, 8, 9, 4, 5, 6, 7]);\n\n Ok(())\n}\n" }, { "path": "tokio-util/tests/context.rs", "content": "#![cfg(feature = \"rt\")]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n#![warn(rust_2018_idioms)]\n\nuse tokio::runtime::Builder;\nuse tokio::time::*;\nuse tokio_util::context::RuntimeExt;\n\n#[test]\nfn tokio_context_with_another_runtime() {\n let rt1 = Builder::new_multi_thread()\n .worker_threads(1)\n // no timer!\n .build()\n .unwrap();\n let rt2 = Builder::new_multi_thread()\n .worker_threads(1)\n .enable_all()\n .build()\n .unwrap();\n\n // Without the `HandleExt.wrap()` there would be a panic because there is\n // no timer running, since it would be referencing runtime r1.\n rt1.block_on(rt2.wrap(async move { sleep(Duration::from_millis(2)).await }));\n}\n" }, { "path": "tokio-util/tests/framed.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio_stream::StreamExt;\nuse tokio_test::assert_ok;\nuse tokio_util::codec::{Decoder, Encoder, Framed, FramedParts};\n\nuse bytes::{Buf, BufMut, BytesMut};\nuse std::io::{self, Read};\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nconst INITIAL_CAPACITY: usize = 8 * 1024;\n\n/// Encode and decode u32 values.\n#[derive(Default)]\nstruct U32Codec {\n read_bytes: usize,\n}\n\nimpl Decoder for U32Codec {\n type Item = u32;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n if buf.len() < 4 {\n return Ok(None);\n }\n\n let n = buf.split_to(4).get_u32();\n self.read_bytes += 4;\n Ok(Some(n))\n }\n}\n\nimpl Encoder for U32Codec {\n type Error = io::Error;\n\n fn encode(&mut self, item: u32, dst: &mut BytesMut) -> io::Result<()> {\n // Reserve space\n dst.reserve(4);\n dst.put_u32(item);\n Ok(())\n }\n}\n\n/// Encode and decode u64 values.\n#[derive(Default)]\nstruct U64Codec {\n read_bytes: usize,\n}\n\nimpl Decoder for U64Codec {\n type Item = u64;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n if buf.len() < 8 {\n return Ok(None);\n }\n\n let n = buf.split_to(8).get_u64();\n self.read_bytes += 8;\n Ok(Some(n))\n }\n}\n\nimpl Encoder for U64Codec {\n type Error = io::Error;\n\n fn encode(&mut self, item: u64, dst: &mut BytesMut) -> io::Result<()> {\n // Reserve space\n dst.reserve(8);\n dst.put_u64(item);\n Ok(())\n }\n}\n\n/// This value should never be used\nstruct DontReadIntoThis;\n\nimpl Read for DontReadIntoThis {\n fn read(&mut self, _: &mut [u8]) -> io::Result {\n Err(io::Error::new(\n io::ErrorKind::Other,\n \"Read into something you weren't supposed to.\",\n ))\n }\n}\n\nimpl tokio::io::AsyncRead for DontReadIntoThis {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &mut tokio::io::ReadBuf<'_>,\n ) -> Poll> {\n unreachable!()\n }\n}\n\n#[tokio::test]\nasync fn can_read_from_existing_buf() {\n let mut parts = FramedParts::new(DontReadIntoThis, U32Codec::default());\n parts.read_buf = BytesMut::from(&[0, 0, 0, 42][..]);\n\n let mut framed = Framed::from_parts(parts);\n let num = assert_ok!(framed.next().await.unwrap());\n\n assert_eq!(num, 42);\n assert_eq!(framed.codec().read_bytes, 4);\n}\n\n#[tokio::test]\nasync fn can_read_from_existing_buf_after_codec_changed() {\n let mut parts = FramedParts::new(DontReadIntoThis, U32Codec::default());\n parts.read_buf = BytesMut::from(&[0, 0, 0, 42, 0, 0, 0, 0, 0, 0, 0, 84][..]);\n\n let mut framed = Framed::from_parts(parts);\n let num = assert_ok!(framed.next().await.unwrap());\n\n assert_eq!(num, 42);\n assert_eq!(framed.codec().read_bytes, 4);\n\n let mut framed = framed.map_codec(|codec| U64Codec {\n read_bytes: codec.read_bytes,\n });\n let num = assert_ok!(framed.next().await.unwrap());\n\n assert_eq!(num, 84);\n assert_eq!(framed.codec().read_bytes, 12);\n}\n\n#[test]\nfn external_buf_grows_to_init() {\n let mut parts = FramedParts::new(DontReadIntoThis, U32Codec::default());\n parts.read_buf = BytesMut::from(&[0, 0, 0, 42][..]);\n\n let framed = Framed::from_parts(parts);\n let FramedParts { read_buf, .. } = framed.into_parts();\n\n assert_eq!(read_buf.capacity(), INITIAL_CAPACITY);\n}\n\n// Regression test: `Framed::from_parts` with an empty read buffer (but with\n// capacity, as produced by `into_parts()` after consuming all data) should NOT\n// call `decode()` before actually reading from the underlying IO.\n//\n// Before the fix, `is_readable` was derived from `buffer.capacity() > 0`, which\n// was always true after `reserve()`. This caused a spurious `decode()` call on\n// the empty buffer before any IO read.\n#[tokio::test]\nasync fn from_parts_empty_read_buf_does_not_spuriously_decode() {\n struct TrackingDecoder {\n decode_count: usize,\n }\n\n impl Decoder for TrackingDecoder {\n type Item = u32;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n self.decode_count += 1;\n if buf.len() < 4 {\n return Ok(None);\n }\n let n = buf.split_to(4).get_u32();\n Ok(Some(n))\n }\n }\n\n impl Encoder for TrackingDecoder {\n type Error = io::Error;\n fn encode(&mut self, item: u32, dst: &mut BytesMut) -> io::Result<()> {\n dst.reserve(4);\n dst.put_u32(item);\n Ok(())\n }\n }\n\n // Underlying IO provides exactly one 4-byte frame.\n let data: &[u8] = &[0, 0, 0, 42];\n let mut parts = FramedParts::new(data, TrackingDecoder { decode_count: 0 });\n // Simulate a buffer recycled from a previous Framed via `into_parts()`:\n // empty but has capacity.\n parts.read_buf = BytesMut::with_capacity(INITIAL_CAPACITY);\n\n let mut framed = Framed::from_parts(parts);\n let num = assert_ok!(framed.next().await.unwrap());\n assert_eq!(num, 42);\n\n // With the fix: decode is called once (after reading data from IO).\n // Before the fix: decode was called twice (once on the empty buffer, then\n // once after reading), because `is_readable` was incorrectly `true`.\n assert_eq!(framed.codec().decode_count, 1);\n}\n\n#[test]\nfn external_buf_does_not_shrink() {\n let mut parts = FramedParts::new(DontReadIntoThis, U32Codec::default());\n parts.read_buf = BytesMut::from(&vec![0; INITIAL_CAPACITY * 2][..]);\n\n let framed = Framed::from_parts(parts);\n let FramedParts { read_buf, .. } = framed.into_parts();\n\n assert_eq!(read_buf.capacity(), INITIAL_CAPACITY * 2);\n}\n" }, { "path": "tokio-util/tests/framed_read.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::io::{AsyncRead, ReadBuf};\nuse tokio_test::assert_ready;\nuse tokio_test::task;\nuse tokio_util::codec::{Decoder, FramedRead};\n\nuse bytes::{Buf, BytesMut};\nuse futures::Stream;\nuse std::collections::VecDeque;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::Poll::{Pending, Ready};\nuse std::task::{Context, Poll};\n\nmacro_rules! mock {\n ($($x:expr,)*) => {{\n let mut v = VecDeque::new();\n v.extend(vec![$($x),*]);\n Mock { calls: v }\n }};\n}\n\nmacro_rules! assert_read {\n ($e:expr, $n:expr) => {{\n let val = assert_ready!($e);\n assert_eq!(val.unwrap().unwrap(), $n);\n }};\n}\n\nmacro_rules! pin {\n ($id:ident) => {\n Pin::new(&mut $id)\n };\n}\n\nstruct U32Decoder;\n\nimpl Decoder for U32Decoder {\n type Item = u32;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n if buf.len() < 4 {\n return Ok(None);\n }\n\n let n = buf.split_to(4).get_u32();\n Ok(Some(n))\n }\n}\n\nstruct U64Decoder;\n\nimpl Decoder for U64Decoder {\n type Item = u64;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n if buf.len() < 8 {\n return Ok(None);\n }\n\n let n = buf.split_to(8).get_u64();\n Ok(Some(n))\n }\n}\n\n#[test]\nfn read_multi_frame_in_packet() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x02\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert_read!(pin!(framed).poll_next(cx), 2);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_multi_frame_across_packets() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x00\".to_vec()),\n Ok(b\"\\x00\\x00\\x00\\x01\".to_vec()),\n Ok(b\"\\x00\\x00\\x00\\x02\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert_read!(pin!(framed).poll_next(cx), 2);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_multi_frame_in_packet_after_codec_changed() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x08\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0x04);\n\n let mut framed = framed.map_decoder(|_| U64Decoder);\n assert_read!(pin!(framed).poll_next(cx), 0x08);\n\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_not_ready() {\n let mut task = task::spawn(());\n let mock = mock! {\n Err(io::Error::new(io::ErrorKind::WouldBlock, \"\")),\n Ok(b\"\\x00\\x00\\x00\\x00\".to_vec()),\n Ok(b\"\\x00\\x00\\x00\\x01\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert!(pin!(framed).poll_next(cx).is_pending());\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_partial_then_not_ready() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\".to_vec()),\n Err(io::Error::new(io::ErrorKind::WouldBlock, \"\")),\n Ok(b\"\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x02\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert!(pin!(framed).poll_next(cx).is_pending());\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert_read!(pin!(framed).poll_next(cx), 2);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_err() {\n let mut task = task::spawn(());\n let mock = mock! {\n Err(io::Error::new(io::ErrorKind::Other, \"\")),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_eq!(\n io::ErrorKind::Other,\n assert_ready!(pin!(framed).poll_next(cx))\n .unwrap()\n .unwrap_err()\n .kind()\n )\n });\n}\n\n#[test]\nfn read_partial_then_err() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\".to_vec()),\n Err(io::Error::new(io::ErrorKind::Other, \"\")),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_eq!(\n io::ErrorKind::Other,\n assert_ready!(pin!(framed).poll_next(cx))\n .unwrap()\n .unwrap_err()\n .kind()\n )\n });\n}\n\n#[test]\nfn read_partial_would_block_then_err() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\".to_vec()),\n Err(io::Error::new(io::ErrorKind::WouldBlock, \"\")),\n Err(io::Error::new(io::ErrorKind::Other, \"\")),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert!(pin!(framed).poll_next(cx).is_pending());\n assert_eq!(\n io::ErrorKind::Other,\n assert_ready!(pin!(framed).poll_next(cx))\n .unwrap()\n .unwrap_err()\n .kind()\n )\n });\n}\n\n#[test]\nfn huge_size() {\n let mut task = task::spawn(());\n let data = &[0; 32 * 1024][..];\n let mut framed = FramedRead::new(data, BigDecoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n\n struct BigDecoder;\n\n impl Decoder for BigDecoder {\n type Item = u32;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> io::Result> {\n if buf.len() < 32 * 1024 {\n return Ok(None);\n }\n buf.advance(32 * 1024);\n Ok(Some(0))\n }\n }\n}\n\n#[test]\nfn data_remaining_is_error() {\n let mut task = task::spawn(());\n let slice = &[0; 5][..];\n let mut framed = FramedRead::new(slice, U32Decoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).unwrap().is_err());\n });\n}\n\n#[test]\nfn multi_frames_on_eof() {\n let mut task = task::spawn(());\n struct MyDecoder(Vec);\n\n impl Decoder for MyDecoder {\n type Item = u32;\n type Error = io::Error;\n\n fn decode(&mut self, _buf: &mut BytesMut) -> io::Result> {\n unreachable!();\n }\n\n fn decode_eof(&mut self, _buf: &mut BytesMut) -> io::Result> {\n if self.0.is_empty() {\n return Ok(None);\n }\n\n Ok(Some(self.0.remove(0)))\n }\n }\n\n let mut framed = FramedRead::new(mock!(), MyDecoder(vec![0, 1, 2, 3]));\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 0);\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert_read!(pin!(framed).poll_next(cx), 2);\n assert_read!(pin!(framed).poll_next(cx), 3);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n#[test]\nfn read_eof_then_resume() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x01\".to_vec()),\n Ok(b\"\".to_vec()),\n Ok(b\"\\x00\\x00\\x00\\x02\".to_vec()),\n Ok(b\"\".to_vec()),\n Ok(b\"\\x00\\x00\\x00\\x03\".to_vec()),\n };\n let mut framed = FramedRead::new(mock, U32Decoder);\n\n task.enter(|cx, _| {\n assert_read!(pin!(framed).poll_next(cx), 1);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n assert_read!(pin!(framed).poll_next(cx), 2);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n assert_read!(pin!(framed).poll_next(cx), 3);\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n assert!(assert_ready!(pin!(framed).poll_next(cx)).is_none());\n });\n}\n\n// ===== Mock ======\n\nstruct Mock {\n calls: VecDeque>>,\n}\n\nimpl AsyncRead for Mock {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n use io::ErrorKind::WouldBlock;\n\n match self.calls.pop_front() {\n Some(Ok(data)) => {\n debug_assert!(buf.remaining() >= data.len());\n buf.put_slice(&data);\n Ready(Ok(()))\n }\n Some(Err(ref e)) if e.kind() == WouldBlock => Pending,\n Some(Err(e)) => Ready(Err(e)),\n None => Ready(Ok(())),\n }\n }\n}\n" }, { "path": "tokio-util/tests/framed_stream.rs", "content": "use futures_core::stream::Stream;\nuse std::{io, pin::Pin};\nuse tokio_test::{assert_ready, io::Builder, task};\nuse tokio_util::codec::{BytesCodec, FramedRead};\n\nmacro_rules! pin {\n ($id:ident) => {\n Pin::new(&mut $id)\n };\n}\n\nmacro_rules! assert_read {\n ($e:expr, $n:expr) => {{\n let val = assert_ready!($e);\n assert_eq!(val.unwrap().unwrap(), $n);\n }};\n}\n\n#[tokio::test]\nasync fn return_none_after_error() {\n let mut io = FramedRead::new(\n Builder::new()\n .read(b\"abcdef\")\n .read_error(io::Error::new(io::ErrorKind::Other, \"Resource errored out\"))\n .read(b\"more data\")\n .build(),\n BytesCodec::new(),\n );\n\n let mut task = task::spawn(());\n\n task.enter(|cx, _| {\n assert_read!(pin!(io).poll_next(cx), b\"abcdef\".to_vec());\n assert!(assert_ready!(pin!(io).poll_next(cx)).unwrap().is_err());\n assert!(assert_ready!(pin!(io).poll_next(cx)).is_none());\n assert_read!(pin!(io).poll_next(cx), b\"more data\".to_vec());\n })\n}\n" }, { "path": "tokio-util/tests/framed_write.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::io::AsyncWrite;\nuse tokio_test::{assert_ready, task};\nuse tokio_util::codec::{Encoder, FramedWrite};\n\nuse bytes::{BufMut, BytesMut};\nuse futures_sink::Sink;\nuse std::collections::VecDeque;\nuse std::io::{self, Write};\nuse std::pin::Pin;\nuse std::task::Poll::{Pending, Ready};\nuse std::task::{Context, Poll};\n\nmacro_rules! mock {\n ($($x:expr,)*) => {{\n let mut v = VecDeque::new();\n v.extend(vec![$($x),*]);\n Mock { calls: v }\n }};\n}\n\nmacro_rules! pin {\n ($id:ident) => {\n Pin::new(&mut $id)\n };\n}\n\nstruct U32Encoder;\n\nimpl Encoder for U32Encoder {\n type Error = io::Error;\n\n fn encode(&mut self, item: u32, dst: &mut BytesMut) -> io::Result<()> {\n // Reserve space\n dst.reserve(4);\n dst.put_u32(item);\n Ok(())\n }\n}\n\nstruct U64Encoder;\n\nimpl Encoder for U64Encoder {\n type Error = io::Error;\n\n fn encode(&mut self, item: u64, dst: &mut BytesMut) -> io::Result<()> {\n // Reserve space\n dst.reserve(8);\n dst.put_u64(item);\n Ok(())\n }\n}\n\n#[test]\nfn write_multi_frame_in_packet() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x01\\x00\\x00\\x00\\x02\".to_vec()),\n };\n let mut framed = FramedWrite::new(mock, U32Encoder);\n\n task.enter(|cx, _| {\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(0).is_ok());\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(1).is_ok());\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(2).is_ok());\n\n // Nothing written yet\n assert_eq!(1, framed.get_ref().calls.len());\n\n // Flush the writes\n assert!(assert_ready!(pin!(framed).poll_flush(cx)).is_ok());\n\n assert_eq!(0, framed.get_ref().calls.len());\n });\n}\n\n#[test]\nfn write_multi_frame_after_codec_changed() {\n let mut task = task::spawn(());\n let mock = mock! {\n Ok(b\"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x08\".to_vec()),\n };\n let mut framed = FramedWrite::new(mock, U32Encoder);\n\n task.enter(|cx, _| {\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(0x04).is_ok());\n\n let mut framed = framed.map_encoder(|_| U64Encoder);\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(0x08).is_ok());\n\n // Nothing written yet\n assert_eq!(1, framed.get_ref().calls.len());\n\n // Flush the writes\n assert!(assert_ready!(pin!(framed).poll_flush(cx)).is_ok());\n\n assert_eq!(0, framed.get_ref().calls.len());\n });\n}\n\n#[test]\nfn write_hits_backpressure() {\n const ITER: usize = 2 * 1024;\n\n let mut mock = mock! {\n // Block the `ITER*2`th write\n Err(io::Error::new(io::ErrorKind::WouldBlock, \"not ready\")),\n Ok(b\"\".to_vec()),\n };\n\n for i in 0..=ITER * 2 {\n let mut b = BytesMut::with_capacity(4);\n b.put_u32(i as u32);\n\n // Append to the end\n match mock.calls.back_mut().unwrap() {\n Ok(ref mut data) => {\n // Write in 2kb chunks\n if data.len() < ITER {\n data.extend_from_slice(&b[..]);\n continue;\n } // else fall through and create a new buffer\n }\n _ => unreachable!(),\n }\n\n // Push a new chunk\n mock.calls.push_back(Ok(b[..].to_vec()));\n }\n // 1 'wouldblock', 8 * 2KB buffers, 1 b-byte buffer\n assert_eq!(mock.calls.len(), 10);\n\n let mut task = task::spawn(());\n let mut framed = FramedWrite::new(mock, U32Encoder);\n framed.set_backpressure_boundary(ITER * 8);\n task.enter(|cx, _| {\n // Send 16KB. This fills up FramedWrite buffer\n for i in 0..ITER * 2 {\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n assert!(pin!(framed).start_send(i as u32).is_ok());\n }\n\n // Now we poll_ready which forces a flush. The mock pops the front message\n // and decides to block.\n assert!(pin!(framed).poll_ready(cx).is_pending());\n\n // We poll again, forcing another flush, which this time succeeds\n // The whole 16KB buffer is flushed\n assert!(assert_ready!(pin!(framed).poll_ready(cx)).is_ok());\n\n // Send more data. This matches the final message expected by the mock\n assert!(pin!(framed).start_send((ITER * 2) as u32).is_ok());\n\n // Flush the rest of the buffer\n assert!(assert_ready!(pin!(framed).poll_flush(cx)).is_ok());\n\n // Ensure the mock is empty\n assert_eq!(0, framed.get_ref().calls.len());\n })\n}\n\n// // ===== Mock ======\n\nstruct Mock {\n calls: VecDeque>>,\n}\n\nimpl Write for Mock {\n fn write(&mut self, src: &[u8]) -> io::Result {\n match self.calls.pop_front() {\n Some(Ok(data)) => {\n assert!(src.len() >= data.len());\n assert_eq!(&data[..], &src[..data.len()]);\n Ok(data.len())\n }\n Some(Err(e)) => Err(e),\n None => panic!(\"unexpected write; {src:?}\"),\n }\n }\n\n fn flush(&mut self) -> io::Result<()> {\n Ok(())\n }\n}\n\nimpl AsyncWrite for Mock {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n match Pin::get_mut(self).write(buf) {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => Pending,\n other => Ready(other),\n }\n }\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n match Pin::get_mut(self).flush() {\n Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => Pending,\n other => Ready(other),\n }\n }\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n unimplemented!()\n }\n}\n" }, { "path": "tokio-util/tests/future.rs", "content": "use std::{\n future::{pending, ready, Future},\n task::{Context, Poll},\n};\n\nuse futures_test::task::new_count_waker;\nuse tokio::pin;\nuse tokio_test::{assert_pending, assert_ready_eq};\nuse tokio_util::{future::FutureExt, sync::CancellationToken};\n\n#[derive(Default)]\nstruct ReadyOnTheSecondPollFuture {\n polled: bool,\n}\n\nimpl Future for ReadyOnTheSecondPollFuture {\n type Output = ();\n\n fn poll(mut self: std::pin::Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n if !self.polled {\n self.polled = true;\n return Poll::Pending;\n }\n Poll::Ready(())\n }\n}\n\n#[test]\nfn ready_fut_with_cancellation_token_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n\n let ready_fut = ready(());\n\n let ready_with_token_fut = ready_fut.with_cancellation_token(&token);\n\n pin!(ready_with_token_fut);\n\n let res = ready_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, Some(()));\n}\n\n#[test]\nfn pending_fut_with_cancellation_token_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token(&token);\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_pending!(res);\n}\n\n#[test]\nfn ready_fut_with_already_cancelled_token_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n token.cancel();\n\n let ready_fut = ready(());\n\n let ready_fut_with_token_fut = ready_fut.with_cancellation_token(&token);\n\n pin!(ready_fut_with_token_fut);\n\n let res = ready_fut_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, None);\n}\n\n#[test]\nfn pending_fut_with_already_cancelled_token_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n token.cancel();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token(&token);\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, None);\n assert_eq!(wake_count, 0);\n}\n\n#[test]\nfn pending_fut_with_token_cancelled_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token(&token);\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n assert_pending!(res);\n\n token.cancel();\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n assert_ready_eq!(res, None);\n assert_eq!(wake_count, 1);\n}\n\n#[test]\nfn pending_only_on_first_poll_with_cancellation_token_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n let fut = ReadyOnTheSecondPollFuture::default().with_cancellation_token(&token);\n pin!(fut);\n\n // first poll, ReadyOnTheSecondPollFuture returned Pending\n let res = fut.as_mut().poll(&mut Context::from_waker(&waker));\n assert_pending!(res);\n\n token.cancel();\n assert_eq!(wake_count, 1);\n\n // due to the polling fairness (biased behavior) of `WithCancellationToken` Future,\n // subsequent polls are biased toward polling ReadyOnTheSecondPollFuture,\n // which results in always returning Ready.\n let res = fut.as_mut().poll(&mut Context::from_waker(&waker));\n assert_ready_eq!(res, Some(()));\n}\n\n#[test]\nfn ready_fut_with_cancellation_owned_token_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n\n let ready_fut = ready(());\n\n let ready_with_token_fut = ready_fut.with_cancellation_token_owned(token);\n\n pin!(ready_with_token_fut);\n\n let res = ready_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, Some(()));\n}\n\n#[test]\nfn pending_fut_with_cancellation_token_owned_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token_owned(token);\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_pending!(res);\n}\n\n#[test]\nfn ready_fut_with_already_cancelled_token_owned_test() {\n let (waker, _) = new_count_waker();\n let token = CancellationToken::new();\n token.cancel();\n\n let ready_fut = ready(());\n\n let ready_fut_with_token_fut = ready_fut.with_cancellation_token_owned(token);\n\n pin!(ready_fut_with_token_fut);\n\n let res = ready_fut_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, None);\n}\n\n#[test]\nfn pending_fut_with_already_cancelled_token_owned_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n token.cancel();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token_owned(token);\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n\n assert_ready_eq!(res, None);\n assert_eq!(wake_count, 0);\n}\n\n#[test]\nfn pending_fut_with_owned_token_cancelled_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n\n let pending_fut = pending::<()>();\n\n let pending_with_token_fut = pending_fut.with_cancellation_token_owned(token.clone());\n\n pin!(pending_with_token_fut);\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n assert_pending!(res);\n\n token.cancel();\n\n let res = pending_with_token_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker));\n assert_ready_eq!(res, None);\n assert_eq!(wake_count, 1);\n}\n\n#[test]\nfn pending_only_on_first_poll_with_cancellation_token_owned_test() {\n let (waker, wake_count) = new_count_waker();\n let token = CancellationToken::new();\n let fut = ReadyOnTheSecondPollFuture::default().with_cancellation_token_owned(token.clone());\n pin!(fut);\n\n // first poll, ReadyOnTheSecondPollFuture returned Pending\n let res = fut.as_mut().poll(&mut Context::from_waker(&waker));\n assert_pending!(res);\n\n token.cancel();\n assert_eq!(wake_count, 1);\n\n // due to the polling fairness (biased behavior) of `WithCancellationToken` Future,\n // subsequent polls are biased toward polling ReadyOnTheSecondPollFuture,\n // which results in always returning Ready.\n let res = fut.as_mut().poll(&mut Context::from_waker(&waker));\n assert_ready_eq!(res, Some(()));\n}\n" }, { "path": "tokio-util/tests/io_inspect.rs", "content": "use std::{\n future::poll_fn,\n io::IoSlice,\n pin::Pin,\n task::{Context, Poll},\n};\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf};\nuse tokio_util::io::{InspectReader, InspectWriter};\n\n/// An AsyncRead implementation that works byte-by-byte, to catch out callers\n/// who don't allow for `buf` being part-filled before the call\nstruct SmallReader {\n contents: Vec,\n}\n\nimpl Unpin for SmallReader {}\n\nimpl AsyncRead for SmallReader {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n if let Some(byte) = self.contents.pop() {\n buf.put_slice(&[byte])\n }\n Poll::Ready(Ok(()))\n }\n}\n\n#[tokio::test]\nasync fn read_tee() {\n let contents = b\"This could be really long, you know\".to_vec();\n let reader = SmallReader {\n contents: contents.clone(),\n };\n let mut altout: Vec = Vec::new();\n let mut teeout = Vec::new();\n {\n let mut tee = InspectReader::new(reader, |bytes| altout.extend(bytes));\n tee.read_to_end(&mut teeout).await.unwrap();\n }\n assert_eq!(teeout, altout);\n assert_eq!(altout.len(), contents.len());\n}\n\n/// An AsyncWrite implementation that works byte-by-byte for poll_write, and\n/// that reads the whole of the first buffer plus one byte from the second in\n/// poll_write_vectored.\n///\n/// This is designed to catch bugs in handling partially written buffers\n#[derive(Debug)]\nstruct SmallWriter {\n contents: Vec,\n}\n\nimpl Unpin for SmallWriter {}\n\nimpl AsyncWrite for SmallWriter {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &[u8],\n ) -> Poll> {\n // Just write one byte at a time\n if buf.is_empty() {\n return Poll::Ready(Ok(0));\n }\n self.contents.push(buf[0]);\n Poll::Ready(Ok(1))\n }\n\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_shutdown(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n ) -> Poll> {\n Poll::Ready(Ok(()))\n }\n\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n bufs: &[IoSlice<'_>],\n ) -> Poll> {\n // Write all of the first buffer, then one byte from the second buffer\n // This should trip up anything that doesn't correctly handle multiple\n // buffers.\n if bufs.is_empty() {\n return Poll::Ready(Ok(0));\n }\n let mut written_len = bufs[0].len();\n self.contents.extend_from_slice(&bufs[0]);\n\n if bufs.len() > 1 {\n let buf = bufs[1];\n if !buf.is_empty() {\n written_len += 1;\n self.contents.push(buf[0]);\n }\n }\n Poll::Ready(Ok(written_len))\n }\n\n fn is_write_vectored(&self) -> bool {\n true\n }\n}\n\n#[tokio::test]\nasync fn write_tee() {\n let mut altout: Vec = Vec::new();\n let mut writeout = SmallWriter {\n contents: Vec::new(),\n };\n {\n let mut tee = InspectWriter::new(&mut writeout, |bytes| altout.extend(bytes));\n tee.write_all(b\"A testing string, very testing\")\n .await\n .unwrap();\n }\n assert_eq!(altout, writeout.contents);\n}\n\n// This is inefficient, but works well enough for test use.\n// If you want something similar for real code, you'll want to avoid all the\n// fun of manipulating `bufs` - ideally, by the time you read this,\n// IoSlice::advance_slices will be stable, and you can use that.\nasync fn write_all_vectored(\n mut writer: W,\n mut bufs: Vec>,\n) -> Result {\n let mut res = 0;\n while !bufs.is_empty() {\n let mut written = poll_fn(|cx| {\n let bufs: Vec = bufs.iter().map(|v| IoSlice::new(v)).collect();\n Pin::new(&mut writer).poll_write_vectored(cx, &bufs)\n })\n .await?;\n res += written;\n while written > 0 {\n let buf_len = bufs[0].len();\n if buf_len <= written {\n bufs.remove(0);\n written -= buf_len;\n } else {\n let buf = &mut bufs[0];\n let drain_len = written.min(buf.len());\n buf.drain(..drain_len);\n written -= drain_len;\n }\n }\n }\n Ok(res)\n}\n\n#[tokio::test]\nasync fn write_tee_vectored() {\n let mut altout: Vec = Vec::new();\n let mut writeout = SmallWriter {\n contents: Vec::new(),\n };\n let original = b\"A very long string split up\";\n let bufs: Vec> = original\n .split(|b| b.is_ascii_whitespace())\n .map(Vec::from)\n .collect();\n assert!(bufs.len() > 1);\n let expected: Vec = {\n let mut out = Vec::new();\n for item in &bufs {\n out.extend_from_slice(item)\n }\n out\n };\n {\n let mut bufcount = 0;\n let tee = InspectWriter::new(&mut writeout, |bytes| {\n bufcount += 1;\n altout.extend(bytes)\n });\n\n assert!(tee.is_write_vectored());\n\n write_all_vectored(tee, bufs.clone()).await.unwrap();\n\n assert!(bufcount >= bufs.len());\n }\n assert_eq!(altout, writeout.contents);\n assert_eq!(writeout.contents, expected);\n}\n" }, { "path": "tokio-util/tests/io_reader_stream.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\nuse tokio::io::{AsyncRead, ReadBuf};\nuse tokio_stream::StreamExt;\n\n/// produces at most `remaining` zeros, that returns error.\n/// each time it reads at most 31 byte.\nstruct Reader {\n remaining: usize,\n}\n\nimpl AsyncRead for Reader {\n fn poll_read(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n buf: &mut ReadBuf<'_>,\n ) -> Poll> {\n let this = Pin::into_inner(self);\n assert_ne!(buf.remaining(), 0);\n if this.remaining > 0 {\n let n = std::cmp::min(this.remaining, buf.remaining());\n let n = std::cmp::min(n, 31);\n for x in &mut buf.initialize_unfilled_to(n)[..n] {\n *x = 0;\n }\n buf.advance(n);\n this.remaining -= n;\n Poll::Ready(Ok(()))\n } else {\n Poll::Ready(Err(std::io::Error::from_raw_os_error(22)))\n }\n }\n}\n\n#[tokio::test]\nasync fn correct_behavior_on_errors() {\n let reader = Reader { remaining: 8000 };\n let mut stream = tokio_util::io::ReaderStream::new(reader);\n let mut zeros_received = 0;\n let mut had_error = false;\n loop {\n let item = stream.next().await.unwrap();\n println!(\"{item:?}\");\n match item {\n Ok(bytes) => {\n let bytes = &*bytes;\n for byte in bytes {\n assert_eq!(*byte, 0);\n zeros_received += 1;\n }\n }\n Err(_) => {\n assert!(!had_error);\n had_error = true;\n break;\n }\n }\n }\n\n assert!(had_error);\n assert_eq!(zeros_received, 8000);\n assert!(stream.next().await.is_none());\n}\n" }, { "path": "tokio-util/tests/io_simplex.rs", "content": "use futures::pin_mut;\nuse futures_test::task::noop_context;\nuse std::io::IoSlice;\nuse std::task::Poll;\nuse tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, ReadBuf};\nuse tokio_test::task::spawn;\nuse tokio_test::{assert_pending, assert_ready};\nuse tokio_util::io::simplex;\n\n/// Sanity check for single-threaded operation.\n#[tokio::test]\nasync fn single_thread() {\n const N: usize = 64;\n const MSG: &[u8] = b\"Hello, world!\";\n const CAPS: &[usize] = &[1, MSG.len() / 2, MSG.len() - 1, MSG.len(), MSG.len() + 1];\n\n // test different buffer capacities to cover edge cases\n for &capacity in CAPS {\n let (mut tx, mut rx) = simplex::new(capacity);\n\n for _ in 0..N {\n let mut read = 0;\n let mut write = 0;\n let mut buf = [0; MSG.len()];\n\n while read < MSG.len() || write < MSG.len() {\n if write < MSG.len() {\n let n = tx.write(&MSG[write..]).await.unwrap();\n write += n;\n }\n\n if read < MSG.len() {\n let n = rx.read(&mut buf[read..]).await.unwrap();\n read += n;\n }\n }\n\n assert_eq!(&buf[..], MSG);\n }\n }\n}\n\n/// Sanity check for multi-threaded operation.\n#[test]\n#[cfg(not(target_os = \"wasi\"))] // No thread on wasi.\nfn multi_thread() {\n use futures::executor::block_on;\n use std::thread;\n\n const N: usize = 64;\n const MSG: &[u8] = b\"Hello, world!\";\n const CAPS: &[usize] = &[1, MSG.len() / 2, MSG.len() - 1, MSG.len(), MSG.len() + 1];\n\n // test different buffer capacities to cover edge cases\n for &capacity in CAPS {\n let (mut tx, mut rx) = simplex::new(capacity);\n\n let jh0 = thread::spawn(move || {\n block_on(async {\n let mut buf = vec![0; MSG.len()];\n for _ in 0..N {\n rx.read_exact(&mut buf).await.unwrap();\n assert_eq!(&buf[..], MSG);\n buf.clear();\n buf.resize(MSG.len(), 0);\n }\n });\n });\n\n let jh1 = thread::spawn(move || {\n block_on(async {\n for _ in 0..N {\n tx.write_all(MSG).await.unwrap();\n }\n });\n });\n\n jh0.join().unwrap();\n jh1.join().unwrap();\n }\n}\n\n#[test]\n#[should_panic(expected = \"capacity must be greater than zero\")]\nfn zero_capacity() {\n let _ = simplex::new(0);\n}\n\n/// The `Receiver::poll_read` should return `Poll::Ready(Ok(()))`\n/// if the `ReadBuf` has zero remaining capacity.\n#[tokio::test]\nasync fn read_buf_is_full() {\n let (_tx, rx) = simplex::new(32);\n let mut buf = ReadBuf::new(&mut []);\n tokio::pin!(rx);\n assert_ready!(rx.as_mut().poll_read(&mut noop_context(), &mut buf)).unwrap();\n assert_eq!(buf.filled().len(), 0);\n}\n\n/// The `Sender::poll_write` should return `Poll::Ready(Ok(0))`\n/// if the input buffer has zero length.\n#[tokio::test]\nasync fn write_buf_is_empty() {\n let (tx, _rx) = simplex::new(32);\n tokio::pin!(tx);\n let n = assert_ready!(tx.as_mut().poll_write(&mut noop_context(), &[])).unwrap();\n assert_eq!(n, 0);\n}\n\n/// The `Sender` should returns error if the `Receiver` has been dropped.\n#[tokio::test]\nasync fn drop_receiver_0() {\n let (mut tx, rx) = simplex::new(32);\n drop(rx);\n\n tx.write_u8(1).await.unwrap_err();\n}\n\n/// The `Sender` should be woken up if the `Receiver` has been dropped.\n#[tokio::test]\nasync fn drop_receiver_1() {\n let (mut tx, rx) = simplex::new(1);\n let mut write_task = spawn(tx.write_u16(1));\n assert_pending!(write_task.poll());\n\n assert!(!write_task.is_woken());\n drop(rx);\n assert!(write_task.is_woken());\n}\n\n/// The `Receiver` should return error if:\n///\n/// - The `Sender` has been dropped.\n/// - AND there is no remaining data in the buffer.\n#[tokio::test]\nasync fn drop_sender_0() {\n const MSG: &[u8] = b\"Hello, world!\";\n\n let (tx, mut rx) = simplex::new(32);\n drop(tx);\n\n let mut buf = vec![0; MSG.len()];\n rx.read_exact(&mut buf).await.unwrap_err();\n}\n\n/// The `Receiver` should be woken up if:\n///\n/// - The `Sender` has been dropped.\n/// - AND there is still remaining data in the buffer.\n#[tokio::test]\nasync fn drop_sender_1() {\n let (mut tx, mut rx) = simplex::new(2);\n let mut buf = vec![];\n let mut read_task = spawn(rx.read_to_end(&mut buf));\n assert_pending!(read_task.poll());\n\n tx.write_u8(1).await.unwrap();\n assert_pending!(read_task.poll());\n\n assert!(!read_task.is_woken());\n drop(tx);\n assert!(read_task.is_woken());\n\n read_task.await.unwrap();\n assert_eq!(buf, vec![1]);\n}\n\n/// All following calls to `Sender::poll_write` and `Sender::poll_flush`\n/// should return error after `shutdown` has been called.\n#[tokio::test]\nasync fn shutdown_sender_0() {\n const MSG: &[u8] = b\"Hello, world!\";\n\n let (mut tx, _rx) = simplex::new(32);\n tx.shutdown().await.unwrap();\n\n tx.write_all(MSG).await.unwrap_err();\n tx.flush().await.unwrap_err();\n}\n\n/// The `Sender::poll_shutdown` should be called multiple times\n/// without error.\n#[tokio::test]\nasync fn shutdown_sender_1() {\n let (mut tx, _rx) = simplex::new(32);\n tx.shutdown().await.unwrap();\n tx.shutdown().await.unwrap();\n}\n\n/// The `Sender::poll_shutdown` should wake up the `Receiver`\n#[tokio::test]\nasync fn shutdown_sender_2() {\n let (mut tx, mut rx) = simplex::new(32);\n\n let mut buf = vec![];\n let mut read_task = spawn(rx.read_to_end(&mut buf));\n assert_pending!(read_task.poll());\n\n tx.write_u8(1).await.unwrap();\n assert_pending!(read_task.poll());\n\n assert!(!read_task.is_woken());\n tx.shutdown().await.unwrap();\n assert!(read_task.is_woken());\n\n read_task.await.unwrap();\n assert_eq!(buf, vec![1]);\n}\n\n/// Both `Sender` and `Receiver` should yield periodically\n/// in a tight-loop.\n#[tokio::test]\n#[cfg(feature = \"rt\")]\nasync fn cooperative_scheduling() {\n // this magic number is copied from\n // https://github.com/tokio-rs/tokio/blob/925c614c89d0a26777a334612e2ed6ad0e7935c3/tokio/src/task/coop/mod.rs#L116\n const INITIAL_BUDGET: usize = 128;\n\n let (tx, _rx) = simplex::new(INITIAL_BUDGET * 2);\n pin_mut!(tx);\n let mut is_pending = false;\n for _ in 0..INITIAL_BUDGET + 1 {\n match tx.as_mut().poll_write(&mut noop_context(), &[0u8; 1]) {\n Poll::Pending => {\n is_pending = true;\n break;\n }\n Poll::Ready(Ok(1)) => {}\n Poll::Ready(Ok(n)) => panic!(\"wrote too many bytes: {n}\"),\n Poll::Ready(Err(e)) => panic!(\"{e}\"),\n }\n }\n assert!(is_pending);\n\n let (tx, _rx) = simplex::new(INITIAL_BUDGET * 2);\n pin_mut!(tx);\n let mut is_pending = false;\n let io_slices = &[IoSlice::new(&[0u8; 1])];\n for _ in 0..INITIAL_BUDGET + 1 {\n match tx\n .as_mut()\n .poll_write_vectored(&mut noop_context(), io_slices)\n {\n Poll::Pending => {\n is_pending = true;\n break;\n }\n Poll::Ready(Ok(1)) => {}\n Poll::Ready(Ok(n)) => panic!(\"wrote too many bytes: {n}\"),\n Poll::Ready(Err(e)) => panic!(\"{e}\"),\n }\n }\n assert!(is_pending);\n\n let (mut tx, rx) = simplex::new(INITIAL_BUDGET * 2);\n tx.write_all(&[0u8; INITIAL_BUDGET + 2]).await.unwrap();\n pin_mut!(rx);\n let mut is_pending = false;\n for _ in 0..INITIAL_BUDGET + 1 {\n let mut buf = [0u8; 1];\n let mut buf = ReadBuf::new(&mut buf);\n match rx.as_mut().poll_read(&mut noop_context(), &mut buf) {\n Poll::Pending => {\n is_pending = true;\n break;\n }\n Poll::Ready(Ok(())) => assert_eq!(buf.filled().len(), 1),\n Poll::Ready(Err(e)) => panic!(\"{e}\"),\n }\n }\n assert!(is_pending);\n}\n\n/// The capacity is exactly same as the total length of the vectored buffers.\n#[tokio::test]\nasync fn poll_write_vectored_0() {\n const MSG1: &[u8] = b\"1\";\n const MSG2: &[u8] = b\"22\";\n const MSG3: &[u8] = b\"333\";\n const MSG_LEN: usize = MSG1.len() + MSG2.len() + MSG3.len();\n\n let io_slices = &[IoSlice::new(MSG1), IoSlice::new(MSG2), IoSlice::new(MSG3)];\n\n let (tx, mut rx) = simplex::new(MSG_LEN);\n tokio::pin!(tx);\n let res = tx.poll_write_vectored(&mut noop_context(), io_slices);\n let n = assert_ready!(res).unwrap();\n assert_eq!(n, MSG_LEN);\n let mut buf = [0; MSG_LEN];\n let n = rx.read_exact(&mut buf).await.unwrap();\n assert_eq!(n, MSG_LEN);\n assert_eq!(&buf, b\"122333\");\n}\n\n/// The capacity is smaller than the total length of the vectored buffers.\n#[tokio::test]\nasync fn poll_write_vectored_1() {\n const MSG1: &[u8] = b\"1\";\n const MSG2: &[u8] = b\"22\";\n const MSG3: &[u8] = b\"333\";\n const CAPACITY: usize = MSG1.len() + MSG2.len() + 1;\n\n let io_slices = &[IoSlice::new(MSG1), IoSlice::new(MSG2), IoSlice::new(MSG3)];\n\n let (tx, mut rx) = simplex::new(CAPACITY);\n tokio::pin!(tx);\n\n // ==== The poll_write_vectored should write MSG1 and MSG2 fully, and MSG3 partially. ====\n let res = tx.poll_write_vectored(&mut noop_context(), io_slices);\n let n = assert_ready!(res).unwrap();\n assert_eq!(n, CAPACITY);\n let mut buf = [0; CAPACITY];\n let n = rx.read_exact(&mut buf).await.unwrap();\n assert_eq!(n, CAPACITY);\n assert_eq!(&buf, b\"1223\");\n}\n\n/// There are two empty buffers in the vectored buffers.\n#[tokio::test]\nasync fn poll_write_vectored_2() {\n const MSG1: &[u8] = b\"1\";\n const MSG2: &[u8] = b\"\";\n const MSG3: &[u8] = b\"22\";\n const MSG4: &[u8] = b\"\";\n const MSG5: &[u8] = b\"333\";\n const MSG_LEN: usize = MSG1.len() + MSG2.len() + MSG3.len() + MSG4.len() + MSG5.len();\n\n let io_slices = &[\n IoSlice::new(MSG1),\n IoSlice::new(MSG2),\n IoSlice::new(MSG3),\n IoSlice::new(MSG4),\n IoSlice::new(MSG5),\n ];\n\n let (tx, mut rx) = simplex::new(MSG_LEN);\n tokio::pin!(tx);\n let res = tx.poll_write_vectored(&mut noop_context(), io_slices);\n let n = assert_ready!(res).unwrap();\n assert_eq!(n, MSG_LEN);\n let mut buf = [0; MSG_LEN];\n let n = rx.read_exact(&mut buf).await.unwrap();\n assert_eq!(n, MSG_LEN);\n assert_eq!(&buf, b\"122333\");\n}\n\n/// The `Sender::poll_write_vectored` should return `Poll::Ready(Ok(0))`\n/// if all the input buffers have zero length.\n#[tokio::test]\nasync fn poll_write_vectored_3() {\n let io_slices = &[IoSlice::new(&[]), IoSlice::new(&[]), IoSlice::new(&[])];\n let (tx, _rx) = simplex::new(32);\n tokio::pin!(tx);\n let n = assert_ready!(tx.poll_write_vectored(&mut noop_context(), io_slices)).unwrap();\n assert_eq!(n, 0);\n}\n" }, { "path": "tokio-util/tests/io_sink_writer.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse bytes::Bytes;\nuse futures_util::SinkExt;\nuse std::io::{self, Error, ErrorKind};\nuse tokio::io::AsyncWriteExt;\nuse tokio_util::codec::{Encoder, FramedWrite};\nuse tokio_util::io::{CopyToBytes, SinkWriter};\nuse tokio_util::sync::PollSender;\n\n#[tokio::test]\nasync fn test_copied_sink_writer() -> Result<(), Error> {\n // Construct a channel pair to send data across and wrap a pollable sink.\n // Note that the sink must mimic a writable object, e.g. have `std::io::Error`\n // as its error type.\n // As `PollSender` requires an owned copy of the buffer, we wrap it additionally\n // with a `CopyToBytes` helper.\n let (tx, mut rx) = tokio::sync::mpsc::channel::(1);\n let mut writer = SinkWriter::new(CopyToBytes::new(\n PollSender::new(tx).sink_map_err(|_| io::Error::from(ErrorKind::BrokenPipe)),\n ));\n\n // Write data to our interface...\n let data: [u8; 4] = [1, 2, 3, 4];\n let _ = writer.write(&data).await;\n\n // ... and receive it.\n assert_eq!(data.to_vec(), rx.recv().await.unwrap().to_vec());\n\n Ok(())\n}\n\n/// A trivial encoder.\nstruct SliceEncoder;\n\nimpl SliceEncoder {\n fn new() -> Self {\n Self {}\n }\n}\n\nimpl<'a> Encoder<&'a [u8]> for SliceEncoder {\n type Error = Error;\n\n fn encode(&mut self, item: &'a [u8], dst: &mut bytes::BytesMut) -> Result<(), Self::Error> {\n // This is where we'd write packet headers, lengths, etc. in a real encoder.\n // For simplicity and demonstration purposes, we just pack a copy of\n // the slice at the end of a buffer.\n dst.extend_from_slice(item);\n Ok(())\n }\n}\n\n#[tokio::test]\nasync fn test_direct_sink_writer() -> Result<(), Error> {\n // We define a framed writer which accepts byte slices\n // and 'reverse' this construction immediately.\n let framed_byte_lc = FramedWrite::new(Vec::new(), SliceEncoder::new());\n let mut writer = SinkWriter::new(framed_byte_lc);\n\n // Write multiple slices to the sink...\n let _ = writer.write(&[1, 2, 3]).await;\n let _ = writer.write(&[4, 5, 6]).await;\n\n // ... and compare it with the buffer.\n assert_eq!(\n writer.into_inner().write_buffer().to_vec().as_slice(),\n &[1, 2, 3, 4, 5, 6]\n );\n\n Ok(())\n}\n" }, { "path": "tokio-util/tests/io_stream_reader.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse bytes::Bytes;\nuse tokio::io::AsyncReadExt;\nuse tokio_stream::iter;\nuse tokio_util::io::StreamReader;\n\n#[tokio::test]\nasync fn test_stream_reader() -> std::io::Result<()> {\n let stream = iter(vec![\n std::io::Result::Ok(Bytes::from_static(&[])),\n Ok(Bytes::from_static(&[0, 1, 2, 3])),\n Ok(Bytes::from_static(&[])),\n Ok(Bytes::from_static(&[4, 5, 6, 7])),\n Ok(Bytes::from_static(&[])),\n Ok(Bytes::from_static(&[8, 9, 10, 11])),\n Ok(Bytes::from_static(&[])),\n ]);\n\n let mut read = StreamReader::new(stream);\n\n let mut buf = [0; 5];\n read.read_exact(&mut buf).await?;\n assert_eq!(buf, [0, 1, 2, 3, 4]);\n\n assert_eq!(read.read(&mut buf).await?, 3);\n assert_eq!(&buf[..3], [5, 6, 7]);\n\n assert_eq!(read.read(&mut buf).await?, 4);\n assert_eq!(&buf[..4], [8, 9, 10, 11]);\n\n assert_eq!(read.read(&mut buf).await?, 0);\n\n Ok(())\n}\n" }, { "path": "tokio-util/tests/io_sync_bridge.rs", "content": "#![cfg(feature = \"io-util\")]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n\nuse std::error::Error;\nuse std::io::{Cursor, Read, Result as IoResult, Write};\nuse tokio::io::{AsyncRead, AsyncReadExt};\nuse tokio_util::io::SyncIoBridge;\n\nasync fn test_reader_len(\n r: impl AsyncRead + Unpin + Send + 'static,\n expected_len: usize,\n) -> IoResult<()> {\n let mut r = SyncIoBridge::new(r);\n let res = tokio::task::spawn_blocking(move || {\n let mut buf = Vec::new();\n r.read_to_end(&mut buf)?;\n Ok::<_, std::io::Error>(buf)\n })\n .await?;\n assert_eq!(res?.len(), expected_len);\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_async_read_to_sync() -> Result<(), Box> {\n test_reader_len(tokio::io::empty(), 0).await?;\n let buf = b\"hello world\";\n test_reader_len(Cursor::new(buf), buf.len()).await?;\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_async_write_to_sync() -> Result<(), Box> {\n let mut dest = Vec::new();\n let src = b\"hello world\";\n let dest = tokio::task::spawn_blocking(move || -> Result<_, String> {\n let mut w = SyncIoBridge::new(Cursor::new(&mut dest));\n std::io::copy(&mut Cursor::new(src), &mut w).map_err(|e| e.to_string())?;\n Ok(dest)\n })\n .await??;\n assert_eq!(dest.as_slice(), src);\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_into_inner() -> Result<(), Box> {\n let mut buf = Vec::new();\n SyncIoBridge::new(tokio::io::empty())\n .into_inner()\n .read_to_end(&mut buf)\n .await\n .unwrap();\n assert_eq!(buf.len(), 0);\n Ok(())\n}\n\n#[tokio::test]\nasync fn test_shutdown() -> Result<(), Box> {\n let (s1, mut s2) = tokio::io::duplex(1024);\n let (_rh, wh) = tokio::io::split(s1);\n tokio::task::spawn_blocking(move || -> std::io::Result<_> {\n let mut wh = SyncIoBridge::new(wh);\n wh.write_all(b\"hello\")?;\n wh.shutdown()?;\n assert!(wh.write_all(b\" world\").is_err());\n Ok(())\n })\n .await??;\n let mut buf = vec![];\n s2.read_to_end(&mut buf).await?;\n assert_eq!(buf, b\"hello\");\n Ok(())\n}\n" }, { "path": "tokio-util/tests/io_write_all_vectored.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse tokio::io::AsyncWrite;\nuse tokio_util::io::write_all_vectored;\n\nuse bytes::BytesMut;\nuse std::io;\nuse std::io::IoSlice;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\n#[tokio::test]\nasync fn test_write_all_vectored() {\n struct Wr {\n buf: BytesMut,\n }\n impl AsyncWrite for Wr {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n // When executing `write_all_buf` with this writer,\n // `poll_write` is not called.\n panic!(\"shouldn't be called\")\n }\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n for buf in bufs {\n self.buf.extend_from_slice(buf);\n }\n let n = self.buf.len();\n Ok(n).into()\n }\n fn is_write_vectored(&self) -> bool {\n // Enable vectored write. (doesn't need to be enabled explicitly for `write_all_vectored`)\n true\n }\n }\n\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n\n let buf = &mut [\n IoSlice::new(&b\"hello\"[..]),\n IoSlice::new(&b\" \"[..]),\n IoSlice::new(&b\"world\"[..]),\n ];\n\n write_all_vectored(&mut wr, buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"hello world\");\n}\n\n#[tokio::test]\nasync fn write_all_vectored_with_empty_slice() {\n struct Wr {\n buf: BytesMut,\n }\n impl AsyncWrite for Wr {\n fn poll_write(\n self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n _buf: &[u8],\n ) -> Poll> {\n panic!(\"shouldn't be called\")\n }\n fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Ok(()).into()\n }\n fn poll_write_vectored(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n bufs: &[io::IoSlice<'_>],\n ) -> Poll> {\n for buf in bufs {\n self.buf.extend_from_slice(buf);\n }\n let n = self.buf.len();\n Ok(n).into()\n }\n fn is_write_vectored(&self) -> bool {\n // Enable vectored write.\n true\n }\n }\n\n // case 1 middle empty slice\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n\n let buf = &mut [\n IoSlice::new(&b\"hello\"[..]),\n IoSlice::new(&[]),\n IoSlice::new(&b\"world\"[..]),\n ];\n\n write_all_vectored(&mut wr, buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"helloworld\");\n\n // case 2 no slices\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n\n let buf = &mut [];\n\n write_all_vectored(&mut wr, buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"\");\n\n // case 3 just an empty slice\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n let buf = &mut [IoSlice::new(&[])];\n\n write_all_vectored(&mut wr, buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"\");\n\n // case 4 ending with empty slice\n let mut wr = Wr {\n buf: BytesMut::with_capacity(64),\n };\n let buf = &mut [IoSlice::new(b\"hello\"), IoSlice::new(&[])];\n\n write_all_vectored(&mut wr, buf).await.unwrap();\n assert_eq!(&wr.buf[..], b\"hello\");\n}\n" }, { "path": "tokio-util/tests/length_delimited.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::io::{AsyncRead, AsyncWrite, ReadBuf};\nuse tokio_test::task;\nuse tokio_test::{\n assert_err, assert_ok, assert_pending, assert_ready, assert_ready_err, assert_ready_ok,\n};\nuse tokio_util::codec::*;\n\nuse bytes::{BufMut, Bytes, BytesMut};\nuse futures::{pin_mut, Sink, Stream};\nuse std::collections::VecDeque;\nuse std::io;\nuse std::pin::Pin;\nuse std::task::{Context, Poll};\n\nmacro_rules! mock {\n ($($x:expr,)*) => {{\n let mut v = VecDeque::new();\n v.extend(vec![$($x),*]);\n Mock { calls: v }\n }};\n}\n\nmacro_rules! assert_next_eq {\n ($io:ident, $expect:expr) => {{\n task::spawn(()).enter(|cx, _| {\n let res = assert_ready!($io.as_mut().poll_next(cx));\n match res {\n Some(Ok(v)) => assert_eq!(v, $expect.as_ref()),\n Some(Err(e)) => panic!(\"error = {:?}\", e),\n None => panic!(\"none\"),\n }\n });\n }};\n}\n\nmacro_rules! assert_next_pending {\n ($io:ident) => {{\n task::spawn(()).enter(|cx, _| match $io.as_mut().poll_next(cx) {\n Poll::Ready(Some(Ok(v))) => panic!(\"value = {:?}\", v),\n Poll::Ready(Some(Err(e))) => panic!(\"error = {:?}\", e),\n Poll::Ready(None) => panic!(\"done\"),\n Poll::Pending => {}\n });\n }};\n}\n\nmacro_rules! assert_next_err {\n ($io:ident) => {{\n task::spawn(()).enter(|cx, _| match $io.as_mut().poll_next(cx) {\n Poll::Ready(Some(Ok(v))) => panic!(\"value = {:?}\", v),\n Poll::Ready(Some(Err(_))) => {}\n Poll::Ready(None) => panic!(\"done\"),\n Poll::Pending => panic!(\"pending\"),\n });\n }};\n}\n\nmacro_rules! assert_done {\n ($io:ident) => {{\n task::spawn(()).enter(|cx, _| {\n let res = assert_ready!($io.as_mut().poll_next(cx));\n match res {\n Some(Ok(v)) => panic!(\"value = {:?}\", v),\n Some(Err(e)) => panic!(\"error = {:?}\", e),\n None => {}\n }\n });\n }};\n}\n\n#[test]\nfn read_empty_io_yields_nothing() {\n let io = Box::pin(FramedRead::new(mock!(), LengthDelimitedCodec::new()));\n pin_mut!(io);\n\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_one_packet() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\\x00\\x09abcdefghi\"),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_one_packet_little_endian() {\n let io = length_delimited::Builder::new()\n .little_endian()\n .new_read(mock! {\n data(b\"\\x09\\x00\\x00\\x00abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_one_packet_native_endian() {\n let d = if cfg!(target_endian = \"big\") {\n b\"\\x00\\x00\\x00\\x09abcdefghi\"\n } else {\n b\"\\x09\\x00\\x00\\x00abcdefghi\"\n };\n let io = length_delimited::Builder::new()\n .native_endian()\n .new_read(mock! {\n data(d),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_multi_frame_one_packet() {\n let mut d: Vec = vec![];\n d.extend_from_slice(b\"\\x00\\x00\\x00\\x09abcdefghi\");\n d.extend_from_slice(b\"\\x00\\x00\\x00\\x03123\");\n d.extend_from_slice(b\"\\x00\\x00\\x00\\x0bhello world\");\n\n let io = FramedRead::new(\n mock! {\n data(&d),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_eq!(io, b\"123\");\n assert_next_eq!(io, b\"hello world\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_multi_packet() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\"),\n data(b\"\\x00\\x09abc\"),\n data(b\"defghi\"),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_multi_frame_multi_packet() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\"),\n data(b\"\\x00\\x09abc\"),\n data(b\"defghi\"),\n data(b\"\\x00\\x00\\x00\\x0312\"),\n data(b\"3\\x00\\x00\\x00\\x0bhello world\"),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_eq!(io, b\"123\");\n assert_next_eq!(io, b\"hello world\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_multi_packet_wait() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\"),\n Poll::Pending,\n data(b\"\\x00\\x09abc\"),\n Poll::Pending,\n data(b\"defghi\"),\n Poll::Pending,\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_pending!(io);\n assert_next_pending!(io);\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_pending!(io);\n assert_done!(io);\n}\n\n#[test]\nfn read_multi_frame_multi_packet_wait() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\"),\n Poll::Pending,\n data(b\"\\x00\\x09abc\"),\n Poll::Pending,\n data(b\"defghi\"),\n Poll::Pending,\n data(b\"\\x00\\x00\\x00\\x0312\"),\n Poll::Pending,\n data(b\"3\\x00\\x00\\x00\\x0bhello world\"),\n Poll::Pending,\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_pending!(io);\n assert_next_pending!(io);\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_pending!(io);\n assert_next_pending!(io);\n assert_next_eq!(io, b\"123\");\n assert_next_eq!(io, b\"hello world\");\n assert_next_pending!(io);\n assert_done!(io);\n}\n\n#[test]\nfn read_incomplete_head() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\"),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_err!(io);\n}\n\n#[test]\nfn read_incomplete_head_multi() {\n let io = FramedRead::new(\n mock! {\n Poll::Pending,\n data(b\"\\x00\"),\n Poll::Pending,\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_pending!(io);\n assert_next_pending!(io);\n assert_next_err!(io);\n}\n\n#[test]\nfn read_incomplete_payload() {\n let io = FramedRead::new(\n mock! {\n data(b\"\\x00\\x00\\x00\\x09ab\"),\n Poll::Pending,\n data(b\"cd\"),\n Poll::Pending,\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n assert_next_pending!(io);\n assert_next_pending!(io);\n assert_next_err!(io);\n}\n\n#[test]\nfn read_max_frame_len() {\n let io = length_delimited::Builder::new()\n .max_frame_length(5)\n .new_read(mock! {\n data(b\"\\x00\\x00\\x00\\x09abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_err!(io);\n}\n\n#[test]\nfn read_update_max_frame_len_at_rest() {\n let io = length_delimited::Builder::new().new_read(mock! {\n data(b\"\\x00\\x00\\x00\\x09abcdefghi\"),\n data(b\"\\x00\\x00\\x00\\x09abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n io.decoder_mut().set_max_frame_length(5);\n assert_next_err!(io);\n}\n\n#[test]\nfn read_update_max_frame_len_in_flight() {\n let io = length_delimited::Builder::new().new_read(mock! {\n data(b\"\\x00\\x00\\x00\\x09abcd\"),\n Poll::Pending,\n data(b\"efghi\"),\n data(b\"\\x00\\x00\\x00\\x09abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_pending!(io);\n io.decoder_mut().set_max_frame_length(5);\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_err!(io);\n}\n\n#[test]\nfn read_one_byte_length_field() {\n let io = length_delimited::Builder::new()\n .length_field_length(1)\n .new_read(mock! {\n data(b\"\\x09abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_header_offset() {\n let io = length_delimited::Builder::new()\n .length_field_length(2)\n .length_field_offset(4)\n .new_read(mock! {\n data(b\"zzzz\\x00\\x09abcdefghi\"),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_multi_frame_one_packet_skip_none_adjusted() {\n let mut d: Vec = vec![];\n d.extend_from_slice(b\"xx\\x00\\x09abcdefghi\");\n d.extend_from_slice(b\"yy\\x00\\x03123\");\n d.extend_from_slice(b\"zz\\x00\\x0bhello world\");\n\n let io = length_delimited::Builder::new()\n .length_field_length(2)\n .length_field_offset(2)\n .num_skip(0)\n .length_adjustment(4)\n .new_read(mock! {\n data(&d),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"xx\\x00\\x09abcdefghi\");\n assert_next_eq!(io, b\"yy\\x00\\x03123\");\n assert_next_eq!(io, b\"zz\\x00\\x0bhello world\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_frame_length_adjusted() {\n let mut d: Vec = vec![];\n d.extend_from_slice(b\"\\x00\\x00\\x0b\\x0cHello world\");\n\n let io = length_delimited::Builder::new()\n .length_field_offset(0)\n .length_field_length(3)\n .length_adjustment(0)\n .num_skip(4)\n .new_read(mock! {\n data(&d),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"Hello world\");\n assert_done!(io);\n}\n\n#[test]\nfn read_single_multi_frame_one_packet_length_includes_head() {\n let mut d: Vec = vec![];\n d.extend_from_slice(b\"\\x00\\x0babcdefghi\");\n d.extend_from_slice(b\"\\x00\\x05123\");\n d.extend_from_slice(b\"\\x00\\x0dhello world\");\n\n let io = length_delimited::Builder::new()\n .length_field_length(2)\n .length_adjustment(-2)\n .new_read(mock! {\n data(&d),\n });\n pin_mut!(io);\n\n assert_next_eq!(io, b\"abcdefghi\");\n assert_next_eq!(io, b\"123\");\n assert_next_eq!(io, b\"hello world\");\n assert_done!(io);\n}\n\n#[test]\nfn write_single_frame_length_adjusted() {\n let io = length_delimited::Builder::new()\n .length_adjustment(-2)\n .new_write(mock! {\n data(b\"\\x00\\x00\\x00\\x0b\"),\n data(b\"abcdefghi\"),\n flush(),\n });\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_nothing_yields_nothing() {\n let io = FramedWrite::new(mock!(), LengthDelimitedCodec::new());\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.poll_flush(cx));\n });\n}\n\n#[test]\nfn write_single_frame_one_packet() {\n let io = FramedWrite::new(\n mock! {\n data(b\"\\x00\\x00\\x00\\x09\"),\n data(b\"abcdefghi\"),\n flush(),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_single_multi_frame_one_packet() {\n let io = FramedWrite::new(\n mock! {\n data(b\"\\x00\\x00\\x00\\x09\"),\n data(b\"abcdefghi\"),\n data(b\"\\x00\\x00\\x00\\x03\"),\n data(b\"123\"),\n data(b\"\\x00\\x00\\x00\\x0b\"),\n data(b\"hello world\"),\n flush(),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"123\")));\n\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"hello world\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_single_multi_frame_multi_packet() {\n let io = FramedWrite::new(\n mock! {\n data(b\"\\x00\\x00\\x00\\x09\"),\n data(b\"abcdefghi\"),\n flush(),\n data(b\"\\x00\\x00\\x00\\x03\"),\n data(b\"123\"),\n flush(),\n data(b\"\\x00\\x00\\x00\\x0b\"),\n data(b\"hello world\"),\n flush(),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"123\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"hello world\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_single_frame_would_block() {\n let io = FramedWrite::new(\n mock! {\n Poll::Pending,\n data(b\"\\x00\\x00\"),\n Poll::Pending,\n data(b\"\\x00\\x09\"),\n data(b\"abcdefghi\"),\n flush(),\n },\n LengthDelimitedCodec::new(),\n );\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n\n assert_pending!(io.as_mut().poll_flush(cx));\n assert_pending!(io.as_mut().poll_flush(cx));\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_single_frame_little_endian() {\n let io = length_delimited::Builder::new()\n .little_endian()\n .new_write(mock! {\n data(b\"\\x09\\x00\\x00\\x00\"),\n data(b\"abcdefghi\"),\n flush(),\n });\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_single_frame_with_short_length_field() {\n let io = length_delimited::Builder::new()\n .length_field_length(1)\n .new_write(mock! {\n data(b\"\\x09\"),\n data(b\"abcdefghi\"),\n flush(),\n });\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdefghi\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_max_frame_len() {\n let io = length_delimited::Builder::new()\n .max_frame_length(5)\n .new_write(mock! {});\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_err!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_update_max_frame_len_at_rest() {\n let io = length_delimited::Builder::new().new_write(mock! {\n data(b\"\\x00\\x00\\x00\\x06\"),\n data(b\"abcdef\"),\n flush(),\n });\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n io.encoder_mut().set_max_frame_length(5);\n\n assert_err!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_update_max_frame_len_in_flight() {\n let io = length_delimited::Builder::new().new_write(mock! {\n data(b\"\\x00\\x00\\x00\\x06\"),\n data(b\"ab\"),\n Poll::Pending,\n data(b\"cdef\"),\n flush(),\n });\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n\n assert_pending!(io.as_mut().poll_flush(cx));\n\n io.encoder_mut().set_max_frame_length(5);\n\n assert_ready_ok!(io.as_mut().poll_flush(cx));\n\n assert_err!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn write_zero() {\n let io = length_delimited::Builder::new().new_write(mock! {});\n pin_mut!(io);\n\n task::spawn(()).enter(|cx, _| {\n assert_ready_ok!(io.as_mut().poll_ready(cx));\n assert_ok!(io.as_mut().start_send(Bytes::from(\"abcdef\")));\n\n assert_ready_err!(io.as_mut().poll_flush(cx));\n\n assert!(io.get_ref().calls.is_empty());\n });\n}\n\n#[test]\nfn encode_overflow() {\n // Test reproducing tokio-rs/tokio#681.\n let mut codec = length_delimited::Builder::new().new_codec();\n let mut buf = BytesMut::with_capacity(1024);\n\n // Put some data into the buffer without resizing it to hold more.\n let some_as = std::iter::repeat(b'a').take(1024).collect::>();\n buf.put_slice(&some_as[..]);\n\n // Trying to encode the length header should resize the buffer if it won't fit.\n codec.encode(Bytes::from(\"hello\"), &mut buf).unwrap();\n}\n\n#[test]\nfn frame_does_not_fit() {\n let codec = LengthDelimitedCodec::builder()\n .length_field_length(1)\n .max_frame_length(256)\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), 255);\n}\n\n#[test]\nfn neg_adjusted_frame_does_not_fit() {\n let codec = LengthDelimitedCodec::builder()\n .length_field_length(1)\n .length_adjustment(-1)\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), 254);\n}\n\n#[test]\nfn pos_adjusted_frame_does_not_fit() {\n let codec = LengthDelimitedCodec::builder()\n .length_field_length(1)\n .length_adjustment(1)\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), 256);\n}\n\n#[test]\nfn max_allowed_frame_fits() {\n let codec = LengthDelimitedCodec::builder()\n .length_field_length(std::mem::size_of::())\n .max_frame_length(usize::MAX)\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), usize::MAX);\n}\n\n#[test]\nfn smaller_frame_len_not_adjusted() {\n let codec = LengthDelimitedCodec::builder()\n .max_frame_length(10)\n .length_field_length(std::mem::size_of::())\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), 10);\n}\n\n#[test]\nfn max_allowed_length_field() {\n let codec = LengthDelimitedCodec::builder()\n .length_field_length(8)\n .max_frame_length(usize::MAX)\n .new_codec();\n\n assert_eq!(codec.max_frame_length(), usize::MAX);\n}\n\n// ===== Test utils =====\n\nstruct Mock {\n calls: VecDeque>>,\n}\n\nenum Op {\n Data(Vec),\n Flush,\n}\n\nimpl AsyncRead for Mock {\n fn poll_read(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n dst: &mut ReadBuf<'_>,\n ) -> Poll> {\n match self.calls.pop_front() {\n Some(Poll::Ready(Ok(Op::Data(data)))) => {\n debug_assert!(dst.remaining() >= data.len());\n dst.put_slice(&data);\n Poll::Ready(Ok(()))\n }\n Some(Poll::Ready(Ok(_))) => panic!(),\n Some(Poll::Ready(Err(e))) => Poll::Ready(Err(e)),\n Some(Poll::Pending) => Poll::Pending,\n None => Poll::Ready(Ok(())),\n }\n }\n}\n\nimpl AsyncWrite for Mock {\n fn poll_write(\n mut self: Pin<&mut Self>,\n _cx: &mut Context<'_>,\n src: &[u8],\n ) -> Poll> {\n match self.calls.pop_front() {\n Some(Poll::Ready(Ok(Op::Data(data)))) => {\n let len = data.len();\n assert!(src.len() >= len, \"expect={data:?}; actual={src:?}\");\n assert_eq!(&data[..], &src[..len]);\n Poll::Ready(Ok(len))\n }\n Some(Poll::Ready(Ok(_))) => panic!(),\n Some(Poll::Ready(Err(e))) => Poll::Ready(Err(e)),\n Some(Poll::Pending) => Poll::Pending,\n None => Poll::Ready(Ok(0)),\n }\n }\n\n fn poll_flush(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n match self.calls.pop_front() {\n Some(Poll::Ready(Ok(Op::Flush))) => Poll::Ready(Ok(())),\n Some(Poll::Ready(Ok(_))) => panic!(),\n Some(Poll::Ready(Err(e))) => Poll::Ready(Err(e)),\n Some(Poll::Pending) => Poll::Pending,\n None => Poll::Ready(Ok(())),\n }\n }\n\n fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll> {\n Poll::Ready(Ok(()))\n }\n}\n\nimpl<'a> From<&'a [u8]> for Op {\n fn from(src: &'a [u8]) -> Op {\n Op::Data(src.into())\n }\n}\n\nimpl From> for Op {\n fn from(src: Vec) -> Op {\n Op::Data(src)\n }\n}\n\nfn data(bytes: &[u8]) -> Poll> {\n Poll::Ready(Ok(bytes.into()))\n}\n\nfn flush() -> Poll> {\n Poll::Ready(Ok(Op::Flush))\n}\n" }, { "path": "tokio-util/tests/mpsc.rs", "content": "use futures::sink::SinkExt;\nuse std::future::poll_fn;\nuse tokio::sync::mpsc::channel;\nuse tokio_test::task::spawn;\nuse tokio_test::{\n assert_ok, assert_pending, assert_ready, assert_ready_eq, assert_ready_err, assert_ready_ok,\n};\nuse tokio_util::sync::PollSender;\n\n#[tokio::test]\nasync fn simple() {\n let (send, mut recv) = channel(3);\n let mut send = PollSender::new(send);\n\n for i in 1..=3i32 {\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(i).unwrap();\n }\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n\n assert_eq!(recv.recv().await.unwrap(), 1);\n assert!(reserve.is_woken());\n assert_ready_ok!(reserve.poll());\n\n drop(recv);\n\n send.send_item(42).unwrap();\n}\n\n#[tokio::test]\nasync fn simple_ref() {\n let v = [1, 2, 3i32];\n\n let (send, mut recv) = channel(3);\n let mut send = PollSender::new(send);\n\n for vi in v.iter() {\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(vi).unwrap();\n }\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n\n assert_eq!(*recv.recv().await.unwrap(), 1);\n assert!(reserve.is_woken());\n assert_ready_ok!(reserve.poll());\n drop(recv);\n send.send_item(&42).unwrap();\n}\n\n#[tokio::test]\nasync fn repeated_poll_reserve() {\n let (send, mut recv) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n assert_ready_ok!(reserve.poll());\n send.send_item(1).unwrap();\n\n assert_eq!(recv.recv().await.unwrap(), 1);\n}\n\n#[tokio::test]\nasync fn abort_send() {\n let (send, mut recv) = channel(3);\n let mut send = PollSender::new(send);\n let send2 = send.get_ref().cloned().unwrap();\n\n for i in 1..=3i32 {\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(i).unwrap();\n }\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n assert_eq!(recv.recv().await.unwrap(), 1);\n assert!(reserve.is_woken());\n assert_ready_ok!(reserve.poll());\n\n let mut send2_send = spawn(send2.send(5));\n assert_pending!(send2_send.poll());\n assert!(send.abort_send());\n assert!(send2_send.is_woken());\n assert_ready_ok!(send2_send.poll());\n\n assert_eq!(recv.recv().await.unwrap(), 2);\n assert_eq!(recv.recv().await.unwrap(), 3);\n assert_eq!(recv.recv().await.unwrap(), 5);\n}\n\n#[tokio::test]\nasync fn close_sender_last() {\n let (send, mut recv) = channel::(3);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n send.close();\n\n assert!(recv_task.is_woken());\n assert!(assert_ready!(recv_task.poll()).is_none());\n}\n\n#[tokio::test]\nasync fn close_sender_not_last() {\n let (send, mut recv) = channel::(3);\n let mut send = PollSender::new(send);\n let send2 = send.get_ref().cloned().unwrap();\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n send.close();\n\n assert!(!recv_task.is_woken());\n assert_pending!(recv_task.poll());\n\n drop(send2);\n\n assert!(recv_task.is_woken());\n assert!(assert_ready!(recv_task.poll()).is_none());\n}\n\n#[tokio::test]\nasync fn close_sender_before_reserve() {\n let (send, mut recv) = channel::(3);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n send.close();\n\n assert!(recv_task.is_woken());\n assert!(assert_ready!(recv_task.poll()).is_none());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_err!(reserve.poll());\n}\n\n#[tokio::test]\nasync fn close_sender_after_pending_reserve() {\n let (send, mut recv) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(1).unwrap();\n\n assert!(recv_task.is_woken());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n drop(reserve);\n\n send.close();\n\n assert!(send.is_closed());\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_err!(reserve.poll());\n}\n\n#[tokio::test]\nasync fn close_sender_after_successful_reserve() {\n let (send, mut recv) = channel::(3);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n drop(reserve);\n\n send.close();\n assert!(send.is_closed());\n assert!(!recv_task.is_woken());\n assert_pending!(recv_task.poll());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n}\n\n#[tokio::test]\nasync fn abort_send_after_pending_reserve() {\n let (send, mut recv) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(1).unwrap();\n\n assert_eq!(send.get_ref().unwrap().capacity(), 0);\n assert!(!send.abort_send());\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n\n assert!(send.abort_send());\n assert_eq!(send.get_ref().unwrap().capacity(), 0);\n}\n\n#[tokio::test]\nasync fn abort_send_after_successful_reserve() {\n let (send, mut recv) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n assert_eq!(send.get_ref().unwrap().capacity(), 1);\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n assert_eq!(send.get_ref().unwrap().capacity(), 0);\n\n assert!(send.abort_send());\n assert_eq!(send.get_ref().unwrap().capacity(), 1);\n}\n\n#[tokio::test]\nasync fn closed_when_receiver_drops() {\n let (send, _) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_err!(reserve.poll());\n}\n\n#[should_panic]\n#[test]\nfn start_send_panics_when_idle() {\n let (send, _) = channel::(3);\n let mut send = PollSender::new(send);\n\n send.send_item(1).unwrap();\n}\n\n#[should_panic]\n#[test]\nfn start_send_panics_when_acquiring() {\n let (send, _) = channel::(1);\n let mut send = PollSender::new(send);\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_ready_ok!(reserve.poll());\n send.send_item(1).unwrap();\n\n let mut reserve = spawn(poll_fn(|cx| send.poll_reserve(cx)));\n assert_pending!(reserve.poll());\n send.send_item(2).unwrap();\n}\n\n#[test]\nfn sink_send_then_flush() {\n let (send, mut recv) = channel(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_ready_ok!(ready.poll());\n assert_ok!(send.start_send_unpin(()));\n\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_pending!(ready.poll());\n\n let mut flush = spawn(poll_fn(|cx| send.poll_flush_unpin(cx)));\n assert_ready_ok!(flush.poll());\n\n // Flushing does not mean that the sender becomes ready.\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_pending!(ready.poll());\n\n assert_ready_eq!(recv_task.poll(), Some(()));\n assert!(ready.is_woken());\n assert_ready_ok!(ready.poll());\n}\n\n#[test]\nfn sink_send_then_close() {\n let (send, mut recv) = channel(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_ready_ok!(ready.poll());\n assert_ok!(send.start_send_unpin(1));\n\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_pending!(ready.poll());\n\n assert!(recv_task.is_woken());\n assert_ready_eq!(recv_task.poll(), Some(1));\n\n assert!(ready.is_woken());\n assert_ready_ok!(ready.poll());\n\n drop(recv_task);\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n assert_ok!(send.start_send_unpin(2));\n\n let mut close = spawn(poll_fn(|cx| send.poll_close_unpin(cx)));\n assert_ready_ok!(close.poll());\n\n assert!(recv_task.is_woken());\n assert_ready_eq!(recv_task.poll(), Some(2));\n\n drop(recv_task);\n let mut recv_task = spawn(recv.recv());\n assert_ready_eq!(recv_task.poll(), None);\n}\n\n#[test]\nfn sink_send_ref() {\n let data = \"data\".to_owned();\n let (send, mut recv) = channel(1);\n let mut send = PollSender::new(send);\n\n let mut recv_task = spawn(recv.recv());\n assert_pending!(recv_task.poll());\n\n let mut ready = spawn(poll_fn(|cx| send.poll_ready_unpin(cx)));\n assert_ready_ok!(ready.poll());\n\n assert_ok!(send.start_send_unpin(data.as_str()));\n\n let mut flush = spawn(poll_fn(|cx| send.poll_flush_unpin(cx)));\n assert_ready_ok!(flush.poll());\n\n assert_ready_eq!(recv_task.poll(), Some(\"data\"));\n}\n" }, { "path": "tokio-util/tests/panic.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(all(feature = \"full\", not(target_os = \"wasi\")))] // Wasi doesn't support panic recovery\n#![cfg(panic = \"unwind\")]\n\nuse parking_lot::{const_mutex, Mutex};\nuse std::error::Error;\nuse std::panic;\nuse std::sync::Arc;\nuse tokio::runtime::Runtime;\nuse tokio::sync::mpsc::channel;\nuse tokio::time::{Duration, Instant};\nuse tokio_test::task;\nuse tokio_util::io::SyncIoBridge;\nuse tokio_util::sync::PollSender;\nuse tokio_util::task::LocalPoolHandle;\nuse tokio_util::time::DelayQueue;\n\n// Taken from tokio-util::time::wheel, if that changes then\nconst MAX_DURATION_MS: u64 = (1 << (36)) - 1;\n\nfn test_panic(func: Func) -> Option {\n static PANIC_MUTEX: Mutex<()> = const_mutex(());\n\n {\n let _guard = PANIC_MUTEX.lock();\n let panic_file: Arc>> = Arc::new(Mutex::new(None));\n\n let prev_hook = panic::take_hook();\n {\n let panic_file = panic_file.clone();\n panic::set_hook(Box::new(move |panic_info| {\n let panic_location = panic_info.location().unwrap();\n panic_file\n .lock()\n .clone_from(&Some(panic_location.file().to_string()));\n }));\n }\n\n let result = panic::catch_unwind(func);\n // Return to the previously set panic hook (maybe default) so that we get nice error\n // messages in the tests.\n panic::set_hook(prev_hook);\n\n if result.is_err() {\n panic_file.lock().clone()\n } else {\n None\n }\n }\n}\n\n#[test]\nfn sync_bridge_new_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = SyncIoBridge::new(tokio::io::empty());\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn poll_sender_send_item_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let (send, _) = channel::(3);\n let mut send = PollSender::new(send);\n\n let _ = send.send_item(42);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_pool_handle_new_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let _ = LocalPoolHandle::new(0);\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn local_pool_handle_spawn_pinned_by_idx_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n\n rt.block_on(async {\n let handle = LocalPoolHandle::new(2);\n handle.spawn_pinned_by_idx(|| async { \"test\" }, 3);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n#[test]\nfn delay_queue_insert_at_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::with_capacity(3));\n\n //let st = std::time::Instant::from(SystemTime::UNIX_EPOCH);\n let _k = queue.insert_at(\n \"1\",\n Instant::now() + Duration::from_millis(MAX_DURATION_MS + 1),\n );\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn delay_queue_insert_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::with_capacity(3));\n\n let _k = queue.insert(\"1\", Duration::from_millis(MAX_DURATION_MS + 1));\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn delay_queue_remove_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::with_capacity(3));\n\n let key = queue.insert_at(\"1\", Instant::now());\n queue.remove(&key);\n queue.remove(&key);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn delay_queue_reset_at_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::with_capacity(3));\n\n let key = queue.insert_at(\"1\", Instant::now());\n queue.reset_at(\n &key,\n Instant::now() + Duration::from_millis(MAX_DURATION_MS + 1),\n );\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn delay_queue_reset_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::with_capacity(3));\n\n let key = queue.insert_at(\"1\", Instant::now());\n queue.reset(&key, Duration::from_millis(MAX_DURATION_MS + 1));\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn delay_queue_reserve_panic_caller() -> Result<(), Box> {\n let panic_location_file = test_panic(|| {\n let rt = basic();\n rt.block_on(async {\n let mut queue = task::spawn(DelayQueue::::with_capacity(3));\n\n queue.reserve((1 << 30) as usize);\n });\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\n#[test]\nfn future_ext_to_panic_caller() -> Result<(), Box> {\n use tokio::{sync::oneshot, time::Duration};\n use tokio_util::future::FutureExt;\n\n let panic_location_file = test_panic(|| {\n let (_tx, rx) = oneshot::channel::<()>();\n // this panics because there is no runtime available\n let _res = rx.timeout(Duration::from_millis(10));\n });\n\n // The panic location should be in this file\n assert_eq!(&panic_location_file.unwrap(), file!());\n\n Ok(())\n}\n\nfn basic() -> Runtime {\n tokio::runtime::Builder::new_current_thread()\n .enable_all()\n .build()\n .unwrap()\n}\n" }, { "path": "tokio-util/tests/poll_semaphore.rs", "content": "use std::future::Future;\nuse std::sync::Arc;\nuse std::task::Poll;\nuse tokio::sync::{OwnedSemaphorePermit, Semaphore};\nuse tokio_util::sync::PollSemaphore;\n\ntype SemRet = Option;\n\nfn semaphore_poll(\n sem: &mut PollSemaphore,\n) -> tokio_test::task::Spawn + '_> {\n let fut = std::future::poll_fn(move |cx| sem.poll_acquire(cx));\n tokio_test::task::spawn(fut)\n}\n\nfn semaphore_poll_many(\n sem: &mut PollSemaphore,\n permits: u32,\n) -> tokio_test::task::Spawn + '_> {\n let fut = std::future::poll_fn(move |cx| sem.poll_acquire_many(cx, permits));\n tokio_test::task::spawn(fut)\n}\n\n#[tokio::test]\nasync fn it_works() {\n let sem = Arc::new(Semaphore::new(1));\n let mut poll_sem = PollSemaphore::new(sem.clone());\n\n let permit = sem.acquire().await.unwrap();\n let mut poll = semaphore_poll(&mut poll_sem);\n assert!(poll.poll().is_pending());\n drop(permit);\n\n assert!(matches!(poll.poll(), Poll::Ready(Some(_))));\n drop(poll);\n\n sem.close();\n\n assert!(semaphore_poll(&mut poll_sem).await.is_none());\n\n // Check that it is fused.\n assert!(semaphore_poll(&mut poll_sem).await.is_none());\n assert!(semaphore_poll(&mut poll_sem).await.is_none());\n}\n\n#[tokio::test]\nasync fn can_acquire_many_permits() {\n let sem = Arc::new(Semaphore::new(4));\n let mut poll_sem = PollSemaphore::new(sem.clone());\n\n let permit1 = semaphore_poll(&mut poll_sem).poll();\n assert!(matches!(permit1, Poll::Ready(Some(_))));\n\n let permit2 = semaphore_poll_many(&mut poll_sem, 2).poll();\n assert!(matches!(permit2, Poll::Ready(Some(_))));\n\n assert_eq!(sem.available_permits(), 1);\n\n drop(permit2);\n\n let mut permit4 = semaphore_poll_many(&mut poll_sem, 4);\n assert!(permit4.poll().is_pending());\n\n drop(permit1);\n\n let permit4 = permit4.poll();\n assert!(matches!(permit4, Poll::Ready(Some(_))));\n assert_eq!(sem.available_permits(), 0);\n}\n\n#[tokio::test]\nasync fn can_poll_different_amounts_of_permits() {\n let sem = Arc::new(Semaphore::new(4));\n let mut poll_sem = PollSemaphore::new(sem.clone());\n assert!(semaphore_poll_many(&mut poll_sem, 5).poll().is_pending());\n assert!(semaphore_poll_many(&mut poll_sem, 4).poll().is_ready());\n\n let permit = sem.acquire_many(4).await.unwrap();\n assert!(semaphore_poll_many(&mut poll_sem, 5).poll().is_pending());\n assert!(semaphore_poll_many(&mut poll_sem, 4).poll().is_pending());\n drop(permit);\n assert!(semaphore_poll_many(&mut poll_sem, 5).poll().is_pending());\n assert!(semaphore_poll_many(&mut poll_sem, 4).poll().is_ready());\n}\n" }, { "path": "tokio-util/tests/reusable_box.rs", "content": "use futures::future::FutureExt;\nuse std::alloc::Layout;\nuse std::future::Future;\nuse std::marker::PhantomPinned;\nuse std::pin::Pin;\nuse std::rc::Rc;\nuse std::task::{Context, Poll};\nuse tokio_util::sync::ReusableBoxFuture;\n\n#[test]\n// Clippy false positive; it's useful to be able to test the trait impls for any lifetime\n#[allow(clippy::extra_unused_lifetimes)]\nfn traits<'a>() {\n fn assert_traits() {}\n // Use a type that is !Unpin\n assert_traits::>();\n // Use a type that is !Send + !Sync\n assert_traits::>>();\n}\n\n#[test]\nfn test_different_futures() {\n let fut = async move { 10 };\n // Not zero sized!\n assert_eq!(Layout::for_value(&fut).size(), 1);\n\n let mut b = ReusableBoxFuture::new(fut);\n\n assert_eq!(b.get_pin().now_or_never(), Some(10));\n\n b.try_set(async move { 20 })\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(20));\n\n b.try_set(async move { 30 })\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(30));\n}\n\n#[test]\nfn test_different_sizes() {\n let fut1 = async move { 10 };\n let val = [0u32; 1000];\n let fut2 = async move { val[0] };\n let fut3 = ZeroSizedFuture {};\n\n assert_eq!(Layout::for_value(&fut1).size(), 1);\n assert_eq!(Layout::for_value(&fut2).size(), 4004);\n assert_eq!(Layout::for_value(&fut3).size(), 0);\n\n let mut b = ReusableBoxFuture::new(fut1);\n assert_eq!(b.get_pin().now_or_never(), Some(10));\n b.set(fut2);\n assert_eq!(b.get_pin().now_or_never(), Some(0));\n b.set(fut3);\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n}\n\nstruct ZeroSizedFuture {}\nimpl Future for ZeroSizedFuture {\n type Output = u32;\n fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll {\n Poll::Ready(5)\n }\n}\n\n#[test]\nfn test_zero_sized() {\n let fut = ZeroSizedFuture {};\n // Zero sized!\n assert_eq!(Layout::for_value(&fut).size(), 0);\n\n let mut b = ReusableBoxFuture::new(fut);\n\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n\n b.try_set(ZeroSizedFuture {})\n .unwrap_or_else(|_| panic!(\"incorrect size\"));\n\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n assert_eq!(b.get_pin().now_or_never(), Some(5));\n}\n" }, { "path": "tokio-util/tests/spawn_pinned.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support threads\n\nuse std::rc::Rc;\nuse std::sync::Arc;\nuse tokio::sync::Barrier;\nuse tokio_util::task;\n\n/// Simple test of running a !Send future via spawn_pinned\n#[tokio::test]\nasync fn can_spawn_not_send_future() {\n let pool = task::LocalPoolHandle::new(1);\n\n let output = pool\n .spawn_pinned(|| {\n // Rc is !Send + !Sync\n let local_data = Rc::new(\"test\");\n\n // This future holds an Rc, so it is !Send\n async move { local_data.to_string() }\n })\n .await\n .unwrap();\n\n assert_eq!(output, \"test\");\n}\n\n/// Dropping the join handle still lets the task execute\n#[test]\nfn can_drop_future_and_still_get_output() {\n let pool = task::LocalPoolHandle::new(1);\n let (sender, receiver) = std::sync::mpsc::channel();\n\n pool.spawn_pinned(move || {\n // Rc is !Send + !Sync\n let local_data = Rc::new(\"test\");\n\n // This future holds an Rc, so it is !Send\n async move {\n let _ = sender.send(local_data.to_string());\n }\n });\n\n assert_eq!(receiver.recv(), Ok(\"test\".to_string()));\n}\n\n#[test]\n#[should_panic(expected = \"assertion failed: pool_size > 0\")]\nfn cannot_create_zero_sized_pool() {\n let _pool = task::LocalPoolHandle::new(0);\n}\n\n/// We should be able to spawn multiple futures onto the pool at the same time.\n#[tokio::test]\nasync fn can_spawn_multiple_futures() {\n let pool = task::LocalPoolHandle::new(2);\n\n let join_handle1 = pool.spawn_pinned(|| {\n let local_data = Rc::new(\"test1\");\n async move { local_data.to_string() }\n });\n let join_handle2 = pool.spawn_pinned(|| {\n let local_data = Rc::new(\"test2\");\n async move { local_data.to_string() }\n });\n\n assert_eq!(join_handle1.await.unwrap(), \"test1\");\n assert_eq!(join_handle2.await.unwrap(), \"test2\");\n}\n\n/// A panic in the spawned task causes the join handle to return an error.\n/// But, you can continue to spawn tasks.\n#[tokio::test]\n#[cfg(panic = \"unwind\")]\nasync fn task_panic_propagates() {\n let pool = task::LocalPoolHandle::new(1);\n\n let join_handle = pool.spawn_pinned(|| async {\n panic!(\"Test panic\");\n });\n\n let result = join_handle.await;\n assert!(result.is_err());\n let error = result.unwrap_err();\n assert!(error.is_panic());\n let panic_str = error.into_panic().downcast::<&'static str>().unwrap();\n assert_eq!(*panic_str, \"Test panic\");\n\n // Trying again with a \"safe\" task still works\n let join_handle = pool.spawn_pinned(|| async { \"test\" });\n let result = join_handle.await;\n assert!(result.is_ok());\n assert_eq!(result.unwrap(), \"test\");\n}\n\n/// A panic during task creation causes the join handle to return an error.\n/// But, you can continue to spawn tasks.\n#[tokio::test]\n#[cfg(panic = \"unwind\")]\nasync fn callback_panic_does_not_kill_worker() {\n let pool = task::LocalPoolHandle::new(1);\n\n let join_handle = pool.spawn_pinned(|| {\n panic!(\"Test panic\");\n #[allow(unreachable_code)]\n async {}\n });\n\n let result = join_handle.await;\n assert!(result.is_err());\n let error = result.unwrap_err();\n assert!(error.is_panic());\n let panic_str = error.into_panic().downcast::<&'static str>().unwrap();\n assert_eq!(*panic_str, \"Test panic\");\n\n // Trying again with a \"safe\" callback works\n let join_handle = pool.spawn_pinned(|| async { \"test\" });\n let result = join_handle.await;\n assert!(result.is_ok());\n assert_eq!(result.unwrap(), \"test\");\n}\n\n/// Canceling the task via the returned join handle cancels the spawned task\n/// (which has a different, internal join handle).\n#[tokio::test]\nasync fn task_cancellation_propagates() {\n let pool = task::LocalPoolHandle::new(1);\n let notify_dropped = Arc::new(());\n let weak_notify_dropped = Arc::downgrade(¬ify_dropped);\n\n let (start_sender, start_receiver) = tokio::sync::oneshot::channel();\n let (drop_sender, drop_receiver) = tokio::sync::oneshot::channel::<()>();\n let join_handle = pool.spawn_pinned(|| async move {\n let _drop_sender = drop_sender;\n // Move the Arc into the task\n let _notify_dropped = notify_dropped;\n let _ = start_sender.send(());\n\n // Keep the task running until it gets aborted\n futures::future::pending::<()>().await;\n });\n\n // Wait for the task to start\n let _ = start_receiver.await;\n\n join_handle.abort();\n\n // Wait for the inner task to abort, dropping the sender.\n // The top level join handle aborts quicker than the inner task (the abort\n // needs to propagate and get processed on the worker thread), so we can't\n // just await the top level join handle.\n let _ = drop_receiver.await;\n\n // Check that the Arc has been dropped. This verifies that the inner task\n // was canceled as well.\n assert!(weak_notify_dropped.upgrade().is_none());\n}\n\n/// Tasks should be given to the least burdened worker. When spawning two tasks\n/// on a pool with two empty workers the tasks should be spawned on separate\n/// workers.\n#[tokio::test]\nasync fn tasks_are_balanced() {\n let pool = task::LocalPoolHandle::new(2);\n\n // Spawn a task so one thread has a task count of 1\n let (start_sender1, start_receiver1) = tokio::sync::oneshot::channel();\n let (end_sender1, end_receiver1) = tokio::sync::oneshot::channel();\n let join_handle1 = pool.spawn_pinned(|| async move {\n let _ = start_sender1.send(());\n let _ = end_receiver1.await;\n std::thread::current().id()\n });\n\n // Wait for the first task to start up\n let _ = start_receiver1.await;\n\n // This task should be spawned on the other thread\n let (start_sender2, start_receiver2) = tokio::sync::oneshot::channel();\n let join_handle2 = pool.spawn_pinned(|| async move {\n let _ = start_sender2.send(());\n std::thread::current().id()\n });\n\n // Wait for the second task to start up\n let _ = start_receiver2.await;\n\n // Allow the first task to end\n let _ = end_sender1.send(());\n\n let thread_id1 = join_handle1.await.unwrap();\n let thread_id2 = join_handle2.await.unwrap();\n\n // Since the first task was active when the second task spawned, they should\n // be on separate workers/threads.\n assert_ne!(thread_id1, thread_id2);\n}\n\n#[tokio::test]\nasync fn spawn_by_idx() {\n let pool = task::LocalPoolHandle::new(3);\n let barrier = Arc::new(Barrier::new(4));\n let barrier1 = barrier.clone();\n let barrier2 = barrier.clone();\n let barrier3 = barrier.clone();\n\n let handle1 = pool.spawn_pinned_by_idx(\n || async move {\n barrier1.wait().await;\n std::thread::current().id()\n },\n 0,\n );\n pool.spawn_pinned_by_idx(\n || async move {\n barrier2.wait().await;\n std::thread::current().id()\n },\n 0,\n );\n let handle2 = pool.spawn_pinned_by_idx(\n || async move {\n barrier3.wait().await;\n std::thread::current().id()\n },\n 1,\n );\n\n let loads = pool.get_task_loads_for_each_worker();\n barrier.wait().await;\n assert_eq!(loads[0], 2);\n assert_eq!(loads[1], 1);\n assert_eq!(loads[2], 0);\n\n let thread_id1 = handle1.await.unwrap();\n let thread_id2 = handle2.await.unwrap();\n\n assert_ne!(thread_id1, thread_id2);\n}\n" }, { "path": "tokio-util/tests/sync_cancellation_token.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::pin;\nuse tokio::sync::oneshot;\nuse tokio_util::sync::{CancellationToken, WaitForCancellationFuture};\n\nuse core::future::Future;\nuse core::task::{Context, Poll};\nuse futures_test::task::new_count_waker;\n\n#[test]\nfn cancel_token() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n assert!(!token.is_cancelled());\n\n let wait_fut = token.cancelled();\n pin!(wait_fut);\n\n assert_eq!(\n Poll::Pending,\n wait_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n let wait_fut_2 = token.cancelled();\n pin!(wait_fut_2);\n\n token.cancel();\n assert_eq!(wake_counter, 1);\n assert!(token.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n wait_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n wait_fut_2.as_mut().poll(&mut Context::from_waker(&waker))\n );\n}\n\n#[test]\nfn cancel_token_owned() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n assert!(!token.is_cancelled());\n\n let wait_fut = token.clone().cancelled_owned();\n pin!(wait_fut);\n\n assert_eq!(\n Poll::Pending,\n wait_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n let wait_fut_2 = token.clone().cancelled_owned();\n pin!(wait_fut_2);\n\n token.cancel();\n assert_eq!(wake_counter, 1);\n assert!(token.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n wait_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n wait_fut_2.as_mut().poll(&mut Context::from_waker(&waker))\n );\n}\n\n#[test]\nfn cancel_token_owned_drop_test() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n\n let future = token.cancelled_owned();\n pin!(future);\n\n assert_eq!(\n Poll::Pending,\n future.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n // let future be dropped while pinned and under pending state to\n // find potential memory related bugs.\n}\n\n#[test]\nfn cancel_child_token_through_parent() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n\n let child_token = token.child_token();\n assert!(!child_token.is_cancelled());\n\n let child_fut = child_token.cancelled();\n pin!(child_fut);\n let parent_fut = token.cancelled();\n pin!(parent_fut);\n\n assert_eq!(\n Poll::Pending,\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n token.cancel();\n assert_eq!(wake_counter, 2);\n assert!(token.is_cancelled());\n assert!(child_token.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n}\n\n#[test]\nfn cancel_grandchild_token_through_parent_if_child_was_dropped() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n\n let intermediate_token = token.child_token();\n let child_token = intermediate_token.child_token();\n drop(intermediate_token);\n assert!(!child_token.is_cancelled());\n\n let child_fut = child_token.cancelled();\n pin!(child_fut);\n let parent_fut = token.cancelled();\n pin!(parent_fut);\n\n assert_eq!(\n Poll::Pending,\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n token.cancel();\n assert_eq!(wake_counter, 2);\n assert!(token.is_cancelled());\n assert!(child_token.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n}\n\n#[test]\nfn cancel_child_token_without_parent() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n\n let child_token_1 = token.child_token();\n\n let child_fut = child_token_1.cancelled();\n pin!(child_fut);\n let parent_fut = token.cancelled();\n pin!(parent_fut);\n\n assert_eq!(\n Poll::Pending,\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n child_token_1.cancel();\n assert_eq!(wake_counter, 1);\n assert!(!token.is_cancelled());\n assert!(child_token_1.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n let child_token_2 = token.child_token();\n let child_fut_2 = child_token_2.cancelled();\n pin!(child_fut_2);\n\n assert_eq!(\n Poll::Pending,\n child_fut_2.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n token.cancel();\n assert_eq!(wake_counter, 3);\n assert!(token.is_cancelled());\n assert!(child_token_2.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n child_fut_2.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n}\n\n#[test]\nfn create_child_token_after_parent_was_cancelled() {\n for drop_child_first in [true, false].iter().cloned() {\n let (waker, wake_counter) = new_count_waker();\n let token = CancellationToken::new();\n token.cancel();\n\n let child_token = token.child_token();\n assert!(child_token.is_cancelled());\n\n {\n let child_fut = child_token.cancelled();\n pin!(child_fut);\n let parent_fut = token.cancelled();\n pin!(parent_fut);\n\n assert_eq!(\n Poll::Ready(()),\n child_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n }\n\n if drop_child_first {\n drop(child_token);\n drop(token);\n } else {\n drop(token);\n drop(child_token);\n }\n }\n}\n\n#[test]\nfn drop_multiple_child_tokens() {\n for drop_first_child_first in &[true, false] {\n let token = CancellationToken::new();\n let mut child_tokens = [None, None, None];\n for child in &mut child_tokens {\n *child = Some(token.child_token());\n }\n\n assert!(!token.is_cancelled());\n assert!(!child_tokens[0].as_ref().unwrap().is_cancelled());\n\n for i in 0..child_tokens.len() {\n if *drop_first_child_first {\n child_tokens[i] = None;\n } else {\n child_tokens[child_tokens.len() - 1 - i] = None;\n }\n assert!(!token.is_cancelled());\n }\n\n drop(token);\n }\n}\n\n#[test]\nfn cancel_only_all_descendants() {\n // ARRANGE\n let (waker, wake_counter) = new_count_waker();\n\n let parent_token = CancellationToken::new();\n let token = parent_token.child_token();\n let sibling_token = parent_token.child_token();\n let child1_token = token.child_token();\n let child2_token = token.child_token();\n let grandchild_token = child1_token.child_token();\n let grandchild2_token = child1_token.child_token();\n let great_grandchild_token = grandchild_token.child_token();\n\n assert!(!parent_token.is_cancelled());\n assert!(!token.is_cancelled());\n assert!(!sibling_token.is_cancelled());\n assert!(!child1_token.is_cancelled());\n assert!(!child2_token.is_cancelled());\n assert!(!grandchild_token.is_cancelled());\n assert!(!grandchild2_token.is_cancelled());\n assert!(!great_grandchild_token.is_cancelled());\n\n let parent_fut = parent_token.cancelled();\n let fut = token.cancelled();\n let sibling_fut = sibling_token.cancelled();\n let child1_fut = child1_token.cancelled();\n let child2_fut = child2_token.cancelled();\n let grandchild_fut = grandchild_token.cancelled();\n let grandchild2_fut = grandchild2_token.cancelled();\n let great_grandchild_fut = great_grandchild_token.cancelled();\n\n pin!(parent_fut);\n pin!(fut);\n pin!(sibling_fut);\n pin!(child1_fut);\n pin!(child2_fut);\n pin!(grandchild_fut);\n pin!(grandchild2_fut);\n pin!(great_grandchild_fut);\n\n assert_eq!(\n Poll::Pending,\n parent_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n sibling_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n child1_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n child2_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n grandchild_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n grandchild2_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Pending,\n great_grandchild_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 0);\n\n // ACT\n token.cancel();\n\n // ASSERT\n assert_eq!(wake_counter, 6);\n assert!(!parent_token.is_cancelled());\n assert!(token.is_cancelled());\n assert!(!sibling_token.is_cancelled());\n assert!(child1_token.is_cancelled());\n assert!(child2_token.is_cancelled());\n assert!(grandchild_token.is_cancelled());\n assert!(grandchild2_token.is_cancelled());\n assert!(great_grandchild_token.is_cancelled());\n\n assert_eq!(\n Poll::Ready(()),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n child1_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n child2_fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n grandchild_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n grandchild2_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(\n Poll::Ready(()),\n great_grandchild_fut\n .as_mut()\n .poll(&mut Context::from_waker(&waker))\n );\n assert_eq!(wake_counter, 6);\n}\n\n#[test]\nfn drop_parent_before_child_tokens() {\n let token = CancellationToken::new();\n let child1 = token.child_token();\n let child2 = token.child_token();\n\n drop(token);\n assert!(!child1.is_cancelled());\n\n drop(child1);\n drop(child2);\n}\n\n#[test]\nfn derives_send_sync() {\n fn assert_send() {}\n fn assert_sync() {}\n\n assert_send::();\n assert_sync::();\n\n assert_send::>();\n assert_sync::>();\n}\n\n#[test]\nfn run_until_cancelled_test() {\n let (waker, _) = new_count_waker();\n\n {\n let token = CancellationToken::new();\n\n let fut = token.run_until_cancelled(std::future::pending::<()>());\n pin!(fut);\n\n assert_eq!(\n Poll::Pending,\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n token.cancel();\n\n assert_eq!(\n Poll::Ready(None),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n }\n\n {\n let (tx, rx) = oneshot::channel::<()>();\n\n let token = CancellationToken::new();\n let fut = token.run_until_cancelled(async move {\n rx.await.unwrap();\n 42\n });\n pin!(fut);\n\n assert_eq!(\n Poll::Pending,\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n tx.send(()).unwrap();\n\n assert_eq!(\n Poll::Ready(Some(42)),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n }\n\n // Do not poll the future when token is already cancelled.\n {\n let token = CancellationToken::new();\n\n let fut = token.run_until_cancelled(async { panic!(\"fut polled after cancellation\") });\n pin!(fut);\n\n token.cancel();\n\n assert_eq!(\n Poll::Ready(None),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n }\n}\n\n#[test]\nfn run_until_cancelled_owned_test() {\n let (waker, _) = new_count_waker();\n\n {\n let token = CancellationToken::new();\n let to_cancel = token.clone();\n\n let takes_ownership = move |token: CancellationToken| {\n token.run_until_cancelled_owned(std::future::pending::<()>())\n };\n\n let fut = takes_ownership(token);\n pin!(fut);\n\n assert_eq!(\n Poll::Pending,\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n to_cancel.cancel();\n\n assert_eq!(\n Poll::Ready(None),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n }\n\n {\n let (tx, rx) = oneshot::channel::<()>();\n\n let token = CancellationToken::new();\n let takes_ownership = move |token: CancellationToken, rx: oneshot::Receiver<()>| {\n token.run_until_cancelled_owned(async move {\n rx.await.unwrap();\n 42\n })\n };\n let fut = takes_ownership(token, rx);\n pin!(fut);\n\n assert_eq!(\n Poll::Pending,\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n\n tx.send(()).unwrap();\n\n assert_eq!(\n Poll::Ready(Some(42)),\n fut.as_mut().poll(&mut Context::from_waker(&waker))\n );\n }\n}\n" }, { "path": "tokio-util/tests/task_join_map.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(feature = \"join-map\")]\n\nuse std::panic::AssertUnwindSafe;\n\nuse futures::future::{pending, FutureExt};\nuse tokio::sync::oneshot;\nuse tokio::task::LocalSet;\nuse tokio::time::Duration;\nuse tokio_util::task::JoinMap;\n\nfn rt() -> tokio::runtime::Runtime {\n tokio::runtime::Builder::new_current_thread()\n .build()\n .unwrap()\n}\n\n// Spawn `N` tasks that return their index (`i`).\nfn spawn_index_tasks(map: &mut JoinMap, n: usize, on: Option<&LocalSet>) {\n for i in 0..n {\n let rc = std::rc::Rc::new(i);\n match on {\n None => map.spawn_local(i, async move { *rc }),\n Some(local) => map.spawn_local_on(i, async move { *rc }, local),\n };\n }\n}\n\n// Spawn `N` “pending” tasks that own a `oneshot::Sender`.\n// When the task is aborted the sender is dropped, which is observed\n// via the returned `Receiver`s.\nfn spawn_pending_tasks(\n map: &mut JoinMap,\n receivers: &mut Vec>,\n n: usize,\n on: Option<&LocalSet>,\n) {\n for i in 0..n {\n let (tx, rx) = oneshot::channel::<()>();\n receivers.push(rx);\n\n let fut = async move {\n pending::<()>().await;\n drop(tx);\n };\n match on {\n None => map.spawn_local(i, fut),\n Some(local) => map.spawn_local_on(i, fut, local),\n };\n }\n}\n\n/// Await every task in JoinMap and assert every task returns its own key.\nasync fn drain_joinmap_and_assert(mut map: JoinMap, n: usize) {\n let mut seen = vec![false; n];\n while let Some((k, res)) = map.join_next().await {\n let v = res.expect(\"task panicked\");\n assert_eq!(k, v);\n seen[v] = true;\n }\n assert!(seen.into_iter().all(|b| b));\n assert!(map.is_empty());\n}\n\n// Await every receiver and assert they all return `Err` because the\n// corresponding sender (inside an aborted task) was dropped.\nasync fn await_receivers_and_assert(receivers: Vec>) {\n for rx in receivers {\n assert!(\n rx.await.is_err(),\n \"task should have been aborted and sender dropped\"\n );\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn test_with_sleep() {\n let mut map = JoinMap::new();\n\n for i in 0..10 {\n map.spawn(i, async move { i });\n assert_eq!(map.len(), 1 + i);\n }\n map.detach_all();\n assert_eq!(map.len(), 0);\n\n assert!(map.join_next().await.is_none());\n\n for i in 0..10 {\n map.spawn(i, async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n i\n });\n assert_eq!(map.len(), 1 + i);\n }\n\n let mut seen = [false; 10];\n while let Some((k, res)) = map.join_next().await {\n seen[k] = true;\n assert_eq!(res.expect(\"task should have completed successfully\"), k);\n }\n\n for was_seen in &seen {\n assert!(was_seen);\n }\n assert!(map.join_next().await.is_none());\n\n // Do it again.\n for i in 0..10 {\n map.spawn(i, async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n i\n });\n }\n\n let mut seen = [false; 10];\n while let Some((k, res)) = map.join_next().await {\n seen[k] = true;\n assert_eq!(res.expect(\"task should have completed successfully\"), k);\n }\n\n for was_seen in &seen {\n assert!(was_seen);\n }\n assert!(map.join_next().await.is_none());\n}\n\n#[tokio::test]\nasync fn test_abort_on_drop() {\n let mut map = JoinMap::new();\n\n let mut recvs = Vec::new();\n\n for i in 0..16 {\n let (send, recv) = oneshot::channel::<()>();\n recvs.push(recv);\n\n map.spawn(i, async {\n // This task will never complete on its own.\n futures::future::pending::<()>().await;\n drop(send);\n });\n }\n\n drop(map);\n\n for recv in recvs {\n // The task is aborted soon and we will receive an error.\n assert!(recv.await.is_err());\n }\n}\n\n#[tokio::test]\nasync fn alternating() {\n let mut map = JoinMap::new();\n\n assert_eq!(map.len(), 0);\n map.spawn(1, async {});\n assert_eq!(map.len(), 1);\n map.spawn(2, async {});\n assert_eq!(map.len(), 2);\n\n for i in 0..16 {\n let (_, res) = map.join_next().await.unwrap();\n assert!(res.is_ok());\n assert_eq!(map.len(), 1);\n map.spawn(i, async {});\n assert_eq!(map.len(), 2);\n }\n}\n\n#[tokio::test]\nasync fn test_keys() {\n use std::collections::HashSet;\n\n let mut map = JoinMap::new();\n\n assert_eq!(map.len(), 0);\n map.spawn(1, async {});\n assert_eq!(map.len(), 1);\n map.spawn(2, async {});\n assert_eq!(map.len(), 2);\n\n let keys = map.keys().collect::>();\n assert!(keys.contains(&1));\n assert!(keys.contains(&2));\n\n let _ = map.join_next().await.unwrap();\n let _ = map.join_next().await.unwrap();\n\n assert_eq!(map.len(), 0);\n let keys = map.keys().collect::>();\n assert!(keys.is_empty());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn abort_by_key() {\n let mut map = JoinMap::new();\n let mut num_canceled = 0;\n let mut num_completed = 0;\n for i in 0..16 {\n map.spawn(i, async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n });\n }\n\n for i in 0..16 {\n if i % 2 != 0 {\n // abort odd-numbered tasks.\n map.abort(&i);\n }\n }\n\n while let Some((key, res)) = map.join_next().await {\n match res {\n Ok(()) => {\n num_completed += 1;\n assert_eq!(key % 2, 0);\n assert!(!map.contains_key(&key));\n }\n Err(e) => {\n num_canceled += 1;\n assert!(e.is_cancelled());\n assert_ne!(key % 2, 0);\n assert!(!map.contains_key(&key));\n }\n }\n }\n\n assert_eq!(num_canceled, 8);\n assert_eq!(num_completed, 8);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn abort_by_predicate() {\n let mut map = JoinMap::new();\n let mut num_canceled = 0;\n let mut num_completed = 0;\n for i in 0..16 {\n map.spawn(i, async move {\n tokio::time::sleep(Duration::from_secs(i as u64)).await;\n });\n }\n\n // abort odd-numbered tasks.\n map.abort_matching(|key| key % 2 != 0);\n\n while let Some((key, res)) = map.join_next().await {\n match res {\n Ok(()) => {\n num_completed += 1;\n assert_eq!(key % 2, 0);\n assert!(!map.contains_key(&key));\n }\n Err(e) => {\n num_canceled += 1;\n assert!(e.is_cancelled());\n assert_ne!(key % 2, 0);\n assert!(!map.contains_key(&key));\n }\n }\n }\n\n assert_eq!(num_canceled, 8);\n assert_eq!(num_completed, 8);\n}\n\n#[test]\nfn runtime_gone() {\n let mut map = JoinMap::new();\n {\n let rt = rt();\n map.spawn_on(\"key\", async { 1 }, rt.handle());\n drop(rt);\n }\n\n let (key, res) = rt().block_on(map.join_next()).unwrap();\n assert_eq!(key, \"key\");\n assert!(res.unwrap_err().is_cancelled());\n}\n\n// This ensures that `join_next` works correctly when the coop budget is\n// exhausted.\n#[tokio::test(flavor = \"current_thread\")]\nasync fn join_map_coop() {\n // Large enough to trigger coop.\n const TASK_NUM: u32 = 1000;\n\n static SEM: tokio::sync::Semaphore = tokio::sync::Semaphore::const_new(0);\n\n let mut map = JoinMap::new();\n\n for i in 0..TASK_NUM {\n map.spawn(i, async move {\n SEM.add_permits(1);\n i\n });\n }\n\n // Wait for all tasks to complete.\n //\n // Since this is a `current_thread` runtime, there's no race condition\n // between the last permit being added and the task completing.\n let _ = SEM.acquire_many(TASK_NUM).await.unwrap();\n\n let mut count = 0;\n let mut coop_count = 0;\n loop {\n match map.join_next().now_or_never() {\n Some(Some((key, Ok(i)))) => assert_eq!(key, i),\n Some(Some((key, Err(err)))) => panic!(\"failed[{key}]: {err}\"),\n None => {\n coop_count += 1;\n tokio::task::yield_now().await;\n continue;\n }\n Some(None) => break,\n }\n\n count += 1;\n }\n assert!(coop_count >= 1);\n assert_eq!(count, TASK_NUM);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn abort_all() {\n let mut map: JoinMap = JoinMap::new();\n\n for i in 0..5 {\n map.spawn(i, futures::future::pending());\n }\n for i in 5..10 {\n map.spawn(i, async {\n tokio::time::sleep(Duration::from_secs(1)).await;\n });\n }\n\n // The join map will now have 5 pending tasks and 5 ready tasks.\n tokio::time::sleep(Duration::from_secs(2)).await;\n\n map.abort_all();\n assert_eq!(map.len(), 10);\n\n let mut count = 0;\n let mut seen = [false; 10];\n while let Some((k, res)) = map.join_next().await {\n seen[k] = true;\n if let Err(err) = res {\n assert!(err.is_cancelled());\n }\n count += 1;\n }\n assert_eq!(count, 10);\n assert_eq!(map.len(), 0);\n for was_seen in &seen {\n assert!(was_seen);\n }\n}\n\n#[tokio::test]\nasync fn duplicate_keys() {\n let mut map = JoinMap::new();\n map.spawn(1, async { 1 });\n map.spawn(1, async { 2 });\n\n assert_eq!(map.len(), 1);\n\n let (key, res) = map.join_next().await.unwrap();\n assert_eq!(key, 1);\n assert_eq!(res.unwrap(), 2);\n\n assert!(map.join_next().await.is_none());\n}\n\n#[tokio::test]\nasync fn duplicate_keys2() {\n let (send, recv) = oneshot::channel::<()>();\n\n let mut map = JoinMap::new();\n map.spawn(1, async { 1 });\n map.spawn(1, async {\n recv.await.unwrap();\n 2\n });\n\n assert_eq!(map.len(), 1);\n\n tokio::select! {\n biased;\n res = map.join_next() => match res {\n Some((_key, res)) => panic!(\"Task {res:?} exited.\"),\n None => panic!(\"Phantom task completion.\"),\n },\n () = tokio::task::yield_now() => {},\n }\n\n send.send(()).unwrap();\n\n let (key, res) = map.join_next().await.unwrap();\n assert_eq!(key, 1);\n assert_eq!(res.unwrap(), 2);\n\n assert!(map.join_next().await.is_none());\n}\n\n#[cfg_attr(not(panic = \"unwind\"), ignore)]\n#[tokio::test]\nasync fn duplicate_keys_drop() {\n #[derive(Hash, Debug, PartialEq, Eq)]\n struct Key;\n impl Drop for Key {\n fn drop(&mut self) {\n panic!(\"drop called for key\");\n }\n }\n\n let (send, recv) = oneshot::channel::<()>();\n\n let mut map = JoinMap::new();\n\n map.spawn(Key, async { recv.await.unwrap() });\n\n // replace the task, force it to drop the key and abort the task\n // we should expect it to panic when dropping the key.\n let _ = std::panic::catch_unwind(AssertUnwindSafe(|| map.spawn(Key, async {}))).unwrap_err();\n\n // don't panic when this key drops.\n let (key, _) = map.join_next().await.unwrap();\n std::mem::forget(key);\n\n // original task should have been aborted, so the sender should be dangling.\n assert!(send.is_closed());\n\n assert!(map.join_next().await.is_none());\n}\n\nmod spawn_local {\n use super::*;\n\n #[test]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n fn panic_outside_any_runtime() {\n let mut map = JoinMap::new();\n map.spawn_local((), async {});\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n async fn panic_in_multi_thread_runtime() {\n let mut map = JoinMap::new();\n map.spawn_local((), async {});\n }\n\n #[cfg(tokio_unstable)]\n mod local_runtime {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n\n let mut map = JoinMap::new();\n spawn_index_tasks(&mut map, N, None);\n\n assert!(map.join_next().now_or_never().is_none());\n drain_joinmap_and_assert(map, N).await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinMap`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, None);\n assert!(map.join_next().now_or_never().is_none());\n\n map.shutdown().await;\n assert!(map.is_empty());\n await_receivers_and_assert(receivers).await;\n }\n\n /// Spawn several pending-forever tasks, and then drop the [`JoinMap`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, None);\n assert!(map.join_next().now_or_never().is_none());\n\n drop(map);\n await_receivers_and_assert(receivers).await;\n }\n }\n\n mod local_set {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async move {\n let mut map = JoinMap::new();\n spawn_index_tasks(&mut map, N, None);\n drain_joinmap_and_assert(map, N).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinMap`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async {\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, None);\n assert!(map.join_next().now_or_never().is_none());\n\n map.shutdown().await;\n assert!(map.is_empty());\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then drop the [`JoinMap`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n let local = LocalSet::new();\n\n local\n .run_until(async {\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, None);\n assert!(map.join_next().now_or_never().is_none());\n\n drop(map);\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n }\n}\n\nmod spawn_local_on {\n use super::*;\n\n #[cfg(tokio_unstable)]\n mod local_runtime {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n\n let local = LocalSet::new();\n let mut map = JoinMap::new();\n\n spawn_index_tasks(&mut map, N, Some(&local));\n assert!(map.join_next().now_or_never().is_none());\n\n local\n .run_until(async move {\n drain_joinmap_and_assert(map, N).await;\n })\n .await;\n }\n }\n\n mod local_set {\n use super::*;\n\n /// Spawn several tasks, and then join all tasks.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_join_next() {\n const N: usize = 8;\n let local = LocalSet::new();\n let mut pending_map = JoinMap::new();\n\n spawn_index_tasks(&mut pending_map, N, Some(&local));\n assert!(pending_map.join_next().now_or_never().is_none());\n\n local\n .run_until(async move {\n drain_joinmap_and_assert(pending_map, N).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks, and then shutdown the [`JoinMap`].\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_shutdown() {\n const N: usize = 8;\n let local = LocalSet::new();\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, Some(&local));\n assert!(map.join_next().now_or_never().is_none());\n\n local\n .run_until(async move {\n map.shutdown().await;\n assert!(map.is_empty());\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n\n /// Spawn several pending-forever tasks and then drop the [`JoinMap`]\n /// before the `LocalSet` is driven and while the `LocalSet` is already driven.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n\n {\n let local = LocalSet::new();\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, Some(&local));\n assert!(map.join_next().now_or_never().is_none());\n\n drop(map);\n local\n .run_until(async move { await_receivers_and_assert(receivers).await })\n .await;\n }\n\n {\n let local = LocalSet::new();\n let mut map = JoinMap::new();\n let mut receivers = Vec::new();\n\n spawn_pending_tasks(&mut map, &mut receivers, N, Some(&local));\n assert!(map.join_next().now_or_never().is_none());\n\n local\n .run_until(async move {\n drop(map);\n await_receivers_and_assert(receivers).await;\n })\n .await;\n }\n }\n }\n}\n" }, { "path": "tokio-util/tests/task_join_queue.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse tokio::sync::oneshot;\nuse tokio::task::yield_now;\nuse tokio::time::Duration;\nuse tokio_test::{assert_pending, assert_ready, task};\nuse tokio_util::task::JoinQueue;\n\n#[tokio::test]\nasync fn test_join_queue_no_spurious_wakeups() {\n let (tx, rx) = oneshot::channel::<()>();\n let mut join_queue = JoinQueue::new();\n join_queue.spawn(async move {\n let _ = rx.await;\n 42\n });\n\n let mut join_next = task::spawn(join_queue.join_next());\n\n assert_pending!(join_next.poll());\n\n assert!(!join_next.is_woken());\n\n let _ = tx.send(());\n yield_now().await;\n\n assert!(join_next.is_woken());\n\n let output = assert_ready!(join_next.poll());\n assert_eq!(output.unwrap().unwrap(), 42);\n}\n\n#[tokio::test]\nasync fn test_join_queue_abort_on_drop() {\n let mut queue = JoinQueue::new();\n\n let mut recvs = Vec::new();\n\n for _ in 0..16 {\n let (send, recv) = oneshot::channel::<()>();\n recvs.push(recv);\n\n queue.spawn(async move {\n // This task will never complete on its own.\n futures::future::pending::<()>().await;\n drop(send);\n });\n }\n\n drop(queue);\n\n for recv in recvs {\n // The task is aborted soon and we will receive an error.\n assert!(recv.await.is_err());\n }\n}\n\n#[tokio::test]\nasync fn test_join_queue_alternating() {\n let mut queue = JoinQueue::new();\n\n assert_eq!(queue.len(), 0);\n queue.spawn(async {});\n assert_eq!(queue.len(), 1);\n queue.spawn(async {});\n assert_eq!(queue.len(), 2);\n\n for _ in 0..16 {\n let res = queue.join_next().await.unwrap();\n assert!(res.is_ok());\n assert_eq!(queue.len(), 1);\n queue.spawn(async {});\n assert_eq!(queue.len(), 2);\n }\n}\n\n#[tokio::test(start_paused = true)]\nasync fn test_join_queue_abort_all() {\n let mut queue: JoinQueue<()> = JoinQueue::new();\n\n for _ in 0..5 {\n queue.spawn(futures::future::pending());\n }\n for _ in 0..5 {\n queue.spawn(async {\n tokio::time::sleep(Duration::from_secs(1)).await;\n });\n }\n\n // The join queue will now have 5 pending tasks and 5 ready tasks.\n tokio::time::sleep(Duration::from_secs(2)).await;\n\n queue.abort_all();\n assert_eq!(queue.len(), 10);\n\n let mut count = 0;\n while let Some(res) = queue.join_next().await {\n if count < 5 {\n assert!(res.unwrap_err().is_cancelled());\n } else {\n assert!(res.is_ok());\n }\n count += 1;\n }\n assert_eq!(count, 10);\n assert!(queue.is_empty());\n}\n\n#[tokio::test]\nasync fn test_join_queue_join_all() {\n let mut queue = JoinQueue::new();\n let mut senders = Vec::new();\n for i in 0..5 {\n let (tx, rx) = oneshot::channel::<()>();\n senders.push(tx);\n queue.spawn(async move {\n let _ = rx.await;\n i\n });\n }\n // Complete all tasks in reverse order\n while let Some(tx) = senders.pop() {\n let _ = tx.send(());\n }\n let results = queue.join_all().await;\n assert_eq!(results, vec![0, 1, 2, 3, 4]);\n}\n\n#[tokio::test]\nasync fn test_join_queue_shutdown() {\n let mut queue = JoinQueue::new();\n let mut senders = Vec::new();\n\n for _ in 0..5 {\n let (tx, rx) = oneshot::channel::<()>();\n senders.push(tx);\n queue.spawn(async move {\n let _ = rx.await;\n });\n }\n\n queue.shutdown().await;\n assert!(queue.is_empty());\n while let Some(tx) = senders.pop() {\n assert!(tx.is_closed());\n }\n}\n\n#[tokio::test]\nasync fn test_join_queue_with_manual_abort() {\n let mut queue = JoinQueue::new();\n let mut num_canceled = 0;\n let mut num_completed = 0;\n let mut senders = Vec::new();\n for i in 0..16 {\n let (tx, rx) = oneshot::channel::<()>();\n senders.push(tx);\n let abort = queue.spawn(async move {\n let _ = rx.await;\n i\n });\n\n if i % 2 != 0 {\n // abort odd-numbered tasks.\n abort.abort();\n }\n }\n // Complete all tasks in reverse order\n while let Some(tx) = senders.pop() {\n let _ = tx.send(());\n }\n while let Some(res) = queue.join_next().await {\n match res {\n Ok(res) => {\n assert_eq!(res, num_completed * 2);\n num_completed += 1;\n }\n Err(e) => {\n assert!(e.is_cancelled());\n num_canceled += 1;\n }\n }\n }\n\n assert_eq!(num_canceled, 8);\n assert_eq!(num_completed, 8);\n}\n\n#[tokio::test]\nasync fn test_join_queue_join_next_with_id() {\n const TASK_NUM: u32 = 1000;\n\n let (send, recv) = tokio::sync::watch::channel(());\n\n let mut queue = JoinQueue::new();\n let mut spawned = Vec::with_capacity(TASK_NUM as usize);\n\n for _ in 0..TASK_NUM {\n let mut recv = recv.clone();\n let handle = queue.spawn(async move { recv.changed().await.unwrap() });\n\n spawned.push(handle.id());\n }\n drop(recv);\n\n send.send_replace(());\n send.closed().await;\n\n let mut count = 0;\n let mut joined = Vec::with_capacity(TASK_NUM as usize);\n while let Some(res) = queue.join_next_with_id().await {\n match res {\n Ok((id, ())) => {\n count += 1;\n joined.push(id);\n }\n Err(err) => panic!(\"failed: {err}\"),\n }\n }\n\n assert_eq!(count, TASK_NUM);\n assert_eq!(joined, spawned);\n}\n\n#[tokio::test]\nasync fn test_join_queue_try_join_next() {\n let mut queue = JoinQueue::new();\n let (tx1, rx1) = oneshot::channel::<()>();\n queue.spawn(async {\n let _ = rx1.await;\n });\n let (tx2, rx2) = oneshot::channel::<()>();\n queue.spawn(async {\n let _ = rx2.await;\n });\n let (tx3, rx3) = oneshot::channel::<()>();\n queue.spawn(async {\n let _ = rx3.await;\n });\n\n // This function also checks that calling `queue.try_join_next()` repeatedly when\n // no task is ready is idempotent, i.e. that it does not change the queue state.\n fn check_try_join_next_is_noop(queue: &mut JoinQueue<()>) {\n let len = queue.len();\n for _ in 0..5 {\n assert!(queue.try_join_next().is_none());\n assert_eq!(queue.len(), len);\n }\n }\n\n assert_eq!(queue.len(), 3);\n check_try_join_next_is_noop(&mut queue);\n\n tx1.send(()).unwrap();\n tokio::task::yield_now().await;\n\n assert_eq!(queue.len(), 3);\n assert!(queue.try_join_next().is_some());\n assert_eq!(queue.len(), 2);\n check_try_join_next_is_noop(&mut queue);\n\n tx3.send(()).unwrap();\n tokio::task::yield_now().await;\n\n assert_eq!(queue.len(), 2);\n check_try_join_next_is_noop(&mut queue);\n\n tx2.send(()).unwrap();\n tokio::task::yield_now().await;\n\n assert_eq!(queue.len(), 2);\n assert!(queue.try_join_next().is_some());\n assert_eq!(queue.len(), 1);\n assert!(queue.try_join_next().is_some());\n assert!(queue.is_empty());\n check_try_join_next_is_noop(&mut queue);\n}\n\n#[tokio::test]\nasync fn test_join_queue_try_join_next_disabled_coop() {\n // This number is large enough to trigger coop. Without using `tokio::task::coop::unconstrained`\n // inside `try_join_next` this test fails on `assert!(coop_count == 0)`.\n const TASK_NUM: u32 = 1000;\n\n let sem: std::sync::Arc =\n std::sync::Arc::new(tokio::sync::Semaphore::new(0));\n\n let mut queue = JoinQueue::new();\n\n for _ in 0..TASK_NUM {\n let sem = sem.clone();\n queue.spawn(async move {\n sem.add_permits(1);\n });\n }\n\n let _ = sem.acquire_many(TASK_NUM).await.unwrap();\n\n let mut count = 0;\n let mut coop_count = 0;\n while !queue.is_empty() {\n match queue.try_join_next() {\n Some(Ok(())) => count += 1,\n Some(Err(err)) => panic!(\"failed: {err}\"),\n None => {\n coop_count += 1;\n tokio::task::yield_now().await;\n }\n }\n }\n assert_eq!(coop_count, 0);\n assert_eq!(count, TASK_NUM);\n}\n\n#[tokio::test]\nasync fn test_join_queue_try_join_next_with_id_disabled_coop() {\n // Note that this number is large enough to trigger coop as in\n // `test_join_queue_try_join_next_coop` test. Without using\n // `tokio::task::coop::unconstrained` inside `try_join_next_with_id`\n // this test fails on `assert_eq!(count, TASK_NUM)`.\n const TASK_NUM: u32 = 1000;\n\n let (send, recv) = tokio::sync::watch::channel(());\n\n let mut queue = JoinQueue::new();\n let mut spawned = Vec::with_capacity(TASK_NUM as usize);\n\n for _ in 0..TASK_NUM {\n let mut recv = recv.clone();\n let handle = queue.spawn(async move { recv.changed().await.unwrap() });\n\n spawned.push(handle.id());\n }\n drop(recv);\n\n assert!(queue.try_join_next_with_id().is_none());\n\n send.send_replace(());\n send.closed().await;\n\n let mut count = 0;\n let mut coop_count = 0;\n let mut joined = Vec::with_capacity(TASK_NUM as usize);\n while !queue.is_empty() {\n match queue.try_join_next_with_id() {\n Some(Ok((id, ()))) => {\n count += 1;\n joined.push(id);\n }\n Some(Err(err)) => panic!(\"failed: {err}\"),\n None => {\n coop_count += 1;\n tokio::task::yield_now().await;\n }\n }\n }\n\n assert_eq!(coop_count, 0);\n assert_eq!(count, TASK_NUM);\n assert_eq!(joined, spawned);\n}\n\n#[test]\n#[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n)]\nfn spawn_local_panic_outside_any_runtime() {\n let mut queue = JoinQueue::new();\n queue.spawn_local(async {});\n}\n\n#[tokio::test(flavor = \"multi_thread\")]\n#[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n)]\nasync fn spawn_local_panic_in_multi_thread_runtime() {\n let mut queue = JoinQueue::new();\n queue.spawn_local(async {});\n}\n" }, { "path": "tokio-util/tests/task_tracker.rs", "content": "#![warn(rust_2018_idioms)]\n\nuse futures::future::pending;\n#[cfg(tokio_unstable)]\nuse std::rc::Rc;\nuse tokio::sync::mpsc;\nuse tokio::task::LocalSet;\nuse tokio_test::{assert_pending, assert_ready, task};\nuse tokio_util::task::TaskTracker;\n\n#[test]\nfn open_close() {\n let tracker = TaskTracker::new();\n assert!(!tracker.is_closed());\n assert!(tracker.is_empty());\n assert_eq!(tracker.len(), 0);\n\n tracker.close();\n assert!(tracker.is_closed());\n assert!(tracker.is_empty());\n assert_eq!(tracker.len(), 0);\n\n tracker.reopen();\n assert!(!tracker.is_closed());\n tracker.reopen();\n assert!(!tracker.is_closed());\n\n assert!(tracker.is_empty());\n assert_eq!(tracker.len(), 0);\n\n tracker.close();\n assert!(tracker.is_closed());\n tracker.close();\n assert!(tracker.is_closed());\n\n assert!(tracker.is_empty());\n assert_eq!(tracker.len(), 0);\n}\n\n#[test]\nfn token_len() {\n let tracker = TaskTracker::new();\n\n let mut tokens = Vec::new();\n for i in 0..10 {\n assert_eq!(tracker.len(), i);\n tokens.push(tracker.token());\n }\n\n assert!(!tracker.is_empty());\n assert_eq!(tracker.len(), 10);\n\n for (i, token) in tokens.into_iter().enumerate() {\n drop(token);\n assert_eq!(tracker.len(), 9 - i);\n }\n}\n\n#[test]\nfn notify_immediately() {\n let tracker = TaskTracker::new();\n tracker.close();\n\n let mut wait = task::spawn(tracker.wait());\n assert_ready!(wait.poll());\n}\n\n#[test]\nfn notify_immediately_on_reopen() {\n let tracker = TaskTracker::new();\n tracker.close();\n\n let mut wait = task::spawn(tracker.wait());\n tracker.reopen();\n assert_ready!(wait.poll());\n}\n\n#[test]\nfn notify_on_close() {\n let tracker = TaskTracker::new();\n\n let mut wait = task::spawn(tracker.wait());\n\n assert_pending!(wait.poll());\n tracker.close();\n assert_ready!(wait.poll());\n}\n\n#[test]\nfn notify_on_close_reopen() {\n let tracker = TaskTracker::new();\n\n let mut wait = task::spawn(tracker.wait());\n\n assert_pending!(wait.poll());\n tracker.close();\n tracker.reopen();\n assert_ready!(wait.poll());\n}\n\n#[test]\nfn notify_on_last_task() {\n let tracker = TaskTracker::new();\n tracker.close();\n let token = tracker.token();\n\n let mut wait = task::spawn(tracker.wait());\n assert_pending!(wait.poll());\n drop(token);\n assert_ready!(wait.poll());\n}\n\n#[test]\nfn notify_on_last_task_respawn() {\n let tracker = TaskTracker::new();\n tracker.close();\n let token = tracker.token();\n\n let mut wait = task::spawn(tracker.wait());\n assert_pending!(wait.poll());\n drop(token);\n let token2 = tracker.token();\n assert_ready!(wait.poll());\n drop(token2);\n}\n\n#[test]\nfn no_notify_on_respawn_if_open() {\n let tracker = TaskTracker::new();\n let token = tracker.token();\n\n let mut wait = task::spawn(tracker.wait());\n assert_pending!(wait.poll());\n drop(token);\n let token2 = tracker.token();\n assert_pending!(wait.poll());\n drop(token2);\n}\n\n#[test]\nfn close_during_exit() {\n const ITERS: usize = 5;\n\n for close_spot in 0..=ITERS {\n let tracker = TaskTracker::new();\n let tokens: Vec<_> = (0..ITERS).map(|_| tracker.token()).collect();\n\n let mut wait = task::spawn(tracker.wait());\n\n for (i, token) in tokens.into_iter().enumerate() {\n assert_pending!(wait.poll());\n if i == close_spot {\n tracker.close();\n assert_pending!(wait.poll());\n }\n drop(token);\n }\n\n if close_spot == ITERS {\n assert_pending!(wait.poll());\n tracker.close();\n }\n\n assert_ready!(wait.poll());\n }\n}\n\n#[test]\nfn notify_many() {\n let tracker = TaskTracker::new();\n\n let mut waits: Vec<_> = (0..10).map(|_| task::spawn(tracker.wait())).collect();\n\n for wait in &mut waits {\n assert_pending!(wait.poll());\n }\n\n tracker.close();\n\n for wait in &mut waits {\n assert_ready!(wait.poll());\n }\n}\n\n#[cfg(tokio_unstable)]\nmod spawn {\n use super::*;\n\n /// Spawn several tasks, and then close the [`TaskTracker`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_close() {\n const N: usize = 8;\n\n let tracker = TaskTracker::new();\n\n for _ in 0..N {\n tracker.spawn(async {});\n }\n\n for _ in 0..N {\n tracker.spawn_on(async {}, &tokio::runtime::Handle::current());\n }\n\n tracker.close();\n tracker.wait().await;\n\n assert!(tracker.is_empty());\n assert!(tracker.is_closed());\n }\n}\n\n#[cfg(tokio_unstable)]\nmod spawn_local {\n use super::*;\n\n #[test]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n fn panic_outside_any_runtime() {\n let tracker = TaskTracker::new();\n tracker.spawn_local(async {});\n }\n\n #[tokio::test(flavor = \"multi_thread\")]\n #[should_panic(\n expected = \"`spawn_local` called from outside of a `task::LocalSet` or `runtime::LocalRuntime`\"\n )]\n async fn panic_in_multi_thread_runtime() {\n let tracker = TaskTracker::new();\n tracker.spawn_local(async {});\n }\n\n /// Spawn several tasks, and then close the [`TaskTracker`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_close() {\n const N: usize = 8;\n\n let tracker = TaskTracker::new();\n\n for _ in 0..N {\n let rc = Rc::new(());\n tracker.spawn_local(async move {\n drop(rc);\n });\n }\n\n tracker.close();\n tracker.wait().await;\n\n assert!(tracker.is_empty());\n assert!(tracker.is_closed());\n }\n\n /// Close the [`TaskTracker`], and then spawn several tasks\n #[tokio::test(flavor = \"local\")]\n async fn spawn_after_close() {\n const N: usize = 8;\n\n let tracker = TaskTracker::new();\n\n tracker.close();\n\n for _ in 0..N {\n let rc = Rc::new(());\n tracker.spawn_local(async move {\n drop(rc);\n });\n }\n\n tracker.wait().await;\n\n assert!(tracker.is_closed());\n assert!(tracker.is_empty());\n }\n}\n\nmod spawn_local_on {\n use super::*;\n\n #[cfg(tokio_unstable)]\n mod local_runtime {\n use super::*;\n\n /// Spawn several tasks, and then close the [`TaskTracker`].\n #[tokio::test(flavor = \"local\")]\n async fn spawn_then_close() {\n const N: usize = 8;\n let local_set = LocalSet::new();\n\n let tracker = TaskTracker::new();\n\n for _ in 0..N {\n let rc = Rc::new(());\n tracker.spawn_local_on(\n async move {\n drop(rc);\n },\n &local_set,\n );\n }\n\n local_set\n .run_until(async {\n tracker.close();\n tracker.wait().await;\n\n assert!(tracker.is_empty());\n assert!(tracker.is_closed());\n })\n .await;\n }\n }\n\n mod local_set {\n use super::*;\n\n /// Spawn several pending-forever tasks, and then drop the [`TaskTracker`]\n /// while the `LocalSet` is already driven.\n #[tokio::test(flavor = \"current_thread\")]\n async fn spawn_then_drop() {\n const N: usize = 8;\n let local = LocalSet::new();\n let tracker = TaskTracker::new();\n let (tx, mut rx) = mpsc::unbounded_channel::<()>();\n\n for _i in 0..N {\n let tx = tx.clone();\n tracker.spawn_local_on(\n async move {\n pending::<()>().await;\n drop(tx);\n },\n &local,\n );\n }\n drop(tx);\n\n local\n .run_until(async move {\n drop(tracker);\n tokio::task::yield_now().await;\n\n use tokio::sync::mpsc::error::TryRecvError;\n\n assert!(matches!(rx.try_recv(), Err(TryRecvError::Empty)));\n })\n .await;\n }\n\n /// Close the tracker first, spawn several pending-forever tasks,\n /// then wait while the`LocalSet` is already driven.\n #[tokio::test(flavor = \"current_thread\")]\n async fn close_then_spawn() {\n const N: usize = 8;\n let local = LocalSet::new();\n let tracker = TaskTracker::new();\n\n tracker.close();\n\n for _ in 0..N {\n let rc = std::rc::Rc::new(());\n tracker.spawn_local_on(\n async move {\n drop(rc);\n },\n &local,\n );\n }\n\n local\n .run_until(async move {\n tracker.wait().await;\n assert!(tracker.is_closed());\n assert!(tracker.is_empty());\n })\n .await;\n }\n }\n}\n" }, { "path": "tokio-util/tests/time_delay_queue.rs", "content": "#![allow(clippy::disallowed_names)]\n#![warn(rust_2018_idioms)]\n#![cfg(feature = \"full\")]\n\nuse futures::StreamExt;\nuse tokio::time::{self, sleep, sleep_until, Duration, Instant};\nuse tokio_test::{assert_pending, assert_ready, task};\nuse tokio_util::time::DelayQueue;\n\nmacro_rules! poll {\n ($queue:ident) => {\n $queue.enter(|cx, mut queue| queue.poll_expired(cx))\n };\n}\n\nmacro_rules! assert_ready_some {\n ($e:expr) => {{\n match assert_ready!($e) {\n Some(v) => v,\n None => panic!(\"None\"),\n }\n }};\n}\n\n#[tokio::test]\nasync fn single_immediate_delay() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n let _key = queue.insert_at(\"foo\", Instant::now());\n\n // Advance time by 1ms to handle thee rounding\n sleep(ms(1)).await;\n\n assert_ready_some!(poll!(queue));\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none())\n}\n\n#[tokio::test]\nasync fn multi_immediate_delays() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let _k = queue.insert_at(\"1\", Instant::now());\n let _k = queue.insert_at(\"2\", Instant::now());\n let _k = queue.insert_at(\"3\", Instant::now());\n\n sleep(ms(1)).await;\n\n let mut res = vec![];\n\n while res.len() < 3 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none());\n\n res.sort_unstable();\n\n assert_eq!(\"1\", res[0]);\n assert_eq!(\"2\", res[1]);\n assert_eq!(\"3\", res[2]);\n}\n\n#[tokio::test]\nasync fn single_short_delay() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n let _key = queue.insert_at(\"foo\", Instant::now() + ms(5));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(1)).await;\n\n assert!(!queue.is_woken());\n\n sleep(ms(5)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(*entry.get_ref(), \"foo\");\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none());\n}\n\n#[tokio::test]\n#[cfg_attr(miri, ignore)] // Too slow on miri.\nasync fn multi_delay_at_start() {\n time::pause();\n\n let long = 262_144 + 9 * 4096;\n let delays = &[1000, 2, 234, long, 60, 10];\n\n let mut queue = task::spawn(DelayQueue::new());\n\n // Setup the delays\n for &i in delays {\n let _key = queue.insert_at(i, Instant::now() + ms(i));\n }\n\n assert_pending!(poll!(queue));\n assert!(!queue.is_woken());\n\n let start = Instant::now();\n for elapsed in 0..1200 {\n println!(\"elapsed: {elapsed:?}\");\n let elapsed = elapsed + 1;\n tokio::time::sleep_until(start + ms(elapsed)).await;\n\n if delays.contains(&elapsed) {\n assert!(queue.is_woken());\n assert_ready!(poll!(queue));\n assert_pending!(poll!(queue));\n } else if queue.is_woken() {\n let cascade = &[192, 960];\n assert!(\n cascade.contains(&elapsed),\n \"elapsed={} dt={:?}\",\n elapsed,\n Instant::now() - start\n );\n\n assert_pending!(poll!(queue));\n }\n }\n println!(\"finished multi_delay_start\");\n}\n\n#[tokio::test]\nasync fn insert_in_past_fires_immediately() {\n println!(\"running insert_in_past_fires_immediately\");\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n let now = Instant::now();\n\n sleep(ms(10)).await;\n\n queue.insert_at(\"foo\", now);\n\n assert_ready!(poll!(queue));\n println!(\"finished insert_in_past_fires_immediately\");\n}\n\n#[tokio::test]\nasync fn remove_entry() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let key = queue.insert_at(\"foo\", Instant::now() + ms(5));\n\n assert_pending!(poll!(queue));\n\n let entry = queue.remove(&key);\n assert_eq!(entry.into_inner(), \"foo\");\n\n sleep(ms(10)).await;\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none());\n}\n\n#[tokio::test]\nasync fn reset_entry() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n let key = queue.insert_at(\"foo\", now + ms(5));\n\n assert_pending!(poll!(queue));\n sleep(ms(1)).await;\n\n queue.reset_at(&key, now + ms(10));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(7)).await;\n\n assert!(!queue.is_woken());\n\n assert_pending!(poll!(queue));\n\n sleep(ms(3)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(*entry.get_ref(), \"foo\");\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none())\n}\n\n// Reproduces tokio-rs/tokio#849.\n#[tokio::test]\nasync fn reset_much_later() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n sleep(ms(1)).await;\n\n let key = queue.insert_at(\"foo\", now + ms(200));\n assert_pending!(poll!(queue));\n\n sleep(ms(3)).await;\n\n queue.reset_at(&key, now + ms(10));\n\n sleep(ms(20)).await;\n\n assert!(queue.is_woken());\n}\n\n// Reproduces tokio-rs/tokio#849.\n#[tokio::test]\nasync fn reset_twice() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n let now = Instant::now();\n\n sleep(ms(1)).await;\n\n let key = queue.insert_at(\"foo\", now + ms(200));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(3)).await;\n\n queue.reset_at(&key, now + ms(50));\n\n sleep(ms(20)).await;\n\n queue.reset_at(&key, now + ms(40));\n\n sleep(ms(20)).await;\n\n assert!(queue.is_woken());\n}\n\n/// Regression test: Given an entry inserted with a deadline in the past, so\n/// that it is placed directly on the expired queue, reset the entry to a\n/// deadline in the future. Validate that this leaves the entry and queue in an\n/// internally consistent state by running an additional reset on the entry\n/// before polling it to completion.\n#[tokio::test]\nasync fn repeatedly_reset_entry_inserted_as_expired() {\n time::pause();\n\n // Instants before the start of the test seem to break in wasm.\n time::sleep(ms(1000)).await;\n\n let mut queue = task::spawn(DelayQueue::new());\n let now = Instant::now();\n\n let key = queue.insert_at(\"foo\", now - ms(100));\n\n queue.reset_at(&key, now + ms(100));\n queue.reset_at(&key, now + ms(50));\n\n assert_pending!(poll!(queue));\n\n time::sleep_until(now + ms(60)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"foo\");\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none());\n}\n\n#[tokio::test]\nasync fn remove_expired_item() {\n time::pause();\n\n let mut queue = DelayQueue::new();\n\n let now = Instant::now();\n\n sleep(ms(10)).await;\n\n let key = queue.insert_at(\"foo\", now);\n\n let entry = queue.remove(&key);\n assert_eq!(entry.into_inner(), \"foo\");\n}\n\n/// Regression test: it should be possible to remove entries which fall in the\n/// 0th slot of the internal timer wheel — that is, entries whose expiration\n/// (a) falls at the beginning of one of the wheel's hierarchical levels and (b)\n/// is equal to the wheel's current elapsed time.\n#[tokio::test]\nasync fn remove_at_timer_wheel_threshold() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let key1 = queue.insert_at(\"foo\", now + ms(64));\n let key2 = queue.insert_at(\"bar\", now + ms(64));\n\n sleep(ms(80)).await;\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n\n match entry {\n \"foo\" => {\n let entry = queue.remove(&key2).into_inner();\n assert_eq!(entry, \"bar\");\n }\n \"bar\" => {\n let entry = queue.remove(&key1).into_inner();\n assert_eq!(entry, \"foo\");\n }\n other => panic!(\"other: {other:?}\"),\n }\n}\n\n#[tokio::test]\nasync fn expires_before_last_insert() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"foo\", now + ms(10_000));\n\n // Delay should be set to 8.192s here.\n assert_pending!(poll!(queue));\n\n // Delay should be set to the delay of the new item here\n queue.insert_at(\"bar\", now + ms(600));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(600)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"bar\");\n}\n\n#[tokio::test]\nasync fn multi_reset() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let one = queue.insert_at(\"one\", now + ms(200));\n let two = queue.insert_at(\"two\", now + ms(250));\n\n assert_pending!(poll!(queue));\n\n queue.reset_at(&one, now + ms(300));\n queue.reset_at(&two, now + ms(350));\n queue.reset_at(&one, now + ms(400));\n\n sleep(ms(310)).await;\n\n assert_pending!(poll!(queue));\n\n sleep(ms(50)).await;\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(*entry.get_ref(), \"two\");\n\n assert_pending!(poll!(queue));\n\n sleep(ms(50)).await;\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(*entry.get_ref(), \"one\");\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none())\n}\n\n#[tokio::test]\nasync fn expire_first_key_when_reset_to_expire_earlier() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let one = queue.insert_at(\"one\", now + ms(200));\n queue.insert_at(\"two\", now + ms(250));\n\n assert_pending!(poll!(queue));\n\n queue.reset_at(&one, now + ms(100));\n\n sleep(ms(100)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"one\");\n}\n\n#[tokio::test]\nasync fn expire_second_key_when_reset_to_expire_earlier() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"one\", now + ms(200));\n let two = queue.insert_at(\"two\", now + ms(250));\n\n assert_pending!(poll!(queue));\n\n queue.reset_at(&two, now + ms(100));\n\n sleep(ms(100)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"two\");\n}\n\n#[tokio::test]\nasync fn reset_first_expiring_item_to_expire_later() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let one = queue.insert_at(\"one\", now + ms(200));\n let _two = queue.insert_at(\"two\", now + ms(250));\n\n assert_pending!(poll!(queue));\n\n queue.reset_at(&one, now + ms(300));\n sleep(ms(250)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"two\");\n}\n\n#[tokio::test]\nasync fn insert_before_first_after_poll() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let _one = queue.insert_at(\"one\", now + ms(200));\n\n assert_pending!(poll!(queue));\n\n let _two = queue.insert_at(\"two\", now + ms(100));\n\n sleep(ms(99)).await;\n\n assert_pending!(poll!(queue));\n\n sleep(ms(1)).await;\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"two\");\n}\n\n#[tokio::test]\nasync fn insert_after_ready_poll() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"1\", now + ms(100));\n queue.insert_at(\"2\", now + ms(100));\n queue.insert_at(\"3\", now + ms(100));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(100)).await;\n\n assert!(queue.is_woken());\n\n let mut res = vec![];\n\n while res.len() < 3 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n queue.insert_at(\"foo\", now + ms(500));\n }\n\n res.sort_unstable();\n\n assert_eq!(\"1\", res[0]);\n assert_eq!(\"2\", res[1]);\n assert_eq!(\"3\", res[2]);\n}\n\n#[tokio::test]\nasync fn reset_later_after_slot_starts() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let foo = queue.insert_at(\"foo\", now + ms(100));\n\n assert_pending!(poll!(queue));\n\n sleep_until(now + Duration::from_millis(80)).await;\n\n assert!(!queue.is_woken());\n\n // At this point the queue hasn't been polled, so `elapsed` on the wheel\n // for the queue is still at 0 and hence the 1ms resolution slots cover\n // [0-64). Resetting the time on the entry to 120 causes it to get put in\n // the [64-128) slot. As the queue knows that the first entry is within\n // that slot, but doesn't know when, it must wake immediately to advance\n // the wheel.\n queue.reset_at(&foo, now + ms(120));\n assert!(queue.is_woken());\n\n assert_pending!(poll!(queue));\n\n sleep_until(now + Duration::from_millis(119)).await;\n assert!(!queue.is_woken());\n\n sleep(ms(1)).await;\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"foo\");\n}\n\n#[tokio::test]\nasync fn reset_inserted_expired() {\n time::pause();\n\n // Instants before the start of the test seem to break in wasm.\n time::sleep(ms(1000)).await;\n\n let mut queue = task::spawn(DelayQueue::new());\n let now = Instant::now();\n\n let key = queue.insert_at(\"foo\", now - ms(100));\n\n // this causes the panic described in #2473\n queue.reset_at(&key, now + ms(100));\n\n assert_eq!(1, queue.len());\n\n sleep(ms(200)).await;\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"foo\");\n\n assert_eq!(queue.len(), 0);\n}\n\n#[tokio::test]\nasync fn reset_earlier_after_slot_starts() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let foo = queue.insert_at(\"foo\", now + ms(200));\n\n assert_pending!(poll!(queue));\n\n sleep_until(now + Duration::from_millis(80)).await;\n\n assert!(!queue.is_woken());\n\n // At this point the queue hasn't been polled, so `elapsed` on the wheel\n // for the queue is still at 0 and hence the 1ms resolution slots cover\n // [0-64). Resetting the time on the entry to 120 causes it to get put in\n // the [64-128) slot. As the queue knows that the first entry is within\n // that slot, but doesn't know when, it must wake immediately to advance\n // the wheel.\n queue.reset_at(&foo, now + ms(120));\n assert!(queue.is_woken());\n\n assert_pending!(poll!(queue));\n\n sleep_until(now + Duration::from_millis(119)).await;\n assert!(!queue.is_woken());\n\n sleep(ms(1)).await;\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"foo\");\n}\n\n#[tokio::test]\nasync fn insert_in_past_after_poll_fires_immediately() {\n time::pause();\n\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"foo\", now + ms(200));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(80)).await;\n\n assert!(!queue.is_woken());\n queue.insert_at(\"bar\", now + ms(40));\n\n assert!(queue.is_woken());\n\n let entry = assert_ready_some!(poll!(queue)).into_inner();\n assert_eq!(entry, \"bar\");\n}\n\n#[tokio::test]\nasync fn delay_queue_poll_expired_when_empty() {\n let mut delay_queue = task::spawn(DelayQueue::new());\n let key = delay_queue.insert(0, std::time::Duration::from_secs(10));\n assert_pending!(poll!(delay_queue));\n\n delay_queue.remove(&key);\n assert!(assert_ready!(poll!(delay_queue)).is_none());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn compact_expire_empty() {\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"foo1\", now + ms(10));\n queue.insert_at(\"foo2\", now + ms(10));\n\n sleep(ms(10)).await;\n\n let mut res = vec![];\n while res.len() < 2 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n queue.compact();\n\n assert_eq!(queue.len(), 0);\n assert_eq!(queue.capacity(), 0);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn compact_remove_empty() {\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let key1 = queue.insert_at(\"foo1\", now + ms(10));\n let key2 = queue.insert_at(\"foo2\", now + ms(10));\n\n queue.remove(&key1);\n queue.remove(&key2);\n\n queue.compact();\n\n assert_eq!(queue.len(), 0);\n assert_eq!(queue.capacity(), 0);\n}\n\n#[tokio::test(start_paused = true)]\n// Trigger a re-mapping of keys in the slab due to a `compact` call and\n// test removal of re-mapped keys\nasync fn compact_remove_remapped_keys() {\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n queue.insert_at(\"foo1\", now + ms(10));\n queue.insert_at(\"foo2\", now + ms(10));\n\n // should be assigned indices 3 and 4\n let key3 = queue.insert_at(\"foo3\", now + ms(20));\n let key4 = queue.insert_at(\"foo4\", now + ms(20));\n\n sleep(ms(10)).await;\n\n let mut res = vec![];\n while res.len() < 2 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n // items corresponding to `foo3` and `foo4` will be assigned\n // new indices here\n queue.compact();\n\n queue.insert_at(\"foo5\", now + ms(10));\n\n // test removal of re-mapped keys\n let expired3 = queue.remove(&key3);\n let expired4 = queue.remove(&key4);\n\n assert_eq!(expired3.into_inner(), \"foo3\");\n assert_eq!(expired4.into_inner(), \"foo4\");\n\n queue.compact();\n assert_eq!(queue.len(), 1);\n assert_eq!(queue.capacity(), 1);\n}\n\n#[tokio::test(start_paused = true)]\nasync fn compact_change_deadline() {\n let mut queue = task::spawn(DelayQueue::new());\n\n let mut now = Instant::now();\n\n queue.insert_at(\"foo1\", now + ms(10));\n queue.insert_at(\"foo2\", now + ms(10));\n\n // should be assigned indices 3 and 4\n queue.insert_at(\"foo3\", now + ms(20));\n let key4 = queue.insert_at(\"foo4\", now + ms(20));\n\n sleep(ms(10)).await;\n\n let mut res = vec![];\n while res.len() < 2 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n // items corresponding to `foo3` and `foo4` should be assigned\n // new indices\n queue.compact();\n\n now = Instant::now();\n\n queue.insert_at(\"foo5\", now + ms(10));\n let key6 = queue.insert_at(\"foo6\", now + ms(10));\n\n queue.reset_at(&key4, now + ms(20));\n queue.reset_at(&key6, now + ms(20));\n\n // foo3 and foo5 will expire\n sleep(ms(10)).await;\n\n while res.len() < 4 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n sleep(ms(10)).await;\n\n while res.len() < 6 {\n let entry = assert_ready_some!(poll!(queue));\n res.push(entry.into_inner());\n }\n\n let entry = assert_ready!(poll!(queue));\n assert!(entry.is_none());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn item_expiry_greater_than_wheel() {\n // This function tests that a delay queue that has existed for at least 2^36 milliseconds won't panic when a new item is inserted.\n let mut queue = DelayQueue::new();\n for _ in 0..2 {\n tokio::time::advance(Duration::from_millis(1 << 35)).await;\n queue.insert(0, Duration::from_millis(0));\n queue.next().await;\n }\n // This should not panic\n let no_panic = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n queue.insert(1, Duration::from_millis(1));\n }));\n assert!(no_panic.is_ok());\n}\n\n#[cfg_attr(target_os = \"wasi\", ignore = \"FIXME: Does not seem to work with WASI\")]\n#[tokio::test(start_paused = true)]\n#[cfg(panic = \"unwind\")]\nasync fn remove_after_compact() {\n let now = Instant::now();\n let mut queue = DelayQueue::new();\n\n let foo_key = queue.insert_at(\"foo\", now + ms(10));\n queue.insert_at(\"bar\", now + ms(20));\n queue.remove(&foo_key);\n queue.compact();\n\n let panic = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n queue.remove(&foo_key);\n }));\n assert!(panic.is_err());\n}\n\n#[cfg_attr(target_os = \"wasi\", ignore = \"FIXME: Does not seem to work with WASI\")]\n#[tokio::test(start_paused = true)]\n#[cfg(panic = \"unwind\")]\nasync fn remove_after_compact_poll() {\n let now = Instant::now();\n let mut queue = task::spawn(DelayQueue::new());\n\n let foo_key = queue.insert_at(\"foo\", now + ms(10));\n queue.insert_at(\"bar\", now + ms(20));\n\n sleep(ms(10)).await;\n assert_eq!(assert_ready_some!(poll!(queue)).key(), foo_key);\n\n queue.compact();\n\n let panic = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n queue.remove(&foo_key);\n }));\n assert!(panic.is_err());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn peek() {\n let mut queue = task::spawn(DelayQueue::new());\n\n let now = Instant::now();\n\n let key = queue.insert_at(\"foo\", now + ms(5));\n let key2 = queue.insert_at(\"bar\", now);\n let key3 = queue.insert_at(\"baz\", now + ms(10));\n\n assert_eq!(queue.peek(), Some(key2));\n\n sleep(ms(6)).await;\n\n assert_eq!(queue.peek(), Some(key2));\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(entry.get_ref(), &\"bar\");\n\n assert_eq!(queue.peek(), Some(key));\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(entry.get_ref(), &\"foo\");\n\n assert_eq!(queue.peek(), Some(key3));\n\n assert_pending!(poll!(queue));\n\n sleep(ms(5)).await;\n\n assert_eq!(queue.peek(), Some(key3));\n\n let entry = assert_ready_some!(poll!(queue));\n assert_eq!(entry.get_ref(), &\"baz\");\n\n assert!(queue.peek().is_none());\n}\n\n#[tokio::test(start_paused = true)]\nasync fn wake_after_remove_last() {\n let mut queue = task::spawn(DelayQueue::new());\n let key = queue.insert(\"foo\", ms(1000));\n\n assert_pending!(poll!(queue));\n\n queue.remove(&key);\n\n assert!(queue.is_woken());\n assert!(assert_ready!(poll!(queue)).is_none());\n}\n\nfn ms(n: u64) -> Duration {\n Duration::from_millis(n)\n}\n" }, { "path": "tokio-util/tests/udp.rs", "content": "#![warn(rust_2018_idioms)]\n#![cfg(not(target_os = \"wasi\"))] // Wasi doesn't support UDP\n#![cfg(not(miri))] // No `socket` in Miri.\n#![cfg(not(loom))] // No udp / UdpFramed in loom\n\nuse tokio::net::UdpSocket;\nuse tokio_stream::StreamExt;\nuse tokio_util::codec::{Decoder, Encoder, LinesCodec};\nuse tokio_util::udp::UdpFramed;\n\nuse bytes::{BufMut, BytesMut};\nuse futures::future::try_join;\nuse futures::future::FutureExt;\nuse futures::sink::SinkExt;\nuse std::io;\nuse std::sync::Arc;\n\n#[cfg_attr(\n any(\n target_os = \"macos\",\n target_os = \"ios\",\n target_os = \"tvos\",\n target_os = \"watchos\",\n target_os = \"visionos\"\n ),\n allow(unused_assignments)\n)]\n#[tokio::test]\nasync fn send_framed_byte_codec() -> std::io::Result<()> {\n let mut a_soc = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let mut b_soc = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let a_addr = a_soc.local_addr()?;\n let b_addr = b_soc.local_addr()?;\n\n // test sending & receiving bytes\n {\n let mut a = UdpFramed::new(a_soc, ByteCodec);\n let mut b = UdpFramed::new(b_soc, ByteCodec);\n\n let msg = b\"4567\";\n\n let send = a.send((msg, b_addr));\n let recv = b.next().map(|e| e.unwrap());\n let (_, received) = try_join(send, recv).await.unwrap();\n\n let (data, addr) = received;\n assert_eq!(msg, &*data);\n assert_eq!(a_addr, addr);\n\n a_soc = a.into_inner();\n b_soc = b.into_inner();\n }\n\n #[cfg(not(any(\n target_os = \"macos\",\n target_os = \"ios\",\n target_os = \"tvos\",\n target_os = \"watchos\",\n target_os = \"visionos\"\n )))]\n // test sending & receiving an empty message\n {\n let mut a = UdpFramed::new(a_soc, ByteCodec);\n let mut b = UdpFramed::new(b_soc, ByteCodec);\n\n let msg = b\"\";\n\n let send = a.send((msg, b_addr));\n let recv = b.next().map(|e| e.unwrap());\n let (_, received) = try_join(send, recv).await.unwrap();\n\n let (data, addr) = received;\n assert_eq!(msg, &*data);\n assert_eq!(a_addr, addr);\n }\n\n Ok(())\n}\n\npub struct ByteCodec;\n\nimpl Decoder for ByteCodec {\n type Item = Vec;\n type Error = io::Error;\n\n fn decode(&mut self, buf: &mut BytesMut) -> Result>, io::Error> {\n let len = buf.len();\n Ok(Some(buf.split_to(len).to_vec()))\n }\n}\n\nimpl Encoder<&[u8]> for ByteCodec {\n type Error = io::Error;\n\n fn encode(&mut self, data: &[u8], buf: &mut BytesMut) -> Result<(), io::Error> {\n buf.reserve(data.len());\n buf.put_slice(data);\n Ok(())\n }\n}\n\n#[tokio::test]\nasync fn send_framed_lines_codec() -> std::io::Result<()> {\n let a_soc = UdpSocket::bind(\"127.0.0.1:0\").await?;\n let b_soc = UdpSocket::bind(\"127.0.0.1:0\").await?;\n\n let a_addr = a_soc.local_addr()?;\n let b_addr = b_soc.local_addr()?;\n\n let mut a = UdpFramed::new(a_soc, ByteCodec);\n let mut b = UdpFramed::new(b_soc, LinesCodec::new());\n\n let msg = b\"1\\r\\n2\\r\\n3\\r\\n\".to_vec();\n a.send((&msg, b_addr)).await?;\n\n assert_eq!(b.next().await.unwrap().unwrap(), (\"1\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"2\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"3\".to_string(), a_addr));\n\n Ok(())\n}\n\n#[tokio::test]\nasync fn framed_half() -> std::io::Result<()> {\n let a_soc = Arc::new(UdpSocket::bind(\"127.0.0.1:0\").await?);\n let b_soc = a_soc.clone();\n\n let a_addr = a_soc.local_addr()?;\n let b_addr = b_soc.local_addr()?;\n\n let mut a = UdpFramed::new(a_soc, ByteCodec);\n let mut b = UdpFramed::new(b_soc, LinesCodec::new());\n\n let msg = b\"1\\r\\n2\\r\\n3\\r\\n\".to_vec();\n a.send((&msg, b_addr)).await?;\n\n let msg = b\"4\\r\\n5\\r\\n6\\r\\n\".to_vec();\n a.send((&msg, b_addr)).await?;\n\n assert_eq!(b.next().await.unwrap().unwrap(), (\"1\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"2\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"3\".to_string(), a_addr));\n\n assert_eq!(b.next().await.unwrap().unwrap(), (\"4\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"5\".to_string(), a_addr));\n assert_eq!(b.next().await.unwrap().unwrap(), (\"6\".to_string(), a_addr));\n\n Ok(())\n}\n" } ]