[ { "path": ".github/ISSUE_TEMPLATE/bug_report.yaml", "content": "name: Bug Report\ndescription: File a bug report\nlabels: [\"bug\"]\nbody:\n - type: markdown\n attributes:\n value: >\n If you observed a crash in the library, or saw unexpected behavior in it, report\n your findings here. If you're not reasonably sure that it's not the intended\n behavior, you should ask through one of the support venues listed on the\n previous page.\n - type: checkboxes\n attributes:\n label: Things to check first\n options:\n - label: >\n I have checked that my issue does not already have a solution in the\n [FAQ](https://apscheduler.readthedocs.io/en/master/faq.html)\n required: true\n - label: >\n I have searched the existing issues and didn't find my bug already reported\n there\n required: true\n - label: >\n I have checked that my bug is still present in the latest release\n required: true\n - type: input\n id: version\n attributes:\n label: Version\n description: What version of APScheduler were you running?\n validations:\n required: true\n - type: textarea\n id: what-happened\n attributes:\n label: What happened?\n description: >\n Unless you are reporting a crash, tell us what you expected to happen instead.\n validations:\n required: true\n - type: textarea\n id: mwe\n attributes:\n label: How can we reproduce the bug?\n description: >\n In order to investigate the bug, we need to be able to reproduce it on our own.\n Please create a\n [minimum workable example](https://stackoverflow.com/help/minimal-reproducible-example)\n that demonstrates the problem. List any third party libraries required for this,\n but avoid using them unless absolutely necessary.\n validations:\n required: true\n" }, { "path": ".github/ISSUE_TEMPLATE/config.yml", "content": "blank_issues_enabled: false\ncontact_links:\n - name: Stack Overflow\n url: https://stackoverflow.com/questions/tagged/apscheduler\n about: The preferred site for asking questions\n - name: GitHub Discussions\n url: https://github.com/agronholm/apscheduler/discussions/categories/q-a\n about: An alternative for StackOverflow (if you don't want to register there)\n - name: Support chat on Gitter\n url: https://gitter.im/apscheduler/Lobby\n about: Technical support chat\n" }, { "path": ".github/ISSUE_TEMPLATE/features_request.yaml", "content": "name: Feature request\ndescription: Suggest a new feature\nlabels: [\"enhancement\"]\nbody:\n - type: markdown\n attributes:\n value: >\n If you have thought of a new feature that would increase the usefulness of this\n project, please use this form to send us your idea.\n - type: checkboxes\n attributes:\n label: Things to check first\n options:\n - label: >\n I have searched the existing issues and didn't find my feature already\n requested there\n required: true\n - type: textarea\n id: feature\n attributes:\n label: Feature description\n description: >\n Describe the feature in detail. The more specific the description you can give,\n the easier it should be to implement this feature.\n validations:\n required: true\n - type: textarea\n id: usecase\n attributes:\n label: Use case\n description: >\n Explain why you need this feature, and why you think it would be useful to\n others too.\n validations:\n required: true\n" }, { "path": ".github/dependabot.yml", "content": "# Keep GitHub Actions up to date with GitHub's Dependabot...\n# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot\n# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem\nversion: 2\nupdates:\n - package-ecosystem: github-actions\n directory: /\n groups:\n github-actions:\n patterns:\n - \"*\" # Group all Actions updates into a single larger pull request\n schedule:\n interval: monthly\n" }, { "path": ".github/pull_request_template.md", "content": "\n## Changes\n\nFixes #. \n\n\n\n## Checklist\n\nIf this is a user-facing code change, like a bugfix or a new feature, please ensure that\nyou've fulfilled the following conditions (where applicable):\n\n- [ ] You've added tests (in `tests/`) added which would fail without your patch\n- [ ] You've updated the documentation (in `docs/`, in case of behavior changes or new\nfeatures)\n- [ ] You've added a new changelog entry (in `docs/versionhistory.rst`).\n\nIf this is a trivial change, like a typo fix or a code reformatting, then you can ignore\nthese instructions.\n\n### Updating the changelog\n\nIf there are no entries after the last release, use `**UNRELEASED**` as the version.\nIf, say, your patch fixes issue #999, the entry should look like this:\n\n`* Fix big bad boo-boo in the async scheduler\n (`#123 `_; PR by @yourgithubaccount)\n\nIf there's no issue linked, just link to your pull request instead by updating the\nchangelog after you've created the PR.\n" }, { "path": ".github/workflows/publish.yml", "content": "name: Publish packages to PyPI\n\non:\n push:\n tags:\n - \"[0-9]+.[0-9]+.[0-9]+\"\n - \"[0-9]+.[0-9]+.[0-9]+.post[0-9]+\"\n - \"[0-9]+.[0-9]+.[0-9]+[a-b][0-9]+\"\n - \"[0-9]+.[0-9]+.[0-9]+rc[0-9]+\"\n\njobs:\n build:\n name: Build the source tarball and the wheel\n runs-on: ubuntu-latest\n environment: release\n steps:\n - uses: actions/checkout@v6\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: 3.x\n - name: Install dependencies\n run: pip install build\n - name: Create packages\n run: python -m build\n - name: Archive packages\n uses: actions/upload-artifact@v6\n with:\n name: dist\n path: dist\n\n publish:\n name: Publish build artifacts to the PyPI\n needs: build\n runs-on: ubuntu-latest\n environment: release\n permissions:\n id-token: write\n steps:\n - name: Retrieve packages\n uses: actions/download-artifact@v7\n with:\n name: dist\n path: dist\n - name: Upload packages\n uses: pypa/gh-action-pypi-publish@release/v1\n\n release:\n name: Create a GitHub release\n needs: build\n runs-on: ubuntu-latest\n permissions:\n contents: write\n steps:\n - uses: actions/checkout@v6\n - id: changelog\n uses: agronholm/release-notes@v1\n with:\n path: docs/versionhistory.rst\n - uses: ncipollo/release-action@v1\n with:\n body: ${{ steps.changelog.outputs.changelog }}\n" }, { "path": ".github/workflows/test.yml", "content": "name: test suite\n\non:\n push:\n branches: [master]\n pull_request:\n\njobs:\n test-linux:\n strategy:\n fail-fast: false\n matrix:\n python-version: [\"3.10\", \"3.11\", \"3.12\", \"3.13\", \"3.14\"]\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v6\n with:\n python-version: ${{ matrix.python-version }}\n allow-prereleases: true\n cache: pip\n cache-dependency-path: pyproject.toml\n - name: Start external services\n run: docker compose up -d\n - name: Ensure pip >= v25.1\n run: python -m pip install \"pip >= 25.1\"\n - name: Install the project and its dependencies\n run: pip install --group test -e .\n - name: Test with pytest\n run: coverage run -m pytest -v\n - name: Generate coverage report\n run: coverage xml\n - name: Upload Coverage\n uses: coverallsapp/github-action@v2\n with:\n parallel: true\n file: coverage.xml\n\n test-pypy:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v6\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v6\n with:\n python-version: pypy-3.11\n cache: pip\n cache-dependency-path: pyproject.toml\n - name: Start external services\n run: docker compose up -d\n - name: Ensure pip >= v25.1\n run: python -m pip install \"pip >= 25.1\"\n - name: Install the project and its dependencies\n run: pip install --group test -e .\n - name: Test with pytest\n run: pytest -v\n\n test-others:\n strategy:\n fail-fast: false\n matrix:\n os: [macos-latest, windows-latest]\n python-version: [\"3.10\", \"3.14\"]\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@v6\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v6\n with:\n python-version: ${{ matrix.python-version }}\n allow-prereleases: true\n cache: pip\n cache-dependency-path: pyproject.toml\n - name: Ensure pip >= v25.1\n run: python -m pip install \"pip >= 25.1\"\n - name: Install the project and its dependencies\n run: pip install --group test -e .\n - name: Test with pytest\n run: coverage run -m pytest -v -m \"not external_service\"\n - name: Generate coverage report\n run: coverage xml\n - name: Upload Coverage\n uses: coverallsapp/github-action@v2\n with:\n parallel: true\n file: coverage.xml\n\n coveralls:\n name: Finish Coveralls\n needs:\n - test-linux\n - test-others\n runs-on: ubuntu-latest\n steps:\n - name: Finished\n uses: coverallsapp/github-action@v2\n with:\n parallel-finished: true\n" }, { "path": ".gitignore", "content": ".project\n.pydevproject\n.idea/\n.coverage\n.cache/\n.mypy_cache\n.pytest_cache/\n.tox/\n.eggs/\n*.egg-info/\n*.pyc\ndist/\ndocs/_build/\nbuild/\nvirtualenv/\nvenv*/\nexample.sqlite\n" }, { "path": ".mailmap", "content": "Alex Grönholm agronholm \nAlex Grönholm demigod \n" }, { "path": ".pre-commit-config.yaml", "content": "# This is the configuration file for pre-commit (https://pre-commit.com/).\n# To use:\n# * Install pre-commit (https://pre-commit.com/#installation)\n# * Copy this file as \".pre-commit-config.yaml\"\n# * Run \"pre-commit install\".\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v6.0.0\n hooks:\n - id: check-added-large-files\n - id: check-case-conflict\n - id: check-merge-conflict\n - id: check-symlinks\n - id: check-toml\n - id: check-yaml\n - id: debug-statements\n - id: end-of-file-fixer\n - id: mixed-line-ending\n args: [ \"--fix=lf\" ]\n - id: trailing-whitespace\n\n - repo: https://github.com/astral-sh/ruff-pre-commit\n rev: v0.14.10\n hooks:\n - id: ruff-check\n args: [--fix, --show-fixes]\n - id: ruff-format\n\n - repo: https://github.com/pre-commit/mirrors-mypy\n rev: v1.19.1\n hooks:\n - id: mypy\n additional_dependencies:\n - attrs == 23.2.0\n - pymongo == 4.8.0\n - redis == 5.0.7\n - sqlalchemy == 2.0.31\n - tzlocal == 5.2\n exclude: docs/conf.py\n stages: [manual]\n\n - repo: https://github.com/codespell-project/codespell\n rev: v2.4.1\n hooks:\n - id: codespell\n\n - repo: https://github.com/pre-commit/pygrep-hooks\n rev: v1.10.0\n hooks:\n - id: rst-backticks\n - id: rst-directive-colons\n - id: rst-inline-touching-normal\n\nci:\n autoupdate_schedule: quarterly\n" }, { "path": ".readthedocs.yml", "content": "version: 2\n\nbuild:\n os: ubuntu-22.04\n tools:\n python: \"3.12\"\n jobs:\n install:\n - python -m pip install --no-cache-dir \"pip >= 25.1\"\n - python -m pip install --upgrade --upgrade-strategy only-if-needed --no-cache-dir --group doc .\n\nsphinx:\n configuration: docs/conf.py\n fail_on_warning: true\n" }, { "path": "LICENSE.txt", "content": "The MIT License (MIT)\n\nCopyright (c) 2009 Alex Grönholm\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so,\nsubject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n" }, { "path": "README.rst", "content": ".. image:: https://github.com/agronholm/apscheduler/actions/workflows/test.yml/badge.svg\n :target: https://github.com/agronholm/apscheduler/actions/workflows/test.yml\n :alt: Build Status\n.. image:: https://coveralls.io/repos/github/agronholm/apscheduler/badge.svg?branch=master\n :target: https://coveralls.io/github/agronholm/apscheduler?branch=master\n :alt: Code Coverage\n.. image:: https://readthedocs.org/projects/apscheduler/badge/?version=latest\n :target: https://apscheduler.readthedocs.io/en/master/?badge=latest\n :alt: Documentation\n\n.. warning:: The v4.0 series is provided as a **pre-release** and may change in a\n backwards incompatible fashion without any migration pathway, so do NOT use this\n release in production!\n\nAdvanced Python Scheduler (APScheduler) is a task scheduler and task queue system for\nPython. It can be used solely as a job queuing system if you have no need for task\nscheduling. It scales both up and down, and is suitable for both trivial, single-process\nuse cases as well as large deployments spanning multiple nodes. Multiple schedulers and\nworkers can be deployed to use a shared data store to provide both a degree of high\navailability and horizontal scaling.\n\nAPScheduler comes in both synchronous and asynchronous flavors, making it a good fit for\nboth traditional, thread-based applications, and asynchronous (asyncio or Trio_)\napplications. Documentation and examples are provided for integrating with either WSGI_\nor ASGI_ compatible web applications.\n\nSupport is provided for persistent storage of schedules and jobs. This means that they\ncan be shared among multiple scheduler/worker instances and will survive process and\nnode restarts.\n\nThe built-in persistent data store back-ends are:\n\n* PostgreSQL\n* MySQL and derivatives\n* SQLite\n* MongoDB\n\nThe built-in event brokers (needed in scenarios with multiple schedulers and/or\nworkers):\n\n* PostgreSQL\n* Redis\n* MQTT\n\nThe built-in scheduling mechanisms (*triggers*) are:\n\n* Cron-style scheduling\n* Interval-based scheduling (runs tasks on even intervals)\n* Calendar-based scheduling (runs tasks on intervals of X years/months/weeks/days,\n always at the same time of day)\n* One-off scheduling (runs a task once, at a specific date/time)\n\nDifferent scheduling mechanisms can even be combined with so-called *combining triggers*\n(see the documentation_ for details).\n\nYou can also implement your custom scheduling logic by building your own trigger class.\nThese will be treated no differently than the built-in ones.\n\nOther notable features include:\n\n* You can limit the maximum number of simultaneous jobs for a given task (function)\n* You can limit the amount of time a job is allowed to start late\n* Jitter (adjustable, random delays added to the run time of each scheduled job)\n\n.. _Trio: https://pypi.org/project/trio/\n.. _WSGI: https://wsgi.readthedocs.io/en/latest/what.html\n.. _ASGI: https://asgi.readthedocs.io/en/latest/index.html\n.. _documentation: https://apscheduler.readthedocs.io/en/master/\n\nDocumentation\n=============\n\nDocumentation can be found\n`here `_.\n\nSource\n======\n\nThe source can be browsed at `Github `_.\n\nReporting bugs\n==============\n\nA `bug tracker `_ is provided by\nGitHub.\n\nGetting help\n============\n\nIf you have problems or other questions, you can either:\n\n* Ask in the `apscheduler `_ room on Gitter\n* Post a question on `GitHub discussions`_, or\n* Post a question on StackOverflow_ and add the ``apscheduler`` tag\n\n.. _GitHub discussions: https://github.com/agronholm/apscheduler/discussions/categories/q-a\n.. _StackOverflow: http://stackoverflow.com/questions/tagged/apscheduler\n" }, { "path": "docker-compose.yml", "content": "services:\n postgresql:\n image: postgres\n ports:\n - 127.0.0.1:5432:5432\n environment:\n POSTGRES_DB: testdb\n POSTGRES_PASSWORD: secret\n\n mysql:\n image: mysql\n ports:\n - 127.0.0.1:3306:3306\n environment:\n MYSQL_DATABASE: testdb\n MYSQL_ROOT_PASSWORD: secret\n\n mongodb:\n image: mongo\n ports:\n - 127.0.0.1:27017:27017\n\n emqx:\n image: emqx/emqx\n ports:\n - 127.0.0.1:1883:1883\n\n redis:\n image: redis\n ports:\n - 127.0.0.1:6379:6379\n" }, { "path": "docs/api.rst", "content": "API reference\n=============\n\nData structures\n---------------\n\n.. autoclass:: apscheduler.Task\n.. autoclass:: apscheduler.TaskDefaults\n.. autoclass:: apscheduler.Schedule\n.. autoclass:: apscheduler.ScheduleResult\n.. autoclass:: apscheduler.Job\n.. autoclass:: apscheduler.JobResult\n\nDecorators\n----------\n\n.. autodecorator:: apscheduler.task\n\nSchedulers\n----------\n\n.. autoclass:: apscheduler.Scheduler\n.. autoclass:: apscheduler.AsyncScheduler\n\nJob executors\n-------------\n\n.. autoclass:: apscheduler.abc.JobExecutor\n.. autoclass:: apscheduler.executors.async_.AsyncJobExecutor\n.. autoclass:: apscheduler.executors.subprocess.ProcessPoolJobExecutor\n.. autoclass:: apscheduler.executors.qt.QtJobExecutor\n.. autoclass:: apscheduler.executors.thread.ThreadPoolJobExecutor\n\nData stores\n-----------\n\n.. autoclass:: apscheduler.abc.DataStore\n.. autoclass:: apscheduler.datastores.memory.MemoryDataStore\n.. autoclass:: apscheduler.datastores.sqlalchemy.SQLAlchemyDataStore\n.. autoclass:: apscheduler.datastores.mongodb.MongoDBDataStore\n\nEvent brokers\n-------------\n\n.. autoclass:: apscheduler.abc.EventBroker\n.. autoclass:: apscheduler.abc.Subscription\n.. autoclass:: apscheduler.eventbrokers.local.LocalEventBroker\n.. autoclass:: apscheduler.eventbrokers.asyncpg.AsyncpgEventBroker\n.. autoclass:: apscheduler.eventbrokers.psycopg.PsycopgEventBroker\n.. autoclass:: apscheduler.eventbrokers.mqtt.MQTTEventBroker\n.. autoclass:: apscheduler.eventbrokers.redis.RedisEventBroker\n\nSerializers\n-----------\n\n.. autoclass:: apscheduler.abc.Serializer\n.. autoclass:: apscheduler.serializers.cbor.CBORSerializer\n.. autoclass:: apscheduler.serializers.json.JSONSerializer\n.. autoclass:: apscheduler.serializers.pickle.PickleSerializer\n\nTriggers\n--------\n\n.. autoclass:: apscheduler.abc.Trigger\n :special-members: __getstate__, __setstate__\n\n.. autoclass:: apscheduler.triggers.date.DateTrigger\n.. autoclass:: apscheduler.triggers.interval.IntervalTrigger\n.. autoclass:: apscheduler.triggers.calendarinterval.CalendarIntervalTrigger\n.. autoclass:: apscheduler.triggers.combining.AndTrigger\n.. autoclass:: apscheduler.triggers.combining.OrTrigger\n.. autoclass:: apscheduler.triggers.cron.CronTrigger\n\nEvents\n------\n\n.. autoclass:: apscheduler.Event\n.. autoclass:: apscheduler.DataStoreEvent\n.. autoclass:: apscheduler.TaskAdded\n.. autoclass:: apscheduler.TaskUpdated\n.. autoclass:: apscheduler.TaskRemoved\n.. autoclass:: apscheduler.ScheduleAdded\n.. autoclass:: apscheduler.ScheduleUpdated\n.. autoclass:: apscheduler.ScheduleRemoved\n.. autoclass:: apscheduler.JobAdded\n.. autoclass:: apscheduler.JobRemoved\n.. autoclass:: apscheduler.ScheduleDeserializationFailed\n.. autoclass:: apscheduler.JobDeserializationFailed\n.. autoclass:: apscheduler.SchedulerEvent\n.. autoclass:: apscheduler.SchedulerStarted\n.. autoclass:: apscheduler.SchedulerStopped\n.. autoclass:: apscheduler.JobAcquired\n.. autoclass:: apscheduler.JobReleased\n\nEnumerated types\n----------------\n\n.. autoclass:: apscheduler.SchedulerRole()\n :show-inheritance:\n\n.. autoclass:: apscheduler.RunState()\n :show-inheritance:\n\n.. autoclass:: apscheduler.JobOutcome()\n :show-inheritance:\n\n.. autoclass:: apscheduler.ConflictPolicy()\n :show-inheritance:\n\n.. autoclass:: apscheduler.CoalescePolicy()\n :show-inheritance:\n\nContext variables\n-----------------\n\nSee the :mod:`contextvars` module for information on how to work with context variables.\n\n.. data:: apscheduler.current_scheduler\n :type: ~contextvars.ContextVar[Scheduler]\n\n The current scheduler.\n\n.. data:: apscheduler.current_async_scheduler\n :type: ~contextvars.ContextVar[AsyncScheduler]\n\n The current asynchronous scheduler.\n\n.. data:: apscheduler.current_job\n :type: ~contextvars.ContextVar[Job]\n\n The job being currently run (available when running the job's target callable).\n\nExceptions\n----------\n\n.. autoexception:: apscheduler.TaskLookupError\n.. autoexception:: apscheduler.ScheduleLookupError\n.. autoexception:: apscheduler.JobLookupError\n.. autoexception:: apscheduler.CallableLookupError\n.. autoexception:: apscheduler.JobResultNotReady\n.. autoexception:: apscheduler.JobCancelled\n.. autoexception:: apscheduler.JobDeadlineMissed\n.. autoexception:: apscheduler.ConflictingIdError\n.. autoexception:: apscheduler.SerializationError\n.. autoexception:: apscheduler.DeserializationError\n.. autoexception:: apscheduler.MaxIterationsReached\n\nSupport classes for retrying failures\n-------------------------------------\n\n.. autoclass:: apscheduler.RetrySettings\n.. autoclass:: apscheduler.RetryMixin\n\nSupport classes for unset options\n---------------------------------\n\n.. data:: apscheduler.unset\n\n Sentinel value for unset option values.\n\n.. autoclass:: apscheduler.UnsetValue\n" }, { "path": "docs/conf.py", "content": "# Configuration file for the Sphinx documentation builder.\n#\n# This file only contains a selection of the most common options. For a full\n# list see the documentation:\n# https://www.sphinx-doc.org/en/master/usage/configuration.html\n\nfrom __future__ import annotations\n\n# -- Path setup --------------------------------------------------------------\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\n# import os\n# import sys\n# sys.path.insert(0, os.path.abspath('.'))\n\n\n# -- Project information -----------------------------------------------------\n\nproject = \"APScheduler\"\ncopyright = \"Alex Grönholm\"\nauthor = \"Alex Grönholm\"\n\n\n# -- General configuration ---------------------------------------------------\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\nextensions = [\n \"sphinx.ext.autodoc\",\n \"sphinx.ext.intersphinx\",\n \"sphinx_tabs.tabs\",\n \"sphinx_autodoc_typehints\",\n \"sphinx_rtd_theme\",\n]\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = [\"_templates\"]\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = []\n\nautodoc_default_options = {\"members\": True}\nautodoc_mock_imports = [\n \"asyncpg\",\n \"bson\",\n \"cbor2\",\n \"paho\",\n \"pymongo\",\n \"psycopg\",\n \"redis\",\n \"sqlalchemy\",\n \"PyQt6\",\n]\nautodoc_type_aliases = {\n \"datetime\": \"datetime.datetime\",\n \"UUID\": \"uuid.UUID\",\n \"AsyncEngine\": \"sqlalchemy.ext.asyncio.AsyncEngine\",\n \"RetrySettings\": \"apscheduler.RetrySettings\",\n \"Serializer\": \"apscheduler.abc.Serializer\",\n}\nnitpick_ignore = [(\"py:class\", \"datetime\")]\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\nhtml_theme = \"sphinx_rtd_theme\"\n\nintersphinx_mapping = {\n \"python\": (\"https://docs.python.org/3/\", None),\n \"anyio\": (\"https://anyio.readthedocs.io/en/latest/\", None),\n \"asyncpg\": (\"https://magicstack.github.io/asyncpg/current/\", None),\n \"cbor2\": (\"https://cbor2.readthedocs.io/en/latest/\", None),\n \"psycopg\": (\"https://www.psycopg.org/psycopg3/docs/\", None),\n \"pymongo\": (\"https://pymongo.readthedocs.io/en/stable\", None),\n \"sqlalchemy\": (\"https://docs.sqlalchemy.org/en/20/\", None),\n \"tenacity\": (\"https://tenacity.readthedocs.io/en/latest/\", None),\n}\n" }, { "path": "docs/contributing.rst", "content": "Contributing to APScheduler\n===========================\n\n.. highlight:: bash\n\nIf you wish to contribute a fix or feature to APScheduler, please follow the following\nguidelines.\n\nWhen you make a pull request against the main APScheduler codebase, Github runs the test\nsuite against your modified code. Before making a pull request, you should ensure that\nthe modified code passes tests and code quality checks locally.\n\nRunning the test suite\n----------------------\n\nThe test suite has dependencies on several external services, such as database servers.\nTo make this easy for the developer, a `docker compose`_ configuration is provided.\nTo use it, you need Docker_ (or a suitable replacement). On Linux, unless you're using\nDocker Desktop, you may need to also install the compose (v2) plugin (named\n``docker-compose-plugin``, or similar) separately.\n\nOnce you have the necessary tools installed, you can start the services with this\ncommand::\n\n docker compose up -d\n\nYou can run the test suite two ways: either with tox_, or by running pytest_ directly.\n\nTo run tox_ against all supported (of those present on your system) Python versions::\n\n tox\n\nTox will handle the installation of dependencies in separate virtual environments.\n\nTo pass arguments to the underlying pytest_ command, you can add them after ``--``, like\nthis::\n\n tox -- -k somekeyword\n\nTo use pytest directly, you can set up a virtual environment and install the project in\ndevelopment mode along with its test dependencies (virtualenv activation demonstrated\nfor Linux and macOS; on Windows you need ``venv\\Scripts\\activate`` instead)::\n\n python -m venv venv\n source venv/bin/activate\n pip install --group test -e .\n\nNow you can just run pytest_::\n\n pytest\n\nBuilding the documentation\n--------------------------\n\nTo build the documentation, run ``tox -e docs``. This will place the documentation in\n``build/sphinx/html`` where you can open ``index.html`` to view the formatted\ndocumentation.\n\nAPScheduler uses ReadTheDocs_ to automatically build the documentation so the above\nprocedure is only necessary if you are modifying the documentation and wish to check the\nresults before committing.\n\nAPScheduler uses pre-commit_ to perform several code style/quality checks. It is\nrecommended to activate pre-commit_ on your local clone of the repository (using\n``pre-commit install``) to ensure that your changes will pass the same checks on GitHub.\n\nMaking a pull request on Github\n-------------------------------\n\nTo get your changes merged to the main codebase, you need a Github account.\n\n#. Fork the repository (if you don't have your own fork of it yet) by navigating to the\n `main APScheduler repository`_ and clicking on \"Fork\" near the top right corner.\n#. Clone the forked repository to your local machine with\n ``git clone git@github.com/yourusername/apscheduler``.\n#. Create a branch for your pull request, like ``git checkout -b myfixname``\n#. Make the desired changes to the code base.\n#. Commit your changes locally. If your changes close an existing issue, add the text\n ``Fixes #XXX.`` or ``Closes #XXX.`` to the commit message (where XXX is the issue\n number).\n#. Push the changeset(s) to your forked repository (``git push``)\n#. Navigate to Pull requests page on the original repository (not your fork) and click\n \"New pull request\"\n#. Click on the text \"compare across forks\".\n#. Select your own fork as the head repository and then select the correct branch name.\n#. Click on \"Create pull request\".\n\nIf you have trouble, consult the `pull request making guide`_ on opensource.com.\n\n.. _Docker: https://docs.docker.com/desktop/#download-and-install\n.. _docker compose: https://docs.docker.com/compose/\n.. _tox: https://tox.readthedocs.io/en/latest/install.html\n.. _pre-commit: https://pre-commit.com/#installation\n.. _pytest: https://pypi.org/project/pytest/\n.. _ReadTheDocs: https://readthedocs.org/\n.. _main APScheduler repository: https://github.com/agronholm/apscheduler\n.. _pull request making guide: https://opensource.com/article/19/7/create-pull-request-github\n" }, { "path": "docs/extending.rst", "content": "#####################\nExtending APScheduler\n#####################\n\n.. py:currentmodule:: apscheduler\n\nThis document is meant to explain how to develop your custom triggers and data stores.\n\nCustom triggers\n---------------\n\nThe built-in triggers cover the needs of the majority of all users, particularly so when\ncombined using :class:`~triggers.combining.AndTrigger` and\n:class:`~triggers.combining.OrTrigger`. However, some users may need specialized\nscheduling logic. This can be accomplished by creating your own custom trigger class.\n\nTo implement your scheduling logic, create a new class that inherits from the\n:class:`~abc.Trigger` interface class::\n\n from __future__ import annotations\n\n from apscheduler.abc import Trigger\n\n class MyCustomTrigger(Trigger):\n def next() -> datetime | None:\n ... # Your custom logic here\n\n def __getstate__():\n ... # Return the serializable state here\n\n def __setstate__(state):\n ... # Restore the state from the return value of __getstate__()\n\nRequirements and constraints for trigger classes:\n\n* :meth:`~abc.Trigger.next` must always either return a timezone aware\n :class:`~datetime.datetime` object or :data:`None` if a new run time cannot be\n calculated\n* :meth:`~abc.Trigger.next` must never return the same :class:`~datetime.datetime`\n twice and never one that is earlier than the previously returned one\n* :meth:`~abc.Trigger.__setstate__` must accept the return value of\n :meth:`~abc.Trigger.__getstate__` and restore the trigger to the functionally same\n state as the original\n* :meth:`~abc.Trigger.__getstate__` may only return an object containing types\n serializable by :class:`~abc.Serializer`\n\nTriggers are stateful objects. The :meth:`~abc.Trigger.next` method is where you\ndetermine the next run time based on the current state of the trigger. The trigger's\ninternal state needs to be updated before returning to ensure that the trigger won't\nreturn the same datetime on the next call. The trigger code does **not** need to be\nthread-safe.\n\nCustom job executors\n--------------------\n\n.. py:currentmodule:: apscheduler\n\nIf you need the ability to use third party frameworks or services to handle the\nactual execution of jobs, you will need a custom job executor.\n\nA job executor needs to inherit from :class:`~abc.JobExecutor`. This interface contains\none abstract method you're required to implement: :meth:`~abc.JobExecutor.run_job`.\nThis method is called with two arguments:\n\n#. ``func``: the callable you're supposed to call\n#. ``job``: the :class:`Job` instance\n\nThe :meth:`~abc.JobExecutor.run_job` implementation needs to call ``func`` with the\npositional and keyword arguments attached to the job (``job.args`` and ``job.kwargs``,\nrespectively). The return value of the callable must be returned from the method.\n\nHere's an example of a simple job executor that runs a (synchronous) callable in a\nthread::\n\n from contextlib import AsyncExitStack\n from functools import partial\n\n from anyio import to_thread\n from apscheduler import Job\n from apscheduler.abc import JobExecutor\n\n class ThreadJobExecutor(JobExecutor):\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n wrapped = partial(func, *job.args, **job.kwargs)\n return await to_thread.run_sync(wrapped)\n\nIf you need to initialize some underlying services, you can override the\n:meth:`~abc.JobExecutor.start` method. For example, the executor above could be improved\nto take a maximum number of threads and create an AnyIO\n:class:`~anyio.CapacityLimiter`::\n\n from contextlib import AsyncExitStack\n from functools import partial\n\n from anyio import CapacityLimiter, to_thread\n from apscheduler import Job\n from apscheduler.abc import JobExecutor\n\n class ThreadJobExecutor(JobExecutor):\n _limiter: CapacityLimiter\n\n def __init__(self, max_threads: int):\n self.max_threads = max_threads\n\n async def start(self, exit_stack: AsyncExitStack) -> None:\n self._limiter = CapacityLimiter(self.max_workers)\n\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n wrapped = partial(func, *job.args, **job.kwargs)\n return await to_thread.run_sync(wrapped, limiter=self._limiter)\n\nCustom data stores\n------------------\n\nIf you want to make use of some external service to store the scheduler data, and it's\nnot covered by a built-in data store implementation, you may want to create a custom\ndata store class.\n\nA data store implementation needs to inherit from :class:`~abc.DataStore` and implement\nseveral abstract methods:\n\n* :meth:`~abc.DataStore.start`\n* :meth:`~abc.DataStore.add_task`\n* :meth:`~abc.DataStore.remove_task`\n* :meth:`~abc.DataStore.get_task`\n* :meth:`~abc.DataStore.get_tasks`\n* :meth:`~abc.DataStore.add_schedule`\n* :meth:`~abc.DataStore.remove_schedules`\n* :meth:`~abc.DataStore.get_schedules`\n* :meth:`~abc.DataStore.acquire_schedules`\n* :meth:`~abc.DataStore.release_schedules`\n* :meth:`~abc.DataStore.get_next_schedule_run_time`\n* :meth:`~abc.DataStore.add_job`\n* :meth:`~abc.DataStore.get_jobs`\n* :meth:`~abc.DataStore.acquire_jobs`\n* :meth:`~abc.DataStore.release_job`\n* :meth:`~abc.DataStore.get_job_result`\n* :meth:`~abc.DataStore.extend_acquired_schedule_leases`\n* :meth:`~abc.DataStore.extend_acquired_job_leases`\n* :meth:`~abc.DataStore.cleanup`\n\nThe :meth:`~abc.DataStore.start` method is where your implementation can perform any\ninitialization, including starting any background tasks. This method is called with two\narguments:\n\n#. ``exit_stack``: an :class:`~contextlib.AsyncExitStack` object that can be used to\n work with context managers\n#. ``event_broker``: the event broker that the store should be using to send events to\n other components of the system (including other schedulers)\n\nThe data store class needs to inherit from :class:`~abc.DataStore`::\n\n from contextlib import AsyncExitStack\n\n from apscheduler.abc import DataStore, EventBroker\n\n class MyCustomDataStore(DataStore):\n _event_broker: EventBroker\n\n async def start(self, exit_stack: AsyncExitStack, event_broker: EventBroker) -> None:\n # Save the event broker in a member attribute and initialize the store\n self._event_broker = event_broker\n\n # See the interface class for the rest of the abstract methods\n\nHandling temporary failures\n+++++++++++++++++++++++++++\n\nIf you plan to make your data store implementation public, it is strongly recommended\nthat you make an effort to ensure that the implementation can tolerate the loss of\nconnectivity to the backing store. The Tenacity_ library is used for this purpose by the\nbuilt-in stores to retry operations in case of a disconnection. If you use it to retry\noperations when exceptions are raised, it is important to only do that in cases of\n*temporary* errors, like connectivity loss, and not in cases like authentication\nfailure, missing database and so forth. See the built-in data store implementations and\nTenacity_ documentation for more information on how to pick the exceptions on which to\nretry the operations.\n\n.. _Tenacity: https://pypi.org/project/tenacity/\n" }, { "path": "docs/faq.rst", "content": "##########################\nFrequently Asked Questions\n##########################\n\nIs there a graphical user interface for APScheduler?\n====================================================\n\nNo graphical interface is provided by the library itself. However, there are some third\nparty implementations, but APScheduler developers are not responsible for them. Here is\na potentially incomplete list:\n\n* django_apscheduler_\n* apschedulerweb_\n* `Nextdoor scheduler`_\n\n.. warning:: As of this writing, these third party offerings have not been updated to\n work with APScheduler 4.\n\n.. _django_apscheduler: https://pypi.org/project/django-apscheduler/\n.. _Flask-APScheduler: https://pypi.org/project/flask-apscheduler/\n.. _aiohttp: https://pypi.org/project/aiohttp/\n.. _apschedulerweb: https://github.com/marwinxxii/apschedulerweb\n.. _Nextdoor scheduler: https://github.com/Nextdoor/ndscheduler\n" }, { "path": "docs/index.rst", "content": "Advanced Python Scheduler\n=========================\n\n.. include:: ../README.rst\n :end-before: Documentation\n\n\nTable of Contents\n=================\n\n.. toctree::\n :maxdepth: 1\n\n userguide\n integrations\n versionhistory\n migration\n contributing\n extending\n faq\n api\n" }, { "path": "docs/integrations.rst", "content": "Integrating with application frameworks\n=======================================\n\n.. py:currentmodule:: apscheduler\n\nWSGI\n----\n\nTo integrate APScheduler with web frameworks using WSGI_ (Web Server Gateway Interface),\nyou need to use the synchronous scheduler and start it as a side effect of importing the\nmodule that contains your application instance::\n\n from apscheduler import Scheduler\n\n\n def app(environ, start_response):\n \"\"\"Trivial example of a WSGI application.\"\"\"\n response_body = b\"Hello, World!\"\n response_headers = [\n (\"Content-Type\", \"text/plain\"),\n (\"Content-Length\", str(len(response_body))),\n ]\n start_response(200, response_headers)\n return [response_body]\n\n\n scheduler = Scheduler()\n scheduler.start_in_background()\n\nAssuming you saved this as ``example.py``, you can now start the application with uWSGI_\nwith:\n\n.. code-block:: bash\n\n uwsgi --enable-threads --http :8080 --wsgi-file example.py\n\nThe ``--enable-threads`` (or ``-T``) option is necessary because uWSGI disables threads\nby default which then prevents the scheduler from working. See the\n`uWSGI documentation `_ for more details.\n\n.. note::\n The :meth:`Scheduler.start_in_background` method installs an\n :mod:`atexit` hook that shuts down the scheduler gracefully when the worker process\n exits.\n\n.. _WSGI: https://wsgi.readthedocs.io/en/latest/what.html\n.. _uWSGI: https://www.fullstackpython.com/uwsgi.html\n.. _uWSGI-threads: https://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html#a-note-on-python-threads\n\nASGI\n----\n\nTo integrate APScheduler with web frameworks using ASGI_ (Asynchronous Server Gateway\nInterface), you need to use the asynchronous scheduler and tie its lifespan to the\nlifespan of the application by wrapping it in middleware, as follows::\n\n from apscheduler import AsyncScheduler\n\n\n async def app(scope, receive, send):\n \"\"\"Trivial example of an ASGI application.\"\"\"\n if scope[\"type\"] == \"http\":\n await receive()\n await send(\n {\n \"type\": \"http.response.start\",\n \"status\": 200,\n \"headers\": [\n [b\"content-type\", b\"text/plain\"],\n ],\n }\n )\n await send(\n {\n \"type\": \"http.response.body\",\n \"body\": b\"Hello, world!\",\n \"more_body\": False,\n }\n )\n elif scope[\"type\"] == \"lifespan\":\n while True:\n message = await receive()\n if message[\"type\"] == \"lifespan.startup\":\n await send({\"type\": \"lifespan.startup.complete\"})\n elif message[\"type\"] == \"lifespan.shutdown\":\n await send({\"type\": \"lifespan.shutdown.complete\"})\n return\n\n\n async def scheduler_middleware(scope, receive, send):\n if scope['type'] == 'lifespan':\n async with AsyncScheduler() as scheduler:\n await app(scope, receive, send)\n else:\n await app(scope, receive, send)\n\nAssuming you saved this as ``example.py``, you can then run this with Hypercorn_:\n\n.. code-block:: bash\n\n hypercorn example:scheduler_middleware\n\nor with Uvicorn_:\n\n.. code-block:: bash\n\n uvicorn example:scheduler_middleware\n\n.. _ASGI: https://asgi.readthedocs.io/en/latest/index.html\n.. _Hypercorn: https://gitlab.com/pgjones/hypercorn/\n.. _Uvicorn: https://www.uvicorn.org/\n" }, { "path": "docs/migration.rst", "content": "###############################################\nMigrating from previous versions of APScheduler\n###############################################\n\n.. py:currentmodule:: apscheduler\n\nFrom v3.x to v4.0\n=================\n\nAPScheduler 4.0 has undergone a partial rewrite since the 3.x series.\n\nThere is currently no way to automatically import schedules from a persistent 3.x job\nstore, but this shortcoming will be rectified before the final v4.0 release.\n\nTerminology and architectural design changes\n--------------------------------------------\n\nThe concept of a *job* has been split into :class:`Task`, :class:`Schedule` and\n:class:`Job`. See the documentation of each class (and read the tutorial) to understand\ntheir roles.\n\n**Data stores**, previously called *job stores*, have been redesigned to work with\nmultiple running schedulers and workers, both for purposes of scalability and fault\ntolerance. Many data store implementations were dropped because they were either too\nburdensome to support, or the backing services were not sophisticated enough to handle\nthe increased requirements.\n\n**Event brokers** are a new component in v4.0. They relay events between schedulers and\nworkers, enabling them to work together with a shared data store. External (as opposed\nto local) event broker services are required in multi-node or multi-process deployment\nscenarios.\n\n**Triggers** are now stateful. This change was found to be necessary to properly support\ncombining triggers (:class:`~.triggers.combining.AndTrigger` and\n:class:`~.triggers.combining.OrTrigger`), as they needed to keep track of the next run\ntimes of all the triggers contained within. This change also enables some more\nsophisticated custom trigger implementations.\n\n**Time zone** support has been revamped to use :mod:`zoneinfo` (or `backports.zoneinfo`_\non Python versions earlier than 3.9) zones instead of pytz zones. You should not use\npytz with APScheduler anymore.\n\n`Entry points`_ are no longer used or supported, as they were more trouble than they\nwere worth, particularly with packagers like py2exe or PyInstaller which by default did\nnot package distribution metadata. Thus, triggers and data stores have to be explicitly\ninstantiated.\n\n.. _backports.zoneinfo: https://pypi.org/project/backports.zoneinfo/\n.. _Entry points: https://packaging.python.org/en/latest/specifications/entry-points/\n\nScheduler changes\n-----------------\n\nThe ``add_job()`` method is now :meth:`~Scheduler.add_schedule`. The scheduler still has\na method named :meth:`~Scheduler.add_job`, but this is meant for making one-off runs of\na task. Previously you would have had to call ``add_job()`` with a\n:class:`~triggers.date.DateTrigger` using the current time as the run time.\n\nThe two most commonly used schedulers, ``BlockingScheduler`` and\n``BackgroundScheduler``, have often caused confusion among users and have thus been\ncombined into :class:`~Scheduler`. This new unified scheduler class has two methods that\nreplace the ``start()`` method used previously: :meth:`~Scheduler.run_until_stopped` and\n:meth:`~Scheduler.start_in_background`. The former should be used if you previously used\n``BlockingScheduler``, and the latter if you used ``BackgroundScheduler``.\n\nThe asyncio scheduler has been replaced with a more generic :class:`AsyncScheduler`,\nwhich is based on AnyIO_ and thus also supports Trio_ in addition to :mod:`asyncio`.\nThe API of the async scheduler differs somewhat from its synchronous counterpart. In\nparticular, it **requires** itself to be used as an async context manager – whereas with\nthe synchronous scheduler, use as a context manager is recommended but not required.\n\nAll other scheduler implementations have been dropped because they were either too\nburdensome to support, or did not seem necessary anymore. Some of the dropped\nimplementations (particularly Qt) are likely to be re-added before v4.0 final.\n\nSchedulers no longer support multiple data stores. If you need this capability, you\nshould run multiple schedulers instead.\n\nConfiguring and running the scheduler has been radically simplified. The ``configure()``\nmethod is gone, and all configuration is now passed as keyword arguments to the\nscheduler class.\n\n.. _AnyIO: https://pypi.org/project/anyio/\n.. _Trio: https://pypi.org/project/trio/\n\nTrigger changes\n---------------\n\nAs the scheduler is no longer used to create triggers, any supplied datetimes will be\nassumed to be in the local time zone. If you wish to change the local time zone, you\nshould set the ``TZ`` environment variable to either the name of the desired timezone\n(e.g. ``Europe/Helsinki``) or to a path of a time zone file. See the tzlocal_\ndocumentation for more information.\n\n**Jitter** support has been moved from individual triggers to the schedule level.\nThis not only simplified trigger design, but also enabled the scheduler to provide\ninformation about the randomized jitter and the original run time to the user.\n\n:class:`~triggers.cron.CronTrigger` was changed to respect the standard order of\nweekdays, so that Sunday is now 0 and Saturday is 6. If you used numbered weekdays\nbefore, you must change your trigger configuration to match. If in doubt, use\nabbreviated weekday names (e.g. ``sun``, ``fri``) instead.\n\n:class:`~triggers.interval.IntervalTrigger` was changed to start immediately, instead\nof waiting for the first interval to pass. If you have workarounds in place to \"fix\"\nthe previous behavior, you should remove them.\n\n.. _tzlocal: https://pypi.org/project/tzlocal/\n\nFrom v3.0 to v3.2\n=================\n\nPrior to v3.1, the scheduler inadvertently exposed the ability to fetch and manipulate\njobs before the scheduler had been started. The scheduler now requires you to call\n``scheduler.start()`` before attempting to access any of the jobs in the job stores. To\nensure that no old jobs are mistakenly executed, you can start the scheduler in paused\nmode (``scheduler.start(paused=True)``) (introduced in v3.2) to avoid any premature job\nprocessing.\n\n\nFrom v2.x to v3.0\n=================\n\nThe 3.0 series is API incompatible with previous releases due to a design overhaul.\n\nScheduler changes\n-----------------\n\n* The concept of \"standalone mode\" is gone. For ``standalone=True``, use\n ``BlockingScheduler`` instead, and for ``standalone=False``, use\n ``BackgroundScheduler``. BackgroundScheduler matches the old default semantics.\n* Job defaults (like ``misfire_grace_time`` and ``coalesce``) must now be passed in a\n dictionary as the ``job_defaults`` option to ``BaseScheduler.configure()``. When\n supplying an ini-style configuration as the first argument, they will need a\n corresponding ``job_defaults.`` prefix.\n* The configuration key prefix for job stores was changed from ``jobstore.`` to\n ``jobstores.`` to match the dict-style configuration better.\n* The ``max_runs`` option has been dropped since the run counter could not be reliably\n preserved when replacing a job with another one with the same ID. To make up for this,\n the ``end_date`` option was added to cron and interval triggers.\n* The old thread pool is gone, replaced by ``ThreadPoolExecutor``.\n This means that the old ``threadpool`` options are no longer valid.\n* The trigger-specific scheduling methods have been removed entirely from the scheduler.\n Use the generic ``BaseScheduler.add_job()`` method or the\n ``@BaseScheduler.scheduled_job`` decorator instead. The signatures of these methods\n were changed significantly.\n* The ``shutdown_threadpool`` and ``close_jobstores`` options have been removed from the\n ``BaseScheduler.shutdown()`` method.\n Executors and job stores are now always shut down on scheduler shutdown.\n* ``Scheduler.unschedule_job()`` and ``Scheduler.unschedule_func()`` have been replaced\n by ``BaseScheduler.remove_job()``. You can also unschedule a job by using the job\n handle returned from ``BaseScheduler.add_job()``.\n\nJob store changes\n-----------------\n\nThe job store system was completely overhauled for both efficiency and forwards\ncompatibility. Unfortunately, this means that the old data is not compatible with the\nnew job stores. If you need to migrate existing data from APScheduler 2.x to 3.x,\ncontact the APScheduler author.\n\nThe Shelve job store had to be dropped because it could not support the new job store\ndesign. Use SQLAlchemyJobStore with SQLite instead.\n\nTrigger changes\n---------------\n\nFrom 3.0 onwards, triggers now require a pytz timezone. This is normally provided by the\nscheduler, but if you were instantiating triggers manually before, then one must be\nsupplied as the ``timezone`` argument.\n\nThe only other backwards incompatible change was that ``get_next_fire_time()`` takes two\narguments now: the previous fire time and the current datetime.\n\n\nFrom v1.x to 2.0\n================\n\nThere have been some API changes since the 1.x series. This document\nexplains the changes made to v2.0 that are incompatible with the v1.x API.\n\nAPI changes\n-----------\n\n* The behavior of cron scheduling with regards to default values for omitted\n fields has been made more intuitive -- omitted fields lower than the least\n significant explicitly defined field will default to their minimum values\n except for the week number and weekday fields\n* SchedulerShutdownError has been removed -- jobs are now added tentatively\n and scheduled for real when/if the scheduler is restarted\n* Scheduler.is_job_active() has been removed -- use\n ``job in scheduler.get_jobs()`` instead\n* dump_jobs() is now print_jobs() and prints directly to the given file or\n sys.stdout if none is given\n* The ``repeat`` parameter was removed from\n ``Scheduler.add_interval_job()`` and ``@Scheduler.interval_schedule`` in favor of the\n universal ``max_runs`` option\n* ``Scheduler.unschedule_func()`` now raises a :exc:`KeyError` if the given function is\n not scheduled\n* The semantics of ``Scheduler.shutdown()`` have changed – the method no longer accepts\n a numeric argument, but two booleans\n\n\nConfiguration changes\n---------------------\n\n* The scheduler can no longer be reconfigured while it's running\n" }, { "path": "docs/userguide.rst", "content": "##########\nUser guide\n##########\n\n.. py:currentmodule:: apscheduler\n\nInstallation\n============\n\nThe preferred installation method is by using\n`pip `_::\n\n $ pip install apscheduler\n\nIf you don't have pip installed, you need to\n`install that first `_.\n\nInterfacing with certain external services may need extra dependencies which are\ninstallable as extras:\n\n* ``asyncpg``: for the AsyncPG event broker\n* ``cbor``: for the CBOR serializer\n* ``mongodb``: for the MongoDB data store\n* ``mqtt``: for the MQTT event broker\n* ``psycopg``: for the Psycopg event broker\n* ``redis``: for the Redis event broker\n* ``sqlalchemy``: for the SQLAlchemy data store\n\nUsing the extras instead of adding the corresponding libraries separately helps ensure\nthat you will have compatible versions of the dependent libraries going forward.\n\nYou can install any number of these extras with APScheduler by providing them as a comma\nseparated list inside the brackets, like this::\n\n pip install apscheduler[psycopg,sqlalchemy]\n\nCode examples\n=============\n\nThe source distribution contains the :file:`examples` directory where you can find many\nworking examples for using APScheduler in different ways. The examples can also be\n`browsed online\n`_.\n\n\nIntroduction\n============\n\nThe core concept of APScheduler is to give the user the ability to queue Python code to\nbe executed, either as soon as possible, later at a given time, or on a recurring\nschedule.\n\nThe *scheduler* is the user-facing interface of the system. When it's running, it does\ntwo things concurrently. The first is processing *schedules*. From its *data store*,\nit fetches :ref:`schedules ` due to be run. For each such schedule, it then\nuses the schedule's trigger_ to calculate run times up to the present. The scheduler\nthen creates one or more jobs (controllable by configuration) based on these run times\nand adds them to the data store.\n\nThe second role of the scheduler is running :ref:`jobs `. The scheduler asks the\n`data store`_ for jobs, and then starts running those jobs. If the data store signals\nthat it has new jobs, the scheduler will try to acquire those jobs if it is capable of\naccommodating more. When a scheduler completes a job, it will then also ask the data\nstore for as many more jobs as it can handle.\n\nBy default, schedulers operate in both of these roles, but can be configured to only\nprocess schedules or run jobs if deemed necessary. It may even be desirable to use the\nscheduler only as an interface to an external data store while leaving schedule and job\nprocessing to other scheduler instances running elsewhere.\n\nBasic concepts / glossary\n=========================\n\nThese are the basic components and concepts of APScheduler which will be referenced\nlater in this guide.\n\n.. _callable:\n\nA *callable* is any object that returns ``True`` from :func:`callable`. These are:\n\n* A free function (``def something(...): ...``)\n* An instance method (``class Foo: ... def something(self, ...): ...``)\n* A class method (``class Foo: ... @classmethod ... def something(cls, ...): ...``)\n* A static method (``class Foo: ... @staticmethod ... def something(...): ...``)\n* A lambda (``lambda a, b: a + b``)\n* An instance of a class that contains a method named ``__call__``)\n\n.. _task:\n\nA *task* encapsulates a callable_ and a number of configuration parameters. They are\noften implicitly defined as a side effect of the user creating a new schedule against a\ncallable_, but can also be :ref:`explicitly defined beforehand `.\n\nTasks have three different roles:\n\n#. They provide the target callable to be run when a job is started\n#. They provide a key (task ID) on which to limit the maximum number of concurrent jobs,\n even between different schedules\n#. They provide a template from which certain parameters, like job executor and misfire\n grace time, are copied to schedules and jobs derived from the task\n\n.. _trigger:\n\nA trigger_ contains the logic and state used to calculate when a scheduled task_ should\nbe run.\n\n.. _schedule:\n\nA *schedule* combines a task_ with a trigger_, plus a number of configuration\nparameters.\n\n.. _job:\n\nA *job* is request for a task_ to be run. It can be created automatically from a\nschedule when a scheduler processes it, or it can be directly created by the user if\nthey directly request a task_ to be run.\n\n.. _data store:\n\nA *data store* is used to store :ref:`schedules ` and :ref:`jobs `, and\nto keep track of :ref:`tasks `.\n\n.. _job executor:\n\nA *job executor* runs the job_, by calling the function associated with the job's task.\nAn executor could directly call the callable_, or do it in another thread, subprocess or\neven some external service.\n\n.. _event broker:\n\nAn *event broker* delivers published events to all interested parties. It facilitates\nthe cooperation between schedulers by notifying them of new or updated\n:ref:`schedules ` and :ref:`jobs `.\n\n.. _scheduler:\n\nA *scheduler* is the main interface of this library. It houses both a `data store`_ and\nan `event broker`_, plus one or more :ref:`job executors `. It contains\nmethods users can use to work with tasks, schedules and jobs. Behind the scenes, it also\nprocesses due schedules, spawning jobs and updating the next run times. It also\nprocesses available jobs, making the appropriate :ref:`job executors ` to\nrun them, and then sending back the results to the `data store`_.\n\nRunning the scheduler\n=====================\n\nThe scheduler_ comes in two flavors: synchronous and asynchronous. The synchronous\nscheduler actually runs an asynchronous scheduler behind the scenes in a dedicated\nthread, so if your app runs on :mod:`asyncio` or Trio_, you should prefer the\nasynchronous scheduler.\n\nThe scheduler can run either in the foreground, blocking on a call to\n:meth:`~Scheduler.run_until_stopped`, or in the background where it does its work while\nletting the rest of the program run.\n\nIf the only intent of your program is to run scheduled tasks, then you should start the\nscheduler with :meth:`~Scheduler.run_until_stopped`. But if you need to do other things\ntoo, then you should call :meth:`~Scheduler.start_in_background` before running the rest\nof the program.\n\nIn almost all cases, the scheduler should be used as a context manager. This initializes\nthe underlying `data store`_ and `event broker`_, allowing you to use the scheduler for\nmanipulating :ref:`tasks `, :ref:`schedules ` and jobs prior to starting\nthe processing of schedules and jobs. Exiting the context manager will shut down the\nscheduler and its underlying services. This mode of operation is mandatory for the\nasynchronous scheduler when running it in the background, but it is preferred for the\nsynchronous scheduler too.\n\nAs a special consideration (for use with WSGI_ based web frameworks), the synchronous\nscheduler can be run in the background without being used as a context manager. In this\nscenario, the scheduler adds an :mod:`atexit` hook that will perform an orderly shutdown\nof the scheduler before the process terminates.\n\n.. _WSGI: https://wsgi.readthedocs.io/en/latest/what.html\n.. _Trio: https://trio.readthedocs.io/en/stable/\n\n.. warning:: If you start the scheduler in the background and let the script finish\n execution, the scheduler will automatically shut down as well.\n\n.. tabs::\n\n .. code-tab:: python Synchronous (run in foreground)\n\n from apscheduler import Scheduler\n\n with Scheduler() as scheduler:\n # Add schedules, configure tasks here\n scheduler.run_until_stopped()\n\n .. code-tab:: python Synchronous (background thread; preferred method)\n\n from apscheduler import Scheduler\n\n with Scheduler() as scheduler:\n # Add schedules, configure tasks here\n scheduler.start_in_background()\n\n .. code-tab:: python Synchronous (background thread; WSGI alternative)\n\n from apscheduler import Scheduler\n\n scheduler = Scheduler()\n # Add schedules, configure tasks here\n scheduler.start_in_background()\n\n .. code-tab:: python Asynchronous (run in foreground)\n\n import asyncio\n\n from apscheduler import AsyncScheduler\n\n async def main():\n async with AsyncScheduler() as scheduler:\n # Add schedules, configure tasks here\n await scheduler.run_until_stopped()\n\n asyncio.run(main())\n\n .. code-tab:: python Asynchronous (background task)\n\n import asyncio\n\n from apscheduler import AsyncScheduler\n\n async def main():\n async with AsyncScheduler() as scheduler:\n # Add schedules, configure tasks here\n await scheduler.start_in_background()\n\n asyncio.run(main())\n\n.. _configuring-tasks:\n\nConfiguring tasks\n=================\n\nIn order to add :ref:`schedules ` or :ref:`jobs ` to the `data store`_,\nyou need to have a task_ that defines which callable_ will be called when each job_ is\nrun.\n\nIn most cases, you don't need to go through this step, and instead have a task_\nimplicitly created for you by the methods that add schedules or jobs.\n\nExplicitly configuring a task is generally only necessary in the following cases:\n\n* You need to have more than one task with the same callable\n* You need to set any of the task settings to non-default values\n* You need to add schedules/jobs targeting lambdas, nested functions or instances of\n unserializable classes\n\nThere are two ways to explicitly configure tasks:\n\n#. Call the :meth:`~Scheduler.configure_task` scheduler method\n#. Decorate your target function with :func:`@task `\n\n.. seealso:: :ref:`settings_inheritance`\n\nLimiting the number of concurrently executing instances of a job\n----------------------------------------------------------------\n\n**Option**: ``max_running_jobs``\n\nIt is possible to control the maximum number of concurrently running jobs for a\nparticular task. By default, only one job is allowed to be run for every task.\nThis means that if the job is about to be run but there is another job for the same task\nstill running, the later job is terminated with the outcome of\n:attr:`~JobOutcome.missed_start_deadline`.\n\nTo allow more jobs to be concurrently running for a task, pass the desired maximum\nnumber as the ``max_running_jobs`` keyword argument to :meth:`~Scheduler.add_schedule`.\n\n.. _controlling-how-much-a-job-can-be-started-late:\n\nControlling how much a job can be started late\n----------------------------------------------\n\n**Option**: ``misfire_grace_time``\n\nThis option applies to scheduled jobs. Some tasks are time sensitive, and should not be\nrun at all if they fail to be started on time (like, for example, if the scheduler(s)\nwere down while they were supposed to be running the scheduled jobs). When a scheduler\nacquires jobs, the data store discards any jobs that have passed their start deadlines\n(scheduled time + ``misfire_grace_time``). Such jobs are released with the outcome of\n:attr:`~JobOutcome.missed_start_deadline`.\n\nAdding custom metadata\n----------------------\n\n**Option**: ``metadata``\n\nThis option allows adding custom, JSON compatible metadata to tasks, schedules and jobs.\nHere, \"JSON compatible\" means the following restrictions:\n\n* The top-level metadata object must be a :class:`dict`\n* All :class:`dict` keys must be strings\n* Values can be :class:`int`, :class:`float`, :class:`str`, :class:`bool` or\n :data:`None`\n\n.. note:: Top level metadata keys are merged with any explicitly passed values, in such\n a way that explicitly passed values override any values from the task level.\n\n.. _settings_inheritance:\n\nInheritance of settings\n-----------------------\n\nWhen tasks are configured, or schedules or jobs created, they will inherit the settings\nof any \"parent\" object according to the following rules:\n\n* Task configuration parameters are resolving according to the following, descending\n priority order:\n\n #. Parameters passed directly to :meth:`~AsyncScheduler.configure_task`\n #. Parameters bound to the target function via :func:`@task `\n #. The scheduler's task defaults\n* Schedules inherit settings from the their respective tasks\n* Jobs created from schedules inherit the settings from their parent schedules\n* Jobs created directly inherit the settings from their parent tasks\n\nThe ``metadata`` parameter works a bit differently. Top level keys will be merged in\nsuch a way that keys on a more explicit configuration level keys will overwrite keys\nfrom a more generic level.\n\nIf any parameter is unset, it will be looked up on the next level. Here is an example\nthat illustrates the lookup order::\n\n from apscheduler import Scheduler, TaskDefaults, task\n\n @task(max_running_jobs=3, metadata={\"foo\": [\"taskfunc\"]})\n def mytaskfunc():\n print(\"running stuff\")\n\n task_defaults = TaskDefaults(\n misfire_grace_time=15,\n job_executor=\"processpool\",\n metadata={\"global\": 3, \"foo\": [\"bar\"]}\n )\n with Scheduler(task_defaults=task_defaults) as scheduler:\n scheduler.configure_task(\n \"sometask\",\n func=mytaskfunc,\n job_executor=\"threadpool\",\n metadata={\"direct\": True}\n )\n\nThe resulting task will have the following parameters:\n\n* ``id``: ``'sometask'`` (from the :meth:`~AsyncScheduler.configure_task` call)\n* ``job_executor``: ``'threadpool'`` (from the :meth:`~AsyncScheduler.configure_task`\n call, where it overrides the scheduler-level default)\n* ``max_running_jobs``: 3 (from the decorator)\n* ``misfire_grace_time``: 15 (from the scheduler-level default)\n* ``metadata``: ``{\"global\": 3, \"foo\": [\"taskfunc\"], \"direct\": True}``\n\nScheduling tasks\n================\n\nTo create a schedule for running a task, you need, at the minimum:\n\n* A preconfigured task_, OR a callable_ to be run\n* A trigger_\n\nIf you've configured a task (as per the previous section), you can pass the task object\nor its ID to :meth:`Scheduler.add_schedule`. As a shortcut, you can pass a callable_\ninstead, in which case a task will be automatically created for you if necessary.\n\nIf the callable you're trying to schedule is either a lambda or a nested function, then\nyou need to explicitly create a task beforehand, as it is not possible to create a\nreference (``package.module:varname``) to these types of callables.\n\nThe trigger determines the scheduling logic for your schedule. In other words, it is\nused to calculate the datetimes on which the task will be run. APScheduler comes with a\nnumber of built-in trigger classes:\n\n* :class:`~triggers.date.DateTrigger`:\n use when you want to run the task just once at a certain point of time\n* :class:`~triggers.interval.IntervalTrigger`:\n use when you want to run the task at fixed intervals of time\n* :class:`~triggers.cron.CronTrigger`:\n use when you want to run the task periodically at certain time(s) of day\n* :class:`~triggers.calendarinterval.CalendarIntervalTrigger`:\n use when you want to run the task on calendar-based intervals, at a specific time of\n day\n\nCombining multiple triggers\n---------------------------\n\nOccasionally, you may find yourself in a situation where your scheduling needs are too\ncomplex to be handled with any of the built-in triggers directly.\n\nOne examples of such a need would be when you want the task to run at 10:00 from Monday\nto Friday, but also at 11:00 from Saturday to Sunday.\nA single :class:`~triggers.cron.CronTrigger` would not be able to handle\nthis case, but an :class:`~triggers.combining.OrTrigger` containing two cron\ntriggers can::\n\n from apscheduler.triggers.combining import OrTrigger\n from apscheduler.triggers.cron import CronTrigger\n\n trigger = OrTrigger(\n CronTrigger(day_of_week=\"mon-fri\", hour=10),\n CronTrigger(day_of_week=\"sat-sun\", hour=11),\n )\n\nOn the first run, :class:`~triggers.combining.OrTrigger` generates the next\nrun times from both cron triggers and saves them internally. It then returns the\nearliest one. On the next run, it generates a new run time from the trigger that\nproduced the earliest run time on the previous run, and then again returns the earliest\nof the two run times. This goes on until all the triggers have been exhausted, if ever.\n\nAnother example would be a case where you want the task to be run every 2 months at\n10:00, but not on weekends (Saturday or Sunday)::\n\n from apscheduler.triggers.calendarinterval import CalendarIntervalTrigger\n from apscheduler.triggers.combining import AndTrigger\n from apscheduler.triggers.cron import CronTrigger\n\n trigger = AndTrigger(\n CalendarIntervalTrigger(months=2, hour=10),\n CronTrigger(day_of_week=\"mon-fri\", hour=10),\n )\n\nOn the first run, :class:`~triggers.combining.AndTrigger` generates the next\nrun times from both the\n:class:`~triggers.calendarinterval.CalendarIntervalTrigger` and\n:class:`~triggers.cron.CronTrigger`. If the run times coincide, it will\nreturn that run time. Otherwise, it will calculate a new run time from the trigger that\nproduced the earliest run time. It will keep doing this until a match is found, one of\nthe triggers has been exhausted or the maximum number of iterations (1000 by default) is\nreached.\n\nIf this trigger is created on 2022-06-07 at 09:00:00, its first run times would be:\n\n* 2022-06-07 10:00:00\n* 2022-10-07 10:00:00\n* 2022-12-07 10:00:00\n\nNotably, 2022-08-07 is skipped because it falls on a Sunday.\n\nRemoving schedules\n------------------\n\nTo remove a previously added schedule, call\n:meth:`~Scheduler.remove_schedule`. Pass the identifier of\nthe schedule you want to remove as an argument. This is the ID you got from\n:meth:`~Scheduler.add_schedule`.\n\nNote that removing a schedule does not cancel any jobs derived from it, but does prevent\nfurther jobs from being created from that schedule.\n\nPausing schedules\n-----------------\n\nTo pause a schedule, call :meth:`~Scheduler.pause_schedule`. Pass the identifier of the\nschedule you want to pause as an argument. This is the ID you got from\n:meth:`~Scheduler.add_schedule`.\n\nPausing a schedule prevents any new jobs from being created from it, but does not cancel\nany jobs that have already been created from that schedule.\n\nThe schedule can be unpaused by calling :meth:`~Scheduler.unpause_schedule` with the\nidentifier of the schedule you want to unpause.\n\nBy default the schedule will retain the next fire time it had when it was paused, which\nmay result in the schedule being considered to have misfired when it is unpaused,\nresulting in whatever misfire behavior it has configured\n(see :ref:`controlling-how-much-a-job-can-be-started-late` for more details).\n\nThe ``resume_from`` parameter can be used to specify the time from which the schedule\nshould be resumed. This can be used to avoid the misfire behavior mentioned above. It\ncan be either a datetime object, or the string ``\"now\"`` as a convenient shorthand for\nthe current datetime. If this parameter is provided, the schedules trigger will be\nrepeatedly advanced to determine a next fire time that is at or after the specified time\nto resume from.\n\nControlling how jobs are queued from schedules\n----------------------------------------------\n\nIn most cases, when a scheduler processes a schedule, it queues a new job using the\nrun time currently marked for the schedule. Then it updates the next run time using the\nschedule's trigger and releases the schedule back to the data store. But sometimes a\nsituation occurs where the schedule did not get processed often or quickly enough, and\none or more next run times produced by the trigger are actually in the past.\n\nIn a situation like that, the scheduler needs to decide what to do: to queue a job for\nevery run time produced, or to *coalesce* them all into a single job, effectively just\nkicking off a single job. To control this, pass the ``coalesce`` argument to\n:meth:`~Scheduler.add_schedule`.\n\nThe possible values are:\n\n* :data:`~CoalescePolicy.latest`: queue exactly one job, using the\n **latest** run time as the designated run time\n* :data:`~CoalescePolicy.earliest`: queue exactly one job, using the\n **earliest** run time as the designated run time\n* :data:`~CoalescePolicy.all`: queue one job for **each** of the calculated\n run times\n\nThe biggest difference between the first two options is how the designated run time, and\nby extension, the starting deadline for the job is selected. With the first option,\nthe job is less likely to be skipped due to being started late since the latest of all\nthe collected run times is used for the deadline calculation.\n\nAs explained in the previous section, the starting\ndeadline is *misfire grace time*\naffects the newly queued job.\n\nRunning tasks without scheduling\n================================\n\nIn some cases, you want to run tasks directly, without involving schedules:\n\n* You're only interested in using the scheduler system as a job queue\n* You're interested in the job's return value\n\nTo queue a job and wait for its completion and get the result, the easiest way is to\nuse :meth:`~Scheduler.run_job`. If you prefer to just launch a job and not wait for its\nresult, use :meth:`~Scheduler.add_job` instead. If you want to get the results later,\nyou need to pass an appropriate ``result_expiration_time`` parameter to\n:meth:`~Scheduler.add_job` so that the result is saved. Then, you can call\n:meth:`~Scheduler.get_job_result` with the job ID you got from\n:meth:`~Scheduler.add_job` to retrieve the result.\n\nContext variables\n=================\n\nSchedulers provide certain `context variables`_ available to the tasks being run:\n\n* The current (synchronous) scheduler: :data:`~current_scheduler`\n* The current asynchronous scheduler: :data:`~current_async_scheduler`\n* Information about the job being currently run: :data:`~current_job`\n\nHere's an example::\n\n from apscheduler import current_job\n\n def my_task_function():\n job_info = current_job.get().id\n print(\n f\"This is job {job_info.id} and was spawned from schedule \"\n f\"{job_info.schedule_id}\"\n )\n\n.. _context variables: :mod:`contextvars`\n\n.. _scheduler-events:\n\nSubscribing to events\n=====================\n\nSchedulers have the ability to notify listeners when some event occurs in the scheduler\nsystem. Examples of such events would be schedulers or workers starting up or shutting\ndown, or schedules or jobs being created or removed from the data store.\n\nTo listen to events, you need a callable_ that takes a single positional argument\nwhich is the event object. Then, you need to decide which events you're interested in:\n\n.. tabs::\n\n .. code-tab:: python Synchronous\n\n from apscheduler import Event, JobAcquired, JobReleased\n\n def listener(event: Event) -> None:\n print(f\"Received {event.__class__.__name__}\")\n\n scheduler.subscribe(listener, {JobAcquired, JobReleased})\n\n .. code-tab:: python Asynchronous\n\n from apscheduler import Event, JobAcquired, JobReleased\n\n async def listener(event: Event) -> None:\n print(f\"Received {event.__class__.__name__}\")\n\n scheduler.subscribe(listener, {JobAcquired, JobReleased})\n\nThis example subscribes to the :class:`~JobAcquired` and\n:class:`~JobReleased` event types. The callback will receive an event of\neither type, and prints the name of the class of the received event.\n\nAsynchronous schedulers and workers support both synchronous and asynchronous callbacks,\nbut their synchronous counterparts only support synchronous callbacks.\n\nWhen **distributed** event brokers (that is, other than the default one) are being used,\nevents other than the ones relating to the life cycles of schedulers and workers, will\nbe sent to all schedulers and workers connected to that event broker.\n\nClean-up of expired jobs, job results and schedules\n===================================================\n\nEach scheduler runs the data store's :meth:`~.abc.DataStore.cleanup` method\nperiodically, configurable via the ``cleanup_interval`` scheduler parameter. This\nensures that the data store doesn't get filled with unused data over time.\n\nDeployment\n==========\n\nUsing persistent data stores\n----------------------------\n\nThe default data store, :class:`~datastores.memory.MemoryDataStore`, stores\ndata only in memory so all the schedules and jobs that were added to it will be erased\nif the process crashes.\n\nWhen you need your schedules and jobs to survive the application shutting down, you need\nto use a *persistent data store*. Such data stores do have additional considerations,\ncompared to the memory data store:\n\n* Task arguments must be *serializable*\n* You must either trust the data store, or use an alternate *serializer*\n* A *conflict policy* and an *explicit identifier* must be defined for schedules that\n are added at application startup\n\nThese requirements warrant some explanation. The first point means that since persisting\ndata means saving it externally, either in a file or sending to a database server, all\nthe objects involved are converted to bytestrings. This process is called\n*serialization*. By default, this is done using :mod:`pickle`, which guarantees the best\ncompatibility but is notorious for being vulnerable to simple injection attacks. This\nbrings us to the second point. If you cannot be sure that nobody can maliciously alter\nthe externally stored serialized data, it would be best to use another serializer. The\nbuilt-in alternatives are:\n\n* :class:`~serializers.cbor.CBORSerializer`\n* :class:`~serializers.json.JSONSerializer`\n\nThe former requires the cbor2_ library, but supports a wider variety of types natively.\nThe latter has no dependencies but has very limited support for different types.\n\nThe third point relates to situations where you're essentially adding the same schedule\nto the data store over and over again. If you don't specify a static identifier for\nthe schedules added at the start of the application, you will end up with an increasing\nnumber of redundant schedules doing the same thing, which is probably not what you want.\nTo that end, you will need to come up with some identifying name which will ensure that\nthe same schedule will not be added over and over again (as data stores are required to\nenforce the uniqueness of schedule identifiers). You'll also need to decide what to do\nif the schedule already exists in the data store (that is, when the application is\nstarted the second time) by passing the ``conflict_policy`` argument. Usually you want\nthe :data:`~ConflictPolicy.replace` option, which replaces the existing\nschedule with the new one.\n\n.. seealso:: You can find practical examples of persistent data stores in the\n :file:`examples/standalone` directory (``async_postgres.py`` and\n ``async_mysql.py``).\n\n.. _cbor2: https://pypi.org/project/cbor2/\n\nUsing multiple schedulers\n-------------------------\n\nThere are several situations in which you would want to run several schedulers against\nthe same data store at once:\n\n* Running a server application (usually a web app) with multiple worker processes\n* You need fault tolerance (scheduling will continue even if a node or process running\n a scheduler goes down)\n\nWhen you have multiple schedulers running at once, they need to be able to coordinate\ntheir efforts so that the schedules don't get processed more than once and the\nschedulers know when to wake up even if another scheduler added the next due schedule to\nthe data store. To this end, a shared *event broker* must be configured.\n\n.. seealso:: You can find practical examples of data store sharing in the\n :file:`examples/web` directory.\n\nUsing a scheduler without running it\n------------------------------------\n\nSome deployment scenarios may warrant the use of a scheduler for only interfacing with\nan external data store, for things like configuring tasks, adding schedules or queuing\njobs. One such practical use case is a web application that needs to run heavy\ncomputations elsewhere so they don't cause performance issues with the web application\nitself.\n\nYou can then run one or more schedulers against the same data store and event broker\nelsewhere where they don't disturb the web application. These schedulers will do all the\nheavy lifting like processing schedules and running jobs.\n\n.. seealso:: A practical example of this separation of concerns can be found in the\n :file:`examples/separate_worker` directory.\n\nExplicitly assigning an identity to the scheduler\n-------------------------------------------------\n\nIf you're running one or more schedulers against a persistent data store in a production\nsetting, it'd be wise to assign each scheduler a custom identity. The reason for this is\ntwofold:\n\n#. It helps you figure out which jobs are being run where\n#. It allows crashed jobs to cleared out quicker, as other schedulers aren't allowed to\n clean them up until the jobs' timeouts expire\n\nThe best choice would be something that the environment guarantees to be unique among\nall the scheduler instances but stays the same when the scheduler instance is restarted.\nFor example, on Kubernetes, this would be the name of the pod where the scheduler is\nrunning, assuming of course that there is only one scheduler running in each pod against\nthe same data store.\n\nOf course, if you're only ever running one scheduler against a persistent data store,\nyou can just use a static scheduler ID.\n\nIf no ID is explicitly given, the scheduler generates an ID by concatenating the\nfollowing:\n\n* the current host name\n* the current process ID\n* the ID of the scheduler instance\n\n.. _troubleshooting:\n\nTroubleshooting\n===============\n\nIf something isn't working as expected, it will be helpful to increase the logging level\nof the ``apscheduler`` logger to the ``DEBUG`` level.\n\nIf you do not yet have logging enabled in the first place, you can do this::\n\n import logging\n\n logging.basicConfig()\n logging.getLogger('apscheduler').setLevel(logging.DEBUG)\n\nThis should provide lots of useful information about what's going on inside the\nscheduler and/or worker.\n\nAlso make sure that you check the :doc:`faq` section to see if your problem already has\na solution.\n\nReporting bugs\n==============\n\nA `bug tracker `_ is provided by\nGitHub.\n\nGetting help\n============\n\nIf you have problems or other questions, you can either:\n\n* Ask in the `apscheduler `_ room on Gitter\n* Post a question on `GitHub discussions`_, or\n* Post a question on StackOverflow_ and add the ``apscheduler`` tag\n\n.. _GitHub discussions: https://github.com/agronholm/apscheduler/discussions/categories/q-a\n.. _StackOverflow: http://stackoverflow.com/questions/tagged/apscheduler\n" }, { "path": "docs/versionhistory.rst", "content": "Version history\n===============\n\nTo find out how to migrate your application from a previous version of\nAPScheduler, see the :doc:`migration section `.\n\n**UNRELEASED**\n\n- **BREAKING** Switched the MongoDB data store to use the asynchronous API in\n ``pymongo`` and bumped the minimum ``pymongo`` version to v4.13.0\n- Dropped support for Python 3.9\n- Fixed an issue where ``CronTrigger`` does not convert ``start_time`` to ``self.timezone``\n (`#1061 `_; PR by @jonasitzmann)\n- Fixed an issue where ``CronTrigger.next()`` returned a non-existing date on a DST change\n (`#1059 `_; PR by @jonasitzmann)\n- Fixed jobs that were being run when the scheduler was gracefully stopped being left in\n an acquired state (`#946 `_)\n\n**4.0.0a6**\n\n- **BREAKING** Refactored ``AsyncpgEventBroker`` to directly accept a connection string,\n thus eliminating the need for the ``AsyncpgEventBroker.from_dsn()`` class method\n- **BREAKING** Added the ``extend_acquired_schedule_leases()`` data store method to\n prevent other schedulers from acquiring schedules already being processed by a\n scheduler, if that's taking unexpectedly long for some reason\n- **BREAKING** Added the ``extend_acquired_job_leases()`` data store method to prevent\n jobs from being cleaned up as if they had been abandoned\n (`#864 `_)\n- **BREAKING** Changed the ``cleanup()`` data store method to also be responsible for\n releasing jobs whose leases have expired (so the schedulers responsible for them have\n probably died)\n- **BREAKING** Changed most attributes in ``Task`` and ``Schedule`` classes to be\n read-only\n- **BREAKING** Refactored the ``release_schedules()`` data store method to take a\n sequence of ``ScheduleResult`` instances instead of a sequence of schedules, to enable\n the memory data store to handle schedule updates more efficiently\n- **BREAKING** Replaced the data store ``lock_expiration_delay`` parameter with a new\n scheduler-level parameter, ``lease_duration`` which is then used to call the various\n data store methods\n- **BREAKING** Added the ``job_result_expiration_time`` field to the ``Schedule`` class,\n to allow the job results from scheduled jobs to stay around for some time\n (`#927 `_)\n- **BREAKING** Added an index for the ``created_at`` job field, so acquiring jobs would\n be faster when there are a lot of them\n- **BREAKING** Removed the ``job_executor`` and ``max_running_jobs`` parameters from\n ``add_schedule()`` and ``add_run_job()`` (explicitly configure the task using\n ``configure_task()`` or by using the new ``@task`` decorator\n- **BREAKING** Replaced the ``default_job_executor`` scheduler parameter with a more\n comprehensive ``task_defaults`` parameter\n- Added the ``@task`` decorator for specifying task configuration parameters bound to a\n function\n- **BREAKING** Changed tasks to only function as job templates as well as buckets to\n limit maximum concurrent job execution\n- **BREAKING** Changed the ``timezone`` argument to ``CronTrigger.from_crontab()`` into\n a keyword-only argument\n- **BREAKING** Added the ``metadata`` field to tasks, schedules and jobs\n- **BREAKING** Added logic to store ``last_fire_time`` in datastore implementations\n (PR by @hlobit)\n- **BREAKING** Added the ``reap_abandoned_jobs()`` abstract method to ``DataStore``\n which the scheduler calls before processing any jobs in order to immediately mark jobs\n left in an acquired state when the scheduler crashed\n- Added the ``start_time`` and ``end_time`` arguments to ``CronTrigger.from_crontab()``\n (`#676 `_)\n- Added the ``psycopg`` event broker\n- Added useful indexes and removed useless ones in ``SQLAlchemyDatastore`` and\n ``MongoDBDataStore``\n- Changed the ``lock_expiration_delay`` parameter of built-in data stores to accept a\n ``timedelta`` as well as ``int`` or ``float``\n- Fixed serialization error with ``CronTrigger`` when pausing a schedule\n (`#864 `_)\n- Fixed ``TypeError: object NoneType can't be used in 'await' expression`` at teardown\n of ``SQLAlchemyDataStore`` when it was passed a URL that implicitly created a\n synchronous engine\n- Fixed serializers raising their own exceptions instead of ``SerializationError`` and\n ``DeserializationError`` as appropriate\n- Fixed ``repr()`` outputs of schedulers, data stores and event brokers to be much more\n useful and reasonable\n- Fixed race condition in ``MongoDBDataStore`` that allowed multiple schedulers to\n acquire the same schedules at once\n- Changed ``SQLAlchemyDataStore`` to automatically create the explicitly specified\n schema if it's missing (PR by @zhu0629)\n- Fixed an issue with ``CronTrigger`` infinitely looping to get next date when DST ends\n (`#980 `_; PR by @hlobit)\n- Skip dispatching extend_acquired_job_leases with no jobs (PR by @JacobHayes)\n- Fixed schedulers not immediately processing schedules that the scheduler left in an\n acquired state after a crash\n- Fixed the job lease extension task exiting prematurely while the scheduler is starting\n (PR by @JacobHayes)\n- Migrated test and documentation dependencies from extras to dependency groups\n- Fixed ``add_job()`` overwriting task configuration (PR by @mattewid)\n\n**4.0.0a5**\n\n- **BREAKING** Added the ``cleanup()`` scheduler method and a configuration option\n (``cleanup_interval``). A corresponding abstract method was added to the ``DataStore``\n class. This method purges expired job results and schedules that have exhausted their\n triggers and have no more associated jobs running. Previously, schedules were\n automatically deleted instantly once their triggers could no longer produce any fire\n times.\n- **BREAKING** Made publishing ``JobReleased`` events the responsibility of the\n ``DataStore`` implementation, rather than the scheduler, for consistency with the\n ``acquire_jobs()`` method\n- **BREAKING** The ``started_at`` field was moved from ``Job`` to ``JobResult``\n- **BREAKING** Removed the ``from_url()`` class methods of ``SQLAlchemyDataStore``,\n ``MongoDBDataStore`` and ``RedisEventBroker`` in favor of the ability to pass a\n connection url to the initializer\n- Added the ability to pause and unpause schedules (PR by @WillDaSilva)\n- Added the ``scheduled_start`` field to the ``JobAcquired`` event\n- Added the ``scheduled_start`` and ``started_at`` fields to the ``JobReleased`` event\n- Fixed large parts of ``MongoDBDataStore`` still calling blocking functions in the\n event loop thread\n- Fixed JSON serialization of triggers that had been used at least once\n- Fixed dialect name checks in the SQLAlchemy job store\n- Fixed JSON and CBOR serializers unable to serialize enums\n- Fixed infinite loop in CalendarIntervalTrigger with UTC timezone (PR by unights)\n- Fixed scheduler not resuming job processing when ``max_concurrent_jobs`` had been\n reached and then a job was completed, thus making job processing possible again\n (PR by MohammadAmin Vahedinia)\n- Fixed the shutdown procedure of the Redis event broker\n- Fixed ``SQLAlchemyDataStore`` not respecting custom schema name when creating enums\n- Fixed skipped intervals with overlapping schedules in ``AndTrigger``\n (#911 _; PR by Bennett Meares)\n- Fixed implicitly created client instances in data stores and event brokers not being\n closed along with the store/broker\n\n**4.0.0a4**\n\n- **BREAKING** Renamed any leftover fields named ``executor`` to ``job_executor``\n (this breaks data store compatibility)\n- **BREAKING** Switched to using the timezone aware timestamp column type on Oracle\n- **BREAKING** Fixed precision issue with interval columns on MySQL\n- **BREAKING** Fixed datetime comparison issues on SQLite and MySQL\n- **BREAKING** Worked around datetime microsecond precision issue on MongoDB\n- **BREAKING** Renamed the ``worker_id`` field to ``scheduler_id`` in the\n ``JobAcquired`` and ``JobReleased`` events\n- **BREAKING** Added the ``task_id`` attribute to the ``ScheduleAdded``,\n ``ScheduleUpdated`` and ``ScheduleRemoved`` events\n- **BREAKING** Added the ``finished`` attribute to the ``ScheduleRemoved`` event\n- **BREAKING** Added the ``logger`` parameter to ``Datastore.start()`` and\n ``EventBroker.start()`` to make both use the scheduler's assigned logger\n- **BREAKING** Made the ``apscheduler.marshalling`` module private\n- Added the ``configure_task()`` and ``get_tasks()`` scheduler methods\n- Fixed out of order delivery of events delivered using worker threads\n- Fixed schedule processing not setting job start deadlines correctly\n\n**4.0.0a3**\n\n- **BREAKING** The scheduler classes were moved to be importable (only) directly from\n the ``apscheduler`` package (``apscheduler.Scheduler`` and\n ``apscheduler.AsyncScheduler``)\n- **BREAKING** Removed the \"tags\" field in schedules and jobs (this will be added back\n when the feature has been fully thought through)\n- **BREAKING** Removed the ``JobInfo`` class in favor of just using the ``Job`` class\n (which is now immutable)\n- **BREAKING** Workers were merged into schedulers. As the ``Worker`` and\n ``AsyncWorker`` classes have been removed, you now need to pass\n ``role=SchedulerRole.scheduler`` to the scheduler to prevent it from processing due\n jobs. The worker event classes (``WorkerEvent``, ``WorkerStarted``, ``WorkerStopped``)\n have also been removed.\n- **BREAKING** The synchronous interfaces for event brokers and data stores have been\n removed. Synchronous libraries can still be used to implement these services through\n the use of ``anyio.to_thread.run_sync()``.\n- **BREAKING** The ``current_worker`` context variable has been removed\n- **BREAKING** The ``current_scheduler`` context variable is now specified to only\n contain the currently running instance of a **synchronous** scheduler\n (``apscheduler.Scheduler``). The asynchronous scheduler instance can be fetched from\n the new ``current_async_scheduler`` context variable, and will always be available\n when a scheduler is running in the current context, while ``current_scheduler`` is\n only available when the synchronous wrapper is being run.\n- **BREAKING** Changed the initialization of data stores and event brokers to use a\n single ``start()`` method that accepts an ``AsyncExitStack`` (and, depending on the\n interface, other arguments too)\n- **BREAKING** Added a concept of \"job executors\". This determines how the task function\n is executed once picked up by a worker. Several data structures and scheduler methods\n have a new field/parameter for this, ``job_executor``. This addition requires database\n schema changes too.\n- Dropped support for Python 3.7\n- Added support for Python 3.12\n- Added the ability to run jobs in worker processes, courtesy of the ``processpool``\n executor\n- Added the ability to run jobs in the Qt event loop via the ``qt`` executor\n- Added the ``get_jobs()`` scheduler method\n- The synchronous scheduler now runs an asyncio event loop in a thread, acting as a\n façade for ``AsyncScheduler``\n- Fixed the ``schema`` parameter in ``SQLAlchemyDataStore`` not being applied\n- Fixed SQLalchemy 2.0 compatibility\n\n**4.0.0a2**\n\n- **BREAKING** Changed the scheduler API to always require a call to either\n ``run_until_stopped()`` or ``start_in_background()`` to start the scheduler (using it\n as a context manager is no longer enough)\n- **BREAKING** Replaced ``from_asyncpg_pool()`` with ``from_dsn()`` in the asyncpg event\n broker\n- Added an async Redis event broker\n- Added automatic reconnection to the Redis event brokers (sync and async)\n- Added automatic reconnection to the asyncpg event broker\n- Changed ``from_async_sqla_engine()`` in asyncpg event broker to only copy the\n connection options instead of directly using the engine\n- Simplified the MQTT event broker by providing a default ``client`` instance if omitted\n- Fixed ``CancelledError`` being reported as a crash on Python 3.7\n- Fixed JSON/CBOR serialization of ``JobReleased`` events\n\n**4.0.0a1**\n\nThis was a major rewrite/redesign of most parts of the project. See the\n:doc:`migration section ` section for details.\n\n.. warning:: The v4.0 series is provided as a **pre-release** and may change in a\n backwards incompatible fashion without any migration pathway, so do NOT use this\n release in production!\n\n- Made persistent data stores shareable between multiple processes and nodes\n- Enhanced data stores to be more resilient against temporary connectivity failures\n- Refactored executors (now called *workers*) to pull jobs from the data store so they\n can be run independently from schedulers\n- Added full async support (:mod:`asyncio` and Trio_) via AnyIO_\n- Added type annotations to the code base\n- Added the ability to queue jobs directly without scheduling them\n- Added alternative serializers (CBOR, JSON)\n- Added the ``CalendarInterval`` trigger\n- Added the ability to access the current scheduler (under certain circumstances),\n current worker and the currently running job via context-local variables\n- Added schedule level support for jitter\n- Made triggers stateful\n- Added threshold support for ``AndTrigger``\n- Migrated from ``pytz`` time zones to standard library ``zoneinfo`` zones\n- Allowed a wider range of tzinfo implementations to be used (though ``zoneinfo`` is\n preferred)\n- Changed ``IntervalTrigger`` to start immediately instead of first waiting for one\n interval\n- Changed ``CronTrigger`` to use Sunday as weekday number 0, as per the crontab standard\n- Dropped support for Python 2.X, 3.5 and 3.6\n- Dropped support for the Qt, Twisted, Tornado and Gevent schedulers\n- Dropped support for the Redis, RethinkDB and Zookeeper job stores\n\n.. _Trio: https://pypi.org/project/trio/\n.. _AnyIO: https://github.com/agronholm/anyio\n\n**3.9.1**\n\n* Removed a leftover check for pytz ``localize()`` and ``normalize()`` methods\n\n**3.9.0**\n\n- Added support for PySide6 to the Qt scheduler\n- No longer enforce pytz time zones (support for others is experimental in the 3.x series)\n- Fixed compatibility with PyMongo 4\n- Fixed pytz deprecation warnings\n- Fixed RuntimeError when shutting down the scheduler from a scheduled job\n\n**3.8.1**\n\n- Allowed the use of tzlocal v4.0+ in addition to v2.*\n\n**3.8.0**\n\n- Allowed passing through keyword arguments to the underlying stdlib executors in the\n thread/process pool executors (PR by Albert Xu)\n\n**3.7.0**\n\n- Dropped support for Python 3.4\n- Added PySide2 support (PR by Abdulla Ibrahim)\n- Pinned ``tzlocal`` to a version compatible with pytz\n- Ensured that jitter is always non-negative to prevent triggers from firing more often than\n intended\n- Changed ``AsyncIOScheduler`` to obtain the event loop in ``start()`` instead of ``__init__()``,\n to prevent situations where the scheduler won't run because it's using a different event loop\n than then one currently running\n- Made it possible to create weak references to ``Job`` instances\n- Made the schedulers explicitly raise a descriptive ``TypeError`` when serialization is attempted\n- Fixed Zookeeper job store using backslashes instead of forward slashes for paths\n on Windows (PR by Laurel-rao)\n- Fixed deprecation warnings on the MongoDB job store and increased the minimum PyMongo\n version to 3.0\n- Fixed ``BlockingScheduler`` and ``BackgroundScheduler`` shutdown hanging after the user has\n erroneously tried to start it twice\n- Fixed memory leak when coroutine jobs raise exceptions (due to reference cycles in tracebacks)\n- Fixed inability to schedule wrapped functions with extra arguments when the wrapped function\n cannot accept them but the wrapper can (original PR by Egor Malykh)\n- Fixed potential ``where`` clause error in the SQLAlchemy job store when a subclass uses more than\n one search condition\n- Fixed a problem where bound methods added as jobs via textual references were called with an\n unwanted extra ``self`` argument (PR by Pengjie Song)\n- Fixed ``BrokenPoolError`` in ``ProcessPoolExecutor`` so that it will automatically replace the\n broken pool with a fresh instance\n\n**3.6.3**\n\n- Fixed Python 2.7 accidentally depending on the ``trollius`` package (regression from v3.6.2)\n\n**3.6.2**\n\n- Fixed handling of :func:`~functools.partial` wrapped coroutine functions in ``AsyncIOExecutor``\n and ``TornadoExecutor`` (PR by shipmints)\n\n**3.6.1**\n\n- Fixed OverflowError on Qt scheduler when the wait time is very long\n- Fixed methods inherited from base class could not be executed by processpool executor\n (PR by Yang Jian)\n\n**3.6.0**\n\n- Adapted ``RedisJobStore`` to v3.0 of the ``redis`` library\n- Adapted ``RethinkDBJobStore`` to v2.4 of the ``rethink`` library\n- Fixed ``DeprecationWarnings`` about ``collections.abc`` on Python 3.7 (PR by Roman Levin)\n\n**3.5.3**\n\n- Fixed regression introduced in 3.5.2: Class methods were mistaken for instance methods and thus\n were broken during serialization\n- Fixed callable name detection for methods in old style classes\n\n**3.5.2**\n\n- Fixed scheduling of bound methods on persistent job stores (the workaround of scheduling\n ``YourClass.methodname`` along with an explicit ``self`` argument is no longer necessary as this\n is now done automatically for you)\n- Added the FAQ section to the docs\n- Made ``BaseScheduler.start()`` raise a ``RuntimeError`` if running under uWSGI with threads\n disabled\n\n**3.5.1**\n\n- Fixed ``OverflowError`` on Windows when the wait time is too long\n- Fixed ``CronTrigger`` sometimes producing fire times beyond ``end_date`` when jitter is enabled\n (thanks to gilbsgilbs for the tests)\n- Fixed ISO 8601 UTC offset information being silently discarded from string formatted datetimes by\n adding support for parsing them\n\n**3.5.0**\n\n- Added the ``engine_options`` option to ``SQLAlchemyJobStore``\n- Added the ``jitter`` options to ``IntervalTrigger`` and ``CronTrigger`` (thanks to gilbsgilbs)\n- Added combining triggers (``AndTrigger`` and ``OrTrigger``)\n- Added better validation for the steps and ranges of different expressions in ``CronTrigger``\n- Added support for named months (``jan`` – ``dec``) in ``CronTrigger`` month expressions\n- Added support for creating a ``CronTrigger`` from a crontab expression\n- Allowed spaces around commas in ``CronTrigger`` fields\n- Fixed memory leak due to a cyclic reference when jobs raise exceptions\n (thanks to gilbsgilbs for help on solving this)\n- Fixed passing ``wait=True`` to ``AsyncIOScheduler.shutdown()`` (although it doesn't do much)\n- Cancel all pending futures when ``AsyncIOExecutor`` is shut down\n\n**3.4.0**\n\n- Dropped support for Python 3.3\n- Added the ability to specify the table schema for ``SQLAlchemyJobStore``\n (thanks to Meir Tseitlin)\n- Added a workaround for the ``ImportError`` when used with PyInstaller and the likes\n (caused by the missing packaging metadata when APScheduler is packaged with these tools)\n\n**3.3.1**\n\n- Fixed Python 2.7 compatibility in ``TornadoExecutor``\n\n**3.3.0**\n\n- The asyncio and Tornado schedulers can now run jobs targeting coroutine functions\n (requires Python 3.5; only native coroutines (``async def``) are supported)\n- The Tornado scheduler now uses TornadoExecutor as its default executor (see above as for why)\n- Added ZooKeeper job store (thanks to Jose Ignacio Villar for the patch)\n- Fixed job store failure (``get_due_jobs()``) causing the scheduler main loop to exit (it now\n waits a configurable number of seconds before retrying)\n- Fixed ``@scheduled_job`` not working when serialization is required (persistent job stores and\n ``ProcessPoolScheduler``)\n- Improved import logic in ``ref_to_obj()`` to avoid errors in cases where traversing the path with\n ``getattr()`` would not work (thanks to Jarek Glowacki for the patch)\n- Fixed CronTrigger's weekday position expressions failing on Python 3\n- Fixed CronTrigger's range expressions sometimes allowing values outside the given range\n\n**3.2.0**\n\n- Added the ability to pause and unpause the scheduler\n- Fixed pickling problems with persistent jobs when upgrading from 3.0.x\n- Fixed AttributeError when importing apscheduler with setuptools < 11.0\n- Fixed some events missing from ``apscheduler.events.__all__`` and\n ``apscheduler.events.EVENTS_ALL``\n- Fixed wrong run time being set for date trigger when the timezone isn't the same as the local one\n- Fixed builtin ``id()`` erroneously used in MongoDBJobStore's ``JobLookupError()``\n- Fixed endless loop with CronTrigger that may occur when the computer's clock resolution is too\n low (thanks to Jinping Bai for the patch)\n\n**3.1.0**\n\n- Added RethinkDB job store (contributed by Allen Sanabria)\n- Added method chaining to the ``modify_job()``, ``reschedule_job()``, ``pause_job()`` and\n ``resume_job()`` methods in ``BaseScheduler`` and the corresponding methods in the ``Job`` class\n- Added the EVENT_JOB_SUBMITTED event that indicates a job has been submitted to its executor.\n- Added the EVENT_JOB_MAX_INSTANCES event that indicates a job's execution was skipped due to its\n maximum number of concurrently running instances being reached\n\n- Added the time zone to the repr() output of ``CronTrigger`` and ``IntervalTrigger``\n- Fixed rare race condition on scheduler ``shutdown()``\n- Dropped official support for CPython 2.6 and 3.2 and PyPy3\n- Moved the connection logic in database backed job stores to the ``start()`` method\n- Migrated to setuptools_scm for versioning\n- Deprecated the various version related variables in the ``apscheduler`` module\n (``apscheduler.version_info``, ``apscheduler.version``, ``apscheduler.release``,\n ``apscheduler.__version__``)\n\n**3.0.6**\n\n- Fixed bug in the cron trigger that produced off-by-1-hour datetimes when crossing the daylight\n saving threshold (thanks to Tim Strazny for reporting)\n\n**3.0.5**\n\n- Fixed cron trigger always coalescing missed run times into a single run time\n (contributed by Chao Liu)\n- Fixed infinite loop in the cron trigger when an out-of-bounds value was given in an expression\n- Fixed debug logging displaying the next wakeup time in the UTC timezone instead of the\n scheduler's configured timezone\n- Allowed unicode function references in Python 2\n\n**3.0.4**\n\n- Fixed memory leak in the base executor class (contributed by Stefan Nordhausen)\n\n**3.0.3**\n\n- Fixed compatibility with pymongo 3.0\n\n**3.0.2**\n\n- Fixed ValueError when the target callable has a default keyword argument that wasn't overridden\n- Fixed wrong job sort order in some job stores\n- Fixed exception when loading all jobs from the redis job store when there are paused jobs in it\n- Fixed AttributeError when printing a job list when there were pending jobs\n- Added setuptools as an explicit requirement in install requirements\n\n**3.0.1**\n\n- A wider variety of target callables can now be scheduled so that the jobs are still serializable\n (static methods on Python 3.3+, unbound methods on all except Python 3.2)\n- Attempting to serialize a non-serializable Job now raises a helpful exception during\n serialization. Thanks to Jeremy Morgan for pointing this out.\n- Fixed table creation with SQLAlchemyJobStore on MySQL/InnoDB\n- Fixed start date getting set too far in the future with a timezone different from the local one\n- Fixed _run_job_error() being called with the incorrect number of arguments in most executors\n\n**3.0.0**\n\n- Added support for timezones (special thanks to Curtis Vogt for help with this one)\n- Split the old Scheduler class into BlockingScheduler and BackgroundScheduler and added\n integration for asyncio (PEP 3156), Gevent, Tornado, Twisted and Qt event loops\n- Overhauled the job store system for much better scalability\n- Added the ability to modify, reschedule, pause and resume jobs\n- Dropped the Shelve job store because it could not work with the new job store system\n- Dropped the max_runs option and run counting of jobs since it could not be implemented reliably\n- Adding jobs is now done exclusively through ``add_job()`` -- the shortcuts to triggers were\n removed\n- Added the ``end_date`` parameter to cron and interval triggers\n- It is now possible to add a job directly to an executor without scheduling, by omitting the\n trigger argument\n- Replaced the thread pool with a pluggable executor system\n- Added support for running jobs in subprocesses (via the ``processpool`` executor)\n- Switched from nose to py.test for running unit tests\n\n**2.1.0**\n\n- Added Redis job store\n- Added a \"standalone\" mode that runs the scheduler in the calling thread\n- Fixed disk synchronization in ShelveJobStore\n- Switched to PyPy 1.9 for PyPy compatibility testing\n- Dropped Python 2.4 support\n- Fixed SQLAlchemy 0.8 compatibility in SQLAlchemyJobStore\n- Various documentation improvements\n\n**2.0.3**\n\n- The scheduler now closes the job store that is being removed, and all job stores on shutdown() by\n default\n- Added the ``last`` expression in the day field of CronTrigger (thanks rcaselli)\n- Raise a TypeError when fields with invalid names are passed to CronTrigger (thanks Christy\n O'Reilly)\n- Fixed the persistent.py example by shutting down the scheduler on Ctrl+C\n- Added PyPy 1.8 and CPython 3.3 to the test suite\n- Dropped PyPy 1.4 - 1.5 and CPython 3.1 from the test suite\n- Updated setup.cfg for compatibility with distutils2/packaging\n- Examples, documentation sources and unit tests are now packaged in the source distribution\n\n**2.0.2**\n\n- Removed the unique constraint from the \"name\" column in the SQLAlchemy job store\n- Fixed output from Scheduler.print_jobs() which did not previously output a line ending at the end\n\n**2.0.1**\n\n- Fixed cron style jobs getting wrong default values\n\n**2.0.0**\n\n- Added configurable job stores with several persistent back-ends (shelve, SQLAlchemy and MongoDB)\n- Added the possibility to listen for job events (execution, error, misfire, finish) on a scheduler\n- Added an optional start time for cron-style jobs\n- Added optional job execution coalescing for situations where several executions of the job are\n due\n- Added an option to limit the maximum number of concurrently executing instances of the job\n- Allowed configuration of misfire grace times on a per-job basis\n- Allowed jobs to be explicitly named\n- All triggers now accept dates in string form (YYYY-mm-dd HH:MM:SS)\n- Jobs are now run in a thread pool; you can either supply your own PEP 3148 compliant thread pool\n or let APScheduler create its own\n- Maximum run count can be configured for all jobs, not just those using interval-based scheduling\n- Fixed a v1.x design flaw that caused jobs to be executed twice when the scheduler thread was\n woken up while still within the allowable range of their previous execution time (issues #5, #7)\n- Changed defaults for cron-style jobs to be more intuitive -- it will now default to all\n minimum values for fields lower than the least significant explicitly defined field\n\n**1.3.1**\n\n- Fixed time difference calculation to take into account shifts to and from daylight saving time\n\n**1.3.0**\n\n- Added __repr__() implementations to expressions, fields, triggers, and jobs to help with\n debugging\n- Added the dump_jobs method on Scheduler, which gives a helpful listing of all jobs scheduled on\n it\n- Fixed positional weekday (3th fri etc.) expressions not working except in some edge cases\n (fixes #2)\n- Removed autogenerated API documentation for modules which are not part of the public API, as it\n might confuse some users\n\n.. Note:: Positional weekdays are now used with the **day** field, not\n **weekday**.\n\n**1.2.1**\n\n- Fixed regression: add_cron_job() in Scheduler was creating a CronTrigger with the wrong\n parameters (fixes #1, #3)\n- Fixed: if the scheduler is restarted, clear the \"stopped\" flag to allow jobs to be scheduled\n again\n\n**1.2.0**\n\n- Added the ``week`` option for cron schedules\n- Added the ``daemonic`` configuration option\n- Fixed a bug in cron expression lists that could cause valid firing times to be missed\n- Fixed unscheduling bound methods via unschedule_func()\n- Changed CronTrigger constructor argument names to match those in Scheduler\n\n**1.01**\n\n- Fixed a corner case where the combination of hour and day_of_week parameters would cause\n incorrect timing for a cron trigger\n" }, { "path": "examples/README.rst", "content": "APScheduler practical examples\n==============================\n\n.. highlight:: bash\n\nThis directory contains a number of practical examples for running APScheduler in a\nvariety of configurations.\n\nPrerequisites\n-------------\n\nMost examples use one or more external services for data sharing and synchronization.\nTo start these services, you need Docker_ installed. Each example lists the services\nit needs (if any) in the module file, so you can start these services selectively.\n\nOn Linux, if you're using the vendor provided system package for Docker instead of\nDocker Desktop, you may need to install the compose (v2) plugin (named\n``docker-compose-plugin``, or similar) separately.\n\n.. note:: If you're still using the Python-based docker-compose tool (aka compose v1),\n replace ``docker compose`` with ``docker-compose``.\n\nTo start all the services, run this command anywhere within the project directory::\n\n docker compose up -d\n\nTo start just a specific service, you can pass its name as an argument::\n\n docker compose up -d postgresql\n\nTo shut down the services and delete all their data::\n\n docker compose down -v\n\nIn addition to having these background services running, you may need to install\nspecific extra dependencies, like database drivers. Each example module has its required\ndependencies listed in the module comment at the top.\n\n.. _Docker: https://docs.docker.com/desktop/#download-and-install\n\nStandalone examples\n-------------------\n\nThe examples in the ``standalone`` directory demonstrate how to run the scheduler in the\nforeground, without anything else going on in the same process.\n\nThe directory contains four modules:\n\n- ``async_memory.py``: Basic asynchronous scheduler using the default memory-based data\n store\n- ``async_postgres.py``: Basic asynchronous scheduler using the asynchronous SQLAlchemy\n data store with a PostgreSQL back-end\n- ``async_mysql.py``: Basic asynchronous scheduler using the asynchronous SQLAlchemy\n data store with a MySQL back-end\n- ``sync_mysql.py``: Basic synchronous scheduler using the default memory-based data\n store\n\nSchedulers in web apps\n----------------------\n\nThe examples in the ``web`` directory demonstrate how to run the scheduler inside a web\napplication (ASGI_ or WSGI_).\n\nThe directory contains five modules:\n\n- ``asgi_noframework.py``: Trivial ASGI_ application, with middleware that starts and\n stops the scheduler as part of the ASGI lifecycle\n- ``asgi_fastapi.py``: Trivial FastAPI_ application, with middleware that starts and\n stops the scheduler as part of the ASGI_ lifecycle\n- ``asgi_starlette.py``: Trivial Starlette_ application, with middleware that starts and\n stops the scheduler as part of the ASGI_ lifecycle\n- ``wsgi_noframework.py``: Trivial WSGI_ application where the scheduler is started in a\n background thread\n- ``wsgi_flask.py``: Trivial Flask_ application where the scheduler is started in a\n background thread\n\n.. note:: There is no Django example available yet.\n\nTo run any of the ASGI_ examples::\n\n uvicorn :app\n\nTo run any of the WSGI_ examples::\n\n uwsgi -T --http :8000 --wsgi-file \n\n.. _ASGI: https://asgi.readthedocs.io/en/latest/introduction.html\n.. _WSGI: https://wsgi.readthedocs.io/en/latest/what.html\n.. _FastAPI: https://fastapi.tiangolo.com/\n.. _Starlette: https://www.starlette.io/\n.. _Flask: https://flask.palletsprojects.com/\n\nSeparate scheduler and worker\n-----------------------------\n\nThe example in the ``separate_worker`` directory demonstrates the ability to run\nschedulers and workers separately. The directory contains three modules:\n\n- ``sync_scheduler.py``: Runs a scheduler (without an internal worker) and adds/updates\n a schedule\n- ``sync_worker.py``: Runs a worker only\n- ``tasks.py``: Contains the task code (don't try to run this directly; it does nothing)\n\nThe reason for the task function being in a separate module is because when you run\neither the ``sync_scheduler`` or ``sync_worker`` script, that script is imported as the\n``__main__`` module, so if the scheduler schedules ``__main__:tick`` as the task, then\nthe worker would not be able to find it because its own script would also be named\n``__main__``.\n\nTo run the example, you need to have both the worker and scheduler scripts running at\nthe same time. To run the worker::\n\n python sync_worker.py\n\nTo run the scheduler::\n\n python sync_scheduler.py\n\nYou can run multiple schedulers and workers at the same time within this example. If you\nrun multiple workers, the message might be printed on the console of a different worker\neach time the job is run. Running multiple schedulers should have no visible effect, and\nas long as at least one scheduler is running, the scheduled task should keep running\nperiodically on one of the workers.\n" }, { "path": "examples/gui/qt_executor.py", "content": "from __future__ import annotations\n\nimport sys\nfrom datetime import datetime\n\nfrom apscheduler import Scheduler\nfrom apscheduler.executors.qt import QtJobExecutor\nfrom apscheduler.triggers.interval import IntervalTrigger\n\ntry:\n from PySide6.QtWidgets import QApplication, QLabel, QMainWindow\nexcept ImportError:\n try:\n from PyQt6.QtWidgets import QApplication, QLabel, QMainWindow\n except ImportError:\n raise ImportError(\"Either PySide6 or PyQt6 is needed to run this\") from None\n\n\nclass MainWindow(QMainWindow):\n def __init__(self):\n super().__init__()\n\n self.setWindowTitle(\"APScheduler demo\")\n\n self.clock = QLabel()\n font = self.clock.font()\n font.setPointSize(30)\n self.clock.setFont(font)\n\n self.update_time()\n self.setCentralWidget(self.clock)\n\n def update_time(self) -> None:\n now = datetime.now()\n self.clock.setText(f\"The time is now {now:%H:%M:%S}\")\n\n\napp = QApplication(sys.argv)\nwindow = MainWindow()\nwindow.show()\nwith Scheduler() as scheduler:\n scheduler.job_executors[\"qt\"] = QtJobExecutor()\n scheduler.add_schedule(\n window.update_time, IntervalTrigger(seconds=1), job_executor=\"qt\"\n )\n scheduler.start_in_background()\n app.exec()\n" }, { "path": "examples/separate_worker/async_scheduler.py", "content": "\"\"\"\nThis is an example demonstrating the use of the scheduler as only an interface to the\nscheduling system. This script adds or updates a single schedule and then exits. To see\nthe schedule acted on, you need to run the corresponding worker script (either\nasync_worker.py or sync_worker.py).\n\nThis script requires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg\nTo run: python async_scheduler.py\n\"\"\"\n\nfrom __future__ import annotations\n\nimport asyncio\nimport logging\n\nfrom example_tasks import tick\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\nasync def main():\n engine = create_async_engine(\n \"postgresql+asyncpg://postgres:secret@localhost/testdb\"\n )\n data_store = SQLAlchemyDataStore(engine)\n event_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n\n # Uncomment the next two lines to use the Redis event broker instead\n # from apscheduler.eventbrokers.redis import RedisEventBroker\n # event_broker = RedisEventBroker.from_url(\"redis://localhost\")\n\n async with AsyncScheduler(data_store, event_broker) as scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n # Note: we don't actually start the scheduler here!\n\n\nlogging.basicConfig(level=logging.INFO)\nasyncio.run(main())\n" }, { "path": "examples/separate_worker/async_worker.py", "content": "\"\"\"\nThis is an example demonstrating how to run a scheduler to process schedules added by\nanother scheduler elsewhere. Prior to starting this script, you need to run the script\n(either async_scheduler.py or sync_scheduler.py) that adds or updates a schedule to the\ndata store. This script will then pick up that schedule and start spawning jobs that\nwill print a line on the console on one-second intervals.\n\nThis script requires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg\nTo run: python async_worker.py\n\"\"\"\n\nfrom __future__ import annotations\n\nimport asyncio\nimport logging\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\n\n\nasync def main():\n async with AsyncScheduler(data_store, event_broker) as scheduler:\n await scheduler.run_until_stopped()\n\n\nlogging.basicConfig(level=logging.INFO)\nengine = create_async_engine(\"postgresql+asyncpg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n\n# Uncomment the next two lines to use the Redis event broker instead\n# from apscheduler.eventbrokers.redis import RedisEventBroker\n# event_broker = RedisEventBroker.from_url(\"redis://localhost\")\n\nasyncio.run(main())\n" }, { "path": "examples/separate_worker/example_tasks.py", "content": "\"\"\"\nThis module contains just the code for the scheduled task.\nIt should not be run directly.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n" }, { "path": "examples/separate_worker/sync_scheduler.py", "content": "\"\"\"\nThis is an example demonstrating the use of the scheduler as only an interface to the\nscheduling system. This script adds or updates a single schedule and then exits. To see\nthe schedule acted on, you need to run the corresponding worker script (either\nasync_worker.py or sync_worker.py).\n\nThis script requires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg\nTo run: python sync_scheduler.py\n\"\"\"\n\nfrom __future__ import annotations\n\nimport logging\n\nfrom example_tasks import tick\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import Scheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\nlogging.basicConfig(level=logging.INFO)\nengine = create_async_engine(\"postgresql+asyncpg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n\n# Uncomment the next two lines to use the MQTT event broker instead\n# from apscheduler.eventbrokers.mqtt import MQTTEventBroker\n# event_broker = MQTTEventBroker()\n\nwith Scheduler(data_store, event_broker) as scheduler:\n scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n # Note: we don't actually start the scheduler here!\n" }, { "path": "examples/separate_worker/sync_worker.py", "content": "\"\"\"\nThis is an example demonstrating how to run a scheduler to process schedules added by\nanother scheduler elsewhere. Prior to starting this script, you need to run the script\n(either async_scheduler.py or sync_scheduler.py) that adds or updates a schedule to the\ndata store. This script will then pick up that schedule and start spawning jobs that\nwill print a line on the console on one-second intervals.\n\nThis script requires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg\nTo run: python sync_worker.py\n\"\"\"\n\nfrom __future__ import annotations\n\nimport logging\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import Scheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\n\nlogging.basicConfig(level=logging.INFO)\nengine = create_async_engine(\"postgresql+asyncpg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n\n# Uncomment the next two lines to use the MQTT event broker instead\n# from apscheduler.eventbrokers.mqtt import MQTTEventBroker\n# event_broker = MQTTEventBroker()\n\nwith Scheduler(data_store, event_broker) as scheduler:\n scheduler.run_until_stopped()\n" }, { "path": "examples/standalone/async_memory.py", "content": "\"\"\"\nExample demonstrating use of the asynchronous scheduler in a simple asyncio app.\n\nTo run: python async_memory.py\n\nIt should print a line on the console on a one-second interval.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom asyncio import run\nfrom datetime import datetime\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nasync def main():\n async with AsyncScheduler() as scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1))\n await scheduler.run_until_stopped()\n\n\nrun(main())\n" }, { "path": "examples/standalone/async_mysql.py", "content": "\"\"\"\nExample demonstrating use of the asynchronous scheduler with persistence via MySQL or\nMariaDB in a simple asyncio app.\n\nRequires the \"mysql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncmy\nTo run: python async_mysql.py\n\nIt should print a line on the console on a one-second interval.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom asyncio import run\nfrom datetime import datetime\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nasync def main():\n engine = create_async_engine(\n \"mysql+asyncmy://root:secret@localhost/testdb?charset=utf8mb4\"\n )\n data_store = SQLAlchemyDataStore(engine)\n async with AsyncScheduler(data_store) as scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n await scheduler.run_until_stopped()\n\n\nrun(main())\n" }, { "path": "examples/standalone/async_postgres.py", "content": "\"\"\"\nExample demonstrating use of the asynchronous scheduler with persistence via PostgreSQL\nin a simple asyncio app.\n\nRequires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg\nTo run: python async_postgres.py\n\nIt should print a line on the console on a one-second interval.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom asyncio import run\nfrom datetime import datetime\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nasync def main():\n engine = create_async_engine(\n \"postgresql+asyncpg://postgres:secret@localhost/testdb\"\n )\n data_store = SQLAlchemyDataStore(engine)\n async with AsyncScheduler(data_store) as scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n await scheduler.run_until_stopped()\n\n\nrun(main())\n" }, { "path": "examples/standalone/sync_memory.py", "content": "\"\"\"\nExample demonstrating use of the synchronous scheduler.\n\nTo run: python sync_memory.py\n\nIt should print a line on the console on a one-second interval.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\nfrom apscheduler import Scheduler\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nwith Scheduler() as scheduler:\n scheduler.add_schedule(tick, IntervalTrigger(seconds=1))\n scheduler.run_until_stopped()\n" }, { "path": "examples/web/asgi_fastapi.py", "content": "\"\"\"\nExample demonstrating use with the FastAPI web framework.\n\nRequires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asycnpg fastapi uvicorn\nTo run: uvicorn asgi_fastapi:app\n\nIt should print a line on the console on a one-second interval while running a\nbasic web app at http://localhost:8000.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom collections.abc import AsyncGenerator\nfrom contextlib import asynccontextmanager\nfrom datetime import datetime\n\nfrom fastapi import FastAPI\nfrom fastapi.responses import PlainTextResponse, Response\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\n@asynccontextmanager\nasync def lifespan(app: FastAPI) -> AsyncGenerator[None]:\n engine = create_async_engine(\n \"postgresql+asyncpg://postgres:secret@localhost/testdb\"\n )\n data_store = SQLAlchemyDataStore(engine)\n event_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n scheduler = AsyncScheduler(data_store, event_broker)\n\n async with scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n await scheduler.start_in_background()\n yield\n\n\nasync def root() -> Response:\n return PlainTextResponse(\"Hello, world!\")\n\n\napp = FastAPI(lifespan=lifespan)\napp.add_api_route(\"/\", root)\n" }, { "path": "examples/web/asgi_noframework.py", "content": "\"\"\"\nExample demonstrating use with ASGI (raw ASGI application, no framework).\n\nRequires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asyncpg uvicorn\nTo run: uvicorn asgi_noframework:app\n\nIt should print a line on the console on a one-second interval while running a\nbasic web app at http://localhost:8000.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nasync def original_app(scope, receive, send):\n \"\"\"Trivial example of an ASGI application.\"\"\"\n if scope[\"type\"] == \"http\":\n await receive()\n await send(\n {\n \"type\": \"http.response.start\",\n \"status\": 200,\n \"headers\": [\n [b\"content-type\", b\"text/plain\"],\n ],\n }\n )\n await send(\n {\n \"type\": \"http.response.body\",\n \"body\": b\"Hello, world!\",\n \"more_body\": False,\n }\n )\n elif scope[\"type\"] == \"lifespan\":\n while True:\n message = await receive()\n if message[\"type\"] == \"lifespan.startup\":\n await send({\"type\": \"lifespan.startup.complete\"})\n elif message[\"type\"] == \"lifespan.shutdown\":\n await send({\"type\": \"lifespan.shutdown.complete\"})\n return\n\n\nasync def scheduler_middleware(scope, receive, send):\n if scope[\"type\"] == \"lifespan\":\n engine = create_async_engine(\n \"postgresql+asyncpg://postgres:secret@localhost/testdb\"\n )\n data_store = SQLAlchemyDataStore(engine)\n event_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n async with AsyncScheduler(data_store, event_broker) as scheduler:\n await scheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\n await scheduler.start_in_background()\n await original_app(scope, receive, send)\n else:\n await original_app(scope, receive, send)\n\n\n# This is just for consistency with the other ASGI examples\napp = scheduler_middleware\n" }, { "path": "examples/web/asgi_starlette.py", "content": "\"\"\"\nExample demonstrating use with the Starlette web framework.\n\nRequires the \"postgresql\" service to be running.\nTo install prerequisites: pip install sqlalchemy asycnpg starlette uvicorn\nTo run: uvicorn asgi_starlette:app\n\nIt should print a line on the console on a one-second interval while running a\nbasic web app at http://localhost:8000.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\nfrom sqlalchemy.ext.asyncio import create_async_engine\nfrom starlette.applications import Starlette\nfrom starlette.middleware import Middleware\nfrom starlette.requests import Request\nfrom starlette.responses import PlainTextResponse, Response\nfrom starlette.routing import Route\nfrom starlette.types import ASGIApp, Receive, Scope, Send\n\nfrom apscheduler import AsyncScheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\nclass SchedulerMiddleware:\n def __init__(\n self,\n app: ASGIApp,\n scheduler: AsyncScheduler,\n ) -> None:\n self.app = app\n self.scheduler = scheduler\n\n async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:\n if scope[\"type\"] == \"lifespan\":\n async with self.scheduler:\n await self.scheduler.add_schedule(\n tick, IntervalTrigger(seconds=1), id=\"tick\"\n )\n await self.scheduler.start_in_background()\n await self.app(scope, receive, send)\n else:\n await self.app(scope, receive, send)\n\n\nasync def root(request: Request) -> Response:\n return PlainTextResponse(\"Hello, world!\")\n\n\nengine = create_async_engine(\"postgresql+asyncpg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\nscheduler = AsyncScheduler(data_store, event_broker)\nroutes = [Route(\"/\", root)]\nmiddleware = [Middleware(SchedulerMiddleware, scheduler=scheduler)]\napp = Starlette(routes=routes, middleware=middleware)\n" }, { "path": "examples/web/wsgi_flask.py", "content": "\"\"\"\nExample demonstrating use with WSGI (raw WSGI application, no framework).\n\nRequires the \"postgresql\" and \"redis\" services to be running.\nTo install prerequisites: pip install sqlalchemy psycopg flask uwsgi\nTo run: uwsgi -T --http :8000 --wsgi-file wsgi_flask.py\n\nIt should print a line on the console on a one-second interval while running a\nbasic web app at http://localhost:8000.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\nfrom flask import Flask\nfrom sqlalchemy.future import create_engine\n\nfrom apscheduler import Scheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.redis import RedisEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\napp = Flask(__name__)\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\n@app.route(\"/\")\ndef hello_world():\n return \"

Hello, World!

\"\n\n\nengine = create_engine(\"postgresql+psycopg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = RedisEventBroker(\"redis://localhost\")\nscheduler = Scheduler(data_store, event_broker)\nscheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\nscheduler.start_in_background()\n" }, { "path": "examples/web/wsgi_noframework.py", "content": "\"\"\"\nExample demonstrating use with WSGI (raw WSGI application, no framework).\n\nRequires the \"postgresql\" and \"redis\" services to be running.\nTo install prerequisites: pip install sqlalchemy psycopg uwsgi\nTo run: uwsgi -T --http :8000 --wsgi-file wsgi_noframework.py\n\nIt should print a line on the console on a one-second interval while running a\nbasic web app at http://localhost:8000.\n\"\"\"\n\nfrom __future__ import annotations\n\nfrom datetime import datetime\n\nfrom sqlalchemy.future import create_engine\n\nfrom apscheduler import Scheduler\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.eventbrokers.redis import RedisEventBroker\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef tick():\n print(\"Hello, the time is\", datetime.now())\n\n\ndef application(environ, start_response):\n response_body = b\"Hello, World!\"\n response_headers = [\n (\"Content-Type\", \"text/plain\"),\n (\"Content-Length\", str(len(response_body))),\n ]\n start_response(\"200 OK\", response_headers)\n return [response_body]\n\n\nengine = create_engine(\"postgresql+psycopg://postgres:secret@localhost/testdb\")\ndata_store = SQLAlchemyDataStore(engine)\nevent_broker = RedisEventBroker(\"redis://localhost\")\nscheduler = Scheduler(data_store, event_broker)\nscheduler.add_schedule(tick, IntervalTrigger(seconds=1), id=\"tick\")\nscheduler.start_in_background()\n" }, { "path": "pyproject.toml", "content": "[build-system]\nrequires = [\n \"setuptools >= 77\",\n \"setuptools_scm >= 6.4\"\n]\nbuild-backend = \"setuptools.build_meta\"\n\n[project]\nname = \"APScheduler\"\ndescription = \"In-process task scheduler with Cron-like capabilities\"\nreadme = \"README.rst\"\nauthors = [{name = \"Alex Grönholm\", email = \"alex.gronholm@nextday.fi\"}]\nclassifiers = [\n \"Development Status :: 5 - Production/Stable\",\n \"Intended Audience :: Developers\",\n \"Framework :: AnyIO\",\n \"Typing :: Typed\",\n \"Programming Language :: Python\",\n \"Programming Language :: Python :: 3 :: Only\",\n \"Programming Language :: Python :: 3.10\",\n \"Programming Language :: Python :: 3.11\",\n \"Programming Language :: Python :: 3.12\",\n \"Programming Language :: Python :: 3.13\",\n \"Programming Language :: Python :: 3.14\",\n]\nkeywords = [\"scheduling\", \"cron\"]\nlicense = \"MIT\"\nrequires-python = \">= 3.10\"\ndependencies = [\n \"anyio ~= 4.0\",\n \"attrs >= 22.1\",\n \"tenacity >= 8.0, < 10.0\",\n \"tzlocal >= 3.0\",\n \"typing_extensions >= 4.0; python_version < '3.11'\"\n]\ndynamic = [\"version\"]\n\n[project.urls]\nDocumentation = \"https://apscheduler.readthedocs.io/en/master/\"\nChangelog = \"https://apscheduler.readthedocs.io/en/master/versionhistory.html\"\n\"Source code\" = \"https://github.com/agronholm/apscheduler\"\n\"Issue tracker\" = \"https://github.com/agronholm/apscheduler/issues\"\n\n[project.optional-dependencies]\nasyncpg = [\"asyncpg >= 0.20\"]\ncbor = [\"cbor2 >= 5.0\"]\nmongodb = [\"pymongo >= 4.13.0\"]\nmqtt = [\"paho-mqtt >= 2.0\"]\nredis = [\"redis >= 5.0.1\"]\nsqlalchemy = [\"sqlalchemy[asyncio] >= 2.0.24\"]\n\n[dependency-groups]\ntest = [\n \"APScheduler[cbor,mongodb,mqtt,redis,sqlalchemy]\",\n \"asyncpg >= 0.20; python_implementation == 'CPython' and python_version < '3.13'\",\n \"aiosqlite >= 0.19\",\n \"anyio[trio]\",\n \"asyncmy >= 0.2.5; python_implementation == 'CPython'\",\n \"coverage >= 7\",\n \"psycopg[binary]\",\n \"pymongo >= 4\",\n \"pymysql[rsa]\",\n \"PySide6 >= 6.6; python_implementation == 'CPython' and python_version < '3.13'\",\n \"pytest >= 7.4\",\n \"pytest-lazy-fixtures\",\n \"pytest-mock\",\n \"time-machine >= 2.13.0; python_implementation == 'CPython'\",\n \"\"\"\\\n uwsgi >= 2.0.31; python_implementation == 'CPython' and platform_system == 'Linux'\\\n and python_version < '3.13'\\\n \"\"\",\n]\ndoc = [\n \"sphinx\",\n \"sphinx-autodoc-typehints >= 2.2.3\",\n \"sphinx-rtd-theme >= 1.3.0\",\n \"sphinx-tabs >= 3.3.1\",\n]\n\n[tool.setuptools_scm]\nversion_scheme = \"post-release\"\nlocal_scheme = \"dirty-tag\"\n\n[tool.pytest.ini_options]\naddopts = \"-rsx --tb=short\"\ntestpaths = \"tests\"\nfilterwarnings = \"always\"\nmarkers = [\n \"external_service: marks tests as requiring some external service\",\n]\n\n[tool.coverage.run]\nsource = [\"apscheduler\"]\n\n[tool.coverage.report]\nshow_missing = true\n\n[tool.ruff.lint]\nextend-select = [\n \"ASYNC\", # flake8-async\n \"B\", # flake8-bugbear\n \"C4\", # flake8-comprehensions\n \"G\", # flake8-logging-format\n \"I\", # isort\n \"ISC\", # flake8-implicit-str-concat\n \"PERF\", # flake8-performance\n \"PGH\", # pygrep-hooks\n \"RUF100\", # unused noqa (yesqa)\n \"T201\", # print\n \"UP\", # pyupgrade\n \"W\", # pycodestyle warnings\n]\nignore = [\n \"PERF203\",\n \"RUF001\",\n \"RUF002\",\n]\n\n[tool.ruff.lint.isort]\nknown-first-party = [\"apscheduler\"]\nrequired-imports = [\"from __future__ import annotations\"]\n\n[tool.ruff.lint.per-file-ignores]\n\"examples/**/*.py\" = [\"T201\"]\n\n[tool.mypy]\npython_version = \"3.14\"\nignore_missing_imports = true\ndisable_error_code = \"type-abstract\"\n\n[tool.tox]\nenv_list = [\"py310\", \"py311\", \"py312\", \"py313\", \"py314\", \"pypy3\"]\nskip_missing_interpreters = true\nrequires = [\"tox >= 4.22\"]\n\n[tool.tox.env_run_base]\ncommands = [[\"pytest\", { replace = \"posargs\", extend = true }]]\npackage = \"editable\"\ndependency_groups = [\"test\"]\n\n[tool.tox.env.pyright]\ncommands = [[\"pyright\", \"--verifytypes\", \"apscheduler\"]]\ndeps = [\"pyright\"]\n\n[tool.tox.env.docs]\ncommands = [[\"sphinx-build\", \"-W\", \"-n\", \"docs\", \"build/sphinx\"]]\ndependency_groups = [\"doc\"]\n" }, { "path": "src/apscheduler/__init__.py", "content": "from __future__ import annotations\n\nfrom typing import Any\n\nfrom ._context import current_async_scheduler as current_async_scheduler\nfrom ._context import current_job as current_job\nfrom ._context import current_scheduler as current_scheduler\nfrom ._decorators import task as task\nfrom ._enums import CoalescePolicy as CoalescePolicy\nfrom ._enums import ConflictPolicy as ConflictPolicy\nfrom ._enums import JobOutcome as JobOutcome\nfrom ._enums import RunState as RunState\nfrom ._enums import SchedulerRole as SchedulerRole\nfrom ._events import DataStoreEvent as DataStoreEvent\nfrom ._events import Event as Event\nfrom ._events import JobAcquired as JobAcquired\nfrom ._events import JobAdded as JobAdded\nfrom ._events import JobDeserializationFailed as JobDeserializationFailed\nfrom ._events import JobReleased as JobReleased\nfrom ._events import JobRemoved as JobRemoved\nfrom ._events import ScheduleAdded as ScheduleAdded\nfrom ._events import ScheduleDeserializationFailed as ScheduleDeserializationFailed\nfrom ._events import ScheduleRemoved as ScheduleRemoved\nfrom ._events import SchedulerEvent as SchedulerEvent\nfrom ._events import SchedulerStarted as SchedulerStarted\nfrom ._events import SchedulerStopped as SchedulerStopped\nfrom ._events import ScheduleUpdated as ScheduleUpdated\nfrom ._events import TaskAdded as TaskAdded\nfrom ._events import TaskRemoved as TaskRemoved\nfrom ._events import TaskUpdated as TaskUpdated\nfrom ._exceptions import CallableLookupError as CallableLookupError\nfrom ._exceptions import ConflictingIdError as ConflictingIdError\nfrom ._exceptions import DeserializationError as DeserializationError\nfrom ._exceptions import JobCancelled as JobCancelled\nfrom ._exceptions import JobDeadlineMissed as JobDeadlineMissed\nfrom ._exceptions import JobLookupError as JobLookupError\nfrom ._exceptions import JobResultNotReady as JobResultNotReady\nfrom ._exceptions import MaxIterationsReached as MaxIterationsReached\nfrom ._exceptions import ScheduleLookupError as ScheduleLookupError\nfrom ._exceptions import SerializationError as SerializationError\nfrom ._exceptions import TaskLookupError as TaskLookupError\nfrom ._retry import RetryMixin as RetryMixin\nfrom ._retry import RetrySettings as RetrySettings\nfrom ._schedulers.async_ import AsyncScheduler as AsyncScheduler\nfrom ._schedulers.sync import Scheduler as Scheduler\nfrom ._structures import Job as Job\nfrom ._structures import JobResult as JobResult\nfrom ._structures import Schedule as Schedule\nfrom ._structures import ScheduleResult as ScheduleResult\nfrom ._structures import Task as Task\nfrom ._structures import TaskDefaults as TaskDefaults\nfrom ._utils import UnsetValue as UnsetValue\n\n# Re-export imports, so they look like they live directly in this package\nvalue: Any\nfor value in list(locals().values()):\n if getattr(value, \"__module__\", \"\").startswith(\"apscheduler.\"):\n value.__module__ = __name__\n" }, { "path": "src/apscheduler/_context.py", "content": "from __future__ import annotations\n\nfrom contextvars import ContextVar\nfrom typing import TYPE_CHECKING\n\nif TYPE_CHECKING:\n from ._schedulers.async_ import AsyncScheduler\n from ._schedulers.sync import Scheduler\n from ._structures import Job\n\n#: The currently running (local) scheduler\ncurrent_scheduler: ContextVar[Scheduler | None] = ContextVar(\n \"current_scheduler\", default=None\n)\ncurrent_async_scheduler: ContextVar[AsyncScheduler | None] = ContextVar(\n \"current_async_scheduler\", default=None\n)\n#: Metadata about the current job\ncurrent_job: ContextVar[Job] = ContextVar(\"job_info\")\n" }, { "path": "src/apscheduler/_converters.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom datetime import date, datetime, timedelta, timezone, tzinfo\nfrom typing import Any\nfrom uuid import UUID\nfrom zoneinfo import ZoneInfo\n\nfrom tzlocal import get_localzone\n\n\ndef as_int(value: int | str) -> int:\n if isinstance(value, str):\n return int(value)\n\n return value\n\n\ndef as_datetime(value: datetime | str) -> datetime:\n if isinstance(value, str):\n # Before Python 3.11, fromisoformat() could not handle the \"Z\" suffix\n if value.upper().endswith(\"Z\"):\n value = value[:-1] + \"+00:00\"\n\n value = datetime.fromisoformat(value)\n\n return value\n\n\ndef as_aware_datetime(value: datetime | str) -> datetime:\n value_as_datetime = as_datetime(value)\n if isinstance(value_as_datetime, datetime) and value_as_datetime.tzinfo is None:\n value_as_datetime = value_as_datetime.astimezone(get_localzone())\n\n return value_as_datetime\n\n\ndef as_date(value: date | str) -> date:\n if isinstance(value, str):\n return date.fromisoformat(value)\n\n return value\n\n\ndef as_timezone(value: tzinfo | str) -> tzinfo:\n if isinstance(value, str):\n return get_localzone() if value == \"local\" else ZoneInfo(value)\n elif value is timezone.utc:\n return ZoneInfo(\"UTC\")\n\n return value\n\n\ndef as_uuid(value: UUID | str) -> UUID:\n if isinstance(value, str):\n return UUID(value)\n\n return value\n\n\ndef as_timedelta(value: timedelta | int) -> timedelta:\n if isinstance(value, (float, int)):\n return timedelta(seconds=value)\n\n return value\n\n\ndef as_enum(enum_class: Any) -> Callable[[Any], Any]:\n def converter(value: Any) -> Any:\n if isinstance(value, str):\n return enum_class[value]\n\n return value\n\n return converter\n\n\ndef list_converter(converter: Callable[[Any], Any]) -> Callable[[Any], Any]:\n def convert(value: Any) -> Any:\n if isinstance(value, list):\n return [converter(item) for item in value]\n\n return value\n\n return convert\n" }, { "path": "src/apscheduler/_decorators.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom datetime import timedelta\nfrom typing import Any, TypeVar\n\nimport attrs\nfrom attr.validators import instance_of, optional\n\nfrom ._converters import as_timedelta\nfrom ._structures import MetadataType, TaskDefaults\nfrom ._utils import UnsetValue, unset\nfrom ._validators import if_not_unset, valid_metadata\n\nT = TypeVar(\"T\", bound=\"Callable[..., Any]\")\n\nTASK_PARAMETERS_KEY = \"_apscheduler_taskdef\"\n\n\n@attrs.define(kw_only=True)\nclass TaskParameters(TaskDefaults):\n id: str | UnsetValue = attrs.field(default=unset)\n job_executor: str | UnsetValue = attrs.field(\n validator=if_not_unset(instance_of(str)), default=unset\n )\n max_running_jobs: int | None | UnsetValue = attrs.field(\n validator=if_not_unset(optional(instance_of(int))), default=unset\n )\n misfire_grace_time: timedelta | None | UnsetValue = attrs.field(\n converter=as_timedelta,\n validator=if_not_unset(optional(instance_of(timedelta))),\n default=unset,\n )\n metadata: MetadataType | UnsetValue = attrs.field(\n validator=if_not_unset(valid_metadata), default=unset\n )\n\n\ndef task(\n id: str | UnsetValue = unset,\n *,\n job_executor: str | UnsetValue = unset,\n max_running_jobs: int | None | UnsetValue = unset,\n misfire_grace_time: int | timedelta | None | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n) -> Callable[[T], T]:\n \"\"\"\n Decorate a function to have implied defaults as an APScheduler task.\n\n :param id: the task ID to use\n :param str job_executor: name of the job executor that will run the task\n :param int | None max_running_jobs: maximum number of instances of the task that are\n allowed to run concurrently\n :param ~datetime.timedelta | None misfire_grace_time: maximum number of seconds the\n run time of jobs created for the task are allowed to be late, compared to the\n scheduled run time\n :param metadata: key-value pairs for storing JSON compatible custom information\n \"\"\"\n\n def wrapper(func: T) -> T:\n if not isinstance(func, Callable):\n raise ValueError(\"only functions can be decorated with @task\")\n\n if hasattr(func, TASK_PARAMETERS_KEY):\n raise ValueError(\n \"this function already has APScheduler task parameters set\"\n )\n\n setattr(\n func,\n TASK_PARAMETERS_KEY,\n TaskParameters(\n id=id,\n job_executor=job_executor,\n max_running_jobs=max_running_jobs,\n misfire_grace_time=misfire_grace_time,\n metadata=metadata,\n ),\n )\n return func\n\n return wrapper\n\n\ndef get_task_params(func: Callable[..., Any]) -> TaskParameters:\n return getattr(func, TASK_PARAMETERS_KEY, None) or TaskParameters()\n" }, { "path": "src/apscheduler/_enums.py", "content": "from __future__ import annotations\n\nfrom enum import Enum, auto\n\n\nclass SchedulerRole(Enum):\n \"\"\"\n Specifies what the scheduler should be doing when it's running.\n\n .. attribute:: scheduler\n\n processes due schedules, but won't run jobs\n\n .. attribute:: worker\n\n runs due jobs, but won't process schedules\n\n .. attribute:: both\n\n processes schedules and runs due jobs\n \"\"\"\n\n scheduler = auto()\n worker = auto()\n both = auto()\n\n\nclass RunState(Enum):\n \"\"\"\n Used to track the running state of schedulers.\n\n .. attribute:: starting\n\n not running yet, but in the process of starting\n\n .. attribute:: started\n\n running\n\n .. attribute:: stopping\n\n still running but in the process of shutting down\n\n .. attribute:: stopped\n\n not running\n \"\"\"\n\n starting = auto()\n started = auto()\n stopping = auto()\n stopped = auto()\n\n\nclass JobOutcome(Enum):\n \"\"\"\n Used to indicate how the execution of a job ended.\n\n .. attribute:: success\n\n the job completed successfully\n\n .. attribute:: error\n\n the job raised an exception\n\n .. attribute:: missed_start_deadline\n\n the job's execution was delayed enough for it to miss its start deadline\n (scheduled time + misfire grace time)\n\n .. attribute:: deserialization_failed\n\n the deserialization operation failed\n\n .. attribute:: cancelled\n\n the job's execution was cancelled\n\n .. attribute:: abandoned\n\n the worker running the job stopped unexpectedly and the job was never marked\n as done\n \"\"\"\n\n success = auto()\n error = auto()\n missed_start_deadline = auto()\n deserialization_failed = auto()\n cancelled = auto()\n abandoned = auto()\n\n\nclass ConflictPolicy(Enum):\n \"\"\"\n Used to indicate what to do when trying to add a schedule whose ID conflicts with an\n existing schedule.\n\n .. attribute:: replace\n\n replace the existing schedule with a new one\n\n .. attribute:: do_nothing\n\n keep the existing schedule as-is and drop the new schedule\n\n .. attribute:: exception\n\n raise an exception if a conflict is detected\n \"\"\"\n\n replace = auto()\n do_nothing = auto()\n exception = auto()\n\n\nclass CoalescePolicy(Enum):\n \"\"\"\n Used to indicate how to queue jobs for a schedule that has accumulated multiple\n run times since the last scheduler iteration.\n\n .. attribute:: earliest\n\n run once, with the earliest fire time\n\n .. attribute:: latest\n\n run once, with the latest fire time\n\n .. attribute:: all\n\n submit one job for every accumulated fire time\n \"\"\"\n\n earliest = auto()\n latest = auto()\n all = auto()\n" }, { "path": "src/apscheduler/_events.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timezone\nfrom functools import partial\nfrom traceback import format_tb\nfrom typing import Any, TypeVar\nfrom uuid import UUID\n\nimport attrs\nfrom attrs.converters import optional\n\nfrom ._converters import as_aware_datetime, as_enum, as_uuid\nfrom ._enums import JobOutcome\nfrom ._structures import Job, JobResult\nfrom ._utils import qualified_name\n\nT_Event = TypeVar(\"T_Event\", bound=\"Event\")\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass Event:\n \"\"\"\n Base class for all events.\n\n :ivar timestamp: the time when the event occurred\n \"\"\"\n\n timestamp: datetime = attrs.field(\n factory=partial(datetime.now, timezone.utc), converter=as_aware_datetime\n )\n\n def marshal(self) -> dict[str, Any]:\n return attrs.asdict(self)\n\n @classmethod\n def unmarshal(cls, marshalled: dict[str, Any]) -> Event:\n return cls(**marshalled)\n\n\n#\n# Data store events\n#\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass DataStoreEvent(Event):\n \"\"\"Base class for events originating from a data store.\"\"\"\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass TaskAdded(DataStoreEvent):\n \"\"\"\n Signals that a new task was added to the store.\n\n :ivar task_id: ID of the task that was added\n \"\"\"\n\n task_id: str\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass TaskUpdated(DataStoreEvent):\n \"\"\"\n Signals that a task was updated in a data store.\n\n :ivar task_id: ID of the task that was updated\n \"\"\"\n\n task_id: str\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass TaskRemoved(DataStoreEvent):\n \"\"\"\n Signals that a task was removed from the store.\n\n :ivar task_id: ID of the task that was removed\n \"\"\"\n\n task_id: str\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass ScheduleAdded(DataStoreEvent):\n \"\"\"\n Signals that a new schedule was added to the store.\n\n :ivar schedule_id: ID of the schedule that was added\n :ivar task_id: ID of the task the schedule belongs to\n :ivar next_fire_time: the first run time calculated for the schedule\n \"\"\"\n\n schedule_id: str\n task_id: str\n next_fire_time: datetime | None = attrs.field(converter=optional(as_aware_datetime))\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass ScheduleUpdated(DataStoreEvent):\n \"\"\"\n Signals that a schedule has been updated in the store.\n\n :ivar schedule_id: ID of the schedule that was updated\n :ivar task_id: ID of the task the schedule belongs to\n :ivar next_fire_time: the next time the schedule will run\n \"\"\"\n\n schedule_id: str\n task_id: str\n next_fire_time: datetime | None = attrs.field(converter=optional(as_aware_datetime))\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass ScheduleRemoved(DataStoreEvent):\n \"\"\"\n Signals that a schedule was removed from the store.\n\n :ivar schedule_id: ID of the schedule that was removed\n :ivar task_id: ID of the task the schedule belongs to\n :ivar finished: ``True`` if the schedule was removed automatically because its\n trigger had no more fire times left\n \"\"\"\n\n schedule_id: str\n task_id: str\n finished: bool\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass JobAdded(DataStoreEvent):\n \"\"\"\n Signals that a new job was added to the store.\n\n :ivar job_id: ID of the job that was added\n :ivar task_id: ID of the task the job would run\n :ivar schedule_id: ID of the schedule the job was created from\n \"\"\"\n\n job_id: UUID = attrs.field(converter=as_uuid)\n task_id: str\n schedule_id: str | None\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass JobRemoved(DataStoreEvent):\n \"\"\"\n Signals that a job was removed from the store.\n\n :ivar job_id: ID of the job that was removed\n :ivar task_id: ID of the task the job would have run\n\n \"\"\"\n\n job_id: UUID = attrs.field(converter=as_uuid)\n task_id: str\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass ScheduleDeserializationFailed(DataStoreEvent):\n \"\"\"\n Signals that the deserialization of a schedule has failed.\n\n :ivar schedule_id: ID of the schedule that failed to deserialize\n :ivar exception: the exception that was raised during deserialization\n \"\"\"\n\n schedule_id: str\n exception: BaseException\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass JobDeserializationFailed(DataStoreEvent):\n \"\"\"\n Signals that the deserialization of a job has failed.\n\n :ivar job_id: ID of the job that failed to deserialize\n :ivar exception: the exception that was raised during deserialization\n \"\"\"\n\n job_id: UUID = attrs.field(converter=as_uuid)\n exception: BaseException\n\n\n#\n# Scheduler events\n#\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass SchedulerEvent(Event):\n \"\"\"Base class for events originating from a scheduler.\"\"\"\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass SchedulerStarted(SchedulerEvent):\n pass\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass SchedulerStopped(SchedulerEvent):\n \"\"\"\n Signals that a scheduler has stopped.\n\n :ivar exception: the exception that caused the scheduler to stop, if any\n \"\"\"\n\n exception: BaseException | None = None\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass JobAcquired(SchedulerEvent):\n \"\"\"\n Signals that a scheduler has acquired a job for processing.\n\n :param job_id: the ID of the job that was acquired\n :param scheduler_id: the ID of the scheduler that acquired the job\n :param task_id: ID of the task the job belongs to\n :param schedule_id: ID of the schedule that\n :param scheduled_start: the time the job was scheduled to start via a schedule (if\n any)\n \"\"\"\n\n job_id: UUID = attrs.field(converter=as_uuid)\n scheduler_id: str\n task_id: str\n schedule_id: str | None = None\n scheduled_start: datetime | None = attrs.field(converter=as_aware_datetime)\n\n @classmethod\n def from_job(cls, job: Job, scheduler_id: str) -> JobAcquired:\n \"\"\"\n Create a new job-acquired event from a job and a scheduler ID.\n\n :param job: the job that was acquired\n :param scheduler_id: the ID of the scheduler that acquired the job\n :return: a new job-acquired event\n\n \"\"\"\n return cls(\n job_id=job.id,\n scheduler_id=scheduler_id,\n task_id=job.task_id,\n schedule_id=job.schedule_id,\n scheduled_start=job.scheduled_fire_time,\n )\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass JobReleased(SchedulerEvent):\n \"\"\"\n Signals that a scheduler has finished processing of a job.\n\n :param uuid.UUID job_id: the ID of the job that was released\n :param scheduler_id: the ID of the scheduler that released the job\n :param task_id: ID of the task run by the job\n :param schedule_id: ID of the schedule (if any) that created the job\n :param scheduled_start: the time the job was scheduled to start via the schedule (if\n any)\n :param started_at: the time the executor actually started running the job (``None``\n if the job was skipped due to missing its start deadline)\n :param outcome: the outcome of the job\n :param exception_type: the fully qualified name of the exception if ``outcome`` is\n :attr:`JobOutcome.error`\n :param exception_message: the result of ``str(exception)`` if ``outcome`` is\n :attr:`JobOutcome.error`\n :param exception_traceback: the traceback lines from the exception if ``outcome`` is\n :attr:`JobOutcome.error`\n \"\"\"\n\n job_id: UUID = attrs.field(converter=as_uuid)\n scheduler_id: str\n task_id: str\n schedule_id: str | None = None\n scheduled_start: datetime | None = attrs.field(converter=as_aware_datetime)\n started_at: datetime | None = attrs.field(converter=as_aware_datetime)\n outcome: JobOutcome = attrs.field(converter=as_enum(JobOutcome))\n exception_type: str | None = None\n exception_message: str | None = None\n exception_traceback: list[str] | None = None\n\n @classmethod\n def from_result(\n cls,\n result: JobResult,\n scheduler_id: str,\n task_id: str,\n schedule_id: str | None,\n scheduled_fire_time: datetime | None = None,\n ) -> JobReleased:\n \"\"\"\n Create a new job-released event from a job, the job result and a scheduler ID.\n\n :param result: the result of the job\n :param scheduler_id: the ID of the scheduler that acquired the job\n :param task_id: the job's task ID\n :param schedule_id: ID of the schedule (if any) from which the job was spawned\n :param scheduled_fire_time: the time the job was scheduled to start (if the job\n was spawned from a schedule)\n :return: a new job-released event\n\n \"\"\"\n if result.exception is not None:\n exception_type: str | None = qualified_name(result.exception.__class__)\n exception_message: str | None = str(result.exception)\n exception_traceback: list[str] | None = format_tb(\n result.exception.__traceback__\n )\n else:\n exception_type = exception_message = exception_traceback = None\n\n return cls(\n timestamp=result.finished_at,\n job_id=result.job_id,\n scheduler_id=scheduler_id,\n task_id=task_id,\n schedule_id=schedule_id,\n outcome=result.outcome,\n scheduled_start=scheduled_fire_time,\n started_at=result.started_at,\n exception_type=exception_type,\n exception_message=exception_message,\n exception_traceback=exception_traceback,\n )\n\n def marshal(self) -> dict[str, Any]:\n marshalled = super().marshal()\n return marshalled\n" }, { "path": "src/apscheduler/_exceptions.py", "content": "from __future__ import annotations\n\nfrom uuid import UUID\n\n\nclass TaskLookupError(LookupError):\n \"\"\"Raised by a data store when it cannot find the requested task.\"\"\"\n\n def __init__(self, task_id: str):\n super().__init__(f\"No task by the id of {task_id!r} was found\")\n\n\nclass ScheduleLookupError(LookupError):\n \"\"\"Raised by a scheduler when it cannot find the requested schedule.\"\"\"\n\n def __init__(self, schedule_id: str):\n super().__init__(f\"No schedule by the id of {schedule_id!r} was found\")\n\n\nclass JobLookupError(LookupError):\n \"\"\"Raised when the job store cannot find a job for update or removal.\"\"\"\n\n def __init__(self, job_id: UUID):\n super().__init__(f\"No job by the id of {job_id} was found\")\n\n\nclass CallableLookupError(LookupError):\n \"\"\"Raised when the target callable for a job could not be found.\"\"\"\n\n\nclass JobResultNotReady(Exception):\n \"\"\"\n Raised by :meth:`~Scheduler.get_job_result` if the job result is\n not ready.\n \"\"\"\n\n def __init__(self, job_id: UUID):\n super().__init__(f\"No job by the id of {job_id} was found\")\n\n\nclass JobCancelled(Exception):\n \"\"\"\n Raised by :meth:`~Scheduler.get_job_result` if the job was\n cancelled.\n \"\"\"\n\n\nclass JobDeadlineMissed(Exception):\n \"\"\"\n Raised by :meth:`~Scheduler.get_job_result` if the job failed to\n start within the allotted time.\n \"\"\"\n\n\nclass ConflictingIdError(KeyError):\n \"\"\"\n Raised when trying to add a schedule to a store that already contains a schedule by\n that ID, and the conflict policy of ``exception`` is used.\n \"\"\"\n\n def __init__(self, schedule_id):\n super().__init__(\n f\"This data store already contains a schedule with the identifier \"\n f\"{schedule_id!r}\"\n )\n\n\nclass SerializationError(Exception):\n \"\"\"Raised when a serializer fails to serialize the given object.\"\"\"\n\n\nclass DeserializationError(Exception):\n \"\"\"Raised when a serializer fails to deserialize the given object.\"\"\"\n\n\nclass MaxIterationsReached(Exception):\n \"\"\"\n Raised when a trigger has reached its maximum number of allowed computation\n iterations when trying to calculate the next fire time.\n \"\"\"\n" }, { "path": "src/apscheduler/_marshalling.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom datetime import tzinfo\nfrom functools import partial\nfrom inspect import isclass, ismethod, ismethoddescriptor\nfrom typing import Any\nfrom zoneinfo import ZoneInfo\n\nfrom ._exceptions import DeserializationError, SerializationError\n\n\ndef marshal_object(obj) -> tuple[str, Any]:\n return (\n f\"{obj.__class__.__module__}:{obj.__class__.__qualname__}\",\n obj.__getstate__(),\n )\n\n\ndef unmarshal_object(ref: str, state: Any) -> Any:\n cls = callable_from_ref(ref)\n if not isinstance(cls, type):\n raise TypeError(f\"{ref} is not a class\")\n\n instance = cls.__new__(cls)\n instance.__setstate__(state)\n return instance\n\n\ndef marshal_timezone(value: tzinfo) -> str:\n if isinstance(value, ZoneInfo):\n return value.key\n elif hasattr(value, \"zone\"): # pytz timezones\n return value.zone\n\n raise SerializationError(\n f\"Unserializable time zone: {value!r}\\n\"\n f\"Only time zones from the zoneinfo or pytz modules can be serialized.\"\n )\n\n\ndef unmarshal_timezone(value: str) -> ZoneInfo:\n return ZoneInfo(value)\n\n\ndef callable_to_ref(func: Callable) -> str:\n \"\"\"\n Return a reference to the given callable.\n\n :raises SerializationError: if the given object is not callable, is a partial(),\n bound method, lambda or local function or does not have the ``__module__`` and\n ``__qualname__`` attributes\n\n \"\"\"\n if isinstance(func, partial):\n raise SerializationError(\"Cannot create a reference to a partial()\")\n\n if ismethod(func):\n if not isclass(func.__self__):\n raise SerializationError(\"Cannot create a reference to an instance method\")\n\n return f\"{func.__module__}:{func.__self__.__qualname__}.{func.__name__}\"\n\n if ismethoddescriptor(func):\n return f\"{func.__objclass__.__module__}:{func.__qualname__}\"\n\n if not hasattr(func, \"__module__\"):\n raise SerializationError(\"Callable has no __module__ attribute\")\n\n if not hasattr(func, \"__qualname__\"):\n raise SerializationError(\"Callable has no __qualname__ attribute\")\n\n if \"\" in func.__qualname__:\n raise SerializationError(\"Cannot create a reference to a lambda\")\n\n if \"\" in func.__qualname__:\n raise SerializationError(\"Cannot create a reference to a nested function\")\n\n return f\"{func.__module__}:{func.__qualname__}\"\n\n\ndef callable_from_ref(ref: str) -> Callable:\n \"\"\"\n Return the callable pointed to by ``ref``.\n\n :raises DeserializationError: if the reference could not be resolved or the looked\n up object is not callable\n\n \"\"\"\n if \":\" not in ref:\n raise ValueError(f\"Invalid reference: {ref}\")\n\n modulename, rest = ref.split(\":\", 1)\n try:\n obj = __import__(modulename, fromlist=[rest])\n except ImportError as exc:\n raise LookupError(\n f\"Error resolving reference {ref!r}: could not import module\"\n ) from exc\n\n try:\n for name in rest.split(\".\"):\n obj = getattr(obj, name)\n except Exception as exc:\n raise DeserializationError(\n f\"Error resolving reference {ref!r}: error looking up object\"\n ) from exc\n\n if not callable(obj):\n raise DeserializationError(\n f\"{ref!r} points to an object of type \"\n f\"{obj.__class__.__qualname__} which is not callable\"\n )\n\n return obj\n" }, { "path": "src/apscheduler/_retry.py", "content": "from __future__ import annotations\n\nfrom logging import Logger\n\nimport attrs\nfrom attr.validators import instance_of\nfrom tenacity import (\n AsyncRetrying,\n RetryCallState,\n retry_if_exception_type,\n stop_after_delay,\n wait_exponential,\n)\nfrom tenacity.stop import stop_base\nfrom tenacity.wait import wait_base\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass RetrySettings:\n \"\"\"\n Settings for retrying an operation with Tenacity.\n\n :param stop: defines when to stop trying\n :param wait: defines how long to wait between attempts\n \"\"\"\n\n stop: stop_base = attrs.field(\n validator=instance_of(stop_base),\n default=stop_after_delay(60),\n )\n wait: wait_base = attrs.field(\n validator=instance_of(wait_base),\n default=wait_exponential(min=0.5, max=20),\n )\n\n\n@attrs.define(kw_only=True, slots=False)\nclass RetryMixin:\n \"\"\"\n Mixin that provides support for retrying operations.\n\n :param retry_settings: Tenacity settings for retrying operations in case of a\n database connecitivty problem\n \"\"\"\n\n retry_settings: RetrySettings = attrs.field(default=RetrySettings())\n _logger: Logger = attrs.field(init=False)\n\n @property\n def _temporary_failure_exceptions(self) -> tuple[type[Exception], ...]:\n \"\"\"\n Tuple of exception classes which indicate that the operation should be retried.\n\n \"\"\"\n return ()\n\n def _retry(self) -> AsyncRetrying:\n def after_attempt(retry_state: RetryCallState) -> None:\n self._logger.warning(\n \"Temporary data store error (attempt %d): %s\",\n retry_state.attempt_number,\n retry_state.outcome.exception(),\n )\n\n return AsyncRetrying(\n stop=self.retry_settings.stop,\n wait=self.retry_settings.wait,\n retry=retry_if_exception_type(self._temporary_failure_exceptions),\n after=after_attempt,\n reraise=True,\n )\n" }, { "path": "src/apscheduler/_schedulers/__init__.py", "content": "" }, { "path": "src/apscheduler/_schedulers/async_.py", "content": "from __future__ import annotations\n\nimport os\nimport platform\nimport random\nimport sys\nfrom collections.abc import Callable, Iterable, Mapping, MutableMapping, Sequence\nfrom contextlib import AsyncExitStack\nfrom datetime import datetime, timedelta, timezone\nfrom functools import partial\nfrom inspect import isbuiltin, isclass, ismethod, ismodule\nfrom logging import Logger, getLogger\nfrom types import TracebackType\nfrom typing import Any, Literal, TypeAlias, TypeVar, cast, overload\nfrom uuid import UUID, uuid4\n\nimport anyio\nimport attrs\nfrom anyio import (\n TASK_STATUS_IGNORED,\n CancelScope,\n create_task_group,\n get_cancelled_exc_class,\n move_on_after,\n sleep,\n)\nfrom anyio.abc import TaskGroup, TaskStatus\nfrom attr.validators import instance_of, optional\n\nfrom .. import JobAdded, SerializationError, TaskLookupError\nfrom .._context import current_async_scheduler, current_job\nfrom .._converters import as_enum, as_timedelta\nfrom .._decorators import TaskParameters, get_task_params\nfrom .._enums import CoalescePolicy, ConflictPolicy, JobOutcome, RunState, SchedulerRole\nfrom .._events import (\n Event,\n JobReleased,\n ScheduleAdded,\n SchedulerStarted,\n SchedulerStopped,\n ScheduleUpdated,\n T_Event,\n)\nfrom .._exceptions import (\n CallableLookupError,\n DeserializationError,\n JobCancelled,\n JobDeadlineMissed,\n JobLookupError,\n ScheduleLookupError,\n)\nfrom .._marshalling import callable_from_ref, callable_to_ref\nfrom .._structures import (\n Job,\n JobResult,\n MetadataType,\n Schedule,\n ScheduleResult,\n Task,\n TaskDefaults,\n)\nfrom .._utils import UnsetValue, create_repr, merge_metadata, unset\nfrom .._validators import non_negative_number\nfrom ..abc import DataStore, EventBroker, JobExecutor, Subscription, Trigger\nfrom ..datastores.memory import MemoryDataStore\nfrom ..eventbrokers.local import LocalEventBroker\nfrom ..executors.async_ import AsyncJobExecutor\nfrom ..executors.subprocess import ProcessPoolJobExecutor\nfrom ..executors.thread import ThreadPoolJobExecutor\n\nif sys.version_info >= (3, 11):\n from typing import Self\nelse:\n from typing_extensions import Self\n\n_microsecond_delta = timedelta(microseconds=1)\n_zero_timedelta = timedelta()\n\nTaskType: TypeAlias = \"Task | str | Callable[..., Any]\"\nT = TypeVar(\"T\")\n\n\n@attrs.define(eq=False, repr=False)\nclass AsyncScheduler:\n \"\"\"\n An asynchronous (AnyIO based) scheduler implementation.\n\n Requires either :mod:`asyncio` or Trio_ to work.\n\n .. note:: If running on Trio, ensure that the data store and event broker are\n compatible with Trio.\n\n .. _AnyIO: https://pypi.org/project/anyio/\n .. _Trio: https://pypi.org/project/trio/\n\n :param data_store: the data store for tasks, schedules and jobs\n :param event_broker: the event broker to use for publishing an subscribing events\n :param identity: the unique identifier of the scheduler\n :param role: specifies what the scheduler should be doing when running (scheduling\n only, job running only, or both)\n :param max_concurrent_jobs: Maximum number of jobs the scheduler will run at once\n :param job_executors: a mutable mapping of executor names to executor instances\n :param task_defaults: default settings for newly configured tasks\n :param cleanup_interval: interval (as seconds or timedelta) between automatic\n calls to :meth:`cleanup` – ``None`` to disable automatic clean-up\n :param lease_duration: maximum amount of time (as seconds or timedelta) that\n the scheduler can keep a lock on a schedule or task\n :param logger: the logger instance used to log events from the scheduler, data store\n and event broker\n \"\"\"\n\n data_store: DataStore = attrs.field(\n validator=instance_of(DataStore), factory=MemoryDataStore\n )\n event_broker: EventBroker = attrs.field(\n validator=instance_of(EventBroker), factory=LocalEventBroker\n )\n identity: str = attrs.field(kw_only=True, validator=instance_of(str), default=\"\")\n role: SchedulerRole = attrs.field(\n kw_only=True, converter=as_enum(SchedulerRole), default=SchedulerRole.both\n )\n task_defaults: TaskDefaults = attrs.field(kw_only=True, factory=TaskDefaults)\n max_concurrent_jobs: int = attrs.field(\n kw_only=True, validator=non_negative_number, default=100\n )\n job_executors: MutableMapping[str, JobExecutor] = attrs.field(\n kw_only=True, validator=instance_of(MutableMapping), factory=dict\n )\n cleanup_interval: timedelta | None = attrs.field(\n kw_only=True,\n converter=as_timedelta,\n validator=optional(instance_of(timedelta)),\n default=timedelta(minutes=15),\n )\n lease_duration: timedelta = attrs.field(converter=as_timedelta, default=30)\n logger: Logger = attrs.field(kw_only=True, default=getLogger(__name__))\n\n _state: RunState = attrs.field(init=False, default=RunState.stopped)\n _services_task_group: TaskGroup | None = attrs.field(init=False, default=None)\n _exit_stack: AsyncExitStack = attrs.field(init=False)\n _services_initialized: bool = attrs.field(init=False, default=False)\n _scheduler_cancel_scope: CancelScope | None = attrs.field(init=False, default=None)\n _running_jobs: set[Job] = attrs.field(init=False, factory=set)\n _task_callables: dict[str, Callable] = attrs.field(init=False, factory=dict)\n\n def __attrs_post_init__(self) -> None:\n if not self.identity:\n self.identity = f\"{platform.node()}-{os.getpid()}-{id(self)}\"\n\n if not self.job_executors:\n self.job_executors = {\n \"async\": AsyncJobExecutor(),\n \"threadpool\": ThreadPoolJobExecutor(),\n \"processpool\": ProcessPoolJobExecutor(),\n }\n\n if self.task_defaults.job_executor is unset:\n self.task_defaults.job_executor = next(iter(self.job_executors))\n elif self.task_defaults.job_executor not in self.job_executors:\n valid_executors = \", \".join(self.job_executors)\n raise ValueError(\n f\"the default job executor must be one of the given job executors \"\n f\"({valid_executors})\"\n )\n\n if self.task_defaults.max_running_jobs is unset:\n self.task_defaults.max_running_jobs = 1\n\n if self.task_defaults.misfire_grace_time is unset:\n self.task_defaults.misfire_grace_time = None\n\n async def __aenter__(self) -> Self:\n async with AsyncExitStack() as exit_stack:\n await self._ensure_services_initialized(exit_stack)\n self._services_task_group = await exit_stack.enter_async_context(\n create_task_group()\n )\n exit_stack.callback(setattr, self, \"_services_task_group\", None)\n exit_stack.push_async_callback(self.stop)\n self._exit_stack = exit_stack.pop_all()\n\n return self\n\n async def __aexit__(\n self,\n exc_type: type[BaseException] | None,\n exc_val: BaseException | None,\n exc_tb: TracebackType | None,\n ) -> None:\n await self._exit_stack.__aexit__(exc_type, exc_val, exc_tb)\n\n def __repr__(self) -> str:\n return create_repr(self, \"identity\", \"role\", \"data_store\", \"event_broker\")\n\n async def _ensure_services_initialized(self, exit_stack: AsyncExitStack) -> None:\n \"\"\"\n Initialize the data store and event broker if this hasn't already been done.\n\n \"\"\"\n if not self._services_initialized:\n self._services_initialized = True\n exit_stack.callback(setattr, self, \"_services_initialized\", False)\n\n await self.event_broker.start(exit_stack, self.logger)\n await self.data_store.start(exit_stack, self.event_broker, self.logger)\n\n def _check_initialized(self) -> None:\n \"\"\"Raise RuntimeError if the services have not been initialized yet.\"\"\"\n if not self._services_initialized:\n raise RuntimeError(\n \"The scheduler has not been initialized yet. Use the scheduler as an \"\n \"async context manager (async with ...) in order to call methods other \"\n \"than run_until_stopped().\"\n )\n\n async def _cleanup_loop(self) -> None:\n delay = self.cleanup_interval.total_seconds()\n assert delay > 0\n while self._state in (RunState.starting, RunState.started):\n await self.cleanup()\n await sleep(delay)\n\n @property\n def state(self) -> RunState:\n \"\"\"The current running state of the scheduler.\"\"\"\n return self._state\n\n async def cleanup(self) -> None:\n \"\"\"Clean up expired job results and finished schedules.\"\"\"\n await self.data_store.cleanup()\n self.logger.info(\"Cleaned up expired job results and finished schedules\")\n\n @overload\n def subscribe(\n self,\n callback: Callable[[T_Event], Any],\n event_types: type[T_Event],\n *,\n one_shot: bool = ...,\n is_async: bool = ...,\n ) -> Subscription: ...\n\n @overload\n def subscribe(\n self,\n callback: Callable[[Event], Any],\n event_types: Iterable[type[Event]] | None = None,\n *,\n one_shot: bool = False,\n is_async: bool = True,\n ) -> Subscription: ...\n\n def subscribe(\n self,\n callback: Callable[[T_Event], Any],\n event_types: type[T_Event] | Iterable[type[T_Event]] | None = None,\n *,\n one_shot: bool = False,\n is_async: bool = True,\n ) -> Subscription:\n \"\"\"\n Subscribe to events.\n\n To unsubscribe, call the :meth:`~abc.Subscription.unsubscribe` method on the\n returned object.\n\n :param callback: callable to be called with the event object when an event is\n published\n :param event_types: an event class or an iterable event classes to subscribe to\n :param one_shot: if ``True``, automatically unsubscribe after the first matching\n event\n :param is_async: ``True`` if the (synchronous) callback should be called on the\n event loop thread, ``False`` if it should be called in a worker thread.\n If ``callback`` is a coroutine function, this flag is ignored.\n\n \"\"\"\n self._check_initialized()\n if isclass(event_types):\n event_types = {event_types}\n\n return self.event_broker.subscribe(\n callback, event_types, is_async=is_async, one_shot=one_shot\n )\n\n @overload\n async def get_next_event(self, event_types: type[T_Event]) -> T_Event: ...\n\n @overload\n async def get_next_event(self, event_types: Iterable[type[Event]]) -> Event: ...\n\n async def get_next_event(\n self, event_types: type[Event] | Iterable[type[Event]]\n ) -> Event:\n \"\"\"\n Wait until the next event matching one of the given types arrives.\n\n :param event_types: an event class or an iterable event classes to subscribe to\n\n \"\"\"\n received_event: Event | None = None\n\n def receive_event(ev: Event) -> None:\n nonlocal received_event\n received_event = ev\n event.set()\n\n event = anyio.Event()\n with self.subscribe(receive_event, event_types, one_shot=True):\n await event.wait()\n return received_event\n\n async def configure_task(\n self,\n func_or_task_id: TaskType,\n *,\n func: Callable[..., Any] | UnsetValue = unset,\n job_executor: str | UnsetValue = unset,\n misfire_grace_time: float | timedelta | None | UnsetValue = unset,\n max_running_jobs: int | None | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n ) -> Task:\n \"\"\"\n Add or update a :ref:`task ` definition.\n\n Any options not explicitly passed to this method will use their default values\n (from ``task_defaults``) when a new task is created:\n\n * ``job_executor``: the value of ``default_job_executor`` scheduler attribute\n * ``misfire_grace_time``: ``None``\n * ``max_running_jobs``: 1\n\n When updating a task, any options not explicitly passed will remain the same.\n\n If a callable is passed as the first argument, its fully qualified name will be\n used as the task ID.\n\n :param func_or_task_id: either a task, task ID or a callable\n :param func: a callable that will be associated with the task (can be omitted if\n the callable is already passed as ``func_or_task_id``)\n :param job_executor: name of the job executor to run the task with\n :param misfire_grace_time: maximum number of seconds the scheduled job's actual\n run time is allowed to be late, compared to the scheduled run time\n :param max_running_jobs: maximum number of instances of the task that are\n allowed to run concurrently\n :param metadata: key-value pairs for storing JSON compatible custom information\n :raises TypeError: if ``func_or_task_id`` is neither a task, task ID or a\n callable\n :return: the created or updated task definition\n\n \"\"\"\n func_ref: str | None = None\n task: Task | None = None\n if callable(func_or_task_id):\n task_params = get_task_params(func_or_task_id)\n if task_params.id is unset:\n task_params.id = callable_to_ref(func_or_task_id)\n\n if func is unset:\n func = func_or_task_id\n elif isinstance(func_or_task_id, Task):\n task_params = TaskParameters(\n id=func_or_task_id.id,\n job_executor=func_or_task_id.job_executor,\n max_running_jobs=func_or_task_id.max_running_jobs,\n misfire_grace_time=func_or_task_id.misfire_grace_time,\n metadata=func_or_task_id.metadata,\n )\n elif isinstance(func_or_task_id, str) and func_or_task_id:\n try:\n task = await self.data_store.get_task(func_or_task_id)\n task_params = TaskParameters(\n id=task.id,\n job_executor=task.job_executor,\n max_running_jobs=task.max_running_jobs,\n misfire_grace_time=task.misfire_grace_time,\n metadata=task.metadata,\n )\n except TaskLookupError:\n task_params = (\n get_task_params(func) if callable(func) else TaskParameters()\n )\n task_params.id = func_or_task_id\n else:\n raise TypeError(\n \"func_or_task_id must be either a task, its identifier or a callable\"\n )\n\n assert task_params.id\n\n # Apply any settings passed directly to this function as arguments\n if job_executor is not unset:\n task_params.job_executor = job_executor\n if max_running_jobs is not unset:\n task_params.max_running_jobs = max_running_jobs\n if misfire_grace_time is not unset:\n task_params.misfire_grace_time = misfire_grace_time\n\n # Fill in unset values with the defaults\n if task_params.job_executor is unset:\n task_params.job_executor = self.task_defaults.job_executor\n if task_params.max_running_jobs is unset:\n task_params.max_running_jobs = self.task_defaults.max_running_jobs\n if task_params.misfire_grace_time is unset:\n task_params.misfire_grace_time = self.task_defaults.misfire_grace_time\n\n # Merge the metadata from the defaults, task definition and explicitly passed\n # metadata\n task_params.metadata = merge_metadata(\n self.task_defaults.metadata, task_params.metadata, metadata\n )\n\n if callable(func):\n self._task_callables[task_params.id] = func\n try:\n func_ref = callable_to_ref(func)\n except SerializationError:\n pass\n\n modified = False\n try:\n task = task or await self.data_store.get_task(cast(str, task_params.id))\n except TaskLookupError:\n task = Task(\n id=task_params.id,\n func=func_ref,\n job_executor=task_params.job_executor,\n max_running_jobs=task_params.max_running_jobs,\n misfire_grace_time=task_params.misfire_grace_time,\n metadata=task_params.metadata,\n )\n modified = True\n else:\n changes: dict[str, Any] = {}\n if func is not unset and task.func != func_ref:\n changes[\"func\"] = func_ref\n\n if task_params.job_executor != task.job_executor:\n changes[\"job_executor\"] = task_params.job_executor\n\n if task_params.max_running_jobs != task.max_running_jobs:\n changes[\"max_running_jobs\"] = task_params.max_running_jobs\n\n if task_params.misfire_grace_time != task.misfire_grace_time:\n changes[\"misfire_grace_time\"] = task_params.misfire_grace_time\n\n if task_params.metadata != task.metadata:\n changes[\"metadata\"] = task_params.metadata\n\n if changes:\n task = attrs.evolve(task, **changes)\n modified = True\n\n if modified:\n await self.data_store.add_task(task)\n\n return task\n\n async def get_tasks(self) -> Sequence[Task]:\n \"\"\"\n Retrieve all currently defined tasks.\n\n :return: a sequence of tasks, sorted by ID\n\n \"\"\"\n self._check_initialized()\n return await self.data_store.get_tasks()\n\n async def add_schedule(\n self,\n func_or_task_id: TaskType,\n trigger: Trigger,\n *,\n id: str | None = None,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n paused: bool = False,\n coalesce: CoalescePolicy = CoalescePolicy.latest,\n job_executor: str | UnsetValue = unset,\n misfire_grace_time: float | timedelta | None | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n max_jitter: float | timedelta | None = None,\n job_result_expiration_time: float | timedelta = 0,\n conflict_policy: ConflictPolicy = ConflictPolicy.do_nothing,\n ) -> str:\n \"\"\"\n Schedule a task to be run one or more times in the future.\n\n :param func_or_task_id: either a callable or an ID of an existing task\n definition\n :param trigger: determines the times when the task should be run\n :param id: an explicit identifier for the schedule (if omitted, a random, UUID\n based ID will be assigned)\n :param args: positional arguments to be passed to the task function\n :param kwargs: keyword arguments to be passed to the task function\n :param paused: whether the schedule is paused\n :param job_executor: name of the job executor to run the scheduled jobs with\n (overrides the executor specified in the task settings)\n :param coalesce: determines what to do when processing the schedule if multiple\n fire times have become due for this schedule since the last processing\n :param misfire_grace_time: maximum number of seconds the scheduled job's actual\n run time is allowed to be late, compared to the scheduled run time\n :param metadata: key-value pairs for storing JSON compatible custom information\n :param max_jitter: maximum time (in seconds, or as a timedelta) to randomly add\n to the scheduled time for each job created from this schedule\n :param job_result_expiration_time: minimum time (in seconds, or as a timedelta)\n to keep the job results in storage from the jobs created by this schedule\n :param conflict_policy: determines what to do if a schedule with the same ID\n already exists in the data store\n :return: the ID of the newly added schedule\n\n \"\"\"\n self._check_initialized()\n schedule_id = id or str(uuid4())\n args = tuple(args or ())\n kwargs = dict(kwargs or {})\n\n # Unpack the function and positional + keyword arguments from a partial()\n if isinstance(func_or_task_id, partial):\n args = func_or_task_id.args + args\n kwargs.update(func_or_task_id.keywords)\n func_or_task_id = func_or_task_id.func\n\n # For instance methods, use the unbound function as the function, and the\n # \"self\" argument as the first positional argument\n if ismethod(func_or_task_id):\n args = (func_or_task_id.__self__, *args)\n func_or_task_id = func_or_task_id.__func__\n elif (\n isbuiltin(func_or_task_id)\n and func_or_task_id.__self__ is not None\n and not ismodule(func_or_task_id.__self__)\n ):\n args = (func_or_task_id.__self__, *args)\n method_class = type(func_or_task_id.__self__)\n func_or_task_id = getattr(method_class, func_or_task_id.__name__)\n\n task = await self.configure_task(func_or_task_id)\n schedule = Schedule(\n id=schedule_id,\n task_id=task.id,\n trigger=trigger,\n args=args,\n kwargs=kwargs,\n paused=paused,\n coalesce=coalesce,\n misfire_grace_time=task.misfire_grace_time\n if misfire_grace_time is unset\n else misfire_grace_time,\n metadata=task.metadata.copy()\n if metadata is unset\n else merge_metadata(task.metadata, metadata),\n max_jitter=max_jitter,\n job_executor=task.job_executor if job_executor is unset else job_executor,\n job_result_expiration_time=job_result_expiration_time,\n )\n schedule.next_fire_time = trigger.next()\n await self.data_store.add_schedule(schedule, conflict_policy)\n self.logger.info(\n \"Added new schedule (task=%r, trigger=%r); next run time at %s\",\n task.id,\n trigger,\n schedule.next_fire_time,\n )\n return schedule.id\n\n async def get_schedule(self, id: str) -> Schedule:\n \"\"\"\n Retrieve a schedule from the data store.\n\n :param id: the unique identifier of the schedule\n :raises ScheduleLookupError: if the schedule could not be found\n\n \"\"\"\n self._check_initialized()\n schedules = await self.data_store.get_schedules({id})\n if schedules:\n return schedules[0]\n else:\n raise ScheduleLookupError(id)\n\n async def get_schedules(self) -> list[Schedule]:\n \"\"\"\n Retrieve all schedules from the data store.\n\n :return: a list of schedules, in an unspecified order\n\n \"\"\"\n self._check_initialized()\n return await self.data_store.get_schedules()\n\n async def remove_schedule(self, id: str) -> None:\n \"\"\"\n Remove the given schedule from the data store.\n\n :param id: the unique identifier of the schedule\n\n \"\"\"\n self._check_initialized()\n await self.data_store.remove_schedules({id})\n\n async def pause_schedule(self, id: str) -> None:\n \"\"\"Pause the specified schedule.\"\"\"\n self._check_initialized()\n await self.data_store.add_schedule(\n schedule=attrs.evolve(await self.get_schedule(id), paused=True),\n conflict_policy=ConflictPolicy.replace,\n )\n\n async def unpause_schedule(\n self,\n id: str,\n *,\n resume_from: datetime | Literal[\"now\"] | None = None,\n ) -> None:\n \"\"\"\n Unpause the specified schedule.\n\n\n :param resume_from: the time to resume the schedules from, or ``'now'`` as a\n shorthand for ``datetime.now(tz=UTC)`` or ``None`` to resume from where the\n schedule left off which may cause it to misfire\n\n \"\"\"\n self._check_initialized()\n schedule = await self.get_schedule(id)\n\n if resume_from == \"now\":\n resume_from = datetime.now(tz=timezone.utc)\n\n if resume_from is None:\n next_fire_time = schedule.next_fire_time\n elif (\n schedule.next_fire_time is not None\n and schedule.next_fire_time >= resume_from\n ):\n next_fire_time = schedule.next_fire_time\n else:\n # Advance `next_fire_time` until its at or past `resume_from`, or until it's\n # exhausted\n while next_fire_time := schedule.trigger.next():\n if next_fire_time is None or next_fire_time >= resume_from:\n break\n\n await self.data_store.add_schedule(\n schedule=attrs.evolve(\n schedule,\n paused=False,\n next_fire_time=next_fire_time,\n ),\n conflict_policy=ConflictPolicy.replace,\n )\n\n async def add_job(\n self,\n func_or_task_id: TaskType,\n *,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n job_executor: str | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n result_expiration_time: timedelta | float = 0,\n ) -> UUID:\n \"\"\"\n Add a job to the data store.\n\n :param func_or_task_id:\n Either the ID of a pre-existing task, or a function/method. If a function is\n given, a task will be created with the fully qualified name of the function\n as the task ID (unless that task already exists of course).\n :param args: positional arguments to call the target callable with\n :param kwargs: keyword arguments to call the target callable with\n :param job_executor: name of the job executor to run the task with\n (overrides the executor in the task definition, if any)\n :param metadata: key-value pairs for storing JSON compatible custom information\n :param result_expiration_time: the minimum time (as seconds, or timedelta) to\n keep the result of the job available for fetching (the result won't be\n saved at all if that time is 0)\n :return: the ID of the newly created job\n\n \"\"\"\n self._check_initialized()\n args = tuple(args or ())\n kwargs = dict(kwargs or {})\n\n # Unpack the function and positional + keyword arguments from a partial()\n if isinstance(func_or_task_id, partial):\n args = func_or_task_id.args + args\n kwargs.update(func_or_task_id.keywords)\n func_or_task_id = func_or_task_id.func\n\n # For instance methods, use the unbound function as the function, and the\n # \"self\" argument as the first positional argument\n if ismethod(func_or_task_id):\n args = (func_or_task_id.__self__, *args)\n func_or_task_id = func_or_task_id.__func__\n elif (\n isbuiltin(func_or_task_id)\n and func_or_task_id.__self__ is not None\n and not ismodule(func_or_task_id.__self__)\n ):\n args = (func_or_task_id.__self__, *args)\n method_class = type(func_or_task_id.__self__)\n func_or_task_id = getattr(method_class, func_or_task_id.__name__)\n\n task = await self.configure_task(func_or_task_id)\n job = Job(\n task_id=task.id,\n args=args or (),\n kwargs=kwargs or {},\n executor=task.job_executor if job_executor is unset else job_executor,\n result_expiration_time=result_expiration_time,\n metadata=merge_metadata(task.metadata, metadata),\n )\n await self.data_store.add_job(job)\n return job.id\n\n async def get_jobs(self) -> Sequence[Job]:\n \"\"\"Retrieve all jobs from the data store.\"\"\"\n self._check_initialized()\n return await self.data_store.get_jobs()\n\n async def get_job_result(\n self, job_id: UUID, *, wait: bool = True\n ) -> JobResult | None:\n \"\"\"\n Retrieve the result of a job.\n\n :param job_id: the ID of the job\n :param wait: if ``True``, wait until the job has ended (one way or another),\n ``False`` to raise an exception if the result is not yet available\n :returns: the job result, or ``None`` if the job finished but didn't record a\n result (``result_expiration_time`` was 0 or a similarly short time interval\n that did not allow for the result to be fetched before it was deleted)\n :raises JobLookupError: if neither the job or its result exist in the data\n store, or the job exists but the result is not ready yet and ``wait=False``\n is set\n\n \"\"\"\n self._check_initialized()\n wait_event = anyio.Event()\n\n def listener(event: JobReleased) -> None:\n if event.job_id == job_id:\n wait_event.set()\n\n with self.event_broker.subscribe(listener, {JobReleased}):\n job_exists = bool(await self.data_store.get_jobs([job_id]))\n if result := await self.data_store.get_job_result(job_id):\n return result\n\n if job_exists and wait:\n await wait_event.wait()\n else:\n raise JobLookupError(job_id)\n\n return await self.data_store.get_job_result(job_id)\n\n async def run_job(\n self,\n func_or_task_id: str | Callable[..., Any],\n *,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n job_executor: str | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n ) -> Any:\n \"\"\"\n Convenience method to add a job and then return its result.\n\n If the job raised an exception, that exception will be reraised here.\n\n :param func_or_task_id: either a callable or an ID of an existing task\n definition\n :param args: positional arguments to be passed to the task function\n :param kwargs: keyword arguments to be passed to the task function\n :param job_executor: name of the job executor to run the task with\n (overrides the executor in the task definition, if any)\n :param metadata: key-value pairs for storing JSON compatible custom information\n :returns: the return value of the task function\n\n \"\"\"\n self._check_initialized()\n job_complete_event = anyio.Event()\n\n def listener(event: JobReleased) -> None:\n if event.job_id == job_id:\n job_complete_event.set()\n\n job_id: UUID | None = None\n with self.event_broker.subscribe(listener, {JobReleased}):\n job_id = await self.add_job(\n func_or_task_id,\n args=args,\n kwargs=kwargs,\n job_executor=job_executor,\n metadata=metadata,\n result_expiration_time=timedelta(minutes=15),\n )\n await job_complete_event.wait()\n\n result = await self.get_job_result(job_id)\n if result is None:\n raise RuntimeError(\n \"Job completed but job result not found - report this as a bug!\"\n )\n\n if result.exception:\n assert result.outcome is JobOutcome.error\n raise result.exception\n\n if result.outcome is JobOutcome.success:\n return result.return_value\n elif result.outcome is JobOutcome.missed_start_deadline:\n raise JobDeadlineMissed\n elif result.outcome is JobOutcome.cancelled:\n raise JobCancelled\n else:\n raise RuntimeError(f\"Unknown job outcome: {result.outcome}\")\n\n async def stop(self) -> None:\n \"\"\"\n Signal the scheduler that it should stop processing schedules.\n\n This method does not wait for the scheduler to actually stop.\n For that, see :meth:`wait_until_stopped`.\n\n \"\"\"\n if self._state is RunState.started and self._scheduler_cancel_scope:\n self._state = RunState.stopping\n self._scheduler_cancel_scope.cancel()\n\n async def wait_until_stopped(self) -> None:\n \"\"\"\n Wait until the scheduler is in the :attr:`~RunState.stopped` or\n :attr:`~RunState.stopping` state.\n\n If the scheduler is already stopped or in the process of stopping, this method\n returns immediately. Otherwise, it waits until the scheduler posts the\n :class:`SchedulerStopped` event.\n\n \"\"\"\n if self._state not in (RunState.stopped, RunState.stopping):\n await self.get_next_event(SchedulerStopped)\n\n async def start_in_background(self) -> None:\n self._check_initialized()\n await self._services_task_group.start(\n self.run_until_stopped,\n name=f\"Scheduler {self.identity!r} main task\",\n )\n\n async def run_until_stopped(\n self, *, task_status: TaskStatus[None] = TASK_STATUS_IGNORED\n ) -> None:\n \"\"\"Run the scheduler until explicitly stopped.\"\"\"\n if self._state is not RunState.stopped:\n raise RuntimeError(\n f'Cannot start the scheduler when it is in the \"{self._state}\" state'\n )\n\n self._state = RunState.starting\n async with AsyncExitStack() as exit_stack:\n await self._ensure_services_initialized(exit_stack)\n\n # Set this scheduler as the current scheduler\n token = current_async_scheduler.set(self)\n exit_stack.callback(current_async_scheduler.reset, token)\n\n exception: BaseException | None = None\n try:\n async with create_task_group() as task_group:\n self._scheduler_cancel_scope = task_group.cancel_scope\n exit_stack.callback(setattr, self, \"_scheduler_cancel_scope\", None)\n\n # Start periodic cleanups\n if self.cleanup_interval:\n task_group.start_soon(\n self._cleanup_loop,\n name=f\"Scheduler {self.identity!r} clean-up loop\",\n )\n self.logger.debug(\n \"Started internal cleanup loop with interval: %s\",\n self.cleanup_interval,\n )\n\n # Start processing due schedules, if configured to do so\n if self.role in (SchedulerRole.scheduler, SchedulerRole.both):\n await task_group.start(\n self._process_schedules,\n name=f\"Scheduler {self.identity!r} schedule processing loop\",\n )\n\n # Start processing due jobs, if configured to do so\n if self.role in (SchedulerRole.worker, SchedulerRole.both):\n await task_group.start(\n self._process_jobs,\n name=f\"Scheduler {self.identity!r} job processing loop\",\n )\n\n # Signal that the scheduler has started\n self._state = RunState.started\n self.logger.info(\"Scheduler started\")\n task_status.started()\n await self.event_broker.publish_local(SchedulerStarted())\n except BaseException as exc:\n exception = exc\n raise\n finally:\n self._state = RunState.stopped\n\n if not exception or isinstance(exception, get_cancelled_exc_class()):\n self.logger.info(\"Scheduler stopped\")\n elif isinstance(exception, Exception):\n self.logger.exception(\"Scheduler crashed\")\n elif exception:\n self.logger.info(\n \"Scheduler stopped due to %s\", exception.__class__.__name__\n )\n\n with move_on_after(3, shield=True):\n await self.event_broker.publish_local(\n SchedulerStopped(exception=exception)\n )\n\n async def _process_schedules(self, *, task_status: TaskStatus[None]) -> None:\n wakeup_event = anyio.Event()\n wakeup_deadline: datetime | None = None\n\n async def schedule_added_or_modified(event: Event) -> None:\n event_ = cast(\"ScheduleAdded | ScheduleUpdated\", event)\n if not wakeup_deadline or (\n event_.next_fire_time and event_.next_fire_time < wakeup_deadline\n ):\n self.logger.debug(\n \"Detected a %s event – waking up the scheduler to process \"\n \"schedules\",\n type(event).__name__,\n )\n wakeup_event.set()\n\n async def extend_schedule_leases(schedules: Sequence[Schedule]) -> None:\n schedule_ids = {schedule.id for schedule in schedules}\n while True:\n await sleep(self.lease_duration.total_seconds() / 2)\n await self.data_store.extend_acquired_schedule_leases(\n self.identity, schedule_ids, self.lease_duration\n )\n\n subscription = self.event_broker.subscribe(\n schedule_added_or_modified, {ScheduleAdded, ScheduleUpdated}\n )\n with subscription:\n # Signal that we are ready, and wait for the scheduler start event\n task_status.started()\n await self.get_next_event(SchedulerStarted)\n\n while self._state is RunState.started:\n schedules = await self.data_store.acquire_schedules(\n self.identity, self.lease_duration, 100\n )\n async with AsyncExitStack() as exit_stack:\n tg = await exit_stack.enter_async_context(create_task_group())\n tg.start_soon(\n extend_schedule_leases,\n schedules,\n name=(\n f\"Scheduler {self.identity!r} schedule lease extension loop\"\n ),\n )\n exit_stack.callback(tg.cancel_scope.cancel)\n\n now = datetime.now(timezone.utc)\n results: list[ScheduleResult] = []\n for schedule in schedules:\n # Calculate a next fire time for the schedule, if possible\n fire_times = [schedule.next_fire_time]\n calculate_next = schedule.trigger.next\n while True:\n try:\n fire_time = calculate_next()\n except Exception:\n self.logger.exception(\n \"Error computing next fire time for schedule %r of \"\n \"task %r – removing schedule\",\n schedule.id,\n schedule.task_id,\n )\n break\n\n # Stop if the calculated fire time is in the future\n if fire_time is None or fire_time > now:\n next_fire_time = fire_time\n break\n\n # Only keep all the fire times if coalesce policy = \"all\"\n if schedule.coalesce is CoalescePolicy.all:\n fire_times.append(fire_time)\n elif schedule.coalesce is CoalescePolicy.latest:\n fire_times[0] = fire_time\n\n # Add one or more jobs to the job queue\n max_jitter = (\n schedule.max_jitter.total_seconds()\n if schedule.max_jitter\n else 0\n )\n for i, fire_time in enumerate(fire_times):\n # Calculate a jitter if max_jitter > 0\n jitter = _zero_timedelta\n if max_jitter:\n if i + 1 < len(fire_times):\n following_fire_time = fire_times[i + 1]\n else:\n following_fire_time = next_fire_time\n\n if following_fire_time is not None:\n # Jitter must never be so high that it would cause a\n # fire time to equal or exceed the next fire time\n max_jitter = min(\n [\n max_jitter,\n (\n following_fire_time\n - fire_time\n - _microsecond_delta\n ).total_seconds(),\n ]\n )\n\n jitter = timedelta(\n seconds=random.uniform(0, max_jitter)\n )\n fire_time += jitter\n\n if schedule.misfire_grace_time is None:\n start_deadline: datetime | None = None\n else:\n start_deadline = fire_time + schedule.misfire_grace_time\n\n job = Job(\n task_id=schedule.task_id,\n args=schedule.args,\n kwargs=schedule.kwargs,\n schedule_id=schedule.id,\n scheduled_fire_time=fire_time,\n jitter=jitter,\n start_deadline=start_deadline,\n executor=schedule.job_executor,\n result_expiration_time=schedule.job_result_expiration_time,\n metadata=schedule.metadata.copy(),\n )\n await self.data_store.add_job(job)\n\n results.append(\n ScheduleResult(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n trigger=schedule.trigger,\n last_fire_time=fire_times[-1],\n next_fire_time=next_fire_time,\n )\n )\n\n # Update the schedules (and release the scheduler's claim on them)\n await self.data_store.release_schedules(self.identity, results)\n\n # If we received fewer schedules than the maximum amount, sleep\n # until the next schedule is due or the scheduler is explicitly\n # woken up\n wait_time = None\n if len(schedules) < 100:\n wakeup_deadline = await self.data_store.get_next_schedule_run_time()\n if wakeup_deadline:\n wait_time = (\n wakeup_deadline - datetime.now(timezone.utc)\n ).total_seconds()\n self.logger.debug(\n \"Sleeping %.3f seconds until the next fire time (%s)\",\n wait_time,\n wakeup_deadline,\n )\n else:\n self.logger.debug(\"Waiting for any due schedules to appear\")\n\n with move_on_after(wait_time):\n await wakeup_event.wait()\n wakeup_event = anyio.Event()\n else:\n self.logger.debug(\"Processing more schedules on the next iteration\")\n\n def _get_task_callable(self, task: Task) -> Callable:\n try:\n return self._task_callables[task.id]\n except KeyError as exc:\n if task.func:\n try:\n func = self._task_callables[task.id] = callable_from_ref(task.func)\n except DeserializationError as exc:\n raise CallableLookupError(\n f\"Error looking up the callable ({task.func!r}) for task \"\n f\"{task.id!r}\"\n ) from exc\n\n return func\n\n raise CallableLookupError(\n f\"Task {task.id} requires a locally defined callable to be run, but no \"\n f\"such callable has been defined. Call \"\n f\"scheduler.configure_task({task.id!r}, func=...) to define the local \"\n f\"callable.\"\n ) from exc\n\n async def _process_jobs(self, *, task_status: TaskStatus[None]) -> None:\n wakeup_event = anyio.Event()\n\n async def check_queue_capacity(event: Event) -> None:\n if len(self._running_jobs) < self.max_concurrent_jobs:\n wakeup_event.set()\n\n async def extend_job_leases() -> None:\n while self._state in (RunState.starting, RunState.started):\n await sleep(self.lease_duration.total_seconds() / 2)\n if job_ids := {job.id for job in self._running_jobs}:\n await self.data_store.extend_acquired_job_leases(\n self.identity, job_ids, self.lease_duration\n )\n\n # If there are any jobs marked as being acquired by this scheduler, release them\n # with the \"abandoned\" outcome right away\n await self.data_store.reap_abandoned_jobs(self.identity)\n\n async with AsyncExitStack() as exit_stack:\n # Start the job executors\n for job_executor in self.job_executors.values():\n await job_executor.start(exit_stack)\n\n task_group = await exit_stack.enter_async_context(create_task_group())\n task_group.start_soon(\n extend_job_leases,\n name=f\"Scheduler {self.identity!r} job lease extension loop\",\n )\n\n # Fetch new jobs every time\n exit_stack.enter_context(\n self.event_broker.subscribe(\n check_queue_capacity, {JobAdded, JobReleased}\n )\n )\n\n # Signal that we are ready, and wait for the scheduler start event\n task_status.started()\n await self.get_next_event(SchedulerStarted)\n\n while self._state is RunState.started:\n limit = self.max_concurrent_jobs - len(self._running_jobs)\n if limit > 0:\n jobs = await self.data_store.acquire_jobs(\n self.identity, self.lease_duration, limit\n )\n for job in jobs:\n task = await self.data_store.get_task(job.task_id)\n func = self._get_task_callable(task)\n self._running_jobs.add(job)\n task_group.start_soon(\n self._run_job,\n job,\n func,\n job.executor,\n name=(\n f\"Scheduler {self.identity!r} job {job.id} \"\n f\"({job.executor!r})\"\n ),\n )\n\n await wakeup_event.wait()\n wakeup_event = anyio.Event()\n\n async def _run_job(self, job: Job, func: Callable[..., Any], executor: str) -> None:\n try:\n # Check if the job started before the deadline\n start_time = datetime.now(timezone.utc)\n if job.start_deadline is not None and start_time > job.start_deadline:\n result = JobResult.from_job(\n job, JobOutcome.missed_start_deadline, finished_at=start_time\n )\n await self.data_store.release_job(self.identity, job, result)\n return\n\n try:\n job_executor = self.job_executors[executor]\n except KeyError:\n return\n\n token = current_job.set(job)\n try:\n retval = await job_executor.run_job(func, job)\n except get_cancelled_exc_class():\n self.logger.info(\"Job %s was cancelled\", job.id)\n with CancelScope(shield=True):\n result = JobResult.from_job(\n job, JobOutcome.cancelled, started_at=start_time\n )\n await self.data_store.release_job(self.identity, job, result)\n except BaseException as exc:\n if isinstance(exc, Exception):\n self.logger.exception(\"Job %s raised an exception\", job.id)\n else:\n self.logger.error(\n \"Job %s was aborted due to %s\", job.id, exc.__class__.__name__\n )\n\n result = JobResult.from_job(\n job,\n JobOutcome.error,\n started_at=start_time,\n exception=exc,\n )\n await self.data_store.release_job(self.identity, job, result)\n if not isinstance(exc, Exception):\n raise\n else:\n self.logger.info(\"Job %s completed successfully\", job.id)\n result = JobResult.from_job(\n job,\n JobOutcome.success,\n started_at=start_time,\n return_value=retval,\n )\n await self.data_store.release_job(self.identity, job, result)\n finally:\n current_job.reset(token)\n finally:\n self._running_jobs.remove(job)\n" }, { "path": "src/apscheduler/_schedulers/sync.py", "content": "from __future__ import annotations\n\nimport atexit\nimport sys\nimport threading\nfrom collections.abc import Callable, Iterable, Mapping, MutableMapping, Sequence\nfrom contextlib import ExitStack\nfrom datetime import datetime, timedelta\nfrom functools import partial\nfrom logging import Logger\nfrom types import TracebackType\nfrom typing import Any, Literal, overload\nfrom uuid import UUID\n\nimport attrs\nfrom anyio.from_thread import BlockingPortal, start_blocking_portal\n\nfrom .. import current_scheduler\nfrom .._enums import CoalescePolicy, ConflictPolicy, RunState, SchedulerRole\nfrom .._events import Event, T_Event\nfrom .._structures import Job, JobResult, MetadataType, Schedule, Task, TaskDefaults\nfrom .._utils import UnsetValue, create_repr, unset\nfrom ..abc import DataStore, EventBroker, JobExecutor, Subscription, Trigger\nfrom .async_ import AsyncScheduler, TaskType\n\nif sys.version_info >= (3, 11):\n from typing import Self\nelse:\n from typing_extensions import Self\n\n\n@attrs.define(init=False, repr=False)\nclass Scheduler:\n \"\"\"\n A synchronous wrapper for :class:`AsyncScheduler`.\n\n When started, this wrapper launches an asynchronous event loop in a separate thread\n and runs the asynchronous scheduler there. This thread is shut down along with the\n scheduler.\n\n See the documentation of the :class:`AsyncScheduler` class for the documentation of\n the configuration options.\n \"\"\"\n\n _async_scheduler: AsyncScheduler\n _exit_stack: ExitStack = attrs.field(init=False, factory=ExitStack)\n _portal: BlockingPortal | None = attrs.field(init=False, default=None)\n _lock: threading.Lock = attrs.field(init=False, factory=threading.Lock)\n\n def __init__(\n self,\n data_store: DataStore | None = None,\n event_broker: EventBroker | None = None,\n *,\n identity: str = \"\",\n role: SchedulerRole = SchedulerRole.both,\n max_concurrent_jobs: int = 100,\n cleanup_interval: float | timedelta | None = None,\n lease_duration: timedelta = timedelta(seconds=30),\n job_executors: MutableMapping[str, JobExecutor] | None = None,\n task_defaults: TaskDefaults | None = None,\n logger: Logger | None = None,\n ):\n kwargs: dict[str, Any] = {}\n if data_store is not None:\n kwargs[\"data_store\"] = data_store\n\n if event_broker is not None:\n kwargs[\"event_broker\"] = event_broker\n\n if logger is not None:\n kwargs[\"logger\"] = logger\n\n if task_defaults is None:\n task_defaults = TaskDefaults()\n\n if task_defaults.job_executor is unset:\n task_defaults.job_executor = \"threadpool\"\n\n async_scheduler = AsyncScheduler(\n identity=identity,\n role=role,\n task_defaults=task_defaults,\n max_concurrent_jobs=max_concurrent_jobs,\n job_executors=job_executors or {},\n cleanup_interval=cleanup_interval,\n lease_duration=lease_duration,\n **kwargs,\n )\n self.__attrs_init__(async_scheduler=async_scheduler)\n\n @property\n def logger(self) -> Logger:\n return self._async_scheduler.logger\n\n @property\n def data_store(self) -> DataStore:\n return self._async_scheduler.data_store\n\n @property\n def event_broker(self) -> EventBroker:\n return self._async_scheduler.event_broker\n\n @property\n def identity(self) -> str:\n return self._async_scheduler.identity\n\n @property\n def role(self) -> SchedulerRole:\n return self._async_scheduler.role\n\n @property\n def max_concurrent_jobs(self) -> int:\n return self._async_scheduler.max_concurrent_jobs\n\n @property\n def cleanup_interval(self) -> timedelta | None:\n return self._async_scheduler.cleanup_interval\n\n @property\n def lease_duration(self) -> timedelta:\n return self._async_scheduler.lease_duration\n\n @property\n def job_executors(self) -> MutableMapping[str, JobExecutor]:\n return self._async_scheduler.job_executors\n\n @property\n def task_defaults(self) -> TaskDefaults:\n return self._async_scheduler.task_defaults\n\n @property\n def state(self) -> RunState:\n \"\"\"The current running state of the scheduler.\"\"\"\n return self._async_scheduler.state\n\n def __enter__(self: Self) -> Self:\n self._ensure_services_ready(self._exit_stack)\n return self\n\n def __exit__(\n self,\n exc_type: type[BaseException] | None,\n exc_val: BaseException | None,\n exc_tb: TracebackType | None,\n ) -> None:\n self._exit_stack.__exit__(exc_type, exc_val, exc_tb)\n\n def _ensure_services_ready(\n self, exit_stack: ExitStack | None = None\n ) -> BlockingPortal:\n \"\"\"Ensure that the underlying asynchronous scheduler has been initialized.\"\"\"\n with self._lock:\n if self._portal is None:\n if exit_stack is None:\n self._exit_stack = exit_stack = ExitStack()\n atexit.register(self._exit_stack.close)\n else:\n exit_stack = self._exit_stack\n\n # Set this scheduler as the current synchronous scheduler\n token = current_scheduler.set(self)\n exit_stack.callback(current_scheduler.reset, token)\n\n self._portal = exit_stack.enter_context(start_blocking_portal())\n exit_stack.callback(setattr, self, \"_portal\", None)\n exit_stack.enter_context(\n self._portal.wrap_async_context_manager(self._async_scheduler)\n )\n\n return self._portal\n\n def __repr__(self) -> str:\n return create_repr(self, \"identity\", \"role\", \"data_store\", \"event_broker\")\n\n def cleanup(self) -> None:\n portal = self._ensure_services_ready()\n return portal.call(self._async_scheduler.cleanup)\n\n @overload\n def subscribe(\n self,\n callback: Callable[[T_Event], Any],\n event_types: type[T_Event],\n *,\n one_shot: bool = ...,\n ) -> Subscription: ...\n\n @overload\n def subscribe(\n self,\n callback: Callable[[Event], Any],\n event_types: Iterable[type[Event]] | None = None,\n *,\n one_shot: bool = False,\n ) -> Subscription: ...\n\n def subscribe(\n self,\n callback: Callable[[T_Event], Any],\n event_types: type[T_Event] | Iterable[type[T_Event]] | None = None,\n *,\n one_shot: bool = False,\n ) -> Subscription:\n \"\"\"\n Subscribe to events.\n\n To unsubscribe, call the :meth:`~abc.Subscription.unsubscribe` method on the\n returned object.\n\n :param callback: callable to be called with the event object when an event is\n published\n :param event_types: an iterable of concrete Event classes to subscribe to\n :param one_shot: if ``True``, automatically unsubscribe after the first matching\n event\n\n \"\"\"\n portal = self._ensure_services_ready()\n return portal.call(\n partial(\n self._async_scheduler.subscribe,\n callback,\n event_types,\n is_async=False,\n one_shot=one_shot,\n )\n )\n\n @overload\n def get_next_event(self, event_types: type[T_Event]) -> T_Event: ...\n\n @overload\n def get_next_event(self, event_types: Iterable[type[Event]]) -> Event: ...\n\n def get_next_event(self, event_types: type[Event] | Iterable[type[Event]]) -> Event:\n portal = self._ensure_services_ready()\n return portal.call(partial(self._async_scheduler.get_next_event, event_types))\n\n def configure_task(\n self,\n func_or_task_id: TaskType,\n *,\n func: Callable[..., Any] | UnsetValue = unset,\n job_executor: str | UnsetValue = unset,\n misfire_grace_time: float | timedelta | None | UnsetValue = unset,\n max_running_jobs: int | None | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n ) -> Task:\n portal = self._ensure_services_ready()\n return portal.call(\n partial(\n self._async_scheduler.configure_task,\n func_or_task_id,\n func=func,\n job_executor=job_executor,\n misfire_grace_time=misfire_grace_time,\n max_running_jobs=max_running_jobs,\n metadata=metadata,\n )\n )\n\n def get_tasks(self) -> Sequence[Task]:\n portal = self._ensure_services_ready()\n return portal.call(self._async_scheduler.get_tasks)\n\n def add_schedule(\n self,\n func_or_task_id: TaskType,\n trigger: Trigger,\n *,\n id: str | None = None,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n paused: bool = False,\n coalesce: CoalescePolicy = CoalescePolicy.latest,\n job_executor: str | UnsetValue = unset,\n misfire_grace_time: float | timedelta | None | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n max_jitter: float | timedelta | None = None,\n job_result_expiration_time: float | timedelta = 0,\n conflict_policy: ConflictPolicy = ConflictPolicy.do_nothing,\n ) -> str:\n portal = self._ensure_services_ready()\n return portal.call(\n partial(\n self._async_scheduler.add_schedule,\n func_or_task_id,\n trigger,\n id=id,\n args=args,\n kwargs=kwargs,\n paused=paused,\n job_executor=job_executor,\n coalesce=coalesce,\n misfire_grace_time=misfire_grace_time,\n max_jitter=max_jitter,\n job_result_expiration_time=job_result_expiration_time,\n metadata=metadata,\n conflict_policy=conflict_policy,\n )\n )\n\n def get_schedule(self, id: str) -> Schedule:\n portal = self._ensure_services_ready()\n return portal.call(self._async_scheduler.get_schedule, id)\n\n def get_schedules(self) -> list[Schedule]:\n portal = self._ensure_services_ready()\n return portal.call(self._async_scheduler.get_schedules)\n\n def remove_schedule(self, id: str) -> None:\n portal = self._ensure_services_ready()\n portal.call(self._async_scheduler.remove_schedule, id)\n\n def pause_schedule(self, id: str) -> None:\n portal = self._ensure_services_ready()\n portal.call(self._async_scheduler.pause_schedule, id)\n\n def unpause_schedule(\n self,\n id: str,\n *,\n resume_from: datetime | Literal[\"now\"] | None = None,\n ) -> None:\n portal = self._ensure_services_ready()\n portal.call(\n partial(\n self._async_scheduler.unpause_schedule,\n id,\n resume_from=resume_from,\n )\n )\n\n def add_job(\n self,\n func_or_task_id: TaskType,\n *,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n job_executor: str | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n result_expiration_time: timedelta | float = 0,\n ) -> UUID:\n portal = self._ensure_services_ready()\n return portal.call(\n partial(\n self._async_scheduler.add_job,\n func_or_task_id,\n args=args,\n kwargs=kwargs,\n job_executor=job_executor,\n metadata=metadata,\n result_expiration_time=result_expiration_time,\n )\n )\n\n def get_jobs(self) -> Sequence[Job]:\n portal = self._ensure_services_ready()\n return portal.call(self._async_scheduler.get_jobs)\n\n def get_job_result(self, job_id: UUID, *, wait: bool = True) -> JobResult | None:\n portal = self._ensure_services_ready()\n return portal.call(\n partial(self._async_scheduler.get_job_result, job_id, wait=wait)\n )\n\n def run_job(\n self,\n func_or_task_id: str | Callable[..., Any],\n *,\n args: Iterable[Any] | None = None,\n kwargs: Mapping[str, Any] | None = None,\n job_executor: str | UnsetValue = unset,\n metadata: MetadataType | UnsetValue = unset,\n ) -> Any:\n portal = self._ensure_services_ready()\n return portal.call(\n partial(\n self._async_scheduler.run_job,\n func_or_task_id,\n args=args,\n kwargs=kwargs,\n job_executor=job_executor,\n metadata=metadata,\n )\n )\n\n def start_in_background(self) -> None:\n \"\"\"\n Launch the scheduler in a new thread.\n\n This method registers :mod:`atexit` hooks to shut down the scheduler and wait\n for the thread to finish.\n\n :raises RuntimeError: if the scheduler is not in the ``stopped`` state\n\n \"\"\"\n # Check if we're running under uWSGI with threads disabled\n uwsgi_module = sys.modules.get(\"uwsgi\")\n if not getattr(uwsgi_module, \"has_threads\", True):\n raise RuntimeError(\n \"The scheduler seems to be running under uWSGI, but threads have \"\n \"been disabled. You must run uWSGI with the --enable-threads \"\n \"option for the scheduler to work.\"\n )\n\n portal = self._ensure_services_ready()\n portal.call(self._async_scheduler.start_in_background)\n\n def stop(self) -> None:\n if self._portal is not None:\n self._portal.call(self._async_scheduler.stop)\n\n def wait_until_stopped(self) -> None:\n if self._portal is not None:\n self._portal.call(self._async_scheduler.wait_until_stopped)\n\n def run_until_stopped(self) -> None:\n with ExitStack() as exit_stack:\n # Run the async scheduler\n portal = self._ensure_services_ready(exit_stack)\n portal.call(self._async_scheduler.run_until_stopped)\n\n\n# Copy the docstrings from the async variant\nfor attrname in dir(AsyncScheduler):\n if attrname.startswith(\"_\"):\n continue\n\n value = getattr(AsyncScheduler, attrname)\n if callable(value):\n sync_method = getattr(Scheduler, attrname, None)\n if sync_method and not sync_method.__doc__:\n sync_method.__doc__ = value.__doc__\n" }, { "path": "src/apscheduler/_structures.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timedelta, timezone\nfrom functools import partial\nfrom typing import Any, TypeAlias\nfrom uuid import UUID, uuid4\n\nimport attrs\nfrom attr.setters import frozen\nfrom attrs.validators import and_, gt, instance_of, matches_re, min_len, optional\n\nfrom ._converters import as_aware_datetime, as_enum, as_timedelta\nfrom ._enums import CoalescePolicy, JobOutcome\nfrom ._utils import UnsetValue, unset\nfrom ._validators import if_not_unset, valid_metadata\nfrom .abc import Serializer, Trigger\n\nMetadataType: TypeAlias = dict[\n str, str | int | bool | None | list[\"MetadataType\"] | dict[str, \"MetadataType\"]\n]\n\n\ndef serialize(inst: Any, field: attrs.Attribute, value: Any) -> Any:\n if isinstance(value, frozenset):\n return list(value)\n\n return value\n\n\n@attrs.define(kw_only=True, order=False)\nclass Task:\n \"\"\"\n Represents a callable and its surrounding configuration parameters.\n\n :var str id: the unique identifier of this task\n :var ~collections.abc.Callable func: the callable that is called when this task is\n run\n :var str job_executor: name of the job executor that will run this task\n :var int | None max_running_jobs: maximum number of instances of this task that are\n allowed to run concurrently\n :var ~datetime.timedelta | None misfire_grace_time: maximum number of seconds the\n run time of jobs created for this task are allowed to be late, compared to the\n scheduled run time\n :var metadata: key-value pairs for storing JSON compatible custom information\n \"\"\"\n\n id: str = attrs.field(validator=[instance_of(str), min_len(1)], on_setattr=frozen)\n func: str | None = attrs.field(\n validator=optional(and_(instance_of(str), matches_re(r\".+:.+\"))),\n on_setattr=frozen,\n )\n job_executor: str = attrs.field(validator=instance_of(str), on_setattr=frozen)\n max_running_jobs: int | None = attrs.field(\n default=None,\n validator=optional(and_(instance_of(int), gt(0))),\n on_setattr=frozen,\n )\n misfire_grace_time: timedelta | None = attrs.field(\n default=None,\n converter=as_timedelta,\n validator=optional(instance_of(timedelta)),\n on_setattr=frozen,\n )\n metadata: MetadataType = attrs.field(validator=valid_metadata, factory=dict)\n running_jobs: int = attrs.field(default=0)\n\n def marshal(self, serializer: Serializer) -> dict[str, Any]:\n return attrs.asdict(self, value_serializer=serialize)\n\n @classmethod\n def unmarshal(cls, serializer: Serializer, marshalled: dict[str, Any]) -> Task:\n return cls(**marshalled)\n\n def __hash__(self) -> int:\n return hash(self.id)\n\n def __eq__(self, other: object) -> bool:\n if isinstance(other, Task):\n return self.id == other.id\n\n return NotImplemented\n\n def __lt__(self, other: object) -> bool:\n if isinstance(other, Task):\n return self.id < other.id\n\n return NotImplemented\n\n\n@attrs.define(kw_only=True)\nclass TaskDefaults:\n \"\"\"\n Contains default values for tasks that will be applied when no matching\n configuration value has been explicitly provided.\n\n :param str job_executor: name of the job executor that will run this task\n :param int | None max_running_jobs: maximum number of instances of this task that are\n allowed to run concurrently\n :param ~datetime.timedelta | None misfire_grace_time: maximum number of seconds the\n run time of jobs created for this task are allowed to be late, compared to the\n scheduled run time\n :var metadata: key-value pairs for storing JSON compatible custom information\n \"\"\"\n\n job_executor: str | UnsetValue = attrs.field(\n validator=if_not_unset(instance_of(str)), default=unset\n )\n max_running_jobs: int | None | UnsetValue = attrs.field(\n validator=optional(instance_of(int)), default=1\n )\n misfire_grace_time: timedelta | None = attrs.field(\n converter=as_timedelta,\n validator=optional(instance_of(timedelta)),\n default=None,\n )\n metadata: MetadataType = attrs.field(validator=valid_metadata, factory=dict)\n\n\n@attrs.define(kw_only=True, order=False)\nclass Schedule:\n \"\"\"\n Represents a schedule on which a task will be run.\n\n :var str id: the unique identifier of this schedule\n :var str task_id: unique identifier of the task to be run on this schedule\n :var Trigger trigger: the trigger that determines when the task will be run\n :var tuple args: positional arguments to pass to the task callable\n :var dict[str, Any] kwargs: keyword arguments to pass to the task callable\n :var bool paused: whether the schedule is paused\n :var CoalescePolicy coalesce: determines what to do when processing the schedule if\n multiple fire times have become due for this schedule since the last processing\n :var ~datetime.timedelta | None misfire_grace_time: maximum number of seconds the\n scheduled job's actual run time is allowed to be late, compared to the scheduled\n run time\n :var ~datetime.timedelta | None max_jitter: maximum number of seconds to randomly\n add to the scheduled time for each job created from this schedule\n :var ~datetime.timedelta job_result_expiration_time: minimum time to keep the job\n results in storage from the jobs created by this schedule\n :var metadata: key-value pairs for storing JSON compatible custom information\n :var ~datetime.datetime next_fire_time: the next time the task will be run\n :var ~datetime.datetime | None last_fire_time: the last time the task was scheduled\n to run\n :var str | None acquired_by: ID of the scheduler that has acquired this schedule for\n processing\n :var str | None acquired_until: the time after which other schedulers are free to\n acquire the schedule for processing even if it is still marked as acquired\n \"\"\"\n\n id: str = attrs.field(validator=[instance_of(str), min_len(1)], on_setattr=frozen)\n task_id: str = attrs.field(\n validator=[instance_of(str), min_len(1)], on_setattr=frozen\n )\n trigger: Trigger = attrs.field(\n validator=instance_of(Trigger), # type: ignore[type-abstract]\n on_setattr=frozen,\n )\n args: tuple = attrs.field(converter=tuple, default=())\n kwargs: dict[str, Any] = attrs.field(converter=dict, default=())\n paused: bool = attrs.field(default=False)\n coalesce: CoalescePolicy = attrs.field(\n default=CoalescePolicy.latest,\n converter=as_enum(CoalescePolicy),\n validator=instance_of(CoalescePolicy),\n on_setattr=frozen,\n )\n misfire_grace_time: timedelta | None = attrs.field(\n default=None,\n converter=as_timedelta,\n validator=optional(instance_of(timedelta)),\n on_setattr=frozen,\n )\n max_jitter: timedelta | None = attrs.field(\n converter=as_timedelta,\n default=None,\n validator=optional(instance_of(timedelta)),\n on_setattr=frozen,\n )\n job_executor: str = attrs.field(validator=instance_of(str), on_setattr=frozen)\n job_result_expiration_time: timedelta = attrs.field(\n default=0,\n converter=as_timedelta,\n validator=optional(instance_of(timedelta)),\n on_setattr=frozen,\n )\n metadata: MetadataType = attrs.field(validator=valid_metadata, factory=dict)\n next_fire_time: datetime | None = attrs.field(\n converter=as_aware_datetime,\n default=None,\n )\n last_fire_time: datetime | None = attrs.field(\n converter=as_aware_datetime,\n default=None,\n )\n acquired_by: str | None = attrs.field(default=None)\n acquired_until: datetime | None = attrs.field(\n converter=as_aware_datetime, default=None\n )\n\n def marshal(self, serializer: Serializer) -> dict[str, Any]:\n marshalled = attrs.asdict(self, recurse=False, value_serializer=serialize)\n marshalled[\"trigger\"] = serializer.serialize(self.trigger)\n marshalled[\"args\"] = serializer.serialize(self.args)\n marshalled[\"kwargs\"] = serializer.serialize(self.kwargs)\n if not self.acquired_by:\n del marshalled[\"acquired_by\"]\n del marshalled[\"acquired_until\"]\n\n return marshalled\n\n @classmethod\n def unmarshal(cls, serializer: Serializer, marshalled: dict[str, Any]) -> Schedule:\n marshalled[\"trigger\"] = serializer.deserialize(marshalled[\"trigger\"])\n marshalled[\"args\"] = serializer.deserialize(marshalled[\"args\"])\n marshalled[\"kwargs\"] = serializer.deserialize(marshalled[\"kwargs\"])\n return cls(**marshalled)\n\n def __hash__(self) -> int:\n return hash(self.id)\n\n def __eq__(self, other: object) -> bool:\n if isinstance(other, Schedule):\n return self.id == other.id\n\n return NotImplemented\n\n def __lt__(self, other: object) -> bool:\n if isinstance(other, Schedule):\n # Sort by next_fire_time first, exhausted schedules last\n if self.next_fire_time is not None and other.next_fire_time is not None:\n return self.next_fire_time < other.next_fire_time\n elif self.next_fire_time is None:\n return False\n elif other.next_fire_time is None:\n return True\n\n # In all other cases, sort by schedule ID\n return self.id < other.id\n\n return NotImplemented\n\n\n@attrs.define(kw_only=True, frozen=True)\nclass ScheduleResult:\n \"\"\"\n Represents a result of a schedule processing operation.\n\n :ivar schedule_id: ID of the schedule\n :ivar task_id: ID of the schedule's task\n :ivar trigger: the schedule's trigger\n :ivar last_fire_time: the schedule's trigger\n :ivar next_fire_time: the next\n \"\"\"\n\n schedule_id: str\n task_id: str\n trigger: Trigger\n last_fire_time: datetime\n next_fire_time: datetime | None\n\n\n@attrs.define(kw_only=True, order=False)\nclass Job:\n \"\"\"\n Represents a queued request to run a task.\n\n :var ~uuid.UUID id: autogenerated unique identifier of the job\n :var str task_id: unique identifier of the task to be run\n :var tuple args: positional arguments to pass to the task callable\n :var dict[str, Any] kwargs: keyword arguments to pass to the task callable\n :var str schedule_id: unique identifier of the associated schedule\n (if the job was derived from a schedule)\n :var ~datetime.datetime | None scheduled_fire_time: the time the job was scheduled\n to run at (if the job was derived from a schedule; includes jitter)\n :var ~datetime.timedelta | None jitter: the time that was randomly added to the\n calculated scheduled run time (if the job was derived from a schedule)\n :var ~datetime.datetime | None start_deadline: if the job is started in the\n scheduler after this time, it is considered to be misfired and will be aborted\n :var ~datetime.timedelta result_expiration_time: minimum amount of time to keep the\n result available for fetching in the data store\n :var metadata: key-value pairs for storing JSON compatible custom information\n :var ~datetime.datetime created_at: the time at which the job was created\n :var str | None acquired_by: the unique identifier of the scheduler that has\n acquired the job for execution\n :var str | None acquired_until: the time after which other schedulers are free to\n acquire the job for processing even if it is still marked as acquired\n \"\"\"\n\n id: UUID = attrs.field(factory=uuid4, on_setattr=frozen)\n task_id: str = attrs.field(on_setattr=frozen)\n args: tuple = attrs.field(\n converter=tuple, default=(), repr=False, on_setattr=frozen\n )\n kwargs: dict[str, Any] = attrs.field(\n converter=dict, factory=dict, repr=False, on_setattr=frozen\n )\n schedule_id: str | None = attrs.field(default=None, on_setattr=frozen)\n scheduled_fire_time: datetime | None = attrs.field(\n converter=as_aware_datetime, default=None, on_setattr=frozen\n )\n executor: str = attrs.field(on_setattr=frozen)\n jitter: timedelta = attrs.field(\n converter=as_timedelta, factory=timedelta, repr=False, on_setattr=frozen\n )\n start_deadline: datetime | None = attrs.field(\n converter=as_aware_datetime, default=None, repr=False, on_setattr=frozen\n )\n result_expiration_time: timedelta = attrs.field(\n converter=as_timedelta, default=timedelta(), repr=False, on_setattr=frozen\n )\n metadata: MetadataType = attrs.field(validator=valid_metadata, factory=dict)\n created_at: datetime = attrs.field(\n converter=as_aware_datetime,\n factory=partial(datetime.now, timezone.utc),\n on_setattr=frozen,\n )\n acquired_by: str | None = attrs.field(default=None, repr=False)\n acquired_until: datetime | None = attrs.field(\n converter=as_aware_datetime, default=None, repr=False\n )\n\n @property\n def original_scheduled_time(self) -> datetime | None:\n \"\"\"The scheduled time without any jitter included.\"\"\"\n if self.scheduled_fire_time is None:\n return None\n\n return self.scheduled_fire_time - self.jitter\n\n def marshal(self, serializer: Serializer) -> dict[str, Any]:\n marshalled = attrs.asdict(self, recurse=False, value_serializer=serialize)\n marshalled[\"args\"] = serializer.serialize(self.args)\n marshalled[\"kwargs\"] = serializer.serialize(self.kwargs)\n if not self.acquired_by:\n del marshalled[\"acquired_by\"]\n del marshalled[\"acquired_until\"]\n\n return marshalled\n\n @classmethod\n def unmarshal(cls, serializer: Serializer, marshalled: dict[str, Any]) -> Job:\n if args := marshalled[\"args\"]:\n marshalled[\"args\"] = serializer.deserialize(args)\n\n if kwargs := marshalled[\"kwargs\"]:\n marshalled[\"kwargs\"] = serializer.deserialize(kwargs)\n\n return cls(**marshalled)\n\n def __hash__(self) -> int:\n return hash(self.id)\n\n def __eq__(self, other: object) -> bool:\n if isinstance(other, Job):\n return self.id == other.id\n\n return NotImplemented\n\n\n@attrs.define(kw_only=True, frozen=True, eq=False)\nclass JobResult:\n \"\"\"\n Represents the result of running a job.\n\n :var ~uuid.UUID job_id: the unique identifier of the job\n :var JobOutcome outcome: indicates how the job ended\n :var ~datetime.datetime started_at: the time when the job was submitted to the\n executor (``None`` if the job never started in the first place)\n :var ~datetime.datetime finished_at: the time when the job finished running, or was\n discarded during the job acquisition process\n :var ~datetime.datetime expires_at: the time when the result will expire\n :var BaseException | None exception: the exception object if the job ended due to an\n exception being raised\n :var return_value: the return value from the task function (if the job ran to\n completion successfully)\n \"\"\"\n\n job_id: UUID\n outcome: JobOutcome = attrs.field(converter=as_enum(JobOutcome))\n started_at: datetime | None = attrs.field(converter=as_aware_datetime, default=None)\n finished_at: datetime = attrs.field(converter=as_aware_datetime)\n expires_at: datetime = attrs.field(converter=as_aware_datetime, repr=False)\n exception: BaseException | None = attrs.field(default=None, repr=False)\n return_value: Any = attrs.field(default=None, repr=False)\n\n @classmethod\n def from_job(\n cls,\n job: Job,\n outcome: JobOutcome,\n *,\n finished_at: datetime | None = None,\n started_at: datetime | None = None,\n exception: BaseException | None = None,\n return_value: Any = None,\n ) -> JobResult:\n real_finished_at = finished_at or datetime.now(timezone.utc)\n expires_at = real_finished_at + job.result_expiration_time\n return cls(\n job_id=job.id,\n outcome=outcome,\n started_at=started_at,\n finished_at=real_finished_at,\n expires_at=expires_at,\n exception=exception,\n return_value=return_value,\n )\n\n def marshal(self, serializer: Serializer) -> dict[str, Any]:\n marshalled = attrs.asdict(self, value_serializer=serialize)\n if self.outcome is JobOutcome.error:\n marshalled[\"exception\"] = serializer.serialize(self.exception)\n else:\n del marshalled[\"exception\"]\n\n if self.outcome is JobOutcome.success:\n marshalled[\"return_value\"] = serializer.serialize(self.return_value)\n else:\n del marshalled[\"return_value\"]\n\n return marshalled\n\n @classmethod\n def unmarshal(cls, serializer: Serializer, marshalled: dict[str, Any]) -> JobResult:\n if marshalled.get(\"exception\"):\n marshalled[\"exception\"] = serializer.deserialize(marshalled[\"exception\"])\n elif marshalled.get(\"return_value\"):\n marshalled[\"return_value\"] = serializer.deserialize(\n marshalled[\"return_value\"]\n )\n\n return cls(**marshalled)\n\n def __hash__(self) -> int:\n return hash(self.job_id)\n\n def __eq__(self, other: object) -> bool:\n if isinstance(other, JobResult):\n return self.job_id == other.job_id\n\n return NotImplemented\n" }, { "path": "src/apscheduler/_utils.py", "content": "\"\"\"This module contains several handy functions primarily meant for internal use.\"\"\"\n\nfrom __future__ import annotations\n\nimport asyncio\nfrom datetime import datetime, tzinfo\nfrom typing import TYPE_CHECKING, Any, NoReturn, TypeVar\nfrom zoneinfo import ZoneInfo\n\nfrom ._exceptions import DeserializationError\nfrom .abc import Trigger\n\ntry:\n import sniffio\nexcept ImportError:\n sniffio = None\n\nif TYPE_CHECKING:\n from ._structures import MetadataType\n\nT = TypeVar(\"T\")\n\n\nclass UnsetValue:\n \"\"\"The type of :data:`unset`.\"\"\"\n\n __slots__ = ()\n\n def __new__(cls) -> UnsetValue:\n try:\n return unset\n except NameError:\n return super().__new__(cls)\n\n def __getstate__(self) -> NoReturn:\n raise RuntimeError(\"Internal error: attempted to serialize an unset value\")\n\n def __repr__(self) -> str:\n return \"\"\n\n\nunset = UnsetValue()\n\n\ndef timezone_repr(timezone: tzinfo) -> str:\n if isinstance(timezone, ZoneInfo):\n return timezone.key\n else:\n return repr(timezone)\n\n\ndef absolute_datetime_diff(dateval1: datetime, dateval2: datetime) -> float:\n return dateval1.timestamp() - dateval2.timestamp()\n\n\ndef qualified_name(cls: type) -> str:\n module = getattr(cls, \"__module__\", None)\n if module is None or module == \"builtins\":\n return cls.__qualname__\n else:\n return f\"{module}.{cls.__qualname__}\"\n\n\ndef require_state_version(\n trigger: Trigger, state: dict[str, Any], max_version: int\n) -> None:\n try:\n if state[\"version\"] > max_version:\n raise DeserializationError(\n f\"{trigger.__class__.__name__} received a serialized state with \"\n f\"version {state['version']}, but it only supports up to version \"\n f\"{max_version}. This can happen when an older version of APScheduler \"\n f\"is being used with a data store that was previously used with a \"\n f\"newer APScheduler version.\"\n )\n except KeyError as exc:\n raise DeserializationError(\n 'Missing \"version\" key in the serialized state'\n ) from exc\n\n\ndef merge_metadata(\n base_metadata: MetadataType, *overlays: MetadataType | UnsetValue\n) -> MetadataType:\n new_metadata = base_metadata.copy()\n for metadata in overlays:\n if isinstance(metadata, UnsetValue):\n continue\n\n new_metadata.update(metadata)\n\n return new_metadata\n\n\ndef create_repr(instance: object, *attrnames: str, **kwargs) -> str:\n kv_pairs: list[tuple[str, object]] = []\n for attrname in attrnames:\n value = getattr(instance, attrname)\n if value is not unset and value is not None:\n kv_pairs.append((attrname, value))\n\n for key, value in kwargs.items():\n if value is not unset and value is not None:\n kv_pairs.append((key, value))\n\n rendered_attrs = \", \".join(f\"{key}={value!r}\" for key, value in kv_pairs)\n return f\"{instance.__class__.__name__}({rendered_attrs})\"\n\n\ndef time_exists(dt: datetime) -> bool:\n \"\"\"\n Determine whether a datetime exists in its time zone.\n\n :return: ``False`` if the given datetime falls within a gap created by a\n forward daylight savings shift, otherwise ``True``\n\n \"\"\"\n return dt == datetime.fromtimestamp(dt.timestamp(), dt.tzinfo)\n\n\ndef current_async_library() -> str:\n \"\"\"Return the name of the currently used async library.\"\"\"\n if sniffio is not None:\n return sniffio.current_async_library() or \"(unknown)\"\n\n try:\n asyncio.get_running_loop()\n return \"asyncio\"\n except RuntimeError:\n return \"(unknown)\"\n" }, { "path": "src/apscheduler/_validators.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom typing import Any\n\nfrom attrs import Attribute\n\nfrom apscheduler._utils import unset\n\n\ndef positive_number(instance: Any, attribute: Attribute, value: Any) -> None:\n if value <= 0:\n raise ValueError(f\"{attribute.name} must be positive, got: {value}\")\n\n\ndef non_negative_number(instance: Any, attribute: Attribute, value: Any) -> None:\n if value < 0:\n raise ValueError(f\"{attribute.name} must be non-negative, got: {value}\")\n\n\ndef aware_datetime(instance: Any, attribute: Attribute, value: Any) -> None:\n if not value.tzinfo:\n raise ValueError(f\"{attribute.name} must be a timezone aware datetime\")\n\n\ndef if_not_unset(validator: Callable[[Any, Any, Any], None]) -> None:\n def validate(instance: Any, attribute: Any, value: Any) -> None:\n if value is unset:\n return\n\n validator(instance, attribute, value)\n\n\ndef valid_metadata(instance: Any, attribute: Attribute, value: Any) -> None:\n def check_value(path: str, val: object) -> None:\n if val is None:\n return\n\n if isinstance(val, list):\n for index, item in enumerate(val):\n check_value(f\"{path}[{index}]\", item)\n elif isinstance(val, dict):\n for k, v in val.items():\n if not isinstance(k, str):\n raise ValueError(f\"{path} has a non-string key ({k!r})\")\n\n check_value(f\"{path}[{k!r}]\", v)\n elif not isinstance(val, (str, int, float, bool)):\n raise ValueError(\n f\"{path} has a value that is not JSON compatible: ({val!r})\"\n )\n\n if not isinstance(value, dict):\n raise ValueError(f\"{attribute.name} must be a dict, got: {value!r}\")\n\n for key, val in value.items():\n check_value(key, val)\n" }, { "path": "src/apscheduler/abc.py", "content": "from __future__ import annotations\n\nimport sys\nfrom abc import ABCMeta, abstractmethod\nfrom collections.abc import Callable, Iterable, Iterator, Sequence\nfrom contextlib import AsyncExitStack\nfrom datetime import datetime, timedelta\nfrom logging import Logger\nfrom typing import TYPE_CHECKING, Any\nfrom uuid import UUID\n\nif sys.version_info >= (3, 11):\n from typing import Self\nelse:\n from typing_extensions import Self\n\nif TYPE_CHECKING:\n from ._enums import ConflictPolicy\n from ._events import Event, T_Event\n from ._structures import Job, JobResult, Schedule, ScheduleResult, Task\n\n\nclass Trigger(Iterator[datetime], metaclass=ABCMeta):\n \"\"\"\n Abstract base class that defines the interface that every trigger must implement.\n \"\"\"\n\n __slots__ = ()\n\n @abstractmethod\n def next(self) -> datetime | None:\n \"\"\"\n Return the next datetime to fire on.\n\n If no such datetime can be calculated, ``None`` is returned.\n\n :raises MaxIterationsReached: if the trigger's internal logic has exceeded a set\n maximum of iterations (used to detect potentially infinite loops)\n\n \"\"\"\n\n @abstractmethod\n def __getstate__(self) -> Any:\n \"\"\"Return the serializable state of the trigger.\"\"\"\n\n @abstractmethod\n def __setstate__(self, state: Any) -> None:\n \"\"\"Initialize an empty instance from an existing state.\"\"\"\n\n def __iter__(self) -> Self:\n return self\n\n def __next__(self) -> datetime:\n dateval = self.next()\n if dateval is None:\n raise StopIteration\n else:\n return dateval\n\n\nclass Serializer(metaclass=ABCMeta):\n \"\"\"Interface for classes that implement (de)serialization.\"\"\"\n\n __slots__ = ()\n\n @abstractmethod\n def serialize(self, obj: object) -> bytes:\n \"\"\"\n Turn the given object into a bytestring.\n\n Must handle the serialization of at least any JSON type, plus the following:\n\n * ``datetime.date`` (using :meth:`datetime.date.isoformat`)\n * ``datetime.timedelta`` (using :meth:`datetime.timedelta.total_seconds`)\n * ``datetime.tzinfo`` (by extracting the time zone name)\n\n :return: a bytestring that can be later restored using :meth:`deserialize`\n \"\"\"\n\n @abstractmethod\n def deserialize(self, serialized: bytes) -> Any:\n \"\"\"\n Restore a previously serialized object from bytestring\n\n :param serialized: a bytestring previously received from :meth:`serialize`\n :return: a copy of the original object\n \"\"\"\n\n\nclass Subscription(metaclass=ABCMeta):\n \"\"\"\n Represents a subscription with an event source.\n\n If used as a context manager, unsubscribes on exit.\n \"\"\"\n\n def __enter__(self) -> Subscription:\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb) -> None:\n self.unsubscribe()\n\n @abstractmethod\n def unsubscribe(self) -> None:\n \"\"\"\n Cancel this subscription.\n\n Does nothing if the subscription has already been cancelled.\n \"\"\"\n\n\nclass EventBroker(metaclass=ABCMeta):\n \"\"\"\n Interface for objects that can be used to publish notifications to interested\n subscribers.\n \"\"\"\n\n @abstractmethod\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n \"\"\"\n Start the event broker.\n\n :param exit_stack: an asynchronous exit stack which will be processed when the\n scheduler is shut down\n :param logger: the logger object the event broker should use to log events\n \"\"\"\n\n @abstractmethod\n async def publish(self, event: Event) -> None:\n \"\"\"Publish an event.\"\"\"\n\n @abstractmethod\n async def publish_local(self, event: Event) -> None:\n \"\"\"Publish an event, but only to local subscribers.\"\"\"\n\n @abstractmethod\n def subscribe(\n self,\n callback: Callable[[T_Event], Any],\n event_types: Iterable[type[T_Event]] | None = None,\n *,\n is_async: bool = True,\n one_shot: bool = False,\n ) -> Subscription:\n \"\"\"\n Subscribe to events from this event broker.\n\n :param callback: callable to be called with the event object when an event is\n published\n :param event_types: an iterable of concrete Event classes to subscribe to\n :param is_async: ``True`` if the (synchronous) callback should be called on the\n event loop thread, ``False`` if it should be called in a scheduler thread.\n If the callback is a coroutine function, this flag is ignored.\n :param one_shot: if ``True``, automatically unsubscribe after the first matching\n event\n \"\"\"\n\n\nclass DataStore(metaclass=ABCMeta):\n \"\"\"\n Interface for data stores.\n\n Data stores keep track of tasks, schedules and jobs. When these objects change, the\n data store publishes events to the associated event broker accordingly.\n \"\"\"\n\n @abstractmethod\n async def start(\n self, exit_stack: AsyncExitStack, event_broker: EventBroker, logger: Logger\n ) -> None:\n \"\"\"\n Start the event broker.\n\n :param exit_stack: an asynchronous exit stack which will be processed when the\n scheduler is shut down\n :param event_broker: the event broker shared between the scheduler, scheduler\n (if any) and this data store\n :param logger: the logger object the data store should use to log events\n \"\"\"\n\n @abstractmethod\n async def add_task(self, task: Task) -> None:\n \"\"\"\n Add the given task to the store.\n\n If a task with the same ID already exists, it replaces the old one but does NOT\n affect task accounting (# of running jobs).\n\n :param task: the task to be added\n \"\"\"\n\n @abstractmethod\n async def remove_task(self, task_id: str) -> None:\n \"\"\"\n Remove the task with the given ID.\n\n :param task_id: ID of the task to be removed\n :raises TaskLookupError: if no matching task was found\n \"\"\"\n\n @abstractmethod\n async def get_task(self, task_id: str) -> Task:\n \"\"\"\n Get an existing task definition.\n\n :param task_id: ID of the task to be returned\n :return: the matching task\n :raises TaskLookupError: if no matching task was found\n \"\"\"\n\n @abstractmethod\n async def get_tasks(self) -> list[Task]:\n \"\"\"\n Get all the tasks in this store.\n\n :return: a list of tasks, sorted by ID\n \"\"\"\n\n @abstractmethod\n async def get_schedules(self, ids: set[str] | None = None) -> list[Schedule]:\n \"\"\"\n Get schedules from the data store.\n\n :param ids: a specific set of schedule IDs to return, or ``None`` to return all\n schedules\n :return: the list of matching schedules, in unspecified order\n \"\"\"\n\n @abstractmethod\n async def add_schedule(\n self, schedule: Schedule, conflict_policy: ConflictPolicy\n ) -> None:\n \"\"\"\n Add or update the given schedule in the data store.\n\n :param schedule: schedule to be added\n :param conflict_policy: policy that determines what to do if there is an\n existing schedule with the same ID\n \"\"\"\n\n @abstractmethod\n async def remove_schedules(self, ids: Iterable[str]) -> None:\n \"\"\"\n Remove schedules from the data store.\n\n :param ids: a specific set of schedule IDs to remove\n \"\"\"\n\n @abstractmethod\n async def acquire_schedules(\n self, scheduler_id: str, lease_duration: timedelta, limit: int\n ) -> list[Schedule]:\n \"\"\"\n Acquire unclaimed due schedules for processing.\n\n This method claims up to the requested number of schedules for the given\n scheduler and returns them.\n\n For a stored schedule to be eligible for acquisition, it must fulfill one of the\n following conditions:\n\n * It is unclaimed (``acquired_until`` is ``None``)\n * Its claim has expired (``acquired_until`` is less than the current datetime)\n * It is claimed by the given scheduler (``acquired_by`` equals ``scheduler_id``)\n\n :param scheduler_id: unique identifier of the scheduler\n :param lease_duration: the duration of the lease, after which the schedules can be\n acquired by another scheduler even if ``acquired_by`` is not ``None``\n :param limit: maximum number of schedules to claim\n :return: the list of claimed schedules\n \"\"\"\n\n @abstractmethod\n async def release_schedules(\n self, scheduler_id: str, results: Sequence[ScheduleResult]\n ) -> None:\n \"\"\"\n Release the claims on the given schedules and update them on the store.\n\n The data store is responsible for updating the following fields on stored\n schedules:\n\n * ``last_fire_time``\n * ``next_fire_time``\n * ``trigger``\n * ``acquired_by`` (must beset to ``None``)\n * ``acquired_until`` (must be set to ``None``)\n\n :param scheduler_id: unique identifier of the scheduler\n :param results: list of schedule processing results\n \"\"\"\n\n @abstractmethod\n async def get_next_schedule_run_time(self) -> datetime | None:\n \"\"\"\n Return the earliest upcoming run time of all the schedules in the store, or\n ``None`` if there are no active schedules.\n \"\"\"\n\n @abstractmethod\n async def add_job(self, job: Job) -> None:\n \"\"\"\n Add a job to be executed by an eligible scheduler.\n\n :param job: the job object\n \"\"\"\n\n @abstractmethod\n async def get_jobs(self, ids: Iterable[UUID] | None = None) -> list[Job]:\n \"\"\"\n Get the list of pending jobs.\n\n :param ids: a specific set of job IDs to return, or ``None`` to return all jobs\n :return: the list of matching pending jobs, in the order they will be given to\n schedulers\n \"\"\"\n\n @abstractmethod\n async def acquire_jobs(\n self, scheduler_id: str, lease_duration: timedelta, limit: int | None = None\n ) -> list[Job]:\n \"\"\"\n Acquire unclaimed jobs for execution.\n\n This method claims up to the requested number of jobs for the given scheduler\n and returns them.\n\n :param scheduler_id: unique identifier of the scheduler\n :param lease_duration: the duration of the lease, after which the jobs will be\n considered to be dead if the scheduler doesn't extend the lease duration\n :param limit: maximum number of jobs to claim and return\n :return: the list of claimed jobs\n \"\"\"\n\n @abstractmethod\n async def release_job(self, scheduler_id: str, job: Job, result: JobResult) -> None:\n \"\"\"\n Release the claim on the given job and record the result.\n\n :param scheduler_id: unique identifier of the scheduler\n :param job: the job to be released\n :param result: the result of the job\n \"\"\"\n\n @abstractmethod\n async def get_job_result(self, job_id: UUID) -> JobResult | None:\n \"\"\"\n Retrieve the result of a job.\n\n The result is removed from the store after retrieval.\n\n :param job_id: the identifier of the job\n :return: the result, or ``None`` if the result was not found\n \"\"\"\n\n @abstractmethod\n async def extend_acquired_schedule_leases(\n self, scheduler_id: str, schedule_ids: set[str], duration: timedelta\n ) -> None:\n \"\"\"\n Extend the leases of specified schedules acquired by the given scheduler.\n\n :param scheduler_id: unique identifier of the scheduler\n :param schedule_ids: the identifiers of the schedules the scheduler is currently\n processing\n :param duration: the duration by which to extend the leases\n \"\"\"\n\n @abstractmethod\n async def extend_acquired_job_leases(\n self, scheduler_id: str, job_ids: set[UUID], duration: timedelta\n ) -> None:\n \"\"\"\n Extend the leases of specified jobs acquired by the given scheduler.\n\n :param scheduler_id: unique identifier of the scheduler\n :param job_ids: the identifiers of the jobs the scheduler is running\n :param duration: the duration by which to extend the leases\n \"\"\"\n\n @abstractmethod\n async def reap_abandoned_jobs(self, scheduler_id: str) -> None:\n \"\"\"\n Find jobs marked as acquired by the given scheduler ID and release them with the\n outcome of :attr:`~apscheduler.JobOutcome.abandoned`.\n\n Implementers must ensure that the proper :class:`~apscheduler.JobReleased`\n events are published.\n\n This method is called once during the scheduler startup sequence.\n\n :param scheduler_id: unique identifier of the scheduler\n \"\"\"\n\n @abstractmethod\n async def cleanup(self) -> None:\n \"\"\"\n Perform clean-up operations on the data store.\n\n This method must perform the following operations (in this order):\n\n * Purge expired job results (where ``expires_at`` is less or equal to the\n current time)\n * Release jobs with expired leases with the ``cancelled`` outcome\n * Purge finished schedules (where ``next_run_time`` is ``None``) that have no\n running jobs associated with them\n \"\"\"\n\n\nclass JobExecutor(metaclass=ABCMeta):\n async def start(self, exit_stack: AsyncExitStack) -> None: # noqa: B027\n \"\"\"\n Start the job executor.\n\n :param exit_stack: an asynchronous exit stack which will be processed when the\n scheduler is shut down\n \"\"\"\n\n @abstractmethod\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n \"\"\"\n Run the given job by calling the given function.\n\n :param func: the function to call\n :param job: the associated job\n :return: the return value of ``func`` (potentially awaiting on the returned\n aawaitable, if any)\n \"\"\"\n" }, { "path": "src/apscheduler/datastores/__init__.py", "content": "" }, { "path": "src/apscheduler/datastores/base.py", "content": "from __future__ import annotations\n\nfrom contextlib import AsyncExitStack\nfrom logging import Logger\n\nimport attrs\n\nfrom .._retry import RetryMixin\nfrom ..abc import DataStore, EventBroker, Serializer\nfrom ..serializers.pickle import PickleSerializer\n\n\n@attrs.define(kw_only=True)\nclass BaseDataStore(DataStore):\n \"\"\"Base class for data stores.\"\"\"\n\n _event_broker: EventBroker = attrs.field(init=False)\n _logger: Logger = attrs.field(init=False)\n\n async def start(\n self, exit_stack: AsyncExitStack, event_broker: EventBroker, logger: Logger\n ) -> None:\n self._event_broker = event_broker\n self._logger = logger\n\n\n@attrs.define(kw_only=True)\nclass BaseExternalDataStore(BaseDataStore, RetryMixin):\n \"\"\"\n Base class for data stores using an external service such as a database.\n\n :param serializer: the serializer used to (de)serialize tasks, schedules and jobs\n for storage\n :param start_from_scratch: erase all existing data during startup (useful for test\n suites)\n \"\"\"\n\n serializer: Serializer = attrs.field(factory=PickleSerializer)\n start_from_scratch: bool = attrs.field(default=False)\n" }, { "path": "src/apscheduler/datastores/memory.py", "content": "from __future__ import annotations\n\nfrom bisect import bisect_left, bisect_right, insort_right\nfrom collections import defaultdict\nfrom collections.abc import Iterable, Sequence\nfrom datetime import MAXYEAR, datetime, timedelta, timezone\nfrom functools import partial\nfrom uuid import UUID\n\nimport attrs\n\nfrom .. import JobOutcome\nfrom .._enums import ConflictPolicy\nfrom .._events import (\n JobAcquired,\n JobAdded,\n JobReleased,\n ScheduleAdded,\n ScheduleRemoved,\n ScheduleUpdated,\n TaskAdded,\n TaskRemoved,\n TaskUpdated,\n)\nfrom .._exceptions import ConflictingIdError, TaskLookupError\nfrom .._structures import Job, JobResult, Schedule, ScheduleResult, Task\nfrom .._utils import create_repr\nfrom .base import BaseDataStore\n\nmax_datetime = datetime(MAXYEAR, 12, 31, 23, 59, 59, 999999, tzinfo=timezone.utc)\n\n\n@attrs.define(eq=False, repr=False)\nclass MemoryDataStore(BaseDataStore):\n \"\"\"\n Stores scheduler data in memory, without serializing it.\n\n Can be shared between multiple schedulers within the same event loop.\n \"\"\"\n\n _tasks: dict[str, Task] = attrs.Factory(dict)\n _schedules: list[Schedule] = attrs.Factory(list)\n _schedules_by_id: dict[str, Schedule] = attrs.Factory(dict)\n _schedules_by_task_id: dict[str, set[Schedule]] = attrs.Factory(\n partial(defaultdict, set)\n )\n _jobs_by_id: dict[UUID, Job] = attrs.Factory(dict)\n _jobs_by_task_id: dict[str, set[Job]] = attrs.Factory(partial(defaultdict, set))\n _jobs_by_schedule_id: dict[str, set[Job]] = attrs.Factory(partial(defaultdict, set))\n _job_results: dict[UUID, JobResult] = attrs.Factory(dict)\n\n def __repr__(self) -> str:\n return create_repr(self)\n\n def _find_schedule_index(self, schedule: Schedule) -> int:\n left_index = bisect_left(self._schedules, schedule)\n right_index = bisect_right(self._schedules, schedule)\n return self._schedules.index(schedule, left_index, right_index + 1)\n\n async def get_schedules(self, ids: set[str] | None = None) -> list[Schedule]:\n if ids is None:\n return self._schedules.copy()\n\n return [\n schedule\n for schedule in self._schedules\n if ids is None or schedule.id in ids\n ]\n\n async def add_task(self, task: Task) -> None:\n if task.id in self._tasks:\n task.running_jobs = self._tasks[task.id].running_jobs\n self._tasks[task.id] = task\n await self._event_broker.publish(TaskUpdated(task_id=task.id))\n else:\n self._tasks[task.id] = task\n await self._event_broker.publish(TaskAdded(task_id=task.id))\n\n async def remove_task(self, task_id: str) -> None:\n try:\n del self._tasks[task_id]\n except KeyError:\n raise TaskLookupError(task_id) from None\n\n await self._event_broker.publish(TaskRemoved(task_id=task_id))\n\n async def get_task(self, task_id: str) -> Task:\n try:\n return self._tasks[task_id]\n except KeyError:\n raise TaskLookupError(task_id) from None\n\n async def get_tasks(self) -> list[Task]:\n return sorted(self._tasks.values())\n\n async def add_schedule(\n self, schedule: Schedule, conflict_policy: ConflictPolicy\n ) -> None:\n old_schedule = self._schedules_by_id.get(schedule.id)\n if old_schedule is not None:\n if conflict_policy is ConflictPolicy.do_nothing:\n return\n elif conflict_policy is ConflictPolicy.exception:\n raise ConflictingIdError(schedule.id)\n\n index = self._find_schedule_index(old_schedule)\n del self._schedules[index]\n self._schedules_by_task_id[old_schedule.task_id].remove(old_schedule)\n\n self._schedules_by_id[schedule.id] = schedule\n self._schedules_by_task_id[schedule.task_id].add(schedule)\n insort_right(self._schedules, schedule)\n\n event: ScheduleUpdated | ScheduleAdded\n if old_schedule is not None:\n event = ScheduleUpdated(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n else:\n event = ScheduleAdded(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n\n await self._event_broker.publish(event)\n\n async def remove_schedules(\n self, ids: Iterable[str], *, finished: bool = False\n ) -> None:\n for schedule_id in ids:\n schedule = self._schedules_by_id.pop(schedule_id, None)\n if schedule:\n self._schedules.remove(schedule)\n event = ScheduleRemoved(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n finished=finished,\n )\n await self._event_broker.publish(event)\n\n async def acquire_schedules(\n self, scheduler_id: str, lease_duration: timedelta, limit: int\n ) -> list[Schedule]:\n now = datetime.now(timezone.utc)\n acquired_until = now + lease_duration\n schedules: list[Schedule] = []\n for schedule in self._schedules:\n if schedule.next_fire_time is None or schedule.next_fire_time > now:\n # The schedule is either exhausted or not yet due. There will be no\n # schedules that are due after this one, so we can stop here.\n break\n elif schedule.paused:\n # The schedule is paused\n continue\n elif schedule.acquired_until is not None:\n if (\n schedule.acquired_by != scheduler_id\n and now <= schedule.acquired_until\n ):\n # The schedule has been acquired by another scheduler and the\n # timeout has not expired yet\n continue\n\n schedules.append(schedule)\n schedule.acquired_by = scheduler_id\n schedule.acquired_until = acquired_until\n if len(schedules) == limit:\n break\n\n return schedules\n\n async def release_schedules(\n self, scheduler_id: str, results: Sequence[ScheduleResult]\n ) -> None:\n # Send update events for schedules\n for result in results:\n # Remove the schedule\n schedule = self._schedules_by_id[result.schedule_id]\n index = self._find_schedule_index(schedule)\n del self._schedules[index]\n\n # Re-add the schedule to its new position\n schedule.last_fire_time = result.last_fire_time\n schedule.next_fire_time = result.next_fire_time\n schedule.acquired_by = None\n schedule.acquired_until = None\n insort_right(self._schedules, schedule)\n event = ScheduleUpdated(\n schedule_id=result.schedule_id,\n task_id=schedule.task_id,\n next_fire_time=result.next_fire_time,\n )\n await self._event_broker.publish(event)\n\n async def get_next_schedule_run_time(self) -> datetime | None:\n return self._schedules[0].next_fire_time if self._schedules else None\n\n async def add_job(self, job: Job) -> None:\n self._jobs_by_id[job.id] = job\n self._jobs_by_task_id[job.task_id].add(job)\n if job.schedule_id is not None:\n self._jobs_by_schedule_id[job.schedule_id].add(job)\n\n event = JobAdded(\n job_id=job.id,\n task_id=job.task_id,\n schedule_id=job.schedule_id,\n )\n await self._event_broker.publish(event)\n\n async def get_jobs(self, ids: Iterable[UUID] | None = None) -> list[Job]:\n if ids is not None:\n ids = frozenset(ids)\n\n if ids is None:\n return list(self._jobs_by_id.values())\n\n return [\n job for job in self._jobs_by_id.values() if ids is None or job.id in ids\n ]\n\n async def acquire_jobs(\n self, scheduler_id: str, lease_duration: timedelta, limit: int | None = None\n ) -> list[Job]:\n now = datetime.now(timezone.utc)\n acquired_until = now + lease_duration\n jobs: list[Job] = []\n job_results: dict[Job, JobResult] = {}\n for job in self._jobs_by_id.values():\n task = self._tasks[job.task_id]\n\n # Skip already acquired jobs (unless the acquisition lock has expired)\n if job.acquired_until is not None:\n if job.acquired_until >= now:\n continue\n else:\n task.running_jobs -= 1\n\n # Discard the job if its start deadline has passed\n if job.start_deadline and job.start_deadline < now:\n job_results[job] = JobResult(\n job_id=job.id,\n outcome=JobOutcome.missed_start_deadline,\n finished_at=now,\n expires_at=now + job.result_expiration_time,\n )\n continue\n\n # Skip the job if no more slots are available\n if (\n task.max_running_jobs is not None\n and task.running_jobs >= task.max_running_jobs\n ):\n self._logger.debug(\n \"Skipping job %s because task %r has the maximum number of %d jobs \"\n \"already running\",\n job.id,\n job.task_id,\n task.running_jobs,\n )\n continue\n\n # Mark the job as acquired by this worker\n jobs.append(job)\n job.acquired_by = scheduler_id\n job.acquired_until = acquired_until\n\n # Increment the number of running jobs for this task\n task.running_jobs += 1\n\n # Exit the loop if enough jobs have been acquired\n if len(jobs) == limit:\n break\n\n # Publish the appropriate events\n for job in jobs:\n await self._event_broker.publish(\n JobAcquired.from_job(job, scheduler_id=scheduler_id)\n )\n\n # Discard the jobs that could not start\n for job, result in job_results.items():\n await self.release_job(scheduler_id, job, result)\n\n return jobs\n\n async def release_job(self, scheduler_id: str, job: Job, result: JobResult) -> None:\n # Record the job result\n if result.expires_at > result.finished_at:\n self._job_results[result.job_id] = result\n\n # Decrement the number of running jobs for this task\n if job.acquired_by:\n self._tasks[job.task_id].running_jobs -= 1\n\n # Delete the job\n job = self._jobs_by_id.pop(result.job_id)\n\n # Remove the job from the jobs belonging to its task\n task_jobs = self._jobs_by_task_id[job.task_id]\n task_jobs.remove(job)\n if not task_jobs:\n del self._jobs_by_task_id[job.task_id]\n\n # If this was a scheduled job, remove the job from the set of jobs belonging to\n # this schedule\n if job.schedule_id:\n schedule_jobs = self._jobs_by_schedule_id[job.schedule_id]\n schedule_jobs.remove(job)\n if not schedule_jobs:\n del self._jobs_by_schedule_id[job.schedule_id]\n\n # Notify other schedulers\n await self._event_broker.publish(\n JobReleased.from_result(\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n )\n )\n\n async def get_job_result(self, job_id: UUID) -> JobResult | None:\n return self._job_results.pop(job_id, None)\n\n async def extend_acquired_schedule_leases(\n self, scheduler_id: str, schedule_ids: set[str], duration: timedelta\n ) -> None:\n acquired_until = datetime.now(timezone.utc) + duration\n for schedule in self._schedules:\n if schedule.acquired_by == scheduler_id and schedule.id in schedule_ids:\n schedule.acquired_until = acquired_until\n\n async def extend_acquired_job_leases(\n self, scheduler_id: str, job_ids: set[UUID], duration: timedelta\n ) -> None:\n acquired_until = datetime.now(timezone.utc) + duration\n for job in self._jobs_by_id.values():\n if job.acquired_by == scheduler_id and job.id in job_ids:\n job.acquired_until = acquired_until\n\n async def reap_abandoned_jobs(self, scheduler_id: str) -> None:\n now = datetime.now(timezone.utc)\n for job in list(self._jobs_by_id.values()):\n if job.acquired_by == scheduler_id:\n result = JobResult.from_job(\n job=job, outcome=JobOutcome.abandoned, finished_at=now\n )\n await self.release_job(job.acquired_by, job, result)\n\n async def cleanup(self) -> None:\n # Clean up expired job results\n now = datetime.now(timezone.utc)\n expired_job_ids = [\n result.job_id\n for result in self._job_results.values()\n if result.expires_at <= now\n ]\n for job_id in expired_job_ids:\n del self._job_results[job_id]\n\n # Finish any jobs whose leases have expired\n expired_jobs = [\n job\n for job in self._jobs_by_id.values()\n if job.acquired_until is not None and job.acquired_until < now\n ]\n for job in expired_jobs:\n result = JobResult.from_job(\n job=job, outcome=JobOutcome.abandoned, finished_at=now\n )\n assert job.acquired_by is not None\n await self.release_job(job.acquired_by, job, result)\n\n # Clean up finished schedules that have no running jobs\n finished_schedule_ids = [\n schedule_id\n for schedule_id, schedule in self._schedules_by_id.items()\n if schedule.next_fire_time is None\n and schedule_id not in self._jobs_by_schedule_id\n ]\n await self.remove_schedules(finished_schedule_ids, finished=True)\n" }, { "path": "src/apscheduler/datastores/mongodb.py", "content": "from __future__ import annotations\n\nimport operator\nfrom collections.abc import (\n Callable,\n Iterable,\n Mapping,\n Sequence,\n)\nfrom contextlib import AsyncExitStack\nfrom datetime import datetime, timedelta, timezone\nfrom logging import Logger\nfrom typing import Any, ClassVar, TypeVar, cast\nfrom uuid import UUID\n\nimport attrs\nimport pymongo\nfrom attrs.validators import instance_of\nfrom bson import CodecOptions, UuidRepresentation\nfrom bson.codec_options import TypeEncoder, TypeRegistry\nfrom pymongo import ASCENDING, DeleteOne, UpdateOne\nfrom pymongo.asynchronous.client_session import AsyncClientSession\nfrom pymongo.asynchronous.collection import AsyncCollection\nfrom pymongo.asynchronous.mongo_client import AsyncMongoClient\nfrom pymongo.errors import ConnectionFailure, DuplicateKeyError\n\nfrom .._enums import CoalescePolicy, ConflictPolicy, JobOutcome\nfrom .._events import (\n DataStoreEvent,\n JobAcquired,\n JobAdded,\n JobReleased,\n ScheduleAdded,\n ScheduleRemoved,\n ScheduleUpdated,\n TaskAdded,\n TaskRemoved,\n TaskUpdated,\n)\nfrom .._exceptions import (\n ConflictingIdError,\n DeserializationError,\n SerializationError,\n TaskLookupError,\n)\nfrom .._structures import Job, JobResult, Schedule, ScheduleResult, Task\nfrom .._utils import create_repr\nfrom ..abc import EventBroker\nfrom .base import BaseExternalDataStore\n\nT = TypeVar(\"T\", bound=Mapping[str, Any])\n\n\nclass CustomEncoder(TypeEncoder):\n def __init__(self, python_type: type, encoder: Callable):\n self._python_type = python_type\n self._encoder = encoder\n\n @property\n def python_type(self) -> type:\n return self._python_type\n\n def transform_python(self, value: Any) -> Any:\n return self._encoder(value)\n\n\ndef marshal_timestamp(timestamp: datetime | None, key: str) -> Mapping[str, Any]:\n if timestamp is None:\n return {key: None, key + \"_utcoffset\": None}\n\n return {\n key: timestamp.timestamp(),\n key + \"_utcoffset\": cast(timedelta, timestamp.utcoffset()).total_seconds()\n // 60,\n }\n\n\ndef marshal_document(document: dict[str, Any]) -> None:\n if \"id\" in document:\n document[\"_id\"] = document.pop(\"id\")\n\n for key, value in list(document.items()):\n if isinstance(value, datetime):\n document.update(marshal_timestamp(document[key], key))\n\n\ndef unmarshal_timestamps(document: dict[str, Any]) -> None:\n for key in list(document):\n if key.endswith(\"_utcoffset\") and (value := document.pop(key)) is not None:\n offset = timedelta(seconds=value * 60)\n tzinfo = timezone(offset)\n time_micro = document[key[:-10]]\n document[key[:-10]] = datetime.fromtimestamp(time_micro, tzinfo)\n\n\n@attrs.define(eq=False, repr=False)\nclass MongoDBDataStore(BaseExternalDataStore):\n \"\"\"\n Uses a MongoDB server to store data.\n\n When started, this data store creates the appropriate indexes on the given database\n if they're not already present.\n\n Operations are retried (in accordance to ``retry_settings``) when an operation\n raises :exc:`pymongo.errors.ConnectionFailure`.\n\n :param client_or_uri: an asynchronous PyMongo client or a MongoDB connection URI\n :param database: name of the database to use\n\n .. note:: The data store will not manage the life cycle of any client instance\n passed to it, so you need to close the client after you're done with it.\n\n .. note:: Datetimes are stored as integers along with their UTC offsets instead of\n BSON datetimes due to the BSON datetimes only being accurate to the millisecond\n while Python datetimes are accurate to the microsecond.\n \"\"\"\n\n client_or_uri: AsyncMongoClient | str = attrs.field(\n validator=instance_of((AsyncMongoClient, str))\n )\n database: str = attrs.field(\n default=\"apscheduler\", kw_only=True, validator=instance_of(str)\n )\n\n _client: AsyncMongoClient = attrs.field(init=False)\n _close_on_exit: bool = attrs.field(init=False, default=False)\n _task_attrs: ClassVar[list[str]] = [field.name for field in attrs.fields(Task)]\n _schedule_attrs: ClassVar[list[str]] = [\n field.name for field in attrs.fields(Schedule)\n ]\n _job_attrs: ClassVar[list[str]] = [field.name for field in attrs.fields(Job)]\n _tasks: AsyncCollection = attrs.field(init=False)\n _schedules: AsyncCollection = attrs.field(init=False)\n _jobs: AsyncCollection = attrs.field(init=False)\n _jobs_results: AsyncCollection = attrs.field(init=False)\n\n @property\n def _temporary_failure_exceptions(self) -> tuple[type[Exception], ...]:\n return (ConnectionFailure,)\n\n def __attrs_post_init__(self) -> None:\n if isinstance(self.client_or_uri, str):\n self._client = AsyncMongoClient(self.client_or_uri)\n self._close_on_exit = True\n else:\n self._client = self.client_or_uri\n\n type_registry = TypeRegistry(\n [\n CustomEncoder(timedelta, timedelta.total_seconds),\n CustomEncoder(ConflictPolicy, operator.attrgetter(\"name\")),\n CustomEncoder(CoalescePolicy, operator.attrgetter(\"name\")),\n CustomEncoder(JobOutcome, operator.attrgetter(\"name\")),\n ]\n )\n codec_options: CodecOptions = CodecOptions(\n type_registry=type_registry,\n uuid_representation=UuidRepresentation.STANDARD,\n )\n database = self._client.get_database(self.database, codec_options=codec_options)\n self._tasks = database[\"tasks\"]\n self._schedules = database[\"schedules\"]\n self._jobs = database[\"jobs\"]\n self._jobs_results = database[\"job_results\"]\n\n def __repr__(self) -> str:\n server_descriptions = self._client.topology_description.server_descriptions()\n return create_repr(self, host=list(server_descriptions))\n\n async def _initialize(self) -> None:\n async with self._client.start_session() as session:\n if self.start_from_scratch:\n await self._tasks.delete_many({}, session=session)\n await self._schedules.delete_many({}, session=session)\n await self._jobs.delete_many({}, session=session)\n await self._jobs_results.delete_many({}, session=session)\n\n await self._schedules.create_index(\"task_id\", session=session)\n await self._schedules.create_index(\"next_fire_time\", session=session)\n await self._schedules.create_index(\"acquired_by\", session=session)\n await self._jobs.create_index(\"task_id\", session=session)\n await self._jobs.create_index(\"schedule_id\", session=session)\n await self._jobs.create_index(\"created_at\", session=session)\n await self._jobs.create_index(\"acquired_by\", session=session)\n await self._jobs_results.create_index(\"expires_at\", session=session)\n\n async def start(\n self, exit_stack: AsyncExitStack, event_broker: EventBroker, logger: Logger\n ) -> None:\n if self._close_on_exit:\n exit_stack.push_async_callback(self._client.close)\n\n await super().start(exit_stack, event_broker, logger)\n server_info = await self._client.server_info()\n if server_info[\"versionArray\"] < [4, 0]:\n raise RuntimeError(\n f\"MongoDB server must be at least v4.0; current version = \"\n f\"{server_info['version']}\"\n )\n\n async for attempt in self._retry():\n with attempt:\n await self._initialize()\n\n async def add_task(self, task: Task) -> None:\n async for attempt in self._retry():\n with attempt:\n previous = await self._tasks.find_one_and_update(\n {\"_id\": task.id},\n {\"$set\": task.marshal(self.serializer)},\n upsert=True,\n )\n\n if previous:\n await self._event_broker.publish(TaskUpdated(task_id=task.id))\n else:\n await self._event_broker.publish(TaskAdded(task_id=task.id))\n\n async def remove_task(self, task_id: str) -> None:\n async for attempt in self._retry():\n with attempt:\n if not await self._tasks.find_one_and_delete({\"_id\": task_id}):\n raise TaskLookupError(task_id)\n\n await self._event_broker.publish(TaskRemoved(task_id=task_id))\n\n async def get_task(self, task_id: str) -> Task:\n async for attempt in self._retry():\n with attempt:\n document = await self._tasks.find_one(\n {\"_id\": task_id}, projection=self._task_attrs\n )\n\n if not document:\n raise TaskLookupError(task_id)\n\n document[\"id\"] = document.pop(\"_id\")\n task = Task.unmarshal(self.serializer, document)\n return task\n\n async def get_tasks(self) -> list[Task]:\n async for attempt in self._retry():\n with attempt:\n tasks: list[Task] = []\n async with self._tasks.find(\n projection=self._task_attrs, sort=[(\"_id\", pymongo.ASCENDING)]\n ) as cursor:\n async for document in cursor:\n document[\"id\"] = document.pop(\"_id\")\n tasks.append(Task.unmarshal(self.serializer, document))\n\n return tasks\n\n async def get_schedules(self, ids: set[str] | None = None) -> list[Schedule]:\n filters = {\"_id\": {\"$in\": list(ids)}} if ids is not None else {}\n async for attempt in self._retry():\n with attempt:\n schedules: list[Schedule] = []\n async with self._schedules.find(filters).sort(\"_id\") as cursor:\n async for document in cursor:\n document[\"id\"] = document.pop(\"_id\")\n unmarshal_timestamps(document)\n try:\n schedule = Schedule.unmarshal(self.serializer, document)\n except DeserializationError:\n self._logger.warning(\n \"Failed to deserialize schedule %r\", document[\"_id\"]\n )\n continue\n\n schedules.append(schedule)\n\n return schedules\n\n async def add_schedule(\n self, schedule: Schedule, conflict_policy: ConflictPolicy\n ) -> None:\n event: DataStoreEvent\n document = schedule.marshal(self.serializer)\n marshal_document(document)\n try:\n async for attempt in self._retry():\n with attempt:\n await self._schedules.insert_one(document)\n except DuplicateKeyError:\n if conflict_policy is ConflictPolicy.exception:\n raise ConflictingIdError(schedule.id) from None\n elif conflict_policy is ConflictPolicy.replace:\n async for attempt in self._retry():\n with attempt:\n await self._schedules.replace_one(\n {\"_id\": schedule.id},\n document,\n True,\n )\n\n event = ScheduleUpdated(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n await self._event_broker.publish(event)\n else:\n event = ScheduleAdded(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n await self._event_broker.publish(event)\n\n async def remove_schedules(self, ids: Iterable[str]) -> None:\n filters = {\"_id\": {\"$in\": list(ids)}} if ids is not None else {}\n async for attempt in self._retry():\n with attempt:\n async with (\n self._client.start_session() as session,\n self._schedules.find(\n filters, projection=[\"_id\", \"task_id\"], session=session\n ) as cursor,\n ):\n new_ids = [(doc[\"_id\"], doc[\"task_id\"]) async for doc in cursor]\n if new_ids:\n await self._schedules.delete_many(filters, session=session)\n\n for schedule_id, task_id in new_ids:\n await self._event_broker.publish(\n ScheduleRemoved(\n schedule_id=schedule_id, task_id=task_id, finished=False\n )\n )\n\n async def acquire_schedules(\n self, scheduler_id: str, lease_duration: timedelta, limit: int\n ) -> list[Schedule]:\n schedules: list[Schedule] = []\n\n # Fetch up to {limit} schedules\n while len(schedules) < limit:\n async for attempt in self._retry():\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(attempt)\n session = await exit_stack.enter_async_context(\n self._client.start_session()\n )\n now = datetime.now(timezone.utc)\n cursor = await exit_stack.enter_async_context(\n self._schedules.find(\n {\n \"next_fire_time\": {\"$lte\": now.timestamp()},\n \"$and\": [\n {\n \"$or\": [\n {\"paused\": {\"$exists\": False}},\n {\"paused\": False},\n ]\n },\n {\n \"$or\": [\n {\"acquired_by\": scheduler_id},\n {\"acquired_until\": {\"$exists\": False}},\n {\n \"acquired_until\": {\n \"$lt\": now.timestamp()\n }\n },\n ]\n },\n ],\n },\n session=session,\n )\n .sort(\"next_fire_time\")\n .limit(limit - len(schedules))\n )\n documents = [doc async for doc in cursor]\n\n # Bail out if there are no more schedules to be acquired\n if not documents:\n return schedules\n\n now = datetime.now(timezone.utc)\n acquired_until = now + lease_duration\n schedule_ids = [doc[\"_id\"] for doc in documents]\n result = await self._schedules.update_many(\n {\n \"_id\": {\"$in\": schedule_ids},\n \"$or\": [\n {\"acquired_until\": {\"$exists\": False}},\n {\"acquired_until\": {\"$lt\": now.timestamp()}},\n ],\n },\n {\n \"$set\": {\n \"acquired_by\": scheduler_id,\n **marshal_timestamp(acquired_until, \"acquired_until\"),\n }\n },\n session=session,\n )\n\n # If the number of modified schedules was smaller than expected,\n # manually check which ones were successfully acquired\n if result.modified_count != len(schedule_ids):\n async with self._schedules.find(\n {\n \"_id\": {\"$in\": schedule_ids},\n \"acquired_by\": scheduler_id,\n },\n sort=[(\"created_at\", ASCENDING)],\n projection=[\"_id\"],\n session=session,\n ) as cursor:\n acquired_schedule_ids = {doc[\"_id\"] async for doc in cursor}\n documents = [\n doc\n for doc in documents\n if doc[\"_id\"] in acquired_schedule_ids\n ]\n\n for doc in documents:\n # Deserialize the schedule\n doc[\"id\"] = doc.pop(\"_id\")\n unmarshal_timestamps(doc)\n schedules.append(Schedule.unmarshal(self.serializer, doc))\n\n return schedules\n\n async def release_schedules(\n self, scheduler_id: str, results: Sequence[ScheduleResult]\n ) -> None:\n updated_schedules: list[tuple[str, datetime | None]] = []\n finished_schedule_ids: list[str] = []\n task_ids = {result.schedule_id: result.task_id for result in results}\n\n requests: list[UpdateOne | DeleteOne] = []\n for result in results:\n filters = {\"_id\": result.schedule_id, \"acquired_by\": scheduler_id}\n try:\n serialized_trigger = self.serializer.serialize(result.trigger)\n except SerializationError:\n self._logger.exception(\n \"Error serializing schedule %r – removing from data store\",\n result.schedule_id,\n )\n requests.append(DeleteOne(filters))\n finished_schedule_ids.append(result.schedule_id)\n continue\n\n update = {\n \"$unset\": {\n \"acquired_by\": True,\n \"acquired_until\": True,\n \"acquired_until_utcoffset\": True,\n },\n \"$set\": {\n \"trigger\": serialized_trigger,\n **marshal_timestamp(result.last_fire_time, \"last_fire_time\"),\n **marshal_timestamp(result.next_fire_time, \"next_fire_time\"),\n },\n }\n requests.append(UpdateOne(filters, update))\n updated_schedules.append((result.schedule_id, result.next_fire_time))\n\n if requests:\n async for attempt in self._retry():\n with attempt:\n async with self._client.start_session() as session:\n await self._schedules.bulk_write(\n requests, ordered=False, session=session\n )\n\n for schedule_id, next_fire_time in updated_schedules:\n event = ScheduleUpdated(\n schedule_id=schedule_id,\n task_id=task_ids[schedule_id],\n next_fire_time=next_fire_time,\n )\n await self._event_broker.publish(event)\n\n for schedule_id in finished_schedule_ids:\n await self._event_broker.publish(\n ScheduleRemoved(\n schedule_id=schedule_id,\n task_id=task_ids[schedule_id],\n finished=True,\n )\n )\n\n async def get_next_schedule_run_time(self) -> datetime | None:\n async for attempt in self._retry():\n with attempt:\n document = await self._schedules.find_one(\n {\"next_fire_time\": {\"$ne\": None}},\n projection=[\"next_fire_time\", \"next_fire_time_utcoffset\"],\n sort=[(\"next_fire_time\", ASCENDING)],\n )\n\n if document:\n unmarshal_timestamps(document)\n return document[\"next_fire_time\"]\n else:\n return None\n\n async def add_job(self, job: Job) -> None:\n document = job.marshal(self.serializer)\n marshal_document(document)\n async for attempt in self._retry():\n with attempt:\n await self._jobs.insert_one(document)\n\n event = JobAdded(\n job_id=job.id,\n task_id=job.task_id,\n schedule_id=job.schedule_id,\n )\n await self._event_broker.publish(event)\n\n async def get_jobs(self, ids: Iterable[UUID] | None = None) -> list[Job]:\n filters = {\"_id\": {\"$in\": list(ids)}} if ids is not None else {}\n async for attempt in self._retry():\n with attempt:\n jobs: list[Job] = []\n async with self._jobs.find(filters).sort(\"_id\") as cursor:\n async for document in cursor:\n document[\"id\"] = document.pop(\"_id\")\n unmarshal_timestamps(document)\n try:\n job = Job.unmarshal(self.serializer, document)\n except DeserializationError:\n self._logger.warning(\n \"Failed to deserialize job %r\", document[\"id\"]\n )\n continue\n\n jobs.append(job)\n\n return jobs\n\n async def acquire_jobs(\n self, scheduler_id: str, lease_duration: timedelta, limit: int | None = None\n ) -> list[Job]:\n events: list[JobAcquired | JobReleased] = []\n async for attempt in self._retry():\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(attempt)\n session = await exit_stack.enter_async_context(\n self._client.start_session()\n )\n\n # Fetch up to {limit} jobs\n now = datetime.now(timezone.utc)\n async with self._jobs.find(\n {\n \"$or\": [\n {\"acquired_until\": {\"$exists\": False}},\n {\"acquired_until\": {\"$lt\": now.timestamp()}},\n ]\n },\n sort=[(\"created_at\", ASCENDING)],\n limit=limit,\n session=session,\n ) as cursor:\n documents = [doc async for doc in cursor]\n\n # Mark them as acquired by this scheduler\n acquired_until = now + lease_duration\n job_ids = [doc[\"_id\"] for doc in documents]\n result = await self._jobs.update_many(\n {\n \"_id\": {\"$in\": job_ids},\n \"$or\": [\n {\"acquired_until\": {\"$exists\": False}},\n {\"acquired_until\": {\"$lt\": now.timestamp()}},\n ],\n },\n {\n \"$set\": {\n \"acquired_by\": scheduler_id,\n **marshal_timestamp(acquired_until, \"acquired_until\"),\n }\n },\n )\n\n # If the number of modified jobs was smaller than expected, manually\n # check which jobs were successfully acquired\n if result.modified_count != len(job_ids):\n async with self._jobs.find(\n {\n \"_id\": {\"$in\": job_ids},\n \"acquired_by\": scheduler_id,\n },\n sort=[(\"created_at\", ASCENDING)],\n projection=[\"_id\"],\n session=session,\n ) as cursor:\n acquired_job_ids = {doc[\"_id\"] async for doc in cursor}\n documents = [\n doc for doc in documents if doc[\"_id\"] in acquired_job_ids\n ]\n\n acquired_jobs: list[Job] = []\n skipped_job_ids: list[UUID] = []\n for doc in documents:\n # Deserialize the job\n doc[\"id\"] = doc.pop(\"_id\")\n unmarshal_timestamps(doc)\n try:\n job = Job.unmarshal(self.serializer, doc)\n except DeserializationError as exc:\n # Deserialization failed, so record the exception as the job\n # result\n result = JobResult(\n job_id=doc[\"id\"],\n outcome=JobOutcome.missed_start_deadline,\n finished_at=now,\n expires_at=now\n + timedelta(seconds=doc[\"result_expiration_time\"]),\n exception=exc,\n )\n events.append(\n await self._release_job(\n session,\n result,\n scheduler_id,\n doc[\"task_id\"],\n doc[\"schedule_id\"],\n doc[\"scheduled_fire_time\"],\n decrement_running_job_count=False,\n )\n )\n continue\n\n # Discard the job if its start deadline has passed\n if job.start_deadline and job.start_deadline < now:\n result = JobResult.from_job(\n job,\n JobOutcome.missed_start_deadline,\n finished_at=now,\n )\n events.append(\n await self._release_job(\n session,\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n decrement_running_job_count=False,\n )\n )\n continue\n\n # Try to increment the task's running jobs count\n update_task_result = await self._tasks.update_one(\n {\n \"_id\": job.task_id,\n \"$or\": [\n {\"max_running_jobs\": None},\n {\n \"$expr\": {\n \"$gt\": [\n \"$max_running_jobs\",\n \"$running_jobs\",\n ]\n }\n },\n ],\n },\n {\"$inc\": {\"running_jobs\": 1}},\n session=session,\n )\n if not update_task_result.matched_count:\n self._logger.debug(\n \"Skipping job %s because task %r has the maximum number of \"\n \"jobs already running\",\n job.id,\n job.task_id,\n )\n skipped_job_ids.append(job.id)\n continue\n\n job.acquired_by = scheduler_id\n job.acquired_until = now + lease_duration\n acquired_jobs.append(job)\n events.append(JobAcquired.from_job(job, scheduler_id=scheduler_id))\n\n # Release jobs skipped due to max job slots being reached\n if skipped_job_ids:\n await self._jobs.update_many(\n {\n \"_id\": {\"$in\": skipped_job_ids},\n \"acquired_by\": scheduler_id,\n },\n {\n \"$unset\": {\n \"acquired_by\": True,\n \"acquired_until\": True,\n \"acquired_until_utcoffset\": True,\n },\n },\n )\n\n # Publish the appropriate events\n for event in events:\n await self._event_broker.publish(event)\n\n return acquired_jobs\n\n async def _release_job(\n self,\n session: AsyncClientSession,\n result: JobResult,\n scheduler_id: str,\n task_id: str,\n schedule_id: str | None = None,\n scheduled_fire_time: datetime | None = None,\n *,\n decrement_running_job_count: bool = True,\n ) -> JobReleased:\n # Record the job result\n if result.expires_at > result.finished_at:\n document = result.marshal(self.serializer)\n document[\"_id\"] = document.pop(\"job_id\")\n marshal_document(document)\n await self._jobs_results.insert_one(document, session=session)\n\n # Delete the job\n await self._jobs.delete_one({\"_id\": result.job_id}, session=session)\n\n # Decrement the running jobs counter if the job had been successfully acquired\n if decrement_running_job_count:\n await self._tasks.find_one_and_update(\n {\"_id\": task_id},\n {\"$inc\": {\"running_jobs\": -1}},\n session=session,\n )\n\n # Notify other schedulers\n return JobReleased.from_result(\n result, scheduler_id, task_id, schedule_id, scheduled_fire_time\n )\n\n async def release_job(self, scheduler_id: str, job: Job, result: JobResult) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._client.start_session() as session:\n event = await self._release_job(\n session,\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n )\n\n # Notify other schedulers\n await self._event_broker.publish(event)\n\n async def get_job_result(self, job_id: UUID) -> JobResult | None:\n async for attempt in self._retry():\n with attempt:\n document = await self._jobs_results.find_one_and_delete({\"_id\": job_id})\n\n if document:\n document[\"job_id\"] = document.pop(\"_id\")\n unmarshal_timestamps(document)\n return JobResult.unmarshal(self.serializer, document)\n else:\n return None\n\n async def extend_acquired_schedule_leases(\n self, scheduler_id: str, schedule_ids: set[str], duration: timedelta\n ) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._client.start_session() as session:\n new_acquired_until = (\n datetime.now(timezone.utc) + duration\n ).timestamp()\n await self._schedules.update_many(\n filter={\n \"acquired_by\": scheduler_id,\n \"_id\": {\"$in\": list(schedule_ids)},\n },\n update={\"$set\": {\"acquired_until\": new_acquired_until}},\n session=session,\n )\n\n async def extend_acquired_job_leases(\n self, scheduler_id: str, job_ids: set[UUID], duration: timedelta\n ) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._client.start_session() as session:\n new_acquired_until = (\n datetime.now(timezone.utc) + duration\n ).timestamp()\n await self._jobs.update_many(\n filter={\n \"acquired_by\": scheduler_id,\n \"_id\": {\"$in\": list(job_ids)},\n },\n update={\"$set\": {\"acquired_until\": new_acquired_until}},\n session=session,\n )\n\n async def reap_abandoned_jobs(self, scheduler_id: str) -> None:\n async for attempt in self._retry():\n events: list[JobReleased] = []\n with attempt:\n async with (\n self._client.start_session() as session,\n self._jobs.find(\n filter={\"acquired_by\": scheduler_id},\n sort=[(\"created_at\", ASCENDING)],\n ) as cursor,\n ):\n async for doc in cursor:\n doc[\"id\"] = doc.pop(\"_id\")\n unmarshal_timestamps(doc)\n job = Job.unmarshal(\n self.serializer, {**doc, \"args\": (), \"kwargs\": {}}\n )\n result = JobResult.from_job(job, JobOutcome.abandoned)\n event = await self._release_job(\n session,\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n )\n events.append(event)\n\n for event in events:\n await self._event_broker.publish(event)\n\n async def cleanup(self) -> None:\n events: list[JobReleased | ScheduleRemoved] = []\n async for attempt in self._retry():\n with attempt:\n async with self._client.start_session() as session:\n # Purge expired job results\n now = datetime.now(timezone.utc)\n await self._jobs_results.delete_many(\n {\"expires_at\": {\"$lte\": now.timestamp()}}, session=session\n )\n\n # Find finished schedules\n async with self._schedules.find(\n {\"next_fire_time\": None},\n projection=[\"_id\", \"task_id\"],\n session=session,\n ) as cursor:\n if finished_schedules := {\n item[\"_id\"]: item[\"task_id\"] async for item in cursor\n }:\n # Find distinct schedule IDs of jobs associated with these\n # schedules\n for schedule_id in await self._jobs.distinct(\n \"schedule_id\",\n {\"schedule_id\": {\"$in\": list(finished_schedules)}},\n session=session,\n ):\n finished_schedules.pop(schedule_id)\n\n # Finish any jobs whose leases have expired\n filters = {\"acquired_until\": {\"$lt\": now.timestamp()}}\n async with self._jobs.find(\n filters,\n projection=[\n \"_id\",\n \"acquired_by\",\n \"task_id\",\n \"schedule_id\",\n \"scheduled_fire_time\",\n \"scheduled_fire_time_utcoffset\",\n \"result_expiration_time\",\n ],\n ) as cursor:\n async for doc in cursor:\n unmarshal_timestamps(doc)\n result = JobResult(\n job_id=doc[\"_id\"],\n outcome=JobOutcome.abandoned,\n finished_at=now,\n expires_at=now\n + timedelta(seconds=doc[\"result_expiration_time\"]),\n )\n events.append(\n await self._release_job(\n session,\n result,\n doc[\"acquired_by\"],\n doc[\"task_id\"],\n doc[\"schedule_id\"],\n doc[\"scheduled_fire_time\"],\n )\n )\n\n # Delete finished schedules that not having any associated jobs\n if finished_schedules:\n await self._schedules.delete_many(\n {\"_id\": {\"$in\": list(finished_schedules)}},\n session=session,\n )\n for schedule_id, task_id in finished_schedules.items():\n events.append(\n ScheduleRemoved(\n schedule_id=schedule_id,\n task_id=task_id,\n finished=True,\n )\n )\n\n # Publish any events produced from the operations\n for event in events:\n await self._event_broker.publish(event)\n" }, { "path": "src/apscheduler/datastores/sqlalchemy.py", "content": "from __future__ import annotations\n\nfrom collections import defaultdict\nfrom collections.abc import AsyncGenerator, Iterable, Mapping, Sequence\nfrom contextlib import AsyncExitStack, asynccontextmanager\nfrom datetime import datetime, timedelta, timezone\nfrom functools import partial\nfrom logging import Logger\nfrom typing import Any, cast\nfrom uuid import UUID\n\nimport anyio\nimport attrs\nimport tenacity\nfrom anyio import CancelScope, to_thread\nfrom attr.validators import instance_of\nfrom sqlalchemy import (\n JSON,\n BigInteger,\n Boolean,\n Column,\n DateTime,\n Enum,\n Integer,\n Interval,\n LargeBinary,\n MetaData,\n SmallInteger,\n Table,\n TypeDecorator,\n Unicode,\n Uuid,\n and_,\n bindparam,\n create_engine,\n false,\n or_,\n select,\n)\nfrom sqlalchemy.engine import URL, Dialect, Result\nfrom sqlalchemy.exc import (\n CompileError,\n IntegrityError,\n InterfaceError,\n InvalidRequestError,\n ProgrammingError,\n)\nfrom sqlalchemy.ext.asyncio import AsyncConnection, AsyncEngine, create_async_engine\nfrom sqlalchemy.future import Connection, Engine\nfrom sqlalchemy.schema import CreateSchema\nfrom sqlalchemy.sql import Executable\nfrom sqlalchemy.sql.ddl import DropTable\nfrom sqlalchemy.sql.elements import BindParameter, literal\nfrom sqlalchemy.sql.type_api import TypeEngine\n\nfrom .._enums import CoalescePolicy, ConflictPolicy, JobOutcome\nfrom .._events import (\n DataStoreEvent,\n Event,\n JobAcquired,\n JobAdded,\n JobDeserializationFailed,\n JobReleased,\n ScheduleAdded,\n ScheduleDeserializationFailed,\n ScheduleRemoved,\n ScheduleUpdated,\n TaskAdded,\n TaskRemoved,\n TaskUpdated,\n)\nfrom .._exceptions import (\n ConflictingIdError,\n DeserializationError,\n SerializationError,\n TaskLookupError,\n)\nfrom .._structures import Job, JobResult, Schedule, ScheduleResult, Task\nfrom .._utils import create_repr, current_async_library\nfrom ..abc import EventBroker\nfrom .base import BaseExternalDataStore\n\n\nclass EmulatedTimestampTZ(TypeDecorator[datetime]):\n impl = Unicode(32)\n cache_ok = True\n\n def process_bind_param(\n self, value: datetime | None, dialect: Dialect\n ) -> str | None:\n return value.isoformat() if value is not None else None\n\n def process_result_value(\n self, value: str | None, dialect: Dialect\n ) -> datetime | None:\n return datetime.fromisoformat(value) if value is not None else None\n\n\nclass EmulatedInterval(TypeDecorator[timedelta]):\n impl = BigInteger()\n cache_ok = True\n\n def process_bind_param(\n self, value: timedelta | None, dialect: Dialect\n ) -> float | None:\n return value.total_seconds() * 1000000 if value is not None else None\n\n def process_result_value(\n self, value: int | None, dialect: Dialect\n ) -> timedelta | None:\n return timedelta(seconds=value / 1000000) if value is not None else None\n\n\ndef marshal_timestamp(timestamp: datetime | None, key: str) -> Mapping[str, Any]:\n if timestamp is None:\n return {key: None, key + \"_utcoffset\": None}\n\n return {\n key: int(timestamp.timestamp() * 1000_000),\n key + \"_utcoffset\": cast(timedelta, timestamp.utcoffset()).total_seconds()\n // 60,\n }\n\n\n@attrs.define(eq=False, frozen=True)\nclass _JobDiscard:\n job_id: UUID\n outcome: JobOutcome\n task_id: str\n schedule_id: str | None\n scheduled_fire_time: datetime | None\n result_expires_at: datetime\n exception: Exception | None = None\n\n\n@attrs.define(eq=False, repr=False)\nclass SQLAlchemyDataStore(BaseExternalDataStore):\n \"\"\"\n Uses a relational database to store data.\n\n When started, this data store creates the appropriate tables on the given database\n if they're not already present.\n\n Operations are retried (in accordance to ``retry_settings``) when an operation\n raises either :exc:`OSError` or :exc:`sqlalchemy.exc.InterfaceError`.\n\n This store has been tested to work with:\n\n * PostgreSQL (asyncpg and psycopg drivers)\n * MySQL (asyncmy driver)\n * aiosqlite (not recommended right now, as issues like\n `#1032 `_ exist)\n\n :param engine_or_url: a SQLAlchemy URL or engine (preferably asynchronous, but can\n be synchronous)\n :param schema: a database schema name to use, if not the default\n\n .. note:: The data store will not manage the life cycle of any engine instance\n passed to it, so you need to close the engine afterwards when you're done with\n it.\n\n .. warning:: Do not use SQLite when sharing the data store with multiple schedulers,\n as there is an unresolved issue with that\n (`#959 `_).\n \"\"\"\n\n engine_or_url: str | URL | Engine | AsyncEngine = attrs.field(\n validator=instance_of((str, URL, Engine, AsyncEngine))\n )\n schema: str | None = attrs.field(kw_only=True, default=None)\n\n _engine: Engine | AsyncEngine = attrs.field(init=False)\n _close_on_exit: bool = attrs.field(init=False, default=False)\n _supports_update_returning: bool = attrs.field(init=False, default=False)\n _supports_tzaware_timestamps: bool = attrs.field(init=False, default=False)\n _supports_native_interval: bool = attrs.field(init=False, default=False)\n _metadata: MetaData = attrs.field(init=False)\n _t_metadata: Table = attrs.field(init=False)\n _t_tasks: Table = attrs.field(init=False)\n _t_schedules: Table = attrs.field(init=False)\n _t_jobs: Table = attrs.field(init=False)\n _t_job_results: Table = attrs.field(init=False)\n\n def __attrs_post_init__(self) -> None:\n if isinstance(self.engine_or_url, (str, URL)):\n try:\n self._engine = create_async_engine(self.engine_or_url)\n except InvalidRequestError:\n self._engine = create_engine(self.engine_or_url)\n\n self._close_on_exit = True\n else:\n self._engine = self.engine_or_url\n\n # Generate the table definitions\n prefix = f\"{self.schema}.\" if self.schema else \"\"\n self._supports_tzaware_timestamps = self._engine.dialect.name in (\n \"postgresql\",\n \"oracle\",\n )\n self._supports_native_interval = self._engine.dialect.name == \"postgresql\"\n self._metadata = self.get_table_definitions()\n self._t_metadata = self._metadata.tables[prefix + \"metadata\"]\n self._t_tasks = self._metadata.tables[prefix + \"tasks\"]\n self._t_schedules = self._metadata.tables[prefix + \"schedules\"]\n self._t_jobs = self._metadata.tables[prefix + \"jobs\"]\n self._t_job_results = self._metadata.tables[prefix + \"job_results\"]\n\n def __repr__(self) -> str:\n return create_repr(self, url=repr(self._engine.url), schema=self.schema)\n\n def _retry(self) -> tenacity.AsyncRetrying:\n def after_attempt(retry_state: tenacity.RetryCallState) -> None:\n self._logger.warning(\n \"Temporary data store error (attempt %d): %s\",\n retry_state.attempt_number,\n retry_state.outcome.exception(),\n )\n\n # OSError is raised by asyncpg if it can't connect\n return tenacity.AsyncRetrying(\n stop=self.retry_settings.stop,\n wait=self.retry_settings.wait,\n retry=tenacity.retry_if_exception_type((InterfaceError, OSError)),\n after=after_attempt,\n sleep=anyio.sleep,\n reraise=True,\n )\n\n @asynccontextmanager\n async def _begin_transaction(\n self,\n ) -> AsyncGenerator[Connection | AsyncConnection, None]:\n # A shielded cancel scope is injected to the exit stack to allow finalization\n # to occur even when the surrounding cancel scope is cancelled\n async with AsyncExitStack() as exit_stack:\n if isinstance(self._engine, AsyncEngine):\n async_cm = self._engine.begin()\n conn = await async_cm.__aenter__()\n exit_stack.enter_context(CancelScope(shield=True))\n exit_stack.push_async_exit(async_cm.__aexit__)\n else:\n cm = self._engine.begin()\n conn = await to_thread.run_sync(cm.__enter__)\n exit_stack.enter_context(CancelScope(shield=True))\n exit_stack.push_async_exit(partial(to_thread.run_sync, cm.__exit__))\n\n yield conn\n\n async def _create_metadata(self, conn: Connection | AsyncConnection) -> None:\n if isinstance(conn, AsyncConnection):\n await conn.run_sync(self._metadata.create_all)\n else:\n await to_thread.run_sync(self._metadata.create_all, conn)\n\n async def _execute(\n self,\n conn: Connection | AsyncConnection,\n statement: Executable,\n parameters: Sequence | Mapping | None = None,\n ):\n if isinstance(conn, AsyncConnection):\n return await conn.execute(statement, parameters)\n else:\n return await to_thread.run_sync(conn.execute, statement, parameters)\n\n @property\n def _temporary_failure_exceptions(self) -> tuple[type[Exception], ...]:\n # SQlite does not use the network, so it doesn't have \"temporary\" failures\n if self._engine.dialect.name == \"sqlite\":\n return ()\n\n return InterfaceError, OSError\n\n def _convert_incoming_fire_times(self, data: dict[str, Any]) -> dict[str, Any]:\n for field in (\"last_fire_time\", \"next_fire_time\"):\n if not self._supports_tzaware_timestamps:\n utcoffset_minutes = data.pop(f\"{field}_utcoffset\", None)\n if utcoffset_minutes is not None:\n tz = timezone(timedelta(minutes=utcoffset_minutes))\n timestamp = data[field] / 1000_000\n data[field] = datetime.fromtimestamp(timestamp, tz=tz)\n\n return data\n\n def _convert_outgoing_fire_times(self, data: dict[str, Any]) -> dict[str, Any]:\n for field in (\"last_fire_time\", \"next_fire_time\"):\n if not self._supports_tzaware_timestamps:\n field_value = data[field]\n if field_value is not None:\n data[field] = int(field_value.timestamp() * 1000_000)\n data[f\"{field}_utcoffset\"] = (\n field_value.utcoffset().total_seconds() // 60\n )\n else:\n data[f\"{field}_utcoffset\"] = None\n\n return data\n\n def get_table_definitions(self) -> MetaData:\n if self._supports_tzaware_timestamps:\n timestamp_type: TypeEngine[datetime] = DateTime(timezone=True)\n last_fire_time_tzoffset_columns: tuple[Column, ...] = (\n Column(\"last_fire_time\", timestamp_type),\n )\n next_fire_time_tzoffset_columns: tuple[Column, ...] = (\n Column(\"next_fire_time\", timestamp_type, index=True),\n )\n else:\n timestamp_type = EmulatedTimestampTZ()\n last_fire_time_tzoffset_columns = (\n Column(\"last_fire_time\", BigInteger),\n Column(\"last_fire_time_utcoffset\", SmallInteger),\n )\n next_fire_time_tzoffset_columns = (\n Column(\"next_fire_time\", BigInteger, index=True),\n Column(\"next_fire_time_utcoffset\", SmallInteger),\n )\n\n if self._supports_native_interval:\n interval_type: TypeDecorator[timedelta] = Interval(second_precision=6)\n else:\n interval_type = EmulatedInterval()\n\n if self._engine.dialect.name == \"postgresql\":\n from sqlalchemy.dialects.postgresql import JSONB\n\n json_type = JSONB\n else:\n json_type = JSON\n\n metadata = MetaData(schema=self.schema)\n Table(\"metadata\", metadata, Column(\"schema_version\", Integer, nullable=False))\n Table(\n \"tasks\",\n metadata,\n Column(\"id\", Unicode(500), primary_key=True),\n Column(\"func\", Unicode(500)),\n Column(\"job_executor\", Unicode(500), nullable=False),\n Column(\"max_running_jobs\", Integer),\n Column(\"misfire_grace_time\", interval_type),\n Column(\"metadata\", json_type, nullable=False),\n Column(\"running_jobs\", Integer, nullable=False, server_default=literal(0)),\n )\n Table(\n \"schedules\",\n metadata,\n Column(\"id\", Unicode(500), primary_key=True),\n Column(\"task_id\", Unicode(500), nullable=False, index=True),\n Column(\"trigger\", LargeBinary),\n Column(\"args\", LargeBinary),\n Column(\"kwargs\", LargeBinary),\n Column(\"paused\", Boolean, nullable=False, server_default=literal(False)),\n Column(\"coalesce\", Enum(CoalescePolicy, metadata=metadata), nullable=False),\n Column(\"misfire_grace_time\", interval_type),\n Column(\"max_jitter\", interval_type),\n Column(\"job_executor\", Unicode(500), nullable=False),\n Column(\"job_result_expiration_time\", interval_type),\n Column(\"metadata\", json_type, nullable=False),\n *last_fire_time_tzoffset_columns,\n *next_fire_time_tzoffset_columns,\n Column(\"acquired_by\", Unicode(500), index=True),\n Column(\"acquired_until\", timestamp_type),\n )\n Table(\n \"jobs\",\n metadata,\n Column(\"id\", Uuid, primary_key=True),\n Column(\"task_id\", Unicode(500), nullable=False, index=True),\n Column(\"args\", LargeBinary, nullable=False),\n Column(\"kwargs\", LargeBinary, nullable=False),\n Column(\"schedule_id\", Unicode(500), index=True),\n Column(\"scheduled_fire_time\", timestamp_type),\n Column(\"executor\", Unicode(500), nullable=False),\n Column(\"jitter\", interval_type),\n Column(\"start_deadline\", timestamp_type),\n Column(\"result_expiration_time\", interval_type),\n Column(\"metadata\", json_type, nullable=False),\n Column(\"created_at\", timestamp_type, nullable=False, index=True),\n Column(\"acquired_by\", Unicode(500), index=True),\n Column(\"acquired_until\", timestamp_type),\n )\n Table(\n \"job_results\",\n metadata,\n Column(\"job_id\", Uuid, primary_key=True),\n Column(\"outcome\", Enum(JobOutcome, metadata=metadata), nullable=False),\n Column(\"started_at\", timestamp_type, index=True),\n Column(\"finished_at\", timestamp_type, nullable=False),\n Column(\"expires_at\", timestamp_type, nullable=False, index=True),\n Column(\"exception\", LargeBinary),\n Column(\"return_value\", LargeBinary),\n )\n return metadata\n\n async def start(\n self, exit_stack: AsyncExitStack, event_broker: EventBroker, logger: Logger\n ) -> None:\n if (asynclib := current_async_library()) != \"asyncio\":\n raise RuntimeError(\n f\"This data store requires asyncio; currently running: {asynclib}\"\n )\n\n if self._close_on_exit:\n if isinstance(self._engine, AsyncEngine):\n exit_stack.push_async_callback(self._engine.dispose)\n else:\n exit_stack.callback(self._engine.dispose)\n\n await super().start(exit_stack, event_broker, logger)\n\n # Verify that the schema is in place\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n # Create the schema first if it doesn't exist yet\n if self.schema:\n await self._execute(\n conn, CreateSchema(name=self.schema, if_not_exists=True)\n )\n\n if self.start_from_scratch:\n for table in self._metadata.sorted_tables:\n await self._execute(conn, DropTable(table, if_exists=True))\n\n await self._create_metadata(conn)\n query = select(self._t_metadata.c.schema_version)\n result = await self._execute(conn, query)\n version = result.scalar()\n if version is None:\n await self._execute(\n conn, self._t_metadata.insert(), {\"schema_version\": 1}\n )\n elif version > 1:\n raise RuntimeError(\n f\"Unexpected schema version ({version}); \"\n f\"only version 1 is supported by this version of \"\n f\"APScheduler\"\n )\n\n # Find out if the dialect supports UPDATE...RETURNING\n async for attempt in self._retry():\n with attempt:\n update = (\n self._t_metadata.update()\n .values(schema_version=self._t_metadata.c.schema_version)\n .returning(self._t_metadata.c.schema_version)\n )\n async with self._begin_transaction() as conn:\n try:\n await self._execute(conn, update)\n except (CompileError, ProgrammingError):\n pass # the support flag is False by default\n else:\n self._supports_update_returning = True\n\n async def _deserialize_schedules(self, result: Result) -> list[Schedule]:\n schedules: list[Schedule] = []\n for row in result:\n try:\n schedules.append(\n Schedule.unmarshal(\n self.serializer,\n self._convert_incoming_fire_times(row._asdict()),\n )\n )\n except SerializationError as exc:\n await self._event_broker.publish(\n ScheduleDeserializationFailed(schedule_id=row.id, exception=exc)\n )\n\n return schedules\n\n async def _deserialize_jobs(self, result: Result) -> list[Job]:\n jobs: list[Job] = []\n for row in result:\n try:\n jobs.append(Job.unmarshal(self.serializer, row._asdict()))\n except SerializationError as exc:\n await self._event_broker.publish(\n JobDeserializationFailed(job_id=row.id, exception=exc)\n )\n\n return jobs\n\n async def add_task(self, task: Task) -> None:\n insert = self._t_tasks.insert().values(\n id=task.id,\n func=task.func,\n job_executor=task.job_executor,\n max_running_jobs=task.max_running_jobs,\n misfire_grace_time=task.misfire_grace_time,\n metadata=task.metadata,\n )\n try:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n await self._execute(conn, insert)\n except IntegrityError:\n update = (\n self._t_tasks.update()\n .values(\n func=task.func,\n job_executor=task.job_executor,\n max_running_jobs=task.max_running_jobs,\n misfire_grace_time=task.misfire_grace_time,\n metadata=task.metadata,\n )\n .where(self._t_tasks.c.id == task.id)\n )\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n await self._execute(conn, update)\n\n await self._event_broker.publish(TaskUpdated(task_id=task.id))\n else:\n await self._event_broker.publish(TaskAdded(task_id=task.id))\n\n async def remove_task(self, task_id: str) -> None:\n delete = self._t_tasks.delete().where(self._t_tasks.c.id == task_id)\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, delete)\n if result.rowcount == 0:\n raise TaskLookupError(task_id)\n else:\n await self._event_broker.publish(TaskRemoved(task_id=task_id))\n\n async def get_task(self, task_id: str) -> Task:\n query = self._t_tasks.select().where(self._t_tasks.c.id == task_id)\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, query)\n row = result.first()\n\n if row:\n return Task.unmarshal(self.serializer, row._asdict())\n else:\n raise TaskLookupError(task_id)\n\n async def get_tasks(self) -> list[Task]:\n query = self._t_tasks.select().order_by(self._t_tasks.c.id)\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, query)\n tasks = [\n Task.unmarshal(self.serializer, row._asdict()) for row in result\n ]\n\n return tasks\n\n async def add_schedule(\n self, schedule: Schedule, conflict_policy: ConflictPolicy\n ) -> None:\n event: DataStoreEvent\n values = self._convert_outgoing_fire_times(schedule.marshal(self.serializer))\n insert = self._t_schedules.insert().values(**values)\n try:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n await self._execute(conn, insert)\n except IntegrityError:\n if conflict_policy is ConflictPolicy.exception:\n raise ConflictingIdError(schedule.id) from None\n elif conflict_policy is ConflictPolicy.replace:\n del values[\"id\"]\n update = (\n self._t_schedules.update()\n .where(self._t_schedules.c.id == schedule.id)\n .values(**values)\n )\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n await self._execute(conn, update)\n\n event = ScheduleUpdated(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n await self._event_broker.publish(event)\n else:\n event = ScheduleAdded(\n schedule_id=schedule.id,\n task_id=schedule.task_id,\n next_fire_time=schedule.next_fire_time,\n )\n await self._event_broker.publish(event)\n\n async def remove_schedules(self, ids: Iterable[str]) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n if self._supports_update_returning:\n delete_returning = (\n self._t_schedules.delete()\n .where(self._t_schedules.c.id.in_(ids))\n .returning(\n self._t_schedules.c.id, self._t_schedules.c.task_id\n )\n )\n removed_ids: list[tuple[str, str]] = [\n (row[0], row[1])\n for row in await self._execute(conn, delete_returning)\n ]\n else:\n query = select(\n self._t_schedules.c.id, self._t_schedules.c.task_id\n ).where(self._t_schedules.c.id.in_(ids))\n ids_to_remove: list[str] = []\n removed_ids = []\n for schedule_id, task_id in await self._execute(conn, query):\n ids_to_remove.append(schedule_id)\n removed_ids.append((schedule_id, task_id))\n\n delete = self._t_schedules.delete().where(\n self._t_schedules.c.id.in_(ids_to_remove)\n )\n await self._execute(conn, delete)\n\n for schedule_id, task_id in removed_ids:\n await self._event_broker.publish(\n ScheduleRemoved(\n schedule_id=schedule_id, task_id=task_id, finished=False\n )\n )\n\n async def get_schedules(self, ids: set[str] | None = None) -> list[Schedule]:\n query = self._t_schedules.select().order_by(self._t_schedules.c.id)\n if ids:\n query = query.where(self._t_schedules.c.id.in_(ids))\n\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, query)\n\n return await self._deserialize_schedules(result)\n\n async def acquire_schedules(\n self, scheduler_id: str, lease_duration: timedelta, limit: int\n ) -> list[Schedule]:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n now = datetime.now(timezone.utc)\n acquired_until = now + lease_duration\n if self._supports_tzaware_timestamps:\n comparison = self._t_schedules.c.next_fire_time <= now\n else:\n comparison = self._t_schedules.c.next_fire_time <= int(\n now.timestamp() * 1000_000\n )\n\n schedules_cte = (\n select(self._t_schedules.c.id)\n .where(\n and_(\n self._t_schedules.c.next_fire_time.isnot(None),\n comparison,\n self._t_schedules.c.paused == false(),\n or_(\n self._t_schedules.c.acquired_by == scheduler_id,\n self._t_schedules.c.acquired_until.is_(None),\n self._t_schedules.c.acquired_until < now,\n ),\n )\n )\n .order_by(self._t_schedules.c.next_fire_time)\n .limit(limit)\n .with_for_update(skip_locked=True)\n .cte()\n )\n subselect = select(schedules_cte.c.id)\n update = (\n self._t_schedules.update()\n .where(self._t_schedules.c.id.in_(subselect))\n .values(acquired_by=scheduler_id, acquired_until=acquired_until)\n )\n if self._supports_update_returning:\n update = update.returning(*self._t_schedules.columns)\n result = await self._execute(conn, update)\n else:\n await self._execute(conn, update)\n query = self._t_schedules.select().where(\n and_(self._t_schedules.c.acquired_by == scheduler_id)\n )\n result = await self._execute(conn, query)\n\n schedules = await self._deserialize_schedules(result)\n\n return schedules\n\n async def release_schedules(\n self, scheduler_id: str, results: Sequence[ScheduleResult]\n ) -> None:\n task_ids = {result.schedule_id: result.task_id for result in results}\n next_fire_times = {\n result.schedule_id: result.next_fire_time for result in results\n }\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n update_events: list[ScheduleUpdated] = []\n finished_schedule_ids: list[str] = []\n update_args: list[dict[str, Any]] = []\n for result in results:\n try:\n serialized_trigger = self.serializer.serialize(\n result.trigger\n )\n except SerializationError:\n self._logger.exception(\n \"Error serializing trigger for schedule %r – \"\n \"removing from data store\",\n result.schedule_id,\n )\n finished_schedule_ids.append(result.schedule_id)\n continue\n\n if self._supports_tzaware_timestamps:\n update_args.append(\n {\n \"p_id\": result.schedule_id,\n \"p_trigger\": serialized_trigger,\n \"p_last_fire_time\": result.last_fire_time,\n \"p_next_fire_time\": result.next_fire_time,\n }\n )\n else:\n update_args.append(\n {\n \"p_id\": result.schedule_id,\n \"p_trigger\": serialized_trigger,\n **marshal_timestamp(\n result.last_fire_time, \"p_last_fire_time\"\n ),\n **marshal_timestamp(\n result.next_fire_time, \"p_next_fire_time\"\n ),\n }\n )\n\n # Update schedules\n if update_args:\n extra_values: dict[str, BindParameter] = {}\n p_id: BindParameter = bindparam(\"p_id\")\n p_trigger: BindParameter = bindparam(\"p_trigger\")\n p_last_fire_time: BindParameter = bindparam(\"p_last_fire_time\")\n p_next_fire_time: BindParameter = bindparam(\"p_next_fire_time\")\n if not self._supports_tzaware_timestamps:\n extra_values[\"last_fire_time_utcoffset\"] = bindparam(\n \"p_last_fire_time_utcoffset\"\n )\n extra_values[\"next_fire_time_utcoffset\"] = bindparam(\n \"p_next_fire_time_utcoffset\"\n )\n\n update = (\n self._t_schedules.update()\n .where(\n and_(\n self._t_schedules.c.id == p_id,\n self._t_schedules.c.acquired_by == scheduler_id,\n )\n )\n .values(\n trigger=p_trigger,\n last_fire_time=p_last_fire_time,\n next_fire_time=p_next_fire_time,\n acquired_by=None,\n acquired_until=None,\n **extra_values,\n )\n )\n # TODO: actually check which rows were updated?\n await self._execute(conn, update, update_args)\n updated_ids = list(next_fire_times)\n\n for schedule_id in updated_ids:\n event = ScheduleUpdated(\n schedule_id=schedule_id,\n task_id=task_ids[schedule_id],\n next_fire_time=next_fire_times[schedule_id],\n )\n update_events.append(event)\n\n # Remove schedules that failed to serialize\n if finished_schedule_ids:\n delete = self._t_schedules.delete().where(\n self._t_schedules.c.id.in_(finished_schedule_ids)\n )\n await self._execute(conn, delete)\n\n for event in update_events:\n await self._event_broker.publish(event)\n\n for schedule_id in finished_schedule_ids:\n await self._event_broker.publish(\n ScheduleRemoved(\n schedule_id=schedule_id,\n task_id=task_ids[schedule_id],\n finished=True,\n )\n )\n\n async def get_next_schedule_run_time(self) -> datetime | None:\n columns = [self._t_schedules.c.next_fire_time]\n if not self._supports_tzaware_timestamps:\n columns.append(self._t_schedules.c.next_fire_time_utcoffset)\n\n statenent = (\n select(*columns)\n .where(\n self._t_schedules.c.next_fire_time.isnot(None),\n self._t_schedules.c.paused == false(),\n self._t_schedules.c.acquired_by.is_(None),\n )\n .order_by(self._t_schedules.c.next_fire_time)\n .limit(1)\n )\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, statenent)\n\n if not self._supports_tzaware_timestamps:\n if row := result.first():\n tz = timezone(timedelta(minutes=row[1]))\n return datetime.fromtimestamp(row[0] / 1000_000, tz=tz)\n else:\n return None\n\n return result.scalar()\n\n async def add_job(self, job: Job) -> None:\n marshalled = job.marshal(self.serializer)\n insert = self._t_jobs.insert().values(**marshalled)\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n await self._execute(conn, insert)\n\n event = JobAdded(\n job_id=job.id,\n task_id=job.task_id,\n schedule_id=job.schedule_id,\n )\n await self._event_broker.publish(event)\n\n async def get_jobs(self, ids: Iterable[UUID] | None = None) -> list[Job]:\n query = self._t_jobs.select().order_by(self._t_jobs.c.id)\n if ids:\n job_ids = list(ids)\n query = query.where(self._t_jobs.c.id.in_(job_ids))\n\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n result = await self._execute(conn, query)\n\n return await self._deserialize_jobs(result)\n\n async def acquire_jobs(\n self, scheduler_id: str, lease_duration: timedelta, limit: int | None = None\n ) -> list[Job]:\n events: list[JobAcquired | JobReleased] = []\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n now = datetime.now(timezone.utc)\n acquired_until = now + lease_duration\n query = (\n select(\n self._t_jobs,\n self._t_tasks.c.max_running_jobs,\n self._t_tasks.c.running_jobs,\n )\n .join(\n self._t_tasks, self._t_tasks.c.id == self._t_jobs.c.task_id\n )\n .where(\n or_(\n self._t_jobs.c.acquired_until.is_(None),\n self._t_jobs.c.acquired_until < now,\n )\n )\n .order_by(self._t_jobs.c.created_at)\n .with_for_update(\n skip_locked=True,\n of=[\n self._t_tasks.c.running_jobs,\n self._t_jobs.c.acquired_by,\n self._t_jobs.c.acquired_until,\n ],\n )\n .limit(limit)\n )\n\n result = await self._execute(conn, query)\n if not result:\n return []\n\n acquired_jobs: list[Job] = []\n discarded_jobs: list[_JobDiscard] = []\n task_job_slots_left: dict[str, float] = defaultdict(\n lambda: float(\"inf\")\n )\n running_job_count_increments: dict[str, int] = defaultdict(\n lambda: 0\n )\n for row in result:\n job_dict = row._asdict()\n task_max_running_jobs = job_dict.pop(\"max_running_jobs\")\n task_running_jobs = job_dict.pop(\"running_jobs\")\n if task_max_running_jobs is not None:\n task_job_slots_left.setdefault(\n row.task_id, task_max_running_jobs - task_running_jobs\n )\n\n # Deserialize the job\n try:\n job = Job.unmarshal(self.serializer, job_dict)\n except DeserializationError as exc:\n # Deserialization failed, so record the exception as the job\n # result\n discarded_jobs.append(\n _JobDiscard(\n job_id=row.id,\n outcome=JobOutcome.deserialization_failed,\n task_id=row.task_id,\n schedule_id=row.schedule_id,\n scheduled_fire_time=row.scheduled_fire_time,\n result_expires_at=now + row.result_expiration_time,\n exception=exc,\n )\n )\n continue\n\n # Discard the job if its start deadline has passed\n if job.start_deadline and job.start_deadline < now:\n discarded_jobs.append(\n _JobDiscard(\n job_id=row.id,\n outcome=JobOutcome.missed_start_deadline,\n task_id=row.task_id,\n schedule_id=row.schedule_id,\n scheduled_fire_time=row.scheduled_fire_time,\n result_expires_at=now + row.result_expiration_time,\n )\n )\n continue\n\n # Skip the job if no more slots are available\n if not task_job_slots_left[job.task_id]:\n self._logger.debug(\n \"Skipping job %s because task %r has the maximum \"\n \"number of %d jobs already running\",\n job.id,\n job.task_id,\n task_max_running_jobs,\n )\n continue\n\n task_job_slots_left[job.task_id] -= 1\n running_job_count_increments[job.task_id] += 1\n job.acquired_by = scheduler_id\n job.acquired_until = acquired_until\n acquired_jobs.append(job)\n events.append(\n JobAcquired.from_job(job, scheduler_id=scheduler_id)\n )\n\n if acquired_jobs:\n # Mark the acquired jobs as acquired by this worker\n acquired_job_ids = [job.id for job in acquired_jobs]\n update = (\n self._t_jobs.update()\n .values(\n acquired_by=scheduler_id, acquired_until=acquired_until\n )\n .where(self._t_jobs.c.id.in_(acquired_job_ids))\n )\n await self._execute(conn, update)\n\n # Increment the running job counters on each task\n p_id: BindParameter = bindparam(\"p_id\")\n p_increment: BindParameter = bindparam(\"p_increment\")\n params = [\n {\"p_id\": task_id, \"p_increment\": increment}\n for task_id, increment in running_job_count_increments.items()\n ]\n update = (\n self._t_tasks.update()\n .values(\n running_jobs=self._t_tasks.c.running_jobs + p_increment\n )\n .where(self._t_tasks.c.id == p_id)\n )\n await self._execute(conn, update, params)\n\n # Discard the jobs that could not start\n for discard in discarded_jobs:\n result = JobResult(\n job_id=discard.job_id,\n outcome=discard.outcome,\n finished_at=now,\n expires_at=discard.result_expires_at,\n exception=discard.exception,\n )\n events.append(\n await self._release_job(\n conn,\n result,\n scheduler_id,\n discard.task_id,\n discard.schedule_id,\n discard.scheduled_fire_time,\n decrement_running_job_count=False,\n )\n )\n\n # Publish the appropriate events\n for event in events:\n await self._event_broker.publish(event)\n\n return acquired_jobs\n\n async def _release_job(\n self,\n conn: Connection | AsyncConnection,\n result: JobResult,\n scheduler_id: str,\n task_id: str,\n schedule_id: str | None = None,\n scheduled_fire_time: datetime | None = None,\n *,\n decrement_running_job_count: bool = True,\n ) -> JobReleased:\n # Record the job result\n if result.expires_at > result.finished_at:\n marshalled = result.marshal(self.serializer)\n insert = self._t_job_results.insert().values(**marshalled)\n await self._execute(conn, insert)\n\n # Decrement the number of running jobs for this task\n if decrement_running_job_count:\n update = (\n self._t_tasks.update()\n .values(running_jobs=self._t_tasks.c.running_jobs - 1)\n .where(self._t_tasks.c.id == task_id)\n )\n await self._execute(conn, update)\n\n # Delete the job\n delete = self._t_jobs.delete().where(self._t_jobs.c.id == result.job_id)\n await self._execute(conn, delete)\n\n # Create the event, to be sent after commit\n return JobReleased.from_result(\n result, scheduler_id, task_id, schedule_id, scheduled_fire_time\n )\n\n async def release_job(self, scheduler_id: str, job: Job, result: JobResult) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n event = await self._release_job(\n conn,\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n )\n\n # Notify other schedulers\n await self._event_broker.publish(event)\n\n async def get_job_result(self, job_id: UUID) -> JobResult | None:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n # Retrieve the result\n query = self._t_job_results.select().where(\n self._t_job_results.c.job_id == job_id\n )\n if row := (await self._execute(conn, query)).one_or_none():\n # Delete the result\n delete = self._t_job_results.delete().where(\n self._t_job_results.c.job_id == job_id\n )\n await self._execute(conn, delete)\n\n return JobResult.unmarshal(self.serializer, row._asdict()) if row else None\n\n async def extend_acquired_schedule_leases(\n self, scheduler_id: str, schedule_ids: set[str], duration: timedelta\n ) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n new_acquired_until = datetime.now(timezone.utc) + duration\n update = (\n self._t_schedules.update()\n .values(acquired_until=new_acquired_until)\n .where(\n self._t_schedules.c.acquired_by == scheduler_id,\n self._t_schedules.c.id.in_(schedule_ids),\n )\n )\n await self._execute(conn, update)\n\n async def extend_acquired_job_leases(\n self, scheduler_id: str, job_ids: set[UUID], duration: timedelta\n ) -> None:\n async for attempt in self._retry():\n with attempt:\n async with self._begin_transaction() as conn:\n new_acquired_until = datetime.now(timezone.utc) + duration\n update = (\n self._t_jobs.update()\n .values(acquired_until=new_acquired_until)\n .where(\n self._t_jobs.c.acquired_by == scheduler_id,\n self._t_jobs.c.id.in_(job_ids),\n )\n )\n await self._execute(conn, update)\n\n async def reap_abandoned_jobs(self, scheduler_id: str) -> None:\n query = (\n select(self._t_jobs)\n .where(self._t_jobs.c.acquired_by == scheduler_id)\n .with_for_update()\n )\n async for attempt in self._retry():\n events: list[JobReleased] = []\n with attempt:\n async with self._begin_transaction() as conn:\n if results := await self._execute(conn, query):\n for row in results:\n job_dict = self._convert_incoming_fire_times(row._asdict())\n job = Job.unmarshal(\n self.serializer, {**job_dict, \"args\": (), \"kwargs\": {}}\n )\n result = JobResult.from_job(job, JobOutcome.abandoned)\n event = await self._release_job(\n conn,\n result,\n scheduler_id,\n job.task_id,\n job.schedule_id,\n job.scheduled_fire_time,\n )\n events.append(event)\n\n for event in events:\n await self._event_broker.publish(event)\n\n async def cleanup(self) -> None:\n async for attempt in self._retry():\n with attempt:\n events: list[Event] = []\n async with self._begin_transaction() as conn:\n # Purge expired job results\n delete = self._t_job_results.delete().where(\n self._t_job_results.c.expires_at <= datetime.now(timezone.utc)\n )\n await self._execute(conn, delete)\n\n # Finish any jobs whose leases have expired\n now = datetime.now(timezone.utc)\n query = select(\n self._t_jobs.c.id,\n self._t_jobs.c.task_id,\n self._t_jobs.c.schedule_id,\n self._t_jobs.c.scheduled_fire_time,\n self._t_jobs.c.acquired_by,\n self._t_jobs.c.result_expiration_time,\n ).where(\n self._t_jobs.c.acquired_by.isnot(None),\n self._t_jobs.c.acquired_until < now,\n )\n for row in await self._execute(conn, query):\n result = JobResult(\n job_id=row.id,\n outcome=JobOutcome.abandoned,\n finished_at=now,\n expires_at=now + row.result_expiration_time,\n )\n events.append(\n await self._release_job(\n conn,\n result,\n row.acquired_by,\n row.task_id,\n row.schedule_id,\n row.scheduled_fire_time,\n )\n )\n\n # Clean up finished schedules that have no running jobs\n query = (\n select(self._t_schedules.c.id, self._t_schedules.c.task_id)\n .outerjoin(\n self._t_jobs,\n self._t_jobs.c.schedule_id == self._t_schedules.c.id,\n )\n .where(\n self._t_schedules.c.next_fire_time.is_(None),\n self._t_jobs.c.id.is_(None),\n )\n )\n results = await self._execute(conn, query)\n if finished_schedule_ids := dict(results.all()):\n delete = self._t_schedules.delete().where(\n self._t_schedules.c.id.in_(finished_schedule_ids)\n )\n await self._execute(conn, delete)\n\n for schedule_id, task_id in finished_schedule_ids.items():\n events.append(\n ScheduleRemoved(\n schedule_id=schedule_id,\n task_id=task_id,\n finished=True,\n )\n )\n\n # Publish any events produced from the operations\n for event in events:\n await self._event_broker.publish(event)\n" }, { "path": "src/apscheduler/eventbrokers/__init__.py", "content": "" }, { "path": "src/apscheduler/eventbrokers/asyncpg.py", "content": "from __future__ import annotations\n\nfrom collections.abc import AsyncGenerator, Mapping\nfrom contextlib import AsyncExitStack, asynccontextmanager\nfrom logging import Logger\nfrom typing import TYPE_CHECKING, Any\n\nimport asyncpg\nimport attrs\nfrom anyio import (\n EndOfStream,\n create_memory_object_stream,\n move_on_after,\n)\nfrom anyio.abc import TaskStatus\nfrom anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream\nfrom asyncpg import Connection, InterfaceError\nfrom attr.validators import instance_of\n\nfrom .._events import Event\nfrom .._exceptions import SerializationError\nfrom .._utils import create_repr\nfrom .base import BaseExternalEventBroker\n\nif TYPE_CHECKING:\n from sqlalchemy.ext.asyncio import AsyncEngine\n\n\n@attrs.define(eq=False, repr=False)\nclass AsyncpgEventBroker(BaseExternalEventBroker):\n \"\"\"\n An asynchronous, asyncpg_ based event broker that uses a PostgreSQL server to\n broadcast events using its ``NOTIFY`` mechanism.\n\n .. _asyncpg: https://pypi.org/project/asyncpg/\n\n :param dsn: a libpq connection string (e.g.\n ``postgres://user:pass@host:port/dbname``)\n :param options: extra keyword arguments passed to :func:`asyncpg.connection.connect`\n :param channel: the ``NOTIFY`` channel to use\n :param max_idle_time: maximum time to let the connection go idle, before sending a\n ``SELECT 1`` query to prevent a connection timeout\n \"\"\"\n\n dsn: str\n options: Mapping[str, Any] = attrs.field(\n factory=dict, validator=instance_of(Mapping)\n )\n channel: str = attrs.field(kw_only=True, default=\"apscheduler\")\n max_idle_time: float = attrs.field(kw_only=True, default=10)\n\n _send: MemoryObjectSendStream[str] = attrs.field(init=False)\n\n @classmethod\n def from_async_sqla_engine(\n cls,\n engine: AsyncEngine,\n options: Mapping[str, Any] | None = None,\n **kwargs: Any,\n ) -> AsyncpgEventBroker:\n \"\"\"\n Create a new asyncpg event broker from an SQLAlchemy engine.\n\n The engine will only be used to create the appropriate options for\n :func:`asyncpg.connection.connect`.\n\n :param engine: an asynchronous SQLAlchemy engine using asyncpg as the driver\n :type engine: ~sqlalchemy.ext.asyncio.AsyncEngine\n :param options: extra keyword arguments passed to\n :func:`asyncpg.connection.connect`\n :param kwargs: keyword arguments to pass to the initializer of this class\n :return: the newly created event broker\n\n \"\"\"\n if engine.dialect.driver != \"asyncpg\":\n raise ValueError(\n f'The driver in the engine must be \"asyncpg\" (current: '\n f\"{engine.dialect.driver})\"\n )\n\n dsn = engine.url.render_as_string(hide_password=False).replace(\"+asyncpg\", \"\")\n return cls(dsn, options or {}, **kwargs)\n\n def __repr__(self) -> str:\n return create_repr(self, \"dsn\")\n\n @property\n def _temporary_failure_exceptions(self) -> tuple[type[Exception], ...]:\n return OSError, InterfaceError\n\n @asynccontextmanager\n async def _connect(self) -> AsyncGenerator[asyncpg.Connection, None]:\n async for attempt in self._retry():\n with attempt:\n conn = await asyncpg.connect(self.dsn, **self.options)\n try:\n yield conn\n finally:\n with move_on_after(5, shield=True):\n await conn.close(timeout=3)\n\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n await super().start(exit_stack, logger)\n self._send, receive = create_memory_object_stream[str](100)\n await exit_stack.enter_async_context(self._send)\n await self._task_group.start(self._listen_notifications, receive)\n\n async def _listen_notifications(\n self, receive: MemoryObjectReceiveStream[str], *, task_status: TaskStatus[None]\n ) -> None:\n conn: Connection\n\n def listen_callback(\n connection: Connection, pid: int, channel: str, payload: str\n ) -> None:\n event = self.reconstitute_event_str(payload)\n if event is not None:\n self._task_group.start_soon(self.publish_local, event)\n\n async def unsubscribe() -> None:\n if not conn.is_closed():\n with move_on_after(3, shield=True):\n await conn.remove_listener(self.channel, listen_callback)\n\n task_started_sent = False\n with receive:\n while True:\n async with AsyncExitStack() as exit_stack:\n conn = await exit_stack.enter_async_context(self._connect())\n self._logger.info(\"Connection established\")\n try:\n await conn.add_listener(self.channel, listen_callback)\n exit_stack.push_async_callback(unsubscribe)\n if not task_started_sent:\n task_status.started()\n task_started_sent = True\n\n while True:\n notification: str | None = None\n with move_on_after(self.max_idle_time):\n try:\n notification = await receive.receive()\n except EndOfStream:\n self._logger.info(\"Stream finished\")\n return\n\n if notification:\n await conn.execute(\n \"SELECT pg_notify($1, $2)\",\n self.channel,\n notification,\n )\n else:\n await conn.execute(\"SELECT 1\")\n except InterfaceError as exc:\n self._logger.error(\"Connection error: %s\", exc)\n\n async def publish(self, event: Event) -> None:\n notification = self.generate_notification_str(event)\n if len(notification) > 7999:\n raise SerializationError(\n \"Serialized event object exceeds 7999 bytes in size\"\n )\n\n await self._send.send(notification)\n" }, { "path": "src/apscheduler/eventbrokers/base.py", "content": "from __future__ import annotations\n\nfrom base64 import b64decode, b64encode\nfrom collections.abc import Callable, Iterable\nfrom contextlib import AsyncExitStack\nfrom inspect import iscoroutine\nfrom logging import Logger\nfrom typing import Any\n\nimport attrs\nfrom anyio import CapacityLimiter, create_task_group, to_thread\nfrom anyio.abc import TaskGroup\n\nfrom .. import _events\nfrom .._events import Event\nfrom .._exceptions import DeserializationError\nfrom .._retry import RetryMixin\nfrom ..abc import EventBroker, Serializer, Subscription\nfrom ..serializers.json import JSONSerializer\n\n\n@attrs.define(eq=False, frozen=True)\nclass LocalSubscription(Subscription):\n callback: Callable[[Event], Any]\n event_types: set[type[Event]] | None\n one_shot: bool\n is_async: bool\n token: object\n _source: BaseEventBroker\n\n def unsubscribe(self) -> None:\n self._source.unsubscribe(self.token)\n\n\n@attrs.define(kw_only=True)\nclass BaseEventBroker(EventBroker):\n _logger: Logger = attrs.field(init=False)\n _subscriptions: dict[object, LocalSubscription] = attrs.field(\n init=False, factory=dict\n )\n _task_group: TaskGroup = attrs.field(init=False)\n _thread_limiter: CapacityLimiter = attrs.field(init=False)\n\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n self._logger = logger\n self._task_group = await exit_stack.enter_async_context(create_task_group())\n self._thread_limiter = CapacityLimiter(1)\n\n def subscribe(\n self,\n callback: Callable[[Event], Any],\n event_types: Iterable[type[Event]] | None = None,\n *,\n is_async: bool = True,\n one_shot: bool = False,\n ) -> Subscription:\n types = set(event_types) if event_types else None\n token = object()\n subscription = LocalSubscription(\n callback, types, one_shot, is_async, token, self\n )\n self._subscriptions[token] = subscription\n return subscription\n\n def unsubscribe(self, token: object) -> None:\n self._subscriptions.pop(token, None)\n\n async def publish_local(self, event: Event) -> None:\n event_type = type(event)\n one_shot_tokens: list[object] = []\n for subscription in self._subscriptions.values():\n if (\n subscription.event_types is None\n or event_type in subscription.event_types\n ):\n self._task_group.start_soon(self._deliver_event, subscription, event)\n if subscription.one_shot:\n one_shot_tokens.append(subscription.token)\n\n for token in one_shot_tokens:\n self.unsubscribe(token)\n\n async def _deliver_event(\n self, subscription: LocalSubscription, event: Event\n ) -> None:\n try:\n if subscription.is_async:\n retval = subscription.callback(event)\n if iscoroutine(retval):\n await retval\n else:\n await to_thread.run_sync(\n subscription.callback, event, limiter=self._thread_limiter\n )\n except Exception:\n self._logger.exception(\n \"Error delivering %s event\", event.__class__.__name__\n )\n\n\n@attrs.define(kw_only=True)\nclass BaseExternalEventBroker(BaseEventBroker, RetryMixin):\n \"\"\"\n Base class for event brokers that use an external service.\n\n :param serializer: the serializer used to (de)serialize events for transport\n \"\"\"\n\n serializer: Serializer = attrs.field(factory=JSONSerializer)\n\n def generate_notification(self, event: Event) -> bytes:\n serialized = self.serializer.serialize(event.marshal())\n return event.__class__.__name__.encode(\"ascii\") + b\" \" + serialized\n\n def generate_notification_str(self, event: Event) -> str:\n serialized = self.serializer.serialize(event.marshal())\n return event.__class__.__name__ + \" \" + b64encode(serialized).decode(\"ascii\")\n\n def _reconstitute_event(self, event_type: str, serialized: bytes) -> Event | None:\n try:\n kwargs = self.serializer.deserialize(serialized)\n except DeserializationError:\n self._logger.exception(\n \"Failed to deserialize an event of type %s\",\n event_type,\n extra={\"serialized\": serialized},\n )\n return None\n\n try:\n event_class = getattr(_events, event_type)\n except AttributeError:\n self._logger.error(\n \"Receive notification for a nonexistent event type: %s\",\n event_type,\n extra={\"serialized\": serialized},\n )\n return None\n\n try:\n return event_class.unmarshal(kwargs)\n except Exception:\n self._logger.exception(\"Error reconstituting event of type %s\", event_type)\n return None\n\n def reconstitute_event(self, payload: bytes) -> Event | None:\n try:\n event_type_bytes, serialized = payload.split(b\" \", 1)\n except ValueError:\n self._logger.error(\n \"Received malformatted notification\", extra={\"payload\": payload}\n )\n return None\n\n event_type = event_type_bytes.decode(\"ascii\", errors=\"replace\")\n return self._reconstitute_event(event_type, serialized)\n\n def reconstitute_event_str(self, payload: str) -> Event | None:\n try:\n event_type, b64_serialized = payload.split(\" \", 1)\n except ValueError:\n self._logger.error(\n \"Received malformatted notification\", extra={\"payload\": payload}\n )\n return None\n\n return self._reconstitute_event(event_type, b64decode(b64_serialized))\n" }, { "path": "src/apscheduler/eventbrokers/local.py", "content": "from __future__ import annotations\n\nimport attrs\n\nfrom .._events import Event\nfrom .._utils import create_repr\nfrom .base import BaseEventBroker\n\n\n@attrs.define(eq=False, repr=False)\nclass LocalEventBroker(BaseEventBroker):\n \"\"\"\n Asynchronous, local event broker.\n\n This event broker only broadcasts within the process it runs in, and is therefore\n not suitable for multi-node or multiprocess use cases.\n\n Does not serialize events.\n \"\"\"\n\n def __repr__(self) -> str:\n return create_repr(self)\n\n async def publish(self, event: Event) -> None:\n await self.publish_local(event)\n" }, { "path": "src/apscheduler/eventbrokers/mqtt.py", "content": "from __future__ import annotations\n\nimport sys\nfrom concurrent.futures import Future\nfrom contextlib import AsyncExitStack\nfrom logging import Logger\nfrom ssl import SSLContext\nfrom typing import Any\n\nimport attrs\nfrom anyio import to_thread\nfrom anyio.from_thread import BlockingPortal\nfrom attr.validators import in_, instance_of, optional\nfrom paho.mqtt.client import Client, MQTTMessage\nfrom paho.mqtt.enums import CallbackAPIVersion\n\nfrom .._events import Event\nfrom .._utils import create_repr\nfrom .base import BaseExternalEventBroker\n\nALLOWED_TRANSPORTS = (\"mqtt\", \"mqtts\", \"ws\", \"wss\", \"unix\")\n\n\n@attrs.define(eq=False, repr=False)\nclass MQTTEventBroker(BaseExternalEventBroker):\n \"\"\"\n An event broker that uses an MQTT (v3.1 or v5) broker to broadcast events.\n\n Requires the paho-mqtt_ library (v2.0 or later) to be installed.\n\n .. _paho-mqtt: https://pypi.org/project/paho-mqtt/\n\n :param host: MQTT broker host (or UNIX socket path)\n :param port: MQTT broker port (for ``tcp`` or ``websocket`` transports)\n :param transport: one of ``tcp``, ``websocket`` or ``unix`` (default: ``tcp``)\n :param client_id: MQTT client ID (needed to resume an MQTT session if a connection\n is broken)\n :param ssl: either ``True`` or a custom SSL context to enable SSL/TLS, ``False`` to\n disable\n :param topic: topic on which to send the messages\n :param subscribe_qos: MQTT QoS to use for subscribing messages\n :param publish_qos: MQTT QoS to use for publishing messages\n \"\"\"\n\n host: str = attrs.field(default=\"localhost\", validator=instance_of(str))\n port: int | None = attrs.field(default=None, validator=optional(instance_of(int)))\n transport: str = attrs.field(\n default=\"tcp\", validator=in_([\"tcp\", \"websocket\", \"unix\"])\n )\n client_id: str | None = attrs.field(\n default=None, validator=optional(instance_of(str))\n )\n ssl: bool | SSLContext = attrs.field(\n default=False, validator=instance_of((bool, SSLContext))\n )\n topic: str = attrs.field(\n kw_only=True, default=\"apscheduler\", validator=instance_of(str)\n )\n subscribe_qos: int = attrs.field(kw_only=True, default=0, validator=in_([0, 1, 2]))\n publish_qos: int = attrs.field(kw_only=True, default=0, validator=in_([0, 1, 2]))\n\n _use_tls: bool = attrs.field(init=False, default=False)\n _client: Client = attrs.field(init=False)\n _portal: BlockingPortal = attrs.field(init=False)\n _ready_future: Future[None] = attrs.field(init=False)\n\n def __attrs_post_init__(self) -> None:\n if self.port is None:\n if self.transport == \"tcp\":\n self.port = 8883 if self.ssl else 1883\n elif self.transport == \"websocket\":\n self.port = 443 if self.ssl else 80\n\n self._client = Client(\n callback_api_version=CallbackAPIVersion.VERSION2,\n client_id=self.client_id,\n transport=self.transport,\n )\n if isinstance(self.ssl, SSLContext):\n self._client.tls_set_context(self.ssl)\n elif self.ssl:\n self._client.tls_set()\n\n def __repr__(self) -> str:\n return create_repr(self, \"host\", \"port\", \"transport\")\n\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n await super().start(exit_stack, logger)\n self._portal = await exit_stack.enter_async_context(BlockingPortal())\n self._ready_future = Future()\n self._client.on_connect = self._on_connect\n self._client.on_connect_fail = self._on_connect_fail\n self._client.on_disconnect = self._on_disconnect\n self._client.on_message = self._on_message\n self._client.on_subscribe = self._on_subscribe\n self._client.connect_async(self.host, self.port)\n self._client.loop_start()\n exit_stack.push_async_callback(to_thread.run_sync, self._client.loop_stop)\n\n # Wait for the connection attempt to be done\n await to_thread.run_sync(self._ready_future.result, 10)\n\n # Schedule a disconnection for when the exit stack is exited\n exit_stack.callback(self._client.disconnect)\n\n def _on_connect(self, client: Client, *_: Any) -> None:\n self._logger.info(\"%s: Connected\", self.__class__.__name__)\n try:\n client.subscribe(self.topic, qos=self.subscribe_qos)\n except Exception as exc:\n self._ready_future.set_exception(exc)\n raise\n\n def _on_connect_fail(self, *_: Any) -> None:\n exc = sys.exc_info()[1]\n self._logger.error(\"%s: Connection failed (%s)\", self.__class__.__name__, exc)\n\n def _on_disconnect(self, *args: Any) -> None:\n reason_code = args[3] if len(args) == 5 else args[2]\n self._logger.error(\n \"%s: Disconnected (code: %s)\", self.__class__.__name__, reason_code\n )\n\n def _on_subscribe(self, *_: Any) -> None:\n self._logger.info(\"%s: Subscribed\", self.__class__.__name__)\n self._ready_future.set_result(None)\n\n def _on_message(self, _: Any, __: Any, msg: MQTTMessage) -> None:\n event = self.reconstitute_event(msg.payload)\n if event is not None:\n self._portal.call(self.publish_local, event)\n\n async def publish(self, event: Event) -> None:\n notification = self.generate_notification(event)\n await to_thread.run_sync(\n lambda: self._client.publish(self.topic, notification, qos=self.publish_qos)\n )\n" }, { "path": "src/apscheduler/eventbrokers/psycopg.py", "content": "from __future__ import annotations\n\nfrom collections.abc import AsyncGenerator, Mapping\nfrom contextlib import AsyncExitStack, asynccontextmanager\nfrom logging import Logger\nfrom typing import TYPE_CHECKING, Any\n\nimport attrs\nfrom anyio import (\n EndOfStream,\n create_memory_object_stream,\n move_on_after,\n)\nfrom anyio.abc import TaskStatus\nfrom anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream\nfrom attr.validators import instance_of\nfrom psycopg import AsyncConnection, InterfaceError\n\nfrom .._events import Event\nfrom .._exceptions import SerializationError\nfrom .._utils import create_repr\nfrom .._validators import positive_number\nfrom .base import BaseExternalEventBroker\n\nif TYPE_CHECKING:\n from sqlalchemy.ext.asyncio import AsyncEngine\n\n\ndef convert_options(value: Mapping[str, Any]) -> dict[str, Any]:\n return dict(value, autocommit=True)\n\n\n@attrs.define(eq=False, repr=False)\nclass PsycopgEventBroker(BaseExternalEventBroker):\n \"\"\"\n An asynchronous, psycopg_ based event broker that uses a PostgreSQL server to\n broadcast events using its ``NOTIFY`` mechanism.\n\n .. _psycopg: https://pypi.org/project/psycopg/\n\n :param conninfo: a libpq connection string (e.g.\n ``postgres://user:pass@host:port/dbname``)\n :param options: extra keyword arguments passed to\n :meth:`psycopg.AsyncConnection.connect`\n :param channel: the ``NOTIFY`` channel to use\n :param max_idle_time: maximum time (in seconds) to let the connection go idle,\n before sending a ``SELECT 1`` query to prevent a connection timeout\n \"\"\"\n\n conninfo: str = attrs.field(validator=instance_of(str))\n options: Mapping[str, Any] = attrs.field(\n factory=dict, converter=convert_options, validator=instance_of(Mapping)\n )\n channel: str = attrs.field(\n kw_only=True, default=\"apscheduler\", validator=instance_of(str)\n )\n max_idle_time: float = attrs.field(\n kw_only=True, default=10, validator=[instance_of((int, float)), positive_number]\n )\n\n _send: MemoryObjectSendStream[str] = attrs.field(init=False)\n\n @classmethod\n def from_async_sqla_engine(\n cls,\n engine: AsyncEngine,\n options: Mapping[str, Any] | None = None,\n **kwargs: Any,\n ) -> PsycopgEventBroker:\n \"\"\"\n Create a new psycopg event broker from a SQLAlchemy engine.\n\n The engine will only be used to create the appropriate options for\n :meth:`psycopg.AsyncConnection.connect`.\n\n :param engine: an asynchronous SQLAlchemy engine using psycopg as the driver\n :type engine: ~sqlalchemy.ext.asyncio.AsyncEngine\n :param options: extra keyword arguments passed to\n :meth:`psycopg.AsyncConnection.connect`\n :param kwargs: keyword arguments to pass to the initializer of this class\n :return: the newly created event broker\n\n \"\"\"\n if engine.dialect.driver != \"psycopg\":\n raise ValueError(\n f'The driver in the engine must be \"psycopg\" (current: '\n f\"{engine.dialect.driver})\"\n )\n\n conninfo = engine.url.render_as_string(hide_password=False).replace(\n \"+psycopg\", \"\"\n )\n return cls(conninfo, options or {}, **kwargs)\n\n def __repr__(self) -> str:\n return create_repr(self, \"conninfo\")\n\n @property\n def _temporary_failure_exceptions(self) -> tuple[type[Exception], ...]:\n return OSError, InterfaceError\n\n @asynccontextmanager\n async def _connect(self) -> AsyncGenerator[AsyncConnection, None]:\n async for attempt in self._retry():\n with attempt:\n conn = await AsyncConnection.connect(self.conninfo, **self.options)\n try:\n yield conn\n finally:\n with move_on_after(5, shield=True):\n await conn.close()\n\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n await super().start(exit_stack, logger)\n self._send, receive = create_memory_object_stream[str](100)\n try:\n await exit_stack.enter_async_context(self._send)\n await self._task_group.start(self._listen_notifications)\n exit_stack.callback(self._task_group.cancel_scope.cancel)\n await self._task_group.start(self._publish_notifications, receive)\n except BaseException:\n receive.close()\n raise\n\n async def _listen_notifications(self, *, task_status: TaskStatus[None]) -> None:\n task_started_sent = False\n while True:\n async with self._connect() as conn:\n try:\n await conn.execute(f\"LISTEN {self.channel}\")\n\n if not task_started_sent:\n task_status.started()\n task_started_sent = True\n\n self._logger.debug(\"Listen connection established\")\n async for notify in conn.notifies():\n if event := self.reconstitute_event_str(notify.payload):\n await self.publish_local(event)\n except InterfaceError as exc:\n self._logger.error(\"Connection error: %s\", exc)\n\n async def _publish_notifications(\n self, receive: MemoryObjectReceiveStream[str], *, task_status: TaskStatus[None]\n ) -> None:\n task_started_sent = False\n with receive:\n while True:\n async with self._connect() as conn:\n if not task_started_sent:\n task_status.started()\n task_started_sent = True\n\n self._logger.debug(\"Publish connection established\")\n notification: str | None = None\n while True:\n with move_on_after(self.max_idle_time):\n try:\n notification = await receive.receive()\n except EndOfStream:\n return\n\n if notification:\n await conn.execute(\n \"SELECT pg_notify(%t, %t)\", [self.channel, notification]\n )\n else:\n await conn.execute(\"SELECT 1\")\n\n async def publish(self, event: Event) -> None:\n notification = self.generate_notification_str(event)\n if len(notification) > 7999:\n raise SerializationError(\n \"Serialized event object exceeds 7999 bytes in size\"\n )\n\n await self._send.send(notification)\n" }, { "path": "src/apscheduler/eventbrokers/redis.py", "content": "from __future__ import annotations\n\nfrom asyncio import CancelledError\nfrom contextlib import AsyncExitStack\nfrom logging import Logger\n\nimport anyio\nimport attrs\nimport tenacity\nfrom anyio import move_on_after\nfrom attr.validators import instance_of\nfrom redis import ConnectionError\nfrom redis.asyncio import Redis\nfrom redis.asyncio.client import PubSub\nfrom redis.asyncio.connection import ConnectionPool\n\nfrom .._events import Event\nfrom .._utils import create_repr\nfrom .base import BaseExternalEventBroker\n\n\n@attrs.define(eq=False, repr=False)\nclass RedisEventBroker(BaseExternalEventBroker):\n \"\"\"\n An event broker that uses a Redis server to broadcast events.\n\n Requires the redis_ library to be installed.\n\n .. _redis: https://pypi.org/project/redis/\n\n :param client_or_url: an asynchronous Redis client or a Redis URL\n (```redis://...```)\n :param channel: channel on which to send the messages\n :param stop_check_interval: interval (in seconds) on which the channel listener\n should check if it should stop (higher values mean slower reaction time but less\n CPU use)\n\n .. note:: The event broker will not manage the life cycle of any client instance\n passed to it, so you need to close the client afterwards when you're done with\n it.\n \"\"\"\n\n client_or_url: Redis | str = attrs.field(validator=instance_of((Redis, str)))\n channel: str = attrs.field(kw_only=True, default=\"apscheduler\")\n stop_check_interval: float = attrs.field(kw_only=True, default=1)\n\n _client: Redis = attrs.field(init=False)\n _close_on_exit: bool = attrs.field(init=False, default=False)\n _stopped: bool = attrs.field(init=False, default=True)\n\n def __attrs_post_init__(self) -> None:\n if isinstance(self.client_or_url, str):\n pool = ConnectionPool.from_url(self.client_or_url)\n self._client = Redis(connection_pool=pool)\n self._close_on_exit = True\n else:\n self._client = self.client_or_url\n\n def __repr__(self) -> str:\n return create_repr(self, \"client_or_url\")\n\n def _retry(self) -> tenacity.AsyncRetrying:\n def after_attempt(retry_state: tenacity.RetryCallState) -> None:\n self._logger.warning(\n \"%s: connection failure (attempt %d): %s\",\n self.__class__.__name__,\n retry_state.attempt_number,\n retry_state.outcome.exception(),\n )\n\n return tenacity.AsyncRetrying(\n stop=self.retry_settings.stop,\n wait=self.retry_settings.wait,\n retry=tenacity.retry_if_exception_type(ConnectionError),\n after=after_attempt,\n sleep=anyio.sleep,\n reraise=True,\n )\n\n async def _close_client(self) -> None:\n with move_on_after(5, shield=True):\n await self._client.aclose(close_connection_pool=True)\n\n async def start(self, exit_stack: AsyncExitStack, logger: Logger) -> None:\n # Close the client and its connection pool if this broker was created using\n # .from_url()\n if self._close_on_exit:\n exit_stack.push_async_callback(self._close_client)\n\n pubsub = await exit_stack.enter_async_context(self._client.pubsub())\n await pubsub.subscribe(self.channel)\n await super().start(exit_stack, logger)\n\n self._stopped = False\n exit_stack.callback(setattr, self, \"_stopped\", True)\n self._task_group.start_soon(\n self._listen_messages, pubsub, name=\"Redis subscriber\"\n )\n\n async def _listen_messages(self, pubsub: PubSub) -> None:\n while not self._stopped:\n try:\n async for attempt in self._retry():\n with attempt:\n msg = await pubsub.get_message(\n ignore_subscribe_messages=True,\n timeout=self.stop_check_interval,\n )\n\n if msg and isinstance(msg[\"data\"], bytes):\n event = self.reconstitute_event(msg[\"data\"])\n if event is not None:\n await self.publish_local(event)\n except Exception as exc:\n # CancelledError is a subclass of Exception in Python 3.7\n if not isinstance(exc, CancelledError):\n self._logger.exception(\n \"%s listener crashed\", self.__class__.__name__\n )\n\n await pubsub.aclose()\n raise\n\n async def publish(self, event: Event) -> None:\n notification = self.generate_notification(event)\n async for attempt in self._retry():\n with attempt:\n await self._client.publish(self.channel, notification)\n" }, { "path": "src/apscheduler/executors/__init__.py", "content": "" }, { "path": "src/apscheduler/executors/async_.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom inspect import isawaitable\nfrom typing import Any\n\nfrom .._structures import Job\nfrom ..abc import JobExecutor\n\n\nclass AsyncJobExecutor(JobExecutor):\n \"\"\"\n Executes functions directly on the event loop thread.\n\n If the function returns a coroutine object (or another kind of awaitable), that is\n awaited on and its return value is used as the job's return value.\n \"\"\"\n\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n retval = func(*job.args, **job.kwargs)\n if isawaitable(retval):\n retval = await retval\n\n return retval\n" }, { "path": "src/apscheduler/executors/qt.py", "content": "from __future__ import annotations\n\nimport sys\nfrom collections.abc import Callable\nfrom concurrent.futures import Future\nfrom contextlib import AsyncExitStack\nfrom typing import Any, TypeVar\n\nimport anyio\nimport attrs\nfrom anyio.from_thread import BlockingPortal\n\nfrom apscheduler import Job, current_job\nfrom apscheduler.abc import JobExecutor\n\nif \"PySide6\" in sys.modules:\n from PySide6.QtCore import QObject, Signal\nelif \"PyQt6\" in sys.modules:\n from PyQt6.QtCore import QObject\n from PyQt6.QtCore import pyqtSignal as Signal\nelse:\n try:\n from PySide6.QtCore import QObject, Signal\n except ImportError:\n from PyQt6.QtCore import QObject\n from PyQt6.QtCore import pyqtSignal as Signal\n\nT_Retval = TypeVar(\"T_Retval\")\n\n\nclass _SchedulerSignals(QObject):\n run_job = Signal(tuple)\n\n\n@attrs.define(eq=False)\nclass QtJobExecutor(JobExecutor):\n _signals: _SchedulerSignals = attrs.field(init=False, factory=_SchedulerSignals)\n _portal: BlockingPortal = attrs.field(init=False)\n\n def __attrs_post_init__(self):\n self._signals.run_job.connect(self.run_in_qt_thread)\n\n async def start(self, exit_stack: AsyncExitStack) -> None:\n self._portal = await exit_stack.enter_async_context(BlockingPortal())\n\n async def run_job(self, func: Callable[..., T_Retval], job: Job) -> Any:\n future: Future[T_Retval] = Future()\n event = anyio.Event()\n self._signals.run_job.emit((func, job, future, event))\n await event.wait()\n return future.result(0)\n\n def run_in_qt_thread(\n self,\n parameters: tuple[Callable[..., T_Retval], Job, Future[T_Retval], anyio.Event],\n ) -> Any:\n func, job, future, event = parameters\n token = current_job.set(job)\n try:\n retval = func(*job.args, **job.kwargs)\n except BaseException as exc:\n future.set_exception(exc)\n if not isinstance(exc, Exception):\n raise\n else:\n future.set_result(retval)\n finally:\n current_job.reset(token)\n self._portal.call(event.set)\n" }, { "path": "src/apscheduler/executors/subprocess.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom contextlib import AsyncExitStack\nfrom functools import partial\nfrom typing import Any\n\nimport attrs\nfrom anyio import CapacityLimiter, to_process\n\nfrom .._structures import Job\nfrom ..abc import JobExecutor\n\n\n@attrs.define(eq=False, kw_only=True)\nclass ProcessPoolJobExecutor(JobExecutor):\n \"\"\"\n Executes functions in a process pool.\n\n :param max_workers: the maximum number of worker processes to keep\n \"\"\"\n\n max_workers: int = 40\n _limiter: CapacityLimiter = attrs.field(init=False)\n\n async def start(self, exit_stack: AsyncExitStack) -> None:\n self._limiter = CapacityLimiter(self.max_workers)\n\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n wrapped = partial(func, *job.args, **job.kwargs)\n return await to_process.run_sync(\n wrapped, cancellable=True, limiter=self._limiter\n )\n" }, { "path": "src/apscheduler/executors/thread.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Callable\nfrom contextlib import AsyncExitStack\nfrom functools import partial\nfrom typing import Any\n\nimport attrs\nfrom anyio import CapacityLimiter, to_thread\n\nfrom .._structures import Job\nfrom ..abc import JobExecutor\n\n\n@attrs.define(eq=False, kw_only=True)\nclass ThreadPoolJobExecutor(JobExecutor):\n \"\"\"\n Executes functions in a thread pool.\n\n :param max_workers: the maximum number of worker threads to keep\n \"\"\"\n\n max_workers: int = 40\n _limiter: CapacityLimiter = attrs.field(init=False)\n\n async def start(self, exit_stack: AsyncExitStack) -> None:\n self._limiter = CapacityLimiter(self.max_workers)\n\n async def run_job(self, func: Callable[..., Any], job: Job) -> Any:\n wrapped = partial(func, *job.args, **job.kwargs)\n return await to_thread.run_sync(wrapped, limiter=self._limiter)\n" }, { "path": "src/apscheduler/py.typed", "content": "" }, { "path": "src/apscheduler/serializers/__init__.py", "content": "" }, { "path": "src/apscheduler/serializers/cbor.py", "content": "from __future__ import annotations\n\nfrom datetime import date, timedelta, tzinfo\nfrom enum import Enum\nfrom typing import Any\n\nimport attrs\nfrom cbor2 import CBORDecoder, CBOREncoder, CBOREncodeTypeError, CBORTag, dumps, loads\n\nfrom .. import DeserializationError, SerializationError\nfrom .._marshalling import marshal_object, marshal_timezone, unmarshal_object\nfrom ..abc import Serializer\n\n\n@attrs.define(kw_only=True, eq=False)\nclass CBORSerializer(Serializer):\n \"\"\"\n Serializes objects using CBOR (:rfc:`8949`).\n\n Can serialize types not normally CBOR serializable, if they implement\n ``__getstate__()`` and ``__setstate__()``.\n\n :param type_tag: CBOR tag number for indicating arbitrary serialized object\n :param dump_options: keyword arguments passed to :func:`cbor2.dumps`\n :param load_options: keyword arguments passed to :func:`cbor2.loads`\n \"\"\"\n\n type_tag: int = 4664\n dump_options: dict[str, Any] = attrs.field(factory=dict)\n load_options: dict[str, Any] = attrs.field(factory=dict)\n\n def __attrs_post_init__(self) -> None:\n self.dump_options.setdefault(\"default\", self._default_hook)\n self.load_options.setdefault(\"tag_hook\", self._tag_hook)\n\n def _default_hook(self, encoder: CBOREncoder, value: object) -> None:\n if isinstance(value, date):\n encoder.encode(value.isoformat())\n elif isinstance(value, timedelta):\n encoder.encode(value.total_seconds())\n elif isinstance(value, tzinfo):\n encoder.encode(marshal_timezone(value))\n elif isinstance(value, Enum):\n encoder.encode(value.name)\n elif hasattr(value, \"__getstate__\"):\n marshalled = marshal_object(value)\n encoder.encode(CBORTag(self.type_tag, marshalled))\n else:\n raise CBOREncodeTypeError(\n f\"cannot serialize type {value.__class__.__name__}\"\n )\n\n def _tag_hook(\n self, decoder: CBORDecoder, tag: CBORTag, shareable_index: int | None = None\n ) -> object:\n if tag.tag == self.type_tag:\n cls_ref, state = tag.value\n return unmarshal_object(cls_ref, state)\n\n def serialize(self, obj: object) -> bytes:\n try:\n return dumps(obj, **self.dump_options)\n except Exception as exc:\n raise SerializationError from exc\n\n def deserialize(self, serialized: bytes):\n try:\n return loads(serialized, **self.load_options)\n except Exception as exc:\n raise DeserializationError from exc\n" }, { "path": "src/apscheduler/serializers/json.py", "content": "from __future__ import annotations\n\nfrom datetime import date, timedelta, tzinfo\nfrom enum import Enum\nfrom json import dumps, loads\nfrom typing import Any\nfrom uuid import UUID\n\nimport attrs\n\nfrom .. import DeserializationError, SerializationError\nfrom .._marshalling import (\n marshal_object,\n marshal_timezone,\n unmarshal_object,\n)\nfrom ..abc import Serializer\n\n\n@attrs.define(kw_only=True, eq=False)\nclass JSONSerializer(Serializer):\n \"\"\"\n Serializes objects using JSON.\n\n Can serialize types not normally CBOR serializable, if they implement\n ``__getstate__()`` and ``__setstate__()``. These objects are serialized into dicts\n that contain the necessary information for deserialization in ``magic_key``.\n\n :param magic_key: name of a specially handled dict key that indicates that a dict\n contains a serialized instance of an arbitrary type\n :param dump_options: keyword arguments passed to :func:`json.dumps`\n :param load_options: keyword arguments passed to :func:`json.loads`\n \"\"\"\n\n magic_key: str = \"_apscheduler_json\"\n dump_options: dict[str, Any] = attrs.field(factory=dict)\n load_options: dict[str, Any] = attrs.field(factory=dict)\n\n def __attrs_post_init__(self):\n self.dump_options[\"default\"] = self._default_hook\n self.load_options[\"object_hook\"] = self._object_hook\n\n def _default_hook(self, obj):\n if isinstance(obj, date):\n return obj.isoformat()\n elif isinstance(obj, timedelta):\n return obj.total_seconds()\n elif isinstance(obj, tzinfo):\n return marshal_timezone(obj)\n elif isinstance(obj, UUID):\n return str(obj)\n elif isinstance(obj, Enum):\n return obj.name\n elif hasattr(obj, \"__getstate__\"):\n cls_ref, state = marshal_object(obj)\n return {self.magic_key: [cls_ref, state]}\n\n raise TypeError(\n f\"Object of type {obj.__class__.__name__!r} is not JSON serializable\"\n )\n\n def _object_hook(self, obj_state: dict[str, Any]):\n if self.magic_key in obj_state:\n ref, state = obj_state[self.magic_key]\n return unmarshal_object(ref, state)\n\n return obj_state\n\n def serialize(self, obj: object) -> bytes:\n try:\n return dumps(obj, ensure_ascii=False, **self.dump_options).encode(\"utf-8\")\n except Exception as exc:\n raise SerializationError from exc\n\n def deserialize(self, serialized: bytes):\n try:\n return loads(serialized, **self.load_options)\n except Exception as exc:\n raise DeserializationError from exc\n" }, { "path": "src/apscheduler/serializers/pickle.py", "content": "from __future__ import annotations\n\nfrom pickle import dumps, loads\n\nimport attrs\n\nfrom .. import DeserializationError, SerializationError\nfrom ..abc import Serializer\n\n\n@attrs.define(kw_only=True, eq=False)\nclass PickleSerializer(Serializer):\n \"\"\"\n Uses the :mod:`pickle` module to (de)serialize objects.\n\n As this serialization method is native to Python, it is able to serialize a wide\n range of types, at the expense of being insecure. Do **not** use this serializer\n unless you can fully trust the entire system to not have maliciously injected data.\n Such data can be made to call arbitrary functions with arbitrary arguments on\n unpickling.\n\n :param protocol: the pickle protocol number to use\n \"\"\"\n\n protocol: int = 4\n\n def serialize(self, obj: object) -> bytes:\n try:\n return dumps(obj, self.protocol)\n except Exception as exc:\n raise SerializationError from exc\n\n def deserialize(self, serialized: bytes):\n try:\n return loads(serialized)\n except Exception as exc:\n raise DeserializationError from exc\n" }, { "path": "src/apscheduler/triggers/__init__.py", "content": "" }, { "path": "src/apscheduler/triggers/calendarinterval.py", "content": "from __future__ import annotations\n\nfrom datetime import date, datetime, time, timedelta, tzinfo\nfrom typing import Any\n\nimport attrs\nfrom attr.validators import instance_of, optional\n\nfrom .._converters import as_aware_datetime, as_date, as_timezone\nfrom .._utils import require_state_version, timezone_repr\nfrom ..abc import Trigger\n\n\n@attrs.define(kw_only=True)\nclass CalendarIntervalTrigger(Trigger):\n \"\"\"\n Runs the task on specified calendar-based intervals always at the same exact time of\n day.\n\n When calculating the next date, the ``years`` and ``months`` parameters are first\n added to the previous date while keeping the day of the month constant. This is\n repeated until the resulting date is valid. After that, the ``weeks`` and ``days``\n parameters are added to that date. Finally, the date is combined with the given time\n (hour, minute, second) to form the final datetime.\n\n This means that if the ``days`` or ``weeks`` parameters are not used, the task will\n always be executed on the same day of the month at the same wall clock time,\n assuming the date and time are valid.\n\n If the resulting datetime is invalid due to a daylight saving forward shift, the\n date is discarded and the process moves on to the next date. If instead the datetime\n is ambiguous due to a backward DST shift, the earlier of the two resulting datetimes\n is used.\n\n If no previous run time is specified when requesting a new run time (like when\n starting for the first time or resuming after being paused), ``start_date`` is used\n as a reference and the next valid datetime equal to or later than the current time\n will be returned. Otherwise, the next valid datetime starting from the previous run\n time is returned, even if it's in the past.\n\n .. warning:: Be wary of setting a start date near the end of the month (29. – 31.)\n if you have ``months`` specified in your interval, as this will skip the months\n when those days do not exist. Likewise, setting the start date on the leap day\n (February 29th) and having ``years`` defined may cause some years to be skipped.\n\n Users are also discouraged from using a time inside the target timezone's DST\n switching period (typically around 2 am) since a date could either be skipped or\n repeated due to the specified wall clock time either occurring twice or not at\n all.\n\n :param years: number of years to wait\n :param months: number of months to wait\n :param weeks: number of weeks to wait\n :param days: number of days to wait\n :param hour: hour to run the task at\n :param minute: minute to run the task at\n :param second: second to run the task at\n :param start_date: first date to trigger on (defaults to current date if omitted)\n :param end_date: latest possible date to trigger on\n :param timezone: time zone to use for calculating the next fire time\n \"\"\"\n\n years: int = 0\n months: int = 0\n weeks: int = 0\n days: int = 0\n hour: int = 0\n minute: int = 0\n second: int = 0\n start_date: date = attrs.field(\n converter=as_date, validator=instance_of(date), factory=date.today\n )\n end_date: date | None = attrs.field(\n converter=as_date, validator=optional(instance_of(date)), default=None\n )\n timezone: tzinfo = attrs.field(\n converter=as_timezone, validator=instance_of(tzinfo), default=\"local\"\n )\n _time: time = attrs.field(init=False, eq=False)\n _last_fire_date: date | None = attrs.field(\n init=False, eq=False, converter=as_aware_datetime, default=None\n )\n\n def __attrs_post_init__(self) -> None:\n self._time = time(self.hour, self.minute, self.second, tzinfo=self.timezone)\n\n if self.years == self.months == self.weeks == self.days == 0:\n raise ValueError(\"interval must be at least 1 day long\")\n\n if self.start_date and self.end_date and self.start_date > self.end_date:\n raise ValueError(\"end_date cannot be earlier than start_date\")\n\n def next(self) -> datetime | None:\n previous_date: date = self._last_fire_date\n while True:\n if previous_date:\n year, month = previous_date.year, previous_date.month\n while True:\n month += self.months\n year += self.years + (month - 1) // 12\n month = (month - 1) % 12 + 1\n try:\n next_date = date(year, month, previous_date.day)\n except ValueError:\n pass # Nonexistent date\n else:\n next_date += timedelta(self.days + self.weeks * 7)\n break\n else:\n next_date = self.start_date\n\n # Don't return any date past end_date\n if self.end_date and next_date > self.end_date:\n return None\n\n # Combine the date with the designated time and normalize the result\n timestamp = datetime.combine(next_date, self._time).timestamp()\n next_time = datetime.fromtimestamp(timestamp, self.timezone)\n\n # Check if the time is off due to normalization and a forward DST shift\n if next_time.timetz() != self._time:\n previous_date = next_time.date()\n else:\n self._last_fire_date = next_date\n return next_time\n\n def __getstate__(self) -> dict[str, Any]:\n return {\n \"version\": 1,\n \"interval\": [self.years, self.months, self.weeks, self.days],\n \"time\": [self._time.hour, self._time.minute, self._time.second],\n \"start_date\": self.start_date,\n \"end_date\": self.end_date,\n \"timezone\": self.timezone,\n \"last_fire_date\": self._last_fire_date,\n }\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n self.years, self.months, self.weeks, self.days = state[\"interval\"]\n self.start_date = state[\"start_date\"]\n self.end_date = state[\"end_date\"]\n self.timezone = state[\"timezone\"]\n self._time = time(*state[\"time\"], tzinfo=self.timezone)\n self._last_fire_date = state[\"last_fire_date\"]\n\n def __repr__(self) -> str:\n fields = []\n for field in \"years\", \"months\", \"weeks\", \"days\":\n value = getattr(self, field)\n if value > 0:\n fields.append(f\"{field}={value}\")\n\n fields.append(f\"time={self._time.isoformat()!r}\")\n fields.append(f\"start_date='{self.start_date}'\")\n if self.end_date:\n fields.append(f\"end_date='{self.end_date}'\")\n\n fields.append(f\"timezone={timezone_repr(self.timezone)!r}\")\n return f\"{self.__class__.__name__}({', '.join(fields)})\"\n" }, { "path": "src/apscheduler/triggers/combining.py", "content": "from __future__ import annotations\n\nfrom abc import abstractmethod\nfrom datetime import datetime, timedelta\nfrom typing import Any\n\nimport attrs\n\nfrom .._converters import as_aware_datetime, as_timedelta, list_converter\nfrom .._exceptions import MaxIterationsReached\nfrom .._marshalling import marshal_object, unmarshal_object\nfrom .._utils import require_state_version\nfrom ..abc import Trigger\n\n\n@attrs.define\nclass BaseCombiningTrigger(Trigger):\n triggers: list[Trigger]\n _next_fire_times: list[datetime | None] = attrs.field(\n init=False, eq=False, converter=list_converter(as_aware_datetime), factory=list\n )\n\n def __getstate__(self) -> dict[str, Any]:\n return {\n \"version\": 1,\n \"triggers\": [marshal_object(trigger) for trigger in self.triggers],\n \"next_fire_times\": self._next_fire_times,\n }\n\n @abstractmethod\n def __setstate__(self, state: dict[str, Any]) -> None:\n self.triggers = [\n unmarshal_object(*trigger_state) for trigger_state in state[\"triggers\"]\n ]\n self._next_fire_times = state[\"next_fire_times\"]\n\n\n@attrs.define\nclass AndTrigger(BaseCombiningTrigger):\n \"\"\"\n Fires on times produced by the enclosed triggers whenever the fire times are within\n the given threshold.\n\n If the produced fire times are not within the given threshold of each other, the\n trigger(s) that produced the earliest fire time will be asked for their next fire\n time and the iteration is restarted. If instead all the triggers agree on a fire\n time, all the triggers are asked for their next fire times and the earliest of the\n previously produced fire times will be returned.\n\n This trigger will be finished when any of the enclosed trigger has finished.\n\n :param triggers: triggers to combine\n :param threshold: maximum time difference between the next fire times of the\n triggers in order for the earliest of them to be returned from :meth:`next` (in\n seconds, or as timedelta)\n :param max_iterations: maximum number of iterations of fire time calculations before\n giving up\n \"\"\"\n\n threshold: timedelta = attrs.field(converter=as_timedelta, default=1)\n max_iterations: int | None = 10000\n\n def next(self) -> datetime | None:\n if not self._next_fire_times:\n # Fill out the fire times on the first run\n self._next_fire_times = [t.next() for t in self.triggers]\n\n for _ in range(self.max_iterations):\n # Find the earliest and latest fire times\n earliest_fire_time: datetime | None = None\n latest_fire_time: datetime | None = None\n for fire_time in self._next_fire_times:\n # If any of the fire times is None, this trigger is finished\n if fire_time is None:\n return None\n\n if earliest_fire_time is None or earliest_fire_time > fire_time:\n earliest_fire_time = fire_time\n\n if latest_fire_time is None or latest_fire_time < fire_time:\n latest_fire_time = fire_time\n\n # Replace all the fire times that were within the threshold\n for i, _trigger in enumerate(self.triggers):\n if self._next_fire_times[i] - earliest_fire_time <= self.threshold:\n self._next_fire_times[i] = self.triggers[i].next()\n\n # If all the fire times were within the threshold, return the earliest one\n if latest_fire_time - earliest_fire_time <= self.threshold:\n return earliest_fire_time\n else:\n raise MaxIterationsReached\n\n def __getstate__(self) -> dict[str, Any]:\n state = super().__getstate__()\n state[\"threshold\"] = self.threshold\n state[\"max_iterations\"] = self.max_iterations\n return state\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n super().__setstate__(state)\n self.threshold = state[\"threshold\"]\n self.max_iterations = state[\"max_iterations\"]\n\n def __repr__(self) -> str:\n return (\n f\"{self.__class__.__name__}({self.triggers}, \"\n f\"threshold={self.threshold.total_seconds()}, \"\n f\"max_iterations={self.max_iterations})\"\n )\n\n\n@attrs.define\nclass OrTrigger(BaseCombiningTrigger):\n \"\"\"\n Fires on every fire time of every trigger in chronological order.\n If two or more triggers produce the same fire time, it will only be used once.\n\n This trigger will be finished when none of the enclosed triggers can produce any new\n fire times.\n\n :param triggers: triggers to combine\n \"\"\"\n\n def next(self) -> datetime | None:\n # Fill out the fire times on the first run\n if not self._next_fire_times:\n self._next_fire_times = [t.next() for t in self.triggers]\n\n # Find out the earliest of the fire times\n earliest_time: datetime | None = min(\n (fire_time for fire_time in self._next_fire_times if fire_time is not None),\n default=None,\n )\n if earliest_time is not None:\n # Generate new fire times for the trigger(s) that generated the earliest\n # fire time\n for i, fire_time in enumerate(self._next_fire_times):\n if fire_time == earliest_time:\n self._next_fire_times[i] = self.triggers[i].next()\n\n return earliest_time\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n super().__setstate__(state)\n\n def __repr__(self) -> str:\n return f\"{self.__class__.__name__}({self.triggers})\"\n" }, { "path": "src/apscheduler/triggers/cron/__init__.py", "content": "from __future__ import annotations\n\nfrom collections.abc import Sequence\nfrom datetime import datetime, tzinfo\nfrom typing import Any, ClassVar\n\nimport attrs\nfrom attr.validators import instance_of, optional\nfrom tzlocal import get_localzone\n\nfrom ..._converters import as_aware_datetime, as_datetime, as_timezone\nfrom ..._utils import require_state_version, time_exists, timezone_repr\nfrom ...abc import Trigger\nfrom .fields import (\n DEFAULT_VALUES,\n BaseField,\n DayOfMonthField,\n DayOfWeekField,\n MonthField,\n WeekField,\n)\n\n\n@attrs.define(kw_only=True)\nclass CronTrigger(Trigger):\n \"\"\"\n Triggers when current time matches all specified time constraints, similarly to how\n the UNIX cron scheduler works.\n\n :param year: 4-digit year\n :param month: month (1-12)\n :param day: day of the (1-31)\n :param week: ISO week (1-53)\n :param day_of_week: number or name of weekday (0-7 or sun,mon,tue,wed,thu,fri,sat,\n sun)\n :param hour: hour (0-23)\n :param minute: minute (0-59)\n :param second: second (0-59)\n :param start_time: earliest possible date/time to trigger on (defaults to current\n time)\n :param end_time: latest possible date/time to trigger on\n :param timezone: time zone to use for the date/time calculations\n (defaults to the local timezone)\n\n .. note:: The first weekday is always **monday**.\n \"\"\"\n\n FIELDS_MAP: ClassVar[list[tuple[str, type[BaseField]]]] = [\n (\"year\", BaseField),\n (\"month\", MonthField),\n (\"day\", DayOfMonthField),\n (\"week\", WeekField),\n (\"day_of_week\", DayOfWeekField),\n (\"hour\", BaseField),\n (\"minute\", BaseField),\n (\"second\", BaseField),\n ]\n\n year: int | str | None = None\n month: int | str | None = None\n day: int | str | None = None\n week: int | str | None = None\n day_of_week: int | str | None = None\n hour: int | str | None = None\n minute: int | str | None = None\n second: int | str | None = None\n start_time: datetime = attrs.field(\n converter=as_datetime,\n validator=instance_of(datetime),\n factory=datetime.now,\n )\n end_time: datetime | None = attrs.field(\n converter=as_datetime,\n validator=optional(instance_of(datetime)),\n default=None,\n )\n timezone: tzinfo = attrs.field(\n converter=as_timezone, validator=instance_of(tzinfo), factory=get_localzone\n )\n _fields: list[BaseField] = attrs.field(init=False, eq=False, factory=list)\n _last_fire_time: datetime | None = attrs.field(\n converter=as_aware_datetime, init=False, eq=False, default=None\n )\n\n def __attrs_post_init__(self) -> None:\n self.start_time = self._to_trigger_timezone(self.start_time, \"start_time\")\n self.end_time = self._to_trigger_timezone(self.end_time, \"end_time\")\n self._set_fields(\n [\n self.year,\n self.month,\n self.day,\n self.week,\n self.day_of_week,\n self.hour,\n self.minute,\n self.second,\n ]\n )\n\n def _set_fields(self, values: Sequence[int | str | None]) -> None:\n self._fields = []\n assigned_values = {\n field_name: value\n for (field_name, _), value in zip(self.FIELDS_MAP, values, strict=True)\n if value is not None\n }\n for field_name, field_class in self.FIELDS_MAP:\n exprs = assigned_values.pop(field_name, None)\n if exprs is None:\n exprs = \"*\" if assigned_values else DEFAULT_VALUES[field_name]\n\n field = field_class(field_name, exprs)\n self._fields.append(field)\n\n @classmethod\n def from_crontab(\n cls,\n expr: str,\n *,\n start_time: datetime | None = None,\n end_time: datetime | None = None,\n timezone: tzinfo | str = \"local\",\n ) -> CronTrigger:\n \"\"\"\n Create a :class:`~CronTrigger` from a standard crontab expression.\n\n See https://en.wikipedia.org/wiki/Cron for more information on the format\n accepted here.\n\n :param expr: minute, hour, day of month, month, day of week\n :param start_time: earliest possible date/time to trigger on (defaults to current\n time)\n :param end_time: latest possible date/time to trigger on\n :param timezone: time zone to use for the date/time calculations\n (defaults to local timezone if omitted)\n\n \"\"\"\n values = expr.split()\n if len(values) != 5:\n raise ValueError(f\"Wrong number of fields; got {len(values)}, expected 5\")\n\n return cls(\n minute=values[0],\n hour=values[1],\n day=values[2],\n month=values[3],\n day_of_week=values[4],\n start_time=start_time or datetime.now(),\n end_time=end_time,\n timezone=timezone,\n )\n\n def _to_trigger_timezone(self, dt: datetime | None, name: str) -> datetime | None:\n if dt is None:\n return None\n\n if dt.tzinfo is None:\n dt = dt.replace(tzinfo=self.timezone)\n else:\n dt = dt.astimezone(self.timezone)\n\n if not time_exists(dt):\n raise ValueError(f\"{name}={dt} does not exist\")\n\n return dt\n\n def _increment_field_value(\n self, dateval: datetime, fieldnum: int\n ) -> tuple[datetime, int]:\n \"\"\"\n Increments the designated field and resets all less significant fields to their\n minimum values.\n\n :return: a tuple containing the new date, and the number of the field that was\n actually incremented\n \"\"\"\n\n values = {}\n i = 0\n while i < len(self._fields):\n field = self._fields[i]\n if not field.real:\n if i == fieldnum:\n fieldnum -= 1\n i -= 1\n else:\n i += 1\n continue\n\n if i < fieldnum:\n values[field.name] = field.get_value(dateval)\n i += 1\n elif i > fieldnum:\n values[field.name] = field.get_min(dateval)\n i += 1\n else:\n value = field.get_value(dateval)\n maxval = field.get_max(dateval)\n if value == maxval:\n fieldnum -= 1\n i -= 1\n else:\n values[field.name] = value + 1\n i += 1\n\n difference = datetime(**values) - dateval.replace(tzinfo=None)\n dateval = datetime.fromtimestamp(\n dateval.timestamp() + difference.total_seconds(), self.timezone\n )\n return dateval, fieldnum\n\n def _set_field_value(\n self, dateval: datetime, fieldnum: int, new_value: int\n ) -> datetime:\n values = {}\n for i, field in enumerate(self._fields):\n if field.real:\n if i < fieldnum:\n values[field.name] = field.get_value(dateval)\n elif i > fieldnum:\n values[field.name] = field.get_min(dateval)\n else:\n values[field.name] = new_value\n\n return datetime(**values, tzinfo=self.timezone, fold=dateval.fold)\n\n def next(self) -> datetime | None:\n if self._last_fire_time:\n next_time = datetime.fromtimestamp(\n self._last_fire_time.timestamp() + 1, self.timezone\n )\n else:\n next_time = self.start_time\n\n fieldnum = 0\n while 0 <= fieldnum < len(self._fields):\n field = self._fields[fieldnum]\n curr_value = field.get_value(next_time)\n next_value = field.get_next_value(next_time)\n\n if next_value is None:\n # No valid value was found\n next_time, fieldnum = self._increment_field_value(\n next_time, fieldnum - 1\n )\n elif next_value > curr_value:\n # A valid, but higher than the starting value, was found\n if field.real:\n next_time = self._set_field_value(next_time, fieldnum, next_value)\n if time_exists(next_time):\n fieldnum += 1\n else:\n # skip non-existent date\n next_time, fieldnum = self._increment_field_value(\n next_time, fieldnum\n )\n else:\n next_time, fieldnum = self._increment_field_value(\n next_time, fieldnum\n )\n else:\n # A valid value was found, no changes necessary\n fieldnum += 1\n\n # Return if the date has rolled past the end date\n if self.end_time and next_time > self.end_time:\n return None\n\n if fieldnum >= 0:\n self._last_fire_time = next_time\n return next_time\n\n return None\n\n def __getstate__(self) -> dict[str, Any]:\n return {\n \"version\": 1,\n \"timezone\": self.timezone,\n \"fields\": [str(f) for f in self._fields],\n \"start_time\": self.start_time,\n \"end_time\": self.end_time,\n \"last_fire_time\": self._last_fire_time,\n }\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n self.timezone = state[\"timezone\"]\n self.start_time = state[\"start_time\"]\n self.end_time = state[\"end_time\"]\n self._last_fire_time = state[\"last_fire_time\"]\n self._set_fields(state[\"fields\"])\n\n def __repr__(self) -> str:\n fields = [f\"{field.name}={str(field)!r}\" for field in self._fields]\n fields.append(f\"start_time={self.start_time.isoformat()!r}\")\n if self.end_time:\n fields.append(f\"end_time={self.end_time.isoformat()!r}\")\n\n fields.append(f\"timezone={timezone_repr(self.timezone)!r}\")\n return f\"CronTrigger({', '.join(fields)})\"\n" }, { "path": "src/apscheduler/triggers/cron/expressions.py", "content": "\"\"\"This module contains the expressions applicable for CronTrigger's fields.\"\"\"\n\nfrom __future__ import annotations\n\nimport re\nfrom calendar import monthrange\nfrom datetime import datetime\nfrom re import Pattern\nfrom typing import TYPE_CHECKING, ClassVar\n\nimport attrs\nfrom attr.validators import instance_of, optional\n\nfrom ..._converters import as_int\nfrom ..._validators import non_negative_number, positive_number\n\nif TYPE_CHECKING:\n from .fields import BaseField\n\nWEEKDAYS = [\"mon\", \"tue\", \"wed\", \"thu\", \"fri\", \"sat\", \"sun\"]\nMONTHS = [\n \"jan\",\n \"feb\",\n \"mar\",\n \"apr\",\n \"may\",\n \"jun\",\n \"jul\",\n \"aug\",\n \"sep\",\n \"oct\",\n \"nov\",\n \"dec\",\n]\n\n\ndef get_weekday_index(weekday: str) -> int:\n try:\n return WEEKDAYS.index(weekday.lower())\n except ValueError:\n raise ValueError(f\"Invalid weekday name {weekday!r}\") from None\n\n\n@attrs.define(slots=True)\nclass AllExpression:\n value_re: ClassVar[Pattern] = re.compile(r\"\\*(?:/(?P\\d+))?$\")\n\n step: int | None = attrs.field(\n converter=as_int,\n validator=optional([instance_of(int), positive_number]),\n default=None,\n )\n\n def validate_range(self, field_name: str, min_value: int, max_value: int) -> None:\n value_range = max_value - min_value\n if self.step and self.step > value_range:\n raise ValueError(\n f\"the step value ({self.step}) is higher than the total range of the \"\n f\"expression ({value_range})\"\n )\n\n def get_next_value(self, dateval: datetime, field: BaseField) -> int | None:\n start = field.get_value(dateval)\n minval = field.get_min(dateval)\n maxval = field.get_max(dateval)\n start = max(start, minval)\n\n if not self.step:\n nextval = start\n else:\n distance_to_next = (self.step - (start - minval)) % self.step\n nextval = start + distance_to_next\n\n return nextval if nextval <= maxval else None\n\n def __str__(self) -> str:\n return f\"*/{self.step}\" if self.step else \"*\"\n\n\n@attrs.define(kw_only=True)\nclass RangeExpression(AllExpression):\n value_re: ClassVar[Pattern] = re.compile(\n r\"(?P\\d+)(?:-(?P\\d+))?(?:/(?P\\d+))?$\"\n )\n\n first: int = attrs.field(\n converter=as_int, validator=[instance_of(int), non_negative_number]\n )\n last: int | None = attrs.field(\n converter=as_int,\n validator=optional([instance_of(int), non_negative_number]),\n default=None,\n )\n\n def __attrs_post_init__(self) -> None:\n if self.last is None and self.step is None:\n self.last = self.first\n\n if self.last is not None and self.first > self.last:\n raise ValueError(\n \"The minimum value in a range must not be higher than the maximum\"\n )\n\n def validate_range(self, field_name: str, min_value: int, max_value: int) -> None:\n super().validate_range(field_name, min_value, max_value)\n if self.first < min_value:\n raise ValueError(\n f\"the first value ({self.first}) is lower than the minimum value \"\n f\"({min_value})\"\n )\n if self.last is not None and self.last > max_value:\n raise ValueError(\n f\"the last value ({self.last}) is higher than the maximum value \"\n f\"({max_value})\"\n )\n value_range = (self.last or max_value) - self.first\n if self.step and self.step > value_range:\n raise ValueError(\n f\"the step value ({self.step}) is higher than the total range of the \"\n f\"expression ({value_range})\"\n )\n\n def get_next_value(self, dateval: datetime, field: BaseField) -> int | None:\n startval = field.get_value(dateval)\n minval = field.get_min(dateval)\n maxval = field.get_max(dateval)\n\n # Apply range limits\n minval = max(minval, self.first)\n maxval = min(maxval, self.last) if self.last is not None else maxval\n nextval = max(minval, startval)\n\n # Apply the step if defined\n if self.step:\n distance_to_next = (self.step - (nextval - minval)) % self.step\n nextval += distance_to_next\n\n return nextval if nextval <= maxval else None\n\n def __str__(self) -> str:\n if self.last != self.first and self.last is not None:\n rangeval = f\"{self.first}-{self.last}\"\n else:\n rangeval = str(self.first)\n\n if self.step:\n return f\"{rangeval}/{self.step}\"\n\n return rangeval\n\n\nclass MonthRangeExpression(RangeExpression):\n value_re: ClassVar[Pattern] = re.compile(\n r\"(?P[a-z]+)(?:-(?P[a-z]+))?\", re.IGNORECASE\n )\n\n def __init__(self, first: str, last: str | None = None):\n try:\n first_num = MONTHS.index(first.lower()) + 1\n except ValueError:\n raise ValueError(f\"Invalid month name {first!r}\") from None\n\n if last:\n try:\n last_num = MONTHS.index(last.lower()) + 1\n except ValueError:\n raise ValueError(f\"Invalid month name {last!r}\") from None\n else:\n last_num = None\n\n super().__init__(first=first_num, last=last_num)\n\n def __str__(self) -> str:\n if self.last != self.first and self.last is not None:\n return f\"{MONTHS[self.first - 1]}-{MONTHS[self.last - 1]}\"\n\n return MONTHS[self.first - 1]\n\n\n@attrs.define(kw_only=True, init=False)\nclass WeekdayRangeExpression(RangeExpression):\n value_re: ClassVar[Pattern] = re.compile(\n r\"(?P[a-z]+)(?:-(?P[a-z]+))?\", re.IGNORECASE\n )\n\n def __init__(self, first: str, last: str | None = None):\n first_num = get_weekday_index(first)\n last_num = get_weekday_index(last) if last else None\n self.__attrs_init__(first=first_num, last=last_num)\n\n def __str__(self) -> str:\n if self.last != self.first and self.last is not None:\n return f\"{WEEKDAYS[self.first]}-{WEEKDAYS[self.last]}\"\n\n return WEEKDAYS[self.first]\n\n\n@attrs.define(kw_only=True, init=False)\nclass WeekdayPositionExpression(AllExpression):\n options: ClassVar[tuple[str, ...]] = (\"1st\", \"2nd\", \"3rd\", \"4th\", \"5th\", \"last\")\n value_re: ClassVar[Pattern] = re.compile(\n f\"(?P{'|'.join(options)}) +(?P(?:\\\\d+|\\\\w+))\",\n re.IGNORECASE,\n )\n\n option_num: int\n weekday: int\n\n def __init__(self, *, option_name: str, weekday_name: str):\n option_num = self.options.index(option_name.lower())\n try:\n weekday = WEEKDAYS.index(weekday_name.lower())\n except ValueError:\n raise ValueError(f\"Invalid weekday name {weekday_name!r}\") from None\n\n self.__attrs_init__(option_num=option_num, weekday=weekday)\n\n def get_next_value(self, dateval: datetime, field: BaseField) -> int | None:\n # Figure out the weekday of the month's first day and the number of days in that\n # month\n first_day_wday, last_day = monthrange(dateval.year, dateval.month)\n\n # Calculate which day of the month is the first of the target weekdays\n first_hit_day = self.weekday - first_day_wday + 1\n if first_hit_day <= 0:\n first_hit_day += 7\n\n # Calculate what day of the month the target weekday would be\n if self.option_num < 5:\n target_day = first_hit_day + self.option_num * 7\n else:\n target_day = first_hit_day + ((last_day - first_hit_day) // 7) * 7\n\n if last_day >= target_day >= dateval.day:\n return target_day\n else:\n return None\n\n def __str__(self) -> str:\n return f\"{self.options[self.option_num]} {WEEKDAYS[self.weekday]}\"\n\n\nclass LastDayOfMonthExpression(AllExpression):\n value_re: ClassVar[Pattern] = re.compile(r\"last\", re.IGNORECASE)\n\n def __init__(self) -> None:\n super().__init__(None)\n\n def get_next_value(self, dateval: datetime, field: BaseField) -> int | None:\n return monthrange(dateval.year, dateval.month)[1]\n\n def __str__(self) -> str:\n return \"last\"\n" }, { "path": "src/apscheduler/triggers/cron/fields.py", "content": "\"\"\"\nFields represent CronTrigger options which map to :class:`~datetime.datetime` fields.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport re\nfrom calendar import monthrange\nfrom collections.abc import Mapping, Sequence\nfrom datetime import datetime\nfrom typing import Any, ClassVar\n\nfrom .expressions import (\n WEEKDAYS,\n AllExpression,\n LastDayOfMonthExpression,\n MonthRangeExpression,\n RangeExpression,\n WeekdayPositionExpression,\n WeekdayRangeExpression,\n get_weekday_index,\n)\n\nMIN_VALUES = {\n \"year\": 1970,\n \"month\": 1,\n \"day\": 1,\n \"week\": 1,\n \"day_of_week\": 0,\n \"hour\": 0,\n \"minute\": 0,\n \"second\": 0,\n}\nMAX_VALUES = {\n \"year\": 9999,\n \"month\": 12,\n \"day\": 31,\n \"week\": 53,\n \"day_of_week\": 7,\n \"hour\": 23,\n \"minute\": 59,\n \"second\": 59,\n}\nDEFAULT_VALUES: Mapping[str, str | int] = {\n \"year\": \"*\",\n \"month\": 1,\n \"day\": 1,\n \"week\": \"*\",\n \"day_of_week\": \"*\",\n \"hour\": 0,\n \"minute\": 0,\n \"second\": 0,\n}\nSEPARATOR = re.compile(\" *, *\")\n\n\nclass BaseField:\n __slots__ = \"expressions\", \"name\"\n\n real: ClassVar[bool] = True\n compilers: ClassVar[Any] = (AllExpression, RangeExpression)\n\n def __init_subclass__(cls, real: bool = True, extra_compilers: Sequence = ()):\n cls.real = real\n if extra_compilers:\n cls.compilers += extra_compilers\n\n def __init__(self, name: str, exprs: int | str):\n self.name = name\n self.expressions: list = []\n for expr in SEPARATOR.split(str(exprs).strip()):\n self.append_expression(expr)\n\n def get_min(self, dateval: datetime) -> int:\n return MIN_VALUES[self.name]\n\n def get_max(self, dateval: datetime) -> int:\n return MAX_VALUES[self.name]\n\n def get_value(self, dateval: datetime) -> int:\n return getattr(dateval, self.name)\n\n def get_next_value(self, dateval: datetime) -> int | None:\n smallest = None\n for expr in self.expressions:\n value = expr.get_next_value(dateval, self)\n if smallest is None or (value is not None and value < smallest):\n smallest = value\n\n return smallest\n\n def append_expression(self, expr: str) -> None:\n for compiler in self.compilers:\n match = compiler.value_re.match(expr)\n if match:\n compiled_expr = compiler(**match.groupdict())\n\n try:\n compiled_expr.validate_range(\n self.name, MIN_VALUES[self.name], MAX_VALUES[self.name]\n )\n except ValueError as exc:\n raise ValueError(\n f\"Error validating expression {expr!r}: {exc}\"\n ) from exc\n\n self.expressions.append(compiled_expr)\n return\n\n raise ValueError(f\"Unrecognized expression {expr!r} for field {self.name!r}\")\n\n def __str__(self) -> str:\n expr_strings = (str(e) for e in self.expressions)\n return \",\".join(expr_strings)\n\n\nclass WeekField(BaseField, real=False):\n __slots__ = ()\n\n def get_value(self, dateval: datetime) -> int:\n return dateval.isocalendar()[1]\n\n\nclass DayOfMonthField(\n BaseField, extra_compilers=(WeekdayPositionExpression, LastDayOfMonthExpression)\n):\n __slots__ = ()\n\n def get_max(self, dateval: datetime) -> int:\n return monthrange(dateval.year, dateval.month)[1]\n\n\nclass DayOfWeekField(BaseField, real=False, extra_compilers=(WeekdayRangeExpression,)):\n __slots__ = ()\n\n def append_expression(self, expr: str) -> None:\n # Convert numeric weekday expressions into textual ones\n match = RangeExpression.value_re.match(expr)\n if match:\n groups = match.groups()\n first = int(groups[0]) - 1\n first = 6 if first < 0 else first\n if groups[1]:\n last = int(groups[1]) - 1\n last = 6 if last < 0 else last\n else:\n last = first\n\n expr = f\"{WEEKDAYS[first]}-{WEEKDAYS[last]}\"\n\n # For expressions like Sun-Tue or Sat-Mon, add two expressions that together\n # cover the expected weekdays\n match = WeekdayRangeExpression.value_re.match(expr)\n if match and match.groups()[1]:\n groups = match.groups()\n first_index = get_weekday_index(groups[0])\n last_index = get_weekday_index(groups[1])\n if first_index > last_index:\n super().append_expression(f\"{WEEKDAYS[0]}-{groups[1]}\")\n super().append_expression(f\"{groups[0]}-{WEEKDAYS[-1]}\")\n return\n\n super().append_expression(expr)\n\n def get_value(self, dateval: datetime) -> int:\n return dateval.weekday()\n\n\nclass MonthField(BaseField, extra_compilers=(MonthRangeExpression,)):\n __slots__ = ()\n" }, { "path": "src/apscheduler/triggers/date.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime\nfrom typing import Any\n\nimport attrs\nfrom attr.validators import instance_of\n\nfrom .._converters import as_aware_datetime\nfrom .._utils import require_state_version\nfrom ..abc import Trigger\n\n\n@attrs.define\nclass DateTrigger(Trigger):\n \"\"\"\n Triggers once on the given date/time.\n\n :param run_time: the date/time to run the job at\n \"\"\"\n\n run_time: datetime = attrs.field(\n converter=as_aware_datetime, validator=instance_of(datetime)\n )\n _completed: bool = attrs.field(init=False, eq=False, default=False)\n\n def next(self) -> datetime | None:\n if not self._completed:\n self._completed = True\n return self.run_time\n else:\n return None\n\n def __getstate__(self) -> dict[str, Any]:\n return {\n \"version\": 1,\n \"run_time\": self.run_time,\n \"completed\": self._completed,\n }\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n self.run_time = state[\"run_time\"]\n self._completed = state[\"completed\"]\n\n def __repr__(self) -> str:\n return f\"{self.__class__.__name__}('{self.run_time}')\"\n" }, { "path": "src/apscheduler/triggers/interval.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timedelta\nfrom typing import Any\n\nimport attrs\nfrom attr.validators import instance_of, optional\n\nfrom .._converters import as_aware_datetime\nfrom .._utils import require_state_version\nfrom ..abc import Trigger\n\n\n@attrs.define(kw_only=True)\nclass IntervalTrigger(Trigger):\n \"\"\"\n Triggers on specified intervals.\n\n The first trigger time is on ``start_time`` which is the moment the trigger was\n created unless specifically overridden. If ``end_time`` is specified, the last\n trigger time will be at or before that time. If no ``end_time`` has been given, the\n trigger will produce new trigger times as long as the resulting datetimes are valid\n datetimes in Python.\n\n :param weeks: number of weeks to wait\n :param days: number of days to wait\n :param hours: number of hours to wait\n :param minutes: number of minutes to wait\n :param seconds: number of seconds to wait\n :param microseconds: number of microseconds to wait\n :param start_time: first trigger date/time (defaults to current date/time if\n omitted)\n :param end_time: latest possible date/time to trigger on\n \"\"\"\n\n weeks: float = 0\n days: float = 0\n hours: float = 0\n minutes: float = 0\n seconds: float = 0\n microseconds: float = 0\n start_time: datetime = attrs.field(\n converter=as_aware_datetime,\n factory=datetime.now,\n validator=instance_of(datetime),\n )\n end_time: datetime | None = attrs.field(\n converter=as_aware_datetime,\n validator=optional(instance_of(datetime)),\n default=None,\n )\n _interval: timedelta = attrs.field(init=False, eq=False, repr=False)\n _last_fire_time: datetime | None = attrs.field(\n init=False, eq=False, converter=as_aware_datetime, default=None\n )\n\n def __attrs_post_init__(self) -> None:\n self._interval = timedelta(\n weeks=self.weeks,\n days=self.days,\n hours=self.hours,\n minutes=self.minutes,\n seconds=self.seconds,\n microseconds=self.microseconds,\n )\n\n if self._interval.total_seconds() <= 0:\n raise ValueError(\"The time interval must be positive\")\n\n if self.end_time and self.end_time < self.start_time:\n raise ValueError(\"end_time cannot be earlier than start_time\")\n\n def next(self) -> datetime | None:\n if self._last_fire_time is None:\n self._last_fire_time = self.start_time\n else:\n self._last_fire_time += self._interval\n\n if self.end_time is None or self._last_fire_time <= self.end_time:\n return self._last_fire_time\n else:\n return None\n\n def __getstate__(self) -> dict[str, Any]:\n return {\n \"version\": 1,\n \"interval\": [\n self.weeks,\n self.days,\n self.hours,\n self.minutes,\n self.seconds,\n self.microseconds,\n ],\n \"start_time\": self.start_time,\n \"end_time\": self.end_time,\n \"last_fire_time\": self._last_fire_time,\n }\n\n def __setstate__(self, state: dict[str, Any]) -> None:\n require_state_version(self, state, 1)\n (\n self.weeks,\n self.days,\n self.hours,\n self.minutes,\n self.seconds,\n self.microseconds,\n ) = state[\"interval\"]\n self.start_time = state[\"start_time\"]\n self.end_time = state[\"end_time\"]\n self._last_fire_time = state[\"last_fire_time\"]\n self._interval = timedelta(\n weeks=self.weeks,\n days=self.days,\n hours=self.hours,\n minutes=self.minutes,\n seconds=self.seconds,\n microseconds=self.microseconds,\n )\n\n def __repr__(self) -> str:\n fields = []\n for field in \"weeks\", \"days\", \"hours\", \"minutes\", \"seconds\", \"microseconds\":\n value = getattr(self, field)\n if value > 0:\n fields.append(f\"{field}={value}\")\n\n fields.append(f\"start_time='{self.start_time}'\")\n if self.end_time:\n fields.append(f\"end_time='{self.end_time}'\")\n\n return f\"{self.__class__.__name__}({', '.join(fields)})\"\n" }, { "path": "tests/conftest.py", "content": "from __future__ import annotations\n\nimport logging\nfrom collections.abc import AsyncGenerator, Generator\nfrom contextlib import AsyncExitStack\nfrom logging import Logger\nfrom pathlib import Path\nfrom typing import Any, cast\nfrom zoneinfo import ZoneInfo\n\nimport pytest\nfrom _pytest.fixtures import SubRequest\nfrom pytest_lazy_fixtures import lf\n\nfrom apscheduler.abc import DataStore, EventBroker, Serializer\nfrom apscheduler.datastores.memory import MemoryDataStore\nfrom apscheduler.serializers.cbor import CBORSerializer\nfrom apscheduler.serializers.json import JSONSerializer\nfrom apscheduler.serializers.pickle import PickleSerializer\n\n\n@pytest.fixture(scope=\"session\")\ndef timezone() -> ZoneInfo:\n return ZoneInfo(\"Europe/Berlin\")\n\n\n@pytest.fixture(scope=\"session\")\ndef utc_timezone() -> ZoneInfo:\n return ZoneInfo(\"UTC\")\n\n\n@pytest.fixture(\n params=[\n pytest.param(PickleSerializer, id=\"pickle\"),\n pytest.param(CBORSerializer, id=\"cbor\"),\n pytest.param(JSONSerializer, id=\"json\"),\n ]\n)\ndef serializer(request) -> Serializer | None:\n return request.param() if request.param else None\n\n\n@pytest.fixture\ndef anyio_backend() -> str:\n return \"asyncio\"\n\n\n@pytest.fixture\ndef local_broker() -> EventBroker:\n from apscheduler.eventbrokers.local import LocalEventBroker\n\n return LocalEventBroker()\n\n\n@pytest.fixture\nasync def redis_broker(serializer: Serializer) -> EventBroker:\n from apscheduler.eventbrokers.redis import RedisEventBroker\n\n broker = RedisEventBroker(\n \"redis://localhost:6379\", serializer=serializer, stop_check_interval=0.05\n )\n await broker._client.flushdb()\n return broker\n\n\n@pytest.fixture\ndef mqtt_broker(serializer: Serializer) -> EventBroker:\n from apscheduler.eventbrokers.mqtt import MQTTEventBroker\n\n return MQTTEventBroker(serializer=serializer)\n\n\n@pytest.fixture\nasync def asyncpg_broker(serializer: Serializer) -> EventBroker:\n pytest.importorskip(\"asyncpg\", reason=\"asyncpg is not installed\")\n from apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\n\n broker = AsyncpgEventBroker(\n \"postgres://postgres:secret@localhost:5432/testdb\", serializer=serializer\n )\n return broker\n\n\n@pytest.fixture\nasync def psycopg_broker(serializer: Serializer) -> EventBroker:\n pytest.importorskip(\"psycopg\", reason=\"psycopg is not installed\")\n from apscheduler.eventbrokers.psycopg import PsycopgEventBroker\n\n broker = PsycopgEventBroker(\n \"postgres://postgres:secret@localhost:5432/testdb\", serializer=serializer\n )\n return broker\n\n\n@pytest.fixture(\n params=[\n pytest.param(lf(\"local_broker\"), id=\"local\"),\n pytest.param(\n lf(\"asyncpg_broker\"),\n id=\"asyncpg\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"psycopg_broker\"),\n id=\"psycopg\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"redis_broker\"),\n id=\"redis\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"mqtt_broker\"), id=\"mqtt\", marks=[pytest.mark.external_service]\n ),\n ]\n)\nasync def raw_event_broker(request: SubRequest) -> EventBroker:\n return cast(EventBroker, request.param)\n\n\n@pytest.fixture\nasync def event_broker(\n raw_event_broker: EventBroker, logger: Logger\n) -> AsyncGenerator[EventBroker, Any]:\n async with AsyncExitStack() as exit_stack:\n await raw_event_broker.start(exit_stack, logger)\n yield raw_event_broker\n\n\n@pytest.fixture\ndef memory_store() -> Generator[DataStore, None, None]:\n yield MemoryDataStore()\n\n\n@pytest.fixture\nasync def mongodb_store() -> AsyncGenerator[DataStore, Any]:\n from pymongo.asynchronous.mongo_client import AsyncMongoClient\n\n from apscheduler.datastores.mongodb import MongoDBDataStore\n\n async with AsyncMongoClient(tz_aware=True, serverSelectionTimeoutMS=1000) as client:\n yield MongoDBDataStore(client, start_from_scratch=True)\n\n\n@pytest.fixture\nasync def psycopg_async_store() -> AsyncGenerator[DataStore, None]:\n from sqlalchemy import text\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n engine = create_async_engine(\n \"postgresql+psycopg://postgres:secret@localhost/testdb\"\n )\n try:\n async with engine.begin() as conn:\n await conn.execute(text(\"CREATE SCHEMA IF NOT EXISTS psycopg_async\"))\n\n yield SQLAlchemyDataStore(\n engine, schema=\"psycopg_async\", start_from_scratch=True\n )\n assert \"Current Checked out connections: 0\" in engine.pool.status()\n finally:\n await engine.dispose()\n\n\n@pytest.fixture\ndef psycopg_sync_store() -> Generator[DataStore, None, None]:\n from sqlalchemy import create_engine, text\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n engine = create_engine(\"postgresql+psycopg://postgres:secret@localhost/testdb\")\n try:\n with engine.begin() as conn:\n conn.execute(text(\"CREATE SCHEMA IF NOT EXISTS psycopg_sync\"))\n\n yield SQLAlchemyDataStore(\n engine, schema=\"psycopg_sync\", start_from_scratch=True\n )\n assert \"Current Checked out connections: 0\" in engine.pool.status()\n finally:\n engine.dispose()\n\n\n@pytest.fixture\ndef pymysql_store() -> Generator[DataStore, None, None]:\n from sqlalchemy import create_engine\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n engine = create_engine(\"mysql+pymysql://root:secret@localhost/testdb\")\n try:\n yield SQLAlchemyDataStore(engine, start_from_scratch=True)\n assert \"Current Checked out connections: 0\" in engine.pool.status()\n finally:\n engine.dispose()\n\n\n@pytest.fixture\nasync def aiosqlite_store(tmp_path: Path) -> AsyncGenerator[DataStore, None]:\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n engine = create_async_engine(f\"sqlite+aiosqlite:///{tmp_path}/test.db\")\n try:\n yield SQLAlchemyDataStore(engine)\n finally:\n await engine.dispose()\n\n\n@pytest.fixture\nasync def asyncpg_store() -> AsyncGenerator[DataStore, None]:\n pytest.importorskip(\"asyncpg\", reason=\"asyncpg is not installed\")\n from asyncpg import compat\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n # Workaround for AnyIO 4.0.0rc1 compatibility\n async def patched_wait_for(fut, timeout):\n import asyncio\n\n if timeout is None:\n return await fut\n fut = asyncio.ensure_future(fut)\n try:\n return await asyncio.wait_for(fut, timeout)\n except asyncio.CancelledError:\n if fut.done() and not fut.cancelled():\n return fut.result()\n else:\n raise\n\n compat.wait_for = patched_wait_for\n\n engine = create_async_engine(\n \"postgresql+asyncpg://postgres:secret@localhost/testdb\", future=True\n )\n try:\n yield SQLAlchemyDataStore(engine, start_from_scratch=True)\n assert \"Current Checked out connections: 0\" in engine.pool.status()\n finally:\n await engine.dispose()\n\n\n@pytest.fixture\nasync def asyncmy_store() -> AsyncGenerator[DataStore, None]:\n pytest.importorskip(\"asyncmy\", reason=\"asyncmy is not installed\")\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\n\n engine = create_async_engine(\n \"mysql+asyncmy://root:secret@localhost/testdb?charset=utf8mb4\", future=True\n )\n try:\n yield SQLAlchemyDataStore(engine, start_from_scratch=True)\n assert \"Current Checked out connections: 0\" in engine.pool.status()\n finally:\n await engine.dispose()\n\n\n@pytest.fixture(\n params=[\n pytest.param(\n lf(\"memory_store\"),\n id=\"memory\",\n ),\n pytest.param(\n lf(\"aiosqlite_store\"),\n id=\"aiosqlite\",\n ),\n pytest.param(\n lf(\"asyncpg_store\"),\n id=\"asyncpg\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"asyncmy_store\"),\n id=\"asyncmy\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"psycopg_async_store\"),\n id=\"psycopg_async\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"psycopg_sync_store\"),\n id=\"psycopg_sync\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"pymysql_store\"),\n id=\"pymysql\",\n marks=[pytest.mark.external_service],\n ),\n pytest.param(\n lf(\"mongodb_store\"),\n id=\"mongodb\",\n marks=[pytest.mark.external_service],\n ),\n ]\n)\nasync def raw_datastore(request: SubRequest) -> DataStore:\n return cast(DataStore, request.param)\n\n\n@pytest.fixture(scope=\"session\")\ndef logger() -> Logger:\n return logging.getLogger(\"apscheduler\")\n" }, { "path": "tests/test_datastores.py", "content": "from __future__ import annotations\n\nimport platform\nfrom collections.abc import AsyncGenerator\nfrom contextlib import AsyncExitStack, asynccontextmanager\nfrom datetime import datetime, timedelta, timezone\nfrom logging import Logger\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING\nfrom unittest.mock import Mock\n\nimport anyio\nimport pytest\nfrom anyio import CancelScope\nfrom pytest_mock.plugin import MockerFixture\n\nfrom apscheduler import (\n CoalescePolicy,\n ConflictPolicy,\n DeserializationError,\n Event,\n Job,\n JobOutcome,\n JobResult,\n Schedule,\n ScheduleAdded,\n ScheduleRemoved,\n ScheduleUpdated,\n Task,\n TaskAdded,\n TaskLookupError,\n TaskUpdated,\n)\nfrom apscheduler._structures import ScheduleResult\nfrom apscheduler.abc import DataStore, EventBroker, Serializer\nfrom apscheduler.datastores.base import BaseExternalDataStore\nfrom apscheduler.datastores.memory import MemoryDataStore\nfrom apscheduler.datastores.mongodb import MongoDBDataStore\nfrom apscheduler.datastores.sqlalchemy import SQLAlchemyDataStore\nfrom apscheduler.triggers.date import DateTrigger\n\nif TYPE_CHECKING:\n from time_machine import TimeMachineFixture\n\npytestmark = pytest.mark.anyio\n\n\n@pytest.fixture\nasync def datastore(\n raw_datastore: DataStore, local_broker: EventBroker, logger: Logger\n) -> AsyncGenerator[DataStore, None]:\n async with AsyncExitStack() as exit_stack:\n await local_broker.start(exit_stack, logger)\n await raw_datastore.start(exit_stack, local_broker, logger)\n yield raw_datastore\n\n\n@pytest.fixture\ndef schedules() -> list[Schedule]:\n trigger = DateTrigger(datetime(2020, 9, 13, tzinfo=timezone.utc))\n schedule1 = Schedule(\n id=\"s1\", task_id=\"task1\", job_executor=\"async\", trigger=trigger\n )\n schedule1.next_fire_time = trigger.next()\n\n trigger = DateTrigger(datetime(2020, 9, 14, tzinfo=timezone.utc))\n schedule2 = Schedule(\n id=\"s2\", task_id=\"task2\", job_executor=\"async\", trigger=trigger\n )\n schedule2.next_fire_time = trigger.next()\n\n trigger = DateTrigger(datetime(2020, 9, 15, tzinfo=timezone.utc))\n schedule3 = Schedule(\n id=\"s3\", task_id=\"task1\", job_executor=\"async\", trigger=trigger\n )\n schedule3.next_fire_time = trigger.next()\n\n return [schedule1, schedule2, schedule3]\n\n\n@asynccontextmanager\nasync def capture_events(\n datastore: DataStore,\n limit: int,\n event_types: set[type[Event]] | None = None,\n) -> AsyncGenerator[list[Event], None]:\n def listener(event: Event) -> None:\n events.append(event)\n if len(events) == limit:\n limit_event.set()\n subscription.unsubscribe()\n\n events: list[Event] = []\n limit_event = anyio.Event()\n subscription = datastore._event_broker.subscribe(listener, event_types)\n yield events\n if limit:\n with anyio.fail_after(3):\n await limit_event.wait()\n\n\nasync def test_add_replace_task(datastore: DataStore) -> None:\n event_types = {TaskAdded, TaskUpdated}\n async with capture_events(datastore, 3, event_types) as events:\n await datastore.add_task(\n Task(id=\"test_task\", func=\"builtins:print\", job_executor=\"async\")\n )\n await datastore.add_task(\n Task(id=\"test_task2\", func=\"math:ceil\", job_executor=\"async\")\n )\n await datastore.add_task(\n Task(id=\"test_task\", func=\"builtins:repr\", job_executor=\"async\")\n )\n\n tasks = await datastore.get_tasks()\n assert len(tasks) == 2\n assert tasks[0].id == \"test_task\"\n assert tasks[0].func == \"builtins:repr\"\n assert tasks[1].id == \"test_task2\"\n assert tasks[1].func == \"math:ceil\"\n\n received_event = events.pop(0)\n assert isinstance(received_event, TaskAdded)\n assert received_event.task_id == \"test_task\"\n\n received_event = events.pop(0)\n assert isinstance(received_event, TaskAdded)\n assert received_event.task_id == \"test_task2\"\n\n received_event = events.pop(0)\n assert isinstance(received_event, TaskUpdated)\n assert received_event.task_id == \"test_task\"\n\n assert not events\n\n\nasync def test_add_schedules(datastore: DataStore, schedules: list[Schedule]) -> None:\n async with capture_events(datastore, 3, {ScheduleAdded}) as events:\n for schedule in schedules:\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n assert await datastore.get_schedules() == schedules\n assert await datastore.get_schedules({\"s1\", \"s2\", \"s3\"}) == schedules\n assert await datastore.get_schedules({\"s1\"}) == [schedules[0]]\n assert await datastore.get_schedules({\"s2\"}) == [schedules[1]]\n assert await datastore.get_schedules({\"s3\"}) == [schedules[2]]\n\n for event, schedule in zip(events, schedules, strict=True):\n assert isinstance(event, ScheduleAdded)\n assert event.schedule_id == schedule.id\n assert event.task_id == schedule.task_id\n assert event.next_fire_time == schedule.next_fire_time\n\n\nasync def test_replace_schedules(\n datastore: DataStore, schedules: list[Schedule]\n) -> None:\n async with capture_events(datastore, 1, {ScheduleUpdated}) as events:\n for schedule in schedules:\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n trigger = DateTrigger(datetime(2020, 9, 16, tzinfo=timezone.utc))\n next_fire_time = trigger.next()\n schedule = Schedule(\n id=\"s3\",\n task_id=\"foo\",\n trigger=trigger,\n args=(),\n kwargs={},\n job_executor=\"async\",\n coalesce=CoalescePolicy.earliest,\n misfire_grace_time=None,\n )\n schedule.next_fire_time = next_fire_time\n await datastore.add_schedule(schedule, ConflictPolicy.replace)\n\n schedules = await datastore.get_schedules({schedule.id})\n assert schedules[0].task_id == \"foo\"\n assert schedules[0].next_fire_time == next_fire_time\n assert schedules[0].args == ()\n assert schedules[0].kwargs == {}\n assert schedules[0].coalesce is CoalescePolicy.earliest\n assert schedules[0].misfire_grace_time is None\n\n received_event = events.pop(0)\n assert isinstance(received_event, ScheduleUpdated)\n assert received_event.schedule_id == \"s3\"\n assert received_event.task_id == \"foo\"\n assert received_event.next_fire_time == datetime(2020, 9, 16, tzinfo=timezone.utc)\n assert not events\n\n\nasync def test_remove_schedules(\n datastore: DataStore, schedules: list[Schedule]\n) -> None:\n async with capture_events(datastore, 2, {ScheduleRemoved}) as events:\n for schedule in schedules:\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n await datastore.remove_schedules([\"s1\", \"s2\"])\n assert await datastore.get_schedules() == [schedules[2]]\n\n received_event = events.pop(0)\n assert isinstance(received_event, ScheduleRemoved)\n assert received_event.schedule_id == \"s1\"\n\n received_event = events.pop(0)\n assert isinstance(received_event, ScheduleRemoved)\n assert received_event.schedule_id == \"s2\"\n\n assert not events\n\n\n@pytest.mark.skipif(\n platform.python_implementation() != \"CPython\",\n reason=\"time-machine is not available\",\n)\nasync def test_acquire_release_schedules(\n datastore: DataStore, schedules: list[Schedule], time_machine: TimeMachineFixture\n) -> None:\n time_machine.move_to(datetime(2020, 9, 14, tzinfo=timezone.utc))\n\n event_types = {ScheduleRemoved, ScheduleUpdated}\n async with capture_events(datastore, 2, event_types) as events:\n for schedule in schedules:\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n # The first scheduler gets the first due schedule\n schedules1 = await datastore.acquire_schedules(\n \"dummy-id1\", timedelta(seconds=30), 1\n )\n assert len(schedules1) == 1\n assert schedules1[0].id == \"s1\"\n\n # The second scheduler gets the second due schedule\n schedules2 = await datastore.acquire_schedules(\n \"dummy-id2\",\n timedelta(seconds=30),\n 1,\n )\n assert len(schedules2) == 1\n assert schedules2[0].id == \"s2\"\n\n # The third scheduler gets nothing\n schedules3 = await datastore.acquire_schedules(\n \"dummy-id3\",\n timedelta(seconds=30),\n 1,\n )\n assert not schedules3\n\n # Update the schedules and check that the job store actually deletes the\n # first one and updates the second one\n results1: list[ScheduleResult] = []\n results2: list[ScheduleResult] = []\n results1.append(\n ScheduleResult(\n schedule_id=schedules1[0].id,\n task_id=schedules1[0].task_id,\n trigger=schedules1[0].trigger,\n last_fire_time=datetime(2020, 9, 14, tzinfo=timezone.utc),\n next_fire_time=None,\n )\n )\n results2.append(\n ScheduleResult(\n schedule_id=schedules2[0].id,\n task_id=schedules2[0].task_id,\n trigger=schedules1[0].trigger,\n last_fire_time=datetime(2020, 9, 14, tzinfo=timezone.utc),\n next_fire_time=datetime(2020, 9, 15, tzinfo=timezone.utc),\n )\n )\n\n # Release all the schedules\n await datastore.release_schedules(\"dummy-id1\", results1)\n await datastore.release_schedules(\"dummy-id2\", results2)\n\n # Check that the first schedule has its next fire time nullified\n schedules = await datastore.get_schedules()\n assert len(schedules) == 3\n schedules.sort(key=lambda s: s.id)\n assert schedules[0].id == \"s1\"\n assert schedules[0].last_fire_time == datetime(2020, 9, 14, tzinfo=timezone.utc)\n assert schedules[0].next_fire_time is None\n assert schedules[1].id == \"s2\"\n assert schedules[1].last_fire_time == datetime(2020, 9, 14, tzinfo=timezone.utc)\n assert schedules[1].next_fire_time == datetime(2020, 9, 15, tzinfo=timezone.utc)\n assert schedules[2].id == \"s3\"\n assert schedules[2].last_fire_time is None\n assert schedules[2].next_fire_time == datetime(2020, 9, 15, tzinfo=timezone.utc)\n\n # Check for the appropriate update and delete events\n received_event = events.pop(0)\n assert isinstance(received_event, ScheduleUpdated)\n assert received_event.schedule_id == \"s1\"\n assert received_event.next_fire_time is None\n\n received_event = events.pop(0)\n assert isinstance(received_event, ScheduleUpdated)\n assert received_event.schedule_id == \"s2\"\n assert received_event.next_fire_time == datetime(2020, 9, 15, tzinfo=timezone.utc)\n\n assert not events\n\n\nasync def test_release_schedule_two_identical_fire_times(datastore: DataStore) -> None:\n \"\"\"Regression test for #616.\"\"\"\n for i in range(1, 3):\n trigger = DateTrigger(datetime(2020, 9, 13, tzinfo=timezone.utc))\n schedule = Schedule(\n id=f\"s{i}\", task_id=\"task1\", job_executor=\"async\", trigger=trigger\n )\n schedule.next_fire_time = trigger.next()\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n schedules = await datastore.acquire_schedules(\n \"foo\",\n timedelta(seconds=30),\n 3,\n )\n results = [\n ScheduleResult(\n schedule_id=schedules[0].id,\n task_id=schedules[0].task_id,\n trigger=schedules[0].trigger,\n last_fire_time=datetime(2020, 9, 10, tzinfo=timezone.utc),\n next_fire_time=None,\n ),\n ]\n await datastore.release_schedules(\"foo\", results)\n\n remaining = await datastore.get_schedules({s.id for s in schedules})\n assert len(remaining) == 2\n remaining.sort(key=lambda s: s.id)\n assert remaining[0].id == schedules[0].id\n assert remaining[0].next_fire_time is None\n assert remaining[1].id == schedules[1].id\n assert remaining[1].next_fire_time\n\n\nasync def test_release_two_schedules_at_once(datastore: DataStore) -> None:\n \"\"\"Regression test for #621.\"\"\"\n for i in range(2):\n trigger = DateTrigger(datetime(2020, 9, 13, tzinfo=timezone.utc))\n schedule = Schedule(\n id=f\"s{i}\", task_id=\"task1\", job_executor=\"async\", trigger=trigger\n )\n schedule.next_fire_time = trigger.next()\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n schedules = await datastore.acquire_schedules(\"foo\", timedelta(seconds=30), 3)\n results = [\n ScheduleResult(\n schedule_id=schedules[0].id,\n task_id=schedules[0].task_id,\n trigger=schedules[0].trigger,\n last_fire_time=datetime(2020, 9, 10, tzinfo=timezone.utc),\n next_fire_time=None,\n ),\n ]\n await datastore.release_schedules(\"foo\", results)\n\n remaining = await datastore.get_schedules({s.id for s in schedules})\n assert len(remaining) == 2\n\n\n@pytest.mark.skipif(\n platform.python_implementation() != \"CPython\",\n reason=\"time-machine is not available\",\n)\nasync def test_acquire_schedules_lock_timeout(\n datastore: DataStore,\n schedules: list[Schedule],\n time_machine: TimeMachineFixture,\n) -> None:\n \"\"\"\n Test that a scheduler can acquire schedules that were acquired by another\n scheduler but not released within the lock timeout period.\n\n \"\"\"\n time_machine.move_to(datetime.now(timezone.utc), tick=False)\n await datastore.add_schedule(schedules[0], ConflictPolicy.exception)\n\n # First, one scheduler acquires the first available schedule\n acquired1 = await datastore.acquire_schedules(\n \"dummy-id1\",\n timedelta(seconds=30),\n 1,\n )\n assert len(acquired1) == 1\n assert acquired1[0].id == \"s1\"\n\n # Try to acquire the schedule just at the threshold (now == acquired_until).\n # This should not yield any schedules.\n time_machine.shift(30)\n acquired2 = await datastore.acquire_schedules(\n \"dummy-id2\",\n timedelta(seconds=30),\n 1,\n )\n assert not acquired2\n\n # Right after that, the schedule should be available\n time_machine.shift(1)\n acquired3 = await datastore.acquire_schedules(\n \"dummy-id2\",\n timedelta(seconds=30),\n 1,\n )\n assert len(acquired3) == 1\n assert acquired3[0].id == \"s1\"\n\n\nasync def test_acquire_multiple_workers(datastore: DataStore) -> None:\n await datastore.add_task(\n Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n jobs = [Job(task_id=\"task1\", executor=\"async\") for _ in range(2)]\n for job in jobs:\n await datastore.add_job(job)\n\n # The first worker gets the first job in the queue\n jobs1 = await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 1)\n assert len(jobs1) == 1\n assert jobs1[0].id == jobs[0].id\n\n # The second worker gets the second job\n jobs2 = await datastore.acquire_jobs(\"worker2\", timedelta(seconds=30), 1)\n assert len(jobs2) == 1\n assert jobs2[0].id == jobs[1].id\n\n # The third worker gets nothing\n jobs3 = await datastore.acquire_jobs(\"worker3\", timedelta(seconds=30), 1)\n assert not jobs3\n\n\nasync def test_job_release_success(datastore: DataStore) -> None:\n await datastore.add_task(\n Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(minutes=1)\n )\n await datastore.add_job(job)\n\n acquired = await datastore.acquire_jobs(\"worker_id\", timedelta(seconds=30), 2)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n await datastore.release_job(\n \"worker_id\",\n acquired[0],\n JobResult.from_job(\n acquired[0],\n JobOutcome.success,\n return_value=\"foo\",\n ),\n )\n result = await datastore.get_job_result(acquired[0].id)\n assert result\n assert result.outcome is JobOutcome.success\n assert result.exception is None\n assert result.return_value == \"foo\"\n\n # Check that the job and its result are gone\n assert not await datastore.get_jobs({acquired[0].id})\n assert not await datastore.get_job_result(acquired[0].id)\n\n\nasync def test_job_release_failure(datastore: DataStore) -> None:\n await datastore.add_task(\n Task(id=\"task1\", job_executor=\"async\", func=\"contextlib:asynccontextmanager\")\n )\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(minutes=1)\n )\n await datastore.add_job(job)\n\n acquired = await datastore.acquire_jobs(\"worker_id\", timedelta(seconds=30), 2)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n await datastore.release_job(\n \"worker_id\",\n acquired[0],\n JobResult.from_job(\n acquired[0],\n JobOutcome.error,\n exception=ValueError(\"foo\"),\n ),\n )\n result = await datastore.get_job_result(acquired[0].id)\n assert result\n assert result.outcome is JobOutcome.error\n assert isinstance(result.exception, ValueError)\n assert result.exception.args == (\"foo\",)\n assert result.return_value is None\n\n # Check that the job and its result are gone\n assert not await datastore.get_jobs({acquired[0].id})\n assert not await datastore.get_job_result(acquired[0].id)\n\n\nasync def test_job_release_missed_deadline(datastore: DataStore):\n await datastore.add_task(\n Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(minutes=1)\n )\n await datastore.add_job(job)\n\n acquired = await datastore.acquire_jobs(\"worker_id\", timedelta(seconds=30), 2)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n await datastore.release_job(\n \"worker_id\",\n acquired[0],\n JobResult.from_job(\n acquired[0],\n JobOutcome.missed_start_deadline,\n ),\n )\n result = await datastore.get_job_result(acquired[0].id)\n assert result\n assert result.outcome is JobOutcome.missed_start_deadline\n assert result.exception is None\n assert result.return_value is None\n\n # Check that the job and its result are gone\n assert not await datastore.get_jobs({acquired[0].id})\n assert not await datastore.get_job_result(acquired[0].id)\n\n\nasync def test_job_release_cancelled(datastore: DataStore) -> None:\n await datastore.add_task(\n Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(minutes=1)\n )\n await datastore.add_job(job)\n\n acquired = await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 2)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n await datastore.release_job(\n \"worker1\",\n acquired[0],\n JobResult.from_job(acquired[0], JobOutcome.cancelled),\n )\n result = await datastore.get_job_result(acquired[0].id)\n assert result\n assert result.outcome is JobOutcome.cancelled\n assert result.exception is None\n assert result.return_value is None\n\n # Check that the job and its result are gone\n assert not await datastore.get_jobs({acquired[0].id})\n assert not await datastore.get_job_result(acquired[0].id)\n\n\n@pytest.mark.skipif(\n platform.python_implementation() != \"CPython\",\n reason=\"time-machine is not available\",\n)\nasync def test_acquire_jobs_lock_timeout(\n datastore: DataStore, time_machine: TimeMachineFixture\n) -> None:\n \"\"\"\n Test that a worker can acquire jobs that were acquired by another scheduler but\n not released within the lock timeout period.\n\n \"\"\"\n await datastore.add_task(\n Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(minutes=1)\n )\n await datastore.add_job(job)\n\n # First, one worker acquires the first available job\n time_machine.move_to(datetime.now(timezone.utc), tick=False)\n acquired = await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 1)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n # Try to acquire the job just at the threshold (now == acquired_until).\n # This should not yield any jobs.\n time_machine.shift(30)\n assert not await datastore.acquire_jobs(\"worker2\", timedelta(seconds=30), 1)\n\n # Right after that, the job should be available\n time_machine.shift(1)\n acquired = await datastore.acquire_jobs(\"worker2\", timedelta(seconds=30), 1)\n assert len(acquired) == 1\n assert acquired[0].id == job.id\n\n\nasync def test_acquire_jobs_max_number_exceeded(datastore: DataStore) -> None:\n await datastore.add_task(\n Task(\n id=\"task1\",\n func=\"contextlib:asynccontextmanager\",\n job_executor=\"async\",\n max_running_jobs=2,\n )\n )\n assert (await datastore.get_task(\"task1\")).running_jobs == 0\n\n jobs = [\n Job(task_id=\"task1\", executor=\"async\"),\n Job(task_id=\"task1\", executor=\"async\"),\n Job(task_id=\"task1\", executor=\"async\"),\n ]\n for job in jobs:\n await datastore.add_job(job)\n\n # Check that only 2 jobs are returned from acquire_jobs() even though the limit\n # was 3\n acquired_jobs = await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 3)\n assert len(acquired_jobs) == 2\n assert [job.id for job in acquired_jobs] == [job.id for job in jobs[:2]]\n assert (await datastore.get_task(\"task1\")).running_jobs == 2\n for job in acquired_jobs:\n assert job.acquired_by == \"worker1\"\n assert job.acquired_until\n\n # Check that no jobs are acquired now that the task is at capacity\n assert not await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 3)\n\n # Release one job, and the worker should be able to acquire the third job\n await datastore.release_job(\n \"worker1\",\n acquired_jobs[0],\n JobResult.from_job(\n acquired_jobs[0],\n JobOutcome.success,\n return_value=None,\n ),\n )\n assert (await datastore.get_task(\"task1\")).running_jobs == 1\n remaining_jobs = await datastore.get_jobs()\n assert len(remaining_jobs) == 2\n\n acquired_jobs = await datastore.acquire_jobs(\"worker1\", timedelta(seconds=30), 3)\n assert [job.id for job in acquired_jobs] == [jobs[2].id]\n assert (await datastore.get_task(\"task1\")).running_jobs == 2\n\n\nasync def test_add_get_task(datastore: DataStore) -> None:\n with pytest.raises(TaskLookupError):\n await datastore.get_task(\"dummyid\")\n\n await datastore.add_task(\n Task(id=\"dummyid\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n )\n task = await datastore.get_task(\"dummyid\")\n assert task.id == \"dummyid\"\n assert task.func == \"contextlib:asynccontextmanager\"\n\n\nasync def test_cancel_start(\n raw_datastore: DataStore, local_broker: EventBroker, logger: Logger\n) -> None:\n with CancelScope() as scope:\n scope.cancel()\n async with AsyncExitStack() as exit_stack:\n await raw_datastore.start(exit_stack, local_broker, logger)\n\n\nasync def test_cancel_stop(\n raw_datastore: DataStore, local_broker: EventBroker, logger: Logger\n) -> None:\n with CancelScope() as scope:\n async with AsyncExitStack() as exit_stack:\n await raw_datastore.start(exit_stack, local_broker, logger)\n scope.cancel()\n\n\nasync def test_next_schedule_run_time(datastore: DataStore, schedules: list[Schedule]):\n next_schedule_run_time = await datastore.get_next_schedule_run_time()\n assert next_schedule_run_time is None\n\n for schedule in schedules:\n await datastore.add_schedule(schedule, ConflictPolicy.exception)\n\n next_schedule_run_time = await datastore.get_next_schedule_run_time()\n assert next_schedule_run_time == datetime(2020, 9, 13, tzinfo=timezone.utc)\n\n\n@pytest.mark.skipif(\n platform.python_implementation() != \"CPython\",\n reason=\"time-machine is not available\",\n)\nasync def test_extend_acquired_schedule_leases(\n datastore: DataStore, time_machine: TimeMachineFixture, schedules: list[Schedule]\n) -> None:\n \"\"\"\n Test that the leases on acquired schedules are updated to prevent other schedulers\n from acquiring them.\n\n \"\"\"\n time_machine.move_to(datetime(2020, 9, 14, tzinfo=timezone.utc))\n\n # Add a schedule to the data store\n await datastore.add_schedule(schedules[0], ConflictPolicy.exception)\n\n # Acquire the schedule\n schedules = await datastore.acquire_schedules(\n \"scheduler_id\",\n timedelta(seconds=30),\n 1,\n )\n assert len(schedules) == 1\n\n # Move 20 seconds forward, then call extend_acquired_schedule_leases(). This should\n # set the acquired_until timestamp to 30 seconds from the new current time.\n time_machine.shift(20)\n await datastore.extend_acquired_schedule_leases(\n \"scheduler_id\", {schedules[0].id}, timedelta(seconds=30)\n )\n\n # The schedule was acquired by scheduler_id so scheduler2_id should not be able to\n # acquire it, as it's still within the original lock expiration delay\n assert not await datastore.acquire_schedules(\n \"scheduler2_id\",\n timedelta(seconds=30),\n 1,\n )\n\n # Move 20 more seconds forward (beyond the initial lock expiration delay), then try\n # to have the second scheduler acquire the schedule again. This should also fail.\n time_machine.shift(20)\n assert not await datastore.acquire_schedules(\n \"scheduler2_id\",\n timedelta(seconds=30),\n 1,\n )\n\n # Move 20 more seconds forward - this time the schedule should be available\n time_machine.shift(20)\n schedules = await datastore.acquire_schedules(\n \"scheduler2_id\",\n timedelta(seconds=30),\n 1,\n )\n assert len(schedules) == 1\n\n\n@pytest.mark.skipif(\n platform.python_implementation() != \"CPython\",\n reason=\"time-machine is not available\",\n)\nasync def test_extend_acquired_job_leases(\n datastore: DataStore, time_machine: TimeMachineFixture\n) -> None:\n \"\"\"\n Test that the leases on acquired jobs are updated to prevent them from being cleaned\n up as if they had been abandoned.\n\n \"\"\"\n time_machine.move_to(datetime(2020, 9, 14, tzinfo=timezone.utc))\n\n # Add a task to the data store\n task = Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n await datastore.add_task(task)\n\n # Add a job to the data store\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(seconds=30)\n )\n await datastore.add_job(job)\n\n # Acquire the job\n jobs = await datastore.acquire_jobs(\"scheduler_id\", timedelta(seconds=30), 1)\n assert len(jobs) == 1\n\n # Move 20 seconds forward, then call extend_acquired_job_leases(). This should set\n # the acquired_until timestamp to 30 seconds from the new current time.\n time_machine.shift(20)\n await datastore.extend_acquired_job_leases(\n \"scheduler_id\", {job.id}, timedelta(seconds=30)\n )\n\n # The job was acquired by scheduler_id so scheduler2_id should not be able to\n # acquire it\n assert not await datastore.acquire_jobs(\"scheduler2_id\", timedelta(seconds=30), 1)\n\n # Move 20 more seconds forward (beyond the initial lock expiration delay), then make\n # sure that the job is still within the data store.\n time_machine.shift(20)\n await datastore.cleanup()\n jobs = await datastore.get_jobs({job.id})\n assert len(jobs) == 1\n\n # Move 20 more seconds forward - this time the job's lease will have expired\n time_machine.shift(20)\n await datastore.cleanup()\n assert not await datastore.get_jobs({job.id})\n\n # Check that a result was recorded, with the \"abandoned\" outcome\n result = await datastore.get_job_result(job.id)\n assert result\n assert result.outcome is JobOutcome.abandoned\n\n\nasync def test_acquire_jobs_deserialization_failure(\n datastore: DataStore, mocker: MockerFixture\n) -> None:\n if not isinstance(datastore, BaseExternalDataStore):\n pytest.skip(\"Only applicable to external data stores\")\n\n # Add a task to the data store\n task = Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n await datastore.add_task(task)\n\n # Add a job to the data store\n job = Job(\n task_id=\"task1\", executor=\"async\", result_expiration_time=timedelta(seconds=30)\n )\n await datastore.add_job(job)\n\n # Make the serializer fail deserialization\n assert isinstance(datastore, BaseExternalDataStore)\n datastore.serializer = Mock(Serializer)\n datastore.serializer.deserialize.configure_mock(side_effect=DeserializationError)\n\n # This should not yield any jobs\n assert await datastore.acquire_jobs(\"scheduler_id\", timedelta(seconds=30), 1) == []\n\n\nasync def test_reap_abandoned_jobs(\n datastore: DataStore, local_broker: EventBroker, logger: Logger\n) -> None:\n # Add a task, schedule and job and acquire the latter two\n task = Task(id=\"task1\", func=\"contextlib:asynccontextmanager\", job_executor=\"async\")\n await datastore.add_task(task)\n\n job = Job(\n task_id=\"task1\",\n executor=\"async\",\n result_expiration_time=timedelta(seconds=30),\n )\n await datastore.add_job(job)\n await datastore.reap_abandoned_jobs(\"testscheduler\")\n jobs = await datastore.acquire_jobs(\"testscheduler\", timedelta(seconds=30), 1)\n assert len(jobs) == 1\n\n await datastore.reap_abandoned_jobs(\"testscheduler\")\n assert not await datastore.get_jobs()\n abandoned_job_result = await datastore.get_job_result(jobs[0].id)\n assert abandoned_job_result.outcome is JobOutcome.abandoned\n\n task = await datastore.get_task(\"task1\")\n assert task.running_jobs == 0\n\n\nclass TestRepr:\n async def test_memory(self, memory_store: MemoryDataStore) -> None:\n assert repr(memory_store) == \"MemoryDataStore()\"\n\n async def test_sqlite(self, tmp_path: Path) -> None:\n from sqlalchemy import create_engine\n\n expected_path = str(tmp_path).replace(\"\\\\\", \"\\\\\\\\\")\n engine = create_engine(f\"sqlite:///{tmp_path}\")\n data_store = SQLAlchemyDataStore(engine)\n assert repr(data_store) == (\n f\"SQLAlchemyDataStore(url='sqlite:///{expected_path}')\"\n )\n\n async def test_psycopg(self) -> None:\n from sqlalchemy.ext.asyncio import create_async_engine\n\n pytest.importorskip(\"psycopg\", reason=\"psycopg not available\")\n engine = create_async_engine(\n \"postgresql+psycopg://postgres:secret@localhost/testdb\"\n )\n data_store = SQLAlchemyDataStore(engine, schema=\"myschema\")\n assert repr(data_store) == (\n \"SQLAlchemyDataStore(url='postgresql+psycopg://postgres:***@localhost/\"\n \"testdb', schema='myschema')\"\n )\n\n async def test_mongodb(self) -> None:\n from pymongo.asynchronous.mongo_client import AsyncMongoClient\n\n async with AsyncMongoClient() as client:\n data_store = MongoDBDataStore(client)\n assert repr(data_store) == \"MongoDBDataStore(host=[('localhost', 27017)])\"\n" }, { "path": "tests/test_eventbrokers.py", "content": "from __future__ import annotations\n\nimport sys\nfrom contextlib import AsyncExitStack\nfrom datetime import datetime, timezone\nfrom logging import Logger\n\nimport pytest\nfrom _pytest.logging import LogCaptureFixture\nfrom anyio import CancelScope, create_memory_object_stream, fail_after\n\nfrom apscheduler import Event, ScheduleAdded\nfrom apscheduler.abc import EventBroker\n\nif sys.version_info >= (3, 11):\n from datetime import UTC\nelse:\n UTC = timezone.utc\n\npytestmark = pytest.mark.anyio\n\n\nasync def test_publish_subscribe(event_broker: EventBroker) -> None:\n send, receive = create_memory_object_stream[Event](2)\n with send, receive:\n event_broker.subscribe(send.send)\n event_broker.subscribe(send.send_nowait)\n event = ScheduleAdded(\n schedule_id=\"schedule1\",\n task_id=\"task1\",\n next_fire_time=datetime(2021, 9, 11, 12, 31, 56, 254867, UTC),\n )\n await event_broker.publish(event)\n\n with fail_after(3):\n event1 = await receive.receive()\n event2 = await receive.receive()\n\n assert event1 == event2\n assert isinstance(event1, ScheduleAdded)\n assert isinstance(event1.timestamp, datetime)\n assert event1.schedule_id == \"schedule1\"\n assert event1.task_id == \"task1\"\n assert event1.next_fire_time == datetime(2021, 9, 11, 12, 31, 56, 254867, UTC)\n\n\nasync def test_subscribe_one_shot(event_broker: EventBroker) -> None:\n send, receive = create_memory_object_stream[Event](2)\n with send, receive:\n event_broker.subscribe(send.send, one_shot=True)\n event = ScheduleAdded(\n schedule_id=\"schedule1\",\n task_id=\"task1\",\n next_fire_time=datetime(2021, 9, 11, 12, 31, 56, 254867, UTC),\n )\n await event_broker.publish(event)\n event = ScheduleAdded(\n schedule_id=\"schedule2\",\n task_id=\"task1\",\n next_fire_time=datetime(2021, 9, 12, 8, 42, 11, 968481, UTC),\n )\n await event_broker.publish(event)\n\n with fail_after(3):\n received_event = await receive.receive()\n\n with pytest.raises(TimeoutError), fail_after(0.1):\n await receive.receive()\n\n assert isinstance(received_event, ScheduleAdded)\n assert received_event.schedule_id == \"schedule1\"\n assert received_event.task_id == \"task1\"\n\n\nasync def test_unsubscribe(event_broker: EventBroker) -> None:\n send, receive = create_memory_object_stream[Event]()\n with send, receive:\n subscription = event_broker.subscribe(send.send)\n await event_broker.publish(Event())\n with fail_after(3):\n await receive.receive()\n\n subscription.unsubscribe()\n await event_broker.publish(Event())\n with pytest.raises(TimeoutError), fail_after(0.1):\n await receive.receive()\n\n\nasync def test_publish_no_subscribers(\n event_broker: EventBroker, caplog: LogCaptureFixture\n) -> None:\n await event_broker.publish(Event())\n assert not caplog.text\n\n\nasync def test_publish_exception(\n event_broker: EventBroker, caplog: LogCaptureFixture\n) -> None:\n def bad_subscriber(event: Event) -> None:\n raise Exception(\"foo\")\n\n timestamp = datetime.now(UTC)\n send, receive = create_memory_object_stream[Event]()\n with send, receive:\n event_broker.subscribe(bad_subscriber)\n event_broker.subscribe(send.send)\n await event_broker.publish(Event(timestamp=timestamp))\n\n received_event = await receive.receive()\n assert received_event.timestamp == timestamp\n assert \"Error delivering Event\" in caplog.text\n\n\nasync def test_cancel_start(raw_event_broker: EventBroker, logger: Logger) -> None:\n with CancelScope() as scope:\n scope.cancel()\n async with AsyncExitStack() as exit_stack:\n await raw_event_broker.start(exit_stack, logger)\n\n\nasync def test_cancel_stop(raw_event_broker: EventBroker, logger: Logger) -> None:\n with CancelScope() as scope:\n async with AsyncExitStack() as exit_stack:\n await raw_event_broker.start(exit_stack, logger)\n scope.cancel()\n\n\ndef test_asyncpg_broker_from_async_engine() -> None:\n pytest.importorskip(\"asyncpg\", reason=\"asyncpg is not installed\")\n from sqlalchemy import URL\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.eventbrokers.asyncpg import AsyncpgEventBroker\n\n url = URL(\n \"postgresql+asyncpg\",\n \"myuser\",\n \"c /%@\",\n \"localhost\",\n 7654,\n \"dbname\",\n {\"opt1\": \"foo\", \"opt2\": \"bar\"},\n )\n engine = create_async_engine(url)\n broker = AsyncpgEventBroker.from_async_sqla_engine(engine)\n assert isinstance(broker, AsyncpgEventBroker)\n assert broker.dsn == (\n \"postgresql://myuser:c %2F%25%40@localhost:7654/dbname?opt1=foo&opt2=bar\"\n )\n\n\ndef test_psycopg_broker_from_async_engine() -> None:\n pytest.importorskip(\n \"psycopg\", exc_type=ImportError, reason=\"psycopg is not installed\"\n )\n from sqlalchemy import URL\n from sqlalchemy.ext.asyncio import create_async_engine\n\n from apscheduler.eventbrokers.psycopg import PsycopgEventBroker\n\n url = URL(\n \"postgresql+psycopg\",\n \"myuser\",\n \"c /%@\",\n \"localhost\",\n 7654,\n \"dbname\",\n {\"opt1\": \"foo\", \"opt2\": \"bar\"},\n )\n engine = create_async_engine(url)\n broker = PsycopgEventBroker.from_async_sqla_engine(engine)\n assert isinstance(broker, PsycopgEventBroker)\n assert broker.conninfo == (\n \"postgresql://myuser:c %2F%25%40@localhost:7654/dbname?opt1=foo&opt2=bar\"\n )\n" }, { "path": "tests/test_marshalling.py", "content": "from __future__ import annotations\n\nimport sys\nfrom datetime import timedelta\nfrom functools import partial\nfrom types import ModuleType\n\nimport pytest\n\nfrom apscheduler import SerializationError\nfrom apscheduler._marshalling import callable_from_ref, callable_to_ref\n\n\nclass DummyClass:\n def meth(self):\n pass\n\n @staticmethod\n def staticmeth():\n pass\n\n @classmethod\n def classmeth(cls):\n pass\n\n def __call__(self):\n pass\n\n class InnerDummyClass:\n @classmethod\n def innerclassmeth(cls):\n pass\n\n\nclass InheritedDummyClass(DummyClass):\n pass\n\n\nclass TestCallableToRef:\n @pytest.mark.parametrize(\n \"obj, error\",\n [\n (partial(DummyClass.meth), \"Cannot create a reference to a partial()\"),\n (lambda: None, \"Cannot create a reference to a lambda\"),\n ],\n ids=[\"partial\", \"lambda\"],\n )\n def test_errors(self, obj, error):\n exc = pytest.raises(SerializationError, callable_to_ref, obj)\n assert str(exc.value) == error\n\n def test_nested_function_error(self):\n def nested():\n pass\n\n exc = pytest.raises(SerializationError, callable_to_ref, nested)\n assert str(exc.value) == \"Cannot create a reference to a nested function\"\n\n @pytest.mark.parametrize(\n \"input,expected\",\n [\n (DummyClass.meth, \"test_marshalling:DummyClass.meth\"),\n (DummyClass.classmeth, \"test_marshalling:DummyClass.classmeth\"),\n (\n DummyClass.InnerDummyClass.innerclassmeth,\n \"test_marshalling:DummyClass.InnerDummyClass.innerclassmeth\",\n ),\n (DummyClass.staticmeth, \"test_marshalling:DummyClass.staticmeth\"),\n (\n InheritedDummyClass.classmeth,\n \"test_marshalling:InheritedDummyClass.classmeth\",\n ),\n (timedelta, \"datetime:timedelta\"),\n ],\n ids=[\n \"unbound method\",\n \"class method\",\n \"inner class method\",\n \"static method\",\n \"inherited class method\",\n \"timedelta\",\n ],\n )\n def test_valid_refs(self, input, expected):\n assert callable_to_ref(input) == expected\n\n\nclass TestCallableFromRef:\n def test_valid_ref(self):\n from logging.handlers import RotatingFileHandler\n\n assert (\n callable_from_ref(\"logging.handlers:RotatingFileHandler\")\n is RotatingFileHandler\n )\n\n def test_complex_path(self):\n pkg1 = ModuleType(\"pkg1\")\n pkg1.pkg2 = \"blah\"\n pkg2 = ModuleType(\"pkg1.pkg2\")\n pkg2.varname = lambda: None\n sys.modules[\"pkg1\"] = pkg1\n sys.modules[\"pkg1.pkg2\"] = pkg2\n assert callable_from_ref(\"pkg1.pkg2:varname\") == pkg2.varname\n\n @pytest.mark.parametrize(\n \"input,error\",\n [(object(), TypeError), (\"module\", ValueError), (\"module:blah\", LookupError)],\n ids=[\"raw object\", \"module\", \"module attribute\"],\n )\n def test_lookup_error(self, input, error):\n pytest.raises(error, callable_from_ref, input)\n" }, { "path": "tests/test_schedulers.py", "content": "from __future__ import annotations\n\nimport os\nimport sys\nimport threading\nimport time\nfrom collections import defaultdict\nfrom collections.abc import Callable\nfrom contextlib import AsyncExitStack\nfrom datetime import datetime, timedelta, timezone\nfrom functools import partial\nfrom inspect import signature\nfrom queue import Queue\nfrom types import ModuleType\nfrom typing import Any, cast\nfrom unittest.mock import patch\n\nimport anyio\nimport pytest\nfrom anyio import (\n Lock,\n WouldBlock,\n create_memory_object_stream,\n fail_after,\n move_on_after,\n sleep,\n)\nfrom pytest import MonkeyPatch\nfrom pytest_mock import MockerFixture, MockFixture\n\nfrom apscheduler import (\n AsyncScheduler,\n CoalescePolicy,\n Event,\n Job,\n JobAcquired,\n JobAdded,\n JobLookupError,\n JobOutcome,\n JobReleased,\n RunState,\n ScheduleAdded,\n ScheduleLookupError,\n Scheduler,\n ScheduleRemoved,\n SchedulerRole,\n SchedulerStarted,\n SchedulerStopped,\n ScheduleUpdated,\n TaskAdded,\n TaskDefaults,\n TaskUpdated,\n current_async_scheduler,\n current_job,\n task,\n)\nfrom apscheduler.abc import DataStore\nfrom apscheduler.datastores.base import BaseExternalDataStore\nfrom apscheduler.datastores.memory import MemoryDataStore\nfrom apscheduler.eventbrokers.local import LocalEventBroker\nfrom apscheduler.executors.async_ import AsyncJobExecutor\nfrom apscheduler.executors.subprocess import ProcessPoolJobExecutor\nfrom apscheduler.executors.thread import ThreadPoolJobExecutor\nfrom apscheduler.triggers.cron import CronTrigger\nfrom apscheduler.triggers.date import DateTrigger\nfrom apscheduler.triggers.interval import IntervalTrigger\n\nif sys.version_info >= (3, 11):\n from datetime import UTC\nelse:\n UTC = timezone.utc\n from exceptiongroup import ExceptionGroup\n\nfrom zoneinfo import ZoneInfo\n\npytestmark = pytest.mark.anyio\n\n\nasync def dummy_async_job(delay: float = 0, fail: bool = False) -> str:\n await anyio.sleep(delay)\n if fail:\n raise RuntimeError(\"failing as requested\")\n else:\n return \"returnvalue\"\n\n\ndef dummy_sync_job(delay: float = 0, fail: bool = False) -> str:\n time.sleep(delay)\n if fail:\n raise RuntimeError(\"failing as requested\")\n else:\n return \"returnvalue\"\n\n\n@task(\n job_executor=\"threadpool\",\n max_running_jobs=3,\n misfire_grace_time=timedelta(seconds=6),\n)\ndef decorated_job() -> None:\n pass\n\n\nclass DummyClass:\n def __init__(self, value: int):\n self.value = value\n\n @staticmethod\n async def dummy_static_method() -> str:\n return \"static\"\n\n @staticmethod\n async def dummy_async_static_method() -> str:\n return \"static\"\n\n @classmethod\n def dummy_class_method(cls) -> str:\n return \"class\"\n\n @classmethod\n async def dummy_async_class_method(cls) -> str:\n return \"class\"\n\n def dummy_instance_method(self) -> int:\n return self.value\n\n async def dummy_async_instance_method(self) -> int:\n return self.value\n\n def __call__(self) -> int:\n return self.value\n\n\nclass TestAsyncScheduler:\n def test_repr(self) -> None:\n scheduler = AsyncScheduler(identity=\"my identity\")\n assert repr(scheduler) == (\n \"AsyncScheduler(identity='my identity', role=, \"\n \"data_store=MemoryDataStore(), event_broker=LocalEventBroker())\"\n )\n\n async def test_use_before_initialized(self) -> None:\n scheduler = AsyncScheduler()\n with pytest.raises(\n RuntimeError, match=\"The scheduler has not been initialized yet\"\n ):\n await scheduler.add_job(dummy_async_job)\n\n async def test_properties(self) -> None:\n async with AsyncScheduler() as scheduler:\n assert isinstance(scheduler.data_store, MemoryDataStore)\n assert isinstance(scheduler.event_broker, LocalEventBroker)\n assert scheduler.role is SchedulerRole.both\n assert isinstance(scheduler.identity, str)\n assert len(scheduler.job_executors) == 3\n assert isinstance(scheduler.job_executors[\"async\"], AsyncJobExecutor)\n assert isinstance(\n scheduler.job_executors[\"threadpool\"], ThreadPoolJobExecutor\n )\n assert isinstance(\n scheduler.job_executors[\"processpool\"], ProcessPoolJobExecutor\n )\n assert scheduler.task_defaults.job_executor == \"async\"\n assert scheduler.state is RunState.stopped\n\n @pytest.mark.parametrize(\"as_default\", [False, True])\n async def test_async_executor(self, as_default: bool) -> None:\n async with AsyncScheduler() as scheduler:\n await scheduler.start_in_background()\n if as_default:\n thread_id = await scheduler.run_job(threading.get_ident)\n else:\n thread_id = await scheduler.run_job(\n threading.get_ident, job_executor=\"async\"\n )\n\n assert thread_id == threading.get_ident()\n\n async def test_threadpool_executor(self) -> None:\n async with AsyncScheduler() as scheduler:\n await scheduler.start_in_background()\n thread_id = await scheduler.run_job(\n threading.get_ident, job_executor=\"threadpool\"\n )\n assert thread_id != threading.get_ident()\n\n async def test_processpool_executor(self) -> None:\n async with AsyncScheduler() as scheduler:\n await scheduler.start_in_background()\n pid = await scheduler.run_job(os.getpid, job_executor=\"processpool\")\n assert pid != os.getpid()\n\n async def test_configure_task(self, raw_datastore: DataStore) -> None:\n send, receive = create_memory_object_stream[Event](2)\n with send, receive:\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n scheduler.subscribe(send.send)\n await scheduler.configure_task(\"mytask\", func=dummy_async_job)\n await scheduler.configure_task(\"mytask\", misfire_grace_time=2)\n tasks = await scheduler.get_tasks()\n assert len(tasks) == 1\n assert tasks[0].id == \"mytask\"\n assert tasks[0].func == f\"{__name__}:dummy_async_job\"\n assert tasks[0].misfire_grace_time == timedelta(seconds=2)\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n assert event.task_id == \"mytask\"\n\n event = await receive.receive()\n assert isinstance(event, TaskUpdated)\n assert event.task_id == \"mytask\"\n\n async def test_configure_task_with_decorator(self) -> None:\n async with AsyncScheduler() as scheduler:\n await scheduler.configure_task(\"taskfunc\", func=decorated_job)\n tasks = await scheduler.get_tasks()\n assert len(tasks) == 1\n assert tasks[0].max_running_jobs == 3\n assert tasks[0].misfire_grace_time == timedelta(seconds=6)\n assert tasks[0].job_executor == \"threadpool\"\n\n async def test_configure_local_task_with_decorator(self) -> None:\n @task(\n id=\"taskfunc\",\n job_executor=\"threadpool\",\n max_running_jobs=3,\n misfire_grace_time=timedelta(seconds=6),\n metadata={\"local\": 6},\n )\n def taskfunc() -> None:\n pass\n\n task_defaults = TaskDefaults(metadata={\"global\": \"foo\"})\n async with AsyncScheduler(task_defaults=task_defaults) as scheduler:\n await scheduler.configure_task(taskfunc, metadata={\"direct\": [1, 9]})\n tasks = await scheduler.get_tasks()\n assert len(tasks) == 1\n assert tasks[0].id == \"taskfunc\"\n assert tasks[0].max_running_jobs == 3\n assert tasks[0].misfire_grace_time == timedelta(seconds=6)\n assert tasks[0].job_executor == \"threadpool\"\n assert tasks[0].metadata == {\"global\": \"foo\", \"local\": 6, \"direct\": [1, 9]}\n\n async def test_add_pause_unpause_remove_schedule(\n self, raw_datastore: DataStore, timezone: ZoneInfo\n ) -> None:\n send, receive = create_memory_object_stream[Event](5)\n with send, receive:\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n scheduler.subscribe(send.send)\n now = datetime.now(timezone)\n trigger = DateTrigger(now)\n schedule_id = await scheduler.add_schedule(\n dummy_async_job, trigger, id=\"foo\"\n )\n assert schedule_id == \"foo\"\n\n schedules = await scheduler.get_schedules()\n assert len(schedules) == 1\n assert schedules[0].id == \"foo\"\n assert schedules[0].task_id == f\"{__name__}:dummy_async_job\"\n\n await scheduler.pause_schedule(\"foo\")\n schedule = await scheduler.get_schedule(\"foo\")\n assert schedule.paused\n assert schedule.next_fire_time == now\n\n await scheduler.unpause_schedule(\"foo\")\n schedule = await scheduler.get_schedule(\"foo\")\n assert not schedule.paused\n assert schedule.next_fire_time == now\n\n await scheduler.remove_schedule(schedule_id)\n assert not await scheduler.get_schedules()\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n\n event = await receive.receive()\n assert isinstance(event, ScheduleAdded)\n assert event.schedule_id == \"foo\"\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.next_fire_time == now\n\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n assert event.schedule_id == \"foo\"\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.next_fire_time == now\n\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n assert event.schedule_id == \"foo\"\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.next_fire_time == now\n\n event = await receive.receive()\n assert isinstance(event, ScheduleRemoved)\n assert event.schedule_id == \"foo\"\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert not event.finished\n\n async def test_add_job_wait_result(self, raw_datastore: DataStore) -> None:\n send, receive = create_memory_object_stream[Event](2)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore)\n )\n assert await scheduler.get_jobs() == []\n\n scheduler.subscribe(send.send)\n job_id = await scheduler.add_job(dummy_async_job, result_expiration_time=10)\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.job_id == job_id\n\n jobs = await scheduler.get_jobs()\n assert len(jobs) == 1\n assert jobs[0].id == job_id\n assert jobs[0].task_id == f\"{__name__}:dummy_async_job\"\n\n with pytest.raises(JobLookupError):\n await scheduler.get_job_result(job_id, wait=False)\n\n await scheduler.start_in_background()\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, SchedulerStarted)\n\n event = await receive.receive()\n assert isinstance(event, JobAcquired)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.schedule_id is None\n assert event.scheduled_start is None\n acquired_at = event.timestamp\n\n event = await receive.receive()\n assert isinstance(event, JobReleased)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.schedule_id is None\n assert event.scheduled_start is None\n assert event.started_at is not None\n assert event.started_at >= acquired_at\n assert event.outcome is JobOutcome.success\n\n result = await scheduler.get_job_result(job_id)\n assert result\n assert result.outcome is JobOutcome.success\n assert result.return_value == \"returnvalue\"\n\n @pytest.mark.parametrize(\"success\", [True, False])\n async def test_run_job(self, raw_datastore: DataStore, success: bool) -> None:\n send, receive = create_memory_object_stream[Event](4)\n with send, receive:\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n await scheduler.start_in_background()\n scheduler.subscribe(send.send)\n try:\n result = await scheduler.run_job(\n dummy_async_job, kwargs={\"fail\": not success}\n )\n except RuntimeError as exc:\n assert str(exc) == \"failing as requested\"\n else:\n assert result == \"returnvalue\"\n\n assert not await scheduler.get_jobs()\n\n with fail_after(3):\n # The task was added\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n\n # The job was added\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n job_id = event.job_id\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n\n # The scheduler acquired the job\n event = await receive.receive()\n assert isinstance(event, JobAcquired)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.schedule_id is None\n assert event.scheduled_start is None\n assert event.scheduler_id == scheduler.identity\n acquired_at = event.timestamp\n\n # The scheduler released the job\n event = await receive.receive()\n assert isinstance(event, JobReleased)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n assert event.schedule_id is None\n assert event.scheduled_start is None\n assert event.started_at is not None\n assert event.started_at >= acquired_at\n assert event.scheduler_id == scheduler.identity\n\n # The scheduler was stopped\n event = await receive.receive()\n assert isinstance(event, SchedulerStopped)\n\n # There should be no more events on the list\n with pytest.raises(WouldBlock):\n receive.receive_nowait()\n\n @pytest.mark.parametrize(\n \"target, expected_result\",\n [\n pytest.param(dummy_async_job, \"returnvalue\", id=\"async_func\"),\n pytest.param(dummy_sync_job, \"returnvalue\", id=\"sync_func\"),\n pytest.param(DummyClass.dummy_static_method, \"static\", id=\"staticmethod\"),\n pytest.param(\n DummyClass.dummy_async_static_method, \"static\", id=\"async_staticmethod\"\n ),\n pytest.param(DummyClass.dummy_class_method, \"class\", id=\"classmethod\"),\n pytest.param(\n DummyClass.dummy_async_class_method, \"class\", id=\"async_classmethod\"\n ),\n pytest.param(DummyClass(5).dummy_instance_method, 5, id=\"instancemethod\"),\n pytest.param(\n DummyClass(6).dummy_async_instance_method, 6, id=\"async_instancemethod\"\n ),\n pytest.param(bytes, b\"\", id=\"builtin_function\"),\n pytest.param(\n datetime(2023, 10, 19, tzinfo=UTC).timestamp,\n 1697673600.0,\n id=\"builtin_method\",\n ),\n pytest.param(partial(bytes, \"foo\", \"ascii\"), b\"foo\", id=\"partial\"),\n ],\n )\n @pytest.mark.parametrize(\n \"use_scheduling\",\n [\n pytest.param(False, id=\"job\"),\n pytest.param(True, id=\"schedule\"),\n ],\n )\n async def test_callable_types(\n self,\n target: Callable[..., Any],\n expected_result: object,\n use_scheduling: bool,\n raw_datastore: DataStore,\n timezone: ZoneInfo,\n ) -> None:\n now = datetime.now(timezone)\n send, receive = create_memory_object_stream[Event](4)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore)\n )\n scheduler.subscribe(send.send, {JobReleased})\n await scheduler.start_in_background()\n if use_scheduling:\n trigger = DateTrigger(now)\n await scheduler.add_schedule(target, trigger, id=\"foo\")\n else:\n await scheduler.add_job(target, result_expiration_time=10)\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, JobReleased)\n\n if not use_scheduling:\n result = await scheduler.get_job_result(event.job_id)\n assert result\n assert result.outcome is JobOutcome.success\n assert result.return_value == expected_result\n\n async def test_scheduled_job_missed_deadline(\n self, raw_datastore: DataStore, timezone: ZoneInfo\n ) -> None:\n one_second_in_past = datetime.now(timezone) - timedelta(seconds=1)\n trigger = DateTrigger(one_second_in_past)\n scheduler_send, scheduler_receive = create_memory_object_stream[Event](4)\n worker_send, worker_receive = create_memory_object_stream[Event](2)\n with scheduler_send, scheduler_receive, worker_send, worker_receive:\n async with AsyncExitStack() as exit_stack:\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore)\n )\n await scheduler.add_schedule(\n dummy_async_job, trigger, misfire_grace_time=0, id=\"foo\"\n )\n await scheduler.start_in_background()\n exit_stack.enter_context(\n scheduler.subscribe(\n scheduler_send.send, {JobAdded, ScheduleUpdated}\n )\n )\n exit_stack.enter_context(\n scheduler.subscribe(worker_send.send, {JobAcquired, JobReleased})\n )\n with fail_after(3):\n # The schedule was processed and a job was added for it\n event = await scheduler_receive.receive()\n assert isinstance(event, JobAdded)\n assert event.schedule_id == \"foo\"\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n job_id = event.job_id\n\n # The schedule was updated with a null next fire time\n event = await scheduler_receive.receive()\n assert isinstance(event, ScheduleUpdated)\n assert event.schedule_id == \"foo\"\n assert event.next_fire_time is None\n\n # The new job was acquired\n event = await worker_receive.receive()\n assert isinstance(event, JobReleased)\n assert event.job_id == job_id\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n assert event.schedule_id == \"foo\"\n assert event.scheduled_start == one_second_in_past\n assert event.started_at is None\n assert event.outcome is JobOutcome.missed_start_deadline\n\n # There should be no more events on the list\n with pytest.raises(WouldBlock):\n scheduler_receive.receive_nowait()\n\n @pytest.mark.parametrize(\n \"coalesce, expected_jobs, first_fire_time_delta\",\n [\n pytest.param(\n CoalescePolicy.all, 4, timedelta(minutes=3, seconds=5), id=\"all\"\n ),\n pytest.param(\n CoalescePolicy.earliest,\n 1,\n timedelta(minutes=3, seconds=5),\n id=\"earliest\",\n ),\n pytest.param(CoalescePolicy.latest, 1, timedelta(seconds=5), id=\"latest\"),\n ],\n )\n async def test_coalesce_policy(\n self,\n coalesce: CoalescePolicy,\n expected_jobs: int,\n first_fire_time_delta: timedelta,\n raw_datastore: DataStore,\n timezone: ZoneInfo,\n ) -> None:\n now = datetime.now(timezone)\n first_start_time = now - timedelta(minutes=3, seconds=5)\n trigger = IntervalTrigger(minutes=1, start_time=first_start_time)\n send, receive = create_memory_object_stream[Event](4)\n with send, receive:\n async with AsyncScheduler(\n data_store=raw_datastore,\n role=SchedulerRole.scheduler,\n cleanup_interval=None,\n ) as scheduler:\n await scheduler.add_schedule(\n dummy_async_job, trigger, id=\"foo\", coalesce=coalesce\n )\n scheduler.subscribe(send.send)\n await scheduler.start_in_background()\n\n with fail_after(3):\n # The scheduler was started\n event = await receive.receive()\n assert isinstance(event, SchedulerStarted)\n\n # The schedule was processed and one or more jobs weres added\n for _ in range(expected_jobs):\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.schedule_id == \"foo\"\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n assert event.next_fire_time == now + timedelta(seconds=55)\n\n expected_scheduled_fire_time = now - first_fire_time_delta\n jobs = await scheduler.get_jobs()\n for job in sorted(jobs, key=lambda job: job.scheduled_fire_time):\n assert job.scheduled_fire_time\n assert job.scheduled_fire_time < now\n assert job.scheduled_fire_time == expected_scheduled_fire_time\n expected_scheduled_fire_time += timedelta(minutes=1)\n\n # The scheduler was stopped\n event = await receive.receive()\n assert isinstance(event, SchedulerStopped)\n\n # There should be no more events on the list\n with pytest.raises(WouldBlock):\n receive.receive_nowait()\n\n @pytest.mark.parametrize(\n \"max_jitter, expected_upper_bound\",\n [pytest.param(2, 2, id=\"within\"), pytest.param(4, 2.999999, id=\"exceed\")],\n )\n async def test_jitter(\n self,\n mocker: MockerFixture,\n timezone: ZoneInfo,\n max_jitter: float,\n expected_upper_bound: float,\n raw_datastore: DataStore,\n ) -> None:\n jitter = 1.569374\n now = datetime.now(timezone)\n fake_uniform = mocker.patch(\"random.uniform\")\n fake_uniform.configure_mock(side_effect=lambda a, b: jitter)\n send, receive = create_memory_object_stream[Event](4)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore, role=SchedulerRole.scheduler)\n )\n scheduler.subscribe(send.send)\n trigger = DateTrigger(now)\n schedule_id = await scheduler.add_schedule(\n dummy_async_job, trigger, id=\"foo\", max_jitter=max_jitter\n )\n schedule = await scheduler.get_schedule(schedule_id)\n assert schedule.max_jitter == timedelta(seconds=max_jitter)\n\n await scheduler.start_in_background()\n\n with fail_after(3):\n # The task was added\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n\n # The schedule was added\n event = await receive.receive()\n assert isinstance(event, ScheduleAdded)\n assert event.schedule_id == \"foo\"\n assert event.next_fire_time == now\n\n # The scheduler was started\n event = await receive.receive()\n assert isinstance(event, SchedulerStarted)\n\n # The schedule was processed and a job was added for it\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.schedule_id == \"foo\"\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n\n # Check that the job was created with the proper amount of jitter in its\n # scheduled time\n jobs = await scheduler.get_jobs()\n assert len(jobs) == 1\n assert jobs[0].jitter == timedelta(seconds=jitter)\n assert jobs[0].scheduled_fire_time == now + timedelta(seconds=jitter)\n assert jobs[0].original_scheduled_time == now\n\n async def test_add_job_get_result_success(self, raw_datastore: DataStore) -> None:\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n job_id = await scheduler.add_job(\n dummy_async_job, kwargs={\"delay\": 0.2}, result_expiration_time=5\n )\n await scheduler.start_in_background()\n with fail_after(3):\n result = await scheduler.get_job_result(job_id)\n\n assert result\n assert result.job_id == job_id\n assert result.outcome is JobOutcome.success\n assert result.return_value == \"returnvalue\"\n\n async def test_add_job_get_result_empty(self, raw_datastore: DataStore) -> None:\n send, receive = create_memory_object_stream[Event](4)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore)\n )\n await scheduler.start_in_background()\n\n scheduler.subscribe(send.send)\n job_id = await scheduler.add_job(dummy_async_job)\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.job_id == job_id\n\n event = await receive.receive()\n assert isinstance(event, JobAcquired)\n assert event.job_id == job_id\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n assert event.schedule_id is None\n\n event = await receive.receive()\n assert isinstance(event, JobReleased)\n assert event.job_id == job_id\n assert event.task_id == \"test_schedulers:dummy_async_job\"\n assert event.schedule_id is None\n\n with pytest.raises(JobLookupError):\n await scheduler.get_job_result(job_id, wait=False)\n\n async def test_add_job_get_result_error(self) -> None:\n async with AsyncScheduler() as scheduler:\n job_id = await scheduler.add_job(\n dummy_async_job,\n kwargs={\"delay\": 0.2, \"fail\": True},\n result_expiration_time=5,\n )\n await scheduler.start_in_background()\n with fail_after(3):\n result = await scheduler.get_job_result(job_id)\n\n assert result\n assert result.job_id == job_id\n assert result.outcome is JobOutcome.error\n assert isinstance(result.exception, RuntimeError)\n assert str(result.exception) == \"failing as requested\"\n\n async def test_add_job_get_result_no_ready_yet(self) -> None:\n send, receive = create_memory_object_stream[Event](4)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(AsyncScheduler())\n scheduler.subscribe(send.send)\n job_id = await scheduler.add_job(dummy_async_job, kwargs={\"delay\": 0.2})\n\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, TaskAdded)\n\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.job_id == job_id\n\n with pytest.raises(JobLookupError), fail_after(1):\n await scheduler.get_job_result(job_id, wait=False)\n\n async def test_add_job_not_rewriting_task_config(\n self, raw_datastore: DataStore\n ) -> None:\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n TASK_ID = \"task_dummy_async_job\"\n JOB_EXECUTOR = \"async\"\n MISFIRE_GRACE_TIME = timedelta(seconds=10)\n MAX_RUNNING_JOBS = 5\n METADATA = {\"key\": \"value\"}\n\n await scheduler.configure_task(\n func_or_task_id=TASK_ID,\n func=dummy_async_job,\n job_executor=JOB_EXECUTOR,\n misfire_grace_time=MISFIRE_GRACE_TIME,\n max_running_jobs=MAX_RUNNING_JOBS,\n metadata=METADATA,\n )\n\n assert await scheduler.add_job(TASK_ID)\n\n task = await scheduler.data_store.get_task(TASK_ID)\n assert task.job_executor == JOB_EXECUTOR\n assert task.misfire_grace_time == MISFIRE_GRACE_TIME\n assert task.max_running_jobs == MAX_RUNNING_JOBS\n assert task.metadata == METADATA\n\n async def test_contextvars(self, mocker: MockerFixture, timezone: ZoneInfo) -> None:\n def check_contextvars() -> None:\n assert current_async_scheduler.get() is scheduler\n info = current_job.get()\n assert isinstance(info, Job)\n assert info.task_id == \"contextvars\"\n assert info.schedule_id == \"foo\"\n assert info.original_scheduled_time == now\n assert info.scheduled_fire_time == now + timedelta(seconds=2.16)\n assert info.jitter == timedelta(seconds=2.16)\n assert info.start_deadline == now + timedelta(seconds=2.16) + timedelta(\n seconds=10\n )\n\n fake_uniform = mocker.patch(\"random.uniform\")\n fake_uniform.configure_mock(return_value=2.16)\n now = datetime.now(timezone)\n send, receive = create_memory_object_stream[Event](1)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(AsyncScheduler())\n await scheduler.configure_task(\"contextvars\", func=check_contextvars)\n await scheduler.add_schedule(\n \"contextvars\",\n DateTrigger(now),\n id=\"foo\",\n max_jitter=3,\n misfire_grace_time=10,\n )\n scheduler.subscribe(send.send, {JobReleased})\n await scheduler.start_in_background()\n\n with fail_after(3):\n event = await receive.receive()\n\n assert event.outcome is JobOutcome.success\n\n async def test_explicit_cleanup(self, raw_datastore: DataStore) -> None:\n send, receive = create_memory_object_stream[Event](1)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(raw_datastore, cleanup_interval=None)\n )\n scheduler.subscribe(send.send, {ScheduleRemoved})\n event = anyio.Event()\n scheduler.subscribe(lambda _: event.set(), {JobReleased}, one_shot=True)\n await scheduler.start_in_background()\n\n # Add a job whose result expires after 1 ms\n job_id = await scheduler.add_job(\n dummy_async_job, result_expiration_time=0.001\n )\n with fail_after(3):\n await event.wait()\n\n # After the sleeping past the expiration time and performing a cleanup, the\n # result should not be there anymore\n await sleep(0.1)\n await scheduler.cleanup()\n with pytest.raises(JobLookupError):\n await scheduler.get_job_result(job_id)\n\n # Add a schedule to immediately set the event\n event = anyio.Event()\n scheduler.subscribe(lambda _: event.set(), {JobReleased}, one_shot=True)\n await scheduler.add_schedule(\n dummy_async_job, DateTrigger(datetime.now(timezone.utc)), id=\"event_set\"\n )\n with fail_after(3):\n await event.wait()\n\n # The schedule should still be around, but with a null next_fire_time\n schedule = await scheduler.get_schedule(\"event_set\")\n assert schedule.next_fire_time is None\n\n # After the cleanup, the schedule should be gone\n await scheduler.cleanup()\n with pytest.raises(ScheduleLookupError):\n await scheduler.get_schedule(\"event_set\")\n\n # Check that the corresponding event was received\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, ScheduleRemoved)\n assert event.schedule_id == schedule.id\n assert event.finished\n\n async def test_explicit_cleanup_avoid_schedules_still_having_jobs(\n self, raw_datastore: DataStore\n ) -> None:\n send, receive = create_memory_object_stream[Event](4)\n async with AsyncExitStack() as exit_stack:\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(raw_datastore, cleanup_interval=None)\n )\n scheduler.subscribe(send.send, {ScheduleUpdated, JobAdded, JobReleased})\n await scheduler.start_in_background()\n\n # Add a schedule to immediately set the event\n dummy_event = anyio.Event()\n await scheduler.configure_task(\"event_set\", func=dummy_event.wait)\n schedule_id = await scheduler.add_schedule(\n \"event_set\", DateTrigger(datetime.now(timezone.utc)), id=\"event_set\"\n )\n\n # Wait for the job to be submitted\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.schedule_id == schedule_id\n\n # Wait for the schedule to be updated\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n assert event.schedule_id == schedule_id\n assert event.next_fire_time is None\n\n # Check that there is a job for the schedule\n jobs = await scheduler.get_jobs()\n assert len(jobs) == 1\n assert jobs[0].schedule_id == schedule_id\n\n # After the cleanup, the schedule should still be around, with a\n # null next_fire_time\n await scheduler.cleanup()\n schedule = await scheduler.get_schedule(\"event_set\")\n assert schedule.next_fire_time is None\n\n # Wait for the job to finish\n dummy_event.set()\n event = await receive.receive()\n assert isinstance(event, JobReleased)\n\n async def test_implicit_cleanup(self, mocker: MockerFixture) -> None:\n \"\"\"\n Test that the scheduler's cleanup() method is called when the scheduler is\n started.\n\n \"\"\"\n async with AsyncScheduler() as scheduler:\n event = anyio.Event()\n mocker.patch.object(scheduler.data_store, \"cleanup\", side_effect=event.set)\n await scheduler.start_in_background()\n with fail_after(3):\n await event.wait()\n\n async def test_wait_until_stopped(self) -> None:\n async with AsyncScheduler() as scheduler:\n await scheduler.add_job(scheduler.stop)\n await scheduler.wait_until_stopped()\n\n # This should be a no-op\n await scheduler.wait_until_stopped()\n\n async def test_max_concurrent_jobs(self) -> None:\n lock = Lock()\n scheduler = AsyncScheduler(max_concurrent_jobs=1)\n tasks_done = 0\n\n async def acquire_release() -> None:\n nonlocal tasks_done\n lock.acquire_nowait()\n await sleep(0.1)\n tasks_done += 1\n if tasks_done == 2:\n await scheduler.stop()\n\n lock.release()\n\n with fail_after(3):\n async with scheduler:\n await scheduler.configure_task(\"dummyjob\", func=acquire_release)\n await scheduler.add_job(\"dummyjob\")\n await scheduler.add_job(\"dummyjob\")\n await scheduler.run_until_stopped()\n\n @pytest.mark.parametrize(\n \"trigger_type, run_job\",\n [\n pytest.param(\"cron\", False, id=\"cron\"),\n pytest.param(\"date\", True, id=\"date\"),\n ],\n )\n async def test_pause_unpause_schedule(\n self,\n raw_datastore: DataStore,\n timezone: ZoneInfo,\n trigger_type: str,\n run_job: bool,\n ) -> None:\n if trigger_type == \"cron\":\n trigger = CronTrigger()\n else:\n trigger = DateTrigger(datetime.now(timezone))\n\n async with AsyncExitStack() as exit_stack:\n send, receive = create_memory_object_stream[Event](4)\n exit_stack.enter_context(send)\n exit_stack.enter_context(receive)\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore, role=SchedulerRole.scheduler)\n )\n schedule_id = await scheduler.add_schedule(\n dummy_async_job, trigger, id=\"foo\"\n )\n scheduler.subscribe(send.send, {ScheduleUpdated, JobAdded})\n\n # Pause the schedule and wait for the schedule update event\n await scheduler.pause_schedule(schedule_id)\n schedule = await scheduler.get_schedule(schedule_id)\n assert schedule.paused\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n\n if run_job:\n # Make sure that no jobs are added when the scheduler is started\n await scheduler.start_in_background()\n assert not await scheduler.get_jobs()\n\n # Unpause the schedule and wait for the schedule update event\n await scheduler.unpause_schedule(schedule_id)\n schedule = await scheduler.get_schedule(schedule_id)\n assert not schedule.paused\n event = await receive.receive()\n assert isinstance(event, ScheduleUpdated)\n\n if run_job:\n with fail_after(3):\n job_added_event = await receive.receive()\n\n assert isinstance(job_added_event, JobAdded)\n assert job_added_event.schedule_id == schedule_id\n\n async def test_schedule_job_result_expiration_time(\n self, raw_datastore: DataStore, timezone: ZoneInfo\n ) -> None:\n trigger = DateTrigger(datetime.now(timezone))\n send, receive = create_memory_object_stream[Event](4)\n with send, receive:\n async with AsyncExitStack() as exit_stack:\n scheduler = await exit_stack.enter_async_context(\n AsyncScheduler(data_store=raw_datastore)\n )\n await scheduler.add_schedule(\n dummy_async_job, trigger, id=\"foo\", job_result_expiration_time=10\n )\n exit_stack.enter_context(scheduler.subscribe(send.send, {JobAdded}))\n await scheduler.start_in_background()\n\n # Wait for the scheduled job to be added\n with fail_after(3):\n event = await receive.receive()\n assert isinstance(event, JobAdded)\n assert event.schedule_id == \"foo\"\n\n # Get its result\n result = await scheduler.get_job_result(event.job_id)\n\n assert result\n assert result.outcome is JobOutcome.success\n assert result.return_value == \"returnvalue\"\n\n async def test_scheduler_crash_restart_schedule_immediately(\n self, raw_datastore: DataStore, timezone: ZoneInfo\n ) -> None:\n \"\"\"\n Test that the scheduler can immediately start processing a schedule it had\n acquired while the crash occurred.\n\n \"\"\"\n scheduler = AsyncScheduler(data_store=raw_datastore)\n error_patch = patch.object(\n raw_datastore, \"release_schedules\", side_effect=RuntimeError(\"Fake failure\")\n )\n with pytest.raises(ExceptionGroup) as exc_info, error_patch:\n async with scheduler:\n await scheduler.add_schedule(\n dummy_async_job, IntervalTrigger(minutes=1), id=\"foo\"\n )\n with move_on_after(3):\n await scheduler.run_until_stopped()\n\n pytest.fail(\"The scheduler did not crash\")\n\n exc = exc_info.value\n while isinstance(exc, ExceptionGroup) and len(exc.exceptions) == 1:\n exc = exc.exceptions[0]\n\n assert isinstance(exc, RuntimeError)\n assert exc.args == (\"Fake failure\",)\n\n # Don't clear the data store at launch\n if isinstance(raw_datastore, BaseExternalDataStore):\n raw_datastore.start_from_scratch = False\n\n # Now reinitialize the scheduler and make sure the schedule gets processed\n # immediately\n async with scheduler:\n # Check that the schedule was left in an acquired state\n schedules = await scheduler.get_schedules()\n assert len(schedules) == 1\n assert schedules[0].acquired_by == scheduler.identity\n assert schedules[0].acquired_until > datetime.now(timezone)\n\n # Start the scheduler and wait for the schedule to be processed\n await scheduler.start_in_background()\n with fail_after(scheduler.lease_duration.total_seconds() / 2):\n job_added_event = await scheduler.get_next_event(JobAdded)\n\n assert job_added_event.schedule_id == \"foo\"\n\n async def test_scheduler_crash_reap_abandoned_jobs(\n self, raw_datastore: DataStore, timezone: ZoneInfo\n ) -> None:\n \"\"\"\n Test that after the scheduler has crashed and been restarted, it immediately\n detects an abandoned job and releases it with the appropriate result code.\n\n \"\"\"\n scheduler = AsyncScheduler(data_store=raw_datastore)\n error_patch = patch.object(\n raw_datastore, \"release_job\", side_effect=RuntimeError(\"Fake failure\")\n )\n with pytest.raises(ExceptionGroup) as exc_info, error_patch:\n async with scheduler:\n job_id = await scheduler.add_job(dummy_async_job)\n with move_on_after(3):\n await scheduler.run_until_stopped()\n\n pytest.fail(\"The scheduler did not crash\")\n\n exc = exc_info.value\n while isinstance(exc, ExceptionGroup) and len(exc.exceptions) == 1:\n exc = exc.exceptions[0]\n\n assert isinstance(exc, RuntimeError)\n assert exc.args == (\"Fake failure\",)\n\n # Don't clear the data store at launch\n if isinstance(raw_datastore, BaseExternalDataStore):\n raw_datastore.start_from_scratch = False\n\n # Now reinitialize the scheduler and make sure the job gets processed\n # immediately\n async with scheduler:\n # Check that the job was left in an acquired state\n jobs = await scheduler.get_jobs()\n assert len(jobs) == 1\n assert jobs[0].acquired_by == scheduler.identity\n assert jobs[0].acquired_until > datetime.now(timezone)\n\n trigger_event = anyio.Event()\n job_released_event: JobReleased | None = None\n\n def event_callback(event: Event) -> None:\n nonlocal job_released_event\n job_released_event = cast(JobReleased, event)\n trigger_event.set()\n\n # Start the scheduler and wait for the job to be processed\n with scheduler.subscribe(event_callback, {JobReleased}):\n await scheduler.start_in_background()\n with fail_after(scheduler.lease_duration.total_seconds() / 2):\n await trigger_event.wait()\n\n assert job_released_event\n assert job_released_event.job_id == job_id\n assert job_released_event.outcome is JobOutcome.abandoned\n assert not await scheduler.get_jobs()\n\n async def test_stop_scheduler_while_job_running(\n self, raw_datastore: DataStore\n ) -> None:\n event = anyio.Event()\n\n async def delay_job() -> None:\n event.set()\n await sleep(8)\n\n async with AsyncScheduler(data_store=raw_datastore) as scheduler:\n await scheduler.configure_task(\"delay_job\", func=delay_job)\n await scheduler.add_job(\"delay_job\", result_expiration_time=15)\n await scheduler.start_in_background()\n\n with fail_after(5):\n await event.wait()\n\n assert len(scheduler._running_jobs) == 1\n job_id = next(iter(scheduler._running_jobs)).id\n event = anyio.Event()\n scheduler.subscribe(lambda _: event.set(), SchedulerStopped, one_shot=True)\n await scheduler.stop()\n with fail_after(5):\n await event.wait()\n\n assert len(scheduler._running_jobs) == 0\n\n # Check that the task has 0 running jobs\n datastore_tasks = await scheduler.get_tasks()\n assert len(datastore_tasks) == 1\n assert datastore_tasks[0].running_jobs == 0\n\n # Check that the job was removed\n datastore_jobs = await scheduler.get_jobs()\n assert len(datastore_jobs) == 0\n\n # Check that the job outcome was set to \"cancelled\"\n result = await scheduler.get_job_result(job_id)\n assert result.outcome is JobOutcome.cancelled\n\n\nclass TestSyncScheduler:\n def test_interface_parity(self) -> None:\n \"\"\"\n Ensure that the sync scheduler has the same properties and methods as the async\n schedulers, and the method parameters match too.\n\n \"\"\"\n actual_attributes = set(dir(Scheduler))\n expected_attributes = sorted(\n attrname for attrname in dir(AsyncScheduler) if not attrname.startswith(\"_\")\n )\n for attrname in expected_attributes:\n if attrname not in actual_attributes:\n pytest.fail(f\"SyncScheduler is missing the {attrname} attribute\")\n\n async_attrval = getattr(AsyncScheduler, attrname)\n sync_attrval = getattr(Scheduler, attrname)\n if callable(async_attrval):\n async_sig = signature(async_attrval)\n async_args: dict[int, list] = defaultdict(list)\n for param in async_sig.parameters.values():\n if param.name not in (\"task_status\", \"is_async\"):\n async_args[param.kind].append(param)\n\n sync_sig = signature(sync_attrval)\n sync_args: dict[int, list] = defaultdict(list)\n for param in sync_sig.parameters.values():\n sync_args[param.kind].append(param)\n\n for kind, args in async_args.items():\n assert args == sync_args[kind], (\n f\"Parameter mismatch for {attrname}(): {args} != {sync_args[kind]}\"\n )\n\n def test_repr(self) -> None:\n scheduler = Scheduler(identity=\"my identity\")\n assert repr(scheduler) == (\n \"Scheduler(identity='my identity', role=, \"\n \"data_store=MemoryDataStore(), event_broker=LocalEventBroker())\"\n )\n\n def test_configure(self) -> None:\n executor = ThreadPoolJobExecutor()\n task_defaults = TaskDefaults(job_executor=\"executor1\")\n scheduler = Scheduler(\n identity=\"identity\",\n role=SchedulerRole.scheduler,\n max_concurrent_jobs=150,\n cleanup_interval=5,\n job_executors={\"executor1\": executor},\n task_defaults=task_defaults,\n )\n assert scheduler.identity == \"identity\"\n assert scheduler.role is SchedulerRole.scheduler\n assert scheduler.max_concurrent_jobs == 150\n assert scheduler.cleanup_interval == timedelta(seconds=5)\n assert scheduler.job_executors == {\"executor1\": executor}\n assert scheduler.task_defaults == task_defaults\n\n @pytest.mark.parametrize(\"as_default\", [False, True])\n def test_threadpool_executor(self, as_default: bool) -> None:\n with Scheduler() as scheduler:\n scheduler.start_in_background()\n if as_default:\n thread_id = scheduler.run_job(threading.get_ident)\n else:\n thread_id = scheduler.run_job(\n threading.get_ident, job_executor=\"threadpool\"\n )\n\n assert thread_id != threading.get_ident()\n\n def test_processpool_executor(self) -> None:\n with Scheduler() as scheduler:\n scheduler.start_in_background()\n pid = scheduler.run_job(os.getpid, job_executor=\"processpool\")\n assert pid != os.getpid()\n\n def test_properties(self) -> None:\n with Scheduler() as scheduler:\n assert isinstance(scheduler.data_store, MemoryDataStore)\n assert isinstance(scheduler.event_broker, LocalEventBroker)\n assert scheduler.role is SchedulerRole.both\n assert isinstance(scheduler.identity, str)\n assert len(scheduler.job_executors) == 3\n assert isinstance(scheduler.job_executors[\"async\"], AsyncJobExecutor)\n assert isinstance(\n scheduler.job_executors[\"threadpool\"], ThreadPoolJobExecutor\n )\n assert isinstance(\n scheduler.job_executors[\"processpool\"], ProcessPoolJobExecutor\n )\n assert isinstance(scheduler.task_defaults, TaskDefaults)\n assert scheduler.state is RunState.stopped\n\n def test_use_without_contextmanager(self, mocker: MockFixture) -> None:\n fake_atexit_register = mocker.patch(\"atexit.register\")\n scheduler = Scheduler()\n scheduler.subscribe(lambda event: None)\n fake_atexit_register.assert_called_once_with(scheduler._exit_stack.close)\n scheduler._exit_stack.close()\n\n def test_configure_task(self) -> None:\n queue: Queue[Event] = Queue()\n with Scheduler() as scheduler:\n scheduler.subscribe(queue.put_nowait)\n scheduler.configure_task(\"mytask\", func=dummy_sync_job)\n scheduler.configure_task(\"mytask\", misfire_grace_time=2)\n tasks = scheduler.get_tasks()\n assert len(tasks) == 1\n assert tasks[0].id == \"mytask\"\n assert tasks[0].func == f\"{__name__}:dummy_sync_job\"\n assert tasks[0].misfire_grace_time == timedelta(seconds=2)\n\n event = queue.get(timeout=1)\n assert isinstance(event, TaskAdded)\n assert event.task_id == \"mytask\"\n\n event = queue.get(timeout=1)\n assert isinstance(event, TaskUpdated)\n assert event.task_id == \"mytask\"\n\n def test_add_remove_schedule(self, timezone: ZoneInfo) -> None:\n queue: Queue[Event] = Queue()\n with Scheduler() as scheduler:\n scheduler.subscribe(queue.put_nowait)\n now = datetime.now(timezone)\n trigger = DateTrigger(now)\n schedule_id = scheduler.add_schedule(dummy_async_job, trigger, id=\"foo\")\n assert schedule_id == \"foo\"\n\n schedules = scheduler.get_schedules()\n assert len(schedules) == 1\n assert schedules[0].id == \"foo\"\n assert schedules[0].task_id == f\"{__name__}:dummy_async_job\"\n\n schedule = scheduler.get_schedule(schedule_id)\n assert schedules[0] == schedule\n\n scheduler.remove_schedule(schedule_id)\n assert not scheduler.get_schedules()\n\n event = queue.get(timeout=1)\n assert isinstance(event, TaskAdded)\n assert event.task_id == f\"{__name__}:dummy_async_job\"\n\n event = queue.get(timeout=1)\n assert isinstance(event, ScheduleAdded)\n assert event.schedule_id == \"foo\"\n assert event.next_fire_time == now\n\n event = queue.get(timeout=1)\n assert isinstance(event, ScheduleRemoved)\n assert event.schedule_id == \"foo\"\n assert not event.finished\n\n def test_add_job_wait_result(self) -> None:\n queue: Queue[Event] = Queue()\n with Scheduler() as scheduler:\n assert scheduler.get_jobs() == []\n\n scheduler.subscribe(queue.put_nowait)\n job_id = scheduler.add_job(dummy_sync_job, result_expiration_time=10)\n\n event = queue.get(timeout=1)\n assert isinstance(event, TaskAdded)\n assert event.task_id == f\"{__name__}:dummy_sync_job\"\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobAdded)\n assert event.job_id == job_id\n\n jobs = scheduler.get_jobs()\n assert len(jobs) == 1\n assert jobs[0].id == job_id\n assert jobs[0].task_id == f\"{__name__}:dummy_sync_job\"\n\n with pytest.raises(JobLookupError):\n scheduler.get_job_result(job_id, wait=False)\n\n scheduler.start_in_background()\n\n event = queue.get(timeout=1)\n assert isinstance(event, SchedulerStarted)\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobAcquired)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_sync_job\"\n assert event.schedule_id is None\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobReleased)\n assert event.job_id == job_id\n assert event.task_id == f\"{__name__}:dummy_sync_job\"\n assert event.schedule_id is None\n assert event.outcome is JobOutcome.success\n\n result = scheduler.get_job_result(job_id)\n assert result\n assert result.outcome is JobOutcome.success\n assert result.return_value == \"returnvalue\"\n\n def test_wait_until_stopped(self) -> None:\n queue: Queue[Event] = Queue()\n with Scheduler() as scheduler:\n scheduler.configure_task(\"stop\", func=scheduler.stop)\n scheduler.add_job(\"stop\")\n scheduler.subscribe(queue.put_nowait)\n scheduler.start_in_background()\n scheduler.wait_until_stopped()\n\n event = queue.get(timeout=1)\n assert isinstance(event, SchedulerStarted)\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobAcquired)\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobReleased)\n\n event = queue.get(timeout=1)\n assert isinstance(event, SchedulerStopped)\n\n def test_explicit_cleanup(self) -> None:\n with Scheduler(cleanup_interval=None) as scheduler:\n event = threading.Event()\n scheduler.add_schedule(\n event.set, DateTrigger(datetime.now(timezone.utc)), id=\"event_set\"\n )\n scheduler.start_in_background()\n assert event.wait(3)\n\n # The schedule should still be around, but with a null next_fire_time\n schedule = scheduler.get_schedule(\"event_set\")\n assert schedule.next_fire_time is None\n\n # After the cleanup, the schedule should be gone\n scheduler.cleanup()\n with pytest.raises(ScheduleLookupError):\n scheduler.get_schedule(\"event_set\")\n\n def test_run_until_stopped(self) -> None:\n queue: Queue[Event] = Queue()\n with Scheduler() as scheduler:\n scheduler.configure_task(\"stop\", func=scheduler.stop)\n scheduler.add_job(\"stop\")\n scheduler.subscribe(queue.put_nowait)\n scheduler.run_until_stopped()\n\n event = queue.get(timeout=1)\n assert isinstance(event, SchedulerStarted)\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobAcquired)\n\n event = queue.get(timeout=1)\n assert isinstance(event, JobReleased)\n\n event = queue.get(timeout=1)\n assert isinstance(event, SchedulerStopped)\n\n def test_uwsgi_threads_error(self, monkeypatch: MonkeyPatch) -> None:\n mod = ModuleType(\"uwsgi\")\n monkeypatch.setitem(sys.modules, \"uwsgi\", mod)\n mod.has_threads = False\n with pytest.raises(\n RuntimeError, match=\"The scheduler seems to be running under uWSGI\"\n ):\n Scheduler().start_in_background()\n" }, { "path": "tests/test_serializers.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timezone\nfrom typing import NoReturn\nfrom uuid import uuid4\n\nimport pytest\n\nfrom apscheduler import (\n DeserializationError,\n Event,\n JobAdded,\n JobOutcome,\n JobReleased,\n SerializationError,\n)\nfrom apscheduler.abc import Serializer\n\n\n@pytest.mark.parametrize(\n \"event\",\n [\n pytest.param(\n JobAdded(\n job_id=uuid4(),\n task_id=\"task\",\n schedule_id=\"schedule\",\n ),\n id=\"job_added\",\n ),\n pytest.param(\n JobReleased(\n job_id=uuid4(),\n scheduler_id=\"testscheduler\",\n task_id=\"task\",\n schedule_id=\"schedule\",\n outcome=JobOutcome.success,\n scheduled_start=datetime.now(timezone.utc),\n started_at=datetime.now(timezone.utc),\n ),\n id=\"job_released\",\n ),\n ],\n)\ndef test_serialize_event(event: Event, serializer: Serializer) -> None:\n payload = serializer.serialize(event.marshal())\n deserialized = type(event).unmarshal(serializer.deserialize(payload))\n assert deserialized == event\n\n\ndef test_serialization_error(serializer: Serializer) -> None:\n class Unserializable:\n def __getstate__(self) -> NoReturn:\n raise ValueError(\"cannot be serialized\")\n\n # An open file cannot be serialized\n with pytest.raises(SerializationError) as exc:\n serializer.serialize(Unserializable())\n\n assert isinstance(exc.value.__cause__, ValueError)\n\n\n@pytest.mark.parametrize(\n \"payload\",\n [\n pytest.param(b\"\", id=\"empty\"),\n pytest.param(b\"\\x61\\x98\", id=\"invalid\"),\n ],\n)\ndef test_deserialization_error(payload: bytes, serializer: Serializer) -> None:\n with pytest.raises(DeserializationError) as exc:\n serializer.deserialize(payload)\n\n assert exc.value.__cause__ is not None\n" }, { "path": "tests/triggers/test_calendarinterval.py", "content": "from __future__ import annotations\n\nfrom datetime import date, datetime\n\nimport pytest\n\nfrom apscheduler.triggers.calendarinterval import CalendarIntervalTrigger\n\n\ndef test_bad_interval(timezone):\n exc = pytest.raises(ValueError, CalendarIntervalTrigger, timezone=timezone)\n exc.match(\"interval must be at least 1 day long\")\n\n\ndef test_bad_start_end_dates(timezone):\n exc = pytest.raises(\n ValueError,\n CalendarIntervalTrigger,\n days=1,\n start_date=date(2016, 3, 4),\n end_date=date(2016, 3, 3),\n timezone=timezone,\n )\n exc.match(\"end_date cannot be earlier than start_date\")\n\n\ndef test_end_date(timezone, serializer):\n \"\"\"Test that end_date is respected.\"\"\"\n start_end_date = date(2020, 12, 31)\n trigger = CalendarIntervalTrigger(\n days=1, start_date=start_end_date, end_date=start_end_date, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next().date() == start_end_date\n assert trigger.next() is None\n\n\ndef test_missing_time(timezone, serializer):\n \"\"\"\n Test that if the designated time does not exist on a day due to a forward DST shift,\n the day is skipped entirely.\n\n \"\"\"\n trigger = CalendarIntervalTrigger(\n days=1, hour=2, minute=30, start_date=date(2016, 3, 27), timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2016, 3, 28, 2, 30, tzinfo=timezone)\n\n\ndef test_repeated_time(timezone, serializer):\n \"\"\"\n Test that if the designated time is repeated during a day due to a backward DST\n shift, the task is executed on the earlier occurrence of that time.\n\n \"\"\"\n trigger = CalendarIntervalTrigger(\n days=2, hour=2, minute=30, start_date=date(2016, 10, 30), timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2016, 10, 30, 2, 30, tzinfo=timezone, fold=0)\n\n\ndef test_nonexistent_days(timezone, serializer):\n \"\"\"Test that invalid dates are skipped.\"\"\"\n trigger = CalendarIntervalTrigger(\n months=1, start_date=date(2016, 3, 31), timezone=timezone\n )\n assert trigger.next() == datetime(2016, 3, 31, tzinfo=timezone)\n\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2016, 5, 31, tzinfo=timezone)\n\n\ndef test_repr(timezone, serializer):\n trigger = CalendarIntervalTrigger(\n years=1,\n months=5,\n weeks=6,\n days=8,\n hour=3,\n second=8,\n start_date=date(2016, 3, 5),\n end_date=date(2020, 12, 25),\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert repr(trigger) == (\n \"CalendarIntervalTrigger(years=1, months=5, weeks=6, days=8, \"\n \"time='03:00:08', start_date='2016-03-05', end_date='2020-12-25', \"\n \"timezone='Europe/Berlin')\"\n )\n\n\ndef test_utc_timezone(utc_timezone):\n trigger = CalendarIntervalTrigger(\n days=1, hour=1, start_date=date(2016, 3, 31), timezone=utc_timezone\n )\n assert trigger.next() == datetime(2016, 3, 31, 1, tzinfo=utc_timezone)\n" }, { "path": "tests/triggers/test_combining.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timedelta, timezone\n\nimport pytest\n\nfrom apscheduler import MaxIterationsReached\nfrom apscheduler.triggers.calendarinterval import CalendarIntervalTrigger\nfrom apscheduler.triggers.combining import AndTrigger, OrTrigger\nfrom apscheduler.triggers.cron import CronTrigger\nfrom apscheduler.triggers.date import DateTrigger\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\nclass TestAndTrigger:\n @pytest.mark.parametrize(\"threshold\", [1, 0])\n def test_two_datetriggers(self, timezone, serializer, threshold):\n date1 = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n date2 = datetime(2020, 5, 16, 14, 17, 31, 254212, tzinfo=timezone)\n trigger = AndTrigger(\n [DateTrigger(date1), DateTrigger(date2)], threshold=threshold\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n if threshold:\n # date2 was within the threshold so it will not be used\n assert trigger.next() == date1\n\n assert trigger.next() is None\n\n def test_max_iterations(self, timezone, serializer):\n start_time = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n trigger = AndTrigger(\n [\n IntervalTrigger(seconds=4, start_time=start_time),\n IntervalTrigger(\n seconds=4, start_time=start_time + timedelta(seconds=2)\n ),\n ]\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n pytest.raises(MaxIterationsReached, trigger.next)\n\n def test_repr(self, timezone, serializer):\n start_time = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n trigger = AndTrigger(\n [\n IntervalTrigger(seconds=4, start_time=start_time),\n IntervalTrigger(\n seconds=4, start_time=start_time + timedelta(seconds=2)\n ),\n ]\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert repr(trigger) == (\n \"AndTrigger([IntervalTrigger(seconds=4, \"\n \"start_time='2020-05-16 14:17:30.254212+02:00'), IntervalTrigger(\"\n \"seconds=4, start_time='2020-05-16 14:17:32.254212+02:00')], \"\n \"threshold=1.0, max_iterations=10000)\"\n )\n\n @pytest.mark.parametrize(\n \"left_trigger,right_trigger,expected_datetimes\",\n [\n (\n IntervalTrigger(\n hours=6, start_time=datetime(2024, 5, 1, tzinfo=timezone.utc)\n ),\n IntervalTrigger(\n hours=12, start_time=datetime(2024, 5, 1, tzinfo=timezone.utc)\n ),\n [\n datetime(2024, 5, 1, 0, tzinfo=timezone.utc),\n datetime(2024, 5, 1, 12, tzinfo=timezone.utc),\n datetime(2024, 5, 2, 0, tzinfo=timezone.utc),\n ],\n ),\n (\n IntervalTrigger(\n days=1, start_time=datetime(2024, 5, 1, tzinfo=timezone.utc)\n ),\n IntervalTrigger(\n weeks=1, start_time=datetime(2024, 5, 1, tzinfo=timezone.utc)\n ),\n [\n datetime(2024, 5, 1, tzinfo=timezone.utc),\n datetime(2024, 5, 8, tzinfo=timezone.utc),\n datetime(2024, 5, 15, tzinfo=timezone.utc),\n ],\n ),\n (\n CronTrigger(\n day_of_week=\"mon-fri\",\n hour=\"*\",\n timezone=timezone.utc,\n start_time=datetime(2024, 5, 3, tzinfo=timezone.utc),\n ),\n IntervalTrigger(\n hours=12, start_time=datetime(2024, 5, 3, tzinfo=timezone.utc)\n ),\n [\n datetime(2024, 5, 3, 0, tzinfo=timezone.utc),\n datetime(2024, 5, 3, 12, tzinfo=timezone.utc),\n datetime(2024, 5, 6, 0, tzinfo=timezone.utc),\n ],\n ),\n (\n CronTrigger(\n day_of_week=\"mon-fri\",\n timezone=timezone.utc,\n start_time=datetime(2024, 5, 13, tzinfo=timezone.utc),\n ),\n IntervalTrigger(\n days=4, start_time=datetime(2024, 5, 13, tzinfo=timezone.utc)\n ),\n [\n datetime(2024, 5, 13, tzinfo=timezone.utc),\n datetime(2024, 5, 17, tzinfo=timezone.utc),\n datetime(2024, 5, 21, tzinfo=timezone.utc),\n datetime(2024, 5, 29, tzinfo=timezone.utc),\n ],\n ),\n (\n CalendarIntervalTrigger(\n months=1,\n timezone=timezone.utc,\n start_date=datetime(2024, 1, 1, tzinfo=timezone.utc),\n ),\n CronTrigger(\n day_of_week=\"mon-fri\",\n timezone=timezone.utc,\n start_time=datetime(2024, 1, 1, tzinfo=timezone.utc),\n ),\n [\n datetime(2024, 1, 1, tzinfo=timezone.utc),\n datetime(2024, 2, 1, tzinfo=timezone.utc),\n datetime(2024, 3, 1, tzinfo=timezone.utc),\n datetime(2024, 4, 1, tzinfo=timezone.utc),\n datetime(2024, 5, 1, tzinfo=timezone.utc),\n datetime(2024, 7, 1, tzinfo=timezone.utc),\n datetime(2024, 8, 1, tzinfo=timezone.utc),\n datetime(2024, 10, 1, tzinfo=timezone.utc),\n datetime(2024, 11, 1, tzinfo=timezone.utc),\n ],\n ),\n ],\n )\n def test_overlapping_triggers(\n self, left_trigger, right_trigger, expected_datetimes\n ):\n \"\"\"\n Verify that the `AndTrigger` fires at the intersection of two triggers.\n \"\"\"\n and_trigger = AndTrigger([left_trigger, right_trigger])\n for expected_datetime in expected_datetimes:\n next_datetime = and_trigger.next()\n assert next_datetime == expected_datetime\n\n\nclass TestOrTrigger:\n def test_two_datetriggers(self, timezone, serializer):\n date1 = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n date2 = datetime(2020, 5, 18, 15, 1, 53, 940564, tzinfo=timezone)\n trigger = OrTrigger([DateTrigger(date1), DateTrigger(date2)])\n\n assert trigger.next() == date1\n\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == date2\n assert trigger.next() is None\n\n def test_two_interval_triggers(self, timezone, serializer):\n start_time = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n end_time1 = start_time + timedelta(seconds=16)\n end_time2 = start_time + timedelta(seconds=18)\n trigger = OrTrigger(\n [\n IntervalTrigger(seconds=4, start_time=start_time, end_time=end_time1),\n IntervalTrigger(seconds=6, start_time=start_time, end_time=end_time2),\n ]\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == start_time\n assert trigger.next() == start_time + timedelta(seconds=4)\n assert trigger.next() == start_time + timedelta(seconds=6)\n assert trigger.next() == start_time + timedelta(seconds=8)\n assert trigger.next() == start_time + timedelta(seconds=12)\n assert trigger.next() == start_time + timedelta(seconds=16)\n # The end time of the 4 second interval has been reached\n assert trigger.next() == start_time + timedelta(seconds=18)\n # The end time of the 6 second interval has been reached\n assert trigger.next() is None\n\n def test_repr(self, timezone):\n date1 = datetime(2020, 5, 16, 14, 17, 30, 254212, tzinfo=timezone)\n date2 = datetime(2020, 5, 18, 15, 1, 53, 940564, tzinfo=timezone)\n trigger = OrTrigger([DateTrigger(date1), DateTrigger(date2)])\n assert repr(trigger) == (\n \"OrTrigger([DateTrigger('2020-05-16 14:17:30.254212+02:00'), \"\n \"DateTrigger('2020-05-18 15:01:53.940564+02:00')])\"\n )\n" }, { "path": "tests/triggers/test_cron.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime\nfrom zoneinfo import ZoneInfo\n\nimport pytest\n\nfrom apscheduler.triggers.cron import CronTrigger\n\n\ndef test_invalid_expression():\n exc = pytest.raises(ValueError, CronTrigger, year=\"2009-fault\")\n exc.match(\"Unrecognized expression '2009-fault' for field 'year'\")\n\n\ndef test_invalid_step():\n exc = pytest.raises(ValueError, CronTrigger, year=\"2009/0\")\n exc.match(\"step must be positive, got: 0\")\n\n\ndef test_invalid_range():\n exc = pytest.raises(ValueError, CronTrigger, year=\"2009-2008\")\n exc.match(\"The minimum value in a range must not be higher than the maximum\")\n\n\n@pytest.mark.parametrize(\"expr\", [\"fab\", \"jan-fab\"], ids=[\"start\", \"end\"])\ndef test_invalid_month_name(expr):\n exc = pytest.raises(ValueError, CronTrigger, month=expr)\n exc.match(\"Invalid month name 'fab'\")\n\n\n@pytest.mark.parametrize(\"expr\", [\"web\", \"mon-web\"], ids=[\"start\", \"end\"])\ndef test_invalid_weekday_name(expr):\n exc = pytest.raises(ValueError, CronTrigger, day_of_week=expr)\n exc.match(\"Invalid weekday name 'web'\")\n\n\ndef test_invalid_weekday_position_name():\n exc = pytest.raises(ValueError, CronTrigger, day=\"1st web\")\n exc.match(\"Invalid weekday name 'web'\")\n\n\n@pytest.mark.parametrize(\n \"values, expected\",\n [\n (\n {\"day\": \"*/31\"},\n r\"Error validating expression '\\*/31': the step value \\(31\\) is higher \"\n r\"than the total range of the expression \\(30\\)\",\n ),\n (\n {\"day\": \"4-6/3\"},\n r\"Error validating expression '4-6/3': the step value \\(3\\) is higher \"\n r\"than the total range of the expression \\(2\\)\",\n ),\n (\n {\"hour\": \"0-24\"},\n r\"Error validating expression '0-24': the last value \\(24\\) is higher \"\n r\"than the maximum value \\(23\\)\",\n ),\n (\n {\"day\": \"0-3\"},\n r\"Error validating expression '0-3': the first value \\(0\\) is lower \"\n r\"than the minimum value \\(1\\)\",\n ),\n ],\n ids=[\n \"too_large_step_all\",\n \"too_large_step_range\",\n \"too_high_last\",\n \"too_low_first\",\n ],\n)\ndef test_invalid_ranges(values, expected):\n pytest.raises(ValueError, CronTrigger, **values).match(expected)\n\n\ndef test_cron_trigger_1(timezone, serializer):\n start_time = datetime(2008, 12, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=\"2009/2\",\n month=\"1-4/3\",\n day=\"5-6\",\n start_time=start_time,\n timezone=timezone,\n )\n\n # since `next` is modifying the trigger, we call it before serializing\n # to make sure the serialization works correctly also for modified triggers\n assert trigger.next() == datetime(2009, 1, 5, tzinfo=timezone)\n\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 1, 6, tzinfo=timezone)\n assert trigger.next() == datetime(2009, 4, 5, tzinfo=timezone)\n assert trigger.next() == datetime(2009, 4, 6, tzinfo=timezone)\n assert trigger.next() == datetime(2011, 1, 5, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2009/2', month='1-4/3', day='5-6', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2008-12-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_cron_trigger_2(timezone, serializer):\n start_time = datetime(2009, 10, 14, tzinfo=timezone)\n trigger = CronTrigger(\n year=\"2009/2\", month=\"1-3\", day=\"5\", start_time=start_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2011, 1, 5, tzinfo=timezone)\n assert trigger.next() == datetime(2011, 2, 5, tzinfo=timezone)\n assert trigger.next() == datetime(2011, 3, 5, tzinfo=timezone)\n assert trigger.next() == datetime(2013, 1, 5, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2009/2', month='1-3', day='5', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2009-10-14T00:00:00+02:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_cron_trigger_3(timezone, serializer):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=\"2009\",\n month=\"feb-dec\",\n hour=\"8-9\",\n start_time=start_time,\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 2, 1, 8, tzinfo=timezone)\n assert trigger.next() == datetime(2009, 2, 1, 9, tzinfo=timezone)\n assert trigger.next() == datetime(2009, 2, 2, 8, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='feb-dec', day='*', week='*', \"\n \"day_of_week='*', hour='8-9', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_cron_trigger_4(timezone, serializer):\n start_time = datetime(2012, 2, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=\"2012\", month=\"2\", day=\"last\", start_time=start_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2012, 2, 29, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2012', month='2', day='last', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2012-02-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\n@pytest.mark.parametrize(\"expr\", [\"3-5\", \"wed-fri\"], ids=[\"numeric\", \"text\"])\ndef test_weekday_overlap(timezone, serializer, expr):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009,\n month=1,\n day=\"6-10\",\n day_of_week=expr,\n start_time=start_time,\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 1, 7, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='1', day='6-10', week='*', \"\n \"day_of_week='wed-fri', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_weekday_range(timezone, serializer):\n start_time = datetime(2020, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2020,\n month=1,\n week=1,\n day_of_week=\"fri-sun\",\n start_time=start_time,\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2020, 1, 3, tzinfo=timezone)\n assert trigger.next() == datetime(2020, 1, 4, tzinfo=timezone)\n assert trigger.next() == datetime(2020, 1, 5, tzinfo=timezone)\n assert trigger.next() is None\n assert repr(trigger) == (\n \"CronTrigger(year='2020', month='1', day='*', week='1', \"\n \"day_of_week='fri-sun', hour='0', minute='0', second='0', \"\n \"start_time='2020-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_last_weekday(timezone, serializer):\n start_time = datetime(2020, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2020, day=\"last sun\", start_time=start_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2020, 1, 26, tzinfo=timezone)\n assert trigger.next() == datetime(2020, 2, 23, tzinfo=timezone)\n assert trigger.next() == datetime(2020, 3, 29, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2020', month='*', day='last sun', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2020-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_increment_weekday(timezone, serializer):\n \"\"\"\n Tests that incrementing the weekday field in the process of calculating the next\n matching date won't cause problems.\n\n \"\"\"\n start_time = datetime(2009, 9, 25, 7, tzinfo=timezone)\n trigger = CronTrigger(hour=\"5-6\", start_time=start_time, timezone=timezone)\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 9, 26, 5, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='*', hour='5-6', minute='0', second='0', \"\n \"start_time='2009-09-25T07:00:00+02:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_month_rollover(timezone, serializer):\n start_time = datetime(2016, 2, 1, tzinfo=timezone)\n trigger = CronTrigger(day=30, start_time=start_time, timezone=timezone)\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2016, 3, 30, tzinfo=timezone)\n assert trigger.next() == datetime(2016, 4, 30, tzinfo=timezone)\n\n\n@pytest.mark.parametrize(\"weekday\", [\"1,0\", \"mon,sun\"], ids=[\"numeric\", \"text\"])\ndef test_weekday_nomatch(timezone, serializer, weekday):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009,\n month=1,\n day=\"6-10\",\n day_of_week=weekday,\n start_time=start_time,\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() is None\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='1', day='6-10', week='*', \"\n \"day_of_week='mon,sun', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_weekday_positional(timezone, serializer):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009, month=1, day=\"4th wed\", start_time=start_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 1, 28, tzinfo=timezone)\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='1', day='4th wed', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_end_time(timezone, serializer):\n \"\"\"Test that next() won't produce\"\"\"\n start_time = datetime(2014, 4, 13, 2, tzinfo=timezone)\n end_time = datetime(2014, 4, 13, 4, tzinfo=timezone)\n trigger = CronTrigger(\n hour=4, start_time=start_time, end_time=end_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2014, 4, 13, 4, tzinfo=timezone)\n assert trigger.next() is None\n assert repr(trigger) == (\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='*', hour='4', minute='0', second='0', \"\n \"start_time='2014-04-13T02:00:00+02:00', \"\n \"end_time='2014-04-13T04:00:00+02:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_week_1(timezone, serializer):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009, month=2, week=8, start_time=start_time, timezone=timezone\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n for day in range(16, 23):\n assert trigger.next() == datetime(2009, 2, day, tzinfo=timezone)\n\n assert trigger.next() is None\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='2', day='*', week='8', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\n@pytest.mark.parametrize(\"weekday\", [3, \"wed\"], ids=[\"numeric\", \"text\"])\ndef test_week_2(timezone, serializer, weekday):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009,\n week=15,\n day_of_week=weekday,\n start_time=start_time,\n timezone=timezone,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == datetime(2009, 4, 8, tzinfo=timezone)\n assert trigger.next() is None\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='*', day='*', week='15', \"\n \"day_of_week='wed', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\n@pytest.mark.parametrize(\n \"trigger_args, start_time, start_time_fold, correct_next_date,\"\n \"correct_next_date_fold\",\n [\n ({\"hour\": 2}, datetime(2013, 3, 9, 20), 0, datetime(2013, 3, 11, 2), 0),\n ({\"hour\": 8}, datetime(2013, 3, 9, 12), 0, datetime(2013, 3, 10, 8), 0),\n ({\"hour\": 8}, datetime(2013, 11, 2, 12), 0, datetime(2013, 11, 3, 8), 0),\n (\n {\"minute\": \"*/30\"},\n datetime(2013, 3, 10, 1, 35),\n 0,\n datetime(2013, 3, 10, 3),\n 0,\n ),\n (\n {\"minute\": \"*/30\"},\n datetime(2013, 11, 3, 1, 35),\n 0,\n datetime(2013, 11, 3, 1),\n 1,\n ),\n ],\n ids=[\n \"spring_skip_hour\",\n \"absolute_spring\",\n \"absolute_autumn\",\n \"interval_spring\",\n \"interval_autumn\",\n ],\n)\ndef test_dst_change(\n trigger_args,\n start_time,\n start_time_fold,\n correct_next_date,\n correct_next_date_fold,\n serializer,\n):\n \"\"\"\n Making sure that CronTrigger works correctly when crossing the DST switch threshold.\n Note that you should explicitly compare datetimes as strings to avoid the internal\n datetime comparison which would test for equality in the UTC timezone.\n\n \"\"\"\n timezone = ZoneInfo(\"US/Eastern\")\n start_time = start_time.replace(tzinfo=timezone, fold=start_time_fold)\n trigger = CronTrigger(timezone=timezone, start_time=start_time, **trigger_args)\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == correct_next_date.replace(\n tzinfo=timezone, fold=correct_next_date_fold\n )\n\n\n@pytest.mark.parametrize(\n \"minute, start_time, correct_next_dates\",\n [\n (\n 0,\n datetime(2024, 10, 27, 2, 0, 0, 0),\n [\n (datetime(2024, 10, 27, 2, 0, 0, 0), 0),\n (datetime(2024, 10, 27, 2, 0, 0, 0), 1),\n (datetime(2024, 10, 27, 3, 0, 0, 0), 0),\n ],\n ),\n (\n 1,\n datetime(2024, 10, 27, 2, 1, 0, 0),\n [\n (datetime(2024, 10, 27, 2, 1, 0, 0), 0),\n (datetime(2024, 10, 27, 2, 1, 0, 0), 1),\n (datetime(2024, 10, 27, 3, 1, 0, 0), 0),\n ],\n ),\n ],\n ids=[\"dst_change_0\", \"dst_change_1\"],\n)\ndef test_dst_change2(\n minute,\n start_time,\n correct_next_dates,\n timezone,\n):\n trigger = CronTrigger(minute=minute, timezone=timezone)\n trigger.start_time = start_time.replace(tzinfo=timezone)\n for correct_next_date, fold in correct_next_dates:\n correct_next_date = correct_next_date.replace(tzinfo=timezone, fold=fold)\n next_date = trigger.next()\n assert next_date == correct_next_date\n assert str(next_date) == str(correct_next_date)\n\n\ndef test_zero_value(timezone):\n start_time = datetime(2020, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(\n year=2009, month=2, hour=0, start_time=start_time, timezone=timezone\n )\n assert repr(trigger) == (\n \"CronTrigger(year='2009', month='2', day='*', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2020-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n\n\ndef test_year_list(timezone, serializer):\n start_time = datetime(2009, 1, 1, tzinfo=timezone)\n trigger = CronTrigger(year=\"2009,2008\", start_time=start_time, timezone=timezone)\n assert (\n repr(trigger) == \"CronTrigger(year='2009,2008', month='1', day='1', week='*', \"\n \"day_of_week='*', hour='0', minute='0', second='0', \"\n \"start_time='2009-01-01T00:00:00+01:00', timezone='Europe/Berlin')\"\n )\n assert trigger.next() == datetime(2009, 1, 1, tzinfo=timezone)\n assert trigger.next() is None\n\n\n@pytest.mark.parametrize(\n \"expr, expected_repr\",\n [\n (\n \"* * * * *\",\n \"CronTrigger(year='*', month='*', day='*', week='*', day_of_week='*', \"\n \"hour='*', minute='*', second='0', start_time='2020-05-19T19:53:22+02:00', \"\n \"timezone='Europe/Berlin')\",\n ),\n (\n \"0-14 * 14-28 jul fri\",\n \"CronTrigger(year='*', month='jul', day='14-28', week='*', \"\n \"day_of_week='fri', hour='*', minute='0-14', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n (\n \" 0-14 * 14-28 jul fri\",\n \"CronTrigger(year='*', month='jul', day='14-28', week='*', \"\n \"day_of_week='fri', hour='*', minute='0-14', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n (\n \"* * * * 1-5\",\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='mon-fri', hour='*', minute='*', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n (\n \"* * * * 0-3\",\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='mon-wed,sun', hour='*', minute='*', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n (\n \"* * * * 6-1\",\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='mon,sat-sun', hour='*', minute='*', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n (\n \"* * * * 6-7\",\n \"CronTrigger(year='*', month='*', day='*', week='*', \"\n \"day_of_week='sat-sun', hour='*', minute='*', second='0', \"\n \"start_time='2020-05-19T19:53:22+02:00', timezone='Europe/Berlin')\",\n ),\n ],\n ids=[\n \"always\",\n \"assorted\",\n \"multiple_spaces_in_format\",\n \"working_week\",\n \"sunday_first\",\n \"saturday_first\",\n \"weekend\",\n ],\n)\ndef test_from_crontab(expr, expected_repr, timezone, serializer):\n trigger = CronTrigger.from_crontab(expr, timezone=timezone)\n trigger.start_time = datetime(2020, 5, 19, 19, 53, 22, tzinfo=timezone)\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert repr(trigger) == expected_repr\n\n\ndef test_from_crontab_wrong_number_of_fields():\n exc = pytest.raises(ValueError, CronTrigger.from_crontab, \"*\")\n exc.match(\"Wrong number of fields; got 1, expected 5\")\n\n\ndef test_from_crontab_start_end_time(timezone: ZoneInfo) -> None:\n start_time = datetime(2020, 5, 19, 19, 53, 22, tzinfo=timezone)\n end_time = datetime(2020, 5, 24, 19, 53, 22, tzinfo=timezone)\n trigger = CronTrigger.from_crontab(\n \"* * * * *\", start_time=start_time, end_time=end_time\n )\n assert trigger.start_time == start_time\n assert trigger.end_time == end_time\n\n\ndef test_aware_start_time_timezone_conversion() -> None:\n est = ZoneInfo(\"America/New_York\")\n cst = ZoneInfo(\"America/Chicago\")\n start_time = datetime(2009, 9, 26, 10, 16, tzinfo=cst)\n trigger = CronTrigger(hour=11, minute=\"*/5\", timezone=est, start_time=start_time)\n correct_next_time = datetime(2009, 9, 26, 11, 20, tzinfo=est)\n next_time = trigger.next()\n assert str(next_time) == str(correct_next_time)\n\n\ndef test_aware_end_time_timezone_conversion() -> None:\n est = ZoneInfo(\"America/New_York\")\n cst = ZoneInfo(\"America/Chicago\")\n start_time = datetime(2009, 9, 26, 10, 16, tzinfo=cst)\n end_time = datetime(2009, 9, 26, 11, tzinfo=est)\n trigger = CronTrigger(\n hour=10, minute=\"*/5\", timezone=cst, start_time=start_time, end_time=end_time\n )\n next_time = trigger.next()\n assert next_time is None\n\n\ndef test_non_existing_naive_start_time() -> None:\n tz = ZoneInfo(\"Europe/Berlin\")\n start_time = datetime(2025, 3, 30, 2, 30, tzinfo=tz)\n with pytest.raises(ValueError):\n CronTrigger(timezone=tz, start_time=start_time)\n\n\ndef test_non_existing_naive_end_time() -> None:\n tz = ZoneInfo(\"Europe/Berlin\")\n start_time = datetime(2025, 3, 30, 1, 30)\n CronTrigger(timezone=tz, start_time=start_time) # start time is ok\n end_time = datetime(2025, 3, 30, 2, 30)\n with pytest.raises(ValueError):\n CronTrigger(timezone=tz, start_time=start_time, end_time=end_time)\n" }, { "path": "tests/triggers/test_date.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime\n\nfrom apscheduler.triggers.date import DateTrigger\n\n\ndef test_run_time(timezone, serializer):\n run_time = datetime(2020, 5, 14, 11, 56, 12, tzinfo=timezone)\n trigger = DateTrigger(run_time)\n if serializer:\n payload = serializer.serialize(trigger)\n trigger = serializer.deserialize(payload)\n\n assert trigger.next() == run_time\n assert trigger.next() is None\n assert repr(trigger) == \"DateTrigger('2020-05-14 11:56:12+02:00')\"\n" }, { "path": "tests/triggers/test_interval.py", "content": "from __future__ import annotations\n\nfrom datetime import datetime, timedelta\n\nimport pytest\n\nfrom apscheduler.triggers.interval import IntervalTrigger\n\n\ndef test_bad_interval():\n exc = pytest.raises(ValueError, IntervalTrigger)\n exc.match(\"The time interval must be positive\")\n\n\ndef test_bad_end_time(timezone):\n start_time = datetime(2020, 5, 16, tzinfo=timezone)\n end_time = datetime(2020, 5, 15, tzinfo=timezone)\n exc = pytest.raises(\n ValueError, IntervalTrigger, seconds=1, start_time=start_time, end_time=end_time\n )\n exc.match(\"end_time cannot be earlier than start_time\")\n\n\ndef test_end_time(timezone, serializer):\n start_time = datetime(2020, 5, 16, 19, 32, 44, 649521, tzinfo=timezone)\n end_time = datetime(2020, 5, 16, 22, 33, 1, tzinfo=timezone)\n interval = timedelta(hours=1, seconds=6)\n trigger = IntervalTrigger(\n start_time=start_time, end_time=end_time, hours=1, seconds=6\n )\n assert trigger.next() == start_time\n\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert trigger.next() == start_time + interval\n assert trigger.next() == start_time + interval * 2\n assert trigger.next() is None\n\n\ndef test_repr(timezone, serializer):\n start_time = datetime(2020, 5, 15, 12, 55, 32, 954032, tzinfo=timezone)\n end_time = datetime(2020, 6, 4, 16, 18, 49, 306942, tzinfo=timezone)\n trigger = IntervalTrigger(\n weeks=1,\n days=2,\n hours=3,\n minutes=4,\n seconds=5,\n microseconds=123525,\n start_time=start_time,\n end_time=end_time,\n )\n if serializer:\n trigger = serializer.deserialize(serializer.serialize(trigger))\n\n assert repr(trigger) == (\n \"IntervalTrigger(weeks=1, days=2, hours=3, minutes=4, seconds=5, \"\n \"microseconds=123525, start_time='2020-05-15 12:55:32.954032+02:00', \"\n \"end_time='2020-06-04 16:18:49.306942+02:00')\"\n )\n" } ]