Showing preview only (401K chars total). Download the full file or copy to clipboard to get everything.
Repository: mher/flower
Branch: master
Commit: 9bead2b6bae7
Files: 114
Total size: 373.7 KB
Directory structure:
gitextract_3ow7j80i/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── build.yml
│ └── docker.yml
├── .gitignore
├── .pylintrc
├── .readthedocs.yaml
├── CONTRIBUTORS
├── Dockerfile
├── LICENSE
├── MANIFEST.in
├── README.rst
├── docker-compose.yml
├── docs/
│ ├── Makefile
│ ├── _static/
│ │ └── .keep
│ ├── _templates/
│ │ ├── localtoc.html
│ │ ├── page.html
│ │ ├── sidebarintro.html
│ │ └── sidebarlogo.html
│ ├── _theme/
│ │ └── celery/
│ │ ├── static/
│ │ │ └── celery.css_t
│ │ └── theme.conf
│ ├── api.ipynb
│ ├── api.rst
│ ├── auth.rst
│ ├── conf.py
│ ├── config.rst
│ ├── features.rst
│ ├── index.rst
│ ├── install.rst
│ ├── man.rst
│ ├── prometheus-integration.rst
│ ├── reverse-proxy.rst
│ ├── tasks.py
│ └── tasks_filter.rst
├── examples/
│ ├── celery-monitoring-grafana-dashboard.json
│ ├── celeryconfig.py
│ ├── nginx.conf
│ ├── prometheus-alerts.yaml
│ ├── pycharm-configurations/
│ │ ├── Grafana.run.xml
│ │ ├── Prometheus.run.xml
│ │ └── Redis.run.xml
│ └── tasks.py
├── flower/
│ ├── __init__.py
│ ├── __main__.py
│ ├── api/
│ │ ├── __init__.py
│ │ ├── control.py
│ │ ├── tasks.py
│ │ └── workers.py
│ ├── app.py
│ ├── command.py
│ ├── events.py
│ ├── inspector.py
│ ├── options.py
│ ├── static/
│ │ ├── css/
│ │ │ └── flower.css
│ │ ├── js/
│ │ │ └── flower.js
│ │ └── swagger.json
│ ├── templates/
│ │ ├── 404.html
│ │ ├── base.html
│ │ ├── broker.html
│ │ ├── error.html
│ │ ├── navbar.html
│ │ ├── task.html
│ │ ├── tasks.html
│ │ ├── worker.html
│ │ └── workers.html
│ ├── urls.py
│ ├── utils/
│ │ ├── __init__.py
│ │ ├── broker.py
│ │ ├── search.py
│ │ ├── tasks.py
│ │ └── template.py
│ └── views/
│ ├── __init__.py
│ ├── auth.py
│ ├── broker.py
│ ├── error.py
│ ├── monitor.py
│ ├── tasks.py
│ └── workers.py
├── prometheus.yml
├── requirements/
│ ├── default.txt
│ ├── dev.txt
│ ├── docs.txt
│ └── test.txt
├── scss/
│ ├── build.sh
│ └── flower.scss
├── setup.cfg
├── setup.py
├── tests/
│ ├── __init__.py
│ ├── call-tasks.sh
│ ├── load.py
│ ├── run-unit-tests.sh
│ └── unit/
│ ├── __init__.py
│ ├── __main__.py
│ ├── api/
│ │ ├── __init__.py
│ │ ├── test_auth.py
│ │ ├── test_control.py
│ │ ├── test_tasks.py
│ │ └── test_workers.py
│ ├── test_command.py
│ ├── utils/
│ │ ├── __init__.py
│ │ ├── test_broker.py
│ │ ├── test_search.py
│ │ ├── test_template.py
│ │ └── test_utils.py
│ └── views/
│ ├── __init__.py
│ ├── test_auth.py
│ ├── test_broker.py
│ ├── test_error.py
│ ├── test_monitor.py
│ ├── test_tasks.py
│ ├── test_url_handlers.py
│ └── test_workers.py
└── tox.ini
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**System information**
Output of `python -c 'from flower.utils import bugreport; print(bugreport())'` command
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
================================================
FILE: .github/dependabot.yml
================================================
# Keep GitHub Actions up to date with GitHub's Dependabot...
# https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot
# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem
version: 2
updates:
- package-ecosystem: github-actions
directory: /
groups:
github-actions:
patterns:
- "*" # Group all Actions updates into a single larger pull request
schedule:
interval: weekly
================================================
FILE: .github/workflows/build.yml
================================================
name: Build
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ['3.9', '3.10', '3.11', '3.12']
celery-version: ['5.2.*', '5.3.*', '5.4.*', '5.5.*']
tornado-version: ['6.0']
exclude: # https://docs.celeryq.dev/en/v5.2.7/whatsnew-5.2.html#step-5-upgrade-to-celery-5-2
- python-version: '3.12'
celery-version: '5.2.*'
steps:
- uses: actions/checkout@v6
- name: Set up python
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install celery==${{ matrix.celery-version }} \
tornado==${{ matrix.tornado-version }} \
-r requirements/dev.txt
- name: Run unit tests
run: |
python -m flower --version
python -m tests.unit
- name: Lint with pylint
run: |
pylint flower --rcfile .pylintrc
================================================
FILE: .github/workflows/docker.yml
================================================
name: docker
on:
push:
branches:
- master
tags:
- '*'
pull_request:
branches:
- master
jobs:
docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: docker/setup-qemu-action@v4
- uses: docker/setup-buildx-action@v4
- id: meta
uses: docker/metadata-action@v6
with:
images: mher/flower
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- uses: docker/login-action@v4
if: github.event_name != 'pull_request'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- uses: docker/build-push-action@v7
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
================================================
FILE: .gitignore
================================================
.DS_Store
*.pyc
*$py.class
*~
.*.sw[po]
dist/
*.egg-info
*.egg
*.egg/
*.eggs
doc/__build/*
build/
.build/
pip-log.txt
.directory
erl_crash.dump
*.db
Documentation/
.tox/
.ropeproject/
.project
.pydevproject
.idea
.vagrant
env
venv
*.retry
.cache/
.mypy_cache/
Pipfile*
.vscode/
data/
.python-version
================================================
FILE: .pylintrc
================================================
[MASTER]
disable=
C0114, # missing-module-docstring
C0115, # missing-class-docstring
C0116, # missing-function-docstring
C0301, # line-too-long
W0223, # abstract-method
R0903, # too-few-public-methods
R0902, # too-many-instance-attributes
W0622, # redefined-builtin
C0415, # import-outside-toplevel
W0718, # broad-exception-caught
R1735, # use-dict-literal
R0917, # too-many-positional-arguments
[BASIC]
good-names=i,e,n,x,logger,tz,db,dt
================================================
FILE: .readthedocs.yaml
================================================
# .readthedocs.yaml
# Read the Docs configuration file
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.11"
sphinx:
configuration: docs/conf.py
python:
install:
- requirements: requirements/docs.txt
================================================
FILE: CONTRIBUTORS
================================================
======================================
Contributors (in chronological order)
======================================
Mher Movsisyan
Ask Solem
Lukasz Marcin Dobrzanski
Alexander Koshelev
Gary Linscott
Tommaso Barbugli
Miguel Gaiowski
Matt Hughes
Romain Commandé
Andres Riancho
Jet Zheung
Audrius Butkevicius
Yulian Slobodyan
Rob O'Dwyer
Horace Thomas
Kit Sunde
Adam Greig
Luciano Pacheco
Miki Tebeka
Michael J. Schultz
TJ Kells
Geoff Jukes
Peter De Vries
Lisa Chung
Sabeel Saif Hakim
Gaurav Dadhania
Charlie Marshall
Benjamin Drung
David Thorman
Hong Minhee
John Costa
Iuri de Silvio
Balthazar Rouberol
Alexandre Ferland
Florian Glesser
Tomasz Pazurkiewicz
Benjamin Toueg
Rob Hoelz
Tadej Janež
Corey Farwell
Thomas Grainger
Tom Mortimer-Jones
Konstantinos Koukopoulos
Samuel Cormier-Iijima
David Matson
Paulo SantAnna
Sanchit Arora
Ilya Lebedev
Wendy Liu
Mike Helmick
Ilya Georgievsky
Raghuram Onti Srinivasan
Michael Kahn
Gaurav Kumar
Simon Westphahl
Pedro Ferreira
Danilo Resende
Kevin Wu
Vinay Karanam
Rodrigo Pinheiro Matias
Thomas Boquet
Misha Behersky
Sebastian Kalinowski
Jingyu Zhou
Maxim Krivodaev
Alli Witheford
Alexander Zaitsev
Anton Prokhorov
Sharang Phadke
Moinuddin Quadri
John Arnold
Scott Kruger
David Schneider
S M Ahasanul Haque
Leo Singer
Pavel Savchenko
Bhargav Srinivasan
Josiah Berkebile
Deniz Dogan
Aliaksei Urbanski
Mike Dearman
Francisco J. Capdevila
Scott Allen
Johan Adami
Ray Marc Marcellones
Wen YE
Waleed Darwish
Bjorn Stiel
Fabio Todaro
Simon Gurcke
Jason Held
Lukas Matta
Tomasz Kluczkowski
Alexey Nikitenko
Sergey Klyuykov
Louis Frament
================================================
FILE: Dockerfile
================================================
FROM python:alpine
# Get latest root certificates and update openssl to fix vulnerabilities
RUN apk add --no-cache ca-certificates tzdata && \
apk upgrade --no-cache openssl && \
update-ca-certificates
# Install the required packages
RUN pip install --no-cache-dir redis flower
# PYTHONUNBUFFERED: Force stdin, stdout and stderr to be totally unbuffered. (equivalent to `python -u`)
# PYTHONHASHSEED: Enable hash randomization (equivalent to `python -R`)
# PYTHONDONTWRITEBYTECODE: Do not write byte files to disk, since we maintain it as readonly. (equivalent to `python -B`)
ENV PYTHONUNBUFFERED=1 PYTHONHASHSEED=random PYTHONDONTWRITEBYTECODE=1
# Default port
EXPOSE 5555
ENV FLOWER_DATA_DIR /data
ENV PYTHONPATH ${FLOWER_DATA_DIR}
WORKDIR $FLOWER_DATA_DIR
# Add a user with an explicit UID/GID and create necessary directories
RUN set -eux; \
addgroup -g 1000 flower; \
adduser -u 1000 -G flower flower -D; \
mkdir -p "$FLOWER_DATA_DIR"; \
chown flower:flower "$FLOWER_DATA_DIR"
USER flower
VOLUME $FLOWER_DATA_DIR
CMD ["celery", "flower"]
================================================
FILE: LICENSE
================================================
Copyright (c) 2012, Mher Movsisyan and individual contributors.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the Celery Flower nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: MANIFEST.in
================================================
include AUTHORS
include CHANGES
include LICENSE
include MANIFEST.in
include README.rst
recursive-include docs *
recursive-include flower/static *
recursive-include flower/templates *
recursive-include tests *
recursive-include requirements *.txt
================================================
FILE: README.rst
================================================
Flower
======
.. image:: https://img.shields.io/pypi/dm/flower.svg
:target: https://pypistats.org/packages/flower
:alt: PyPI - Downloads
.. image:: https://img.shields.io/docker/pulls/mher/flower.svg
:target: https://hub.docker.com/r/mher/flower
:alt: Docker Pulls
.. image:: https://github.com/mher/flower/workflows/Build/badge.svg
:target: https://github.com/mher/flower/actions
.. image:: https://img.shields.io/pypi/v/flower.svg
:target: https://pypi.python.org/pypi/flower
Flower is an open-source web application for monitoring and managing Celery clusters.
It provides real-time information about the status of Celery workers and tasks.
Features
--------
- Real-time monitoring using Celery Events
- View task progress and history
- View task details (arguments, start time, runtime, and more)
- Remote Control
- View worker status and statistics
- Shutdown and restart worker instances
- Control worker pool size and autoscale settings
- View and modify the queues a worker instance consumes from
- View currently running tasks
- View scheduled tasks (ETA/countdown)
- View reserved and revoked tasks
- Apply time and rate limits
- Revoke or terminate tasks
- Broker monitoring
- View statistics for all Celery queues
- HTTP Basic Auth, Google, Github, Gitlab and Okta OAuth
- Prometheus integration
- API
Installation
------------
Installing `flower` with `pip <http://www.pip-installer.org/>`_ is simple ::
$ pip install flower
The development version can be installed from Github ::
$ pip install https://github.com/mher/flower/zipball/master#egg=flower
Usage
-----
To run Flower, you need to provide the broker URL ::
$ celery --broker=amqp://guest:guest@localhost:5672// flower
Or use the configuration of `celery application <https://docs.celeryq.dev/en/stable/userguide/application.html>`_ ::
$ celery -A tasks.app flower
By default, flower runs on port 5555, which can be modified with the `port` option ::
$ celery -A tasks.app flower --port=5001
You can also run Flower using the docker image ::
$ docker run -v examples:/data -p 5555:5555 mher/flower celery --app=tasks.app flower
In this example, Flower is using the `tasks.app` defined in the `examples/tasks.py <https://github.com/mher/flower/blob/master/examples/tasks.py>`_ file
API
---
Flower API enables to manage the cluster via HTTP `REST API`.
For example you can restart worker's pool by: ::
$ curl -X POST http://localhost:5555/api/worker/pool/restart/myworker
Or call a task by: ::
$ curl -X POST -d '{"args":[1,2]}' http://localhost:5555/api/task/async-apply/tasks.add
Or terminate executing task by: ::
$ curl -X POST -d 'terminate=True' http://localhost:5555/api/task/revoke/8a4da87b-e12b-4547-b89a-e92e4d1f8efd
For more info checkout `API Reference`_
.. _API Reference: https://flower.readthedocs.io/en/latest/api.html
Documentation
-------------
Documentation is available at `Read the Docs`_
.. _Read the Docs: https://flower.readthedocs.io
License
-------
Flower is licensed under BSD 3-Clause License.
See the `License`_ file for the full license text.
.. _`License`: https://github.com/mher/flower/blob/master/LICENSE
================================================
FILE: docker-compose.yml
================================================
version: '3'
services:
redis:
image: redis:alpine
ports:
- 6379:6379
prometheus:
image: prom/prometheus
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- 9090:9090
grafana:
image: grafana/grafana
depends_on:
- prometheus
ports:
- 3000:3000
worker:
build: ./
entrypoint: celery
command: -A tasks worker -l info -E
user: nobody
volumes:
- ./examples:/data
environment:
CELERY_BROKER_URL: redis://redis
CELERY_RESULT_BACKEND: redis://redis
PYTHONPATH: /data
depends_on:
- redis
flower:
build: ./
command: celery -A tasks flower
volumes:
- ./examples:/data
working_dir: /data
ports:
- 5555:5555
environment:
CELERY_BROKER_URL: redis://redis
CELERY_RESULT_BACKEND: redis://redis
depends_on:
- worker
- redis
================================================
FILE: docs/Makefile
================================================
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = .build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/flower.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/flower.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/flower"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/flower"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
================================================
FILE: docs/_static/.keep
================================================
================================================
FILE: docs/_templates/localtoc.html
================================================
{%- if display_toc %}
<h3><a href="{{ pathto(master_doc) }}">{{ _('Table Of Contents') }}</a></h3>
{{ toctree(collapse=False) }}
{%- endif %}
================================================
FILE: docs/_templates/page.html
================================================
{% extends "layout.html" %}
{% block body %}
<div class="deck">
</div>
{{ body }}
{% endblock %}
================================================
FILE: docs/_templates/sidebarintro.html
================================================
<p>
<iframe src="http://ghbtns.com/github-btn.html?user=mher&repo=flower&type=watch&count=true&size=large"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
{%- if display_toc %}
<h3><a href="{{ pathto(master_doc) }}">{{ _('Table Of Contents') }}</a></h3>
{{ toctree(collapse=False) }}
{%- endif %}
================================================
FILE: docs/_templates/sidebarlogo.html
================================================
<p>
<iframe src="http://ghbtns.com/github-btn.html?user=mher&repo=flower&type=watch&count=true&size=large"
allowtransparency="true" frameborder="0" scrolling="0" width="200px" height="35px"></iframe>
</p>
================================================
FILE: docs/_theme/celery/static/celery.css_t
================================================
/*
* celery.css_t
* ~~~~~~~~~~~~
*
* :copyright: Copyright 2010 by Armin Ronacher.
* :license: BSD, see LICENSE for details.
*/
{% set page_width = 940 %}
{% set sidebar_width = 220 %}
{% set body_font_stack = 'Optima, Segoe, "Segoe UI", Candara, Calibri, Arial, sans-serif' %}
{% set headline_font_stack = 'Futura, "Trebuchet MS", Arial, sans-serif' %}
{% set code_font_stack = "'Consolas', 'Menlo', 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace" %}
@import url("basic.css");
/* -- page layout ----------------------------------------------------------- */
body {
font-family: {{ body_font_stack }};
font-size: 17px;
background-color: white;
color: #000;
margin: 30px 0 0 0;
padding: 0;
}
div.document {
width: {{ page_width }}px;
margin: 0 auto;
}
div.deck {
font-size: 18px;
}
p.developmentversion {
color: red;
}
div.related {
width: {{ page_width - 20 }}px;
padding: 5px 10px;
background: #F2FCEE;
margin: 15px auto 15px auto;
}
div.documentwrapper {
float: left;
width: 100%;
}
div.bodywrapper {
margin: 0 0 0 {{ sidebar_width }}px;
}
div.sphinxsidebar {
width: {{ sidebar_width }}px;
}
hr {
border: 1px solid #B1B4B6;
}
div.body {
background-color: #ffffff;
color: #3E4349;
padding: 0 30px 0 30px;
}
img.celerylogo {
padding: 0 0 10px 10px;
float: right;
}
div.footer {
width: {{ page_width - 15 }}px;
margin: 10px auto 30px auto;
padding-right: 15px;
font-size: 14px;
color: #888;
text-align: right;
}
div.footer a {
color: #888;
}
div.sphinxsidebar a {
color: #444;
text-decoration: none;
border-bottom: 1px dashed #DCF0D5;
}
div.sphinxsidebar a:hover {
border-bottom: 1px solid #999;
}
div.sphinxsidebar {
font-size: 14px;
line-height: 1.5;
}
div.sphinxsidebarwrapper {
padding: 7px 10px;
}
div.sphinxsidebarwrapper p.logo {
padding: 0 0 20px 0;
margin: 0;
}
div.sphinxsidebar h3,
div.sphinxsidebar h4 {
font-family: {{ headline_font_stack }};
color: #444;
font-size: 24px;
font-weight: normal;
margin: 0 0 5px 0;
padding: 0;
}
div.sphinxsidebar h4 {
font-size: 20px;
}
div.sphinxsidebar h3 a {
color: #444;
}
div.sphinxsidebar p.logo a,
div.sphinxsidebar h3 a,
div.sphinxsidebar p.logo a:hover,
div.sphinxsidebar h3 a:hover {
border: none;
}
div.sphinxsidebar p {
color: #555;
margin: 10px 0;
}
div.sphinxsidebar ul {
margin: 10px 0;
padding: 0;
color: #000;
}
div.sphinxsidebar input {
border: 1px solid #ccc;
font-family: {{ body_font_stack }};
font-size: 1em;
}
/* -- body styles ----------------------------------------------------------- */
a {
color: #348613;
text-decoration: underline;
}
a:hover {
color: #59B833;
text-decoration: underline;
}
div.body h1,
div.body h2,
div.body h3,
div.body h4,
div.body h5,
div.body h6 {
font-family: {{ headline_font_stack }};
font-weight: normal;
margin: 30px 0px 10px 0px;
padding: 0;
}
div.body h1 { margin-top: 0; padding-top: 0; font-size: 200%; }
div.body h2 { font-size: 180%; }
div.body h3 { font-size: 150%; }
div.body h4 { font-size: 130%; }
div.body h5 { font-size: 100%; }
div.body h6 { font-size: 100%; }
div.body h1 a.toc-backref,
div.body h2 a.toc-backref,
div.body h3 a.toc-backref,
div.body h4 a.toc-backref,
div.body h5 a.toc-backref,
div.body h6 a.toc-backref {
color: inherit!important;
text-decoration: none;
}
a.headerlink {
color: #ddd;
padding: 0 4px;
text-decoration: none;
}
a.headerlink:hover {
color: #444;
background: #eaeaea;
}
div.body p, div.body dd, div.body li {
line-height: 1.4em;
}
div.admonition {
background: #fafafa;
margin: 20px -30px;
padding: 10px 30px;
border-top: 1px solid #ccc;
border-bottom: 1px solid #ccc;
}
div.admonition p.admonition-title {
font-family: {{ headline_font_stack }};
font-weight: normal;
font-size: 24px;
margin: 0 0 10px 0;
padding: 0;
line-height: 1;
}
div.admonition p.last {
margin-bottom: 0;
}
div.highlight{
background-color: white;
}
dt:target, .highlight {
background: #FAF3E8;
}
div.note {
background-color: #eee;
border: 1px solid #ccc;
}
div.seealso {
background-color: #ffc;
border: 1px solid #ff6;
}
div.topic {
background-color: #eee;
}
div.warning {
background-color: #ffe4e4;
border: 1px solid #f66;
}
p.admonition-title {
display: inline;
}
p.admonition-title:after {
content: ":";
}
pre, tt {
font-family: {{ code_font_stack }};
font-size: 0.9em;
}
img.screenshot {
}
tt.descname, tt.descclassname {
font-size: 0.95em;
}
tt.descname {
padding-right: 0.08em;
}
img.screenshot {
-moz-box-shadow: 2px 2px 4px #eee;
-webkit-box-shadow: 2px 2px 4px #eee;
box-shadow: 2px 2px 4px #eee;
}
table.docutils {
border: 1px solid #888;
-moz-box-shadow: 2px 2px 4px #eee;
-webkit-box-shadow: 2px 2px 4px #eee;
box-shadow: 2px 2px 4px #eee;
}
table.docutils td, table.docutils th {
border: 1px solid #888;
padding: 0.25em 0.7em;
}
table.field-list, table.footnote {
border: none;
-moz-box-shadow: none;
-webkit-box-shadow: none;
box-shadow: none;
}
table.footnote {
margin: 15px 0;
width: 100%;
border: 1px solid #eee;
background: #fdfdfd;
font-size: 0.9em;
}
table.footnote + table.footnote {
margin-top: -15px;
border-top: none;
}
table.field-list th {
padding: 0 0.8em 0 0;
}
table.field-list td {
padding: 0;
}
table.footnote td.label {
width: 0px;
padding: 0.3em 0 0.3em 0.5em;
}
table.footnote td {
padding: 0.3em 0.5em;
}
dl {
margin: 0;
padding: 0;
}
dl dd {
margin-left: 30px;
}
blockquote {
margin: 0 0 0 30px;
padding: 0;
}
ul {
margin: 10px 0 10px 30px;
padding: 0;
}
pre {
background: #F0FFEB;
padding: 7px 10px;
margin: 15px 0;
border: 1px solid #C7ECB8;
border-radius: 2px;
-moz-border-radius: 2px;
-webkit-border-radius: 2px;
line-height: 1.3em;
}
tt {
background: #F0FFEB;
color: #222;
/* padding: 1px 2px; */
}
tt.xref, a tt {
background: #F0FFEB;
border-bottom: 1px solid white;
}
a.reference {
text-decoration: none;
border-bottom: 1px dashed #DCF0D5;
}
a.reference:hover {
border-bottom: 1px solid #6D4100;
}
a.footnote-reference {
text-decoration: none;
font-size: 0.7em;
vertical-align: top;
border-bottom: 1px dashed #DCF0D5;
}
a.footnote-reference:hover {
border-bottom: 1px solid #6D4100;
}
a:hover tt {
background: #EEE;
}
================================================
FILE: docs/_theme/celery/theme.conf
================================================
[theme]
inherit = basic
stylesheet = celery.css
[options]
================================================
FILE: docs/api.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# flower REST API\n",
"\n",
"This document shows how to use the flower [REST API](https://github.com/mher/flower#api). \n",
"\n",
"We will use [requests](http://www.python-requests.org/en/latest/) for accessing the API. (See [here](http://www.python-requests.org/en/latest/user/install/) on how to install it.) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Code\n",
"We'll use the following code throughout the documentation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## tasks.py"
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from celery import Celery\n",
"from time import sleep\n",
"\n",
"celery = Celery()\n",
"celery.config_from_object({\n",
" 'BROKER_URL': 'amqp://localhost',\n",
" 'CELERY_RESULT_BACKEND': 'amqp://',\n",
" 'CELERYD_POOL_RESTARTS': True, # Required for /worker/pool/restart API\n",
"})\n",
"\n",
"\n",
"@celery.task\n",
"def add(x, y):\n",
" return x + y\n",
"\n",
"\n",
"@celery.task\n",
"def sub(x, y):\n",
" sleep(30) # Simulate work\n",
" return x - y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Running\n",
"You'll need a celery worker instance and a flower instance running. In one terminal window run\n",
"\n",
" celery worker --loglevel INFO -A proj -E --autoscale 10,3\n",
"\n",
"and in another terminal run\n",
"\n",
" celery flower -A proj"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tasks API\n",
"The tasks API is *async*, meaning calls will return immediately and you'll need to poll on task status."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Done once for the whole docs\n",
"import requests, json\n",
"api_root = 'http://localhost:5555/api'\n",
"task_api = '{}/task'.format(api_root)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## async-apply"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/async-apply/tasks.add\n"
]
},
{
"data": {
"text/plain": [
"{u'state': u'PENDING', u'task-id': u'f4a53407-30f3-42af-869f-b7f8f4fbd684'}"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"args = {'args': [1, 2]}\n",
"url = '{}/async-apply/tasks.add'.format(task_api)\n",
"print(url)\n",
"resp = requests.post(url, data=json.dumps(args))\n",
"reply = resp.json()\n",
"reply"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see that we created a new task and it's pending. Note that the API is *async*, meaning it won't wait until the task finish."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## apply"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For create task and wait results you can use 'apply' API."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/apply/tasks.add\n"
]
},
{
"data": {
"text/plain": [
"{u'result': 3,\n",
" u'state': u'SUCCESS',\n",
" u'task-id': u'ced6fd57-419e-4b8e-8d99-0770be717cb4'}"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"args = {'args': [1, 2]}\n",
"url = '{}/apply/tasks.add'.format(task_api)\n",
"print(url)\n",
"resp = requests.post(url, data=json.dumps(args))\n",
"reply = resp.json()\n",
"reply"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## result\n",
"Gets the task result. This is *async* and will return immediately even if the task didn't finish (with state 'PENDING')"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/result/ced6fd57-419e-4b8e-8d99-0770be717cb4\n"
]
},
{
"data": {
"text/plain": [
"{u'result': 3,\n",
" u'state': u'SUCCESS',\n",
" u'task-id': u'ced6fd57-419e-4b8e-8d99-0770be717cb4'}"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/result/{}'.format(task_api, reply['task-id'])\n",
"print(url)\n",
"resp = requests.get(url)\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## revoke\n",
"Revoke a running task."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/revoke/bcb4ac2e-cb2d-4a4b-a402-8eb3a3b0c8e8\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"Revoked 'bcb4ac2e-cb2d-4a4b-a402-8eb3a3b0c8e8'\"}"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Run a task\n",
"args = {'args': [1, 2]}\n",
"resp = requests.post('{}/async-apply/tasks.sub'.format(task_api), data=json.dumps(args))\n",
"reply = resp.json()\n",
"\n",
"# Now revoke it\n",
"url = '{}/revoke/{}'.format(task_api, reply['task-id'])\n",
"print(url)\n",
"resp = requests.post(url, data='terminate=True')\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## rate-limit\n",
"Update [rate limit](https://docs.celeryq.dev/en/latest/userguide/tasks.html#Task.rate_limit) for a task."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/rate-limit/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u'new rate limit set successfully'}"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"worker = 'miki-manjaro' # You'll need to get the worker name from the worker API (seel below)\n",
"url = '{}/rate-limit/{}'.format(task_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'taskname': 'tasks.add', 'ratelimit': '10'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## timeout\n",
"Set timeout (both [hard](https://docs.celeryq.dev/en/latest/userguide/tasks.html#Task.time_limit) and [soft](https://docs.celeryq.dev/en/latest/userguide/tasks.html#Task.soft_time_limit)) for a task."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/task/timeout/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u'time limits set successfully'}"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/timeout/{}'.format(task_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'taskname': 'tasks.add', 'hard': '3.14', 'soft': '3'}) # You can omit soft or hard\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Worker API"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Once for the documentation\n",
"worker_api = '{}/worker'.format(api_root)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## workers\n",
"List workers."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/workers\n"
]
},
{
"data": {
"text/plain": [
"{u'miki-manjaro': {u'completed_tasks': 0,\n",
" u'concurrency': 1,\n",
" u'queues': [u'celery'],\n",
" u'running_tasks': 0,\n",
" u'status': True}}"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/workers'.format(api_root) # Only one not under /worker\n",
"print(url)\n",
"resp = requests.get(url)\n",
"workers = resp.json()\n",
"workers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## pool/shutdown\n",
"Shutdown a worker."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/shutdown/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u'Shutting down!'}"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"worker = workers.keys()[0]\n",
"url = '{}/shutdown/{}'.format(worker_api, worker)\n",
"print(url)\n",
"resp = requests.post(url)\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## pool/restart\n",
"Restart a worker pool, you need to have [CELERYD_POOL_RESTARTS](https://docs.celeryq.dev/en/latest/configuration.html#std:setting-CELERYD_POOL_RESTARTS) enabled in your configuration)."
]
},
{
"cell_type": "code",
"execution_count": 43,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/pool/restart/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"Restarting 'miki-manjaro' worker's pool\"}"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pool_api = '{}/pool'.format(worker_api)\n",
"url = '{}/restart/{}'.format(pool_api, worker)\n",
"print(url)\n",
"resp = requests.post(url)\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## pool/grow\n",
"Grows worker pool."
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/pool/grow/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"Growing 'miki-manjaro' worker's pool\"}"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/grow/{}'.format(pool_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'n': '10'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## pool/shrink\n",
"Shrink worker pool."
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/pool/shrink/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"Shrinking 'miki-manjaro' worker's pool\"}"
]
},
"execution_count": 54,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/shrink/{}'.format(pool_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'n': '3'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## pool/autoscale\n",
"[Autoscale](https://docs.celeryq.dev/en/latest/userguide/workers.html#autoscaling) a pool."
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/pool/autoscale/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"Autoscaling 'miki-manjaro' worker\"}"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/autoscale/{}'.format(pool_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'min': '3', 'max': '10'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## queue/add-consumer\n",
"[Add a consumer](https://docs.celeryq.dev/en/latest/userguide/workers.html#std:control-add_consumer) to a queue."
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/queue/add-consumer/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u\"add consumer u'jokes'\"}"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"queue_api = '{}/queue'.format(worker_api)\n",
"url = '{}/add-consumer/{}'.format(queue_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'queue': 'jokes'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## queue/cancel-consumer\n",
"[Cancel a consumer](https://docs.celeryq.dev/en/latest/userguide/workers.html#queues-cancelling-consumers) queue."
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/worker/queue/cancel-consumer/miki-manjaro\n"
]
},
{
"data": {
"text/plain": [
"{u'message': u'no longer consuming from jokes'}"
]
},
"execution_count": 63,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/cancel-consumer/{}'.format(queue_api, worker)\n",
"print(url)\n",
"resp = requests.post(url, params={'queue': 'jokes'})\n",
"resp.json()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Queue API\n",
"\n",
"We assume that we've two queues; the default one 'celery' and 'all'"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"http://localhost:5555/api/queues/length\n"
]
},
{
"data": {
"text/plain": [
"{u'active_queues': [{u'messages': 2, u'name': u'all'},\n",
" {u'messages': 1, u'name': u'celery'}]}"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"url = '{}/queues/length'.format(api_root)\n",
"print(url)\n",
"resp = requests.get(url)\n",
"resp.json()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: docs/api.rst
================================================
API Reference
=============
For security reasons, the Flower API is disabled by default when authentication is not enabled.
To enable the API for unauthenticated environments, you can set the FLOWER_UNAUTHENTICATED_API
environment variable to true.
.. autotornado:: flower.app:Flower()
================================================
FILE: docs/auth.rst
================================================
.. _authentication:
Authentication
==============
Flower supports a variety of authentication methods, including Basic Authentication, Google, GitHub,
GitLab, and Okta OAuth. You can also customize and use your own authentication method.
The following endpoints are exempt from authentication:
- /healthcheck
- /metrics
.. _basic-authentication:
HTTP Basic Authentication
-------------------------
Flower supports Basic Authentication as a built-in authentication method, allowing you to secure access to the
Flower using simple username and password credentials. This authentication method is commonly used for
straightforward authentication requirements.
To enable basic authentication, use :ref:`basic_auth` option. This option allows you to specify a list of
username and password pairs for authentication.
For example, running Flower with the following :ref:`basic_auth` option will protect the Flower UI and
only allow access to users providing the username user and the password pswd::
$ celery flower --basic-auth=user:pswd
See also :ref:`reverse-proxy`
.. _google-oauth:
Google OAuth
------------
Flower provides authentication support using Google OAuth, enabling you to authenticate users through their Google accounts.
This integration simplifies the authentication process and offers a seamless experience for users who are already logged into Google.
Follow the steps below to configure and use Google OAuth authentication:
1. Go to the `Google Developer Console`_
2. Select a project, or create a new one.
3. In the sidebar on the left, select Credentials.
4. Click CREATE CREDENTIALS and click OAuth client ID.
5. Under Application type, select Web application.
6. Name OAuth 2.0 client and click Create.
7. Copy the "Client secret" and "Client ID"
8. Add redirect URI to the list of Authorized redirect URIs
Here's an example configuration file with the Google OAuth options:
.. code-block:: python
auth_provider="flower.views.auth.GoogleAuth2LoginHandler"
auth="allowed-emails.*@gmail.com"
oauth2_key="<your_client_id>"
oauth2_secret="<your_client_secret>"
oauth2_redirect_uri="http://localhost:5555/login"
Replace `<your_client_id>` and `<your_client_secret>` with the actual Client ID and secret obtained from
the Google Developer Console.
.. _Google Developer Console: https://console.developers.google.com
.. _github-oauth:
GitHub OAuth
------------
Flower also supports GitHub OAuth. Before getting started, Flower should be registered in
`Github Settings`_.
Github OAuth is activated by setting :ref:`auth_provider` to `flower.views.auth.GithubLoginHandler`.
Here's an example configuration file with the Github OAuth options:
.. code-block:: python
auth_provider="flower.views.auth.GithubLoginHandler"
auth="allowed-emails.*@gmail.com"
oauth2_key="<your_client_id>"
oauth2_secret="<your_client_secret>"
oauth2_redirect_uri="http://localhost:5555/login"
Replace `<your_client_id>` and `<your_client_secret>` with the actual Client ID and secret obtained from
the Github Settings.
See `GitHub OAuth API`_ docs for more info.
.. _Github Settings: https://github.com/settings/applications/new
.. _GitHub OAuth API: https://developer.github.com/v3/oauth/
.. _okta-oauth:
Okta OAuth
----------
Flower also supports Okta OAuth. Before getting started, you need to register Flower in `Okta`_.
Okta OAuth is activated by setting :ref:`auth_provider` option to `flower.views.auth.OktaLoginHandler`.
Okta OAuth requires `oauth2_key`, `oauth2_secret` and `oauth2_redirect_uri` options which should be obtained from Okta.
Okta OAuth also uses `FLOWER_OAUTH2_OKTA_BASE_URL` environment variable.
See Okta `Okta OAuth API`_ docs for more info.
.. _Okta: https://developer.okta.com/docs/guides/add-an-external-idp/openidconnect/main/
.. _Okta OAuth API: https://developer.okta.com/docs/reference/api/oidc/
.. _gitlab-oauth:
GitLab OAuth
------------
Flower also supports GitLab OAuth for authentication. To enable GitLab OAuth, follow the steps below:
1. Register Flower as an application at GitLab. You can refer to the `GitLab OAuth documentation`_ for detailed instructions on how to do this.
2. Once registered, you will obtain the credentials for Flower configuration.
3. In your Flower configuration, set the following options to activate GitLab OAuth:
- :ref:`auth_provider` to `flower.views.auth.GitLabLoginHandler`.
- :ref:`oauth2_key` to the "Application ID" obtained from GitLab.
- :ref:`oauth2_secret` to the "Secret" obtained from GitLab.
- :ref:`oauth2_redirect_uri`: Set this to the redirect URI configured in GitLab.
4. (Optional) To restrict access to specific GitLab groups, you can utilize the `FLOWER_GITLAB_AUTH_ALLOWED_GROUPS` environment variable. Set it to a comma-separated list of allowed groups. You can include subgroups by using the `/` character. For example: `group1,group2/subgroup`.
5. (Optional) The default minimum required group access level can be adjusted using the `FLOWER_GITLAB_MIN_ACCESS_LEVEL` environment variable.
6. (Optional) The custom GitHub Domain can be adjusted using the `FLOWER_GITLAB_OAUTH_DOMAIN` environment variable.
For further details on GitLab OAuth and its implementation, refer to the `Group and project members API`_ documentation.
It provides comprehensive information and guidelines on working with GitLab's OAuth functionality.
See also `GitLab OAuth2 API`_ documentation for more info.
.. _GitLab OAuth documentation: https://docs.gitlab.com/ee/integration/oauth_provider.htm
.. _GitLab OAuth2 API: https://docs.gitlab.com/ee/api/oauth2.html
.. _Group and project members API: https://docs.gitlab.com/ee/api/members.html
================================================
FILE: docs/conf.py
================================================
# -*- coding: utf-8 -*-
#
# flower documentation build configuration file, created by
# sphinx-quickstart on Fri Apr 11 17:26:01 2014.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys
import os
sys.path.insert(0, os.path.abspath('..'))
import flower # noqa: E402
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.intersphinx',
'sphinxcontrib.httpdomain',
'sphinxcontrib.autohttp.tornado',
'sphinxcontrib.redoc',
]
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
# source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = 'Flower'
copyright = '2023, Mher Movsisyan'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '.'.join(map(str, flower.VERSION[0:2]))
# The full version, including alpha/beta/rc tags.
release = flower.__version__.rstrip('-dev')
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
# language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['.build']
# The reST default role (used for this markup: `text`) to use for all
# documents.
# default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
# add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
# keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'celery'
html_theme_path = ['_theme']
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
# html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
# html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
# html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
# html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
# html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
html_sidebars = {
'index': ['sidebarintro.html', 'sourcelink.html', 'searchbox.html'],
'**': ['sidebarlogo.html', 'localtoc.html', 'relations.html',
'sourcelink.html', 'searchbox.html']
}
# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}
# If false, no module index is generated.
html_domain_indices = False
# If false, no index is generated.
html_use_index = False
# If true, the index is split into individual pages for each letter.
# html_split_index = False
# If true, links to the reST sources are added to the pages.
html_show_sourcelink = False
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
html_show_sphinx = False
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
# html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'flowerdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
# ' pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
# 'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(
'index',
'flower.tex',
'flower Documentation',
'Mher Movsisyan',
'manual'
),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False
# If true, show page references after internal links.
# latex_show_pagerefs = False
# If true, show URL addresses after external links.
# latex_show_urls = False
# Documents to append as an appendix to all manuals.
# latex_appendices = []
# If false, no module index is generated.
# latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('man', 'flower', 'flower Documentation',
['Mher Movsisyan'], 1)
]
# If true, show URL addresses after external links.
# man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(
'index',
'flower',
'flower Documentation',
'Mher Movsisyan',
'flower',
'One line description of project.',
'Miscellaneous'
),
]
# Documents to append as an appendix to all manuals.
# texinfo_appendices = []
# If false, no module index is generated.
# texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
# texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
# texinfo_no_detailmenu = False
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'http://docs.python.org/': None}
================================================
FILE: docs/config.rst
================================================
:tocdepth: 2
Configuration
=============
Flower is highly customizable. You can pass configuration options through the command line,
configuration file, or environment variables. For a full list of options, see the `Option Reference`_ section.
Command line
------------
Flower operates as a sub-command of Celery, allowing you to pass both Celery and Flower
configuration options from the command line. The template for this is as follows ::
celery [celery options] flower [flower options]
Celery options should be specified after the celery command, while Flower options
should be specified after the flower sub-command.
For example ::
$ celery --broker=redis:// flower --unix-socket=/tmp/flower.sock
See `Celery Configuration reference`_ for a comprehensive listing of all available settings
and their default values.
.. _`Celery Configuration reference`: https://docs.celeryq.dev/en/latest/userguide/configuration.html
Configuration file
------------------
Flower tries to load configuration from the :file:`flowerconfig.py` file by default.
You can override the name of the configuration file with the `conf`_ option.
The configuration file is a simple Python file that contains key-value pairs:
.. code-block:: python
# Set RabbitMQ management api
broker_api = 'http://guest:guest@localhost:15672/api/'
# Enable debug logging
logging = 'DEBUG'
Environment variables
---------------------
Flower configuration options can also be passed through environment variables.
All Flower options must be prefixed with `FLOWER_`.
For example, to set the basic_auth option to foo:bar, you would set the
`FLOWER_BASIC_AUTH` environment variable to `foo:bar` ::
export FLOWER_BASIC_AUTH=foo:bar
celery flower
.. _options_referance:
Option Reference
-----------------
.. contents::
:local:
:depth: 1
.. _address:
address
~~~~~~~
Default: '' (empty string)
Sets the address on which the Flower HTTP server should listen.
The address may be either an IP address or a hostname. If a hostname is provided,
the server will listen on all IP addresses associated with that name.
To listen on all available interfaces, set the address to an empty string.
Example:
Listen on all available interfaces::
$ celery flower --address='0.0.0.0'
Listen only on the loopback interface::
$ celery flower --address='localhost'
Listen on all IP addresses associated with 'example.com'::
$ celery flower --address='example.com'
.. _auth:
auth
~~~~
Default: '' (empty string)
Enables authentication. `auth` is a regular expression of emails to grant access.
The `auth` option allows you to enable authentication in Flower. By default, the `auth` option is set to an empty string, indicating that authentication is disabled.
To enable authentication and restrict access to specific email addresses, set the `auth` option to a regular expression pattern that matches the desired email addresses. The `auth` option supports a basic regex syntax, including:
- Single email: Use a single email address, such as `user@example.com`.
- Wildcard: Use a wildcard pattern with `.*` to match multiple email addresses with the same domain, such as `.*@example.com`.
- List of emails: Use a list of emails separated by pipes (`|`), such as `one@example.com|two@example.com`.
Please note that for security reasons, the `auth` option only supports a basic regex syntax and does not provide advanced regex features.
For more information and detailed usage examples, refer to the :ref:`Authentication` section of the Flower documentation.
.. _auto_refresh:
auto_refresh
~~~~~~~~~~~~
Default: True
Enables automatic refresh for the Workers view.
By default, the Workers view automatically refreshes at regular intervals to provide up-to-date
information about the workers. Set this option to `False` to disable automatic refreshing.
.. _basic_auth:
basic_auth
~~~~~~~~~~
Default: None
Enables HTTP Basic authentication. It accepts a comma-separated list of `username:password` pairs.
Each pair represents a valid username and password combination for authentication.
Example:
Enable HTTP Basic authentication with multiple users::
$ celery flower --basic-auth="user1:password1,user2:password2"
See :ref:`basic-authentication` for more information.
.. _broker_api:
broker_api
~~~~~~~~~~
Default: None
The URL of the broker API used by Flower to retrieve information about queues.
Flower uses the RabbitMQ Management Plugin to gather information about queues.
The `broker_api` option should be set to the URL of the RabbitMQ HTTP API, including user credentials if required.
Example::
$ celery flower broker-api="http://username:password@rabbitmq-server-name:15672/api/"
.. Note:: By default, the RabbitMQ Management Plugin is not enabled. To enable it, run the following command::
$ rabbitmq-plugins enable rabbitmq_management
.. Note:: The port number for RabbitMQ versions prior to 3.0 is 55672.
For more information refer to the `RabbitMQ Management Plugin`_ documentation.
.. _`RabbitMQ Management Plugin`: https://www.rabbitmq.com/management.html
.. _ca_certs:
ca_certs
~~~~~~~~
Default: None
Sets the path to the `ca_certs` file containing a set of concatenated "certification authority" certificates.
The `ca_certs` file is used to validate certificates received from the other end of the connection.
It contains a collection of trusted root certificates. Set the `ca_certs` option to the path of the `ca_certs` file.
If not specified, certificate validation will not be performed.
For more information about certificate validation in Python, refer to the `Python SSL`_ documentation.
.. _`Python SSL`: https://docs.python.org/3/library/ssl.html
.. _certfile:
certfile
~~~~~~~~
Default: None
Sets the path to the SSL certificate file.
The `certfile` option specifies the path to the SSL certificate file used for SSL/TLS encryption.
The certificate file contains the public key certificate for the Flower server.
If not specified, SSL/TLS encryption will not be used.
.. _conf:
conf
~~~~
Default: flowerconfig.py
Sets the configuration file to be used by Flower.
Example::
$ celery flower --conf="./examples/celeryconfig.py"
.. _db:
db
~~
Default: flower
Sets the database file to use if persistent mode is enabled.
If the `persistent`_ mode is enabled, the `db` option specifies the database file
to be used by Flower for storing task results, events, or other persistent data.
Example::
$ celery flower --persistent=True --db="flower_db"
.. _debug:
debug
~~~~~
Default: False
Enables the debug mode
.. Note:: When debug mode is enabled, Flower may print sensitive information
.. _enable_events:
enable_events
~~~~~~~~~~~~~
Default: True
When enabled, Flower periodically sends Celery `enable_events` commands to all workers.
Enabling Celery events allows Flower to receive real-time updates about task events
from the Celery workers.
You can also enable events directly when running Celery workers by using the `-E` flag.
For more information, refer to the `Celery documentation <https://docs.celeryq.dev/en/stable/reference/cli.html#cmdoption-celery-worker-E>`_:
.. _format_task:
format_task
~~~~~~~~~~~
Default: None
Modifies the default task formatting.
The `format_task` function allows to modify the default formatting of tasks.
By defining the `format_task` function in the `flowerconfig.py` configuration file,
you can customize the task object before it is displayed. The `format_task` function accepts a task object
as a parameter and should return the modified version of the task.
This function is particularly useful for filtering out sensitive information or limiting display lengths of task arguments, kwargs, or results.
The example below shows how to filter arguments and limit display lengths:
.. code-block:: python
from flower.utils.template import humanize
def format_task(task):
task.args = humanize(task.args, length=10)
task.kwargs.pop('credit_card_number')
task.result = humanize(task.result, length=20)
return task
.. _inspect_timeout:
inspect_timeout
~~~~~~~~~~~~~~~
Default: 1000
Sets the timeout for the worker inspect commands in milliseconds.
Worker inspection involves retrieving information about the workers, such as their current status, tasks, and resource usage.
.. _keyfile:
keyfile
~~~~~~~
Default: None
Sets the path to the SSL key file.
The key file contains the private key corresponding to the SSL certificate.
If not specified, or set to `None`, SSL/TLS encryption will not be used.
.. _max_workers:
max_workers
~~~~~~~~~~~
Default: 5000
Sets the maximum number of workers to keep in memory
.. _max_tasks:
max_tasks
~~~~~~~~~
Default: 100000
Sets the maximum number of tasks to keep in memory
.. _natural_time:
natural_time
~~~~~~~~~~~~
Default: False
Enables showing time relative to the page refresh time in a more human-readable format.
When enabled, timestamps will be shown as relative time such as "2 minutes ago" or "1 hour ago" instead of the exact timestamp.
.. _persistent:
persistent
~~~~~~~~~~
Default: False
Enables persistent mode in Flower.
When persistent mode is enabled, Flower saves its current state and reloads it upon restart.
This ensures that Flower retains its state and configuration across restarts.
Flower stores its state in a database file specified by the `db`_ option.
.. _port:
port
~~~~
Default: 5555
Sets the port number for running the Flower HTTP server.
.. _state_save_interval:
state_save_interval
~~~~~~~~~~~~~~~~~~~
Default: 0
Sets the interval for saving the Flower state.
Flower state includes information about workers, tasks. The state is saved periodically to ensure data persistence and recovery upon restart.
By default, periodic saving is disabled. Flower will not automatically save its state at regular intervals.
If you want to enable periodic state saving, set the `state_save_interval` option to a positive integer value representing the interval in milliseconds.
.. _xheaders:
xheaders
~~~~~~~~
Default: False
Enables support for `X-Real-Ip` and `X-Scheme` headers.
The `xheaders` option allows Flower to enable support for `X-Real-Ip` and `X-Scheme` headers.
These headers are commonly used in proxy or load balancer configurations to preserve the original client IP address and scheme.
.. _tasks_columns:
tasks_columns
~~~~~~~~~~~~~
Default: name,uuid,state,args,kwargs,result,received,started,runtime,worker
Specifies the list of comma-delimited columns to display on the `/tasks` page.
The `tasks_columns` option allows you to customize the columns displayed on the `/tasks` page in Flower.
By default, the specified columns are: name, uuid, state, args, kwargs, result, received, started, runtime, and worker.
Available columns are:
- `name`
- `uuid`
- `state`
- `args`
- `kwargs`
- `result`
- `received`
- `started`
- `runtime`
- `worker`
- `retries`
- `revoked`
- `exception`
- `expires`
- `eta`
Example::
$ celery flower --tasks-columns='name,uuid,state,args,kwargs,result,received,started,runtime,worker,retries,revoked,exception,expires,eta'
In the above example, all available columns are displayed.
.. _url_prefix:
url_prefix
~~~~~~~~~~
Default: '' (empty string)
Enables deploying Flower on a non-root URL.
The `url_prefix` option allows you to deploy Flower on a non-root URL.
By default, Flower is deployed on the root URL. However, if you need to run Flower on a specific path,
such as `http://example.com/flower`, you can specify the desired URL prefix using the `url_prefix` option.
.. _unix_socket:
unix_socket
~~~~~~~~~~~
Default: '' (empty string)
Runs Flower using a UNIX socket file.
The `unix_socket` option allows you to run Flower using a UNIX socket file instead of a network port.
By default, the `unix_socket` option is set to an empty string, indicating that Flower should not use a UNIX socket.
To run Flower using a UNIX socket file, set the `unix_socket` option to the desired path of the UNIX socket file.
Flower will then bind to the specified socket file instead of a network port.
Example::
$ celery flower --unix-socket='/var/run/flower.sock'
.. _cookie_secret:
cookie_secret
~~~~~~~~~~~~~
Default: token_urlsafe(64) (random string)
Sets a secret key for signing cookies.
The `cookie_secret` option allows you to set a secret key used for signing cookies in Flower.
By default, the `cookie_secret` option is set to 'token_urlsafe(64)', which generates a random string of length 64 characters as the secret key.
This provides a good level of security for signing cookies. If you want to specify a custom secret key, you can set the `cookie_secret` option to the desired string.
.. _auth_provider:
auth_provider
~~~~~~~~~~~~~
Default: None
Sets the authentication provider for Flower.
The `auth_provider` option allows you to set the authentication provider for Flower.
By default, the `auth_provider` option is set to `None`, indicating that no authentication provider is configured.
To enable authentication and specify an authentication provider, set the `auth_provider` option to one of the following values:
- Google `flower.views.auth.GoogleAuth2LoginHandler`
- GitHub `flower.views.auth.GithubLoginHandler`
- GitLab `flower.views.auth.GitLabLoginHandler`
- Okta `flower.views.auth.OktaLoginHandler`
See also :ref:`Authentication` for usage examples
.. _purge_offline_workers:
purge_offline_workers
~~~~~~~~~~~~~~~~~~~~~
Default: None
Time (in seconds) after which offline workers are automatically removed from the Workers view.
By default, offline workers will remain on the dashboard indefinitely.
.. _task_runtime_metric_buckets:
task_runtime_metric_buckets
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Default: 'Histogram.DEFAULT_BUCKETS' (default prometheus buckets)
Sets the task runtime latency buckets.
You can provide the `buckets` value as a comma-separated list of values.
Example::
$ celery flower --task-runtime-metric-buckets=1,5,10,inf
The buckets represent the upper bounds of the latency intervals.
You can specify them as integer or float values. The `inf` value represents positive infinity, indicating
that the last bucket captures all values greater than or equal to the previous bucket.
.. _oauth2_key:
oauth2_key
~~~~~~~~~~
Default: None
Sets the OAuth 2.0 key (client ID) issued by the OAuth 2.0 provider
`oauth2_key` option should be used with :ref:`auth`, :ref:`auth_provider`, :ref:`oauth2_redirect_uri` and :ref:`oauth2_secret` options.
.. _oauth2_secret:
oauth2_secret
~~~~~~~~~~~~~
Default: None
Sets the OAuth 2.0 secret issued by the OAuth 2.0 provider
`oauth2_secret` option should be used with :ref:`auth`, :ref:`auth_provider`, :ref:`oauth2_redirect_uri` and :ref:`oauth2_key` options.
.. _oauth2_redirect_uri:
oauth2_redirect_uri
~~~~~~~~~~~~~~~~~~~
Default: None
Sets the URI to which an OAuth 2.0 server redirects the user after successful authentication and authorization.
`oauth2_redirect_uri` option should be used with :ref:`auth`, :ref:`auth_provider`, :ref:`oauth2_key` and :ref:`oauth2_secret` options.
================================================
FILE: docs/features.rst
================================================
Features
--------
- Real-time monitoring using Celery Events
- View task progress and history
- View task details (arguments, start time, runtime, and more)
- Remote Control
- View worker status and statistics
- Shutdown and restart worker instances
- Control worker pool size and autoscale settings
- View and modify the queues a worker instance consumes from
- View currently running tasks
- View scheduled tasks (ETA/countdown)
- View reserved and revoked tasks
- Apply time and rate limits
- Revoke or terminate tasks
- Broker monitoring
- View statistics for all Celery queues
- HTTP Basic Auth, Google, Github, Gitlab and Okta OAuth
- Prometheus integration
- API
================================================
FILE: docs/index.rst
================================================
======
Flower
======
Flower is an open-source web application for monitoring and managing `Celery`_ clusters.
It provides real-time information about the status of Celery workers and tasks.
.. _Celery: https://docs.celeryq.dev/en/stable/#
Contents
========
.. toctree::
features
install
config
tasks_filter
auth
reverse-proxy
prometheus-integration
api
Flower is licensed under BSD 3-Clause License.
See the `License`_ file for the full license text.
Flower development and support is provided through its GitHub `repository`_.
.. _`License`: https://github.com/mher/flower/blob/master/LICENSE
.. _`repository`: https://github.com/mher/flower
================================================
FILE: docs/install.rst
================================================
Getting started
===============
Installation
------------
Installing `flower` with `pip <http://www.pip-installer.org/>`_ is simple ::
$ pip install flower
The development version can be installed from Github ::
$ pip install https://github.com/mher/flower/zipball/master#egg=flower
Usage
-----
To run Flower, you need to provide the broker URL ::
$ celery --broker=amqp://guest:guest@localhost:5672// flower
Or use the configuration of `celery application <https://docs.celeryq.dev/en/stable/userguide/application.html>`_ ::
$ celery -A tasks.app flower
By default, flower runs on port 5555, which can be modified with the :ref:`port` option ::
$ celery -A tasks.app flower --port=5001
You can also run Flower using the docker image ::
$ docker run -v examples:/data -p 5555:5555 mher/flower celery --app=tasks.app flower
In this example, Flower is using the `tasks.app` defined in the `examples/tasks.py <https://github.com/mher/flower/blob/master/examples/tasks.py>`_ file
================================================
FILE: docs/man.rst
================================================
========
flower
========
SYNOPSIS
========
``flower`` [*OPTIONS*]
DESCRIPTION
===========
Flower is a web based tool for monitoring and administrating Celery clusters.
It has these features:
- Real-time monitoring using Celery Events
- Task progress and history
- Ability to show task details (arguments, start time, runtime, and more)
- Graphs and statistics
- Remote Control
- View worker status and statistics
- Shutdown and restart worker instances
- Control worker pool size and autoscale settings
- View and modify the queues a worker instance consumes from
- View currently running tasks
- View scheduled tasks (ETA/countdown)
- View reserved and revoked tasks
- Apply time and rate limits
- Configuration viewer
- Revoke or terminate tasks
- Broker monitoring
- View statistics for all Celery queues
- Queue length graphs
- HTTP API
- Basic Auth and Google OpenID authentication
- Prometheus integration
OPTIONS
=======
--address run on the given address
--auth regexp of emails to grant access
--auth_provider sets authentication provider class
--auto_refresh refresh workers automatically (default *True*)
--basic_auth colon separated user-password to enable
basic auth
--broker_api inspect broker e.g.
http://guest:guest@localhost:15672/api/
--ca_certs path to SSL certificate authority (CA) file
--certfile path to SSL certificate file
--conf flower configuration file path (default *flowerconfig.py*)
--cookie_secret secure cookie secret
--db flower database file (default *flower*)
--debug run in debug mode (default *False*)
--enable_events periodically enable Celery events (default *True*)
--format_task use custom task formatter
--help show this help information
--inspect inspect workers (default *True*)
--inspect_timeout inspect timeout (in milliseconds) (default
*1000*)
--keyfile path to SSL key file
--max_workers maximum number of workers to keep in memory
(default *5000*)
--max_tasks maximum number of tasks to keep in memory
(default *10000*)
--natural_time show time in relative format (default *False*)
--persistent enable persistent mode (default *False*)
--port run on the given port (default *5555*)
--purge_offline_workers time (in seconds) after which offline workers are purged
from workers
--state_save_interval state save interval (in milliseconds) (default *0*)
--tasks_columns slugs of columns on /tasks/ page, delimited by comma
(default *name,uuid,state,args,kwargs,result,received,started,runtime,worker*)
--unix_socket path to unix socket to bind flower server to
--url_prefix base url prefix
--xheaders enable support for the 'X-Real-Ip' and
'X-Scheme' headers. (default *False*)
--task_runtime_metric_buckets task runtime prometheus latency metric buckets (default prometheus latency buckets)
TORNADO OPTIONS
===============
--log_file_max_size max size of log files before rollover
(default *100000000*)
--log_file_num_backups number of log files to keep (default *10*)
--log_file_prefix=PATH Path prefix for log files. Note that if you
are running multiple tornado processes,
log_file_prefix must be different for each
of them (e.g. include the port number)
--log_to_stderr Send log output to stderr (colorized if
possible). By default use stderr if
``--log_file_prefix`` is not set and no other
logging is configured.
--logging=debug|info|warning|error|none
Set the Python log level. If *none*, tornado
won't touch the logging configuration.
(default *info*)
USAGE
=====
Launch the Flower server at specified port other than default 5555 (open the UI at http://localhost:5566): ::
$ celery flower --port=5566
Specify Celery application path with address and port for Flower: ::
$ celery -A proj flower --address=127.0.0.6 --port=5566
Broker URL and other configuration options can be passed through the standard Celery options (notice that they are after
Celery command and before Flower sub-command): ::
$ celery -A proj --broker=amqp://guest:guest@localhost:5672// flower
================================================
FILE: docs/prometheus-integration.rst
================================================
Prometheus Integration
======================
Flower exports several celery worker and task metrics in Prometheus' format.
The ``/metrics`` endpoint is available from the get go after you have installed Flower.
By default on your local machine Flower's metrics are available at: ``localhost:5555/metrics``.
Read further for more information about configuration and available metrics please.
Complete guide on integration of Celery, Flower, Prometheus and Grafana is here: `Grafana Integration Guide`_.
Configure Prometheus to scrape Flower metrics
---------------------------------------------
To integrate with Prometheus you have to add Flower as the target in Prometheus's configuration.
In this example we are assuming your Flower and Prometheus are installed on your local machine
with their defaults and available at ``localhost:<port number>``.
To add Flower's metrics to Prometheus go to its config file ``prometheus.yml`` which initially
will look like this:
.. code-block:: yaml
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
and alter the ``scrape_configs`` definition to be:
.. code-block:: yaml
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
- job_name: flower
static_configs:
- targets: ['localhost:5555']
You can also just point Prometheus at the example ``prometheus.yml`` file in the root of the `Flower repository <https://github.com/mher/flower/prometheus.yml>`
when you start it from the command line (note that you would have to set ``flower`` to point at ``localhost`` in your ``etc/hosts`` config for the DNS to resolve correctly)::
./prometheus --config.file=prometheus.yml
Available Metrics
-----------------
Below you will find a table of available Prometheus metrics exposed by Flower.
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| Name | Description | Labels | Instrument Type |
+===================================================+======================================================================+====================+=================+
| flower_events_total | Number of times a celery task event was registered by Flower. | task, type, worker | counter |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| flower_task_prefetch_time_seconds | The time the task spent waiting at the celery worker to be executed. | task, worker | gauge |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| flower_worker_prefetched_tasks | Number of tasks of given type prefetched at a worker. | task, worker | gauge |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| flower_task_runtime_seconds | The time it took to run the task. | task, worker | histogram |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| flower_worker_online | Shows celery worker's online status. | worker | gauge |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
| flower_worker_number_of_currently_executing_tasks | Number of tasks currently executing at this worker. | worker | gauge |
+---------------------------------------------------+----------------------------------------------------------------------+--------------------+-----------------+
Using Metric Labels
-------------------
You can filter received data in prometheus using ``promql`` syntax to present information only for selected labels.
We have the following labels available:
* **task** - task name, i.e. ``tasks.add``, ``tasks.multiply``.
* **type** - task event type, i.e. ``task-started``, ``task-succeeded``. Note that worker related events **will not be counted**.
For more info on task event types see: `task events in celery <https://docs.celeryq.dev/en/stable/userguide/monitoring.html#task-events>`_.
* **worker** - celery worker name, i.e ``celery@<your computer name>``.
Example Prometheus Alerts
-------------------------
See example `Prometheus alerts <https://github.com/mher/flower/tree/master/examples/prometheus-alerts.yaml>`_.
Add the rules to your ``alertmanager.yml`` config as in the `alert manager's documentation <https://prometheus.io/docs/alerting/latest/configuration/>`_.
Example Grafana Dashboard
-------------------------
See example `Grafana dashboard <https://github.com/mher/flower/tree/master/examples/celery-monitoring-grafana-dashboard.json>`_.
You can import it easily in Grafana.
Hover over the + button in the side bar menu -> Import -> Upload JSON file.
The dashboard should give you a nice starting point for monitoring of your celery cluster.
Grafana Integration Guide
=========================
In this guide you will learn how to setup each part of the stack to make it talk to the next one and achieve Celery
monitoring solution with help of Flower.
Same as above we assume localhost usage and for ease of deployment I will use Pycharm configurations to start docker
containers with necessary images. If you do not have docker installed on your system: `download and install it please <https://www.docker.com/get-started>`_.
Start Celery Broker
-------------------
Easiest is to use `Redis Pycharm run configuration <https://github.com/mher/flower/tree/master/examples/pycharm-configurations/Redis.run.xml>`_.
Or run::
docker run --name redis -d -p 6379:6379 redis
Set Up Your Celery Application
-------------------------------
We are assuming that your Celery application has tasks in `tasks.py` file. The `-E` argument makes Celery send events
which are required to produce Prometheus metrics.
Create `celeryconfig.py` in root of your Celery app. We are setting Celery to use Redis DB as the broker/backend in this
example. Skip this if you configure your broker/backend already in another way (make sure to adjust further steps to that).
.. code-block:: python
broker_url = 'redis://localhost:6379/0'
celery_result_backend = 'redis://localhost:6379/0'
Or download it from `here <https://github.com/mher/flower/tree/master/examples/celeryconfig.py>`_.
Start your Celery app::
celery -A tasks worker -l INFO -E
When the app starts you should see this line::
-- ******* ---- .> task events: ON
Start Flower Monitoring
-----------------------
In your Celery application folder run this command (Flower needs to be installed)::
celery -A tasks --broker=redis://localhost:6379/0 flower
Configure and Start Prometheus
------------------------------
Create `prometheus.yml` file. Note its absolute path - we will use it to start the Prometheus docker image.
For ease of use put it in the root of your Celery project (so that you can use Pycharm configuration below without any changes).
.. code-block:: yaml
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
- job_name: flower
static_configs:
- targets: ['localhost:5555']
Run Prometheus inside docker:
You can use `Prometheus Pycharm run configuration <https://github.com/mher/flower/tree/master/examples/pycharm-configurations/Prometheus.run.xml>`_ (may need to adjust the `prometheus.yml` path if it is not in root of your Celery project).
Or just start it via command line::
docker run --name Prometheus -v <ABSOLUTE PATH TO YOUR prometheus.yml FILE>:/etc/prometheus/prometheus.yml -p 9090:9090 --network host prom/prometheus
Now go to `localhost:9090` and check that Prometheus is running.
If everything so far was set up and started correctly, you should be able to see metrics provided by Flower in your
Prometheus's GUI. Go to `Graph` tab and start typing `flower` - the autocomplete should show you all available metrics.
.. image:: screenshots/flower-metrics-in-prometheus.png
:width: 100%
Start Grafana
-------------
You can use `Grafana Pycharm run configuration <https://github.com/mher/flower/tree/master/examples/pycharm-configurations/Grafana.run.xml>`_.
Or run it from the terminal::
docker run --name Grafana -d -v grafana-storage:/var/lib/grafana -p 3000:3000 --network host grafana/grafana
try to access its web GUI now by going to `localhost:3000`, use `admin/admin` for credentials. It will ask you to set up
a new password - you may click skip for now.
Add Prometheus As a Data Source In Grafana
------------------------------------------
Click `Configuration` (settings icon) in the left side-bar. Then the blue `Add data source` button.
.. image:: screenshots/grafana-add-data-source.png
:width: 100%
Search for Prometheus data source and click it (it should be at the top).
.. image:: screenshots/grafana-add-prometheus-data-source.png
:width: 100%
Once in Prometheus data source configuration, use all defaults and enter the HTTP/URL parameter as below (which is the placeholder by the way)::
http://localhost:9090
.. image:: screenshots/grafana-configure-prometheus-data-source.png
:width: 100%
Scroll down and click `Save & Test`, if all is good a green banner will pop up saying `Data source is working`
.. image:: screenshots/grafana-test-prometheus-data-source.png
:width: 100%
Import The Celery Monitoring Dashboard In Grafana
-------------------------------------------------
Download `Grafana dashboard <https://github.com/mher/flower/tree/master/examples/celery-monitoring-grafana-dashboard.json>`_.
Hover over the `+` icon in the left side-bar and click `Import` button.
.. image:: screenshots/grafana-import-dashboard.png
:width: 30%
Click `Upload JSON file` button and select the `celery-monitoring-grafana-dashboard.json` you have just downloaded.
.. image:: screenshots/grafana-import-celery-monitoring-dashboard.png
:width: 100%
Click on the `Prometheus` field and select a Prometheus data source.
.. image:: screenshots/grafana-configure-imported-dashboard.png
:width: 100%
Click `Import` to finish the process.
You should see a dashboard as on the image below. Congratulations!
.. image:: screenshots/grafana-dashboard.png
:width: 100%
================================================
FILE: docs/reverse-proxy.rst
================================================
.. _reverse-proxy:
Running behind reverse proxy
============================
To run `Flower` behind a reverse proxy, remember to set the correct `Host`
header to the request to make sure Flower can generate correct URLs.
The following block represents the minimal `nginx` configuration:
.. code-block:: nginx
server {
listen 80;
server_name flower.example.com;
location / {
proxy_pass http://localhost:5555;
}
}
If you run Flower behind custom location, make sure :ref:`url_prefix` option
value equals to the location path.
You have to use either environment variable `FLOWER_URL_PREFIX=flower`
or command parameter `--url_prefix=flower` when you run it
via `celery`. With that being set you need the following `nginx` configuration:
.. code-block:: nginx
server {
listen 80;
server_name example.com;
location /flower/ {
proxy_pass http://localhost:5555/flower/;
}
}
without `url_prefix` Flower frontend won't be able to generate
correct static links, and without `/flower/` at the end of `proxy_pass`
parameter, the browser will lead you to 404.
Note that you should not expose this site to the public internet without
any sort of authentication! If you have a `htpasswd` file with user
credentials you can make `nginx` use this file by adding the following
lines to the location block:
.. code-block:: nginx
auth_basic "Restricted";
auth_basic_user_file htpasswd;
================================================
FILE: docs/tasks.py
================================================
from celery import Celery
from time import sleep
celery = Celery()
celery.config_from_object({
'BROKER_URL': 'amqp://10.0.2.2',
'CELERY_RESULT_BACKEND': 'amqp://',
'CELERYD_POOL_RESTARTS': True,
})
@celery.task
def add(x, y):
return x + y
@celery.task
def sub(x, y):
sleep(30) # Simulate work
return x - y
================================================
FILE: docs/tasks_filter.rst
================================================
Tasks filtering
===============
By now, tasks can be filtered by worker, type, state, received and started datetime.
Also, filtering by args/kwargs/result/state value available.
Flower uses github-style syntax for args/kwargs/result filtering.
- `foo` find all tasks containing foo in args, kwargs or result
- `args:foo` find all tasks containing foo in arguments
- `kwargs:foo=bar` find all tasks containing foo=bar keyword
- `result:foo` find all tasks containing foo in result
- `state:FAILURE` find all failed tasks
If the search term contains spaces it should be enclosed in " (e.g. `args:"hello world"`).
For examples, see `tests/utils/test_search.py`.
================================================
FILE: examples/celery-monitoring-grafana-dashboard.json
================================================
{
"__inputs": [
{
"name": "DS_PROMETHEUS",
"label": "Prometheus",
"description": "",
"type": "datasource",
"pluginId": "prometheus",
"pluginName": "Prometheus"
}
],
"__requires": [
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "7.5.2"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "datasource",
"id": "prometheus",
"name": "Prometheus",
"version": "1.0.0"
}
],
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"description": "Basic celery monitoring example",
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"id": null,
"links": [],
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel shows status of celery workers. 1 = online, 0 = offline.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 0
},
"hiddenSeries": false,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "flower_worker_online",
"interval": "",
"legendFormat": "{{ worker }}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Celery Worker Status",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:150",
"format": "short",
"label": "",
"logBase": 1,
"max": "1",
"min": "0",
"show": true
},
{
"$$hashKey": "object:151",
"decimals": null,
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel shows number of tasks currently executing at worker.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 0
},
"hiddenSeries": false,
"id": 9,
"legend": {
"alignAsTable": true,
"avg": true,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "flower_worker_number_of_currently_executing_tasks",
"interval": "",
"legendFormat": "{{worker}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Number of Tasks Currently Executing at Worker",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:79",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:80",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel shows average task runtime at worker by worker and task name.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 24,
"x": 0,
"y": 8
},
"hiddenSeries": false,
"id": 11,
"legend": {
"alignAsTable": true,
"avg": true,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "rate(flower_task_runtime_seconds_sum[5m]) / rate(flower_task_runtime_seconds_count[5m])",
"interval": "",
"legendFormat": "{{task}}, {{worker}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Average Task Runtime at Worker",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:337",
"format": "s",
"label": "",
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:338",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel shows task prefetch time at worker by worker and task name.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 24,
"x": 0,
"y": 17
},
"hiddenSeries": false,
"id": 12,
"legend": {
"alignAsTable": true,
"avg": true,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "flower_task_prefetch_time_seconds",
"interval": "",
"legendFormat": "{{task}}, {{worker}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Task Prefetch Time at Worker",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:337",
"format": "s",
"label": "",
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:338",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel shows number of tasks prefetched at worker by task and worker name.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 24,
"x": 0,
"y": 26
},
"hiddenSeries": false,
"id": 10,
"legend": {
"alignAsTable": true,
"avg": true,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "flower_worker_prefetched_tasks",
"interval": "",
"legendFormat": "{{task}}, {{worker}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Number of Tasks Prefetched At Worker",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:337",
"format": "short",
"label": "",
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:338",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel presents average task success ratio over time by task name.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 12,
"x": 0,
"y": 35
},
"hiddenSeries": false,
"id": 2,
"legend": {
"alignAsTable": true,
"avg": false,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "(sum(avg_over_time(flower_events_total{type=\"task-succeeded\"}[15m])) by (task) / sum(avg_over_time(flower_events_total{type=~\"task-failed|task-succeeded\"}[15m])) by (task)) * 100",
"interval": "",
"legendFormat": "{{ task }}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Task Success Ratio",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:63",
"format": "percent",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:64",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"description": "This panel presents average task failure ratio over time by task name.",
"fieldConfig": {
"defaults": {},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 12,
"x": 12,
"y": 35
},
"hiddenSeries": false,
"id": 7,
"legend": {
"alignAsTable": true,
"avg": false,
"current": true,
"max": true,
"min": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.5.2",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"exemplar": true,
"expr": "(sum(avg_over_time(flower_events_total{type=\"task-failed\"}[15m])) by (task) / sum(avg_over_time(flower_events_total{type=~\"task-failed|task-succeeded\"}[15m])) by (task)) * 100",
"interval": "",
"legendFormat": "{{ task }}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Task Failure Ratio",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:63",
"format": "percent",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:64",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
],
"refresh": "10s",
"schemaVersion": 27,
"style": "dark",
"tags": [
"celery",
"monitoring",
"flower"
],
"templating": {
"list": []
},
"time": {
"from": "now-15m",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Celery Monitoring",
"uid": "3OBI1flGz",
"version": 9
}
================================================
FILE: examples/celeryconfig.py
================================================
broker_url = 'redis://localhost:6379/0'
celery_result_backend = 'redis://localhost:6379/0'
task_send_sent_event = False
================================================
FILE: examples/nginx.conf
================================================
server {
listen 80;
# with url_prefix=`flower`
location /flower/ {
proxy_pass http://localhost:5555;
}
}
================================================
FILE: examples/prometheus-alerts.yaml
================================================
- alert: CeleryWorkerOffline
expr: flower_worker_online == 0
for: 2m
labels:
severity: critical
context: celery-worker
annotations:
summary: Celery worker offline
description: Celery worker {{ $labels.worker }} has been offline for more than 2 minutes.
- alert: TaskFailureRatioTooHigh
expr: (sum(avg_over_time(flower_events_total{type="task-failed"}[15m])) by (task) / sum(avg_over_time(flower_events_total{type=~"task-failed|task-succeeded"}[15m])) by (task)) * 100 > 1
for: 5m
labels:
severity: critical
context: celery-task
annotations:
summary: Task Failure Ratio Too High.
description: Average task failure ratio for task {{ $labels.task }} is {{ $value }}.
- alert: TaskPrefetchTimeTooHigh
expr: sum(avg_over_time(flower_task_prefetch_time_seconds[15m])) by (task, worker) > 1
for: 5m
labels:
severity: critical
context: celery-task
annotations:
summary: Average Task Prefetch Time Too High.
description: Average task prefetch time at worker for task {{ $labels.task }} and worker {{ $labels.worker }} is {{ $value }}.
================================================
FILE: examples/pycharm-configurations/Grafana.run.xml
================================================
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Grafana" type="docker-deploy" factoryName="docker-image" server-name="Docker">
<deployment type="docker-image">
<settings>
<option name="imageTag" value="grafana/grafana" />
<option name="command" value="" />
<option name="containerName" value="Grafana" />
<option name="entrypoint" value="" />
<option name="commandLineOptions" value="-d -p 3000:3000 --network=host -v grafana-storage:/var/lib/grafana" />
</settings>
</deployment>
<method v="2" />
</configuration>
</component>
================================================
FILE: examples/pycharm-configurations/Prometheus.run.xml
================================================
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Prometheus" type="docker-deploy" factoryName="docker-image" server-name="Docker">
<deployment type="docker-image">
<settings>
<option name="imageTag" value="prom/prometheus" />
<option name="command" value="" />
<option name="containerName" value="Prometheus" />
<option name="entrypoint" value="" />
<option name="commandLineOptions" value="-p 9090:9090 -v $PROJECT_DIR$/prometheus.yml:/etc/prometheus/prometheus.yml --network=host" />
</settings>
</deployment>
<method v="2" />
</configuration>
</component>
================================================
FILE: examples/pycharm-configurations/Redis.run.xml
================================================
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Redis" type="docker-deploy" factoryName="docker-image" server-name="Docker">
<deployment type="docker-image">
<settings>
<option name="imageTag" value="redis" />
<option name="command" value="" />
<option name="containerName" value="redis" />
<option name="entrypoint" value="" />
<option name="commandLineOptions" value="-d -p 6379:6379" />
</settings>
</deployment>
<method v="2" />
</configuration>
</component>
================================================
FILE: examples/tasks.py
================================================
import os
import time
from datetime import datetime
from celery import Celery
app = Celery("tasks",
broker=os.environ.get('CELERY_BROKER_URL', 'redis://'),
backend=os.environ.get('CELERY_RESULT_BACKEND', 'redis'))
app.conf.accept_content = ['pickle', 'json', 'msgpack', 'yaml']
app.conf.worker_send_task_events = True
@app.task
def add(x, y):
return x + y
@app.task
def sleep(seconds):
time.sleep(seconds)
@app.task
def echo(msg, timestamp=False):
return "%s: %s" % (datetime.now(), msg) if timestamp else msg
@app.task
def error(msg):
raise Exception(msg)
if __name__ == "__main__":
app.start()
================================================
FILE: flower/__init__.py
================================================
VERSION = (2, 0, 0)
__version__ = '.'.join(map(str, VERSION)) + '-dev'
================================================
FILE: flower/__main__.py
================================================
import sys
from celery.bin.celery import main as _main, celery
from flower.command import flower
def main():
celery.add_command(flower)
sys.exit(_main())
if __name__ == "__main__":
main()
================================================
FILE: flower/api/__init__.py
================================================
import os
import tornado.web
from ..utils import strtobool
from ..views import BaseHandler
class BaseApiHandler(BaseHandler):
def prepare(self):
enable_api = strtobool(os.environ.get(
'FLOWER_UNAUTHENTICATED_API') or "false")
if not (self.application.options.basic_auth or self.application.options.auth) and not enable_api:
raise tornado.web.HTTPError(
401, "FLOWER_UNAUTHENTICATED_API environment variable is required to enable API without authentication")
def write_error(self, status_code, **kwargs):
exc_info = kwargs.get('exc_info')
log_message = exc_info[1].log_message
if log_message:
self.write(log_message)
self.set_status(status_code)
self.finish()
================================================
FILE: flower/api/control.py
================================================
import logging
from tornado import web
from . import BaseApiHandler
logger = logging.getLogger(__name__)
class ControlHandler(BaseApiHandler):
def is_worker(self, workername):
return workername and workername in self.application.workers
def error_reason(self, workername, response):
"extracts error message from response"
for res in response:
try:
return res[workername].get('error', 'Unknown reason')
except KeyError:
pass
logger.error("Failed to extract error reason from '%s'", response)
return 'Unknown reason'
class WorkerShutDown(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Shut down a worker
**Example request**:
.. sourcecode:: http
POST /api/worker/shutdown/celery@worker2 HTTP/1.1
Content-Length: 0
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 29
Content-Type: application/json; charset=UTF-8
{
"message": "Shutting down!"
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
logger.info("Shutting down '%s' worker", workername)
self.capp.control.broadcast('shutdown', destination=[workername])
self.write(dict(message="Shutting down!"))
class WorkerPoolRestart(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Restart worker's pool
**Example request**:
.. sourcecode:: http
POST /api/worker/pool/restart/celery@worker2 HTTP/1.1
Content-Length: 0
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 56
Content-Type: application/json; charset=UTF-8
{
"message": "Restarting 'celery@worker2' worker's pool"
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: pool restart is not enabled (see CELERYD_POOL_RESTARTS)
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
logger.info("Restarting '%s' worker's pool", workername)
response = self.capp.control.broadcast(
'pool_restart', arguments={'reload': False},
destination=[workername], reply=True)
if response and 'ok' in response[0][workername]:
self.write(dict(message=f"Restarting '{workername}' worker's pool"))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to restart the '{workername}' pool: {reason}")
class WorkerPoolGrow(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Grow worker's pool
**Example request**:
.. sourcecode:: http
POST /api/worker/pool/grow/celery@worker2?n=3 HTTP/1.1
Content-Length: 0
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 58
Content-Type: application/json; charset=UTF-8
{
"message": "Growing 'celery@worker2' worker's pool by 3"
}
:query n: number of pool processes to grow, default is 1
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: failed to grow
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
n = self.get_argument('n', default=1, type=int)
logger.info("Growing '%s' worker's pool by '%s'", workername, n)
response = self.capp.control.pool_grow(
n=n, reply=True, destination=[workername])
if response and 'ok' in response[0][workername]:
self.write(dict(message=f"Growing '{workername}' worker's pool by {n}"))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to grow '{workername}' worker's pool: {reason}")
class WorkerPoolShrink(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Shrink worker's pool
**Example request**:
.. sourcecode:: http
POST /api/worker/pool/shrink/celery@worker2 HTTP/1.1
Content-Length: 0
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 60
Content-Type: application/json; charset=UTF-8
{
"message": "Shrinking 'celery@worker2' worker's pool by 1"
}
:query n: number of pool processes to shrink, default is 1
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: failed to shrink
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
n = self.get_argument('n', default=1, type=int)
logger.info("Shrinking '%s' worker's pool by '%s'", workername, n)
response = self.capp.control.pool_shrink(
n=n, reply=True, destination=[workername])
if response and 'ok' in response[0][workername]:
self.write(dict(message=f"Shrinking '{workername}' worker's pool by {n}"))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to shrink '{workername}' worker's pool: {reason}")
class WorkerPoolAutoscale(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Autoscale worker pool
**Example request**:
.. sourcecode:: http
POST /api/worker/pool/autoscale/celery@worker2?min=3&max=10 HTTP/1.1
Content-Length: 0
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 66
Content-Type: application/json; charset=UTF-8
{
"message": "Autoscaling 'celery@worker2' worker (min=3, max=10)"
}
:query min: minimum number of pool processes
:query max: maximum number of pool processes
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: autoscaling is not enabled (see CELERYD_AUTOSCALER)
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
min = self.get_argument('min', type=int)
max = self.get_argument('max', type=int)
logger.info("Autoscaling '%s' worker by '%s'",
workername, (min, max))
response = self.capp.control.broadcast(
'autoscale', arguments={'min': min, 'max': max},
destination=[workername], reply=True)
if response and 'ok' in response[0][workername]:
self.write(dict(message=f"Autoscaling '{workername}' worker "
"(min={min}, max={max})"))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to autoscale '{workername}' worker: {reason}")
class WorkerQueueAddConsumer(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Start consuming from a queue
**Example request**:
.. sourcecode:: http
POST /api/worker/queue/add-consumer/celery@worker2?queue=sample-queue
Content-Length: 0
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 40
Content-Type: application/json; charset=UTF-8
{
"message": "add consumer sample-queue"
}
:query queue: the name of a new queue
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: failed to add consumer
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
queue = self.get_argument('queue')
logger.info("Adding consumer '%s' to worker '%s'",
queue, workername)
response = self.capp.control.broadcast(
'add_consumer', arguments={'queue': queue},
destination=[workername], reply=True)
if response and 'ok' in response[0][workername]:
self.write(dict(message=response[0][workername]['ok']))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to add '{queue}' consumer to '{workername}' worker: {reason}")
class WorkerQueueCancelConsumer(ControlHandler):
@web.authenticated
def post(self, workername):
"""
Stop consuming from a queue
**Example request**:
.. sourcecode:: http
POST /api/worker/queue/cancel-consumer/celery@worker2?queue=sample-queue
Content-Length: 0
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 52
Content-Type: application/json; charset=UTF-8
{
"message": "no longer consuming from sample-queue"
}
:query queue: the name of queue
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 403: failed to cancel consumer
:statuscode 404: unknown worker
"""
if not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
queue = self.get_argument('queue')
logger.info("Canceling consumer '%s' from worker '%s'",
queue, workername)
response = self.capp.control.broadcast(
'cancel_consumer', arguments={'queue': queue},
destination=[workername], reply=True)
if response and 'ok' in response[0][workername]:
self.write(dict(message=response[0][workername]['ok']))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(workername, response)
self.write(f"Failed to cancel '{queue}' consumer from '{workername}' worker: {reason}")
class TaskRevoke(ControlHandler):
@web.authenticated
def post(self, taskid):
"""
Revoke a task
**Example request**:
.. sourcecode:: http
POST /api/task/revoke/1480b55c-b8b2-462c-985e-24af3e9158f9?terminate=true
Content-Length: 0
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 61
Content-Type: application/json; charset=UTF-8
{
"message": "Revoked '1480b55c-b8b2-462c-985e-24af3e9158f9'"
}
:query terminate: terminate the task if it is running
:query signal: name of signal to send to process if terminate (default: 'SIGTERM')
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
"""
logger.info("Revoking task '%s'", taskid)
terminate = self.get_argument('terminate', default=False, type=bool)
signal = self.get_argument('signal', default='SIGTERM', type=str)
self.capp.control.revoke(taskid, terminate=terminate, signal=signal)
self.write(dict(message=f"Revoked '{taskid}'"))
class TaskTimout(ControlHandler):
@web.authenticated
def post(self, taskname):
"""
Change soft and hard time limits for a task
**Example request**:
.. sourcecode:: http
POST /api/task/timeout/tasks.sleep HTTP/1.1
Content-Length: 44
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
soft=30&hard=100&workername=celery%40worker1
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 46
Content-Type: application/json; charset=UTF-8
{
"message": "time limits set successfully"
}
:query workername: worker name
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task/worker
"""
workername = self.get_argument('workername')
hard = self.get_argument('hard', default=None, type=float)
soft = self.get_argument('soft', default=None, type=float)
if taskname not in self.capp.tasks:
raise web.HTTPError(404, f"Unknown task '{taskname}'")
if workername is not None and not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
logger.info("Setting timeouts for '%s' task (%s, %s)",
taskname, soft, hard)
destination = [workername] if workername is not None else None
response = self.capp.control.time_limit(
taskname, reply=True, hard=hard, soft=soft,
destination=destination)
if response and 'ok' in response[0][workername]:
self.write(dict(message=response[0][workername]['ok']))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(taskname, response)
self.write(f"Failed to set timeouts: '{reason}'")
class TaskRateLimit(ControlHandler):
@web.authenticated
def post(self, taskname):
"""
Change rate limit for a task
**Example request**:
.. sourcecode:: http
POST /api/task/rate-limit/tasks.sleep HTTP/1.1
Content-Length: 41
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:5555
ratelimit=200&workername=celery%40worker1
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 61
Content-Type: application/json; charset=UTF-8
{
"message": "new rate limit set successfully"
}
:query workername: worker name
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task/worker
"""
workername = self.get_argument('workername')
ratelimit = self.get_argument('ratelimit')
if taskname not in self.capp.tasks:
raise web.HTTPError(404, f"Unknown task '{taskname}'")
if workername is not None and not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
logger.info("Setting '%s' rate limit for '%s' task",
ratelimit, taskname)
destination = [workername] if workername is not None else None
response = self.capp.control.rate_limit(
taskname, ratelimit, reply=True, destination=destination)
if response and 'ok' in response[0][workername]:
self.write(dict(message=response[0][workername]['ok']))
else:
logger.error(response)
self.set_status(403)
reason = self.error_reason(taskname, response)
self.write(f"Failed to set rate limit: '{reason}'")
================================================
FILE: flower/api/tasks.py
================================================
import json
import logging
from collections import OrderedDict
from datetime import datetime
from celery import states
from celery.backends.base import DisabledBackend
from celery.contrib.abortable import AbortableAsyncResult
from celery.result import AsyncResult
from tornado import web
from tornado.escape import json_decode
from tornado.ioloop import IOLoop
from tornado.web import HTTPError
from ..utils import tasks
from ..utils.broker import Broker
from . import BaseApiHandler
logger = logging.getLogger(__name__)
class BaseTaskHandler(BaseApiHandler):
DATE_FORMAT = '%Y-%m-%d %H:%M:%S.%f'
def get_task_args(self):
try:
body = self.request.body
options = json_decode(body) if body else {}
except ValueError as e:
raise HTTPError(400, str(e)) from e
if not isinstance(options, dict):
raise HTTPError(400, 'invalid options')
args = options.pop('args', [])
kwargs = options.pop('kwargs', {})
if not isinstance(args, (list, tuple)):
raise HTTPError(400, 'args must be an array')
return args, kwargs, options
@staticmethod
def backend_configured(result):
return not isinstance(result.backend, DisabledBackend)
def write_error(self, status_code, **kwargs):
self.set_status(status_code)
def update_response_result(self, response, result):
if result.state == states.FAILURE:
response.update({'result': self.safe_result(result.result),
'traceback': result.traceback})
else:
response.update({'result': self.safe_result(result.result)})
def normalize_options(self, options):
if 'eta' in options:
options['eta'] = datetime.strptime(options['eta'],
self.DATE_FORMAT)
if 'countdown' in options:
options['countdown'] = float(options['countdown'])
if 'expires' in options:
expires = options['expires']
try:
expires = float(expires)
except ValueError:
expires = datetime.strptime(expires, self.DATE_FORMAT)
options['expires'] = expires
def safe_result(self, result):
"returns json encodable result"
try:
json.dumps(result)
except TypeError:
return repr(result)
return result
class TaskApply(BaseTaskHandler):
@web.authenticated
async def post(self, taskname):
"""
Execute a task by name and wait results
**Example request**:
.. sourcecode:: http
POST /api/task/apply/tasks.add HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate, compress
Content-Length: 16
Content-Type: application/json; charset=utf-8
Host: localhost:5555
{
"args": [1, 2]
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 71
Content-Type: application/json; charset=UTF-8
{
"state": "SUCCESS",
"task-id": "c60be250-fe52-48df-befb-ac66174076e6",
"result": 3
}
:query args: a list of arguments
:query kwargs: a dictionary of arguments
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task
"""
args, kwargs, options = self.get_task_args()
logger.debug("Invoking a task '%s' with '%s' and '%s'",
taskname, args, kwargs)
try:
task = self.capp.tasks[taskname]
except KeyError as exc:
raise HTTPError(404, f"Unknown task '{taskname}'") from exc
try:
self.normalize_options(options)
except ValueError as exc:
raise HTTPError(400, 'Invalid option') from exc
result = task.apply_async(args=args, kwargs=kwargs, **options)
response = {'task-id': result.task_id}
response = await IOLoop.current().run_in_executor(
None, self.wait_results, result, response)
self.write(response)
def wait_results(self, result, response):
# Wait until task finished and do not raise anything
result.get(propagate=False)
# Write results and finish async function
self.update_response_result(response, result)
if self.backend_configured(result):
response.update(state=result.state)
return response
class TaskAsyncApply(BaseTaskHandler):
@web.authenticated
def post(self, taskname):
"""
Execute a task
**Example request**:
.. sourcecode:: http
POST /api/task/async-apply/tasks.add HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate, compress
Content-Length: 16
Content-Type: application/json; charset=utf-8
Host: localhost:5555
{
"args": [1, 2]
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 71
Content-Type: application/json; charset=UTF-8
Date: Sun, 13 Apr 2014 15:55:00 GMT
{
"state": "PENDING",
"task-id": "abc300c7-2922-4069-97b6-a635cc2ac47c"
}
:query args: a list of arguments
:query kwargs: a dictionary of arguments
:query options: a dictionary of `apply_async` keyword arguments
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task
"""
args, kwargs, options = self.get_task_args()
logger.debug("Invoking a task '%s' with '%s' and '%s'",
taskname, args, kwargs)
try:
task = self.capp.tasks[taskname]
except KeyError as exc:
raise HTTPError(404, f"Unknown task '{taskname}'") from exc
try:
self.normalize_options(options)
except ValueError as exc:
raise HTTPError(400, 'Invalid option') from exc
result = task.apply_async(args=args, kwargs=kwargs, **options)
response = {'task-id': result.task_id}
if self.backend_configured(result):
response.update(state=result.state)
self.write(response)
class TaskSend(BaseTaskHandler):
@web.authenticated
def post(self, taskname):
"""
Execute a task by name (doesn't require task sources)
**Example request**:
.. sourcecode:: http
POST /api/task/send-task/tasks.add HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate, compress
Content-Length: 16
Content-Type: application/json; charset=utf-8
Host: localhost:5555
{
"args": [1, 2]
}
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 71
Content-Type: application/json; charset=UTF-8
{
"state": "SUCCESS",
"task-id": "c60be250-fe52-48df-befb-ac66174076e6"
}
:query args: a list of arguments
:query kwargs: a dictionary of arguments
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task
"""
args, kwargs, options = self.get_task_args()
logger.debug("Invoking task '%s' with '%s' and '%s'",
taskname, args, kwargs)
result = self.capp.send_task(
taskname, args=args, kwargs=kwargs, **options)
response = {'task-id': result.task_id}
if self.backend_configured(result):
response.update(state=result.state)
self.write(response)
class TaskResult(BaseTaskHandler):
@web.authenticated
def get(self, taskid):
"""
Get a task result
**Example request**:
.. sourcecode:: http
GET /api/task/result/c60be250-fe52-48df-befb-ac66174076e6 HTTP/1.1
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 84
Content-Type: application/json; charset=UTF-8
{
"result": 3,
"state": "SUCCESS",
"task-id": "c60be250-fe52-48df-befb-ac66174076e6"
}
:query timeout: how long to wait, in seconds, before the operation times out
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 503: result backend is not configured
"""
timeout = self.get_argument('timeout', None)
timeout = float(timeout) if timeout is not None else None
result = AsyncResult(taskid)
if not self.backend_configured(result):
raise HTTPError(503)
response = {'task-id': taskid, 'state': result.state}
if timeout:
result.get(timeout=timeout, propagate=False)
self.update_response_result(response, result)
elif result.ready():
self.update_response_result(response, result)
self.write(response)
class TaskAbort(BaseTaskHandler):
@web.authenticated
def post(self, taskid):
"""
Abort a running task
**Example request**:
.. sourcecode:: http
POST /api/task/abort/c60be250-fe52-48df-befb-ac66174076e6 HTTP/1.1
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 61
Content-Type: application/json; charset=UTF-8
{
"message": "Aborted '1480b55c-b8b2-462c-985e-24af3e9158f9'"
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 503: result backend is not configured
"""
logger.info("Aborting task '%s'", taskid)
result = AbortableAsyncResult(taskid)
if not self.backend_configured(result):
raise HTTPError(503)
result.abort()
self.write(dict(message=f"Aborted '{taskid}'"))
class GetQueueLengths(BaseTaskHandler):
@web.authenticated
async def get(self):
"""
Return length of all active queues
**Example request**:
.. sourcecode:: http
GET /api/queues/length
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 94
Content-Type: application/json; charset=UTF-8
{
"active_queues": [
{"name": "celery", "messages": 0},
{"name": "video-queue", "messages": 5}
]
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 503: result backend is not configured
"""
app = self.application
http_api = None
if app.transport == 'amqp' and app.options.broker_api:
http_api = app.options.broker_api
broker = Broker(app.capp.connection().as_uri(include_password=True),
http_api=http_api, broker_options=self.capp.conf.broker_transport_options,
broker_use_ssl=self.capp.conf.broker_use_ssl)
queues = await broker.queues(self.get_active_queue_names())
self.write({'active_queues': queues})
class ListTasks(BaseTaskHandler):
@web.authenticated
def get(self):
"""
List tasks
**Example request**:
.. sourcecode:: http
GET /api/tasks HTTP/1.1
Host: localhost:5555
User-Agent: HTTPie/0.8.0
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 1109
Content-Type: application/json; charset=UTF-8
Etag: "b2478118015c8b825f7b88ce6b660e5449746c37"
Server: TornadoServer/3.1.1
{
"e42ceb2d-8730-47b5-8b4d-8e0d2a1ef7c9": {
"args": "[3, 4]",
"client": null,
"clock": 1079,
"eta": null,
"exception": null,
"exchange": null,
"expires": null,
"failed": null,
"kwargs": "{}",
"name": "tasks.add",
"received": 1398505411.107885,
"result": "'7'",
"retried": null,
"retries": 0,
"revoked": null,
"routing_key": null,
"runtime": 0.01610181899741292,
"sent": null,
"started": 1398505411.108985,
"state": "SUCCESS",
"succeeded": 1398505411.124802,
"timestamp": 1398505411.124802,
"traceback": null,
"uuid": "e42ceb2d-8730-47b5-8b4d-8e0d2a1ef7c9",
"worker": "celery@worker1"
},
"f67ea225-ae9e-42a8-90b0-5de0b24507e0": {
"args": "[1, 2]",
"client": null,
"clock": 1042,
"eta": null,
"exception": null,
"exchange": null,
"expires": null,
"failed": null,
"kwargs": "{}",
"name": "tasks.add",
"received": 1398505395.327208,
"result": "'3'",
"retried": null,
"retries": 0,
"revoked": null,
"routing_key": null,
"runtime": 0.012884548006695695,
"sent": null,
"started": 1398505395.3289,
"state": "SUCCESS",
"succeeded": 1398505395.341089,
"timestamp": 1398505395.341089,
"traceback": null,
"uuid": "f67ea225-ae9e-42a8-90b0-5de0b24507e0",
"worker": "celery@worker1"
}
}
:query limit: maximum number of tasks
:query offset: skip first n tasks
:query sort_by: sort tasks by attribute (name, state, received, started)
:query workername: filter task by workername
:query taskname: filter tasks by taskname
:query state: filter tasks by state
:query received_start: filter tasks by received date (must be greater than) format %Y-%m-%d %H:%M
:query received_end: filter tasks by received date (must be less than) format %Y-%m-%d %H:%M
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
"""
app = self.application
limit = self.get_argument('limit', None)
offset = self.get_argument('offset', default=0, type=int)
worker = self.get_argument('workername', None)
type = self.get_argument('taskname', None)
state = self.get_argument('state', None)
received_start = self.get_argument('received_start', None)
received_end = self.get_argument('received_end', None)
sort_by = self.get_argument('sort_by', None)
search = self.get_argument('search', None)
limit = limit and int(limit)
offset = max(offset, 0)
worker = worker if worker != 'All' else None
type = type if type != 'All' else None
state = state if state != 'All' else None
result = []
for task_id, task in tasks.iter_tasks(
app.events, limit=limit, offset=offset, sort_by=sort_by, type=type,
worker=worker, state=state,
received_start=received_start,
received_end=received_end,
search=search
):
task = tasks.as_dict(task)
worker = task.pop('worker', None)
if worker is not None:
task['worker'] = worker.hostname
result.append((task_id, task))
self.write(OrderedDict(result))
class ListTaskTypes(BaseTaskHandler):
@web.authenticated
def get(self):
"""
List (seen) task types
**Example request**:
.. sourcecode:: http
GET /api/task/types HTTP/1.1
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 44
Content-Type: application/json; charset=UTF-8
{
"task-types": [
"tasks.add",
"tasks.sleep"
]
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
"""
seen_task_types = self.application.events.state.task_types()
response = {}
response['task-types'] = seen_task_types
self.write(response)
class TaskInfo(BaseTaskHandler):
@web.authenticated
def get(self, taskid):
"""
Get a task info
**Example request**:
.. sourcecode:: http
GET /api/task/info/91396550-c228-4111-9da4-9d88cfd5ddc6 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, compress
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 575
Content-Type: application/json; charset=UTF-8
{
"args": "[2, 2]",
"client": null,
"clock": 25,
"eta": null,
"exception": null,
"exchange": null,
"expires": null,
"failed": null,
"kwargs": "{}",
"name": "tasks.add",
"received": 1400806241.970742,
"result": "'4'",
"retried": null,
"retries": null,
"revoked": null,
"routing_key": null,
"runtime": 2.0037889280356467,
"sent": null,
"started": 1400806241.972624,
"state": "SUCCESS",
"succeeded": 1400806243.975336,
"task-id": "91396550-c228-4111-9da4-9d88cfd5ddc6",
"timestamp": 1400806243.975336,
"traceback": null,
"worker": "celery@worker1"
}
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
:statuscode 404: unknown task
"""
task = tasks.get_task_by_id(self.application.events, taskid)
if not task:
raise HTTPError(404, f"Unknown task '{taskid}'")
response = task.as_dict()
if task.worker is not None:
response['worker'] = task.worker.hostname
self.write(response)
================================================
FILE: flower/api/workers.py
================================================
import asyncio
import logging
from tornado import web
from .control import ControlHandler
logger = logging.getLogger(__name__)
class ListWorkers(ControlHandler):
@web.authenticated
async def get(self):
"""
List workers
**Example request**:
.. sourcecode:: http
GET /api/workers HTTP/1.1
Host: localhost:5555
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Content-Length: 1526
Content-Type: application/json; charset=UTF-8
Date: Tue, 28 Jul 2015 01:32:38 GMT
Etag: "fcdd75d85a82b4052275e28871d199aac1ece21c"
Server: TornadoServer/4.0.2
{
"celery@worker1": {
"active_queues": [
{
"alias": null,
"auto_delete": false,
"binding_arguments": null,
"bindings": [],
"durable": true,
"exchange": {
"arguments": null,
"auto_delete": false,
"delivery_mode": 2,
"durable": true,
"name": "celery",
"passive": false,
"type": "direct"
},
"exclusive": false,
"name": "celery",
"no_ack": false,
"queue_arguments": null,
"routing_key": "celery"
}
],
"conf": {
"CELERYBEAT_SCHEDULE": {},
"CELERY_INCLUDE": [
"celery.app.builtins",
"__main__"
],
"CELERY_SEND_TASK_SENT_EVENT": true,
"CELERY_TIMEZONE": "UTC"
},
"registered": [
"tasks.add",
"tasks.echo",
"tasks.error",
"tasks.retry",
"tasks.sleep"
],
"stats": {
"broker": {
"alternates": [],
"connect_timeout": 4,
"heartbeat": null,
"hostname": "127.0.0.1",
"insist": false,
"login_method": "AMQPLAIN",
"port": 5672,
"ssl": false,
"transport": "amqp",
"transport_options": {},
"uri_prefix": null,
"userid": "guest",
"virtual_host": "/"
},
"clock": "918",
"pid": 90494,
"pool": {
"max-concurrency": 4,
"max-tasks-per-child": "N/A",
"processes": [
90499,
90500,
90501,
90502
],
"put-guarded-by-semaphore": false,
"timeouts": [
0,
0
],
"writes": {
"all": "100.00%",
"avg": "100.00%",
"inqueues": {
"active": 0,
"total": 4
},
"raw": "1",
"total": 1
}
},
"prefetch_count": 16,
"rusage": {
"idrss": 0,
"inblock": 211,
"isrss": 0,
"ixrss": 0,
"majflt": 6,
"maxrss": 26996736,
"minflt": 11450,
"msgrcv": 4968,
"msgsnd": 1227,
"nivcsw": 1367,
"nsignals": 0,
"nswap": 0,
"nvcsw": 1855,
"oublock": 93,
"stime": 0.414564,
"utime": 0.975726
},
"total": {
"tasks.add": 1
}
},
"timestamp": 1438049312.073402
}
}
:query refresh: run inspect to get updated list of workers
:query workername: get info for workername
:query status: only get worker status info
:reqheader Authorization: optional OAuth token to authenticate
:statuscode 200: no error
:statuscode 401: unauthorized request
"""
refresh = self.get_argument('refresh', default=False, type=bool)
status = self.get_argument('status', default=False, type=bool)
workername = self.get_argument('workername', default=None)
if refresh:
try:
await asyncio.wait(self.application.update_workers(workername=workername))
except Exception as e:
msg = f"Failed to update workers: {e}"
logger.error(msg)
raise web.HTTPError(503, msg)
if status:
info = {}
for name, worker in self.application.events.state.workers.items():
info[name] = worker.alive
self.write(info)
return
if self.application.workers and not refresh and\
workername in self.application.workers:
self.write({workername: self.application.workers[workername]})
return
if workername and not self.is_worker(workername):
raise web.HTTPError(404, f"Unknown worker '{workername}'")
if workername:
self.write({workername: self.application.workers[workername]})
else:
self.write(self.application.workers)
================================================
FILE: flower/app.py
================================================
import sys
import logging
from concurrent.futures import ThreadPoolExecutor
import celery
import tornado.web
from tornado import ioloop
from tornado.httpserver import HTTPServer
from tornado.web import url
from .urls import handlers as default_handlers
from .events import Events
from .inspector import Inspector
from .options import default_options
logger = logging.getLogger(__name__)
if sys.version_info[0] == 3 and sys.version_info[1] >= 8 and sys.platform.startswith('win'):
import asyncio
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
# pylint: disable=consider-using-f-string
def rewrite_handler(handler, url_prefix):
if isinstance(handler, url):
return url("/{}{}".format(url_prefix.strip("/"), handler.regex.pattern),
handler.handler_class, handler.kwargs, handler.name)
return ("/{}{}".format(url_prefix.strip("/"), handler[0]), handler[1])
class Flower(tornado.web.Application):
pool_executor_cls = ThreadPoolExecutor
max_workers = None
def __init__(self, options=None, capp=None, events=None,
io_loop=None, **kwargs):
handlers = default_handlers
if options is not None and options.url_prefix:
handlers = [rewrite_handler(h, options.url_prefix) for h in handlers]
kwargs.update(handlers=handlers)
super().__init__(**kwargs)
self.options = options or default_options
self.io_loop = io_loop or ioloop.IOLoop.instance()
self.ssl_options = kwargs.get('ssl_options', None)
self.capp = capp or celery.Celery()
self.capp.loader.import_default_modules()
self.executor = self.pool_executor_cls(max_workers=self.max_workers)
self.io_loop.set_default_executor(self.executor)
self.inspector = Inspector(self.io_loop, self.capp, self.options.inspect_timeout / 1000.0)
self.events = events or Events(
self.capp,
db=self.options.db,
persistent=self.options.persistent,
state_save_interval=self.options.state_save_interval,
enable_events=self.options.enable_events,
io_loop=self.io_loop,
max_workers_in_memory=self.options.max_workers,
max_tasks_in_memory=self.options.max_tasks)
self.started = False
def start(self):
self.events.start()
if not self.options.unix_socket:
self.listen(self.options.port, address=self.options.address,
ssl_options=self.ssl_options,
xheaders=self.options.xheaders)
else:
from tornado.netutil import bind_unix_socket
server = HTTPServer(self)
socket = bind_unix_socket(self.options.unix_socket, mode=0o777)
server.add_socket(socket)
self.started = True
self.update_workers()
self.io_loop.start()
def stop(self):
if self.started:
self.events.stop()
logging.debug("Stopping executors...")
self.executor.shutdown(wait=False)
logging.debug("Stopping event loop...")
self.io_loop.stop()
self.started = False
@property
def transport(self):
return getattr(self.capp.connection().transport, 'driver_type', None)
@property
def workers(self):
return self.inspector.workers
def update_workers(self, workername=None):
return self.inspector.inspect(workername)
================================================
FILE: flower/command.py
================================================
import os
import sys
import atexit
import signal
import logging
from pprint import pformat
from logging import NullHandler
import click
from tornado.options import options
from tornado.options import parse_command_line, parse_config_file
from tornado.log import enable_pretty_logging
from celery.bin.base import CeleryCommand
from .app import Flower
from .urls import settings
from .utils import abs_path, prepend_url, strtobool
from .options import DEFAULT_CONFIG_FILE, default_options
from .views.auth import validate_auth_option
logger = logging.getLogger(__name__)
ENV_VAR_PREFIX = 'FLOWER_'
def sigterm_handler(signum, _):
logger.info('%s detected, shutting down', signum)
sys.exit(0)
@click.command(cls=CeleryCommand,
context_settings={
'ignore_unknown_options': True
})
@click.argument("tornado_argv", nargs=-1, type=click.UNPROCESSED)
@click.pass_context
def flower(ctx, tornado_argv):
"""Web based tool for monitoring and administrating Celery clusters."""
warn_about_celery_args_used_in_flower_command(ctx, tornado_argv)
apply_env_options()
apply_options(sys.argv[0], tornado_argv)
extract_settings()
setup_logging()
app = ctx.obj.app
flower_app = Flower(capp=app, options=options, **settings)
atexit.register(flower_app.stop)
signal.signal(signal.SIGTERM, sigterm_handler)
if not ctx.obj.quiet:
print_banner(app, 'ssl_options' in settings)
try:
flower_app.start()
except (KeyboardInterrupt, SystemExit):
pass
def apply_env_options():
"apply options passed through environment variables"
env_options = filter(is_flower_envvar, os.environ)
for env_var_name in env_options:
name = env_var_name.replace(ENV_VAR_PREFIX, '', 1).lower()
value = os.environ[env_var_name]
try:
option = options._options[name] # pylint: disable=protected-access
except KeyError:
option = options._options[name.replace('_', '-')] # pylint: disable=protected-access
if option.multiple:
value = [option.type(i) for i in value.split(',')]
else:
if option.type is bool:
value = bool(strtobool(value))
else:
value = option.type(value)
setattr(options, name, value)
def apply_options(prog_name, argv):
"apply options passed through the configuration file"
argv = list(filter(is_flower_option, argv))
# parse the command line to get --conf option
parse_command_line([prog_name] + argv)
try:
parse_config_file(os.path.abspath(options.conf), final=False)
parse_command_line([prog_name] + argv)
except IOError:
if os.path.basename(options.conf) != DEFAULT_CONFIG_FILE:
raise
def warn_about_celery_args_used_in_flower_command(ctx, flower_args):
celery_options = [option for param in ctx.parent.command.params for option in param.opts]
incorrectly_used_args = []
for arg in flower_args:
arg_name, _, _ = arg.partition("=")
if arg_name in celery_options:
incorrectly_used_args.append(arg_name)
if incorrectly_used_args:
logger.warning(
'You have incorrectly specified the following celery arguments after flower command:'
' %s. '
'Please specify them after celery command instead following this template: '
'celery [celery args] flower [flower args].', incorrectly_used_args
)
def setup_logging():
if options.debug and options.logging == 'info':
options.logging = 'debug'
enable_pretty_logging()
else:
logging.getLogger("tornado.access").addHandler(NullHandler())
logging.getLogger("tornado.access").propagate = False
def extract_settings():
settings['debug'] = options.debug
if options.cookie_secret:
settings['cookie_secret'] = options.cookie_secret
if options.url_prefix:
for name in ['login_url', 'static_url_prefix']:
settings[name] = prepend_url(settings[name], options.url_prefix)
if options.auth:
settings['oauth'] = {
'key': options.oauth2_key or os.environ.get('FLOWER_OAUTH2_KEY'),
'secret': options.oauth2_secret or os.environ.get('FLOWER_OAUTH2_SECRET'),
'redirect_uri': options.oauth2_redirect_uri or os.environ.get('FLOWER_OAUTH2_REDIRECT_URI'),
}
if options.certfile and options.keyfile:
settings['ssl_options'] = dict(certfile=abs_path(options.certfile),
keyfile=abs_path(options.keyfile))
if options.ca_certs:
settings['ssl_options']['ca_certs'] = abs_path(options.ca_certs)
if options.auth and not validate_auth_option(options.auth):
logger.error("Invalid '--auth' option: %s", options.auth)
sys.exit(1)
def is_flower_option(arg):
name, _, _ = arg.lstrip('-').partition("=")
name = name.replace('-', '_')
return hasattr(options, name)
def is_flower_envvar(name):
return name.startswith(ENV_VAR_PREFIX) and \
name[len(ENV_VAR_PREFIX):].lower() in default_options
def print_banner(app, ssl):
if not options.unix_socket:
if options.url_prefix:
prefix_str = f'/{options.url_prefix}/'
else:
prefix_str = ''
logger.info(
"Visit me at http%s://%s:%s%s", 's' if ssl else '',
options.address or '0.0.0.0', options.port,
prefix_str
)
else:
logger.info("Visit me via unix socket file: %s", options.unix_socket)
logger.info('Broker: %s', app.connection().as_uri())
logger.info(
'Registered tasks: \n%s',
pformat(sorted(app.tasks.keys()))
)
logger.debug('Settings: %s', pformat(settings))
================================================
FILE: flower/events.py
================================================
import collections
import logging
import shelve
import threading
import time
from collections import Counter
from functools import partial
from celery.events import EventReceiver
from celery.events.state import State
from prometheus_client import Counter as PrometheusCounter
from prometheus_client import Gauge, Histogram
from tornado.ioloop import PeriodicCallback
from tornado.options import options
logger = logging.getLogger(__name__)
PROMETHEUS_METRICS = None
def get_prometheus_metrics():
global PROMETHEUS_METRICS # pylint: disable=global-statement
if PROMETHEUS_METRICS is None:
PROMETHEUS_METRICS = PrometheusMetrics()
return PROMETHEUS_METRICS
class PrometheusMetrics:
def __init__(self):
self.events = PrometheusCounter('flower_events_total', "Number of events", ['worker', 'type', 'task'])
self.runtime = Histogram(
'flower_task_runtime_seconds',
"Task runtime",
['worker', 'task'],
buckets=options.task_runtime_metric_buckets
)
self.prefetch_time = Gauge(
'flower_task_prefetch_time_seconds',
"The time the task spent waiting at the celery worker to be executed.",
['worker', 'task']
)
self.number_of_prefetched_tasks = Gauge(
'flower_worker_prefetched_tasks',
'Number of tasks of given type prefetched at a worker',
['worker', 'task']
)
self.worker_online = Gauge('flower_worker_online', "Worker online status", ['worker'])
self.worker_number_of_currently_executing_tasks = Gauge(
'flower_worker_number_of_currently_executing_tasks',
"Number of tasks currently executing at a worker",
['worker']
)
class EventsState(State):
# EventsState object is created and accessed only from ioloop thread
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.counter = collections.defaultdict(Counter)
self.metrics = get_prometheus_metrics()
def event(self, event):
# Save the event
super().event(event)
worker_name = event['hostname']
event_type = event['type']
self.counter[worker_name][event_type] += 1
if event_type.startswith('task-'):
task_id = event['uuid']
task = self.tasks.get(task_id)
task_name = event.get('name', '')
if not task_name and task_id in self.tasks:
task_name = task.name or ''
self.metrics.events.labels(worker_name, event_type, task_name).inc()
runtime = event.get('runtime', 0)
if runtime:
self.metrics.runtime.labels(worker_name, task_name).observe(runtime)
task_started = task.started
task_received = task.received
if event_type == 'task-received' and not task.eta and task_received:
self.metrics.number_of_prefetched_tasks.labels(worker_name, task_name).inc()
if event_type == 'task-started' and not task.eta and task_started and task_received:
self.metrics.prefetch_time.labels(worker_name, task_name).set(task_started - task_received)
self.metrics.number_of_prefetched_tasks.labels(worker_name, task_name).dec()
if event_type in ['task-succeeded', 'task-failed'] and not task.eta and task_started and task_received:
self.metrics.prefetch_time.labels(worker_name, task_name).set(0)
if event_type == 'worker-online':
self.metrics.worker_online.labels(worker_name).set(1)
if event_type == 'worker-heartbeat':
self.metrics.worker_online.labels(worker_name).set(1)
num_executing_tasks = event.get('active')
if num_executing_tasks is not None:
self.metrics.worker_number_of_currently_executing_tasks.labels(worker_name).set(num_executing_tasks)
if event_type == 'worker-offline':
self.metrics.worker_online.labels(worker_name).set(0)
class Events(threading.Thread):
events_enable_interval = 5000
# pylint: disable=too-many-arguments
def __init__(self, capp, io_loop, db=None, persistent=False,
enable_events=True, state_save_interval=0,
**kwargs):
threading.Thread.__init__(self)
self.daemon = True
self.io_loop = io_loop
self.capp = capp
self.db = db
self.persistent = persistent
self.enable_events = enable_events
self.state = None
self.state_save_timer = None
if self.persistent:
logger.debug("Loading state from '%s'...", self.db)
state = shelve.open(self.db)
if state:
self.state = state['events']
state.close()
if state_save_interval:
self.state_save_timer = PeriodicCallback(self.save_state,
state_save_interval)
if not self.state:
self.state = EventsState(**kwargs)
self.timer = PeriodicCallback(self.on_enable_events,
self.events_enable_interval)
def start(self):
threading.Thread.start(self)
if self.enable_events:
logger.debug("Starting enable events timer...")
self.timer.start()
if self.state_save_timer:
logger.debug("Starting state save timer...")
self.state_save_timer.start()
def stop(self):
if self.enable_events:
logger.debug("Stopping enable events timer...")
self.timer.stop()
if self.state_save_timer:
logger.debug("Stopping state save timer...")
self.state_save_timer.stop()
if self.persistent:
self.save_state()
def run(self):
try_interval = 1
while True:
try:
try_interval *= 2
with self.capp.connection() as conn:
recv = EventReceiver(conn,
handlers={"*": self.on_event},
app=self.capp)
try_interval = 1
logger.debug("Capturing events...")
recv.capture(limit=None, timeout=None, wakeup=True)
except (KeyboardInterrupt, SystemExit):
try:
import _thread as thread
except ImportError:
import thread
thread.interrupt_main()
except Exception as e:
logger.error("Failed to capture events: '%s', "
"trying again in %s seconds.",
e, try_interval)
logger.debug(e, exc_info=True)
time.sleep(try_interval)
def save_state(self):
logger.debug("Saving state to '%s'...", self.db)
state = shelve.open(self.db, flag='n')
state['events'] = self.state
state.close()
def on_enable_events(self):
# Periodically enable events for workers
# launched after flower
self.io_loop.run_in_executor(None, self.capp.control.enable_events)
def on_event(self, event):
# Call EventsState.event in ioloop thread to avoid synchronization
self.io_loop.add_callback(partial(self.state.event, event))
================================================
FILE: flower/inspector.py
================================================
import collections
import logging
import time
from functools import partial
logger = logging.getLogger(__name__)
class Inspector:
methods = ('stats', 'active_queues', 'registered', 'scheduled',
'active', 'reserved', 'revoked', 'conf')
def __init__(self, io_loop, capp, timeout):
self.io_loop = io_loop
self.capp = capp
self.timeout = timeout
self.workers = collections.defaultdict(dict)
def inspect(self, workername=None):
feutures = []
for method in self.methods:
feutures.append(self.io_loop.run_in_executor(None, partial(self._inspect, method, workername)))
return feutures
def _on_update(self, workername, method, response):
info = self.workers[workername]
info[method] = response
info['timestamp'] = time.time()
def _inspect(self, method, workername):
destination = [workername] if workername else None
inspect = self.capp.control.inspect(timeout=self.timeout, destination=destination)
logger.debug('Sending %s inspect command', method)
start = time.time()
result = (
getattr(inspect, method)()
if method != 'active'
else getattr(inspect, method)(safe=True)
)
logger.debug("Inspect command %s took %.2fs to complete", method, time.time() - start)
if result is None or 'error' in result:
logger.warning("Inspect method %s failed", method)
return
for worker, response in result.items():
if response is not None:
self.io_loop.add_callback(partial(self._on_update, worker, method, response))
================================================
FILE: flower/options.py
================================================
import types
from secrets import token_urlsafe
from prometheus_client import Histogram
from tornado.options import define, options
DEFAULT_CONFIG_FILE = 'flowerconfig.py'
define("port", default=5555,
help="run on the given port", type=int)
define("address", default='',
help="run on the given address", type=str)
define("unix_socket", default='',
help="path to unix socket to bind", type=str)
define("debug", default=False,
help="run in debug mode", type=bool)
define("inspect_timeout", default=1000.0, type=float,
help="inspect timeout (in milliseconds)")
define("auth", default='', type=str,
help="regexp of emails to grant access")
define("basic_auth", type=str, default=None, multiple=True,
help="enable http basic authentication")
define("oauth2_key", type=str, default=None,
help="OAuth2 key (requires --auth)")
define("oauth2_secret", type=str, default=None,
help="OAuth2 secret (requires --auth)")
define("oauth2_redirect_uri", type=str, default=None,
help="OAuth2 redirect uri (requires --auth)")
define("max_workers", type=int, default=5000,
help="maximum number of workers to keep in memory")
define("max_tasks", type=int, default=100000,
help="maximum number of tasks to keep in memory")
define("db", type=str, default='flower',
help="flower database file")
define("persistent", type=bool, default=False,
help="enable persistent mode")
define("state_save_interval", type=int, default=0,
help="state save interval (in milliseconds)")
define("broker_api", type=str, default=None,
help="inspect broker e.g. http://guest:guest@localhost:15672/api/")
define("ca_certs", type=str, default=None,
help="SSL certificate authority (CA) file")
define("certfile", type=str, default=None,
help="SSL certificate file")
define("keyfile", type=str, default=None,
help="SSL key file")
define("xheaders", type=bool, default=False,
help="enable support for the 'X-Real-Ip' and 'X-Scheme' headers.")
define("auto_refresh", default=True,
help="refresh workerss", type=bool)
define("purge_offline_workers", default=None, type=int,
help="time (in seconds) after which offline workers are purged from workers")
define("cookie_secret", type=str, default=token_urlsafe(64),
help="secure cookie secret")
define("conf", default=DEFAULT_CONFIG_FILE,
help="configuration file")
define("enable_events", type=bool, default=True,
help="periodically enable Celery events")
define("format_task", type=types.FunctionType, default=None,
help="use custom task formatter")
define("natural_time", type=bool, default=False,
help="show time in relative format")
define("tasks_columns", type=str,
default="name,uuid,state,args,kwargs,result,received,started,runtime,worker",
help="slugs of columns on /tasks/ page, delimited by comma")
define("auth_provider", default=None, type=str, help="auth handler class")
define("url_prefix", type=str, help="base url prefix")
define("task_runtime_metric_buckets", type=float, default=Histogram.DEFAULT_BUCKETS,
multiple=True, help="histogram latency bucket value")
default_options = options
================================================
FILE: flower/static/css/flower.css
================================================
.bg-green {
background-color: #f0ffeb;
}
.dataTables_wrapper {
border: 1px solid #c7ecb8;
}
.dataTables_filter input {
width: 50%;
text-indent: 5px;
}
.dataTables_length {
margin: 10px;
}
div.dataTables_wrapper .dataTables_filter input {
width: 100%;
margin: 10px;
border: 1px solid #c7ecb8;
}
.dataTables_info {
padding: 5px;
}
@media (min-width: 768px) {
div.dataTables_wrapper .dataTables_filter input {
width: 500px;
}
}
.overflow-auto {
max-width: 400px;
text-overflow: ellipsis;
}
.overflow-auto::-webkit-scrollbar {
display: none;
}
================================================
FILE: flower/static/js/flower.js
================================================
/*jslint browser: true */
/*global $, WebSocket, jQuery */
var flower = (function () {
"use strict";
var alertContainer = document.getElementById('alert-container');
function show_alert(message, type) {
var wrapper = document.createElement('div');
wrapper.innerHTML = `
<div class="alert alert-${type} alert-dismissible" role="alert">
<div>${message}</div>
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
</div>`;
alertContainer.appendChild(wrapper);
}
function url_prefix() {
var prefix = $('#url_prefix').val();
if (prefix) {
prefix = prefix.replace(/\/+$/, '');
if (prefix.startsWith('/')) {
return prefix;
} else {
return '/' + prefix;
}
}
return '';
}
//https://github.com/DataTables/DataTables/blob/1.10.11/media/js/jquery.dataTables.js#L14882
function htmlEscapeEntities(d) {
return typeof d === 'string' ?
d.replace(/</g, '<').replace(/>/g, '>').replace(/"/g, '"') :
d;
}
function active_page(name) {
var pathname = $(location).attr('pathname');
if (name === '/') {
return pathname === (url_prefix() + name);
}
else {
return pathname.startsWith(url_prefix() + name);
}
}
$('#worker-refresh').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
$('.dropdown-toggle').dropdown('hide');
var workername = $('#workername').text();
$.ajax({
type: 'GET',
url: url_prefix() + '/api/workers',
dataType: 'json',
data: {
workername: unescape(workername),
refresh: 1
},
success: function (data) {
show_alert(data.message || 'Successfully refreshed', 'success');
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-refresh-all').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
$('.dropdown-toggle').dropdown('hide');
$.ajax({
type: 'GET',
url: url_prefix() + '/api/workers',
dataType: 'json',
data: {
refresh: 1
},
success: function (data) {
show_alert(data.message || 'Refreshed All Workers', 'success');
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-pool-restart').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
$('.dropdown-toggle').dropdown('hide');
var workername = $('#workername').text();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/pool/restart/' + workername,
dataType: 'json',
data: {
workername: workername
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-shutdown').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
$('.dropdown-toggle').dropdown('hide');
var workername = $('#workername').text();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/shutdown/' + workername,
dataType: 'json',
data: {
workername: workername
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-pool-grow').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var workername = $('#workername').text(),
grow_size = $('#pool-size').val();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/pool/grow/' + workername,
dataType: 'json',
data: {
'workername': workername,
'n': grow_size,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-pool-shrink').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var workername = $('#workername').text(),
shrink_size = $('#pool-size').val();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/pool/shrink/' + workername,
dataType: 'json',
data: {
'workername': workername,
'n': shrink_size,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-pool-autoscale').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var workername = $('#workername').text(),
min = $('#min-autoscale').val(),
max = $('#max-autoscale').val();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/pool/autoscale/' + workername,
dataType: 'json',
data: {
'workername': workername,
'min': min,
'max': max,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-add-consumer').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var workername = $('#workername').text(),
queue = $('#add-consumer-name').val();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/queue/add-consumer/' + workername,
dataType: 'json',
data: {
'workername': workername,
'queue': queue,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#worker-queues').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
if (!event.target.id.startsWith("worker-cancel-consumer")) {
return;
}
var workername = $('#workername').text(),
queue = $(event.target).closest("tr").children("td:eq(0)").text();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/worker/queue/cancel-consumer/' + workername,
dataType: 'json',
data: {
'workername': workername,
'queue': queue,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#limits-table').on('click', function (event) {
if (event.target.id.startsWith("task-timeout-")) {
var timeout = parseInt($(event.target).siblings().closest("input").val()),
type = $(event.target).text().toLowerCase(),
taskname = $(event.target).closest("tr").children("td:eq(0)").text(),
post_data = {'workername': $('#workername').text()};
taskname = taskname.split(' ')[0]; // removes [rate_limit=xxx]
post_data[type] = timeout;
if (!Number.isInteger(timeout)) {
show_alert("Invalid timeout value", "danger");
return;
}
$.ajax({
type: 'POST',
url: url_prefix() + '/api/task/timeout/' + taskname,
dataType: 'json',
data: post_data,
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert($(data.responseText).text(), "danger");
}
});
} else if (event.target.id.startsWith("task-rate-limit-")) {
var taskname = $(event.target).closest("tr").children("td:eq(0)").text(),
workername = $('#workername').text(),
ratelimit = parseInt($(event.target).prev().val());
taskname = taskname.split(' ')[0]; // removes [rate_limit=xxx]
$.ajax({
type: 'POST',
url: url_prefix() + '/api/task/rate-limit/' + taskname,
dataType: 'json',
data: {
'workername': workername,
'ratelimit': ratelimit,
},
success: function (data) {
show_alert(data.message, "success");
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
}
});
$('#task-revoke').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var taskid = $('#taskid').text();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/task/revoke/' + taskid,
dataType: 'json',
data: {
'terminate': false,
},
success: function (data) {
show_alert(data.message, "success");
document.getElementById("task-revoke").disabled = true;
setTimeout(function() {location.reload();}, 5000);
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
$('#task-terminate').on('click', function (event) {
event.preventDefault();
event.stopPropagation();
var taskid = $('#taskid').text();
$.ajax({
type: 'POST',
url: url_prefix() + '/api/task/revoke/' + taskid,
dataType: 'json',
data: {
'terminate': true,
},
success: function (data) {
show_alert(data.message, "success");
document.getElementById("task-terminate").disabled = true;
setTimeout(function() {location.reload();}, 5000);
},
error: function (data) {
show_alert(data.responseText, "danger");
}
});
});
function sum(a, b) {
return parseInt(a, 10) + parseInt(b, 10);
}
function format_time(timestamp) {
var time = $('#time').val(),
prefix = time.startsWith('natural-time') ? 'natural-time' : 'time',
tz = time.substr(prefix.length + 1) || 'UTC';
if (prefix === 'natural-time') {
return moment.unix(timestamp).tz(tz).fromNow();
}
return moment.unix(timestamp).tz(tz).format('YYYY-MM-DD HH:mm:ss.SSS');
}
function isColumnVisible(name) {
var columns = $('#columns').val();
if (columns === "all")
return true;
if (columns) {
columns = columns.split(',').map(function (e) {
return e.trim();
});
return columns.indexOf(name) !== -1;
}
return true;
}
$.urlParam = function (name) {
var results = new RegExp('[\\?&]' + name + '=([^&#]*)').exec(window.location.href);
return (results && results[1]) || 0;
};
$(document).ready(function () {
//https://github.com/twitter/bootstrap/issues/1768
var shiftWindow = function () {
scrollBy(0, -50);
};
if (location.hash) {
shiftWindow();
}
window.addEventListener("hashchange", shiftWindow);
// Make bootstrap tabs persistent
$(document).ready(function () {
if (location.hash !== '') {
$('a[href="' + location.hash + '"]').tab('show');
}
// Listen for tab shown events and update the URL hash fragment accordingly
$('.nav-tabs a[data-bs-toggle="tab"]').on('shown.bs.tab', function (event) {
const tabPaneId = $(event.target).attr('href').substr(1);
if (tabPaneId) {
window.location.hash = tabPaneId;
}
});
});
});
$(document).ready(function () {
gitextract_3ow7j80i/ ├── .github/ │ ├── ISSUE_TEMPLATE/ │ │ ├── bug_report.md │ │ └── feature_request.md │ ├── dependabot.yml │ └── workflows/ │ ├── build.yml │ └── docker.yml ├── .gitignore ├── .pylintrc ├── .readthedocs.yaml ├── CONTRIBUTORS ├── Dockerfile ├── LICENSE ├── MANIFEST.in ├── README.rst ├── docker-compose.yml ├── docs/ │ ├── Makefile │ ├── _static/ │ │ └── .keep │ ├── _templates/ │ │ ├── localtoc.html │ │ ├── page.html │ │ ├── sidebarintro.html │ │ └── sidebarlogo.html │ ├── _theme/ │ │ └── celery/ │ │ ├── static/ │ │ │ └── celery.css_t │ │ └── theme.conf │ ├── api.ipynb │ ├── api.rst │ ├── auth.rst │ ├── conf.py │ ├── config.rst │ ├── features.rst │ ├── index.rst │ ├── install.rst │ ├── man.rst │ ├── prometheus-integration.rst │ ├── reverse-proxy.rst │ ├── tasks.py │ └── tasks_filter.rst ├── examples/ │ ├── celery-monitoring-grafana-dashboard.json │ ├── celeryconfig.py │ ├── nginx.conf │ ├── prometheus-alerts.yaml │ ├── pycharm-configurations/ │ │ ├── Grafana.run.xml │ │ ├── Prometheus.run.xml │ │ └── Redis.run.xml │ └── tasks.py ├── flower/ │ ├── __init__.py │ ├── __main__.py │ ├── api/ │ │ ├── __init__.py │ │ ├── control.py │ │ ├── tasks.py │ │ └── workers.py │ ├── app.py │ ├── command.py │ ├── events.py │ ├── inspector.py │ ├── options.py │ ├── static/ │ │ ├── css/ │ │ │ └── flower.css │ │ ├── js/ │ │ │ └── flower.js │ │ └── swagger.json │ ├── templates/ │ │ ├── 404.html │ │ ├── base.html │ │ ├── broker.html │ │ ├── error.html │ │ ├── navbar.html │ │ ├── task.html │ │ ├── tasks.html │ │ ├── worker.html │ │ └── workers.html │ ├── urls.py │ ├── utils/ │ │ ├── __init__.py │ │ ├── broker.py │ │ ├── search.py │ │ ├── tasks.py │ │ └── template.py │ └── views/ │ ├── __init__.py │ ├── auth.py │ ├── broker.py │ ├── error.py │ ├── monitor.py │ ├── tasks.py │ └── workers.py ├── prometheus.yml ├── requirements/ │ ├── default.txt │ ├── dev.txt │ ├── docs.txt │ └── test.txt ├── scss/ │ ├── build.sh │ └── flower.scss ├── setup.cfg ├── setup.py ├── tests/ │ ├── __init__.py │ ├── call-tasks.sh │ ├── load.py │ ├── run-unit-tests.sh │ └── unit/ │ ├── __init__.py │ ├── __main__.py │ ├── api/ │ │ ├── __init__.py │ │ ├── test_auth.py │ │ ├── test_control.py │ │ ├── test_tasks.py │ │ └── test_workers.py │ ├── test_command.py │ ├── utils/ │ │ ├── __init__.py │ │ ├── test_broker.py │ │ ├── test_search.py │ │ ├── test_template.py │ │ └── test_utils.py │ └── views/ │ ├── __init__.py │ ├── test_auth.py │ ├── test_broker.py │ ├── test_error.py │ ├── test_monitor.py │ ├── test_tasks.py │ ├── test_url_handlers.py │ └── test_workers.py └── tox.ini
SYMBOL INDEX (445 symbols across 46 files)
FILE: docs/tasks.py
function add (line 13) | def add(x, y):
function sub (line 18) | def sub(x, y):
FILE: examples/tasks.py
function add (line 16) | def add(x, y):
function sleep (line 21) | def sleep(seconds):
function echo (line 26) | def echo(msg, timestamp=False):
function error (line 31) | def error(msg):
FILE: flower/__main__.py
function main (line 6) | def main():
FILE: flower/api/__init__.py
class BaseApiHandler (line 9) | class BaseApiHandler(BaseHandler):
method prepare (line 10) | def prepare(self):
method write_error (line 17) | def write_error(self, status_code, **kwargs):
FILE: flower/api/control.py
class ControlHandler (line 10) | class ControlHandler(BaseApiHandler):
method is_worker (line 11) | def is_worker(self, workername):
method error_reason (line 14) | def error_reason(self, workername, response):
class WorkerShutDown (line 25) | class WorkerShutDown(ControlHandler):
method post (line 27) | def post(self, workername):
class WorkerPoolRestart (line 64) | class WorkerPoolRestart(ControlHandler):
method post (line 66) | def post(self, workername):
class WorkerPoolGrow (line 112) | class WorkerPoolGrow(ControlHandler):
method post (line 114) | def post(self, workername):
class WorkerPoolShrink (line 163) | class WorkerPoolShrink(ControlHandler):
method post (line 165) | def post(self, workername):
class WorkerPoolAutoscale (line 214) | class WorkerPoolAutoscale(ControlHandler):
method post (line 216) | def post(self, workername):
class WorkerQueueAddConsumer (line 271) | class WorkerQueueAddConsumer(ControlHandler):
method post (line 273) | def post(self, workername):
class WorkerQueueCancelConsumer (line 324) | class WorkerQueueCancelConsumer(ControlHandler):
method post (line 326) | def post(self, workername):
class TaskRevoke (line 377) | class TaskRevoke(ControlHandler):
method post (line 379) | def post(self, taskid):
class TaskTimout (line 417) | class TaskTimout(ControlHandler):
method post (line 419) | def post(self, taskname):
class TaskRateLimit (line 477) | class TaskRateLimit(ControlHandler):
method post (line 479) | def post(self, taskname):
FILE: flower/api/tasks.py
class BaseTaskHandler (line 22) | class BaseTaskHandler(BaseApiHandler):
method get_task_args (line 25) | def get_task_args(self):
method backend_configured (line 44) | def backend_configured(result):
method write_error (line 47) | def write_error(self, status_code, **kwargs):
method update_response_result (line 50) | def update_response_result(self, response, result):
method normalize_options (line 57) | def normalize_options(self, options):
method safe_result (line 71) | def safe_result(self, result):
class TaskApply (line 80) | class TaskApply(BaseTaskHandler):
method post (line 82) | async def post(self, taskname):
method wait_results (line 143) | def wait_results(self, result, response):
class TaskAsyncApply (line 153) | class TaskAsyncApply(BaseTaskHandler):
method post (line 156) | def post(self, taskname):
class TaskSend (line 218) | class TaskSend(BaseTaskHandler):
method post (line 220) | def post(self, taskname):
class TaskResult (line 270) | class TaskResult(BaseTaskHandler):
method get (line 272) | def get(self, taskid):
class TaskAbort (line 319) | class TaskAbort(BaseTaskHandler):
method post (line 321) | def post(self, taskid):
class GetQueueLengths (line 360) | class GetQueueLengths(BaseTaskHandler):
method get (line 362) | async def get(self):
class ListTasks (line 407) | class ListTasks(BaseTaskHandler):
method get (line 409) | def get(self):
class ListTaskTypes (line 533) | class ListTaskTypes(BaseTaskHandler):
method get (line 535) | def get(self):
class TaskInfo (line 572) | class TaskInfo(BaseTaskHandler):
method get (line 574) | def get(self, taskid):
FILE: flower/api/workers.py
class ListWorkers (line 11) | class ListWorkers(ControlHandler):
method get (line 13) | async def get(self):
FILE: flower/app.py
function rewrite_handler (line 27) | def rewrite_handler(handler, url_prefix):
class Flower (line 34) | class Flower(tornado.web.Application):
method __init__ (line 38) | def __init__(self, options=None, capp=None, events=None,
method start (line 68) | def start(self):
method stop (line 85) | def stop(self):
method transport (line 95) | def transport(self):
method workers (line 99) | def workers(self):
method update_workers (line 102) | def update_workers(self, workername=None):
FILE: flower/command.py
function sigterm_handler (line 27) | def sigterm_handler(signum, _):
function flower (line 38) | def flower(ctx, tornado_argv):
function apply_env_options (line 62) | def apply_env_options():
function apply_options (line 82) | def apply_options(prog_name, argv):
function warn_about_celery_args_used_in_flower_command (line 95) | def warn_about_celery_args_used_in_flower_command(ctx, flower_args):
function setup_logging (line 113) | def setup_logging():
function extract_settings (line 122) | def extract_settings():
function is_flower_option (line 150) | def is_flower_option(arg):
function is_flower_envvar (line 156) | def is_flower_envvar(name):
function print_banner (line 161) | def print_banner(app, ssl):
FILE: flower/events.py
function get_prometheus_metrics (line 21) | def get_prometheus_metrics():
class PrometheusMetrics (line 29) | class PrometheusMetrics:
method __init__ (line 30) | def __init__(self):
class EventsState (line 57) | class EventsState(State):
method __init__ (line 60) | def __init__(self, *args, **kwargs):
method event (line 65) | def event(self, event):
class Events (line 113) | class Events(threading.Thread):
method __init__ (line 117) | def __init__(self, capp, io_loop, db=None, persistent=False,
method start (line 149) | def start(self):
method stop (line 159) | def stop(self):
method run (line 171) | def run(self):
method save_state (line 197) | def save_state(self):
method on_enable_events (line 203) | def on_enable_events(self):
method on_event (line 208) | def on_event(self, event):
FILE: flower/inspector.py
class Inspector (line 9) | class Inspector:
method __init__ (line 13) | def __init__(self, io_loop, capp, timeout):
method inspect (line 19) | def inspect(self, workername=None):
method _on_update (line 25) | def _on_update(self, workername, method, response):
method _inspect (line 30) | def _inspect(self, method, workername):
FILE: flower/static/js/flower.js
function show_alert (line 8) | function show_alert(message, type) {
function url_prefix (line 18) | function url_prefix() {
function htmlEscapeEntities (line 32) | function htmlEscapeEntities(d) {
function active_page (line 38) | function active_page(name) {
function sum (line 365) | function sum(a, b) {
function format_time (line 369) | function format_time(timestamp) {
function isColumnVisible (line 380) | function isColumnVisible(name) {
FILE: flower/utils/__init__.py
function gen_cookie_secret (line 8) | def gen_cookie_secret():
function bugreport (line 12) | def bugreport(app=None):
function abs_path (line 31) | def abs_path(path):
function prepend_url (line 39) | def prepend_url(url, prefix):
function strtobool (line 43) | def strtobool(val):
FILE: flower/utils/broker.py
class BrokerBase (line 20) | class BrokerBase:
method __init__ (line 21) | def __init__(self, broker_url, *_, **__):
method queues (line 33) | async def queues(self, names):
class RabbitMQ (line 37) | class RabbitMQ(BrokerBase):
method __init__ (line 38) | def __init__(self, broker_url, http_api, io_loop=None, **__):
method queues (line 58) | async def queues(self, names):
method validate_http_api (line 82) | def validate_http_api(cls, http_api):
class RedisBase (line 88) | class RedisBase(BrokerBase):
method __init__ (line 92) | def __init__(self, broker_url, *_, **kwargs):
method _q_for_pri (line 105) | def _q_for_pri(self, queue, pri):
method queues (line 111) | async def queues(self, names):
class Redis (line 123) | class Redis(RedisBase):
method __init__ (line 125) | def __init__(self, broker_url, *args, **kwargs):
method _prepare_virtual_host (line 132) | def _prepare_virtual_host(self, vhost):
method _get_redis_client_args (line 144) | def _get_redis_client_args(self):
method _get_redis_client (line 153) | def _get_redis_client(self):
class RedisSentinel (line 157) | class RedisSentinel(RedisBase):
method __init__ (line 159) | def __init__(self, broker_url, *args, **kwargs):
method _prepare_virtual_host (line 169) | def _prepare_virtual_host(self, vhost):
method _prepare_master_name (line 181) | def _prepare_master_name(self, broker_options):
method _get_redis_client (line 188) | def _get_redis_client(self, broker_options, broker_use_ssl):
class RedisSocket (line 203) | class RedisSocket(RedisBase):
method __init__ (line 205) | def __init__(self, broker_url, *args, **kwargs):
class RedisSsl (line 211) | class RedisSsl(Redis):
method __init__ (line 218) | def __init__(self, broker_url, *args, **kwargs):
method _get_redis_client_args (line 224) | def _get_redis_client_args(self):
class Broker (line 232) | class Broker:
method __new__ (line 243) | def __new__(cls, broker_url, *args, **kwargs):
method queues (line 257) | async def queues(self, names):
function main (line 261) | async def main():
FILE: flower/utils/search.py
function parse_search_terms (line 6) | def parse_search_terms(raw_search_value):
function satisfies_search_terms (line 37) | def satisfies_search_terms(task, search_terms):
function stringified_dict_contains_value (line 62) | def stringified_dict_contains_value(key, value, str_dict):
function preprocess_search_value (line 83) | def preprocess_search_value(raw_value):
function task_args_contains_search_args (line 87) | def task_args_contains_search_args(task_args, search_args):
FILE: flower/utils/tasks.py
function iter_tasks (line 8) | def iter_tasks(events, limit=None, offset=0, type=None, worker=None, sta...
function sort_tasks (line 53) | def sort_tasks(tasks, sort_by):
function get_task_by_id (line 65) | def get_task_by_id(events, task_id):
function as_dict (line 69) | def as_dict(task):
FILE: flower/utils/template.py
function format_time (line 13) | def format_time(time, tz):
function humanize (line 18) | def humanize(obj, type=None, length=None):
FILE: flower/views/__init__.py
class BaseHandler (line 17) | class BaseHandler(tornado.web.RequestHandler):
method set_default_headers (line 18) | def set_default_headers(self):
method options (line 26) | def options(self, *_, **__):
method render (line 30) | def render(self, *args, **kwargs):
method write_error (line 38) | def write_error(self, status_code, **kwargs):
method get_current_user (line 65) | def get_current_user(self):
method get_argument (line 95) | def get_argument(self, name, default=[], strip=True, type=None):
method capp (line 114) | def capp(self):
method format_task (line 118) | def format_task(self, task):
method get_active_queue_names (line 127) | def get_active_queue_names(self):
FILE: flower/views/auth.py
function authenticate (line 19) | def authenticate(pattern, email):
function validate_auth_option (line 28) | def validate_auth_option(pattern):
class GoogleAuth2LoginHandler (line 38) | class GoogleAuth2LoginHandler(BaseHandler, tornado.auth.GoogleOAuth2Mixin):
method get (line 41) | async def get(self):
method _on_auth (line 58) | async def _on_auth(self, user):
class LoginHandler (line 84) | class LoginHandler(BaseHandler):
method __new__ (line 85) | def __new__(cls, *args, **kwargs):
class GithubLoginHandler (line 89) | class GithubLoginHandler(BaseHandler, tornado.auth.OAuth2Mixin):
method get_authenticated_user (line 98) | async def get_authenticated_user(self, redirect_uri, code):
method get (line 118) | async def get(self):
method _on_auth (line 135) | async def _on_auth(self, user):
class GitLabLoginHandler (line 163) | class GitLabLoginHandler(BaseHandler, tornado.auth.OAuth2Mixin):
method get_authenticated_user (line 171) | async def get_authenticated_user(self, redirect_uri, code):
method get (line 190) | async def get(self):
method _on_auth (line 207) | async def _on_auth(self, user):
class OktaLoginHandler (line 255) | class OktaLoginHandler(BaseHandler, tornado.auth.OAuth2Mixin):
method base_url (line 260) | def base_url(self):
method _OAUTH_AUTHORIZE_URL (line 264) | def _OAUTH_AUTHORIZE_URL(self):
method _OAUTH_ACCESS_TOKEN_URL (line 268) | def _OAUTH_ACCESS_TOKEN_URL(self):
method _OAUTH_USER_INFO_URL (line 272) | def _OAUTH_USER_INFO_URL(self):
method get_access_token (line 275) | async def get_access_token(self, redirect_uri, code):
method get (line 295) | async def get(self):
method _on_auth (line 321) | async def _on_auth(self, access_token_response):
FILE: flower/views/broker.py
class BrokerView (line 11) | class BrokerView(BaseHandler):
method get (line 13) | async def get(self):
FILE: flower/views/error.py
class NotFoundErrorHandler (line 6) | class NotFoundErrorHandler(BaseHandler):
method get (line 7) | def get(self):
method post (line 10) | def post(self):
FILE: flower/views/monitor.py
class Metrics (line 6) | class Metrics(BaseHandler):
method get (line 7) | async def get(self):
class Healthcheck (line 12) | class Healthcheck(BaseHandler):
method get (line 13) | async def get(self):
FILE: flower/views/tasks.py
class TaskView (line 13) | class TaskView(BaseHandler):
method get (line 15) | def get(self, task_id):
class Comparable (line 25) | class Comparable:
method __init__ (line 31) | def __init__(self, value):
method __eq__ (line 34) | def __eq__(self, other):
method __lt__ (line 37) | def __lt__(self, other):
class TasksDataTable (line 44) | class TasksDataTable(BaseHandler):
method get (line 46) | def get(self):
method maybe_normalize_for_sort (line 82) | def maybe_normalize_for_sort(cls, tasks, sort_by):
method post (line 94) | def post(self):
method format_task (line 97) | def format_task(self, task):
class TasksView (line 109) | class TasksView(BaseHandler):
method get (line 111) | def get(self):
FILE: flower/views/workers.py
class WorkerView (line 12) | class WorkerView(BaseHandler):
method get (line 14) | async def get(self, name):
class WorkersView (line 30) | class WorkersView(BaseHandler):
method get (line 32) | async def get(self):
method _as_dict (line 78) | def _as_dict(cls, worker):
method _info (line 84) | def _info(cls, worker):
FILE: setup.py
function get_package_version (line 11) | def get_package_version():
function get_requirements (line 22) | def get_requirements(filename):
FILE: tests/load.py
function main (line 14) | def main():
FILE: tests/unit/__init__.py
class AsyncHTTPTestCase (line 15) | class AsyncHTTPTestCase(tornado.testing.AsyncHTTPTestCase):
method _get_celery_app (line 17) | def _get_celery_app(self):
method get_app (line 20) | def get_app(self, capp=None):
method get (line 28) | def get(self, url, **kwargs):
method post (line 31) | def post(self, url, **kwargs):
method mock_option (line 36) | def mock_option(self, name, value):
FILE: tests/unit/__main__.py
function all (line 7) | def all():
FILE: tests/unit/api/__init__.py
class BaseApiTestCase (line 6) | class BaseApiTestCase(AsyncHTTPTestCase):
method setUp (line 7) | def setUp(self):
method tearDown (line 11) | def tearDown(self):
FILE: tests/unit/api/test_auth.py
class BasicAuthTests (line 4) | class BasicAuthTests(AsyncHTTPTestCase):
method test_auth (line 5) | def test_auth(self):
FILE: tests/unit/api/test_control.py
class UnknownWorkerControlTests (line 11) | class UnknownWorkerControlTests(BaseApiTestCase):
method test_unknown_worker (line 12) | def test_unknown_worker(self):
class WorkerControlTests (line 17) | class WorkerControlTests(BaseApiTestCase):
method setUp (line 18) | def setUp(self):
method tearDown (line 23) | def tearDown(self):
method test_shutdown (line 27) | def test_shutdown(self):
method test_pool_restart (line 35) | def test_pool_restart(self):
method test_pool_grow (line 47) | def test_pool_grow(self):
method test_pool_shrink (line 55) | def test_pool_shrink(self):
method test_pool_autoscale (line 63) | def test_pool_autoscale(self):
method test_add_consumer (line 74) | def test_add_consumer(self):
method test_cancel_consumer (line 86) | def test_cancel_consumer(self):
method test_task_timeout (line 98) | def test_task_timeout(self):
method test_task_ratelimit (line 112) | def test_task_ratelimit(self):
method test_task_ratelimit_non_integer (line 123) | def test_task_ratelimit_non_integer(self):
method test_param_escape (line 134) | def test_param_escape(self):
class TaskControlTests (line 147) | class TaskControlTests(BaseApiTestCase):
method test_revoke (line 148) | def test_revoke(self):
method test_terminate (line 157) | def test_terminate(self):
method test_terminate_signal (line 166) | def test_terminate_signal(self):
class ControlAuthTests (line 177) | class ControlAuthTests(WorkerControlTests):
method test_auth (line 178) | def test_auth(self):
method test_auth_without_env_var (line 186) | def test_auth_without_env_var(self):
FILE: tests/unit/api/test_tasks.py
class ApplyTests (line 17) | class ApplyTests(BaseApiTestCase):
method test_apply (line 18) | def test_apply(self):
class AsyncApplyTests (line 39) | class AsyncApplyTests(BaseApiTestCase):
method test_async_apply (line 40) | def test_async_apply(self):
method test_async_apply_eta (line 48) | def test_async_apply_eta(self):
method test_async_apply_countdown (line 59) | def test_async_apply_countdown(self):
method test_async_apply_expires (line 69) | def test_async_apply_expires(self):
method test_async_apply_expires_datetime (line 79) | def test_async_apply_expires_datetime(self):
class MockTasks (line 91) | class MockTasks:
method get_task_by_id (line 94) | def get_task_by_id(events, task_id):
class TaskTests (line 99) | class TaskTests(BaseApiTestCase):
method setUp (line 100) | def setUp(self):
method get_app (line 104) | def get_app(self, capp=None):
method test_task_info (line 108) | def test_task_info(self):
method test_tasks_pagination (line 111) | def test_tasks_pagination(self):
FILE: tests/unit/api/test_workers.py
class ListWorkersTest (line 23) | class ListWorkersTest(BaseApiTestCase):
method test_refresh_cache (line 25) | def test_refresh_cache(self, m_inspect):
method test_refresh_cache_with_empty_response (line 49) | def test_refresh_cache_with_empty_response(self, m_inspect):
FILE: tests/unit/test_command.py
class TestFlowerCommand (line 17) | class TestFlowerCommand(AsyncHTTPTestCase):
method test_task_runtime_metric_buckets_read_from_cmd_line (line 18) | def test_task_runtime_metric_buckets_read_from_cmd_line(self):
method test_task_runtime_metric_buckets_no_cmd_line_arg (line 22) | def test_task_runtime_metric_buckets_no_cmd_line_arg(self):
method test_task_runtime_metric_buckets_read_from_env (line 26) | def test_task_runtime_metric_buckets_read_from_env(self):
method test_task_runtime_metric_buckets_no_env_value_provided (line 31) | def test_task_runtime_metric_buckets_no_env_value_provided(self):
method test_port (line 35) | def test_port(self):
method test_address (line 40) | def test_address(self):
method test_auto_refresh (line 45) | def test_auto_refresh(self):
method test_autodiscovery (line 70) | def test_autodiscovery(self):
class TestPrintBanner (line 86) | class TestPrintBanner(AsyncHTTPTestCase):
method test_print_banner (line 87) | def test_print_banner(self):
method test_print_banner_with_ssl (line 95) | def test_print_banner_with_ssl(self):
method test_print_banner_unix_socket (line 103) | def test_print_banner_unix_socket(self):
class TestWarnAboutCeleryArgsUsedInFlowerCommand (line 111) | class TestWarnAboutCeleryArgsUsedInFlowerCommand(AsyncHTTPTestCase):
method test_does_not_log_warning (line 113) | def test_does_not_log_warning(self, mock_warning):
method test_logs_warning (line 129) | def test_logs_warning(self, mock_warning):
class TestConfOption (line 149) | class TestConfOption(AsyncHTTPTestCase):
method test_error_conf (line 150) | def test_error_conf(self):
method test_default_option (line 157) | def test_default_option(self):
method test_empty_conf (line 161) | def test_empty_conf(self):
method test_conf_abs (line 166) | def test_conf_abs(self):
method test_conf_relative (line 175) | def test_conf_relative(self):
method test_all_options_documented (line 184) | def test_all_options_documented(self):
FILE: tests/unit/utils/__init__.py
class HtmlTableParser (line 8) | class HtmlTableParser(HTMLParser):
method __init__ (line 9) | def __init__(self, *args, **kwargs):
method handle_starttag (line 14) | def handle_starttag(self, tag, attrs):
method handle_endtag (line 23) | def handle_endtag(self, tag):
method handle_data (line 29) | def handle_data(self, data):
method parse (line 33) | def parse(self, source):
method query (line 36) | def query(self, pattern):
method rows (line 40) | def rows(self):
method get_row (line 43) | def get_row(self, row_id):
function task_succeeded_events (line 51) | def task_succeeded_events(worker, id=None, name=None, runtime=0.1234, re...
function task_failed_events (line 62) | def task_failed_events(worker, id=None, name=None):
FILE: tests/unit/utils/test_broker.py
class TestRabbitMQ (line 12) | class TestRabbitMQ(unittest.TestCase):
method test_init (line 13) | def test_init(self):
method test_url (line 19) | def test_url(self):
method test_url_vhost_slash (line 27) | def test_url_vhost_slash(self):
method test_url_defaults_rabbitmq (line 35) | def test_url_defaults_rabbitmq(self):
method test_url_defaults_redis (line 44) | def test_url_defaults_redis(self):
method test_invalid_http_api (line 53) | def test_invalid_http_api(self):
class TestRedis (line 59) | class TestRedis(unittest.TestCase):
method test_init (line 60) | def test_init(self):
method test_priority_steps (line 65) | def test_priority_steps(self):
method test_custom_sep (line 73) | def test_custom_sep(self):
method test_url (line 81) | def test_url(self):
method test_url_defaults (line 87) | def test_url_defaults(self):
method test_url_with_password (line 95) | def test_url_with_password(self):
method test_url_with_user_and_password (line 102) | def test_url_with_user_and_password(self):
method test_ipv6 (line 110) | def test_ipv6(self):
class TestRedisSentinel (line 117) | class TestRedisSentinel(unittest.TestCase):
method test_init (line 118) | def test_init(self):
method test_priority_steps (line 124) | def test_priority_steps(self):
method test_url (line 132) | def test_url(self):
method test_url_defaults (line 139) | def test_url_defaults(self):
method test_url_with_password (line 148) | def test_url_with_password(self):
class TestRedisSsl (line 157) | class TestRedisSsl(unittest.TestCase):
method test_init_with_broker_use_ssl (line 165) | def test_init_with_broker_use_ssl(self):
method test_init_without_broker_use_ssl (line 170) | def test_init_without_broker_use_ssl(self):
method test_redis_client_args (line 174) | def test_redis_client_args(self):
class TestRedisSocket (line 187) | class TestRedisSocket(unittest.TestCase):
method test_init (line 188) | def test_init(self):
method test_url (line 193) | def test_url(self):
FILE: tests/unit/utils/test_search.py
class TestSearchParser (line 8) | class TestSearchParser(unittest.TestCase):
method test_any_value (line 9) | def test_any_value(self):
method test_result_value (line 15) | def test_result_value(self):
method test_kwargs (line 21) | def test_kwargs(self):
method test_partial_kwargs (line 31) | def test_partial_kwargs(self):
method test_args (line 41) | def test_args(self):
method test_strip_spaces (line 51) | def test_strip_spaces(self):
method test_quotes (line 61) | def test_quotes(self):
class TestStringfiedDictChecker (line 72) | class TestStringfiedDictChecker(unittest.TestCase):
method test_stringifies_args (line 73) | def test_stringifies_args(self):
method test_works_for_no_kwargs (line 79) | def test_works_for_no_kwargs(self):
method test_works_for_nonexisting_kwargs (line 85) | def test_works_for_nonexisting_kwargs(self):
method test_works_for_kwargs_in_different_parts_of_string (line 91) | def test_works_for_kwargs_in_different_parts_of_string(self):
class TestTaskFiltering (line 99) | class TestTaskFiltering(unittest.TestCase):
method _create_task (line 100) | def _create_task(self, result=None, args=None, kwargs='{}'):
method setUp (line 105) | def setUp(self):
method test_kwarg_search_works (line 112) | def test_kwarg_search_works(self):
method test_args_search_works (line 130) | def test_args_search_works(self):
method test_result_search_handles_none (line 144) | def test_result_search_handles_none(self):
FILE: tests/unit/utils/test_template.py
class TestHumanize (line 9) | class TestHumanize(unittest.TestCase):
method test_none (line 10) | def test_none(self):
method test_bool (line 13) | def test_bool(self):
method test_numbers (line 17) | def test_numbers(self):
method test_keywords (line 22) | def test_keywords(self):
method test_uuid (line 41) | def test_uuid(self):
method test_sequences (line 45) | def test_sequences(self):
method test_time (line 51) | def test_time(self):
method test_natural_time (line 56) | def test_natural_time(self):
method test_strings (line 62) | def test_strings(self):
method test_truncate (line 68) | def test_truncate(self):
FILE: tests/unit/utils/test_utils.py
class BugreportTests (line 11) | class BugreportTests(unittest.TestCase):
method test_default (line 12) | def test_default(self):
method test_with_app (line 19) | def test_with_app(self):
method test_when_unable_to_generate_report (line 27) | def test_when_unable_to_generate_report(self):
class TestGenCookieSecret (line 36) | class TestGenCookieSecret(unittest.TestCase):
method test_cookie_secret_generation (line 37) | def test_cookie_secret_generation(self):
class TestAbsPath (line 43) | class TestAbsPath(unittest.TestCase):
method test_absolute_path (line 44) | def test_absolute_path(self):
method test_relative_path (line 48) | def test_relative_path(self):
method test_relative_path_with_pwd (line 59) | def test_relative_path_with_pwd(self):
method test_home_path (line 63) | def test_home_path(self):
class TestStrtobool (line 67) | class TestStrtobool(unittest.TestCase):
method test_true_values (line 68) | def test_true_values(self):
method test_false_values (line 76) | def test_false_values(self):
method test_invalid_value (line 84) | def test_invalid_value(self):
FILE: tests/unit/views/test_auth.py
class BasicAuthTests (line 5) | class BasicAuthTests(AsyncHTTPTestCase):
method test_with_single_creds (line 6) | def test_with_single_creds(self):
method test_with_multiple_creds (line 15) | def test_with_multiple_creds(self):
class AuthTests (line 27) | class AuthTests(AsyncHTTPTestCase):
method test_validate_auth_option (line 28) | def test_validate_auth_option(self):
method test_authenticate_single_email (line 39) | def test_authenticate_single_email(self):
method test_authenticate_email_list (line 47) | def test_authenticate_email_list(self):
method test_authenticate_wildcard_email (line 56) | def test_authenticate_wildcard_email(self):
FILE: tests/unit/views/test_broker.py
class TestBrokerView (line 4) | class TestBrokerView(AsyncHTTPTestCase):
method test_empty_page (line 5) | def test_empty_page(self):
FILE: tests/unit/views/test_error.py
class ErrorTests (line 4) | class ErrorTests(AsyncHTTPTestCase):
method test_404 (line 5) | def test_404(self):
FILE: tests/unit/views/test_monitor.py
class PrometheusTests (line 13) | class PrometheusTests(AsyncHTTPTestCase):
method setUp (line 14) | def setUp(self):
method get_app (line 18) | def get_app(self, capp=None):
method test_metrics (line 21) | def test_metrics(self):
method test_task_prefetch_time_metric (line 47) | def test_task_prefetch_time_metric(self):
method test_task_prefetch_time_metric_successful_task_resets_metric_to_zero (line 72) | def test_task_prefetch_time_metric_successful_task_resets_metric_to_ze...
method test_task_prefetch_time_metric_failed_task_resets_metric_to_zero (line 97) | def test_task_prefetch_time_metric_failed_task_resets_metric_to_zero(s...
method test_task_prefetch_time_metric_does_not_compute_prefetch_time_if_task_has_eta (line 122) | def test_task_prefetch_time_metric_does_not_compute_prefetch_time_if_t...
method test_worker_online_metric_worker_is_offline (line 143) | def test_worker_online_metric_worker_is_offline(self):
method test_worker_prefetched_tasks_metric (line 158) | def test_worker_prefetched_tasks_metric(self):
class HealthcheckTests (line 201) | class HealthcheckTests(AsyncHTTPTestCase):
method setUp (line 202) | def setUp(self):
method get_app (line 206) | def get_app(self, capp=None):
method test_healthcheck_route (line 209) | def test_healthcheck_route(self):
FILE: tests/unit/views/test_tasks.py
class TaskTest (line 11) | class TaskTest(AsyncHTTPTestCase):
method test_unknown_task (line 12) | def test_unknown_task(self):
class TasksTest (line 18) | class TasksTest(AsyncHTTPTestCase):
method setUp (line 19) | def setUp(self):
method get_app (line 23) | def get_app(self, capp=None):
method test_no_task (line 26) | def test_no_task(self):
method test_succeeded_task (line 32) | def test_succeeded_task(self):
method test_failed_task (line 64) | def test_failed_task(self):
method test_sort_runtime (line 96) | def test_sort_runtime(self):
method test_sort_incomparable (line 148) | def test_sort_incomparable(self):
method test_pagination (line 189) | def test_pagination(self):
FILE: tests/unit/views/test_url_handlers.py
class UrlsTests (line 10) | class UrlsTests(AsyncHTTPTestCase):
method test_workers_url (line 11) | def test_workers_url(self):
method test_root_url (line 15) | def test_root_url(self):
method test_tasks_api_url (line 19) | def test_tasks_api_url(self):
class URLPrefixTests (line 25) | class URLPrefixTests(AsyncHTTPTestCase):
method setUp (line 26) | def setUp(self):
method test_tuple_handler_rewrite (line 31) | def test_tuple_handler_rewrite(self):
method test_root_url (line 35) | def test_root_url(self):
method test_tasks_api_url (line 39) | def test_tasks_api_url(self):
method test_base_url_no_longer_working (line 44) | def test_base_url_no_longer_working(self):
class RewriteHandlerTests (line 49) | class RewriteHandlerTests(AsyncHTTPTestCase):
method target (line 50) | def target(self):
method test_url_rewrite_using_URLSpec (line 53) | def test_url_rewrite_using_URLSpec(self):
method test_url_rewrite_using_tuple (line 61) | def test_url_rewrite_using_tuple(self):
FILE: tests/unit/views/test_workers.py
class WorkersTests (line 15) | class WorkersTests(AsyncHTTPTestCase):
method setUp (line 16) | def setUp(self):
method get_app (line 20) | def get_app(self, capp=None):
method test_default_page (line 23) | def test_default_page(self):
method test_no_workers (line 28) | def test_no_workers(self):
method test_unknown_worker (line 35) | def test_unknown_worker(self):
method test_single_workers_offline (line 41) | def test_single_workers_offline(self):
method test_purge_offline_workers (line 61) | def test_purge_offline_workers(self):
method test_single_workers_online (line 80) | def test_single_workers_online(self):
method test_task_received (line 99) | def test_task_received(self):
method test_task_started (line 128) | def test_task_started(self):
method test_task_succeeded (line 158) | def test_task_succeeded(self):
method test_task_failed (line 190) | def test_task_failed(self):
method test_task_retried (line 222) | def test_task_retried(self):
method test_tasks (line 256) | def test_tasks(self):
method test_workers_view_json (line 291) | def test_workers_view_json(self):
method test_workers_view_refresh (line 303) | def test_workers_view_refresh(self):
method test_workers_page (line 315) | def test_workers_page(self):
Condensed preview — 114 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (408K chars).
[
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 577,
"preview": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: bug\nassignees: ''\n\n---\n\n**Describe the "
},
{
"path": ".github/ISSUE_TEMPLATE/feature_request.md",
"chars": 374,
"preview": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: enhancement\nassignees: ''\n\n---\n\n**Is"
},
{
"path": ".github/dependabot.yml",
"chars": 579,
"preview": "# Keep GitHub Actions up to date with GitHub's Dependabot...\n# https://docs.github.com/en/code-security/dependabot/worki"
},
{
"path": ".github/workflows/build.yml",
"chars": 1074,
"preview": "name: Build\n\non: [push, pull_request]\n\njobs:\n build:\n runs-on: ubuntu-latest\n\n strategy:\n fail-fast: false\n "
},
{
"path": ".github/workflows/docker.yml",
"chars": 1064,
"preview": "name: docker\n\non:\n push:\n branches:\n - master\n tags:\n - '*'\n pull_request:\n branches:\n - maste"
},
{
"path": ".gitignore",
"chars": 300,
"preview": ".DS_Store\n*.pyc\n*$py.class\n*~\n.*.sw[po]\ndist/\n*.egg-info\n*.egg\n*.egg/\n*.eggs\ndoc/__build/*\nbuild/\n.build/\npip-log.txt\n.d"
},
{
"path": ".pylintrc",
"chars": 488,
"preview": "[MASTER]\ndisable=\n C0114, # missing-module-docstring\n C0115, # missing-class-docstring\n C0116, # missing-functi"
},
{
"path": ".readthedocs.yaml",
"chars": 223,
"preview": "# .readthedocs.yaml\n# Read the Docs configuration file\n\nversion: 2\n\nbuild:\n os: ubuntu-22.04\n tools:\n python: \"3.11"
},
{
"path": "CONTRIBUTORS",
"chars": 1570,
"preview": "======================================\n Contributors (in chronological order)\n======================================\n\nMh"
},
{
"path": "Dockerfile",
"chars": 1077,
"preview": "FROM python:alpine\n\n# Get latest root certificates and update openssl to fix vulnerabilities\nRUN apk add --no-cache ca-c"
},
{
"path": "LICENSE",
"chars": 1545,
"preview": "Copyright (c) 2012, Mher Movsisyan and individual contributors.\nAll rights reserved. \n\nRedistribution and use in source "
},
{
"path": "MANIFEST.in",
"chars": 246,
"preview": "include AUTHORS\ninclude CHANGES\ninclude LICENSE\ninclude MANIFEST.in\ninclude README.rst\nrecursive-include docs *\nrecursiv"
},
{
"path": "README.rst",
"chars": 3253,
"preview": "Flower\n======\n\n.. image:: https://img.shields.io/pypi/dm/flower.svg\n :target: https://pypistats.org/packages/flower\n "
},
{
"path": "docker-compose.yml",
"chars": 914,
"preview": "version: '3'\nservices:\n redis:\n image: redis:alpine\n ports:\n - 6379:6379\n prometheus:\n image: prom/prome"
},
{
"path": "docs/Makefile",
"chars": 6762,
"preview": "# Makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD "
},
{
"path": "docs/_static/.keep",
"chars": 0,
"preview": ""
},
{
"path": "docs/_templates/localtoc.html",
"chars": 146,
"preview": "{%- if display_toc %}\n <h3><a href=\"{{ pathto(master_doc) }}\">{{ _('Table Of Contents') }}</a></h3>\n {{ toctree(collap"
},
{
"path": "docs/_templates/page.html",
"chars": 102,
"preview": "{% extends \"layout.html\" %}\n{% block body %}\n<div class=\"deck\">\n\n</div>\n {{ body }}\n{% endblock %}\n"
},
{
"path": "docs/_templates/sidebarintro.html",
"chars": 357,
"preview": "<p>\n <iframe src=\"http://ghbtns.com/github-btn.html?user=mher&repo=flower&type=watch&count=true&size=large\"\n allowtr"
},
{
"path": "docs/_templates/sidebarlogo.html",
"chars": 211,
"preview": "<p>\n <iframe src=\"http://ghbtns.com/github-btn.html?user=mher&repo=flower&type=watch&count=true&size=large\"\n allowtr"
},
{
"path": "docs/_theme/celery/static/celery.css_t",
"chars": 6684,
"preview": "/*\n * celery.css_t\n * ~~~~~~~~~~~~\n *\n * :copyright: Copyright 2010 by Armin Ronacher.\n * :license: BSD, see LICENSE for"
},
{
"path": "docs/_theme/celery/theme.conf",
"chars": 59,
"preview": "[theme]\ninherit = basic\nstylesheet = celery.css\n\n[options]\n"
},
{
"path": "docs/api.ipynb",
"chars": 16993,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# flower REST API\\n\",\n \"\\n\",\n "
},
{
"path": "docs/api.rst",
"chars": 288,
"preview": "API Reference\n=============\n\nFor security reasons, the Flower API is disabled by default when authentication is not enab"
},
{
"path": "docs/auth.rst",
"chars": 5703,
"preview": ".. _authentication:\n\nAuthentication\n==============\n\nFlower supports a variety of authentication methods, including Basic"
},
{
"path": "docs/conf.py",
"chars": 8851,
"preview": "# -*- coding: utf-8 -*-\n#\n# flower documentation build configuration file, created by\n# sphinx-quickstart on Fri Apr 11 "
},
{
"path": "docs/config.rst",
"chars": 15214,
"preview": ":tocdepth: 2\n\nConfiguration\n=============\n\nFlower is highly customizable. You can pass configuration options through the"
},
{
"path": "docs/features.rst",
"chars": 724,
"preview": "Features\n--------\n\n- Real-time monitoring using Celery Events\n - View task progress and history\n - View task detai"
},
{
"path": "docs/index.rst",
"chars": 675,
"preview": "======\nFlower\n======\n\nFlower is an open-source web application for monitoring and managing `Celery`_ clusters.\nIt provid"
},
{
"path": "docs/install.rst",
"chars": 1016,
"preview": "Getting started\n===============\n\nInstallation\n------------\n\nInstalling `flower` with `pip <http://www.pip-installer.org/"
},
{
"path": "docs/man.rst",
"chars": 5363,
"preview": "========\n flower\n========\n\nSYNOPSIS\n========\n\n``flower`` [*OPTIONS*]\n\nDESCRIPTION\n===========\n\nFlower is a web based too"
},
{
"path": "docs/prometheus-integration.rst",
"chars": 11209,
"preview": "Prometheus Integration\n======================\n\nFlower exports several celery worker and task metrics in Prometheus' form"
},
{
"path": "docs/reverse-proxy.rst",
"chars": 1494,
"preview": ".. _reverse-proxy:\n\nRunning behind reverse proxy\n============================\n\nTo run `Flower` behind a reverse proxy, r"
},
{
"path": "docs/tasks.py",
"chars": 336,
"preview": "from celery import Celery\nfrom time import sleep\n\ncelery = Celery()\ncelery.config_from_object({\n 'BROKER_URL': 'amqp:"
},
{
"path": "docs/tasks_filter.rst",
"chars": 669,
"preview": "Tasks filtering\n===============\n\nBy now, tasks can be filtered by worker, type, state, received and started datetime.\nAl"
},
{
"path": "examples/celery-monitoring-grafana-dashboard.json",
"chars": 17301,
"preview": "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type"
},
{
"path": "examples/celeryconfig.py",
"chars": 120,
"preview": "broker_url = 'redis://localhost:6379/0'\ncelery_result_backend = 'redis://localhost:6379/0'\ntask_send_sent_event = False\n"
},
{
"path": "examples/nginx.conf",
"chars": 130,
"preview": "server {\n listen 80;\n\n # with url_prefix=`flower`\n location /flower/ {\n proxy_pass http://localhost:5555"
},
{
"path": "examples/prometheus-alerts.yaml",
"chars": 1100,
"preview": "- alert: CeleryWorkerOffline\n expr: flower_worker_online == 0\n for: 2m\n labels:\n severity: critical\n context: c"
},
{
"path": "examples/pycharm-configurations/Grafana.run.xml",
"chars": 634,
"preview": "<component name=\"ProjectRunConfigurationManager\">\n <configuration default=\"false\" name=\"Grafana\" type=\"docker-deploy\" f"
},
{
"path": "examples/pycharm-configurations/Prometheus.run.xml",
"chars": 664,
"preview": "<component name=\"ProjectRunConfigurationManager\">\n <configuration default=\"false\" name=\"Prometheus\" type=\"docker-deploy"
},
{
"path": "examples/pycharm-configurations/Redis.run.xml",
"chars": 569,
"preview": "<component name=\"ProjectRunConfigurationManager\">\n <configuration default=\"false\" name=\"Redis\" type=\"docker-deploy\" fac"
},
{
"path": "examples/tasks.py",
"chars": 655,
"preview": "import os\nimport time\nfrom datetime import datetime\n\nfrom celery import Celery\n\n\napp = Celery(\"tasks\",\n brok"
},
{
"path": "flower/__init__.py",
"chars": 71,
"preview": "VERSION = (2, 0, 0)\n__version__ = '.'.join(map(str, VERSION)) + '-dev'\n"
},
{
"path": "flower/__main__.py",
"chars": 204,
"preview": "import sys\nfrom celery.bin.celery import main as _main, celery\nfrom flower.command import flower\n\n\ndef main():\n celer"
},
{
"path": "flower/api/__init__.py",
"chars": 780,
"preview": "import os\n\nimport tornado.web\n\nfrom ..utils import strtobool\nfrom ..views import BaseHandler\n\n\nclass BaseApiHandler(Base"
},
{
"path": "flower/api/control.py",
"chars": 15691,
"preview": "import logging\n\nfrom tornado import web\n\nfrom . import BaseApiHandler\n\nlogger = logging.getLogger(__name__)\n\n\nclass Cont"
},
{
"path": "flower/api/tasks.py",
"chars": 17367,
"preview": "import json\nimport logging\nfrom collections import OrderedDict\nfrom datetime import datetime\n\nfrom celery import states\n"
},
{
"path": "flower/api/workers.py",
"chars": 5541,
"preview": "import asyncio\nimport logging\n\nfrom tornado import web\n\nfrom .control import ControlHandler\n\nlogger = logging.getLogger("
},
{
"path": "flower/app.py",
"chars": 3500,
"preview": "import sys\nimport logging\n\nfrom concurrent.futures import ThreadPoolExecutor\n\nimport celery\nimport tornado.web\n\nfrom tor"
},
{
"path": "flower/command.py",
"chars": 5858,
"preview": "import os\nimport sys\nimport atexit\nimport signal\nimport logging\n\nfrom pprint import pformat\n\nfrom logging import NullHan"
},
{
"path": "flower/events.py",
"chars": 7502,
"preview": "import collections\nimport logging\nimport shelve\nimport threading\nimport time\nfrom collections import Counter\nfrom functo"
},
{
"path": "flower/inspector.py",
"chars": 1689,
"preview": "import collections\nimport logging\nimport time\nfrom functools import partial\n\nlogger = logging.getLogger(__name__)\n\n\nclas"
},
{
"path": "flower/options.py",
"chars": 3216,
"preview": "import types\nfrom secrets import token_urlsafe\n\nfrom prometheus_client import Histogram\nfrom tornado.options import defi"
},
{
"path": "flower/static/css/flower.css",
"chars": 582,
"preview": ".bg-green {\n background-color: #f0ffeb;\n}\n\n.dataTables_wrapper {\n border: 1px solid #c7ecb8;\n}\n\n.dataTables_filter inp"
},
{
"path": "flower/static/js/flower.js",
"chars": 23116,
"preview": "/*jslint browser: true */\n/*global $, WebSocket, jQuery */\n\nvar flower = (function () {\n \"use strict\";\n\n var alert"
},
{
"path": "flower/static/swagger.json",
"chars": 12574,
"preview": "{\n \"tags\": [\n \n ],\n \"paths\": {\n \"\\/api\\/tasks\": {\n \"get\": {\n \"responses\": {\n \"200\": {\n "
},
{
"path": "flower/templates/404.html",
"chars": 262,
"preview": "{% extends \"base.html\" %}\n\n{% block container %}\n <div class=\"col-12\">\n <p>\n {% if message %}\n "
},
{
"path": "flower/templates/base.html",
"chars": 1257,
"preview": "{% import pprint %}\n<!doctype html>\n<html lang=\"en\">\n <head>\n <meta charset=\"utf-8\">\n <meta name=\"viewport\" conte"
},
{
"path": "flower/templates/broker.html",
"chars": 1073,
"preview": "{% extends \"base.html\" %}\n\n{% block navbar %}\n{% module Template(\"navbar.html\", active_tab=\"broker\") %}\n{% end %}\n\n{% bl"
},
{
"path": "flower/templates/error.html",
"chars": 487,
"preview": "{% extends \"base.html\" %}\n\n{% block container %}\n {% if debug %}\n <div class=\"col-12\">\n <p>It looks like yo"
},
{
"path": "flower/templates/navbar.html",
"chars": 2519,
"preview": "<nav class=\"navbar navbar-expand-lg navbar-light bg-green mx-2\">\n <a class=\"navbar-brand\" href=\"{{ reverse_url('main') "
},
{
"path": "flower/templates/task.html",
"chars": 3621,
"preview": "{% extends \"base.html\" %}\n\n{% block navbar %}\n {% module Template(\"navbar.html\", active_tab=\"tasks\") %}\n{% end %}\n\n{% b"
},
{
"path": "flower/templates/tasks.html",
"chars": 2095,
"preview": "{% extends \"base.html\" %}\n\n{% block navbar %}\n {% module Template(\"navbar.html\", active_tab=\"tasks\") %}\n{% end %}\n\n\n{% "
},
{
"path": "flower/templates/worker.html",
"chars": 17329,
"preview": "{% extends \"base.html\" %}\n\n{% block navbar %}\n{% module Template(\"navbar.html\", active_tab=\"workers\") %}\n{% end %}\n\n{% b"
},
{
"path": "flower/templates/workers.html",
"chars": 1656,
"preview": "{% extends \"base.html\" %}\n\n{% block navbar %}\n{% module Template(\"navbar.html\", active_tab=\"workers\")%}\n{% end %}\n\n{% bl"
},
{
"path": "flower/urls.py",
"chars": 2436,
"preview": "import os\n\nfrom tornado.web import StaticFileHandler, url\n\nfrom .api import control, tasks, workers\nfrom .utils import g"
},
{
"path": "flower/utils/__init__.py",
"chars": 1523,
"preview": "import base64\nimport os.path\nimport uuid\n\nfrom .. import __version__\n\n\ndef gen_cookie_secret():\n return base64.b64enc"
},
{
"path": "flower/utils/broker.py",
"chars": 9491,
"preview": "import asyncio\nimport json\nimport logging\nimport numbers\nimport socket\nimport sys\nfrom urllib.parse import quote, unquot"
},
{
"path": "flower/utils/search.py",
"chars": 3627,
"preview": "import re\n\nfrom kombu.utils.encoding import safe_str\n\n\ndef parse_search_terms(raw_search_value):\n search_regexp = r'("
},
{
"path": "flower/utils/tasks.py",
"chars": 2187,
"preview": "import datetime\nimport time\n\nfrom .search import parse_search_terms, satisfies_search_terms\n\n\n# pylint: disable=too-many"
},
{
"path": "flower/utils/template.py",
"chars": 1682,
"preview": "import re\nfrom datetime import datetime, timedelta\n\nfrom celery import current_app\nfrom humanize import naturaltime\nfrom"
},
{
"path": "flower/views/__init__.py",
"chars": 5133,
"preview": "import re\nimport inspect\nimport traceback\nimport copy\nimport logging\nimport hmac\n\nfrom base64 import b64decode\n\nimport t"
},
{
"path": "flower/views/auth.py",
"chars": 13517,
"preview": "import json\nimport os\nimport re\nimport uuid\nfrom urllib.parse import urlencode\n\nimport tornado.auth\nimport tornado.gen\ni"
},
{
"path": "flower/views/broker.py",
"chars": 1167,
"preview": "import logging\n\nfrom tornado import web\n\nfrom ..utils.broker import Broker\nfrom ..views import BaseHandler\n\nlogger = log"
},
{
"path": "flower/views/error.py",
"chars": 217,
"preview": "import tornado.web\n\nfrom ..views import BaseHandler\n\n\nclass NotFoundErrorHandler(BaseHandler):\n def get(self):\n "
},
{
"path": "flower/views/monitor.py",
"chars": 307,
"preview": "import prometheus_client\n\nfrom ..views import BaseHandler\n\n\nclass Metrics(BaseHandler):\n async def get(self):\n "
},
{
"path": "flower/views/tasks.py",
"chars": 3710,
"preview": "import copy\nimport logging\nfrom functools import total_ordering\n\nfrom tornado import web\n\nfrom ..utils.tasks import as_d"
},
{
"path": "flower/views/workers.py",
"chars": 3072,
"preview": "import logging\nimport time\n\nfrom tornado import web\n\nfrom ..options import options\nfrom ..views import BaseHandler\n\nlogg"
},
{
"path": "prometheus.yml",
"chars": 152,
"preview": "global:\n scrape_interval: 15s\n evaluation_interval: 15s\n\nscrape_configs:\n - job_name: flower\n static_configs:\n"
},
{
"path": "requirements/default.txt",
"chars": 75,
"preview": "celery>=5.0.5\ntornado>=5.0.0,<7.0.0\nprometheus_client>=0.8.0\nhumanize\npytz\n"
},
{
"path": "requirements/dev.txt",
"chars": 47,
"preview": "-r default.txt\n-r test.txt\nredis>=4.3.6\npylint\n"
},
{
"path": "requirements/docs.txt",
"chars": 89,
"preview": "-r default.txt\nSphinx\nsphinxcontrib-fulltoc\nsphinxcontrib-httpdomain\nsphinxcontrib-redoc\n"
},
{
"path": "requirements/test.txt",
"chars": 1,
"preview": "\n"
},
{
"path": "scss/build.sh",
"chars": 839,
"preview": "set -euxo pipefail\n\nBOOTSTRAP_VERSION=\"${1:-5.2.3}\"\nBOOTSTRAP_ZIP=\"v$BOOTSTRAP_VERSION.zip\" \nBOOTSTRAP_DIR=\"bootstrap-$B"
},
{
"path": "scss/flower.scss",
"chars": 150,
"preview": "// flower.scss\n\n$primary: #348613;\n$link-color: #348613;\n$border-color: #c7ecb8;\n$table-striped-bg: #f0ffeb;\n$bg-light: "
},
{
"path": "setup.cfg",
"chars": 20,
"preview": "[wheel]\nuniversal=1\n"
},
{
"path": "setup.py",
"chars": 2199,
"preview": "#!/usr/bin/env python\nimport os\nimport re\n\nfrom setuptools import setup, find_packages\n\n\nversion = re.compile(r'VERSION\\"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/call-tasks.sh",
"chars": 312,
"preview": "#!/bin/bash\nset -e\n\nexport ROOT_DIR=$(git rev-parse --show-toplevel)\nexport PYTHONPATH=\"$PYTHONPATH:$ROOT_DIR/examples\"\n"
},
{
"path": "tests/load.py",
"chars": 523,
"preview": "import time\nimport random\n\nimport os\nimport sys\ntests_dir = os.path.dirname(__file__)\nexamples_dir = os.path.join(tests_"
},
{
"path": "tests/run-unit-tests.sh",
"chars": 41,
"preview": "#!/bin/bash\nset -e\n\npython -m tests.unit\n"
},
{
"path": "tests/unit/__init__.py",
"chars": 1149,
"preview": "from unittest.mock import patch\nfrom urllib.parse import urlencode\n\nimport celery\nimport tornado.testing\nfrom tornado.io"
},
{
"path": "tests/unit/__main__.py",
"chars": 358,
"preview": "import unittest\nfrom glob import glob\n\nimport tornado.testing\n\n\ndef all():\n test_modules = list(map(lambda x: x.rstri"
},
{
"path": "tests/unit/api/__init__.py",
"chars": 304,
"preview": "import os\n\nfrom tests.unit import AsyncHTTPTestCase\n\n\nclass BaseApiTestCase(AsyncHTTPTestCase):\n def setUp(self):\n "
},
{
"path": "tests/unit/api/test_auth.py",
"chars": 664,
"preview": "from tests.unit import AsyncHTTPTestCase\n\n\nclass BasicAuthTests(AsyncHTTPTestCase):\n def test_auth(self):\n wit"
},
{
"path": "tests/unit/api/test_control.py",
"chars": 7445,
"preview": "import os\nfrom unittest.mock import MagicMock, patch\n\nfrom tornado.options import options\n\nfrom flower.api.control impor"
},
{
"path": "tests/unit/api/test_tasks.py",
"chars": 8174,
"preview": "import json\nimport time\nfrom collections import OrderedDict\nfrom datetime import datetime, timedelta\nfrom unittest.mock "
},
{
"path": "tests/unit/api/test_workers.py",
"chars": 2085,
"preview": "import json\nfrom unittest import mock\n\nfrom flower.inspector import Inspector\n\nfrom . import BaseApiTestCase\n\ninspect_re"
},
{
"path": "tests/unit/test_command.py",
"chars": 7812,
"preview": "import os\nimport subprocess\nimport sys\nimport tempfile\nimport unittest\nfrom unittest.mock import Mock, patch\n\nimport cel"
},
{
"path": "tests/unit/utils/__init__.py",
"chars": 2284,
"preview": "import xml.etree.ElementTree as ET\nfrom html.parser import HTMLParser\n\nfrom celery.events import Event\nfrom celery.utils"
},
{
"path": "tests/unit/utils/test_broker.py",
"chars": 7521,
"preview": "import unittest\nfrom unittest.mock import MagicMock\n\nfrom flower.utils import broker\nfrom flower.utils.broker import (Br"
},
{
"path": "tests/unit/utils/test_search.py",
"chars": 4717,
"preview": "import unittest\nfrom collections import namedtuple\n\nfrom flower.utils.search import (parse_search_terms, satisfies_searc"
},
{
"path": "tests/unit/utils/test_template.py",
"chars": 2388,
"preview": "import time\nimport unittest\n\nfrom pytz import utc\n\nfrom flower.utils.template import format_time, humanize\n\n\nclass TestH"
},
{
"path": "tests/unit/utils/test_utils.py",
"chars": 3035,
"preview": "import os\nimport tempfile\nimport unittest\nfrom unittest.mock import Mock, patch\n\nfrom celery import Celery\n\nfrom flower."
},
{
"path": "tests/unit/views/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/unit/views/test_auth.py",
"chars": 3532,
"preview": "from flower.views.auth import authenticate, validate_auth_option\nfrom tests.unit import AsyncHTTPTestCase\n\n\nclass BasicA"
},
{
"path": "tests/unit/views/test_broker.py",
"chars": 189,
"preview": "from tests.unit import AsyncHTTPTestCase\n\n\nclass TestBrokerView(AsyncHTTPTestCase):\n def test_empty_page(self):\n "
},
{
"path": "tests/unit/views/test_error.py",
"chars": 175,
"preview": "from tests.unit import AsyncHTTPTestCase\n\n\nclass ErrorTests(AsyncHTTPTestCase):\n def test_404(self):\n r = self"
},
{
"path": "tests/unit/views/test_monitor.py",
"chars": 7386,
"preview": "import re\nimport time\nfrom datetime import datetime, timedelta\n\nfrom celery.events import Event\nfrom kombu import uuid\n\n"
},
{
"path": "tests/unit/views/test_tasks.py",
"chars": 9378,
"preview": "import json\nimport time\n\nfrom celery.events import Event\n\nfrom flower.events import EventsState\nfrom tests.unit import A"
},
{
"path": "tests/unit/views/test_url_handlers.py",
"chars": 2078,
"preview": "import os\nfrom unittest.mock import patch\n\nfrom tornado.web import url\n\nfrom flower.app import rewrite_handler\nfrom test"
},
{
"path": "tests/unit/views/test_workers.py",
"chars": 13127,
"preview": "import json\nimport time\nimport unittest\nfrom unittest.mock import patch\n\nfrom celery.events import Event\nfrom celery.uti"
},
{
"path": "tox.ini",
"chars": 1082,
"preview": "[tox]\nenvlist =\n # Celery 5.2: only py38–py311 (py312 excluded)\n {py39,py310,py311}-celery52-{tornado60,tornado61,"
}
]
About this extraction
This page contains the full source code of the mher/flower GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 114 files (373.7 KB), approximately 94.5k tokens, and a symbol index with 445 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.