[ { "path": ".github/ISSUE_TEMPLATE/any-questions-topics-you-want-to-share.md", "content": "---\nname: Any Questions/Topics you want to share\nabout: Describe anything you want to talk about related to pygwalker\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: \"[BUG] pygwalker bug report\"\nlabels: bug\nassignees: ''\n\n---\n\n**Describe the bug**\nA clear and concise description of what the bug is.\n\n**To Reproduce**\nSteps to reproduce the behavior:\n1. Go to '...'\n2. Click on '....'\n3. Scroll down to '....'\n4. See error\n\n**Expected behavior**\nA clear and concise description of what you expected to happen.\n\n**Screenshots**\nIf applicable, add screenshots to help explain your problem.\n\n**Versions**\n- pygwalker version: \n- python version\n- browser\n\n**Additional context**\nAdd any other context about the problem here.\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Is your feature request related to a problem? Please describe.**\nA clear and concise description of what the problem is. Ex. I'm always frustrated when [...]\n\n**Describe the solution you'd like**\nA clear and concise description of what you want to happen.\n\n**Describe alternatives you've considered**\nA clear and concise description of any alternative solutions or features you've considered.\n\n**Additional context**\nAdd any other context or screenshots about the feature request here.\n" }, { "path": ".github/workflows/auto-ci.yml", "content": "# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node\n# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs\n\nname: Auto CI\n\non:\n push:\n branches: [ \"main\", \"dev\" ]\n pull_request:\n branches: [ \"main\" ]\n workflow_dispatch:\n\njobs:\n build-js:\n\n runs-on: ubuntu-latest\n\n strategy:\n matrix:\n node-version: [22.x]\n # See supported Node.js release schedule at https://nodejs.org/en/about/releases/\n\n steps:\n - uses: actions/checkout@v4\n with:\n submodules: 'recursive'\n - name: Use Node.js ${{ matrix.node-version }}\n uses: actions/setup-node@v4\n with:\n node-version: ${{ matrix.node-version }}\n cache: 'yarn'\n cache-dependency-path: ./app/yarn.lock\n - name: Try Compiling\n working-directory: ./\n run: |\n chmod u+x ./scripts/compile.sh\n ./scripts/compile.sh\n - name: Uploading dist\n uses: actions/upload-artifact@v4\n with:\n name: pygwalker-app\n path: ./pygwalker/templates/dist/*\n \n build-py:\n needs: build-js\n timeout-minutes: 90\n\n strategy:\n fail-fast: false\n matrix:\n python-version: ['3.12', '3.13']\n os-version: [ubuntu-latest, windows-latest, macos-latest]\n\n runs-on: ${{ matrix.os-version }}\n steps:\n - uses: actions/checkout@v4\n - name: Download PyGWalkerApp\n uses: actions/download-artifact@v4\n id: build-py\n with:\n name: pygwalker-app\n path: ./pygwalker/templates/dist\n - name: Use Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n - name: Download test data\n working-directory: ./\n run: |\n chmod u+x ./scripts/test-init.py\n python ./scripts/test-init.py\n - name: Try Installing\n working-directory: ./\n env:\n PIP_ONLY_BINARY: \"duckdb\"\n run: |\n pip install \".[export]\"\n - name: Try Install Modin In Linux Os\n if: ${{ matrix.os-version == 'ubuntu-latest' && matrix.python-version == '3.11' }}\n run: |\n pip install aiohttp==3.8.6\n pip install \"modin==0.23.1\" \"modin[ray]==0.23.1\"\n pip install pydantic==1.10.9\n - name: Try Running\n working-directory: ./tests/\n run: |\n pip install ipykernel nbconvert pandas polars\n python -m ipykernel install --name python --user\n jupyter kernelspec list\n jupyter nbconvert --execute --ExecutePreprocessor.kernel_name=python --to html *.ipynb\n - name: Try Pytest\n timeout-minutes: 20\n working-directory: ./\n run: |\n pip install duckdb_engine\n pip install pytest\n python ./scripts/ci_run_pytest.py\n - name: Uploading notebooks\n uses: actions/upload-artifact@v4\n if: ${{ matrix.python-version == '3.12' && matrix.os-version == 'ubuntu-latest' }}\n with:\n name: notebook\n path: |\n ./tests/main.html\n ./tests/offline.html\n ./tests/stress-test.html\n" }, { "path": ".github/workflows/publish.yml", "content": "name: Publish PyPI\n\non:\n workflow_dispatch:\n\njobs:\n build-js:\n\n runs-on: ubuntu-latest\n\n strategy:\n matrix:\n node-version: [22.x]\n # See supported Node.js release schedule at https://nodejs.org/en/about/releases/\n\n steps:\n - uses: actions/checkout@v4\n with:\n submodules: 'recursive'\n - name: Use Node.js ${{ matrix.node-version }}\n uses: actions/setup-node@v4\n with:\n node-version: ${{ matrix.node-version }}\n cache: 'yarn'\n cache-dependency-path: ./app/yarn.lock\n - name: Try Compiling\n working-directory: ./\n run: |\n chmod u+x ./scripts/compile.sh\n ./scripts/compile.sh\n - name: Uploading dist\n if: ${{ matrix.node-version == '22.x' }}\n uses: actions/upload-artifact@v4\n with:\n name: pygwalker-app\n path: ./pygwalker/templates/dist/*\n\n build-py:\n needs: [build-js]\n\n strategy:\n fail-fast: false\n matrix:\n python-version: ['3.12']\n os-version: [ubuntu-latest]\n\n runs-on: ${{ matrix.os-version }}\n steps:\n - uses: actions/checkout@v4\n - name: Download PyGWalkerApp\n uses: actions/download-artifact@v4\n with:\n name: pygwalker-app\n path: ./pygwalker/templates/dist\n - name: Use Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n - name: Try building\n working-directory: ./\n run: |\n pip install build twine\n python -m build .\n - name: Uploading packages\n if: ${{ matrix.python-version == '3.12' && matrix.os-version == 'ubuntu-latest' }}\n uses: actions/upload-artifact@v4\n with:\n name: pygwalker\n path: ./dist/*\n - name: Uploading PyPI\n if: ${{ matrix.python-version == '3.12' && matrix.os-version == 'ubuntu-latest' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags')) }}\n uses: pypa/gh-action-pypi-publish@v1.12.4\n with:\n # PyPI user\n # Password for your PyPI user or an access token\n password: ${{ secrets.PYPI_TOKEN }}\n # The target directory for distribution\n packages-dir: dist\n # Check metadata before uploading\n verify-metadata: true\n # Do not fail if a Python package distribution exists in the target package index\n skip-existing: true\n # Show verbose output.\n verbose: true\n # Show hash values of files to be uploaded\n print-hash: true\n" }, { "path": ".gitignore", "content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\npip-wheel-metadata/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n*.py,cover\n.hypothesis/\n.pytest_cache/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n.python-version\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# PEP 582; used by e.g. github.com/David-OConnor/pyflow\n__pypackages__/\n\n# Celery stuff\ncelerybeat-schedule\ncelerybeat.pid\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n__pycache__\ndist\n!/pygwalker/templates/dist/*\n.DS_Store\nvenv\n.ipynb_checkpoints\nnode_modules\npoetry.lock\n\npygwalker/templates/graphic-walker.umd.js\npygwalker/templates/graphic-walker.iife.js\n\ntests/*.csv\n\n__pycache__/\n.states\n*.db\n.web\n*.py[cod]\nassets/external/" }, { "path": ".gitmodules", "content": "" }, { "path": ".pylintrc", "content": "# This Pylint rcfile contains a best-effort configuration to uphold the\n# best-practices and style described in the Google Python style guide:\n# https://google.github.io/styleguide/pyguide.html\n#\n# Its canonical open-source location is:\n# https://google.github.io/styleguide/pylintrc\n\n[MASTER]\n\n# Files or directories to be skipped. They should be base names, not paths.\nignore=third_party\n\n# Files or directories matching the regex patterns are skipped. The regex\n# matches against base names, not paths.\nignore-patterns=\n\n# Pickle collected data for later comparisons.\npersistent=no\n\n# List of plugins (as comma separated values of python modules names) to load,\n# usually to register additional checkers.\nload-plugins=\n\n# Use multiple processes to speed up Pylint.\njobs=4\n\n# Allow loading of arbitrary C extensions. Extensions are imported into the\n# active Python interpreter and may run arbitrary code.\nunsafe-load-any-extension=no\n\n\n[MESSAGES CONTROL]\n\n# Only show warnings with the listed confidence levels. Leave empty to show\n# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED\nconfidence=\n\n# Enable the message, report, category or checker with the given id(s). You can\n# either give multiple identifier separated by comma (,) or put this option\n# multiple time (only on the command line, not in the configuration file where\n# it should appear only once). See also the \"--disable\" option for examples.\n#enable=\n\n# Disable the message, report, category or checker with the given id(s). You\n# can either give multiple identifiers separated by comma (,) or put this\n# option multiple times (only on the command line, not in the configuration\n# file where it should appear only once).You can also use \"--disable=all\" to\n# disable everything first and then reenable specific checks. For example, if\n# you want to run only the similarities checker, you can use \"--disable=all\n# --enable=similarities\". If you want to run only the classes checker, but have\n# no Warning level messages displayed, use\"--disable=all --enable=classes\n# --disable=W\"\ndisable=abstract-method,\n apply-builtin,\n arguments-differ,\n attribute-defined-outside-init,\n backtick,\n bad-option-value,\n basestring-builtin,\n buffer-builtin,\n c-extension-no-member,\n consider-using-enumerate,\n cmp-builtin,\n cmp-method,\n coerce-builtin,\n coerce-method,\n delslice-method,\n div-method,\n duplicate-code,\n eq-without-hash,\n execfile-builtin,\n file-builtin,\n filter-builtin-not-iterating,\n fixme,\n getslice-method,\n global-statement,\n hex-method,\n idiv-method,\n implicit-str-concat,\n import-error,\n import-self,\n import-star-module-level,\n inconsistent-return-statements,\n input-builtin,\n intern-builtin,\n invalid-str-codec,\n locally-disabled,\n long-builtin,\n long-suffix,\n map-builtin-not-iterating,\n misplaced-comparison-constant,\n missing-function-docstring,\n missing-module-docstring,\n metaclass-assignment,\n next-method-called,\n next-method-defined,\n no-absolute-import,\n no-else-break,\n no-else-continue,\n no-else-raise,\n no-else-return,\n no-init, # added\n no-member,\n no-name-in-module,\n no-self-use,\n nonzero-method,\n oct-method,\n old-division,\n old-ne-operator,\n old-octal-literal,\n old-raise-syntax,\n parameter-unpacking,\n print-statement,\n raising-string,\n range-builtin-not-iterating,\n raw_input-builtin,\n rdiv-method,\n reduce-builtin,\n relative-import,\n reload-builtin,\n round-builtin,\n setslice-method,\n signature-differs,\n standarderror-builtin,\n suppressed-message,\n sys-max-int,\n too-few-public-methods,\n too-many-ancestors,\n too-many-arguments,\n too-many-boolean-expressions,\n too-many-branches,\n too-many-instance-attributes,\n too-many-locals,\n too-many-nested-blocks,\n too-many-public-methods,\n too-many-return-statements,\n too-many-statements,\n trailing-newlines,\n unichr-builtin,\n unicode-builtin,\n unnecessary-pass,\n unpacking-in-except,\n useless-else-on-loop,\n useless-object-inheritance,\n useless-suppression,\n using-cmp-argument,\n wrong-import-order,\n xrange-builtin,\n zip-builtin-not-iterating,\n\n\n[REPORTS]\n\n# Set the output format. Available formats are text, parseable, colorized, msvs\n# (visual studio) and html. You can also give a reporter class, eg\n# mypackage.mymodule.MyReporterClass.\noutput-format=text\n\n# Tells whether to display a full report or only the messages\nreports=no\n\n# Python expression which should return a note less than 10 (10 is the highest\n# note). You have access to the variables errors warning, statement which\n# respectively contain the number of errors / warnings messages and the total\n# number of statements analyzed. This is used by the global evaluation report\n# (RP0004).\nevaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)\n\n# Template used to display messages. This is a python new-style format string\n# used to format the message information. See doc for all details\n#msg-template=\n\n\n[BASIC]\n\n# Good variable names which should always be accepted, separated by a comma\ngood-names=main,_\n\n# Bad variable names which should always be refused, separated by a comma\nbad-names=\n\n# Colon-delimited sets of names that determine each other's naming style when\n# the name regexes allow several styles.\nname-group=\n\n# Include a hint for the correct naming format with invalid-name\ninclude-naming-hint=no\n\n# List of decorators that produce properties, such as abc.abstractproperty. Add\n# to this list to register other decorators that produce valid properties.\nproperty-classes=abc.abstractproperty,cached_property.cached_property,cached_property.threaded_cached_property,cached_property.cached_property_with_ttl,cached_property.threaded_cached_property_with_ttl\n\n# Regular expression matching correct function names\nfunction-rgx=^(?:(?PsetUp|tearDown|setUpModule|tearDownModule)|(?P_?[A-Z][a-zA-Z0-9]*)|(?P_?[a-z][a-z0-9_]*))$\n\n# Regular expression matching correct variable names\nvariable-rgx=^[a-z][a-z0-9_]*$\n\n# Regular expression matching correct constant names\nconst-rgx=^(_?[A-Z][A-Z0-9_]*|__[a-z0-9_]+__|_?[a-z][a-z0-9_]*)$\n\n# Regular expression matching correct attribute names\nattr-rgx=^_{0,2}[a-z][a-z0-9_]*$\n\n# Regular expression matching correct argument names\nargument-rgx=^[a-z][a-z0-9_]*$\n\n# Regular expression matching correct class attribute names\nclass-attribute-rgx=^(_?[A-Z][A-Z0-9_]*|__[a-z0-9_]+__|_?[a-z][a-z0-9_]*)$\n\n# Regular expression matching correct inline iteration names\ninlinevar-rgx=^[a-z][a-z0-9_]*$\n\n# Regular expression matching correct class names\nclass-rgx=^_?[A-Z][a-zA-Z0-9]*$\n\n# Regular expression matching correct module names\nmodule-rgx=^(_?[a-z][a-z0-9_]*|__init__)$\n\n# Regular expression matching correct method names\nmethod-rgx=(?x)^(?:(?P_[a-z0-9_]+__|runTest|setUp|tearDown|setUpTestCase|tearDownTestCase|setupSelf|tearDownClass|setUpClass|(test|assert)_*[A-Z0-9][a-zA-Z0-9_]*|next)|(?P_{0,2}[A-Z][a-zA-Z0-9_]*)|(?P_{0,2}[a-z][a-z0-9_]*))$\n\n# Regular expression which should only match function or class names that do\n# not require a docstring.\nno-docstring-rgx=(__.*__|main|test.*|.*test|.*Test)$\n\n# Minimum line length for functions/classes that require docstrings, shorter\n# ones are exempt.\ndocstring-min-length=10\n\n\n[TYPECHECK]\n\n# List of decorators that produce context managers, such as\n# contextlib.contextmanager. Add to this list to register other decorators that\n# produce valid context managers.\ncontextmanager-decorators=contextlib.contextmanager,contextlib2.contextmanager\n\n# Tells whether missing members accessed in mixin class should be ignored. A\n# mixin class is detected if its name ends with \"mixin\" (case insensitive).\nignore-mixin-members=yes\n\n# List of module names for which member attributes should not be checked\n# (useful for modules/projects where namespaces are manipulated during runtime\n# and thus existing member attributes cannot be deduced by static analysis. It\n# supports qualified module names, as well as Unix pattern matching.\nignored-modules=\n\n# List of class names for which member attributes should not be checked (useful\n# for classes with dynamically set attributes). This supports the use of\n# qualified names.\nignored-classes=optparse.Values,thread._local,_thread._local\n\n# List of members which are set dynamically and missed by pylint inference\n# system, and so shouldn't trigger E1101 when accessed. Python regular\n# expressions are accepted.\ngenerated-members=\n\n\n[FORMAT]\n\n# Maximum number of characters on a single line.\nmax-line-length=180\n\n# TODO(https://github.com/PyCQA/pylint/issues/3352): Direct pylint to exempt\n# lines made too long by directives to pytype.\n\n# Regexp for a line that is allowed to be longer than the limit.\nignore-long-lines=(?x)(\n ^\\s*(\\#\\ )??$|\n ^\\s*(from\\s+\\S+\\s+)?import\\s+.+$)\n\n# Allow the body of an if to be on the same line as the test if there is no\n# else.\nsingle-line-if-stmt=yes\n\n# Maximum number of lines in a module\nmax-module-lines=99999\n\n# String used as indentation unit. The internal Google style guide mandates 2\n# spaces. Google's externaly-published style guide says 4, consistent with\n# PEP 8. Here, we use 2 spaces, for conformity with many open-sourced Google\n# projects (like TensorFlow).\nindent-string=' '\n\n# Number of spaces of indent required inside a hanging or continued line.\nindent-after-paren=4\n\n# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.\nexpected-line-ending-format=\n\n\n[MISCELLANEOUS]\n\n# List of note tags to take in consideration, separated by a comma.\nnotes=TODO\n\n\n[STRING]\n\n# This flag controls whether inconsistent-quotes generates a warning when the\n# character used as a quote delimiter is used inconsistently within a module.\ncheck-quote-consistency=no\n\n\n[VARIABLES]\n\n# Tells whether we should check for unused import in __init__ files.\ninit-import=no\n\n# A regular expression matching the name of dummy variables (i.e. expectedly\n# not used).\ndummy-variables-rgx=^\\*{0,2}(_$|unused_|dummy_)\n\n# List of additional names supposed to be defined in builtins. Remember that\n# you should avoid to define new builtins when possible.\nadditional-builtins=\n\n# List of strings which can identify a callback function by name. A callback\n# name must start or end with one of those strings.\ncallbacks=cb_,_cb\n\n# List of qualified module names which can have objects that can redefine\n# builtins.\nredefining-builtins-modules=six,six.moves,past.builtins,future.builtins,functools\n\n\n[LOGGING]\n\n# Logging modules to check that the string format arguments are in logging\n# function parameter format\nlogging-modules=logging,absl.logging,tensorflow.io.logging\n\n\n[SIMILARITIES]\n\n# Minimum lines number of a similarity.\nmin-similarity-lines=4\n\n# Ignore comments when computing similarities.\nignore-comments=yes\n\n# Ignore docstrings when computing similarities.\nignore-docstrings=yes\n\n# Ignore imports when computing similarities.\nignore-imports=no\n\n\n[SPELLING]\n\n# Spelling dictionary name. Available dictionaries: none. To make it working\n# install python-enchant package.\nspelling-dict=\n\n# List of comma separated words that should not be checked.\nspelling-ignore-words=\n\n# A path to a file that contains private dictionary; one word per line.\nspelling-private-dict-file=\n\n# Tells whether to store unknown words to indicated private dictionary in\n# --spelling-private-dict-file option instead of raising a message.\nspelling-store-unknown-words=no\n\n\n[IMPORTS]\n\n# Deprecated modules which should not be used, separated by a comma\ndeprecated-modules=regsub,\n TERMIOS,\n Bastion,\n rexec,\n sets\n\n# Create a graph of every (i.e. internal and external) dependencies in the\n# given file (report RP0402 must not be disabled)\nimport-graph=\n\n# Create a graph of external dependencies in the given file (report RP0402 must\n# not be disabled)\next-import-graph=\n\n# Create a graph of internal dependencies in the given file (report RP0402 must\n# not be disabled)\nint-import-graph=\n\n# Force import order to recognize a module as part of the standard\n# compatibility libraries.\nknown-standard-library=\n\n# Force import order to recognize a module as part of a third party library.\nknown-third-party=enchant, absl\n\n# Analyse import fallback blocks. This can be used to support both Python 2 and\n# 3 compatible code, which means that the block might have code that exists\n# only in one or another interpreter, leading to false positives when analysed.\nanalyse-fallback-blocks=no\n\n\n[CLASSES]\n\n# List of method names used to declare (i.e. assign) instance attributes.\ndefining-attr-methods=__init__,\n __new__,\n setUp\n\n# List of member names, which should be excluded from the protected access\n# warning.\nexclude-protected=_asdict,\n _fields,\n _replace,\n _source,\n _make\n\n# List of valid names for the first argument in a class method.\nvalid-classmethod-first-arg=cls,\n class_\n\n# List of valid names for the first argument in a metaclass class method.\nvalid-metaclass-classmethod-first-arg=mcs\n\n\n[EXCEPTIONS]\n\n# Exceptions that will emit a warning when being caught. Defaults to\n# \"Exception\"\novergeneral-exceptions=builtins.StandardError,\n builtins.Exception,\n builtins.BaseException" }, { "path": "CITATION.cff", "content": "# This CITATION.cff file was generated with cffinit.\n# Visit https://bit.ly/cffinit to generate yours today!\n\ncff-version: 1.2.0\ntitle: PyGWalker\nmessage: >-\n If you use this software, please cite it using the\n metadata from this file.\ntype: software\nauthors:\n - website: 'https://kanaries.net/'\n name: Kanaries Open Source Community\nrepository-code: 'https://github.com/Kanaries/pygwalker'\nurl: 'https://kanaries.net/pygwalker'\nabstract: >-\n PyGWalker is a Python Library for Exploratory Data\n Analysis with Visualization that can simplify your Jupyter\n Notebook data analysis and data visualization workflow, by\n turning your pandas dataframe into an interactive user\n interface for visual exploration.\nkeywords:\n - Data Analysis\n - Exploratory Data Analysis\n - Data Visualization tools\n - Python Library\n - interactive\nlicense: Apache-2.0\n" }, { "path": "CONDUCT.md", "content": "# Code of Conduct\n\n## Our Pledge\n\nWe as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.\n\nWe pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.\n\n## Our Standards\n\nExamples of behavior that contributes to a positive environment for our community include:\n\n- Demonstrating empathy and kindness toward other people\n- Being respectful of differing opinions, viewpoints, and experiences\n- Giving and gracefully accepting constructive feedback\n- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience\n- Focusing on what is best not just for us as individuals, but for the overall community\n\nExamples of unacceptable behavior include:\n\n- The use of sexualized language or imagery, and sexual attention or advances of any kind\n- Trolling, insulting or derogatory comments, and personal or political attacks\n- Public or private harassment\n- Publishing others’ private information, such as a physical or email address, without their explicit permission\n- Other conduct which could reasonably be considered inappropriate in a professional setting\n\n## Enforcement Responsibilities\n\nCommunity leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.\n\nCommunity leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.\n\n## Scope\n\nThis Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.\n\n## Enforcement\n\nInstances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at [support@kanaries.net](mailto:support@kanaries.net). All complaints will be reviewed and investigated promptly and fairly.\n\nAll community leaders are obligated to respect the privacy and security of the reporter of any incident.\n\n## Enforcement Guidelines\n\nCommunity leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:\n\n#### 1. Correction \nCommunity Impact: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.\n\nConsequence: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.\n\n#### 2. Warning\nCommunity Impact: A violation through a single incident or series of actions.\n\nConsequence: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.\n\n#### 3. Temporary Ban\nCommunity Impact: A serious violation of community standards, including sustained inappropriate behavior.\n\nConsequence: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.\n\n#### 4. Permanent Ban\nCommunity Impact: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.\n\nConsequence: A permanent ban from any sort of public interaction within the community.\n\n## Attribution\n\nThis Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.\n" }, { "path": "LICENSE", "content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [2023] [Kanaries Data, Inc.]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License." }, { "path": "MANIFEST.in", "content": "include LICENSE\ninclude README.md\ngraft pygwalker/templates/**" }, { "path": "README.md", "content": "[English](README.md) | [Español](./docs/README.es.md) | [Français](./docs/README.fr.md) | [Deutsch](./docs/README.de.md) | [中文](./docs/README.zh.md) | [Türkçe](./docs/README.tr.md) | [日本語](./docs/README.ja.md) | [한국어](./docs/README.ko.md) | [Русский](./docs/README.ru.md)\n\n\n

\"\"

\n\n

PyGWalker: A Python Library for Exploratory Data Analysis with Visualization

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker) can simplify your Jupyter Notebook data analysis and data visualization workflow, by turning your pandas dataframe into an interactive user interface for visual exploration.\n\n**PyGWalker** (pronounced like \"Pig Walker\", just for fun) is named as an abbreviation of \"**Py**thon binding of **G**raphic **Walker**\". It integrates Jupyter Notebook with [Graphic Walker](https://github.com/Kanaries/graphic-walker), an open-source alternative to Tableau. It allows data scientists to visualize / clean / annotates the data with simple drag-and-drop operations and even natural language queries.\n \n\nhttps://github.com/Kanaries/pygwalker/assets/22167673/2b940e11-cf8b-4cde-b7f6-190fb10ee44b\n\n> [!TIP]\n> If you want more AI features, we also build [runcell](https://runcell.dev), an AI Code Agent in Jupyter that understands your code/data/cells and generate code, execute cells and take actions for you. It can be used in jupyter lab with `pip install runcell`\n\n\n\nhttps://github.com/user-attachments/assets/9ec64252-864d-4bd1-8755-83f9b0396d38\n\n\n\n\nVisit [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) or [Graphic Walker Online Demo](https://graphic-walker.kanaries.net/) to test it out!\n\n> If you prefer using R, check [GWalkR](https://github.com/Kanaries/GWalkR), the R wrapper of Graphic Walker.\n> If you prefer a Desktop App that can be used offline and without any coding, check out [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip).\n\n\n# Features\nPyGWalker is a Python library that simplifies data analysis and visualization workflows by turning pandas DataFrames into interactive visual interfaces.\nIt offers a variety of features that make it a powerful tool for data exploration:\n- ##### Interactive Data Exploration:\n - Drag-and-drop interface for easy visualization creation.   \n - Real-time updates as you make changes to the visualization.\n - Ability to zoom, pan, and filter the data.   \n- ##### Data Cleaning and Transformation:\n - Visual data cleaning tools to identify and remove outliers or inconsistencies.   \n - Ability to create new variables and features based on existing data.   \n- ##### Advanced Visualization Capabilities:\n - Support for various chart types (bar charts, line charts, scatter plots, etc.). \n - Customization options for colors, labels, and other visual elements.   \n - Interactive features like tooltips and drill-down capabilities.   \n- ##### Integration with Jupyter Notebooks:\n - Seamless integration with Jupyter Notebooks for a smooth workflow.   \n- ##### Open-Source and Free:\n - Available for free and allows for customization and extension.\n\n\n\n# Getting Started\n> Check our video tutorial about using pygwalker, pygwalker + streamlit and pygwalker + snowflake, [How to explore data with PyGWalker in Python\n](https://youtu.be/rprn79wfB9E?si=lAsJn1cAQnb-EklD)\n\n| [Run in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) | [Run in Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## Setup pygwalker\n\nBefore using pygwalker, make sure to install the packages through the command line using pip or conda.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **Note**\n> \n> For an early trial, you can install with `pip install pygwalker --upgrade` to keep your version up to date with the latest release or even `pip install pygwalker --upgrade --pre` to obtain latest features and bug-fixes.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\nor\n```bash\nmamba install -c conda-forge pygwalker\n```\nSee [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock) for more help.\n\n\n## Use pygwalker in Jupyter Notebook\n\n### Quick Start\n\nImport pygwalker and pandas to your Jupyter Notebook to get started.\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\nYou can use pygwalker without breaking your existing workflow. For example, you can call up PyGWalker with the dataframe loaded in this way:\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nThat's it. Now you have an interactive UI to analyze and visualize data with simple drag-and-drop operations.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nCool things you can do with PyGwalker:\n\n+ You can change the mark type into others to make different charts, for example, a line chart:\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ To compare different measures, you can create a concat view by adding more than one measure into rows/columns.\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ To make a facet view of several subviews divided by the value in dimension, put dimensions into rows or columns to make a facets view.\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n\n+ PyGWalker contains a powerful data table, which provides a quick view of data and its distribution, profiling. You can also add filters or change the data types in the table.\n\"pygwalker-data-preview\"\n\n+ You can save the data exploration result to a local file\n\n### Better Practices\n\nThere are some important parameters you should know when using pygwalker: \n\n+ `spec`: for save/load chart config (json string or file path)\n+ `kernel_computation`: for using duckdb as computing engine which allows you to handle larger dataset faster in your local machine.\n+ `use_kernel_calc`: Deprecated, use `kernel_computation` instead.\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # this json file will save your chart state, you need to click save button in ui mannual when you finish a chart, 'autosave' will be supported in the future.\n kernel_computation=True, # set `kernel_computation=True`, pygwalker will use duckdb as computing engine, it support you explore bigger dataset(<=100GB).\n)\n```\n\n### Example in local notebook\n\n* Notebook Code: [Click Here](https://github.com/Kanaries/pygwalker-offline-example)\n* Preview Notebook Html: [Click Here](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### Example in cloud notebook\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n### Programmatic Export of Charts\n\nAfter saving a chart from the UI, you can retrieve the image directly from Python.\n\n```python\nwalker = pyg.walk(df, spec=\"./chart_meta_0.json\")\n# edit the chart in the UI and click the save button\nwalker.save_chart_to_file(\"Chart 1\", \"chart1.svg\", save_type=\"svg\")\npng_bytes = walker.export_chart_png(\"Chart 1\")\nsvg_bytes = walker.export_chart_svg(\"Chart 1\")\n```\n\n## Use pygwalker in Streamlit\nStreamlit allows you to host a web version of pygwalker without figuring out details of how web application works.\n\nHere are some of the app examples build with pygwalker and streamlit:\n+ [PyGWalker + streamlit for Bike sharing dataset](https://pygwalkerdemo-cxz7f7pt5oc.streamlit.app/)\n+ [Earthquake Dashboard](https://earthquake-dashboard-pygwalker.streamlit.app/)\n\n[![](https://user-images.githubusercontent.com/22167673/271170853-5643c3b1-6216-4ade-87f4-41c6e6893eab.png)](https://earthquake-dashboard-pygwalker.streamlit.app/)\n\n```python\nfrom pygwalker.api.streamlit import StreamlitRenderer\nimport pandas as pd\nimport streamlit as st\n\n# Adjust the width of the Streamlit page\nst.set_page_config(\n page_title=\"Use Pygwalker In Streamlit\",\n layout=\"wide\"\n)\n\n# Add Title\nst.title(\"Use Pygwalker In Streamlit\")\n\n# You should cache your pygwalker renderer, if you don't want your memory to explode\n@st.cache_resource\ndef get_pyg_renderer() -> \"StreamlitRenderer\":\n df = pd.read_csv(\"./bike_sharing_dc.csv\")\n # If you want to use feature of saving chart config, set `spec_io_mode=\"rw\"`\n return StreamlitRenderer(df, spec=\"./gw_config.json\", spec_io_mode=\"rw\")\n\n\nrenderer = get_pyg_renderer()\n\nrenderer.explorer()\n```\n\n## [API Reference](https://pygwalker-docs.vercel.app/api-reference/jupyter)\n\n### [pygwalker.walk](https://pygwalker-docs.vercel.app/api-reference/jupyter#walk)\n\n\n| Parameter | Type | Default | Description |\n|------------------------|-----------------------------------------------------------|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|\n| dataset | Union[DataFrame, Connector] | - | The dataframe or connector to be used. |\n| gid | Union[int, str] | None | ID for the GraphicWalker container div, formatted as 'gwalker-{gid}'. |\n| env | Literal['Jupyter', 'JupyterWidget'] | 'JupyterWidget' | Environment using pygwalker. |\n| field_specs | Optional[Dict[str, FieldSpec]] | None | Specifications of fields. Will be automatically inferred from `dataset` if not specified. |\n| hide_data_source_config | bool | True | If True, hides DataSource import and export button. |\n| theme_key | Literal['vega', 'g2'] | 'g2' | Theme type for the GraphicWalker. |\n| appearance | Literal['media', 'light', 'dark'] | 'media' | Theme setting. 'media' will auto-detect the OS theme. |\n| spec | str | \"\" | Chart configuration data. Can be a configuration ID, JSON, or remote file URL. |\n| use_preview | bool | True | If True, uses the preview function. |\n| kernel_computation | bool | False | If True, uses kernel computation for data. |\n| **kwargs | Any | - | Additional keyword arguments. |\n\n## Development\n\nRefer it: [local-development](https://docs.kanaries.net/pygwalker/installation#local-development)\n\n## Tested Environments\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab\n- [x] Jupyter Lite\n- [x] Databricks Notebook (Since version `0.1.4a0`)\n- [x] Jupyter Extension for Visual Studio Code (Since version `0.1.4a0`)\n- [x] Most web applications compatiable with IPython kernels. (Since version `0.1.4a0`)\n- [x] **Streamlit (Since version `0.1.4.9`)**, enabled with `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (Since version `0.1.4a0`)\n- [x] Panel. See [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker).\n- [x] marimo (Since version `0.4.9.11`)\n- [ ] Hex Projects \n- [ ] ...feel free to raise an issue for more environments.\n\n## Configuration And Privacy Policy(pygwalker >= 0.3.10)\n\nYou can use `pygwalker config` to set your privacy configuration.\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: ~/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND. Events data will bind with a unique id, which is generated by pygwalker when it is installed based on timestamp. We will not collect any other information about you.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\nMore details, refer it: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# License\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# Contribution Guideline\nYou are encouraged to contribute to PyGWalker in any way that suits your interests. This may include:\n- Answering questions and providing support\n- Sharing ideas for new features\n- Reporting bugs and glitches\n- Contributing code to the project\n- Offering suggestions for website improvements and better documentation\n\n# Resources\n\n> PyGWalker Cloud is released! You can now save your charts to cloud, publish the interactive cell as a web app and use advanced GPT-powered features. Check out the [PyGWalker Cloud](https://kanaries.net/pygwalker?from=gh_md) for more details.\n\n+ Check out more resources about PyGWalker on [Kanaries PyGWalker](https://kanaries.net/pygwalker)\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ We are also working on [RATH](https://kanaries.net): an Open Source, Automate exploratory data analysis software that redefines the workflow of data wrangling, exploration and visualization with AI-powered automation. Check out the [Kanaries website](https://kanaries.net) and [RATH GitHub](https://github.com/Kanaries/Rath) for more!\n+ [Youtube: How to explore data with PyGWalker in Python\n](https://youtu.be/rprn79wfB9E?si=lAsJn1cAQnb-EklD)\n+ [Use pygwalker to build visual analysis app in streamlit](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ Use [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker) to build data visualization apps with Panel.\n+ If you encounter any issues and need support, please join our [Discord](https://discord.gg/Z4ngFWXz2U) channel or raise an issue on github.\n+ Share pygwalker on these social media platforms if you like it!\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-alternative%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-alternative%20UI)\n" }, { "path": "app/.gitignore", "content": "node_modules/\n!lib" }, { "path": "app/components.json", "content": "{\n \"$schema\": \"https://ui.shadcn.com/schema.json\",\n \"style\": \"new-york\",\n \"rsc\": false,\n \"tsx\": true,\n \"tailwind\": {\n \"config\": \"tailwind.config.js\",\n \"css\": \"src/index.css\",\n \"baseColor\": \"zinc\",\n \"cssVariables\": true\n },\n \"aliases\": {\n \"components\": \"@/components\",\n \"utils\": \"@/lib/utils\"\n }\n}" }, { "path": "app/index.html", "content": "\n\n\n \n \n \n PyGWalker App Preview\n\n\n
\n \n\n\n" }, { "path": "app/package.json", "content": "{\n \"name\": \"pygwalker-app\",\n \"version\": \"0.0.1\",\n \"main\": \"index.ts\",\n \"license\": \"Apache License 2.0\",\n \"private\": true,\n \"scripts\": {\n \"build\": \"vite build && vite build --mode=dsl_to_workflow && vite build --mode=vega_to_dsl\",\n \"build:app\": \"vite build\",\n \"dev:preinstall\": \"(cd ../graphic-walker/packages/graphic-walker; yarn --frozen-lockfile && yarn build)\",\n \"dev:server\": \"vite --host\",\n \"dev\": \"vite\",\n \"test:front_end\": \"vite --host\",\n \"test\": \"npm run test:front_end\",\n \"serve\": \"vite preview\"\n },\n \"dependencies\": {\n \"@anywidget/react\": \"^0.0.8\",\n \"@headlessui/react\": \"^1.7.14\",\n \"@heroicons/react\": \"^2.0.8\",\n \"@kanaries/graphic-walker\": \"0.5.0-alpha.2\",\n \"@kanaries/gw-dsl-parser\": \"0.1.49\",\n \"@radix-ui/react-checkbox\": \"^1.3.3\",\n \"@radix-ui/react-dialog\": \"^1.1.15\",\n \"@radix-ui/react-icons\": \"^1.3.2\",\n \"@radix-ui/react-label\": \"^2.1.8\",\n \"@radix-ui/react-select\": \"^2.2.6\",\n \"@radix-ui/react-slot\": \"^1.2.4\",\n \"@radix-ui/react-tabs\": \"^1.1.13\",\n \"@radix-ui/react-toggle\": \"^1.1.10\",\n \"@radix-ui/react-toggle-group\": \"^1.1.11\",\n \"@segment/analytics-next\": \"^1.69.0\",\n \"autoprefixer\": \"^10.3.5\",\n \"buffer\": \"^6.0.3\",\n \"class-variance-authority\": \"^0.7.0\",\n \"clsx\": \"^2.0.0\",\n \"html-to-image\": \"^1.11.11\",\n \"lucide-react\": \"^0.341.0\",\n \"mobx\": \"^6.9.0\",\n \"mobx-react-lite\": \"^3.4.3\",\n \"postcss\": \"^8.3.7\",\n \"react\": \"19.x\",\n \"react-dom\": \"19.x\",\n \"react-syntax-highlighter\": \"^15.5.0\",\n \"streamlit-component-lib\": \"^2.0.0\",\n \"styled-components\": \"^5.3.6\",\n \"tailwind-merge\": \"^1.14.0\",\n \"tailwindcss\": \"^3.2.4\",\n \"tailwindcss-animate\": \"^1.0.7\",\n \"uuid\": \"^8.3.2\"\n },\n \"devDependencies\": {\n \"@rollup/plugin-commonjs\": \"^24.0.x\",\n \"@rollup/plugin-replace\": \"^5.0.x\",\n \"@rollup/plugin-terser\": \"^0.4.x\",\n \"@rollup/plugin-typescript\": \"^11.0.x\",\n \"@types/node\": \"^20.7.2\",\n \"@types/react\": \"^19.x\",\n \"@types/react-dom\": \"^19.x\",\n \"@types/react-syntax-highlighter\": \"^15.5.7\",\n \"@types/styled-components\": \"^5.1.26\",\n \"@vitejs/plugin-react\": \"^3.1.x\",\n \"typescript\": \"^5.4.2\",\n \"vite\": \"4.1.5\",\n \"vite-plugin-wasm\": \"^3.2.2\"\n },\n \"resolutions\": {\n \"react\": \"^19.x\",\n \"react-dom\": \"^19.x\",\n \"@types/react\": \"^19.x\",\n \"@types/react-dom\": \"^19.x\"\n },\n \"peerDependencies\": {},\n \"prettier\": {\n \"tabWidth\": 4,\n \"printWidth\": 160\n }\n}\n" }, { "path": "app/postcss.config.js", "content": "module.exports = {\n plugins: {\n tailwindcss: {},\n autoprefixer: {},\n },\n}\n" }, { "path": "app/src/components/codeExportModal/index.tsx", "content": "import React, { useEffect, useState, useCallback } from \"react\";\nimport { observer } from \"mobx-react-lite\";\nimport { Light as SyntaxHighlighter } from \"react-syntax-highlighter\";\nimport json from \"react-syntax-highlighter/dist/esm/languages/hljs/json\";\nimport py from \"react-syntax-highlighter/dist/esm/languages/hljs/python\";\nimport atomOneLight from \"react-syntax-highlighter/dist/esm/styles/hljs/atom-one-light\";\nimport atomOneDark from \"react-syntax-highlighter/dist/esm/styles/hljs/atom-one-dark\";\nimport type { VizSpecStore } from \"@kanaries/graphic-walker/store/visualSpecStore\";\nimport type { IChart } from \"@kanaries/graphic-walker/interfaces\";\nimport { Tabs, TabsContent, TabsList, TabsTrigger } from \"@/components/ui/tabs\";\nimport commonStore from \"@/store/common\";\nimport { darkModeContext } from \"@/store/context\";\nimport { Button } from \"@/components/ui/button\";\nimport { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from \"@/components/ui/dialog\";\nimport { tracker } from \"@/utils/tracker\";\n\nimport { usePythonCode } from \"./usePythonCode\";\n\nSyntaxHighlighter.registerLanguage(\"json\", json);\nSyntaxHighlighter.registerLanguage(\"python\", py);\n\ninterface ICodeExport {\n globalStore: React.MutableRefObject;\n sourceCode: string;\n open: boolean;\n setOpen: (open: boolean) => void;\n}\n\nconst CodeExport: React.FC = observer((props) => {\n const { globalStore, sourceCode, open, setOpen } = props;\n const [visSpec, setVisSpec] = useState([]);\n const [tips, setTips] = useState(\"\");\n const darkMode = React.useContext(darkModeContext);\n\n const { pyCode } = usePythonCode({\n sourceCode,\n visSpec,\n version: commonStore.version,\n });\n\n const closeModal = useCallback(() => {\n setOpen(false);\n }, [setOpen]);\n\n const copyToCliboard = async (content: string) => {\n try {\n navigator.clipboard.writeText(content);\n setOpen(false);\n } catch(e) {\n setTips(\"The Clipboard API has been blocked in this environment. Please copy manully.\");\n }\n };\n\n useEffect(() => {\n if (open && globalStore.current) {\n const res = globalStore.current.exportCode();\n setVisSpec(res);\n }\n }, [open]);\n\n return (\n {\n setOpen(show);\n }}\n >\n \n \n Code Export\n \n Export the code of all charts in PyGWalker.\n \n \n
\n \n \n Python\n JSON(Graphic Walker)\n \n \n

PyGWalker Code

\n \n {pyCode}\n \n
\n

{tips}

\n
\n
\n {\n copyToCliboard(pyCode);\n }}\n >\n Copy to Clipboard\n \n \n
\n \n \n

Graphic Walker Specification

\n \n {JSON.stringify(visSpec, null, 2)}\n \n
\n

{tips}

\n
\n
\n {\n copyToCliboard(JSON.stringify(visSpec, null, 2));\n tracker.track(\"click\", {\"entity\": \"copy_code_button\"});\n }}\n >\n Copy to Clipboard\n \n \n
\n \n \n
\n \n \n );\n});\n\nexport default CodeExport;\n" }, { "path": "app/src/components/codeExportModal/usePythonCode.ts", "content": "import { chartToWorkflow } from \"@kanaries/graphic-walker/utils/workflow\";\nimport type { IChart } from \"@kanaries/graphic-walker/interfaces\";\nimport { useMemo } from \"react\"\n\nexport function usePythonCode (props: {\n sourceCode: string;\n visSpec: IChart[];\n version: string;\n}) {\n const { sourceCode, visSpec, version } = props;\n const pygConfig = useMemo(() => {\n return JSON.stringify({\n \"config\": visSpec,\n \"chart_map\": {},\n \"workflow_list\": visSpec.map((spec) => chartToWorkflow(spec)),\n version\n })\n }, [visSpec])\n const pyCode = useMemo(() => {\n const preCode = sourceCode.replace(\"'____pyg_walker_spec_params____'\", \"vis_spec\")\n return `vis_spec = r\"\"\"${pygConfig}\"\"\"\\n${preCode}`;\n }, [sourceCode, pygConfig])\n return {\n pyCode\n }\n}" }, { "path": "app/src/components/initModal/index.tsx", "content": "import React from \"react\";\nimport { observer } from \"mobx-react-lite\";\nimport { Dialog, DialogContent, DialogTitle } from \"@/components/ui/dialog\";\n\nimport commonStore from \"../../store/common\";\n\ninterface IInitModal {}\n\nconst InitModal: React.FC = observer((props) => {\n return (\n \n \n {commonStore.initModalInfo.title}\n
\n {commonStore.initModalInfo.title}\n \n {commonStore.initModalInfo.curIndex} / {commonStore.initModalInfo.total}\n \n
\n
\n
\n \n \n \n );\n});\n\nexport default InitModal;\n" }, { "path": "app/src/components/options.tsx", "content": "import React, { useEffect, useState } from \"react\";\nimport type { IAppProps } from \"../interfaces\";\n\nconst copyToClipboard = async (text: string) => {\n return navigator.clipboard.writeText(text);\n};\n\ninterface ISolutionProps {\n header: string;\n cmd: string;\n}\n\nconst updateSolutions: ISolutionProps[] = [\n {\n header: \"Using pip:\",\n cmd: \"pip install pygwalker --upgrade\",\n },\n {\n header: \"Using anaconda:\",\n cmd: \"conda install -c conda-forge pygwalker\",\n },\n];\n\nconst Solution: React.FC = (props) => {\n const [busy, setBusy] = useState(false);\n\n return (\n <>\n
{props.header}
\n
\n {props.cmd}\n {\n if (busy) {\n return;\n }\n setBusy(true);\n copyToClipboard(props.cmd).finally(() => {\n setTimeout(() => {\n setBusy(false);\n }, 2_000);\n });\n }}\n >\n {busy ? \"Copied\" : \"Copy\"}\n {busy ? \"\\u2705\" : \"\\u274f\"}\n
\n \n \n );\n};\n\nconst RAND_HASH = Math.random().toString(16).split(\".\").at(1) + new Date().getTime().toString(16).padStart(16, \"0\");\nconst Options: React.FC = (props: IAppProps) => {\n const [outdated, setOutDated] = useState(false);\n const [appMeta, setAppMeta] = useState({});\n const [showUpdateHint, setShowUpdateHint] = useState(false);\n if (window.localStorage.getItem(\"HASH\") === null) {\n window.localStorage.setItem(\"HASH\", RAND_HASH);\n }\n const UPDATE_URL = \"https://5agko11g7e.execute-api.us-west-1.amazonaws.com/default/check_updates\";\n const VERSION = (window as any)?.__GW_VERSION || \"current\";\n const HASH = window.localStorage.getItem(\"HASH\");\n useEffect(() => {\n if (props.userConfig?.privacy !== \"offline\") {\n const req = `${UPDATE_URL}?pkg=pygwalker-app&v=${VERSION}&hashcode=${HASH}&env=${process.env.NODE_ENV}&kernelHashcode=${props.hashcode}`;\n fetch(req, {\n headers: {\n \"Content-Type\": \"application/json\",\n },\n })\n .then((resp) => resp.json())\n .then((res) => {\n setAppMeta({ data: res.data });\n setOutDated(res?.data?.outdated || false);\n });\n }\n }, []);\n\n useEffect(() => {\n setShowUpdateHint(false);\n }, [outdated]);\n\n useEffect(() => {\n if (!showUpdateHint) {\n return;\n }\n const handleDismiss = () => {\n setShowUpdateHint(false);\n };\n document.addEventListener(\"click\", handleDismiss);\n return () => {\n document.removeEventListener(\"click\", handleDismiss);\n };\n }, [showUpdateHint]);\n\n useEffect(() => {\n setShowUpdateHint(false);\n }, [outdated]);\n\n useEffect(() => {\n if (!showUpdateHint) {\n return;\n }\n const handleDismiss = () => {\n setShowUpdateHint(false);\n };\n document.addEventListener(\"click\", handleDismiss);\n return () => {\n document.removeEventListener(\"click\", handleDismiss);\n };\n }, [showUpdateHint]);\n\n return (\n <>\n \n {outdated && (\n
e.stopPropagation()}>\n

\n \n {\"Update: \"}\n {`${VERSION}\\u2191`}\n {` ${appMeta?.data?.latest?.release?.version || \"latest\"}`}\n \n |\n setShowUpdateHint((s) => !s)}>\n {`${showUpdateHint ? \"Hide\" : \" Cmd\"} \\u274f`}\n \n

\n {showUpdateHint && (\n
\n {updateSolutions.map((sol, i) => (\n \n ))}\n
\n )}\n
\n )}\n \n );\n};\nexport default Options;\n" }, { "path": "app/src/components/preview/index.tsx", "content": "import React from \"react\";\nimport { observer } from \"mobx-react-lite\";\nimport { PureRenderer, IRow } from '@kanaries/graphic-walker';\nimport type { IDarkMode, IThemeKey } from '@kanaries/graphic-walker/interfaces';\n\nimport { Tabs, TabsContent, TabsList, TabsTrigger } from \"@/components/ui/tabs\";\n// @ts-ignore\nimport style from '@/index.css?inline'\n\ninterface IPreviewProps {\n gid: string;\n themeKey: string;\n dark: IDarkMode;\n charts: {\n visSpec: any;\n data: IRow[];\n }[];\n}\n\nconst Preview: React.FC = observer((props) => {\n const { charts, themeKey } = props;\n\n return (\n \n
\n \n \n
\n \n {charts.map((chart, index) => {\n return {chart.visSpec.name}\n })}\n \n
\n {charts.map((chart, index) => {\n return \n { return chart.data }}\n appearance={props.dark as IDarkMode}\n />\n \n })}\n \n
\n \n );\n});\n\ninterface IChartPreviewProps {\n themeKey: string;\n dark: IDarkMode;\n visSpec: any;\n data: IRow[];\n title: string;\n desc: string;\n}\n\nconst ChartPreview: React.FC = observer((props) => {\n\n return (\n \n
\n

{ props.title }

\n

{ props.desc }

\n { return props.data }}\n appearance={props.dark as IDarkMode}\n />\n
\n \n );\n});\n\n\nexport {\n Preview,\n ChartPreview,\n};\n\nexport type {\n IPreviewProps,\n IChartPreviewProps\n}\n" }, { "path": "app/src/components/runcellBanner/index.tsx", "content": "import React, { useState } from \"react\";\nimport { tracker } from \"@/utils/tracker\";\n\nconst RUNCELL_LOGO_URL = \"https://www.runcell.dev/runcell-logo.svg\";\nconst RUNCELL_WEBSITE = \"https://www.runcell.dev?utm_source=pygwalker\";\n\nconst LLM_LOGOS = [\n \"https://www.runcell.dev/llm-icons/openai.svg\",\n \"https://www.runcell.dev/llm-icons/claude-color.svg\",\n \"https://www.runcell.dev/llm-icons/gemini-color.svg\",\n \"https://www.runcell.dev/llm-icons/deepseek-color.svg\",\n];\n\ninterface RuncellBannerProps {\n env?: string;\n}\n\nconst checkRuncellInstalled = (): boolean => {\n try {\n // Runcell JupyterLab plugin stores user status in localStorage\n const runcellUser = window.parent.localStorage.getItem(\"plugin_auth_user_v2\");\n return runcellUser !== null;\n } catch {\n return false;\n }\n};\n\nexport const RuncellBanner: React.FC = ({ env }) => {\n const [dismissed, setDismissed] = useState(false);\n\n // Only show in Jupyter environments\n if (env !== \"jupyter_widgets\" && env !== \"anywidget\") {\n return null;\n }\n\n // Don't show if runcell is installed or user dismissed\n if (checkRuncellInstalled() || dismissed) {\n return null;\n }\n\n const handleClick = () => {\n tracker.track(\"click\", { entity: \"runcell_banner\" });\n window.open(RUNCELL_WEBSITE, \"_blank\");\n };\n\n const handleDismiss = (e: React.MouseEvent) => {\n e.stopPropagation();\n tracker.track(\"click\", { entity: \"runcell_banner_dismiss\" });\n setDismissed(true);\n };\n\n return (\n \n
\n \n \n Enable AI Agent for data analysis with pip install runcell\n \n
\n
\n
\n {LLM_LOGOS.map((logo, index) => (\n \n ))}\n
\n \n ×\n \n
\n \n );\n};\n\nexport default RuncellBanner;\n" }, { "path": "app/src/components/ui/badge.tsx", "content": "import * as React from \"react\"\nimport { cva, type VariantProps } from \"class-variance-authority\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst badgeVariants = cva(\n \"inline-flex items-center rounded-md border px-2.5 py-0.5 text-xs font-semibold transition-colors focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2\",\n {\n variants: {\n variant: {\n default:\n \"border-transparent bg-primary text-primary-foreground shadow hover:bg-primary/80\",\n secondary:\n \"border-transparent bg-secondary text-secondary-foreground hover:bg-secondary/80\",\n destructive:\n \"border-transparent bg-destructive text-destructive-foreground shadow hover:bg-destructive/80\",\n outline: \"text-foreground\",\n },\n },\n defaultVariants: {\n variant: \"default\",\n },\n }\n)\n\nexport interface BadgeProps\n extends React.HTMLAttributes,\n VariantProps {}\n\nfunction Badge({ className, variant, ...props }: BadgeProps) {\n return (\n
\n )\n}\n\nexport { Badge, badgeVariants }\n" }, { "path": "app/src/components/ui/button.tsx", "content": "import * as React from \"react\"\nimport { Slot } from \"@radix-ui/react-slot\"\nimport { cva, type VariantProps } from \"class-variance-authority\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst buttonVariants = cva(\n \"inline-flex items-center justify-center whitespace-nowrap rounded-md text-sm font-medium transition-colors focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-ring disabled:pointer-events-none disabled:opacity-50\",\n {\n variants: {\n variant: {\n default:\n \"bg-primary text-primary-foreground shadow hover:bg-primary/90\",\n destructive:\n \"bg-destructive text-destructive-foreground shadow-sm hover:bg-destructive/90\",\n outline:\n \"border border-input bg-background shadow-sm hover:bg-accent hover:text-accent-foreground\",\n secondary:\n \"bg-secondary text-secondary-foreground shadow-sm hover:bg-secondary/80\",\n ghost: \"hover:bg-accent hover:text-accent-foreground\",\n link: \"text-primary underline-offset-4 hover:underline\",\n },\n size: {\n default: \"h-9 px-4 py-2\",\n sm: \"h-8 rounded-md px-3 text-xs\",\n lg: \"h-10 rounded-md px-8\",\n icon: \"h-9 w-9\",\n },\n },\n defaultVariants: {\n variant: \"default\",\n size: \"default\",\n },\n }\n)\n\nexport interface ButtonProps\n extends React.ButtonHTMLAttributes,\n VariantProps {\n asChild?: boolean\n}\n\nconst Button = React.forwardRef(\n ({ className, variant, size, asChild = false, ...props }, ref) => {\n const Comp = asChild ? Slot : \"button\"\n return (\n \n )\n }\n)\nButton.displayName = \"Button\"\n\nexport { Button, buttonVariants }\n" }, { "path": "app/src/components/ui/checkbox.tsx", "content": "import * as React from \"react\"\nimport * as CheckboxPrimitive from \"@radix-ui/react-checkbox\"\nimport { CheckIcon } from \"@radix-ui/react-icons\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst Checkbox = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n \n \n \n \n))\nCheckbox.displayName = CheckboxPrimitive.Root.displayName\n\nexport { Checkbox }\n" }, { "path": "app/src/components/ui/dialog.tsx", "content": "import * as React from \"react\"\nimport * as DialogPrimitive from \"@radix-ui/react-dialog\"\nimport { Cross2Icon } from \"@radix-ui/react-icons\"\n\nimport { cn } from \"@/lib/utils\"\nimport { portalContainerContext } from \"@/store/context\"\n\nconst Dialog = DialogPrimitive.Root\n\nconst DialogTrigger = DialogPrimitive.Trigger\n\nconst DialogPortal = DialogPrimitive.Portal\n\nconst DialogClose = DialogPrimitive.Close\n\nconst DialogOverlay = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nDialogOverlay.displayName = DialogPrimitive.Overlay.displayName\n\nconst DialogContent = React.forwardRef<\n React.ElementRef,\n { hideClose?: boolean } & React.ComponentPropsWithoutRef\n>(({ className, children, hideClose, ...props }, ref) => (\n \n \n \n {children}\n { !hideClose && (\n \n \n Close\n \n )}\n \n \n))\nDialogContent.displayName = DialogPrimitive.Content.displayName\n\nconst DialogHeader = ({\n className,\n ...props\n}: React.HTMLAttributes) => (\n \n)\nDialogHeader.displayName = \"DialogHeader\"\n\nconst DialogFooter = ({\n className,\n ...props\n}: React.HTMLAttributes) => (\n \n)\nDialogFooter.displayName = \"DialogFooter\"\n\nconst DialogTitle = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nDialogTitle.displayName = DialogPrimitive.Title.displayName\n\nconst DialogDescription = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nDialogDescription.displayName = DialogPrimitive.Description.displayName\n\nexport {\n Dialog,\n DialogPortal,\n DialogOverlay,\n DialogTrigger,\n DialogClose,\n DialogContent,\n DialogHeader,\n DialogFooter,\n DialogTitle,\n DialogDescription,\n}\n" }, { "path": "app/src/components/ui/input.tsx", "content": "import * as React from \"react\"\n\nimport { cn } from \"@/lib/utils\"\n\nexport interface InputProps\n extends React.InputHTMLAttributes {}\n\nconst Input = React.forwardRef(\n ({ className, type, ...props }, ref) => {\n return (\n \n )\n }\n)\nInput.displayName = \"Input\"\n\nexport { Input }\n" }, { "path": "app/src/components/ui/label.tsx", "content": "import * as React from \"react\"\nimport * as LabelPrimitive from \"@radix-ui/react-label\"\nimport { cva, type VariantProps } from \"class-variance-authority\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst labelVariants = cva(\n \"text-sm font-medium leading-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70\"\n)\n\nconst Label = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef &\n VariantProps\n>(({ className, ...props }, ref) => (\n \n))\nLabel.displayName = LabelPrimitive.Root.displayName\n\nexport { Label }\n" }, { "path": "app/src/components/ui/select.tsx", "content": "import * as React from \"react\"\nimport {\n CaretSortIcon,\n CheckIcon,\n ChevronDownIcon,\n ChevronUpIcon,\n} from \"@radix-ui/react-icons\"\nimport * as SelectPrimitive from \"@radix-ui/react-select\"\n\nimport { cn } from \"@/lib/utils\"\nimport { portalContainerContext } from \"@/store/context\"\n\nconst Select = SelectPrimitive.Root\n\nconst SelectGroup = SelectPrimitive.Group\n\nconst SelectValue = SelectPrimitive.Value\n\nconst SelectTrigger = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, children, ...props }, ref) => (\n span]:line-clamp-1\",\n className\n )}\n {...props}\n >\n {children}\n \n \n \n \n))\nSelectTrigger.displayName = SelectPrimitive.Trigger.displayName\n\nconst SelectScrollUpButton = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n \n \n))\nSelectScrollUpButton.displayName = SelectPrimitive.ScrollUpButton.displayName\n\nconst SelectScrollDownButton = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n \n \n))\nSelectScrollDownButton.displayName =\n SelectPrimitive.ScrollDownButton.displayName\n\nconst SelectContent = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, children, position = \"popper\", ...props }, ref) => (\n \n \n \n \n {children}\n \n \n \n \n))\nSelectContent.displayName = SelectPrimitive.Content.displayName\n\nconst SelectLabel = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nSelectLabel.displayName = SelectPrimitive.Label.displayName\n\nconst SelectItem = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, children, ...props }, ref) => (\n \n \n \n \n \n \n {children}\n \n))\nSelectItem.displayName = SelectPrimitive.Item.displayName\n\nconst SelectSeparator = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nSelectSeparator.displayName = SelectPrimitive.Separator.displayName\n\nexport {\n Select,\n SelectGroup,\n SelectValue,\n SelectTrigger,\n SelectContent,\n SelectLabel,\n SelectItem,\n SelectSeparator,\n SelectScrollUpButton,\n SelectScrollDownButton,\n}\n" }, { "path": "app/src/components/ui/tabs.tsx", "content": "import * as React from \"react\"\nimport * as TabsPrimitive from \"@radix-ui/react-tabs\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst Tabs = TabsPrimitive.Root\n\nconst TabsList = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nTabsList.displayName = TabsPrimitive.List.displayName\n\nconst TabsTrigger = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nTabsTrigger.displayName = TabsPrimitive.Trigger.displayName\n\nconst TabsContent = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef\n>(({ className, ...props }, ref) => (\n \n))\nTabsContent.displayName = TabsPrimitive.Content.displayName\n\nexport { Tabs, TabsList, TabsTrigger, TabsContent }\n" }, { "path": "app/src/components/ui/toggle-group.tsx", "content": "import * as React from \"react\"\nimport * as ToggleGroupPrimitive from \"@radix-ui/react-toggle-group\"\nimport { VariantProps } from \"class-variance-authority\"\n\nimport { cn } from \"@/lib/utils\"\nimport { toggleVariants } from \"@/components/ui/toggle\"\n\nconst ToggleGroupContext = React.createContext<\n VariantProps\n>({\n size: \"default\",\n variant: \"default\",\n})\n\nconst ToggleGroup = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef &\n VariantProps\n>(({ className, variant, size, children, ...props }, ref) => (\n \n \n {children}\n \n \n))\n\nToggleGroup.displayName = ToggleGroupPrimitive.Root.displayName\n\nconst ToggleGroupItem = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef &\n VariantProps\n>(({ className, children, variant, size, ...props }, ref) => {\n const context = React.useContext(ToggleGroupContext)\n\n return (\n \n {children}\n \n )\n})\n\nToggleGroupItem.displayName = ToggleGroupPrimitive.Item.displayName\n\nexport { ToggleGroup, ToggleGroupItem }\n" }, { "path": "app/src/components/ui/toggle.tsx", "content": "import * as React from \"react\"\nimport * as TogglePrimitive from \"@radix-ui/react-toggle\"\nimport { cva, type VariantProps } from \"class-variance-authority\"\n\nimport { cn } from \"@/lib/utils\"\n\nconst toggleVariants = cva(\n \"inline-flex items-center justify-center rounded-md text-sm font-medium transition-colors hover:bg-muted hover:text-muted-foreground focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-ring disabled:pointer-events-none disabled:opacity-50 data-[state=on]:bg-accent data-[state=on]:text-accent-foreground\",\n {\n variants: {\n variant: {\n default: \"bg-transparent\",\n outline:\n \"border border-input bg-transparent shadow-sm hover:bg-accent hover:text-accent-foreground\",\n },\n size: {\n default: \"h-9 px-3\",\n sm: \"h-8 px-2\",\n lg: \"h-10 px-3\",\n },\n },\n defaultVariants: {\n variant: \"default\",\n size: \"default\",\n },\n }\n)\n\nconst Toggle = React.forwardRef<\n React.ElementRef,\n React.ComponentPropsWithoutRef &\n VariantProps\n>(({ className, variant, size, ...props }, ref) => (\n \n))\n\nToggle.displayName = TogglePrimitive.Root.displayName\n\nexport { Toggle, toggleVariants }\n" }, { "path": "app/src/components/uploadChartModal/index.tsx", "content": "import React, { useState, useEffect } from \"react\";\nimport { observer } from \"mobx-react-lite\";\nimport type { IGWHandler } from \"@kanaries/graphic-walker/interfaces\";\nimport type { VizSpecStore } from '@kanaries/graphic-walker/store/visualSpecStore'\nimport { chartToWorkflow } from \"@kanaries/graphic-walker\"\nimport { tracker } from \"@/utils/tracker\";\n\nimport communicationStore from \"../../store/communication\";\nimport commonStore from \"../../store/common\";\nimport { Button } from \"@/components/ui/button\";\nimport { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from \"@/components/ui/dialog\";\nimport { Input } from \"@/components/ui/input\";\nimport { Label } from \"@/components/ui/label\";\nimport { Checkbox } from \"@/components/ui/checkbox\";\n\ninterface IUploadChartModal {\n gwRef: React.MutableRefObject;\n storeRef: React.MutableRefObject;\n dark: string;\n}\n\nconst UploadChartModal: React.FC = observer((props) => {\n const [uploading, setUploading] = useState(false);\n const [chartName, setChartName] = useState(\"\");\n const [datasetName, setDatasetName] = useState(\"\");\n const [isPublic, setIsPublic] = useState(true);\n const [isCreateDashboard, setIsCreateDashboard] = useState(true);\n const [instanceType, setInstanceType] = useState(\"\");\n\n useEffect(() => {\n if (commonStore.uploadChartModalOpen) {\n const instanceType = (props.storeRef.current?.exportCode().length || 0) > 1 ? \"dashboard\" : \"chart\";\n setChartName(`${instanceType}-${new Date().getTime().toString(16).padStart(16, \"0\")}`);\n setDatasetName(`dataset-${new Date().getTime().toString(16).padStart(16, \"0\")}`);\n setIsPublic(true);\n setInstanceType(instanceType);\n }\n }, [commonStore.uploadChartModalOpen])\n\n const uploadSuccess = (instanceType: string, instanceId: string, datasetId: string) => {\n const managerUrl = instanceType === \"chart\" ? `https://kanaries.net/analytics/c/${instanceId}` : `https://kanaries.net/analytics/d/${instanceId}`\n const shareUrl = instanceType === \"chart\" ? `https://kanaries.net/analytics/chart/${instanceId}/share?theme=${props.dark}` : `https://kanaries.net/analytics/dashboard/${instanceId}/share?theme=${props.dark}`\n const datsetUrl = `https://kanaries.net/analytics/detail/${datasetId}`\n if (instanceType === \"dashboard\" && instanceId === \"\" ) {\n commonStore.setNotification(\n {\n type: \"success\",\n title: \"Success\",\n message: (\n <>\n

Upload success, you can view and manager it at:\n \n {datsetUrl}\n \n

\n \n ),\n },\n 30_000\n );\n } else {\n commonStore.setNotification(\n {\n type: \"success\",\n title: \"Success\",\n message: (\n <>\n

Upload success, you can view and manager it at:\n \n {managerUrl}\n \n

\n
\n {isPublic &&

Since you set the {instanceType} to public, you can also share it with others by:\n \n {shareUrl}\n \n

}\n \n ),\n },\n 30_000\n );\n }\n };\n\n const onClick = async () => {\n if (uploading) return;\n setUploading(true);\n tracker.track(\"click\", {\"entity\": \"upload_chart_button\"});\n\n const visSpec = props.storeRef.current?.exportCode()!;\n try {\n if (instanceType === \"dashboard\") {\n const resp = await communicationStore.comm?.sendMsg(\n \"upload_to_cloud_dashboard\",\n {\n chartName: chartName,\n datasetName: datasetName,\n isPublic: isPublic,\n isCreateDashboard: isCreateDashboard,\n visSpec: visSpec,\n workflowList: visSpec.map(spec => chartToWorkflow(spec).workflow),\n },\n 120_000\n );\n uploadSuccess(instanceType, resp?.data.dashboardId, resp?.data.datasetId);\n } else {\n const resp = await communicationStore.comm?.sendMsg(\n \"upload_to_cloud_charts\",\n {\n chartName: chartName,\n datasetName: datasetName,\n isPublic: isPublic,\n visSpec: visSpec,\n workflow: chartToWorkflow(visSpec[0]).workflow,\n },\n 120_000\n );\n uploadSuccess(instanceType, resp?.data.chartId, resp?.data.datasetId);\n }\n commonStore.setUploadChartModalOpen(false);\n } finally {\n setUploading(false);\n }\n };\n\n return (\n {\n commonStore.setUploadChartModalOpen(show)\n }}\n >\n \n \n Upload Chart\n \n upload your charts to dashboard of kanaries cloud.\n \n \n
\n
\n
\n \n {\n setDatasetName(e.target.value);\n }}\n type=\"text\"\n placeholder=\"please input dataset name\"\n id=\"dataset-name-input\"\n className=\"mb-1\"\n />\n
\n
\n \n {\n setChartName(e.target.value);\n }}\n disabled={!isCreateDashboard}\n type=\"text\"\n placeholder=\"please input chart name\"\n id=\"chart-name-input\"\n className=\"mb-1\"\n />\n
\n
\n {setIsPublic(checked as boolean);}}\n />\n
\n \n

\n Make {instanceType} publicly accessible.\n

\n
\n
\n {instanceType === \"dashboard\" && (\n
\n {setIsCreateDashboard(checked as boolean);}}\n />\n
\n \n

\n Create a dashboard containing all charts, or upload charts individually.\n

\n
\n
\n )}\n
\n
\n \n
\n
\n \n \n );\n});\n\nexport default UploadChartModal;\n" }, { "path": "app/src/components/uploadSpecModal/index.tsx", "content": "import React, { useEffect, useState } from \"react\";\nimport { observer } from \"mobx-react-lite\";\nimport type { VizSpecStore } from '@kanaries/graphic-walker/store/visualSpecStore'\nimport { chartToWorkflow } from \"@kanaries/graphic-walker/utils/workflow\";\nimport { tracker } from \"@/utils/tracker\";\n\nimport communicationStore from \"../../store/communication\";\nimport commonStore from \"../../store/common\";\nimport { Button } from \"@/components/ui/button\";\nimport { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from \"@/components/ui/dialog\";\nimport { Input } from \"@/components/ui/input\";\nimport { Label } from \"@/components/ui/label\";\nimport { Checkbox } from \"@/components/ui/checkbox\";\nimport { badgeVariants } from \"@/components/ui/badge\";\n\ninterface IUploadSpecModal {\n setGwIsChanged: React.Dispatch>;\n storeRef: React.MutableRefObject;\n}\n\nconst UploadSpecModal: React.FC = observer((props) => {\n const [uploading, setUploading] = useState(false);\n const [specName, setSpecName] = useState(\"\");\n const [isSetToken, setIsSetToken] = useState(false);\n const [token, setToken] = useState(\"\");\n const [contentType, setContentType] = useState<\"onboarding\" | \"upload\">(\"onboarding\");\n\n const uploadSuccess = (path: string) => {\n commonStore.setNotification(\n {\n type: \"success\",\n title: \"Success\",\n message: (\n <>\n

upload spec success, please use

\n

`pyg.walk(df, spec=\"ksf://{path}\")`

\n

to rerun pygwalker.

\n \n ),\n },\n 30_000\n );\n };\n\n const onClick = async () => {\n if (uploading) return;\n tracker.track(\"click\", {\"entity\": \"upload_spec_to_cloud_button\"});\n setUploading(true);\n\n try {\n\n const resp = await communicationStore.comm?.sendMsg(\n \"upload_spec_to_cloud\",\n {\"fileName\": specName, \"newToken\": isSetToken ? token : \"\"},\n 30_000\n );\n commonStore.setUploadSpecModalOpen(false);\n uploadSuccess(resp?.data[\"specFilePath\"]);\n props.setGwIsChanged(false);\n } finally {\n setUploading(false);\n }\n };\n\n const onClickSetToken = (checked: boolean) => {\n setIsSetToken(checked);\n setToken(\"\");\n }\n\n const saveSpecToLocal = () => {\n tracker.track(\"click\", {\"entity\": \"save_spec_to_local_file_button\"});\n const visSpec = props.storeRef.current?.exportCode();\n const configObj = {\n config: visSpec,\n chart_map: {},\n version: commonStore.version,\n workflow_list: visSpec?.map(spec => chartToWorkflow(spec).workflow),\n }\n const blob = new Blob([JSON.stringify(configObj)], {type: \"text/plain;charset=utf-8\"});\n const url = URL.createObjectURL(blob);\n const tempLink = document.createElement(\"a\");\n tempLink.href = url;\n tempLink.download = `pygwalker_spec_${new Date().getTime()}.json`\n tempLink.click();\n URL.revokeObjectURL(url);\n commonStore.setUploadSpecModalOpen(false);\n props.setGwIsChanged(false);\n };\n\n useEffect(() => {\n setContentType(\"onboarding\");\n }, [commonStore.uploadSpecModalOpen]);\n\n const OnboardingContent = (\n \n \n Save Spec\n \n
\n {\n setContentType(\"upload\");\n tracker.track(\"click\", {\"entity\": \"select_upload_spec_to_cloud_button\"});\n }}\n >\n
\n upload as cloud file\n
\n \n \n
\n save as local file\n
\n \n
\n \n )\n\n const UpdateSpecContent = (\n \n \n Upload Sepc\n \n

Because you currently don't pass in the spec parameter or the passed in spec parameter is not a writable spec file.

\n

Currently the spec configuration is already in your ui cache, you may need to upload kanaries cloud to save it.

\n

If you don't have kanaries_token, you need to get it: Kanaries

\n \n \n
\n
\n {\n setSpecName(e.target.value);\n }}\n type=\"text\"\n placeholder=\"please input spec file name\"\n id=\"chart-name-input\"\n className=\"mb-1\"\n />\n
\n
\n {\n onClickSetToken(checked as boolean);\n }}\n />\n \n
\n {\n isSetToken && (\n
\n {\n setToken(e.target.value);\n }}\n type=\"text\"\n autoComplete=\"off\"\n placeholder=\"please input new kanaries token\"\n id=\"token-input\"\n />\n
\n )\n }\n
\n \n
\n
\n \n )\n\n return (\n {\n commonStore.setUploadSpecModalOpen(show);\n }}\n >\n {contentType === \"upload\" && UpdateSpecContent }\n {contentType === \"onboarding\" && OnboardingContent }\n \n );\n});\n\nexport default UploadSpecModal;\n" }, { "path": "app/src/dataSource/index.tsx", "content": "import type { IDataSourceProps } from \"../interfaces\";\nimport type { IRow, IDataQueryPayload } from \"@kanaries/graphic-walker/interfaces\";\nimport commonStore from \"../store/common\";\nimport communicationStore from \"../store/communication\"\nimport { parser_dsl_with_meta } from \"@kanaries/gw-dsl-parser\";\n\ninterface MessagePayload extends IDataSourceProps {\n action: \"requestData\" | \"postData\" | \"finishData\";\n dataSourceId: string;\n partId: number;\n curIndex: number;\n total: number;\n data?: IRow[];\n}\n\ninterface ICommPostDataMessage {\n dataSourceId: string;\n data?: IRow[];\n total: number;\n curIndex: number;\n}\n\nexport async function loadDataSource(props: IDataSourceProps): Promise {\n const { dataSourceId } = props;\n\n return new Promise((resolve, reject) => {\n const data = new Array();\n const timeout = () => {\n reject(\"timeout\");\n };\n let timer = setTimeout(timeout, 100_000);\n const onmessage = (ev: MessageEvent) => {\n try {\n if (ev.data.dataSourceId === dataSourceId) {\n clearTimeout(timer);\n timer = setTimeout(timeout, 100_000);\n if (ev.data.action === \"postData\") {\n commonStore.setInitModalOpen(true);\n commonStore.setInitModalInfo({\n total: ev.data.total,\n curIndex: ev.data.curIndex,\n title: \"Loading Data\",\n });\n data.push(...(ev.data.data ?? []));\n } else if (ev.data.action === \"finishData\") {\n window.removeEventListener(\"message\", onmessage);\n resolve(data);\n }\n }\n } catch (err) {\n reject({ message: \"handler\", error: err });\n }\n };\n window.addEventListener(\"message\", onmessage);\n });\n}\n\nexport function postDataService(msg: ICommPostDataMessage) {\n window.postMessage(\n {\n action: \"postData\",\n dataSourceId: msg.dataSourceId,\n total: msg.total,\n curIndex: msg.curIndex,\n data: msg.data,\n } as MessagePayload,\n \"*\"\n )\n}\n\nexport function finishDataService(msg: any) {\n window.postMessage(\n {\n action: \"finishData\",\n dataSourceId: msg.dataSourceId,\n } as MessagePayload,\n \"*\"\n )\n}\n\ninterface IBatchGetDatasTask {\n query: any;\n resolve: (value: any) => void;\n reject: (reason?: any) => void;\n}\n\nfunction initBatchGetDatas(action: string) {\n const taskList = [] as IBatchGetDatasTask[];\n\n const batchGetDatas = async(taskList: IBatchGetDatasTask[]) => {\n const result = await communicationStore.comm?.sendMsg(\n action,\n {\"queryList\": taskList.map(task => task.query)},\n 60_000\n );\n if (result) {\n for (let i = 0; i < taskList.length; i++) {\n taskList[i].resolve(result[\"data\"][\"datas\"][i]);\n }\n } else {\n for (let i = 0; i < taskList.length; i++) {\n taskList[i].reject(\"get result error\");\n }\n }\n }\n\n const getDatas = (query: any) => {\n return new Promise((resolve, reject) => {\n taskList.push({ query, resolve, reject });\n if (taskList.length === 1) {\n setTimeout(() => {\n batchGetDatas(taskList.splice(0, taskList.length));\n }, 100);\n }\n })\n }\n\n return {\n getDatas\n }\n}\n\nconst batchGetDatasBySql = initBatchGetDatas(\"batch_get_datas_by_sql\");\nconst batchGetDatasByPayload = initBatchGetDatas(\"batch_get_datas_by_payload\");\nconst DEFAULT_LIMIT = 50_000;\n\nfunction notifyDataLimit() {\n commonStore.setNotification({\n type: \"warning\",\n title: \"Data Limit Reached\",\n message: (<>\n The current computation has returned more than 50,000 rows of data. \n To ensure optimal performance, we are currently rendering only the first 50,000 rows. \n If you need to render the entire all datas, please use the 'limit' tool \n (\n \n ) to manually set the maximum number of rows to be returned.\n )\n }, 60_000);\n}\n\nexport function getDatasFromKernelBySql(fieldMetas: any) {\n return async (payload: IDataQueryPayload) => {\n const sql = parser_dsl_with_meta(\n \"pygwalker_mid_table\",\n JSON.stringify({...payload, limit: payload.limit ?? DEFAULT_LIMIT}),\n JSON.stringify({\"pygwalker_mid_table\": fieldMetas})\n );\n const result = await batchGetDatasBySql.getDatas(sql) ?? [];\n if (!payload.limit && result.length === DEFAULT_LIMIT) {\n notifyDataLimit();\n }\n return result as IRow[];\n }\n}\n\nexport async function getDatasFromKernelByPayload(payload: IDataQueryPayload) {\n const result = await batchGetDatasByPayload.getDatas({...payload, limit: payload.limit ?? DEFAULT_LIMIT}) ?? [];\n if (!payload.limit && result.length === DEFAULT_LIMIT) {\n notifyDataLimit();\n }\n return result as IRow[];\n}\n" }, { "path": "app/src/index.css", "content": "@tailwind base;\n@tailwind components;\n@tailwind utilities;\n\n@layer base {\n :root {\n --background: 0 0% 100%;\n --foreground: 240 10% 3.9%;\n \n --card: 0 0% 100%;\n --card-foreground: 240 10% 3.9%;\n \n --popover: 0 0% 100%;\n --popover-foreground: 240 10% 3.9%;\n \n --primary: 240 5.9% 10%;\n --primary-foreground: 0 0% 98%;\n \n --secondary: 240 4.8% 95.9%;\n --secondary-foreground: 240 5.9% 10%;\n \n --muted: 240 4.8% 95.9%;\n --muted-foreground: 240 3.8% 46.1%;\n \n --accent: 240 4.8% 95.9%;\n --accent-foreground: 240 5.9% 10%;\n \n --destructive: 0 84.2% 60.2%;\n --destructive-foreground: 0 0% 98%;\n \n --border: 240 5.9% 90%;\n --input: 240 5.9% 90%;\n --ring: 240 10% 3.9%;\n \n --radius: 0.5rem;\n }\n \n .dark {\n --background: 240 10% 3.9%;\n --foreground: 0 0% 98%;\n \n --card: 240 10% 3.9%;\n --card-foreground: 0 0% 98%;\n \n --popover: 240 10% 3.9%;\n --popover-foreground: 0 0% 98%;\n \n --primary: 0 0% 98%;\n --primary-foreground: 240 5.9% 10%;\n \n --secondary: 240 3.7% 15.9;\n --secondary-foreground: 0 0% 98%;\n \n --muted: 240 3.7% 15.9%;\n --muted-foreground: 240 5% 64.9%;\n \n --accent: 240 3.7% 15.9%;\n --accent-foreground: 0 0% 98%;\n \n --destructive: 0 62.8% 30.6%;\n --destructive-foreground: 0 0% 98%;\n \n --border: 240 3.7% 15.9%;\n --input: 240 3.7% 15.9%;\n --ring: 240 4.9% 83.9%;\n }\n}\n\n" }, { "path": "app/src/index.tsx", "content": "import React, { useCallback, useContext, useEffect, useState } from 'react';\nimport { createRoot } from 'react-dom/client';\nimport { observer } from \"mobx-react-lite\";\nimport { reaction } from \"mobx\"\nimport { GraphicWalker, PureRenderer, GraphicRenderer, TableWalker } from '@kanaries/graphic-walker'\nimport type { VizSpecStore } from '@kanaries/graphic-walker/store/visualSpecStore'\nimport type { IGWHandler, IViewField, ISegmentKey, IDarkMode, IChatMessage, IRow } from '@kanaries/graphic-walker/interfaces';\nimport { Tabs, TabsContent, TabsList, TabsTrigger } from \"@/components/ui/tabs\";\nimport { Streamlit, withStreamlitConnection } from \"streamlit-component-lib\"\nimport { createRender, useModel } from \"@anywidget/react\";\n\nimport Options from './components/options';\nimport { IAppProps } from './interfaces';\n\nimport { loadDataSource, postDataService, finishDataService, getDatasFromKernelBySql, getDatasFromKernelByPayload } from './dataSource';\n\nimport commonStore from \"./store/common\";\nimport { initJupyterCommunication, initHttpCommunication, streamlitComponentCallback, initAnywidgetCommunication } from \"./utils/communication\";\nimport communicationStore from \"./store/communication\"\nimport { setConfig } from './utils/userConfig';\nimport CodeExportModal from './components/codeExportModal';\nimport type { IPreviewProps, IChartPreviewProps } from './components/preview';\nimport { Preview, ChartPreview } from './components/preview';\nimport UploadSpecModal from \"./components/uploadSpecModal\"\nimport UploadChartModal from './components/uploadChartModal';\nimport InitModal from './components/initModal';\nimport { getSaveTool } from './tools/saveTool';\nimport { getExportTool } from './tools/exportTool';\nimport { getExportDataframeTool } from './tools/exportDataframe';\nimport { getRuncellTool } from './tools/runcellTool';\nimport { formatExportedChartDatas } from \"./utils/save\";\nimport { tracker } from \"@/utils/tracker\";\nimport Notification from \"./notify\"\nimport initDslParser from \"@kanaries/gw-dsl-parser\";\nimport {\n Select,\n SelectContent,\n SelectItem,\n SelectTrigger,\n SelectValue,\n} from \"@/components/ui/select\"\nimport {\n ToggleGroup,\n ToggleGroupItem,\n} from \"@/components/ui/toggle-group\"\nimport { SunIcon, MoonIcon, DesktopIcon, ChevronLeftIcon, ChevronRightIcon } from \"@radix-ui/react-icons\"\n\n// @ts-ignore\nimport style from './index.css?inline'\nimport { currentMediaTheme } from './utils/theme';\nimport { AppContext, darkModeContext } from './store/context';\nimport FormatSpec from './utils/formatSpec';\nimport { getOpenDesktopTool } from './tools/openDesktop';\nimport RuncellBanner from './components/runcellBanner';\n\n\nconst initChart = async (gwRef: React.MutableRefObject, total: number, props: IAppProps) => {\n if (props.needInitChart && props.env === \"jupyter_widgets\" && total !== 0) {\n commonStore.setInitModalOpen(true);\n commonStore.setInitModalInfo({\n title: \"Recover Charts\",\n curIndex: 0,\n total: total,\n });\n for await (const chart of gwRef.current?.exportChartList(\"data-url\")!) {\n await communicationStore.comm?.sendMsg(\"save_chart\", await formatExportedChartDatas(chart.data));\n commonStore.setInitModalInfo({\n title: \"Recover Charts\",\n curIndex: chart.index + 1,\n total: chart.total,\n });\n }\n }\n commonStore.setInitModalOpen(false);\n}\n\nconst getComputationCallback = (props: IAppProps) => {\n if (props.useKernelCalc && props.parseDslType === \"client\") {\n return getDatasFromKernelBySql(props.fieldMetas);\n }\n if (props.useKernelCalc && props.parseDslType === \"server\") {\n return getDatasFromKernelByPayload;\n }\n}\n\nconst MainApp = observer((props: {children: React.ReactNode, darkMode: \"dark\" | \"light\" | \"media\", hideToolBar?: boolean, gid?: string, sendMessage?: boolean}) => {\n const [portal, setPortal] = useState(null);\n const [selectedDarkMode, setSelectedDarkMode] = useState(props.darkMode);\n const [darkMode, setDarkMode] = useState(currentMediaTheme(props.darkMode));\n\n const sendAppearanceMessageToParent = useCallback((appearance: IDarkMode) => {\n if (!props.sendMessage) return;\n window.parent.postMessage({\n action: \"changeAppearance\",\n gid: props.gid,\n appearance: appearance\n }, \"*\");\n }, [props.gid, props.sendMessage]);\n\n useEffect(() => {\n if (selectedDarkMode === \"media\") {\n setDarkMode(currentMediaTheme(selectedDarkMode));\n sendAppearanceMessageToParent(currentMediaTheme(selectedDarkMode));\n const media = window.matchMedia('(prefers-color-scheme: dark)');\n const listener = (e: MediaQueryListEvent) => {\n setDarkMode(e.matches ? \"dark\" : \"light\");\n }\n media.addEventListener(\"change\", listener);\n return () => media.removeEventListener(\"change\", listener);\n } else {\n sendAppearanceMessageToParent(selectedDarkMode);\n setDarkMode(selectedDarkMode);\n }\n }, [selectedDarkMode])\n\n return (\n \n
\n
\n \n
\n { props.children }\n
\n {!props.hideToolBar && (\n
\n {value && setSelectedDarkMode(value as IDarkMode)}}\n >\n \n \n \n \n \n \n \n \n \n \n
\n )}\n \n
\n
\n
\n \n )\n})\n\nconst ExploreApp: React.FC = (props) => {\n const gwRef = React.useRef(null);\n const { userConfig } = props;\n const [exportOpen, setExportOpen] = useState(false);\n const [mode, setMode] = useState(\"walker\");\n const [visSpec, setVisSpec] = useState(props.visSpec);\n const [hideModeOption, _] = useState(true);\n const [isChanged, setIsChanged] = useState(false);\n const storeRef = React.useRef(null);\n const disposerRef = React.useRef<() => void>();\n const storeRefProxied = React.useMemo(\n () =>\n new Proxy(storeRef, {\n set(target, prop, value) {\n if (prop === \"current\") {\n if (value) {\n disposerRef.current?.();\n const store = value as VizSpecStore;\n disposerRef.current = reaction(\n () => store.currentVis,\n () => {\n setIsChanged((value as VizSpecStore).canUndo);\n streamlitComponentCallback({\n event: \"spec_change\",\n data: store.exportCode()\n });\n },\n );\n }\n }\n return Reflect.set(target, prop, value);\n },\n }),\n []\n );\n\n commonStore.setVersion(props.version!);\n\n useEffect(() => {\n commonStore.setShowCloudTool(props.showCloudTool);\n tracker.setUserId(props.hashcode ?? \"\");\n if (userConfig) {\n setConfig(userConfig);\n tracker.setOpen(userConfig.privacy === \"events\");\n };\n }, []);\n\n useEffect(() => {\n if (props.initChartFlag) {\n setTimeout(() => { initChart(gwRef, visSpec.length, props) }, 0);\n }\n }, [props.initChartFlag]);\n\n useEffect(() => {\n setTimeout(() => {\n storeRef.current?.setSegmentKey(props.defaultTab as ISegmentKey);\n }, 0);\n }, [mode]);\n\n const runcellTool = getRuncellTool();\n const exportTool = getExportTool(setExportOpen);\n const openInDesktopTool = getOpenDesktopTool(props, storeRef);\n\n const tools = [runcellTool, exportTool, openInDesktopTool];\n if (props.env && [\"jupyter_widgets\", \"streamlit\", \"gradio\", \"marimo\", \"anywidget\", \"web_server\"].indexOf(props.env) !== -1 && props.useSaveTool) {\n const saveTool = getSaveTool(props, gwRef, storeRef, isChanged, setIsChanged);\n tools.push(saveTool);\n }\n if (props.isExportDataFrame) {\n const exportDataFrameTool = getExportDataframeTool(props, storeRef);\n tools.push(exportDataFrameTool);\n }\n\n const toolbarConfig = {\n exclude: [\"export_code\"],\n extra: tools\n }\n\n const enhanceAPI = React.useMemo(() => {\n if (props.showCloudTool) {\n const features: Record = {};\n if (props.enableAskViz) {\n features[\"askviz\"] = async (metas: IViewField[], query: string) => {\n const resp = await communicationStore.comm?.sendMsg(\"get_spec_by_text\", { metas, query });\n return resp?.data.data;\n };\n }\n if (props.enableVlChat) {\n features[\"vlChat\"] = async (metas: IViewField[], chats: IChatMessage[]) => {\n const resp = await communicationStore.comm?.sendMsg(\"get_chart_by_chats\", { metas, chats });\n return resp?.data.data;\n };\n }\n if (Object.keys(features).length > 0) {\n return { features };\n }\n }\n return undefined;\n }, [props.showCloudTool, props.enableAskViz, props.enableVlChat]);\n\n const computationCallback = React.useMemo(() => getComputationCallback(props), []);\n\n const modeChange = (value: string) => {\n if (mode === \"walker\") {\n setVisSpec(storeRef.current?.exportCode());\n }\n setMode(value);\n }\n \n return (\n \n \n \n \n \n {\n !hideModeOption &&\n \n }\n {\n mode === \"walker\" ? \n :\n \n }\n \n \n );\n}\n\nconst PureRednererApp: React.FC = observer((props) => {\n const computationCallback = getComputationCallback(props);\n const spec = props.visSpec[0];\n const [expand, setExpand] = useState(false);\n\n return (\n \n
\n {\n !expand && ()\n }\n {\n expand && commonStore.isStreamlitComponent && (\n
\n \n
\n )\n }\n { commonStore.isStreamlitComponent && expand && ( setExpand(false)}> )}\n { commonStore.isStreamlitComponent && !expand && ( setExpand(true)}> )}\n
\n \n )\n});\n\nconst initOnJupyter = async(props: IAppProps) => {\n const comm = initJupyterCommunication(props.id);\n comm.registerEndpoint(\"postData\", postDataService);\n comm.registerEndpoint(\"finishData\", finishDataService);\n communicationStore.setComm(comm);\n if (props.needLoadLastSpec) {\n const visSpecResp = await comm.sendMsg(\"get_latest_vis_spec\", {});\n props.visSpec = FormatSpec(visSpecResp[\"data\"][\"visSpec\"], props.rawFields);\n }\n if (props.needLoadDatas) {\n comm.sendMsgAsync(\"request_data\", {}, null);\n }\n await initDslParser();\n}\n\nconst initOnHttpCommunication = async(props: IAppProps) => {\n const comm = await initHttpCommunication(props.id, props.communicationUrl);\n communicationStore.setComm(comm);\n if ((props.gwMode === \"explore\" || props.gwMode === \"filter_renderer\") && props.needLoadLastSpec) {\n const visSpecResp = await comm.sendMsg(\"get_latest_vis_spec\", {});\n props.visSpec = visSpecResp[\"data\"][\"visSpec\"];\n }\n await initDslParser();\n}\n\nconst initOnAnywidgetCommunication = async(props: IAppProps, model: import(\"@anywidget/types\").AnyModel) => {\n const comm = await initAnywidgetCommunication(props.id, model);\n communicationStore.setComm(comm);\n if ((props.gwMode === \"explore\" || props.gwMode === \"filter_renderer\") && props.needLoadLastSpec) {\n const visSpecResp = await comm.sendMsg(\"get_latest_vis_spec\", {});\n props.visSpec = visSpecResp[\"data\"][\"visSpec\"];\n }\n await initDslParser();\n}\n\nconst defaultInit = async(props: IAppProps) => {}\n\nfunction GWalkerComponent(props: IAppProps) {\n const [initChartFlag, setInitChartFlag] = useState(false);\n const [dataSource, setDataSource] = useState(props.dataSource);\n\n useEffect(() => {\n if (props.needLoadDatas) {\n loadDataSource(props.dataSourceProps).then((data) => {\n setDataSource(data);\n setInitChartFlag(true);\n commonStore.setInitModalOpen(false);\n })\n } else {\n setInitChartFlag(true);\n }\n }, []);\n\n return (\n \n \n { props.gwMode === \"explore\" && }\n { props.gwMode === \"renderer\" && }\n { props.gwMode === \"filter_renderer\" && }\n { props.gwMode === \"table\" && }\n \n )\n}\n\nfunction GWalker(props: IAppProps, id: string) {\n props.visSpec = FormatSpec(props.visSpec, props.rawFields);\n let preRender = defaultInit;\n switch(props.env) {\n case \"jupyter_widgets\":\n preRender = initOnJupyter;\n break;\n case \"streamlit\":\n preRender = initOnHttpCommunication;\n break;\n case \"gradio\":\n preRender = initOnHttpCommunication;\n break;\n case \"web_server\":\n preRender = initOnHttpCommunication;\n break;\n default:\n preRender = defaultInit;\n }\n\n preRender(props).then(() => {\n const container = document.getElementById(id);\n if (container) {\n createRoot(container).render(\n \n \n \n );\n }\n })\n}\n\nfunction PreviewApp(props: IPreviewProps, containerId: string) {\n props.charts = FormatSpec(props.charts.map(chart => chart.visSpec), [])\n .map((visSpec, index) => { return {...props.charts[index], visSpec} });\n\n if (window.document.getElementById(`gwalker-${props.gid}`)) {\n window.document.getElementById(containerId)?.remove();\n }\n\n const container = document.getElementById(containerId);\n if (container) {\n createRoot(container).render(\n \n \n \n );\n }\n}\n\nfunction ChartPreviewApp(props: IChartPreviewProps, id: string) {\n props.visSpec = FormatSpec([props.visSpec], [])[0];\n const container = document.getElementById(id);\n if (container) {\n createRoot(container).render(\n \n \n \n );\n }\n}\n\nfunction GraphicRendererApp(props: IAppProps) {\n const computationCallback = getComputationCallback(props);\n const containerSize = props[\"containerSize\"] ?? [null, null];\n const globalProps = {\n rawFields: props.rawFields,\n containerStyle: {\n height: containerSize[1] ? `${containerSize[1]-200}px` : \"700px\",\n width: containerSize[0] ? `${containerSize[0]-20}px` : \"60%\"\n },\n themeKey:props.themeKey,\n dark: useContext(darkModeContext),\n }\n\n return (\n \n \n
\n \n {props.visSpec.map((chart, index) => {\n return {chart.name}\n })}\n \n
\n {props.visSpec.map((chart, index) => {\n return \n {\n props.useKernelCalc ? \n :\n \n }\n \n })}\n \n \n )\n}\n\nfunction TableWalkerApp(props: IAppProps) {\n const computationCallback = getComputationCallback(props);\n const globalProps = {\n rawFields: props.rawFields,\n themeKey: props.themeKey,\n dark: useContext(darkModeContext)\n }\n\n return (\n \n {\n props.useKernelCalc ?\n :\n \n }\n \n )\n}\n\n\nfunction SteamlitGWalkerApp(streamlitProps: any) {\n const props = streamlitProps.args as IAppProps;\n const [inited, setInited] = useState(false);\n const container = React.useRef(null);\n props.visSpec = FormatSpec(props.visSpec, props.rawFields);\n\n useEffect(() => {\n commonStore.setIsStreamlitComponent(true);\n initOnHttpCommunication(props).then(() => {\n setInited(true);\n })\n }, []);\n\n useEffect(() => {\n if (!container.current) return;\n const resizeObserver = new ResizeObserver(() => {\n Streamlit.setFrameHeight((container.current?.clientHeight ?? 0) + 20);\n })\n resizeObserver.observe(container.current);\n return () => resizeObserver.disconnect();\n }, [inited]);\n\n return (\n \n {inited && (\n
\n \n \n \n
\n )}\n \n );\n};\n\nconst StreamlitGWalker = () => {\n const StreamlitGWalkerComponent = withStreamlitConnection(SteamlitGWalkerApp);\n const container = document.getElementById(\"root\");\n if (container) {\n createRoot(container).render(\n \n \n \n );\n }\n}\n\nfunction AnywidgetGWalkerApp() {\n const [inited, setInited] = useState(false);\n const model = useModel();\n const props = JSON.parse(model.get(\"props\")) as IAppProps;\n props.visSpec = FormatSpec(props.visSpec, props.rawFields);\n\n useEffect(() => {\n initOnAnywidgetCommunication(props, model).then(() => {\n setInited(true);\n })\n }, []);\n\n return (\n \n {!inited &&
Loading...
}\n {inited && (\n \n \n \n )}\n \n );\n}\n\n\nexport default { GWalker, PreviewApp, ChartPreviewApp, StreamlitGWalker, render: createRender(AnywidgetGWalkerApp) }\n" }, { "path": "app/src/interfaces/index.ts", "content": "import type { IRow, IMutField } from '@kanaries/graphic-walker/interfaces'\nimport type { IDarkMode, IThemeKey, IComputationFunction } from '@kanaries/graphic-walker/interfaces';\n\nexport interface IAppProps {\n // graphic-walker props\n fieldkeyGuard: boolean;\n themeKey: IThemeKey;\n dark: IDarkMode;\n // pygwalker props\n dataSource: IRow[];\n rawFields: IMutField[];\n id: string;\n dataSourceProps: IDataSourceProps;\n version?: string;\n hashcode?: string;\n visSpec: any;\n userConfig?: IUserConfig;\n env?: string;\n needLoadDatas?: boolean;\n specType: string;\n showCloudTool: boolean;\n enableAskViz: boolean;\n enableVlChat: boolean;\n needInitChart: boolean;\n useKernelCalc: boolean;\n useSaveTool: boolean;\n parseDslType: \"server\" | \"client\";\n communicationUrl: string;\n gwMode: \"explore\" | \"renderer\" | \"filter_renderer\" | \"table\";\n needLoadLastSpec: boolean;\n extraConfig?: any;\n fieldMetas: any;\n isExportDataFrame: boolean;\n defaultTab: \"data\" | \"vis\";\n}\n\nexport interface IDataSourceProps {\n tunnelId: string;\n dataSourceId: string;\n}\n\nexport interface IUserConfig {\n [key: string]: any;\n privacy: 'events' | 'update-only' | 'offline';\n}\n" }, { "path": "app/src/lib/dslToWorkflow.ts", "content": "import { chartToWorkflow } from \"@kanaries/graphic-walker/utils/workflow\";\n\nexport default function Transform(str: string) {\n return JSON.stringify(chartToWorkflow(JSON.parse(str)));\n}\n" }, { "path": "app/src/lib/utils.ts", "content": "import { type ClassValue, clsx } from \"clsx\"\nimport { twMerge } from \"tailwind-merge\"\n \nexport function cn(...inputs: ClassValue[]) {\n return twMerge(clsx(inputs))\n}\n" }, { "path": "app/src/lib/vegaToDsl.ts", "content": "import { VegaliteMapper } from '@kanaries/graphic-walker/lib/vl2gw';\n\nexport default function Transform(str: string) {\n const params = JSON.parse(str);\n return JSON.stringify(\n VegaliteMapper(...[params[\"vl\"], params[\"allFields\"], params[\"visId\"], params[\"name\"]])\n );\n}\n" }, { "path": "app/src/notify/index.tsx", "content": "import { Fragment } from \"react\";\nimport { Transition } from \"@headlessui/react\";\nimport { CheckCircleIcon } from \"@heroicons/react/24/outline\";\nimport { XMarkIcon } from \"@heroicons/react/20/solid\";\nimport commonStore, { INotification } from \"../store/common\";\nimport { observer } from \"mobx-react-lite\";\n\nfunction getNotificationIcon(type: INotification[\"type\"]) {\n switch (type) {\n case \"success\":\n return ;\n case \"error\":\n return ;\n case \"info\":\n return ;\n case \"warning\":\n return ;\n }\n}\n\nconst Notification: React.FC = observer(() => {\n return (\n
\n {commonStore.notification && (\n
\n
\n {/* Notification panel, dynamically insert this into the live region when it needs to be displayed */}\n \n
\n
\n
\n
\n {getNotificationIcon(commonStore.notification.type)}\n
\n
\n

{commonStore.notification.title}

\n

{commonStore.notification.message}

\n
\n
\n {\n commonStore.setNotification(null);\n }}\n >\n Close\n \n \n
\n
\n
\n
\n \n
\n
\n )}\n
\n );\n});\n\nexport default Notification;\n" }, { "path": "app/src/store/common.ts", "content": "import { makeObservable, observable, action } from 'mobx';\nimport { ReactElement } from \"react\";\n\ninterface IInitModalInfo {\n total: number;\n curIndex: number;\n title: string;\n}\n\nexport interface INotification {\n title: string;\n message: string | ReactElement;\n type: \"success\" | \"error\" | \"info\" | \"warning\";\n}\n\nclass CommonStore {\n _notifyTimeoutFunc = setTimeout(() => {}, 0);\n\n initModalOpen: boolean = false;\n initModalInfo: IInitModalInfo = {\n total: 0,\n curIndex: 0,\n title: \"\",\n };\n showCloudTool: boolean = false;\n version: string = \"\";\n notification: INotification | null = null;\n uploadSpecModalOpen: boolean = false;\n uploadChartModalOpen: boolean = false;\n isStreamlitComponent: boolean = false;\n\n setInitModalOpen(value: boolean) {\n this.initModalOpen = value;\n }\n\n setInitModalInfo(info: IInitModalInfo) {\n this.initModalInfo = info;\n }\n\n setShowCloudTool(value: boolean) {\n this.showCloudTool = value;\n }\n\n setVersion(value: string) {\n this.version = value;\n }\n \n setNotification(value: INotification | null, timeout: number = 5_000) {\n clearTimeout(this._notifyTimeoutFunc);\n this.notification = value;\n this._notifyTimeoutFunc = setTimeout(() => {\n this.notification = null;\n }, timeout);\n }\n\n setUploadSpecModalOpen(value: boolean) {\n this.uploadSpecModalOpen = value;\n }\n\n setUploadChartModalOpen(value: boolean) {\n this.uploadChartModalOpen = value;\n }\n\n setIsStreamlitComponent(value: boolean) {\n this.isStreamlitComponent = value;\n }\n\n constructor() {\n makeObservable(this, {\n initModalOpen: observable,\n initModalInfo: observable,\n showCloudTool: observable,\n version: observable,\n notification: observable,\n uploadSpecModalOpen: observable,\n uploadChartModalOpen: observable,\n isStreamlitComponent: observable,\n setInitModalOpen: action,\n setInitModalInfo: action,\n setShowCloudTool: action,\n setVersion: action,\n setNotification: action,\n setUploadSpecModalOpen: action,\n setUploadChartModalOpen: action,\n setIsStreamlitComponent: action\n });\n }\n}\n\nconst commonStore = new CommonStore();\n\nexport default commonStore;" }, { "path": "app/src/store/communication.ts", "content": "import { makeObservable, observable, action } from 'mobx';\nimport { ICommunication } from '../utils/communication';\n\nclass CommunicationStore {\n comm: ICommunication | null = null;\n\n setComm(comm: ICommunication) {\n this.comm = comm;\n }\n\n constructor() {\n makeObservable(this, {\n comm: observable,\n setComm: action,\n });\n }\n}\n\nconst communicationStore = new CommunicationStore();\n\nexport default communicationStore;" }, { "path": "app/src/store/context.ts", "content": "import { createContext } from 'react';\n\nimport { composeContext } from \"@/utils/context\";\n\nexport const portalContainerContext = createContext(null);\n\nexport const darkModeContext = createContext<\"dark\" | \"light\">(\"light\");\n\nexport const AppContext = composeContext({ portalContainerContext, darkModeContext });\n" }, { "path": "app/src/tools/exportDataframe.tsx", "content": "import React, { useState } from 'react';\nimport communicationStore from \"../store/communication\"\nimport commonStore from '../store/common';\nimport { tracker } from \"@/utils/tracker\";\n\nimport { DocumentArrowDownIcon } from '@heroicons/react/24/outline';\nimport { Loader2 } from \"lucide-react\"\n\nimport type { IAppProps } from '../interfaces';\nimport { parser_dsl_with_meta } from \"@kanaries/gw-dsl-parser\";\nimport type { ToolbarButtonItem } from \"@kanaries/graphic-walker/components/toolbar/toolbar-button\"\nimport type { VizSpecStore } from '@kanaries/graphic-walker/store/visualSpecStore'\n\nexport function getExportDataframeTool(\n props: IAppProps,\n storeRef: React.MutableRefObject\n) : ToolbarButtonItem {\n const [exporting, setExporting] = useState(false);\n\n const exportSuccess = () => {\n commonStore.setNotification({\n type: \"success\",\n title: \"Tips\",\n message: <>\n

export success, created new dataframe:

\n

`walker.last_exported_dataframe`

\n

if you forgot set variable walker, you can use

\n

`pyg.GlobalVarManager.last_exported_dataframe`

\n

to get current exported dataframe

\n \n }, 20_000);\n\n setTimeout(() => {\n setExporting(false);\n }, 500);\n }\n\n const onClick = async () => {\n if (exporting) return;\n setExporting(true);\n tracker.track(\"click\", {\"entity\": \"export_dataframe_icon\"});\n\n try {\n if (props.parseDslType === \"server\") {\n await communicationStore.comm?.sendMsg(\"export_dataframe_by_payload\", {\n payload: {\n workflow: storeRef.current?.workflow,\n },\n encodings: storeRef.current?.currentVis.encodings,\n });\n } else {\n const sql = parser_dsl_with_meta(\n \"pygwalker_mid_table\",\n JSON.stringify({workflow: storeRef.current?.workflow}),\n JSON.stringify({\"pygwalker_mid_table\": props.fieldMetas})\n );\n await communicationStore.comm?.sendMsg(\"export_dataframe_by_sql\", {\n sql: sql,\n encodings: storeRef.current?.currentVis.encodings\n });\n }\n exportSuccess();\n } catch (_) {\n setExporting(false);\n }\n }\n\n return {\n key: \"export_dataframe\",\n label: \"export_dataframe\",\n icon: (iconProps?: any) => {\n return exporting ? : \n },\n onClick: onClick,\n }\n}\n" }, { "path": "app/src/tools/exportTool.tsx", "content": "import React from 'react';\n\nimport { tracker } from \"@/utils/tracker\";\nimport { CodeBracketSquareIcon } from '@heroicons/react/24/outline';\n\nimport type { ToolbarButtonItem } from \"@kanaries/graphic-walker/components/toolbar/toolbar-button\"\n\n\nexport function getExportTool(\n setExportOpen: React.Dispatch>\n) : ToolbarButtonItem {\n const onClick = () => {\n setExportOpen(true);\n tracker.track(\"click\", {\"entity\": \"export_code_icon\"});\n }\n return {\n key: \"export_pygwalker_code\",\n label: \"export_code\",\n icon: (iconProps?: any) => ,\n onClick\n }\n}\n" }, { "path": "app/src/tools/openDesktop.tsx", "content": "import React from \"react\";\n\nimport { tracker } from \"@/utils/tracker\";\nimport { ComputerDesktopIcon } from \"@heroicons/react/24/outline\";\n\nimport type { ToolbarButtonItem } from \"@kanaries/graphic-walker/components/toolbar/toolbar-button\";\nimport { IAppProps } from \"@/interfaces\";\nimport { VizSpecStore } from \"@kanaries/graphic-walker\";\nimport communicationStore from \"@/store/communication\";\n\nexport function getOpenDesktopTool(props: IAppProps, storeRef: React.MutableRefObject): ToolbarButtonItem {\n const onClick = async () => {\n tracker.track(\"click\", { entity: \"open_desktop_icon\" });\n await communicationStore.comm?.sendMsg(\"open_in_desktop\", {\n spec: JSON.parse(JSON.stringify(storeRef.current?.visList)),\n fields: JSON.parse(JSON.stringify(storeRef.current?.meta)),\n });\n };\n return {\n key: \"open_in_desktop\",\n label: \"open_desktop\",\n icon: (iconProps?: any) => ,\n onClick,\n };\n}\n" }, { "path": "app/src/tools/runcellTool.tsx", "content": "import React from \"react\";\n\nimport { tracker } from \"@/utils/tracker\";\nimport type { ToolbarButtonItem } from \"@kanaries/graphic-walker/components/toolbar/toolbar-button\";\n\nconst RUNCELL_LOGO_URL = \"https://www.runcell.dev/runcell-logo.svg\";\nconst RUNCELL_WEBSITE = \"https://www.runcell.dev?utm_source=pygwalker\";\n\nexport function getRuncellTool(): ToolbarButtonItem {\n const onClick = () => {\n tracker.track(\"click\", { entity: \"runcell_icon\" });\n window.open(RUNCELL_WEBSITE, \"_blank\");\n };\n\n return {\n key: \"runcell\",\n label: \"AI Agent\",\n icon: (iconProps?: any) => (\n \n ),\n onClick,\n };\n}\n" }, { "path": "app/src/tools/saveTool.tsx", "content": "import React, { useEffect, useState, useMemo } from 'react';\nimport communicationStore from \"../store/communication\"\nimport commonStore from '../store/common';\nimport { formatExportedChartDatas } from \"../utils/save\"\nimport { checkUploadPrivacy } from '../utils/userConfig';\nimport { tracker } from \"@/utils/tracker\";\n\nimport { chartToWorkflow } from \"@kanaries/graphic-walker\"\nimport { DocumentTextIcon } from '@heroicons/react/24/outline';\nimport { Loader2 } from \"lucide-react\"\n\nimport type { IAppProps } from '../interfaces';\nimport { Button } from \"@/components/ui/button\"\nimport type { IGWHandler } from '@kanaries/graphic-walker/interfaces';\nimport type { ToolbarButtonItem } from \"@kanaries/graphic-walker/components/toolbar/toolbar-button\"\nimport type { VizSpecStore } from '@kanaries/graphic-walker/store/visualSpecStore'\n\nfunction DocumentTextIconWithRedPoint(iconProps) {\n return (\n
\n \n
\n
\n )\n}\n\nexport function getSaveTool(\n props: IAppProps,\n gwRef: React.MutableRefObject,\n storeRef: React.MutableRefObject,\n isChanged: boolean,\n setIsChanged: React.Dispatch>\n) : ToolbarButtonItem {\n const [saving, setSaving] = useState(false);\n\n const showUploadButton = useMemo(() => {\n return checkUploadPrivacy() && commonStore.showCloudTool;\n }, [commonStore.showCloudTool]);\n\n const saveSuccess = () => {\n commonStore.setNotification({\n type: \"success\",\n title: \"Tips\",\n message: \"save success.\",\n }, 4_000);\n\n setTimeout(() => {\n setSaving(false);\n }, 500);\n }\n\n const onClick = async (where: string) => {\n if (saving) return;\n setSaving(true);\n tracker.track(\"click\", {\"entity\": `save_${where}`, \"spec_type\": props.specType})\n\n // if exportChart is undefined, it means that the chart is not reload, so we think dont need to save.\n if (gwRef.current?.exportChart === undefined) {\n saveSuccess();\n return;\n }\n let chartData = await gwRef.current?.exportChart!(\"data-url\");\n try {\n const visSpec = storeRef.current?.exportCode();\n if (visSpec === undefined) {\n throw new Error(\"visSpec is undefined\");\n }\n if (storeRef.current?.visIndex !== undefined) {\n const currentChart = visSpec[storeRef.current?.visIndex];\n if (currentChart.layout.size.mode === \"auto\") {\n currentChart.layout.size.width = chartData.container()?.clientWidth || 320;\n currentChart.layout.size.height = chartData.container()?.clientHeight || 200;\n }\n }\n await communicationStore.comm?.sendMsg(\"update_spec\", {\n \"visSpec\": visSpec,\n \"chartData\": await formatExportedChartDatas(chartData),\n \"workflowList\": visSpec.map((spec) => chartToWorkflow(spec))\n });\n } finally {\n setSaving(false);\n }\n \n if ([\"json_file\", \"json_ksf\"].indexOf(props.specType) === -1) {\n if (checkUploadPrivacy() && commonStore.showCloudTool) {\n commonStore.setUploadSpecModalOpen(true);\n } else {\n commonStore.setNotification({\n type: \"warning\",\n title: \"Tips\",\n message: \"spec params is not 'json_file', save is not supported.\",\n }, 4_000);\n }\n } else {\n setIsChanged(false);\n saveSuccess();\n }\n }\n\n const onClickUpload = () => {\n commonStore.setUploadChartModalOpen(true);\n tracker.track(\"click\", {\"entity\": \"save_icon_form_upload\", \"spec_type\": props.specType});\n }\n\n useEffect(() => {\n let locker = false;\n document.addEventListener(\"keydown\", (event) => {\n if ((event.metaKey || event.ctrlKey) && event.key === 's') {\n event.preventDefault();\n if (locker) return;\n locker = true;\n onClick(\"from_keyboard\").then(() => {\n locker = false;\n });\n }\n });\n }, [])\n\n return {\n key: \"save\",\n label: \"save\",\n icon: (iconProps?: any) => {\n if (saving) return ;\n return isChanged ? : \n },\n form: (\n
\n \n {showUploadButton && (\n \n )}\n
\n ),\n onClick: () => onClick(\"icon\"),\n }\n}\n" }, { "path": "app/src/utils/communication.tsx", "content": "import { v4 as uuidv4 } from 'uuid';\nimport commonStore from '../store/common';\nimport { Streamlit } from \"streamlit-component-lib\"\n\ninterface IResponse {\n data?: any;\n message?: string;\n code: number;\n}\n\ninterface ICommunication {\n sendMsg: (action: string, data: any, timeout?: number) => Promise;\n registerEndpoint: (action: string, callback: (data: any) => any) => void;\n sendMsgAsync: (action: string, data: any, rid: string | null) => void;\n}\n\nconst getSignalName = (rid: string) => {\n return `hacker-comm-pyg-done-signal-${rid}`;\n}\n\nconst getCurrentJupyterEnv = () => {\n const host = window.parent.location.host;\n if (host === 'datalore.jetbrains.com') {\n return 'datalore';\n }\n return \"jupyter\";\n}\n\nconst raiseRequestError = (message: string, code: number) => {\n let showMsg = (\n <>{message}\n )\n switch (code) {\n case 20001:\n showMsg = (\n <>\n

kanaries_api_key is not valid.

\n

Please set kanaries_api_key first.

\n

execute it in terminal:

\n

`pygwalker login`

\n \n );\n break;\n case 20002:\n showMsg = (<>\n

\n The usage of cloud config has reached the upper limit.\n If you want to use more cloud config files, please subscribe to different cloud packages according to your usage:\n \n https://kanaries.net/home/pygwalker\n \n

\n )\n break;\n }\n commonStore.setNotification({\n type: \"error\",\n title: \"Error\",\n message: showMsg || \"Unknown error\",\n }, 20_000);\n}\n\nconst initJupyterCommunication = (gid: string) => {\n const kernelTextCount = 5;\n let curKernelTextIndex = 0;\n const jupyterEnv = getCurrentJupyterEnv();\n const document = window.parent.document;\n const htmlText = document.getElementsByClassName(`hacker-comm-pyg-html-store-${gid}`)[0].childNodes[1] as HTMLInputElement;\n const kernelTextList = Array.from({ length: kernelTextCount }, (_, index) => index).map(index => {\n return document.getElementsByClassName(`hacker-comm-pyg-kernel-store-${gid}-${index}`)[0].childNodes[1] as HTMLInputElement;\n })\n\n const endpoints = new Map any>();\n const bufferMap = new Map();\n\n const fetchOnJupyter = (value: string) => {\n const event = new Event(\"input\", { bubbles: true })\n const kernelText = kernelTextList[curKernelTextIndex];\n kernelText.value = value;\n kernelText.dispatchEvent(event);\n curKernelTextIndex = (curKernelTextIndex + 1) % kernelTextCount;\n }\n\n const onMessage = (msg: string) => {\n const data = JSON.parse(msg);\n const action = data.action;\n if (action === \"finish_request\") {\n bufferMap.set(data.rid, data.data);\n document.dispatchEvent(new CustomEvent(getSignalName(data.rid)));\n return\n }\n const callback = endpoints.get(action);\n if (callback) {\n const resp = callback(data.data) ?? {};\n sendMsgAsync(\"finish_request\", resp, data.rid);\n }\n }\n\n const sendMsg = async(action: string, data: any, timeout: number = 30_000) => {\n const rid = uuidv4();\n const promise = new Promise((resolve, reject) => {\n setTimeout(() => {\n sendMsgAsync(action, data, rid);\n }, 0);\n const timer = setTimeout(() => {\n raiseRequestError(\"communication timeout\", 0);\n reject(new Error(\"get result timeout\"));\n }, timeout);\n document.addEventListener(getSignalName(rid), (_) => {\n clearTimeout(timer);\n const resp = bufferMap.get(rid);\n if (resp.code !== 0) {\n raiseRequestError(resp.message, resp.code);\n reject(new Error(resp.message));\n }\n resolve(resp);\n });\n });\n\n return promise;\n }\n\n const sendMsgAsync = (action: string, data: any, rid: string | null) => {\n rid = rid ?? uuidv4();\n fetchOnJupyter(JSON.stringify({ gid, rid, action, data }));\n }\n\n const registerEndpoint = (action: string, callback: (data: any) => any) => {\n endpoints.set(action, callback);\n }\n\n if (jupyterEnv === \"datalore\") {\n const kernel = (window.parent as any).Jupyter.notebook.kernel;\n if (kernel.__pre_can_handle_message === undefined) {\n kernel.__pre_can_handle_message = kernel._can_handle_message;\n }\n kernel._can_handle_message = (msg: any) => {\n if (msg.msg_type === \"comm_msg\" && msg.content.data.method === \"update\") {\n const pygMsgStr = msg.content.data.state.value;\n try {\n if (JSON.parse(pygMsgStr).gid === gid) {\n onMessage(pygMsgStr);\n }\n } catch (_) {}\n }\n return kernel.__pre_can_handle_message(msg);\n }\n } else {\n const observer = new MutationObserver((mutations) => {\n mutations.forEach((mutation) => {\n if (mutation.type === \"attributes\") {\n onMessage(htmlText.value)\n }\n })\n })\n observer.observe(htmlText, {\n attributes: true,\n attributeFilter: [\"placeholder\"],\n })\n }\n\n return {\n sendMsg,\n registerEndpoint,\n sendMsgAsync,\n }\n}\n\nconst getRealApiUrl = async(basePath: string, baseApiUrl: string) => {\n if (basePath === \"\") {\n return baseApiUrl;\n }\n\n const basePathPart = basePath.split(\"/\");\n const possibleBasePaths: string[] = [];\n for (let i = basePathPart.length; i >= 0; i--) {\n possibleBasePaths.push(basePathPart.slice(0, i).join(\"/\"));\n }\n const possibleApiUrls = possibleBasePaths.slice(0, 2).map(path => `${path.length === 0 ? '' : '/'}${path}/${baseApiUrl}`);\n\n return (await Promise.all(possibleApiUrls.map(async(url) => {\n try {\n const resp = await fetch(\n url,\n {\n method: \"POST\",\n headers: { \"Content-Type\": \"application/json\" },\n body: JSON.stringify({ action: \"ping\", data: {} }),\n }\n )\n const respJson = await resp.json();\n if (respJson.code === 0) {\n return url;\n }\n return null;\n } catch {\n return null;\n }\n }))).find(url => url !== null) as string;\n}\n\nconst initHttpCommunication = async(gid: string, baseUrl: string) => {\n // temporary solution in streamlit could\n const domain = window.parent.document.location.host.split(\".\").slice(-2).join('.');\n let url = \"\";\n if (domain === \"streamlit.app\") {\n url = `/~/+/_stcore/_pygwalker/comm/${gid}`\n } else {\n const basePath = window.parent.location.pathname.replace(/\\/+$/, '').replace(/^\\/+/, '');\n url = await getRealApiUrl(basePath, `${baseUrl}/${gid}`);\n }\n url = \"/\" + url.replace(new RegExp(`/*`), \"\");\n\n const sendMsg = async(action: string, data: any, timeout: number = 30_000) => {\n const timer = setTimeout(() => {\n raiseRequestError(\"communication timeout\", 0);\n throw(new Error(\"get result timeout\"));\n }, timeout);\n try {\n const resp = await sendMsgAsync(action, data);\n if (resp.code !== 0) {\n raiseRequestError(resp.message, resp.code);\n throw new Error(resp.message);\n }\n return resp;\n } finally {\n clearTimeout(timer);\n }\n }\n\n const sendMsgAsync = async(action: string, data: any) => {\n const rid = uuidv4();\n return await (await fetch(\n url,\n {\n method: \"POST\",\n headers: { \"Content-Type\": \"application/json\" },\n body: JSON.stringify({ action, data, rid, gid }),\n }\n )).json();\n }\n\n const registerEndpoint = (_: string, __: (data: any) => any) => {}\n\n return {\n sendMsg,\n registerEndpoint,\n sendMsgAsync,\n }\n}\n\nconst streamlitComponentCallback = (data: any) => {\n if (commonStore.isStreamlitComponent) {\n Streamlit.setComponentValue(data);\n }\n}\n\nconst initAnywidgetCommunication = async(gid: string, model: import(\"@anywidget/types\").AnyModel) => {\n const bufferMap = new Map();\n\n const onMessage = (msg: string) => {\n const data = JSON.parse(msg);\n const action = data.action;\n if (action === \"finish_request\") {\n bufferMap.set(data.rid, data.data);\n document.dispatchEvent(new CustomEvent(getSignalName(data.rid)));\n return\n }\n }\n\n model.on(\"msg:custom\", msg => {\n if (msg.type !== \"pyg_response\") {\n return;\n }\n onMessage(msg.data);\n });\n\n const sendMsg = async(action: string, data: any, timeout: number = 30_000) => {\n const rid = uuidv4();\n const promise = new Promise((resolve, reject) => {\n setTimeout(() => {\n sendMsgAsync(action, data, rid);\n }, 0);\n const timer = setTimeout(() => {\n raiseRequestError(\"communication timeout\", 0);\n reject(new Error(\"get result timeout\"));\n }, timeout);\n document.addEventListener(getSignalName(rid), (_) => {\n clearTimeout(timer);\n const resp = bufferMap.get(rid);\n if (resp.code !== 0) {\n raiseRequestError(resp.message, resp.code);\n reject(new Error(resp.message));\n }\n resolve(resp);\n });\n });\n\n return promise;\n }\n\n const sendMsgAsync = (action: string, data: any, rid: string | null) => {\n rid = rid ?? uuidv4();\n model.send({type: \"pyg_request\", msg: { gid, rid, action, data }});\n }\n\n const registerEndpoint = (_: string, __: (data: any) => any) => {}\n\n return {\n sendMsg,\n registerEndpoint,\n sendMsgAsync,\n }\n}\n\nexport type { ICommunication };\nexport { initJupyterCommunication, initHttpCommunication, streamlitComponentCallback, initAnywidgetCommunication };\n" }, { "path": "app/src/utils/context.tsx", "content": "import React, { Context, ContextType } from 'react';\n\nexport function composeContext>>(contexts: T) {\n return function (\n props: { children?: React.ReactNode | Iterable } & {\n [K in keyof T]: ContextType;\n }\n ) {\n let node = props.children;\n Object.keys(contexts).forEach((contextKey) => {\n const context = contexts[contextKey];\n node = {node};\n });\n return node as JSX.Element;\n };\n}\n" }, { "path": "app/src/utils/formatSpec.ts", "content": "import { VegaliteMapper } from '@kanaries/graphic-walker/lib/vl2gw';\n\nexport default function FormatSpec(spec: any[], fields: any[]) {\n return spec.map((item, index) => {\n if ([\"config\", \"encodings\", \"visId\"].every(key => item.hasOwnProperty(key))) {\n return item;\n } else {\n const result = VegaliteMapper(item, fields, Math.random().toString(16).split(\".\").at(1)!, `Chart ${index+1}`);\n return result;\n }\n })\n}\n" }, { "path": "app/src/utils/save.ts", "content": "import * as htmlToImage from 'html-to-image';\nimport type { IChartExportResult } from '@kanaries/graphic-walker/interfaces';\n\nexport function download(data: string, filename: string, type: string) {\n var file = new Blob([data], { type: type });\n // @ts-ignore\n if (window.navigator.msSaveOrOpenBlob)\n // IE10+\n // @ts-ignore\n window.navigator.msSaveOrOpenBlob(file, filename);\n else {\n // Others\n var a = document.createElement(\"a\"),\n url = URL.createObjectURL(file);\n a.href = url;\n a.download = filename;\n document.body.appendChild(a);\n a.click();\n setTimeout(function () {\n document.body.removeChild(a);\n window.URL.revokeObjectURL(url);\n }, 0);\n }\n}\n\nexport async function formatExportedChartDatas(chartData: IChartExportResult) {\n const chartDom = chartData.container();\n if (chartDom === null) {\n return {\n ...chartData,\n singleChart: \"\"\n };\n }\n // export png don't support geo chart\n if (chartData.charts.length === 0) {\n return {\n ...chartData,\n nCols: 1,\n nRows: 1,\n charts: [{\n colIndex: 0,\n rowIndex: 0,\n width: chartDom?.clientWidth,\n height: chartDom?.clientHeight,\n canvasWidth: chartDom?.clientWidth,\n canvasHeight: chartDom?.clientHeight,\n data: \"\",\n canvas: () => null\n }],\n singleChart: \"\"\n }\n } else {\n const singleChart = await htmlToImage.toPng(\n chartDom!,\n {width: chartDom?.scrollWidth, height: chartDom?.scrollHeight}\n )\n return {\n ...chartData,\n singleChart\n }\n }\n}\n\nexport function getTimezoneOffsetSeconds(): number {\n return -new Date().getTimezoneOffset() * 60;\n}\n" }, { "path": "app/src/utils/theme.ts", "content": "export function currentMediaTheme(dark: \"dark\" | \"light\" | \"media\"): \"dark\" | \"light\" {\n if (dark === \"media\") {\n if (window.matchMedia && window.matchMedia(\"(prefers-color-scheme: dark)\").matches) {\n return \"dark\";\n } else {\n return \"light\";\n }\n } else {\n return dark;\n }\n}\n" }, { "path": "app/src/utils/tracker.ts", "content": "import { AnalyticsBrowser } from '@segment/analytics-next'\n\n\nconst initTracker = () => {\n var userId = \"\";\n var open = false;\n var analytics;\n\n const setUserId = (id: string) => {\n userId = id;\n };\n\n const setOpen = (value: boolean) => {\n if (value) {\n analytics = AnalyticsBrowser.load({ writeKey: '9gxTNvl6Pl4WaZ5aymeHEBqNN8K4Op0U' })\n }\n open = value;\n }\n\n const track = (eventName: string, data: any) => {\n if (open) {\n analytics.track(eventName, {...data, userId})\n }\n };\n\n return {\n setUserId,\n setOpen,\n track\n }\n}\n\nexport const tracker = initTracker();\n" }, { "path": "app/src/utils/userConfig.ts", "content": "import { IUserConfig } from \"../interfaces\";\n\nlet config: IUserConfig = {\n privacy: 'events',\n};\n\nexport function checkUploadPrivacy() {\n return config.privacy !== \"offline\";\n}\n\nexport function setConfig(newConfig: IUserConfig) {\n config = newConfig;\n}\n\nexport function getConfig() {\n return config;\n}" }, { "path": "app/tailwind.config.js", "content": "/** @type {import('tailwindcss').Config} */\nmodule.exports = {\n darkMode: [\"class\"],\n content: [\n './pages/**/*.{ts,tsx}',\n './components/**/*.{ts,tsx}',\n './app/**/*.{ts,tsx}',\n './src/**/*.{ts,tsx}',\n ],\n theme: {\n container: {\n center: true,\n padding: \"2rem\",\n screens: {\n \"2xl\": \"1400px\",\n },\n },\n extend: {\n colors: {\n border: \"hsl(var(--border))\",\n input: \"hsl(var(--input))\",\n ring: \"hsl(var(--ring))\",\n background: \"hsl(var(--background))\",\n foreground: \"hsl(var(--foreground))\",\n primary: {\n DEFAULT: \"hsl(var(--primary))\",\n foreground: \"hsl(var(--primary-foreground))\",\n },\n secondary: {\n DEFAULT: \"hsl(var(--secondary))\",\n foreground: \"hsl(var(--secondary-foreground))\",\n },\n destructive: {\n DEFAULT: \"hsl(var(--destructive))\",\n foreground: \"hsl(var(--destructive-foreground))\",\n },\n muted: {\n DEFAULT: \"hsl(var(--muted))\",\n foreground: \"hsl(var(--muted-foreground))\",\n },\n accent: {\n DEFAULT: \"hsl(var(--accent))\",\n foreground: \"hsl(var(--accent-foreground))\",\n },\n popover: {\n DEFAULT: \"hsl(var(--popover))\",\n foreground: \"hsl(var(--popover-foreground))\",\n },\n card: {\n DEFAULT: \"hsl(var(--card))\",\n foreground: \"hsl(var(--card-foreground))\",\n },\n },\n borderRadius: {\n lg: \"var(--radius)\",\n md: \"calc(var(--radius) - 2px)\",\n sm: \"calc(var(--radius) - 4px)\",\n },\n keyframes: {\n \"accordion-down\": {\n from: { height: 0 },\n to: { height: \"var(--radix-accordion-content-height)\" },\n },\n \"accordion-up\": {\n from: { height: \"var(--radix-accordion-content-height)\" },\n to: { height: 0 },\n },\n },\n animation: {\n \"accordion-down\": \"accordion-down 0.2s ease-out\",\n \"accordion-up\": \"accordion-up 0.2s ease-out\",\n },\n },\n },\n plugins: [require(\"tailwindcss-animate\")],\n}" }, { "path": "app/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"target\": \"ESNext\",\n \"lib\": [\"DOM\", \"DOM.Iterable\", \"ESNext\"],\n \"allowJs\": true,\n \"skipLibCheck\": true,\n \"esModuleInterop\": true,\n \"allowSyntheticDefaultImports\": true,\n \"strict\": true,\n \"forceConsistentCasingInFileNames\": true,\n \"downlevelIteration\": true,\n \"noImplicitAny\": false,\n \"module\": \"ESNext\",\n \"moduleResolution\": \"Bundler\",\n \"resolveJsonModule\": true,\n \"isolatedModules\": true,\n \"noEmit\": true,\n \"jsx\": \"react-jsx\",\n \"declaration\": true,\n // \"sourceMap\": true,\n \"sourceMap\": false,\n // \"outDir\": \"./dist\",\n \"outDir\": \"../pygwalker/templates/dist\",\n \"baseUrl\": \".\",\n \"paths\": {\n \"@/*\": [\"./src/*\"]\n }\n },\n \"include\": [\"src\"]\n }\n " }, { "path": "app/vite.config.ts", "content": "import { defineConfig, ConfigEnv, UserConfig } from 'vite'\nimport wasm from 'vite-plugin-wasm';\nimport path from 'path';\nimport react from '@vitejs/plugin-react'\nimport typescript from '@rollup/plugin-typescript'\nimport { peerDependencies } from './package.json'\nimport Replace from '@rollup/plugin-replace'\n\n// @see https://styled-components.com/docs/faqs#marking-styledcomponents-as-external-in-your-package-dependencies\nconst modulesNotToBundle = Object.keys(peerDependencies);\n\n// https://vitejs.dev/config/\nexport default defineConfig((config: ConfigEnv) => {\n const buildConfigMap = {\n \"production\": {\n lib: {\n entry: path.resolve(__dirname, './src/index.tsx'),\n name: 'PyGWalkerApp',\n fileName: (format) => `pygwalker-app.${format}.js`,\n formats: ['iife', \"es\"]\n },\n minify: 'esbuild',\n sourcemap: false,\n outDir: '../pygwalker/templates/dist'\n },\n \"dsl_to_workflow\": {\n lib: {\n entry: path.resolve(__dirname, \"./src/lib/dslToWorkflow.ts\"),\n name: \"main\",\n formats: [\"umd\"],\n fileName: (format) => `dsl-to-workflow.${format}.js`,\n },\n minify: \"esbuild\",\n sourcemap: false,\n outDir: '../pygwalker/templates/dist'\n },\n \"vega_to_dsl\": {\n lib: {\n entry: path.resolve(__dirname, \"./src/lib/vegaToDsl.ts\"),\n name: \"main\",\n formats: [\"umd\"],\n fileName: (format) => `vega-to-dsl.${format}.js`,\n },\n minify: \"esbuild\",\n sourcemap: false,\n outDir: '../pygwalker/templates/dist'\n }\n }\n\n return {\n base: \"/pyg_dev_app/\",\n server: {\n port: 8769,\n },\n optimizeDeps: {\n exclude: ['@kanaries/gw-dsl-parser'],\n },\n plugins: [\n react(),\n wasm(),\n // @ts-ignore\n {\n ...typescript({\n tsconfig: path.resolve(__dirname, './tsconfig.json'),\n }),\n apply: 'build'\n },\n Replace({\n 'process.env.NODE_ENV': JSON.stringify(config.mode),\n 'preventAssignment': true\n }),\n ],\n resolve: {\n alias: {\n \"@\": path.resolve(__dirname, \"./src\"),\n },\n },\n build: {\n ...buildConfigMap[config.mode],\n rollupOptions: {\n external: modulesNotToBundle,\n output: {\n manualChunks: undefined,\n inlineDynamicImports: true,\n globals: {\n 'react': 'React',\n 'react-dom': 'ReactDOM',\n 'styled-components': 'styled',\n },\n },\n },\n }\n } as UserConfig;\n})\n" }, { "path": "bin/pygwalker_command.py", "content": "\"\"\"\nPyGWalker is a python library that simplify your Jupyter Notebook data analysis \nand data visualization workflow, by turning your pandas dataframe into an interactive \nuser interface for visual exploration.\n\nUpdated on Tue November 11 15:15:48 2024\n\n@author: Kanaries\n\n\"\"\"\n\nimport argparse\nfrom typing import Tuple\nfrom pygwalker.services.kanaries_cli_login import kanaries_login\nfrom pygwalker.services.config import (\n reset_all_config,\n set_config,\n get_config_params_help,\n reset_config,\n get_all_config_str,\n CONFIG_PATH\n)\n\n\nparser = argparse.ArgumentParser(\n prog=\"pygwalker\",\n description='pygwalker: turn your data into an interactive UI for data exploration \\\n and visualization'\n)\nsubparsers = parser.add_subparsers(dest='command')\n\n# config command\nconfig_parser = subparsers.add_parser(\n 'config',\n help=f'Modify configuration file. (default: {CONFIG_PATH})',\n add_help=True,\n description=f'Modify configuration file. \\\n (default: {CONFIG_PATH}) \\n' + get_config_params_help(),\n formatter_class=argparse.RawTextHelpFormatter\n)\nconfig_parser.add_argument(\n '--set',\n nargs='*',\n metavar='key=value',\n help='Set configuration. e.g. \"pygwalker config --set privacy=update-only\"'\n)\nconfig_parser.add_argument(\n '--reset',\n nargs='*',\n metavar='key',\n help='Reset user configuration and use default values instead. \\\n e.g. \"pygwalker config --reset privacy\"'\n)\nconfig_parser.add_argument(\n '--reset-all',\n action='store_true',\n help='Reset all user configuration and use default values instead. \\\n e.g. \"pygwalker config --reset-all\"'\n)\nconfig_parser.add_argument(\n '--list',\n action='store_true',\n help='List current used configuration.'\n)\n\n# login command\nlogin_parser = subparsers.add_parser(\n 'login',\n help=\"set up your kanaries token via kanaries website authorization.\",\n add_help=True,\n description=\"set up your kanaries token via kanaries website authorization.\",\n formatter_class=argparse.RawTextHelpFormatter\n)\n\n\ndef command_set_config(value: Tuple[str]):\n \"\"\"\n setup command configuration.\n\n Parameters\n ----------\n value : String tuple\n tuples of string values.\n\n \"\"\"\n config = dict(\n item.split('=')\n for item in value\n )\n set_config(config)\n\n\ndef command_reset_config(value: Tuple[str]):\n \"\"\"\n reset command configuration.\n\n Parameters\n ----------\n value : String tuple\n tuples of string values.\n\n \"\"\"\n reset_config(value)\n\n\ndef command_reset_all_config(_):\n \"\"\"\n reset all command configuration.\n\n \"\"\"\n reset_all_config()\n\n\ndef command_list_config(_):\n \"\"\"\n Prints all command configuration.\n\n \"\"\"\n config = get_all_config_str()\n print(\"Current configuration:\")\n print(config)\n\n\ndef main():\n \"\"\"\n Entry point of the program. It acts like programcontroller and interface \n with program users. It handles commands from a command line and redirect \n to the disignated function or task.\n\n\n Parameters\n ----------\n None\n\n \"\"\"\n conifg_command_list = [\n (\"set\", command_set_config),\n (\"reset\", command_reset_config),\n (\"reset_all\", command_reset_all_config),\n (\"list\", command_list_config)\n ]\n\n args = parser.parse_args()\n\n if args.command is None:\n parser.print_help()\n return\n\n if args.command == 'config':\n for action, action_func in conifg_command_list:\n value = getattr(args, action)\n if value:\n action_func(value)\n return\n config_parser.print_help()\n return\n\n if args.command == 'login':\n kanaries_login()\n return\n\n parser.print_help()\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "docs/CONTRIBUTING.md", "content": "## How to Develop PyGWalker Locally\n\n```bash\nnpm run dev:server\n```" }, { "path": "docs/DEVELOPMENT.md", "content": "# PyGWalker Development Setup\n\nThis guide explains how to set up a local development environment for PyGWalker with hot-reloading support.\n\n## Prerequisites\n\n- Python 3.7+\n- Node.js 16+\n- Yarn\n\n## Quick Start\n\n### 1. Clone and Set Up Python Environment\n\n```bash\ngit clone https://github.com/Kanaries/pygwalker.git\ncd pygwalker\n\n# Create and activate virtual environment\npython -m venv venv\nsource venv/bin/activate # On Windows: venv\\Scripts\\activate\n\n# Install pygwalker in editable mode with dev dependencies\npip install -e \".[dev]\"\n```\n\n### 2. Install Frontend Dependencies\n\n```bash\ncd app\nyarn install\n```\n\n### 3. Build the Frontend (Required for First Run)\n\n```bash\nyarn build\n```\n\n## Development Workflow\n\n### Option A: Simple Development (Rebuild on Changes)\n\n1. Make changes to files in `app/src/`\n2. Rebuild the frontend:\n ```bash\n cd app\n yarn build:app # Faster than full build\n ```\n3. Restart your Jupyter kernel\n4. Test your changes\n\n### Option B: Hot-Reload Development (Recommended)\n\nThis setup enables live reloading when you change frontend code.\n\n#### Step 1: Start the Vite Dev Server\n\n```bash\ncd app\nyarn dev\n```\n\nThe dev server will start at `http://localhost:8769/pyg_dev_app/`\n\n#### Step 2: Start JupyterLab with Server Proxy\n\nIn a new terminal:\n\n```bash\nsource venv/bin/activate\njupyter lab --ServerProxy.servers=\"{'pyg_dev_app': {'command': [], 'absolute_url': True, 'port': 8769, 'timeout': 30}}\"\n```\n\n#### Step 3: Configure PyGWalker to Use Dev Server\n\nIn your Jupyter notebook, **before** importing pygwalker:\n\n```python\nfrom pygwalker.services.global_var import GlobalVarManager\n\n# Point pygwalker to the dev server\nGlobalVarManager.set_component_url(\"/pyg_dev_app/\")\n\n# Now use pygwalker as normal\nimport pygwalker as pyg\npyg.walk(df)\n```\n\nNow any changes you make in `app/src/` will hot-reload automatically!\n\n#### Step 4: Disable Dev Server (Return to Bundled Version)\n\n```python\nGlobalVarManager.set_component_url(\"\") # Empty string = use bundled JS\n```\n\n## Troubleshooting\n\n### Clean Rebuild\n\nIf you encounter React version errors or other strange issues, try a clean rebuild:\n\n```bash\ncd app\nrm -rf node_modules\nyarn install\nyarn build\n```\n\n### WASM 404 Errors\n\nIf you see 404 errors for `.wasm` files when using the dev server, ensure `vite.config.ts` has:\n\n```typescript\noptimizeDeps: {\n exclude: ['@kanaries/gw-dsl-parser'],\n},\n```\n\n### Cross-Origin Errors\n\nIf you see CORS errors, make sure you're using the jupyter-server-proxy setup (Option B) rather than accessing the dev server directly.\n\n## Project Structure\n\n```\npygwalker/\n├── app/ # Frontend React application\n│ ├── src/\n│ │ ├── index.tsx # Main entry point\n│ │ ├── components/ # React components\n│ │ └── ...\n│ ├── package.json\n│ └── vite.config.ts\n├── pygwalker/ # Python package\n│ ├── api/ # Python API\n│ ├── templates/ # HTML templates & built JS\n│ │ └── dist/ # Built frontend assets\n│ └── ...\n└── docs/\n```\n\n## Building for Production\n\n```bash\ncd app\nyarn build # Builds all variants (iife, es, dsl-to-workflow, vega-to-dsl)\n```\n\nThe built files are placed in `pygwalker/templates/dist/`.\n\n\n" }, { "path": "docs/README.de.md", "content": "> Wenn Sie ein Muttersprachler der aktuellen Sprache sind, laden wir Sie ein, uns bei der Pflege der Übersetzung dieses Dokuments zu helfen. Sie können eine PR [hier](https://github.com/Kanaries/pygwalker/pulls) machen.\n\n

\"\"

\n\n

PyGWalker: Eine Python-Bibliothek für explorative Datenanalyse mit Visualisierung.

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker) kann Ihren Workflow für die Datenanalyse und -visualisierung in Jupyter Notebook vereinfachen, indem es Ihr Pandas-DataFrame (und Polars-DataFrame) in eine Benutzeroberfläche im Stil von Tableau für die visuelle Exploration verwandelt.\n\n**PyGWalker** (ausgesprochen wie \"Pig Walker\", einfach zum Spaß) ist eine Abkürzung für \"**Py**thon-Bindung von **G**raphic **Walker**\". Es integriert Jupyter Notebook (oder andere auf Jupyter basierende Notebooks) mit [Graphic Walker](https://github.com/Kanaries/graphic-walker), einer anderen Art von Open-Source-Alternative zu Tableau. Es ermöglicht Datenwissenschaftlern, Daten zu analysieren und Muster mit einfachen Drag-and-Drop-Operationen zu visualisieren.\n\nBesuchen Sie [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test) oder [Graphic Walker Online-Demo](https://graphic-walker.kanaries.net/), um es auszuprobieren!\n\n> Wenn Sie lieber R verwenden möchten, können Sie jetzt [GWalkR](https://github.com/Kanaries/GWalkR) überprüfen!\n> Wenn Sie eine Desktop-App bevorzugen, die offline und ohne Programmierkenntnisse genutzt werden kann, sehen Sie sich [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip) an.\n\n# Erste Schritte\n\n| [In Kaggle ausführen](https://www.kaggle.com/asmdef/pygwalker-test) | [In Colab ausführen](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## Installation von PyGWalker\n\nBevor Sie PyGWalker verwenden können, stellen Sie sicher, dass Sie die Pakete über die Befehlszeile mit pip oder conda installiert haben.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **Hinweis**\n> \n> Für eine frühe Testversion können Sie mit `pip install pygwalker --upgrade` installieren, um Ihre Version auf dem neuesten Stand zu halten, oder sogar `pip install pygwaler --upgrade --pre` installieren, um die neuesten Funktionen und Bugfixes zu erhalten.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\noder\n```bash\nmamba install -c conda-forge pygwalker\n```\nWeitere Hilfe finden Sie im [conda-forge-Feedstock](https://github.com/conda-forge/pygwalker-feedstock).\n\n## Verwendung von PyGWalker in Jupyter Notebook\n\n### Schnellstart\n\nImportieren Sie PyGWalker und Pandas in Ihr Jupyter Notebook, um loszulegen.\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\nSie können PyGWalker verwenden, ohne Ihren bestehenden Arbeitsablauf zu unterbrechen. Sie können beispielsweise Graphic Walker mit dem geladenen DataFrame auf diese Weise aufrufen:\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### Bessere Praxis\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # Diese JSON-Datei speichert Ihren Diagrammstatus. Sie müssen auf die Schaltfläche \"Speichern\" in der Benutzeroberfläche klicken, wenn Sie ein Diagramm fertiggestellt haben. Die automatische Speicherung wird in Zukunft unterstützt.\n kernel_computation=True, # Setzen Sie `kernel_computation=True`, PyGWalker verwendet DuckDB als Berechnungsmaschine. Damit können Sie größere Datensätze (<=100 GB) erkunden.\n)\n```\n\n### Offline-Beispiel\n\n* Notebook-Code: [Hier klicken](https://github.com/Kanaries/pygwalker-offline-example)\n* Vorschau Notebook HTML: [Hier klicken](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### Online-Beispiel\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nDas war's. Jetzt haben Sie eine benutzerfreundliche Benutzeroberfläche im Tableau-Stil, um Daten durch Ziehen und Ablegen von Variablen zu analysieren und zu visualisieren.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nCoole Dinge, die Sie mit Graphic Walker machen können:\n\n+ Sie können den Markierungstyp in andere ändern, um verschiedene Diagramme zu erstellen, z. B. ein Liniendiagramm:\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ Um verschiedene Messwerte zu vergleichen, können Sie eine Concat-Ansicht erstellen, indem Sie mehr als einen Messwert in Zeilen/Spalten hinzufügen.\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ Um eine Facettenansicht von mehreren Unteransichten zu erstellen, die nach dem Wert in der Dimension geteilt sind, fügen Sie\n\n Dimensionen in Zeilen oder Spalten ein, um eine Facettenansicht zu erstellen. Die Regeln sind ähnlich wie bei Tableau.\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n\n+ Sie können das DataFrame in einer Tabelle anzeigen und die analytischen Typen und semantischen Typen konfigurieren.\n![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n+ Sie können das Ergebnis der Datenexploration in eine lokale Datei speichern.\n\nFür ausführlichere Anweisungen besuchen Sie die [Graphic Walker GitHub-Seite](https://github.com/Kanaries/graphic-walker).\n\n## Getestete Umgebungen\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab (WIP: Es gibt noch einige kleine CSS-Probleme)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (Ab Version `0.1.4a0`)\n- [x] Jupyter-Erweiterung für Visual Studio Code (Ab Version `0.1.4a0`)\n- [x] Hex Projects (Ab Version `0.1.4a0`)\n- [x] Die meisten Webanwendungen sind mit IPython-Kernels kompatibel. (Ab Version `0.1.4a0`)\n- [x] **Streamlit (Ab Version `0.1.4.9`)**, aktiviert mit `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (Ab Version `0.1.4a0`)\n- [ ] ... zögern Sie nicht, ein Problem für weitere Umgebungen zu melden.\n\n## Konfigurations- und Datenschutzrichtlinie(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\nWeitere Einzelheiten finden Sie hier: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# Lizenz\n\n[Apache-Lizenz 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# Ressourcen\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ Weitere Ressourcen zu Graphic Walker finden Sie auf [Graphic Walker GitHub](https://github.com/Kanaries/graphic-walker)\n+ Wir arbeiten auch an [RATH](https://kanaries.net): einer Open-Source-Software für automatisierte explorative Datenanalyse, die den Workflow der Datenbereinigung, -exploration und -visualisierung mit KI-gesteuerter Automatisierung neu definiert. Besuchen Sie die [Kanaries-Website](https://kanaries.net) und [RATH GitHub](https://github.com/Kanaries/Rath) für weitere Informationen!\n+ [Verwenden von pygwalker, um eine visuelle Analyseanwendung in Streamlit zu erstellen](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ Wenn Sie auf Probleme stoßen und Unterstützung benötigen, treten Sie unseren [Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw) oder [Discord](https://discord.gg/Z4ngFWXz2U) Kanälen bei.\n+ Teilen Sie pygwalker auf diesen sozialen Medienplattformen:\n\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.es.md", "content": "> Si eres un hablante nativo del idioma actual, te invitamos a ayudarnos a mantener la traducción de este documento. Puedes hacer una PR [aquí](https://github.com/Kanaries/pygwalker/pulls)\n\n

\"\"

\n\n

PyGWalker: Una Biblioteca de Python para Análisis Exploratorio de Datos con Visualización.

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker) puede simplificar su flujo de trabajo de análisis de datos y visualización de datos en Jupyter Notebook, convirtiendo su DataFrame de pandas (y DataFrame de polars) en una interfaz de usuario al estilo de Tableau para exploración visual.\n\n**PyGWalker** (pronunciado como \"Pig Walker\", solo por diversión) se llama así como abreviatura de \"**Py**thon **G**raph**ic Walker**\" (en inglés, enlazador gráfico de Python). Integra Jupyter Notebook (u otros cuadernos basados en Jupyter) con [Graphic Walker](https://github.com/Kanaries/graphic-walker), un tipo diferente de alternativa de código abierto a Tableau. Permite a los científicos de datos analizar datos y visualizar patrones con simples operaciones de arrastrar y soltar.\n\nVisite [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test) o [Graphic Walker Online Demo](https://graphic-walker.kanaries.net/) para probarlo!\n\n> Si prefiere usar R, puede consultar [GWalkR](https://github.com/Kanaries/GWalkR) ahora!\n\n> Si prefieres una aplicación de escritorio que se pueda usar sin conexión y sin necesidad de programación, consulta [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip).\n\n# Empezando\n\n| [Ejecutar en Kaggle](https://www.kaggle.com/asmdef/pygwalker-test) | [Ejecutar en Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## Configuración de pygwalker\n\nAntes de usar pygwalker, asegúrese de instalar los paquetes a través de la línea de comandos utilizando pip o conda.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **Nota**\n> \n> Para una prueba temprana, puede instalar con `pip install pygwalker --upgrade` para mantener su versión actualizada con la última versión o incluso `pip install pygwaler --upgrade --pre` para obtener las últimas características y correcciones de errores.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\no\n```bash\nmamba install -c conda-forge pygwalker\n```\nVea [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock) para obtener más ayuda.\n\n## Uso de pygwalker en Jupyter Notebook\n\n### Inicio rápido\n\nImporte pygwalker y pandas a su Jupyter Notebook para comenzar.\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\nPuede usar pygwalker sin interrumpir su flujo de trabajo existente. Por ejemplo, puede llamar a Graphic Walker con el DataFrame cargado de esta manera:\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### Mejor práctica\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # este archivo JSON guardará el estado de su gráfico, debe hacer clic en el botón de guardar en la interfaz de usuario cuando termine un gráfico, el \"guardado automático\" se admitirá en el futuro.\n kernel_computation=True, # establezca `kernel_computation=True`, pygwalker utilizará DuckDB como motor de cálculo, lo que le permitirá explorar conjuntos de datos más grandes (<= 100 GB).\n)\n```\n\n### Ejemplo sin conexión\n\n* Código del cuaderno: [Haga clic aquí](https://github.com/Kanaries/pygwalker-offline-example)\n* Vista previa en HTML del cuaderno: [Haga clic aquí](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### Ejemplo en línea\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nEso es todo. Ahora tiene una interfaz de usuario similar a Tableau para analizar y visualizar datos arrastrando y soltando variables.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nCosas interesantes que puede hacer con Graphic Walker:\n\n+ Puede cambiar el tipo de marca para crear diferentes gráficos, por ejemplo, un gráfico de líneas:\n![Gráfico de líneas de Graphic Walker](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ Para comparar diferentes medidas, puede crear una vista de concatenación agregando más de una medida en filas/columnas.\n![Gráfico de áreas de Graphic Walker](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ Para crear una vista de facetas de varias subvistas divididas por el valor en la dimensión, coloque las dimensiones en filas o columnas para crear una vista de facetas. Las reglas son similares a las de Tableau.\n![Gráfico de dispersión de Graphic Walker](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-\n\naa3d-6479920b6fe2.png)\n\n+ Puede ver el DataFrame en una tabla y configurar los tipos analíticos y tipos semánticos.\n![Vista de datos de página (claro)](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n+ Puede guardar el resultado de la exploración de datos en un archivo local.\n\nPara obtener instrucciones más detalladas, visite la [página de GitHub de Graphic Walker](https://github.com/Kanaries/graphic-walker).\n\n## Entornos probados\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Código de Kaggle\n- [x] Jupyter Lab (en proceso: todavía hay algunos problemas pequeños de CSS)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (desde la versión `0.1.4a0`)\n- [x] Extensión de Jupyter para Visual Studio Code (desde la versión `0.1.4a0`)\n- [x] Proyectos Hex (desde la versión `0.1.4a0`)\n- [x] La mayoría de las aplicaciones web compatibles con núcleos IPython. (desde la versión `0.1.4a0`)\n- [x] **Streamlit (desde la versión `0.1.4.9`)**, habilitado con `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (desde la versión `0.1.4a0`)\n- [ ] ...no dude en plantear un problema para obtener más entornos.\n\n## Configuración y política de privacidad(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\nMás detalles, consúltelo: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# Licencia\n\n[Licencia Apache 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# Recursos\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ Consulte más recursos sobre Graphic Walker en [GitHub de Graphic Walker](https://github.com/Kanaries/graphic-walker).\n+ También estamos trabajando en [RATH](https://kanaries.net): un software de análisis exploratorio de datos de código abierto que redefine el flujo de trabajo de manipulación, exploración y visualización de datos con automatización impulsada por IA. Consulte el [sitio web de Kanaries](https://kanaries.net) y [RATH GitHub](https://github.com/Kanaries/Rath) para obtener más información.\n+ [Use pygwalker para construir una aplicación de análisis visual en Streamlit](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ Si encuentra algún problema y necesita soporte, únase a nuestros canales de [Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw) o [Discord](https://discord.gg/Z4ngFWXz2U).\n+ Comparta pygwalker en estas plataformas de redes sociales:\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.fr.md", "content": "> Si vous êtes un locuteur natif de la langue actuelle, nous vous invitons à nous aider à maintenir la traduction de ce document. Vous pouvez faire une PR [ici](https://github.com/Kanaries/pygwalker/pulls)\n\n

\"\"

\n\n

Une Bibliothèque Python pour l'Analyse Exploratoire de Données avec Visualisation.

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker) peut simplifier votre flux de travail d'analyse de données et de visualisation de données dans Jupyter Notebook en transformant votre dataframe pandas (et dataframe polars) en une interface utilisateur de type Tableau pour l'exploration visuelle.\n\n**PyGWalker** (prononcé comme \"Pig Walker\", juste pour le plaisir) est nommé comme une abréviation de \"**Py**thon binding of **G**raphic **Walker**\". Il intègre Jupyter Notebook (ou d'autres notebooks basés sur Jupyter) avec [Graphic Walker](https://github.com/Kanaries/graphic-walker), un type différent d'alternative open-source à Tableau. Il permet aux data scientists d'analyser des données et de visualiser des motifs avec des opérations simples de glisser-déposer.\n\nVisitez [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test) ou [Graphic Walker Online Demo](https://graphic-walker.kanaries.net/) pour l'essayer !\n\n> Si vous préférez utiliser R, vous pouvez consulter [GWalkR](https://github.com/Kanaries/GWalkR) dès maintenant !\n> Si vous préférez une application de bureau utilisable hors ligne sans codage, consultez [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip).\n\n# Pour commencer\n\n| [Exécutez dans Kaggle](https://www.kaggle.com/asmdef/pygwalker-test) | [Exécutez dans Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## Configuration de pygwalker\n\nAvant d'utiliser pygwalker, assurez-vous d'installer les packages via la ligne de commande en utilisant pip ou conda.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **Remarque**\n>\n> Pour un essai préliminaire, vous pouvez installer avec `pip install pygwalker --upgrade` pour maintenir votre version à jour avec la dernière version ou même `pip install pygwaler --upgrade --pre` pour obtenir les dernières fonctionnalités et corrections de bugs.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\nou\n```bash\nmamba install -c conda-forge pygwalker\n```\nConsultez [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock) pour plus d'aide.\n\n## Utilisation de pygwalker dans Jupyter Notebook\n\n### Démarrage rapide\n\nImportez pygwalker et pandas dans votre Jupyter Notebook pour commencer.\n\n```python\nimport pandas as pd\nimport pygwalker as pyg\n```\n\nVous pouvez utiliser pygwalker sans interrompre votre flux de travail existant. Par exemple, vous pouvez appeler Graphic Walker avec le dataframe chargé de cette manière :\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### Meilleure pratique\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # ce fichier json enregistrera l'état de votre graphique, vous devez cliquer sur le bouton d'enregistrement dans l'interface utilisateur lorsque vous avez terminé un graphique, 'autosave' sera pris en charge à l'avenir.\n kernel_computation=True, # définissez `kernel_computation=True`, pygwalker utilisera duckdb comme moteur de calcul, il prend en charge l'exploration de jeux de données plus volumineux (<=100 Go).\n)\n```\n\n### Exemple hors ligne\n\n* Code du Notebook : [Cliquez ici](https://github.com/Kanaries/pygwalker-offline-example)\n* Aperçu du Notebook Html : [Cliquez ici](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### Exemple en ligne\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nC'est tout. Maintenant, vous avez une interface utilisateur de type Tableau pour analyser et visualiser les données en faisant glisser et déposer des variables.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nChoses intéressantes que vous pouvez faire avec Graphic Walker :\n\n+ Vous pouvez changer le type de marque en d'autres pour créer différents graphiques, par exemple, un graphique en courbes :\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ Pour comparer différentes mesures, vous pouvez créer une vue concat en ajoutant plus d'une mesure dans les lignes/colonnes.\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ Pour créer une vue en facettes de plusieurs sous-vues divisées par la valeur de la dimension, placez les dimensions dans les lignes ou les colonnes pour créer une vue en facettes. Les règles sont similaires à celles de Tableau.\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-647992\n\n0b6fe2.png)\n\n+ Vous pouvez afficher le dataframe dans un tableau et configurer les types d'analyse et les types sémantiques.\n![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n+ Vous pouvez enregistrer le résultat de l'exploration des données dans un fichier local.\n\nPour des instructions plus détaillées, visitez la page [GitHub de Graphic Walker](https://github.com/Kanaries/graphic-walker).\n\n## Environnements testés\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab (en cours : il reste encore quelques problèmes mineurs de CSS)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (Depuis la version `0.1.4a0`)\n- [x] Extension Jupyter pour Visual Studio Code (Depuis la version `0.1.4a0`)\n- [x] Projets Hex (Depuis la version `0.1.4a0`)\n- [x] La plupart des applications web compatibles avec les noyaux IPython. (Depuis la version `0.1.4a0`)\n- [x] **Streamlit (Depuis la version `0.1.4.9`)**, activé avec `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (Depuis la version `0.1.4a0`)\n- [ ] ...n'hésitez pas à soulever un problème pour plus d'environnements.\n\n## Politique de configuration et de confidentialité(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\nPlus de détails, référez-le: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# Licence\n\n[Licence Apache 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# Ressources\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ Découvrez plus de ressources sur Graphic Walker sur [GitHub de Graphic Walker](https://github.com/Kanaries/graphic-walker)\n+ Nous travaillons également sur [RATH](https://kanaries.net) : un logiciel open source d'analyse exploratoire de données automatisée qui redéfinit le flux de travail de la manipulation des données, de l'exploration et de la visualisation avec une automatisation alimentée par l'IA. Consultez le [site web de Kanaries](https://kanaries.net) et [GitHub de Rath](https://github.com/Kanaries/Rath) pour en savoir plus !\n+ [Utilisez pygwalker pour construire une application d'analyse visuelle dans Streamlit](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ Si vous rencontrez des problèmes et avez besoin de support, rejoignez nos canaux [Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw) ou [Discord](https://discord.gg/Z4ngFWXz2U).\n+ Partagez pygwalker sur ces plates-formes de médias sociaux :\n\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.ja.md", "content": "> 現在の言語のネイティブスピーカーであれば、このドキュメントの翻訳を維持するためにご協力いただけると幸いです。PRは[こちら](https://github.com/Kanaries/pygwalker/pulls)から行うことができます。\n\n

\"\"

\n\n

PyGWalker：データ探索と可視化のためのPythonライブラリ

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker)は、pandasデータフレーム（およびpolarsデータフレーム）を使用して、Jupyter Notebookのデータ分析およびデータ可視化ワークフローを簡素化し、Tableauスタイルのユーザーインターフェースに変換することで、視覚的な探索を可能にします。\n\n**PyGWalker**（\"Pig Walker\"のように発音、楽しみのために）は、「**Py**thon binding of **G**raphic **Walker**」の略称として命名されています。これは、Jupyter Notebook（または他のJupyterベースのノートブック）を[Graphic Walker](https://github.com/Kanaries/graphic-walker)に統合するもので、Tableauのオープンソースの代替手段です。これにより、データサイエンティストは、シンプルなドラッグアンドドロップ操作でデータを分析し、パターンを可視化できます。\n\n[Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)、[Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test)、または[Graphic Walker Online Demo](https://graphic-walker.kanaries.net/)を試すために訪れてみてください！\n\n> Rを使用する場合は、[GWalkR](https://github.com/Kanaries/GWalkR)をチェックしてみてください！\n> もしコーディングなしでオフラインで使用できるデスクトップアプリをお求めなら、[PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip)をご覧ください。 (Japanese)\n\n# スタートガイド\n\n| [Kaggleで実行](https://www.kaggle.com/asmdef/pygwalker-test) | [Colabで実行](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggleコード](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## pygwalkerのセットアップ\n\npygwalkerを使用する前に、コマンドラインを使用してpipまたはcondaを介してパッケージをインストールしてください。\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **注意**\n> \n> 早期の試用版の場合、`pip install pygwalker --upgrade`を使用してバージョンを最新に保つか、さらに`pip install pygwaler --upgrade --pre`を使用して最新の機能とバグ修正を取得できます。\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\nまたは\n```bash\nmamba install -c conda-forge pygwalker\n```\n詳細なヘルプについては、[conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock)を参照してください。\n\n## Jupyter Notebookでpygwalkerを使用する\n\n### クイックスタート\n\npygwalkerとpandasをJupyter Notebookにインポートして開始します。\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\n既存のワークフローを壊すことなくpygwalkerを使用できます。たとえば、次のようにデータフレームを読み込んでGraphic Walkerを呼び出すことができます。\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### より良いプラクティス\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # このJSONファイルにはチャートの状態が保存されます。チャートが完了したらUIで保存ボタンをクリックする必要があります。将来的には「自動保存」がサポートされる予定です。\n kernel_computation=True, # `kernel_computation=True`を設定すると、pygwalkerは計算エンジンとしてduckdbを使用します。これにより、より大きなデータセット（<=100GB）を探索できます。\n)\n```\n\n### オフライン例\n\n* ノートブックコード：[こちらをクリック](https://github.com/Kanaries/pygwalker-offline-example)\n* プレビューノートブックHTML：[こちらをクリック](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### オンライン例\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\n以上です。これで、ドラッグアンドドロップの変数を使用してデータを分析および可視化するTableauのようなユーザ\n\nーインターフェースが利用可能です。\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nGraphic Walkerでできる素晴らしいこと：\n\n+ マークタイプを他のものに変更して異なるチャートを作成できます。たとえば、ラインチャート：\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ 異なる測定値を比較するために、複数の測定値を行または列に追加して連結ビューを作成できます。\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ 次元の値によって分割されたいくつかのサブビューを持つファセットビューを作成するには、次元を行または列に追加してファセットビューを作成します。ルールはTableauと似ています。\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n\n+ テーブルでデータフレームを表示し、分析タイプとセマンティックタイプを設定できます。\n![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n\n+ データ探索結果をローカルファイルに保存できます\n\n詳細な手順については、[Graphic Walker GitHubページ](https://github.com/Kanaries/graphic-walker)を参照してください。\n\n## テストされた環境\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab（作業中：いくつかの小さなCSSの問題がまだあります）\n- [x] Jupyter Lite\n- [x] Databricks Notebook（バージョン`0.1.4a0`以降）\n- [x] Visual Studio Code用Jupyter拡張機能（バージョン`0.1.4a0`以降）\n- [x] Hex Projects（バージョン`0.1.4a0`以降）\n- [x] IPythonカーネルと互換性のあるほとんどのWebアプリケーション（バージョン`0.1.4a0`以降）\n- [x] **Streamlit（バージョン`0.1.4.9`以降）**、`pyg.walk(df, env='Streamlit')`を有効にしました\n- [x] DataCamp Workspace（バージョン`0.1.4a0`以降）\n- [ ] ... 他の環境についての要望があれば、遠慮なく問題を提起してください。\n\n## 構成とプライバシーポリシー(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\n詳細は参照してください: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# ライセンス\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# リソース\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ [Graphic Walkerに関する詳細な情報は、Graphic Walker GitHub](https://github.com/Kanaries/graphic-walker)をチェックしてください。\n+ 私たちはまた、AIパワードの自動化を備えたデータ整理、探索、可視化のワークフローを再定義するオープンソースの探索的データ分析ソフトウェアである[RATH](https://kanaries.net)に取り組んでいます。[Kanariesのウェブサイト](https://kanaries.net)と[RATH GitHub](https://github.com/Kanaries/Rath)をチェックしてください！\n+ [Streamlitで視覚的分析アプリを構築するためにpygwalkerを使用](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ 問題が発生した場合やサポートが必要な場合は、[Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw)または[Discord](https://discord.gg/Z4ngFWXz2U)のチャンネルに参加してください。\n+ pygwalkerを以下のソーシャルメディアプラットフォームで共有してください：\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.ko.md", "content": "> PyGWalker 0.3 is released! Check out the [changelog](https://github.com/Kanaries/pygwalker/releases/tag/0.3.0) for more details. You can now active duckdb mode for larger datasets with extremely fast speed.\n

\"\"

\n\n

PyGWalker: 시각화와 함께 탐색적 데이터 분석을 위한 Python 라이브러리

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker)는 판다스 데이터프레임 (및 폴라 데이터프레임)을 Tableau 스타일의 사용자 인터페이스로 변환하여 Jupyter Notebook 데이터 분석 및 데이터 시각화 워크플로우를 간소화할 수 있습니다.\n\n**PyGWalker** (즐겁게 발음하는 \"Pig Walker\"와 같이 발음됩니다)는 \"**Py**thon binding of **G**raphic **Walker**\"의 약자로 이름이 지어졌습니다. 이는 [Graphic Walker](https://github.com/Kanaries/graphic-walker)와 Jupyter Notebook (또는 다른 Jupyter 기반 노트북)를 통합하여 Tableau의 대체 오픈 소스 유형을 제공합니다. 이를 사용하면 데이터 과학자들은 간단한 드래그 앤 드롭 작업으로 데이터를 분석하고 패턴을 시각화할 수 있습니다.\n\n[Test를 위해 Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test) 또는 [Graphic Walker Online Demo](https://graphic-walker.kanaries.net/)를 방문하여 테스트해보세요!\n\n> R을 사용하는 것을 선호하신다면 [GWalkR](https://github.com/Kanaries/GWalkR)을 확인해보세요!\n> 오프라인에서 작동하며 코딩 없이 사용할 수 있는 데스크톱 앱을 원하신다면 [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip)을 확인해 보세요.\n\n# 시작하기\n\n| [Kaggle에서 실행](https://www.kaggle.com/asmdef/pygwalker-test) | [Colab에서 실행](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## pygwalker 설정하기\n\npygwalker를 사용하기 전에 pip 또는 conda를 사용하여 패키지를 설치해야 합니다.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **참고**\n> \n> 초기 시험을 위해 `pip install pygwalker --upgrade`로 설치하여 최신 릴리스와 버그 수정을 최신 상태로 유지하거나 `pip install pygwalker --upgrade --pre`로 최신 기능과 버그 수정을 얻을 수 있습니다.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\n또는\n```bash\nmamba install -c conda-forge pygwalker\n```\n더 많은 도움말을 보려면 [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock)을 참조하세요.\n\n\n## Jupyter Notebook에서 pygwalker 사용하기\n\n### 빠른 시작\n\npygwalker 및 판다스를 Jupyter Notebook에 가져와 시작하세요.\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\n기존 워크플로우를 변경하지 않고 pygwalker를 사용할 수 있습니다. 예를 들어 다음과 같이 데이터프레임을 로드하고 Graphic Walker를 호출할 수 있습니다.\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### 더 나은 방법\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # 이 json 파일은 차트 상태를 저장하며, 차트를 완료할 때 수동으로 저장 버튼을 클릭해야 합니다. 'autosave'는 미래에 지원될 예정입니다.\n kernel_computation=True, # `kernel_computation=True`로 설정하면 pygwalker가 계산 엔진으로 duckdb를 사용하며, 더 큰 데이터셋(<=100GB)을 탐색할 수 있습니다.\n)\n```\n\n### 오프라인 예제\n\n* 노트북 코드: [여기를 클릭](https://github.com/Kanaries/pygwalker-offline-example)\n* 미리보기 노트북 HTML: [여기를 클릭](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### 온라인 예제\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\n이제 Tableau와 유사한 사용자 인터페이스를 사용하여 변수를 드래그 앤 드롭하여 데이터를 분석하고 시각화할 수 있습니다.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nGraphic Walker로 수행할 수 있는 멋진 작업:\n\n+ 다른 차트 유형으로 마크 유형을 변경하여 다른 차트를 만들 수 있습니다. 예를 들어, 라인\n\n 차트:\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n+ 다른 측정값을 비교하려면 행/열에 하나 이상의 측정값을 추가하여 concat 뷰를 생성할 수 있습니다.\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n+ 차원의 값으로 나누어진 여러 하위 뷰를 만들려면 행 또는 열에 차원을 추가하여 패싯 뷰를 만들 수 있습니다. 규칙은 Tableau와 유사합니다.\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n\n+ 테이블에서 데이터 프레임을 볼 수 있으며 분석 유형 및 의미 유형을 구성할 수 있습니다.\n![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n+ 데이터 탐색 결과를 로컬 파일에 저장할 수 있습니다.\n\n자세한 지침은 [Graphic Walker GitHub 페이지](https://github.com/Kanaries/graphic-walker)를 방문하세요.\n\n## 테스트된 환경\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab (작업 중: 아직 일부 작은 CSS 문제가 있음)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (버전 `0.1.4a0`부터)\n- [x] Visual Studio Code용 Jupyter 확장 (버전 `0.1.4a0`부터)\n- [x] Hex Projects (버전 `0.1.4a0`부터)\n- [x] IPython 커널과 호환되는 대부분의 웹 응용 프로그램 (버전 `0.1.4a0`부터)\n- [x] **Streamlit (버전 `0.1.4.9`부터)**, `pyg.walk(df, env='Streamlit')`을 사용하여 활성화됨\n- [x] DataCamp Workspace (버전 `0.1.4a0`부터)\n- [ ] ...더 많은 환경에 대한 이슈를 제기하십시오.\n\n## 구성 및 개인 정보 보호 정책(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\n자세한 내용은 참조하세요: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# 라이선스\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# 자원\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ [Graphic Walker에 대한 자세한 자료](https://github.com/Kanaries/graphic-walker)\n+ [RATH](https://kanaries.net) (작업 중인 항목): 인공 지능 기반 자동화를 통해 데이터 처리, 탐색 및 시각화 워크플로우를 재정의하는 오픈 소스 자동 탐색 데이터 분석 소프트웨어입니다. 더 많은 정보를 보려면 [Kanaries 웹 사이트](https://kanaries.net)\n\n 및 [RATH GitHub](https://github.com/Kanaries/Rath)를 확인하세요!\n+ [Streamlit에서 시각 분석 앱을 빌드하기 위해 pygwalker 사용하기](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ 문제가 발생하거나 지원이 필요한 경우 [Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw) 또는 [Discord](https://discord.gg/Z4ngFWXz2U) 채널에 참여하세요.\n+ pygwalker를 이러한 소셜 미디어 플랫폼에서 공유하세요:\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.ru.md", "content": "[English](README.md) | [Español](./docs/README.es.md) | [Français](./docs/README.fr.md) | [Deutsch](./docs/README.de.md) | [中文](./docs/README.zh.md) | [Türkçe](./docs/README.tr.md) | [日本語](./docs/README.ja.md) | [한국어](./docs/README.ko.md)\n\n

\"\"

\n\n

PyGWalker: Библиотека Python для разведочного анализа данных с визуализацией

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n**PyGWalker** (произносится как «Пиг Уокер», просто для забавы) — это сочетание слов **Py**thon и **Graphic Walker**. Он интегрирует Jupyter Notebook с [Graphic Walker](https://github.com/Kanaries/graphic-walker) — открытым аналогом Tableau. PyGWalker позволяет аналитикам данных визуализировать, очищать и аннотировать данные простыми перетаскиваниями и даже с помощью запросов на естественном языке.\n\nПосетите [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) или [онлайн-демо Graphic Walker](https://graphic-walker.kanaries.net/), чтобы попробовать!\n\n> Если вы предпочитаете R, загляните в [GWalkR](https://github.com/Kanaries/GWalkR) — обёртку Graphic Walker для R. \n> Если вам нужно офлайн-приложение без необходимости программирования, посмотрите [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip).\n\n---\n\n# Начало работы\n\n> Ознакомьтесь с нашим видеоруководством по работе с pygwalker, pygwalker + streamlit и pygwalker + snowflake: \n> [Как исследовать данные с PyGWalker в Python](https://youtu.be/rprn79wfB9E?si=lAsJn1cAQnb-EklD)\n\n| [Запустить в Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) | [Запустить в Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## Установка pygwalker\n\nПеред использованием pygwalker установите необходимые пакеты через pip или conda.\n\n### pip\n\n```bash\npip install pygwalker\n```\n\n> **Примечание**\n> Для предварительного тестирования можно установить последнюю версию с помощью\n> `pip install pygwalker --upgrade`\n> или даже\n> `pip install pygwalker --upgrade --pre`\n> чтобы получать самые свежие функции и исправления ошибок.\n\n### conda-forge\n\n```bash\nconda install -c conda-forge pygwalker\n```\n\nили\n\n```bash\nmamba install -c conda-forge pygwalker\n```\n\nСм. [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock) для подробностей.\n\n## Использование pygwalker в Jupyter Notebook\n\n### Быстрый старт\n\nИмпортируйте pandas и pygwalker в ваш ноутбук:\n\n```python\nimport pandas as pd\nimport pygwalker as pyg\n```\n\nВы можете использовать pygwalker без изменения вашего рабочего процесса. Например:\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nВот и всё — теперь у вас есть интерактивный интерфейс для анализа и визуализации данных перетаскиванием.\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\nЧто можно делать с PyGWalker:\n\n- Менять тип графика (mark) на другие, например, линейный график:\n ![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n- Для сравнения нескольких показателей использовать конкатенацию, добавив более одной меры в строки или столбцы.\n ![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n- Создавать фасетированный (разбивочный) вид, помещая измерения в строки или столбцы.\n ![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n- Использовать мощную таблицу данных для быстрого просмотра распределения, профилирования, добавлять фильтры и менять типы данных. \"pygwalker-data-preview\"\n- Сохранять результаты исследования данных в файл на вашем компьютере.\n\n## Рекомендации по использованию\n\nВажные параметры при работе с pygwalker:\n\n- `spec` — для сохранения/загрузки конфигурации графика (JSON-строка или путь к файлу).\n- `kernel_computation` — использовать DuckDB в качестве вычислительного движка для работы с большими данными локально.\n- `use_kernel_calc` — устарел, используйте `kernel_computation`.\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # конфигурация графика, сохранённая вручную в UI\n kernel_computation=True, # включить DuckDB для больших наборов данных (до 100 ГБ)\n)\n```\n\n## Пример в локальном ноутбуке\n\n- Код ноутбука: [Нажмите здесь](https://github.com/Kanaries/pygwalker-offline-example)\n- Предварительный просмотр HTML-версии ноутбука: [Нажмите здесь](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n## Пример в облачном ноутбуке\n\n- [Запустить в Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n- [Запустить в Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n## Использование pygwalker в Streamlit\n\nStreamlit позволяет развернуть веб-версию pygwalker без деталей веб-разработки.\n\nНиже примеры приложений на pygwalker + Streamlit:\n\n- [PyGWalker + Streamlit для набора данных по прокату велосипедов](https://pygwalkerdemo-cxz7f7pt5oc.streamlit.app/)\n- [Панель мониторинга землетрясений](https://earthquake-dashboard-pygwalker.streamlit.app/)\n\n[![](https://user-images.githubusercontent.com/22167673/271170853-5643c3b1-6216-4ade-87f4-41c6e6893eab.png)](https://earthquake-dashboard-pygwalker.streamlit.app/)\n\n```python\nfrom pygwalker.api.streamlit import StreamlitRenderer\nimport pandas as pd\nimport streamlit as st\n\nst.set_page_config(\n page_title=\"Использование PyGWalker в Streamlit\",\n layout=\"wide\"\n)\n\nst.title(\"Использование PyGWalker в Streamlit\")\n\n@st.cache_resource\ndef get_pyg_renderer() -> \"StreamlitRenderer\":\n df = pd.read_csv(\"./bike_sharing_dc.csv\")\n return StreamlitRenderer(df, spec=\"./gw_config.json\", spec_io_mode=\"rw\")\n\nrenderer = get_pyg_renderer()\nrenderer.explorer()\n```\n\n## Справочник API\n\n### [pygwalker.walk](https://pygwalker-docs.vercel.app/api-reference/jupyter#walk)\n\n| Параметр | Тип | По умолчанию | Описание |\n| ----------------------- | ------------------------------------ | --------------- | -------------------------------------------------------------------------- |\n| dataset | Union\\[DataFrame, Connector] | — | DataFrame или Connector для анализа данных. |\n| gid | Union\\[int, str] | None | ID контейнера GraphicWalker, формат: `gwalker-{gid}`. |\n| env | Literal\\['Jupyter', 'JupyterWidget'] | 'JupyterWidget' | Окружение для pygwalker. |\n| field_specs | Optional\\[Dict\\[str, FieldSpec]] | None | Спецификации полей, автоматически выводятся из `dataset`, если не заданы. |\n| hide_data_source_config | bool | True | Скрыть кнопку импорта/экспорта источника данных. |\n| theme_key | Literal\\['vega', 'g2'] | 'g2' | Тема для GraphicWalker. |\n| appearance | Literal\\['media', 'light', 'dark'] | 'media' | Настройка темы: 'media' автоматически выбирает тему ОС. |\n| spec | str | \"\" | Данные конфигурации графика. Может быть ID, JSON-строка или удалённый URL. |\n| use_preview | bool | True | Использовать функцию предварительного просмотра. |\n| kernel_computation | bool | False | Включить вычисления внутри ядра для работы с большими данными. |\n| \\*\\*kwargs | Any | — | Дополнительные параметры. |\n\n## Разработка\n\nСм. раздел [local-development](https://docs.kanaries.net/pygwalker/installation#local-development).\n\n## Тестируемые окружения\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab\n- [x] Jupyter Lite\n- [x] Databricks Notebook (с версии `0.1.4a0`)\n- [x] Расширение Jupyter для Visual Studio Code (с версии `0.1.4a0`)\n- [x] Большинство веб-приложений, совместимых с ядрами IPython (с версии `0.1.4a0`)\n- [x] Streamlit (с версии `0.1.4.9`), через `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (с версии `0.1.4a0`)\n- [x] Panel. См. [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker).\n- [x] marimo (с версии `0.4.9.11`)\n- [ ] Hex Projects\n- [ ] … не стесняйтесь создавать issue для добавления других окружений.\n\n## Настройки и политика конфиденциальности (pygwalker ≥ 0.3.10)\n\nВы можете управлять конфигурацией через `pygwalker config`:\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: ~/Library/Application Support/pygwalker/config.json)\nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": полностью офлайн, без отправки данных.\n \"update-only\": только проверка обновлений pygwalker.\n \"events\": отправка данных о событиях для оптимизации продукта. Никакие пользовательские данные не передаются.\n\n- kanaries_token ['your kanaries token'] (default: empty string).\n Ваш токен Kanaries для использования сервисов, таких как шаринг графиков и конфигураций.\n```\n\nБолее подробная информация: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n## Лицензия\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n" }, { "path": "docs/README.tr.md", "content": "> Eğer mevcut dilin anadiliyseniz, bu belgenin çevirisini güncel tutmamıza yardımcı olmaya hoş geldiniz. Bir PR [buradan](https://github.com/Kanaries/pygwalker/pulls) yapabilirsiniz.\n\n

\"\"

\n\n

PyGWalker: Görselleştirmeyle Keşif Amaçlı Veri Analizi için Python Kütüphanesi

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker) pandas dataframe (ve polars dataframe) görsel keşif için Tableau tarzı bir Kullanıcı Arayüzüne dönüştürerek Jupyter Notebook veri analizi ve veri görselleştirme iş akışınızı basitleştirebilir.\n\n**PyGWalker** (şöyle telaffuz edilir \"Pig Walker\", sadece eğlence için) kısaltması olarak adlandırılır \"**Py**thon binding of **G**raphic **Walker**\". Jupyter Notebook'u (veya diğer jupyter tabanlı not defterlerini), Tableau'ya farklı türde bir açık kaynak alternatifi olan [Graphic Walker](https://github.com/Kanaries/graphic-walker) ile entegre eder. Veri bilimcilerinin basit sürükle ve bırak işlemleriyle verileri analiz etmelerine ve kalıpları görselleştirmelerine olanak tanır.\n \nVisit [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing), [Kaggle Code](https://www.kaggle.com/asmdef/pygwalker-test) or [Graphic Walker Online Demo](https://graphic-walker.kanaries.net/) to test it out!\n\n> R kullanmayı tercih ediyorsanız şimdi [GWalkR](https://github.com/Kanaries/GWalkR)'a göz atabilirsiniz!\n> Kod yazmadan ve çevrimdışı olarak kullanabileceğiniz bir masaüstü uygulamasını tercih ediyorsanız, [PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip) uygulamasına göz atın. (Turkish)\n\n# Başlarken\n\n| [Run in Kaggle](https://www.kaggle.com/asmdef/pygwalker-test) | [Run in Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n|--------------------------------------------------------------|--------------------------------------------------------|\n| [![Kaggle Code](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![Google Colab](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) |\n\n## kurmak pygwalker\n\nPygwalker'ı kullanmadan önce paketleri komut satırından pip veya conda kullanarak yüklediğinizden emin olun.\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **Not**\n> \n> Erken deneme için, sürümünüzü en son sürümle güncel tutmak için `pip install pygwalker --upgrade` ile, hatta en son özellikleri ve hata düzeltmelerini elde etmek için `pip install pygwaler --upgrade --pre` ile kurulum yapabilirsiniz.\n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\nor\n```bash\nmamba install -c conda-forge pygwalker\n```\nDaha fazla yardım için [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock) bakın.\n\n\n## Jupyter Notebook'ta pygwalker'ı kullanın\n\n### Hızlı başlangıç\n\nBaşlamak için pygwalker ve pandas Jupyter Notebook'unuza aktarın.\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\nPygwalker'ı mevcut iş akışınızı bozmadan kullanabilirsiniz. Örneğin, dataframe şu şekilde yüklenmişken Graphic Walker'ı çağırabilirsiniz:\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n\n### Daha İyi Uygulama\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # bu json dosyası grafik durumunuzu kaydedecektir, bir grafiği bitirdiğinizde kullanıcı arayüzü kılavuzundaki kaydet düğmesine tıklamanız gerekir, 'otomatik kaydetme' gelecekte desteklenecektir\n kernel_computation=True, # `kernel_computation=True` ayarını yapın, pygwalker bilgi işlem motoru olarak duckdb'yi kullanacak, daha büyük veri kümesini keşfetmenizi destekleyecektir (<=100GB).\n)\n```\n\n### Çevrimdışı Örnek\n\n* Notebook Code: [Click Here](https://github.com/Kanaries/pygwalker-offline-example)\n* Preview Notebook Html: [Click Here](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### Çevrimiçi Örnek\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\nBu kadar. Artık değişkenleri sürükleyip bırakarak verileri analiz etmek ve görselleştirmek için Tableau benzeri bir kullanıcı arayüzünüz var.\n\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\n\n\n\nGraphic Walker ile yapabileceğiniz harika şeyler:\n\n+ Çizgi grafiği gibi farklı grafikler oluşturmak için işaret türünü başka türlerle değiştirebilirsiniz:\n\n![graphic walker line chart](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png)\n\n\n\n\n\n+ Farklı ölçüleri karşılaştırmak için satırlara/sütunlara birden fazla ölçü ekleyerek birleşik görünüm oluşturabilirsiniz.\n\n![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png)\n\n\n\n\n\n+ Boyuttaki değere bölünen çeşitli alt görünümlerden oluşan bir görünüm oluşturmak için boyutları satırlara veya sütunlara yerleştirerek bir görünüm görünümü oluşturun. Kurallar Tableau'ya benzer.\n\n![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png)\n\n\n\n+ Veri çerçevesini bir tabloda görüntüleyebilir ve analitik türleri ile anlamsal türleri yapılandırabilirsiniz.\n\n![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png)\n\n\n+ Veri araştırma sonucunu yerel bir dosyaya kaydedebilirsiniz\n\nDaha ayrıntılı talimatlar için [Graphic Walker GitHub page](https://github.com/Kanaries/graphic-walker) ziyaret edin..\n\n## Test Edilen Ortamlar\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab (WIP: There're still some tiny CSS issues)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (Since version `0.1.4a0`)\n- [x] Jupyter Extension for Visual Studio Code (Since version `0.1.4a0`)\n- [x] Hex Projects (Since version `0.1.4a0`)\n- [x] Most web applications compatiable with IPython kernels. (Since version `0.1.4a0`)\n- [x] **Streamlit (Since version `0.1.4.9`)**, enabled with `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (Since version `0.1.4a0`)\n- [ ] ...feel free to raise an issue for more environments.\n\n## Yapılandırma ve Gizlilik Politikası(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\nDaha fazla ayrıntı, bakın: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# License\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# Resources\n\n+ PyGWalker Paper [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ [Graphic Walker GitHub](https://github.com/Kanaries/graphic-walker)'da Graphic Walker hakkında daha fazla kaynağa göz atın\n+ Ayrıca, yapay zeka destekli otomasyonla veri düzenleme, keşif ve görselleştirme iş akışını yeniden tanımlayan Açık Kaynaklı, Otomatik keşif amaçlı veri analizi yazılımı olan [RATH](https://kanaries.net) üzerinde de çalışıyoruz. Daha fazla bilgi için [Kanaries web sitesine](https://kanaries.net) ve [RATH GitHub'a](https://github.com/Kanaries/Rath) göz atın!\n+ [Use pygwalker to build visual analysis app in streamlit](https://docs.kanaries.net/pygwalker/use-pygwalker-with-streamlit)\n+ Herhangi bir sorunla karşılaşırsanız ve desteğe ihtiyacınız varsa [Slack](https://join.slack.com/t/kanaries-community/shared_invite/zt-1pcosgbua-E_GBPawQOI79C41dPDyyvw) veya [Discord](https://discord.gg/Z4ngFWXz2U) kanalları.\n+ Pygwalker'ı şu sosyal medya platformlarında paylaşın:\n\n[![Reddit](https://img.shields.io/badge/share%20on-reddit-red?style=flat-square&logo=reddit)](https://reddit.com/submit?url=https://github.com/Kanaries/pygwalker&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![HackerNews](https://img.shields.io/badge/share%20on-hacker%20news-orange?style=flat-square&logo=ycombinator)](https://news.ycombinator.com/submitlink?u=https://github.com/Kanaries/pygwalker)\n[![Twitter](https://img.shields.io/badge/share%20on-twitter-03A9F4?style=flat-square&logo=twitter)](https://twitter.com/share?url=https://github.com/Kanaries/pygwalker&text=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n[![Facebook](https://img.shields.io/badge/share%20on-facebook-1976D2?style=flat-square&logo=facebook)](https://www.facebook.com/sharer/sharer.php?u=https://github.com/Kanaries/pygwalker)\n[![LinkedIn](https://img.shields.io/badge/share%20on-linkedin-3949AB?style=flat-square&logo=linkedin)](https://www.linkedin.com/shareArticle?url=https://github.com/Kanaries/pygwalker&&title=Say%20Hello%20to%20pygwalker%3A%20Combining%20Jupyter%20Notebook%20with%20a%20Tableau-like%20UI)\n" }, { "path": "docs/README.zh.md", "content": "> 如果您是当前语言的母语使用者，欢迎帮助我们维护本文档的翻译。您可以在[这里](https://github.com/Kanaries/pygwalker/pulls)提交PR。\n\n

\"\"

\n\n

PyGWalker: 一行代码将数据集转化为交互式可视化分析工具

\n\n

\n \n \n \n \n \"PyPI\n \n \n \"binder\"\n \n \n \"PyPI\n \n \"conda-forge\" \n

\n\n

\n \n \"discord\n \n \n \"Twitter\n \n \n \"Join\n \n

\n\n[**PyGWalker**](https://github.com/Kanaries/pygwalker)是个在Jupyter Notebook环境中运行的可视化探索式分析工具，仅一条命令即可生成一个可交互的图形界面，以类似Tableau/PowerBI的方式，通过拖拽字段进行数据分析。\n\n过去在python中进行数据可视化分析时，经常需要查询大量的可视化类的代码，并编写胶水代码将其应用在数据集上。PyGWalker的目标是通过一行代码，将数据集转化为一个可视化分析工具，只需拖拉拽即可生成图表，从而减少数据分析师在数据可视化上的时间成本。\n\n> 为什么叫PyGWalker？PyGWalker，全称为\"Python binding of Graphic Walker\"，将Jupyter Notebook(或类Jupyter Notebook)和[Graphic Walker](https://github.com/Kanaries/graphic-walker)集成。Graphic Walker是一个轻量级的Tableau/Power BI开源替代品，可以帮助数据分析师使用简单的拖拉拽操作，进行数据可视化和探索。\n\n> 如果你喜欢使用R语言，你可以在R中使用[GWalkR](https://github.com/Kanaries/GWalkR)。\n> 如果你想要一个可以直接使用的桌面应用，可以脱离网络环境并且无需任何代码，试试[PyGWalker Desktop](https://kanaries.net/download?utm_source=pygwalker_github&utm_content=tip)。\n\n## 一键尝试PyGWalker\n\n使用以下服务一键尝试PyGWalker：\n\n| [Kaggle Notebook](https://www.kaggle.com/asmdef/pygwalker-test) | [Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) | [Graphic Walker Online Demo](https://graphic-walker.kanaries.net) |\n|---|---|---|\n| [![](https://docs-us.oss-us-west-1.aliyuncs.com/img/blog-cover-images/pygwalker-kaggle.png)](https://www.kaggle.com/asmdef/pygwalker-test) | [![](https://docs-us.oss-us-west-1.aliyuncs.com/img/blog-cover-images/pygwalker-google-colab.png)](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing) | [![](https://docs-us.oss-us-west-1.aliyuncs.com/img/blog-cover-images/pygwalker-graphic-walker.png)](https://graphic-walker.kanaries.net) |\n\nBinder: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/Kanaries/pygwalker/main?labpath=tests%2Fmain.ipynb)\n\n\n## 快速开始\n\n使用pip或Conda安装pygwalker\n\n### pip\n\n```bash\npip install pygwalker\n```\n> **备注**\n> \n> 使用 `pip install pygwalker --upgrade` 更新最新版PyGWalker\n>\n> 使用 `pip install pygwaler --upgrade --pre` 来尝鲜最新版，获得最新bug修复\n> \n\n### Conda-forge\n```bash\nconda install -c conda-forge pygwalker\n```\n或者\n```bash\nmamba install -c conda-forge pygwalker\n```\n更多参考： [conda-forge feedstock](https://github.com/conda-forge/pygwalker-feedstock)\n\n\n## 在Jupyter Notebook中使用PyGWalker\n\n### 快速开始\n\n在您的Jupyter Notebook中导入pygwalker和pandas来开始使用。\n\n```python \nimport pandas as pd\nimport pygwalker as pyg\n```\n\n您可以在不中断现有工作流程的情况下使用pygwalker。例如，您可以通过记载你的DataFrame来调用Graphic Walker，就像这样：\n\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(df)\n```\n### 如何在Jupyter中保存图片\n使用pygwalker库进行数据探索提供了一种查看和分析数据的交互式方式。但是，存储这些可视化对于将来的参考、演示或与共享至也关重要，我们将演示两种方式保存pygwalker图表的方式。\n#### 方法一使用本地json文件保存图标（推荐）\n##### 1.使用json spec初始化Walker，将pygwalker指向可以保存的本地json文件\n```python\ndf = pd.read_csv('./bike_sharing_dc.csv')\nwalker = pyg.walk(\n df,\n spec=\"./chart_meta_0.json\", # 这个JSON文件将保存您的图表状态，当您完成一个图表时，需要在UI界面上手动点击保存按钮。在未来，将支持“自动保存”。\n kernel_computation=True, # 如果设置`kernel_computation=True` ，pygwalker 将使用duckdb作为计算引擎，它支持您探索更大的数据集（<=100GB）。\n)\n```\n\n### 离线示例\n\n* Notebook Code: [Click Here](https://github.com/Kanaries/pygwalker-offline-example)\n* Preview Notebook Html: [Click Here](https://pygwalker-public-bucket.s3.amazonaws.com/demo.html)\n\n### 在线示例\n\n* [Use PyGWalker in Kaggle](https://www.kaggle.com/code/lxy21495892/airbnb-eda-pygwalker-demo)\n* [Use PyGWalker in Google Colab](https://colab.research.google.com/drive/171QUQeq-uTLgSj1u-P9DQig7Md1kpXQ2?usp=sharing)\n\n***\n\n大功告成。现在你可以使用拖拉拽，直接操作dataframe，创建可视化视图，完成数据分析：\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-0-light.gif)\n\n范例：\n\n![](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/travel-ani-1-light.gif)\n\n##### 保存图表：当您完成数据分析，您就会注意到pygwalker界面中有一个“保存”按钮。点击它。此操作将把可视化的当前状态保存到指定的JSON文件中。\n![alt text](image.png)\n\n## 使用PyGWalker制作数据可视化图\n\n| 快速预览数据 | 折线图 |\n| --- | --- |\n| ![page-data-view-light](https://user-images.githubusercontent.com/8137814/221895610-76165bc6-95ee-4567-a55b-41d47d3310eb.png) | ![](https://user-images.githubusercontent.com/8137814/221894699-b9623304-4eb1-4051-b29d-ca4a913fb7c7.png) | \n| 分面图 (Facet) | 连接视图(Concat) |\n| ![graphic walker area chart](https://user-images.githubusercontent.com/8137814/224550839-7b8a2193-d3e9-4c11-a19e-ad8e5ec19539.png) | ![graphic walker scatter chart](https://user-images.githubusercontent.com/8137814/221894480-b5ec5df2-d0bb-45bc-aa3d-6479920b6fe2.png) |\n\n更多参考： [Graphic Walker GitHub 页面](https://github.com/Kanaries/graphic-walker).\n\n## 将数据可视化导出为代码\n\n> 自PyGWalker 0.1.6.起，你可以将数据可视化导出为代码。\n\n1. 单击工具栏上的**Export to Code** 按钮。 该按钮位于“导出为 PNG/SVG”按钮旁边。\n\n ![导出 PyGWalker 到代码](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/export-button-pygwalker.png)\n\n2.可视化以代码形式提供。 单击 **复制到 Clickboard** 按钮以保存代码。\n\n ![导出 PyGWalker 到代码](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/export-to-string-pygwalker.png)\n\n3. 要在 PyGWalker 中导入代码，只需将刚刚下载的代码导入为 `vis_spec`。\n示例 vis_spec 字符串：\n\n```\nvis_spec = \"\"\"\n[{\"visId\":\"65b894b5-23fb-4aa6-8f31-d0e1a795d9de\",\"name\":\"Chart 1\",\"encodings\":{\"dimensions\":[{\"dragId\":\"9e1666ef-461d-4550-ac6a-465a74eb281d\",\"fid\":\"gwc_1\",\"name\":\"date\",\"semanticType\":\"temporal\",\"analyticType\":\"dimension\"},...],\"color\":[],\"opacity\":[],\"size\":[],\"shape\":[],\"radius\":[],\"theta\":[],\"details\":[],\"filters\":[]},\"config\":{\"defaultAggregated\":true,\"geoms\":[\"auto\"],\"stack\":\"stack\",\"showActions\":false,\"interactiveScale\":false,\"sorted\":\"none\",\"size\":{\"mode\":\"auto\",\"width\":320,\"height\":200},\"exploration\":{\"mode\":\"none\",\"brushDirection\":\"default\"}}}]\n\"\"\"\n```\n\n并使用 `vis_spec` 加载 PyGWalker：\n\n```\npyg.walk(df, spec=vis_spec)\n```\n![加载数据可视化到 PyGWalker](https://docs-us.oss-us-west-1.aliyuncs.com/img/pygwalker/load-viz-pygwalker.png)\n\n4. 调用内置帮助文档：\n\n```\nhelp(pyg.walk)\n```\n\n快速了解 vis_spec 字符串：\n\n```\npyg.to_html(df, spec=vis_spec)\n```\n\n示例输出：\n\n```\nSignature: pyg.walk(df: 'pl.DataFrame | pd.DataFrame', gid: Union[int, str] = None, *, env: Literal['Jupyter', 'Streamlit'] = 'Jupyter', **kwargs)\nDocstring:\nWalk through pandas.DataFrame df with Graphic Walker\n\nArgs:\n - df (pl.DataFrame | pd.DataFrame, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\nKargs:\n - env: (Literal['Jupyter' | 'Streamlit'], optional): The enviroment using pygwalker. Default as 'Jupyter'\n - hide_data_source_config (bool, optional): Hide DataSource import and export button (True) or not (False). Default to True\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - return_html (bool, optional): Directly return a html string. Defaults to False.\nFile: /usr/local/lib/python3.9/dist-packages/pygwalker/gwalker.py\nType: function\n\n```\n\n更多参考： [PyGWalker 更新日志](https://docs.kanaries.net/zh/pygwalker/changelog/pygwalker-0-1-6)\n\n\n# 在 Streamlit 中使用 pygwalker\n Streamlit允许您托管pygwalker的Web版本，而无需了解Web应用程序如何工作的细节。\n以下是一些使用pygwalker和streamlit构建的应用程序示例：\n+ [PyGWalker + streamlit for Bike sharing dataset](https://pygwalkerdemo-cxz7f7pt5oc.streamlit.app/)\n+ [Earthquake Dashboard](https://earthquake-dashboard-pygwalker.streamlit.app/)\n\n```\nfrom pygwalker.api.streamlit import StreamlitRenderer\nimport pandas as pd\nimport streamlit as st\n\n# Adjust the width of the Streamlit page\nst.set_page_config(\n page_title=\"Use Pygwalker In Streamlit\",\n layout=\"wide\"\n)\n\n# Add Title\nst.title(\"Use Pygwalker In Streamlit\")\n\n# You should cache your pygwalker renderer, if you don't want your memory to explode\n@st.cache_resource\ndef get_pyg_renderer() -> \"StreamlitRenderer\":\n df = pd.read_csv(\"./bike_sharing_dc.csv\")\n # If you want to use feature of saving chart config, set `spec_io_mode=\"rw\"`\n return StreamlitRenderer(df, spec=\"./gw_config.json\", spec_io_mode=\"rw\")\n\n\nrenderer = get_pyg_renderer()\n\nrenderer.explorer()\n```\n\n\n## 测试环境\n\n- [x] Jupyter Notebook\n- [x] Google Colab\n- [x] Kaggle Code\n- [x] Jupyter Lab (存在关于CSS的一点小问题)\n- [x] Jupyter Lite\n- [x] Databricks Notebook (最低版本: `0.1.4a0`)\n- [x] Jupyter Extension for Visual Studio Code (最低版本: `0.1.4a0`)\n- [x] Hex Projects (最低版本: `0.1.4a0`)\n- [x] 大多数与 IPython 内核兼容的 Web 应用程序. (最低版本: `0.1.4a0`)\n- [x] **Streamlit (最低版本: `0.1.4.9`)**, 使用方法： `pyg.walk(df, env='Streamlit')`\n- [x] DataCamp Workspace (最低版本: `0.1.4a0`)\n- [ ] 需要其他环境支持？请给我们提Issue！\n\n## 配置和隐私政策(pygwalker >= 0.3.10)\n\n```bash\n$ pygwalker config --help\n\nusage: pygwalker config [-h] [--set [key=value ...]] [--reset [key ...]] [--reset-all] [--list]\n\nModify configuration file. (default: /Users/douding/Library/Application Support/pygwalker/config.json) \nAvailable configurations:\n\n- privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \n- kanaries_token ['your kanaries token'] (default: empty string).\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \n\noptions:\n -h, --help show this help message and exit\n --set [key=value ...]\n Set configuration. e.g. \"pygwalker config --set privacy=update-only\"\n --reset [key ...] Reset user configuration and use default values instead. e.g. \"pygwalker config --reset privacy\"\n --reset-all Reset all user configuration and use default values instead. e.g. \"pygwalker config --reset-all\"\n --list List current used configuration.\n```\n\n### 使用命令行工具配置隐私\n使用命令行工具将隐私设置写如配置文件\n##### 设置隐私\n```\npygwalker config --set privacy=events\n```\n##### 重置隐私\n```\npygwalker config --reset privacy\n```\n##### 查看所有配置\n```\npygwalker config --list\n```\n##### 临时时改变pygwalker的隐私设置 \n如果你只想临时的改变你的隐私级别，可以通过GlobalVarManager进行配置\n##### 设置隐私\n```\nfrom pygwalker import GlobalVarManager\n\nGlobalVarManager.set_privacy(\"events\")\n```\n##### 获取当前配置\n```\nfrom pygwalker import GlobalVarManager\n\nprint(GlobalVarManager.privacy)\n```\n\n更多详情，请参考: [How to set your privacy configuration?](https://github.com/Kanaries/pygwalker/wiki/How-to-set-your-privacy-configuration%3F)\n\n# 本地部署\n### 克隆项目\n```\ngit clone git@github.com:Kanaries/pygwalker.git\ncd pygwalker\n```\n### 启动web\n```\n# pygwalker/app\ncd app\nyarn install\nyarn dev\n```\n### 下载依赖\n```\n# pygwalker\npip install -e .\npip install jupyterlab jupyter_server_proxy\n```\n### 启动jupyterlab\n```\n# pygwalker\njupyter lab --ServerProxy.servers=\"{'pyg_dev_app': {'absolute_url': True, 'port': 8769, 'timeout': 30}}\"\n```\n### 在jupyterlab中启动pygwalker\n```\n# jupyterlab\nimport pandas as pd\nimport pygwalker as pyg\n \npyg.GlobalVarManager.set_component_url(\"/pyg_dev_app/\") # use dev frontend app\n \ndf = pd.read_csv(\"xxxx\")\n \nwalker = pyg.walk(df)\n```\n\n# License\n\n[Apache License 2.0](https://github.com/Kanaries/pygwalker/blob/main/LICENSE)\n\n# 更多阅读\n\n+ PyGWalker论文 [PyGWalker: On-the-fly Assistant for Exploratory Visual Data Analysis\n](https://arxiv.org/abs/2406.11637)\n+ 关于Graphic Walker，参考 [Graphic Walker GitHub](https://github.com/Kanaries/graphic-walker)\n+ 我们也在开发新一代增强分析型BI：[RATH](https://kanaries.net)。RATH是新一代智能化数据分析工具。借助AI，因果推断，智能可视化引擎协助你进行数据分析，体验前所未有的自动化。更多请访问：[RATH GitHub](https://github.com/Kanaries/Rath)\n" }, { "path": "environment.yml", "content": "name: pygwalker\nchannels:\n - conda-forge\n - defaults\ndependencies:\n - ipython\n - jinja2\n - pandas\n - python>=3.5\n - pip\n - polars\n - pip:\n - pygwalker>=0.1" }, { "path": "examples/README.md", "content": "# Examples\nThis folder contains example implementations of Pygwalker across different interfaces.\n\n- [`component_demo.ipynb`](component_demo.ipynb): Creating various visualizations using Pygwalker's core components\n- [`dash_demo.py`](dash_demo.py): Integration of Pygwalker with Dash framework\n- [`gradio_demo.py`](gradio_demo.py): Using Pygwalker within Gradio applications\n- [`html_demo.py`](html_demo.py): Generating standalone HTML outputs with Pygwalker\n- [`jupyter_demo.ipynb`](jupyter_demo.ipynb): Basic Pygwalker usage in Jupyter environments\n- [`marimo_demo.py`](marimo_demo.py): Interactive data exploration using Pygwalker in Marimo notebooks\n- [`streamlit_demo.py`](streamlit_demo.py): Embedding Pygwalker visualizations in Streamlit apps\n- [`reflex_demo.py`](reflex_demo.py): Example of using Pygwalker in a Reflex application\n- [`web_server_demo.py`](web_server_demo.py): Setting up Pygwalker with a web server\n\n## Running examples\nEach example includes its own set of requirements and setup instructions within the file. Make sure you have Pygwalker installed:\n```shell\npip install pygwalker\n```\n\nAdditional dependencies may be required based on the specific interface you're using (e.g., streamlit, dash, gradio).\nFor the Reflex demo, install the optional Reflex plugin:\n```shell\npip install \"pygwalker[reflex]\"\n```\n" }, { "path": "examples/component_demo.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"2142be9e-460c-45af-88aa-d356954b0caa\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pygwalker as pyg\\n\",\n \"import pandas as pd\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"7384b136-9098-482d-8914-1663547dc6ae\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"df = pd.read_csv(\\\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"45cbe3c8-6606-498b-a741-455425798cc9\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"pyg.component(df).bar().encode(x=\\\"season\\\", y=\\\"sum(casual)\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"cf5f40ff-a2ed-44c6-9f85-d633b75fa020\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component = pyg.component(df)\\n\",\n \"component_0 = component.encode(x=\\\"season\\\", y=\\\"sum(casual)\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"a98cf2a9-5286-4f4a-9279-d51a69d166bb\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.bar()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"d81d0c87-4c6f-42f8-8d22-8b32fc61384f\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.area()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"b5f1b05a-4b43-4f73-9558-1ce2edae81a3\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.line()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"9d8f9721-688f-4ade-aaaa-577be9bda56a\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.trail()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"901ecec4-cddc-4b74-afac-c82a7c52e98e\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component.scatter().encode(x=\\\"feeling_temp\\\", y=\\\"temperature\\\", color=\\\"humidity\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"98bb748f-0aaf-4b43-acbe-0524eafe10af\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component.circle().encode(x=\\\"feeling_temp\\\", y=\\\"temperature\\\", color=\\\"humidity\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"80c52a23-b3e3-4957-860f-21dd80b6ef47\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"(\\n\",\n \"component\\n\",\n \" .rect()\\n\",\n \" .encode(x='bin(\\\"feeling_temp\\\", 6)', y='bin(\\\"temperature\\\", 6)', color=\\\"MEAN(humidity)\\\")\\n\",\n \" .layout(height=400, width=460)\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"4cbf2edb-8bef-4190-986f-62b48517e7f4\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component.arc().encode(theta=\\\"SUM(registered)\\\", color=\\\"season\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f622848b-6122-4ee4-a7b1-d8521520b63f\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.bar().explorer()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"aafdd3b4-6819-480f-9c34-70c8d1aa410e\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"component_0.profiling()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"bff819f3-209d-4a78-b723-103b70dfb2a5\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.7\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "examples/dash_demo.py", "content": "import dash\nimport dash_dangerously_set_inner_html\nimport pandas as pd\nimport pygwalker as pyg\n\ndf = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n\napp = dash.Dash()\n\npyg_html_code = pyg.to_html(df, spec=\"./gw_config.json\")\n\napp.layout = dash.html.Div([\n dash_dangerously_set_inner_html.DangerouslySetInnerHTML(pyg_html_code),\n])\n\nif __name__ == '__main__':\n app.run_server(debug=True)\n" }, { "path": "examples/gradio_demo.py", "content": "import gradio as gr\nimport pandas as pd\n\nfrom pygwalker.api.gradio import PYGWALKER_ROUTE, get_html_on_gradio\n\nwith gr.Blocks() as demo:\n df = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n pyg_html = get_html_on_gradio(df, spec=\"./gw_config.json\", spec_io_mode=\"rw\")\n gr.HTML(pyg_html)\n\n\napp = demo.launch(app_kwargs={\n \"routes\": [PYGWALKER_ROUTE]\n})\n" }, { "path": "examples/gw_config.json", "content": "{\"config\": [{\"config\": {\"defaultAggregated\": true, \"geoms\": [\"bar\"], \"coordSystem\": \"generic\", \"limit\": -1}, \"encodings\": {\"dimensions\": [{\"dragId\": \"gw_d0SN\", \"fid\": \"date\", \"name\": \"date\", \"basename\": \"date\", \"semanticType\": \"temporal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_fCVU\", \"fid\": \"month\", \"name\": \"month\", \"basename\": \"month\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_xAWV\", \"fid\": \"season\", \"name\": \"season\", \"basename\": \"season\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_ho7q\", \"fid\": \"year\", \"name\": \"year\", \"basename\": \"year\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_1bIC\", \"fid\": \"holiday\", \"name\": \"holiday\", \"basename\": \"holiday\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_K8Ek\", \"fid\": \"work yes or not\", \"name\": \"work yes or not\", \"basename\": \"work yes or not\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_tORa\", \"fid\": \"am or pm\", \"name\": \"am or pm\", \"basename\": \"am or pm\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_RMSm\", \"fid\": \"Day of the week\", \"name\": \"Day of the week\", \"basename\": \"Day of the week\", \"semanticType\": \"quantitative\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_mea_key_fid\", \"fid\": \"gw_mea_key_fid\", \"name\": \"Measure names\", \"analyticType\": \"dimension\", \"semanticType\": \"nominal\"}, {\"fid\": \"gw_hYau\", \"dragId\": \"gw_hYau\", \"name\": \"Weekday [date]\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_hYau\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"weekday\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}]}}, {\"fid\": \"gw_lSdd\", \"dragId\": \"gw_lSdd\", \"name\": \"Quarter [date]\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_lSdd\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"quarter\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}]}}], \"measures\": [{\"dragId\": \"gw_oE-g\", \"fid\": \"hour\", \"name\": \"hour\", \"basename\": \"hour\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_LZNz\", \"fid\": \"temperature\", \"name\": \"temperature\", \"basename\": \"temperature\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_JbdF\", \"fid\": \"feeling_temp\", \"name\": \"feeling_temp\", \"basename\": \"feeling_temp\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_7hAr\", \"fid\": \"humidity\", \"name\": \"humidity\", \"basename\": \"humidity\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_a8mK\", \"fid\": \"winspeed\", \"name\": \"winspeed\", \"basename\": \"winspeed\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_Yb-_\", \"fid\": \"casual\", \"name\": \"casual\", \"basename\": \"casual\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_fdQ9\", \"fid\": \"registered\", \"name\": \"registered\", \"basename\": \"registered\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_Bdj1\", \"fid\": \"count\", \"name\": \"count\", \"basename\": \"count\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_count_fid\", \"fid\": \"gw_count_fid\", \"name\": \"Row count\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"one\", \"params\": [], \"as\": \"gw_count_fid\"}}, {\"dragId\": \"gw_mea_val_fid\", \"fid\": \"gw_mea_val_fid\", \"name\": \"Measure values\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}], \"rows\": [{\"dragId\": \"gw_XI5j\", \"fid\": \"registered\", \"name\": \"registered\", \"basename\": \"registered\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}], \"columns\": [{\"fid\": \"gw_hYau\", \"dragId\": \"gw_Jx83\", \"name\": \"Weekday [date]\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_hYau\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"weekday\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}]}}], \"color\": [{\"fid\": \"gw_lSdd\", \"dragId\": \"gw_BZBe\", \"name\": \"Quarter [date]\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_lSdd\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"quarter\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}]}}], \"opacity\": [], \"size\": [], \"shape\": [], \"radius\": [], \"theta\": [], \"longitude\": [], \"latitude\": [], \"geoId\": [], \"details\": [], \"filters\": [{\"dragId\": \"gw_U38p\", \"fid\": \"month\", \"name\": \"month\", \"basename\": \"month\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\", \"rule\": {\"type\": \"range\", \"value\": [1, 12]}}], \"text\": []}, \"layout\": {\"showActions\": false, \"showTableSummary\": false, \"stack\": \"stack\", \"interactiveScale\": false, \"zeroScale\": true, \"size\": {\"mode\": \"fixed\", \"width\": 350, \"height\": 345}, \"format\": {}, \"geoKey\": \"name\", \"resolve\": {\"x\": false, \"y\": false, \"color\": false, \"opacity\": false, \"shape\": false, \"size\": false}, \"colorPalette\": \"paired\", \"scale\": {\"opacity\": {}, \"size\": {}}, \"scaleIncludeUnmatchedChoropleth\": false, \"useSvg\": false}, \"visId\": \"gw_YvK3\", \"name\": \"Chart 1\"}, {\"config\": {\"defaultAggregated\": true, \"geoms\": [\"auto\"], \"coordSystem\": \"generic\", \"limit\": -1, \"folds\": [\"registered\", \"casual\"]}, \"encodings\": {\"dimensions\": [{\"dragId\": \"gw_iwKS\", \"fid\": \"date\", \"name\": \"date\", \"basename\": \"date\", \"semanticType\": \"temporal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_qttg\", \"fid\": \"month\", \"name\": \"month\", \"basename\": \"month\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_FJZI\", \"fid\": \"season\", \"name\": \"season\", \"basename\": \"season\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_noqw\", \"fid\": \"year\", \"name\": \"year\", \"basename\": \"year\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_S1Op\", \"fid\": \"holiday\", \"name\": \"holiday\", \"basename\": \"holiday\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_FECQ\", \"fid\": \"work yes or not\", \"name\": \"work yes or not\", \"basename\": \"work yes or not\", \"semanticType\": \"ordinal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_F4AV\", \"fid\": \"am or pm\", \"name\": \"am or pm\", \"basename\": \"am or pm\", \"semanticType\": \"nominal\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_Srun\", \"fid\": \"Day of the week\", \"name\": \"Day of the week\", \"basename\": \"Day of the week\", \"semanticType\": \"quantitative\", \"analyticType\": \"dimension\"}, {\"dragId\": \"gw_mea_key_fid\", \"fid\": \"gw_mea_key_fid\", \"name\": \"Measure names\", \"analyticType\": \"dimension\", \"semanticType\": \"nominal\"}], \"measures\": [{\"dragId\": \"gw_KeT-\", \"fid\": \"hour\", \"name\": \"hour\", \"basename\": \"hour\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_rDyp\", \"fid\": \"temperature\", \"name\": \"temperature\", \"basename\": \"temperature\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_G71D\", \"fid\": \"feeling_temp\", \"name\": \"feeling_temp\", \"basename\": \"feeling_temp\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_Gjrm\", \"fid\": \"humidity\", \"name\": \"humidity\", \"basename\": \"humidity\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_2SZj\", \"fid\": \"winspeed\", \"name\": \"winspeed\", \"basename\": \"winspeed\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_Pjzq\", \"fid\": \"casual\", \"name\": \"casual\", \"basename\": \"casual\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_dBk7\", \"fid\": \"registered\", \"name\": \"registered\", \"basename\": \"registered\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_Bju8\", \"fid\": \"count\", \"name\": \"count\", \"basename\": \"count\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}, {\"dragId\": \"gw_count_fid\", \"fid\": \"gw_count_fid\", \"name\": \"Row count\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\", \"computed\": true, \"expression\": {\"op\": \"one\", \"params\": [], \"as\": \"gw_count_fid\"}}, {\"dragId\": \"gw_mea_val_fid\", \"fid\": \"gw_mea_val_fid\", \"name\": \"Measure values\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}], \"rows\": [{\"dragId\": \"gw_PW8u\", \"fid\": \"gw_mea_val_fid\", \"name\": \"Measure values\", \"analyticType\": \"measure\", \"semanticType\": \"quantitative\", \"aggName\": \"sum\"}], \"columns\": [{\"dragId\": \"gw_8tGb\", \"fid\": \"date\", \"name\": \"date\", \"basename\": \"date\", \"semanticType\": \"temporal\", \"analyticType\": \"dimension\"}], \"color\": [{\"dragId\": \"gw_NK9S\", \"fid\": \"gw_mea_key_fid\", \"name\": \"Measure names\", \"analyticType\": \"dimension\", \"semanticType\": \"nominal\"}], \"opacity\": [], \"size\": [], \"shape\": [], \"radius\": [], \"theta\": [], \"longitude\": [], \"latitude\": [], \"geoId\": [], \"details\": [], \"filters\": [{\"dragId\": \"gw_RBu-\", \"fid\": \"date\", \"name\": \"date\", \"basename\": \"date\", \"semanticType\": \"temporal\", \"analyticType\": \"dimension\", \"rule\": {\"type\": \"temporal range\", \"value\": [1293811200000, 1356883200000], \"format\": \"%Y-%m-%d\", \"offset\": -480}}], \"text\": []}, \"layout\": {\"showActions\": false, \"showTableSummary\": false, \"stack\": \"stack\", \"interactiveScale\": false, \"zeroScale\": true, \"size\": {\"mode\": \"fixed\", \"width\": 752, \"height\": 360}, \"format\": {}, \"geoKey\": \"name\", \"resolve\": {\"x\": false, \"y\": false, \"color\": false, \"opacity\": false, \"shape\": false, \"size\": false}, \"scaleIncludeUnmatchedChoropleth\": false}, \"visId\": \"gw_QwuS\", \"name\": \"Chart 2\"}], \"chart_map\": {}, \"version\": \"0.4.5a3\", \"workflow_list\": [{\"workflow\": [{\"type\": \"filter\", \"filters\": [{\"fid\": \"month\", \"rule\": {\"type\": \"range\", \"value\": [1, 12]}}]}, {\"type\": \"transform\", \"transform\": [{\"key\": \"gw_hYau\", \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_hYau\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"weekday\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}, {\"type\": \"displayOffset\", \"value\": -480}]}}, {\"key\": \"gw_lSdd\", \"expression\": {\"op\": \"dateTimeFeature\", \"as\": \"gw_lSdd\", \"params\": [{\"type\": \"field\", \"value\": \"date\"}, {\"type\": \"value\", \"value\": \"quarter\"}, {\"type\": \"format\", \"value\": \"%Y-%m-%d\"}, {\"type\": \"displayOffset\", \"value\": -480}]}}]}, {\"type\": \"view\", \"query\": [{\"op\": \"aggregate\", \"groupBy\": [\"gw_hYau\", \"gw_lSdd\"], \"measures\": [{\"field\": \"registered\", \"agg\": \"sum\", \"asFieldKey\": \"registered_sum\"}]}]}]}, {\"workflow\": [{\"type\": \"filter\", \"filters\": [{\"fid\": \"date\", \"rule\": {\"type\": \"temporal range\", \"value\": [1293811200000, 1356883200000], \"offset\": -480, \"format\": \"%Y-%m-%d\"}}]}, {\"type\": \"view\", \"query\": [{\"op\": \"aggregate\", \"groupBy\": [\"date\"], \"measures\": [{\"field\": \"registered\", \"agg\": \"sum\", \"asFieldKey\": \"registered_sum\"}, {\"field\": \"casual\", \"agg\": \"sum\", \"asFieldKey\": \"casual_sum\"}]}]}]}]}" }, { "path": "examples/html_demo.py", "content": "import pygwalker as pyg\nimport pandas as pd\n\n\ndf = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n\nwith open(\"pyg_demo.html\", \"w\", encoding=\"utf-8\") as f:\n html = pyg.to_html(df)\n f.write(html)\n" }, { "path": "examples/jupyter_demo.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"0934673e-22de-45e6-9f70-438e8c4c49d3\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pygwalker as pyg\\n\",\n \"import pandas as pd\\n\",\n \"\\n\",\n \"df = pd.read_csv(\\\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\\\")\\n\",\n \"\\n\",\n \"walker = pyg.walk(df, spec=\\\"./gw_config.json\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"9e43f350\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"pyg.table(df)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"4557ad08\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"pyg.render(df, spec=\\\"./gw_config.json\\\")\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.11.7\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "examples/marimo_demo.py", "content": "# /// script\n# requires-python = \">=3.12\"\n# dependencies = [\n# \"marimo\",\n# \"pandas==2.2.3\",\n# \"pygwalker==0.4.9.13\",\n# ]\n# ///\n\nimport marimo\n\n__generated_with = \"0.9.15\"\napp = marimo.App(width=\"medium\")\n\n\n@app.cell\ndef __(pd, walk):\n _df = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n walk(_df)\n return\n\n\n@app.cell\ndef __(pd, pyg):\n _df = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n pyg.walk(_df)\n return\n\n\n@app.cell\ndef __():\n # import libraries\n import marimo as mo\n import pandas as pd\n from pygwalker.api.marimo import walk # Usage - directly use walk(\"\")\n import pygwalker.api.marimo as pyg # Usage - directly use pyg.walk(\"\")\n return mo, pd, pyg, walk\n\n\nif __name__ == \"__main__\":\n app.run()\n" }, { "path": "examples/reflex_demo/.gitignore", "content": ".web\n.states\nassets/external/\n*.db\n*.py[cod]\n# Reflex generated files and directories\n.web/\n__pycache__/\nrequirements.txt\n*.pyc\n*.pyo\n*.pyd\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# Environment files\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Database files\nreflex.db\n" }, { "path": "examples/reflex_demo/README.md", "content": "# PyGWalker Reflex Demo\n\nThis demo shows how to integrate PyGWalker with Reflex, a Python web framework for building interactive web applications.\n\n## Prerequisites\n\nMake sure you have PyGWalker installed with the Reflex plugin:\n\n```bash\npip install \"pygwalker[reflex]\"\n```\n\n## Running the Demo\n\n1. Navigate to this directory:\n ```bash\n cd examples/reflex_demo\n ```\n\n2. Initialize Reflex (first time only):\n ```bash\n reflex init\n ```\n\n3. Run the application:\n ```bash\n reflex run\n ```\n\n4. Open your browser and navigate to `http://localhost:3002`\n\n## What This Demo Shows\n\n- **Full PyGWalker + Reflex Integration**: Complete integration with API transformer\n- Proper Reflex package structure (`app/app.py`)\n- Interactive data visualization components within Reflex\n- Automatic fallback when PyGWalker is not available\n- Proper error handling and graceful degradation\n\n## Files in This Demo\n\n- `app/app.py` - Main application file with PyGWalker integration\n- `app/__init__.py` - Package initialization \n- `rxconfig.py` - Reflex configuration file\n- `README.md` - This documentation file\n\n## Package Structure\n\nThe demo follows the standard Reflex package structure:\n\n```\nexamples/reflex_demo/\n├── app/\n│ ├── __init__.py\n│ └── app.py # Main app with PyGWalker integration\n├── rxconfig.py # Reflex configuration\n├── README.md\n└── .gitignore\n```\n\nThis structure allows Reflex to properly import the app as `app.app` where:\n- `app_name=\"app\"` in `rxconfig.py`\n- `app/` directory contains the Python package\n- `app/app.py` contains the `app = rx.App()` object\n\n## Demo Features\n\n- **Full Integration**: Uses PyGWalker's `register_pygwalker_api` transformer for complete functionality\n- **Local Data**: Uses generated sample data (weather data) to avoid network dependencies \n- **Error Handling**: Gracefully handles import and runtime errors with fallback mode\n- **Interactive UI**: Provides full PyGWalker data visualization capabilities\n\n## Fixed Issues\n\nThis demo includes fixes for:\n- ✅ **ASGI Mounting Errors**: Fixed PyGWalker's `register_pygwalker_api` function to properly mount FastAPI sub-applications\n- ✅ **Package Structure**: Proper Reflex package structure following `app_name/app_name.py` convention\n- ✅ **Route Registration**: Improved API route registration using FastAPI mounting instead of direct route appending\n\nThe demo now provides complete PyGWalker integration within a Reflex web application with full API communication support. " }, { "path": "examples/reflex_demo/__init__.py", "content": "\"\"\"Pygwalker examples package.\"\"\" \n\n# Initialize examples package " }, { "path": "examples/reflex_demo/app/__init__.py", "content": "\"\"\"PyGWalker Reflex Demo App Package.\"\"\" " }, { "path": "examples/reflex_demo/app/app.py", "content": "\"\"\"\nPyGWalker + Reflex integration demo.\n\nThis demo shows how to integrate PyGWalker with Reflex, a Python web framework.\n\nTo run this demo:\n1. Make sure you have installed PyGWalker with the Reflex plugin:\n pip install \"pygwalker[reflex]\"\n\n2. Navigate to this directory:\n cd examples/reflex_demo\n\n3. Run the app:\n reflex run\n\"\"\"\n\nimport os\nimport sys\nimport traceback\nimport pandas as pd\nimport reflex as rx\n\n# Add the project root directory to sys.path to ensure pygwalker can be imported\nproject_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..'))\nif project_root not in sys.path:\n sys.path.insert(0, project_root)\n\n# Try to import PyGWalker Reflex components, fall back to basic demo if not available\ntry:\n from pygwalker.api.reflex import get_component\n from pygwalker.communications.reflex_comm import register_pygwalker_api\n PYGWALKER_AVAILABLE = True\n print(\"✅ PyGWalker Reflex integration is available!\")\nexcept Exception as e:\n print(f\"⚠️ PyGWalker Reflex integration not available: {e}\")\n print(\"Running in fallback mode with basic Reflex demo.\")\n PYGWALKER_AVAILABLE = False\n get_component = None\n register_pygwalker_api = None\n\n\nclass State(rx.State):\n \"\"\"The app state.\"\"\"\n pass\n\n\ndef index() -> rx.Component:\n \"\"\"PyGWalker + Reflex demo page.\"\"\"\n if PYGWALKER_AVAILABLE:\n return pygwalker_demo()\n else:\n return fallback_demo()\n\n\ndef pygwalker_demo() -> rx.Component:\n \"\"\"PyGWalker integration demo.\"\"\"\n try:\n # Use local data to avoid network issues\n df = pd.DataFrame({\n 'Date': pd.date_range('2023-01-01', periods=100),\n 'Temperature': [20 + 10 * (i % 10) / 10 for i in range(100)],\n 'Humidity': [50 + 30 * (i % 7) / 7 for i in range(100)],\n 'City': ['New York', 'London', 'Tokyo', 'Paris', 'Sydney'] * 20\n })\n \n # Create a PyGWalker component\n pyg_component = get_component(\n df,\n theme_key=\"g2\",\n appearance=\"media\",\n default_tab=\"vis\"\n )\n \n return rx.vstack(\n rx.heading(\"Use Pygwalker In Reflex\", size=\"3\"),\n rx.text(\"This demo shows PyGWalker integrated with Reflex framework.\"),\n rx.text(\"✅ PyGWalker components with full API integration!\", color=\"green\"),\n pyg_component,\n spacing=\"4\",\n width=\"100%\",\n align_items=\"stretch\",\n )\n except Exception as e:\n print(f\"Error in pygwalker_demo: {e}\")\n print(traceback.format_exc())\n return fallback_demo_with_error(str(e))\n\n\ndef fallback_demo() -> rx.Component:\n \"\"\"Fallback demo when PyGWalker is not available.\"\"\"\n # Sample data\n df = pd.DataFrame({\n 'Date': pd.date_range('2023-01-01', periods=10),\n 'Temperature': [20, 22, 25, 28, 30, 27, 24, 21, 19, 23],\n 'Humidity': [45, 50, 55, 60, 65, 58, 52, 48, 44, 49],\n 'City': ['New York'] * 10\n })\n \n return rx.vstack(\n rx.heading(\"PyGWalker + Reflex Demo\", size=\"3\"),\n rx.text(\"PyGWalker Reflex integration is not available.\", color=\"orange\"),\n rx.text(\"Install it with: pip install 'pygwalker[reflex]'\", font_family=\"monospace\"),\n \n rx.heading(\"Sample Data\", size=\"2\", margin_top=\"20px\"),\n rx.text(f\"Data shape: {df.shape}\"),\n \n rx.box(\n rx.text(\"Temperature Data:\", font_weight=\"bold\"),\n rx.text(f\"Min: {df['Temperature'].min()}°C, Max: {df['Temperature'].max()}°C\"),\n rx.text(\"Humidity Data:\", font_weight=\"bold\"),\n rx.text(f\"Min: {df['Humidity'].min()}%, Max: {df['Humidity'].max()}%\"),\n padding=\"15px\",\n border=\"1px solid #ccc\",\n border_radius=\"8px\",\n margin_top=\"10px\",\n ),\n \n rx.text(\n \"This demo shows the basic Reflex setup working. \"\n \"When PyGWalker Reflex integration is available, \"\n \"this will display an interactive data visualization component.\",\n margin_top=\"20px\",\n font_style=\"italic\"\n ),\n \n spacing=\"4\",\n width=\"100%\",\n align_items=\"stretch\",\n padding=\"20px\",\n )\n\n\ndef fallback_demo_with_error(error_msg: str) -> rx.Component:\n \"\"\"Fallback demo when PyGWalker components fail.\"\"\"\n return rx.vstack(\n rx.heading(\"PyGWalker + Reflex Demo\", size=\"3\"),\n rx.text(\"PyGWalker components encountered an error:\", color=\"red\"),\n rx.code(error_msg, padding=\"10px\", background_color=\"gray.100\"),\n rx.text(\"Falling back to basic demo.\", color=\"orange\"),\n \n rx.text(\n \"This suggests there might be a compatibility issue between \"\n \"PyGWalker and the current version of Reflex. The basic Reflex \"\n \"framework is working correctly.\",\n margin_top=\"20px\",\n font_style=\"italic\"\n ),\n \n spacing=\"4\",\n width=\"100%\",\n align_items=\"stretch\",\n padding=\"20px\",\n )\n\n\n# Create a Reflex app with the fixed PyGWalker API transformer\ntry:\n if PYGWALKER_AVAILABLE:\n try:\n app = rx.App(api_transformer=register_pygwalker_api)\n print(\"✅ Reflex app created with PyGWalker API transformer!\")\n except Exception as e:\n print(f\"⚠️ PyGWalker API transformer failed: {e}\")\n print(\"Creating basic Reflex app without transformer...\")\n app = rx.App()\n print(\"✅ Reflex app created without PyGWalker transformer!\")\n else:\n app = rx.App()\n print(\"✅ Basic Reflex app created!\")\n \n app.add_page(index)\n print(\"✅ Pages added successfully!\")\n \nexcept Exception as e:\n print(f\"❌ Error creating Reflex app: {e}\")\n print(traceback.format_exc())\n raise " }, { "path": "examples/reflex_demo/rxconfig.py", "content": "import reflex as rx\n\nconfig = rx.Config(\n app_name=\"app\",\n backend_port=8000,\n loglevel=\"debug\",\n) " }, { "path": "examples/streamlit_demo.py", "content": "from pygwalker.api.streamlit import StreamlitRenderer\nimport pandas as pd\nimport streamlit as st\n\n# Adjust the width of the Streamlit page\nst.set_page_config(\n page_title=\"Use Pygwalker In Streamlit\",\n layout=\"wide\"\n)\n\n# Add Title\nst.title(\"Use Pygwalker In Streamlit\")\n\n# You should cache your pygwalker renderer, if you don't want your memory to explode\n@st.cache_resource\ndef get_pyg_renderer() -> \"StreamlitRenderer\":\n df = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n # If you want to use feature of saving chart config, set `spec_io_mode=\"rw\"`\n return StreamlitRenderer(df, spec=\"./gw_config.json\")\n\n\nrenderer = get_pyg_renderer()\n\nst.subheader(\"Display Explore UI\")\n\ntab1, tab2, tab3, tab4, tab5 = st.tabs(\n [\"graphic walker\", \"data profiling\", \"graphic renderer\", \"pure chart\", \"table\"]\n)\n\nwith tab1:\n renderer.explorer()\n\nwith tab2:\n renderer.explorer(default_tab=\"data\", key=\"explorer0\")\n\nwith tab3:\n renderer.viewer()\n\nwith tab4:\n st.markdown(\"### registered per weekday\")\n renderer.chart(0)\n st.markdown(\"### registered per day\")\n renderer.chart(1)\n\nwith tab5:\n renderer.table()\n" }, { "path": "examples/web_server_demo.py", "content": "\"\"\"\npygwalker>=0.4.8.6\n\nThis is poc for PygWalker integration with FastAPI.\n\nIf you want to use Graphic-Walker in your web application, the best practice is to use Graphic-Walker as a separate front-end component for development, rather than using pygwalker.\n\nrun it: uvicorn web_server_demo:app --reload --port 8000\nview it: http://127.0.0.1:8000/pyg_html/test0\n\"\"\"\nfrom typing import Dict, Any\nimport json\n\nfrom fastapi import FastAPI, Body\nfrom fastapi.responses import HTMLResponse, JSONResponse\nfrom pygwalker.api.pygwalker import PygWalker\nfrom pygwalker.communications.base import BaseCommunication\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker import GlobalVarManager\nimport pandas as pd\n\napp = FastAPI()\n\n\ndef init_pygwalker_entity_map() -> Dict[str, PygWalker]:\n \"\"\"Register PygWalker entity\"\"\"\n GlobalVarManager.set_privacy(\"offline\")\n df = pd.read_csv(\"https://kanaries-app.s3.ap-northeast-1.amazonaws.com/public-datasets/bike_sharing_dc.csv\")\n walker = PygWalker(\n gid=\"test0\",\n dataset=df,\n field_specs=None,\n spec=\"\",\n source_invoke_code=\"\",\n theme_key=\"vega\",\n appearance=\"light\",\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=True,\n use_save_tool=False,\n is_export_dataframe=False,\n kanaries_api_key=\"\",\n default_tab=\"vis\",\n cloud_computation=False,\n gw_mode=\"explore\",\n )\n comm = BaseCommunication(walker.gid)\n walker._init_callback(comm)\n return {\n walker.gid: walker\n }\n\n\npygwalker_entity_map = init_pygwalker_entity_map()\n\n\n# api path and html path need to have the same prefix\n@app.post(\"/pyg_html/_pygwalker/comm/{gid}\")\ndef pygwalker_comm(gid: str, payload: Dict[str, Any] = Body(...)):\n if gid not in pygwalker_entity_map:\n return {\"success\": False, \"message\": f\"Unknown gid: {gid}\"}\n\n comm_obj = pygwalker_entity_map[gid].comm\n result = comm_obj._receive_msg(payload[\"action\"], payload[\"data\"])\n return JSONResponse(content=json.loads(json.dumps(result, cls=DataFrameEncoder)))\n\n\n# api path and html path need to have the same prefix\n@app.get(\"/pyg_html/{gid}\")\ndef pyg_html(gid: str):\n walker = pygwalker_entity_map[gid]\n props = walker._get_props(\"web_server\")\n props[\"communicationUrl\"] = \"_pygwalker/comm\"\n html = walker._get_render_iframe(props, True)\n return HTMLResponse(content=html)\n" }, { "path": "pygwalker/__init__.py", "content": "from pygwalker.utils.log import init_logging as __init_logging\n\n__init_logging()\n\n# pylint: disable=wrong-import-position\nimport logging\n\nfrom pygwalker.utils.randoms import rand_str as __rand_str\nfrom pygwalker.utils.execute_env_check import check_kaggle as __check_kaggle\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom pygwalker.services.kaggle import show_tips_user_kaggle as __show_tips_user_kaggle\n\n__version__ = \"0.5.0.0\"\n__hash__ = __rand_str()\n\nfrom pygwalker.api.adapter import walk, render, table\nfrom pygwalker.api.html import to_html\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.api.component import component\n\nif GlobalVarManager.privacy == 'offline':\n logging.getLogger(__name__).info(\"Running in offline mode. There might be newer releases available. Please check at https://github.com/Kanaries/pygwalker or https://pypi.org/project/pygwalker.\")\n\nif __check_kaggle():\n __show_tips_user_kaggle()\n\n__all__ = [\"walk\", \"render\", \"table\", \"to_html\", \"FieldSpec\", \"GlobalVarManager\", \"component\"]\n" }, { "path": "pygwalker/_constants.py", "content": "import os\n\nJUPYTER_BYTE_LIMIT = 1 << 24\nJUPYTER_WIDGETS_BYTE_LIMIT = 1 << 20\n\nROOT_DIR = os.path.dirname(os.path.abspath(__file__))\n" }, { "path": "pygwalker/_typing.py", "content": "from typing import TypeVar, TYPE_CHECKING\n\nfrom typing_extensions import Literal\n\ndataframe_types = []\nif TYPE_CHECKING:\n try:\n import pandas as pd\n dataframe_types.append(pd.DataFrame)\n except ModuleNotFoundError:\n pass\n try:\n from modin import pandas as mpd\n dataframe_types.append(mpd.DataFrame)\n except ModuleNotFoundError:\n pass\n try:\n import polars as pl\n dataframe_types.append(pl.DataFrame)\n except ModuleNotFoundError:\n pass\n try:\n from pyspark.sql import DataFrame as SparkDataFrame\n dataframe_types.append(SparkDataFrame)\n except ModuleNotFoundError:\n pass\n\n\nDataFrame = TypeVar(\"DataFrame\", *dataframe_types)\nSeries = TypeVar(\"Series\")\n\nIThemeKey = Literal['vega', 'g2', 'streamlit']\nIAppearance = Literal['media', 'light', 'dark']\nISpecIOMode = Literal[\"r\", \"rw\"]\n" }, { "path": "pygwalker/api/__init__.py", "content": "" }, { "path": "pygwalker/api/adapter.py", "content": "\nfrom typing import Union, List, Optional\n\nfrom typing_extensions import Literal\n\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.utils.runtime_env import get_current_env\nfrom pygwalker.api import jupyter\nfrom pygwalker.api import webserver\n\n\ndef walk(\n dataset: Union[DataFrame, Connector, str],\n gid: Union[int, str] = None,\n *,\n env: Literal['Jupyter', 'JupyterWidget'] = 'JupyterWidget',\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n use_kernel_calc: Optional[bool] = None,\n kernel_computation: Optional[bool] = None,\n cloud_computation: bool = False,\n show_cloud_tool: bool = True,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n):\n \"\"\"Walk through pandas.DataFrame df with Graphic Walker\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - env: (Literal['Jupyter' | 'JupyterWidget'], optional): The enviroment using pygwalker. Default as 'JupyterWidget'\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None, automatically determine whether to use kernel calculation.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n - cloud_computation(bool): Whether to use cloud compute for datas, it upload your data to kanaries cloud. Default to False.\n - port(int): only works in web server mode. port to use for the server. Default to None, which means a random port will be used.\n \"\"\"\n cur_env = get_current_env()\n if cur_env == \"jupyter\":\n return jupyter.walk(\n dataset,\n gid,\n env=env,\n field_specs=field_specs,\n theme_key=theme_key,\n appearance=appearance,\n spec=spec,\n use_kernel_calc=use_kernel_calc,\n kernel_computation=kernel_computation,\n cloud_computation=cloud_computation,\n show_cloud_tool=show_cloud_tool,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n **kwargs\n )\n\n return webserver.walk(\n dataset,\n gid,\n field_specs=field_specs,\n theme_key=theme_key,\n appearance=appearance,\n spec=spec,\n kernel_computation=kernel_computation,\n cloud_computation=cloud_computation,\n show_cloud_tool=show_cloud_tool,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n auto_open=True,\n auto_shutdown=True,\n **kwargs\n )\n\n\ndef render(\n dataset: Union[DataFrame, Connector, str],\n spec: str,\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n **kwargs\n):\n \"\"\"\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - spec (str): chart config data. config id, json, remote file url\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - port(int): only works in web server mode. port to use for the server. Default to None, which means a random port will be used.\n \"\"\"\n cur_env = get_current_env()\n if cur_env == \"jupyter\":\n return jupyter.render(\n dataset,\n spec,\n theme_key=theme_key,\n appearance=appearance,\n kernel_computation=kernel_computation,\n kanaries_api_key=kanaries_api_key,\n **kwargs\n )\n\n return webserver.render(\n dataset,\n spec,\n theme_key=theme_key,\n appearance=appearance,\n kernel_computation=kernel_computation,\n kanaries_api_key=kanaries_api_key,\n auto_open=True,\n auto_shutdown=True,\n **kwargs\n )\n\n\ndef table(\n dataset: Union[DataFrame, Connector, str],\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n **kwargs\n):\n \"\"\"\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - port(int): only works in web server mode. port to use for the server. Default to None, which means a random port will be used.\n \"\"\"\n cur_env = get_current_env()\n if cur_env == \"jupyter\":\n return jupyter.table(\n dataset,\n theme_key=theme_key,\n appearance=appearance,\n kernel_computation=kernel_computation,\n kanaries_api_key=kanaries_api_key,\n **kwargs\n )\n \n return webserver.table(\n dataset,\n theme_key=theme_key,\n appearance=appearance,\n kernel_computation=kernel_computation,\n kanaries_api_key=kanaries_api_key,\n auto_open=True,\n auto_shutdown=True,\n **kwargs\n )\n" }, { "path": "pygwalker/api/anywidget.py", "content": "from typing import Union, List, Optional\nimport inspect\nimport json\nimport pathlib\n\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.services.format_invoke_walk_code import get_formated_spec_params_code_from_frame\nfrom pygwalker.communications.anywidget_comm import AnywidgetCommunication\nimport anywidget\nimport traitlets\n\n\nclass _WalkerWidget(anywidget.AnyWidget):\n \"\"\"WalkerWidget\"\"\"\n _esm = (pathlib.Path(__file__).parent.parent / \"templates\" / \"dist\" / \"pygwalker-app.es.js\").read_text(encoding=\"utf-8\")\n props = traitlets.Unicode(\"\").tag(sync=True)\n\n\ndef walk(\n dataset: Union[DataFrame, Connector, str],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n show_cloud_tool: bool = False,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n):\n \"\"\"Walk through pandas.DataFrame df with Graphic Walker\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n if field_specs is None:\n field_specs = []\n\n source_invoke_code = get_formated_spec_params_code_from_frame(\n inspect.stack()[1].frame\n )\n\n widget = _WalkerWidget()\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=source_invoke_code,\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=False,\n kernel_computation=True,\n use_save_tool=True,\n gw_mode=\"explore\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=False,\n **kwargs\n )\n comm = AnywidgetCommunication(walker.gid)\n\n widget.props = json.dumps(walker._get_props(\"anywidget\", []))\n comm.register_widget(widget)\n walker._init_callback(comm)\n\n return widget\n" }, { "path": "pygwalker/api/component.py", "content": "from typing import List, Optional, Dict, Any, Union\nfrom typing_extensions import Literal\nfrom copy import deepcopy\n\nfrom pydantic import BaseModel, Field\nimport sqlglot\nimport sqlglot.expressions as exp\n\nfrom .pygwalker import PygWalker\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey, ISpecIOMode\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker.utils.randoms import rand_str\n\n\nGRAPHIC_WALKER_AGG_FUNCS = {\n \"sum\", \"mean\", \"median\",\n \"min\", \"max\", \"variance\", \"stddev\",\n}\n\nGRAPHIC_WALKER_FIELD_FUNCS = {\n \"bin\", \"bin_count\"\n}\n\n\ndef _convert_sql_to_field(sql: str, is_agg_sql: bool) -> Dict[str, Any]:\n \"\"\"Convert sql to field info.\"\"\"\n fid = \"gw_\" + rand_str(6)\n field_item = {\n \"fid\": fid,\n \"name\": sql,\n \"analyticType\": \"dimension\",\n \"semanticType\": \"nominal\",\n \"computed\": True,\n \"expression\": {\n \"as\": fid,\n \"op\": \"expr\",\n \"params\": [\n {\"type\": \"sql\", \"value\": sql}\n ]\n }\n }\n if is_agg_sql:\n field_item[\"aggName\"] = \"expr\"\n field_item[\"analyticType\"] = \"measure\"\n field_item[\"semanticType\"] = \"quantitative\"\n return field_item\n\n\ndef _convert_gw_agg_function_to_field(func_name: str, field_name: str) -> Dict[str, Any]:\n \"\"\"Convert graphic walker agg function to field info.\"\"\"\n field_name = field_name.strip(\"\\\"'\")\n field_item = {\n \"fid\": field_name,\n \"name\": field_name,\n \"analyticType\": \"measure\",\n \"semanticType\": \"quantitative\",\n \"aggName\": func_name,\n }\n return field_item\n\n\ndef _convert_gw_bin_function_to_field(func_name: str, field_name: str, num: int) -> Dict[str, Any]:\n \"\"\"Convert graphic walker bin function to field info.\"\"\"\n fid = \"gw_\" + rand_str(6)\n field_name = field_name.strip(\"\\\"'\")\n field_item = {\n \"fid\": fid,\n \"name\": f\"{func_name}{num}({field_name})\",\n \"analyticType\": \"dimension\",\n \"semanticType\": \"ordinal\",\n \"computed\": True,\n \"expression\": {\n \"op\": func_name,\n \"as\": fid,\n \"params\": [\n {\"type\": \"field\", \"value\": field_name},\n ],\n \"num\": num,\n }\n }\n return field_item\n\n\ndef _handle_anonymous(ast: exp.Anonymous, origin_str: str) -> Dict[str, Any]:\n \"\"\"Handle anonymous expression.\"\"\"\n func_name = str(ast.this).lower()\n if func_name in GRAPHIC_WALKER_AGG_FUNCS:\n return _convert_gw_agg_function_to_field(func_name, str(ast.expressions[0]))\n if func_name in GRAPHIC_WALKER_FIELD_FUNCS:\n return _convert_gw_bin_function_to_field(func_name, str(ast.expressions[0]), int(str(ast.expressions[1])))\n return _convert_sql_to_field(origin_str, False)\n\n\ndef _handle_agg_func(ast: exp.AggFunc, origin_str: str) -> Dict[str, Any]:\n \"\"\"Handle agg function expression.\"\"\"\n func_name = ast.sql_name().lower()\n if func_name in GRAPHIC_WALKER_AGG_FUNCS:\n return _convert_gw_agg_function_to_field(func_name, str(ast.this))\n return _convert_sql_to_field(origin_str, True)\n\n\nclass Expression(BaseModel):\n op: str\n as_: str = Field(alias=\"as\")\n params: List[Dict[str, Any]]\n\n\nclass FieldInfo(BaseModel):\n fid: str\n name: str\n semantic_type: str = Field(alias=\"semanticType\")\n analytic_type: str = Field(alias=\"analyticType\")\n computed: bool\n agg_name: str = Field(alias=\"aggName\")\n expression: Expression\n\n\n# pylint: disable=protected-access\nclass Component:\n \"\"\"\n Component class for creating a chain of components.\n\n Kargs:\n - walker (PygWalker): PygWalker instance.\n - render_type (str): render type.\n - field_map (Dict[str, Any]): field map.\n - single_chart_spec (Dict[str, Any]): single chart\n \"\"\"\n def __init__(\n self,\n *,\n walker: PygWalker,\n render_type: str,\n field_map: Dict[str, Any],\n single_chart_spec: Dict[str, Any],\n ):\n self.walker = walker\n self._render_type = render_type\n self._field_map = field_map\n self._single_chart_spec = single_chart_spec\n self._runtime_time = None\n\n def copy(self) -> \"Component\":\n \"\"\"return new copied component.\"\"\"\n return self.__class__(\n walker=self.walker,\n render_type=self._render_type,\n field_map=deepcopy(self._field_map),\n single_chart_spec=deepcopy(self._single_chart_spec)\n )\n\n def _update_single_chart_spec(self, key: str, value: Any) -> Dict[str, Any]:\n \"\"\"update single chart spec.\"\"\"\n cur_obj = self._single_chart_spec\n keys = key.split(\"__\")\n for k in keys[:-1]:\n cur_obj = cur_obj[k]\n cur_obj[keys[-1]] = value\n\n def _convert_string_to_field_info(self, s: str) -> Dict[str, Any]:\n \"\"\"\n example: sum(field_name), field_name\n \"\"\"\n ast = sqlglot.parse_one(s, dialect=\"duckdb\")\n if isinstance(ast, exp.Anonymous):\n return _handle_anonymous(ast, s)\n elif isinstance(ast, exp.AggFunc):\n return _handle_agg_func(ast, s)\n elif isinstance(ast, exp.Func):\n return _convert_sql_to_field(s, False)\n elif isinstance(ast, exp.Column):\n return {\n \"fid\": s,\n \"name\": s,\n \"analyticType\": \"dimension\",\n \"semanticType\": \"nominal\",\n \"computed\": False,\n **self._field_map.get(s, {})\n }\n return {}\n\n def _repr_html_(self) -> str:\n return self.to_html()\n\n def to_html(self) -> str:\n if self._render_type == \"pure_chart\":\n return self._get_single_chart_html()\n if self._render_type == \"explorer\":\n return self._get_explorer_html()\n if self._render_type == \"profiling\":\n return self._get_profiling_html()\n return \"\"\n\n def _get_single_chart_html(self) -> str:\n return self.walker.get_single_chart_html_by_spec(spec=self._single_chart_spec)\n\n def _get_explorer_html(self) -> str:\n all_datas = None\n if self.walker.kernel_computation:\n all_datas = self.walker.data_parser.to_records()\n pyg_props = self.walker._get_props(data_source=all_datas)\n pyg_props[\"visSpec\"] = [self._single_chart_spec]\n\n return self.walker._get_render_iframe(pyg_props)\n\n def _get_profiling_html(self) -> str:\n all_datas = None\n if self.walker.kernel_computation:\n all_datas = self.walker.data_parser.to_records()\n pyg_props = self.walker._get_props(data_source=all_datas)\n pyg_props[\"gwMode\"] = \"table\"\n return self.walker._get_render_iframe(pyg_props)\n\n def bar(self) -> \"Component\":\n \"\"\"Bar chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"bar\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def line(self) -> \"Component\":\n \"\"\"Line chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"line\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def area(self) -> \"Component\":\n \"\"\"Area chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"area\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def trail(self) -> \"Component\":\n \"\"\"Trail chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"trail\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def scatter(self) -> \"Component\":\n \"\"\"Scatter chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"point\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def circle(self) -> \"Component\":\n \"\"\"Circle chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"circle\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def tick(self) -> \"Component\":\n \"\"\"Tick chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"tick\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def rect(self) -> \"Component\":\n \"\"\"Rect chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"rect\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def arc(self) -> \"Component\":\n \"\"\"Arc chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"arc\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def text(self) -> \"Component\":\n \"\"\"Text chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"text\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def box(self) -> \"Component\":\n \"\"\"Box chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"boxplot\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def table(self) -> \"Component\":\n \"\"\"Table chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"table\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"generic\")\n return copied_obj\n\n def poi(self) -> \"Component\":\n \"\"\"Poi chart.\"\"\"\n copied_obj = self.copy()\n copied_obj._update_single_chart_spec(\"config__geoms\", [\"poi\"])\n copied_obj._update_single_chart_spec(\"config__coordSystem\", \"geographic\")\n return copied_obj\n\n # pylint: disable=unused-argument\n def encode(\n self,\n x: Union[str, List[str]] = \"\",\n y: Union[str, List[str]] = \"\",\n color: str = \"\",\n opacity: str = \"\",\n size: str = \"\",\n shape: str = \"\",\n radius: str = \"\",\n theta: str = \"\",\n longitude: str = \"\",\n latitude: str = \"\",\n geoid: str = \"\",\n details: str = \"\",\n text: str = \"\",\n ) -> \"Component\":\n \"\"\"\n Encode fields.\n example: .encode(x=\"field_0\", y=\"field_1\", color=\"field_2\")\n .encode(x=\"field_0\", y=\"SUM(field_1)\")\n \"\"\"\n all_params = {\n key: [value] if isinstance(value, str) else value\n for key, value in locals().items()\n if key != \"self\"\n }\n copied_obj = self.copy()\n params_key_map = {\n \"x\": \"columns\",\n \"y\": \"rows\",\n \"geoid\": \"geoId\",\n }\n\n for key, field_str_list in all_params.items():\n field_list = []\n for field_str in field_str_list:\n if not field_str:\n continue\n field_info = copied_obj._convert_string_to_field_info(field_str)\n if field_info.get(\"aggName\"):\n copied_obj._update_single_chart_spec(\"config__defaultAggregated\", True)\n field_list.append(field_info)\n if field_info[\"fid\"] not in copied_obj._field_map:\n copied_obj._field_map[field_info[\"fid\"]] = field_info\n if field_info[\"analyticType\"] == \"dimension\":\n copied_obj._single_chart_spec[\"encodings\"][\"dimensions\"].append(field_info)\n else:\n copied_obj._single_chart_spec[\"encodings\"][\"measures\"].append(field_info)\n copied_obj._single_chart_spec[\"encodings\"][params_key_map.get(key, key)] = field_list\n\n return copied_obj\n\n # pylint: enable=unused-argument\n def layout(\n self,\n *,\n mode: Optional[Literal[\"auto\", \"fixed\", \"container\"]] = None,\n width: Optional[int] = None,\n height: Optional[int] = None,\n **kwargs\n ) -> \"Component\":\n \"\"\"\n Set layout config.\n example: .layout(resolve__color=False)\n {\n \"colorPalette\": \"paired\",\n \"format\": {},\n \"geoKey\": \"name\",\n \"interactiveScale\": False,\n \"resolve\": {\n \"color\": False,\n \"opacity\": False,\n \"shape\": False,\n \"size\": False,\n \"x\": False,\n \"y\": False,\n },\n \"scale\": {\n \"opacity\": {},\n \"size\": {},\n },\n \"scaleIncludeUnmatchedChoropleth\": False,\n \"showActions\": False,\n \"showTableSummary\": False,\n \"size\": {\n \"mode\": \"fixed\",\n \"width\": 360,\n \"height\": 360,\n },\n \"stack\": \"stack\",\n \"useSvg\": False,\n \"zeroScale\": True,\n }\n \"\"\"\n copied_obj = self.copy()\n layout_info = {\n \"size__mode\": mode,\n \"size__width\": width,\n \"size__height\": height,\n **kwargs\n }\n for key, value in layout_info.items():\n if value is None:\n continue\n copied_obj._update_single_chart_spec(\"layout__\" + key, value)\n\n return copied_obj\n\n def profiling(self) -> \"Component\":\n \"\"\"Profiling mode.\"\"\"\n copied_obj = self.copy()\n copied_obj._render_type = \"profiling\"\n return copied_obj\n\n def explorer(self) -> \"Component\":\n \"\"\"Explorer mode.\"\"\"\n copied_obj = self.copy()\n copied_obj._render_type = \"explorer\"\n return copied_obj\n# pylint: enable=protected-access\n\n\ndef component(\n dataset: Union[DataFrame, Connector, str],\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n spec: str = \"\",\n spec_io_mode: ISpecIOMode = \"rw\",\n theme_key: IThemeKey = \"vega\",\n appearance: IAppearance = \"media\",\n show_cloud_tool: Optional[bool] = False,\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n **kwargs\n) -> Component:\n \"\"\"\n Component class for creating a chain of components.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - spec (str): chart config data. config id, json, remote file url\n - spec_io_mode (ISpecIOMode): spec io mode, Default to \"r\", \"r\" for read, \"rw\" for read and write.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n \"\"\"\n walker = PygWalker(\n gid=None,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=True,\n kernel_computation=isinstance(dataset, (Connector, str)) or kernel_computation,\n use_save_tool=\"w\" in spec_io_mode,\n gw_mode=\"explore\",\n is_export_dataframe=\"w\" in spec_io_mode,\n kanaries_api_key=kanaries_api_key,\n default_tab=\"data\",\n cloud_computation=False,\n **kwargs\n )\n render_type = \"pure_chart\"\n field_map = {\n field[\"fid\"]: field\n for field in walker.data_parser.raw_fields\n }\n single_chart_spec = {\n \"name\": \"Chart 1\",\n \"visId\": \"\",\n \"config\": {\n \"coordSystem\": \"generic\",\n \"defaultAggregated\": False,\n \"geoms\": [\"auto\"],\n \"limit\": -1,\n \"timezoneDisplayOffset\": 0,\n },\n \"encodings\": {\n \"dimensions\": [field for field in walker.data_parser.raw_fields if field[\"analyticType\"] == \"dimension\"],\n \"measures\": [field for field in walker.data_parser.raw_fields if field[\"analyticType\"] == \"measure\"],\n \"rows\": [],\n \"columns\": [],\n \"color\": [],\n \"opacity\": [],\n \"size\": [],\n \"shape\": [],\n \"radius\": [],\n \"theta\": [],\n \"longitude\": [],\n \"latitude\": [],\n \"geoId\": [],\n \"details\": [],\n \"filters\": [],\n \"text\": [],\n },\n \"layout\": {\n \"format\": {},\n \"geoKey\": \"name\",\n \"interactiveScale\": False,\n \"resolve\": {\n \"color\": False,\n \"opacity\": False,\n \"shape\": False,\n \"size\": False,\n \"x\": False,\n \"y\": False,\n },\n \"showActions\": False,\n \"showTableSummary\": False,\n \"size\": {\n \"mode\": \"fixed\",\n \"width\": 360,\n \"height\": 360,\n },\n \"stack\": \"stack\",\n \"zeroScale\": True,\n }\n }\n return Component(\n walker=walker,\n render_type=render_type,\n field_map=field_map,\n single_chart_spec=single_chart_spec,\n )\n" }, { "path": "pygwalker/api/gradio.py", "content": "from typing import Union, List, Optional\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.communications.gradio_comm import (\n BASE_URL_PATH,\n GradioCommunication,\n PYGWALKER_ROUTE\n)\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, ISpecIOMode, IThemeKey\nfrom pygwalker.utils.check_walker_params import check_expired_params\n\n\n# pylint: disable=protected-access\ndef get_html_on_gradio(\n dataset: Union[DataFrame, Connector],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n spec_io_mode: ISpecIOMode = \"r\",\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n) -> str:\n \"\"\"Get pygwalker html render to gradio\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - env: (Literal['Jupyter' | 'Streamlit'], optional): The enviroment using pygwalker. Default as 'Jupyter'\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - spec_io_mode (ISpecIOMode): spec io mode, Default to \"r\", \"r\" for read, \"rw\" for read and write.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to True.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n check_expired_params(kwargs)\n\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs if field_specs is not None else [],\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, Connector) or kernel_computation,\n use_save_tool=\"w\" in spec_io_mode,\n is_export_dataframe=False,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=False,\n gw_mode=\"explore\",\n **kwargs\n )\n\n props = walker._get_props(\"gradio\")\n props[\"communicationUrl\"] = BASE_URL_PATH\n comm = GradioCommunication(str(walker.gid))\n walker._init_callback(comm)\n\n html = walker._get_render_iframe(props, True)\n return html\n" }, { "path": "pygwalker/api/html.py", "content": "from typing import Union, Dict, Optional, Any, List\nimport logging\n\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.services.data_parsers import get_parser\nfrom pygwalker.services.preview_image import render_gw_chart_preview_html\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.utils.randoms import generate_hash_code\nfrom pygwalker.utils.check_walker_params import check_expired_params\n\nlogger = logging.getLogger(__name__)\n\n\ndef _to_html(\n df: DataFrame,\n gid: Union[int, str] = None,\n *,\n spec: str = \"\",\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n gw_mode: Literal['explore', 'renderer', 'filter_renderer', 'table'] = \"explore\",\n width: Optional[int] = None,\n height: Optional[int] = None,\n **kwargs\n) -> str:\n \"\"\"\n Generate embeddable HTML code of Graphic Walker with data of `df`.\n\n Args:\n - df (pl.DataFrame | pd.DataFrame, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - spec (str): chart config data. config id, json, remote file url\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance ('media' | 'light' | 'dark'): 'media': auto detect OS theme.\n \"\"\"\n check_expired_params(kwargs)\n\n if gid is None:\n gid = generate_hash_code()\n\n if field_specs is None:\n field_specs = []\n\n walker = PygWalker(\n gid=gid,\n dataset=df,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=False,\n use_save_tool=False,\n gw_mode=gw_mode,\n is_export_dataframe=False,\n kanaries_api_key=\"\",\n default_tab=default_tab,\n cloud_computation=False,\n **kwargs\n )\n\n return walker.to_html(width, height)\n\n\ndef to_html(\n df: DataFrame,\n gid: Union[int, str] = None,\n *,\n spec: str = \"\",\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n) -> str:\n \"\"\"\n Generate embeddable HTML code of Graphic Walker with data of `df`.\n\n Args:\n - df (pl.DataFrame | pd.DataFrame, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - spec (str): chart config data. config id, json, remote file url\n - theme_key ('vega' | 'g2'): theme type.\n - appearance ('media' | 'light' | 'dark'): 'media': auto detect OS theme.\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n return _to_html(\n df,\n gid,\n spec=spec,\n field_specs=field_specs,\n theme_key=theme_key,\n appearance=appearance,\n default_tab=default_tab,\n **kwargs\n )\n\n\ndef to_table_html(\n df: DataFrame,\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n **kwargs\n) -> str:\n \"\"\"\n Generate embeddable HTML code of Graphic Walker with data of `df`.\n\n Args:\n - df (pl.DataFrame | pd.DataFrame, optional): dataframe.\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance ('media' | 'light' | 'dark'): 'media': auto detect OS theme.\n \"\"\"\n return _to_html(\n df,\n None,\n spec=\"\",\n field_specs=[],\n theme_key=theme_key,\n appearance=appearance,\n gw_mode=\"table\",\n height=\"800px\",\n **kwargs\n )\n\n\ndef to_render_html(\n df: DataFrame,\n spec: str,\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n **kwargs\n) -> str:\n \"\"\"\n Args:\n - df (pl.DataFrame | pd.DataFrame, optional): dataframe.\n - spec (str): chart config data. config id, json, remote file url\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance ('media' | 'light' | 'dark'): 'media': auto detect OS theme.\n \"\"\"\n return _to_html(\n df,\n None,\n spec=spec,\n field_specs=[],\n theme_key=theme_key,\n appearance=appearance,\n gw_mode=\"filter_renderer\",\n **kwargs\n )\n\n\ndef to_chart_html(\n dataset: Union[DataFrame, Connector, str],\n spec: Dict[str, Any],\n *,\n spec_type: Literal[\"graphic-walker\", \"vega\"] = \"graphic-walker\",\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n) -> str:\n \"\"\"\n Generate HTML code of a chart by graphic-walker or vega spec.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataset.\n - spec (Dict[str, Any]): chart config data.\n\n Kargs:\n - spec_type (Literal[\"graphic-walker\", \"vega\"]): type of spec.\n - theme_key ('vega' | 'g2'): theme type.\n - appearance ('media' | 'light' | 'dark'): 'media': auto detect OS theme.\n \"\"\"\n # pylint: disable=import-outside-toplevel\n # Since the compatibility of quick js is not certain, the related methods are lazy loaded.\n from pygwalker.utils.dsl_transform import vega_to_dsl, dsl_to_workflow\n\n data_parser = get_parser(dataset)\n if spec_type == \"vega\":\n gw_dsl = vega_to_dsl(spec, data_parser.raw_fields)\n else:\n gw_dsl = spec\n workflow = dsl_to_workflow(gw_dsl)\n\n data = data_parser.get_datas_by_payload(workflow)\n return render_gw_chart_preview_html(\n single_vis_spec=gw_dsl,\n data=data,\n theme_key=theme_key,\n appearance=appearance,\n title=\"\",\n desc=\"\"\n )\n" }, { "path": "pygwalker/api/jupyter.py", "content": "from typing import Union, List, Optional\nimport inspect\n\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.services.format_invoke_walk_code import get_formated_spec_params_code_from_frame\nfrom pygwalker.services.kaggle import auto_set_kanaries_api_key_on_kaggle, adjust_kaggle_default_font_size\nfrom pygwalker.utils.execute_env_check import check_convert, get_kaggle_run_type, check_kaggle\nfrom pygwalker.utils.check_walker_params import check_expired_params\nfrom pygwalker.utils import fallback_value\n\n\ndef walk(\n dataset: Union[DataFrame, Connector, str],\n gid: Union[int, str] = None,\n *,\n env: Literal['Jupyter', 'JupyterWidget'] = 'JupyterWidget',\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n use_kernel_calc: Optional[bool] = None,\n kernel_computation: Optional[bool] = None,\n cloud_computation: bool = False,\n show_cloud_tool: bool = True,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n):\n \"\"\"Walk through pandas.DataFrame df with Graphic Walker\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - env: (Literal['Jupyter' | 'JupyterWidget'], optional): The enviroment using pygwalker. Default as 'JupyterWidget'\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None, automatically determine whether to use kernel calculation.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n - cloud_computation(bool): Whether to use cloud compute for datas, it upload your data to kanaries cloud. Default to False.\n \"\"\"\n check_expired_params(kwargs)\n\n if field_specs is None:\n field_specs = []\n\n source_invoke_code = get_formated_spec_params_code_from_frame(\n inspect.stack()[1].frame\n )\n\n if check_kaggle():\n auto_set_kanaries_api_key_on_kaggle()\n\n if get_kaggle_run_type() == \"batch\":\n adjust_kaggle_default_font_size()\n env = \"JupyterPreview\"\n elif check_convert():\n env = \"JupyterConvert\"\n\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=source_invoke_code,\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=True,\n kernel_computation=env != \"JupyterConvert\" and (isinstance(dataset, (Connector, str)) or fallback_value(kernel_computation, use_kernel_calc)),\n use_save_tool=True,\n gw_mode=\"explore\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=cloud_computation,\n **kwargs\n )\n\n env_display_map = {\n \"JupyterWidget\": walker.display_on_jupyter_use_widgets,\n \"Jupyter\": walker.display_on_jupyter,\n \"JupyterConvert\": walker.display_on_convert_html,\n \"JupyterPreview\": walker.display_preview_on_jupyter\n }\n\n display_func = env_display_map.get(env, lambda: None)\n display_func()\n\n return walker\n\n\ndef render(\n dataset: Union[DataFrame, Connector, str],\n spec: str,\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n **kwargs\n):\n \"\"\"\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - spec (str): chart config data. config id, json, remote file url\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n \"\"\"\n\n walker = PygWalker(\n gid=None,\n dataset=dataset,\n field_specs=[],\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, (Connector, str)) or kernel_computation,\n use_save_tool=False,\n gw_mode=\"filter_renderer\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=\"vis\",\n cloud_computation=False,\n **kwargs\n )\n\n walker.display_on_jupyter_use_widgets()\n\n\ndef table(\n dataset: Union[DataFrame, Connector, str],\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n **kwargs\n):\n \"\"\"\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n \"\"\"\n walker = PygWalker(\n gid=None,\n dataset=dataset,\n field_specs=[],\n spec=\"\",\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, (Connector, str)) or kernel_computation,\n use_save_tool=False,\n gw_mode=\"table\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=\"vis\",\n cloud_computation=False,\n **kwargs\n )\n\n walker.display_on_jupyter_use_widgets(iframe_height=\"800px\")\n" }, { "path": "pygwalker/api/kanaries_cloud.py", "content": "from typing import List, Optional, Union\nfrom datetime import datetime\n\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker._typing import DataFrame\nfrom pygwalker.utils.display import display_html\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker.services.cloud_service import CloudService\nfrom pygwalker.services.data_parsers import get_parser\n\n\ndef create_cloud_dataset(\n dataset: Union[DataFrame, Connector],\n *,\n name: Optional[str] = None,\n is_public: bool = False,\n kanaries_api_key: str = \"\"\n) -> str:\n \"\"\"\n Create a dataset in kanaries cloud\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataset.\n\n Kargs:\n - name (str): dataset name in kanaries cloud.\n - is_public (bool): whether to make this dataset public.\n\n Returns:\n str: dataset id in kanaries cloud\n \"\"\"\n cloud_service = CloudService(kanaries_api_key)\n data_parser = get_parser(dataset, False, None)\n if name is None:\n name = f\"pygwalker_{datetime.now().strftime('%Y%m%d%H%M')}\"\n\n dataset_id = cloud_service.create_cloud_dataset(data_parser, name, is_public)\n return dataset_id\n\n\ndef create_cloud_walker(\n dataset: DataFrame,\n *,\n chart_name: str,\n workspace_name: str,\n field_specs: Optional[List[FieldSpec]] = None,\n kanaries_api_key: str = \"\"\n) -> str:\n \"\"\"\n (deprecated)\n Create a pygwalker in kanaries cloud\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame, optional): dataframe.\n\n Kargs:\n - chart_name (str): pygwalker chart name in kanaries cloud.\n - workspace_name (str): kanaries workspace name.\n - field_specs (Dict[str, FieldSpec]): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n\n Returns:\n str: pygwalker url in kanaries cloud\n \"\"\"\n if field_specs is None:\n field_specs = []\n\n cloud_service = CloudService(kanaries_api_key)\n data_parser = get_parser(dataset, False, field_specs)\n\n cloud_service.create_cloud_graphic_walker(\n chart_name=chart_name,\n workspace_name=workspace_name,\n dataset_content=data_parser.to_parquet(),\n field_specs=data_parser.raw_fields\n )\n\n\ndef walk_on_cloud(workspace_name: str, chart_name: str, kanaries_api_key: str = \"\"):\n \"\"\"\n (deprecated)\n render a pygwalker in kanaries cloud\n\n Args:\n - chart_name (str): pygwalker chart name in kanaries cloud.\n - workspace_name (str): kanaries workspace name.\n \"\"\"\n cloud_service = CloudService(kanaries_api_key)\n cloud_url = cloud_service.get_cloud_graphic_walker(workspace_name, chart_name)\n\n iframe_html = f\"\"\"\n \n \n \"\"\"\n\n display_html(iframe_html)\n" }, { "path": "pygwalker/api/marimo.py", "content": "from typing import Union, List, Optional\nimport inspect\nimport json\nimport pathlib\n\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.services.format_invoke_walk_code import get_formated_spec_params_code_from_frame\nfrom pygwalker.communications.anywidget_comm import AnywidgetCommunication\nimport marimo as mo\nimport anywidget\nimport traitlets\n\n\nclass _WalkerWidget(anywidget.AnyWidget):\n \"\"\"WalkerWidget\"\"\"\n _esm = (pathlib.Path(__file__).parent.parent / \"templates\" / \"dist\" / \"pygwalker-app.es.js\").read_text(encoding=\"utf-8\")\n props = traitlets.Unicode(\"\").tag(sync=True)\n\n\ndef walk(\n dataset: Union[DataFrame, Connector, str],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n show_cloud_tool: bool = False,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n):\n \"\"\"Walk through pandas.DataFrame df with Graphic Walker\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n if field_specs is None:\n field_specs = []\n\n source_invoke_code = get_formated_spec_params_code_from_frame(\n inspect.stack()[1].frame\n )\n\n widget = _WalkerWidget()\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=source_invoke_code,\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=False,\n kernel_computation=True,\n use_save_tool=True,\n gw_mode=\"explore\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=False,\n **kwargs\n )\n comm = AnywidgetCommunication(walker.gid)\n\n widget.props = json.dumps(walker._get_props(\"marimo\", []))\n comm.register_widget(widget)\n walker._init_callback(comm)\n\n return mo.ui.anywidget(widget)\n" }, { "path": "pygwalker/api/pygwalker.py", "content": "import base64\nfrom typing import List, Dict, Any, Optional, Union\nimport urllib\nimport json\nimport os, sys, subprocess\nimport urllib.parse\nimport zlib\n\nfrom typing_extensions import Literal\nfrom duckdb import ParserException\nimport ipywidgets\nimport pandas as pd\n\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.data_parsers.base import BaseDataParser, FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker.utils.display import display_html\nfrom pygwalker.utils.randoms import rand_str\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom pygwalker.services.render import (\n render_gwalker_html,\n render_gwalker_iframe,\n get_max_limited_datas,\n render_iframe_messages_html\n)\nfrom pygwalker.services.config import set_config\nfrom pygwalker.services.preview_image import (\n PreviewImageTool,\n ChartData,\n render_gw_preview_html,\n render_gw_chart_preview_html\n)\nfrom pygwalker.services.upload_data import (\n BatchUploadDatasToolOnWidgets,\n BatchUploadDatasToolOnJupyter\n)\nfrom pygwalker.services.config import get_local_user_id\nfrom pygwalker.services.spec import get_spec_json, fill_new_fields\nfrom pygwalker.services.data_parsers import get_parser\nfrom pygwalker.services.cloud_service import CloudService\nfrom pygwalker.services.check_update import check_update\nfrom pygwalker.services.track import track_event\nfrom pygwalker.utils.randoms import generate_hash_code\nfrom pygwalker.communications.hacker_comm import HackerCommunication, BaseCommunication\nfrom pygwalker._constants import JUPYTER_BYTE_LIMIT, JUPYTER_WIDGETS_BYTE_LIMIT\nfrom pygwalker import __version__\n\n\nclass PygWalker:\n \"\"\"PygWalker\"\"\"\n def __init__(\n self,\n *,\n gid: Optional[Union[int, str]],\n dataset: Union[DataFrame, Connector, str],\n field_specs: List[FieldSpec],\n spec: str,\n source_invoke_code: str,\n theme_key: IThemeKey,\n appearance: IAppearance,\n show_cloud_tool: Optional[bool],\n use_preview: bool,\n kernel_computation: Optional[bool],\n cloud_computation: Optional[bool],\n use_save_tool: bool,\n is_export_dataframe: bool,\n kanaries_api_key: str,\n default_tab: Literal[\"data\", \"vis\"],\n gw_mode: Literal[\"explore\", \"renderer\", \"filter_renderer\", \"table\"],\n **kwargs\n ):\n self.kanaries_api_key = kanaries_api_key or GlobalVarManager.kanaries_api_key\n if gid is None:\n self.gid = generate_hash_code()\n else:\n self.gid = gid\n self.cloud_service = CloudService(self.kanaries_api_key)\n self.data_parser = self._get_data_parser(\n dataset=dataset,\n field_specs=field_specs,\n cloud_computation=cloud_computation,\n kanaries_api_key=self.kanaries_api_key,\n cloud_service=self.cloud_service\n )\n\n suggest_kernel_computation = self.data_parser.data_size > JUPYTER_BYTE_LIMIT\n self.kernel_computation = suggest_kernel_computation if kernel_computation is None else kernel_computation\n self.origin_data_source = self.data_parser.to_records(500 if self.kernel_computation else None)\n self.field_specs = self.data_parser.raw_fields\n self.spec = spec\n self.source_invoke_code = source_invoke_code\n self.theme_key = theme_key\n self.appearance = appearance\n self.data_source_id = rand_str()\n self.other_props = kwargs\n self.tunnel_id = \"tunnel!\"\n self.show_cloud_tool = bool(self.kanaries_api_key) if show_cloud_tool is None else show_cloud_tool\n self.use_preview = use_preview\n self._init_spec(spec, self.field_specs)\n self.use_save_tool = use_save_tool\n self.parse_dsl_type = self._get_parse_dsl_type(self.data_parser)\n self.gw_mode = gw_mode\n self.dataset_type = self.data_parser.dataset_type\n self.is_export_dataframe = is_export_dataframe\n self._last_exported_dataframe = None\n self.default_tab = default_tab\n self.cloud_computation = cloud_computation\n self.comm = None\n check_update()\n # Temporarily adapt to pandas import module bug\n if self.kernel_computation:\n try:\n self.data_parser.get_datas_by_sql(\"SELECT 1 FROM pygwalker_mid_table LIMIT 1\")\n except Exception:\n pass\n if GlobalVarManager.privacy == \"offline\":\n self.show_cloud_tool = False\n\n @property\n def last_exported_dataframe(self) -> Optional[pd.DataFrame]:\n return self._last_exported_dataframe\n\n def _get_data_parser(\n self,\n *,\n dataset: Union[DataFrame, Connector, str],\n field_specs: List[FieldSpec],\n cloud_computation: bool,\n kanaries_api_key: str,\n cloud_service: CloudService\n ) -> BaseDataParser:\n data_parser = get_parser(\n dataset,\n field_specs,\n other_params={\"kanaries_api_key\": kanaries_api_key}\n )\n if not cloud_computation:\n return data_parser\n\n dataset_id = cloud_service.create_cloud_dataset(\n data_parser,\n f\"temp_{rand_str()}\",\n False,\n True\n )\n\n return get_parser(\n dataset_id,\n field_specs,\n other_params={\"kanaries_api_key\": kanaries_api_key}\n )\n\n def _get_parse_dsl_type(self, data_parser: BaseDataParser) -> Literal[\"server\", \"client\"]:\n if data_parser.dataset_type.startswith(\"connector\"):\n return \"server\"\n if data_parser.dataset_type == \"cloud_dataset\":\n return \"server\"\n return \"client\"\n\n def _init_spec(self, spec: Dict[str, Any], field_specs: List[FieldSpec]):\n spec_obj, spec_type = get_spec_json(spec)\n if spec_type.startswith(\"vega\"):\n self._update_vis_spec(spec_obj[\"config\"])\n else:\n self._update_vis_spec(spec_obj[\"config\"] and fill_new_fields(spec_obj[\"config\"], field_specs))\n self.spec_type = spec_type\n self._chart_map = self._parse_chart_map_dict(spec_obj[\"chart_map\"])\n self.spec_version = spec_obj.get(\"version\", None)\n self.workflow_list = spec_obj.get(\"workflow_list\", [])\n\n def _update_vis_spec(self, vis_spec: List[Dict[str, Any]]):\n self.vis_spec = vis_spec\n self._chart_name_index_map = {\n item[\"name\"]: index\n for index, item in enumerate(vis_spec)\n if \"name\" in item\n }\n\n def _get_chart_map_dict(self, chart_map: Dict[str, ChartData]) -> Dict[str, Any]:\n return {\n key: value.dict(by_alias=True)\n for key, value in chart_map.items()\n }\n\n def _parse_chart_map_dict(self, chart_map_dict: Dict[str, Any]) -> Dict[str, ChartData]:\n return {\n key: ChartData.parse_obj(value)\n for key, value in chart_map_dict.items()\n }\n\n def to_html(self, iframe_width: Optional[str] = None, iframe_height: Optional[str] = None) -> str:\n props = self._get_props()\n return self._get_render_iframe(props, iframe_width=iframe_width, iframe_height=iframe_height)\n\n def to_html_without_iframe(self) -> str:\n props = self._get_props()\n html = render_gwalker_html(self.gid, props)\n return html\n\n def display_on_convert_html(self):\n \"\"\"\n Display on jupyter-nbconvert html.\n \"\"\"\n props = self._get_props(\"jupyter\")\n iframe_html = self._get_render_iframe(props)\n display_html(iframe_html)\n\n def display_on_jupyter(self):\n \"\"\"\n Display on jupyter notebook/lab.\n If share has large data loading, only sample data can be displayed when reload.\n After that, it will be changed to python for data calculation,\n and only a small amount of data will be output to the front end to complete the analysis of big data.\n \"\"\"\n data_source = get_max_limited_datas(self.origin_data_source, JUPYTER_BYTE_LIMIT)\n props = self._get_props(\n \"jupyter\",\n data_source,\n len(self.origin_data_source) > len(data_source)\n )\n iframe_html = self._get_render_iframe(props)\n\n if len(self.origin_data_source) > len(data_source):\n upload_tool = BatchUploadDatasToolOnJupyter()\n display_html(iframe_html)\n upload_tool.run(\n records=self.origin_data_source,\n sample_data_count=0,\n data_source_id=self.data_source_id,\n gid=self.gid,\n tunnel_id=self.tunnel_id,\n )\n else:\n display_html(iframe_html)\n display_html(render_iframe_messages_html(self.gid))\n\n def display_on_jupyter_use_widgets(self, iframe_width: Optional[str] = None, iframe_height: Optional[str] = None):\n \"\"\"\n use ipywidgets, Display on jupyter notebook/lab.\n When the kernel is down, the chart will not be displayed, so use `display_on_jupyter` to share\n \"\"\"\n comm = HackerCommunication(self.gid)\n preview_tool = PreviewImageTool(self.gid)\n data_source = get_max_limited_datas(self.origin_data_source, JUPYTER_WIDGETS_BYTE_LIMIT)\n props = self._get_props(\n \"jupyter_widgets\",\n data_source,\n len(self.origin_data_source) > len(data_source)\n )\n iframe_html = self._get_render_iframe(props, iframe_width=iframe_width, iframe_height=iframe_height)\n\n html_widgets = ipywidgets.Box(\n [ipywidgets.HTML(iframe_html), comm.get_widgets()],\n layout=ipywidgets.Layout(display='block')\n )\n\n self._init_callback(comm, preview_tool)\n\n display_html(html_widgets)\n display_html(render_iframe_messages_html(self.gid))\n preview_tool.init_display()\n preview_tool.async_render_gw_review(self._get_gw_preview_html())\n\n def display_preview_on_jupyter(self):\n \"\"\"\n Display preview on jupyter notebook/lab.\n \"\"\"\n display_html(self._get_gw_preview_html(True))\n\n @property\n def chart_list(self) -> List[str]:\n \"\"\"\n Get the list of saved charts.\n \"\"\"\n return list(self._chart_map.keys())\n\n def save_chart_to_file(self, chart_name: str, path: str, save_type: Literal[\"html\", \"png\", \"svg\"] = \"png\"):\n \"\"\"\n Save the chart to a file.\n \"\"\"\n if save_type == \"html\":\n content = self.export_chart_html(chart_name)\n write_mode = \"w\"\n encoding = \"utf-8\"\n elif save_type == \"png\":\n content = self.export_chart_png(chart_name)\n write_mode = \"wb\"\n encoding = None\n elif save_type == \"svg\":\n content = self.export_chart_svg(chart_name)\n write_mode = \"wb\"\n encoding = None\n else:\n raise ValueError(f\"save_type must be html, png or svg, but got {save_type}\")\n\n with open(path, write_mode, encoding=encoding) as f:\n f.write(content)\n\n def export_chart_html(self, chart_name: str) -> str:\n \"\"\"\n Export the chart as a html string.\n \"\"\"\n return self._get_gw_chart_preview_html(\n chart_name,\n title=\"\",\n desc=\"\"\n )\n\n def export_chart_png(self, chart_name: str) -> bytes:\n \"\"\"\n Export the chart as a png bytes.\n \"\"\"\n chart_data = self._get_chart_by_name(chart_name)\n\n with urllib.request.urlopen(chart_data.single_chart) as png_string:\n return png_string.read()\n\n def export_chart_svg(self, chart_name: str) -> bytes:\n \"\"\"Export the chart as svg bytes.\"\"\"\n chart_data = self._get_chart_by_name(chart_name)\n if len(chart_data.charts) == 0:\n raise ValueError(f\"chart_name: {chart_name} has no svg data\")\n svg_str = chart_data.charts[0].data\n prefix = \"data:image/svg+xml;base64,\"\n if isinstance(svg_str, str) and svg_str.startswith(prefix):\n import base64\n return base64.b64decode(svg_str[len(prefix):])\n if isinstance(svg_str, str):\n return svg_str.encode(\"utf-8\")\n return svg_str\n\n def display_chart(self, chart_name: str, *, title: Optional[str] = None, desc: str = \"\"):\n \"\"\"\n Display the chart in the notebook.\n \"\"\"\n if title is None:\n title = chart_name\n\n html = self._get_gw_chart_preview_html(\n chart_name,\n title=title,\n desc=desc\n )\n display_html(html)\n\n def get_single_chart_html_by_spec(\n self,\n *,\n spec: Dict[str, Any],\n title: str = \"\",\n desc: str = \"\",\n ) -> str:\n # pylint: disable=import-outside-toplevel\n from pygwalker.utils.dsl_transform import dsl_to_workflow\n workflow = dsl_to_workflow(spec)\n data = self.data_parser.get_datas_by_payload(workflow)\n return render_gw_chart_preview_html(\n single_vis_spec=spec,\n data=data,\n theme_key=self.theme_key,\n title=title,\n desc=desc,\n appearance=self.appearance\n )\n\n def _get_chart_by_name(self, chart_name: str) -> ChartData:\n if chart_name not in self._chart_map:\n raise ValueError(f\"chart_name: {chart_name} not found, please confirm whether to save\")\n return self._chart_map[chart_name]\n\n # TODO: using the better way to handle callback\n def _init_callback(self, comm: BaseCommunication, preview_tool: PreviewImageTool = None):\n upload_tool = BatchUploadDatasToolOnWidgets(comm)\n self.comm = comm\n\n def reuqest_data_callback(_):\n upload_tool.run(\n records=self.origin_data_source,\n sample_data_count=0,\n data_source_id=self.data_source_id\n )\n return {}\n\n def get_latest_vis_spec(_):\n return {\"visSpec\": self.vis_spec}\n\n def save_chart_endpoint(data: Dict[str, Any]):\n chart_data = ChartData.parse_obj(data)\n self._chart_map[data[\"title\"]] = chart_data\n\n def update_spec(data: Dict[str, Any]):\n spec_obj = {\n \"config\": data[\"visSpec\"],\n \"chart_map\": {},\n \"version\": __version__,\n \"workflow_list\": data.get(\"workflowList\", [])\n }\n self._update_vis_spec(data[\"visSpec\"])\n self.spec_version = __version__\n self.workflow_list = data.get(\"workflowList\", [])\n\n if self.use_preview:\n preview_tool.async_render_gw_review(self._get_gw_preview_html())\n\n save_chart_endpoint(data[\"chartData\"])\n\n if self.spec_type == \"json_file\":\n with open(self.spec, \"w\", encoding=\"utf-8\") as f:\n f.write(json.dumps(spec_obj))\n if self.spec_type == \"json_ksf\":\n self.cloud_service.write_config_to_cloud(self.spec[6:], json.dumps(spec_obj))\n\n def upload_spec_to_cloud(data: Dict[str, Any]):\n if data[\"newToken\"]:\n set_config({\"kanaries_token\": data[\"newToken\"]})\n GlobalVarManager.kanaries_api_key = data[\"newToken\"]\n spec_obj = {\n \"config\": self.vis_spec,\n \"chart_map\": {},\n \"version\": __version__,\n \"workflow_list\": self.workflow_list,\n }\n file_name = data[\"fileName\"]\n workspace_name = self.cloud_service.get_kanaries_user_info()[\"workspaceName\"]\n path = f\"{workspace_name}/{file_name}\"\n self.cloud_service.write_config_to_cloud(path, json.dumps(spec_obj))\n return {\"specFilePath\": path}\n\n def _get_datas(data: Dict[str, Any]):\n sql = data[\"sql\"]\n datas = self.data_parser.get_datas_by_sql(sql)\n return {\n \"datas\": datas\n }\n\n def _get_datas_by_payload(data: Dict[str, Any]):\n datas = self.data_parser.get_datas_by_payload(data[\"payload\"])\n return {\n \"datas\": datas\n }\n\n def _batch_get_datas_by_sql(data: Dict[str, Any]):\n result = self.data_parser.batch_get_datas_by_sql(data[\"queryList\"])\n return {\n \"datas\": result\n }\n\n def _batch_get_datas_by_payload(data: Dict[str, Any]):\n result = self.data_parser.batch_get_datas_by_payload(data[\"queryList\"])\n return {\n \"datas\": result\n }\n\n def _get_spec_by_text(data: Dict[str, Any]):\n callback = self.other_props.get(\n \"custom_ask_callback\",\n self.cloud_service.get_spec_by_text\n )\n return {\n \"data\": callback(data[\"metas\"], data[\"query\"])\n }\n\n def _get_chart_by_chats(data: Dict[str, Any]):\n callback = self.other_props.get(\n \"custom_chat_callback\",\n self.cloud_service.get_chart_by_chats\n )\n return {\n \"data\": callback(data[\"metas\"], data[\"chats\"])\n }\n\n def _export_dataframe_by_payload(data: Dict[str, Any]):\n df = pd.DataFrame(self.data_parser.get_datas_by_payload(data[\"payload\"]))\n GlobalVarManager.set_last_exported_dataframe(df)\n self._last_exported_dataframe = df\n\n def _export_dataframe_by_sql(data: Dict[str, Any]):\n sql = data[\"sql\"]\n df = pd.DataFrame(self.data_parser.get_datas_by_sql(sql))\n GlobalVarManager.set_last_exported_dataframe(df)\n self._last_exported_dataframe = df\n\n def _upload_to_cloud_charts(data: Dict[str, Any]):\n result = self.cloud_service.upload_cloud_chart(\n data_parser=self.data_parser,\n chart_name=data[\"chartName\"],\n dataset_name=data[\"datasetName\"],\n workflow=data[\"workflow\"],\n spec_list=data[\"visSpec\"],\n is_public=data[\"isPublic\"],\n )\n return {\"chartId\": result[\"chart_id\"], \"datasetId\": result[\"dataset_id\"]}\n\n def _upload_to_cloud_dashboard(data: Dict[str, Any]):\n result = self.cloud_service.upload_cloud_dashboard(\n data_parser=self.data_parser,\n dashboard_name=data[\"chartName\"],\n dataset_name=data[\"datasetName\"],\n workflow_list=data[\"workflowList\"],\n spec_list=data[\"visSpec\"],\n is_public=data[\"isPublic\"],\n create_dashboard_flag=data[\"isCreateDashboard\"],\n appearance=self.appearance\n )\n return {\"dashboardId\": result[\"dashboard_id\"], \"datasetId\": result[\"dataset_id\"]}\n\n def _open_protocol(link):\n if sys.platform == \"win32\":\n os.startfile(link)\n else:\n opener = \"open\" if sys.platform == \"darwin\" else \"xdg-open\"\n subprocess.call([opener, link])\n\n def compress_data(data: str) -> str:\n compress = zlib.compressobj(zlib.Z_BEST_COMPRESSION, zlib.DEFLATED, 15, 8, 0)\n compressed_data = compress.compress(data.encode())\n compressed_data += compress.flush()\n return urllib.parse.quote(base64.b64encode(compressed_data).decode())\n\n def open_in_desktop(data: Dict[str, Any]):\n spec = json.dumps(data['spec'])\n fields = json.dumps(data['fields'])\n data = json.dumps(self.data_parser.to_records(), default=lambda obj: obj.isoformat() if hasattr(obj, 'isoformat') else str(obj))\n _open_protocol(f\"gw://import?data={compress_data(data)}&spec={compress_data(spec)}&fields={compress_data(fields)}\")\n\n comm.register(\"get_latest_vis_spec\", get_latest_vis_spec)\n comm.register(\"request_data\", reuqest_data_callback)\n comm.register(\"ping\", lambda _: {})\n comm.register(\"open_in_desktop\", open_in_desktop)\n\n if self.use_save_tool:\n comm.register(\"upload_spec_to_cloud\", upload_spec_to_cloud)\n comm.register(\"update_spec\", update_spec)\n comm.register(\"save_chart\", save_chart_endpoint)\n\n if self.show_cloud_tool:\n comm.register(\"upload_to_cloud_charts\", _upload_to_cloud_charts)\n comm.register(\"upload_to_cloud_dashboard\", _upload_to_cloud_dashboard)\n comm.register(\"get_spec_by_text\", _get_spec_by_text)\n comm.register(\"get_chart_by_chats\", _get_chart_by_chats)\n\n if self.kernel_computation:\n comm.register(\"get_datas\", _get_datas)\n comm.register(\"get_datas_by_payload\", _get_datas_by_payload)\n comm.register(\"batch_get_datas_by_sql\", _batch_get_datas_by_sql)\n comm.register(\"batch_get_datas_by_payload\", _batch_get_datas_by_payload)\n\n if self.is_export_dataframe:\n comm.register(\"export_dataframe_by_payload\", _export_dataframe_by_payload)\n comm.register(\"export_dataframe_by_sql\", _export_dataframe_by_sql)\n\n def _send_props_track(self, props: Dict[str, Any]):\n needed_fields = {\n \"id\", \"version\", \"hashcode\", \"themeKey\",\n \"dark\", \"env\", \"specType\", \"needLoadDatas\", \"showCloudTool\",\n \"useKernelCalc\", \"useSaveTool\", \"parseDslType\", \"gwMode\", \"datasetType\",\n \"defaultTab\", \"useCloudCalc\"\n }\n event_info = {key: value for key, value in props.items() if key in needed_fields}\n event_info[\"hasKanariesToken\"] = bool(self.kanaries_api_key)\n\n track_event(\"invoke_props\", event_info)\n\n def _get_props(\n self,\n env: str = \"\",\n data_source: Optional[Dict[str, Any]] = None,\n need_load_datas: bool = False\n ) -> Dict[str, Any]:\n if data_source is None:\n data_source = self.origin_data_source\n props = {\n \"id\": self.gid,\n \"dataSource\": data_source,\n \"len\": len(data_source),\n \"version\": __version__,\n \"hashcode\": get_local_user_id(),\n \"userConfig\": {\n \"privacy\": GlobalVarManager.privacy,\n },\n \"visSpec\": self.vis_spec,\n \"rawFields\": [\n {**field, \"offset\": 0}\n for field in self.field_specs\n ],\n \"fieldkeyGuard\": False,\n \"themeKey\": self.theme_key,\n \"dark\": self.appearance,\n \"sourceInvokeCode\": self.source_invoke_code,\n \"dataSourceProps\": {\n 'tunnelId': self.tunnel_id,\n 'dataSourceId': self.data_source_id,\n },\n \"env\": env,\n \"specType\": self.spec_type,\n \"needLoadDatas\": not self.kernel_computation and need_load_datas,\n \"showCloudTool\": self.show_cloud_tool,\n \"enableAskViz\": GlobalVarManager.enable_askviz,\n \"enableVlChat\": GlobalVarManager.enable_vlchat,\n \"needInitChart\": not self._chart_map,\n \"useKernelCalc\": self.kernel_computation,\n \"useSaveTool\": self.use_save_tool,\n \"parseDslType\": self.parse_dsl_type,\n \"gwMode\": self.gw_mode,\n \"needLoadLastSpec\": True,\n \"datasetType\": self.dataset_type,\n \"extraConfig\": self.other_props,\n \"fieldMetas\": self.data_parser.field_metas,\n \"isExportDataFrame\": self.is_export_dataframe,\n \"defaultTab\": self.default_tab,\n \"useCloudCalc\": self.cloud_computation\n }\n\n self._send_props_track(props)\n\n return props\n\n def _get_render_iframe(\n self,\n props: Dict[str, Any],\n return_iframe: bool = True,\n iframe_width: Optional[str] = None,\n iframe_height: Optional[str] = None\n ) -> str:\n \"\"\"Get render iframe html.\"\"\"\n html = render_gwalker_html(self.gid, props)\n if return_iframe:\n return render_gwalker_iframe(self.gid, html, iframe_width, iframe_height, self.appearance)\n else:\n return html\n\n def _get_gw_preview_html(self, manual: bool = False) -> str:\n \"\"\"\n 'manual' represents the user actively calling to obtain preview_html. It will randomly generate a gid, keeping it separate from the logic of walker automatically generating the preview part.\n \"\"\"\n if not self.workflow_list:\n return \"\"\n datas = []\n for workflow in self.workflow_list:\n try:\n datas.append(self.data_parser.get_datas_by_payload(workflow))\n except ParserException:\n datas.append([])\n html = render_gw_preview_html(\n self.vis_spec,\n datas,\n self.theme_key,\n self.gid if not manual else self.gid + rand_str(),\n self.appearance\n )\n\n return html\n\n def _get_gw_chart_preview_html(self, chart_name: int, title: str, desc: str) -> str:\n if chart_name not in self._chart_name_index_map:\n raise ValueError(f\"chart_name: {chart_name} not found.\")\n chart_index = self._chart_name_index_map[chart_name]\n\n if not self.workflow_list:\n return \"\"\n data = self.data_parser.get_datas_by_payload(self.workflow_list[chart_index])\n return render_gw_chart_preview_html(\n single_vis_spec=self.vis_spec[chart_index],\n data=data,\n theme_key=self.theme_key,\n title=title,\n desc=desc,\n appearance=self.appearance\n )\n" }, { "path": "pygwalker/api/reflex.py", "content": "from typing import Union, List, Optional\n\nimport reflex as rx\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.communications.reflex_comm import (\n BASE_URL_PATH,\n ReflexCommunication,\n)\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, ISpecIOMode, IThemeKey\nfrom pygwalker.utils.check_walker_params import check_expired_params\n\n\n# pylint: disable=protected-access\n\ndef get_component(\n dataset: Union[DataFrame, Connector],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = \"g2\",\n appearance: IAppearance = \"media\",\n spec: str = \"\",\n spec_io_mode: ISpecIOMode = \"r\",\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs,\n) -> rx.Component:\n \"\"\"Get a Reflex component that renders Pygwalker.\"\"\"\n check_expired_params(kwargs)\n\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs if field_specs is not None else [],\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, Connector) or kernel_computation,\n use_save_tool=\"w\" in spec_io_mode,\n is_export_dataframe=False,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=False,\n gw_mode=\"explore\",\n **kwargs,\n )\n\n props = walker._get_props(\"reflex\")\n props[\"communicationUrl\"] = BASE_URL_PATH\n comm = ReflexCommunication(str(walker.gid))\n walker._init_callback(comm)\n\n html = walker._get_render_iframe(props, True)\n return rx.html(html)\n" }, { "path": "pygwalker/api/streamlit.py", "content": "from typing import Union, Dict, Optional, List, Any, Tuple\nfrom packaging.version import Version\nfrom copy import deepcopy\nimport json\n\nfrom typing_extensions import Literal, deprecated\nfrom pydantic import BaseModel\nfrom cachetools import cached, TTLCache\nimport arrow\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.communications.streamlit_comm import (\n hack_streamlit_server,\n BASE_URL_PATH,\n StreamlitCommunication\n)\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, ISpecIOMode, IThemeKey\nfrom pygwalker.utils.randoms import rand_str\nfrom pygwalker.utils.check_walker_params import check_expired_params\nfrom pygwalker.utils import fallback_value\nfrom pygwalker.services.streamlit_components import pygwalker_component\nfrom pygwalker.services.data_parsers import get_dataset_hash\n\n\nclass PreFilter(BaseModel):\n \"\"\"\n Pre Filter.\n example:\n 1. use temporal range: pass in millisecond timestamp.\n PreFilter(field=\"date\", op=\"temporal range\", value=[1293840000000, 1297641600000])\n PreFilter(field=\"date\", op=\"temporal range\", value=[\"2019-01-01\", \"2020-01-01\"])\n 2. use range: pass in number.\n PreFilter(field=\"age\", op=\"range\", value=[0, 100])\n 3. use one of: pass in string or number.\n PreFilter(field=\"category\", op=\"one of\", value=[\"a\", \"b\", \"c\"])\n \"\"\"\n field: str\n op: Literal[\"range\", \"temporal range\", \"one of\"]\n value: List[Union[int, float, str]]\n\n\ndef init_streamlit_comm():\n \"\"\"Initialize pygwalker communication in streamlit\"\"\"\n hack_streamlit_server()\n\n\n# pylint: disable=protected-access\nclass StreamlitRenderer:\n \"\"\"Streamlit Renderer\"\"\"\n def __init__(\n self,\n dataset: Union[DataFrame, Connector],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n spec_io_mode: ISpecIOMode = \"r\",\n kernel_computation: Optional[bool] = None,\n use_kernel_calc: Optional[bool] = True,\n show_cloud_tool: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n ):\n \"\"\"Get pygwalker html render to streamlit.\n In Streamlit, pygwalker calculates a somewhat inaccurate gid based on the dataset to \n distinguish between datasets and uses it as the key for the Streamlit component to\n avoid redundant rendering.\n\n In some use case, If user frequently use the same StreamlitRenderer to receive different dataframes,\n and the differences between these dataframes are so small that pygwalker's gid calculation logic cannot distinguish between different datasets,\n user should customize method to generate a gid to differentiate between datasets.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - spec_io_mode (ISpecIOMode): spec io mode, Default to \"r\", \"r\" for read, \"rw\" for read and write.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to True.\n - use_kernel_calc(bool): Deprecated, use kernel_computation instead.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n check_expired_params(kwargs)\n\n init_streamlit_comm()\n\n self.walker = PygWalker(\n gid=gid if gid is not None else get_dataset_hash(dataset),\n dataset=dataset,\n field_specs=field_specs if field_specs is not None else [],\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=False,\n kernel_computation=isinstance(dataset, Connector) or fallback_value(kernel_computation, use_kernel_calc),\n use_save_tool=\"w\" in spec_io_mode,\n is_export_dataframe=False,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=False,\n gw_mode=\"explore\",\n **kwargs\n )\n comm = StreamlitCommunication(str(self.walker.gid))\n self.walker._init_callback(comm)\n self.global_pre_filters = None\n\n @cached(cache=TTLCache(maxsize=256, ttl=1800))\n def _get_html_with_params_str_cache(self, params_str: str) -> str:\n params = dict(json.loads(params_str))\n mode = params.pop(\"mode\")\n vis_spec = params.pop(\"vis_spec\")\n kwargs = params\n\n props = self.walker._get_props(\"streamlit\")\n\n props[\"communicationUrl\"] = BASE_URL_PATH\n props[\"gwMode\"] = mode\n if vis_spec is not None:\n props[\"visSpec\"] = vis_spec\n\n props.update(kwargs)\n\n return self.walker._get_render_iframe(props, False)\n\n def _get_html(\n self,\n *,\n mode: Literal[\"explore\", \"renderer\", \"filter_renderer\"] = \"explore\",\n vis_spec: Optional[List[Dict[str, Any]]] = None,\n **kwargs: Dict[str, Any]\n ) -> str:\n \"\"\"\n Get the html for streamlit.\n Kwargs will update origin props.\n \"\"\"\n params_str = json.dumps(sorted({\n \"mode\": mode,\n \"vis_spec\": vis_spec,\n **kwargs\n }.items()))\n\n return self._get_html_with_params_str_cache(params_str)\n\n def _convert_pre_filters_to_gw_config(\n self,\n pre_filters: List[PreFilter],\n spec_obj: Dict[str, Any]\n ) -> List[Dict[str, Any]]:\n field_map = {\n field[\"name\"]: field\n for field in spec_obj[\"encodings\"][\"dimensions\"] + spec_obj[\"encodings\"][\"measures\"]\n }\n\n gw_filters = []\n for pre_filter in pre_filters:\n if pre_filter.op == \"temporal range\":\n values = [\n int(arrow.get(value).timestamp() * 1000)\n for value in pre_filter.value\n ]\n else:\n values = pre_filter.value\n\n gw_filters.append({\n **field_map[pre_filter.field],\n \"dragId\": \"gw_\" + rand_str(4),\n \"rule\": {\n \"type\": pre_filter.op,\n \"value\": values\n }\n })\n return gw_filters\n\n def set_global_pre_filters(self, pre_filters: List[PreFilter]):\n \"\"\"It will append new filters to exists charts.\"\"\"\n self.global_pre_filters = pre_filters\n\n def viewer(self, *, key: str = \"viewer\"):\n \"\"\"Render filter renderer UI\"\"\"\n key = f\"{self.walker.gid}-{key}\"\n return self._component(key=key, mode=\"filter_renderer\")\n\n @deprecated(\"render_filter_renderer is deprecated, use viewer instead.\")\n def render_filter_renderer(self, *args, **kwargs):\n return self.viewer(*args, **kwargs)\n\n def explorer(\n self,\n *,\n key: str = \"explorer\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\"\n ):\n \"\"\"Render explore UI(it can drag and drop fields)\"\"\"\n key = f\"{self.walker.gid}-{key}\"\n return self._component(key=key, mode=\"explore\", defaultTab=default_tab)\n\n @deprecated(\"render_explore is deprecated, use explorer instead.\")\n def render_explore(self, *args, **kwargs):\n return self.explorer(*args, **kwargs)\n\n def chart(\n self,\n index: int,\n *,\n key: str = \"chart\",\n size: Optional[Tuple[int, int]] = None,\n pre_filters: Optional[List[PreFilter]] = None,\n ):\n \"\"\"\n Render pure chart, index is the order of chart, starting from 0.\n If you set `pre_filters`, it will overwritre global_pre_filters.\n \"\"\"\n cur_spec_obj = deepcopy(self.walker.vis_spec[index])\n key = f\"{self.walker.gid}-{key}-{index}\"\n\n if Version(self.walker.spec_version) > Version(\"0.3.11\"):\n chart_size_config = cur_spec_obj[\"layout\"][\"size\"]\n else:\n chart_size_config = cur_spec_obj[\"config\"][\"size\"]\n\n if pre_filters is None:\n pre_filters = self.global_pre_filters\n\n if pre_filters is not None:\n pre_filters_json = self._convert_pre_filters_to_gw_config(\n pre_filters, cur_spec_obj\n )\n cur_spec_obj[\"encodings\"][\"filters\"].extend(pre_filters_json)\n\n if size is not None:\n chart_size_config[\"mode\"] = \"fixed\"\n chart_size_config[\"width\"] = size[0]\n chart_size_config[\"height\"] = size[1]\n\n return self._component(key=key, mode=\"renderer\", vis_spec=[cur_spec_obj])\n\n @deprecated(\"render_pure_chart is deprecated, use chart instead.\")\n def render_pure_chart(self, *args, **kwargs):\n return self.chart(*args, **kwargs)\n\n def table(\n self,\n *,\n key: str = \"table\",\n ):\n \"\"\"Render pure table UI\"\"\"\n key = f\"{self.walker.gid}-{key}\"\n return self._component(key=key, mode=\"table\")\n\n def _component(\n self,\n *,\n key: str,\n mode: Literal[\"explore\", \"renderer\", \"filter_renderer\", \"table\"],\n vis_spec: Optional[List[Dict[str, Any]]] = None,\n **kwargs: Dict[str, Any]\n ):\n props = self.walker._get_props(\"streamlit\", [])\n props[\"gwMode\"] = mode\n props[\"communicationUrl\"] = BASE_URL_PATH\n if vis_spec is not None:\n props[\"visSpec\"] = vis_spec\n props.update(kwargs)\n\n component_value = pygwalker_component(props, key)\n return component_value\n\n\ndef get_streamlit_html(\n dataset: Union[DataFrame, Connector],\n gid: Union[int, str] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n use_kernel_calc: Optional[bool] = None,\n kernel_computation: Optional[bool] = None,\n show_cloud_tool: Optional[bool] = None,\n spec_io_mode: ISpecIOMode = \"r\",\n kanaries_api_key: str = \"\",\n mode: Literal[\"explore\", \"filter_renderer\", \"table\"] = \"explore\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n **kwargs\n) -> str:\n \"\"\"Get pygwalker html render to streamlit\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - use_kernel_calc(bool): Deprecated, use kernel_computation instead.\n - spec_io_mode (ISpecIOMode): spec io mode, Default to \"r\", \"r\" for read, \"rw\" for read and write.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n \"\"\"\n if field_specs is None:\n field_specs = []\n\n renderer = StreamlitRenderer(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n theme_key=theme_key,\n appearance=appearance,\n spec_io_mode=spec_io_mode,\n use_kernel_calc=use_kernel_calc,\n kernel_computation=kernel_computation,\n show_cloud_tool=show_cloud_tool,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n **kwargs\n )\n\n return renderer._get_html(mode=mode)\n" }, { "path": "pygwalker/api/webserver.py", "content": "from typing import Union, List, Optional\nimport threading\nimport socketserver\nimport http.server\nimport urllib.parse\nimport json\nimport webbrowser\nimport time\n\nfrom typing_extensions import Literal\n\nfrom .pygwalker import PygWalker\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame, IAppearance, IThemeKey\nfrom pygwalker.utils.check_walker_params import check_expired_params\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker.utils.free_port import find_free_port\nfrom pygwalker.communications.base import BaseCommunication\n\n_MAX_HEALTH_TIMEOUT_SECONDS = 3\n_SEND_HEALTH_JS_SCRIPT = \"\"\"\n\n\"\"\"\n\n\nclass _GlobalState:\n def __init__(self, auto_shutdown: bool):\n # why +60? If page not auto open, user manually open page, we should give user more time to interact with page.\n self.last_health_time = time.time() + 60\n self.auto_shutdown = auto_shutdown\n\n\nclass _PygWalkerHandler(http.server.SimpleHTTPRequestHandler):\n _walker: PygWalker\n _state: _GlobalState\n\n def do_GET(self):\n parsed_path = urllib.parse.urlparse(self.path)\n path = parsed_path.path\n\n if path == \"/health\":\n self._state.last_health_time = time.time()\n self.send_response(200)\n self.end_headers()\n self.wfile.write(b\"OK\")\n return\n\n props = self._walker._get_props(\"web_server\")\n props[\"communicationUrl\"] = \"comm\"\n\n html = self._walker._get_render_iframe(props)\n if self._state.auto_shutdown:\n html += _SEND_HEALTH_JS_SCRIPT\n\n self.send_response(200)\n self.send_header(\"Content-type\", \"text/html\")\n self.end_headers()\n self.wfile.write(html.encode(\"utf-8\"))\n \n def do_POST(self):\n parsed_path = urllib.parse.urlparse(self.path)\n path = parsed_path.path\n\n if not path.startswith(\"/comm\"):\n self.send_error(404)\n return\n\n content_length = int(self.headers['Content-Length'])\n post_data = self.rfile.read(content_length).decode('utf-8')\n payload = json.loads(post_data)\n result = self._walker.comm._receive_msg(payload[\"action\"], payload[\"data\"])\n self.send_response(200)\n self.send_header(\"Content-type\", \"application/json\")\n self.end_headers()\n self.wfile.write(json.dumps(result, cls=DataFrameEncoder).encode(\"utf-8\"))\n \n def log_message(self, format, *args):\n pass\n\n\nclass CustomTCPServer(socketserver.TCPServer):\n def log_request(self, *args, **kwargs):\n pass\n\n\ndef _create_handler_with_walker(walker: PygWalker, state: _GlobalState):\n \"\"\"Create a custom handler with walker\"\"\"\n class CustomPygWalkerHandler(_PygWalkerHandler):\n _walker = walker\n _state = state\n \n return CustomPygWalkerHandler\n\n\ndef _open_browser(address: str, delay_ms: int = 1000):\n \"\"\"Open browser with address\"\"\"\n time.sleep(delay_ms / 1000)\n\n try:\n opened = webbrowser.open(address)\n except Exception:\n opened = False\n \n if opened:\n print(f\"Run pygwalker at {address}, close page or press Ctrl+C to end.\")\n else:\n print(f\"Auto open browser failed, please open {address} manually, close page or press Ctrl+C to end.\")\n\n\ndef _start_server(\n walker: PygWalker,\n port: Optional[int],\n *,\n auto_open: bool,\n auto_shutdown: bool,\n):\n \"\"\"Start a server with walker\"\"\"\n state = _GlobalState(auto_shutdown=auto_shutdown)\n walker._init_callback(BaseCommunication(str(walker.gid)))\n\n handler = _create_handler_with_walker(walker, state)\n if port is None:\n port = find_free_port()\n address = f\"http://localhost:{port}\"\n\n def _listen_shutdown():\n while 1:\n time.sleep(1)\n if time.time() - state.last_health_time > _MAX_HEALTH_TIMEOUT_SECONDS:\n httpd.shutdown()\n break\n\n try:\n with CustomTCPServer((\"127.0.0.1\", port), handler) as httpd:\n if auto_open:\n threading.Thread(target=_open_browser, args=(address,)).start()\n if auto_shutdown:\n threading.Thread(target=_listen_shutdown).start()\n httpd.serve_forever()\n except KeyboardInterrupt:\n pass\n\n\ndef walk(\n dataset: Union[DataFrame, Connector, str],\n gid: Optional[Union[int, str]] = None,\n *,\n field_specs: Optional[List[FieldSpec]] = None,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n spec: str = \"\",\n kernel_computation: Optional[bool] = None,\n cloud_computation: bool = False,\n show_cloud_tool: bool = True,\n kanaries_api_key: str = \"\",\n default_tab: Literal[\"data\", \"vis\"] = \"vis\",\n port: Optional[int] = None,\n auto_shutdown: bool = False,\n auto_open: bool = False,\n **kwargs\n):\n \"\"\"Walk through pandas.DataFrame df with Graphic Walker.\n This function was originally designed solely to launch Pygwalker in script mode.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - gid (Union[int, str], optional): GraphicWalker container div's id ('gwalker-{gid}')\n\n Kargs:\n - field_specs (List[FieldSpec], optional): Specifications of some fields. They'll been automatically inferred from `df` if some fields are not specified.\n - theme_key ('vega' | 'g2' | 'streamlit'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - spec (str): chart config data. config id, json, remote file url\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None, automatically determine whether to use kernel calculation.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - default_tab (Literal[\"data\", \"vis\"]): default tab to show. Default to \"vis\"\n - cloud_computation(bool): Whether to use cloud compute for datas, it upload your data to kanaries cloud. Default to False.\n - port (int): port to use for the server. Default to None, which means a random port will be used.\n - auto_shutdown (bool): Whether to shutdown the server when the page is closed. Default to False.\n - auto_open (bool): Whether to open the browser automatically. Default to False.\n \"\"\"\n check_expired_params(kwargs)\n\n if field_specs is None:\n field_specs = []\n\n walker = PygWalker(\n gid=gid,\n dataset=dataset,\n field_specs=field_specs,\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=show_cloud_tool,\n use_preview=False,\n kernel_computation=kernel_computation,\n use_save_tool=True,\n gw_mode=\"explore\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=default_tab,\n cloud_computation=cloud_computation,\n **kwargs\n )\n _start_server(walker, port, auto_open=auto_open, auto_shutdown=auto_shutdown)\n\n\ndef render(\n dataset: Union[DataFrame, Connector, str],\n spec: str,\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n port: Optional[int] = None,\n auto_shutdown: bool = False,\n auto_open: bool = False,\n **kwargs\n):\n \"\"\"\n This function was originally designed solely to launch Pygwalker in script mode.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n - spec (str): chart config data. config id, json, remote file url\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - port (int): port to use for the server. Default to None, which means a random port will be used.\n - auto_shutdown (bool): Whether to shutdown the server when the page is closed. Default to False.\n - auto_open (bool): Whether to open the browser automatically. Default to False.\n \"\"\"\n\n walker = PygWalker(\n gid=None,\n dataset=dataset,\n field_specs=[],\n spec=spec,\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, (Connector, str)) or kernel_computation,\n use_save_tool=False,\n gw_mode=\"filter_renderer\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=\"vis\",\n cloud_computation=False,\n **kwargs\n )\n _start_server(walker, port, auto_open=auto_open, auto_shutdown=auto_shutdown)\n\n\ndef table(\n dataset: Union[DataFrame, Connector, str],\n *,\n theme_key: IThemeKey = 'g2',\n appearance: IAppearance = 'media',\n kernel_computation: Optional[bool] = None,\n kanaries_api_key: str = \"\",\n port: Optional[int] = None,\n auto_shutdown: bool = False,\n auto_open: bool = False,\n **kwargs\n):\n \"\"\"\n This function was originally designed solely to launch Pygwalker in script mode.\n\n Args:\n - dataset (pl.DataFrame | pd.DataFrame | Connector, optional): dataframe.\n\n Kargs:\n - theme_key ('vega' | 'g2'): theme type.\n - appearance (Literal['media' | 'light' | 'dark']): 'media': auto detect OS theme.\n - kernel_computation(bool): Whether to use kernel compute for datas, Default to None.\n - kanaries_api_key (str): kanaries api key, Default to \"\".\n - port (int): port to use for the server. Default to None, which means a random port will be used.\n - auto_shutdown (bool): Whether to shutdown the server when the page is closed. Default to False.\n - auto_open (bool): Whether to open the browser automatically. Default to False.\n \"\"\"\n walker = PygWalker(\n gid=None,\n dataset=dataset,\n field_specs=[],\n spec=\"\",\n source_invoke_code=\"\",\n theme_key=theme_key,\n appearance=appearance,\n show_cloud_tool=False,\n use_preview=False,\n kernel_computation=isinstance(dataset, (Connector, str)) or kernel_computation,\n use_save_tool=False,\n gw_mode=\"table\",\n is_export_dataframe=True,\n kanaries_api_key=kanaries_api_key,\n default_tab=\"vis\",\n cloud_computation=False,\n **kwargs\n )\n _start_server(walker, port, auto_open=auto_open, auto_shutdown=auto_shutdown)\n" }, { "path": "pygwalker/communications/__init__.py", "content": "" }, { "path": "pygwalker/communications/anywidget_comm.py", "content": "from typing import Any, Dict, Optional, List\nimport uuid\nimport json\n\nimport anywidget\n\nfrom .base import BaseCommunication\nfrom pygwalker.utils.encode import DataFrameEncoder\n\n\nclass AnywidgetCommunication(BaseCommunication):\n \"\"\"communication class for anywidget\"\"\"\n def register_widget(self, widget: anywidget.AnyWidget) -> None:\n \"\"\"register widget\"\"\"\n self.widget = widget\n self.widget.on_msg(self._on_mesage)\n\n def send_msg_async(self, action: str, data: Dict[str, Any], rid: Optional[str] = None):\n \"\"\"send message base on anywidget\"\"\"\n if rid is None:\n rid = uuid.uuid1().hex\n msg = {\n \"gid\": self.gid,\n \"rid\": rid,\n \"action\": action,\n \"data\": data\n }\n self.widget.send({\"type\": \"pyg_response\", \"data\": json.dumps(msg, cls=DataFrameEncoder)})\n\n def _on_mesage(self, _: anywidget.AnyWidget, data: Dict[str, Any], buffers: List[Any]):\n if data.get(\"type\", \"\") != \"pyg_request\":\n return\n \n msg = data[\"msg\"]\n action = msg[\"action\"]\n rid = msg[\"rid\"]\n\n if action == \"finish_request\":\n return\n\n resp = self._receive_msg(action, msg[\"data\"])\n self.send_msg_async(\"finish_request\", resp, rid)\n" }, { "path": "pygwalker/communications/base.py", "content": "from typing import Any, Callable, Dict\n\nfrom pygwalker.errors import BaseError, ErrorCode\nfrom pygwalker.services.track import track_event\n\n\ndef _upload_error_info(gid: str, action: str, error: Exception):\n try:\n track_event(\"pygwalker_error\", {\n \"gid\": gid,\n \"action\": action,\n \"error\": str(error),\n \"error_type\": type(error).__name__\n })\n except Exception:\n pass\n\n\nclass BaseCommunication:\n \"\"\"\n Base class for communication\n message format:\n {\n \"rid\": \"xxx\",\n \"action\": \"xxx\",\n \"data\": {}\n }\n \"\"\"\n def __init__(self, gid: str) -> None:\n self._endpoint_map = {}\n self.gid = gid\n\n def send_msg_async(self, action: str, data: Dict[str, Any]):\n raise NotImplementedError\n\n def _receive_msg(self, action: str, data: Dict[str, Any]) -> Dict[str, Any]:\n handler_func = self._endpoint_map.get(action, None)\n if handler_func is None:\n return {\"code\": ErrorCode.UNKNOWN_ERROR, \"data\": None, \"message\": f\"Unknown action: {action}\"}\n try:\n data = handler_func(data)\n return {\"code\": 0, \"data\": data, \"message\": \"success\"}\n except BaseError as e:\n _upload_error_info(self.gid, action, e)\n return {\"code\": e.code, \"data\": data, \"message\": str(e)}\n except Exception as e:\n _upload_error_info(self.gid, action, e)\n return {\"code\": ErrorCode.UNKNOWN_ERROR, \"data\": data, \"message\": str(e)}\n\n def register(self, endpoint: str, func: Callable[[Dict[str, Any]], Any]):\n self._endpoint_map[endpoint] = func\n" }, { "path": "pygwalker/communications/gradio_comm.py", "content": "import json\nimport gc\n\nfrom fastapi import FastAPI\nfrom starlette.routing import Route\nfrom starlette.responses import JSONResponse, Response\nfrom starlette.requests import Request\n\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom .base import BaseCommunication\n\ngradio_comm_map = {}\n\nBASE_URL_PATH = \"/_pygwalker/comm/\".strip(\"/\")\n\n\nasync def _pygwalker_router(req: Request) -> Response:\n gid = req.path_params[\"gid\"]\n comm_obj = gradio_comm_map.get(gid, None)\n if comm_obj is None:\n return JSONResponse({\"success\": False, \"message\": f\"Unknown gid: {gid}\"})\n json_data = await req.json()\n\n # pylint: disable=protected-access\n result = comm_obj._receive_msg(json_data[\"action\"], json_data[\"data\"])\n # pylint: enable=protected-access\n\n result = json.dumps(result, cls=DataFrameEncoder)\n return JSONResponse(json.loads(result))\n\n\nclass GradioCommunication(BaseCommunication):\n \"\"\"\n Hacker streamlit communication class.\n only support receive message.\n \"\"\"\n def __init__(self, gid: str) -> None:\n super().__init__(gid)\n gradio_comm_map[gid] = self\n\n\nPYGWALKER_ROUTE = Route(\n \"/_pygwalker/comm/{gid}\",\n _pygwalker_router,\n methods=[\"POST\"]\n)\n\n\n# it will work when gradio server reload\ndef _hack_gradio_server():\n for obj in gc.get_objects():\n if isinstance(obj, FastAPI):\n for index, route in enumerate(obj.routes):\n if route.path == \"/_pygwalker/comm/{gid}\":\n obj.routes[index] = PYGWALKER_ROUTE\n return\n\n\n_hack_gradio_server()\n" }, { "path": "pygwalker/communications/hacker_comm.py", "content": "from threading import Lock\nfrom typing import Any, Dict, Optional, List\nimport uuid\nimport json\nimport time\n\nfrom ipywidgets import Text, Layout, Box\n\nfrom .base import BaseCommunication\nfrom pygwalker.utils.encode import DataFrameEncoder\n\n\nclass HackerCommunication(BaseCommunication):\n \"\"\"\n Hacker communication class.\n Since it is not a long running service for multiple users,\n some expired buffers and locks will not be cleaned up.\n \"\"\"\n def __init__(self, gid: str) -> None:\n super().__init__(gid)\n self._kernel_widget = self._get_kernel_widget()\n self._html_widget = self._get_html_widget()\n self._send_msg_lock = Lock()\n self.__increase = 0\n\n def send_msg_async(self, action: str, data: Dict[str, Any], rid: Optional[str] = None):\n \"\"\"\n To transmit messages through a widget,\n there will be problems during concurrency,\n because the timing of front-end rendering is not sure,\n so a sleep is temporarily added to solve it violently\n \"\"\"\n if rid is None:\n rid = uuid.uuid1().hex\n msg = {\n \"gid\": self.gid,\n \"rid\": rid,\n \"action\": action,\n \"data\": data\n }\n with self._send_msg_lock:\n self._html_widget.value = json.dumps(msg, cls=DataFrameEncoder)\n self._html_widget.placeholder = str(self.__increase)\n self.__increase += 1\n time.sleep(0.1)\n\n def get_widgets(self) -> Box:\n return Box(\n children=[self._html_widget, *self._kernel_widget],\n layout=Layout(display=\"none\")\n )\n\n def _on_mesage(self, info: Dict[str, Any]):\n self.__increase += 1\n msg_json = json.loads(info[\"new\"])\n action = msg_json[\"action\"]\n data = msg_json[\"data\"]\n rid = msg_json[\"rid\"]\n\n if action == \"finish_request\":\n return\n\n resp = self._receive_msg(action, data)\n self.send_msg_async(\"finish_request\", resp, rid)\n\n def _get_html_widget(self) -> Text:\n text = Text(value=\"\", placeholder=\"\")\n text.add_class(f\"hacker-comm-pyg-html-store-{self.gid}\")\n return text\n\n def _get_kernel_widget(self) -> List[Text]:\n text_list = []\n for index in range(5):\n text = Text(value=\"\", placeholder=\"\")\n text.add_class(f\"hacker-comm-pyg-kernel-store-{self.gid}-{index}\")\n text.observe(self._on_mesage, \"value\")\n text_list.append(text)\n return text_list\n" }, { "path": "pygwalker/communications/reflex_comm.py", "content": "import json\nfrom fastapi import FastAPI, HTTPException\nfrom starlette.routing import Route\nfrom starlette.responses import JSONResponse, Response\nfrom starlette.requests import Request\n\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom .base import BaseCommunication\n\nreflex_comm_map = {}\n\n# Fixed: Consistent path with leading slash to match mount point\nBASE_URL_PATH = \"/_pygwalker/comm\"\n\n\nasync def _pygwalker_router(req: Request) -> Response:\n \"\"\"Legacy router function - kept for compatibility.\"\"\"\n gid = req.path_params[\"gid\"]\n comm_obj = reflex_comm_map.get(gid, None)\n if comm_obj is None:\n return JSONResponse({\"success\": False, \"message\": f\"Unknown gid: {gid}\"})\n \n try:\n json_data = await req.json()\n \n # Fixed: Input validation - check for required keys\n if \"action\" not in json_data:\n return JSONResponse({\"success\": False, \"message\": \"Missing 'action' field in request\"})\n if \"data\" not in json_data:\n return JSONResponse({\"success\": False, \"message\": \"Missing 'data' field in request\"})\n \n result = comm_obj._receive_msg(json_data[\"action\"], json_data[\"data\"])\n \n # Fixed: Proper JSON encoding with DataFrameEncoder\n encoded_result = json.loads(json.dumps(result, cls=DataFrameEncoder))\n return JSONResponse(encoded_result)\n \n except json.JSONDecodeError:\n return JSONResponse({\"success\": False, \"message\": \"Invalid JSON in request body\"})\n except Exception as e:\n return JSONResponse({\"success\": False, \"message\": f\"Internal error: {str(e)}\"})\n\n\nclass ReflexCommunication(BaseCommunication):\n \"\"\"Communication class for Reflex.\"\"\"\n\n def __init__(self, gid: str) -> None:\n super().__init__(gid)\n reflex_comm_map[gid] = self\n\n\n# Create a FastAPI sub-application for PyGWalker API\ndef _create_pygwalker_app() -> FastAPI:\n \"\"\"Create a FastAPI sub-application for PyGWalker API routes.\"\"\"\n pygwalker_app = FastAPI()\n \n @pygwalker_app.post(\"/{gid}\")\n async def pygwalker_endpoint(gid: str, request: Request) -> Response:\n \"\"\"PyGWalker communication endpoint with proper error handling.\"\"\"\n # Get the communication object for this gid\n comm_obj = reflex_comm_map.get(gid, None)\n if comm_obj is None:\n return JSONResponse({\"success\": False, \"message\": f\"Unknown gid: {gid}\"})\n \n try:\n # Process the request with validation\n json_data = await request.json()\n \n # Fixed: Input validation - check for required keys\n if \"action\" not in json_data:\n return JSONResponse({\"success\": False, \"message\": \"Missing 'action' field in request\"})\n if \"data\" not in json_data:\n return JSONResponse({\"success\": False, \"message\": \"Missing 'data' field in request\"})\n \n result = comm_obj._receive_msg(json_data[\"action\"], json_data[\"data\"])\n \n # Fixed: Proper JSON encoding with DataFrameEncoder\n encoded_result = json.loads(json.dumps(result, cls=DataFrameEncoder))\n return JSONResponse(encoded_result)\n \n except json.JSONDecodeError:\n return JSONResponse({\"success\": False, \"message\": \"Invalid JSON in request body\"})\n except Exception as e:\n return JSONResponse({\"success\": False, \"message\": f\"Internal error: {str(e)}\"})\n \n return pygwalker_app\n\n\ndef register_pygwalker_api(app: FastAPI) -> FastAPI:\n \"\"\"Register pygwalker API route into Reflex app.\"\"\"\n # Create a sub-application for PyGWalker routes\n pygwalker_app = _create_pygwalker_app()\n \n # Fixed: Use consistent path (BASE_URL_PATH already has leading slash)\n app.mount(BASE_URL_PATH, pygwalker_app)\n \n return app\n" }, { "path": "pygwalker/communications/streamlit_comm.py", "content": "import gc\nimport json\n\nfrom tornado.web import Application\nfrom streamlit import config\nfrom streamlit.web.server.server_util import make_url_path_regex\nimport tornado.web\nimport streamlit as st\n\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker.errors import StreamlitPygwalkerApiError\nfrom .base import BaseCommunication\n\nstreamlit_comm_map = {}\n\n_STREAMLIT_PREFIX_URL = config.get_option(\"server.baseUrlPath\").strip(\"/\")\nBASE_URL_PATH = \"/_stcore/_pygwalker/comm/\".strip(\"/\")\nPYGWALKER_API_PATH = make_url_path_regex(\n _STREAMLIT_PREFIX_URL,\n r\"/_stcore/_pygwalker/comm/(.+)\"\n)\n\n\nclass PygwalkerHandler(tornado.web.RequestHandler):\n \"\"\"\n Handler for pygwalker communication\n \"\"\"\n def check_xsrf_cookie(self):\n return True\n\n def post(self, gid: str):\n comm_obj = streamlit_comm_map.get(gid, None)\n if comm_obj is None:\n self.write({\"success\": False, \"message\": f\"Unknown gid: {gid}\"})\n return\n json_data = json.loads(self.request.body)\n\n # pylint: disable=protected-access\n result = comm_obj._receive_msg(json_data[\"action\"], json_data[\"data\"])\n # pylint: enable=protected-access\n\n self.write(json.dumps(result, cls=DataFrameEncoder))\n return\n\n\n@st.cache_resource\ndef hack_streamlit_server():\n tornado_obj = None\n for obj in gc.get_objects():\n try:\n if isinstance(obj, Application) and any((\"streamlit\" in str(rule.target) for rule in obj.wildcard_router.rules)):\n tornado_obj = obj\n except Exception:\n pass\n\n if tornado_obj:\n tornado_obj.add_handlers(\".*\", [(PYGWALKER_API_PATH, PygwalkerHandler)])\n else:\n raise StreamlitPygwalkerApiError()\n\n\nclass StreamlitCommunication(BaseCommunication):\n \"\"\"\n Hacker streamlit communication class.\n only support receive message.\n \"\"\"\n def __init__(self, gid: str) -> None:\n super().__init__(gid)\n streamlit_comm_map[gid] = self\n" }, { "path": "pygwalker/data_parsers/__init__.py", "content": "" }, { "path": "pygwalker/data_parsers/base.py", "content": "from typing import Generic, Dict, List, Any, Optional\nfrom typing_extensions import Literal\nfrom functools import lru_cache\nfrom datetime import datetime, date\nfrom datetime import timedelta\nimport abc\nimport io\n\nfrom pydantic import BaseModel\nimport duckdb\nimport arrow\nimport pytz\n\nfrom pygwalker._typing import DataFrame\nfrom pygwalker.utils.payload_to_sql import get_sql_from_payload\nfrom pygwalker.utils.estimate_tools import estimate_average_data_size\n\n\n# pylint: disable=broad-except\nclass FieldSpec(BaseModel):\n \"\"\"Field specification.\n\n Args:\n - fname: str. The field name.\n - semantic_type: '?' | 'nominal' | 'ordinal' | 'temporal' | 'quantitative'. default to '?'.\n - analytic_type: '?' | 'dimension' | 'measure'. default to '?'.\n - display_as: str. The field name displayed. None means using the original column name.\n \"\"\"\n fname: str\n semantic_type: Literal['?', 'nominal', 'ordinal', 'temporal', 'quantitative'] = '?'\n analytic_type: Literal['?', 'dimension', 'measure'] = '?'\n display_as: str = None\n\n\nINFINITY_DATA_SIZE = 1 << 62\n\n\nclass BaseDataParser(abc.ABC):\n \"\"\"Base class for data parser\"\"\"\n\n @abc.abstractmethod\n def __init__(\n self,\n data: Any,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ) -> None:\n raise NotImplementedError\n\n @property\n @abc.abstractmethod\n def raw_fields(self) -> List[Dict[str, str]]:\n \"\"\"get raw fields\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n \"\"\"get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n \"\"\"get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def get_datas_by_payload(self, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n \"\"\"get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def batch_get_datas_by_sql(self, sql_list: List[str]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def batch_get_datas_by_payload(self, payload_list: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def to_csv(self) -> io.BytesIO:\n \"\"\"get records\"\"\"\n raise NotImplementedError\n\n @abc.abstractmethod\n def to_parquet(self) -> io.BytesIO:\n \"\"\"get records\"\"\"\n raise NotImplementedError\n\n @property\n @abc.abstractmethod\n def dataset_type(self) -> str:\n \"\"\"get dataset type\"\"\"\n raise NotImplementedError\n\n @property\n @abc.abstractmethod\n def field_metas(self) -> List[Dict[str, str]]:\n \"\"\"get field metas\"\"\"\n raise NotImplementedError\n\n @property\n @abc.abstractmethod\n def placeholder_table_name(self) -> str:\n \"\"\"get placeholder table name\"\"\"\n raise NotImplementedError\n\n @property\n @abc.abstractmethod\n def data_size(self) -> int:\n \"\"\"Estimate data bytes size\"\"\"\n raise NotImplementedError\n\n\nclass BaseDataFrameDataParser(Generic[DataFrame], BaseDataParser):\n \"\"\"DataFrame property getter\"\"\"\n def __init__(\n self, df: DataFrame,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ):\n self.origin_df = df\n self.df = self._rename_dataframe(df)\n self._example_df = self.df[:1000]\n self.field_specs = field_specs\n self._duckdb_df = self.df\n self.infer_string_to_date = infer_string_to_date\n self.infer_number_to_dimension = infer_number_to_dimension\n self.other_params = other_params\n\n @property\n @lru_cache()\n def field_metas(self) -> List[Dict[str, str]]:\n duckdb.register(\"pygwalker_mid_table\", self._duckdb_df)\n result = duckdb.query(\"SELECT * FROM pygwalker_mid_table LIMIT 1\")\n data = result.fetchone()\n return get_data_meta_type(dict(zip(result.columns, data))) if data else []\n\n @property\n @lru_cache()\n def raw_fields(self) -> List[Dict[str, str]]:\n return [\n self._infer_prop(col, self.field_specs)\n for _, col in enumerate(self._example_df.columns)\n ]\n\n def _infer_prop(\n self, col: str, field_specs: List[FieldSpec] = None\n ) -> Dict[str, str]:\n \"\"\"get IMutField\n\n Returns:\n (IMutField, Dict)\n \"\"\"\n s = self._example_df[col]\n orig_fname = col\n\n field_spec_map = {field_spec.fname: field_spec for field_spec in field_specs}\n\n field_spec = field_spec_map.get(orig_fname, FieldSpec(fname=orig_fname))\n semantic_type = self._infer_semantic(s, orig_fname) if field_spec.semantic_type == '?' else field_spec.semantic_type\n analytic_type = self._infer_analytic(s, orig_fname) if field_spec.analytic_type == '?' else field_spec.analytic_type\n fname = orig_fname if field_spec.display_as is None else field_spec.display_as\n return {\n 'fid': col,\n 'name': fname,\n 'semanticType': semantic_type,\n 'analyticType': analytic_type,\n }\n\n def get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n \"\"\"get datas by duckdb\"\"\"\n try:\n duckdb.query(\"SET TimeZone = 'UTC'\")\n except Exception:\n pass\n\n duckdb.register(\"pygwalker_mid_table\", self._duckdb_df)\n\n result = duckdb.query(sql)\n return [\n dict(zip(result.columns, row))\n for row in result.fetchall()\n ]\n\n def _rename_dataframe(self, df: DataFrame) -> DataFrame:\n \"\"\"rename dataframe\"\"\"\n raise NotImplementedError\n\n def get_datas_by_payload(self, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n sql = get_sql_from_payload(\n \"pygwalker_mid_table\",\n payload,\n {\"pygwalker_mid_table\": self.field_metas}\n )\n return self.get_datas_by_sql(sql)\n\n def batch_get_datas_by_sql(self, sql_list: List[str]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_sql(sql)\n for sql in sql_list\n ]\n\n def batch_get_datas_by_payload(self, payload_list: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_payload(payload)\n for payload in payload_list\n ]\n\n @property\n def dataset_type(self) -> str:\n return \"dataframe_default\"\n\n @property\n def placeholder_table_name(self) -> str:\n return \"pygwalker_mid_table\"\n\n @property\n def data_size(self) -> int:\n datas = self.to_records(300)\n avg_size = estimate_average_data_size(datas)\n return avg_size * self.df.shape[0]\n\n\ndef is_temporal_field(value: Any, infer_string_to_date: bool) -> bool:\n \"\"\"check if field is temporal\"\"\"\n if infer_string_to_date:\n try:\n arrow.get(str(value))\n except Exception:\n return False\n return True\n\n return isinstance(value, (datetime, date))\n\n\ndef is_geo_field(field_name: str) -> bool:\n \"\"\"check if filed is \"\"\"\n field_name = field_name.lower().strip(\" .\")\n return field_name in {\n \"latitude\", \"longitude\",\n \"lat\", \"long\", \"lon\"\n }\n\n\ndef format_temporal_string(value: str) -> str:\n \"\"\"Convert temporal fields to a fixed format\"\"\"\n return arrow.get(value).strftime(\"%Y-%m-%d %H:%M:%S\")\n\n\ndef get_data_meta_type(data: Dict[str, Any]) -> List[Dict[str, str]]:\n meta_types = []\n for key, value in data.items():\n if isinstance(value, datetime):\n field_meta_type = \"datetime\"\n if value.tzinfo:\n field_meta_type = \"datetime_tz\"\n elif isinstance(value, (int, float)):\n field_meta_type = \"number\"\n else:\n field_meta_type = \"string\"\n meta_types.append({\n \"key\": key,\n \"type\": field_meta_type\n })\n return meta_types\n\n\n@lru_cache()\ndef get_timezone_base_offset(offset_seconds: int) -> Optional[str]:\n utc_offset = timedelta(seconds=offset_seconds)\n now = datetime.now(pytz.utc)\n for tz in map(pytz.timezone, pytz.all_timezones_set):\n if now.astimezone(tz).utcoffset() == utc_offset:\n return tz.zone\n" }, { "path": "pygwalker/data_parsers/cloud_dataset_parser.py", "content": "from typing import Any, Dict, List, Optional\nfrom functools import lru_cache\nfrom decimal import Decimal\nimport logging\nimport io\n\nimport pandas as pd\n\nfrom .base import BaseDataParser, get_data_meta_type, INFINITY_DATA_SIZE\nfrom .pandas_parser import PandasDataFrameDataParser\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.services.cloud_service import CloudService\n\nlogger = logging.getLogger(__name__)\n\n\nclass CloudDatasetParser(BaseDataParser):\n \"\"\"data parser for database\"\"\"\n def __init__(\n self,\n dataset_id: str,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ):\n self.dataset_id = dataset_id\n self.field_specs = field_specs\n self.infer_string_to_date = infer_string_to_date\n self.infer_number_to_dimension = infer_number_to_dimension\n self.other_params = other_params\n self._cloud_service = CloudService(other_params.get(\"kanaries_api_key\", \"\"))\n self.example_pandas_df = self._get_example_pandas_df()\n\n def _get_example_pandas_df(self) -> pd.DataFrame:\n datas = self._get_all_datas(1000)\n example_df = pd.DataFrame(datas)\n for column in example_df.columns:\n if any(isinstance(val, Decimal) for val in example_df[column]):\n example_df[column] = example_df[column].astype(float)\n return example_df\n\n @property\n @lru_cache()\n def field_metas(self) -> List[Dict[str, str]]:\n data = self._get_all_datas(1)\n return get_data_meta_type(data[0]) if data else []\n\n @property\n @lru_cache()\n def raw_fields(self) -> List[Dict[str, str]]:\n pandas_parser = PandasDataFrameDataParser(\n self.example_pandas_df,\n self.field_specs,\n self.infer_string_to_date,\n self.infer_number_to_dimension,\n self.other_params\n )\n return [\n {**field, \"fid\": field[\"name\"]}\n for field in pandas_parser.raw_fields\n ]\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n if limit is None:\n df = self.example_pandas_df\n else:\n df = self.example_pandas_df[:limit]\n df = df.replace({float('nan'): None})\n return df.to_dict(orient='records')\n\n def get_datas_by_payload(self, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n result = self._cloud_service.query_from_dataset(self.dataset_id, payload)\n return result\n\n def get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n pass\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.example_pandas_df.toPandas().to_csv(content, index=False)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.example_pandas_df.toPandas().to_parquet(content, index=False, compression=\"snappy\")\n return content\n\n def _get_all_datas(self, limit: int) -> List[Dict[str, Any]]:\n payload = {\"workflow\": [{\"type\": \"view\", \"query\": [{\"op\": \"raw\", \"fields\": [\"*\"]}]}], \"limit\": limit, \"offset\": 0}\n return self.get_datas_by_payload(payload)\n\n def batch_get_datas_by_sql(self, sql_list: List[str]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n pass\n\n def batch_get_datas_by_payload(self, payload_list: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n result = self._cloud_service.batch_query_from_dataset(self.dataset_id, payload_list)\n return [\n item[\"rows\"]\n for item in result\n ]\n\n @property\n def dataset_type(self) -> str:\n return \"cloud_dataset\"\n\n @property\n def placeholder_table_name(self) -> str:\n return \"pygwalker_mid_table\"\n\n @property\n def data_size(self) -> int:\n return INFINITY_DATA_SIZE\n" }, { "path": "pygwalker/data_parsers/database_parser.py", "content": "from typing import Any, Dict, List, Optional\nfrom functools import lru_cache\nfrom decimal import Decimal\nimport logging\nimport json\nimport io\n\nfrom sqlalchemy import create_engine, text\nfrom sqlalchemy.engine import Engine, Connection\nimport pandas as pd\nimport sqlglot.expressions as exp\nimport sqlglot\n\nfrom .base import BaseDataParser, get_data_meta_type, INFINITY_DATA_SIZE\nfrom .pandas_parser import PandasDataFrameDataParser\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.utils.custom_sqlglot import DuckdbDialect\nfrom pygwalker.utils.payload_to_sql import get_sql_from_payload\nfrom pygwalker.errors import ViewSqlSameColumnError\n\nlogger = logging.getLogger(__name__)\n\n\ndef _check_view_sql(sql: str) -> None:\n \"\"\"check view sql, it will raise ViewSqlSameColumnError if view sql contain same column\"\"\"\n select_columns = [\n select.alias_or_name\n for select in sqlglot.parse_one(sql).find(exp.Select)\n ]\n\n has_join = sqlglot.parse_one(sql).find(exp.Join) is not None\n has_select_all = any(column == \"*\" for column in select_columns)\n select_expr_count = len(select_columns)\n hash_same_column = len(set(select_columns)) != select_expr_count\n\n if has_select_all and select_expr_count > 1:\n raise ViewSqlSameColumnError(\"fields with the same name may appear when use select * and select other fields\")\n if has_join and has_select_all:\n raise ViewSqlSameColumnError(\"fields with the same name may appear when multi table join and use select *\")\n if hash_same_column:\n raise ViewSqlSameColumnError(\"view sql can not contain same column\")\n\n\nclass Connector:\n \"\"\"\n database connector, it will cache engine by url.\n\n - url: database url, refer to sqlalchemy doc for url. example: mysql+pymysql://user:password@host:port/database\n - view_sql: view sql, example: SELECT * FROM table_name\n - engine_params: engine params, refer to sqlalchemy doc for params. example: {\"pool_size\": 10}\n \"\"\"\n engine_map = {}\n JSON_TYPE_CODE_SET_MAP = {\n \"snowflake\": {9, 10},\n \"mysql\": {245}\n }\n PRE_INIT_SQL_MAP = {\n \"snowflake\": \"ALTER SESSION SET WEEK_OF_YEAR_POLICY=1, WEEK_START=7, STRICT_JSON_OUTPUT=True;\",\n }\n\n def __init__(self, url: str, view_sql: str, engine_params: Optional[Dict[str, Any]] = None) -> \"Connector\":\n if engine_params is None:\n engine_params = {}\n\n self._init_instance(self._get_or_create_engine(url, engine_params), view_sql)\n\n @classmethod\n def from_sqlalchemy_engine(cls, engine: Engine, view_sql: str) -> \"Connector\":\n \"\"\"Create connector from engine\"\"\"\n instance = cls.__new__(cls)\n instance._init_instance(engine, view_sql)\n return instance\n\n @classmethod\n def from_sqlalchemy_connection(cls, connection: Connection, view_sql: str) -> \"Connector\":\n \"\"\"\n Create a Connector instance from an existing SQLAlchemy connection. \n This adapts the DuckDB connector.\n\n Note:\n - All subsequent queries will use the same connection.\n - The caller is responsible for managing and closing the connection when no longer needed.\n \"\"\"\n instance = cls.__new__(cls)\n instance._init_instance(connection.engine, view_sql)\n instance._existing_conn = connection\n return instance\n\n def _init_instance(self, engine: Engine, view_sql: str):\n _check_view_sql(view_sql)\n self.engine = engine\n self.url = str(engine.url)\n self.view_sql = view_sql\n self._json_type_code_set = self.JSON_TYPE_CODE_SET_MAP.get(self.dialect_name, set())\n self._existing_conn = None \n self._run_pre_init_sql(engine)\n\n def _get_or_create_engine(self, url: str, engine_params: Dict[str, Any]) -> Engine:\n if url not in self.engine_map:\n engine = create_engine(url, **engine_params)\n engine.dialect.requires_name_normalize = False\n self.engine_map[url] = engine\n\n return self.engine_map[url]\n\n def _run_pre_init_sql(self, engine: Engine) -> None:\n if engine.dialect.name in self.PRE_INIT_SQL_MAP:\n pre_init_sql = self.PRE_INIT_SQL_MAP[engine.dialect.name]\n with engine.connect(True) as connection:\n connection.execute(text(pre_init_sql))\n\n def query_datas(self, sql: str) -> List[Dict[str, Any]]:\n field_type_map = {}\n should_close_connection = self._existing_conn is None\n connection = self._existing_conn or self.engine.connect()\n \n try:\n result = connection.execute(text(sql))\n if self.dialect_name in self.JSON_TYPE_CODE_SET_MAP:\n field_type_map = {\n column_desc[0]: column_desc[1]\n for column_desc in result.cursor.description\n }\n return [\n {\n key: json.loads(value) if field_type_map.get(key, -1) in self._json_type_code_set else value\n for key, value in item.items()\n }\n for item in result.mappings()\n ]\n finally:\n if should_close_connection:\n connection.close()\n\n @property\n def dialect_name(self) -> str:\n return self.engine.dialect.name\n\n\nclass DatabaseDataParser(BaseDataParser):\n \"\"\"data parser for database\"\"\"\n sqlglot_dialect_map = {\n \"postgresql\": \"postgres\",\n \"mssql\": \"tsql\",\n }\n\n def __init__(\n self,\n conn: Connector,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ):\n self.conn = conn\n self.example_pandas_df = self._get_example_pandas_df()\n self.field_specs = field_specs\n self.infer_string_to_date = infer_string_to_date\n self.infer_number_to_dimension = infer_number_to_dimension\n self.other_params = other_params\n\n def _get_example_pandas_df(self) -> pd.DataFrame:\n sql = self._format_sql(f\"SELECT * FROM {self.placeholder_table_name} LIMIT 1000\")\n example_df = pd.DataFrame(self.conn.query_datas(sql))\n for column in example_df.columns:\n if any(isinstance(val, Decimal) for val in example_df[column]):\n example_df[column] = example_df[column].astype(float)\n return example_df\n\n def _format_sql(self, sql: str) -> str:\n sqlglot_dialect_name = self.sqlglot_dialect_map.get(self.conn.dialect_name, self.conn.dialect_name)\n\n sub_query = exp.Subquery(\n this=sqlglot.parse(self.conn.view_sql, read=sqlglot_dialect_name)[0],\n alias=exp.TableAlias(this=\"temp_view_name\")\n )\n ast = sqlglot.parse(sql, read=DuckdbDialect)[0]\n for from_exp in ast.find_all(exp.From):\n if str(from_exp.this).strip('\"') == self.placeholder_table_name:\n from_exp.this.replace(sub_query)\n\n sql = ast.sql(sqlglot_dialect_name)\n return sql\n\n @property\n def placeholder_table_name(self) -> str:\n return \"___pygwalker_temp_view_name___\"\n\n @property\n @lru_cache()\n def field_metas(self) -> List[Dict[str, str]]:\n data = self._get_datas_by_sql(f\"SELECT * FROM {self.placeholder_table_name} LIMIT 1\")\n return get_data_meta_type(data[0]) if data else []\n\n @property\n @lru_cache()\n def raw_fields(self) -> List[Dict[str, str]]:\n pandas_parser = PandasDataFrameDataParser(\n self.example_pandas_df,\n self.field_specs,\n self.infer_string_to_date,\n self.infer_number_to_dimension,\n self.other_params\n )\n return [\n {**field, \"fid\": field[\"name\"]}\n for field in pandas_parser.raw_fields\n ]\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n if limit is None:\n df = self.example_pandas_df\n else:\n df = self.example_pandas_df[:limit]\n df = df.replace({float('nan'): None})\n return df.to_dict(orient='records')\n\n def get_datas_by_payload(self, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n sql = get_sql_from_payload(\n self.placeholder_table_name,\n payload,\n {self.placeholder_table_name: self.field_metas}\n )\n sql = self._format_sql(sql)\n result = self.conn.query_datas(sql)\n return result\n\n def get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n pass\n\n def _get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n \"\"\"a private method for get_datas_by_sql\"\"\"\n sql = self._format_sql(sql)\n result = self.conn.query_datas(sql)\n return result\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.example_pandas_df.toPandas().to_csv(content, index=False)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.example_pandas_df.toPandas().to_parquet(content, index=False, compression=\"snappy\")\n return content\n\n def batch_get_datas_by_sql(self, sql_list: List[str]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_sql(sql)\n for sql in sql_list\n ]\n\n def batch_get_datas_by_payload(self, payload_list: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_payload(payload)\n for payload in payload_list\n ]\n\n @property\n def dataset_type(self) -> str:\n return f\"connector_{self.conn.dialect_name}\"\n\n @property\n def data_size(self) -> int:\n return INFINITY_DATA_SIZE\n" }, { "path": "pygwalker/data_parsers/modin_parser.py", "content": "import io\nfrom typing import Any, Dict, List, Optional\n\nfrom modin import pandas as mpd\n\nfrom .base import (\n BaseDataFrameDataParser,\n FieldSpec,\n is_temporal_field,\n is_geo_field\n)\nfrom pygwalker.services.fname_encodings import rename_columns\n\n\nclass ModinPandasDataFrameDataParser(BaseDataFrameDataParser[mpd.DataFrame]):\n \"\"\"prop parser for modin.pandas.DataFrame\"\"\"\n def __init__(\n self,\n df: mpd.DataFrame,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ):\n super().__init__(df, field_specs, infer_string_to_date, infer_number_to_dimension, other_params)\n self._duckdb_df = self.df._to_pandas()\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n df = self.df[:limit] if limit is not None else self.df\n df = df.replace({float('nan'): None})\n return df.to_dict(orient='records')\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.to_csv(content, index=False)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.to_parquet(content, index=False, compression=\"snappy\")\n return content\n\n def _rename_dataframe(self, df: mpd.DataFrame) -> mpd.DataFrame:\n df = df.reset_index(drop=True)\n df.columns = rename_columns(list(df.columns))\n return df\n\n def _infer_semantic(self, s: mpd.Series, field_name: str):\n example_value = s[0]\n kind = s.dtype.kind\n\n if kind in \"fcmiu\" or is_geo_field(field_name):\n return \"quantitative\"\n if kind in \"M\" or (kind in \"bOSUV\" and is_temporal_field(example_value, self.infer_string_to_date)):\n return 'temporal'\n\n return \"nominal\"\n\n def _infer_analytic(self, s: mpd.Series, field_name: str):\n kind = s.dtype.kind\n\n if is_geo_field(field_name):\n return \"dimension\"\n\n if self.infer_number_to_dimension and kind in \"iu\" and len(s.unique()) <= 16:\n return \"dimension\"\n\n if kind in \"fcmiu\":\n return \"measure\"\n\n return \"dimension\"\n\n @property\n def dataset_type(self) -> str:\n return \"modin_dataframe\"\n" }, { "path": "pygwalker/data_parsers/pandas_parser.py", "content": "from typing import Any, Dict, List, Optional\nimport io\n\nimport pandas as pd\n\nfrom .base import (\n BaseDataFrameDataParser,\n is_temporal_field,\n is_geo_field\n)\nfrom pygwalker.services.fname_encodings import rename_columns\n\n\nclass PandasDataFrameDataParser(BaseDataFrameDataParser[pd.DataFrame]):\n \"\"\"prop parser for pandas.DataFrame\"\"\"\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n df = self.df[:limit] if limit is not None else self.df\n df = df.replace({float('nan'): None})\n return df.to_dict(orient='records')\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.to_csv(content, index=False)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.to_parquet(content, index=False, compression=\"snappy\")\n return content\n\n def _rename_dataframe(self, df: pd.DataFrame) -> pd.DataFrame:\n df = df.reset_index(drop=True)\n df.columns = rename_columns(list(df.columns))\n return df\n\n def _infer_semantic(self, s: pd.Series, field_name: str):\n example_value = s[0]\n kind = s.dtype.kind\n\n if kind in \"fcmiu\" or is_geo_field(field_name):\n return \"quantitative\"\n if kind in \"M\" or (kind in \"bOSUV\" and is_temporal_field(example_value, self.infer_string_to_date)):\n return 'temporal'\n\n return \"nominal\"\n\n def _infer_analytic(self, s: pd.Series, field_name: str):\n kind = s.dtype.kind\n\n if is_geo_field(field_name):\n return \"dimension\"\n\n if self.infer_number_to_dimension and kind in \"iu\" and len(s.unique()) <= 16:\n return \"dimension\"\n\n if kind in \"fcmiu\":\n return \"measure\"\n\n return \"dimension\"\n\n @property\n def dataset_type(self) -> str:\n return \"pandas_dataframe\"\n" }, { "path": "pygwalker/data_parsers/polars_parser.py", "content": "from typing import List, Any, Dict, Optional\nimport io\n\nimport polars as pl\n\nfrom .base import (\n BaseDataFrameDataParser,\n is_temporal_field,\n is_geo_field\n)\nfrom pygwalker.services.fname_encodings import rename_columns\n\n\nclass PolarsDataFrameDataParser(BaseDataFrameDataParser[pl.DataFrame]):\n \"\"\"prop parser for polars.DataFrame\"\"\"\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n df = self.df[:limit] if limit is not None else self.df\n df = df.fill_nan(None)\n return df.to_dicts()\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.write_csv(content)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.write_parquet(content, compression=\"snappy\")\n return content\n\n def _rename_dataframe(self, df: pl.DataFrame) -> pl.DataFrame:\n df = df.rename({\n old_col: new_col\n for old_col, new_col in zip(df.columns, rename_columns(df.columns))\n })\n return df\n\n def _infer_semantic(self, s: pl.Series, field_name: str):\n example_value = s[0]\n kind = s.dtype\n\n if kind in pl.NUMERIC_DTYPES or is_geo_field(field_name):\n return \"quantitative\"\n if kind in pl.TEMPORAL_DTYPES or is_temporal_field(example_value, self.infer_string_to_date):\n return \"temporal\"\n\n return \"nominal\"\n\n def _infer_analytic(self, s: pl.Series, field_name: str):\n kind = s.dtype\n\n if is_geo_field(field_name):\n return \"dimension\"\n\n if self.infer_number_to_dimension and kind in pl.INTEGER_DTYPES and len(s.unique()) <= 16:\n return \"dimension\"\n\n if kind in pl.NUMERIC_DTYPES:\n return \"measure\"\n\n return \"dimension\"\n\n @property\n def dataset_type(self) -> str:\n return \"polars_dataframe\"\n" }, { "path": "pygwalker/data_parsers/spark_parser.py", "content": "from typing import Any, Dict, List, Optional\nfrom functools import lru_cache\nimport logging\nimport io\n\nfrom pyspark.sql import DataFrame\nimport sqlglot\n\nfrom .base import BaseDataParser, get_data_meta_type, INFINITY_DATA_SIZE\nfrom .pandas_parser import PandasDataFrameDataParser\nfrom pygwalker.services.fname_encodings import rename_columns\nfrom pygwalker.data_parsers.base import FieldSpec\nfrom pygwalker.utils.payload_to_sql import get_sql_from_payload\n\nlogger = logging.getLogger(__name__)\n\n\nclass SparkDataFrameDataParser(BaseDataParser):\n \"\"\"prop parser for DataFrame of spark\"\"\"\n def __init__(\n self,\n df: DataFrame,\n field_specs: List[FieldSpec],\n infer_string_to_date: bool,\n infer_number_to_dimension: bool,\n other_params: Dict[str, Any]\n ):\n if not df.is_cached:\n logger.warning(\n \"The input dataframe is not cached, which may cause performance issues.\\n\"\n \"If dataframe is the result of a large number of calculations before, please cache it before passing it to pygwalker.\\n\"\n \"Pyspark cache function: `df.cache()`\"\n )\n self.spark = df.sparkSession\n self.origin_df = df\n self.df = self._rename_dataframe(df)\n self.example_pandas_df = df.limit(1000).toPandas()\n self.field_specs = field_specs\n self.infer_string_to_date = infer_string_to_date\n self.infer_number_to_dimension = infer_number_to_dimension\n self.other_params = other_params\n\n @property\n @lru_cache()\n def raw_fields(self) -> List[Dict[str, str]]:\n pandas_parser = PandasDataFrameDataParser(\n self.example_pandas_df,\n self.field_specs,\n self.infer_string_to_date,\n self.infer_number_to_dimension,\n self.other_params\n )\n return pandas_parser.raw_fields\n\n @property\n @lru_cache()\n def field_metas(self) -> List[Dict[str, str]]:\n data = self.get_datas_by_sql(\"SELECT * FROM pygwalker_mid_table LIMIT 1\")\n return get_data_meta_type(data[0]) if data else []\n\n def to_records(self, limit: Optional[int] = None) -> List[Dict[str, Any]]:\n df = self.df.limit(limit) if limit is not None else self.df\n return [row.asDict() for row in df.collect()]\n\n def get_datas_by_sql(self, sql: str) -> List[Dict[str, Any]]:\n self.df.createOrReplaceTempView(\"pygwalker_mid_table\")\n sql = sqlglot.transpile(sql, read=\"duckdb\", write=\"spark\")[0]\n result_df = self.spark.sql(sql)\n return [row.asDict() for row in result_df.collect()]\n\n def get_datas_by_payload(self, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n sql = get_sql_from_payload(\n \"pygwalker_mid_table\",\n payload,\n {\"pygwalker_mid_table\": self.field_metas}\n )\n return self.get_datas_by_sql(sql)\n\n def batch_get_datas_by_sql(self, sql_list: List[str]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_sql(sql)\n for sql in sql_list\n ]\n\n def batch_get_datas_by_payload(self, payload_list: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:\n \"\"\"batch get records\"\"\"\n return [\n self.get_datas_by_payload(payload)\n for payload in payload_list\n ]\n\n def to_csv(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.toPandas().to_csv(content, index=False)\n return content\n\n def to_parquet(self) -> io.BytesIO:\n content = io.BytesIO()\n self.df.toPandas().to_parquet(content, index=False, compression=\"snappy\")\n return content\n\n def _rename_dataframe(self, df: DataFrame) -> DataFrame:\n new_columns = rename_columns(list(df.columns))\n return df.toDF(*new_columns)\n\n @property\n def dataset_type(self) -> str:\n return \"spark_dataframe\"\n\n @property\n def placeholder_table_name(self) -> str:\n return \"pygwalker_mid_table\"\n\n @property\n def data_size(self) -> int:\n return INFINITY_DATA_SIZE\n" }, { "path": "pygwalker/errors.py", "content": "\"\"\"\n This module contains all the custom errors used by pygwalker.\n\"\"\"\nfrom enum import Enum\n\n\nclass ErrorCode(int, Enum):\n UNKNOWN_ERROR = -1\n TOKEN_ERROR = 20001\n CLOUD_CONFIG_LIMIT = 20002\n CLOUD_CHART_NOT_FOUND = 20003\n\n\nclass BaseError(Exception):\n \"\"\"Base class for all exceptions raised by pygwalker.\"\"\"\n def __init__(self, *args, code: ErrorCode = ErrorCode.UNKNOWN_ERROR) -> None:\n super().__init__(*args)\n self.code = code\n\n\nclass InvalidConfigIdError(BaseError):\n \"\"\"Raised when the config is invalid.\"\"\"\n pass\n\n\nclass PrivacyError(BaseError):\n \"\"\"Raised when the privacy setting is invalid.\"\"\"\n pass\n\n\nclass CloudFunctionError(BaseError):\n \"\"\"Raised when the cloud function is invalid.\"\"\"\n pass\n\n\nclass CsvFileTooLargeError(BaseError):\n \"\"\"Raised when the csv file is too large.\"\"\"\n pass\n\n\nclass ViewSqlSameColumnError(BaseError):\n \"\"\"Raised when the view sql is invalid.\"\"\"\n pass\n\n\nclass StreamlitPygwalkerApiError(BaseError):\n \"\"\"Raised when the config is invalid.\"\"\"\n def __init__(self) -> None:\n super().__init__(\n \"Adding pygwalker web api to streamlit failed. If possible, please report this case to the pygwalker team. Thanks!\",\n code=ErrorCode.UNKNOWN_ERROR\n )\n" }, { "path": "pygwalker/services/__init__.py", "content": "" }, { "path": "pygwalker/services/check_update.py", "content": "\"\"\"\nCheck_Update Module: This module enables you to check if\nthe version is up-to-date.\n\nUpdated on Tue November 11 15:15:48 2024\n\n@author: Kanaries\n\n\"\"\"\n\nimport asyncio\nimport logging\nimport sys\nimport json\nfrom typing import Dict, Any, Coroutine\nfrom urllib import request\nfrom threading import Thread\nfrom pygwalker import __version__\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom .config import get_local_user_id\n\n\n_UPDATE_URL = 'https://5agko11g7e.execute-api.us-west-1.amazonaws.com/default/check_updates'\n\nlogger = logging.getLogger(__name__)\n\n\ndef _sync_get_async_result(co: Coroutine[Any, Any, Any]) -> Any:\n \"\"\"\n fetch asynchronous result.\n\n Parameters\n ----------\n co : Coroutine\n asynchronous parameter.\n\n \"\"\"\n loop = asyncio.new_event_loop()\n try:\n return loop.run_until_complete(co)\n finally:\n loop.close()\n\n\nasync def _request_on_pyodide(url: str) -> Dict[str, Any]:\n \"\"\"\n request url.\n\n Parameters\n ----------\n url : string\n url string.\n Returns\n ---------\n Dict\n\n \"\"\"\n import pyodide\n resp = await pyodide.http.pyfetch(url)\n return await resp.json()\n\n\ndef _request_on_python(url: str) -> Dict[str, Any]:\n \"\"\"\n request url on python.\n\n Parameters\n ----------\n url : string\n url string.\n Returns\n ---------\n Dict\n\n \"\"\"\n with request.urlopen(url, timeout=30) as resp:\n return json.loads(resp.read().decode(\"utf-8\"))\n\n\ndef _check_update() -> Dict[str, Any]:\n \"\"\"\n Internal function to Check for updates.\n\n Parameters\n ----------\n None\n\n Returns\n ---------\n Dict\n\n \"\"\"\n payload = {'pkg': 'pygwalker', 'v': __version__, 'hashcode': get_local_user_id()}\n request_func = _request_on_python\n\n if \"pyodide\" in sys.modules:\n payload['pkg'] = 'pygwalker-pyodide'\n request_func = _request_on_pyodide\n\n params = '&'.join([f\"{k}={v}\" for k, v in payload.items()])\n url = f\"{_UPDATE_URL}?{params}\"\n\n try:\n result = request_func(url)\n if isinstance(result, Coroutine):\n result = _sync_get_async_result(result)\n return result\n finally:\n pass\n\n\ndef check_update() -> None:\n \"\"\"\n Check for update.\n\n Parameters\n ----------\n None\n\n \"\"\"\n if GlobalVarManager.privacy != \"offline\":\n Thread(target=_check_update).start()\n" }, { "path": "pygwalker/services/cloud_service.py", "content": "from typing import List, Dict, Any, Optional\nfrom datetime import datetime\nfrom urllib.parse import urlencode\nfrom typing_extensions import Literal\nimport logging\nimport io\nimport json\nimport hashlib\n\nimport requests\n\nfrom .global_var import GlobalVarManager\nfrom pygwalker.services.data_parsers import BaseDataParser\nfrom pygwalker.errors import CloudFunctionError, ErrorCode\n\nlogger = logging.getLogger(__name__)\n\n\ndef _get_database_type_from_dialect_name(dialect_name: str) -> str:\n type_map = {\n \"postgresql\": \"postgres\",\n }\n type_name = type_map.get(dialect_name, dialect_name)\n return type_name[0].upper() + type_name[1:]\n\n\ndef _upload_file_to_s3(url: str, content: io.BytesIO):\n requests.put(url, content.getvalue(), timeout=300)\n\n\ndef _generate_chart_pre_redirect_uri(chart_id: str, auth_code_info: Dict[str, Any]) -> str:\n pre_uri = f\"{GlobalVarManager.kanaries_main_host}/api/pygwalker/authCode\"\n redirect_uri = f\"{GlobalVarManager.kanaries_main_host}/share/pyg_chart/{chart_id}\"\n\n params = {\n **auth_code_info,\n \"successUrl\": f\"{redirect_uri}?mode=edit\",\n \"failUrl\": f\"{redirect_uri}?mode=view\",\n }\n return pre_uri + \"?\" + urlencode(params)\n\n\ndef read_config_from_cloud(path: str) -> str:\n \"\"\"Return config, if not exist, return empty string\"\"\"\n url = f\"{GlobalVarManager.kanaries_api_host}/pygConfig\"\n resp = requests.get(url, params={\"path\": path}, timeout=15)\n return resp.json()[\"data\"][\"config\"]\n\n\nclass PrivateSession(requests.Session):\n \"\"\"A session with kanaries\"\"\"\n def __init__(self, api_key: Optional[str]):\n super().__init__()\n self.kanaries_api_key = api_key\n\n def prepare_request(self, request: requests.Request) -> requests.PreparedRequest:\n req = super().prepare_request(request)\n req.headers[\"kanaries-api-key\"] = self.kanaries_api_key\n return req\n\n def send(self, request: requests.PreparedRequest, **kwargs) -> requests.Response:\n kanaries_api_key = self.kanaries_api_key or GlobalVarManager.kanaries_api_key\n if not kanaries_api_key:\n logger.error((\n \"kanaries_api_key is not valid.\\n\"\n \"Please set kanaries_api_key first.\\n\"\n \"If you are not kanaries user, please register it from 'https://kanaries.net/home/access' \\n\"\n \"Then refer 'https://github.com/Kanaries/pygwalker/wiki/How-to-get-api-key-of-kanaries%3F' to set kanaries_api_key. \\n\"\n ))\n raise CloudFunctionError(\"no kanaries api key. visit: https://docs.kanaries.net/ for setup documentation.\", code=ErrorCode.TOKEN_ERROR)\n resp = super().send(request, **kwargs)\n try:\n resp_json = resp.json()\n except Exception as e:\n raise CloudFunctionError(f\"Request failed: {resp.text}\") from e\n\n if \"success\" in resp_json:\n if resp_json[\"success\"] is False:\n raise CloudFunctionError(\n f\"Request failed: {resp_json['message']}\",\n code=resp_json[\"code\"] if resp_json[\"code\"] != 0 else ErrorCode.UNKNOWN_ERROR\n )\n else:\n if resp.status_code != 200:\n raise CloudFunctionError(\n f\"Request failed: {resp_json['error']['message']}\",\n code=resp_json[\"error\"][\"code\"]\n )\n\n return resp\n\n\nclass CloudService:\n \"\"\"A class to manage cloud service\"\"\"\n def __init__(self, api_key: str):\n self.session = PrivateSession(api_key)\n\n def _upload_file_dataset_meta(\n self,\n name: str,\n file_type: str = Literal[\"parquet\", \"csv\"],\n is_public: bool = True,\n kind: Literal[\"TEMP\", \"FILE\"] = \"FILE\"\n ) -> Dict[str, Any]:\n param_file_type_map = {\n \"csv\": \"TEXT_FILE\",\n \"parquet\": \"PARQUET\"\n }\n\n url = f\"{GlobalVarManager.kanaries_api_host}/dataset/upload\"\n params = {\n \"name\": name,\n \"fileName\": name + \".\" + file_type,\n \"isPublic\": is_public,\n \"desc\": \"\",\n \"meta\": {\n \"extractHeader\": True,\n \"encoding\": \"utf-8\",\n \"type\": param_file_type_map.get(file_type, \"TEXT_FILE\"),\n \"separator\": \",\",\n },\n \"type\": kind,\n }\n resp = self.session.post(url, json=params, timeout=10)\n return resp.json()[\"data\"]\n\n def _upload_dataset_callback(self, dataset_id: str, fid_list: List[str]) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/dataset/callback\"\n params = {\n \"datasetId\": dataset_id,\n \"fidList\": fid_list\n }\n resp = self.session.post(url, json=params, timeout=10)\n return resp.json()\n\n def _create_chart(\n self,\n *,\n dataset_id: str,\n name: str,\n meta: str,\n workflow: List[Dict[str, Any]],\n thumbnail: str,\n is_public: bool,\n ) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/chart\"\n params = {\n \"datasetId\": dataset_id,\n \"meta\": meta,\n \"query\": json.dumps({\"datasetId\": dataset_id, \"workflow\": workflow}),\n \"config\": \"{}\",\n \"name\": name,\n \"desc\": \"\",\n \"isPublic\": is_public,\n \"chartType\": \"\",\n \"thumbnail\": thumbnail,\n }\n resp = self.session.post(url, json={\"chart\": params}, timeout=10)\n return resp.json()[\"data\"]\n\n def _create_notebook(self, title: str, chart_id: str) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/notebook\"\n markdown = \"\\n\".join([\n \"# \" + title,\n f\"::chart[{chart_id}]\"\n ])\n params = {\n \"title\": title,\n \"markdown\": markdown,\n \"isPublic\": True,\n }\n resp = self.session.post(url, json=params, timeout=10)\n return resp.json()[\"data\"]\n\n def _get_chart_by_name(self, name: str, workspace_name: str) -> Optional[Dict[str, Any]]:\n url = f\"{GlobalVarManager.kanaries_main_host}/api/pygwalker/chart\"\n try:\n resp = self.session.get(url, params={\"chartName\": name, \"workspaceName\": workspace_name}, timeout=15)\n except CloudFunctionError as e:\n if e.code == ErrorCode.CLOUD_CHART_NOT_FOUND:\n return None\n raise e\n return resp.json()[\"data\"]\n\n def _get_auth_code_info(self) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/auth/code\"\n resp = self.session.get(url, timeout=15)\n return resp.json()[\"data\"]\n\n def write_config_to_cloud(self, path: str, config: str):\n \"\"\"Write config to cloud\"\"\"\n url = f\"{GlobalVarManager.kanaries_api_host}/pygConfig\"\n self.session.put(url, json={\n \"path\": path,\n \"config\": config\n })\n\n def get_cloud_graphic_walker(self, workspace_name: str, chart_name: str) -> str:\n chart_data = self._get_chart_by_name(chart_name, workspace_name)\n\n if chart_data is None:\n raise CloudFunctionError(\"chart not exists\", code=ErrorCode.CLOUD_CHART_NOT_FOUND)\n\n try:\n auto_code_info = self._get_auth_code_info()\n except CloudFunctionError:\n auto_code_info = {}\n\n pre_redirect_uri = _generate_chart_pre_redirect_uri(chart_data[\"chartId\"], auto_code_info)\n\n return pre_redirect_uri\n\n def create_cloud_graphic_walker(\n self,\n *,\n chart_name: str,\n workspace_name: str,\n dataset_content: io.BytesIO,\n field_specs: List[Dict[str, Any]],\n ) -> str:\n fid_list = [field[\"fid\"] for field in field_specs]\n meta = json.dumps({\n \"dataSources\": [{\n \"id\": \"dataSource-0\",\n \"data\": []\n }],\n \"datasets\": [{\n \"id\": 'dataset-0',\n \"name\": 'DataSet',\n \"rawFields\": field_specs,\n \"dsId\": 'dataSource-0',\n }],\n \"specList\": []\n })\n\n chart_data = self._get_chart_by_name(chart_name, workspace_name)\n\n if chart_data is not None:\n raise CloudFunctionError(\"chart name already exists\", code=ErrorCode.UNKNOWN_ERROR)\n\n dataset_name = f\"pygwalker_{datetime.now().strftime('%Y%m%d%H%M')}\"\n dataset_info = self._upload_file_dataset_meta(dataset_name, \"parquet\")\n dataset_id = dataset_info[\"datasetId\"]\n upload_url = dataset_info[\"uploadUrl\"]\n _upload_file_to_s3(upload_url, dataset_content)\n self._upload_dataset_callback(dataset_id, fid_list)\n\n self._create_chart(\n dataset_id=dataset_id,\n name=chart_name,\n meta=meta,\n workflow={},\n thumbnail=\"\",\n is_public=True\n )\n\n def get_kanaries_user_info(self) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/user/info\"\n resp = self.session.get(url, timeout=15)\n return resp.json()[\"data\"]\n\n def get_spec_by_text(self, metas: List[Dict[str, Any]], text: str) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/vis/text2gw\"\n resp = self.session.post(\n url,\n json={\"metas\": metas, \"messages\": [{\"role\": \"user\", \"content\": text}]},\n timeout=15\n )\n return resp.json()[\"data\"]\n\n def get_chart_by_chats(self, metas: List[Dict[str, Any]], chats: Any) -> Dict[str, Any]:\n url = f\"{GlobalVarManager.kanaries_api_host}/vis/chat2gw\"\n resp = self.session.post(\n url,\n json={\"metas\": metas, \"messages\": chats},\n timeout=30\n )\n return resp.json()[\"data\"]\n\n def create_file_dataset(\n self,\n dataset_name: str,\n dataset_content: io.BytesIO,\n fid_list: List[str],\n is_public: bool,\n kind: Literal[\"TEMP\", \"FILE\"]\n ) -> str:\n dataset_info = self._upload_file_dataset_meta(dataset_name, \"parquet\", is_public, kind=kind)\n dataset_id = dataset_info[\"datasetId\"]\n upload_url = dataset_info[\"uploadUrl\"]\n _upload_file_to_s3(upload_url, dataset_content)\n self._upload_dataset_callback(dataset_id, fid_list)\n return dataset_id\n\n def create_datasource(\n self,\n name: str,\n database_url: str,\n database_type: str,\n ) -> str:\n url = f\"{GlobalVarManager.kanaries_api_host}/datasource\"\n params = {\n \"name\": name,\n \"connectionconfiguration\": {\n \"url\": database_url,\n },\n \"datasourceType\": database_type,\n \"desc\": \"\"\n }\n resp = self.session.post(url, json=params, timeout=15)\n return resp.json()[\"data\"][\"datasourceId\"]\n\n def get_datasource_by_name(self, name: str) -> Optional[str]:\n url = f\"{GlobalVarManager.kanaries_api_host}/datasource/search\"\n resp = self.session.post(url, params={\"fullName\": name}, timeout=15)\n datasources = resp.json()[\"data\"][\"datasourceList\"]\n return datasources[0][\"id\"] if datasources else None\n\n def create_database_dataset(\n self,\n name: str,\n datasource_id: str,\n is_public: bool,\n view_sql: str,\n ) -> str:\n url = f\"{GlobalVarManager.kanaries_api_host}/dataset\"\n params = {\n \"name\": name,\n \"desc\": \"\",\n \"datasourceId\": datasource_id,\n \"isPublic\": is_public,\n \"type\": \"DATABASE\",\n \"meta\": {\n \"viewSql\": view_sql,\n }\n }\n resp = self.session.post(url, json=params, timeout=60)\n return resp.json()[\"data\"][\"datasetId\"]\n\n def query_from_dataset(self, dataset_id: str, payload: Dict[str, Any]) -> List[Dict[str, Any]]:\n url = f\"{GlobalVarManager.kanaries_api_host}/public/query\"\n params = {\n \"datasetId\": dataset_id,\n \"query\": payload,\n }\n resp = self.session.post(url, json=params, timeout=15)\n return resp.json()[\"data\"]\n\n def batch_query_from_dataset(self, dataset_id: str, query_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:\n url = f\"{GlobalVarManager.kanaries_api_host}/v1/dataset/{dataset_id}/query\"\n params = {\n \"query\": query_list,\n }\n resp = self.session.post(url, json=params, timeout=60)\n return resp.json()[\"data\"]\n\n def create_cloud_dataset(\n self,\n data_parser: BaseDataParser,\n name: str,\n is_public: bool,\n is_temp_dataset: bool = False\n ) -> str:\n if name is None:\n name = f\"pygwalker_{datetime.now().strftime('%Y%m%d%H%M')}\"\n\n if data_parser.dataset_type == \"cloud_dataset\":\n raise ValueError(\"dataset is already a cloud dataset\")\n\n if data_parser.dataset_type.startswith(\"connector\"):\n connector = data_parser.conn\n datasource_name = \"pygwalker_\" + hashlib.md5(connector.url.encode()).hexdigest()\n datasource_id = self.get_datasource_by_name(datasource_name)\n if datasource_id is None:\n datasource_id = self.create_datasource(\n datasource_name,\n connector.url,\n _get_database_type_from_dialect_name(connector.dialect_name)\n )\n dataset_id = self.create_database_dataset(\n name,\n datasource_id,\n is_public,\n connector.view_sql\n )\n return dataset_id\n else:\n dataset_id = self.create_file_dataset(\n name,\n data_parser.to_parquet(),\n [field[\"name\"] for field in data_parser.raw_fields],\n is_public,\n kind=\"TEMP\" if is_temp_dataset else \"FILE\"\n )\n\n return dataset_id\n\n def create_dashboard(\n self,\n *,\n name: str,\n layout: List[Any],\n config: Dict[str, Any],\n is_public: bool\n ) -> str:\n url = f\"{GlobalVarManager.kanaries_api_host}/report\"\n params = {\n \"title\": name,\n \"version\": \"0.0.1\",\n \"desc\": \"\",\n \"size\": {},\n \"config\": config,\n \"layout\": layout,\n \"public\": is_public\n }\n resp = self.session.post(url, json=params, timeout=60)\n return resp.json()[\"data\"][\"id\"]\n\n def upload_cloud_chart(\n self,\n *,\n chart_name: str,\n dataset_name: str,\n data_parser: BaseDataParser,\n workflow: List[Dict[str, Any]],\n spec_list: List[Dict[str, Any]],\n is_public: bool,\n ) -> str:\n workspace_name = self.get_kanaries_user_info()[\"workspaceName\"]\n\n chart_data = self._get_chart_by_name(chart_name, workspace_name)\n\n if chart_data is not None:\n raise CloudFunctionError(\"chart name already exists\", code=ErrorCode.UNKNOWN_ERROR)\n\n dataset_id = self.create_cloud_dataset(data_parser, dataset_name, False)\n chart_info = self._create_chart(\n dataset_id=dataset_id,\n name=chart_name,\n meta=json.dumps({\n \"dataSources\": [{\n \"id\": \"dataSource-0\",\n \"data\": []\n }],\n \"datasets\": [{\n \"id\": 'dataset-0',\n \"name\": 'DataSet',\n \"rawFields\": data_parser.raw_fields,\n \"dsId\": 'dataSource-0',\n }],\n \"specList\": spec_list\n }),\n workflow=workflow,\n thumbnail=\"\",\n is_public=is_public\n )\n\n return {\n \"chart_id\": chart_info[\"chartId\"],\n \"dataset_id\": dataset_id\n }\n\n def upload_cloud_dashboard(\n self,\n *,\n dashboard_name: str,\n dataset_name: str,\n data_parser: BaseDataParser,\n workflow_list: List[List[Dict[str, Any]]],\n spec_list: List[Dict[str, Any]],\n is_public: bool,\n appearance: str,\n create_dashboard_flag: bool\n ) -> Dict[str, str]:\n dataset_id = self.create_cloud_dataset(data_parser, dataset_name, False)\n\n chart_info_list = []\n for spec, workflow in zip(spec_list, workflow_list): \n chart_info = self._create_chart(\n dataset_id=dataset_id,\n name=f\"{dashboard_name}-{spec['name']}\",\n meta=json.dumps({\n \"dataSources\": [{\n \"id\": \"dataSource-0\",\n \"data\": []\n }],\n \"datasets\": [{\n \"id\": 'dataset-0',\n \"name\": 'DataSet',\n \"rawFields\": data_parser.raw_fields,\n \"dsId\": 'dataSource-0',\n }],\n \"specList\": [spec]\n }),\n workflow=workflow,\n thumbnail=\"\",\n is_public=is_public\n )\n\n chart_info_list.append(chart_info)\n\n if not create_dashboard_flag:\n return {\n \"dashboard_id\": \"\",\n \"dataset_id\": dataset_id\n }\n\n dashboard_id = self.create_dashboard(\n name=dashboard_name,\n is_public=is_public,\n config={\n \"items\": [\n {\"id\": \"dashboard_title\", \"content\": f\"# {dashboard_name}\", \"type\": \"text\", \"name\": \"Text\"},\n {\n \"id\": \"chart_tab\",\n \"dark\": appearance,\n \"name\": \"Charts\",\n \"type\": \"data\",\n \"tabs\": [\n {\"chartId\": chart_info[\"chartId\"], \"title\": spec[\"name\"]}\n for spec, chart_info in zip(spec_list, chart_info_list)\n ],\n \"mode\": \"gwtabs\",\n }\n ],\n },\n layout=[\n {\"i\": \"dashboard_title\", \"h\": 2, \"w\": 4, \"x\": 0, \"y\": 0},\n {\"i\": \"chart_tab\", \"h\": 20, \"w\": 4, \"x\": 0, \"y\": 2},\n ]\n )\n\n return {\n \"dashboard_id\": dashboard_id,\n \"dataset_id\": dataset_id\n }\n" }, { "path": "pygwalker/services/config.py", "content": "import os\nimport json\nfrom typing import List, Optional, Dict\nfrom functools import lru_cache\n\nfrom pygwalker.utils.randoms import generate_hash_code\n\nfrom appdirs import user_config_dir\n\n\nDEFAULT_CONFIG = {\n \"privacy\": \"events\",\n \"kanaries_token\": \"\",\n}\nCONFIG_KEYS = list(DEFAULT_CONFIG.keys())\nAPP_DIR = user_config_dir(\"pygwalker\")\nCONFIG_PATH = os.path.join(APP_DIR, \"config.json\")\nUSER_CONFIG_PATH = os.path.join(APP_DIR, \"user_config.json\")\n\n\nclass ConfigItem:\n \"\"\"Configuration item.\"\"\"\n def __init__(\n self,\n name: str,\n type_list: List[str],\n default: Optional[str] = None,\n description: str = ''\n ) -> None:\n self.name = name\n self.type_list = type_list\n self.default = default\n self.description = description\n\n def __str__(self) -> str:\n return f\"- {self.name} {self.type_list} (default: {self.default}).{self.description}\"\n\n\nprivacy_item = ConfigItem(\n \"privacy\",\n [\"offline\", \"update-only\", \"events\"],\n default=\"events\",\n description=\"\"\"\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYSIS IS SEND.\n \"\"\"\n)\nkanati_token_item = ConfigItem(\n \"kanaries_token\",\n [\"your kanaries token\"],\n default=\"empty string\",\n description=\"\"\"\n your kanaries token, you can get it from https://kanaries.net.\n refer: https://space.kanaries.net/t/how-to-get-api-key-of-kanaries.\n by kanaries token, you can use kanaries service in pygwalker, such as share chart, share config.\n \"\"\"\n)\nconfig_items = [privacy_item, kanati_token_item]\n\n\ndef get_config_params_help() -> str:\n help_str = \"\"\n help_str += \"Available configurations:\\n\\n\"\n for item in config_items: \n help_str += (str(item) + \"\\n\")\n return help_str\n\n\ndef _read_and_create_file(path: str, default_content: Dict[str, str]) -> Dict[str, str]:\n try:\n if not os.path.exists(path):\n os.makedirs(os.path.dirname(path), exist_ok=True)\n with open(path, 'w', encoding=\"utf-8\") as f:\n json.dump(default_content, f, indent=4)\n\n with open(path, 'r', encoding=\"utf-8\") as f:\n file_content = json.load(f)\n return file_content\n except Exception:\n return default_content\n\n\ndef set_config(new_config: Dict[str, str]):\n \"\"\"Set configuration.\n\n Args:\n configs (dict): key-value map\n save (bool, optional): save to user's config path. Defaults to False.\n \"\"\"\n config = _read_and_create_file(CONFIG_PATH, DEFAULT_CONFIG)\n\n config.update(new_config)\n\n with open(CONFIG_PATH, 'w', encoding=\"utf-8\") as f:\n json.dump(config, f, indent=4)\n\n\ndef reset_config(keys: List[str]):\n \"\"\"Unset user configuration and use default value instead.\n Args:\n keys (List[str], optional): Defaults to None.\n \"\"\"\n config = _read_and_create_file(CONFIG_PATH, DEFAULT_CONFIG)\n\n for key in keys:\n if key in DEFAULT_CONFIG:\n config[key] = DEFAULT_CONFIG[key]\n else:\n config.pop(key, None)\n\n with open(CONFIG_PATH, 'w', encoding=\"utf-8\") as f:\n json.dump(config, f, indent=4)\n\n\ndef reset_all_config():\n \"\"\"Unset all user configuration and use default value instead.\"\"\"\n with open(CONFIG_PATH, 'w', encoding=\"utf-8\") as f:\n json.dump(DEFAULT_CONFIG, f, indent=4)\n\n\ndef get_config(key: str) -> str:\n \"\"\"Get configuration.\n\n Args:\n key (str, optional): Defaults to None.\n default (any, optional): Defaults to None.\n\n Returns:\n value, default_value: value of the key\n \"\"\"\n config = _read_and_create_file(CONFIG_PATH, DEFAULT_CONFIG)\n return config.get(key, \"\")\n\n\ndef get_config_dict() -> Dict[str, str]:\n config = _read_and_create_file(CONFIG_PATH, DEFAULT_CONFIG)\n return config\n\n\ndef get_all_config_str() -> str:\n config = _read_and_create_file(CONFIG_PATH, DEFAULT_CONFIG)\n return json.dumps(config, indent=4)\n\n\n@lru_cache(maxsize=1)\ndef get_local_user_id() -> str:\n return _read_and_create_file(\n USER_CONFIG_PATH,\n {\"user_id\": generate_hash_code()}\n ).get(\"user_id\", \"\")\n" }, { "path": "pygwalker/services/data_parsers.py", "content": "import sys\nimport hashlib\nimport pandas as pd\nfrom typing import Dict, Optional, Union, Any, List, Tuple\nfrom typing_extensions import Literal\n\nfrom pygwalker.data_parsers.base import BaseDataParser, FieldSpec\nfrom pygwalker.data_parsers.database_parser import Connector\nfrom pygwalker._typing import DataFrame\n\n__classname2method = {}\n\nDatasetType = Literal['pandas', 'polars', 'modin', 'pyspark', 'connector', 'cloud_dataset']\n\n\n# pylint: disable=import-outside-toplevel\ndef _get_data_parser(dataset: Union[DataFrame, Connector, str]) -> Tuple[BaseDataParser, DatasetType]:\n \"\"\"\n Get DataFrameDataParser for dataset\n TODO: Maybe you can find a better way to handle the following code\n \"\"\"\n if type(dataset) in __classname2method:\n return __classname2method[type(dataset)]\n\n if isinstance(dataset, pd.DataFrame):\n from pygwalker.data_parsers.pandas_parser import PandasDataFrameDataParser\n __classname2method[pd.DataFrame] = (PandasDataFrameDataParser, \"pandas\")\n return __classname2method[pd.DataFrame]\n\n if 'polars' in sys.modules:\n import polars as pl\n if isinstance(dataset, pl.DataFrame):\n from pygwalker.data_parsers.polars_parser import PolarsDataFrameDataParser\n __classname2method[pl.DataFrame] = (PolarsDataFrameDataParser, \"polars\")\n return __classname2method[pl.DataFrame]\n\n if 'modin.pandas' in sys.modules:\n from modin import pandas as mpd\n if isinstance(dataset, mpd.DataFrame):\n from pygwalker.data_parsers.modin_parser import ModinPandasDataFrameDataParser\n __classname2method[mpd.DataFrame] = (ModinPandasDataFrameDataParser, \"modin\")\n return __classname2method[mpd.DataFrame]\n\n if 'pyspark' in sys.modules:\n from pyspark.sql import DataFrame as SparkDataFrame\n if isinstance(dataset, SparkDataFrame):\n from pygwalker.data_parsers.spark_parser import SparkDataFrameDataParser\n __classname2method[SparkDataFrame] = (SparkDataFrameDataParser, \"pyspark\")\n return __classname2method[SparkDataFrame]\n\n if isinstance(dataset, Connector):\n from pygwalker.data_parsers.database_parser import DatabaseDataParser\n __classname2method[DatabaseDataParser] = (DatabaseDataParser, \"connector\")\n return __classname2method[DatabaseDataParser]\n\n if isinstance(dataset, str):\n from pygwalker.data_parsers.cloud_dataset_parser import CloudDatasetParser\n __classname2method[CloudDatasetParser] = (CloudDatasetParser, \"cloud_dataset\")\n return __classname2method[CloudDatasetParser]\n\n raise TypeError(f\"Unsupported data type: {type(dataset)}\")\n\n\ndef get_parser(\n dataset: Union[DataFrame, Connector, str],\n field_specs: Optional[List[FieldSpec]] = None,\n infer_string_to_date: bool = False,\n infer_number_to_dimension: bool = True,\n other_params: Optional[Dict[str, Any]] = None\n) -> BaseDataParser:\n if field_specs is None:\n field_specs = []\n if other_params is None:\n other_params = {}\n\n parser_func, _ = _get_data_parser(dataset)\n parser = parser_func(\n dataset,\n field_specs,\n infer_string_to_date,\n infer_number_to_dimension,\n other_params\n )\n return parser\n\n\ndef _get_pl_dataset_hash(dataset: DataFrame) -> str:\n \"\"\"Get polars dataset hash value.\"\"\"\n import polars as pl\n row_count = dataset.shape[0]\n other_info = str(dataset.shape) + \"_polars\"\n if row_count > 4000:\n dataset = pl.concat([dataset[:2000], dataset[-2000:]])\n hash_bytes = dataset.hash_rows().to_numpy().tobytes() + other_info.encode()\n return hashlib.md5(hash_bytes).hexdigest()\n\n\ndef _get_pd_dataset_hash(dataset: DataFrame) -> str:\n \"\"\"Get pandas dataset hash value.\"\"\"\n row_count = dataset.shape[0]\n other_info = str(dataset.shape) + \"_pandas\"\n if row_count > 4000:\n dataset = pd.concat([dataset[:2000], dataset[-2000:]])\n hash_bytes = pd.util.hash_pandas_object(dataset).values.tobytes() + other_info.encode()\n return hashlib.md5(hash_bytes).hexdigest()\n\n\ndef _get_modin_dataset_hash(dataset: DataFrame) -> str:\n \"\"\"Get modin dataset hash value.\"\"\"\n import modin.pandas as mpd\n row_count = dataset.shape[0]\n other_info = str(dataset.shape) + \"_modin\"\n if row_count > 4000:\n dataset = mpd.concat([dataset[:2000], dataset[-2000:]])\n dataset = dataset._to_pandas()\n hash_bytes = pd.util.hash_pandas_object(dataset).values.tobytes() + other_info.encode()\n return hashlib.md5(hash_bytes).hexdigest()\n\n\ndef _get_spark_dataset_hash(dataset: DataFrame) -> str:\n \"\"\"Get pyspark dataset hash value.\"\"\"\n shape = ((dataset.count(), len(dataset.columns)))\n row_count = shape[0]\n other_info = str(shape) + \"_pyspark\"\n if row_count > 4000:\n dataset = dataset.limit(4000)\n dataset_pd = dataset.toPandas()\n hash_bytes = pd.util.hash_pandas_object(dataset_pd).values.tobytes() + other_info.encode()\n return hashlib.md5(hash_bytes).hexdigest()\n\n\ndef get_dataset_hash(dataset: Union[DataFrame, Connector, str]) -> str:\n \"\"\"Just a less accurate way to get different dataset hash values.\"\"\"\n _, dataset_type = _get_data_parser(dataset)\n\n if dataset_type == \"polars\":\n return _get_pl_dataset_hash(dataset)\n\n if dataset_type == \"pandas\":\n return _get_pd_dataset_hash(dataset)\n\n if dataset_type == \"modin\":\n return _get_modin_dataset_hash(dataset)\n\n if dataset_type == \"pyspark\":\n return _get_spark_dataset_hash(dataset)\n\n if dataset_type == \"connector\":\n return hashlib.md5(\"_\".join([dataset.url, dataset.view_sql, dataset_type]).encode()).hexdigest()\n\n if dataset_type == \"cloud_dataset\":\n return hashlib.md5(\"_\".join([dataset, dataset_type]).encode()).hexdigest()\n" }, { "path": "pygwalker/services/fname_encodings.py", "content": "from typing import List\nfrom math import ceil\nfrom collections import defaultdict\n\n\ndef base36encode(s: str) -> str:\n \"\"\"Converts an string to a base36 string.\"\"\"\n alphabet = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ'\n number = int.from_bytes(s.encode(), \"big\")\n\n if not isinstance(number, int):\n raise TypeError('number must be an integer')\n\n base36 = ''\n\n if 0 <= number < len(alphabet):\n return alphabet[number]\n\n while number != 0:\n number, i = divmod(number, len(alphabet))\n base36 = alphabet[i] + base36\n\n return base36\n\n\ndef base36decode(s: str) -> str:\n \"\"\"Converts a base36 string to an string.\"\"\"\n number = int(s, 36)\n return number.to_bytes(ceil(number.bit_length() / 8), \"big\").decode()\n\n\ndef fname_encode(fname: str) -> str:\n \"\"\"Encode fname in base32\n\n Args:\n - fname (str): Suppose to be str\n\n Returns:\n str\n \"\"\"\n return \"GW_\" + base36encode(fname)\n\n\ndef fname_decode(encode_fname: str) -> str:\n \"\"\"Decode fname in base32\"\"\"\n return base36decode(encode_fname[3:])\n\n\ndef rename_columns(columns: List[str]) -> List[str]:\n \"\"\"rename columns to avoid duplicate column names\"\"\"\n column_map = defaultdict(lambda: 0)\n renamed_columns = []\n for col in columns:\n col = col.replace(\"\\\\\", \"-\")\n if column_map[col] == 0:\n renamed_columns.append(col)\n else:\n renamed_columns.append(f\"{col}_{column_map[col]}\")\n column_map[col] += 1\n return renamed_columns\n" }, { "path": "pygwalker/services/format_invoke_walk_code.py", "content": "from typing import Optional, List\nfrom types import FrameType\nimport logging\nimport inspect\nimport ast\n\nfrom astor.source_repr import split_lines\nimport astor\n\n_MAX_LINE = 150\n\nlogger = logging.getLogger(__name__)\n\n\ndef _find_walk_func_node(code: str) -> Optional['ast.Call']:\n node_list = [ast.parse(code)]\n while node_list:\n cur_node = node_list.pop()\n if isinstance(cur_node, ast.Call):\n if isinstance(cur_node.func, ast.Name):\n func_name = cur_node.func.id\n else:\n func_name = cur_node.func.attr\n if func_name == 'walk':\n return cur_node\n for node_info in astor.iter_node(cur_node):\n if isinstance(node_info[0], list):\n nodes = node_info[0]\n else:\n nodes = [node_info[0]]\n\n for children_node in nodes:\n node_list.append(children_node)\n\n\ndef _private_astor_pretty_source(source: List[str]) -> str:\n return \"\".join(split_lines(source, maxline=_MAX_LINE))\n\n\ndef _repalce_spec_params_code(func: 'ast.Call') -> str:\n replace_value = ast.Constant(value='____pyg_walker_spec_params____')\n spec_index = -1\n for index, keyword in enumerate(func.keywords):\n if keyword.arg == 'spec':\n spec_index = index\n if spec_index != -1:\n func.keywords[spec_index].value = replace_value\n else:\n func.keywords.insert(0, ast.keyword(arg='spec', value=replace_value))\n\n return astor.to_source(func, pretty_source=_private_astor_pretty_source)\n\n\ndef _get_default_code() -> str:\n return \"pyg.walk(df, spec='____pyg_walker_spec_params____')\"\n\n\ndef get_formated_spec_params_code(code: str) -> str:\n call_func = _find_walk_func_node(code.strip())\n if call_func is None:\n return ''\n return _repalce_spec_params_code(call_func)\n\n\ndef get_formated_spec_params_code_from_frame(frame: FrameType) -> str:\n try:\n source_invoke_code = get_formated_spec_params_code(\n inspect.getsource(frame).split(\"\\n\")[frame.f_lineno-1]\n )\n except Exception:\n return _get_default_code()\n\n if source_invoke_code == '':\n return _get_default_code()\n\n return source_invoke_code\n" }, { "path": "pygwalker/services/global_var.py", "content": "import os\n\nfrom pandas import DataFrame\nfrom typing_extensions import Literal, deprecated\n\nfrom .config import get_config\n\n\nclass GlobalVarManager:\n \"\"\"A class to manage global variables.\"\"\"\n env = None\n privacy = get_config(\"privacy\") or \"events\"\n kanaries_api_key = get_config(\"kanaries_token\") or os.getenv(\"KANARIES_API_KEY\", \"\")\n kanaries_api_host = \"https://api.kanaries.net\"\n kanaries_main_host = \"https://kanaries.net\"\n last_exported_dataframe = None\n max_data_length = 1000 * 1000\n component_url = \"\"\n # Feature flags for AI features (disabled by default)\n enable_askviz = os.getenv(\"PYGWALKER_ENABLE_ASKVIZ\", \"false\").lower() == \"true\"\n enable_vlchat = os.getenv(\"PYGWALKER_ENABLE_VLCHAT\", \"false\").lower() == \"true\"\n\n @classmethod\n def set_env(cls, env: Literal['Jupyter', 'Streamlit']):\n cls.env = env\n\n @classmethod\n def get_env(cls) -> Literal['Jupyter', 'Streamlit']:\n return cls.env\n\n @classmethod\n def set_kanaries_api_key(cls, api_key: str):\n cls.kanaries_api_key = api_key\n\n @classmethod\n def set_kanaries_api_host(cls, api_host: str):\n cls.kanaries_api_host = api_host\n\n @classmethod\n def set_kanaries_main_host(cls, main_host: str):\n cls.kanaries_main_host = main_host\n\n @classmethod\n def set_privacy(cls, privacy: Literal['offline', 'update-only', 'events']):\n cls.privacy = privacy\n\n @classmethod\n def set_last_exported_dataframe(cls, df: DataFrame):\n cls.last_exported_dataframe = df\n\n @classmethod\n @deprecated(\"use ui config instead.\")\n def set_max_data_length(cls, length: int):\n cls.max_data_length = length\n\n @classmethod\n def set_component_url(cls, url: str):\n cls.component_url = url\n" }, { "path": "pygwalker/services/kaggle.py", "content": "from functools import lru_cache\n\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom pygwalker.utils.display import display_html\n\n\n# pylint: disable=import-outside-toplevel\n# pylint: disable=broad-exception-caught\ndef auto_set_kanaries_api_key_on_kaggle():\n \"\"\"Auto set kanaries api key on kaggle.\"\"\"\n from kaggle_secrets import UserSecretsClient\n if not GlobalVarManager.kanaries_api_key:\n try:\n GlobalVarManager.set_kanaries_api_key(\n UserSecretsClient().get_secret(\"kanaries_api_key\")\n )\n except Exception:\n pass\n\n\n@lru_cache()\ndef adjust_kaggle_default_font_size():\n \"\"\"Adjust kaggle default font size.\"\"\"\n display_html(\"\"\"\"\"\")\n\n\ndef show_tips_user_kaggle() -> bool:\n \"\"\"Whether has set kanaries api key.\"\"\"\n from kaggle_secrets import UserSecretsClient\n try:\n has_set_api_key = bool(UserSecretsClient().get_secret(\"kanaries_api_key\"))\n except Exception:\n has_set_api_key = False\n\n tips = \"\"\"\n

Since you haven't set the kannaries_api_key, Pygwalker assumes it's your first time using it on Kaggle.

\n

Due to the persistent file approach in Kaggle, there may be some impact on the user experience when using Pygwalker on Kaggle.

\n

Please take a moment to read this article: Best Practices for Using Pygwalker in Kaggle, which can help you optimize your Pygwalker experience on Kaggle.

\n \"\"\"\n\n if not has_set_api_key:\n display_html(tips)\n" }, { "path": "pygwalker/services/kanaries_cli_login.py", "content": "from http.server import BaseHTTPRequestHandler, HTTPServer\nfrom typing import Any\nfrom urllib.parse import urlparse, parse_qs, quote\nfrom threading import Thread, Lock\nimport webbrowser\n\nfrom pygwalker.services.config import set_config\nfrom pygwalker.utils.free_port import find_free_port\n\nAUTH_HOST = \"https://kanaries.net\"\nauth_info = {}\nwait_lock = Lock()\n\n\nclass TextStyle:\n RESET = '\\033[0m'\n GREEN = '\\033[32m'\n RED = '\\033[31m'\n UNDERLINE = '\\033[4m'\n\n\nclass _CallbackHandler(BaseHTTPRequestHandler):\n \"\"\"A simple HTTP request handler to process the callback from the OAuth server\"\"\"\n\n def log_message(self, _, *args: Any) -> None:\n pass\n\n def do_GET(self):\n parsed_url = urlparse(self.path)\n query_params = parse_qs(parsed_url.query)\n api_key = query_params.get('apiKey', [''])[0]\n user_name = query_params.get('username', [''])[0]\n workspace_name = query_params.get('workspaceName', [''])[0]\n auth_as = quote(f\"workspace: {workspace_name}, user: {user_name}\")\n\n if api_key:\n set_config({\"kanaries_token\": api_key})\n self.send_response(302)\n self.send_header('Location', f\"{AUTH_HOST}/home/cli/success?authedAs={auth_as}\")\n self.end_headers()\n auth_info[\"user_name\"] = user_name\n auth_info[\"workspace_name\"] = workspace_name\n wait_lock.release()\n else:\n self.send_response(404)\n self.end_headers()\n self.wfile.write(bytes(\"auth error, please re-auth\", \"utf-8\"))\n\n\ndef _run_callback_server(port: int):\n server_address = ('localhost', port)\n httpd = HTTPServer(server_address, _CallbackHandler)\n httpd.serve_forever()\n\n\ndef kanaries_login():\n wait_lock.acquire()\n\n port = find_free_port()\n callback_server = Thread(target=_run_callback_server, args=(port,), daemon=True)\n callback_server.start()\n\n callback_url = f'http://localhost:{port}'\n auth_url = f\"{AUTH_HOST}/home/cli?redirect_url={quote(callback_url)}\"\n\n print(f'Please visit {TextStyle.GREEN}{auth_url}{TextStyle.RESET} to log in.')\n print('Waiting for authorization...')\n webbrowser.open_new(auth_url)\n\n wait_flag = wait_lock.acquire(blocking=True, timeout=300)\n if not wait_flag:\n print(f'{TextStyle.RED}Authorization timeout.{TextStyle.RESET}')\n return\n\n print((\n f'{TextStyle.GREEN}Authorization success and kanaries token is configured!{TextStyle.RESET}\\n'\n f'user: {TextStyle.UNDERLINE}{auth_info[\"user_name\"]}{TextStyle.RESET}\\n'\n f'workspace: {TextStyle.UNDERLINE}{auth_info[\"workspace_name\"]}{TextStyle.RESET}'\n ))\n" }, { "path": "pygwalker/services/preview_image.py", "content": "from typing import List, Dict, Any\nfrom concurrent.futures.thread import ThreadPoolExecutor\nimport base64\nimport zlib\nimport json\n\nfrom pydantic import BaseModel, Field\nfrom ipylab import JupyterFrontEnd\n\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker.utils.display import display_html\nfrom pygwalker.utils.randoms import generate_hash_code\nfrom pygwalker.services.render import jinja_env, GWALKER_SCRIPT_BASE64, compress_data\n\n\nclass ImgData(BaseModel):\n row_index: int = Field(..., alias=\"rowIndex\")\n col_index: int = Field(..., alias=\"colIndex\")\n data: str\n height: int\n width: int\n canvas_height: int = Field(..., alias=\"canvasHeight\")\n canvas_width: int = Field(..., alias=\"canvasWidth\")\n\n\nclass ChartData(BaseModel):\n charts: List[ImgData]\n single_chart: str = Field(..., alias=\"singleChart\")\n n_rows: int = Field(..., alias=\"nRows\")\n n_cols: int = Field(..., alias=\"nCols\")\n title: str\n\n\ndef render_gw_preview_html(\n vis_spec_obj: List[Dict[str, Any]],\n datas: List[List[Dict[str, Any]]],\n theme_key: str,\n gid: str,\n appearance: str,\n) -> str:\n \"\"\"\n Render html for previewing gwalker(use purerenderer mode of graphic-wlaker, not png preview)\n \"\"\"\n charts = []\n for vis_spec_item, data in zip(\n vis_spec_obj,\n datas\n ):\n charts.append({\n \"visSpec\": vis_spec_item,\n \"data\": data\n })\n\n props = {\"charts\": charts, \"themeKey\": theme_key, \"dark\": appearance, \"gid\": gid}\n\n container_id = f\"pygwalker-preview-{gid}\"\n template = jinja_env.get_template(\"pygwalker_main_page.html\")\n html = template.render(\n gwalker={\n 'id': container_id,\n 'gw_script': GWALKER_SCRIPT_BASE64,\n \"component_script\": \"PyGWalkerApp.PreviewApp(props, gw_id);\",\n \"props\": compress_data(json.dumps(props, cls=DataFrameEncoder))\n },\n component_url=\"\"\n )\n return html\n\n\ndef render_gw_chart_preview_html(\n *,\n single_vis_spec: Dict[str, Any],\n data: List[Dict[str, Any]],\n theme_key: str,\n title: str,\n desc: str,\n appearance: str,\n) -> str:\n \"\"\"\n Render html for single chart(use purerenderer mode of graphic-wlaker, not png preview)\n \"\"\"\n\n props = {\n \"visSpec\": single_vis_spec,\n \"data\": data,\n \"themeKey\": theme_key,\n \"title\": title,\n \"desc\": desc,\n \"dark\": appearance,\n }\n\n container_id = f\"pygwalker-chart-preview-{generate_hash_code()[:20]}\"\n template = jinja_env.get_template(\"pygwalker_main_page.html\")\n html = template.render(\n gwalker={\n 'id': container_id,\n 'gw_script': GWALKER_SCRIPT_BASE64,\n \"component_script\": \"PyGWalkerApp.ChartPreviewApp(props, gw_id);\",\n \"props\": compress_data(json.dumps(props, cls=DataFrameEncoder))\n },\n component_url=\"\"\n )\n return html\n\n\nclass PreviewImageTool:\n \"\"\"Preview image tool for pygwalker\"\"\"\n def __init__(self, gid: str):\n self.gid = gid\n self.image_slot_id = f\"pygwalker-preview-{gid}\"\n self.t_pool = ThreadPoolExecutor(1)\n try:\n self.command_app = JupyterFrontEnd()\n except Exception:\n self.command_app = None\n\n def init_display(self):\n display_html(\"\", slot_id=self.image_slot_id)\n\n def render_gw_review(self, html: str):\n display_html(html, slot_id=self.image_slot_id)\n\n if self.command_app:\n try:\n self.command_app.commands.execute('docmanager:save')\n except Exception:\n pass\n\n def async_render_gw_review(self, html: str):\n self.t_pool.submit(self.render_gw_review, html)\n" }, { "path": "pygwalker/services/render.py", "content": "import os\nimport json\nimport base64\nimport html as m_html\nfrom typing import Dict, List, Any, Optional\nimport zlib\n\nfrom jinja2 import Environment, PackageLoader\n\nfrom pygwalker._typing import IAppearance\nfrom pygwalker._constants import ROOT_DIR\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker.utils.estimate_tools import estimate_average_data_size\nfrom pygwalker.services.global_var import GlobalVarManager\n\njinja_env = Environment(\n loader=PackageLoader(\"pygwalker\"),\n autoescape=(()), # select_autoescape()\n)\n\n\ndef compress_data(data: str) -> str:\n compress = zlib.compressobj(zlib.Z_BEST_COMPRESSION, zlib.DEFLATED, 15, 8, 0)\n compressed_data = compress.compress(data.encode())\n compressed_data += compress.flush()\n return base64.b64encode(compressed_data).decode()\n\n\nwith open(os.path.join(ROOT_DIR, 'templates', 'dist', 'pygwalker-app.iife.js'), 'r', encoding='utf8') as f:\n GWALKER_SCRIPT = f.read()\n GWALKER_SCRIPT_BASE64 = compress_data(GWALKER_SCRIPT)\n\n\ndef get_max_limited_datas(datas: List[Dict[str, Any]], byte_limit: int) -> List[Dict[str, Any]]:\n if len(datas) > 1024:\n avg_size = estimate_average_data_size(datas)\n n = int(byte_limit / avg_size)\n if len(datas) >= 2 * n:\n return datas[:n]\n return datas\n\n\ndef render_iframe_messages_html(gid: str) -> str:\n return jinja_env.get_template(\"jupyter_iframe_message.html\").render(gid=gid)\n\n\ndef render_gwalker_iframe(\n gid: int,\n html: str,\n width: Optional[str] = None,\n height: Optional[str] = None,\n appearance: IAppearance = \"media\",\n) -> str:\n if height is None:\n height = \"960px\"\n if width is None:\n width = \"100%\"\n\n return jinja_env.get_template(\"pygwalker_iframe.html\").render(\n gid=gid,\n srcdoc=m_html.escape(html),\n height=height,\n width=width,\n appearance=appearance,\n component_url=GlobalVarManager.component_url\n )\n\n\ndef render_gwalker_html(gid: int, props: Dict[str, Any]) -> str:\n container_id = f\"gwalker-div-{gid}\"\n template = jinja_env.get_template(\"pygwalker_main_page.html\")\n html = template.render(\n gwalker={\n 'id': container_id,\n 'gw_script': GWALKER_SCRIPT_BASE64,\n \"component_script\": \"PyGWalkerApp.GWalker(props, gw_id);\",\n \"props\": compress_data(json.dumps(props, cls=DataFrameEncoder)),\n },\n component_url=GlobalVarManager.component_url\n )\n return html\n" }, { "path": "pygwalker/services/spec.py", "content": "from urllib import request\nfrom typing import Tuple, Dict, Any, List, Union\nfrom packaging.version import Version\nfrom copy import deepcopy\nimport json\nimport os\n\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom pygwalker.utils.randoms import rand_str\nfrom pygwalker.services.fname_encodings import rename_columns\nfrom pygwalker.services.cloud_service import read_config_from_cloud\nfrom pygwalker.errors import InvalidConfigIdError, PrivacyError\n\n\ndef _is_json(s: str) -> bool:\n try:\n json.loads(s)\n except ValueError:\n return False\n return True\n\n\ndef _get_spec_from_server(config_id: str) -> str:\n url = f\"https://i4rwxmw117.execute-api.us-east-1.amazonaws.com/default/pygwalker-config?config_id={config_id}\"\n with request.urlopen(url, timeout=30) as resp:\n json_data = json.loads(resp.read().decode(\"utf-8\"))\n\n if json_data[\"code\"] != 0:\n raise InvalidConfigIdError(f\"Invalid config id: {config_id}\")\n\n return json_data[\"data\"][\"config_json\"]\n\n\ndef _get_spec_from_url(url: str) -> str:\n with request.urlopen(url, timeout=15) as resp:\n return resp.read().decode(\"utf-8\")\n\n\ndef _get_spec_from_local(path: str) -> str:\n with open(path, \"r\", encoding=\"utf-8\") as f:\n return f.read()\n\n\ndef _is_config_id(config_id: str) -> bool:\n if len(config_id) != 32:\n return False\n try:\n int(config_id, 16)\n except ValueError:\n return False\n\n return True\n\n\ndef _get_spec_json_from_diff_source(spec: str) -> Tuple[str, str]:\n if not spec:\n return \"\", \"empty_string\"\n\n if _is_json(spec):\n return spec, \"json_string\"\n\n if spec.startswith(\"ksf://\"):\n if GlobalVarManager.privacy == \"offline\":\n raise PrivacyError(\"Due to privacy policy, you can't use this spec offline\")\n return read_config_from_cloud(spec[6:]), \"json_ksf\"\n\n if spec.startswith((\"http:\", \"https:\")):\n if GlobalVarManager.privacy == \"offline\":\n raise PrivacyError(\"Due to privacy policy, you can't use this spec offline\")\n return _get_spec_from_url(spec), \"json_http\"\n\n if _is_config_id(spec):\n if GlobalVarManager.privacy == \"offline\":\n raise PrivacyError(\"Due to privacy policy, you can't use this spec offline\")\n return _get_spec_from_server(spec), \"json_server\"\n\n if len(os.path.basename(spec)) > 200:\n raise ValueError(\"Spec file name too long\")\n\n file_exist = os.path.exists(spec)\n if file_exist:\n return _get_spec_from_local(spec), \"json_file\"\n else:\n with open(spec, \"w\", encoding=\"utf-8\") as f:\n f.write(\"\")\n return \"\", \"json_file\"\n\n\ndef _config_adapter(config: str) -> str:\n config_obj = json.loads(config)\n for chart_item in config_obj:\n old_fid_fname_map = {\n field[\"fid\"]: field[\"name\"]\n for field in chart_item[\"encodings\"][\"dimensions\"] + chart_item[\"encodings\"][\"measures\"]\n if not field.get(\"computed\", False) and field.get(\"fid\") not in [\"gw_mea_val_fid\", \"gw_mea_key_fid\"]\n }\n old_fid_list = []\n fname_list = []\n for old_fid, fname in old_fid_fname_map.items():\n old_fid_list.append(old_fid)\n fname_list.append(fname)\n\n new_fid_list = rename_columns(fname_list)\n for old_fid, new_fid in zip(old_fid_list, new_fid_list):\n config = config.replace(old_fid, new_fid)\n\n return config\n\n\ndef fill_new_fields(config: List[Dict[str, Any]], all_fields: List[Dict[str, str]]) -> List[Dict[str, Any]]:\n \"\"\"when df schema changed, fill new fields to every chart config\"\"\"\n config = deepcopy(config)\n for chart_item in config:\n field_set = {\n field[\"fid\"]\n for field in chart_item[\"encodings\"][\"dimensions\"] + chart_item[\"encodings\"][\"measures\"]\n }\n new_dimension_fields = []\n new_measure_fields = []\n for field in all_fields:\n if field[\"fid\"] not in field_set:\n gw_field = {\n **field,\n \"basename\": field[\"name\"],\n \"dragId\": \"GW_\" + rand_str()\n }\n if field[\"analyticType\"] == \"dimension\":\n new_dimension_fields.append(gw_field)\n else:\n new_measure_fields.append(gw_field)\n\n chart_item[\"encodings\"][\"dimensions\"].extend(new_dimension_fields)\n chart_item[\"encodings\"][\"measures\"].extend(new_measure_fields)\n return config\n\n\ndef _config_adapter_045a5(config: List[Dict[str, Any]]):\n config = deepcopy(config)\n\n for chart_item in config:\n if \"config\" in chart_item and chart_item[\"config\"].get(\"timezoneDisplayOffset\", None) is None:\n chart_item[\"config\"][\"timezoneDisplayOffset\"] = 0\n\n for item_list in chart_item[\"encodings\"].values():\n for item in item_list:\n item[\"offset\"] = 0\n if isinstance(item.get(\"expression\", {}).get(\"params\"), list):\n for param in item[\"expression\"][\"params\"]:\n if param.get(\"type\") == \"offset\":\n param[\"value\"] = 0\n\n return config\n\n\ndef _is_gw_config(config: Dict[str, Any]) -> bool:\n return not bool({\"config\", \"encodings\", \"visId\"} - set(config.keys()))\n\n\ndef _is_pygwalker_config(config: Dict[str, Any]) -> bool:\n return \"config\" in config and isinstance(config[\"config\"], (list, str))\n\n\ndef get_spec_json(spec: Union[str, List[Any], Dict[str, Any]]) -> Tuple[Dict[str, Any], str]:\n if isinstance(spec, str):\n spec, spec_type = _get_spec_json_from_diff_source(spec)\n if not spec:\n return {\"chart_map\": {}, \"config\": [], \"workflow_list\": []}, spec_type\n\n try:\n spec_obj = json.loads(spec)\n except json.decoder.JSONDecodeError as e:\n raise ValueError(\"spec is not a valid json\") from e\n else:\n spec_obj = spec\n spec_type = \"json_obj\"\n\n if isinstance(spec_obj, list):\n if spec_obj and not _is_gw_config(spec_obj[0]):\n return {\"chart_map\": {}, \"config\": spec_obj, \"workflow_list\": []}, \"vega_list\"\n else:\n spec_obj = {\"chart_map\": {}, \"config\": json.dumps(spec_obj), \"workflow_list\": []}\n\n if isinstance(spec_obj, dict) and not _is_pygwalker_config(spec_obj):\n return {\"chart_map\": {}, \"config\": [spec_obj], \"workflow_list\": []}, \"vega_single\"\n\n if Version(spec_obj.get(\"version\", \"0.1.0\")) <= Version(\"0.3.17a4\"):\n spec_obj[\"config\"] = _config_adapter(spec_obj[\"config\"])\n \n\n if isinstance(spec_obj[\"config\"], str):\n spec_obj[\"config\"] = json.loads(spec_obj[\"config\"])\n\n if Version(spec_obj.get(\"version\", \"0.1.0\")) <= Version(\"0.4.7a5\"):\n spec_obj[\"config\"] = _config_adapter_045a5(spec_obj[\"config\"])\n\n return spec_obj, spec_type\n" }, { "path": "pygwalker/services/streamlit_components.py", "content": "from typing import Dict, Any\nimport os\n\nfrom streamlit.components.v1.components import CustomComponent\nimport streamlit.components.v1 as components\n\nfrom pygwalker._constants import ROOT_DIR\n\n_build_dir = os.path.join(ROOT_DIR, \"templates\")\n_component_func = components.declare_component(\"pygwalker_component\", path=_build_dir)\n\n\ndef pygwalker_component(props: Dict[str, Any], key: str) -> CustomComponent:\n return _component_func(\n key=key,\n **props,\n )\n" }, { "path": "pygwalker/services/tip_tools.py", "content": "from threading import Thread\nimport time\n\nfrom pygwalker.utils.display import display_html\n\nWIDGETS_TIPS = \"\"\"\n
\nIf you are using pygwalker on Jupyter Notebook(version<7) and it can't display properly, please execute code to fix it: `pip install \"pygwalker[notebook]\" --pre`.(close after 15 seconds)\n
\n\"\"\"\n\nTIPS_MAP = {\n \"widgets\": WIDGETS_TIPS\n}\n\n\nclass TipOnStartTool:\n \"\"\"Tip on start tool for pygwalker\"\"\"\n\n def __init__(self, gid: str, tip_name: str):\n self.gid = gid\n self.slot_id = f\"user-tips-{gid}\"\n self.tips = TIPS_MAP.get(tip_name, \"\")\n Thread(target=self.hide).start()\n\n def show(self):\n display_html(self.tips, slot_id=self.slot_id)\n\n def hide(self):\n time.sleep(15)\n display_html(\"\", slot_id=self.slot_id)\n" }, { "path": "pygwalker/services/track.py", "content": "from typing import Dict, Any, Optional\n\nimport segment.analytics as analytics\nimport kanaries_track\n\nfrom pygwalker.services.global_var import GlobalVarManager\nfrom pygwalker.services.config import get_local_user_id\n\nanalytics.write_key = 'z58N15R8LShkpUbBSt1ZjdDSdSEF5VpR'\nkanaries_public_key = \"tk-6572d7b34a03d7fcf6cf0c86-cOzZyr6xqd\"\nkanaries_track.config.auth_token = kanaries_public_key\nkanaries_track.config.proxies = {}\nkanaries_track.config.max_retries = 2\n\n\n# pylint: disable=broad-exception-caught\ndef track_event(event: str, properties: Optional[Dict[str, Any]] = None):\n \"\"\"\n Track an event in Segment and Kanaries.\n When privacy config of user is 'events',\n PyGWalker will collect certain events data share which events about which feature is used in pygwalker, it only contains events tag about which feature you arrive for product optimization. No DATA YOU ANALYZE IS SENT.\n We only use these data to improve the user experience of pygwalker. Events data will bind with a unique id, which is generated by pygwalker when it is installed based on timestamp. We will not collect any other information about you.\n EXAMPLE:\n - pygwalker's version\n - pygwalker's mode: 'light', 'dark' or 'auto'\n - pygwalker's spec type: 'json', 'file', 'url'. We won't collect the exact value of spec. No DATA YOU ANALYZE OR THEIR METADATA IS COLLECTED.\n\n - privacy ['offline', 'update-only', 'events'] (default: events).\n \"offline\": fully offline, no data is send or api is requested\n \"update-only\": only check whether this is a new version of pygwalker to update\n \"events\": share which events about which feature is used in pygwalker, it only contains events data about which feature you arrive for product optimization. No DATA YOU ANALYZE IS SENT.\n \"\"\"\n if GlobalVarManager.privacy == \"events\":\n try:\n analytics.track(\n user_id=get_local_user_id(),\n event=event,\n properties=properties\n )\n kanaries_track.track({**properties, \"user_id\": get_local_user_id()})\n except Exception:\n pass\n" }, { "path": "pygwalker/services/upload_data.py", "content": "from typing import Dict, Any, List\nimport time\nimport json\nimport html as m_html\n\nfrom pygwalker.utils.randoms import rand_str\nfrom pygwalker.utils.display import display_html\nfrom pygwalker.utils.encode import DataFrameEncoder\nfrom pygwalker.communications.base import BaseCommunication\nfrom pygwalker import __hash__\n\n\ndef _send_js(js_code: str, slot_id: str):\n display_html(\n f\"\"\"