Repository: okfn/opendataeditor Branch: main Commit: 957c79351781 Files: 104 Total size: 415.0 KB Directory structure: gitextract_cjudfkyp/ ├── .github/ │ ├── issue_template.md │ ├── pull_request_template.md │ ├── stale.yaml │ └── workflows/ │ └── general.yaml ├── .gitignore ├── .python-version ├── LICENSE.md ├── README.md ├── build.py ├── create-deb.sh ├── create-dmg.sh ├── docs/ │ ├── Makefile │ ├── make.bat │ ├── public/ │ │ └── .gitkeep │ ├── requirements.txt │ └── source/ │ ├── conf.py │ ├── contributing/ │ │ ├── contribution-guidelines.md │ │ └── translations.md │ ├── index.rst │ ├── introduction/ │ │ ├── acknowledgements.md │ │ ├── fair-data.md │ │ ├── free-data-literacy-course.md │ │ ├── latest-updates.md │ │ ├── responsible-ai-integration.md │ │ ├── similar-tools-and-differentiators.md │ │ └── what-is-open-data-editor.md │ ├── technical-documentation/ │ │ ├── building-the-application.md │ │ ├── documentation.md │ │ ├── environment.md │ │ ├── making-a-release.md │ │ ├── prerequisites.md │ │ ├── running-tests.md │ │ └── start-the-application.md │ ├── use-cases/ │ │ ├── agricultural-data-ghana.md │ │ ├── climate-data-kenya.md │ │ ├── context.md │ │ ├── data-journalism-mexico.md │ │ ├── defence-data-france.md │ │ ├── financial-data-south-africa.md │ │ ├── government-data-croatia.md │ │ ├── heritage-data-cambodia.md │ │ └── library-data-india.md │ └── user-guide/ │ ├── assets/ │ │ └── table-error-list/ │ │ ├── column-name-missing.csv │ │ ├── duplicate-column-name.csv │ │ ├── empty-row.csv │ │ ├── extra-cell.csv │ │ ├── header-missing.csv │ │ └── wrong-data-type.csv │ ├── deleting-files-or-folders.md │ ├── downloading-ode.md │ ├── editing-errors-in-tables.md │ ├── exporting-your-data.md │ ├── full-list-of-table-errors-detected.md │ ├── how-to-explore-and-edit-metadata.md │ ├── how-to-explore-table-errors.md │ ├── how-to-use-the-ai-component.md │ ├── installing-ode.md │ └── uploading-data.md ├── packaging/ │ ├── linux/ │ │ └── opendataeditor.desktop │ ├── macos/ │ │ ├── entitlements.mac.plist │ │ └── icon.icns │ └── windows/ │ └── installer.nsi ├── pyproject.sublime-workspace ├── pyproject.toml ├── src/ │ └── ode/ │ ├── __init__.py │ ├── assets/ │ │ ├── __init__.py │ │ ├── licenses.json │ │ ├── style.qss │ │ └── translations/ │ │ ├── de.qm │ │ ├── de.ts │ │ ├── es.qm │ │ ├── es.ts │ │ ├── fr.qm │ │ ├── fr.ts │ │ ├── it.qm │ │ ├── it.ts │ │ ├── pt.qm │ │ └── pt.ts │ ├── dialogs/ │ │ ├── __init__.py │ │ ├── delete.py │ │ ├── download.py │ │ ├── llm_dialog_warning.py │ │ ├── loading.py │ │ ├── metadata.py │ │ ├── rename.py │ │ └── upload.py │ ├── file.py │ ├── llama.py │ ├── log_setup.py │ ├── main.py │ ├── panels/ │ │ ├── __init__.py │ │ ├── data.py │ │ ├── errors.py │ │ └── source.py │ ├── paths.py │ ├── shared.py │ └── utils.py └── tests/ ├── __init__.py ├── conftest.py └── ode/ ├── __init__.py ├── test_application.py ├── test_files.py ├── test_frictionless_errors.py └── test_paths.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/issue_template.md ================================================ # Overview Please replace this line with full information about your idea or problem. If it's a bug share as much as possible to reproduce it ================================================ FILE: .github/pull_request_template.md ================================================ - fixes # --- Please make sure that all the checks pass. Please add here any additional information regarding this pull request. It's highly recommended that you link this PR to an issue (please create one if it doesn't exist for this PR) ================================================ FILE: .github/stale.yaml ================================================ # Number of days of inactivity before an issue becomes stale daysUntilStale: 90 # Number of days of inactivity before a stale issue is closed daysUntilClose: 30 # Issues with these labels will never be considered stale exemptLabels: - feature - enhancement - bug # Label to use when marking an issue as stale staleLabel: wontfix # Comment to post when marking an issue as stale. Set to `false` to disable markComment: > This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. # Comment to post when closing a stale issue. Set to `false` to disable closeComment: false ================================================ FILE: .github/workflows/general.yaml ================================================ name: general on: push: branches: - main tags: - v*.*.* pull_request: branches: - main env: # Mandatory when using uv pip workflow. UV_SYSTEM_PYTHON: 1 jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Check pep8 run: | uvx ruff check tests: needs: lint runs-on: ubuntu-latest env: QT_QPA_PLATFORM: 'offscreen' steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Install QT Libs and Dependencies uses: tlambert03/setup-qt-libs@v1 - name: Install python requirements run: | uv sync - name: Check pep8 run: | uvx ruff check - name: Run tests run: | uv run pytest macos-packaging: if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/') needs: tests runs-on: macos-13 steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Install Dependencies run: | uv sync brew install create-dmg - name: Build and notarize the dmg file env: CSC_LINK: ${{ secrets.CSC_LINK }} CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }} APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }} APPLE_ID: ${{ secrets.APPLE_ID }} APPLE_APP_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }} run: | chmod +x create-dmg.sh ./create-dmg.sh - name: Archive build artifacts uses: actions/upload-artifact@v4 with: name: distribution-files-macos path: | *.dmg retention-days: 14 linux-packaging: if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/') needs: tests runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Install OS Dependencies run: | sudo apt-get update sudo gem install fpm fpm --version - name: Install Qt Dependencies run: | # https://forum.qt.io/post/769050 # Fix PyInstaller warnings of Qt Dependencies missing sudo apt-get install synaptic sudo apt-get install libxcb-icccm4 libxcb-image0-dev libxcb-keysyms1 libxcb-render-util0 libxcb-xkb1 libxcb-xinerama0 libxkbcommon-x11-0 libxcb-cursor0 libxcb-shape0-dev - name: Install Dependencies run: | uv sync - name: Build the deb package run: | chmod +x create-deb.sh ./create-deb.sh - name: Archive build artifacts uses: actions/upload-artifact@v4 with: name: distribution-files-deb path: | dist/*.deb retention-days: 14 windows-packaging: if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/') needs: tests runs-on: windows-2022 steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Install Dependencies run: | uv sync - name: Install NSIS run: choco install nsis - name: Build with PyInstaller run: uv run build.py build - name: Compile installer shell: bash # Force bash shell for VERSION command run: | VERSION=$(uv run python -c "from importlib.metadata import version; print(version('opendataeditor'))") makensis -DAPP_VERSION="$VERSION" ./packaging/windows/installer.nsi - name: Upload artifact uses: actions/upload-artifact@v4 with: name: distribution-files-win path: | .\packaging\windows\*.exe retention-days: 14 linux-app-image: if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/') needs: tests runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 - name: Install uv uses: astral-sh/setup-uv@v5 with: version: "0.9.9" - name: Set up Python 3.13 uses: actions/setup-python@v3 with: python-version: "3.13" - name: Install Qt Dependencies run: | # https://forum.qt.io/post/769050 # Fix PyInstaller warnings of Qt Dependencies missing sudo apt-get update sudo apt-get install synaptic sudo apt-get install libxcb-icccm4 libxcb-image0-dev libxcb-keysyms1 libxcb-render-util0 libxcb-xkb1 libxcb-xinerama0 libxkbcommon-x11-0 - name: Install Dependencies run: | uv sync - name: Build the deb package run: | VERSION=$(uv run python -c "from importlib.metadata import version; print(version('opendataeditor'))") uv run build.py build --onefile --name "opendataeditor-$VERSION.AppImage" - name: Archive build artifacts uses: actions/upload-artifact@v4 with: name: distribution-files-app-image path: | dist/*.AppImage retention-days: 14 ================================================ FILE: .gitignore ================================================ venv data/ __pycache__ build/ dist/ tmp/ # Astro node_modules/ .astro package-lock.json /.venv .DS_Store ================================================ FILE: .python-version ================================================ 3.13 ================================================ FILE: LICENSE.md ================================================ The MIT License (MIT) Copyright (c) 2022 Open Knowledge Foundation Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ [![Build](https://img.shields.io/github/actions/workflow/status/frictionlessdata/application/general.yaml?branch=main)](https://github.com/frictionlessdata/application/actions) [![Codebase](https://img.shields.io/badge/codebase-github-brightgreen)](https://github.com/frictionlessdata/application) ![ODE-landscape-full-rgb@3x](https://github.com/okfn/opendataeditor/assets/20649846/01ae62e8-87f6-4e44-9487-790b8111e321) # Open Data Editor (beta) ### Welcome to our Readme! The Open Data Editor (ODE) is a **no-code application** to **explore, validate and publish data** in a simple way. Forever free and **open source project** powered by the **Frictionless Framework**. 📩 [Send us feedback/Report a problem (email)](mailto:info@okfn.org) 🪲 [Create an issue on GitHub](https://github.com/okfn/opendataeditor/issues) 🤔 [Suggest a new feature](https://github.com/okfn/opendataeditor/issues) # Useful links 🔵 [Development guide](https://opendataeditor.okfn.org/contributing/development/) 🔵 [Open Data Editor User Guide and Project Documentation](https://opendataeditor.okfn.org/) 🔵 [Frictionless Framework](https://framework.frictionlessdata.io/) 🔵 [Frictionless Data](https://frictionlessdata.io/) 🔵 [Contributing Guidelines](https://opendataeditor.okfn.org/contributing/contribution-guidelines) 🔵 [Open Data Editor Concept Note](https://opendataeditor.okfn.org/ode-concept-note.pdf) 🔵 For all contributions: [Code of conduct](https://frictionlessdata.io/code-of-conduct/) # How to download the ODE You can download the latest version from the [ODE website](https://okfn.org/en/projects/open-data-editor/) For previous releases, you can download them from Github [RELEASE](https://github.com/okfn/opendataeditor/releases) * For **Windows**:Download the most recent **EXE** file. * For **MacOS**:Download the most recent **DMG** file. * For **Linux**:Download the most recent **AppImage** or **DEB** file. ================================================ FILE: build.py ================================================ import os import platform import PyInstaller.__main__ import subprocess import sys def run(cmd: list[str], cwd: str = "."): """Run a subprocess command.""" subprocess.run(cmd, check=True, cwd=cwd) def docs(): """Build the documentation and run a local server.""" run(["make", "html"], cwd="docs") run(["python", "-m", "http.server", "-d", "build/html"], cwd="docs") def update_translations(): """Generate/update .ts files from the source code.""" run(["pyside6-lupdate", "-extensions", "py", "-recursive", "ode", "-ts", "ode/assets/translations/de.ts", "-target-language", "de_DE"]) run(["pyside6-lupdate", "-extensions", "py", "-recursive", "ode", "-ts", "ode/assets/translations/es.ts", "-target-language", "es_ES"]) run(["pyside6-lupdate", "-extensions", "py", "-recursive", "ode", "-ts", "ode/assets/translations/fr.ts", "-target-language", "fr_FR"]) run(["pyside6-lupdate", "-extensions", "py", "-recursive", "ode", "-ts", "ode/assets/translations/pt.ts", "-target-language", "pt_PT"]) run(["pyside6-lupdate", "-extensions", "py", "-recursive", "ode", "-ts", "ode/assets/translations/it.ts", "-target-language", "it_IT"]) def compile_translations(): """Compile .ts to .qm files.""" run(["pyside6-lrelease", "ode/assets/translations/de.ts", "-qm", "ode/assets/translations/de.qm"]) run(["pyside6-lrelease", "ode/assets/translations/es.ts", "-qm", "ode/assets/translations/es.qm"]) run(["pyside6-lrelease", "ode/assets/translations/fr.ts", "-qm", "ode/assets/translations/fr.qm"]) run(["pyside6-lrelease", "ode/assets/translations/pt.ts", "-qm", "ode/assets/translations/pt.qm"]) run(["pyside6-lrelease", "ode/assets/translations/it.ts", "-qm", "ode/assets/translations/it.qm"]) def build_application(): """Build an executable file for the Application.""" system = platform.system() # Linux Defaults icon_path = "packaging/linux/icon.svg" app_name = "opendataeditor" if system == "Darwin": # macOS icon_path = "packaging/macos/icon.icns" app_name = "OpenDataEditor" elif system == "Windows": icon_path = "packaging/windows/icon.ico" app_name = "opendataeditor" print("Creating executable file for Open Data Editor") params = [ "src/ode/main.py", "--windowed", # Required for Windows install to not open a console. "--collect-all", "frictionless", # Frictionless depends on data files "--collect-all", "ode", # Collect all assets from Open Data Editor "--collect-all", "llama_cpp", # Collect all assets from llama_cpp "--collect-all", "numpy", # Collect all assets from numpy (llama_cpp dependency) "--log-level", "WARN", "--name", app_name, "--noconfirm", "--icon", icon_path, ] if system == "Darwin": params.extend(["--osx-bundle-identifier", "org.okfn.opendataeditor"]) if system == "Windows": # llama_cpp depends on vcomp140.dll and it is not properly collected by PyInstaller as it # is a dependency of shiboken6 as well. This library is only present in Windows if C++ Redistributable # is installed which might not be the case for all of our users. params.extend(["--add-binary", "C:\\Windows\\system32\\vcomp140.dll:."]) # Allow calling `python build.py build` with extra arguments # e.g. when building AppImage `python build.py build --onefile --name "opendataeditor-X.Y.Z.AppImage"` cli_args = sys.argv[2:] if cli_args: params.extend(cli_args) PyInstaller.__main__.run(params) # Clean the spec file generated by PyInstaller if os.path.exists(f"{app_name}.spec"): os.remove(f"{app_name}.spec") def main(): if len(sys.argv) < 2: print("Usage:") print(" python build.py update-translations") print(" python build.py compile-translations") print(" python build.py build") print(" python build.py docs") sys.exit(1) command = sys.argv[1].lower() if command == "update-translations": update_translations() elif command == "compile-translations": compile_translations() elif command == "build": build_application() elif command == "docs": docs() else: print(f"Unknown command: {command}") sys.exit(1) if __name__ == "__main__": main() ================================================ FILE: create-deb.sh ================================================ #!/bin/sh # Script to create a PyQT deb package using fpm # # This script will create a folder structure that will be used as input # for the fpm command and copy into it all the distributable files generated by # pyinstaller. # We will create a folder with the same structure Linux systems expects: # - /opt for our executable and associated files (a.k.a our dist/ folder) # - /usr/share/applications (for the .desktop file) # - /usr/share/icons/hicolor/scalable/apps for our svg icons (only svg in scalable folder) # # More info: # - https://www.pythonguis.com/tutorials/packaging-pyqt5-applications-linux-pyinstaller/ # - https://fpm.readthedocs.io/en/latest/packages/dir.html#dir-local-files # # Create folders [ -e tmp ] && rm -r tmp mkdir -p tmp/opt mkdir -p tmp/usr/share/applications mkdir -p tmp/usr/share/icons/hicolor/scalable/apps # Build the project [ -e build ] && rm -r build [ -e dist ] && rm -r dist uv run build.py build # Copy files cp -r dist/opendataeditor tmp/opt/opendataeditor cp ./packaging/linux/icon.svg tmp/usr/share/icons/hicolor/scalable/apps/org.okfn.opendataeditor.svg cp ./packaging/linux/opendataeditor.desktop tmp/usr/share/applications # Change permissions # Packages retain the permissions of installed files from when they were packaged, # but will be installed by root. In order for ordinary users to be able to run the # application, we need to change the permissions. find tmp/opt/opendataeditor -type f -exec chmod 644 -- {} + find tmp/opt/opendataeditor -type d -exec chmod 755 -- {} + find tmp/usr/share -type f -exec chmod 644 -- {} + chmod +x tmp/opt/opendataeditor/opendataeditor # Create the deb package VERSION=$(uv run python -c "from importlib.metadata import version; print(version('opendataeditor'))") FILENAME=opendataeditor-linux-$VERSION.deb [ -e dist/$FILENAME ] && rm dist/$FILENAME fpm -C tmp -s dir -t deb -n "opendataeditor" -v $VERSION -p dist/$FILENAME ================================================ FILE: create-dmg.sh ================================================ #!/bin/sh # File to create the DMG file using create-dmg tool and notarize it. # # It is intended to work on Github Actions and so it might not be the most optimized # workflow for executing it locally (because of the secrets required for code sign # and notarizing) # # This script expects 6 secrets: # - CSC_LINK: A base64 encoded p12 certificate. # - CSC_KEY_PASSWORD: The password used to encrypt the p12 certificate # - APPLE_TEAM_ID: This is the ID of the team of your Apple Developer Account (Something like S1235Q75WSA) # - APPLE_APPLE_ID: This is the ID of your Apple Developer Account (usually your email) # - APPLE_APP_SPECIFIC_PASSWORD: The Application Specific Password created in your Developer Account. # # How to generate an APPLE_APP_SPECIFIC_PASSWORD # 1) Visit the Apple ID website: https://appleid.apple.com/ # 2) Sign in with your Apple ID credentials # 3) Navigate to the "Security" section # 4) Look for "App-Specific Passwords" # 5) Click "Generate Password..." # 6) Enter a descriptive label (e.g., "macOS App Notarization") # 7) Apple will generate a 16-character app-specific password # 8) Copy this password and use it as the value for the $APPLE_APP_SPECIFIC_PASSWORD environment variable in your notarization workflow # # Context and materials that inspired this script: # - https://www.pythonguis.com/tutorials/packaging-pyqt6-applications-pyinstaller-macos-dmg/ # - https://medium.com/flutter-community/build-sign-and-deliver-flutter-macos-desktop-applications-on-github-actions-5d9b69b0469c # - https://defn.io/2023/09/22/distributing-mac-apps-with-github-actions/ # - https://gist.github.com/txoof/0636835d3cc65245c6288b2374799c43 # # Issues we had with the notarization process: # - https://github.com/pyinstaller/pyinstaller/issues/8927 # Build the project [ -e build ] && rm -r build [ -e dist ] && rm -r dist uv run build.py build mv "dist/OpenDataEditor.app" "dist/Open Data Editor.app" # # Codesign the executable created by pyinstaller echo "Codesigning the executable created by PyInstaller" echo $CSC_LINK | base64 --decode > certificate.p12 security create-keychain -p thisisatemporarypass build.keychain security default-keychain -s build.keychain security unlock-keychain -p thisisatemporarypass build.keychain security import certificate.p12 -k build.keychain -P $CSC_KEY_PASSWORD -T /usr/bin/codesign security set-key-partition-list -S apple-tool:,apple:,codedign: -s -k thisisatemporarypass build.keychain echo "Signing complete application bundle..." /usr/bin/codesign --force --deep --options=runtime --entitlements ./packaging/macos/entitlements.mac.plist -s $APPLE_TEAM_ID --timestamp "dist/Open Data Editor.app" # Verify signature echo "Verifying signature..." codesign -vvv --deep --strict "dist/Open Data Editor.app" echo "Signing process completed." # Create dmg folder and copy our signed executable mkdir -p dist/dmg # We need to use -R to copy the app bundle recursively instead of -r because it doesn't preserver the symlinks otherwise # https://github.com/pyinstaller/pyinstaller/issues/8927 # https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html#requirements-imposed-by-symbolic-links-in-frozen-application cp -R "dist/Open Data Editor.app" "dist/dmg" # We need to detach the volume if it is already mounted # and remove the dmg file if it exists echo "Unmounting any existing volume..." hdiutil detach /Volumes/"Open Data Editor" &>/dev/null || true sleep 5 rm -f *.dmg # Create the dmg file VERSION=$(uv run python -c "from importlib.metadata import version; print(version('opendataeditor'))") FILENAME=opendataeditor-macos-$VERSION.dmg [ -e $FILENAME ] && rm $FILENAME MAX_RETRIES=3 RETRY_COUNT=0 while [ $RETRY_COUNT -lt $MAX_RETRIES ]; do echo "Creating the DMG file" if create-dmg \ --volname "Open Data Editor" \ --volicon "./packaging/macos/icon.icns" \ --window-pos 200 120 \ --window-size 800 400 \ --icon-size 100 \ --icon "Open Data Editor.app" 200 190 \ --hide-extension "Open Data Editor.app" \ --app-drop-link 600 185 \ $FILENAME \ "dist/dmg/";then echo "DMG created: $FILENAME" break else RETRY_COUNT=$((RETRY_COUNT + 1)) echo "Failed to create DMG. Retrying... ($RETRY_COUNT/$MAX_RETRIES)" hdiutil detach "/Volumes/Open Data Editor" -force &>/dev/null || true killall Finder &>/dev/null || true sleep 20 fi done if [ $RETRY_COUNT -eq $MAX_RETRIES ]; then echo "Failed to create DMG after $MAX_RETRIES attempts. Exiting." exit 1 fi if [ ! -f "$FILENAME" ]; then echo "DMG file not found. Exiting." exit 1 fi # Notarize the DMG File # If an error occurs, we can check the logs using # xcrun notarytool log $REPLACE-WITH-RUNNING-HASH --team-id $APPLE_TEAM_ID --apple-id $APPLE_ID --password $APPLE_APP_SPECIFIC_PASSWORD notarization_log.json echo "Notarizing the DMG file" xcrun notarytool submit --verbose --team-id $APPLE_TEAM_ID --apple-id $APPLE_ID --password $APPLE_APP_SPECIFIC_PASSWORD --wait $FILENAME > notarization_output.txt # Staple the file # We check if the notarization was successful if grep -q "status: Accepted" notarization_output.txt; then echo "Notarization successful!" # We wait for 30 seconds to make sure the notarization ticket is available echo "Waiting 30 seconds for notarization ticket to be available..." sleep 30 echo "Stapling the file" xcrun stapler staple $FILENAME else echo "Notarization failed. Check notarization_output.txt for details." cat notarization_output.txt exit 1 fi ================================================ FILE: docs/Makefile ================================================ # Minimal makefile for Sphinx documentation # # You can set these variables from the command line, and also # from the environment for the first two. SPHINXOPTS ?= SPHINXBUILD ?= sphinx-build SOURCEDIR = source BUILDDIR = build # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) ================================================ FILE: docs/make.bat ================================================ @ECHO OFF pushd %~dp0 REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set SOURCEDIR=source set BUILDDIR=build %SPHINXBUILD% >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.https://www.sphinx-doc.org/ exit /b 1 ) if "%1" == "" goto help %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% goto end :help %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% :end popd ================================================ FILE: docs/public/.gitkeep ================================================ ================================================ FILE: docs/requirements.txt ================================================ sphinx myst-parser sphinx_rtd_theme ================================================ FILE: docs/source/conf.py ================================================ # Configuration file for the Sphinx documentation builder. # # For the full list of built-in configuration values, see the documentation: # https://www.sphinx-doc.org/en/master/usage/configuration.html # -- Project information ----------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information project = 'Open Data Editor' copyright = '2025, Open Knowledge Foundation' author = 'Open Knowledge Foundation' release = '1.5.1' # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration extensions = [ 'myst_parser', 'sphinx_rtd_theme', ] myst_enable_extensions = [ "colon_fence", # Admonitions ] templates_path = ['_templates'] exclude_patterns = [] # -- Options for HTML output ------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output html_theme = 'sphinx_rtd_theme' html_static_path = ['_static'] ================================================ FILE: docs/source/contributing/contribution-guidelines.md ================================================ # Contributing ## Contribution Guidelines You don’t need to know how to code to contribute to Open Data Editor, you just need time and attitude. There are many possible ways to contribute to the project: * Use the Open Data Editor and give us feedback * Spread the word about it\! * Improve the documentation * Add a translation * Report issues * Contribute to the code Please read this guide for more details on the contribution process. ### How can you help? #### Giving us feedback Use the Open Data Editor and give us feedback. You can try out all the features and report any issues you might encounter. You can also suggest improvements to the Open Data Editor’s look and layout. For example, *if something feels confusing or hard to find in the tool’s layout, you can suggest changes to make it easier to use.* Feedback from a beginner’s perspective is particularly welcome as it helps us improve usability. To give us feedback, you can either [open an issue in the GitHub repository](https://github.com/okfn/opendataeditor) or reach out by email at info@okfn.org. #### Spread the word If you like the application, tell others about it\! Spreading the word about the Open Data Editor is also a valuable contribution to the project. Whether you share it on social media, write a blog post, create a tutorial video, or simply tell a friend, you’re helping more people discover and use the tool. Every bit of awareness and engagement makes a difference\! #### Documentation ##### Improving documentation Documentation is always the best place to start contributing. If you spot a typo or you’ve ever thought, “I wish the documentation explained this more clearly,” you can help make it easier to understand. You can either flag that, or propose a modification. You can also add missing steps or explanations. ##### Adding examples You can also contribute by adding examples\! If you notice something is not covered in the documentation, try creating a helpful example and sharing it. You can also provide real-world use cases to show how Open Data Editor can be used. Every improvement makes Open Data Editor easier for everyone to use\! All documentation files are contained [in this folder](https://github.com/okfn/opendataeditor/tree/main/portal/content/docs/documentation). You can just propose a change or add examples by opening a PR. Please **always include a short description of what you changed and why**, so it is easier to review. If you don’t feel comfortable using GitHub, you can also send us an email with the proposed changes at info\[at\]okfn.org. ##### Use cases If you have started using Open Data Editor in your daily work and would like to share with the world how the tool is helping you, you can always write a blog about it. You can find a few examples [here](https://blog.okfn.org/tag/ode-use-cases/). If possible, try to include photos and screenshots. If you need guidance, you can always reach out to us at info@okfn.org. You can send us your blog at that same email address. #### Reporting a bug We use GitHub as a code and issues hosting platform. To report a bug or propose a new feature, please open an issue [in the repository](https://github.com/okfn/opendataeditor/issues). Please provide a detailed description of the bug and include screenshots. There are a few predefined issue templates to help you get started. If you don’t feel comfortable using GitHub, you can also send us an email at info@okfn.org. #### Code contributions ##### Pull requests First, please check the [issue tracker](https://github.com/okfn/opendataeditor/issues). Look for issues with “help wanted” or “good first issue.” **If you want to submit a PR, there needs to be a corresponding issue in the issue tracker.** Please link the issue in your PR, and provide a brief explanation of the changes you are pushing. ### What is the review process for your contribution? Your contribution will be **reviewed by the Open Data Editor core team at Open Knowledge Foundation**. In case of documentation and code contributions, a member of the core team will check if it is relevant, helpful, and easy to understand. If small changes are needed, they suggest edits. Once everything looks good, the contribution is approved and added to the project, and the contributor is acknowledged (because every contribution matters\!). In case of bugs and feedback contributions, the core team will discuss if and how they can be implemented. We can include you in the discussion if you wish. Once we have a clear pathway, we will notify you. ### Do you need help? If at any point you need help, feel free to contact us via email at [info@okfn.org](mailto:info@okfn.org). ================================================ FILE: docs/source/contributing/translations.md ================================================ # Translations ODE supports several languages following the [Qt Framework practices for](https://doc.qt.io/qt-6/internationalization.html) internationalisation. ## Internationalisation workflow :::{note} ODE provides a `python build.py update-translations` and `python build.py compile-translations` command for all the supported languages. Check the project’s `build.py` file for reference. ::: 1. Create or update translation files by running `python build.py update-translations` (or the `pyside6-lupdate` command directly if you are on macOS) 2. Update the translation files: 1. Complete `unfinished` translations (this is the actual addition of translated text). 2. Clean `vanished` translations. 3. Compile the translation files by running `python build.py compile-translations` (or the `pyside6-lrelease` command directly if you are on macOS) 4. Commit both the `.ts` and `.qm` files. 5. Create a PR with the changes. All translation files are located in the `ode/assets/translations/` folder. ## Translation Tools For updating translations, you can use either: 1. A text editor to directly update the translation files (\*.ts) 2. Install Qt and use [Qt Linguist](https://doc.qt.io/qt-6/qtlinguist-index.html) application to do the translation using a UI. ## Adding new languages When adding a new language, two extra changes are required: 1. Add the new language to both `python build.py update-translations` and `python build.py compile-translations` commands of the `build.py` file. (after doing this, you can run the `python build.py update-translations` and it will create the `.ts` file for you.) 2. Update the `language` QComboBox of the `main.py` file so the new language appears as an option to the user. Here is a reference Pull Request of what is expected when adding a new language: [https://github.com/okfn/opendataeditor/pull/750](https://github.com/okfn/opendataeditor/pull/750) ================================================ FILE: docs/source/index.rst ================================================ .. Open Data Editor documentation master file, created by sphinx-quickstart on Fri Jul 4 13:22:06 2025. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Open Data Editor Docs ======================================== .. image:: /assets/ODE-logo.png :alt: Open Data Editor logo :width: 200px **Your no-code app for error-free spreadsheets, and guaranteed privacy and FAIR data** This website contains user guides and technical documentation for the Open Data Editor (ODE), as well as information on contributing code and use cases. For more information, please visit the official project page at the Open Knowledge Foundation (OKFN) website: `https://okfn.org/en/projects/open-data-editor/ `_ .. toctree:: :maxdepth: 3 :caption: Introduction introduction/what-is-open-data-editor.md introduction/fair-data.md introduction/responsible-ai-integration.md introduction/free-data-literacy-course.md introduction/similar-tools-and-differentiators.md introduction/acknowledgements.md introduction/latest-updates.md .. toctree:: :maxdepth: 3 :caption: Use Cases use-cases/context.md use-cases/agricultural-data-ghana.md use-cases/climate-data-kenya.md use-cases/data-journalism-mexico.md use-cases/defence-data-france.md use-cases/financial-data-south-africa.md use-cases/government-data-croatia.md use-cases/heritage-data-cambodia.md use-cases/library-data-india.md .. toctree:: :maxdepth: 3 :caption: User Guide user-guide/downloading-ode.md user-guide/installing-ode.md user-guide/uploading-data.md user-guide/how-to-explore-table-errors.md user-guide/editing-errors-in-tables.md user-guide/deleting-files-or-folders.md user-guide/full-list-of-table-errors-detected.md user-guide/how-to-use-the-ai-component.md user-guide/how-to-explore-and-edit-metadata.md user-guide/exporting-your-data.md .. toctree:: :maxdepth: 3 :caption: Technical Documentation technical-documentation/prerequisites.md technical-documentation/environment.md technical-documentation/start-the-application.md technical-documentation/running-tests.md technical-documentation/building-the-application.md technical-documentation/documentation.md technical-documentation/making-a-release.md .. toctree:: :maxdepth: 3 :caption: Contributing contributing/contribution-guidelines.md contributing/translations.md ================================================ FILE: docs/source/introduction/acknowledgements.md ================================================ ## Acknowledgements We are grateful for the support and partnership of the [Patrick J. McGovern Foundation (PJMF)](https://www.mcgovern.org/), without which the development of the Open Data Editor would not have been possible. Learn more about its funding programmes [here](https://www.mcgovern.org/grants/). ================================================ FILE: docs/source/introduction/fair-data.md ================================================ ## FAIR data The Open Data Editor (ODE) improves data quality based on the [FAIR principles](https://www.go-fair.org/fair-principles/). As described by the [GO FAIR initiative](https://www.go-fair.org/), FAIR data refers to data that adheres to the **Findable**, **Accessible**, **Interoperable**, and **Reusable** principles, designed to make research data more discoverable and usable by people and machines, enhancing its value and reuse. **Findable** The first step in (re)using data is to find it. Metadata and data should be easy to find for both humans and computers. **Accessible** Once the user finds the required data, they need to know how it can be accessed, possibly including authentication and authorisation. **Interoperable** The data usually need to be integrated with other data. In addition, the data needs to interoperate with applications or workflows for analysis, storage, and processing. **Reusable** The ultimate goal of FAIR is to optimise the reuse of data. To achieve this, metadata and data should be well-described so that they can be replicated and/or combined in different settings. Learn all about the FAIR principles in [Module 4](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/lessons/introduction-6/) of the ‘Quality and Consistent Data with ODE’ course, available on [School of Data](https://schoolofdata.org/). ================================================ FILE: docs/source/introduction/free-data-literacy-course.md ================================================ ## Free data literacy course Open Data Editor's focus on improving digital literacy and preparing non-technical users to work with data, led us to develop a free course available on School of Data. [‘Quality and Consistent Data with the Open Data Editor’](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/) is an essential open educational resource for anyone who wants to generate knowledge from data. The course is now available in English and Portuguese, and will soon be translated into Spanish and French. It is specially designed for non-technical users working with tabular data (Excel, Google Sheets, CSV) but without advanced technical knowledge. It's 100% online, free, and gamified. You can check the contents and enrol here: [https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/](https://schoolofdata.org/courses/quality-and-consistent-data-with-open-data-editor/) ================================================ FILE: docs/source/introduction/latest-updates.md ================================================ ## Latest updates Open Data Editor is being built in the open. Follow the progress, from feature updates to community stories on the Open Knowledge Blog: [https://blog.okfn.org/category/open-data-editor/](https://blog.okfn.org/category/open-data-editor/) ================================================ FILE: docs/source/introduction/responsible-ai-integration.md ================================================ ## Responsible AI integration The Open Data Editor (ODE) has an AI component to help users better understand their data. The AI component is powered by local models. **No data is sent to the cloud, and all operations take place on the user’s computer.** The introduction of these AI-assisted data quality features significantly expands the tool’s capabilities, enabling faster identification of anomalies, smarter suggestions for corrections, and improved handling of large, dense datasets while retaining a privacy-preserving, local-first design. This deepens Open Knowledge Foundation's commitment to developing simple, sustainable and long-lasting technologies that solve people’s real problems, in line with [The Tech We Want](https://okfn.org/en/projects/the-tech-we-want/) initiative. In order to use the AI feature, ODE will guide users to download the model onto their machine first. :::{note} As the model is local and all computing will be happening in the user’s computer, performance will be affected by the machine’s hardware, and the response quality sometimes will be minor than typical cloud LLMs like ChatGPT, Deepseek or Claude. ::: ODE is trying to balance user experience for non-technical audiences and performance and privacy. ================================================ FILE: docs/source/introduction/similar-tools-and-differentiators.md ================================================ ## Similar tools and differentiators The tools currently available with functions similar to those of the Open Data Editor were created for a specific purpose and have a more technical profile. This makes them difficult for people unfamiliar with code, standards or programming languages to access. The main differences in relation to ODE are listed in each subsection below: ### Data Check Available at: [https://data.humdata.org/tools/datacheck/import](https://data.humdata.org/tools/datacheck/import) Main differences: * Maximum file size: 20 MB. * Works only with the HXL standard. * The table view after the error check is limited, and the user needs to navigate through several tabs if the file has many lines. * Does not include a publication feature. ### IATI Validator Available at: [https://validator.iatistandard.org/](https://validator.iatistandard.org/) Main differences: * Works only with the IATI standard. * Targets a specific sector, the international aid community. * The tool offers five levels of qualification regarding data quality (Success, Success with Advisories, Warning, Error, and Critical), rather than a list of all errors and how to correct them. ### CSV Lint.io Available at: [https://csvlint.io/](https://csvlint.io/) Main differences: * Works only with CSV files, informing the user if the file “is readable” or not. * Agnostic tool; the schema can also be ingested. ### 360Giving Data Quality Checker Available at: [https://dataquality.threesixtygiving.org/](https://dataquality.threesixtygiving.org/) Main differences: * Works only with the 360Giving standard. ================================================ FILE: docs/source/introduction/what-is-open-data-editor.md ================================================ ## What is Open Data Editor [Open Data Editor (ODE)](https://okfn.org/opendataeditor), developed by the [Open Knowledge Foundation (OKFN)](https://okfn.org/en/), is a free, open-source tool designed to help nonprofits, data journalists, activists, and public servants detect errors in their datasets. It is designed for people working with tabular data (Excel, Google Sheets, CSV) who don't know how to code or don't have the programming skills to automatise the data exploration process. The technical mission of ODE is to provide a free, open-source, no-code, cross-platform desktop application that empowers non-technical users working with tabular data to quickly detect and correct errors, enforce data-validation and metadata standards (notably the [FAIR principles](https://www.go-fair.org/fair-principles/)), and output clean, interoperable datasets ready for publication – while preserving privacy (local-first architecture), remaining lightweight (suitable for low-resource or offline contexts), and employing open standards (e.g., the [Frictionless framework](https://framework.frictionlessdata.io/)) for maximum reuse and integration. Since 2025, the Digital Public Goods Alliance (DPGA) has recognised Open Data Editor as a [digital public good](https://blog.okfn.org/2025/10/22/open-data-editor-recognised-as-a-digital-public-good/) (DPG), which means it meets high standards of openness and supports sustainable development globally. ================================================ FILE: docs/source/technical-documentation/building-the-application.md ================================================ ## Building the application ```bash uv run build.py build ``` or ```bash # With the virtual environment activated python build.py build ``` This will create a distributable file for the application in the ‘dist/’ folder. ================================================ FILE: docs/source/technical-documentation/documentation.md ================================================ ## Documentation Documentation is written with [Sphinx](https://www.sphinx-doc.org/en/master/) (in the `docs` directory). The source files are in the `docs/source/` directory. To locally build the documentation, you can execute: ```bash uv run build.py docs ``` or ```bash # With the virtual environment activated python build.py docs ``` It will be automatically published on CloudFlare when merged to the `main`branch, with previews available for pull requests. ================================================ FILE: docs/source/technical-documentation/environment.md ================================================ ## Environment Use `uv` to create a virtualenv and activate it: ```bash uv sync source venv/bin/activate ``` ================================================ FILE: docs/source/technical-documentation/making-a-release.md ================================================ ## Making a release To make a release, follow the following checklist: * Check with the Product Owner that the `main` branch is code complete. * Check that the distributables built on `main` are working by installing them on your machine. * Sometimes PyInstaller cannot compile new dependencies, and the application will fail at runtime. * Create a new PR bumping the version of the application in the `pyproject.toml` file and merge it to main. * Create a New Github Release with a new tag matching the new version number of the application. * Fill in the Release notes. * Create the Release. * Wait until the GitHub Action for the new tag finishes, and then upload the distributable files to the new Release. * Notify the Communications Team to make the announcement and changes to the [OKFN’s Website](https://okfn.org/opendataeditor/). ================================================ FILE: docs/source/technical-documentation/prerequisites.md ================================================ ## Prerequisites We are using 3.13. To start working on the project, you need the following dependencies on your machine: * Python 3.13 * python3.13-dev (For PyInstaller) We are using [uv](https://docs.astral.sh/uv/) as a package manager, so make sure you have it installed. ================================================ FILE: docs/source/technical-documentation/running-tests.md ================================================ ## Running tests ```bash uv run pytest tests/ ``` or ```bash # With the virtual environment activated pytest tests/ ``` ================================================ FILE: docs/source/technical-documentation/start-the-application.md ================================================ ## Start the application ```bash uv run ode ``` or ```bash # With the virtual environment activated python src/ode/main.py ``` ================================================ FILE: docs/source/use-cases/agricultural-data-ghana.md ================================================ ## Agricultural data (Ghana) Open Science Community Ghana (OSCG) used ODE to streamline the work with manually-collected data and reduce the time to identify and correct errors. The ODE interface enabled the team to easily define and enforce a standard data structure, ensuring that every dataset adheres to a consistent format. This directly solved their core problem of harmonising the data that they receive from different sources. ![Ghana](./assets/use-cases/ghana.png) An example of errors detected by ODE in a research dataset: blank cells and wrong formats. Learn more: [https://blog.okfn.org/2025/11/10/open-data-editor-in-action-bridging-the-gap-between-field-data-and-research-insights-in-ghana/](https://blog.okfn.org/2025/11/10/open-data-editor-in-action-bridging-the-gap-between-field-data-and-research-insights-in-ghana/) ================================================ FILE: docs/source/use-cases/climate-data-kenya.md ================================================ ## Climate data (Kenya) The Demography Project used ODE to check and correct errors in a giant spreadsheet of air quality data, enabling accurate analysis. These tables, which compile data from different sources (including two portable air quality monitors, GPS devices, smartphones), had many inconsistencies that were detected in seconds. Once the errors have been identified, they could then correct the missing or incorrect data, increasing the dataset’s quality. ![Kenya](./assets/use-cases/kenya.png) In this spreadsheet, ODE identified 29 inconsistencies and errors in the dataset. Learn more: [https://blog.okfn.org/2025/04/28/open-data-editor-use-case-a-giant-spreadsheet-of-environmental-data-now-accurate-for-analysis/](https://blog.okfn.org/2025/04/28/open-data-editor-use-case-a-giant-spreadsheet-of-environmental-data-now-accurate-for-analysis/) ================================================ FILE: docs/source/use-cases/context.md ================================================ # Context The Open Data Editor was developed in collaboration with user communities worldwide. Between 2024 and 2025, three formal testing and feedback phases were conducted with civil society organisations working in different fields of knowledge. This work has given rise to several use cases, all of which are compiled here: [https://blog.okfn.org/tag/ode-use-cases/](https://blog.okfn.org/tag/ode-use-cases/) Below, we highlight some of them to give you an idea of the types of tasks that ODE helps to perform to improve data workflow and data quality. You can also contribute by documenting a use case. Learn how at **Documentation \> Use cases** in this guide. ================================================ FILE: docs/source/use-cases/data-journalism-mexico.md ================================================ ## Data journalism (Mexico) Data Crítica used ODE to identify reliable variables for journalistic investigations and ensure stories are built on a solid foundation. They used it as a central part of its “data interrogation” methodology in workshops and its own research. ODE instantly flagged missing values, providing a clear, visual overview of a dataset’s completeness by highlighting empty cells. ![Mexico](./assets/use-cases/mexico.png) In the same dataset as the picture above, ODE now identifies errors in columns with the same name and unexpected data types Learn more: [https://blog.okfn.org/2025/11/28/open-data-editor-in-action-interrogating-data-for-investigative-journalism-in-mexico/](https://blog.okfn.org/2025/11/28/open-data-editor-in-action-interrogating-data-for-investigative-journalism-in-mexico/) ================================================ FILE: docs/source/use-cases/defence-data-france.md ================================================ ## Defence data (France) The Observatoire des armements used ODE to turn multiple public spending data sources (arms purchases and sales) into a single quality spreadsheet. They reduced error resolution time from days to seconds, eliminated 95% of manual review work, and enabled the team to focus on what matters. ![France](./assets/use-cases/france.jpg) In this spreadsheet, ODE flagged 48 inconsistencies in seconds. Learn more: [https://blog.okfn.org/2025/04/14/open-data-editor-use-case-multiple-defence-spending-data-sources-in-a-single-quality-spreadsheet/](https://blog.okfn.org/2025/04/14/open-data-editor-use-case-multiple-defence-spending-data-sources-in-a-single-quality-spreadsheet/) ================================================ FILE: docs/source/use-cases/financial-data-south-africa.md ================================================ ## Financial data (South Africa) The Public Affairs Research Institute (PARI) used ODE to restructure messy public financial datasets into a clean, consistent format, making them ready for reliable analysis. ODE automatically profiled the data, instantly highlighting empty cells, type mismatches, and structural inconsistencies that would take days to find manually. This shifted their role from manual detectives to efficient data supervisors. ![South Africa](./assets/use-cases/south-africa.png) Examples of errors detected by ODE in a municipal dataset: blank cells and wrong formats. Learn more: [https://blog.okfn.org/2025/11/10/open-data-editor-in-action-enhancing-fiscal-governance-and-transparency-in-south-african-municipalities/](https://blog.okfn.org/2025/11/10/open-data-editor-in-action-enhancing-fiscal-governance-and-transparency-in-south-african-municipalities/) ================================================ FILE: docs/source/use-cases/government-data-croatia.md ================================================ ## Government data (Croatia) The City of Zagreb used ODE to comply with open data standards and foster a culture of data literacy across different city offices in Zagreb. ODE flagged inconsistencies (e.g., missing values, formatting errors) in seconds, such as a public buildings dataset with 11 columns of unlabeled energy metrics. The team was able to add critical context (e.g., data owners, sourcing methods) directly in ODE’s metadata panel, aligning with FAIR principles. ![Croatia](./assets/use-cases/croatia.jpg) ODE’s metadata panel was central to understanding the importance of interoperability and creating a culture of data literacy in the public administration. Learn more: [https://blog.okfn.org/2025/05/20/open-data-editor-in-action-streamlining-data-governance-and-unlocking-the-potential-value-of-urban-data-in-croatia/](https://blog.okfn.org/2025/05/20/open-data-editor-in-action-streamlining-data-governance-and-unlocking-the-potential-value-of-urban-data-in-croatia/) ================================================ FILE: docs/source/use-cases/heritage-data-cambodia.md ================================================ ## Heritage data (Cambodia) An AI of Our Own (AAOO) used ODE to create AI models that are built on respectful and ethically sourced data from the Global South. ODE allowed the team to identify and rectify formatting inconsistencies. They could standardise date formats and other variables, ensuring that data from different collection methods could be seamlessly unified. A critical feature for AAOO was the ability to add detailed descriptions to each column. This process of adding context and meaning to each data point is fundamental to building a high-quality, culturally nuanced AI dataset. ![Cambodia](./assets/use-cases/cambodia.jpg) Errors flagged showing data inconsistency from the converted unstructured data into a structured format without considering the standard format and time stamps (Data on Indigenous Knowledge Systems on Plant use) Learn more: [https://blog.okfn.org/2025/11/05/open-data-editor-in-action-building-culturally-accurate-and-global-south-sensitive-ai-datasets/](https://blog.okfn.org/2025/11/05/open-data-editor-in-action-building-culturally-accurate-and-global-south-sensitive-ai-datasets/) ================================================ FILE: docs/source/use-cases/library-data-india.md ================================================ ## Library data (India) The Indian Institute of Technology (IIT) Delhi used ODE to check errors in large spreadsheets of publications catalogue and educate colleagues about data quality. ODE automatically flagged a range of issues in complex datasets, including bibliographic information from major indexes like Scopus and Web of Science. By identifying formatting inconsistencies, ODE provided a foundation for the team to clean and standardise fields, creating more reliable datasets for their reports and website. ![India](./assets/use-cases/india.png) Open Data Editor helps automate error detection; above, problems with blank cells and the data types expected for a given column. Learn more: [https://blog.okfn.org/2025/12/09/open-data-editor-in-action-advancing-data-quality-in-academic-library-services-in-india/](https://blog.okfn.org/2025/12/09/open-data-editor-in-action-advancing-data-quality-in-academic-library-services-in-india/) ================================================ FILE: docs/source/user-guide/assets/table-error-list/column-name-missing.csv ================================================ col1, 1,2 3,4 ================================================ FILE: docs/source/user-guide/assets/table-error-list/duplicate-column-name.csv ================================================ col1,col1 1,2 3,4 ================================================ FILE: docs/source/user-guide/assets/table-error-list/empty-row.csv ================================================ col1,col2 1,2 3,4 ================================================ FILE: docs/source/user-guide/assets/table-error-list/extra-cell.csv ================================================ col1,col2 1,2 3,4 5,6,7 ================================================ FILE: docs/source/user-guide/assets/table-error-list/header-missing.csv ================================================ , 1,2 3,4 ================================================ FILE: docs/source/user-guide/assets/table-error-list/wrong-data-type.csv ================================================ col1,col2 1,2 3,4 5,6 7,8 9,10 11,12 13,14 15,16 17,18 19,20 21,bad ================================================ FILE: docs/source/user-guide/deleting-files-or-folders.md ================================================ ## Deleting files or folders To delete a file or folder, click on the three dots next to the file/folder name and select **Delete**. ![Delete button in the file navigator](./assets/deleting-files-folder/delete-option.png) ================================================ FILE: docs/source/user-guide/downloading-ode.md ================================================ ## Downloading ODE Open Data Editor is available on all major platforms: * **For Windows:** Download the most recent **EXE file**. * **For MacOS:** Download the most recent **DMG file**. * **For Debian-based Linux:** Download the most recent **AppImage or DEB file**. * **For other Linux:** Download the most recent **AppImage**. ### From our website You can download ODE from our website: [https://okfn.org/opendataeditor/](https://okfn.org/opendataeditor/) ### From GitHub Releases You can download ODE from the repository: [https://github.com/okfn/opendataeditor/releases](https://github.com/okfn/opendataeditor/releases) ================================================ FILE: docs/source/user-guide/editing-errors-in-tables.md ================================================ ## Editing errors in tables To fix cell errors, you can directly edit the data cells in the viewer/editor. **Step 1:** Locate the cell with the error. For example: ![Error cell](./assets/editing-errors-in-table/cell-with-errors-edit.png) **Step 2:** Double-click on the cell to start editing content: ![Edit cell with errors](./assets/editing-errors-in-table/edit-cell-with-error.png) **Step 3:** To save changes, click on another part of the table to accept the change in the cell, and when the **Save changes** button is activated, click on it. The button will get activated if there are unsaved changes. ![Save changes button](./assets/editing-errors-in-table/save-changes-button.png) After clicking the **Save changes** button, ODE will update the Errors Report. ================================================ FILE: docs/source/user-guide/exporting-your-data.md ================================================ ## Exporting your data You can export your data using the **Export** feature located at the top right of the datagrid: ![Publish button](./assets/exporting-data/export-button.png) Once you click the **Export** button, ODE will display the following dialogue: ![Publish form](./assets/exporting-data/export-dialog.png) ### Download file This option will download the file in CSV format. ### Download file with errors This option will export an Excel file with three sheets: * **Data:** This sheet contains the original table with all the errors painted in red. * **Errors Description:** This sheet contains the description of the errors detected by ODE with the corresponding cell (row and column). * **Blank Rows:** This sheet contains the rows on the original table that did not contain any values. ================================================ FILE: docs/source/user-guide/full-list-of-table-errors-detected.md ================================================ ## Full list of table errors detected Here we describe the list of errors that ODE can detect after users upload tables to the app. All the examples are based on CSV files. :::{note} It is possible to reproduce a subset of these errors using other formats like Excel, but some errors might not be applicable to other formats. ::: To explain and understand errors, we need to illustrate some key elements that are part of tables: * A regular table contains one **header row** (where the names of columns are listed), **rows** and **cells**. * **Cells describing names of columns** are also called **labels**. * Rows contain **cells** called **values**. The table example is represented as follows: ``` [1] [header row] label 1 | label 2 [2] [data row] value 1 | value 2 [3] [data row] value 3 | value 4 ``` ### Errors detected automatically **This type of error occurs when the structure of the data is not as expected.** For example, the number of columns in a row is different from the number of columns in the header. #### Header missing (Blank Label) This error occurs when the **header row is empty**. The header row should contain the names of the columns: ``` , 1,2 3,4 ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/bb8d52a4fb7cb7ce135cda4ac8156173/header-missing.csv) This is how ODE will show the error: ![Header missing error](./assets/table-error-list/header-missing.png) #### Column name missing This error occurs when **one or more column names are missing**: ``` col1, 1,2 3,4 ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/45ba64a900f79509f76ef9865da818fe/column-name-missing.csv) This is how ODE will show the error: ![Column name missing error](./assets/table-error-list/column-name-missing.png) #### Duplicate column name This error occurs when there are **two or more columns with the same name**. Each column should have a unique name. ``` col1,col1 1,2 3,4 ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/c352eb003acbc2caf2f45df0d37631c9/duplicate-column-name.csv) This is how ODE will show the error: ![Duplicated column name error](./assets/table-error-list/duplicate-column-name.png) #### Empty row This error occurs when an **empty row is present in the data**. ``` col1,col2 1,2 3,4 ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/4cd0ff415190ada70a6cf765842f7aff/empty-row.csv) This is how ODE will show the error: ![Empty row error](./assets/table-error-list/empty-row.png) #### Missing cell This error occurs when **a row has fewer cells than the header**. ``` col1,col2 1,2 3,4 5 ``` This is how ODE will show the error: ![Missing cell error](./assets/table-error-list/missing-cell.png) #### Extra cell This error occurs when a row has more cells than the header. Each row should have the same number of cells as the header. ``` col1,col2 1,2 3,4 5,6,7 ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/1c7d6743ad084bf1dbe1de05b961e158/extra-cell.csv) This is how ODE will show the error: ![Extra cell error](./assets/table-error-list/extra-cell.png) #### Wrong data type This error occurs when **a cell contains a value that is not of the expected type**. For example, a cell in a column that should contain numbers contains a string. ``` col1,col2 1,2 3,4 5,6 7,8 9,10 11,12 13,14 15,16 17,18 19,20 21,bad ``` * [Reproduce the error using this file](https://opendataeditor.okfn.org/_downloads/508223106a8c55b55430211419875f01/wrong-data-type.csv) This is how ODE will show the error: ![Wrong data type](./assets/table-error-list/wrong-data-type.png) :::{note} This error can be identified without providing a Table Schema, but only if the data has enough cells of the correct type in the column to infer the intended type. ::: ### Errors Requiring Metadata These errors can only be identified if a Table Dialect or Table Schema is provided by editing the table’s metadata. The Table Schema defines the structure of the data, including the type of each column. Table Schema adds additional constraints to the data, which are used to validate the data. #### Extra column name This error occurs when **a header label in the data is not defined in the Table Schema**. The Table Schema should define all the columns in the data. ``` fields: - name: col1 - name: col2 col1,col2,col3 1,2 3,4 Cell col3 is an extra label ``` #### Missing column name This error occurs when **a column defined in the Table Schema is not present in the header row**. The data should contain all the columns defined in the Table Schema. ``` fields: - name: col1 - name: col2 - name: col3 col1,col2 1,2,3 4,5,6 Missing cell col3 is a missing label. ``` #### Incorrect column name This error occurs when **the header label in the data does not match the label defined in the Table Schema**. The header row should contain the same labels as defined in the Table Schema. ``` fields: - name: col1 - name: col2 col1,col3 1,2 3,4 Cell col3 is an incorrect label. ``` #### Primary Key Error This error occurs when the **primary key constraint defined in the Table Schema is not satisfied**. The primary key constraint ensures that the values in the specified columns are unique. ``` fields: - name: col1 - name: col2 primaryKey: col1 col1,col2 1,2 1,4 Cell 1 in the second data row is not unique. ``` #### Foreign Key Error This error occurs when the **foreign key constraint defined in the Table Schema is not satisfied**. The foreign key constraint ensures that the values in the specified columns are present in another table or satisfies self-referencing constraint. ``` fields: - name: col1 - name: col2 foreignKeys: - fields: col2 reference: fields: col1 col1,col2 1,2 2,4 Cell 4 in the second data row is not present in the col1 column. ``` #### Unique constraint error This error occurs when the **unique constraint defined in the Table Schema is not satisfied**. The unique constraint ensures that the values in the specified columns are unique. ``` fields: - name: col1 - name: col2 unique: true col1,col2 1,2 3,2 Cell 2 in the second data row is not unique. ``` #### Constraint Error This error occurs when **a field constraint defined in the Table Schema is not satisfied**. Read more about Table Schema constraints: [https://datapackage.org/standard/table-schema/\#field-constraints](https://datapackage.org/standard/table-schema/#field-constraints) ``` fields: - name: col1 - name: col2 constraints: - required: true col1,col2 1,2 3 Missing cell 4 in the second data row is required. ``` The following constraints can be defined in the Table Schema and are currently supported by Open Data Editor (please read the section above about unique constraints): - required - enum - minimum - maximum - minLength - maxLength - pattern ================================================ FILE: docs/source/user-guide/how-to-explore-and-edit-metadata.md ================================================ ## How to explore and edit metadata To explore or edit the metadata, select a file from the menu on the left and click on any cell of the header row (first row). ![Metadata Button](./assets/explore-edit-metadata/metadata-button.png) ODE will then display the **Metadata** window: ![Metadata panel](./assets/explore-edit-metadata/metadata-panel.png) You can click on any of the options to start editing the metadata linked to your file. Once you have finished editing the metadata, click on the **Save changes** button to save the changes. :::{note} Saving changes will trigger a validation of the file. ::: ================================================ FILE: docs/source/user-guide/how-to-explore-table-errors.md ================================================ ## How to explore table errors As mentioned in the **Uploading data** section in this guide, if a file has errors, ODE will show a red dot next to the file name on the sidebar. However, if you want to review errors in the table, you can use the datagrid to explore problematic data. ODE will highlight the cell in red if it has a problem. For instance, if it contains text instead of a number. This is how a cell with an error is shown on ODE: ![Cell with errors](./assets/explore-table-errors/cell-with-error-edit.png) You can also explore errors by clicking on the **Errors Report** button, located at the top left of the datagrid: ![Errors panel button](./assets/explore-table-errors/errors-panel-button.png) After clicking the button, ODE will display a panel with the full list of errors: ![Errors panel](./assets/explore-table-errors/errors-panel.png) ================================================ FILE: docs/source/user-guide/how-to-use-the-ai-component.md ================================================ ## How to use the AI component To use the AI assistant, select a file from the sidebar, and then click on the **AI** button located in the top right corner of the app: ![AI Integration button is located in the top panel](./assets/ai-integration/ai-integration-1.png) ODE will show a dialogue to assist the user in downloading the model: ![Downloading model dialog](./assets/ai-integration/ai-integration-2.png) Once the model is downloaded, users will be able to click on the **Next** button to continue. ### AI Use Cases ODE has two use cases for the AI component: 1. Assist users in understanding the columns of a table. 2. Suggest analysis and questions that users can use to query the data. To choose the use case, select it from the dropdown menu. ![AI Use Cases dropdown menu](./assets/ai-integration/ai-integration-3.png) Once selected, click on the **Execute** button to get a response from the AI model. :::{note} Depending on the hardware of the users’ machines, the response time can vary. Usually, it will take around 10 seconds to get a response. ::: #### Assist users in understanding the columns of a table The AI model will analyse the columns’ names, types and some sample data and will generate a description of each column. This is useful to understand the data better, clarify technical or complex names, expand acronyms, etc. ![AI assistance in understanding the columns of a table](./assets/ai-integration/ai-integration-4.png) #### Suggest analysis and questions that users can use to query the data The AI model will analyse the columns’ names, types and some sample data and will generate a list of questions that the user can use to query the data. ![AI assistance in understanding the columns of a table](./assets/ai-integration/ai-integration-5.png) ================================================ FILE: docs/source/user-guide/installing-ode.md ================================================ ## Installing ODE ### Windows Download the most recent **EXE** file as per the above instructions. 1\. If you receive the following message, click ‘Continue download’. ![DOWNLOAD SECURITY](./assets/getting-started/gs-windows-download.png) 2\. After downloading, double-click to run the app. You may encounter the security message window, click ‘More info’ and proceed. ![SECURITY MESSAGE](./assets/getting-started/gs-protection-screen.png) 3\. Click ‘Run anyway’ to run the application. ![SECURITY MESSAGE STEP 2](./assets/getting-started/gs-protection-screen-2.png) ### MacOS Download the most recent **DMG** file as per the above instructions. 1\. If you encounter a security message, click on the question mark and then click the link in the first section. ![DOWNLOAD SECURITY](./assets/getting-started/gs-macos-download.png) 2\. Change settings to allow the app to execute. ![DOWNLOAD SETTINGS](./assets/getting-started/gs-macos-download-step2.png) ### Linux For Linux, there are two options available: * AppImage (for any distributions) * deb (for Ubuntu/Debian) #### Any Distribution Download the most recent **AppImage** file as per the above instructions. After downloading, you have to make it executable: ![MAKE EXECUTABLE](./assets/getting-started/gs-linux-executable.png) Then double-click on the file to start the application. #### Ubuntu/Debian Download the most recent **DEB** file as per the above instructions. Double-click on the file, and it will initiate the installation process. ![MAKE INSTALLATION](./assets/getting-started/gs-ode-installation.png) After installation, you can use it. ![INSTALLED APP](./assets/getting-started/gs-ode-app.png) Optionally, in Debian, you can install it by running the following command: *\# Replace \ with the version you downloaded* sudo dpkg \-i opendataeditor-linux-\.deb ================================================ FILE: docs/source/user-guide/uploading-data.md ================================================ ## Uploading data This section explains how to upload tabular files, folders with tables and online data to ODE. The tool also ingests other types of format files like PDF, JPEG, etc. However, please note, ODE's main objective is to detect errors on tabular files, the application will **ONLY** show previews for tables. **Uploading data to ODE is easy\!** After installing the app and once you open the application on your laptop, you will see this screen: ![Uploading data](./assets/uploading-data/uploading-data.png) You can click on the **Upload your data** button, located in the centre of the screen or at the top left of the sidebar, to start adding files/folders to the app. :::{note} Each time you upload a file or folder to ODE, the application will not ingest the original file/folder from your computer. Instead, it will make a copy of it and add it to the application folder on your laptop. After the ingestion process ends, you can right-click on the file/folder and select the **Open Location** option. By doing so, ODE will redirect you to the exact location where the copy was saved. ::: ![Open file location](./assets/uploading-data/open-location.png) ### Excel, CSV files and folders When clicking on the **Upload your data** button, ODE will display the following dialogue. If you want to upload files or folders, you can do so from the **From Your Computer** section. If you want to add tables that are online, you can do it from **Add External Data**: ![Upload files from your computer](./assets/uploading-data/uploading-data-1.png) You will see there are two options available in the **From your computer** section. To add file/s, click **Select** on the **Add one or more Excel or csv files** feature. If you want to ingest one or many folders, click Select on the **Add one or more folders** box. Once the ingestion process concludes, ODE will add your data to the sidebar of the app: ![Uploading data sidebar](./assets/uploading-data/uploading-data-sidebar.png) Note that while uploading your data, the tool checks your files or folders to find errors according to the validation rules provided by [Frictionless](https://framework.frictionlessdata.io/). Please, check the **Full list of table errors detected** in this guide to learn more. Please keep in mind that since the preview section (datagrid) can only show one tabular file at a time, when uploading folders, and after the ingestion process is done, you will need to click on the folder and select the file you want to visualise on the screen. As soon as you click on a specific file, ODE will start validating your data (looking for possible errors), and the table will be shown on the app. #### Tables published online ODE also allows users to upload online tables. You can upload files from open data portals, Google Sheets or tables from your GitHub repository. To upload online tables, first click the **Upload your data** button and then select the **Add External Data** section: ![Upload online tables](./assets/uploading-data/tables-published-online.png) Now, write or paste the URL to the table and click the **Add** button: ![Upload online table URL input](./assets/uploading-data/tables-published-online-2.png) After that, ODE will start reviewing your file in the background to detect possible errors, and data will be displayed on the main screen. :::{note} Before you upload your online table… 👉🏼 If you are uploading a Google Sheets file, check that the file is published online. If you don’t know how to do it, please visit [this page](https://support.google.com/docs/answer/183965?hl=en&co=GENIE.Platform%3DDesktop) and follow the steps listed there. 👉🏼 For Google Sheets, please make sure you are adding the public version of your file without the HTML term at the end. For example: ✅ https://docs.google.com/spreadsheets/d/1dFVoF6f9VU5pjaGhyyvQaBN0n6ae-iLCtlvsO1N2jhA/edit?gid=0\#gid=0 ❌ https://docs.google.com/spreadsheets/d/e/2PACX-1vQ8w9yb7D-iYEbImb0WD4Kh53\_Yp7H1VOi1bIMcicphWbkrrH9PobXCJhXt9frqyQ/pubhtml 👉🏼 When exporting a file from Google Sheets in CSV and you have columns with numbers, please make sure to use “.” for decimals, instead of commas. Otherwise, Frictionless, the code working behind ODE will interpret the content of your cells with numbers as text. For all tables… 👉🏼 Make sure your file is well-organised (well-formatted: each column must contain a name, there should be no extra rows before the table, or additional elements (like borders or lines) next to the space where the tabular data is located. 👉🏼 Check that the tabular data does not contain cells that are merged. Data producers from Governments, international organizations and internal reports usually add tiles, descriptions, and graphs within sheets, like in this case. If there are extra elements in your file, ODE will ingest your file and show you multiple errors. ::: ================================================ FILE: packaging/linux/opendataeditor.desktop ================================================ [Desktop Entry] # The type of the thing this desktop file refers to Type=Application # The Application Name Name=Open Data Editor # Tooltip comment to show in menus Comment=Data management for humans. # The path (folder) in which the executable is run Path=/opt/opendataeditor # The executable (can include arguments) Exec=/opt/opendataeditor/opendataeditor # The icon we install for the application, use the target filesystem path Icon=org.okfn.opendataeditor ================================================ FILE: packaging/macos/entitlements.mac.plist ================================================ com.apple.security.cs.allow-unsigned-executable-memory com.apple.security.cs.allow-jit ================================================ FILE: packaging/windows/installer.nsi ================================================ ; NSIS Script to create a Windows Installer. ; ; To create a windows installer: ; 1. Install NSIS from https://nsis.sourceforge.io/Download ; 2. Build your application with PyInstaller first: python build.py ; 3. Create the installer: 'C:\Program Files (x86)\NSIS\makensis.exe' .\packaging\windows\installer.nsi ; ; The APP_ID was originaly set by electron-builder in the first versions of the application and we are maintaining it. ; https://www.electron.build/nsis.html#guid-vs-application-name !define APP_ID "42c092cd-67f7-566d-b9a4-980d3103f082" !define APP_NAME "Open Data Editor" !define PUBLISHER "Open Knowledge Foundation" !define INSTALL_DIR "$LOCALAPPDATA\Programs\opendataeditor" ; Required version parameter !ifndef APP_VERSION !error "APP_VERSION must be defined via -DAPP_VERSION=x.y.z command parameter." !endif ; Modern UI setup !include "MUI2.nsh" !include "LogicLib.nsh" !include "FileFunc.nsh" ; To calculate Estimated Size Name "${APP_NAME}" OutFile "opendataeditor-win-${APP_VERSION}.exe" InstallDir "${INSTALL_DIR}" RequestExecutionLevel user ; No admin privileges needed for user-level install ; Registry key for uninstaller !define UNINST_KEY "Software\Microsoft\Windows\CurrentVersion\Uninstall\${APP_ID}" !define MUI_ICON "icon.ico" !define MUI_UNICON "icon.ico" !insertmacro MUI_PAGE_WELCOME !insertmacro MUI_PAGE_INSTFILES !insertmacro MUI_UNPAGE_CONFIRM !insertmacro MUI_UNPAGE_INSTFILES !insertmacro MUI_LANGUAGE "English" Section "Install" ; Remove previous installation RMDir /r "$INSTDIR" ; Create installation directory SetOutPath "$INSTDIR" ; Copy application files from PyInstaller output File /r "..\..\dist\opendataeditor\*.*" ; Calculate installed size ${GetSize} "$INSTDIR" "/S=0K" $0 $1 $2 IntFmt $0 "0x%08X" $0 ; Create shortcuts CreateDirectory "$SMPROGRAMS\${APP_NAME}" CreateShortcut "$SMPROGRAMS\${APP_NAME}\${APP_NAME}.lnk" "$INSTDIR\opendataeditor.exe" CreateShortcut "$SMPROGRAMS\${APP_NAME}\Uninstall.lnk" "$INSTDIR\Uninstall opendataeditor.exe" ; Write uninstaller WriteUninstaller "$INSTDIR\Uninstall opendataeditor.exe" ; Write registry entries WriteRegStr HKCU "${UNINST_KEY}" "DisplayName" "${APP_NAME}" WriteRegStr HKCU "${UNINST_KEY}" "DisplayVersion" "${APP_VERSION}" WriteRegStr HKCU "${UNINST_KEY}" "Publisher" "${PUBLISHER}" WriteRegStr HKCU "${UNINST_KEY}" "UninstallString" '"$INSTDIR\Uninstall opendataeditor.exe" /currentuser' WriteRegStr HKCU "${UNINST_KEY}" "DisplayIcon" "$INSTDIR\opendataeditor.exe" WriteRegStr HKCU "${UNINST_KEY}" "InstallLocation" "$INSTDIR" WriteRegStr HKCU "${UNINST_KEY}" "NoModify" 1 WriteRegStr HKCU "${UNINST_KEY}" "NoRepair" 1 WriteRegDWORD HKCU "${UNINST_KEY}" "EstimatedSize" "$0" SectionEnd Section "Uninstall" ; Remove application files RMDir /r "$INSTDIR" ; Remove shortcuts RMDir /r "$SMPROGRAMS\${APP_NAME}" ; Remove registry entries DeleteRegKey HKCU "${UNINST_KEY}" SectionEnd ================================================ FILE: pyproject.sublime-workspace ================================================ {} ================================================ FILE: pyproject.toml ================================================ [project] name = "opendataeditor" version = "1.7.1" # Update this version number also in ode/__init__.py description = "A no-code application to explore and validate tabular data in a simple way. " readme = "README.md" requires-python = ">=3.13" dependencies = [ "frictionless[excel,github]>=5.18.1", "llama-cpp-python>=0.3.16", "openpyxl>=3.1.5", "pyside6>=6.10.0", "xlrd>=2.0.2", "xlutils>=2.0.0", "xlwt>=1.3.0", ] [project.scripts] ode = "ode.main:main" [build-system] requires = ["uv_build>=0.9.10,<0.10.0"] build-backend = "uv_build" [tool.uv.build-backend] module-name = "ode" [tool.ruff] line-length = 140 [dependency-groups] dev = [ "ipdb>=0.13.13", "mypy>=1.18.2", "myst-parser>=4.0.1", "pyinstaller==6.11.1", "pytest>=9.0.0", "pytest-qt>=4.5.0", "ruff>=0.14.4", "sphinx>=8.2.3", "sphinx-rtd-theme>=3.0.2", ] ================================================ FILE: src/ode/__init__.py ================================================ ================================================ FILE: src/ode/assets/__init__.py ================================================ # Required for PyInstaller to collect all assets ================================================ FILE: src/ode/assets/licenses.json ================================================ [ { "name": "AAL", "path": "https://opensource.org/licenses/AAL", "title": "Attribution Assurance Licenses" }, { "name": "AFL-3.0", "path": "https://opensource.org/licenses/AFL-3.0", "title": "Academic Free License 3.0" }, { "name": "AGPL-3.0", "path": "https://opensource.org/licenses/AGPL-3.0", "title": "GNU Affero General Public License v3" }, { "name": "APL-1.0", "path": "https://opensource.org/licenses/APL-1.0", "title": "Adaptive Public License 1.0" }, { "name": "APSL-2.0", "path": "https://opensource.org/licenses/APSL-2.0", "title": "Apple Public Source License 2.0" }, { "name": "Against-DRM", "path": "https://opendefinition.org/licenses/against-drm", "title": "Against DRM" }, { "name": "Apache-1.1", "path": "https://opensource.org/licenses/Apache-1.1", "title": "Apache Software License 1.1" }, { "name": "Apache-2.0", "path": "https://opensource.org/licenses/Apache-2.0", "title": "Apache Software License 2.0" }, { "name": "Artistic-2.0", "path": "https://opensource.org/licenses/Artistic-2.0", "title": "Artistic License 2.0" }, { "name": "BSD-2-Clause", "path": "https://opensource.org/licenses/BSD-2-Clause", "title": "BSD 2-Clause \"Simplified\" or \"FreeBSD\" License (BSD-2-Clause)" }, { "name": "BSD-3-Clause", "path": "https://opensource.org/licenses/BSD-3-Clause", "title": "BSD 3-Clause \"New\" or \"Revised\" License (BSD-3-Clause)" }, { "name": "BSL-1.0", "path": "https://opensource.org/licenses/BSL-1.0", "title": "Boost Software License 1.0" }, { "name": "BitTorrent-1.1", "path": "https://spdx.org/licenses/BitTorrent-1.1", "title": "BitTorrent Open Source License 1.1" }, { "name": "CATOSL-1.1", "path": "https://opensource.org/licenses/CATOSL-1.1", "title": "Computer Associates Trusted Open Source License 1.1 (CATOSL-1.1)" }, { "name": "CC-BY-4.0", "path": "https://creativecommons.org/licenses/by/4.0/", "title": "Creative Commons Attribution 4.0" }, { "name": "CC-BY-NC-4.0", "path": "https://creativecommons.org/licenses/by-nc/4.0/", "title": "Creative Commons Attribution-NonCommercial 4.0" }, { "name": "CC-BY-NC-ND-4.0", "path": "https://creativecommons.org/licenses/by-nc-nd/4.0/", "title": "Attribution-NonCommercial-NoDerivatives 4.0" }, { "name": "CC-BY-NC-SA-4.0", "path": "https://creativecommons.org/licenses/by-nc-sa/4.0/", "title": "Attribution-NonCommercial-ShareAlike 4.0" }, { "name": "CC-BY-ND-4.0", "path": "https://creativecommons.org/licenses/by-nd/4.0/", "title": "Attribution-NoDerivatives 4.0" }, { "name": "CC-BY-SA-4.0", "path": "https://creativecommons.org/licenses/by-sa/4.0/", "title": "Creative Commons Attribution Share-Alike 4.0" }, { "name": "CC0-1.0", "path": "https://creativecommons.org/publicdomain/zero/1.0/", "title": "CC0 1.0" }, { "name": "CDDL-1.0", "path": "https://opensource.org/licenses/CDDL-1.0", "title": "Common Development and Distribution License 1.0" }, { "name": "CECILL-2.1", "path": "https://opensource.org/licenses/CECILL-2.1", "title": "CeCILL License 2.1" }, { "name": "CNRI-Python", "path": "https://opensource.org/licenses/CNRI-Python", "title": "CNRI Python License" }, { "name": "CPAL-1.0", "path": "https://opensource.org/licenses/CPAL-1.0", "title": "Common Public Attribution License 1.0" }, { "name": "CUA-OPL-1.0", "path": "https://opensource.org/licenses/CUA-OPL-1.0", "title": "CUA Office Public License 1.0" }, { "name": "DSL", "path": "https://opendefinition.org/licenses/dsl", "title": "Design Science License" }, { "name": "ECL-2.0", "path": "https://opensource.org/licenses/ECL-2.0", "title": "Educational Community License 2.0" }, { "name": "EFL-2.0", "path": "https://opensource.org/licenses/EFL-2.0", "title": "Eiffel Forum License 2.0" }, { "name": "EPL-1.0", "path": "https://opensource.org/licenses/EPL-1.0", "title": "Eclipse Public License 1.0" }, { "name": "EPL-2.0", "path": "https://opensource.org/licenses/EPL-2.0", "title": "Eclipse Public License 2.0" }, { "name": "EUDatagrid", "path": "https://opensource.org/licenses/EUDatagrid", "title": "EU DataGrid Software License" }, { "name": "EUPL-1.1", "path": "https://opensource.org/licenses/EUPL-1.1", "title": "European Union Public License 1.1" }, { "name": "Entessa", "path": "https://opensource.org/licenses/Entessa", "title": "Entessa Public License" }, { "name": "FAL-1.3", "path": "https://opendefinition.org/licenses/fal", "title": "Free Art License 1.3" }, { "name": "Fair", "path": "https://opensource.org/licenses/Fair", "title": "Fair License" }, { "name": "Frameworx-1.0", "path": "https://opensource.org/licenses/Frameworx-1.0", "title": "Frameworx License 1.0" }, { "name": "GFDL-1.3-no-cover-texts-no-invariant-sections", "path": "https://opendefinition.org/licenses/gfdl", "title": "GNU Free Documentation License 1.3 with no cover texts and no invariant sections" }, { "name": "GPL-2.0", "path": "https://opensource.org/licenses/GPL-2.0", "title": "GNU General Public License 2.0" }, { "name": "GPL-3.0", "path": "https://opensource.org/licenses/GPL-3.0", "title": "GNU General Public License 3.0" }, { "name": "HPND", "path": "https://opensource.org/licenses/HPND", "title": "Historical Permission Notice and Disclaimer" }, { "name": "IPA", "path": "https://opensource.org/licenses/IPA", "title": "IPA Font License" }, { "name": "IPL-1.0", "path": "https://opensource.org/licenses/IPL-1.0", "title": "IBM Public License 1.0" }, { "name": "ISC", "path": "https://opensource.org/licenses/ISC", "title": "ISC License" }, { "name": "Intel", "path": "https://opensource.org/licenses/Intel", "title": "Intel Open Source License" }, { "name": "LGPL-2.1", "path": "https://opensource.org/licenses/LGPL-2.1", "title": "GNU Lesser General Public License 2.1" }, { "name": "LGPL-3.0", "path": "https://opensource.org/licenses/LGPL-3.0", "title": "GNU Lesser General Public License 3.0" }, { "name": "LO-FR-2.0", "path": "https://www.etalab.gouv.fr/licence-ouverte-open-licence", "title": "Open License 2.0" }, { "name": "LPL-1.0", "path": "https://opensource.org/licenses/LPL-1.0", "title": "Lucent Public License (\"Plan9\") 1.0" }, { "name": "LPL-1.02", "path": "https://opensource.org/licenses/LPL-1.02", "title": "Lucent Public License 1.02" }, { "name": "LPPL-1.3c", "path": "https://opensource.org/licenses/LPPL-1.3c", "title": "LaTeX Project Public License 1.3c" }, { "name": "MIT", "path": "https://opensource.org/licenses/MIT", "title": "MIT License" }, { "name": "MPL-1.0", "path": "https://opensource.org/licenses/MPL-1.0", "title": "Mozilla Public License 1.0" }, { "name": "MPL-1.1", "path": "https://opensource.org/licenses/MPL-1.1", "title": "Mozilla Public License 1.1" }, { "name": "MPL-2.0", "path": "https://opensource.org/licenses/MPL-2.0", "title": "Mozilla Public License 2.0" }, { "name": "MS-PL", "path": "https://opensource.org/licenses/MS-PL", "title": "Microsoft Public License" }, { "name": "MS-RL", "path": "https://opensource.org/licenses/MS-RL", "title": "Microsoft Reciprocal License" }, { "name": "MirOS", "path": "https://opensource.org/licenses/MirOS", "title": "MirOS Licence" }, { "name": "Motosoto", "path": "https://opensource.org/licenses/Motosoto", "title": "Motosoto License" }, { "name": "Multics", "path": "https://opensource.org/licenses/Multics", "title": "Multics License" }, { "name": "NASA-1.3", "path": "https://opensource.org/licenses/NASA-1.3", "title": "NASA Open Source Agreement 1.3" }, { "name": "NCSA", "path": "https://opensource.org/licenses/NCSA", "title": "University of Illinois/NCSA Open Source License" }, { "name": "NGPL", "path": "https://opensource.org/licenses/NGPL", "title": "Nethack General Public License" }, { "name": "NPOSL-3.0", "path": "https://opensource.org/licenses/NPOSL-3.0", "title": "Non-Profit Open Software License 3.0" }, { "name": "NTP", "path": "https://opensource.org/licenses/NTP", "title": "NTP License" }, { "name": "Naumen", "path": "https://opensource.org/licenses/Naumen", "title": "Naumen Public License" }, { "name": "Nokia", "path": "https://opensource.org/licenses/Nokia", "title": "Nokia Open Source License" }, { "name": "OCLC-2.0", "path": "https://opensource.org/licenses/OCLC-2.0", "title": "OCLC Research Public License 2.0" }, { "name": "ODC-BY-1.0", "path": "https://opendefinition.org/licenses/odc-by", "title": "Open Data Commons Attribution License 1.0" }, { "name": "ODbL-1.0", "path": "https://opendefinition.org/licenses/odc-odbl", "title": "Open Data Commons Open Database License 1.0" }, { "name": "OFL-1.1", "path": "https://opensource.org/licenses/OFL-1.1", "title": "Open Font License 1.1" }, { "name": "OGL-Canada-2.0", "path": "https://open.canada.ca/en/open-government-licence-canada", "title": "Open Government License 2.0 (Canada)" }, { "name": "OGL-UK-1.0", "path": "https://www.nationalarchives.gov.uk/doc/open-government-licence/version/1/", "title": "Open Government Licence 1.0 (United Kingdom)" }, { "name": "OGL-UK-2.0", "path": "https://www.nationalarchives.gov.uk/doc/open-government-licence/version/2/", "title": "Open Government Licence 2.0 (United Kingdom)" }, { "name": "OGL-UK-3.0", "path": "https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/", "title": "Open Government Licence 3.0 (United Kingdom)" }, { "name": "OGTSL", "path": "https://opensource.org/licenses/OGTSL", "title": "Open Group Test Suite License" }, { "name": "OSL-3.0", "path": "https://opensource.org/licenses/OSL-3.0", "title": "Open Software License 3.0" }, { "name": "PDDL-1.0", "path": "https://opendefinition.org/licenses/odc-pddl", "title": "Open Data Commons Public Domain Dedication and Licence 1.0" }, { "name": "PHP-3.0", "path": "https://opensource.org/licenses/PHP-3.0", "title": "PHP License 3.0" }, { "name": "PostgreSQL", "path": "https://opensource.org/licenses/PostgreSQL", "title": "PostgreSQL License" }, { "name": "Python-2.0", "path": "https://opensource.org/licenses/Python-2.0", "title": "Python License 2.0" }, { "name": "QPL-1.0", "path": "https://opensource.org/licenses/QPL-1.0", "title": "Q Public License 1.0" }, { "name": "RPL-1.5", "path": "https://opensource.org/licenses/RPL-1.5", "title": "Reciprocal Public License 1.5" }, { "name": "RPSL-1.0", "path": "https://opensource.org/licenses/RPSL-1.0", "title": "RealNetworks Public Source License 1.0" }, { "name": "RSCPL", "path": "https://opensource.org/licenses/RSCPL", "title": "Ricoh Source Code Public License" }, { "name": "SISSL", "path": "https://opensource.org/licenses/SISSL", "title": "Sun Industry Standards Source License 1.1" }, { "name": "SPL-1.0", "path": "https://opensource.org/licenses/SPL-1.0", "title": "Sun Public License 1.0" }, { "name": "SimPL-2.0", "path": "https://opensource.org/licenses/SimPL-2.0", "title": "Simple Public License 2.0" }, { "name": "Sleepycat", "path": "https://opensource.org/licenses/Sleepycat", "title": "Sleepycat License" }, { "name": "Talis", "path": "https://opendefinition.org/licenses/tcl", "title": "Talis Community License" }, { "name": "Unlicense", "path": "https://unlicense.org/", "title": "Unlicense" }, { "name": "VSL-1.0", "path": "https://opensource.org/licenses/VSL-1.0", "title": "Vovida Software License 1.0" }, { "name": "W3C", "path": "https://opensource.org/licenses/W3C", "title": "W3C License" }, { "name": "WXwindows", "path": "https://opensource.org/licenses/WXwindows", "title": "wxWindows Library License" }, { "name": "Watcom-1.0", "path": "https://opensource.org/licenses/Watcom-1.0", "title": "Sybase Open Watcom Public License 1.0" }, { "name": "Xnet", "path": "https://opensource.org/licenses/Xnet", "title": "X.Net License" }, { "name": "ZPL-2.0", "path": "https://opensource.org/licenses/ZPL-2.0", "title": "Zope Public License 2.0" }, { "name": "Zlib", "path": "https://opensource.org/licenses/Zlib", "title": "zlib/libpng license" }, { "name": "dli-model-use", "path": "http://data.library.ubc.ca/datalib/geographic/DMTI/license.html", "title": "Statistics Canada: Data Liberation Initiative (DLI) - Model Data Use Licence" }, { "name": "geogratis", "path": "http://geogratis.gc.ca/geogratis/licenceGG", "title": "Geogratis" }, { "name": "hesa-withrights", "path": "https://web.archive.org/web/20131009082944/http://www.hesa.ac.uk/index.php?option=com_content&task=view&id=2619&Itemid=209", "title": "Higher Education Statistics Agency Copyright with data.gov.uk rights" }, { "name": "localauth-withrights", "path": "", "title": "Local Authority Copyright with data.gov.uk rights" }, { "name": "met-office-cp", "path": "https://www.metoffice.gov.uk/climatechange/science/monitoring/ukcp09/UKCIP08_license_agreement_130709.pdf", "title": "Met Office UK Climate Projections Licence Agreement" }, { "name": "mitre", "path": "https://opensource.org/licenses/CVW", "title": "MITRE Collaborative Virtual Workspace License (CVW License)" }, { "name": "notspecified", "path": "", "title": "License Not Specified" }, { "name": "other-at", "path": "", "title": "Other (Attribution)" }, { "name": "other-closed", "path": "", "title": "Other (Not Open)" }, { "name": "other-nc", "path": "", "title": "Other (Non-Commercial)" }, { "name": "other-open", "path": "", "title": "Other (Open)" }, { "name": "other-pd", "path": "", "title": "Other (Public Domain)" }, { "name": "ukclickusepsi", "path": "", "title": "UK Click Use PSI" }, { "name": "ukcrown", "path": "", "title": "UK Crown Copyright" }, { "name": "ukcrown-withrights", "path": "", "title": "UK Crown Copyright with data.gov.uk rights" }, { "name": "ukpsi", "path": "https://opendefinition.org/licenses/ukpsi", "title": "UK PSI Public Sector Information" } ] ================================================ FILE: src/ode/assets/style.qss ================================================ /* Open Data Editor main Style Sheet. The application uses a fusion style set when creating the QMainWindow object. This stylesheets complements the fusion style with colors and branding. Open Data Editor is design for light-mode so we are enforcing it by explicitly setting the main widgets and components background to white. */ /* Enforcing Light Mode of main elements. */ Content, FrictionlessResourceMetadataWidget, SelectWidget, Toolbar, FieldsForm, SingleFieldForm, SchemaForm, QWidget#fields_form_container, QTableView, QTableView QHeaderView, QTableView QTableCornerButton, QTreeView, QDialog, QPushButton, QComboBox, QComboBox QAbstractItemView, QLineEdit, QLabel, QTabWidget, QTabBar, QScrollBar, QSpinBox, QListWidget, QTextEdit, QSrollArea, QPlainTextEdit, QWidget#central_widget, QGroupBox { background: #FFF; color: #404040; font-size: 19px; /* multi-os support */ } * { gridline-color: lightgrey; } /* multi-os support */ .QComboBox:disabled, QLineEdit:disabled { background: #f0f0f0; } ErrorsReportButton { border: 0; } ErrorsReportButton:hover { background-color: #F0F0F0; } ErrorsReportButton QLabel { font-size: 16px; font-weight: 600; color: #4C5564; background: transparent; } ErrorsReportButton QLabel:disabled { color: grey; } ErrorsReportButton QLabel[error="true"] { border: 1px solid #FECBCA; color: #D32F2F; padding: 0px; } Toolbar QPushButton { font-size: 16px; font-weight: 600; border: 0px solid; color: #4C5564; background: #FFFFFF; padding: 6px 8px; } Toolbar QPushButton:hover { background-color: #F0F0F0; } Toolbar QPushButton:pressed { background-color: #F0F0F0; } Toolbar QPushButton[active="true"]{ border: 2px solid #0078D7; background-color: #F0F0F0; outline: none; } Toolbar QComboBox#excelSheetCombo, Toolbar QLabel#excelSheetLabel { font-size: 16px; font-weight: 600; color: #4C5564; } QPushButton#button_save, QPushButton#button_export, QPushButton#button_ai { font-size: 14px; font-weight: 500; color: #FFF; background-color: #000; border-style: outset; border-width: 1px; border-radius: 4px; border-color: #000; padding: 6px 8px; } QPushButton#button_save:hover, QPushButton#button_export:hover, QPushButton#button_ai:hover { color: #FFF; background: #0288D1; border-color: #0288d1; } QPushButton#button_save:pressed, QPushButton#button_export:pressed, QPushButton#button_ai:pressed { color: #FFF; background-color: #000; border-color: #FFF; } QPushButton#button_save:disabled, QPushButton#button_export:disabled, QPushButton#button_ai:disabled { background-color: #F0F0F0; border-color: #F0F0F0; color: #4C5564; } Sidebar { border-right: 1px solid #000; } Sidebar QPushButton#button_upload { font-size: 14px; font-weight: 500; text-align: center; color: #0288D1; border-style: outset; border-width: 1px; border-radius: 4px; border-color: #0288d1; padding-top: 10px; padding-bottom: 10px; padding-left: 15px; padding-right: 15px; margin-top: 22px; margin-bottom: 22px; } Sidebar QPushButton#button_upload:hover { color: #FFF; background: #0288D1; } Sidebar QPushButton#button_upload:pressed { color: #0288D1; background: #FFF; } Sidebar QTreeView { border: 1px solid #d0d0d0; } Sidebar QTreeView::item:hover { color: #FFF; background: black; } Sidebar QTreeView::item:selected { color: #FFF; background: gray; } Sidebar QPushButton { border: 0px; padding: 3px; text-align: left; } Sidebar QPushButton:hover { background: #D0D0D0; } Sidebar QPushButton:pressed { background: #FFFFFF; } Welcome QPushButton { font-size: 14px; font-weight: 500; color: #FFFFFF; background: #000000; border-style: outset; border-width: 1px; border-radius: 4px; padding: 10px 15px; } Welcome QPushButton:hover { color: #FFFFFF; background: #0288D1; border-color: #0288D1; } Welcome QPushButton:pressed { color: #FFFFFF; background: #000000; border-color: #000000; } FieldsForm QScrollArea { border: none; } QHeaderView::section { background-color: rgb(240, 240, 240); border: 1px solid rgb(200, 200, 200); } QTableCornerButton::section { background-color: rgb(240, 240, 240); border: 1px solid rgb(200, 200, 200); } DataUploadDialog, DeleteDialog, RenameDialog, DownloadDialog, LlamaDialog, LLMWarningDialog, ColumnMetadataDialog, LlamaDownloadingDialog { border: 2px solid #000000; } DataUploadDialog QPushButton, DeleteDialog QPushButton, RenameDialog QPushButton, DownloadDialog QPushButton, LLMWarningDialog QPushButton, ColumnMetadataDialog QPushButton, LlamaDialog QPushButton, LlamaDownloadDialog QPushButton { border: 2px solid #000000; background-color: #F0F0F0; font-size: 16px; font-weight: 500; color: #FFFFFF; background: #000000; border-style: outset; border-width: 1px; border-radius: 4px; padding: 10px 15px; text-align: center; } DeleteDialog QPushButton, RenameDialog QPushButton, DownloadDialog QPushButton, LLMWarningDialog QPushButton, ColumnMetadataDialog QPushButton, LlamaDialog QPushButton, LlamaDownloadDialog QPushButton { padding: 5px; } DataUploadDialog QPushButton:hover, DeleteDialog QPushButton:hover, RenameDialog QPushButton:hover, DownloadDialog QPushButton:hover, ColumnMetadataDialog QPushButton:hover, LLMWarningDialog QPushButton:hover, LlamaDialog QPushButton:hover, LlamaDownloadDialog QPushButton:hover { background: #0288D1; border-color: #0288d1; } DataUploadDialog QPushButton:pressed, DeleteDialog QPushButton:pressed, RenameDialog QPushButton:pressed, DownloadDialog QPushButton:pressed, ColumnMetadataDialog QPushButton:pressed, LLMWarningDialog QPushButton:pressed, LlamaDialog QPushButton:pressed, LlamaDownloadDialog QPushButton:pressed { color: #FFFFFF; background: #000000; border-color: #000000; } DataUploadDialog QPushButton:disabled, DeleteDialog QPushButton:disabled, RenameDialog QPushButton:disabled, DownloadDialog QPushButton:disabled, ColumnMetadataDialog QPushButton:disabled, LLMWarningDialog QPushButton:disabled, LlamaDialog QPushButton:disabled, LlamaDownloadDialog QPushButton:disabled { background-color: #F0F0F0; border-color: #F0F0F0; color: #4C5564; } QTableView::item:selected { background: white; color: black; border: 2px solid #646464; } /* Keyboard Focus Styles for Accessibility */ /* General focus styles for primary UI components */ QPushButton:focus { border: 2px solid #0078D7; outline: none; } QLineEdit:focus { border: 2px solid #0078D7; outline: none; } QComboBox:focus { border: 2px solid #0078D7; outline: none; } QTreeView:focus { border: 2px solid #0078D7; outline: none; } QTableView:focus { border: 2px solid #0078D7; outline: none; } /* Focus styles for save/publish buttons */ QPushButton#button_save:focus, QPushButton#button_export:focus { border: 3px solid #0078D7; outline: none; } /* Focus styles for sidebar elements */ Sidebar QPushButton#button_upload:focus { border: 2px solid #0078D7; outline: none; } Sidebar QPushButton:focus { border: 2px solid #0078D7; background: #D0D0D0; outline: none; } Sidebar QTreeView:focus { border: 2px solid #0078D7; outline: none; } /* Focus styles for welcome screen buttons */ Welcome QPushButton:focus { border: 2px solid #0078D7; outline: none; } /* Focus styles for dialog buttons */ DataUploadDialog QPushButton:focus, DeleteDialog QPushButton:focus, RenameDialog QPushButton:focus, DownloadDialog QPushButton:focus, ColumnMetadataDialog QPushButton:focus, LLMWarningDialog QPushButton:focus, LlamaDialog QPushButton:focus, LlamaDownloadDialog QPushButton:focus { border: 2px solid #0078D7; outline: none; } /* Focus styles for table items */ QTableView::item:focus { border: 2px solid #0078D7; outline: none; } LLMWarningDialog QCheckBox { color: #404040; font-size: 18px; background-color: white; } QPushButton#deleteButton { background-color: black; color: white; } QPushButton#deleteButton:hover { background-color: #0078D7; } QPushButton#deleteButton:disabled { color: gray; background-color: #f0f0f0; } /* Download Button */ QPushButton#downloadButton { background-color: #black; color: white; } QPushButton#downloadButton:hover { background-color: #0078D7; } QPushButton#downloadButton:disabled { color: gray; background-color: #f0f0f0; } QWidget#llamaDownloadModelRow { border: 1px solid #ccc; border-radius: 4px; background-color: #f9f9f9; margin: 2px; } ================================================ FILE: src/ode/assets/translations/de.ts ================================================ ColumnMetadataDialog Column name cannot be empty Spaltenname darf nicht leer sein There is another column in the table with the same name. Please choose a different one Es gibt bereits eine Spalte in der Tabelle mit demselben Namen. Bitte wählen Sie einen anderen Save Speichern Cancel Abbrechen DataUploadDialog Please paste a valid URL. Bitte fügen Sie eine gültige URL ein. Please paste a valid URL starting with http:// or https://. Bitte fügen Sie eine gültige URL ein, die mit http:// oder https:// beginnt. Error: The Google Sheets URL is not valid or the table is not publicly available. Fehler: Die Google Sheets URL ist nicht gültig oder die Tabelle ist nicht öffentlich verfügbar. Error: The URL is not associated with a table Fehler: Die URL ist nicht mit einer Tabelle verknüpft Upload your data Laden Sie Ihre Daten hoch Add one or more Excel or csv files Fügen Sie eine oder mehrere Excel- oder CSV-Dateien hinzu Add a folder Einen Ordner hinzufügen Paste your Google Sheet or csv link to create a local copy in the Open Data Editor Fügen Sie Ihren Google Sheet oder CSV-Link ein, um eine lokale Kopie im Open Data Editor zu erstellen Add Local Files Lokale Dateien hinzufügen Select Auswählen Link to the external table: Link zur externen Tabelle: Enter or paste URL URL eingeben oder einfügen Add Hinzufügen Add External Data Externe Daten hinzufügen DataViewer Preview not available for this item. Vorschau für dieses Element nicht verfügbar. DataWorker Reading file... Datei wird gelesen... Checking errors... Fehler werden überprüft... Drawing table... Tabelle wird gezeichnet... Read and error checking finished. Lesen und Fehlerüberprüfung abgeschlossen. DeleteDialog Cancel Abbrechen Ok OK Are you sure you want to delete this item? Sind Sie sicher, dass Sie dieses Element löschen möchten? Delete file Datei löschen DownloadDialog Download Herunterladen Please, select one of the following options: Bitte wählen Sie eine der folgenden Optionen: Download file Datei herunterladen Download file with errors Datei mit Fehlern herunterladen File downloaded successfully to: {} Datei erfolgreich heruntergeladen nach: {} Success Erfolg Error downloading file: {} Fehler beim Herunterladen der Datei: {} ErrorsMessages Missing header Fehlende Kopfzeile Duplicated header Doppelte Kopfzeile Empty row Leere Zeile Type mismatch Typkonflikt Missing value Fehlender Wert Extra cell Zusätzliche Zelle Blank Label Leeres Label A column in the header row has no name. Every column should have a unique, non-empty header. Eine Spalte in der Kopfzeile hat keinen Namen. Jede Spalte sollte einen eindeutigen, nicht-leeren Kopfzeilennamen haben. Two or more columns share the same name. Column names must be unique. Zwei oder mehr Spalten teilen sich denselben Namen. Spaltennamen müssen eindeutig sein. This row has no data. Rows should contain at least one cell with data. Diese Zeile enthält keine Daten. Zeilen sollten mindestens eine Zelle mit Daten enthalten. A cell value doesn't match the expected data type or format for the column. Ein Zellenwert entspricht nicht dem erwarteten Datentyp oder Format für die Spalte. This cell is missing data Diese Zelle enthält keine Daten This row has more values compared to the header row. Diese Zeile hat mehr Werte im Vergleich zur Kopfzeile. A label in the header row is missing a value. Label should be provided and not be blank. Ein Label in der Kopfzeile hat keinen Wert. Das Label sollte angegeben werden und nicht leer sein. ErrorsWidget Please note that the ODE currently detects errors in tables, with a maximum of Bitte beachten Sie, dass der ODE derzeit Fehler in Tabellen erkennt, mit einem Maximum von FrictionlessTableModel Data Daten LLMWarningDialog AI assistant KI-Assistent Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more Willkommen beim KI-Assistenten des ODE! Diese Funktion hilft Ihnen dabei, bessere Beschreibungen für die Spalten Ihrer Tabelle zu generieren und auch Fragen zur Datenanalyse. Um zu beginnen, müssen Sie die KI-Datei auf Ihrem Computer installieren. Nach der Installation läuft alles lokal, was bedeutet, dass Ihre Daten immer privat und sicher bleiben. Mehr erfahren Don't show again Nicht mehr anzeigen Cancel Abbrechen Ok OK LlamaDialog Generating response... Antwort wird generiert... Execute Ausführen Error Fehler AI assistant KI-Assistent Stop execution Ausführung stoppen Results will be displayed here... Ergebnisse werden hier angezeigt... LlamaDownloadDialog AI assistant KI-Assistent To start using the AI assistant, please download the following model. Um den KI-Assistenten zu verwenden, laden Sie bitte das folgende Modell herunter. The ODE will save the file in this location: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Der ODE wird die Datei an diesem Speicherort speichern: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Next Weiter Delete Löschen Download Herunterladen File exists Datei existiert Do you want to delete it and download it again? Möchten Sie sie löschen und erneut herunterladen? Downloading model Modell wird heruntergeladen Cancel Abbrechen LLM Model Download Progress LLM-Modell Download-Fortschritt Error Occurred Fehler ist aufgetreten Confirm Deletion Löschung bestätigen Are you sure you want to delete {AI_MODEL.name}? Sind Sie sicher, dass Sie {AI_MODEL.name} löschen möchten? LoadingDialog Loading Laden Loading... Laden... MainWindow Error Fehler File Datei Ready. Bereit. Error initializing the LLM: Fehler beim Initialisieren des LLM: Add Hinzufügen File/Folder Datei/Ordner External URL Externe URL View Ansicht Errors panel Fehlerpanel Source panel Quellpanel Help Hilfe User Guide Benutzerhandbuch Report an Issue Ein Problem melden View logs Logs anzeigen About Über Language changed. Sprache geändert. File and Metadata changes saved. Datei- und Metadatenänderungen gespeichert. Last 100 Lines Letzte 100 Zeilen Close Schließen Copy to Clipboard In Zwischenablage kopieren Downloading data with errors... Daten mit Fehlern werden heruntergeladen... File downloaded successfully to: {} Datei erfolgreich heruntergeladen nach: {} Success Erfolg MetadataForm Column Name: Spaltenname: Data Type: Datentyp: Description: Beschreibung: Flag empty cells as errors?: Leere Zellen als Fehler markieren?: Min. Characters in cells: Min. Zeichen in Zellen: Max. Characters in cell Max. Zeichen in Zelle RenameDialog Rename file Datei umbenennen Rename item to: Element umbenennen zu: Cancel Abbrechen OK OK Sidebar Upload your data Laden Sie Ihre Daten hoch User guide Benutzerhandbuch Report an issue Ein Problem melden Rename Umbenennen Open File in Location Datei im Speicherort öffnen Delete Löschen Operation not permitted. Vorgang nicht zulässig. File with this name already exists. Datei mit diesem Namen existiert bereits. Item renamed successfuly. Element erfolgreich umbenannt. Item deleted successfuly. Element erfolgreich gelöscht. Error Fehler Source is a file but destination a directory. Die Quelle ist eine Datei, das Ziel jedoch ein Verzeichnis. Source is a directory but destination a file. Die Quelle ist ein Verzeichnis, das Ziel jedoch eine Datei. SourceViewer This view is only available for CSV files. Diese Ansicht ist nur für CSV-Dateien verfügbar. Toolbar Data Daten Errors Report Fehlerbericht Source code Quellcode Export Exportieren Save changes Änderungen speichern AI KI Sheet: Blatt: Welcome The ODE supports Excel & csv files Der ODE unterstützt Excel- & CSV-Dateien You can also add links to online tables Sie können auch Links zu Online-Tabellen hinzufügen Upload your data Laden Sie Ihre Daten hoch ================================================ FILE: src/ode/assets/translations/es.ts ================================================ ColumnMetadataDialog Column name cannot be empty El nombre de la columna no puede estar vacío There is another column in the table with the same name. Please choose a different one Hay otra columna en la tabla con el mismo nombre. Por favor elige otro Save Guardar Cancel Cancelar DataUploadDialog Please paste a valid URL. Por favor ingresa una URL válida. Please paste a valid URL starting with http:// or https://. Por favor ingresa una URL válida que empiece con http:// o https://. Error: The Google Sheets URL is not valid or the table is not publicly available. Error: La URL de la Planilla de Google no es válida o su contenido no es público. Error: The URL is not associated with a table Error: La URL no está asociada a una tabla Upload your data Carga tus archivos Add one or more Excel or csv files Agrega uno o más archivos Excel of CSV Add a folder Agrega una carpeta Select Seleccionar Link to the external table: Enlace a la tabla externa: Enter or paste URL Introduce o pega la URL Paste your Google Sheet or csv link to create a local copy in the Open Data Editor Pega el enlace de tu hoja de cálculo de Google o CSV para crear una copia local en el Editor de Datos Abiertos Add Agregar Add Local Files Agregar archivos locales Add External Data Agregar Datos Externos DataViewer Preview not available for this item. Vista previa no disponible para este ítem. DataWorker Reading file... Leyendo el archivo... Checking errors... Comprobando errores... Read and error checking finished. Lectura y comprobación de errores finalizada. Drawing table... Renderizando la tabla... DeleteDialog Cancel Cancelar Ok Ok Are you sure you want to delete this item? ¿Está seguro de que desea eliminar este elemento? Delete file Eliminar archivo DownloadDialog Download Descargar Please, select one of the following options: Por favor, selecciona una de las siguientes opciones: Download file Descargar archivo Download file with errors Descargar archivo con errores File downloaded successfully to: {} Archivo descargado exitosamente en: {} Error downloading file: {} Error descargando el archivo: {} Success Éxito ErrorsMessages Missing header Falta el encabezado Duplicated header Encabezado duplicado Empty row Fila vacía Type mismatch Error de tipo Missing value Valor faltante Extra cell Celda adicional Blank Label Etiqueta en blanco A column in the header row has no name. Every column should have a unique, non-empty header. Una columna en la fila de encabezado no tiene nombre. Cada columna debe tener un encabezado único y no vacío. Two or more columns share the same name. Column names must be unique. Dos o más columnas comparten el mismo nombre. Los nombres de columna deben ser únicos. This row has no data. Rows should contain at least one cell with data. Esta fila no tiene datos. Las filas deben contener al menos una celda con datos. A cell value doesn't match the expected data type or format for the column. Un valor de celda no coincide con el tipo de dato o formato esperado para la columna. This cell is missing data Esta celda carece de datos This row has more values compared to the header row. Esta fila tiene más valores en comparación con la fila de encabezado. A label in the header row is missing a value. Label should be provided and not be blank. Falta un valor en una etiqueta de la fila de encabezado. La etiqueta debe proporcionarse y no estar en blanco. ErrorsWidget Please note that the ODE currently detects errors in tables, with a maximum of Por favor, ten en cuenta que la ODE detecta actualmente errores en las tablas, con un máximo de FrictionlessTableModel Data Datos LLMWarningDialog AI assistant Asistente de IA Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more ¡Bienvenido al asistente de IA de ODE! Esta función te ayudará a generar mejores descripciones para las columnas de tu tabla y también preguntas para el análisis de datos. Para comenzar, necesitarás instalar el archivo de IA en tu computadora. Una vez instalado, todo se ejecutará localmente, lo que significa que tus datos siempre permanecerán privados y seguros. Más información Don't show again No mostrar de nuevo Cancel Cancelar Ok Ok LlamaDialog Execute Ejecutar Generating response... Generando respuesta... Error Error AI assistant Asistente de IA Stop execution Detener ejecución Results will be displayed here... Los resultados se mostrarán aquí... LlamaDownloadDialog Delete Eliminar Download Descargar AI assistant Asistente de IA To start using the AI assistant, please download the following model. Para comenzar a usar el asistente de IA, por favor descarga el siguiente modelo. The ODE will save the file in this location: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> El ODE guardará el archivo en esta carpeta: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Next Siguiente File exists El archivo ya existe Do you want to delete it and download it again? ¿Quieres eliminarlo y volver a descargarlo? Downloading model Descargando modelo Cancel Cancelar LLM Model Download Progress Progreso de descarga del modelo LLM Error Occurred Ocurrió un error Confirm Deletion Confirmar eliminación Are you sure you want to delete {AI_MODEL.name}? ¿Estás seguro de que quieres eliminar {AI_MODEL.name}? LoadingDialog Loading Cargando Loading... Cargando... MainWindow Ready. Listo. Error Error Error initializing the LLM: Error al inicializar el LLM: File Archivo Add Agregar File/Folder Archivo/Carpeta External URL URL Externa View Ver Downloading data with errors... Descargando datos con errores... File downloaded successfully to: {} Archivo descargado exitosamente en: {} Success Éxito Errors panel Panel de Errores Source panel Panel de Fuente Help Ayuda User Guide Guía de Usuario Report an Issue Reportar un problema About Acerca de View logs Ver logs Language changed. Lenguaje cambiado. File and Metadata changes saved. Archivo y Metadatos guardados. Last 100 Lines Últimas 100 líneas Close Cerrar Copy to Clipboard Copiar al portapapeles MetadataForm Column Name: Nombre de la columna: Data Type: Tipo de dato: Description: Descripción: Flag empty cells as errors?: ¿Marcar celdas vacías como errores?: Min. Characters in cells: Caracteres mínimos en las celdas: Max. Characters in cell Caracteres máximos en la celda RenameDialog Rename file Renombrar archivo Rename item to: Renombrar archivo a: Cancel Cancelar OK OK Sidebar Upload your data Carga tus archivos User guide Guía de Usuario Report an issue Reportar un problema Rename Renombrar Open File in Location Abrir la ubicacion de archivo Delete Eliminar Error Error Source is a file but destination a directory. El origen es un archivo pero el destino es un directorio. Source is a directory but destination a file. El origen es un directorio pero el destino es un archivo. Operation not permitted. Operación no permitida. File with this name already exists. Ya existe un archivo con este nombre. Item renamed successfuly. Item renombrado exitosamente. Item deleted successfuly. Item eliminado exitosamente. SourceViewer This view is only available for CSV files. Esta vista sólo está disponible para archivos CSV. Toolbar Sheet: Hoja: Data Datos Errors Report Reporte de Errores Source code Codigo fuente Export Exportar Save changes Guardar cambios AI IA Welcome The ODE supports Excel & csv files El ODE admite archivos Excel y CSV You can also add links to online tables También puedes agregar enlaces a tablas en línea Upload your data Carga tus archivos ================================================ FILE: src/ode/assets/translations/fr.ts ================================================ ColumnMetadataDialog Column name cannot be empty Le nom de la colonne ne peut pas être vide There is another column in the table with the same name. Please choose a different one Il y a une autre colonne dans le tableau avec le même nom. Veuillez en choisir un autre Save Enregistrer Cancel Annuler DataUploadDialog Please paste a valid URL. Veuillez coller une URL valide. Please paste a valid URL starting with http:// or https://. Veuillez coller une URL valide commençant par http:// ou https://. Error: The Google Sheets URL is not valid or the table is not publicly available. Erreur: L'URL Google Sheets n'est pas valide ou le tableau n'est pas accessible publiquement. Error: The URL is not associated with a table Erreur: L'URL n'est pas associée à un tableau Upload your data Télécharger vos données Add one or more Excel or csv files Ajouter un ou plusieurs fichiers Excel ou csv Add a folder Ajouter un dossier Select Sélectionner Link to the external table: Lien vers le tableau externe: Enter or paste URL Entrez ou collez l'URL Paste your Google Sheet or csv link to create a local copy in the Open Data Editor Collez le lien de votre feuille de calcul Google ou fichier CSV pour créer une copie locale dans l'Éditeur de Données Ouvertes. Add Ajouter Add Local Files Ajouter fichiers Add External Data Ajouter des données externes DataViewer Preview not available for this item. Aperçu non disponible pour cet article. DataWorker Reading file... Lecture du fichier... Checking errors... Vérification des erreurs... Read and error checking finished. Lecture et vérification des erreurs terminées. Drawing table... Rendu du tableau... DeleteDialog Cancel Annuler Ok Ok Are you sure you want to delete this item? Êtes-vous sûr de vouloir supprimer cet élément? Delete file Supprimer le fichier DownloadDialog Download Télécharger Please, select one of the following options: Veuillez sélectionner l'une des options suivantes : Download file Télécharger le fichier Download file with errors Télécharger le fichier avec erreurs File downloaded successfully to: {} Fichier téléchargé avec succès dans : {} Error downloading file: {} Erreur lors du téléchargement du fichier : {} Success Succès ErrorsMessages Missing header En-tête manquant Duplicated header En-tête dupliqué Empty row Ligne vide Type mismatch Type incompatible Missing value Valeur manquante Extra cell Cellule supplémentaire Blank Label Étiquette vide A column in the header row has no name. Every column should have a unique, non-empty header. Une colonne dans la ligne d'en-tête n'a pas de nom. Chaque colonne doit avoir un en-tête unique et non vide. Two or more columns share the same name. Column names must be unique. Deux colonnes ou plus partagent le même nom. Les noms de colonnes doivent être uniques. This row has no data. Rows should contain at least one cell with data. Cette ligne ne contient aucune donnée. Les lignes doivent contenir au moins une cellule avec des données. A cell value doesn't match the expected data type or format for the column. Une valeur de cellule ne correspond pas au type de données ou au format attendu pour la colonne. This cell is missing data Cette cellule ne contient pas de données This row has more values compared to the header row. Cette ligne a plus de valeurs que la ligne d'en-tête. A label in the header row is missing a value. Label should be provided and not be blank. Une étiquette dans la ligne d'en-tête n'a pas de valeur. L'étiquette doit être fournie et ne pas être vide. ErrorsWidget Please note that the ODE currently detects errors in tables, with a maximum of Veuillez noter que l'ODE détecte actuellement des erreurs dans les tableaux, avec un maximum de FrictionlessTableModel Data Données LLMWarningDialog AI assistant Assistant IA Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more Bienvenue dans l'assistant IA de l'ODE ! Cette fonctionnalité vous aidera à générer de meilleures descriptions pour les colonnes de votre tableau ainsi que des questions pour l'analyse des données. Pour commencer, vous devrez installer le fichier IA sur votre ordinateur. Une fois installé, tout fonctionnera localement, ce qui signifie que vos données restent toujours privées et sécurisées. En savoir plus Don't show again Ne plus afficher Cancel Annuler Ok Ok LlamaDialog Execute Exécuter Generating response... Génération de la réponse... Error Erreur AI assistant Assistant IA Stop execution Arrêter l'exécution Results will be displayed here... Les résultats seront affichés ici... LlamaDownloadDialog AI assistant Assistant IA To start using the AI assistant, please download the following model. Pour commencer à utiliser l'assistant IA, veuillez télécharger le modèle suivant. The ODE will save the file in this location: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> L'ODE enregistrera le fichier à cet emplacement : <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Next Suivant Delete Supprimer Download Télécharger File exists Fichier existant Do you want to delete it and download it again? Voulez-vous le supprimer et le télécharger à nouveau ? Downloading model Téléchargement du modèle Cancel Annuler LLM Model Download Progress Progression du téléchargement du modèle LLM Error Occurred Une erreur s'est produite Confirm Deletion Confirmer la suppression Are you sure you want to delete {AI_MODEL.name}? Êtes-vous sûr de vouloir supprimer {AI_MODEL.name} ? LoadingDialog Loading Chargement... Loading... Chargement... MainWindow Ready. Prêt. Error Erreur Error initializing the LLM: Erreur lors de l'initialisation du LLM : File Fichier Add Ajouter File/Folder Fichier/Dossier External URL URL externe View Affichage Downloading data with errors... Téléchargement des données avec erreurs... File downloaded successfully to: {} Fichier téléchargé avec succès dans : {} Success Succès Errors panel Panneau d'erreurs Source panel Panneau source Help Aide User Guide Guide d'utilisateur Report an Issue Signaler un problème About À propos View logs Voir les logs Language changed. Langue modifiée. File and Metadata changes saved. Modifications du fichier et des métadonnées enregistrées. Last 100 Lines 100 dernières lignes Close Fermer Copy to Clipboard Copier dans le presse-papiers MetadataForm Column Name: Nom de la colonne: Data Type: Type de données: Description: Description: Flag empty cells as errors?: Marquer les cellules vides comme des erreurs ? Min. Characters in cells: Caractères min. dans les cellules: Max. Characters in cell Caractères max. dans la cellule: RenameDialog Rename file Renommer le fichier Rename item to: Renommer l'élément en: Cancel Annuler OK OK Sidebar Upload your data Téléchargez vos données User guide Guide d'utilisateur Report an issue Signaler un problème Rename Renommer Open File in Location Ouvrir le fichier dans l'emplacement Delete Supprimer Error Erreur Source is a file but destination a directory. La source est un fichier mais la destination est un répertoire. Source is a directory but destination a file. La source est un répertoire mais la destination est un fichier. Operation not permitted. Opération non autorisée. File with this name already exists. Un fichier portant ce nom existe déjà. Item renamed successfuly. Élément renommé avec succès. Item deleted successfuly. Élément supprimé avec succès. SourceViewer This view is only available for CSV files. Cette vue est uniquement disponible pour les fichiers CSV. Toolbar Data Données Sheet: Feuille : Errors Report Rapport d'erreurs Source code Code source Export Exporter Save changes Enregistrer les modifications AI IA Welcome The ODE supports Excel & csv files L'ODE prend en charge les fichiers Excel et csv You can also add links to online tables Vous pouvez également ajouter des liens vers des tableaux en ligne Upload your data Téléchargez vos données ================================================ FILE: src/ode/assets/translations/it.ts ================================================ ColumnMetadataDialog Column name cannot be empty Il nome della colonna non può essere vuoto There is another column in the table with the same name. Please choose a different one C'è un altra colonna nella tabella con lo stesso nome. Scegli un nome differente Save Salva Cancel Annulla DataUploadDialog Please paste a valid URL. Incolla una URL valida. Please paste a valid URL starting with http:// or https://. Incollare una URL valida che cominci con http:// o https://. Error: The Google Sheets URL is not valid or the table is not publicly available. Errore: l'URL del Google Fogli non è valida o la tabella non è stata resa pubblica. Error: The URL is not associated with a table Errore: l'URL non è associata ad una tabella Upload your data Carica i tuoi dati Add one or more Excel or csv files Aggiungi uno o più file Excel o csv Add a folder Aggiungi una cartella Select Seleziona Link to the external table: Link alla tabella esterna: Enter or paste URL Inserisci o incolla URL Paste your Google Sheet or csv link to create a local copy in the Open Data Editor Incolla il link di Google Fogli o csv per creare una copia locale nell'Open Data Editor Add Aggiungi Add Local Files Add file locali Add External Data Aggiungi dati esterni DataViewer Preview not available for this item. L'anteprima non è disponibile per questa voce. DataWorker Reading file... Lettura del file... Checking errors... Verifica errori... Read and error checking finished. Lettura e verifica degli errori completata. Drawing table... Sto disegnando la tabella... DeleteDialog Cancel Annulla Ok Ok Are you sure you want to delete this item? Vuoi davvero cancellare questa voce? Delete file Cancella file DownloadDialog Download Download Please, select one of the following options: Seleziona una di queste opzioni: Download file Download file Download file with errors Download file con errori File downloaded successfully to: {} File scaricati con successo in: {} Error downloading file: {} Errore scaricando file: {} Success Successo ErrorsMessages Missing header Intestazione mancante Duplicated header Intestazione duplicata Empty row Riga vuota Type mismatch Tipo non corrispondente Missing value Valore mancante Extra cell Cella extra Blank Label Etichetta vuota A column in the header row has no name. Every column should have a unique, non-empty header. Una colonna nella riga di intestazione non ha nome. Ogni colonna deve avere un'intestazione unica e non vuota. Two or more columns share the same name. Column names must be unique. Due o più colonne condividono lo stesso nome. I nomi delle colonne devono essere univoci. This row has no data. Rows should contain at least one cell with data. Questa riga non contiene dati. Le righe devono contenere almeno una cella con dati. A cell value doesn't match the expected data type or format for the column. Il valore di una cella non corrisponde al tipo di dato o formato previsto per la colonna. This cell is missing data Questa cella manca di dati This row has more values compared to the header row. Questa riga ha più valori rispetto alla riga di intestazione. A label in the header row is missing a value. Label should be provided and not be blank. Un'etichetta nella riga di intestazione manca di un valore. L'etichetta deve essere fornita e non essere vuota. ErrorsWidget Please note that the ODE currently detects errors in tables, with a maximum of Attenzione: ODE attualmente individua errori nelle tabelle con un massimo di FrictionlessTableModel Data Dati LLMWarningDialog AI assistant Assistente AI Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more Benvenuto nell'assistente AI di ODE! Questa funzione ti aiuterà a generare descrizioni migliori per le colonne della tua tabella e anche domande per l'analisi dei dati. Per iniziare, dovrai installare il file AI sul tuo computer. Una volta installato, tutto funzionerà localmente, il che significa che i tuoi dati rimarranno sempre privati e sicuri. Scopri di più Don't show again Non mostrare più Cancel Annulla Ok Ok LlamaDialog Generating response... Generazione della risposta... Execute Esegui Error Errore AI assistant Assistente AI Stop execution Ferma esecuzione Results will be displayed here... I risultati saranno mostrati qui... LlamaDownloadDialog Delete Elimina Download Download AI assistant Assistente AI To start using the AI assistant, please download the following model. Per iniziare ad utilizzare l'assistente AI, scarica il seguente modello. The ODE will save the file in this location: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> ODE salverà il file in questa posizione: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Next Avanti File exists Il file esiste Do you want to delete it and download it again? Vuoi eliminarlo e scaricarlo nuovamente? Downloading model Donwload modello Cancel Annulla LLM Model Download Progress Avanzamento del download del modello LLM Error Occurred Si è verificato un errore Confirm Deletion Conferma eliminazione Are you sure you want to delete {AI_MODEL.name}? Sei sicuro di voler eliminare {AI_MODEL.name}? LoadingDialog Loading Caricamento Loading... Caricamento... MainWindow Ready. Pronto. Error Errore Error initializing the LLM: Errore nell'inizializzazione del LLM: File File Add Aggiungi File/Folder File/Cartella External URL URL Esterna View Vedi Downloading data with errors... Download di dati con errori... File downloaded successfully to: {} File scaricato correttamente in: {} Success Successo Errors panel Pannello degli errori Source panel Pannello Sorgente Help Aiuto User Guide Guida utente Report an Issue Segnala un problema About Info View logs Vedi i log Language changed. Lingua cambiata. File and Metadata changes saved. Cambiamenti nel file e nei metadati salvati. Last 100 Lines Ultime 100 line Close Chiudi Copy to Clipboard Copia nel clipboard MetadataForm Column Name: Nome della colonna: Data Type: Tipo di dato: Description: Descrizione: Flag empty cells as errors?: Contrassegnare le celle vuote come errori?: Min. Characters in cells: numero minimo caratteri nella cella: Max. Characters in cell numero massimo caratteri nella cella RenameDialog Rename file Rinomina file Rename item to: Rinomina la voce in: Cancel Annulla OK OK Sidebar Upload your data Carica i tuoi dati User guide Guida utente Report an issue Segnala un problema Rename Rinominare Open File in Location Apri file nella posizione Delete Elimina Error Errore Source is a file but destination a directory. La sorgente è un file ma la destinazione è una cartella. Source is a directory but destination a file. La sorgente è una cartella ma la destinazione è un file. Operation not permitted. Operazione non permessa. File with this name already exists. Esiste già un file con questo nome. Item renamed successfuly. Voce rinominata. Item deleted successfuly. Voce eliminata. SourceViewer This view is only available for CSV files. Questa vista è disponibile solo per i file CSV. Toolbar Data Dati Errors Report Report Errori Source code Codice Sorgente Export Esporta Save changes Salva modifiche AI AI Sheet: Foglio: Welcome The ODE supports Excel & csv files ODE supporta file Excel & csv You can also add links to online tables Puoi anche aggiungere link a tabelle online Upload your data Carica i tuoi dati ================================================ FILE: src/ode/assets/translations/pt.ts ================================================ ColumnMetadataDialog Column name cannot be empty O nome da coluna não pode estar vazio There is another column in the table with the same name. Please choose a different one Há outra coluna na tabela com o mesmo nome. Por favor, escolha outro Save Salvar Cancel Cancelar DataUploadDialog Please paste a valid URL. Por favor, cole uma URL válida. Please paste a valid URL starting with http:// or https://. Por favor, cole uma URL válida começando com http:// ou https://. Error: The Google Sheets URL is not valid or the table is not publicly available. Erro: A URL do Google Sheets não é válida ou a tabela não está publicamente disponível. Error: The URL is not associated with a table Erro: A URL não está associada a uma tabela Upload your data Carregue seus dados Add one or more Excel or csv files Adicione um ou mais arquivos Excel ou CSV Add a folder Adicione uma pasta Select Selecionar Link to the external table: Link para a tabela externa: Enter or paste URL Digite ou cole a URL Paste your Google Sheet or csv link to create a local copy in the Open Data Editor Cole o link da sua planilha do Google ou CSV para criar uma cópia local no Editor de Dados Abertos. Add Adicionar Add Local Files Adicionar arquivos locais Add External Data Adicionar Dados Externos DataViewer Preview not available for this item. Visualização não disponível para este item. DataWorker Reading file... Lendo arquivo... Checking errors... Verificando erros... Read and error checking finished. Leitura e verificação de erros concluída. Drawing table... Renderizando a tabela... DeleteDialog Cancel Cancelar Ok Ok Are you sure you want to delete this item? Tem certeza que deseja excluir este item? Delete file Excluir arquivo DownloadDialog Download Baixar Please, select one of the following options: Por favor, selecione uma das seguintes opções: Download file Baixar arquivo Download file with errors Baixar arquivo com erros File downloaded successfully to: {} Arquivo baixado com sucesso em: {} Error downloading file: {} Erro ao baixar o arquivo: {} Success Sucesso ErrorsMessages Missing header Cabeçalho faltante Duplicated header Cabeçalho duplicado Empty row Linha vazia Type mismatch Incompatibilidade de tipo Missing value Valor faltante Extra cell Célula extra Blank Label Rótulo em branco A column in the header row has no name. Every column should have a unique, non-empty header. Uma coluna na linha do cabeçalho não tem nome. Cada coluna deve ter um cabeçalho único e não vazio. Two or more columns share the same name. Column names must be unique. Duas ou mais colunas compartilham o mesmo nome. Os nomes das colunas devem ser únicos. This row has no data. Rows should contain at least one cell with data. Esta linha não tem dados. As linhas devem conter pelo menos uma célula com dados. A cell value doesn't match the expected data type or format for the column. Um valor de célula não corresponde ao tipo de dado ou formato esperado para a coluna. This cell is missing data Esta célula está sem dados This row has more values compared to the header row. Esta linha tem mais valores em comparação com a linha do cabeçalho. A label in the header row is missing a value. Label should be provided and not be blank. Falta um valor em um rótulo da linha do cabeçalho. O rótulo deve ser fornecido e não pode estar em branco. ErrorsWidget Please note that the ODE currently detects errors in tables, with a maximum of Por favor, note que a ODE atualmente detecta erros nas tabelas, com um máximo de FrictionlessTableModel Data Dados LLMWarningDialog AI assistant Assistente de IA Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more Bem-vindo ao assistente de IA do ODE! Este recurso irá ajudá-lo a gerar melhores descrições para as colunas da sua tabela e também perguntas para análise de dados. Para começar, você precisará instalar o arquivo de IA no seu computador. Uma vez instalado, tudo será executado localmente, o que significa que seus dados sempre permanecem privados e seguros. Saiba mais Don't show again Não mostrar novamente Cancel Cancelar Ok Ok LlamaDialog Execute Executar Generating response... Gerando resposta... Error Erro AI assistant Assistente de IA Stop execution Parar execução Results will be displayed here... Os resultados serão exibidos aqui... LlamaDownloadDialog Delete Excluir Download Baixar AI assistant Assistente de IA To start using the AI assistant, please download the following model. Para começar a usar o assistente de IA, por favor baixe o seguinte modelo. The ODE will save the file in this location: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> O ODE salvará o arquivo neste local: <i><a href="file://{AI_MODELS_PATH}">{AI_MODELS_PATH}</a></i> Next Próximo File exists Arquivo existente Do you want to delete it and download it again? Deseja excluí-lo e baixá-lo novamente? Downloading model Baixando modelo Cancel Cancelar LLM Model Download Progress Progresso do Download do Modelo LLM Error Occurred Ocorreu um erro Confirm Deletion Confirmar exclusão Are you sure you want to delete {AI_MODEL.name}? Tem certeza de que deseja excluir {AI_MODEL.name}? LlamaWorker Preparing data for analysis... Preparando os dados para análise... LoadingDialog Loading Carregando... Loading... Carregando... MainWindow Ready. Pronto. Error Erro Error initializing the LLM: Erro ao inicializar o LLM: File Arquivo Add Adicionar File/Folder Arquivo/Pasta External URL URL Externa View Visualizar Downloading data with errors... Baixando dados com erros... File downloaded successfully to: {} Arquivo baixado com sucesso em: {} Success Sucesso Errors panel Painel de Erros Source panel Painel de Origem Help Ajuda User Guide Guia do Usuário Report an Issue Reportar um Problema About Sobre View logs Ver logs Language changed. Idioma alterado. File and Metadata changes saved. Alterações no arquivo e metadados salvas. Last 100 Lines Últimas 100 linhas Close Fechar Copy to Clipboard Copiar para área de transferência MetadataForm Column Name: Nome da coluna: Data Type: Tipo de dado: Description: Descrição: Flag empty cells as errors?: Marcar células vazias como erros? Min. Characters in cells: Caracteres mínimos nas células: Max. Characters in cell Caracteres máximos na célula: RenameDialog Rename file Renomear arquivo Rename item to: Renomear item para: Cancel Cancelar OK OK Sidebar Upload your data Carregue seus dados User guide Guia do Usuário Report an issue Reportar um Problema Rename Renomear Open File in Location Abrir Arquivo no Local Delete Excluir Error Erro Source is a file but destination a directory. A origem é um arquivo, mas o destino é um diretório. Source is a directory but destination a file. A origem é um diretório, mas o destino é um arquivo. Operation not permitted. Operação não permitida. File with this name already exists. Um arquivo com este nome já existe. Item renamed successfuly. Item renomeado com sucesso. Item deleted successfuly. Item excluído com sucesso. SourceViewer This view is only available for CSV files. Esta visualização está disponível apenas para arquivos CSV. Toolbar Data Dados Sheet: Planilha: Errors Report Relatório de Erros Source code Código-fonte Export Exportar Save changes Salvar alterações AI IA Welcome The ODE supports Excel & csv files O ODE suporta arquivos Excel e CSV You can also add links to online tables Você também pode adicionar links para tabelas online Upload your data Carregue seus dados ================================================ FILE: src/ode/dialogs/__init__.py ================================================ ================================================ FILE: src/ode/dialogs/delete.py ================================================ from PySide6.QtWidgets import QDialog, QVBoxLayout, QPushButton, QHBoxLayout, QLabel, QWidget class DeleteDialog(QDialog): """ Dialog to delete a file. """ def __init__(self, parent: QWidget, filename: str): super().__init__(parent) self.result_text = None layout = QVBoxLayout() self.delete_dialog_label = QLabel() layout.addWidget(self.delete_dialog_label) button_layout = QHBoxLayout() self.cancel_button = QPushButton() # We set the default button to Cancel when clicking Enter self.cancel_button.setDefault(True) self.ok_button = QPushButton() self.cancel_button.clicked.connect(self.reject) self.ok_button.clicked.connect(self.accept) button_layout.addWidget(self.cancel_button) button_layout.addWidget(self.ok_button) layout.addLayout(button_layout) self.setLayout(layout) self.retranslateUI() @staticmethod def confirm(parent, filename): """Static method that returns a boolean value indicating if the user confirmed the deletion of the file.""" dialog = DeleteDialog(parent, filename) return dialog.exec() == QDialog.DialogCode.Accepted def retranslateUI(self): """Apply translations to class elements.""" self.cancel_button.setText(self.tr("Cancel")) self.ok_button.setText(self.tr("Ok")) self.delete_dialog_label.setText(self.tr("Are you sure you want to delete this item?")) self.setWindowTitle(self.tr("Delete file")) ================================================ FILE: src/ode/dialogs/download.py ================================================ import os import shutil from PySide6.QtWidgets import QVBoxLayout, QPushButton, QDialog, QMessageBox, QLabel, QHBoxLayout from PySide6.QtCore import Qt, Signal, QStandardPaths from pathlib import Path class DownloadDialog(QDialog): """Dialog to export the file and the errors.""" download_data_with_errors = Signal() finished = Signal() def __init__(self, parent, filepath: Path, has_errors:bool) -> None: super().__init__(parent) self.filepath = filepath self.setFixedHeight(200) layout = QVBoxLayout() self.label = QLabel() layout.addWidget(self.label) # Block the main window until the dialog is closed self.setWindowModality(Qt.WindowModality.ApplicationModal) button_layout = QHBoxLayout() self.download_button = QPushButton() self.download_button.clicked.connect(self.download_file) button_layout.addWidget(self.download_button) self.download_error_button = QPushButton() self.download_error_button.clicked.connect(self.download_error_file) if not has_errors: self.download_error_button.setDisabled(True) button_layout.addWidget(self.download_error_button) layout.addLayout(button_layout) self.setLayout(layout) self.retranslateUI() def retranslateUI(self) -> None: """Apply translations to class elements.""" self.setWindowTitle(self.tr("Download")) self.label.setText(self.tr("Please, select one of the following options:")) self.download_button.setText(self.tr("Download file")) self.download_error_button.setText(self.tr("Download file with errors")) def download_file(self): """ Opens a dialog to select the destination directory and copies the file """ downloads_path = QStandardPaths.writableLocation(QStandardPaths.StandardLocation.DownloadLocation) filename = os.path.basename(self.filepath) filepath = Path(downloads_path, filename) try: shutil.copy2(self.filepath, filepath) success_text = self.tr("File downloaded successfully to:\n{}").format(filepath) QMessageBox.information(self, self.tr("Success"), success_text) except Exception as e: error_text = self.tr("Error downloading file:\n{}").format(str(e)) QMessageBox.critical(self, "Error", error_text) def download_error_file(self): self.download_data_with_errors.emit() ================================================ FILE: src/ode/dialogs/llm_dialog_warning.py ================================================ from PySide6.QtWidgets import QDialog, QVBoxLayout, QPushButton, QHBoxLayout, QLabel, QWidget, QCheckBox from PySide6.QtCore import QSettings class LLMWarningDialog(QDialog): """ This dialog informs users that the AI assistant operates entirely on their laptop, ensuring that data from their table is never sent or shared outside the device. """ def __init__(self, parent: QWidget): super().__init__(parent) self.setWindowTitle(self.tr("AI assistant")) self.setFixedSize(600, 270) layout = QVBoxLayout() self.warning_text = QLabel() self.warning_text.setWordWrap(True) self.warning_text.setText( self.tr( "Welcome to the ODE's AI assistant! This feature will help you generating better descriptions for the columns of your table and also questions for data analysis. \n\n" "To get started, you will need to install the AI file in your computer. Once installed, everything will run locally, meaning your data always stays private and secure. Learn more" ) ) self.dont_show_again_checkbox = QCheckBox() self.dont_show_again_checkbox.setText(self.tr("Don't show again")) button_layout = QHBoxLayout() self.cancel_button = QPushButton() self.cancel_button.clicked.connect(self.reject) self.cancel_button.setText(self.tr("Cancel")) button_layout.addWidget(self.cancel_button) self.ok_button = QPushButton() self.ok_button.setDefault(True) button_layout.addWidget(self.ok_button) self.ok_button.clicked.connect(self.accept) self.ok_button.setText(self.tr("Ok")) layout.addWidget(self.warning_text) layout.addWidget(self.dont_show_again_checkbox) layout.addLayout(button_layout) self.setLayout(layout) def accept(self): """Override accept to save the checkbox state""" if self.dont_show_again_checkbox.isChecked(): settings = QSettings() settings.setValue("llm_warning_dialog/dont_show_again", True) super().accept() @staticmethod def confirm(parent): settings = QSettings() # Check if the user has previously chosen to not show the dialog again if settings.value("llm_warning_dialog/dont_show_again", False, type=bool): return True dialog = LLMWarningDialog(parent) return dialog.exec() == QDialog.DialogCode.Accepted ================================================ FILE: src/ode/dialogs/loading.py ================================================ from PySide6.QtWidgets import QDialog, QProgressBar, QVBoxLayout, QLabel from PySide6.QtCore import QTimer, Slot class LoadingDialog(QDialog): """ Dialog to show a loading message while the application is doing some work. """ def __init__(self, parent=None): super().__init__(parent) self.setMinimumSize(350, 80) self.label = QLabel() self.label.setMargin(0) self.progressBar = QProgressBar() self.progressBar.setRange(0, 0) mainLayout = QVBoxLayout(self) mainLayout.addWidget(self.label) mainLayout.addWidget(self.progressBar) self.timer = QTimer() self.timer.setSingleShot(True) self.timer.timeout.connect(self.exec) self.retranslateUI() def retranslateUI(self): self.setWindowTitle(self.tr("Loading")) self.label.setText(self.tr("Loading...")) def cancel_loading_timer(self): if self.timer.isActive(): self.timer.stop() def show(self, millis: int = 300): self.timer.start(millis) def show_immediately(self): self.show() @Slot(str) def show_message(self, message): self.label.setText(message) ================================================ FILE: src/ode/dialogs/metadata.py ================================================ from typing import NamedTuple from PySide6.QtWidgets import ( QDialog, QVBoxLayout, QHBoxLayout, QPushButton, QWidget, QGridLayout, QLabel, QLineEdit, QTextEdit, QComboBox, QSpinBox, ) from PySide6.QtCore import Qt, Signal class ColumnMetadataField(NamedTuple): """ Represents a field in the metadata with its name, type, description, and constraints. """ name: str type: str description: str constraints: dict class DataTypeMapper: """Class to handle bidirectional mapping between internal types and user-displayed types""" DATA_TYPES = [ "number", "date", "string", "any", "array", "boolean", "datetime", "duration", "geojson", "geopoint", "integer", "object", "time", "year", "yearmonth", ] # Mapping from internal types to user-displayed types DISPLAY_MAPPING = { "string": "Text", "number": "Number", "date": "Date", "any": "Any", "array": "Array", "boolean": "Boolean", "datetime": "Date Time", "duration": "Duration", "geojson": "GeoJSON", "geopoint": "Geo Point", "integer": "Integer", "object": "Object", "time": "Time", "year": "Year", "yearmonth": "Year Month", } def __init__(self): # Reverse mapping: from displayed types to internal types self.internal_mapping = {v: k for k, v in self.DISPLAY_MAPPING.items()} def get_display_type(self, internal_type): """Converts an internal type to its user-displayed representation""" return self.DISPLAY_MAPPING.get(internal_type, internal_type) def get_internal_type(self, display_type): """Converts a user-displayed type to its internal representation""" return self.internal_mapping.get(display_type, display_type) def get_all_display_types(self): """Returns all types in user-display format""" return [self.get_display_type(t) for t in self.DATA_TYPES] def get_all_internal_types(self): """Returns all internal types""" return self.DATA_TYPES.copy() class NoWheelComboBox(QComboBox): """QComboBox that disables the mouse wheel event. The current UX when scrolling through FieldsForms is not ideal since as soon as the mouse points a QComboBox the form stops scrolling and it starts changing the value of the QComboBox instead. """ def wheelEvent(self, event): event.ignore() class MetadataForm(QWidget): """ Widget to show the Metadata Fields that will be displayed inside a dialog. """ def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) layout = QGridLayout() # Name self.nameLabel = QLabel() layout.addWidget(self.nameLabel, 0, 0) self.name = QLineEdit() self.name.setMinimumWidth(200) layout.addWidget(self.name, 0, 1, 1, 3) # Type self.typeLabel = QLabel() layout.addWidget(self.typeLabel, 1, 0) self.type = NoWheelComboBox() layout.addWidget(self.type, 1, 1, 1, 3) self.type.currentTextChanged.connect(self.on_type_changed) # Description self.description_label = QLabel() layout.addWidget(self.description_label, 2, 0) self.description = QTextEdit() self.description.setMaximumHeight(60) layout.addWidget(self.description, 2, 1, 1, 3) # Required self.required_label = QLabel() layout.addWidget(self.required_label, 3, 0) self.required = QComboBox() self.required.addItems(["Yes", "No"]) layout.addWidget(self.required, 3, 1, 1, 3) # Min and Max Length self.min_length_label = QLabel() layout.addWidget(self.min_length_label, 4, 0) self.min_length = QSpinBox() layout.addWidget(self.min_length, 4, 1) self.max_length_label = QLabel() layout.addWidget(self.max_length_label, 4, 2) self.max_length = QSpinBox() layout.addWidget(self.max_length, 4, 3) # Error label self.error_label = QLabel() self.error_label.setStyleSheet("color: red;") layout.addWidget(self.error_label, 6, 0, 1, 4) self.error_label.setHidden(True) # Set layout properties layout.setColumnMinimumWidth(1, 150) layout.setColumnMinimumWidth(3, 150) layout.setHorizontalSpacing(20) self.setLayout(layout) self.retranslateUI() def on_type_changed(self, text): """ Updates the min and max length fields based on the selected type. """ if text == "Text": self.min_length.setEnabled(True) self.min_length.setStyleSheet("") self.min_length_label.setStyleSheet("") self.max_length.setEnabled(True) self.max_length.setStyleSheet("") self.max_length_label.setStyleSheet("") else: self.min_length.setEnabled(False) self.min_length.setStyleSheet("color: lightgray;") self.min_length_label.setStyleSheet("color: lightgray;") self.max_length.setEnabled(False) self.max_length.setStyleSheet("color: lightgray;") self.max_length_label.setStyleSheet("color: lightgray;") def retranslateUI(self): """ Applies the translations to the labels. """ self.nameLabel.setText(self.tr("Column Name:")) self.typeLabel.setText(self.tr("Data Type:")) self.description_label.setText(self.tr("Description:")) self.required_label.setText(self.tr("Flag empty cells as errors?:")) self.min_length_label.setText(self.tr("Min. Characters in cells:")) self.max_length_label.setText(self.tr("Max. Characters in cell")) class ColumnMetadataDialog(QDialog): """ Dialog for editing the column's metadata. """ save_clicked = Signal(object) def __init__(self, parent: QWidget, field: ColumnMetadataField, field_index: int, field_names: list): """ Initialize the dialog. Args: parent: The parent widget """ super().__init__(parent) self.parent = parent self.dataTypeMapper = DataTypeMapper() self.field_index = field_index self.field_names = field_names self.field = field # Set up the form self.form = MetadataForm() self.form.name.setText(field.name) self.form.type.addItems(self.dataTypeMapper.get_all_display_types()) self.form.type.setCurrentText(self.dataTypeMapper.get_display_type(field.type)) self.form.description.setText(field.description) self.form.required.setCurrentText("Yes" if field.constraints.get("required") else "No") self.form.min_length.setMinimum(0) self.form.min_length.setMaximum(999999) self.form.min_length.setValue(field.constraints.get("minLength", 0)) self.form.max_length.setMinimum(0) self.form.max_length.setMaximum(999999) self.form.max_length.setValue(field.constraints.get("maxLength", 999)) # Set window modality self.setWindowModality(Qt.WindowModality.WindowModal) # Create buttons self.save_button = QPushButton() self.cancel_button = QPushButton() # Set up the layout self.setup_layout() # Connect signals self.save_button.clicked.connect(self.save_and_close) self.cancel_button.clicked.connect(self.reject) def setup_layout(self): """ Set up the dialog layout. """ layout = QVBoxLayout() layout.addWidget(self.form) # Buttons layout buttons_layout = QHBoxLayout() buttons_layout.addWidget(self.cancel_button) buttons_layout.addWidget(self.save_button) layout.addLayout(buttons_layout) self.setLayout(layout) self.retranslateUI() def save_and_close(self): """ Emits the save_clicked signal with the form data and closes the dialog. """ # Validate the field name field_name_error = None field_name = self.form.name.text() if field_name.strip() == "": field_name_error = self.tr("Column name cannot be empty") elif field_name != self.field.name and field_name in self.field_names: field_name_error = self.tr( "There is another column in the table with the same name. Please choose a different one" ) if field_name_error: self.form.name.setStyleSheet("border: 1px solid red;") self.form.error_label.setText(field_name_error) self.form.error_label.setHidden(False) return self.save_clicked.emit( { "index": self.field_index, "name": field_name, "type": self.dataTypeMapper.get_internal_type(self.form.type.currentText()), "description": self.form.description.toPlainText(), "constraints": { "required": self.form.required.currentText() == "Yes", "minLength": self.form.min_length.value(), "maxLength": self.form.max_length.value(), }, } ) self.accept() def retranslateUI(self): """ Applies the translations to the labels. """ self.save_button.setText(self.tr("Save")) self.cancel_button.setText(self.tr("Cancel")) ================================================ FILE: src/ode/dialogs/rename.py ================================================ from PySide6.QtWidgets import QDialog, QVBoxLayout, QLineEdit, QPushButton, QHBoxLayout, QLabel, QWidget class RenameDialog(QDialog): """ Dialog to rename a file. The dialog will show a text box with the current filename and allow the user to change it. The dialog will save the new filaneme in the result_text attribute when the user clicks OK. """ def __init__(self, parent: QWidget, filename: str): super().__init__(parent) self.result_text = None layout = QVBoxLayout() self.rename_label = QLabel() layout.addWidget(self.rename_label) self.text_edit = QLineEdit(filename) self.text_edit.setMinimumWidth(300) self.text_edit.setMinimumHeight(30) layout.addWidget(self.text_edit) button_layout = QHBoxLayout() self.cancel_button = QPushButton() self.ok_button = QPushButton() # We set the default button to OK when clicking Enter self.ok_button.setDefault(True) self.cancel_button.clicked.connect(self.reject) self.ok_button.clicked.connect(self.accept) button_layout.addWidget(self.cancel_button) button_layout.addWidget(self.ok_button) layout.addLayout(button_layout) self.setLayout(layout) self.retranslateUI() def accept(self): """ Accept the dialog and save the new filename in result_text.""" self.result_text = self.text_edit.text() super().accept() def retranslateUI(self): """Apply translations to class elements.""" self.setWindowTitle(self.tr("Rename file")) self.rename_label.setText(self.tr("Rename item to:")) self.cancel_button.setText(self.tr("Cancel")) self.ok_button.setText(self.tr("OK")) ================================================ FILE: src/ode/dialogs/upload.py ================================================ import re import shutil from frictionless.resources import FileResource, TableResource, FrictionlessException from pathlib import Path from PySide6.QtWidgets import ( QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLabel, QFileDialog, QDialog, QTabWidget, QLineEdit, ) from PySide6.QtGui import QPixmap from PySide6.QtCore import Qt from ode import paths from ode.paths import Paths class SelectWidget(QWidget): """Widget to render the File/Folder upload buttons.""" def __init__(self, icon_path, parent=None): super().__init__(parent) layout = QVBoxLayout() icon_label = QLabel(self) pixmap = QPixmap(icon_path) icon_label.setPixmap(pixmap) icon_label.setAlignment(Qt.AlignmentFlag.AlignCenter) layout.addWidget(icon_label) self.text_label = QLabel() self.text_label.setAlignment(Qt.AlignmentFlag.AlignHCenter) self.text_label.setWordWrap(True) layout.addWidget(self.text_label) self.select_button = QPushButton() layout.addWidget(self.select_button) self.setLayout(layout) def connect_select_action(self, action): self.select_button.clicked.connect(action) class DataUploadDialog(QDialog): """Dialog to Upload File, Folders or URLs. The goal of this Dialog is to have an intuitive UX for people to add files, folders or URLs. For external URLs we rely on frictionless's TableResource to read and write tables hosted in the web or Google Spreadsheets. How to use: dialog = DataUploadDialog(self) ok, path = dialog.upload_dialog() """ def __init__(self, parent, external_first=False): super().__init__(parent) self.setFixedHeight(400) self.setFixedWidth(600) self.target_path = Path() main_layout = QVBoxLayout() main_layout.setContentsMargins(10, 100, 10, 10) # Block the main window until the dialog is closed self.setWindowModality(Qt.WindowModality.ApplicationModal) # Tab Widget self.tab_widget = QTabWidget() main_layout.addWidget(self.tab_widget) # From Your Computer Tab from_computer_tab = QWidget() from_computer_layout = QHBoxLayout() from_computer_tab.setLayout(from_computer_layout) self.file_select_widget = SelectWidget(Paths.asset("icons/upload-file.png")) self.file_select_widget.connect_select_action(self.add_files) self.folder_select_widget = SelectWidget(Paths.asset("icons/upload-folder.png")) self.folder_select_widget.connect_select_action(self.add_folders) from_computer_layout.addWidget(self.file_select_widget) from_computer_layout.addWidget(self.folder_select_widget) # Add External Data Tab external_data_tab = QWidget() external_data_layout = QVBoxLayout() external_data_tab.setLayout(external_data_layout) self.url_label = QLabel() self.url_input = QLineEdit() self.help_text = QLabel() self.help_text.setWordWrap(True) self.help_text.setStyleSheet("font-style:italic; font-size: 15px;") self.paste_button = QPushButton() self.paste_button.clicked.connect(self.load_table_from_url) self.error_text = QLabel() self.error_text.setWordWrap(True) self.error_text.setStyleSheet("color: red; font-style: italic; font-size: 15px;") external_data_layout.addWidget(self.url_label) external_data_layout.addWidget(self.url_input) external_data_layout.addWidget(self.help_text) external_data_layout.addWidget(self.paste_button) external_data_layout.addWidget(self.error_text) # Add Tabs to Tab Widget self.tab_widget.addTab(from_computer_tab, "") self.tab_widget.addTab(external_data_tab, "") if external_first: self.tab_widget.setCurrentIndex(1) self.setLayout(main_layout) self.retranslateUI() def add_files(self): """Copy the selected file to the project path.""" filters = [ "All supported files (*.csv *.xlsx *.xls)", "Comma Separated Values (*.csv)", "Excel 2007-365 (*.xlsx)", "Excel 97-2003 (*.xls)", ] filename, _ = QFileDialog.getOpenFileName(self, filter=";;".join(filters)) if not filename: return self.target_path = Paths.get_unique_destination_filepath(filename) shutil.copy(filename, self.target_path) self.accept() def add_folders(self) -> None: """Copy the selected folder and all its content to the project path.""" source_folder = QFileDialog.getExistingDirectory(self) if not source_folder: self.reject() return folder = Path(source_folder) self.target_path = paths.PROJECT_PATH / folder.name shutil.copytree(folder, self.target_path, dirs_exist_ok=True) self.accept() def load_table_from_url(self): """Load a tabular file from a public URL. This method uses frictionless to read a remote URL. Currently we support Google Spreadsheets and any other URL pointing to a csv file. """ url = self.url_input.text() if not url: self.error_text.setText(self.tr("Please paste a valid URL.")) return if not url.startswith(("http://", "https://")): self.error_text.setText(self.tr("Please paste a valid URL starting with http:// or https://.")) return table = TableResource(path=url) filename = table.name if table.format == "gsheets": try: filename = self._read_url_html_title(url) except FrictionlessException: error = self.tr("Error: The Google Sheets URL is not valid or the table is not publicly available.") self.error_text.setText(error) return self.target_path = Paths.get_unique_destination_filepath(filename + ".csv") try: with open(self.target_path, mode="w") as file: table.write(file.name) self.accept() except Exception: error = self.tr("Error: The URL is not associated with a table") self.error_text.setText(error) def upload_dialog(self) -> tuple[int, Path]: """Shows the dialog and then returns the result code and the path to the uploaded file. This method is inspired in QFileDIalog.getOpenFileName(...) and QInputDialog.getText(..). When called, this method will display the dialog and return the result code + the path where the file/folder has been copied to. """ result = self.exec() return result, self.target_path def _read_url_html_title(self, url): """Return the title of HTML document. We use the `title` attribute of the Google Spreadshet's HTML as name of the file. This attribute is the same as the name of the spreadsheet. """ file = FileResource(path=url) text = file.read_text(size=10000) match = re.search(r"(.*?)", text) if match: title = match.group(1) title = title.rsplit("- Google", 1)[0].strip() return f"{title}" return "google-sheets" def retranslateUI(self): """Apply translations to class elements.""" self.setWindowTitle(self.tr("Upload your data")) self.file_select_widget.text_label.setText(self.tr("Add one or more Excel or csv files")) self.folder_select_widget.text_label.setText(self.tr("Add a folder")) self.file_select_widget.select_button.setText(self.tr("Select")) self.folder_select_widget.select_button.setText(self.tr("Select")) self.url_label.setText(self.tr("Link to the external table: ")) self.url_input.setPlaceholderText(self.tr("Enter or paste URL")) self.help_text.setText( self.tr("Paste your Google Sheet or csv link to create a local copy in the Open Data Editor") ) self.paste_button.setText(self.tr("Add")) self.tab_widget.setTabText(0, self.tr("Add Local Files")) self.tab_widget.setTabText(1, self.tr("Add External Data")) ================================================ FILE: src/ode/file.py ================================================ import json import shutil import logging import xlrd import openpyxl from frictionless import system from frictionless.resources import TableResource from frictionless.formats.excel import ExcelControl from pathlib import Path from ode import paths logger = logging.getLogger(__name__) class File: """Class to interact with a File and it's associated Metadata. In ODE, every file has a corresponding metadata file that stores Fricionless Metadata and any other metadata required by ODE. All metadata files are going to be stored in a `.metadata` folder mimicing the file name and the structure of the project folder. Everytime the name or the location of the file changes, we need to update it's metadata file as well, so this class will implement all methods required to keep synchronized all the information. Metadata file example: { "resource": "{...frictionless descriptor...}" "custom_ode_metadata": "custom_ode_metadata_value" } """ def __init__(self, path: str | Path, sheet_name: str | None = None) -> None: self.path: Path = path if isinstance(path, Path) else Path(path) self.metadata_path: Path = self._get_path_to_metadata_file(self.path, sheet_name) def get_metadata_dict(self, metadata_path) -> dict: """Returns the ODE metadata dictionary for the current file. The difference with get_or_create_metadata is that this method will not return a Frictionless TableResource object in the record key, just a JSON object. """ with open(metadata_path) as file: metadata = json.load(file) return metadata def set_metadata_dict(self, metadata_path: Path, metadata: dict) -> None: with open(metadata_path, mode="w") as file: json.dump(metadata, file) def _get_path_to_metadata_file(self, path: Path, sheet_name: str | None = None) -> Path: """Returns the path to the metadata file of the given file. Example 1: - File: Paths.PROJECT_FOLDER / 'myfile.csv' - Metadata: Paths.PROJECT_FOLDER / '.metadata/myfile.json' Example 2 (subfolder): - File: Paths.PROJECT_FOLDER / 'subfolder/invalid-file.csv' - Metadata: Paths.PROJECT_FOLDER / '.metadata/subfolder/invalid-file.json' Example 3 (input is folder): - Folder: Paths.PROJECT_FOLDER / 'subfolder' - Metadata: Paths.PROJECT_FOLDER / '.metadata/subfolder' """ relative_path = path.parent.relative_to(paths.PROJECT_PATH) metadata_path = paths.METADATA_PATH / relative_path if path.is_dir(): return metadata_path / path.stem if sheet_name: # If the file is an Excel file and a sheet name is provided, we include the sheet name # in the metadata filename to differentiate between different sheets of the same file. safe_sheet_name = "".join(c if c.isalnum() else "_" for c in sheet_name) return metadata_path / (path.stem + f"_sheet_{safe_sheet_name}" + ".json") else: return metadata_path / (path.stem + ".json") def get_or_create_metadata(self, sheet_name: str | None = None): """Get or create a metadata object for the Resource. Sheet name is used to specify which sheet of the Excel file we want to use. """ if self.metadata_path.exists(): metadata = dict() with open(self.metadata_path) as file: metadata = json.load(file) with system.use_context(trusted=True): if sheet_name: logger.info("Using sheet %s for resource %s", sheet_name, self.path) resource = TableResource(metadata["resource"], control=ExcelControl(sheet=sheet_name)) else: logger.info("Using resource %s", self.path) resource = TableResource(metadata["resource"]) resource.infer() metadata["resource"] = resource else: # If the metadata file does not exist, we create it. metadata = self._setup_metadata_first_time(sheet_name) return metadata def _setup_metadata_first_time(self, sheet_name: str | None = None): """ Set up the metadata for the first time when the file is opened. """ metadata = dict() self.metadata_path.parent.mkdir(parents=True, exist_ok=True) with system.use_context(trusted=True): if sheet_name: resource = TableResource(self.path, control=ExcelControl(sheet=sheet_name)) else: resource = TableResource(self.path) resource.infer() with open(self.metadata_path, "w") as f: # Resource is not serializable, converting to dict before writing. metadata["resource"] = resource.to_descriptor() json.dump(metadata, f) # We want to return a Frictionless object, so we are plugging it back. metadata["resource"] = resource return metadata def rename(self, new_name, sheet_names: list[str] | None = None): """Rename a file and the corresponding metadata file. Whenever we rename files we need to update a) the name of the metadata file and b) the Frictionless path attribute. When renaming a folder, we need to ensure that every metadata file of children files are updated as well. """ new_path = self.path.with_stem(new_name) old_path = self.path if new_path.exists(): raise OSError self.path.rename(new_path) self.path = new_path if sheet_names: first = True # If we are renaming an Excel file, we need to create a metadata file for each sheet. for sheet_name in sheet_names: old_metadata_path_with_sheet_name = self._get_path_to_metadata_file(old_path, sheet_name) new_metadata_path_with_sheet_name = self._get_path_to_metadata_file(new_path, sheet_name) self.rename_metadata_file( old_metadata_path_with_sheet_name, new_metadata_path_with_sheet_name, old_path, new_path ) if first: self.metadata_path = new_metadata_path_with_sheet_name first = False else: new_metadata_path = self.metadata_path.with_stem(new_name) self.rename_metadata_file(self.metadata_path, new_metadata_path, old_path, new_path) self.metadata_path = new_metadata_path def rename_metadata_file(self, old_metadata_path: Path, new_metadata_path: str, old_path: Path, new_path: Path): """Rename the metadata file and update the path attribute in the metadata.""" # First we update metadata's path attribute to point to the renamed file/folder if old_metadata_path.is_file(): metadata = self.get_metadata_dict(old_metadata_path) metadata["resource"]["path"] = str(new_path) self.set_metadata_dict(new_metadata_path, metadata) old_metadata_path.unlink() elif old_metadata_path.is_dir(): # If we are renaming a directory, we need to update all existing metadata files. for file in old_metadata_path.rglob("*.json"): metadata = self.get_metadata_dict(file) # When renaming a directory the filename remains but we need to replace its # parent directory. current = metadata["resource"]["path"] metadata["resource"]["path"] = current.replace(str(old_path), str(new_path)) self.set_metadata_dict(file, metadata) old_metadata_path.rename(new_metadata_path) def remove(self, sheet_names: list[str] | None = None): """Remove a file from disk. When uploading a folder, the metadata folder does not exist until the first children file is open and validated. If the user uploads a folder and do not open any file, we will not have a metadata folder. We check if it exist before deleting to ignore errors. """ if self.path.is_file(): self.path.unlink() if sheet_names: for sheet_name in sheet_names: metadata_path_with_sheet_name = self._get_path_to_metadata_file(self.path, sheet_name) if metadata_path_with_sheet_name.exists(): metadata_path_with_sheet_name.unlink() elif self.metadata_path.exists(): self.metadata_path.unlink() elif self.path.is_dir(): shutil.rmtree(self.path) if self.metadata_path.exists(): shutil.rmtree(self.metadata_path) @staticmethod def get_sheets_names(filepath: Path) -> list[str]: """Get the names of the sheets in an Excel file.""" sheet_names = [] if filepath.suffix == ".xls": workbook = xlrd.open_workbook(str(filepath)) sheet_names = workbook.sheet_names() elif filepath.suffix in [".xlsx"]: workbook = openpyxl.load_workbook(filepath, read_only=True) sheet_names = workbook.sheetnames return sheet_names ================================================ FILE: src/ode/llama.py ================================================ import sys import os import logging from pathlib import Path from typing import NamedTuple from enum import Enum from llama_cpp import Llama from PySide6.QtWidgets import ( QDialog, QVBoxLayout, QTextEdit, QPushButton, QLabel, QHBoxLayout, QMessageBox, QProgressDialog, QComboBox, QWidget, ) from PySide6.QtCore import QThread, Signal, QObject, QSaveFile, QIODevice, Slot, Qt from PySide6.QtNetwork import QNetworkReply, QNetworkRequest, QNetworkAccessManager from ode.paths import AI_MODELS_PATH if not os.path.exists(AI_MODELS_PATH): os.makedirs(AI_MODELS_PATH) logger = logging.getLogger(__name__) class PromptKeys(Enum): SELECT = "select" COLUMNS = "columns" ANALYSIS = "analysis" class AIModel(NamedTuple): """Data structure to hold AI model information.""" name: str url: str filename: str AI_MODEL = AIModel( name="Llama 3.2 3B (2.0GB)", url="https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_K_M.gguf?download=true", filename="Llama-3.2-3B-Instruct-Q4_K_M.gguf", ) class LlamaWorkerSignals(QObject): """Define the signals for the LlamaWorker.""" finished = Signal() error = Signal(str) stream_token = Signal(str) stream_token_first_received = Signal() started = Signal() class LlamaWorker(QThread): """ LlamaWorker is a QThread that processes a prompt using the LLM with streaming. """ def __init__(self, llm, prompt): super().__init__() self.llm = llm self.prompt = prompt self.signals = LlamaWorkerSignals() def run(self): try: self.signals.started.emit() messages = [ { "role": "system", "content": "You are an English-speaking Data Analyst in charge of reviewing and improving the quality of tabular files. You are concise and only reply with the answer, nothing else. You always respond in English, regardless of the input language.", }, { "role": "user", "content": self.prompt, }, ] first_response = True for output in self.llm.create_chat_completion(messages, max_tokens=-1, stream=True, temperature=0.2): token = output["choices"][0]["delta"].get("content", "") if first_response: self.signals.stream_token_first_received.emit() first_response = False self.signals.stream_token.emit(token) self.signals.finished.emit() except Exception as e: logger.error("Error during LLM processing", exc_info=True) self.signals.error.emit(str(e)) class LlamaDialog(QDialog): """ Dialog for interacting with the LLM. """ def __init__(self, parent=None): super().__init__(parent) self.llm = None self.worker = None self.init_ui() self.data = None self.prompt = None # Block the AI Assistant window until the dialog is closed self.setWindowModality(Qt.WindowModality.ApplicationModal) def closeEvent(self, event): """Handle the close event to clear the output text.""" if self.worker and self.worker.isRunning(): self.worker.terminate() self.worker.wait() self.prompt_selector.setCurrentIndex(0) self.output_text.clear() self.on_execution_finished() event.accept() super().closeEvent(event) def init_ui(self): """Initialize the UI for the Llama dialog.""" layout = QVBoxLayout(self) self.prompt_label = QLabel( "The AI assistant currently support two use cases. Please, select one of the following options:" ) layout.addWidget(self.prompt_label) self.prompt_selector = QComboBox() options = [ ("Please select a use case", PromptKeys.SELECT.value), ("Generate descriptions for columns", PromptKeys.COLUMNS.value), ("Suggest questions for data analysis", PromptKeys.ANALYSIS.value), ] for i, (text, key) in enumerate(options): self.prompt_selector.addItem(text) self.prompt_selector.setItemData(i, key) layout.addWidget(self.prompt_selector) self.btn_is_running = False self.btn_run = QPushButton() self.btn_run.setEnabled(False) self.btn_run.clicked.connect(self.run) layout.addWidget(self.btn_run) self.btn_stop = QPushButton() self.btn_stop.clicked.connect(self.stop) self.btn_stop.setVisible(False) layout.addWidget(self.btn_stop) self.output_text = QTextEdit() self.output_text.setReadOnly(True) self.output_text.setMinimumHeight(500) self.output_text.setMinimumWidth(700) layout.addWidget(self.output_text) self.prompt_text_label = QLabel( "This answer is AI generated. We recommend users to check responses to make sure they are in accordance with the table's data" ) layout.addWidget(self.prompt_text_label) self.prompt_selector.activated.connect(self.set_prompt) self.retranslateUI() def set_data(self, data): """Set the data for analysis.""" self.data = data def _get_columns_prompt(self): headers = [str(h) for h in self.data[0] if h is not None and h != ""] prompt = f"""# Explain the column names Below is the metadata and sample data of a dataset that you will suggest columns descriptions. ## Describe what each columns means in the context of the dataset 1. Please provide a description for each column name. 2. Assume the user does not know anything about the topic. 3. Use plain language and be verbose when explaining what data the column contains. 4. If there are technical terms, expand the description to explain what that term means in the context of the dataset. 5. Use the content of the First 5 rows to gain context about the columns so you answer is more accurate. # Metadata Current column names: {" | ".join(headers)} First 5 rows: {self.data[1:6]} """ return prompt def _get_analysis_prompt(self): headers = [str(h) for h in self.data[0] if h is not None and h != ""] prompt = f"""# Understand the Data: Below is the metadata and sample data of a dataset that you will suggest analysis for. ## Create questions that cover a wide range of analytical techniques: 1. Descriptive Statistics: (e.g., counts, averages, distributions, min/max) 2. Trend Analysis: (e.g., over time, across categories) 3. Relationship & Correlation: (e.g., between two numeric columns) 4. Segmentation & Comparison: (e.g., comparing groups based on a category) 5. Aggregation & Binning: (e.g., using groups and summary functions) 6. Data Quality & Anomalies: (e.g., missing values, outliers) ## Tailor the Questions: The questions must be specific to the provided column names and inferred context. Do not generate generic questions that could apply to any dataset. ## Value of the Question: For each question you would add a sentence providing what useful information could be get out of the answer and why you consider the question important. # Metadata: 1. Column names: {" | ".join(headers)} 2. First 5 rows: {self.data[1:6]} """ return prompt def set_prompt(self, index): """Set the prompt""" key = self.prompt_selector.itemData(index) self.prompt = "No prompt selected." self.output_text.clear() if key == PromptKeys.SELECT.value: self.prompt = None self.btn_run.setEnabled(False) return if key == PromptKeys.COLUMNS.value: self.prompt = self._get_columns_prompt() elif key == PromptKeys.ANALYSIS.value: self.prompt = self._get_analysis_prompt() if not self.btn_is_running: self.btn_run.setEnabled(True) def init_llm(self, model_path): """Initialize the LLM with the given model path.""" cores = self._calculate_half_cpu_count() self.llm = Llama( model_path=model_path, n_ctx=4096, # chat_format="llama-3", # TODO: Understand if this is being inferred correctly from the model metadata. verbose=False, # Change to True for verbose output when running the model in development. seed=4294967295, # Copied from llama.cpp server. n_threads=cores, n_threads_batch=cores, ) def run(self): """Run the LLM with the selected prompt.""" if self.llm is None or self.data is None: return self.output_text.clear() self.btn_run.setEnabled(False) self.worker = LlamaWorker(self.llm, self.prompt) self.worker.signals.started.connect(self.on_execution_started) self.worker.signals.finished.connect(self.on_execution_finished) self.worker.signals.error.connect(self.on_execution_error) self.worker.signals.stream_token.connect(self.on_stream_token) self.worker.signals.stream_token_first_received.connect(self.on_stream_token_first_received) self.worker.start() def stop(self): """Stop the current execution.""" if self.worker and self.worker.isRunning(): self.worker.terminate() self.worker.wait() self.on_execution_finished() def on_execution_started(self): """Handle the start of execution.""" self.btn_run.setText(self.tr("Generating response...")) self.btn_is_running = True self.prompt_selector.setEnabled(False) def on_execution_finished(self): """Handle the completion of execution.""" self.btn_run.setText(self.tr("Execute")) self.btn_is_running = False self.btn_run.setEnabled(True) self.btn_stop.setVisible(False) self.btn_run.setVisible(True) self.prompt_selector.setEnabled(True) def on_execution_error(self, error_msg): """Handle execution errors.""" self.btn_run.setEnabled(True) self.btn_run.setText(self.tr("Execute")) QMessageBox.critical(self, self.tr("Error"), error_msg) def on_stream_token(self, token): """Inserts token and scrolls down to ensure stream is allways visible.""" self.output_text.insertPlainText(token) self.output_text.ensureCursorVisible() def on_stream_token_first_received(self): """Handle the first token received event by allowing the user to stop the execution.""" self.btn_run.setVisible(False) self.btn_stop.setVisible(True) def retranslateUI(self): """Retranslate the UI elements.""" self.setWindowTitle(self.tr("AI assistant")) self.btn_run.setText(self.tr("Execute")) self.btn_stop.setText(self.tr("Stop execution")) self.output_text.setPlaceholderText(self.tr("Results will be displayed here...")) def _calculate_half_cpu_count(self) -> int: """Returns half of the core number of the current machine. By default LLMs use all of the available cores in the machine causing the computer to freeze as it is using all the resources availables. We are limiting to half. """ cores = os.cpu_count() if cores and isinstance(cores, int): return int(cores / 2) return 4 class LlamaDownloadDialog(QDialog): """Dialog for downloading and selecting LLama models. Based on: https://doc.qt.io/qtforpython-6/examples/example_network_downloader.html """ def __init__(self, parent=None): super().__init__(parent) self.manager = QNetworkAccessManager(self) self.selected_model_path = None self.download_file_path = None self.file = None self.reply = None self.progress_dialog = None self.init_ui() def init_ui(self): """Initialize the UI for the Llama download dialog.""" self.setWindowTitle(self.tr("AI assistant")) layout = QVBoxLayout(self) label_models = QLabel(self.tr("To start using the AI assistant, please download the following model.")) layout.addWidget(label_models) label_download_location = QLabel( self.tr( f'The ODE will save the file in this location: {AI_MODELS_PATH}' ) ) label_download_location.setTextFormat(Qt.TextFormat.RichText) label_download_location.setTextInteractionFlags(Qt.TextInteractionFlag.TextBrowserInteraction) label_download_location.linkActivated.connect(self.open_download_directory) layout.addWidget(label_download_location) # Single model row self.create_model_row(layout) # Next button section next_layout = QHBoxLayout() next_layout.addStretch() # Push button to the right self.btn_next = QPushButton(self.tr("Next")) self.btn_next.clicked.connect(self.on_next) self.btn_next.setDefault(True) next_layout.addWidget(self.btn_next) layout.addLayout(next_layout) # Update button states on startup self.update_ui_state() def create_model_row(self, parent_layout): """Create the single model row with buttons.""" # Model row widget model_row = QWidget() model_row.setObjectName("llamaDownloadModelRow") model_layout = QHBoxLayout(model_row) model_layout.setContentsMargins(10, 8, 10, 8) # Model name label self.model_label = QLabel() model_layout.addWidget(self.model_label) model_layout.addStretch() # Push buttons to the right # Delete button self.btn_delete = QPushButton(self.tr("Delete")) self.btn_delete.setObjectName("deleteButton") self.btn_delete.clicked.connect(self.on_delete_model) model_layout.addWidget(self.btn_delete) # Download button self.btn_download = QPushButton(self.tr("Download")) self.btn_download.setObjectName("downloadButton") # Class name for QSS self.btn_download.clicked.connect(self.on_download_model) model_layout.addWidget(self.btn_download) model_layout.setSpacing(10) parent_layout.addWidget(model_row) def update_ui_state(self): """Update UI elements based on download status.""" is_downloaded = self.is_model_downloaded() # Update label text and style status_text = "Downloaded" if is_downloaded else "Not downloaded" self.model_label.setText(f"{AI_MODEL.name} ({status_text})") # Update button states and styles self.btn_delete.setEnabled(is_downloaded) self.btn_download.setEnabled(not is_downloaded) self.btn_next.setEnabled(is_downloaded) def is_model_downloaded(self): """Check if the model is already downloaded.""" model_path = AI_MODELS_PATH / AI_MODEL.filename return model_path.exists() def on_next(self): """Continue to next step - use the downloaded model.""" model_path = AI_MODELS_PATH / AI_MODEL.filename self.selected_model_path = str(model_path) self.accept() def open_download_directory(self): """Open the directory where models are downloaded.""" path = str(AI_MODELS_PATH) if sys.platform == "win32": os.system(f'explorer.exe /select,"{Path(path)}"') elif sys.platform == "darwin": os.system(f'osascript -e \'tell application "Finder" to reveal (POSIX file "{path}")\'') os.system("osascript -e 'tell application \"Finder\" to activate'") else: cmd_run = f'dbus-send --dest=org.freedesktop.FileManager1 --type=method_call /org/freedesktop/FileManager1 org.freedesktop.FileManager1.ShowItems array:string:"{path}" string:""' os.system(cmd_run) @Slot() def on_download_model(self): """Download the model.""" self.download_file_path = AI_MODELS_PATH / AI_MODEL.filename if self.download_file_path.exists(): ret = QMessageBox.question( self, self.tr("File exists"), self.tr("Do you want to delete it and download it again?"), QMessageBox.Yes | QMessageBox.No, ) if ret == QMessageBox.No: return self.download_file_path.unlink() # Disable download button during download self.btn_download.setEnabled(False) # Create the file in write mode to append bytes self.file = QSaveFile(str(self.download_file_path)) if self.file.open(QIODevice.OpenModeFlag.WriteOnly): self.reply = self.manager.get(QNetworkRequest(AI_MODEL.url)) self.progress_dialog = QProgressDialog(self.tr("Downloading model"), self.tr("Cancel"), 0, 0, self) self.progress_dialog.setWindowTitle(self.tr("LLM Model Download Progress")) self.progress_dialog.setAutoClose(True) self.progress_dialog.setWindowModality(Qt.WindowModality.WindowModal) self.progress_dialog.canceled.connect(self.on_download_abort) self.reply.downloadProgress.connect(self.on_download_progress) self.reply.finished.connect(self.on_download_finished) self.reply.readyRead.connect(self.on_download_ready_read) self.reply.errorOccurred.connect(self.on_download_error) else: error = self.file.errorString() QMessageBox.warning(self, self.tr("Error Occurred"), error) @Slot() def on_delete_model(self): """Delete the model file.""" # Confirm deletion ret = QMessageBox.question( self, self.tr("Confirm Deletion"), self.tr(f"Are you sure you want to delete {AI_MODEL.name}?"), QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No, ) if ret == QMessageBox.StandardButton.No: return model_path = AI_MODELS_PATH / AI_MODEL.filename model_path.unlink(missing_ok=True) self.update_ui_state() @Slot() def on_download_abort(self): """When user press abort button""" if self.reply: self.reply.abort() if self.file: self.file.cancelWriting() # cancelWriting should delete the file but it doesn't seems to be happening. if self.download_file_path.exists(): self.download_file_path.unlink() self.btn_download.setEnabled(True) @Slot() def on_download_finished(self): """Handle the completion of the download.""" if self.reply: self.reply.deleteLater() if self.file: self.file.commit() self.update_ui_state() # Refresh UI state @Slot() def on_download_ready_read(self): """Get available bytes and store them into the file""" if self.reply: if self.reply.error() == QNetworkReply.NetworkError.NoError: self.file.write(self.reply.readAll()) @Slot(int, int) def on_download_progress(self, bytesReceived: int, bytesTotal: int): """Update progress bar""" if self.reply.isOpen() and bytesTotal > 0: # Percentage progress self.progress_dialog.setMaximum(100) percentage = int((bytesReceived / bytesTotal) * 100) self.progress_dialog.setValue(percentage) text = f"{self.format_size(bytesReceived)} / {self.format_size(bytesTotal)}" self.progress_dialog.setLabelText(text) @Slot(QNetworkReply.NetworkError) def on_download_error(self, code: QNetworkReply.NetworkError): """Show a message if an error happen""" if self.reply: QMessageBox.warning(self, self.tr("Error Occurred"), self.reply.errorString()) self.progress_dialog = None @staticmethod def format_size(bytes_size: int) -> str: # Convert bytes to human-readable format for unit in ["B", "KB", "MB", "GB"]: if bytes_size < 1024.0: return f"{bytes_size:.2f} {unit}" bytes_size /= 1024.0 return f"{bytes_size:.2f} TB" class LlamaInitWorker(QObject): """ Worker to initialize the LLM in a separate thread to avoid blocking the UI and show a loading dialog. """ finished = Signal() error = Signal(str) progress = Signal(str) def __init__(self, ai_llama, model_path): super().__init__() self.ai_llama = ai_llama self.model_path = model_path @Slot() def init_llm(self): try: self.progress.emit("Initializing model...") self.ai_llama.init_llm(self.model_path) self.finished.emit() except Exception as e: self.error.emit(str(e)) ================================================ FILE: src/ode/log_setup.py ================================================ import logging import sys from logging.handlers import RotatingFileHandler from PySide6.QtCore import QtMsgType, qInstallMessageHandler from ode import utils from ode.paths import LOGS_PATH def configure_logging(): """Configure logging for the application.""" # Create the logs directory if it doesn't exist LOGS_PATH.mkdir(parents=True, exist_ok=True) root_logger = logging.getLogger() root_logger.setLevel(logging.INFO) formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") # File handler for logging errors file_handler = RotatingFileHandler( LOGS_PATH / "info.log", maxBytes=5 * 1024 * 1024, # 5MB (5 * 1024 * 1024 bytes) backupCount=3, # 5MB, Keep 3 backup files ) file_handler.setLevel(logging.INFO) file_handler.setFormatter(formatter) root_logger.addHandler(file_handler) # Configure the handler for non handled exceptions configure_exception_handling(root_logger) return root_logger def configure_exception_handling(logger): """Configure exception handling to log uncaught exceptions.""" # This will always be called when an exception is raised and not handled by the application # The only downside is that it will be executed even if the exception is handled in another thread. def exception_hook(exctype, value, traceback): logger.error(f"{exctype.__name__}: {value}", exc_info=(exctype, value, traceback)) utils.show_error_dialog( message=f"An unexpected error occurred: {exctype.__name__}: {value}", title="Error", ) sys._excepthook(exctype, value, traceback) # Replace the default exception hook with our custom one # This is necessary to ensure that the original exception hook is called sys._excepthook = sys.excepthook sys.excepthook = exception_hook # Set up logging for PyQt/PySide def qt_message_handler(msg_type, context, message): if msg_type == QtMsgType.QtFatalMsg or msg_type == QtMsgType.QtCriticalMsg: logger.error(f"Qt Error: {message}") qInstallMessageHandler(qt_message_handler) def get_module_logger(module_name): """Get a logger for a specific module.""" return logging.getLogger(module_name) ================================================ FILE: src/ode/main.py ================================================ import logging import os import sys from enum import IntEnum from importlib.metadata import version from pathlib import Path from typing import Callable from PySide6.QtWidgets import ( QApplication, QMainWindow, QWidget, QVBoxLayout, QHBoxLayout, QTreeView, QPushButton, QLabel, QStackedLayout, QComboBox, QMenu, QMessageBox, QToolTip, QTextEdit, QSplitter, ) from PySide6.QtGui import ( QPixmap, QIcon, QDesktopServices, QAction, QFont, QPalette, QColor, QShortcut, QKeySequence, QKeyEvent, ) from PySide6.QtCore import ( Qt, QSize, QFileInfo, QTranslator, QFile, QTextStream, QThreadPool, Slot, Signal, QItemSelectionModel, QEvent, QModelIndex, QStandardPaths, QTimer, QThread, ) # https://bugreports.qt.io/browse/PYSIDE-1914 from PySide6.QtWidgets import QFileSystemModel, QDialog from ode import paths from ode.dialogs.delete import DeleteDialog from ode.dialogs.llm_dialog_warning import LLMWarningDialog from ode.dialogs.loading import LoadingDialog from ode.file import File from ode.llama import LlamaDialog, LlamaDownloadDialog, LlamaInitWorker from ode.paths import Paths from ode.panels.errors import ErrorsWidget from ode.panels.data import FrictionlessTableModel, DataWorker, DataViewer from ode.panels.source import SourceViewer from ode.dialogs.upload import DataUploadDialog from ode.dialogs.rename import RenameDialog from ode.dialogs.download import DownloadDialog from ode.utils import migrate_metadata_store, setup_ode_internal_folders from ode.log_setup import LOGS_PATH, configure_logging configure_logging() logger = logging.getLogger(__name__) logger.info("Starting Open Data Editor") _VERSION = version("opendataeditor") class ContentIndex(IntEnum): """Enum to represent the index of the content panels. They need to be added in this same order to match the stacked layout indices. """ DATA = 0 ERRORS = 1 SOURCE = 2 class CustomTreeView(QTreeView): """An extended QTreeView to handle custom features for ODE. Currently we want the application to show a Welcome widget with an Upload Button whenever the user clicks on the empty space of the QTreeView. """ empty_area_click = Signal() def __init__(self, parent=None): super().__init__(parent) self.clicked.connect(self.item_clicked) def keyPressEvent(self, event: QKeyEvent): """Override keyPressEvent to handle expande/collapse folders.""" if event.key() == Qt.Key_Return or event.key() == Qt.Key_Enter: index = self.currentIndex() model = self.model() if model and model.hasChildren(index): if self.isExpanded(index): self.collapse(index) else: self.expand(index) super().keyPressEvent(event) def mousePressEvent(self, event): """Emits an event if the user clicks on an empty space.""" index = self.indexAt(event.position().toPoint()) if not index.isValid(): self.empty_area_click.emit() super().mousePressEvent(event) def viewportEvent(self, event): """ Show a tooltip with the filename when hovering over a file in the file navigator. """ if event.type() == QEvent.Type.ToolTip: index = self.indexAt(event.pos()) if index.isValid(): global_pos = event.globalPos() file_path = self.model().filePath(index) filename = Path(file_path).name # We cannot change the QToolTip styles through the style.qss file font = QFont() font.setPointSize(14) QToolTip.setFont(font) palette = QToolTip.palette() palette.setColor(QPalette.ToolTipBase, QColor("#D6D6D6")) palette.setColor(QPalette.ToolTipText, QColor("#333333")) QToolTip.setPalette(palette) QToolTip.showText(global_pos, filename) # We return True to indicate that we handled the event and stop the propagation return True else: return super().viewportEvent(event) def item_clicked(self, index: QModelIndex): """ Handle the click event of the QTreeView. If the item has children, we want to expand/collapse it when clicked. """ model = self.model() if model and model.hasChildren(index): if self.isExpanded(index): self.collapse(index) else: self.expand(index) class ClickableLabel(QLabel): """Add a click event to a QLabel. We want an interaction when the user clicks on the ODE logo of the sidebar. """ clicked = Signal() def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.setFocusPolicy(Qt.FocusPolicy.StrongFocus) def mousePressEvent(self, event): self.clicked.emit() super().mousePressEvent(event) class Sidebar(QWidget): """Widget containing the left sidebar of ODE. This class is responsible for: - Rendering all the components of the Sidebar. - All the logic of the context menu of the File Navigator. """ def __init__(self): super().__init__() layout = QVBoxLayout() self.icon_label = ClickableLabel() pixmap = QPixmap(Paths.asset("logo.svg")) self.icon_label.setPixmap(pixmap) self.icon_label.setAlignment(Qt.AlignmentFlag.AlignLeft) self.button_upload = QPushButton(objectName="button_upload") self.file_navigator = CustomTreeView() self.file_model = QFileSystemModel() self.file_navigator.setModel(self.file_model) self.file_navigator.setRootIndex(self.file_model.setRootPath(str(paths.PROJECT_PATH))) self._show_only_name_column_in_file_navigator(self.file_model, self.file_navigator) self.file_navigator.setHeaderHidden(True) self.file_navigator.setContextMenuPolicy(Qt.ContextMenuPolicy.CustomContextMenu) self.file_navigator.customContextMenuRequested.connect(self._show_context_menu) self._setup_file_navigator_context_menu() self.user_guide = QPushButton() self.user_guide.setIcon(QIcon(Paths.asset("icons/24/open-in-new.svg"))) self.user_guide.setIconSize(QSize(20, 20)) self.report_issue = QPushButton() self.report_issue.setIcon(QIcon(Paths.asset("icons/24/open-in-new.svg"))) self.report_issue.setIconSize(QSize(20, 20)) self.language = QComboBox() # We are changing since default SizeAdjustPolicy has a buggy behaviour with the Splitter. self.language.setSizeAdjustPolicy(QComboBox.SizeAdjustPolicy.AdjustToMinimumContentsLengthWithIcon) options = [ ("English", ""), ("Français", "fr"), ("Deutsch", "de"), ("Español", "es"), ("Português", "pt"), ("Italiano", "it"), ] language_icon = QIcon(Paths.asset("icons/24/language.svg")) for i, (text, locale) in enumerate(options): self.language.addItem(text) self.language.setItemData(i, locale) self.language.setItemIcon(i, language_icon) self.language.setStyleSheet("text-align: left;") layout.addWidget(self.icon_label) layout.addWidget(self.button_upload) layout.addWidget(self.file_navigator) layout.addWidget(self.user_guide) layout.addWidget(self.report_issue) layout.addWidget(self.language) self.setLayout(layout) def retranslateUI(self): """Apply translations to class elements.""" self.button_upload.setText(self.tr("Upload your data")) self.user_guide.setText(self.tr("User guide")) self.report_issue.setText(self.tr("Report an issue")) self.rename_action.setText(self.tr("Rename")) self.open_location_action.setText(self.tr("Open File in Location")) self.delete_action.setText(self.tr("Delete")) def _setup_file_navigator_context_menu(self): """Create the context menu for the file navigator.""" self.context_menu = QMenu(self) self.rename_action = QAction() self.open_location_action = QAction() self.delete_action = QAction() self.rename_action.triggered.connect(self._rename_file_navigator_item) self.open_location_action.triggered.connect(self._open_file_navigator_location) self.delete_action.triggered.connect(self._delete_file_navitagor_item) self.context_menu.addAction(self.rename_action) self.context_menu.addAction(self.open_location_action) self.context_menu.addAction(self.delete_action) def _show_context_menu(self, position): """Show the context menu for a specific index at the specific position.""" index = self.file_navigator.indexAt(position) if index.isValid(): global_pos = self.file_navigator.viewport().mapToGlobal(position) self.context_menu.exec(global_pos) def _rename_file_navigator_item(self): """Ask user for the new name for the selected file/folder.""" index = self.file_navigator.currentIndex() if index.isValid(): file = File(self.file_model.filePath(index)) name = file.path.stem dialog = RenameDialog(self, name) dialog.exec() new_name = dialog.result_text if new_name and new_name != name: try: sheets_names = None if file.path.suffix in [".xls", ".xlsx"]: sheets_names = File.get_sheets_names(file.path) file.rename(new_name, sheets_names) except IsADirectoryError: QMessageBox.warning( self, self.tr("Error"), self.tr("Source is a file but destination a directory.") ) except NotADirectoryError: QMessageBox.warning( self, self.tr("Error"), self.tr("Source is a directory but destination a file.") ) except PermissionError: # Since we have a managed PROJECT_PATH this should never happen. QMessageBox.warning(self, self.tr("Error"), self.tr("Operation not permitted.")) except OSError: QMessageBox.warning(self, self.tr("Error"), self.tr("File with this name already exists.")) else: self.window().statusBar().showMessage(self.tr("Item renamed successfuly.")) def _open_file_navigator_location(self): """Open the folder where the file lives using the OS application.""" index = self.file_navigator.currentIndex() if index.isValid(): path = self.file_model.filePath(index) if sys.platform == "win32": os.system(f'explorer.exe /select,"{Path(path)}"') elif sys.platform == "darwin": os.system(f'osascript -e \'tell application "Finder" to reveal (POSIX file "{path}")\'') os.system("osascript -e 'tell application \"Finder\" to activate'") else: cmd_run = f'dbus-send --dest=org.freedesktop.FileManager1 --type=method_call /org/freedesktop/FileManager1 org.freedesktop.FileManager1.ShowItems array:string:"{path}" string:""' os.system(cmd_run) def _delete_file_navitagor_item(self): """Delete a file/folder from the file navigator (and the OS).""" index = self.file_navigator.currentIndex() if index.isValid(): file = File(self.file_model.filePath(index)) is_selected = self.window().selected_file_path == file.path if DeleteDialog.confirm(self, file.path.name): try: sheets_names = None if file.path.suffix in [".xls", ".xlsx"]: sheets_names = File.get_sheets_names(file.path) file.remove(sheets_names) except OSError as e: QMessageBox.warning(self, self.tr("Error"), str(e)) else: if is_selected: self.window().show_welcome_screen() self.window().statusBar().showMessage(self.tr("Item deleted successfuly.")) def _show_only_name_column_in_file_navigator(self, file_model, file_navigator): """Hide all columns except for the name column (column 0)""" for column in range(file_model.columnCount()): if column != 0: # 0 is the name column file_navigator.setColumnHidden(column, True) class ErrorsReportButton(QPushButton): """Toolbar button for the Errors Report that contains Icon, Text and ErrorCount. QPushButton (Icon+Text) is not enough since we need a three part button: Icon+Text+ErrorCount. In order for the ErrorCount Label to be part of the button (background, hover, clickable) we need to extend the basic QPushButton and override its layout and some methods. """ def __init__(self, parent=None): super().__init__(parent) layout = QHBoxLayout(self) layout.setSpacing(2) # Aligns better with QPushButton look & feel layout.setContentsMargins(0, 0, 0, 0) self.icon_label = QLabel() self.icon_label.setFixedSize(20, 20) # Match icon size layout.addWidget(self.icon_label) self.text_label = QLabel() layout.addWidget(self.text_label) self.error_label = QLabel() self.error_label.setProperty("error", True) # For referencing in our style.qss file layout.addWidget(self.error_label) # This is some Qt Magic to properly display the button and of all its labels # (auto-expanding content-based width) layout.setSizeConstraint(QHBoxLayout.SizeConstraint.SetMinimumSize) self.setLayout(layout) def setText(self, text): self.text_label.setText(text) self.updateGeometry() # Force layout recalc def setIcon(self, icon): """Overrides the QPushButton method to set the icon to our icon_label. A difference with QPushButton is that we handle the size here instead of calling QPushButton.setIconSize(). """ if icon.isNull(): self.icon_label.clear() else: pixmap = icon.pixmap(QSize(20, 20)) self.icon_label.setPixmap(pixmap) self.updateGeometry() def enable(self, number): """Enables the button and displays the error number. All children labels should also be enabled so we can use QSS pseudo-states for styling. """ self.setEnabled(True) self.icon_label.setEnabled(True) self.text_label.setEnabled(True) self.error_label.setEnabled(True) if number <= 999: self.error_label.setText(str(number)) else: self.error_label.setText("+999") self.error_label.show() self.updateGeometry() def disable(self): """Disables the button and hides the error number. Disabled button will have a grey color and no hover style. All children labels should also be disabled (so we can use QSS pseudo-states for styling) """ self.setEnabled(False) self.icon_label.setEnabled(False) self.text_label.setEnabled(False) self.error_label.setEnabled(False) self.error_label.hide() self.updateGeometry() class Toolbar(QWidget): """Widget containing ODE's toolbar. The toolbar contains: - Buttons that allow the user to navigate between the panels (Data, Metadata, Errors, Source, etc) - Buttons for the main actions like AI, Export and Save. """ def __init__(self): super().__init__() layout = QHBoxLayout() # Buttons on the left # Setting the cursor to PointingHandCursor to indicate that the button is clickable because # is not working with the style.qss file. self.button_data = QPushButton() self.button_data.setCursor(Qt.CursorShape.PointingHandCursor) self.button_errors = ErrorsReportButton() self.button_errors.setCursor(Qt.CursorShape.PointingHandCursor) self.button_errors.setIcon(QIcon(Paths.asset("icons/24/rule.svg"))) self.button_source = QPushButton() self.button_source.setCursor(Qt.CursorShape.PointingHandCursor) self.button_source.setIcon(QIcon(Paths.asset("icons/24/code.svg"))) self.button_source.setIconSize(QSize(20, 20)) layout.addWidget(self.button_data) layout.addWidget(self.button_errors) layout.addWidget(self.button_source) # Excel Sheet Selection self.excel_sheet_layout = QHBoxLayout() self.excel_sheet_container = QWidget() self.excel_sheet_label = QLabel() self.excel_sheet_label.setObjectName("excelSheetLabel") self.excel_sheet_combo = QComboBox() self.excel_sheet_combo.setObjectName("excelSheetCombo") self.excel_sheet_layout.addWidget(self.excel_sheet_label) self.excel_sheet_layout.addWidget(self.excel_sheet_combo) self.excel_sheet_container.setLayout(self.excel_sheet_layout) layout.addWidget(self.excel_sheet_container) # Spacer to push right-side buttons to the end layout.addStretch() # Buttons on the right self.button_ai = QPushButton(objectName="button_ai") self.button_ai.setIcon(QIcon(Paths.asset("icons/24/wand.svg"))) self.button_ai.setIconSize(QSize(20, 20)) self.button_ai.setFixedWidth(90) self.button_export = QPushButton(objectName="button_export") self.button_export.setIcon(QIcon(Paths.asset("icons/24/file-download.svg"))) self.button_export.setIconSize(QSize(20, 20)) self.button_export.setEnabled(False) self.button_save = QPushButton(objectName="button_save") self.button_save.setMinimumSize(QSize(117, 35)) self.button_save.setIcon(QIcon(Paths.asset("icons/24/check.svg"))) self.button_save.setIconSize(QSize(20, 20)) self.button_save.setEnabled(False) # self.update_qss_button = QPushButton("QSS") # layout.addWidget(self.update_qss_button) layout.addWidget(self.button_ai) layout.addWidget(self.button_export) layout.addWidget(self.button_save) self.setLayout(layout) def retranslateUI(self): """Apply translations to class elements.""" self.button_data.setText(self.tr("Data")) self.button_errors.setText(self.tr("Errors Report")) self.button_source.setText(self.tr("Source code")) self.button_export.setText(self.tr("Export")) self.button_save.setText(self.tr("Save changes")) self.button_ai.setText(self.tr("AI")) self.excel_sheet_label.setText(self.tr("Sheet:")) class Content(QWidget): """Widget to display the main section of the ODE. This widget represents the main content area of the Open Data Editor. If a file is selected, it will display the Toolbar and Panels. If no file is selected it will display a Welcoming widget with an upload button. """ def __init__(self): super().__init__() layout = QVBoxLayout() layout.setSpacing(0) layout.setContentsMargins(0, 0, 0, 0) self.toolbar = Toolbar() layout.addWidget(self.toolbar) self.panels = QWidget(self) self.stacked_layout = QStackedLayout() self.panels.setLayout(self.stacked_layout) self.data_view = DataViewer() self.errors_view = ErrorsWidget() self.source_view = SourceViewer() self.ai_llama = LlamaDialog(self) self.stacked_layout.addWidget(self.data_view) # ContentIndex.DATA = 0 self.stacked_layout.addWidget(self.errors_view) # ContentIndex.ERRORS = 1 self.stacked_layout.addWidget(self.source_view) # ContentIndex.SOURCE = 2 layout.addWidget(self.panels) self.setLayout(layout) class Welcome(QWidget): """Displays an Upload button when no files are selected.""" def __init__(self): super().__init__() main_layout = QVBoxLayout() main_layout.setAlignment(Qt.AlignmentFlag.AlignHCenter | Qt.AlignmentFlag.AlignVCenter) image_label = QLabel(self) pixmap = QPixmap(Paths.asset("images/welcome_screen.png")) image_label.setPixmap(pixmap) image_label.setAlignment(Qt.AlignmentFlag.AlignCenter) self.label_top = QLabel() self.label_top.setAlignment(Qt.AlignmentFlag.AlignCenter) self.label_top.setStyleSheet("font-size: 14px; font-weight: 800;") self.button_upload = QPushButton() self.label_bottom = QLabel() self.label_bottom.setStyleSheet("font-size: 14px;") main_layout.addWidget(image_label) main_layout.addWidget(self.label_top) main_layout.addWidget(self.button_upload) main_layout.addWidget(self.label_bottom) self.setLayout(main_layout) self.retranslateUI() def retranslateUI(self): """Apply translations to class elements.""" self.label_top.setText(self.tr("The ODE supports Excel & csv files")) self.label_bottom.setText(self.tr("You can also add links to online tables")) self.button_upload.setText(self.tr("Upload your data")) class MainWindow(QMainWindow): """Main Window of the Open Data Editor. This class is also the main Controller of the application with two reponsibilites: - Connect signals/slots of all widgets and elements (including children). - Handle custom logic that that requires children interactions with each other. The main window is composed by: - A Sidebar with the file navigator and buttons for several actions. - A Main area that can display two widgets: - A Welcome widget if no file is selected. - A Toolbar + Panel widgets if a file is selected. """ def __init__(self): super().__init__() self.setWindowTitle("Open Data Editor") icon = QIcon(Paths.asset("icons/icon.png")) self.setWindowIcon(icon) self.threadpool = QThreadPool() self.selected_file_path = Path() central_widget = QWidget(objectName="central_widget") layout = QHBoxLayout(central_widget) self.setCentralWidget(central_widget) splitter = QSplitter(Qt.Orientation.Horizontal) layout.addWidget(splitter) self.sidebar = Sidebar() self.main = QWidget() self.main_layout = QStackedLayout() self.welcome = Welcome() self.content = Content() self.main_layout.addWidget(self.welcome) self.main_layout.addWidget(self.content) self.main.setLayout(self.main_layout) splitter.addWidget(self.sidebar) splitter.addWidget(self.main) # Set splitter proportions (15% for sidebar) splitter.setSizes([int(self.width() * 0.15), int(self.width() * 0.85)]) self._menu_bar() # Handle Slot/Signals self.sidebar.button_upload.clicked.connect(self.on_button_upload_click) self.welcome.button_upload.clicked.connect(self.on_button_upload_click) self.sidebar.file_navigator.clicked.connect(self.on_tree_click) self.sidebar.file_navigator.activated.connect(self.on_tree_click) self.sidebar.user_guide.clicked.connect(self.open_user_guide) self.sidebar.report_issue.clicked.connect(self.open_report_issue) self.sidebar.language.activated.connect(self.on_language_change) self.content.toolbar.button_export.clicked.connect(self.on_export_click) self.content.toolbar.button_save.clicked.connect(self.on_save_click) self.content.toolbar.button_ai.clicked.connect(self.on_ai_click) self.content.toolbar.button_data.clicked.connect(lambda: self.change_active_panel(ContentIndex.DATA)) self.content.toolbar.button_errors.clicked.connect(lambda: self.change_active_panel(ContentIndex.ERRORS)) self.content.toolbar.button_source.clicked.connect(lambda: self.change_active_panel(ContentIndex.SOURCE)) self.content.toolbar.excel_sheet_combo.currentTextChanged.connect(self.on_excel_sheet_selection_changed) self.content.data_view.on_save.connect(self.on_data_view_save) self.sidebar.file_navigator.empty_area_click.connect(self.show_welcome_screen) self.sidebar.icon_label.clicked.connect(self.show_welcome_screen) # Shortcuts self.shortcut_f5 = QShortcut(QKeySequence("F5"), self) self.shortcut_f5.activated.connect(self.on_ai_click) # Data Panel self.shortcut_alt_d = QShortcut(QKeySequence(Qt.Modifier.ALT | Qt.Key.Key_D), self) self.shortcut_alt_d.activated.connect(lambda: self.change_active_panel(ContentIndex.DATA)) # Errors Panel self.shortcut_alt_r = QShortcut(QKeySequence(Qt.Modifier.ALT | Qt.Key.Key_R), self) self.shortcut_alt_r.activated.connect(lambda: self.change_active_panel(ContentIndex.ERRORS)) # Source Panel self.shortcut_alt_s = QShortcut(QKeySequence(Qt.Modifier.ALT | Qt.Key.Key_S), self) self.shortcut_alt_s.activated.connect(lambda: self.change_active_panel(ContentIndex.SOURCE)) # Save if sys.platform == "darwin": self.shortcut_control_s = QShortcut(QKeySequence(Qt.MetaModifier | Qt.Key_S), self) else: self.shortcut_control_s = QShortcut(QKeySequence(Qt.Modifier.CTRL | Qt.Key.Key_S), self) self.shortcut_control_s.activated.connect(self.on_save_click) # Translation self.translator = QTranslator() self.retranslateUI() # self.content.toolbar.update_qss_button.clicked.connect(self.apply_stylesheet) self.apply_stylesheet() self._create_status_bar() def _create_status_bar(self): self.statusBar().showMessage(self.tr("Ready.")) def _menu_bar(self): """Creates the menu bar and assign all its actions. Names and titles are going to be set in retranslateUI. """ # File self.menu_file = QMenu() self.menu_file_add = QMenu() self.menu_file_add_action_upload_file = QAction() self.menu_file_add_action_upload_file.triggered.connect(self.upload_data) self.menu_file_add.addAction(self.menu_file_add_action_upload_file) self.menu_file_add_action_upload_external_url = QAction() self.menu_file_add_action_upload_external_url.triggered.connect(lambda: self.upload_data(external_first=True)) self.menu_file_add.addAction(self.menu_file_add_action_upload_external_url) self.menu_file.addMenu(self.menu_file_add) self.menuBar().addMenu(self.menu_file) # View self.menu_view = QMenu() # By default is disabled because not file is selected self.menu_view.setEnabled(False) self.menu_view_action_errors_panel = QAction() self.menu_view_action_errors_panel.triggered.connect(lambda: self.change_active_panel(ContentIndex.ERRORS)) self.menu_view.addAction(self.menu_view_action_errors_panel) self.menu_view_action_source_panel = QAction() self.menu_view_action_source_panel.triggered.connect(lambda: self.change_active_panel(ContentIndex.SOURCE)) self.menu_view.addAction(self.menu_view_action_source_panel) self.menuBar().addMenu(self.menu_view) # Help self.menu_help = QMenu() self.menuBar().addMenu(self.menu_help) self.menu_help_action_user_guide = QAction() self.menu_help_action_user_guide.triggered.connect(self.open_user_guide) self.menu_help.addAction(self.menu_help_action_user_guide) self.menu_help_action_report_issue = QAction() self.menu_help_action_report_issue.triggered.connect(self.open_report_issue) self.menu_help.addAction(self.menu_help_action_report_issue) self.menu_help_action_show_logs = QAction() self.menu_help_action_show_logs.triggered.connect(self.show_logs_content) self.menu_help.addAction(self.menu_help_action_show_logs) self.menu_help_action_about = QAction() self.menu_help_action_about.triggered.connect(self.open_about_dialog) self.menu_help.addAction(self.menu_help_action_about) def apply_stylesheet(self): """Reads our main style QSS file and applies it to the application. Tip: this method can be connected to a button to live reload changes. """ qss_file = QFile(Paths.asset("style.qss")) qss_file.open(QFile.ReadOnly) qss_content = QTextStream(qss_file).readAll() self.setStyleSheet(qss_content) @Slot() def show_welcome_screen(self): """Focus on the welcome screen and clear file navigator selection.""" self.sidebar.file_navigator.selectionModel().clear() self.main_layout.setCurrentIndex(0) # No file is selected, disable the View menu self.menu_view.setEnabled(False) def on_export_click(self): """Handle the click on the Export button.""" # TODO: we are using a proxy variable to check if the file has errors. We should find a # better state variable for it. has_errors = self.content.toolbar.button_errors.isEnabled() download_dialog = DownloadDialog(self, self.selected_file_path, has_errors) download_dialog.download_data_with_errors.connect(self.on_download_error_file) download_dialog.show() def on_data_changed(self): """Action that enables the Save Button.""" self.content.toolbar.button_save.setEnabled(True) def on_ai_click(self): """Handle the click on the AI button.""" if not LLMWarningDialog.confirm(self): return ai_llama_download = LlamaDownloadDialog(self) if ai_llama_download.exec() == QDialog.DialogCode.Accepted: selected_model = ai_llama_download.selected_model_path if selected_model: self.loading_dialog = LoadingDialog(self) self.worker_thread = QThread() self.worker = LlamaInitWorker(self.content.ai_llama, selected_model) # Move to worker thread instead of QThread inheritance to avoid concurrency conflicts self.worker.moveToThread(self.worker_thread) # Connecting signals self.worker_thread.started.connect(self.worker.init_llm) self.worker.finished.connect(self.on_llm_init_finished) self.worker.error.connect(self.on_llm_init_error) self.worker.progress.connect(self.loading_dialog.show_message) # Signal to clean up the thread when the work is done self.worker.finished.connect(self.worker_thread.quit) self.worker.error.connect(self.worker_thread.quit) self.worker_thread.finished.connect(self.worker_thread.deleteLater) # Show the loading dialog and start the thread self.loading_dialog.show_immediately() self.worker_thread.start() @Slot() def on_llm_init_finished(self): """Callback to be called when the LLM is initialized.""" self.loading_dialog.accept() self.content.ai_llama.show() @Slot(str) def on_llm_init_error(self, error_message): """Callback to be called when there is an error initializing the LLM.""" self.loading_dialog.reject() logger.error(f"Error initializing the LLM: {error_message}") QMessageBox.critical(self, self.tr("Error"), self.tr("Error initializing the LLM:\n") + error_message) def change_active_panel(self, panel_index: ContentIndex): """Change the active panel in the content area and highlight its toolbar button. This method changes the active panel in the content area based on the provided panel index and sets the "active" property of the related button in the toolbar. """ if panel_index < 0 or panel_index >= self.content.stacked_layout.count(): raise ValueError("Invalid panel index.") self.content.stacked_layout.setCurrentIndex(panel_index) buttons = [ self.content.toolbar.button_data, # ContentIndex.DATA = 0 self.content.toolbar.button_errors, # ContentIndex.ERRORS = 1 self.content.toolbar.button_source, # ContentIndex.SOURCE = 2 ] for i, button in enumerate(buttons): button.setProperty("active", i == panel_index) button.style().polish(button) # Force the button to update its style # For the Errors label we need to check if it is enabled and update its style # to hide the error label styles if we are in the Errors panel. button_error_label = self.content.toolbar.button_errors.error_label hide_styles_for_error_label = panel_index != ContentIndex.ERRORS button_error_label.setProperty("error", hide_styles_for_error_label) button_error_label.style().polish(button_error_label) def retranslateUI(self): """Set the text of all the UI elements using a translation function. retranslateUI is a pattern used in Qt to handle the dynamic updates of languages. All text that needs to be translated needs to be set inside this function so we can call it to refresh the UI. This method should be call when: a) The application is load for the first time (MainWindow.__init__) b) Every time the user selects a different language (language ComboBox) c) An event related to language change is fired (like the user changing the language in the OS. Not Implemented yet). """ # Update translated text for menus # File menu self.menu_file.setTitle(self.tr("File")) self.menu_file_add.setTitle(self.tr("Add")) self.menu_file_add_action_upload_file.setText(self.tr("File/Folder")) self.menu_file_add_action_upload_external_url.setText(self.tr("External URL")) # View self.menu_view.setTitle(self.tr("View")) self.menu_view_action_errors_panel.setText(self.tr("Errors panel")) self.menu_view_action_source_panel.setText(self.tr("Source panel")) # Help self.menu_help.setTitle(self.tr("Help")) self.menu_help_action_user_guide.setText(self.tr("User Guide")) self.menu_help_action_report_issue.setText(self.tr("Report an Issue")) self.menu_help_action_show_logs.setText(self.tr("View logs")) self.menu_help_action_about.setText(self.tr("About")) # Hook retranslateUI for main widgets self.sidebar.retranslateUI() self.welcome.retranslateUI() self.content.toolbar.retranslateUI() # Hook retranslateUI for all panels (data, errors, metadata, etc) self.content.data_view.retranslateUI() self.content.errors_view.retranslateUI() self.content.source_view.retranslateUI() self.content.ai_llama.retranslateUI() self.excel_sheet_name = None def on_language_change(self, index): """Gets a *.qm translation file and calls retranslateUI. Translation files are generated using Qt tools pyside6-lupdate and pyside6-lrelease. """ locale = self.sidebar.language.itemData(index) app = QApplication.instance() if not locale: app.removeTranslator(self.translator) self.retranslateUI() return filename = locale + ".qm" filepath = Paths.translation(filename) if self.translator.load(filepath): app.installTranslator(self.translator) else: print(f"Error when loading {filepath} translator file. Fallbacking to English.") app.removeTranslator(self.translator) self.retranslateUI() self.statusBar().showMessage(self.tr("Language changed.")) def upload_data(self, external_first=False): """Copy data file to the project folder of ode. After successful upload the file should be selected, validated, and displayed. """ dialog = DataUploadDialog(self, external_first=external_first) ok, path = dialog.upload_dialog() if ok and path: self.selected_file_path = path # Calling file_model.index() has a weird side-effect in the QTreeView that will display the new # uploaded file at the end of the list instead of the default alphabetical order. if path.suffix in [".xls", ".xlsx"]: self.excel_sheet_name = File.get_sheets_names(path)[0] else: self.excel_sheet_name = None index = self.sidebar.file_model.index(str(path)) self.sidebar.file_navigator.selectionModel().select(index, QItemSelectionModel.SelectionFlag.ClearAndSelect) self.read_validate_and_display_file(path) def on_button_upload_click(self): self.upload_data() def on_save_click(self): """Saves changes made in the Table View into the file.""" self.table_model.write_data(self.selected_file_path, sheet_name=self.excel_sheet_name) # TODO: Since the file is already in memory we should only validate/display to avoid unecessary tasks. self.read_validate_and_display_file(self.selected_file_path) self.statusBar().showMessage(self.tr("File and Metadata changes saved.")) def on_data_view_save(self, save_data): """ Reloads the file and updates the views. when is saved in the data view """ if save_data: self.table_model.write_data(self.selected_file_path, sheet_name=self.excel_sheet_name) self.read_validate_and_display_file(self.selected_file_path) @Slot(tuple) def update_views(self, worker_data): """Update all the main views with the data provided by the read worker. This method is connected to the data widget Worker's signal and it will receive the data, the frictionless report and a list of errors. """ filepath, data, errors = worker_data self.table_model = FrictionlessTableModel(data, errors) self.table_model.dataChanged.connect(self.on_data_changed) self.content.data_view.display_data(self.table_model, filepath, sheet_name=self.excel_sheet_name) self.content.errors_view.display_errors(errors, self.table_model) self.content.source_view.open_file(filepath) self.content.ai_llama.set_data(data) self.update_excel_sheet_dropdown(filepath) # Always focus back to the data view. self.main_layout.setCurrentIndex(1) self.change_active_panel(ContentIndex.DATA) def update_excel_sheet_dropdown(self, filepath): """ Update the Excel sheet dropdown with the names of the sheets in the selected file. If there are no sheets, the dropdown and label will be hidden. """ self.content.toolbar.excel_sheet_combo.blockSignals(True) self.content.toolbar.excel_sheet_combo.clear() sheets_names = File.get_sheets_names(filepath) if len(sheets_names) > 0: if self.excel_sheet_name is None: self.excel_sheet_name = sheets_names[0] # We only show the dropdown if there are multiple sheets if len(sheets_names) > 1: self.content.toolbar.excel_sheet_combo.addItems(sheets_names) self.content.toolbar.excel_sheet_combo.setCurrentText(self.excel_sheet_name) self.content.toolbar.excel_sheet_container.setVisible(True) else: self.content.toolbar.excel_sheet_container.setVisible(False) self.excel_sheet_name = None self.content.toolbar.excel_sheet_combo.blockSignals(False) def on_excel_sheet_selection_changed(self, sheet_name: str): """ Handle the change of the selected Excel sheet in the dropdown. Reloads the file with the selected sheet name and updates the views. """ if self.selected_file_path.suffix in [".xls", ".xlsx"]: self.excel_sheet_name = sheet_name self.read_validate_and_display_file(self.selected_file_path) else: raise ValueError("Selected file is not an Excel file.") @Slot(tuple) def update_toolbar(self, worker_data): """ Updates the toolbar based on the data provided by the read worker. This method is connected to the data widget Worker's signal and it will receive the data, the frictionless report and a list of errors. """ _, _, errors = worker_data errors_count = len(errors) # If we don't have errors we don't enable the Errors Report tab. if errors_count == 0: self.content.toolbar.button_errors.disable() else: self.content.toolbar.button_errors.enable(errors_count) # Save button should be disabled everytime we load and display a new file. self.content.toolbar.button_save.setEnabled(False) @Slot(tuple) def update_menu_bar(self, worker_data): """ Updates the menu bar based on the data provided by the read worker. This method is connected to the data widget Worker's signal and it will receive the data, the frictionless report and a list of errors. """ self.menu_view.setEnabled(True) _, _, errors = worker_data errors_count = len(errors) if errors_count == 0: self.menu_view_action_errors_panel.setEnabled(False) else: self.menu_view_action_errors_panel.setEnabled(True) def read_validate_and_display_file(self, file_path, fn_callback: Callable | None = None): """Reads a file, validates it and refresh the whole UI. This method is called when the user selects a file in our QTreeView but there could be other workflows in the application that will require this logic (like Uploading a File). It will create a Worker to read data in the background and display a ProgressDialog if it is taking too long. Reading with a worker is a requirement to display a proper QProgressDialog. Args: file_path (Path): The path to the file to read. fn_finished (Callable, optional): A function to call when the worker finishes. Defaults to None. It cannot be a lambda function since it will not be picked and sent to the worker thread. """ info = QFileInfo(file_path) if info.isFile() and info.suffix() in ["csv", "xls", "xlsx"]: self.loading_dialog = LoadingDialog(self) worker = DataWorker(file_path, self.excel_sheet_name) worker.signals.finished.connect(self.update_views) worker.signals.finished.connect(self.update_toolbar) worker.signals.finished.connect(self.update_menu_bar) worker.signals.finished.connect(self.loading_dialog.close) worker.signals.finished.connect(self.loading_dialog.cancel_loading_timer) if fn_callback: worker.signals.finished.connect(fn_callback) worker.signals.messages.connect(self.statusBar().showMessage) worker.signals.messages.connect(self.loading_dialog.show_message) self.threadpool.start(worker) self.loading_dialog.show() def on_tree_click(self, index): """Handles the click action of our File Navigator.""" self.selected_file_path = Path(self.sidebar.file_model.filePath(index)) if self.selected_file_path.is_file(): # Reset the excel sheet name to None to avoid displaying the previous file's sheet if self.selected_file_path.suffix in [".xls", ".xlsx"]: self.excel_sheet_name = File.get_sheets_names(self.selected_file_path)[0] else: self.excel_sheet_name = None self.read_validate_and_display_file(self.selected_file_path) self.content.toolbar.button_export.setEnabled(True) else: self.content.toolbar.button_export.setEnabled(False) def open_about_dialog(self): text = f"Version: {_VERSION}
Website" QMessageBox.about(self, "Open Data Editor", text) def open_user_guide(self): QDesktopServices.openUrl("https://opendataeditor.okfn.org/documentation/welcome") def open_report_issue(self): QDesktopServices.openUrl("https://github.com/okfn/opendataeditor") # Then define the function that will be executed when the action is triggered def show_logs_content(self): file_path = LOGS_PATH / "info.log" try: # Read only the last 40 lines of the file with open(file_path, "r", encoding="utf-8") as file: # Read all lines and store in a list all_lines = file.readlines() # Get the last 40 lines (or all if less than 40) last_lines = all_lines[-100:] if len(all_lines) > 40 else all_lines # Join the lines into a single string content = "".join(last_lines) # Create a dialog to show the content dialog = QDialog(self) dialog.setWindowTitle(self.tr("Last 100 Lines")) dialog.resize(900, 500) # Create a layout for the dialog layout = QVBoxLayout(dialog) # Create a text widget to display the content text_edit = QTextEdit() font = QFont("Courier New") font.setStyleHint(QFont.StyleHint.Monospace) text_edit.setFont(font) text_edit.setReadOnly(True) text_edit.setText(content) layout.addWidget(text_edit) # Create a horizontal layout for buttons button_layout = QHBoxLayout() # Close button close_button = QPushButton(self.tr("Close")) close_button.clicked.connect(dialog.close) button_layout.addWidget(close_button) # Copy button copy_button = QPushButton(self.tr("Copy to Clipboard")) copy_button.clicked.connect(lambda: QApplication.clipboard().setText(text_edit.toPlainText())) button_layout.addWidget(copy_button) layout.addLayout(button_layout) # Show the dialog dialog.exec() except Exception as e: QMessageBox.critical(self, "Error", f"Could not open file: {str(e)}") def on_download_error_file(self): """ Downloads the file with errors to the user's Downloads folder. """ self.table_model.finished.connect(self.loading_dialog.close) self.table_model.finished.connect(self.loading_dialog.cancel_loading_timer) self.loading_dialog.show_message(self.tr("Downloading data with errors...")) # We are showing the dialog instantly without waiting the 300ms delay self.loading_dialog.show(0) # We use QTimer to ensure the download is performed after the dialog is shown QTimer.singleShot(100, self._perform_download) def _perform_download(self): """ Performs the actual download of the file with errors to the user's Downloads folder. """ downloads_path = QStandardPaths.writableLocation(QStandardPaths.StandardLocation.DownloadLocation) filename = self.selected_file_path.name # TODO: do no overwrite existing files filepath = Path(downloads_path, filename) filepath = filepath.with_stem(f"{filepath.stem}_errors").with_suffix(".xlsx") self.table_model.write_error_xlsx(filepath) success_text = self.tr("File downloaded successfully to:\n{}").format(filepath) QMessageBox.information(self, self.tr("Success"), success_text) def main(): app = QApplication(sys.argv) app.setOrganizationName("Open Knowledge Foundation") app.setApplicationName("Open Data Editor") app.setApplicationVersion(_VERSION) app.setStyle("Fusion") # Migration to ODE 1.4 migrate_metadata_store() setup_ode_internal_folders() window = MainWindow() window.showMaximized() window.show() sys.exit(app.exec()) if __name__ == "__main__": main() ================================================ FILE: src/ode/panels/__init__.py ================================================ """ Open Data Editor main widgets module. This module contains the widgets that implements the main functionalities of ODE: - data.py: MVC for displaying the file contents. - errors.py: Views to display a summary of the errors detected in the data. - source.py: Raw view of the file. This module should contain high level widgets that implements core functionalities, low level widgets like custom buttons, labels or titles should be implemented among the high level widget that uses it. """ ================================================ FILE: src/ode/panels/data.py ================================================ import json import csv import logging from pathlib import Path from frictionless import system from PySide6.QtCore import Qt, QAbstractTableModel, QObject, Signal, Slot, QRunnable, QRect, QEvent from PySide6.QtGui import QColor, QIcon, QKeyEvent, QPen from PySide6.QtWidgets import QWidget, QVBoxLayout, QTableView, QLabel, QApplication, QStyledItemDelegate, QStyle from openpyxl import Workbook, load_workbook from openpyxl.styles import PatternFill import xlwt import xlrd from xlutils.copy import copy from ode import utils from ode.dialogs.metadata import ColumnMetadataDialog, ColumnMetadataField from ode.file import File from ode.shared import COLOR_RED, COLOR_BLUE from ode.paths import Paths DEFAULT_LIMIT_ERRORS = 1000 logger = logging.getLogger(__name__) class DataWorkerSignals(QObject): """Define the signals for the DataWorker.""" finished = Signal(tuple) messages = Signal(str) class DataWorker(QRunnable): """Worker to execute all the reading and validation tasks. This worker will allow us to read and validate the data (that can take several seconds) in the background. By moving this logic to a Worker we can avoid the application to get freeze while reading and instead display proper messages to the user. """ def __init__(self, filepath, sheet_name=None): super().__init__() self.file = File(filepath, sheet_name) self.signals = DataWorkerSignals() self.sheet_name = sheet_name self.resource = self.file.get_or_create_metadata(sheet_name).get("resource") @Slot() def run(self): """Reads and validates the data. We are using Resource.read_cells() because we want to read the data as is in the file in order to properly show data errors. We are using the system context to allow us to read from non-relative paths. This method emits a finished signal with all the data that the main UI requires to display the table and the errors. """ with system.use_context(trusted=True): self.signals.messages.emit(QApplication.translate("DataWorker", "Reading file...")) data = self.resource.read_cells() self.signals.messages.emit(QApplication.translate("DataWorker", "Checking errors...")) report = self.resource.validate(limit_errors=DEFAULT_LIMIT_ERRORS) self.signals.messages.emit(QApplication.translate("DataWorker", "Drawing table...")) errors = [] if not report.valid: try: # report.error is only available for single error report errors.append(report.error) except Exception: errors = report.tasks[0].errors self.signals.messages.emit(QApplication.translate("DataWorker", "Read and error checking finished.")) self.signals.finished.emit((self.file.path, data, errors)) class ColumnMetadataIconDelegate(QStyledItemDelegate): """ Custom delegate to render an icon in the first row of the table. """ icon_clicked = Signal(object) def __init__(self, icon_path, parent=None): super().__init__(parent) self.icon_size = 14 self.icon = QIcon(icon_path) def _get_icon_rect(self, option): """Get the rectangle where the icon should be painted.""" return QRect( option.rect.right() - self.icon_size, option.rect.top(), self.icon_size, self.icon_size, ) def paint(self, painter, option, index): """Paint the icon in the first row of the table adds blue background if mouse over.""" super().paint(painter, option, index) if index.row() == 0: self.icon.paint(painter, self._get_icon_rect(option)) if option.state & QStyle.StateFlag.State_MouseOver: pen = QPen(COLOR_BLUE) pen.setWidth(3) painter.setPen(pen) # We do it like this because it was painting outside the cell rect = option.rect.adjusted(1, 1, -1, -1) painter.drawLine(rect.topLeft(), rect.topRight()) painter.drawLine(rect.topLeft(), rect.bottomLeft()) painter.drawLine(rect.topRight(), rect.bottomRight()) painter.drawLine(rect.bottomLeft(), rect.bottomRight()) def editorEvent(self, event, model, option, index): """Handle mouse events for the first row.""" if index.row() == 0 and event.type() == QEvent.Type.MouseButtonPress: self.icon_clicked.emit(index.column()) return True return super().editorEvent(event, model, option, index) class FrictionlessTableModel(QAbstractTableModel): finished = Signal() def __init__( self, data=[], errors=[], ): super().__init__() self._data = data self._row_count = self._get_row_count() self.errors = self._get_errors(errors) self._column_count = self._get_column_count() def write_data(self, filepath: Path, sheet_name=None): """ Write the data to a file in the format specified by the file extension. """ extension = filepath.suffix.lower() if extension == ".csv": self.write_data_csv(filepath) elif extension == ".xlsx": self.write_data_xlsx(filepath, sheet_name) elif extension == ".xls": self.write_data_xls(str(filepath), sheet_name) else: raise ValueError(f"Unsupported format: {extension}. Use .csv, .xlsx or .xls") def write_data_csv(self, filepath: Path): """ Write the data to a CSV file. """ logger.info(f"Writing data to CSV file: {filepath}") resource = File(filepath).get_or_create_metadata().get("resource") dialect = resource.dialect.to_dict() csv_config = dialect.get("csv", None) # Default delimiter delimiter = "," if csv_config and "delimiter" in csv_config: delimiter = csv_config["delimiter"] with open(filepath, "w", newline="", encoding="utf-8") as csvfile: writer = csv.writer(csvfile, delimiter=delimiter) # Write header writer.writerow(self._data[0]) # Write data rows writer.writerows(self._data[1:]) logger.info(f"Data saved in CSV format: {filepath}") def write_data_xlsx(self, filepath: Path, sheet_name=None): """ Write the data to an Excel file. """ logger.info(f"Writing data to Excel file: {filepath}") wb = load_workbook(filepath) if sheet_name in wb.sheetnames: ws = wb[sheet_name] logger.info(f"Deleting existing data in sheet: {sheet_name}") ws.delete_rows(1, ws.max_row) # Delete all rows else: logger.error(f"Sheet {sheet_name} does not exist in the workbook: {filepath}") raise ValueError(f"Sheet {sheet_name} does not exist in the workbook: {filepath}") # Header row ws.append(self._data[0]) # Data rows rows = self._data[1:] for row in rows: ws.append(row) wb.save(filepath) logger.info(f"Data saved in Excel format: {filepath}") def write_data_xls(self, filepath: str, sheet_name=None): """ Write the data to an Excel XLS file. The filepath must be a string because xlwt does not support Path objects. """ logger.info(f"Writing data to XLS file: {filepath}") wb = xlwt.Workbook() rb = xlrd.open_workbook(filepath, formatting_info=True) try: sheet_name_index = rb.sheet_names().index(sheet_name) except ValueError: raise ValueError(f"Sheet {sheet_name} does not exist in the workbook: {filepath}") # We use xlutils to transform the xlrd book into an xlwt book # This will allow us to modify existing XLS files wb = copy(rb) ws = wb.get_sheet(sheet_name_index) for row_idx, row_data in enumerate(self._data): for col_idx, cell_value in enumerate(row_data): ws.write(row_idx, col_idx, cell_value) wb.save(filepath) logger.info(f"Data saved in XLS format: {filepath}") def write_error_xlsx(self, filepath: Path): """ Write the errors to an Excel file in the specified directory painting with red the cells with errors. """ wb = Workbook() data_sheet = wb.active data_sheet.title = self.tr("Data") errors_sheet = wb.create_sheet("Errors Description") errors_sheet.append(["Row", "Column", "Error Title", "Error Description"]) blank_sheet = wb.create_sheet("Blank Rows") blank_sheet.append(["Row", "Error Description"]) red_fill = PatternFill(start_color="FF0000", end_color="FF0000", fill_type="solid") errors_cells = list() blank_rows = set() for row_index, errors_in_row in enumerate(self.errors): if errors_in_row: for error_column, error_type, error_description in errors_in_row: if error_type == "blank-row": blank_rows.add(row_index) else: errors_cells.append((row_index, error_column, error_type, error_description)) for row_index, row in enumerate(self._data): data_sheet.append(row) errors_cells.sort(key=lambda x: (x[0], x[1])) # Sort by row and column index for row_index, col_index, error_type, error_description in errors_cells: excel_row = row_index + 1 excel_col = col_index + 1 error_title = utils.ErrorTexts.get_error_title(error_type) if not error_title: error_title = error_type.replace("-", " ").title() errors_sheet.append( [ excel_row, excel_col, error_title, utils.ErrorTexts.get_error_description(error_type) or error_description, ] ) # Paint the cell with red if it has an error in the Data Sheet data_sheet.cell(row=excel_row, column=excel_col).fill = red_fill for row_index in blank_rows: excel_row = row_index + 1 blank_sheet.append( [ excel_row, utils.ErrorTexts.get_error_description("blank-row"), ] ) # Paint the cell with red if it has an error in the Data Sheet for col_index in range(1, data_sheet.max_column + 1): data_sheet.cell(row=excel_row, column=col_index).fill = red_fill wb.save(filepath) self.finished.emit() logger.info(f"Errors saved in Excel format: {filepath}") def _get_errors(self, errors): """Return an array with errors information to use when rendering the table. The array has same lenght as our data with None or a Tuple: [None, None, (3, 'blank-label', 'error mesage to be displayed'), None] Main actions: - Moves from Frictionless' 1-index to 0-index - Handles inconsistency in Fricionless Error Object API - Builds an array for easy access to error information to render performant tables. """ result = [None] * self._row_count for error in errors: if error.type == "source-error": # SourceError happens with files that cannot be read and do not have row_number nor field_number. return result elif error.type in ["blank-label", "duplicate-label", "incorrect-label", "missing-label", "extra-label"]: # https://github.com/frictionlessdata/frictionless-py/issues/1710 row = error.row_numbers[0] - 1 column = error.field_number - 1 elif error.type == "blank-row": row = error.row_number - 1 # BlankRow error does not have field_number column = 0 elif not hasattr(error, "row_number"): row = None column = None else: row = error.row_number - 1 column = error.field_number - 1 if row is not None: if result[row] is None: result[row] = list() result[row].append((column, error.type, error.message)) return result def _get_row_count(self): return len(self._data) def _get_column_count(self): """Get the amout of columns. We are expecting malformed CSVs, so the amout of columns should always be the size of the longest row. """ try: return max(map(len, self._data)) except ValueError: return 0 def get_header_data(self): """Returns the first row of the file.""" return self._data[0] def rowCount(self, parent=None): """Returning from a pre-calculated private attribute for performance improvements.""" return self._row_count def columnCount(self, parent=None): """Returning from a pre-calculated private attribute for performance improvements.""" return self._column_count def data(self, index, role): """Returns information to be used to render the Data Table View. For each cell we return: - The value to be displayed in the cell - If there is an error in that cell, a red color for the Background. - If there is an error in that cell, a message for the tooltip. """ if not index.isValid(): return None if role == Qt.ItemDataRole.DisplayRole or role == Qt.ItemDataRole.EditRole: try: # We convert the data as string to avoid PySide6 treating numbers differently value = self._data[index.row()][index.column()] if value is None: return "" return str(value) except IndexError: # Our data could be irregular (missing columns and rows) # So it is okay to return None and keep iterating. return None if not self.errors[index.row()]: return None if role == Qt.ItemDataRole.BackgroundRole: for error_column, error_type, _ in self.errors[index.row()]: if error_type == "blank-row": # BlankRowError does not have field_number, we paint all the cells. return COLOR_RED if error_column == index.column(): return COLOR_RED elif role == Qt.ItemDataRole.ForegroundRole: for error_column, _, _ in self.errors[index.row()]: if error_column == index.column(): return QColor(255, 255, 255) def flags(self, index): """Enable edition mode""" if not index.isValid(): return Qt.ItemFlag.NoItemFlags if index.row() == 0: return super().flags(index) return super().flags(index) | Qt.ItemFlag.ItemIsEditable def setData(self, index, value, role): """Insert the edited value at the specific cell. Since the TableView will always have enough rows and enough columns, when editing the raw data we encounter two scenarios: a) The cell in the raw data exist and therefore we just replace b) The cell in the raw data do not exist and therefore we need to create empty cells until we get to the exact column we want to insert. """ if role == Qt.ItemDataRole.EditRole: currentRow = self._data[index.row()] try: # a) raw cell exist currentRow[index.column()] = value except IndexError: # b) we create empty cells and then insert at specific column currentRow += [None] * (index.column() - len(currentRow)) currentRow.insert(index.column(), value) self.dataChanged.emit(index, index) return True return False class CustomTableView(QTableView): """Custom QTableView to handle specific key events.""" on_click_first_row = Signal(object) def __init__(self, parent=None): super().__init__(parent) def keyPressEvent(self, event: QKeyEvent): """ We check if the user pressed Enter or Return on the first row of the table. If so, we emit a signal with the column index of the clicked cell. """ index = self.currentIndex() if event.key() in (Qt.Key.Key_Return, Qt.Key.Key_Enter) and index.row() == 0 and index.isValid(): self.on_click_first_row.emit(index.column()) return super().keyPressEvent(event) def mouseMoveEvent(self, event): """Changes the cursor to Pointing Hand if positing is in first row.""" index = self.indexAt(event.pos()) if index.row() == 0: self.setCursor(Qt.CursorShape.PointingHandCursor) else: self.unsetCursor() return super().mouseMoveEvent(event) class DataViewer(QWidget): """Widget to display the content of tabular data.""" # Signal to notify that the metadata has been saved on_save = Signal(object) def __init__(self): super().__init__() utils.set_common_style(self) layout = QVBoxLayout() self.setLayout(layout) self.label = QLabel() self.label.setAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignTop) self.table_view = CustomTableView() self.table_view.on_click_first_row.connect(self.show_column_metadata_dialog) # TableView's corner button hangs the application when working with huge datasets so we disable it. self.table_view.setCornerButtonEnabled(False) self.table_view.setTabKeyNavigation(False) self.table_view.hide() self.delegate = ColumnMetadataIconDelegate(Paths.asset("icons/three-lines.png")) self.delegate.icon_clicked.connect(self.show_column_metadata_dialog) layout.addWidget(self.label) layout.addWidget(self.table_view) self.sheet_name = None self.retranslateUI() def display_data(self, model, filepath, sheet_name=None): """Set the model of the QTableView When a tabular file is selected, the main application will create a FrictionlessTableModel and call this function using the model as a parametner. """ self.table_view.setModel(model) self.table_view.setItemDelegate(self.delegate) self.table_view.horizontalHeader().setDefaultSectionSize(120) self.table_view.setMouseTracking(True) self.sheet_name = sheet_name self.metadata = File(filepath, sheet_name).get_or_create_metadata(sheet_name) self.resource = self.metadata.get("resource") self.label.hide() self.table_view.show() def show_column_metadata_dialog(self, field_index): """ Shows a dialog to edit a column's metadata. """ model = self.table_view.model() column_count = model.columnCount() headers = [] for column in range(column_count): index = model.index(0, column) value = model.data(index, Qt.ItemDataRole.DisplayRole) headers.append(value) field = self.resource.schema.fields[field_index] column_metadata_field = ColumnMetadataField( headers[field_index], field.type, field.description, field.constraints ) dialog = ColumnMetadataDialog(self, column_metadata_field, field_index, headers) dialog.save_clicked.connect(self.save_metadata_to_descriptor_file) dialog.exec() def clear(self, model): """Reset the view to the default state. This view depends of the main application self.table_model attribute. This method should always receive an empty model """ self.table_view.setModel(model) self.label.show() self.table_view.hide() def retranslateUI(self): """Apply translations to class elements.""" self.label.setText(self.tr("Preview not available for this item.")) def save_metadata_to_descriptor_file(self, field_form: dict): """Save the metadata to the descriptor file.""" field_index = field_form.get("index") field = self.resource.schema.fields[field_index] field.name = field_form.get("name") field.title = field_form.get("name") field.description = field_form.get("description", "") field.constraints = { "required": field_form.get("constraints").get("required"), } type = field_form.get("type") if type == "string": field.constraints["minLength"] = field_form.get("constraints").get("minLength") field.constraints["maxLength"] = field_form.get("constraints").get("maxLength") else: # If the type is not Text, we remove the minLength and maxLength constraints field.constraints.pop("minLength", None) field.constraints.pop("maxLength", None) # Update the field in the schema self.resource.schema.set_field(field) # Field.type cannot be updated directly, we need to use set_field_type # it needs to be after the set_field to avoid being overridden self.resource.schema.set_field_type(field.name, type) self.metadata["resource"] = self.resource.to_descriptor() file = File(self.resource.path, self.sheet_name) # We remove the dialect from the metatada, because Frictionless will be stuck # on this sheet otherwise self.metadata["resource"].pop("dialect", None) with open(file.metadata_path, "w") as f: print(f"Saving metadata {file.metadata_path}") json.dump(self.metadata, f) # Check if we name was changed, if so we need to update the header model = self.table_view.model() index = model.index(0, field_index) original_name = model.data(index, Qt.ItemDataRole.DisplayRole) table_view_changed = False if original_name != field.name: model.setData(index, field.name, Qt.ItemDataRole.EditRole) table_view_changed = True self.on_save.emit(table_view_changed) ================================================ FILE: src/ode/panels/errors.py ================================================ import collections from PySide6.QtCore import Qt, QSortFilterProxyModel from PySide6.QtWidgets import QWidget, QLabel, QHBoxLayout, QVBoxLayout, QTableView from PySide6.QtGui import QFont from ode import utils from ode.panels.data import DEFAULT_LIMIT_ERRORS from ode.shared import COLOR_RED class ErrorFilterProxyModel(QSortFilterProxyModel): """Proxy model to display only the rows of the given error type. As recommended by Qt, ODE reuses the same FrictionlessTableModel for all the TableViews. For ErrorReports we filter and show only the rows containing the specific error_type we want to display. """ def __init__(self, error_type): super().__init__() self.error_type = error_type def filterAcceptsRow(self, source_row, source_parent): """Accept rows that contains an error. The source_model is a FrictionlessTableModel and its errors attribute contains a list of tuples: [..., (row_number, error_type, error_message), ...] """ source_model = self.sourceModel() if source_model.errors[source_row] is None or len(source_model.errors[source_row]) == 0: return False for error in source_model.errors[source_row]: if error[1] == self.error_type: return True return False def data(self, index, role): """Overrides the data method to set the background color of the cells according the error type.""" if not index.isValid(): return None # Converts the index to the source model so we can map it with the errors list source_index = self.mapToSource(index) source_row = source_index.row() source_column = source_index.column() if role == Qt.ItemDataRole.BackgroundRole: source_model = self.sourceModel() if source_model.errors[source_row] is None or len(source_model.errors[source_row]) == 0: # Default color return None for error in source_model.errors[source_row]: if self.error_type == "blank-row": # BlankRowError does not have field_number, we paint all the cells. return COLOR_RED elif error[0] == source_column and error[1] == self.error_type: return COLOR_RED # Default color return None return super().data(index, role) class ErrorReport(QWidget): """Widget to show a single-type Error report. This widget will be use in the Errors view for every type of error that frictionless validate finds. It display the title, description and table preview for a specific type of error. The error argument is a list of Frictionless errors object of the same type of error. """ def __init__(self, errors, model, *args, **kwargs): super().__init__(*args, **kwargs) self.errors = errors utils.set_common_style(self) # Title of each error report: Label + count title = QWidget() title_layout = QHBoxLayout() title_layout.setContentsMargins(0, 0, 0, 0) self.title_label = QLabel(objectName="title_label") self.count_label = QLabel(objectName="count_label") title_layout.addWidget(self.title_label) title_layout.addWidget(self.count_label) title_layout.addStretch() title.setLayout(title_layout) # Description of the error self.description = QLabel() font = self.description.font() font.setPointSize(12) self.description.setFont(font) self.description.setWordWrap(True) # Previsualization table self.proxy_model = ErrorFilterProxyModel(self.errors[0].type) self.proxy_model.setSourceModel(model) self.table = QTableView() self.table.setModel(self.proxy_model) vbox = QVBoxLayout() vbox.addWidget(title) vbox.addWidget(self.description) vbox.addWidget(self.table) self.setLayout(vbox) self.setStyleSheet( """ QLabel#title_label { font-weight: bold; } QLabel#count_label { background: #D32F2F; color: #FFF; padding: 2px 2px; border-style: outset; border-width: 1px; border-radius: 4px; border-color: #D32F2F; } """ ) self.retranslateUI() def retranslateUI(self): error_title = utils.ErrorTexts.get_error_title(self.errors[0].type) or self.errors[0].title error_count = str(len(self.errors)) errors_description = utils.ErrorTexts.get_error_description(self.errors[0].type) or self.errors[0].description self.title_label.setText(error_title) self.count_label.setText(error_count) self.description.setText(errors_description) class ErrorsWidget(QWidget): """Widget to dynamically show errors reports.""" def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) layout = QVBoxLayout() self.max_errors_label = QLabel() font = QFont() font.setItalic(True) self.max_errors_label.setFont(font) self.max_errors_label.setStyleSheet("font-size: 17px") self.max_errors_label.setAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignTop) self.reports = QWidget() self.reports_layout = QVBoxLayout() self.reports_layout.setContentsMargins(0, 0, 0, 0) self.reports.setLayout(self.reports_layout) layout.addWidget(self.max_errors_label) layout.addWidget(self.reports) self.setLayout(layout) def display_errors(self, errors, model): """Builds and display the entire error report. This method should be called when reading and validating a tabular file. It is currently triggered when the user clicks on a file in the FileTreeNavigator """ self.clear() if not errors: return errors_list = self._sort_frictionless_errors(errors) total_errors = 0 for error in errors_list: errorReport = ErrorReport(error, model) self.reports_layout.addWidget(errorReport) total_errors += len(error) self.reports.show() if total_errors >= DEFAULT_LIMIT_ERRORS: self.max_errors_label.show() else: self.max_errors_label.hide() def clear(self): """Removes all the ErrorReports that have been added to this widget.""" while self.reports_layout.count(): errorReport = self.reports_layout.takeAt(0) errorReport.widget().deleteLater() self.reports.hide() def _sort_frictionless_errors(self, errors): """Splits a list of dictionaries into several lists grouped by type. Frictionless returns an array of Error objects, since we want to create an ErrorReport for each type of error, we rearrange the array into a list of arrays in which each one contains only one error type. """ result = collections.defaultdict(list) for error in errors: result[error.type].append(error) return list(result.values()) def retranslateUI(self): self.max_errors_label.setText( self.tr("Please note that the ODE currently detects errors in tables, with a maximum of ") + str(DEFAULT_LIMIT_ERRORS) ) for i in range(self.reports_layout.count()): self.reports_layout.itemAt(i).widget().retranslateUI() ================================================ FILE: src/ode/panels/source.py ================================================ import sys from PySide6.QtWidgets import QApplication, QWidget, QVBoxLayout, QPlainTextEdit, QLabel from PySide6.QtGui import QFont from PySide6.QtCore import Qt, QFileInfo from ode import utils class SourceViewer(QWidget): """Widget to display files as they are (raw). This class needs to properly detect the enconding of a file. For now UTF-8 and ISO-8859-1 will cover most of our scenarios but we will fail to display source of minority languages. """ def __init__(self): super().__init__() utils.set_common_style(self) layout = QVBoxLayout() self.setLayout(layout) self.label = QLabel() self.label.setAlignment(Qt.AlignmentFlag.AlignLeft | Qt.AlignmentFlag.AlignTop) self.text_edit = QPlainTextEdit() self.text_edit.setLineWrapMode(QPlainTextEdit.LineWrapMode.NoWrap) self.text_edit.setReadOnly(True) self.text_edit.setFont(QFont("Courier, monospace")) self.text_edit.hide() layout.addWidget(self.label) layout.addWidget(self.text_edit) def _read_file(self, filepath): """Function to read the file from disk and return a string. We could implement a more fancy approach with chardet to detect the file encoding but early tests where not performat enough. We are brute-forcing the two most popular encodings in the web. """ content = "" try: with open(filepath, "r", encoding="utf-8") as file: content = file.read() return content except Exception as e: content = f"Error while reading the file with encoding UTF-8: {e}" try: with open(filepath, "r", encoding="iso-8859-1") as file: content = file.read() return content except Exception as e: content += f"\nError while reading the file with encoding ISO-8859-1: {e}" return content def open_file(self, filepath): """Reads the file and sets the QPlainText.""" info = QFileInfo(filepath) if not info.isFile() or info.suffix().lower() not in ["csv"]: self.label.show() self.text_edit.hide() return content = self._read_file(filepath) self.label.hide() self.text_edit.show() self.text_edit.setPlainText(content) def clear(self): """Shows an empty view.""" self.label.show() self.text_edit.hide() def retranslateUI(self): self.label.setText(self.tr("This view is only available for CSV files.")) if __name__ == "__main__": app = QApplication(sys.argv) viewer = SourceViewer() viewer.show() sys.exit(app.exec()) ================================================ FILE: src/ode/paths.py ================================================ import os from pathlib import Path # This is the Project path where all files are stored and # it will be hardcoded to the home folder of the user until # we define how to properly handle projects in Open Data Editor. PROJECT_PATH = Path.home() / ".opendataeditor/tmp" METADATA_PATH = PROJECT_PATH / ".metadata" LOGS_PATH = PROJECT_PATH / ".logs" AI_MODELS_PATH = PROJECT_PATH / ".ai_models" class Paths: """Utility class to handle relative paths.""" base = os.path.dirname(__file__) assets = os.path.join(base, "assets") @classmethod def asset(cls, filename): return os.path.join(cls.assets, filename) @classmethod def translation(cls, filename): return os.path.join(cls.assets, "translations", filename) @classmethod def get_unique_destination_filepath(cls, src_filepath) -> Path: """Returns a unique destination_filepath by appending a number if the file already exists. If the specified file already exists, the method will generate a new filename by appending a number in parentheses to the original name. For example: - For a file named 'myfile.csv', if it doesn't exist, it will return 'myfile.csv'. - If 'myfile.csv' already exists, it will return 'myfile(1).csv'. - If 'myfile(1).csv' also exists, it will return 'myfile(2).csv', and so on. """ src_filepath = Path(src_filepath) if isinstance(src_filepath, str) else src_filepath destination_filepath = PROJECT_PATH / src_filepath.name # If already exists we increment to `filename (n) until we find one not taking counter = 1 while destination_filepath.exists(): destination_filepath = destination_filepath.with_stem(f"{src_filepath.stem}({counter})") counter += 1 return destination_filepath ================================================ FILE: src/ode/shared.py ================================================ from PySide6.QtGui import QColor COLOR_RED = QColor("#D32F2F") COLOR_BLUE = QColor("#0288D1") ================================================ FILE: src/ode/utils.py ================================================ import json import platform import subprocess from pathlib import Path from frictionless.resources import TableResource from frictionless import system from ode import paths from PySide6.QtWidgets import QApplication, QMessageBox def setup_ode_internal_folders(): """Creates the folders to store the files and metadata files.""" paths.METADATA_PATH.mkdir(parents=True, exist_ok=True) if platform.system() == "Windows": # Set the .metadata folder hidden so it is not shown in the ODE file navigator # This is the default behaviour in Linux/MacOs since the directory name starts with a dot. subprocess.run(["attrib", "+H", f"{str(paths.METADATA_PATH)}"], check=True) subprocess.run(["attrib", "+H", f"{str(paths.AI_MODELS_PATH)}"], check=True) subprocess.run(["attrib", "+H", f"{str(paths.LOGS_PATH)}"], check=True) def migrate_metadata_store(): """Migrates all the metadata information to separated files. Previously ODE stored the Frictionless metadata and other info of every file in a single "records" dictionary stored in a metadata.json. This created several issues like: custom logic to deduplicate file names, a third database to link files with records, etc. We will now store metadata on independent files under a `.metadata` folder. Each file will have the same name of the original file with a `metadata.json` append. We will also mimic the folder structure. """ # Path to ODE v1.3 metadata.json file metadata_file_path = paths.PROJECT_PATH / ".opendataeditor/metadata.json" if not metadata_file_path.exists(): # ODE has never been used in this machine. Nothing to migrate. return new_metadata_dir = paths.PROJECT_PATH / ".metadata/" if new_metadata_dir.exists(): # If folder exist we asume migrated and return. return ode_dir = paths.PROJECT_PATH / ".opendataeditor/" if ode_dir.exists() and platform.system() == "Windows": # Hid .opendataeditor directory. This directory is no longer used. subprocess.run(["attrib", "+H", f"{str(ode_dir)}"], check=True) # ODE v1.3 has been used and we need to migrate. with open(metadata_file_path, "r") as file: try: metadata = json.load(file) except Exception as e: # We are receiving json.decoder.JSONDecodeError error reports from users. # So we skip the migration if the file cannot be read. print(f"Cannot read ode v1.3 metadata file. Skipping migration: {e}") return records = metadata["record"] for _, record_data in records.items(): path = record_data["path"] try: # Infer Frictionless Statistics, it is mandatory for the newest version of ODE. with system.use_context(trusted=True): resource = TableResource(record_data.get("resource")) # Patch the original resource path with the absolute path to the file. resource.path = str(paths.PROJECT_PATH / path) resource.infer() record_data["resource"] = resource.to_descriptor() except Exception as e: # This should happen only if the user did file/metadata editing outside ODE. print(f"Error when creating TableResource: {e}") continue # Set metadata file name as .json # Example: my-file.csv -> my-file.json # Example: subfolder/my-file.csv -> subfolder/my-file.json filename = path.rsplit(".", 1)[0] metadata_filename = str(new_metadata_dir / filename) + ".json" # Ensure the directory structure exists # Example: if we are migrating a file located in a subfolder, we need to create # it before the json.dump file. Path(metadata_filename).parent.mkdir(parents=True, exist_ok=True) with open(metadata_filename, "w") as json_file: json.dump(record_data, json_file, indent=4) print("Migration completed successfully!") def set_common_style(widget): widget.setStyleSheet("font-size: 17px;") def show_error_dialog(message=None, title="Error"): if message is None: message = "An unexpected error occurred in the application." error_box = QMessageBox() error_box.setIcon(QMessageBox.Icon.Critical) error_box.setWindowTitle(title) error_box.setText("An error has occurred") error_box.setInformativeText(message) error_box.setStandardButtons(QMessageBox.StandardButton.Ok) return error_box.exec() class ErrorTexts: @classmethod def get_error_title(cls, error_type): """Returns a more user-friendly title if exists.""" ERROR_TITLES = { "missing-label": QApplication.translate("ErrorsMessages", "Missing header"), "duplicate-label": QApplication.translate("ErrorsMessages", "Duplicated header"), "blank-row": QApplication.translate("ErrorsMessages", "Empty row"), "type-error": QApplication.translate("ErrorsMessages", "Type mismatch"), "missing-cell": QApplication.translate("ErrorsMessages", "Missing value"), "extra-cell": QApplication.translate("ErrorsMessages", "Extra cell"), "blank-header": QApplication.translate("ErrorsMessages", "Missing header"), "blank-label": QApplication.translate("ErrorsMessages", "Blank Label"), } return ERROR_TITLES.get(error_type, None) @classmethod def get_error_description(cls, error_type): """Returns a more user-friendly description if exists.""" ERROR_DESCRIPTIONS = { "missing-label": QApplication.translate("ErrorsMessages", "A column in the header row has no name. Every column should have a unique, non-empty header."), "duplicate-label": QApplication.translate("ErrorsMessages", "Two or more columns share the same name. Column names must be unique."), "blank-row": QApplication.translate("ErrorsMessages", "This row has no data. Rows should contain at least one cell with data."), "type-error": QApplication.translate("ErrorsMessages", "A cell value doesn't match the expected data type or format for the column."), "missing-cell": QApplication.translate("ErrorsMessages", "This cell is missing data"), "extra-cell": QApplication.translate("ErrorsMessages", "This row has more values compared to the header row."), "blank-header": QApplication.translate("ErrorsMessages", "A column in the header row has no name. Every column should have a unique, non-empty header."), "blank-label": QApplication.translate("ErrorsMessages", "A label in the header row is missing a value. Label should be provided and not be blank."), } return ERROR_DESCRIPTIONS.get(error_type, None) ================================================ FILE: tests/__init__.py ================================================ ================================================ FILE: tests/conftest.py ================================================ import pytest from ode import paths, main @pytest.fixture(autouse=True) def project_folder(tmp_path): # Patch the PROJECT_PATH to use temporary directory paths.PROJECT_PATH = tmp_path paths.METADATA_PATH = tmp_path / ".metadata" return tmp_path @pytest.fixture(autouse=True) def window(qtbot, project_folder): win = main.MainWindow() qtbot.addWidget(win) win.show() return win ================================================ FILE: tests/ode/__init__.py ================================================ ================================================ FILE: tests/ode/test_application.py ================================================ from PySide6.QtCore import Qt from ode.shared import COLOR_RED def test_file_is_displayed(qtbot, window, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") # Assert Welcome screen is selected by default. assert window.main_layout.currentIndex() == 0 assert window.content.data_view.isVisible() is False # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) qtbot.waitUntil(lambda: window.main_layout.currentIndex() == 1) assert window.main_layout.currentIndex() == 1 assert window.content.data_view.isVisible() # Test our TableView model has 3 rows and 2 columns (data was loaded properly) assert window.content.data_view.table_view.model().rowCount() == 3 assert window.content.data_view.table_view.model().columnCount() == 2 def test_button_errors_displays_error_count(qtbot, window, project_folder): p1 = project_folder / "missing-header.csv" p1.write_text("name,\nAlice,30\nBob,25") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") error_report = window.content.errors_view.reports_layout.takeAt(0) error_text = error_report.widget().description.text() # Test Frictionless error assert "Label should be provided and not be blank." in str(error_text) def test_error_reports_show_two_blank_lines_in_red(qtbot, window, project_folder): """Test that two blank lines are shown in red in the error report.""" p1 = project_folder / "two-blank-lines.csv" p1.write_text("name,surname\n,\n,") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "2") # Get error report error_report = window.content.errors_view.reports_layout.takeAt(0) proxy = error_report.widget().proxy_model total_rows = proxy.rowCount() red_rows = 0 red_background = COLOR_RED.name() for row in range(total_rows): # Get the proxy index for this row index = proxy.index(row, 0) # Get background color background_color = proxy.data(index, Qt.BackgroundRole) # If the color is red, increment counter if background_color is not None and background_color.name() == red_background: red_rows += 1 assert red_rows == 2 def test_error_reports_show_two_errors_in_same_row(qtbot, window, project_folder): """ This test is for the case where there are two errors in the same row, and both errors are shown in red. Error 1: Duplicated header Error 2: Empty header """ p1 = project_folder / "header-errors.csv" p1.write_text("name,name,,empty\nname,surname,empty,empty") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "2") red_background = COLOR_RED.name() # Check that we have two error report tables assert window.content.errors_view.reports_layout.count() == 2 # Check we have on error in each report # Error 1: Duplicated header proxy_model = window.content.errors_view.reports_layout.itemAt(0).widget().proxy_model assert proxy_model.error_type == "duplicate-label" # Check the exact column index = proxy_model.index(0, 1) background_color = proxy_model.data(index, Qt.BackgroundRole) assert background_color.name() == red_background # Error 2: Empty header proxy_model = window.content.errors_view.reports_layout.itemAt(1).widget().proxy_model assert proxy_model.error_type == "blank-label" # Check the exact column index = proxy_model.index(0, 2) background_color = proxy_model.data(index, Qt.BackgroundRole) assert background_color.name() == red_background ================================================ FILE: tests/ode/test_files.py ================================================ import json import pytest from ode.file import File class TestFiles: def test_constructor(self, project_folder): p1 = project_folder / "example.csv" file = File(p1) assert file.path == p1 assert file.metadata_path == (project_folder / ".metadata/example.json") def test_path_to_metadata_file(self, project_folder): p1 = project_folder / "example.csv" m1 = project_folder / ".metadata/example.json" assert File(p1).metadata_path == m1 def test_path_to_metadata_subfolder(self, project_folder): p2 = project_folder / "subfolder/example-1.csv" m2 = project_folder / ".metadata/subfolder/example-1.json" assert File(p2).metadata_path == m2 def test_path_to_metadata_folder(self, project_folder): p3 = project_folder / "subfolder/" p3.mkdir() m3 = project_folder / ".metadata/subfolder/" assert File(p3).metadata_path == m3 def test_get_create_metadata(self, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") file = File(p1) metadata = file.get_or_create_metadata() assert file.metadata_path.exists() assert metadata["resource"] assert metadata["resource"].path == str(p1) def test_get_metadata_dict(self, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") file = File(p1) file.get_or_create_metadata() metadata = file.get_metadata_dict(file.metadata_path) assert metadata assert isinstance(metadata, dict) assert metadata["resource"]["path"] == str(p1) assert (project_folder / ".metadata/example.json").exists() def test_rename_file(self, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") file = File(p1) file.rename("bar") assert not p1.exists() assert (project_folder / "bar.csv").exists() def test_rename_folder(self, project_folder): m1 = project_folder / "subfolder" m1.mkdir() p1 = project_folder / "subfolder/example.csv" p1.write_text("name,age\nAlice,30\nBob,25") ode_folder = File(m1) ode_folder.rename("bar") assert not m1.exists() assert (project_folder / "bar").exists() assert (project_folder / "bar/example.csv").exists() def test_rename_raises_error_if_target_exist(self, project_folder): p1 = project_folder / "foo.csv" p1.write_text("foo") p2 = project_folder / "bar.csv" p2.write_text("bar") file = File(p1) with pytest.raises(OSError): file.rename("bar") def test_rename_also_updates_object_attributes(self, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") file = File(p1) file.get_or_create_metadata() file.rename("bar") assert file.path == (project_folder / "bar.csv") assert str(file.metadata_path) == str(project_folder / ".metadata/bar.json") def test_rename_file_metadata(self, project_folder): p1 = project_folder / "example.csv" p1.write_text("name,age\nAlice,30\nBob,25") file = File(p1) file.get_or_create_metadata() file.rename("bar") metadata = file.get_metadata_dict(file.metadata_path) expected = project_folder / "bar.csv" assert metadata["resource"]["path"] == str(expected) def test_rename_folder_metadata(self, project_folder): """Test that renaming folders updates all metadata files of children files.""" m1 = project_folder / "subfolder" m1.mkdir() p1 = project_folder / "subfolder/foo.csv" p1.write_text("name,age\nAlice,30\nBob,25") File(p1).get_or_create_metadata() p2 = project_folder / "subfolder/bar.csv" p2.write_text("name,age\nAlice,30\nBob,25") File(p2).get_or_create_metadata() file = File(m1) file.rename("new_name") assert (project_folder / ".metadata/new_name/foo.json").exists() assert (project_folder / ".metadata/new_name/bar.json").exists() metadata_path = project_folder / ".metadata/new_name/foo.json" with open(metadata_path) as f: metadata = json.load(f) assert metadata["resource"]["path"] == str(project_folder / "new_name/foo.csv") metadata_path = project_folder / ".metadata/new_name/bar.json" with open(metadata_path) as f: metadata = json.load(f) assert metadata["resource"]["path"] == str(project_folder / "new_name/bar.csv") def test_remove_file_and_metadata(self, project_folder): p1 = project_folder / "example.csv" p1.touch() file = File(p1) file.get_or_create_metadata() file.remove() assert file.path.exists() is False assert file.metadata_path.exists() is False def test_delete_folder_and_metadata(self, project_folder): m1 = project_folder / "subfolder" m1.mkdir() p1 = project_folder / "subfolder/foo.csv" p1.touch() f1 = File(p1) f1.get_or_create_metadata() p2 = project_folder / "subfolder/bar.csv" p2.touch() f2 = File(p2) f2.get_or_create_metadata() p3 = project_folder / "subfolder/zoo.csv" p3.touch() f3 = File(p3) # f3 should not have a metadata fail and it should not fail when deleting. file = File(m1) file.remove() assert p1.exists() is False assert f1.metadata_path.exists() is False assert p2.exists() is False assert f2.metadata_path.exists() is False assert p3.exists() is False assert f3.metadata_path.exists() is False ================================================ FILE: tests/ode/test_frictionless_errors.py ================================================ from PySide6.QtCore import Qt from PySide6.QtWidgets import QDialog from ode.dialogs.metadata import ColumnMetadataDialog from ode.panels.data import ColumnMetadataField from ode.shared import COLOR_RED class TestFrictionlessErrors: """Test that FrictionlessTableModel returns correct background for errors. Our main QTableView calls the data() endpoint of our table_model with Qt.ItemDataRole.BackgroundRole to request wich color should the cell be painted. We also assume that if the value returned is QColor("red") then the table will be displayed properly. """ def test_blank_header_error(self, qtbot, window, project_folder): p1 = project_folder / "blank-header.csv" p1.write_text("name,\nAlice,30\nBob,25") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain only 1 error. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") # Test FrictionlessModel returns a Red Background for the error cell. index = window.table_model.index(0, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED # Test FrictionlessModel do not return a Red Background for other cells. index = window.table_model.index(1, 0) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background != COLOR_RED def test_duplicate_label_error(self, qtbot, window, project_folder): p1 = project_folder / "duplicate-label.csv" p1.write_text("name,name\nAlice,30\nBob,25") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain only 1 error. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") # Test FrictionlessModel returns a Red Background for the error cell. index = window.table_model.index(0, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED # Test FrictionlessModel do not return a Red Background for first header. index = window.table_model.index(0, 0) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background != COLOR_RED def test_blank_row_error(self, qtbot, window, project_folder): p1 = project_folder / "blank-row.csv" p1.write_text("name,age,city\nAlice,30,Barcelona\n,,\nBob,25,Valencia") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain only 1 error. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") # Blank Row error should paint the whole row in red. index = window.table_model.index(2, 0) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED index = window.table_model.index(2, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED index = window.table_model.index(2, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED def test_blank_row_and_duplicated_label_error(self, qtbot, window, project_folder): p1 = project_folder / "blank-row-and-duplicated-label.csv" p1.write_text("name,name\nAlice,30\n,\nBob,25") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain only 2 errors: blank row and duplicated label. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "2") # Blank Row error should paint the whole row in red. index = window.table_model.index(2, 0) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED index = window.table_model.index(2, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED # Duplicated Label should paint the cell index = window.table_model.index(0, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED def test_missing_cell_error(self, qtbot, window, project_folder): p1 = project_folder / "missing-cell.csv" p1.write_text("name,age,city\nAlice,30\nBob,25\nTom,15") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain 3 errors. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "3") # Missing Cell should paint the third column (except the header). index = window.table_model.index(0, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background != COLOR_RED index = window.table_model.index(1, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED index = window.table_model.index(2, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED index = window.table_model.index(3, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED def test_extra_cell_error(self, qtbot, window, project_folder): p1 = project_folder / "extra-cell.csv" p1.write_text("name,age\nAlice,30\nBob,25,extra\nTom,15") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain 1 errors. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") # Extra Cell should paint the extra cell. index = window.table_model.index(2, 2) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background == COLOR_RED # The value of the extra cell should be "extra" value = window.table_model.data(index, Qt.ItemDataRole.DisplayRole) assert value == "extra" # Other cells should not be red index = window.table_model.index(2, 1) background = window.table_model.data(index, Qt.ItemDataRole.BackgroundRole) assert background != COLOR_RED def test_custom_errors_descriptions_are_shown(self, qtbot, window, project_folder): """Test that the application is using our custom error descriptions.""" p1 = project_folder / "blank-row.csv" p1.write_text("name,age,city\nAlice,30,Barcelona\n,,\nBob,25,Valencia") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain 1 errors. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") window.content.toolbar.button_errors.click() # If we want to see the change in the table, we need to update the window window.update() qtbot.wait(100) assert window.content.errors_view.reports_layout.count() == 1 error_report = window.content.errors_view.reports_layout.itemAt(0).widget() assert ( error_report.description.text() == "This row has no data. Rows should contain at least one cell with data." ) assert error_report.title_label.text() == "Empty row" def test_default_frictionless_errors_if_missing_custom(self, qtbot, window, project_folder): """Test that the application fallbacks to Frictionless errors if we do not provide a custom description.""" p1 = project_folder / "blank-label.csv" p1.write_text("name,,city\nAlice,30,Barcelona\nBob,25,Valencia") # Simulate click event index = window.sidebar.file_model.index(str(p1)) window.on_tree_click(index) # This file should contain 1 errors. qtbot.waitUntil(lambda: window.content.toolbar.button_errors.error_label.text() == "1") window.content.toolbar.button_errors.click() # If we want to see the change in the table, we need to update the window window.update() qtbot.wait(100) assert window.content.errors_view.reports_layout.count() == 1 error_report = window.content.errors_view.reports_layout.itemAt(0).widget() # The following are Frictionless default texts: https://framework.frictionlessdata.io/docs/errors/label.html#blank-label assert ( error_report.description.text() == "A label in the header row is missing a value. Label should be provided and not be blank." ) assert error_report.title_label.text() == "Blank Label" def test_changing_column_header_name_fixes_error_with_dialog(self, qtbot, window, project_folder): """Test that changing the header name of a column with a dialog fixes the blank header error.""" # The file should contain a blank header error p0 = project_folder / "temp.csv" p0.write_text("name,\nAlice,A\nBob,B") # Choose the file index = window.sidebar.file_model.index(str(p0)) window.on_tree_click(index) # Check that the file is loaded and has an error qtbot.wait(100) assert window.content.errors_view.reports_layout.count() == 1 # Create and show the dialog blank_field = ColumnMetadataField("", "string", "", {"required": False, "minLength": 0, "maxLength": 100}) field_names = ["name", ""] dialog = ColumnMetadataDialog( parent=window, field=blank_field, field_index=1, field_names=field_names # Index of the blank column ) # Use qtbot to interact with the dialog qtbot.addWidget(dialog) dialog.show() # For debugging purposes # Set the new name in the dialog dialog.form.name.setText("surname") # Verify the dialog state assert dialog.form.name.text() == "surname" # Connect to the save signal to capture the emitted data saved_data = [] dialog.save_clicked.connect(lambda data: saved_data.append(data)) # Click the save button qtbot.mouseClick(dialog.save_button, Qt.MouseButton.LeftButton) # For debugging # qtbot.wait(100) # window.update() # Verify the dialog was accepted and data was emitted assert dialog.result() == QDialog.DialogCode.Accepted assert len(saved_data) == 1 assert saved_data[0]["name"] == "surname" assert saved_data[0]["index"] == 1 window.content.data_view.save_metadata_to_descriptor_file(saved_data[0]) # Wait for the changes to be applied qtbot.wait(1000) window.update() # Check that the error is gone assert window.content.errors_view.reports_layout.count() == 0 def test_changing_column_type_with_metadata_dialog(self, qtbot, window, project_folder): """Test changing for and back the column type with the metadata dialog show and fixes the error.""" # Create file p0 = project_folder / "temp.csv" p0.write_text("name,surname\nAlice,A\nBob,B") # Choose the file index = window.sidebar.file_model.index(str(p0)) window.on_tree_click(index) # Check that the file is loaded and has zero error qtbot.wait(100) assert window.content.errors_view.reports_layout.count() == 0 # Create and show the dialog blank_field = ColumnMetadataField( "surname", "string", "", {"required": False, "minLength": 0, "maxLength": 100} ) field_names = ["name", "surname"] # We are changing the "surname" column type to "integer" dialog = ColumnMetadataDialog(parent=window, field=blank_field, field_index=1, field_names=field_names) # Use qtbot to interact with the dialog qtbot.addWidget(dialog) dialog.show() # For debugging purposes dialog.form.type.setCurrentText("Number") # Verify the dialog state assert dialog.form.type.currentText() == "Number" # Connect to the save signal to capture the emitted data saved_data = [] dialog.save_clicked.connect(lambda data: saved_data.append(data)) # Click the save button qtbot.mouseClick(dialog.save_button, Qt.MouseButton.LeftButton) # Verify the dialog was accepted and data was emitted assert dialog.result() == QDialog.DialogCode.Accepted assert len(saved_data) == 1 assert saved_data[0]["type"] == "number" assert saved_data[0]["index"] == 1 window.content.data_view.save_metadata_to_descriptor_file(saved_data[0]) # Wait for the changes to be applied qtbot.wait(1000) window.update() # Check that we have an error now assert window.content.errors_view.reports_layout.count() == 1 # Change the column type back to "string" to fix the error dialog.form.type.setCurrentText("Text") qtbot.mouseClick(dialog.save_button, Qt.MouseButton.LeftButton) # Verify the dialog was accepted and data was emitted assert dialog.result() == QDialog.DialogCode.Accepted assert len(saved_data) == 2 assert saved_data[1]["type"] == "string" assert saved_data[1]["index"] == 1 window.content.data_view.save_metadata_to_descriptor_file(saved_data[1]) # Wait for the changes to be applied qtbot.wait(1000) window.update() # Check that we have an error now assert window.content.errors_view.reports_layout.count() == 0 ================================================ FILE: tests/ode/test_paths.py ================================================ from ode.paths import Paths class TestPaths: def test_no_conflict(self, project_folder): test_file = project_folder / "tempfile.txt" result = Paths.get_unique_destination_filepath(test_file) assert result == test_file def test_single_conflict(self, project_folder): # Create same temp file to generate a conflict (project_folder / "tempfile.txt").touch() result = Paths.get_unique_destination_filepath(project_folder / "tempfile.txt") assert result == (project_folder / "tempfile(1).txt") def test_multiple_conflicts(self, project_folder): # Create two temp files to see if increased to the next one (project_folder / "tempfile.txt").touch() (project_folder / "tempfile(1).txt").touch() result = Paths.get_unique_destination_filepath(project_folder / "tempfile.txt") assert result == (project_folder / "tempfile(2).txt")