# Connect the World, Frame by Frame
# Connect the World, Frame by Frame
| ### Dual Subtitles --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Cosy2 Voice Clone --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS with my voice --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
If you find VideoLingo helpful, please give me a ⭐️!
================================================ FILE: VideoLingo_colab.ipynb ================================================ { "cells": [ { "cell_type": "markdown", "metadata": { "id": "RkeSbYF2HoM_" }, "source": [ "# Welcome to VideoLingo! 🎉🚀\n", "#### This colab file allows you to quickly experience the full functionality in just 5 minutes! ⏱️✨ Before you begin, you may need to prepare some keys. 🔑🗝️ Please read https://videolingo.io/docs/start to get ready. 📚👍\n", "#### *Please use a T4 GPU to execute this colab for optimal performance." ] }, { "cell_type": "markdown", "metadata": { "id": "h0jE67Gc-1nO" }, "source": [ "## 1. Clone VideoLingo repo 📥" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "NC3i2T7D51oS", "outputId": "19821917-8ee4-4123-a099-fd35405fcdfe" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Cloning into 'VideoLingo'...\n", "remote: Enumerating objects: 2578, done.\u001b[K\n", "remote: Counting objects: 100% (595/595), done.\u001b[K\n", "remote: Compressing objects: 100% (221/221), done.\u001b[K\n", "remote: Total 2578 (delta 408), reused 378 (delta 374), pack-reused 1983 (from 1)\u001b[K\n", "Receiving objects: 100% (2578/2578), 10.44 MiB | 12.60 MiB/s, done.\n", "Resolving deltas: 100% (1644/1644), done.\n" ] } ], "source": [ "!git clone https://github.com/Huanshere/VideoLingo.git\n", "%cd VideoLingo" ] }, { "cell_type": "markdown", "metadata": { "id": "A5sIHzRs8JI1" }, "source": [ "## 2. Installation 🚀\n", "this takes around 4 mins" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "tSymHAEg6Vzr", "outputId": "8c5059e4-37d5-4540-cba5-9c7aa46fe226" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (13.8.1)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich) (3.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich) (2.18.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich) (0.1.2)\n", "\u001b[1;35m╭──────────────────────────╮\u001b[0m\n", "\u001b[1;35m│\u001b[0m\u001b[1;35m \u001b[0m\u001b[1;35mStarting installation...\u001b[0m\u001b[1;35m \u001b[0m\u001b[1;35m│\u001b[0m\n", "\u001b[1;35m╰──────────────────────────╯\u001b[0m\n", "config.py file has been created. Please fill in the API key and base URL in the config.py file.\n", "\u001b[36m╭──────────────────────────────────────────────────────────────────────────────────────────────────╮\u001b[0m\n", "\u001b[36m│\u001b[0m\u001b[36m \u001b[0m\u001b[36mInstalling requests...\u001b[0m\u001b[36m \u001b[0m\u001b[36m \u001b[0m\u001b[36m│\u001b[0m\n", "\u001b[36m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (2.32.3)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests) (3.10)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests) (2.2.3)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests) (2024.8.30)\n", "\u001b[3m Whisper Model Selection \u001b[0m\n", "┏━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1mOption\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mModel \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mDescription \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│\u001b[36m \u001b[0m\u001b[36m1 \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35mwhisperX 💻 \u001b[0m\u001b[35m \u001b[0m│\u001b[32m \u001b[0m\u001b[32mlocal model (can also use online model api)\u001b[0m\u001b[32m \u001b[0m│\n", "│\u001b[36m \u001b[0m\u001b[36m2 \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35mwhisperXapi ☁️\u001b[0m\u001b[35m \u001b[0m│\u001b[32m \u001b[0m\u001b[32monline model through api only \u001b[0m\u001b[32m \u001b[0m│\n", "└────────┴───────────────┴─────────────────────────────────────────────┘\n", "If you're unsure about the differences between models, please see \n", "\u001b[4;94mhttps://github.com/Huanshere/VideoLingo/\u001b[0m\n", "\u001b[36m╭──────────────────────────────────────────────────────────────────────────────────────────────────╮\u001b[0m\n", "\u001b[36m│\u001b[0m\u001b[36m \u001b[0m\u001b[36mInstalling PyTorch with CUDA support...\u001b[0m\u001b[36m \u001b[0m\u001b[36m \u001b[0m\u001b[36m│\u001b[0m\n", "\u001b[36m╰──────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n", "Looking in indexes: https://download.pytorch.org/whl/cu118\n", "Collecting torch==2.0.0\n", " Downloading https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp310-cp310-linux_x86_64.whl (2267.3 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 GB\u001b[0m \u001b[31m626.3 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting torchaudio==2.0.0\n", " Downloading https://download.pytorch.org/whl/cu118/torchaudio-2.0.0%2Bcu118-cp310-cp310-linux_x86_64.whl (4.4 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.4/4.4 MB\u001b[0m \u001b[31m84.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.0.0) (3.16.1)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==2.0.0) (4.12.2)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.0.0) (1.13.3)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.0.0) (3.3)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.0) (3.1.4)\n", "Collecting triton==2.0.0 (from torch==2.0.0)\n", " Downloading https://download.pytorch.org/whl/triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.0) (3.30.3)\n", "Collecting lit (from triton==2.0.0->torch==2.0.0)\n", " Downloading https://download.pytorch.org/whl/lit-15.0.7.tar.gz (132 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m132.3/132.3 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch==2.0.0) (2.1.5)\n", "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.0.0) (1.3.0)\n", "Building wheels for collected packages: lit\n", " Building wheel for lit (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for lit: filename=lit-15.0.7-py3-none-any.whl size=89990 sha256=f9f0ca0ce885e2ddbc5583433ea047a853b4034c8e2712e685d93fbed24f9204\n", " Stored in directory: /root/.cache/pip/wheels/27/2c/b6/3ed2983b1b44fe0dea1bb35234b09f2c22fb8ebb308679c922\n", "Successfully built lit\n", "Installing collected packages: lit, triton, torch, torchaudio\n", " Attempting uninstall: torch\n", " Found existing installation: torch 2.4.1+cu121\n", " Uninstalling torch-2.4.1+cu121:\n", " Successfully uninstalled torch-2.4.1+cu121\n", " Attempting uninstall: torchaudio\n", " Found existing installation: torchaudio 2.4.1+cu121\n", " Uninstalling torchaudio-2.4.1+cu121:\n", " Successfully uninstalled torchaudio-2.4.1+cu121\n", "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "torchvision 0.19.1+cu121 requires torch==2.4.1, but you have torch 2.0.0+cu118 which is incompatible.\u001b[0m\u001b[31m\n", "\u001b[0mSuccessfully installed lit-15.0.7 torch-2.0.0+cu118 torchaudio-2.0.0+cu118 triton-2.0.0\n", "Installing whisperX...\n", "Obtaining file:///content/VideoLingo/third_party/whisperX\n", " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: torch>=2 in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (2.0.0+cu118)\n", "Requirement already satisfied: torchaudio>=2 in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (2.0.0+cu118)\n", "Collecting faster-whisper==1.0.0 (from whisperx==3.1.1)\n", " Downloading faster_whisper-1.0.0-py3-none-any.whl.metadata (14 kB)\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (4.44.2)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (2.2.2)\n", "Requirement already satisfied: setuptools>=65 in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (71.0.4)\n", "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from whisperx==3.1.1) (3.8.1)\n", "Collecting pyannote.audio==3.1.1 (from whisperx==3.1.1)\n", " Downloading pyannote.audio-3.1.1-py2.py3-none-any.whl.metadata (9.3 kB)\n", "Collecting av==11.* (from faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading av-11.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.5 kB)\n", "Collecting ctranslate2<5,>=4.0 (from faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading ctranslate2-4.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (10 kB)\n", "Requirement already satisfied: huggingface-hub>=0.13 in /usr/local/lib/python3.10/dist-packages (from faster-whisper==1.0.0->whisperx==3.1.1) (0.24.7)\n", "Collecting tokenizers<0.16,>=0.13 (from faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)\n", "Collecting onnxruntime<2,>=1.14 (from faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading onnxruntime-1.19.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (4.5 kB)\n", "Collecting asteroid-filterbanks>=0.4 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading asteroid_filterbanks-0.4.0-py3-none-any.whl.metadata (3.3 kB)\n", "Requirement already satisfied: einops>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pyannote.audio==3.1.1->whisperx==3.1.1) (0.8.0)\n", "Collecting lightning>=2.0.1 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading lightning-2.4.0-py3-none-any.whl.metadata (38 kB)\n", "Collecting omegaconf<3.0,>=2.1 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading omegaconf-2.3.0-py3-none-any.whl.metadata (3.9 kB)\n", "Collecting pyannote.core>=5.0.0 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pyannote.core-5.0.0-py3-none-any.whl.metadata (1.4 kB)\n", "Collecting pyannote.database>=5.0.1 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pyannote.database-5.1.0-py3-none-any.whl.metadata (1.2 kB)\n", "Collecting pyannote.metrics>=3.2 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pyannote.metrics-3.2.1-py3-none-any.whl.metadata (1.3 kB)\n", "Collecting pyannote.pipeline>=3.0.1 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pyannote.pipeline-3.0.1-py3-none-any.whl.metadata (897 bytes)\n", "Collecting pytorch-metric-learning>=2.1.0 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pytorch_metric_learning-2.6.0-py3-none-any.whl.metadata (17 kB)\n", "Requirement already satisfied: rich>=12.0.0 in /usr/local/lib/python3.10/dist-packages (from pyannote.audio==3.1.1->whisperx==3.1.1) (13.8.1)\n", "Collecting semver>=3.0.0 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading semver-3.0.2-py3-none-any.whl.metadata (5.0 kB)\n", "Requirement already satisfied: soundfile>=0.12.1 in /usr/local/lib/python3.10/dist-packages (from pyannote.audio==3.1.1->whisperx==3.1.1) (0.12.1)\n", "Collecting speechbrain>=0.5.14 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading speechbrain-1.0.1-py3-none-any.whl.metadata (24 kB)\n", "Collecting tensorboardX>=2.6 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading tensorboardX-2.6.2.2-py2.py3-none-any.whl.metadata (5.8 kB)\n", "Collecting torch-audiomentations>=0.11.0 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading torch_audiomentations-0.11.1-py3-none-any.whl.metadata (14 kB)\n", "Collecting torchmetrics>=0.11.0 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading torchmetrics-1.4.2-py3-none-any.whl.metadata (19 kB)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (3.16.1)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (4.12.2)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (1.13.3)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (3.3)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (3.1.4)\n", "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2->whisperx==3.1.1) (2.0.0)\n", "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=2->whisperx==3.1.1) (3.30.3)\n", "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=2->whisperx==3.1.1) (15.0.7)\n", "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->whisperx==3.1.1) (8.1.7)\n", "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->whisperx==3.1.1) (1.4.2)\n", "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk->whisperx==3.1.1) (2024.9.11)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk->whisperx==3.1.1) (4.66.5)\n", "Requirement already satisfied: numpy>=1.22.4 in /usr/local/lib/python3.10/dist-packages (from pandas->whisperx==3.1.1) (1.26.4)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->whisperx==3.1.1) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->whisperx==3.1.1) (2024.2)\n", "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->whisperx==3.1.1) (2024.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers->whisperx==3.1.1) (24.1)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers->whisperx==3.1.1) (6.0.2)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers->whisperx==3.1.1) (2.32.3)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers->whisperx==3.1.1) (0.4.5)\n", "INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.\n", "Collecting transformers (from whisperx==3.1.1)\n", " Downloading transformers-4.45.1-py3-none-any.whl.metadata (44 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m44.4/44.4 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.45.0-py3-none-any.whl.metadata (44 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m44.4/44.4 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.44.1-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.44.0-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.43.4-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.43.3-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.43.2-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m552.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hINFO: pip is still looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.\n", " Downloading transformers-4.43.1-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.43.0-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.42.4-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.42.3-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.42.2-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hINFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n", " Downloading transformers-4.42.1-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.42.0-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.41.2-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.8/43.8 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.41.1-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.8/43.8 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.41.0-py3-none-any.whl.metadata (43 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.8/43.8 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.40.2-py3-none-any.whl.metadata (137 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.0/138.0 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.40.1-py3-none-any.whl.metadata (137 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.0/138.0 kB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.40.0-py3-none-any.whl.metadata (137 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.6/137.6 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Downloading transformers-4.39.3-py3-none-any.whl.metadata (134 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m11.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13->faster-whisper==1.0.0->whisperx==3.1.1) (2024.6.1)\n", "Collecting lightning-utilities<2.0,>=0.10.0 (from lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading lightning_utilities-0.11.7-py3-none-any.whl.metadata (5.2 kB)\n", "INFO: pip is looking at multiple versions of lightning to determine which version is compatible with other requirements. This could take a while.\n", "Collecting lightning>=2.0.1 (from pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading lightning-2.3.3-py3-none-any.whl.metadata (35 kB)\n", "Collecting pytorch-lightning (from lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pytorch_lightning-2.4.0-py3-none-any.whl.metadata (21 kB)\n", "Collecting antlr4-python3-runtime==4.9.* (from omegaconf<3.0,>=2.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.0/117.0 kB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Collecting coloredlogs (from onnxruntime<2,>=1.14->faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)\n", "Requirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper==1.0.0->whisperx==3.1.1) (24.3.25)\n", "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime<2,>=1.14->faster-whisper==1.0.0->whisperx==3.1.1) (3.20.3)\n", "Requirement already satisfied: sortedcontainers>=2.0.4 in /usr/local/lib/python3.10/dist-packages (from pyannote.core>=5.0.0->pyannote.audio==3.1.1->whisperx==3.1.1) (2.4.0)\n", "Requirement already satisfied: scipy>=1.1 in /usr/local/lib/python3.10/dist-packages (from pyannote.core>=5.0.0->pyannote.audio==3.1.1->whisperx==3.1.1) (1.13.1)\n", "Requirement already satisfied: typer>=0.12.1 in /usr/local/lib/python3.10/dist-packages (from pyannote.database>=5.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (0.12.5)\n", "Requirement already satisfied: scikit-learn>=0.17.1 in /usr/local/lib/python3.10/dist-packages (from pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (1.5.2)\n", "Collecting docopt>=0.6.2 (from pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading docopt-0.6.2.tar.gz (25 kB)\n", " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: tabulate>=0.7.7 in /usr/local/lib/python3.10/dist-packages (from pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (0.9.0)\n", "Requirement already satisfied: matplotlib>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (3.7.1)\n", "Collecting optuna>=3.1 (from pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading optuna-4.0.0-py3-none-any.whl.metadata (16 kB)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->whisperx==3.1.1) (1.16.0)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=12.0.0->pyannote.audio==3.1.1->whisperx==3.1.1) (3.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=12.0.0->pyannote.audio==3.1.1->whisperx==3.1.1) (2.18.0)\n", "Requirement already satisfied: cffi>=1.0 in /usr/local/lib/python3.10/dist-packages (from soundfile>=0.12.1->pyannote.audio==3.1.1->whisperx==3.1.1) (1.17.1)\n", "Collecting hyperpyyaml (from speechbrain>=0.5.14->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading HyperPyYAML-1.2.2-py3-none-any.whl.metadata (7.6 kB)\n", "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from speechbrain>=0.5.14->pyannote.audio==3.1.1->whisperx==3.1.1) (0.2.0)\n", "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2->whisperx==3.1.1) (1.3.0)\n", "Collecting julius<0.3,>=0.2.3 (from torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading julius-0.2.7.tar.gz (59 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.6/59.6 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: librosa>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.10.2.post1)\n", "Collecting torch-pitch-shift>=1.2.2 (from torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading torch_pitch_shift-1.2.5-py3-none-any.whl.metadata (2.5 kB)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2->whisperx==3.1.1) (2.1.5)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->whisperx==3.1.1) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->whisperx==3.1.1) (3.10)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->whisperx==3.1.1) (2.2.3)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->whisperx==3.1.1) (2024.8.30)\n", "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.0->soundfile>=0.12.1->pyannote.audio==3.1.1->whisperx==3.1.1) (2.22)\n", "Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (3.10.6)\n", "Requirement already satisfied: audioread>=2.1.9 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (3.0.1)\n", "Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (4.4.2)\n", "Requirement already satisfied: numba>=0.51.0 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.60.0)\n", "Requirement already satisfied: pooch>=1.1 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (1.8.2)\n", "Requirement already satisfied: soxr>=0.3.2 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.5.0.post1)\n", "Requirement already satisfied: lazy-loader>=0.1 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.4)\n", "Requirement already satisfied: msgpack>=1.0 in /usr/local/lib/python3.10/dist-packages (from librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (1.0.8)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=12.0.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.1.2)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (1.3.0)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (0.12.1)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (4.54.1)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (1.4.7)\n", "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (10.4.0)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib>=2.0.0->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (3.1.4)\n", "Collecting alembic>=1.5.0 (from optuna>=3.1->pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading alembic-1.13.3-py3-none-any.whl.metadata (7.4 kB)\n", "Collecting colorlog (from optuna>=3.1->pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading colorlog-6.8.2-py3-none-any.whl.metadata (10 kB)\n", "Requirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from optuna>=3.1->pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (2.0.35)\n", "Requirement already satisfied: threadpoolctl>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.17.1->pyannote.metrics>=3.2->pyannote.audio==3.1.1->whisperx==3.1.1) (3.5.0)\n", "Collecting primePy>=1.3 (from torch-pitch-shift>=1.2.2->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading primePy-1.3-py3-none-any.whl.metadata (4.8 kB)\n", "Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from typer>=0.12.1->pyannote.database>=5.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (1.5.4)\n", "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime<2,>=1.14->faster-whisper==1.0.0->whisperx==3.1.1)\n", " Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)\n", "Collecting ruamel.yaml>=0.17.28 (from hyperpyyaml->speechbrain>=0.5.14->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading ruamel.yaml-0.18.6-py3-none-any.whl.metadata (23 kB)\n", "INFO: pip is looking at multiple versions of pytorch-lightning to determine which version is compatible with other requirements. This could take a while.\n", "Collecting pytorch-lightning (from lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading pytorch_lightning-2.3.3-py3-none-any.whl.metadata (21 kB)\n", "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (2.4.0)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (24.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (6.1.0)\n", "Requirement already satisfied: yarl<2.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (1.12.1)\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2026.0,>=2022.5.0->lightning>=2.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (4.0.3)\n", "Collecting Mako (from alembic>=1.5.0->optuna>=3.1->pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading Mako-1.3.5-py3-none-any.whl.metadata (2.9 kB)\n", "Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba>=0.51.0->librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (0.43.0)\n", "Requirement already satisfied: platformdirs>=2.5.0 in /usr/local/lib/python3.10/dist-packages (from pooch>=1.1->librosa>=0.6.0->torch-audiomentations>=0.11.0->pyannote.audio==3.1.1->whisperx==3.1.1) (4.3.6)\n", "Collecting ruamel.yaml.clib>=0.2.7 (from ruamel.yaml>=0.17.28->hyperpyyaml->speechbrain>=0.5.14->pyannote.audio==3.1.1->whisperx==3.1.1)\n", " Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (2.2 kB)\n", "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from sqlalchemy>=1.3.0->optuna>=3.1->pyannote.pipeline>=3.0.1->pyannote.audio==3.1.1->whisperx==3.1.1) (3.1.1)\n", "Downloading faster_whisper-1.0.0-py3-none-any.whl (1.5 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m41.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pyannote.audio-3.1.1-py2.py3-none-any.whl (208 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m208.7/208.7 kB\u001b[0m \u001b[31m19.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading av-11.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32.9 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m32.9/32.9 MB\u001b[0m \u001b[31m54.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading transformers-4.39.3-py3-none-any.whl (8.8 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.8/8.8 MB\u001b[0m \u001b[31m68.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading asteroid_filterbanks-0.4.0-py3-none-any.whl (29 kB)\n", "Downloading ctranslate2-4.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (37.2 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m37.2/37.2 MB\u001b[0m \u001b[31m13.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading lightning-2.3.3-py3-none-any.whl (808 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m808.5/808.5 kB\u001b[0m \u001b[31m39.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading omegaconf-2.3.0-py3-none-any.whl (79 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m79.5/79.5 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading onnxruntime-1.19.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (13.2 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.2/13.2 MB\u001b[0m \u001b[31m100.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pyannote.core-5.0.0-py3-none-any.whl (58 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.5/58.5 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pyannote.database-5.1.0-py3-none-any.whl (48 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.1/48.1 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pyannote.metrics-3.2.1-py3-none-any.whl (51 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m51.4/51.4 kB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pyannote.pipeline-3.0.1-py3-none-any.whl (31 kB)\n", "Downloading pytorch_metric_learning-2.6.0-py3-none-any.whl (119 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m119.3/119.3 kB\u001b[0m \u001b[31m11.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading semver-3.0.2-py3-none-any.whl (17 kB)\n", "Downloading speechbrain-1.0.1-py3-none-any.whl (807 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m807.2/807.2 kB\u001b[0m \u001b[31m47.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading tensorboardX-2.6.2.2-py2.py3-none-any.whl (101 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.7/101.7 kB\u001b[0m \u001b[31m9.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.6/3.6 MB\u001b[0m \u001b[31m96.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading torch_audiomentations-0.11.1-py3-none-any.whl (50 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.1/50.1 kB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading torchmetrics-1.4.2-py3-none-any.whl (869 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m869.2/869.2 kB\u001b[0m \u001b[31m51.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading lightning_utilities-0.11.7-py3-none-any.whl (26 kB)\n", "Downloading optuna-4.0.0-py3-none-any.whl (362 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m362.8/362.8 kB\u001b[0m \u001b[31m27.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading torch_pitch_shift-1.2.5-py3-none-any.whl (5.0 kB)\n", "Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading HyperPyYAML-1.2.2-py3-none-any.whl (16 kB)\n", "Downloading pytorch_lightning-2.3.3-py3-none-any.whl (812 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m812.3/812.3 kB\u001b[0m \u001b[31m48.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading alembic-1.13.3-py3-none-any.whl (233 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m233.2/233.2 kB\u001b[0m \u001b[31m21.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading primePy-1.3-py3-none-any.whl (4.0 kB)\n", "Downloading ruamel.yaml-0.18.6-py3-none-any.whl (117 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.8/117.8 kB\u001b[0m \u001b[31m11.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading colorlog-6.8.2-py3-none-any.whl (11 kB)\n", "Downloading ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (526 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m526.7/526.7 kB\u001b[0m \u001b[31m33.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading Mako-1.3.5-py3-none-any.whl (78 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.6/78.6 kB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hBuilding wheels for collected packages: antlr4-python3-runtime, docopt, julius\n", " Building wheel for antlr4-python3-runtime (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.9.3-py3-none-any.whl size=144554 sha256=0d45790bcba89ef25b40e28a352826b1e3b8e0a996f3c4c71c77a2e039838c51\n", " Stored in directory: /root/.cache/pip/wheels/12/93/dd/1f6a127edc45659556564c5730f6d4e300888f4bca2d4c5a88\n", " Building wheel for docopt (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=0a394444811d2361dc868cfb7f23c797a47cad9a1ee5485aa86c20551b33a0fe\n", " Stored in directory: /root/.cache/pip/wheels/fc/ab/d4/5da2067ac95b36618c629a5f93f809425700506f72c9732fac\n", " Building wheel for julius (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for julius: filename=julius-0.2.7-py3-none-any.whl size=21869 sha256=c4d0f47e1e8d2846ed38f7ba03d944e5ed1068278a16c671c7b946053b7f7446\n", " Stored in directory: /root/.cache/pip/wheels/b9/b2/05/f883527ffcb7f2ead5438a2c23439aa0c881eaa9a4c80256f4\n", "Successfully built antlr4-python3-runtime docopt julius\n", "Installing collected packages: primePy, docopt, antlr4-python3-runtime, tensorboardX, semver, ruamel.yaml.clib, omegaconf, Mako, lightning-utilities, humanfriendly, ctranslate2, colorlog, av, ruamel.yaml, pyannote.core, coloredlogs, alembic, tokenizers, optuna, onnxruntime, hyperpyyaml, transformers, pyannote.database, faster-whisper, pyannote.pipeline, pyannote.metrics, torchmetrics, torch-pitch-shift, pytorch-lightning, julius, torch-audiomentations, speechbrain, pytorch-metric-learning, lightning, asteroid-filterbanks, pyannote.audio, whisperx\n", " Attempting uninstall: tokenizers\n", " Found existing installation: tokenizers 0.19.1\n", " Uninstalling tokenizers-0.19.1:\n", " Successfully uninstalled tokenizers-0.19.1\n", " Attempting uninstall: transformers\n", " Found existing installation: transformers 4.44.2\n", " Uninstalling transformers-4.44.2:\n", " Successfully uninstalled transformers-4.44.2\n", " Running setup.py develop for whisperx\n", "Successfully installed Mako-1.3.5 alembic-1.13.3 antlr4-python3-runtime-4.9.3 asteroid-filterbanks-0.4.0 av-11.0.0 coloredlogs-15.0.1 colorlog-6.8.2 ctranslate2-4.4.0 docopt-0.6.2 faster-whisper-1.0.0 humanfriendly-10.0 hyperpyyaml-1.2.2 julius-0.2.7 lightning-2.3.3 lightning-utilities-0.11.7 omegaconf-2.3.0 onnxruntime-1.19.2 optuna-4.0.0 primePy-1.3 pyannote.audio-3.1.1 pyannote.core-5.0.0 pyannote.database-5.1.0 pyannote.metrics-3.2.1 pyannote.pipeline-3.0.1 pytorch-lightning-2.3.3 pytorch-metric-learning-2.6.0 ruamel.yaml-0.18.6 ruamel.yaml.clib-0.2.8 semver-3.0.2 speechbrain-1.0.1 tensorboardX-2.6.2.2 tokenizers-0.15.2 torch-audiomentations-0.11.1 torch-pitch-shift-1.2.5 torchmetrics-1.4.2 transformers-4.39.3 whisperx-3.1.1\n", "Converting requirements.txt to GBK encoding...\n", "Conversion completed.\n", "Installing dependencies from requirements.txt...\n", "Collecting azure-cognitiveservices-speech==1.40.0 (from -r requirements.txt (line 1))\n", " Downloading azure_cognitiveservices_speech-1.40.0-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n", "Requirement already satisfied: librosa==0.10.2.post1 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 2)) (0.10.2.post1)\n", "Requirement already satisfied: moviepy==1.0.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 3)) (1.0.3)\n", "Requirement already satisfied: numpy==1.26.4 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (1.26.4)\n", "Collecting openai==1.47.0 (from -r requirements.txt (line 5))\n", " Downloading openai-1.47.0-py3-none-any.whl.metadata (24 kB)\n", "Requirement already satisfied: opencv-python==4.10.0.84 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 6)) (4.10.0.84)\n", "Requirement already satisfied: openpyxl==3.1.5 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 7)) (3.1.5)\n", "Collecting pandas==2.2.3 (from -r requirements.txt (line 8))\n", " Downloading pandas-2.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (89 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m89.9/89.9 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting pydub==0.25.1 (from -r requirements.txt (line 9))\n", " Downloading pydub-0.25.1-py2.py3-none-any.whl.metadata (1.4 kB)\n", "Requirement already satisfied: PyYAML==6.0.2 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 10)) (6.0.2)\n", "Collecting replicate==0.33.0 (from -r requirements.txt (line 11))\n", " Downloading replicate-0.33.0-py3-none-any.whl.metadata (25 kB)\n", "Requirement already satisfied: requests==2.32.3 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (2.32.3)\n", "Collecting resampy==0.4.3 (from -r requirements.txt (line 13))\n", " Downloading resampy-0.4.3-py3-none-any.whl.metadata (3.0 kB)\n", "Requirement already satisfied: spacy==3.7.6 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 14)) (3.7.6)\n", "Collecting streamlit==1.38.0 (from -r requirements.txt (line 15))\n", " Downloading streamlit-1.38.0-py2.py3-none-any.whl.metadata (8.5 kB)\n", "Collecting yt-dlp==2024.8.6 (from -r requirements.txt (line 16))\n", " Downloading yt_dlp-2024.8.6-py3-none-any.whl.metadata (170 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m170.1/170.1 kB\u001b[0m \u001b[31m11.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting json-repair (from -r requirements.txt (line 17))\n", " Downloading json_repair-0.29.7-py3-none-any.whl.metadata (10 kB)\n", "Requirement already satisfied: audioread>=2.1.9 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (3.0.1)\n", "Requirement already satisfied: scipy>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.13.1)\n", "Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.5.2)\n", "Requirement already satisfied: joblib>=0.14 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.4.2)\n", "Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (4.4.2)\n", "Requirement already satisfied: numba>=0.51.0 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (0.60.0)\n", "Requirement already satisfied: soundfile>=0.12.1 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (0.12.1)\n", "Requirement already satisfied: pooch>=1.1 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.8.2)\n", "Requirement already satisfied: soxr>=0.3.2 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (0.5.0.post1)\n", "Requirement already satisfied: typing-extensions>=4.1.1 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (4.12.2)\n", "Requirement already satisfied: lazy-loader>=0.1 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (0.4)\n", "Requirement already satisfied: msgpack>=1.0 in /usr/local/lib/python3.10/dist-packages (from librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.0.8)\n", "Requirement already satisfied: tqdm<5.0,>=4.11.2 in /usr/local/lib/python3.10/dist-packages (from moviepy==1.0.3->-r requirements.txt (line 3)) (4.66.5)\n", "Requirement already satisfied: proglog<=1.0.0 in /usr/local/lib/python3.10/dist-packages (from moviepy==1.0.3->-r requirements.txt (line 3)) (0.1.10)\n", "Requirement already satisfied: imageio<3.0,>=2.5 in /usr/local/lib/python3.10/dist-packages (from moviepy==1.0.3->-r requirements.txt (line 3)) (2.35.1)\n", "Requirement already satisfied: imageio-ffmpeg>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from moviepy==1.0.3->-r requirements.txt (line 3)) (0.5.1)\n", "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai==1.47.0->-r requirements.txt (line 5)) (3.7.1)\n", "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai==1.47.0->-r requirements.txt (line 5)) (1.7.0)\n", "Collecting httpx<1,>=0.23.0 (from openai==1.47.0->-r requirements.txt (line 5))\n", " Downloading httpx-0.27.2-py3-none-any.whl.metadata (7.1 kB)\n", "Collecting jiter<1,>=0.4.0 (from openai==1.47.0->-r requirements.txt (line 5))\n", " Downloading jiter-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.6 kB)\n", "Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai==1.47.0->-r requirements.txt (line 5)) (2.9.2)\n", "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai==1.47.0->-r requirements.txt (line 5)) (1.3.1)\n", "Requirement already satisfied: et-xmlfile in /usr/local/lib/python3.10/dist-packages (from openpyxl==3.1.5->-r requirements.txt (line 7)) (1.1.0)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas==2.2.3->-r requirements.txt (line 8)) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas==2.2.3->-r requirements.txt (line 8)) (2024.2)\n", "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas==2.2.3->-r requirements.txt (line 8)) (2024.2)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from replicate==0.33.0->-r requirements.txt (line 11)) (24.1)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests==2.32.3->-r requirements.txt (line 12)) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests==2.32.3->-r requirements.txt (line 12)) (3.10)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests==2.32.3->-r requirements.txt (line 12)) (2.2.3)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests==2.32.3->-r requirements.txt (line 12)) (2024.8.30)\n", "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (3.0.12)\n", "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (1.0.5)\n", "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (1.0.10)\n", "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (2.0.8)\n", "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (3.0.9)\n", "Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (8.2.5)\n", "Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (1.1.3)\n", "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (2.4.8)\n", "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (2.0.10)\n", "Requirement already satisfied: weasel<0.5.0,>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (0.4.1)\n", "Requirement already satisfied: typer<1.0.0,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (0.12.5)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (3.1.4)\n", "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (71.0.4)\n", "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.10/dist-packages (from spacy==3.7.6->-r requirements.txt (line 14)) (3.4.1)\n", "Requirement already satisfied: altair<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (4.2.2)\n", "Requirement already satisfied: blinker<2,>=1.0.0 in /usr/lib/python3/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (1.4)\n", "Requirement already satisfied: cachetools<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (5.5.0)\n", "Requirement already satisfied: click<9,>=7.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (8.1.7)\n", "Requirement already satisfied: pillow<11,>=7.1.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (10.4.0)\n", "Requirement already satisfied: protobuf<6,>=3.20 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (3.20.3)\n", "Requirement already satisfied: pyarrow>=7.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (16.1.0)\n", "Requirement already satisfied: rich<14,>=10.14.0 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (13.8.1)\n", "Collecting tenacity<9,>=8.1.0 (from streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading tenacity-8.5.0-py3-none-any.whl.metadata (1.2 kB)\n", "Requirement already satisfied: toml<2,>=0.10.1 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (0.10.2)\n", "Collecting gitpython!=3.1.19,<4,>=3.0.7 (from streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)\n", "Collecting pydeck<1,>=0.8.0b4 (from streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading pydeck-0.9.1-py2.py3-none-any.whl.metadata (4.1 kB)\n", "Requirement already satisfied: tornado<7,>=6.0.3 in /usr/local/lib/python3.10/dist-packages (from streamlit==1.38.0->-r requirements.txt (line 15)) (6.3.3)\n", "Collecting watchdog<5,>=2.1.5 (from streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading watchdog-4.0.2-py3-none-manylinux2014_x86_64.whl.metadata (38 kB)\n", "Collecting brotli (from yt-dlp==2024.8.6->-r requirements.txt (line 16))\n", " Downloading Brotli-1.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (5.5 kB)\n", "Collecting mutagen (from yt-dlp==2024.8.6->-r requirements.txt (line 16))\n", " Downloading mutagen-1.47.0-py3-none-any.whl.metadata (1.7 kB)\n", "Collecting pycryptodomex (from yt-dlp==2024.8.6->-r requirements.txt (line 16))\n", " Downloading pycryptodomex-3.21.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)\n", "Collecting websockets>=12.0 (from yt-dlp==2024.8.6->-r requirements.txt (line 16))\n", " Downloading websockets-13.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)\n", "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (0.4)\n", "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (4.23.0)\n", "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (0.12.1)\n", "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai==1.47.0->-r requirements.txt (line 5)) (1.2.2)\n", "Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3.0.7->streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)\n", "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai==1.47.0->-r requirements.txt (line 5))\n", " Downloading httpcore-1.0.6-py3-none-any.whl.metadata (21 kB)\n", "Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai==1.47.0->-r requirements.txt (line 5))\n", " Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)\n", "Requirement already satisfied: language-data>=1.2 in /usr/local/lib/python3.10/dist-packages (from langcodes<4.0.0,>=3.2.0->spacy==3.7.6->-r requirements.txt (line 14)) (1.2.0)\n", "Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba>=0.51.0->librosa==0.10.2.post1->-r requirements.txt (line 2)) (0.43.0)\n", "Requirement already satisfied: platformdirs>=2.5.0 in /usr/local/lib/python3.10/dist-packages (from pooch>=1.1->librosa==0.10.2.post1->-r requirements.txt (line 2)) (4.3.6)\n", "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai==1.47.0->-r requirements.txt (line 5)) (0.7.0)\n", "Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai==1.47.0->-r requirements.txt (line 5)) (2.23.4)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->spacy==3.7.6->-r requirements.txt (line 14)) (2.1.5)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas==2.2.3->-r requirements.txt (line 8)) (1.16.0)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich<14,>=10.14.0->streamlit==1.38.0->-r requirements.txt (line 15)) (3.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich<14,>=10.14.0->streamlit==1.38.0->-r requirements.txt (line 15)) (2.18.0)\n", "Requirement already satisfied: threadpoolctl>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.20.0->librosa==0.10.2.post1->-r requirements.txt (line 2)) (3.5.0)\n", "Requirement already satisfied: cffi>=1.0 in /usr/local/lib/python3.10/dist-packages (from soundfile>=0.12.1->librosa==0.10.2.post1->-r requirements.txt (line 2)) (1.17.1)\n", "Requirement already satisfied: blis<0.8.0,>=0.7.8 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy==3.7.6->-r requirements.txt (line 14)) (0.7.11)\n", "Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.10/dist-packages (from thinc<8.3.0,>=8.2.2->spacy==3.7.6->-r requirements.txt (line 14)) (0.1.5)\n", "Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from typer<1.0.0,>=0.3.0->spacy==3.7.6->-r requirements.txt (line 14)) (1.5.4)\n", "Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from weasel<0.5.0,>=0.1.0->spacy==3.7.6->-r requirements.txt (line 14)) (0.19.0)\n", "Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in /usr/local/lib/python3.10/dist-packages (from weasel<0.5.0,>=0.1.0->spacy==3.7.6->-r requirements.txt (line 14)) (7.0.4)\n", "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.0->soundfile>=0.12.1->librosa==0.10.2.post1->-r requirements.txt (line 2)) (2.22)\n", "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit==1.38.0->-r requirements.txt (line 15))\n", " Downloading smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)\n", "Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (24.2.0)\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (2023.12.1)\n", "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (0.35.1)\n", "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit==1.38.0->-r requirements.txt (line 15)) (0.20.0)\n", "Requirement already satisfied: marisa-trie>=0.7.7 in /usr/local/lib/python3.10/dist-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy==3.7.6->-r requirements.txt (line 14)) (1.2.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich<14,>=10.14.0->streamlit==1.38.0->-r requirements.txt (line 15)) (0.1.2)\n", "Requirement already satisfied: wrapt in /usr/local/lib/python3.10/dist-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.1.0->spacy==3.7.6->-r requirements.txt (line 14)) (1.16.0)\n", "Downloading azure_cognitiveservices_speech-1.40.0-py3-none-manylinux1_x86_64.whl (40.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40.1/40.1 MB\u001b[0m \u001b[31m24.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading openai-1.47.0-py3-none-any.whl (375 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m375.6/375.6 kB\u001b[0m \u001b[31m24.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pandas-2.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.1/13.1 MB\u001b[0m \u001b[31m73.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n", "Downloading replicate-0.33.0-py3-none-any.whl (45 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.3/45.3 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading resampy-0.4.3-py3-none-any.whl (3.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m69.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading streamlit-1.38.0-py2.py3-none-any.whl (8.7 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.7/8.7 MB\u001b[0m \u001b[31m71.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading yt_dlp-2024.8.6-py3-none-any.whl (3.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m54.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading json_repair-0.29.7-py3-none-any.whl (17 kB)\n", "Downloading GitPython-3.1.43-py3-none-any.whl (207 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.3/207.3 kB\u001b[0m \u001b[31m18.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading httpx-0.27.2-py3-none-any.whl (76 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.4/76.4 kB\u001b[0m \u001b[31m7.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading httpcore-1.0.6-py3-none-any.whl (78 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.0/78.0 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading jiter-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (318 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m318.9/318.9 kB\u001b[0m \u001b[31m24.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pydeck-0.9.1-py2.py3-none-any.whl (6.9 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.9/6.9 MB\u001b[0m \u001b[31m83.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading tenacity-8.5.0-py3-none-any.whl (28 kB)\n", "Downloading watchdog-4.0.2-py3-none-manylinux2014_x86_64.whl (82 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m82.9/82.9 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading websockets-13.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (164 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m164.1/164.1 kB\u001b[0m \u001b[31m13.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading Brotli-1.1.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.0 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.0/3.0 MB\u001b[0m \u001b[31m66.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading mutagen-1.47.0-py3-none-any.whl (194 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.4/194.4 kB\u001b[0m \u001b[31m15.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading pycryptodomex-3.21.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 MB\u001b[0m \u001b[31m64.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading h11-0.14.0-py3-none-any.whl (58 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading smmap-5.0.1-py3-none-any.whl (24 kB)\n", "Installing collected packages: pydub, brotli, websockets, watchdog, tenacity, smmap, pycryptodomex, mutagen, json-repair, jiter, h11, azure-cognitiveservices-speech, yt-dlp, resampy, pydeck, pandas, httpcore, gitdb, httpx, gitpython, replicate, openai, streamlit\n", " Attempting uninstall: tenacity\n", " Found existing installation: tenacity 9.0.0\n", " Uninstalling tenacity-9.0.0:\n", " Successfully uninstalled tenacity-9.0.0\n", " Attempting uninstall: pandas\n", " Found existing installation: pandas 2.2.2\n", " Uninstalling pandas-2.2.2:\n", " Successfully uninstalled pandas-2.2.2\n", "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "cudf-cu12 24.6.1 requires pandas<2.2.3dev0,>=2.0, but you have pandas 2.2.3 which is incompatible.\n", "google-colab 1.0.0 requires pandas==2.2.2, but you have pandas 2.2.3 which is incompatible.\u001b[0m\u001b[31m\n", "\u001b[0mSuccessfully installed azure-cognitiveservices-speech-1.40.0 brotli-1.1.0 gitdb-4.0.11 gitpython-3.1.43 h11-0.14.0 httpcore-1.0.6 httpx-0.27.2 jiter-0.5.0 json-repair-0.29.7 mutagen-1.47.0 openai-1.47.0 pandas-2.2.3 pycryptodomex-3.21.0 pydeck-0.9.1 pydub-0.25.1 replicate-0.33.0 resampy-0.4.3 smmap-5.0.1 streamlit-1.38.0 tenacity-8.5.0 watchdog-4.0.2 websockets-13.1 yt-dlp-2024.8.6\n", "Downloading UVR model: HP2_all_vocals.pth...\n", "Downloaded: 0.01%\n", "HP2_all_vocals.pth downloaded successfully.\n", "Downloading UVR model: VR-DeEchoAggressive.pth...\n", "Downloaded: 0.00%\n", "VR-DeEchoAggressive.pth downloaded successfully.\n", "Downloading FFmpeg...\n", "FFmpeg has been downloaded to ffmpeg.tar.xz\n", "Extracting FFmpeg...\n", "Cleaning up...\n", "FFmpeg extraction completed.\n", "\u001b[1;32m╭───────────────────────────────────────╮\u001b[0m\n", "\u001b[1;32m│\u001b[0m\u001b[1;32m \u001b[0m\u001b[1;32mAll installation steps are completed!\u001b[0m\u001b[1;32m \u001b[0m\u001b[1;32m│\u001b[0m\n", "\u001b[1;32m╰───────────────────────────────────────╯\u001b[0m\n", "Please use the following command to start Streamlit:\n", "\u001b[1;36mstreamlit run st.py\u001b[0m\n" ] } ], "source": [ "!python install.py" ] }, { "cell_type": "markdown", "metadata": { "id": "5J2jjEY2BrM3" }, "source": [ "## 3. Register and Obtain Ngrok Token 🔑\n", "\n", "1. Visit the [Ngrok official website](https://ngrok.com/) and register for an account.\n", "2. After logging in, find the \"Your Authtoken\" section on the dashboard page, or directly visit [Ngrok Token](https://dashboard.ngrok.com/get-started/your-authtoken).\n", "3. Copy your Ngrok Authtoken.\n", "\n", "After completing these steps, please fill in your ngrok token in the next section of code and proceed.\n", "\n", "---\n", "\n", "## 3. 注册并获取 Ngrok 令牌 🔑\n", "\n", "1. 访问 [Ngrok 官方网站](https://ngrok.com/) 并注册账户。\n", "2. 登录后，在仪表板页面找到\"Your Authtoken\"部分，或直接访问 [Ngrok 令牌](https://dashboard.ngrok.com/get-started/your-authtoken)。\n", "3. 复制您的 Ngrok Authtoken。\n", "\n", "完成这些步骤后，请在下一节代码中填入您的 ngrok 令牌并继续。" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "tm25rblnBqhl", "outputId": "93691228-1e7b-48e7-b7f3-c27007a4a13f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: pyngrok in /usr/local/lib/python3.10/dist-packages (7.2.0)\n", "Requirement already satisfied: PyYAML>=5.1 in /usr/local/lib/python3.10/dist-packages (from pyngrok) (6.0.2)\n" ] } ], "source": [ "!pip install pyngrok\n", "from pyngrok import ngrok\n", "\n", "#! SET Ngrok Authtoken Here\n", "ngrok.set_auth_token(\"YOUR_TOKEN_HERE\")" ] }, { "cell_type": "markdown", "metadata": { "id": "MDJXtgMuGXMH" }, "source": [ "## 🎈 4. Streamlit GO !!!\n", "Click the NgrokChannel URL to start your VideoLingo Journey.\n", "\n", "> tips: You can set your Language down the sidebar on the left." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 294 }, "id": "qr_a4mS29k5_", "outputId": "e6915708-d030-45d0-d6a8-cb177960c137" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Collecting usage statistics. To deactivate, set browser.gatherUsageStats to false.\n", "\n" ] }, { "data": { "text/html": [ "╭───────────────────────────────────╮\n",
"│ Streamlit is available at Ngrok ⬇️ │\n",
"╰───────────────────────────────────╯\n",
"\n"
],
"text/plain": [
"╭───────────────────────────────────╮\n",
"│ Streamlit is available at Ngrok ⬇️ │\n",
"╰───────────────────────────────────╯\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Click 👉 NgrokTunnel: \"https://0308-34-125-196-161.ngrok-free.app\" -> \"http://localhost:8501\"\n",
"\n",
" You can now view your Streamlit app in your browser.\n",
"\n",
" Local URL: http://localhost:8501\n",
" Network URL: http://172.28.0.12:8501\n",
" External URL: http://34.125.196.161:8501\n",
"\n",
" Stopping...\n",
"Interrupted by user, shutting down...\n"
]
}
],
"source": [
"import subprocess\n",
"import threading\n",
"import sys\n",
"from pyngrok import ngrok\n",
"from rich import print as rprint\n",
"from rich.panel import Panel\n",
"\n",
"def print_output(process):\n",
" for line in iter(process.stdout.readline, ''):\n",
" sys.stdout.write(line)\n",
" for line in iter(process.stderr.readline, ''):\n",
" sys.stderr.write(line)\n",
"\n",
"# Start Streamlit\n",
"streamlit_process = subprocess.Popen(\n",
" [\"streamlit\", \"run\", \"st.py\"],\n",
" stdout=subprocess.PIPE,\n",
" stderr=subprocess.PIPE,\n",
" universal_newlines=True,\n",
" bufsize=1\n",
")\n",
"\n",
"# Create and start the output printing thread\n",
"output_thread = threading.Thread(target=print_output, args=(streamlit_process,))\n",
"output_thread.start()\n",
"\n",
"# Create a tunnel using ngrok\n",
"public_url = ngrok.connect(8501)\n",
"rprint(Panel(f\"Streamlit is available at Ngrok ⬇️\", expand=False))\n",
"print(f\"Click 👉 {public_url}\")\n",
"\n",
"# Keep the program running\n",
"ngrok_process = ngrok.get_ngrok_process()\n",
"try:\n",
" streamlit_process.wait()\n",
"except KeyboardInterrupt:\n",
" print(\"Interrupted by user, shutting down...\")\n",
"finally:\n",
" ngrok.kill()\n",
" streamlit_process.terminate()\n",
" output_thread.join()\n"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: batch/OneKeyBatch.bat
================================================
@echo off
cd /D "%~dp0"
cd ..
call conda activate videolingo
@rem 运行批处理脚本
call python batch\utils\batch_processor.py
:end
pause
================================================
FILE: batch/README.md
================================================
# VideoLingo Batch Mode
[English](./README.md) | [简体中文](./README.zh.md)
Before utilizing the batch mode, ensure you have used the Streamlit mode and properly configured the parameters in `config.yaml`.
## Usage Guide
### 1. Video File Preparation
- Place your video files in the `input` folder
- YouTube links can be specified in the next step
### 2. Task Configuration
Edit the `tasks_setting.xlsx` file:
| Field | Description | Acceptable Values |
|-------|-------------|-------------------|
| Video File | Video filename (without `input/` prefix) or YouTube URL | - |
| Source Language | Source language | 'en', 'zh', ... or leave empty for default |
| Target Language | Translation language | Use natural language description, or leave empty for default |
| Dubbing | Enable dubbing | 0 or empty: no dubbing; 1: enable dubbing |
Example:
| Video File | Source Language | Target Language | Dubbing |
|------------|-----------------|-----------------|---------|
| https://www.youtube.com/xxx | | German | |
| Kungfu Panda.mp4 | | | 1 |
### 3. Executing Batch Processing
1. Double-click to run `OneKeyBatch.bat`
2. Output files will be saved in the `output` folder
3. Task status can be monitored in the `Status` column of `tasks_setting.xlsx`
> Note: Keep `tasks_setting.xlsx` closed during execution to prevent interruptions due to file access conflicts.
## Important Considerations
### Handling Interruptions
If the command line is closed unexpectedly, language settings in `config.yaml` may be altered. Check settings before retrying.
### Error Management
- Failed files will be moved to the `output/ERROR` folder
- Error messages are recorded in the `Status` column of `tasks_setting.xlsx`
- To retry:
1. Move the single video folder from `ERROR` to the root directory
2. Rename it to `output`
3. Use Streamlit mode to process again
================================================
FILE: batch/README.zh.md
================================================
# VideoLingo Batch Mode
[English](./README.md) | [简体中文](./README.zh.md)
在使用批处理模式前，请确保你已经使用过 Streamlit 模式并正确设置了 `config.yaml` 中的参数。
## 使用方法
### 1. 准备视频文件
- 将要处理的视频文件放入 `input` 文件夹
- YouTube 链接可在下一步填写
### 2. 配置任务
编辑 `tasks_setting.xlsx` 文件：
| 字段 | 说明 | 可选值 |
|------|------|--------|
| Video File | 视频文件名（无需 `input/` 前缀）或 YouTube 链接 | - |
| Source Language | 源语言 | 'en', 'zh', ... 或留空使用默认设置 |
| Target Language | 翻译语言 | 使用自然语言描述，或留空使用默认设置 |
| Dubbing | 是否配音 | 0 或留空：不配音；1：配音 |
示例：
| Video File | Source Language | Target Language | Dubbing |
|------------|-----------------|-----------------|---------|
| https://www.youtube.com/xxx | | German | |
| Kungfu Panda.mp4 | | | 1 |
### 3. 运行批处理
1. 双击运行 `OneKeyBatch.bat`
2. 输出文件将保存在 `output` 文件夹
3. 任务状态可在 `tasks_setting.xlsx` 的 `Status` 列查看
> 注意在运行时保持 `tasks_setting.xlsx` 关闭，否则会因占用无法写入而中断。
## 注意事项
### 中断处理
如果中途关闭命令行，`config.yaml` 中的语言设置可能会改变。重试前请检查设置。
### 错误处理
- 处理失败的文件会被移至 `output/ERROR` 文件夹
- 错误信息记录在 `tasks_setting.xlsx` 的 `Status` 列
- 如需重试：
1. 将 `ERROR` 下的单个视频文件夹移至根目录
2. 重命名为 `output`
3. 使用 Streamlit 模式重新执行
================================================
FILE: batch/utils/batch_processor.py
================================================
import os
import gc
from batch.utils.settings_check import check_settings
from batch.utils.video_processor import process_video
from core.utils.config_utils import load_key, update_key
import pandas as pd
from rich.console import Console
from rich.panel import Panel
import time
import shutil
console = Console()
def record_and_update_config(source_language, target_language):
original_source_lang = load_key('whisper.language')
original_target_lang = load_key('target_language')
if source_language and not pd.isna(source_language):
update_key('whisper.language', source_language)
if target_language and not pd.isna(target_language):
update_key('target_language', target_language)
return original_source_lang, original_target_lang
def process_batch():
if not check_settings():
raise Exception("Settings check failed")
df = pd.read_excel('batch/tasks_setting.xlsx')
for index, row in df.iterrows():
if pd.isna(row['Status']) or 'Error' in str(row['Status']):
total_tasks = len(df)
video_file = row['Video File']
if not pd.isna(row['Status']) and 'Error' in str(row['Status']):
console.print(Panel(f"Retrying failed task: {video_file}\nTask {index + 1}/{total_tasks}",
title="[bold yellow]Retry Task", expand=False))
# Restore files from batch/output/ERROR to output
error_folder = os.path.join('batch', 'output', 'ERROR', os.path.splitext(video_file)[0])
if os.path.exists(error_folder):
# Ensure the output folder exists
os.makedirs('output', exist_ok=True)
# Copy all contents from ERROR folder for the specific video to output
for item in os.listdir(error_folder):
src_path = os.path.join(error_folder, item)
dst_path = os.path.join('output', item)
if os.path.isdir(src_path):
if os.path.exists(dst_path):
shutil.rmtree(dst_path)
shutil.copytree(src_path, dst_path)
else:
if os.path.exists(dst_path):
os.remove(dst_path)
shutil.copy2(src_path, dst_path)
console.print(f"[green]Restored files from ERROR folder for {video_file}")
else:
console.print(f"[yellow]Warning: Error folder not found: {error_folder}")
else:
console.print(Panel(f"Now processing task: {video_file}\nTask {index + 1}/{total_tasks}",
title="[bold blue]Current Task", expand=False))
source_language = row['Source Language']
target_language = row['Target Language']
original_source_lang, original_target_lang = record_and_update_config(source_language, target_language)
try:
dubbing = 0 if pd.isna(row['Dubbing']) else int(row['Dubbing'])
is_retry = not pd.isna(row['Status']) and 'Error' in str(row['Status'])
status, error_step, error_message = process_video(video_file, dubbing, is_retry)
status_msg = "Done" if status else f"Error: {error_step} - {error_message}"
except Exception as e:
status_msg = f"Error: Unhandled exception - {str(e)}"
console.print(f"[bold red]Error processing {video_file}: {status_msg}")
finally:
update_key('whisper.language', original_source_lang)
update_key('target_language', original_target_lang)
df.at[index, 'Status'] = status_msg
df.to_excel('batch/tasks_setting.xlsx', index=False)
gc.collect()
time.sleep(1)
else:
print(f"Skipping task: {row['Video File']} - Status: {row['Status']}")
console.print(Panel("All tasks processed!\nCheck out in `batch/output`!",
title="[bold green]Batch Processing Complete", expand=False))
if __name__ == "__main__":
process_batch()
================================================
FILE: batch/utils/settings_check.py
================================================
import os
import pandas as pd
from rich.console import Console
from rich.panel import Panel
# Constants
SETTINGS_FILE = 'batch/tasks_setting.xlsx'
INPUT_FOLDER = os.path.join('batch', 'input')
VALID_DUBBING_VALUES = [0, 1]
console = Console()
def check_settings():
os.makedirs(INPUT_FOLDER, exist_ok=True)
df = pd.read_excel(SETTINGS_FILE)
input_files = set(os.listdir(INPUT_FOLDER))
excel_files = set(df['Video File'].tolist())
files_not_in_excel = input_files - excel_files
all_passed = True
local_video_tasks = 0
url_tasks = 0
if files_not_in_excel:
console.print(Panel(
"\n".join([f"- {file}" for file in files_not_in_excel]),
title="[bold red]Warning: Files in input folder not mentioned in Excel sheet",
expand=False
))
all_passed = False
for index, row in df.iterrows():
video_file = row['Video File']
source_language = row['Source Language']
dubbing = row['Dubbing']
if video_file.startswith('http'):
url_tasks += 1
elif os.path.isfile(os.path.join(INPUT_FOLDER, video_file)):
local_video_tasks += 1
else:
console.print(Panel(f"Invalid video file or URL 「{video_file}」", title=f"[bold red]Error in row {index + 2}", expand=False))
all_passed = False
if not pd.isna(dubbing):
if int(dubbing) not in VALID_DUBBING_VALUES:
console.print(Panel(f"Invalid dubbing value 「{dubbing}」", title=f"[bold red]Error in row {index + 2}", expand=False))
all_passed = False
if all_passed:
console.print(Panel(f"✅ All settings passed the check!\nDetected {local_video_tasks} local video tasks and {url_tasks} URL tasks.", title="[bold green]Success", expand=False))
return all_passed
if __name__ == "__main__":
check_settings()
================================================
FILE: batch/utils/video_processor.py
================================================
import os
from core.st_utils.imports_and_utils import *
from core.utils.onekeycleanup import cleanup
from core.utils import load_key
import shutil
from functools import partial
from rich.panel import Panel
from rich.console import Console
from core import *
console = Console()
INPUT_DIR = 'batch/input'
OUTPUT_DIR = 'output'
SAVE_DIR = 'batch/output'
ERROR_OUTPUT_DIR = 'batch/output/ERROR'
YTB_RESOLUTION_KEY = "ytb_resolution"
def process_video(file, dubbing=False, is_retry=False):
if not is_retry:
prepare_output_folder(OUTPUT_DIR)
text_steps = [
("🎥 Processing input file", partial(process_input_file, file)),
("🎙️ Transcribing with Whisper", partial(_2_asr.transcribe)),
("✂️ Splitting sentences", split_sentences),
("📝 Summarizing and translating", summarize_and_translate),
("⚡ Processing and aligning subtitles", process_and_align_subtitles),
("🎬 Merging subtitles to video", _7_sub_into_vid.merge_subtitles_to_video),
]
if dubbing:
dubbing_steps = [
("🔊 Generating audio tasks", gen_audio_tasks),
("🎵 Extracting reference audio", _9_refer_audio.extract_refer_audio_main),
("🗣️ Generating audio", _10_gen_audio.gen_audio),
("🔄 Merging full audio", _11_merge_audio.merge_full_audio),
("🎞️ Merging dubbing to video", _12_dub_to_vid.merge_video_audio),
]
text_steps.extend(dubbing_steps)
current_step = ""
for step_name, step_func in text_steps:
current_step = step_name
for attempt in range(3):
try:
console.print(Panel(
f"[bold green]{step_name}[/]",
subtitle=f"Attempt {attempt + 1}/3" if attempt > 0 else None,
border_style="blue"
))
result = step_func()
if result is not None:
globals().update(result)
break
except Exception as e:
if attempt == 2:
error_panel = Panel(
f"[bold red]Error in step '{current_step}':[/]\n{str(e)}",
border_style="red"
)
console.print(error_panel)
cleanup(ERROR_OUTPUT_DIR)
return False, current_step, str(e)
console.print(Panel(
f"[yellow]Attempt {attempt + 1} failed. Retrying...[/]",
border_style="yellow"
))
console.print(Panel("[bold green]All steps completed successfully! 🎉[/]", border_style="green"))
cleanup(SAVE_DIR)
return True, "", ""
def prepare_output_folder(output_folder):
if os.path.exists(output_folder):
shutil.rmtree(output_folder)
os.makedirs(output_folder)
def process_input_file(file):
if file.startswith('http'):
_1_ytdlp.download_video_ytdlp(file, resolution=load_key(YTB_RESOLUTION_KEY))
video_file = _1_ytdlp.find_video_files()
else:
input_file = os.path.join('batch', 'input', file)
output_file = os.path.join(OUTPUT_DIR, file)
shutil.copy(input_file, output_file)
video_file = output_file
return {'video_file': video_file}
def split_sentences():
_3_1_split_nlp.split_by_spacy()
_3_2_split_meaning.split_sentences_by_meaning()
def summarize_and_translate():
_4_1_summarize.get_summary()
_4_2_translate.translate_all()
def process_and_align_subtitles():
_5_split_sub.split_for_sub_main()
_6_gen_sub.align_timestamp_main()
def gen_audio_tasks():
_8_1_audio_task.gen_audio_task_main()
_8_2_dub_chunks.gen_dub_chunks()
================================================
FILE: config.yaml
================================================
# * Settings marked with * are advanced settings that won't appear in the Streamlit page and can only be modified manually in config.py
# recommend to set in streamlit page
# -------------------
# version: "3.0.0"
# author: "Huanshere"
# -------------------
## ======================== Basic Settings ======================== ##
display_language: "zh-CN"
# API settings
api:
key: 'your-api-key'
base_url: 'https://yunwu.ai'
model: 'gpt-4.1-2025-04-14'
llm_support_json: false
# *Number of LLM multi-threaded accesses, set to 1 if using local LLM
max_workers: 4
# Language settings, written into the prompt, can be described in natural language
target_language: '简体中文'
# Whether to use Demucs for vocal separation before transcription
demucs: true
whisper:
# ["large-v3", "large-v3-turbo"]. Note: for zh model will force to use Belle/large-v3
model: 'large-v3'
# Whisper specified recognition language ISO 639-1
language: 'en'
detected_language: 'en'
# Whisper running mode ["local", "cloud", "elevenlabs"]. Specifies where to run, cloud uses 302.ai API
runtime: 'local'
# 302.ai API key
whisperX_302_api_key: 'your_302_api_key'
# ElevenLabs API key (experimental)
elevenlabs_api_key: 'your_elevenlabs_api_key'
# Whether to burn subtitles into the video
burn_subtitles: true
## ======================== Advanced Settings ======================== ##
# *🔬 h264_nvenc GPU acceleration for ffmpeg, make sure your GPU supports it
ffmpeg_gpu: false
# *Youtube settings
youtube:
cookies_path: ''
# *Default resolution for downloading YouTube videos [360, 1080, best]
ytb_resolution: '1080'
subtitle:
# *Maximum length of each subtitle line in characters
max_length: 75
# *Translated subtitles are slightly larger than source subtitles, affecting the reference length for subtitle splitting
target_multiplier: 1.2
# *Summary length, set low to 2k if using local LLM
summary_length: 8000
# *Maximum number of words for the first rough cut, below 18 will cut too finely affecting translation, above 22 is too long and will make subsequent subtitle splitting difficult to align
max_split_length: 20
# *Whether to reflect the translation result in the original text
reflect_translate: true
# *Whether to pause after extracting professional terms and before translation, allowing users to manually adjust the terminology table output\log\terminology.json
pause_before_translate: false
## ======================== Dubbing Settings ======================== ##
# TTS selection [sf_fish_tts, openai_tts, gpt_sovits, azure_tts, fish_tts, edge_tts, custom_tts]
tts_method: 'azure_tts'
# SiliconFlow FishTTS
sf_fish_tts:
# SiliconFlow API key
api_key: 'YOUR_API_KEY'
# only for mode "preset"
voice: 'anna'
# *only for mode "custom", dont set manually
custom_name: ''
voice_id: ''
# preset, custom, dynamic
mode: "preset"
# OpenAI TTS-1 API configuration, 302.ai API only
openai_tts:
api_key: 'YOUR_302_API_KEY'
voice: 'alloy'
# Azure configuration, 302.ai API only
azure_tts:
api_key: 'YOUR_302_API_KEY'
voice: 'zh-CN-YunfengNeural'
# FishTTS configuration, 302.ai API only
fish_tts:
api_key: 'YOUR_302_API_KEY'
character: 'AD学姐'
character_id_dict:
'AD学姐': '7f92f8afb8ec43bf81429cc1c9199cb1'
'丁真': '54a5170264694bfc8e9ad98df7bd89c3'
# SiliconFlow CosyVoice2 Clone
sf_cosyvoice2:
api_key: 'YOUR_SF_KEY'
# Edge TTS configuration
edge_tts:
voice: 'zh-CN-XiaoxiaoNeural'
# SoVITS configuration
gpt_sovits:
character: 'Huanyuv2'
refer_mode: 3
f5tts:
302_api: 'YOUR_302_API_KEY'
# *Audio speed range
speed_factor:
min: 1
accept: 1.2 # Maximum acceptable speed
max: 1.4
# *Merge audio configuration
min_subtitle_duration: 2.5 # Minimum subtitle duration, will be forcibly extended
min_trim_duration: 3.5 # Subtitles shorter than this value won't be split
tolerance: 1.5 # Allowed extension time to the next subtitle
## ======================== Additional settings ======================== ##
# Whisper model directory
model_dir: './_model_cache'
# Supported upload video formats
allowed_video_formats:
- 'mp4'
- 'mov'
- 'avi'
- 'mkv'
- 'flv'
- 'wmv'
- 'webm'
allowed_audio_formats:
- 'wav'
- 'mp3'
- 'flac'
- 'm4a'
# Spacy models
spacy_model_map:
en: 'en_core_web_md'
ru: 'ru_core_news_md'
fr: 'fr_core_news_md'
ja: 'ja_core_news_md'
es: 'es_core_news_md'
de: 'de_core_news_md'
it: 'it_core_news_md'
zh: 'zh_core_web_md'
# Languages that use space as separator
language_split_with_space:
- 'en'
- 'es'
- 'fr'
- 'de'
- 'it'
- 'ru'
# Languages that do not use space as separator
language_split_without_space:
- 'zh'
- 'ja'
================================================
FILE: core/_10_gen_audio.py
================================================
import os
import time
import shutil
import subprocess
from typing import Tuple
import pandas as pd
from pydub import AudioSegment
from rich.console import Console
from rich.progress import Progress
from concurrent.futures import ThreadPoolExecutor, as_completed
from core.utils import *
from core.utils.models import *
from core.asr_backend.audio_preprocess import get_audio_duration
from core.tts_backend.tts_main import tts_main
console = Console()
TEMP_FILE_TEMPLATE = f"{_AUDIO_TMP_DIR}/{{}}_temp.wav"
OUTPUT_FILE_TEMPLATE = f"{_AUDIO_SEGS_DIR}/{{}}.wav"
WARMUP_SIZE = 5
def parse_df_srt_time(time_str: str) -> float:
"""Convert SRT time format to seconds"""
hours, minutes, seconds = time_str.strip().split(':')
seconds, milliseconds = seconds.split('.')
return int(hours) * 3600 + int(minutes) * 60 + int(seconds) + int(milliseconds) / 1000
def adjust_audio_speed(input_file: str, output_file: str, speed_factor: float) -> None:
"""Adjust audio speed and handle edge cases"""
# If the speed factor is close to 1, directly copy the file
if abs(speed_factor - 1.0) < 0.001:
shutil.copy2(input_file, output_file)
return
atempo = speed_factor
cmd = ['ffmpeg', '-i', input_file, '-filter:a', f'atempo={atempo}', '-y', output_file]
input_duration = get_audio_duration(input_file)
max_retries = 2
for attempt in range(max_retries):
try:
subprocess.run(cmd, check=True, stderr=subprocess.PIPE)
output_duration = get_audio_duration(output_file)
expected_duration = input_duration / speed_factor
diff = output_duration - expected_duration
# If the output duration exceeds the expected duration, but the input audio is less than 3 seconds, and the error is within 0.1 seconds, truncate to the expected length
if output_duration >= expected_duration * 1.02 and input_duration < 3 and diff <= 0.1:
audio = AudioSegment.from_wav(output_file)
trimmed_audio = audio[:(expected_duration * 1000)] # pydub uses milliseconds
trimmed_audio.export(output_file, format="wav")
print(f"✂️ Trimmed to expected duration: {expected_duration:.2f} seconds")
return
elif output_duration >= expected_duration * 1.02:
raise Exception(f"Audio duration abnormal: input file={input_file}, output file={output_file}, speed factor={speed_factor}, input duration={input_duration:.2f}s, output duration={output_duration:.2f}s")
return
except subprocess.CalledProcessError as e:
if attempt < max_retries - 1:
rprint(f"[yellow]⚠️ Audio speed adjustment failed, retrying in 1s ({attempt + 1}/{max_retries})[/yellow]")
time.sleep(1)
else:
rprint(f"[red]❌ Audio speed adjustment failed, max retries reached ({max_retries})[/red]")
raise e
def process_row(row: pd.Series, tasks_df: pd.DataFrame) -> Tuple[int, float]:
"""Helper function for processing single row data"""
number = row['number']
lines = eval(row['lines']) if isinstance(row['lines'], str) else row['lines']
real_dur = 0
for line_index, line in enumerate(lines):
temp_file = TEMP_FILE_TEMPLATE.format(f"{number}_{line_index}")
tts_main(line, temp_file, number, tasks_df)
real_dur += get_audio_duration(temp_file)
return number, real_dur
def generate_tts_audio(tasks_df: pd.DataFrame) -> pd.DataFrame:
"""Generate TTS audio sequentially and calculate actual duration"""
tasks_df['real_dur'] = 0
rprint("[bold green]🎯 Starting TTS audio generation...[/bold green]")
with Progress() as progress:
task = progress.add_task("[cyan]🔄 Generating TTS audio...", total=len(tasks_df))
# warm up for first 5 rows
warmup_size = min(WARMUP_SIZE, len(tasks_df))
for _, row in tasks_df.head(warmup_size).iterrows():
try:
number, real_dur = process_row(row, tasks_df)
tasks_df.loc[tasks_df['number'] == number, 'real_dur'] = real_dur
progress.advance(task)
except Exception as e:
rprint(f"[red]❌ Error in warmup: {str(e)}[/red]")
raise e
# for gpt_sovits, do not use parallel to avoid mistakes
max_workers = load_key("max_workers") if load_key("tts_method") != "gpt_sovits" else 1
# parallel processing for remaining tasks
if len(tasks_df) > warmup_size:
remaining_tasks = tasks_df.iloc[warmup_size:].copy()
with ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = [
executor.submit(process_row, row, tasks_df.copy())
for _, row in remaining_tasks.iterrows()
]
for future in as_completed(futures):
try:
number, real_dur = future.result()
tasks_df.loc[tasks_df['number'] == number, 'real_dur'] = real_dur
progress.advance(task)
except Exception as e:
rprint(f"[red]❌ Error: {str(e)}[/red]")
raise e
rprint("[bold green]✨ TTS audio generation completed![/bold green]")
return tasks_df
def process_chunk(chunk_df: pd.DataFrame, accept: float, min_speed: float) -> tuple[float, bool]:
"""Process audio chunk and calculate speed factor"""
chunk_durs = chunk_df['real_dur'].sum()
tol_durs = chunk_df['tol_dur'].sum()
durations = tol_durs - chunk_df.iloc[-1]['tolerance']
all_gaps = chunk_df['gap'].sum() - chunk_df.iloc[-1]['gap']
keep_gaps = True
speed_var_error = 0.1
if (chunk_durs + all_gaps) / accept < durations:
speed_factor = max(min_speed, (chunk_durs + all_gaps) / (durations-speed_var_error))
elif chunk_durs / accept < durations:
speed_factor = max(min_speed, chunk_durs / (durations-speed_var_error))
keep_gaps = False
elif (chunk_durs + all_gaps) / accept < tol_durs:
speed_factor = max(min_speed, (chunk_durs + all_gaps) / (tol_durs-speed_var_error))
else:
speed_factor = chunk_durs / (tol_durs-speed_var_error)
keep_gaps = False
return round(speed_factor, 3), keep_gaps
def merge_chunks(tasks_df: pd.DataFrame) -> pd.DataFrame:
"""Merge audio chunks and adjust timeline"""
rprint("[bold blue]🔄 Starting audio chunks processing...[/bold blue]")
accept = load_key("speed_factor.accept")
min_speed = load_key("speed_factor.min")
chunk_start = 0
tasks_df['new_sub_times'] = None
for index, row in tasks_df.iterrows():
if row['cut_off'] == 1:
chunk_df = tasks_df.iloc[chunk_start:index+1].reset_index(drop=True)
speed_factor, keep_gaps = process_chunk(chunk_df, accept, min_speed)
# 🎯 Step1: Start processing new timeline
chunk_start_time = parse_df_srt_time(chunk_df.iloc[0]['start_time'])
chunk_end_time = parse_df_srt_time(chunk_df.iloc[-1]['end_time']) + chunk_df.iloc[-1]['tolerance'] # 加上tolerance才是这一块的结束
cur_time = chunk_start_time
for i, row in chunk_df.iterrows():
# If i is not 0, which is not the first row of the chunk, cur_time needs to be added with the gap of the previous row, remember to divide by speed_factor
if i != 0 and keep_gaps:
cur_time += chunk_df.iloc[i-1]['gap']/speed_factor
new_sub_times = []
number = row['number']
lines = eval(row['lines']) if isinstance(row['lines'], str) else row['lines']
for line_index, line in enumerate(lines):
# 🔄 Step2: Start speed change and save as OUTPUT_FILE_TEMPLATE
temp_file = TEMP_FILE_TEMPLATE.format(f"{number}_{line_index}")
output_file = OUTPUT_FILE_TEMPLATE.format(f"{number}_{line_index}")
adjust_audio_speed(temp_file, output_file, speed_factor)
ad_dur = get_audio_duration(output_file)
new_sub_times.append([cur_time, cur_time+ad_dur])
cur_time += ad_dur
# 🔄 Step3: Find corresponding main DataFrame index and update new_sub_times
main_df_idx = tasks_df[tasks_df['number'] == row['number']].index[0]
tasks_df.at[main_df_idx, 'new_sub_times'] = new_sub_times
# 🎯 Step4: Choose emoji based on speed_factor and accept comparison
emoji = "⚡" if speed_factor <= accept else "⚠️"
rprint(f"[cyan]{emoji} Processed chunk {chunk_start} to {index} with speed factor {speed_factor}[/cyan]")
# 🔄 Step5: Check if the last row exceeds the range
if cur_time > chunk_end_time:
time_diff = cur_time - chunk_end_time
if time_diff <= 0.6: # If exceeding time is within 0.6 seconds, truncate the last audio
rprint(f"[yellow]⚠️ Chunk {chunk_start} to {index} exceeds by {time_diff:.3f}s, truncating last audio[/yellow]")
# Get the last audio file
last_number = tasks_df.iloc[index]['number']
last_lines = eval(tasks_df.iloc[index]['lines']) if isinstance(tasks_df.iloc[index]['lines'], str) else tasks_df.iloc[index]['lines']
last_line_index = len(last_lines) - 1
last_file = OUTPUT_FILE_TEMPLATE.format(f"{last_number}_{last_line_index}")
# Calculate the duration to keep
audio = AudioSegment.from_wav(last_file)
original_duration = len(audio) / 1000 # Convert to seconds
new_duration = original_duration - time_diff
trimmed_audio = audio[:(new_duration * 1000)] # pydub uses milliseconds
trimmed_audio.export(last_file, format="wav")
# Update the last timestamp
last_times = tasks_df.at[index, 'new_sub_times']
last_times[-1][1] = chunk_end_time
tasks_df.at[index, 'new_sub_times'] = last_times
else:
raise Exception(f"Chunk {chunk_start} to {index} exceeds the chunk end time {chunk_end_time:.2f} seconds with current time {cur_time:.2f} seconds")
chunk_start = index+1
rprint("[bold green]✅ Audio chunks processing completed![/bold green]")
return tasks_df
def gen_audio() -> None:
"""Main function: Generate audio and process timeline"""
rprint("[bold magenta]🚀 Starting audio generation process...[/bold magenta]")
# 🎯 Step1: Create necessary directories
os.makedirs(_AUDIO_TMP_DIR, exist_ok=True)
os.makedirs(_AUDIO_SEGS_DIR, exist_ok=True)
# 📝 Step2: Load task file
tasks_df = pd.read_excel(_8_1_AUDIO_TASK)
rprint("[green]📊 Loaded task file successfully[/green]")
# 🔊 Step3: Generate TTS audio
tasks_df = generate_tts_audio(tasks_df)
# 🔄 Step4: Merge audio chunks
tasks_df = merge_chunks(tasks_df)
# 💾 Step5: Save results
tasks_df.to_excel(_8_1_AUDIO_TASK, index=False)
rprint("[bold green]🎉 Audio generation completed successfully![/bold green]")
if __name__ == "__main__":
gen_audio()
================================================
FILE: core/_11_merge_audio.py
================================================
import os
import pandas as pd
import subprocess
from pydub import AudioSegment
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TaskProgressColumn
from rich.console import Console
from core.utils import *
from core.utils.models import *
console = Console()
DUB_VOCAL_FILE = 'output/dub.mp3'
DUB_SUB_FILE = 'output/dub.srt'
OUTPUT_FILE_TEMPLATE = f"{_AUDIO_SEGS_DIR}/{{}}.wav"
def load_and_flatten_data(excel_file):
"""Load and flatten Excel data"""
df = pd.read_excel(excel_file)
lines = [eval(line) if isinstance(line, str) else line for line in df['lines'].tolist()]
lines = [item for sublist in lines for item in sublist]
new_sub_times = [eval(time) if isinstance(time, str) else time for time in df['new_sub_times'].tolist()]
new_sub_times = [item for sublist in new_sub_times for item in sublist]
return df, lines, new_sub_times
def get_audio_files(df):
"""Generate a list of audio file paths"""
audios = []
for index, row in df.iterrows():
number = row['number']
line_count = len(eval(row['lines']) if isinstance(row['lines'], str) else row['lines'])
for line_index in range(line_count):
temp_file = OUTPUT_FILE_TEMPLATE.format(f"{number}_{line_index}")
audios.append(temp_file)
return audios
def process_audio_segment(audio_file):
"""Process a single audio segment with MP3 compression"""
temp_file = f"{audio_file}_temp.mp3"
ffmpeg_cmd = [
'ffmpeg', '-y',
'-i', audio_file,
'-ar', '16000',
'-ac', '1',
'-b:a', '64k',
temp_file
]
subprocess.run(ffmpeg_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
audio_segment = AudioSegment.from_mp3(temp_file)
os.remove(temp_file)
return audio_segment
def merge_audio_segments(audios, new_sub_times, sample_rate):
merged_audio = AudioSegment.silent(duration=0, frame_rate=sample_rate)
with Progress(SpinnerColumn(), TextColumn("[progress.description]{task.description}"), BarColumn(), TaskProgressColumn()) as progress:
merge_task = progress.add_task("🎵 Merging audio segments...", total=len(audios))
for i, (audio_file, time_range) in enumerate(zip(audios, new_sub_times)):
if not os.path.exists(audio_file):
console.print(f"[bold yellow]⚠️ Warning: File {audio_file} does not exist, skipping...[/bold yellow]")
progress.advance(merge_task)
continue
audio_segment = process_audio_segment(audio_file)
start_time, end_time = time_range
# Add silence segment
if i > 0:
prev_end = new_sub_times[i-1][1]
silence_duration = start_time - prev_end
if silence_duration > 0:
silence = AudioSegment.silent(duration=int(silence_duration * 1000), frame_rate=sample_rate)
merged_audio += silence
elif start_time > 0:
silence = AudioSegment.silent(duration=int(start_time * 1000), frame_rate=sample_rate)
merged_audio += silence
merged_audio += audio_segment
progress.advance(merge_task)
return merged_audio
def create_srt_subtitle():
df, lines, new_sub_times = load_and_flatten_data(_8_1_AUDIO_TASK)
with open(DUB_SUB_FILE, 'w', encoding='utf-8') as f:
for i, ((start_time, end_time), line) in enumerate(zip(new_sub_times, lines), 1):
start_str = f"{int(start_time//3600):02d}:{int((start_time%3600)//60):02d}:{int(start_time%60):02d},{int((start_time*1000)%1000):03d}"
end_str = f"{int(end_time//3600):02d}:{int((end_time%3600)//60):02d}:{int(end_time%60):02d},{int((end_time*1000)%1000):03d}"
f.write(f"{i}\n")
f.write(f"{start_str} --> {end_str}\n")
f.write(f"{line}\n\n")
rprint(f"[bold green]✅ Subtitle file created: {DUB_SUB_FILE}[/bold green]")
def merge_full_audio():
"""Main function: Process the complete audio merging process"""
console.print("\n[bold cyan]🎬 Starting audio merging process...[/bold cyan]")
with console.status("[bold cyan]📊 Loading data from Excel...[/bold cyan]"):
df, lines, new_sub_times = load_and_flatten_data(_8_1_AUDIO_TASK)
console.print("[bold green]✅ Data loaded successfully[/bold green]")
with console.status("[bold cyan]🔍 Getting audio file list...[/bold cyan]"):
audios = get_audio_files(df)
console.print(f"[bold green]✅ Found {len(audios)} audio segments[/bold green]")
with console.status("[bold cyan]📝 Generating subtitle file...[/bold cyan]"):
create_srt_subtitle()
if not os.path.exists(audios[0]):
console.print(f"[bold red]❌ Error: First audio file {audios[0]} does not exist![/bold red]")
return
sample_rate = 16000
console.print(f"[bold green]✅ Sample rate: {sample_rate}Hz[/bold green]")
console.print("[bold cyan]🔄 Starting audio merge process...[/bold cyan]")
merged_audio = merge_audio_segments(audios, new_sub_times, sample_rate)
with console.status("[bold cyan]💾 Exporting final audio file...[/bold cyan]"):
merged_audio = merged_audio.set_frame_rate(16000).set_channels(1)
merged_audio.export(DUB_VOCAL_FILE, format="mp3", parameters=["-b:a", "64k"])
console.print(f"[bold green]✅ Audio file successfully merged![/bold green]")
console.print(f"[bold green]📁 Output file: {DUB_VOCAL_FILE}[/bold green]")
if __name__ == "__main__":
merge_full_audio()
================================================
FILE: core/_12_dub_to_vid.py
================================================
import platform
import subprocess
import cv2
import numpy as np
from rich.console import Console
from core._1_ytdlp import find_video_files
from core.asr_backend.audio_preprocess import normalize_audio_volume
from core.utils import *
from core.utils.models import *
console = Console()
DUB_VIDEO = "output/output_dub.mp4"
DUB_SUB_FILE = 'output/dub.srt'
DUB_AUDIO = 'output/dub.mp3'
TRANS_FONT_SIZE = 17
TRANS_FONT_NAME = 'Arial'
if platform.system() == 'Linux':
TRANS_FONT_NAME = 'NotoSansCJK-Regular'
if platform.system() == 'Darwin':
TRANS_FONT_NAME = 'Arial Unicode MS'
TRANS_FONT_COLOR = '&H00FFFF'
TRANS_OUTLINE_COLOR = '&H000000'
TRANS_OUTLINE_WIDTH = 1
TRANS_BACK_COLOR = '&H33000000'
def merge_video_audio():
"""Merge video and audio, and reduce video volume"""
VIDEO_FILE = find_video_files()
background_file = _BACKGROUND_AUDIO_FILE
if not load_key("burn_subtitles"):
rprint("[bold yellow]Warning: A 0-second black video will be generated as a placeholder as subtitles are not burned in.[/bold yellow]")
# Create a black frame
frame = np.zeros((1080, 1920, 3), dtype=np.uint8)
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(DUB_VIDEO, fourcc, 1, (1920, 1080))
out.write(frame)
out.release()
rprint("[bold green]Placeholder video has been generated.[/bold green]")
return
# Normalize dub audio
normalized_dub_audio = 'output/normalized_dub.wav'
normalize_audio_volume(DUB_AUDIO, normalized_dub_audio)
# Merge video and audio with translated subtitles
video = cv2.VideoCapture(VIDEO_FILE)
TARGET_WIDTH = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
TARGET_HEIGHT = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
video.release()
rprint(f"[bold green]Video resolution: {TARGET_WIDTH}x{TARGET_HEIGHT}[/bold green]")
subtitle_filter = (
f"subtitles={DUB_SUB_FILE}:force_style='FontSize={TRANS_FONT_SIZE},"
f"FontName={TRANS_FONT_NAME},PrimaryColour={TRANS_FONT_COLOR},"
f"OutlineColour={TRANS_OUTLINE_COLOR},OutlineWidth={TRANS_OUTLINE_WIDTH},"
f"BackColour={TRANS_BACK_COLOR},Alignment=2,MarginV=27,BorderStyle=4'"
)
cmd = [
'ffmpeg', '-y', '-i', VIDEO_FILE, '-i', background_file, '-i', normalized_dub_audio,
'-filter_complex',
f'[0:v]scale={TARGET_WIDTH}:{TARGET_HEIGHT}:force_original_aspect_ratio=decrease,'
f'pad={TARGET_WIDTH}:{TARGET_HEIGHT}:(ow-iw)/2:(oh-ih)/2,'
f'{subtitle_filter}[v];'
f'[1:a][2:a]amix=inputs=2:duration=first:dropout_transition=3[a]'
]
if load_key("ffmpeg_gpu"):
rprint("[bold green]Using GPU acceleration...[/bold green]")
cmd.extend(['-map', '[v]', '-map', '[a]', '-c:v', 'h264_nvenc'])
else:
cmd.extend(['-map', '[v]', '-map', '[a]'])
cmd.extend(['-c:a', 'aac', '-b:a', '96k', DUB_VIDEO])
subprocess.run(cmd)
rprint(f"[bold green]Video and audio successfully merged into {DUB_VIDEO}[/bold green]")
if __name__ == '__main__':
merge_video_audio()
================================================
FILE: core/_1_ytdlp.py
================================================
import os,sys
import glob
import re
import subprocess
from core.utils import *
def sanitize_filename(filename):
# Remove or replace illegal characters
filename = re.sub(r'[<>:"/\\|?*]', '', filename)
# Ensure filename doesn't start or end with a dot or space
filename = filename.strip('. ')
# Use default name if filename is empty
return filename if filename else 'video'
def update_ytdlp():
try:
subprocess.check_call([sys.executable, "-m", "pip", "install", "--upgrade", "yt-dlp"])
if 'yt_dlp' in sys.modules:
del sys.modules['yt_dlp']
rprint("[green]yt-dlp updated[/green]")
except subprocess.CalledProcessError as e:
rprint("[yellow]Warning: Failed to update yt-dlp: {e}[/yellow]")
from yt_dlp import YoutubeDL
return YoutubeDL
def download_video_ytdlp(url, save_path='output', resolution='1080'):
os.makedirs(save_path, exist_ok=True)
ydl_opts = {
'format': 'bestvideo+bestaudio/best' if resolution == 'best' else f'bestvideo[height<={resolution}]+bestaudio/best[height<={resolution}]',
'outtmpl': f'{save_path}/%(title)s.%(ext)s',
'noplaylist': True,
'writethumbnail': True,
'postprocessors': [{'key': 'FFmpegThumbnailsConvertor', 'format': 'jpg'}],
}
# Read Youtube Cookie File
cookies_path = load_key("youtube.cookies_path")
if os.path.exists(cookies_path):
ydl_opts["cookiefile"] = str(cookies_path)
# Get YoutubeDL class after updating
YoutubeDL = update_ytdlp()
with YoutubeDL(ydl_opts) as ydl:
ydl.download([url])
# Check and rename files after download
for file in os.listdir(save_path):
if os.path.isfile(os.path.join(save_path, file)):
filename, ext = os.path.splitext(file)
new_filename = sanitize_filename(filename)
if new_filename != filename:
os.rename(os.path.join(save_path, file), os.path.join(save_path, new_filename + ext))
def find_video_files(save_path='output'):
video_files = [file for file in glob.glob(save_path + "/*") if os.path.splitext(file)[1][1:].lower() in load_key("allowed_video_formats")]
# change \\ to /, this happen on windows
if sys.platform.startswith('win'):
video_files = [file.replace("\\", "/") for file in video_files]
video_files = [file for file in video_files if not file.startswith("output/output")]
if len(video_files) != 1:
raise ValueError(f"Number of videos found {len(video_files)} is not unique. Please check.")
return video_files[0]
if __name__ == '__main__':
# Example usage
url = input('Please enter the URL of the video you want to download: ')
resolution = input('Please enter the desired resolution (360/480/720/1080, default 1080): ')
resolution = int(resolution) if resolution.isdigit() else 1080
download_video_ytdlp(url, resolution=resolution)
print(f"🎥 Video has been downloaded to {find_video_files()}")
================================================
FILE: core/_2_asr.py
================================================
from core.utils import *
from core.asr_backend.demucs_vl import demucs_audio
from core.asr_backend.audio_preprocess import process_transcription, convert_video_to_audio, split_audio, save_results, normalize_audio_volume
from core._1_ytdlp import find_video_files
from core.utils.models import *
@check_file_exists(_2_CLEANED_CHUNKS)
def transcribe():
# 1. video to audio
video_file = find_video_files()
convert_video_to_audio(video_file)
# 2. Demucs vocal separation:
if load_key("demucs"):
demucs_audio()
vocal_audio = normalize_audio_volume(_VOCAL_AUDIO_FILE, _VOCAL_AUDIO_FILE, format="mp3")
else:
vocal_audio = _RAW_AUDIO_FILE
# 3. Extract audio
segments = split_audio(_RAW_AUDIO_FILE)
# 4. Transcribe audio by clips
all_results = []
runtime = load_key("whisper.runtime")
if runtime == "local":
from core.asr_backend.whisperX_local import transcribe_audio as ts
rprint("[cyan]🎤 Transcribing audio with local model...[/cyan]")
elif runtime == "cloud":
from core.asr_backend.whisperX_302 import transcribe_audio_302 as ts
rprint("[cyan]🎤 Transcribing audio with 302 API...[/cyan]")
elif runtime == "elevenlabs":
from core.asr_backend.elevenlabs_asr import transcribe_audio_elevenlabs as ts
rprint("[cyan]🎤 Transcribing audio with ElevenLabs API...[/cyan]")
for start, end in segments:
result = ts(_RAW_AUDIO_FILE, vocal_audio, start, end)
all_results.append(result)
# 5. Combine results
combined_result = {'segments': []}
for result in all_results:
combined_result['segments'].extend(result['segments'])
# 6. Process df
df = process_transcription(combined_result)
save_results(df)
if __name__ == "__main__":
transcribe()
================================================
FILE: core/_3_1_split_nlp.py
================================================
from core.spacy_utils import *
from core.utils.models import _3_1_SPLIT_BY_NLP
from core.utils import check_file_exists
@check_file_exists(_3_1_SPLIT_BY_NLP)
def split_by_spacy():
nlp = init_nlp()
split_by_mark(nlp)
split_by_comma_main(nlp)
split_sentences_main(nlp)
split_long_by_root_main(nlp)
return
if __name__ == '__main__':
split_by_spacy()
================================================
FILE: core/_3_2_split_meaning.py
================================================
import concurrent.futures
from difflib import SequenceMatcher
import math
from core.prompts import get_split_prompt
from core.spacy_utils.load_nlp_model import init_nlp
from core.utils import *
from rich.console import Console
from rich.table import Table
from core.utils.models import _3_1_SPLIT_BY_NLP, _3_2_SPLIT_BY_MEANING
console = Console()
def tokenize_sentence(sentence, nlp):
doc = nlp(sentence)
return [token.text for token in doc]
def find_split_positions(original, modified):
split_positions = []
parts = modified.split('[br]')
start = 0
whisper_language = load_key("whisper.language")
language = load_key("whisper.detected_language") if whisper_language == 'auto' else whisper_language
joiner = get_joiner(language)
for i in range(len(parts) - 1):
max_similarity = 0
best_split = None
for j in range(start, len(original)):
original_left = original[start:j]
modified_left = joiner.join(parts[i].split())
left_similarity = SequenceMatcher(None, original_left, modified_left).ratio()
if left_similarity > max_similarity:
max_similarity = left_similarity
best_split = j
if max_similarity < 0.9:
console.print(f"[yellow]Warning: low similarity found at the best split point: {max_similarity}[/yellow]")
if best_split is not None:
split_positions.append(best_split)
start = best_split
else:
console.print(f"[yellow]Warning: Unable to find a suitable split point for the {i+1}th part.[/yellow]")
return split_positions
def split_sentence(sentence, num_parts, word_limit=20, index=-1, retry_attempt=0):
"""Split a long sentence using GPT and return the result as a string."""
split_prompt = get_split_prompt(sentence, num_parts, word_limit)
def valid_split(response_data):
choice = response_data["choice"]
if f'split{choice}' not in response_data:
return {"status": "error", "message": "Missing required key: `split`"}
if "[br]" not in response_data[f"split{choice}"]:
return {"status": "error", "message": "Split failed, no [br] found"}
return {"status": "success", "message": "Split completed"}
response_data = ask_gpt(split_prompt + " " * retry_attempt, resp_type='json', valid_def=valid_split, log_title='split_by_meaning')
choice = response_data["choice"]
best_split = response_data[f"split{choice}"]
split_points = find_split_positions(sentence, best_split)
# split the sentence based on the split points
for i, split_point in enumerate(split_points):
if i == 0:
best_split = sentence[:split_point] + '\n' + sentence[split_point:]
else:
parts = best_split.split('\n')
last_part = parts[-1]
parts[-1] = last_part[:split_point - split_points[i-1]] + '\n' + last_part[split_point - split_points[i-1]:]
best_split = '\n'.join(parts)
if index != -1:
console.print(f'[green]✅ Sentence {index} has been successfully split[/green]')
table = Table(title="")
table.add_column("Type", style="cyan")
table.add_column("Sentence")
table.add_row("Original", sentence, style="yellow")
table.add_row("Split", best_split.replace('\n', ' ||'), style="yellow")
console.print(table)
return best_split
def parallel_split_sentences(sentences, max_length, max_workers, nlp, retry_attempt=0):
"""Split sentences in parallel using a thread pool."""
new_sentences = [None] * len(sentences)
futures = []
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
for index, sentence in enumerate(sentences):
# Use tokenizer to split the sentence
tokens = tokenize_sentence(sentence, nlp)
# print("Tokenization result:", tokens)
num_parts = math.ceil(len(tokens) / max_length)
if len(tokens) > max_length:
future = executor.submit(split_sentence, sentence, num_parts, max_length, index=index, retry_attempt=retry_attempt)
futures.append((future, index, num_parts, sentence))
else:
new_sentences[index] = [sentence]
for future, index, num_parts, sentence in futures:
split_result = future.result()
if split_result:
split_lines = split_result.strip().split('\n')
new_sentences[index] = [line.strip() for line in split_lines]
else:
new_sentences[index] = [sentence]
return [sentence for sublist in new_sentences for sentence in sublist]
@check_file_exists(_3_2_SPLIT_BY_MEANING)
def split_sentences_by_meaning():
"""The main function to split sentences by meaning."""
# read input sentences
with open(_3_1_SPLIT_BY_NLP, 'r', encoding='utf-8') as f:
sentences = [line.strip() for line in f.readlines()]
nlp = init_nlp()
# 🔄 process sentences multiple times to ensure all are split
for retry_attempt in range(3):
sentences = parallel_split_sentences(sentences, max_length=load_key("max_split_length"), max_workers=load_key("max_workers"), nlp=nlp, retry_attempt=retry_attempt)
# 💾 save results
with open(_3_2_SPLIT_BY_MEANING, 'w', encoding='utf-8') as f:
f.write('\n'.join(sentences))
console.print('[green]✅ All sentences have been successfully split![/green]')
if __name__ == '__main__':
# print(split_sentence('Which makes no sense to the... average guy who always pushes the character creation slider all the way to the right.', 2, 22))
split_sentences_by_meaning()
================================================
FILE: core/_4_1_summarize.py
================================================
import json
from core.prompts import get_summary_prompt
import pandas as pd
from core.utils import *
from core.utils.models import _3_2_SPLIT_BY_MEANING, _4_1_TERMINOLOGY
CUSTOM_TERMS_PATH = 'custom_terms.xlsx'
def combine_chunks():
"""Combine the text chunks identified by whisper into a single long text"""
with open(_3_2_SPLIT_BY_MEANING, 'r', encoding='utf-8') as file:
sentences = file.readlines()
cleaned_sentences = [line.strip() for line in sentences]
combined_text = ' '.join(cleaned_sentences)
return combined_text[:load_key('summary_length')] #! Return only the first x characters
def search_things_to_note_in_prompt(sentence):
"""Search for terms to note in the given sentence"""
with open(_4_1_TERMINOLOGY, 'r', encoding='utf-8') as file:
things_to_note = json.load(file)
things_to_note_list = [term['src'] for term in things_to_note['terms'] if term['src'].lower() in sentence.lower()]
if things_to_note_list:
prompt = '\n'.join(
f'{i+1}. "{term["src"]}": "{term["tgt"]}",'
f' meaning: {term["note"]}'
for i, term in enumerate(things_to_note['terms'])
if term['src'] in things_to_note_list
)
return prompt
else:
return None
def get_summary():
src_content = combine_chunks()
custom_terms = pd.read_excel(CUSTOM_TERMS_PATH)
custom_terms_json = {
"terms":
[
{
"src": str(row.iloc[0]),
"tgt": str(row.iloc[1]),
"note": str(row.iloc[2])
}
for _, row in custom_terms.iterrows()
]
}
if len(custom_terms) > 0:
rprint(f"📖 Custom Terms Loaded: {len(custom_terms)} terms")
rprint("📝 Terms Content:", json.dumps(custom_terms_json, indent=2, ensure_ascii=False))
summary_prompt = get_summary_prompt(src_content, custom_terms_json)
rprint("📝 Summarizing and extracting terminology ...")
def valid_summary(response_data):
required_keys = {'src', 'tgt', 'note'}
if 'terms' not in response_data:
return {"status": "error", "message": "Invalid response format"}
for term in response_data['terms']:
if not all(key in term for key in required_keys):
return {"status": "error", "message": "Invalid response format"}
return {"status": "success", "message": "Summary completed"}
summary = ask_gpt(summary_prompt, resp_type='json', valid_def=valid_summary, log_title='summary')
summary['terms'].extend(custom_terms_json['terms'])
with open(_4_1_TERMINOLOGY, 'w', encoding='utf-8') as f:
json.dump(summary, f, ensure_ascii=False, indent=4)
rprint(f'💾 Summary log saved to → `{_4_1_TERMINOLOGY}`')
if __name__ == '__main__':
get_summary()
================================================
FILE: core/_4_2_translate.py
================================================
import pandas as pd
import json
import concurrent.futures
from core.translate_lines import translate_lines
from core._4_1_summarize import search_things_to_note_in_prompt
from core._8_1_audio_task import check_len_then_trim
from core._6_gen_sub import align_timestamp
from core.utils import *
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn
from difflib import SequenceMatcher
from core.utils.models import *
console = Console()
# Function to split text into chunks
def split_chunks_by_chars(chunk_size, max_i):
"""Split text into chunks based on character count, return a list of multi-line text chunks"""
with open(_3_2_SPLIT_BY_MEANING, "r", encoding="utf-8") as file:
sentences = file.read().strip().split('\n')
chunks = []
chunk = ''
sentence_count = 0
for sentence in sentences:
if len(chunk) + len(sentence + '\n') > chunk_size or sentence_count == max_i:
chunks.append(chunk.strip())
chunk = sentence + '\n'
sentence_count = 1
else:
chunk += sentence + '\n'
sentence_count += 1
chunks.append(chunk.strip())
return chunks
# Get context from surrounding chunks
def get_previous_content(chunks, chunk_index):
return None if chunk_index == 0 else chunks[chunk_index - 1].split('\n')[-3:] # Get last 3 lines
def get_after_content(chunks, chunk_index):
return None if chunk_index == len(chunks) - 1 else chunks[chunk_index + 1].split('\n')[:2] # Get first 2 lines
# 🔍 Translate a single chunk
def translate_chunk(chunk, chunks, theme_prompt, i):
things_to_note_prompt = search_things_to_note_in_prompt(chunk)
previous_content_prompt = get_previous_content(chunks, i)
after_content_prompt = get_after_content(chunks, i)
translation, english_result = translate_lines(chunk, previous_content_prompt, after_content_prompt, things_to_note_prompt, theme_prompt, i)
return i, english_result, translation
# Add similarity calculation function
def similar(a, b):
return SequenceMatcher(None, a, b).ratio()
# 🚀 Main function to translate all chunks
@check_file_exists(_4_2_TRANSLATION)
def translate_all():
console.print("[bold green]Start Translating All...[/bold green]")
chunks = split_chunks_by_chars(chunk_size=600, max_i=10)
with open(_4_1_TERMINOLOGY, 'r', encoding='utf-8') as file:
theme_prompt = json.load(file).get('theme')
# 🔄 Use concurrent execution for translation
with Progress(SpinnerColumn(), TextColumn("[progress.description]{task.description}"), transient=True) as progress:
task = progress.add_task("[cyan]Translating chunks...", total=len(chunks))
with concurrent.futures.ThreadPoolExecutor(max_workers=load_key("max_workers")) as executor:
futures = []
for i, chunk in enumerate(chunks):
future = executor.submit(translate_chunk, chunk, chunks, theme_prompt, i)
futures.append(future)
results = []
for future in concurrent.futures.as_completed(futures):
results.append(future.result())
progress.update(task, advance=1)
results.sort(key=lambda x: x[0]) # Sort results based on original order
# 💾 Save results to lists and Excel file
src_text, trans_text = [], []
for i, chunk in enumerate(chunks):
chunk_lines = chunk.split('\n')
src_text.extend(chunk_lines)
# Calculate similarity between current chunk and translation results
chunk_text = ''.join(chunk_lines).lower()
matching_results = [(r, similar(''.join(r[1].split('\n')).lower(), chunk_text))
for r in results]
best_match = max(matching_results, key=lambda x: x[1])
# Check similarity and handle exceptions
if best_match[1] < 0.9:
console.print(f"[yellow]Warning: No matching translation found for chunk {i}[/yellow]")
raise ValueError(f"Translation matching failed (chunk {i})")
elif best_match[1] < 1.0:
console.print(f"[yellow]Warning: Similar match found (chunk {i}, similarity: {best_match[1]:.3f})[/yellow]")
trans_text.extend(best_match[0][2].split('\n'))
# Trim long translation text
df_text = pd.read_excel(_2_CLEANED_CHUNKS)
df_text['text'] = df_text['text'].str.strip('"').str.strip()
df_translate = pd.DataFrame({'Source': src_text, 'Translation': trans_text})
subtitle_output_configs = [('trans_subs_for_audio.srt', ['Translation'])]
df_time = align_timestamp(df_text, df_translate, subtitle_output_configs, output_dir=None, for_display=False)
console.print(df_time)
# apply check_len_then_trim to df_time['Translation'], only when duration > MIN_TRIM_DURATION.
df_time['Translation'] = df_time.apply(lambda x: check_len_then_trim(x['Translation'], x['duration']) if x['duration'] > load_key("min_trim_duration") else x['Translation'], axis=1)
console.print(df_time)
df_time.to_excel(_4_2_TRANSLATION, index=False)
console.print("[bold green]✅ Translation completed and results saved.[/bold green]")
if __name__ == '__main__':
translate_all()
================================================
FILE: core/_5_split_sub.py
================================================
import pandas as pd
from typing import List, Tuple
import concurrent.futures
from core._3_2_split_meaning import split_sentence
from core.prompts import get_align_prompt
from rich.panel import Panel
from rich.console import Console
from rich.table import Table
from core.utils import *
from core.utils.models import *
console = Console()
# ! You can modify your own weights here
# Chinese and Japanese 2.5 characters, Korean 2 characters, Thai 1.5 characters, full-width symbols 2 characters, other English-based and half-width symbols 1 character
def calc_len(text: str) -> float:
text = str(text) # force convert
def char_weight(char):
code = ord(char)
if 0x4E00 <= code <= 0x9FFF or 0x3040 <= code <= 0x30FF: # Chinese and Japanese
return 1.75
elif 0xAC00 <= code <= 0xD7A3 or 0x1100 <= code <= 0x11FF: # Korean
return 1.5
elif 0x0E00 <= code <= 0x0E7F: # Thai
return 1
elif 0xFF01 <= code <= 0xFF5E: # full-width symbols
return 1.75
else: # other characters (e.g. English and half-width symbols)
return 1
return sum(char_weight(char) for char in text)
def align_subs(src_sub: str, tr_sub: str, src_part: str) -> Tuple[List[str], List[str], str]:
align_prompt = get_align_prompt(src_sub, tr_sub, src_part)
def valid_align(response_data):
if 'align' not in response_data:
return {"status": "error", "message": "Missing required key: `align`"}
if len(response_data['align']) < 2:
return {"status": "error", "message": "Align does not contain more than 1 part as expected!"}
return {"status": "success", "message": "Align completed"}
parsed = ask_gpt(align_prompt, resp_type='json', valid_def=valid_align, log_title='align_subs')
align_data = parsed['align']
src_parts = src_part.split('\n')
tr_parts = [item[f'target_part_{i+1}'].strip() for i, item in enumerate(align_data)]
whisper_language = load_key("whisper.language")
language = load_key("whisper.detected_language") if whisper_language == 'auto' else whisper_language
joiner = get_joiner(language)
tr_remerged = joiner.join(tr_parts)
table = Table(title="🔗 Aligned parts")
table.add_column("Language", style="cyan")
table.add_column("Parts", style="magenta")
table.add_row("SRC_LANG", "\n".join(src_parts))
table.add_row("TARGET_LANG", "\n".join(tr_parts))
console.print(table)
return src_parts, tr_parts, tr_remerged
def split_align_subs(src_lines: List[str], tr_lines: List[str]):
subtitle_set = load_key("subtitle")
MAX_SUB_LENGTH = subtitle_set["max_length"]
TARGET_SUB_MULTIPLIER = subtitle_set["target_multiplier"]
remerged_tr_lines = tr_lines.copy()
to_split = []
for i, (src, tr) in enumerate(zip(src_lines, tr_lines)):
src, tr = str(src), str(tr)
if len(src) > MAX_SUB_LENGTH or calc_len(tr) * TARGET_SUB_MULTIPLIER > MAX_SUB_LENGTH:
to_split.append(i)
table = Table(title=f"📏 Line {i} needs to be split")
table.add_column("Type", style="cyan")
table.add_column("Content", style="magenta")
table.add_row("Source Line", src)
table.add_row("Target Line", tr)
console.print(table)
@except_handler("Error in split_align_subs")
def process(i):
split_src = split_sentence(src_lines[i], num_parts=2).strip()
src_parts, tr_parts, tr_remerged = align_subs(src_lines[i], tr_lines[i], split_src)
src_lines[i] = src_parts
tr_lines[i] = tr_parts
remerged_tr_lines[i] = tr_remerged
with concurrent.futures.ThreadPoolExecutor(max_workers=load_key("max_workers")) as executor:
executor.map(process, to_split)
# Flatten `src_lines` and `tr_lines`
src_lines = [item for sublist in src_lines for item in (sublist if isinstance(sublist, list) else [sublist])]
tr_lines = [item for sublist in tr_lines for item in (sublist if isinstance(sublist, list) else [sublist])]
return src_lines, tr_lines, remerged_tr_lines
def split_for_sub_main():
console.print("[bold green]🚀 Start splitting subtitles...[/bold green]")
df = pd.read_excel(_4_2_TRANSLATION)
src = df['Source'].tolist()
trans = df['Translation'].tolist()
subtitle_set = load_key("subtitle")
MAX_SUB_LENGTH = subtitle_set["max_length"]
TARGET_SUB_MULTIPLIER = subtitle_set["target_multiplier"]
for attempt in range(3): # 多次切割
console.print(Panel(f"🔄 Split attempt {attempt + 1}", expand=False))
split_src, split_trans, remerged = split_align_subs(src.copy(), trans)
# 检查是否所有字幕都符合长度要求
if all(len(src) <= MAX_SUB_LENGTH for src in split_src) and \
all(calc_len(tr) * TARGET_SUB_MULTIPLIER <= MAX_SUB_LENGTH for tr in split_trans):
break
# 更新源数据继续下一轮分割
src, trans = split_src, split_trans
# 确保二者有相同的长度，防止报错
if len(src) > len(remerged):
remerged += [None] * (len(src) - len(remerged))
elif len(remerged) > len(src):
src += [None] * (len(remerged) - len(src))
pd.DataFrame({'Source': split_src, 'Translation': split_trans}).to_excel(_5_SPLIT_SUB, index=False)
pd.DataFrame({'Source': src, 'Translation': remerged}).to_excel(_5_REMERGED, index=False)
if __name__ == '__main__':
split_for_sub_main()
================================================
FILE: core/_6_gen_sub.py
================================================
import pandas as pd
import os
import re
from rich.panel import Panel
from rich.console import Console
import autocorrect_py as autocorrect
from core.utils import *
from core.utils.models import *
console = Console()
SUBTITLE_OUTPUT_CONFIGS = [
('src.srt', ['Source']),
('trans.srt', ['Translation']),
('src_trans.srt', ['Source', 'Translation']),
('trans_src.srt', ['Translation', 'Source'])
]
AUDIO_SUBTITLE_OUTPUT_CONFIGS = [
('src_subs_for_audio.srt', ['Source']),
('trans_subs_for_audio.srt', ['Translation'])
]
def convert_to_srt_format(start_time, end_time):
"""Convert time (in seconds) to the format: hours:minutes:seconds,milliseconds"""
def seconds_to_hmsm(seconds):
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
seconds = seconds % 60
milliseconds = int(seconds * 1000) % 1000
return f"{hours:02d}:{minutes:02d}:{int(seconds):02d},{milliseconds:03d}"
start_srt = seconds_to_hmsm(start_time)
end_srt = seconds_to_hmsm(end_time)
return f"{start_srt} --> {end_srt}"
def remove_punctuation(text):
text = re.sub(r'\s+', ' ', text)
text = re.sub(r'[^\w\s]', '', text)
return text.strip()
def show_difference(str1, str2):
"""Show the difference positions between two strings"""
min_len = min(len(str1), len(str2))
diff_positions = []
for i in range(min_len):
if str1[i] != str2[i]:
diff_positions.append(i)
if len(str1) != len(str2):
diff_positions.extend(range(min_len, max(len(str1), len(str2))))
print("Difference positions:")
print(f"Expected sentence: {str1}")
print(f"Actual match: {str2}")
print("Position markers: " + "".join("^" if i in diff_positions else " " for i in range(max(len(str1), len(str2)))))
print(f"Difference indices: {diff_positions}")
def get_sentence_timestamps(df_words, df_sentences):
time_stamp_list = []
# Build complete string and position mapping
full_words_str = ''
position_to_word_idx = {}
for idx, word in enumerate(df_words['text']):
clean_word = remove_punctuation(word.lower())
start_pos = len(full_words_str)
full_words_str += clean_word
for pos in range(start_pos, len(full_words_str)):
position_to_word_idx[pos] = idx
current_pos = 0
for idx, sentence in df_sentences['Source'].items():
clean_sentence = remove_punctuation(sentence.lower()).replace(" ", "")
sentence_len = len(clean_sentence)
match_found = False
while current_pos <= len(full_words_str) - sentence_len:
if full_words_str[current_pos:current_pos+sentence_len] == clean_sentence:
start_word_idx = position_to_word_idx[current_pos]
end_word_idx = position_to_word_idx[current_pos + sentence_len - 1]
time_stamp_list.append((
float(df_words['start'][start_word_idx]),
float(df_words['end'][end_word_idx])
))
current_pos += sentence_len
match_found = True
break
current_pos += 1
if not match_found:
print(f"\n⚠️ Warning: No exact match found for sentence: {sentence}")
show_difference(clean_sentence,
full_words_str[current_pos:current_pos+len(clean_sentence)])
print("\nOriginal sentence:", df_sentences['Source'][idx])
raise ValueError("❎ No match found for sentence.")
return time_stamp_list
def align_timestamp(df_text, df_translate, subtitle_output_configs: list, output_dir: str, for_display: bool = True):
"""Align timestamps and add a new timestamp column to df_translate"""
df_trans_time = df_translate.copy()
# Assign an ID to each word in df_text['text'] and create a new DataFrame
words = df_text['text'].str.split(expand=True).stack().reset_index(level=1, drop=True).reset_index()
words.columns = ['id', 'word']
words['id'] = words['id'].astype(int)
# Process timestamps ⏰
time_stamp_list = get_sentence_timestamps(df_text, df_translate)
df_trans_time['timestamp'] = time_stamp_list
df_trans_time['duration'] = df_trans_time['timestamp'].apply(lambda x: x[1] - x[0])
# Remove gaps 🕳️
for i in range(len(df_trans_time)-1):
delta_time = df_trans_time.loc[i+1, 'timestamp'][0] - df_trans_time.loc[i, 'timestamp'][1]
if 0 < delta_time < 1:
df_trans_time.at[i, 'timestamp'] = (df_trans_time.loc[i, 'timestamp'][0], df_trans_time.loc[i+1, 'timestamp'][0])
# Convert start and end timestamps to SRT format
df_trans_time['timestamp'] = df_trans_time['timestamp'].apply(lambda x: convert_to_srt_format(x[0], x[1]))
# Polish subtitles: replace punctuation in Translation if for_display
if for_display:
df_trans_time['Translation'] = df_trans_time['Translation'].apply(lambda x: re.sub(r'[，。]', ' ', x).strip())
# Output subtitles 📜
def generate_subtitle_string(df, columns):
return ''.join([f"{i+1}\n{row['timestamp']}\n{row[columns[0]].strip()}\n{row[columns[1]].strip() if len(columns) > 1 else ''}\n\n" for i, row in df.iterrows()]).strip()
if output_dir:
os.makedirs(output_dir, exist_ok=True)
for filename, columns in subtitle_output_configs:
subtitle_str = generate_subtitle_string(df_trans_time, columns)
with open(os.path.join(output_dir, filename), 'w', encoding='utf-8') as f:
f.write(subtitle_str)
return df_trans_time
# ✨ Beautify the translation
def clean_translation(x):
if pd.isna(x):
return ''
cleaned = str(x).strip('。').strip('，')
return autocorrect.format(cleaned)
def align_timestamp_main():
df_text = pd.read_excel(_2_CLEANED_CHUNKS)
df_text['text'] = df_text['text'].str.strip('"').str.strip()
df_translate = pd.read_excel(_5_SPLIT_SUB)
df_translate['Translation'] = df_translate['Translation'].apply(clean_translation)
align_timestamp(df_text, df_translate, SUBTITLE_OUTPUT_CONFIGS, _OUTPUT_DIR)
console.print(Panel("[bold green]🎉📝 Subtitles generation completed! Please check in the `output` folder 👀[/bold green]"))
# for audio
df_translate_for_audio = pd.read_excel(_5_REMERGED) # use remerged file to avoid unmatched lines when dubbing
df_translate_for_audio['Translation'] = df_translate_for_audio['Translation'].apply(clean_translation)
align_timestamp(df_text, df_translate_for_audio, AUDIO_SUBTITLE_OUTPUT_CONFIGS, _AUDIO_DIR)
console.print(Panel(f"[bold green]🎉📝 Audio subtitles generation completed! Please check in the `{_AUDIO_DIR}` folder 👀[/bold green]"))
if __name__ == '__main__':
align_timestamp_main()
================================================
FILE: core/_7_sub_into_vid.py
================================================
import os, subprocess, time
from core._1_ytdlp import find_video_files
import cv2
import numpy as np
import platform
from core.utils import *
SRC_FONT_SIZE = 15
TRANS_FONT_SIZE = 17
FONT_NAME = 'Arial'
TRANS_FONT_NAME = 'Arial'
# Linux need to install google noto fonts: apt-get install fonts-noto
if platform.system() == 'Linux':
FONT_NAME = 'NotoSansCJK-Regular'
TRANS_FONT_NAME = 'NotoSansCJK-Regular'
# Mac OS has different font names
elif platform.system() == 'Darwin':
FONT_NAME = 'Arial Unicode MS'
TRANS_FONT_NAME = 'Arial Unicode MS'
SRC_FONT_COLOR = '&HFFFFFF'
SRC_OUTLINE_COLOR = '&H000000'
SRC_OUTLINE_WIDTH = 1
SRC_SHADOW_COLOR = '&H80000000'
TRANS_FONT_COLOR = '&H00FFFF'
TRANS_OUTLINE_COLOR = '&H000000'
TRANS_OUTLINE_WIDTH = 1
TRANS_BACK_COLOR = '&H33000000'
OUTPUT_DIR = "output"
OUTPUT_VIDEO = f"{OUTPUT_DIR}/output_sub.mp4"
SRC_SRT = f"{OUTPUT_DIR}/src.srt"
TRANS_SRT = f"{OUTPUT_DIR}/trans.srt"
def check_gpu_available():
try:
result = subprocess.run(['ffmpeg', '-encoders'], capture_output=True, text=True)
return 'h264_nvenc' in result.stdout
except:
return False
def merge_subtitles_to_video():
video_file = find_video_files()
os.makedirs(os.path.dirname(OUTPUT_VIDEO), exist_ok=True)
# Check resolution
if not load_key("burn_subtitles"):
rprint("[bold yellow]Warning: A 0-second black video will be generated as a placeholder as subtitles are not burned in.[/bold yellow]")
# Create a black frame
frame = np.zeros((1080, 1920, 3), dtype=np.uint8)
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter(OUTPUT_VIDEO, fourcc, 1, (1920, 1080))
out.write(frame)
out.release()
rprint("[bold green]Placeholder video has been generated.[/bold green]")
return
if not os.path.exists(SRC_SRT) or not os.path.exists(TRANS_SRT):
rprint("Subtitle files not found in the 'output' directory.")
exit(1)
video = cv2.VideoCapture(video_file)
TARGET_WIDTH = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
TARGET_HEIGHT = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
video.release()
rprint(f"[bold green]Video resolution: {TARGET_WIDTH}x{TARGET_HEIGHT}[/bold green]")
ffmpeg_cmd = [
'ffmpeg', '-i', video_file,
'-vf', (
f"scale={TARGET_WIDTH}:{TARGET_HEIGHT}:force_original_aspect_ratio=decrease,"
f"pad={TARGET_WIDTH}:{TARGET_HEIGHT}:(ow-iw)/2:(oh-ih)/2,"
f"subtitles={SRC_SRT}:force_style='FontSize={SRC_FONT_SIZE},FontName={FONT_NAME},"
f"PrimaryColour={SRC_FONT_COLOR},OutlineColour={SRC_OUTLINE_COLOR},OutlineWidth={SRC_OUTLINE_WIDTH},"
f"ShadowColour={SRC_SHADOW_COLOR},BorderStyle=1',"
f"subtitles={TRANS_SRT}:force_style='FontSize={TRANS_FONT_SIZE},FontName={TRANS_FONT_NAME},"
f"PrimaryColour={TRANS_FONT_COLOR},OutlineColour={TRANS_OUTLINE_COLOR},OutlineWidth={TRANS_OUTLINE_WIDTH},"
f"BackColour={TRANS_BACK_COLOR},Alignment=2,MarginV=27,BorderStyle=4'"
).encode('utf-8'),
]
ffmpeg_gpu = load_key("ffmpeg_gpu")
if ffmpeg_gpu:
rprint("[bold green]will use GPU acceleration.[/bold green]")
ffmpeg_cmd.extend(['-c:v', 'h264_nvenc'])
ffmpeg_cmd.extend(['-y', OUTPUT_VIDEO])
rprint("🎬 Start merging subtitles to video...")
start_time = time.time()
process = subprocess.Popen(ffmpeg_cmd)
try:
process.wait()
if process.returncode == 0:
rprint(f"\n✅ Done! Time taken: {time.time() - start_time:.2f} seconds")
else:
rprint("\n❌ FFmpeg execution error")
except Exception as e:
rprint(f"\n❌ Error occurred: {e}")
if process.poll() is None:
process.kill()
if __name__ == "__main__":
merge_subtitles_to_video()
================================================
FILE: core/_8_1_audio_task.py
================================================
import datetime
import re
import pandas as pd
from rich.console import Console
from rich.panel import Panel
from core.prompts import get_subtitle_trim_prompt
from core.tts_backend.estimate_duration import init_estimator, estimate_duration
from core.utils import *
from core.utils.models import *
console = Console()
speed_factor = load_key("speed_factor")
TRANS_SUBS_FOR_AUDIO_FILE = 'output/audio/trans_subs_for_audio.srt'
SRC_SUBS_FOR_AUDIO_FILE = 'output/audio/src_subs_for_audio.srt'
ESTIMATOR = None
def check_len_then_trim(text, duration):
global ESTIMATOR
if ESTIMATOR is None:
ESTIMATOR = init_estimator()
estimated_duration = estimate_duration(text, ESTIMATOR) / speed_factor['max']
console.print(f"Subtitle text: {text}, "
f"[bold green]Estimated reading duration: {estimated_duration:.2f} seconds[/bold green]")
if estimated_duration > duration:
rprint(Panel(f"Estimated reading duration {estimated_duration:.2f} seconds exceeds given duration {duration:.2f} seconds, shortening...", title="Processing", border_style="yellow"))
original_text = text
prompt = get_subtitle_trim_prompt(text, duration)
def valid_trim(response):
if 'result' not in response:
return {'status': 'error', 'message': 'No result in response'}
return {'status': 'success', 'message': ''}
try:
response = ask_gpt(prompt, resp_type='json', log_title='sub_trim', valid_def=valid_trim)
shortened_text = response['result']
except Exception:
rprint("[bold red]🚫 AI refused to answer due to sensitivity, so manually remove punctuation[/bold red]")
shortened_text = re.sub(r'[,.!?;:，。！？；：]', ' ', text).strip()
rprint(Panel(f"Subtitle before shortening: {original_text}\nSubtitle after shortening: {shortened_text}", title="Subtitle Shortening Result", border_style="green"))
return shortened_text
else:
return text
def time_diff_seconds(t1, t2, base_date):
"""Calculate the difference in seconds between two time objects"""
dt1 = datetime.datetime.combine(base_date, t1)
dt2 = datetime.datetime.combine(base_date, t2)
return (dt2 - dt1).total_seconds()
def process_srt():
"""Process srt file, generate audio tasks"""
with open(TRANS_SUBS_FOR_AUDIO_FILE, 'r', encoding='utf-8') as file:
content = file.read()
with open(SRC_SUBS_FOR_AUDIO_FILE, 'r', encoding='utf-8') as src_file:
src_content = src_file.read()
subtitles = []
src_subtitles = {}
for block in src_content.strip().split('\n\n'):
lines = [line.strip() for line in block.split('\n') if line.strip()]
if len(lines) < 3:
continue
number = int(lines[0])
src_text = ' '.join(lines[2:])
src_subtitles[number] = src_text
for block in content.strip().split('\n\n'):
lines = [line.strip() for line in block.split('\n') if line.strip()]
if len(lines) < 3:
continue
try:
number = int(lines[0])
start_time, end_time = lines[1].split(' --> ')
start_time = datetime.datetime.strptime(start_time, '%H:%M:%S,%f').time()
end_time = datetime.datetime.strptime(end_time, '%H:%M:%S,%f').time()
duration = time_diff_seconds(start_time, end_time, datetime.date.today())
text = ' '.join(lines[2:])
# Remove content within parentheses (including English and Chinese parentheses)
text = re.sub(r'\([^)]*\)', '', text).strip()
text = re.sub(r'（[^）]*）', '', text).strip()
# Remove '-' character, can continue to add illegal characters that cause errors
text = text.replace('-', '')
# Add the original text from src_subs_for_audio.srt
origin = src_subtitles.get(number, '')
except ValueError as e:
rprint(Panel(f"Unable to parse subtitle block '{block}', error: {str(e)}, skipping this subtitle block.", title="Error", border_style="red"))
continue
subtitles.append({'number': number, 'start_time': start_time, 'end_time': end_time, 'duration': duration, 'text': text, 'origin': origin})
df = pd.DataFrame(subtitles)
i = 0
MIN_SUB_DUR = load_key("min_subtitle_duration")
while i < len(df):
today = datetime.date.today()
if df.loc[i, 'duration'] < MIN_SUB_DUR:
if i < len(df) - 1 and time_diff_seconds(df.loc[i, 'start_time'],df.loc[i+1, 'start_time'],today) < MIN_SUB_DUR:
rprint(f"[bold yellow]Merging subtitles {i+1} and {i+2}[/bold yellow]")
df.loc[i, 'text'] += ' ' + df.loc[i+1, 'text']
df.loc[i, 'origin'] += ' ' + df.loc[i+1, 'origin']
df.loc[i, 'end_time'] = df.loc[i+1, 'end_time']
df.loc[i, 'duration'] = time_diff_seconds(df.loc[i, 'start_time'],df.loc[i, 'end_time'],today)
df = df.drop(i+1).reset_index(drop=True)
else:
if i < len(df) - 1: # Not the last audio
rprint(f"[bold blue]Extending subtitle {i+1} duration to {MIN_SUB_DUR} seconds[/bold blue]")
df.loc[i, 'end_time'] = (datetime.datetime.combine(today, df.loc[i, 'start_time']) +
datetime.timedelta(seconds=MIN_SUB_DUR)).time()
df.loc[i, 'duration'] = MIN_SUB_DUR
else:
rprint(f"[bold red]The last subtitle {i+1} duration is less than {MIN_SUB_DUR} seconds, but not extending[/bold red]")
i += 1
else:
i += 1
df['start_time'] = df['start_time'].apply(lambda x: x.strftime('%H:%M:%S.%f')[:-3])
df['end_time'] = df['end_time'].apply(lambda x: x.strftime('%H:%M:%S.%f')[:-3])
##! No longer perform secondary trim
# check and trim subtitle length, for twice to ensure the subtitle length is within the limit, 允许tolerance
# df['text'] = df.apply(lambda x: check_len_then_trim(x['text'], x['duration']+x['tolerance']), axis=1)
return df
@check_file_exists(_8_1_AUDIO_TASK)
def gen_audio_task_main():
df = process_srt()
console.print(df)
df.to_excel(_8_1_AUDIO_TASK, index=False)
rprint(Panel(f"Successfully generated {_8_1_AUDIO_TASK}", title="Success", border_style="green"))
if __name__ == '__main__':
gen_audio_task_main()
================================================
FILE: core/_8_2_dub_chunks.py
================================================
import datetime
import re
import pandas as pd
from core._8_1_audio_task import time_diff_seconds
from core.asr_backend.audio_preprocess import get_audio_duration
from core.tts_backend.estimate_duration import init_estimator, estimate_duration
from core.utils import *
from core.utils.models import *
SRC_SRT = "output/src.srt"
TRANS_SRT = "output/trans.srt"
MAX_MERGE_COUNT = 5
ESTIMATOR = None
def calc_if_too_fast(est_dur, tol_dur, duration, tolerance):
accept = load_key("speed_factor.accept") # Maximum acceptable speed factor
if est_dur / accept > tol_dur: # Even max speed factor cannot adapt
return 2
elif est_dur > tol_dur: # Speed adjustment needed within acceptable range
return 1
elif est_dur < duration - tolerance: # Speaking speed too slow
return -1
else: # Normal speaking speed
return 0
def merge_rows(df, start_idx, merge_count):
"""Merge multiple rows and calculate cumulative values"""
merged = {
'est_dur': df.iloc[start_idx]['est_dur'],
'tol_dur': df.iloc[start_idx]['tol_dur'],
'duration': df.iloc[start_idx]['duration']
}
while merge_count < MAX_MERGE_COUNT and (start_idx + merge_count) < len(df):
next_row = df.iloc[start_idx + merge_count]
merged['est_dur'] += next_row['est_dur']
merged['tol_dur'] += next_row['tol_dur']
merged['duration'] += next_row['duration']
speed_flag = calc_if_too_fast(
merged['est_dur'],
merged['tol_dur'],
merged['duration'],
df.iloc[start_idx + merge_count]['tolerance']
)
if speed_flag <= 0 or merge_count == 2:
df.at[start_idx + merge_count, 'cut_off'] = 1
return merge_count + 1
merge_count += 1
# If no suitable merge point is found
if merge_count >= MAX_MERGE_COUNT or (start_idx + merge_count) >= len(df):
df.at[start_idx + merge_count - 1, 'cut_off'] = 1
return merge_count
def analyze_subtitle_timing_and_speed(df):
rprint("[🔍 Analyzing] Calculating subtitle timing and speed...")
global ESTIMATOR
if ESTIMATOR is None:
ESTIMATOR = init_estimator()
TOLERANCE = load_key("tolerance")
whole_dur = get_audio_duration(_RAW_AUDIO_FILE)
df['gap'] = 0.0 # Initialize gap column
for i in range(len(df) - 1):
current_end = datetime.datetime.strptime(df.loc[i, 'end_time'], '%H:%M:%S.%f').time()
next_start = datetime.datetime.strptime(df.loc[i + 1, 'start_time'], '%H:%M:%S.%f').time()
df.loc[i, 'gap'] = time_diff_seconds(current_end, next_start, datetime.date.today())
# Set the gap for the last line
last_end = datetime.datetime.strptime(df.iloc[-1]['end_time'], '%H:%M:%S.%f').time()
last_end_seconds = (last_end.hour * 3600 + last_end.minute * 60 +
last_end.second + last_end.microsecond / 1000000)
df.iloc[-1, df.columns.get_loc('gap')] = whole_dur - last_end_seconds
df['tolerance'] = df['gap'].apply(lambda x: TOLERANCE if x > TOLERANCE else x)
df['tol_dur'] = df['duration'] + df['tolerance']
df['est_dur'] = df.apply(lambda x: estimate_duration(x['text'], ESTIMATOR), axis=1)
## Calculate speed indicators
accept = load_key("speed_factor.accept") # Maximum acceptable speed factor
def calc_if_too_fast(row):
est_dur = row['est_dur']
tol_dur = row['tol_dur']
duration = row['duration']
tolerance = row['tolerance']
if est_dur / accept > tol_dur: # Even max speed factor cannot adapt
return 2
elif est_dur > tol_dur: # Speed adjustment needed within acceptable range
return 1
elif est_dur < duration - tolerance: # Speaking speed too slow
return -1
else: # Normal speaking speed
return 0
df['if_too_fast'] = df.apply(calc_if_too_fast, axis=1)
return df
def process_cutoffs(df):
rprint("[✂️ Processing] Generating cutoff points...")
df['cut_off'] = 0 # Initialize cut_off column
df.loc[df['gap'] >= load_key("tolerance"), 'cut_off'] = 1 # Set to 1 when gap is greater than TOLERANCE
idx = 0
while idx < len(df):
# Process marked split points
if df.iloc[idx]['cut_off'] == 1:
if df.iloc[idx]['if_too_fast'] == 2:
rprint(f"[⚠️ Warning] Line {idx} is too fast and cannot be fixed by speed adjustment")
idx += 1
continue
# Process the last line
if idx + 1 >= len(df):
df.at[idx, 'cut_off'] = 1
break
# Process normal or slow lines
if df.iloc[idx]['if_too_fast'] <= 0:
if df.iloc[idx + 1]['if_too_fast'] <= 0:
df.at[idx, 'cut_off'] = 1
idx += 1
else:
idx += merge_rows(df, idx, 1)
# Process fast lines
else:
idx += merge_rows(df, idx, 1)
return df
def gen_dub_chunks():
rprint("[🎬 Starting] Generating dubbing chunks...")
df = pd.read_excel(_8_1_AUDIO_TASK)
rprint("[📊 Processing] Analyzing timing and speed...")
df = analyze_subtitle_timing_and_speed(df)
rprint("[✂️ Processing] Processing cutoffs...")
df = process_cutoffs(df)
rprint("[📝 Reading] Loading transcript files...")
content = open(TRANS_SRT, "r", encoding="utf-8").read()
ori_content = open(SRC_SRT, "r", encoding="utf-8").read()
# Process subtitle content
content_lines = []
ori_content_lines = []
# Process translated subtitles
for block in content.strip().split('\n\n'):
lines = [line.strip() for line in block.split('\n') if line.strip()]
if len(lines) >= 3:
text = ' '.join(lines[2:])
text = re.sub(r'\([^)]*\)|（[^）]*）', '', text).strip().replace('-', '')
content_lines.append(text)
# Process source subtitles (same structure)
for block in ori_content.strip().split('\n\n'):
lines = [line.strip() for line in block.split('\n') if line.strip()]
if len(lines) >= 3:
text = ' '.join(lines[2:])
text = re.sub(r'\([^)]*\)|（[^）]*）', '', text).strip().replace('-', '')
ori_content_lines.append(text)
# Match processing
df['lines'] = None
df['src_lines'] = None
last_idx = 0
def clean_text(text):
"""clean space and punctuation"""
if not text or not isinstance(text, str):
return ''
return re.sub(r'[^\w\s]|[\s]', '', text)
for idx, row in df.iterrows():
target = clean_text(row['text'])
matches = []
current = ''
match_indices = [] # Store indices for matching lines
for i in range(last_idx, len(content_lines)):
line = content_lines[i]
cleaned_line = clean_text(line)
current += cleaned_line
matches.append(line) # 存储原始文本
match_indices.append(i)
if current == target:
df.at[idx, 'lines'] = matches
df.at[idx, 'src_lines'] = [ori_content_lines[i] for i in match_indices]
last_idx = i + 1
break
else: # If no match is found
rprint(f"[❌ Error] Matching failed at line {idx}:")
rprint(f"Target: '{target}'")
rprint(f"Current: '{current}'")
raise ValueError("Matching failed")
# Save results
df.to_excel(_8_1_AUDIO_TASK, index=False)
rprint("[✅ Complete] Matching completed successfully!")
if __name__ == "__main__":
gen_dub_chunks()
================================================
FILE: core/_9_refer_audio.py
================================================
import os
from rich.panel import Panel
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
from core.utils import *
from core.utils.models import *
import pandas as pd
import soundfile as sf
console = Console()
from core.asr_backend.demucs_vl import demucs_audio
from core.utils.models import *
def time_to_samples(time_str, sr):
"""Unified time conversion function"""
h, m, s = time_str.split(':')
s, ms = s.split(',') if ',' in s else (s, '0')
seconds = int(h) * 3600 + int(m) * 60 + float(s) + float(ms) / 1000
return int(seconds * sr)
def extract_audio(audio_data, sr, start_time, end_time, out_file):
"""Simplified audio extraction function"""
start = time_to_samples(start_time, sr)
end = time_to_samples(end_time, sr)
sf.write(out_file, audio_data[start:end], sr)
def extract_refer_audio_main():
demucs_audio() #!!! in case demucs not run
if os.path.exists(os.path.join(_AUDIO_SEGS_DIR, '1.wav')):
rprint(Panel("Audio segments already exist, skipping extraction", title="Info", border_style="blue"))
return
# Create output directory
os.makedirs(_AUDIO_REFERS_DIR, exist_ok=True)
# Read task file and audio data
df = pd.read_excel(_8_1_AUDIO_TASK)
data, sr = sf.read(_VOCAL_AUDIO_FILE)
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
) as progress:
task = progress.add_task("Extracting audio segments...", total=len(df))
for _, row in df.iterrows():
out_file = os.path.join(_AUDIO_REFERS_DIR, f"{row['number']}.wav")
extract_audio(data, sr, row['start_time'], row['end_time'], out_file)
progress.update(task, advance=1)
rprint(Panel(f"Audio segments saved to {_AUDIO_REFERS_DIR}", title="Success", border_style="green"))
if __name__ == "__main__":
extract_refer_audio_main()
================================================
FILE: core/__init__.py
================================================
# use try-except to avoid error when installing
try:
from . import (
_1_ytdlp,
_2_asr,
_3_1_split_nlp,
_3_2_split_meaning,
_4_1_summarize,
_4_2_translate,
_5_split_sub,
_6_gen_sub,
_7_sub_into_vid,
_8_1_audio_task,
_8_2_dub_chunks,
_9_refer_audio,
_10_gen_audio,
_11_merge_audio,
_12_dub_to_vid
)
from .utils import *
from .utils.onekeycleanup import cleanup
from .utils.delete_retry_dubbing import delete_dubbing_files
except ImportError:
pass
__all__ = [
'ask_gpt',
'load_key',
'update_key',
'cleanup',
'delete_dubbing_files',
'_1_ytdlp',
'_2_asr',
'_3_1_split_nlp',
'_3_2_split_meaning',
'_4_1_summarize',
'_4_2_translate',
'_5_split_sub',
'_6_gen_sub',
'_7_sub_into_vid',
'_8_1_audio_task',
'_8_2_dub_chunks',
'_9_refer_audio',
'_10_gen_audio',
'_11_merge_audio',
'_12_dub_to_vid'
]
================================================
FILE: core/asr_backend/__init__.py
================================================
================================================
FILE: core/asr_backend/audio_preprocess.py
================================================
import os, subprocess
import pandas as pd
from typing import Dict, List, Tuple
from pydub import AudioSegment
from core.utils import *
from core.utils.models import *
from pydub import AudioSegment
from pydub.silence import detect_silence
from pydub.utils import mediainfo
from rich import print as rprint
def _ffmpeg_has_encoder(encoder_name: str) -> bool:
"""Check if the current ffmpeg installation supports a given audio encoder."""
try:
result = subprocess.run(
['ffmpeg', '-encoders'], capture_output=True, text=True, timeout=10
)
return encoder_name in result.stdout
except Exception:
return False
def normalize_audio_volume(audio_path, output_path, target_db = -20.0, format = "wav"):
audio = AudioSegment.from_file(audio_path)
change_in_dBFS = target_db - audio.dBFS
normalized_audio = audio.apply_gain(change_in_dBFS)
normalized_audio.export(output_path, format=format)
rprint(f"[green]✅ Audio normalized from {audio.dBFS:.1f}dB to {target_db:.1f}dB[/green]")
return output_path
def convert_video_to_audio(video_file: str):
os.makedirs(_AUDIO_DIR, exist_ok=True)
if not os.path.exists(_RAW_AUDIO_FILE):
rprint(f"[blue]🎬➡️🎵 Converting to high quality audio with FFmpeg ......[/blue]")
if _ffmpeg_has_encoder('libmp3lame'):
cmd = [
'ffmpeg', '-y', '-i', video_file, '-vn',
'-c:a', 'libmp3lame', '-b:a', '32k',
'-ar', '16000', '-ac', '1',
'-metadata', 'encoding=UTF-8', _RAW_AUDIO_FILE
]
else:
# Fallback: conda-forge ffmpeg often lacks libmp3lame.
# Output as WAV (PCM) which all ffmpeg builds support.
# Downstream readers (pydub, librosa, whisperX) detect format by
# file header, not extension, so .mp3 path with WAV content works.
rprint("[yellow]⚠️ libmp3lame not found in ffmpeg, falling back to WAV (PCM) encoding[/yellow]")
cmd = [
'ffmpeg', '-y', '-i', video_file, '-vn',
'-c:a', 'pcm_s16le', '-ar', '16000', '-ac', '1',
'-f', 'wav', _RAW_AUDIO_FILE
]
subprocess.run(cmd, check=True, stderr=subprocess.PIPE)
rprint(f"[green]🎬➡️🎵 Converted <{video_file}> to <{_RAW_AUDIO_FILE}> with FFmpeg\n[/green]")
def get_audio_duration(audio_file: str) -> float:
"""Get the duration of an audio file using ffmpeg."""
cmd = ['ffmpeg', '-i', audio_file]
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
_, stderr = process.communicate()
output = stderr.decode('utf-8', errors='ignore')
try:
duration_str = [line for line in output.split('\n') if 'Duration' in line][0]
duration_parts = duration_str.split('Duration: ')[1].split(',')[0].split(':')
duration = float(duration_parts[0])*3600 + float(duration_parts[1])*60 + float(duration_parts[2])
except Exception as e:
print(f"[red]❌ Error: Failed to get audio duration: {e}[/red]")
duration = 0
return duration
def split_audio(audio_file: str, target_len: float = 30*60, win: float = 60) -> List[Tuple[float, float]]:
## 在 [target_len-win, target_len+win] 区间内用 pydub 检测静默，切分音频
rprint(f"[blue]🎙️ Starting audio segmentation {audio_file} {target_len} {win}[/blue]")
audio = AudioSegment.from_file(audio_file)
duration = float(mediainfo(audio_file)["duration"])
if duration <= target_len + win:
return [(0, duration)]
segments, pos = [], 0.0
safe_margin = 0.5 # 静默点前后安全边界，单位秒
while pos < duration:
if duration - pos <= target_len:
segments.append((pos, duration)); break
threshold = pos + target_len
ws, we = int((threshold - win) * 1000), int((threshold + win) * 1000)
# 获取完整的静默区域
silence_regions = detect_silence(audio[ws:we], min_silence_len=int(safe_margin*1000), silence_thresh=-30)
silence_regions = [(s/1000 + (threshold - win), e/1000 + (threshold - win)) for s, e in silence_regions]
# 筛选长度足够（至少1秒）且位置适合的静默区域
valid_regions = [
(start, end) for start, end in silence_regions
if (end - start) >= (safe_margin * 2) and threshold <= start + safe_margin <= threshold + win
]
if valid_regions:
start, end = valid_regions[0]
split_at = start + safe_margin # 在静默区域起始点后0.5秒处切分
else:
rprint(f"[yellow]⚠️ No valid silence regions found for {audio_file} at {threshold}s, using threshold[/yellow]")
split_at = threshold
segments.append((pos, split_at)); pos = split_at
rprint(f"[green]🎙️ Audio split completed {len(segments)} segments[/green]")
return segments
def process_transcription(result: Dict) -> pd.DataFrame:
all_words = []
for segment in result['segments']:
# Get speaker_id, if not exists, set to None
speaker_id = segment.get('speaker_id', None)
for word in segment['words']:
# Check word length
if len(word["word"]) > 30:
rprint(f"[yellow]⚠️ Warning: Detected word longer than 30 characters, skipping: {word['word']}[/yellow]")
continue
# ! For French, we need to convert guillemets to empty strings
word["word"] = word["word"].replace('»', '').replace('«', '')
if 'start' not in word and 'end' not in word:
if all_words:
# Assign the end time of the previous word as the start and end time of the current word
word_dict = {
'text': word["word"],
'start': all_words[-1]['end'],
'end': all_words[-1]['end'],
'speaker_id': speaker_id
}
all_words.append(word_dict)
else:
# If it's the first word, look next for a timestamp then assign it to the current word
next_word = next((w for w in segment['words'] if 'start' in w and 'end' in w), None)
if next_word:
word_dict = {
'text': word["word"],
'start': next_word["start"],
'end': next_word["end"],
'speaker_id': speaker_id
}
all_words.append(word_dict)
else:
raise Exception(f"No next word with timestamp found for the current word : {word}")
else:
# Normal case, with start and end times
word_dict = {
'text': f'{word["word"]}',
'start': word.get('start', all_words[-1]['end'] if all_words else 0),
'end': word['end'],
'speaker_id': speaker_id
}
all_words.append(word_dict)
return pd.DataFrame(all_words)
def save_results(df: pd.DataFrame):
os.makedirs('output/log', exist_ok=True)
# Remove rows where 'text' is empty
initial_rows = len(df)
df = df[df['text'].str.len() > 0]
removed_rows = initial_rows - len(df)
if removed_rows > 0:
rprint(f"[blue]ℹ️ Removed {removed_rows} row(s) with empty text.[/blue]")
# Check for and remove words longer than 20 characters
long_words = df[df['text'].str.len() > 30]
if not long_words.empty:
rprint(f"[yellow]⚠️ Warning: Detected {len(long_words)} word(s) longer than 30 characters. These will be removed.[/yellow]")
df = df[df['text'].str.len() <= 30]
df['text'] = df['text'].apply(lambda x: f'"{x}"')
df.to_excel(_2_CLEANED_CHUNKS, index=False)
rprint(f"[green]📊 Excel file saved to {_2_CLEANED_CHUNKS}[/green]")
def save_language(language: str):
update_key("whisper.detected_language", language)
================================================
FILE: core/asr_backend/demucs_vl.py
================================================
import os
import torch
from rich.console import Console
from rich import print as rprint
from demucs.pretrained import get_model
from demucs.audio import save_audio
from torch.cuda import is_available as is_cuda_available
from typing import Optional
from demucs.api import Separator
from demucs.apply import BagOfModels
import gc
from core.utils.models import *
class PreloadedSeparator(Separator):
def __init__(self, model: BagOfModels, shifts: int = 1, overlap: float = 0.25,
split: bool = True, segment: Optional[int] = None, jobs: int = 0):
self._model, self._audio_channels, self._samplerate = model, model.audio_channels, model.samplerate
device = "cuda" if is_cuda_available() else "mps" if torch.backends.mps.is_available() else "cpu"
self.update_parameter(device=device, shifts=shifts, overlap=overlap, split=split,
segment=segment, jobs=jobs, progress=True, callback=None, callback_arg=None)
def demucs_audio():
if os.path.exists(_VOCAL_AUDIO_FILE) and os.path.exists(_BACKGROUND_AUDIO_FILE):
rprint(f"[yellow]⚠️ {_VOCAL_AUDIO_FILE} and {_BACKGROUND_AUDIO_FILE} already exist, skip Demucs processing.[/yellow]")
return
console = Console()
os.makedirs(_AUDIO_DIR, exist_ok=True)
console.print("🤖 Loading "{comment.content}"
{comment.author}, {comment.title}
{feature.description}
{user.login}
{description}
| ### Russian Translation --- https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7 | ### GPT-SoVITS Dubbing --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
If you find VideoLingo helpful, please give us a ⭐️!
================================================ FILE: docs/pages/docs/introduction.zh-CN.md ================================================ # VideoLingo: 连接世界的每一帧 **QQ 群：875297969** ## 🌟 简介（[在线体验！](https://videolingo.io)） VideoLingo 是一站式视频翻译本地化配音工具，能够一键生成 Netflix 级别的高质量字幕，告别生硬机翻，告别多行字幕，还能加上高质量的克隆配音，让全世界的知识能够跨越语言的障碍共享。 主要特点和功能： - 🎥 使用 yt-dlp 从 Youtube 链接下载视频 - **🎙️ 使用 WhisperX 进行单词级时间轴字幕识别** - **📝 使用 NLP 和 GPT 根据句意进行字幕分割** - **📚 GPT 总结提取术语知识库，上下文连贯翻译** - **🔄 三步直译、反思、意译，媲美字幕组精翻效果** - **✅ 按照 Netflix 标准检查单行长度，绝无双行字幕** - **🗣️ 使用 GPT-SoVITS 等方法对齐克隆配音** - 🚀 整合包一键启动，在 streamlit 中一键出片 - 📝 详细记录每步操作日志，支持随时中断和恢复进度 与同类项目相比的优势：**绝无多行字幕，最佳的翻译质量，无缝的配音体验** ## 🎥 效果演示| ### 俄语翻译 --- https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7 | ### GPT-SoVITS配音 --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
{t("This stage includes the following steps:")}
1. {t("WhisperX word-level transcription")}
2. {t("Sentence segmentation using NLP and LLM")}
3. {t("Summarization and multi-step translation")}
4. {t("Cutting and aligning long subtitles")}
5. {t("Generating timeline and subtitles")}
6. {t("Merging subtitles into the video")}
""", unsafe_allow_html=True)
if not os.path.exists(SUB_VIDEO):
if st.button(t("Start Processing Subtitles"), key="text_processing_button"):
process_text()
st.rerun()
else:
if load_key("burn_subtitles"):
st.video(SUB_VIDEO)
download_subtitle_zip_button(text=t("Download All Srt Files"))
if st.button(t("Archive to 'history'"), key="cleanup_in_text_processing"):
cleanup()
st.rerun()
return True
def process_text():
with st.spinner(t("Using Whisper for transcription...")):
_2_asr.transcribe()
with st.spinner(t("Splitting long sentences...")):
_3_1_split_nlp.split_by_spacy()
_3_2_split_meaning.split_sentences_by_meaning()
with st.spinner(t("Summarizing and translating...")):
_4_1_summarize.get_summary()
if load_key("pause_before_translate"):
input(t("⚠️ PAUSE_BEFORE_TRANSLATE. Go to `output/log/terminology.json` to edit terminology. Then press ENTER to continue..."))
_4_2_translate.translate_all()
with st.spinner(t("Processing and aligning subtitles...")):
_5_split_sub.split_for_sub_main()
_6_gen_sub.align_timestamp_main()
with st.spinner(t("Merging subtitles to video...")):
_7_sub_into_vid.merge_subtitles_to_video()
st.success(t("Subtitle processing complete! 🎉"))
st.balloons()
def audio_processing_section():
st.header(t("c. Dubbing"))
with st.container(border=True):
st.markdown(f"""
{t("This stage includes the following steps:")}
1. {t("Generate audio tasks and chunks")}
2. {t("Extract reference audio")}
3. {t("Generate and merge audio files")}
4. {t("Merge final audio into video")}
""", unsafe_allow_html=True)
if not os.path.exists(DUB_VIDEO):
if st.button(t("Start Audio Processing"), key="audio_processing_button"):
process_audio()
st.rerun()
else:
st.success(t("Audio processing is complete! You can check the audio files in the `output` folder."))
if load_key("burn_subtitles"):
st.video(DUB_VIDEO)
if st.button(t("Delete dubbing files"), key="delete_dubbing_files"):
delete_dubbing_files()
st.rerun()
if st.button(t("Archive to 'history'"), key="cleanup_in_audio_processing"):
cleanup()
st.rerun()
def process_audio():
with st.spinner(t("Generate audio tasks")):
_8_1_audio_task.gen_audio_task_main()
_8_2_dub_chunks.gen_dub_chunks()
with st.spinner(t("Extract refer audio")):
_9_refer_audio.extract_refer_audio_main()
with st.spinner(t("Generate all audio")):
_10_gen_audio.gen_audio()
with st.spinner(t("Merge full audio")):
_11_merge_audio.merge_full_audio()
with st.spinner(t("Merge dubbing to the video")):
_12_dub_to_vid.merge_video_audio()
st.success(t("Audio processing complete! 🎇"))
st.balloons()
def main():
logo_col, _ = st.columns([1,1])
with logo_col:
st.image("docs/logo.png", width="stretch")
st.markdown(button_style, unsafe_allow_html=True)
welcome_text = t("Hello, welcome to VideoLingo. If you encounter any issues, feel free to get instant answers with our Free QA Agent here! You can also try out our SaaS website at videolingo.io for free!")
st.markdown(f"
{welcome_text}
", unsafe_allow_html=True) # add settings with st.sidebar: page_setting() st.markdown(give_star_button, unsafe_allow_html=True) download_video_section() text_processing_section() audio_processing_section() if __name__ == "__main__": main() ================================================ FILE: translations/README.es.md ================================================
# Conectando el Mundo, Cuadro por Cuadro
| ### Subtítulos Duales --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Clonación de Voz Cosy2 --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS con mi voz --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
Si encuentras útil VideoLingo, ¡por favor dame una ⭐️!
================================================ FILE: translations/README.fr.md ================================================
# Connecter le Monde, Image par Image
| ### Sous-titres Doubles --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Clonage Vocal Cosy2 --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS avec ma voix --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
Si vous trouvez VideoLingo utile, donnez-moi une ⭐️ !
================================================ FILE: translations/README.ja.md ================================================
# フレームごとに世界をつなぐ
| ### デュアル字幕 --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Cosy2 ボイスクローン --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS 吹き替え --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
VideoLingoが役立つと感じた場合は、⭐️をお願いします！
================================================ FILE: translations/README.ru.md ================================================
# Объединяя Мир, Кадр за Кадром
| ### Двойные Субтитры --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Клонирование Голоса Cosy2 --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS с моим голосом --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
# 連結世界，逐格前行
| ### 雙語字幕 --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Cosy2 聲音克隆 --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS 配音 --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |
如果您覺得 VideoLingo 有幫助，請給我一個 ⭐️！
================================================ FILE: translations/README.zh.md ================================================
# 连接世界每一帧
| ### 双语字幕 --- https://github.com/user-attachments/assets/a5c3d8d1-2b29-4ba9-b0d0-25896829d951 | ### Cosy2 声音克隆 --- https://github.com/user-attachments/assets/e065fe4c-3694-477f-b4d6-316917df7c0a | ### GPT-SoVITS 配音 --- https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c |