Repository: FujiwaraChoki/MoneyPrinter Branch: main Commit: 608c3a9cb7a8 Files: 40 Total size: 171.1 KB Directory structure: gitextract__b835fml/ ├── .github/ │ ├── FUNDING.yml │ └── ISSUE_TEMPLATE/ │ ├── bug_report.md │ └── feature_request.md ├── .gitignore ├── AGENTS.md ├── Backend/ │ ├── db.py │ ├── gpt.py │ ├── logstream.py │ ├── main.py │ ├── models.py │ ├── pipeline.py │ ├── repository.py │ ├── search.py │ ├── tiktokvoice.py │ ├── utils.py │ ├── video.py │ ├── worker.py │ └── youtube.py ├── CLAUDE.md ├── Dockerfile ├── Frontend/ │ ├── app.js │ └── index.html ├── LICENSE ├── README.md ├── docker-compose.yml ├── docs/ │ ├── README.md │ ├── architecture.md │ ├── configuration.md │ ├── docker.md │ ├── quickstart.md │ ├── testing.md │ └── troubleshooting.md ├── pyproject.toml ├── setup.sh └── tests/ ├── conftest.py ├── test_api_jobs.py ├── test_api_misc.py ├── test_repository.py ├── test_utils.py └── test_worker.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/FUNDING.yml ================================================ # These are supported funding model platforms github: [FujiwaraChoki] ================================================ FILE: .github/ISSUE_TEMPLATE/bug_report.md ================================================ --- name: Bug report about: Create a report to help us improve title: "[BUG]" labels: '' assignees: FujiwaraChoki --- **Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - OS: [e.g. Linux, Windows] - Browser [e.g. chrome, edge] - Python Version [e.g. 3.9] **Additional context** Add any other context about the problem here. ================================================ FILE: .github/ISSUE_TEMPLATE/feature_request.md ================================================ --- name: Feature request about: Suggest an idea for this project title: '' labels: '' assignees: '' --- **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here. ================================================ FILE: .gitignore ================================================ __pycache__ .env temp/* sounds/* output/* images/* *.zip *.srt *.mp4 *.mp3 .history subtitles/* /venv .venv client_secret.json main.py-oauth2.json .DS_Store Backend/output* Songs/ venv/ ================================================ FILE: AGENTS.md ================================================ # AGENTS Guide for MoneyPrinter This file is the operating manual for coding agents working in this repository. Follow it before making changes. ## 1) Repository Layout - `Backend/`: Flask API, DB-backed job queue, and video generation pipeline. - `Frontend/`: static HTML/JS client served by `python -m http.server`. - `docs/`: source-of-truth setup and runtime docs. - `fonts/`, `Songs/`, `subtitles/`, `temp/`: runtime assets/output folders. - Root output artifact: `output.mp4`. ## 2) Source of Truth and Existing Rules - No `.cursor/rules/` directory found. - No `.cursorrules` file found. - No `.github/copilot-instructions.md` file found. - If any of the above appear later, treat them as higher-priority constraints and update this file. ## 3) Environment and Setup Commands - Python version: `>=3.11` (from `pyproject.toml`). - Dependency manager used in docs: `uv`. - Create local env file: `cp .env.example .env`. - Install dependencies: `uv sync`. - Run backend: `uv run python Backend/main.py`. - Run worker (new terminal): `uv run python Backend/worker.py`. - Run frontend (new terminal): `python3 -m http.server 3000 --directory Frontend`. - Docker workflow: `docker compose up --build`. ## 4) Build, Lint, and Test Commands This project has a baseline `pytest` setup for backend repository tests. Use the commands below as the expected agent workflow. ### 4.1 Build / Runtime Verification - Backend syntax check: `uv run python -m compileall Backend`. - Frontend syntax sanity (lightweight): open `Frontend/index.html` in browser and run generation flow. - API smoke check after backend start: `curl http://localhost:8080/api/models`. - Queue smoke check: `curl -X POST http://localhost:8080/api/generate -H "Content-Type: application/json" -d '{"videoSubject":"test","voice":"en_us_001","paragraphNumber":1,"customPrompt":""}'`. - Full local run: backend + worker + frontend servers, then generate a short sample video. ### 4.2 Lint / Formatting (Recommended) - There is no enforced formatter in-repo today. - Follow existing style and keep diffs minimal. - If linting is requested, prefer adding tooling in a separate PR. - Suggested ad-hoc checks when available locally: - `uv run python -m py_compile Backend/*.py` - `uv run python -m compileall Backend` ### 4.3 Test Commands (Current and Future) - Run all tests: `uv run pytest` - Run one file: `uv run pytest tests/test_file.py` - Run a single test: `uv run pytest tests/test_file.py::test_name` - Run a single class test: `uv run pytest tests/test_file.py::TestClass::test_name` - Current suite location: `tests/`. ## 5) High-Confidence Conventions from Existing Code These conventions are inferred from current source and should guide new changes. ### 5.1 Python Imports - Prefer standard library imports first, then third-party, then local modules. - Use one import per line for readability in long modules. - Avoid wildcard imports in new code (`from module import *`), even if legacy files use them. - Prefer explicit local imports, e.g. `from utils import ENV_FILE, TEMP_DIR`. ### 5.2 Formatting and Structure - Use 4-space indentation in Python. - Keep line length readable; split long calls across multiple lines. - Favor small helper functions for distinct pipeline stages. - Keep side-effectful startup logic near application boot (`load_dotenv`, env checks). ### 5.3 Typing and Signatures - Add type hints to all new/modified function signatures. - Reuse `Optional`, `List`, `Tuple`, `dict` typing already used in backend. - Prefer explicit return types (`-> str`, `-> None`, `-> Tuple[...]`). - Use `Path` for filesystem paths where practical. ### 5.4 Naming Conventions - Python functions/variables: `snake_case`. - Constants/env keys: `UPPER_SNAKE_CASE`. - JS variables/functions in frontend: `camelCase`. - Keep API route names simple and verb-oriented (`/api/generate`, `/api/cancel`). ### 5.5 Error Handling and Logging - Fail fast on missing critical env vars (current code exits early in startup checks). - Catch exceptions at boundary layers (HTTP handlers, external API calls, file IO). - Return user-safe JSON error messages from Flask endpoints. - Log actionable context with existing logger/log-stream helpers. - Do not swallow exceptions silently; at minimum emit error logs. ### 5.6 Filesystem and Path Safety - Prefer `pathlib.Path` operations. - Ensure directories exist before writing (`mkdir(parents=True, exist_ok=True)`). - Sanitize uploaded filenames (`os.path.basename`) before save. - Avoid hardcoded OS-specific paths; rely on env vars and `Path.resolve()`. ### 5.7 Backend API Patterns - Keep endpoint payloads consistent with `{"status": "success|error", ...}`. - Use appropriate HTTP status codes for conflict/client errors (e.g., `409`, `400`). - Long-running work should run in worker process from DB queue, not on request thread. - Preserve cancellation semantics using per-job cancellation and persisted job events. ### 5.8 Frontend Patterns - Use centralized API helper (`apiRequest`) for backend calls. - Validate required fields before firing requests. - Keep user feedback explicit via toasts and status area toggles. - Preserve localStorage key patterns (`Value`). ## 6) Change Scope Rules for Agents - Make minimal, targeted edits. - Do not rename files/modules unless required by task. - Do not introduce new frameworks/toolchains without request. - Keep backward compatibility for existing API payload shape when possible. - Update docs in `docs/` when setup, env vars, or runtime behavior changes. ## 7) Validation Checklist Before Finishing - Ran relevant command(s) from section 4. - Confirmed backend still starts (`uv run python Backend/main.py`). - Confirmed worker still starts (`uv run python Backend/worker.py`). - Confirmed frontend still loads (`python3 -m http.server 3000 --directory Frontend`). - Verified changed endpoints still return JSON and preserve `status` field. - Checked no secrets were added to tracked files. ## 8) Notes for Future Tooling PRs - Keep tests standardized on `pytest` and document exact paths/selectors here. - If adding linting, prefer `ruff` for lint + format and commit config files. - If adding type checks, document command and strictness level (`mypy` or equivalent). - Keep this file updated whenever workflow commands change. ## 9) Agent Workflow Expectations - Prefer minimal diffs and preserve current behavior unless the task requires changes. - Keep API responses machine-parseable and consistent for frontend consumers. - Avoid checking in generated media/output artifacts unless explicitly requested. - Before returning work, include what was validated and what was not validated. - When adding commands or tooling, update this file and `docs/` together. ================================================ FILE: Backend/db.py ================================================ import os from sqlalchemy import create_engine from sqlalchemy.orm import DeclarativeBase, sessionmaker from dotenv import load_dotenv from utils import ENV_FILE load_dotenv(ENV_FILE) class Base(DeclarativeBase): pass def _database_url() -> str: database_url = os.getenv("DATABASE_URL") if database_url: return database_url return "sqlite:///moneyprinter.db" DATABASE_URL = _database_url() engine = create_engine( DATABASE_URL, pool_pre_ping=True, connect_args={"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {}, ) SessionLocal = sessionmaker( bind=engine, autoflush=False, autocommit=False, expire_on_commit=False ) def init_db() -> None: from models import Artifact, GenerationEvent, GenerationJob, Project, Script # noqa: F401 Base.metadata.create_all(bind=engine) ================================================ FILE: Backend/gpt.py ================================================ import re import os import json from ollama import Client, ResponseError from dotenv import load_dotenv from logstream import log from typing import Tuple, List, Optional from utils import ENV_FILE # Load environment variables load_dotenv(ENV_FILE) # Set environment variables OLLAMA_BASE_URL = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434").rstrip("/") OLLAMA_MODEL = os.getenv("OLLAMA_MODEL", "llama3.1:8b") OLLAMA_TIMEOUT = float(os.getenv("OLLAMA_TIMEOUT", "180")) def _ollama_client() -> Client: return Client(host=OLLAMA_BASE_URL, timeout=OLLAMA_TIMEOUT) def _extract_model_name(model_obj) -> str: if hasattr(model_obj, "model") and getattr(model_obj, "model"): return str(getattr(model_obj, "model")).strip() if hasattr(model_obj, "name") and getattr(model_obj, "name"): return str(getattr(model_obj, "name")).strip() if isinstance(model_obj, dict): return str(model_obj.get("model") or model_obj.get("name") or "").strip() return "" def list_ollama_models() -> Tuple[List[str], str]: """ Returns available Ollama model names and configured default model. Returns: Tuple[List[str], str]: (available model names, default model) """ try: response = _ollama_client().list() except Exception as err: raise RuntimeError(f"Failed to fetch Ollama models: {err}") from err models = [] if hasattr(response, "models") and getattr(response, "models") is not None: models = list(getattr(response, "models")) elif isinstance(response, dict): models = response.get("models") or [] model_names = [_extract_model_name(model) for model in models] model_names = [name for name in model_names if name] unique_names = list(dict.fromkeys(model_names)) if OLLAMA_MODEL and OLLAMA_MODEL in unique_names: default_model = OLLAMA_MODEL elif unique_names: default_model = unique_names[0] else: default_model = OLLAMA_MODEL if OLLAMA_MODEL else "" return unique_names, default_model def generate_response(prompt: str, ai_model: str) -> str: """ Generate a script for a video, depending on the subject of the video. Args: video_subject (str): The subject of the video. ai_model (str): The AI model to use for generation. Returns: str: The response from the AI model. """ model_name = (ai_model or "").strip() or OLLAMA_MODEL try: client = _ollama_client() try: response = client.chat( model=model_name, messages=[{"role": "user", "content": prompt}], stream=False, ) except ResponseError as err: if err.status_code == 404: try: response = client.generate( model=model_name, prompt=prompt, stream=False ) except ResponseError as fallback_err: if ( fallback_err.status_code == 404 and "not found" in str(fallback_err).lower() ): available_models, _ = list_ollama_models() available = ( ", ".join(available_models) if available_models else "none" ) raise RuntimeError( f"Ollama model '{model_name}' is not installed. Available models: {available}. " f"Install it with: ollama pull {model_name}" ) from fallback_err raise else: raise except RuntimeError: raise except Exception as err: raise RuntimeError(f"Failed to connect to Ollama: {err}") from err content = "" if hasattr(response, "message") and getattr(response, "message") is not None: message = getattr(response, "message") if hasattr(message, "content") and getattr(message, "content"): content = str(getattr(message, "content")).strip() elif isinstance(message, dict): content = str(message.get("content") or "").strip() if not content: if hasattr(response, "response") and getattr(response, "response"): content = str(getattr(response, "response")).strip() elif isinstance(response, dict): content = ( str(response.get("message", {}).get("content") or "") or str(response.get("response") or "") ).strip() if not content: raise RuntimeError("Ollama returned an empty response.") return content def generate_script( video_subject: str, paragraph_number: int, ai_model: str, voice: str, customPrompt: str, ) -> Optional[str]: """ Generate a script for a video, depending on the subject of the video, the number of paragraphs, and the AI model. Args: video_subject (str): The subject of the video. paragraph_number (int): The number of paragraphs to generate. ai_model (str): The AI model to use for generation. Returns: str: The script for the video. """ # Build prompt if customPrompt: prompt = customPrompt else: prompt = """ Generate a script for a video, depending on the subject of the video. The script is to be returned as a string with the specified number of paragraphs. Here is an example of a string: "This is an example string." Do not under any circumstance reference this prompt in your response. Get straight to the point, don't start with unnecessary things like, "welcome to this video". Obviously, the script should be related to the subject of the video. YOU MUST NOT INCLUDE ANY TYPE OF MARKDOWN OR FORMATTING IN THE SCRIPT, NEVER USE A TITLE. YOU MUST WRITE THE SCRIPT IN THE LANGUAGE SPECIFIED IN [LANGUAGE]. ONLY RETURN THE RAW CONTENT OF THE SCRIPT. DO NOT INCLUDE "VOICEOVER", "NARRATOR" OR SIMILAR INDICATORS OF WHAT SHOULD BE SPOKEN AT THE BEGINNING OF EACH PARAGRAPH OR LINE. YOU MUST NOT MENTION THE PROMPT, OR ANYTHING ABOUT THE SCRIPT ITSELF. ALSO, NEVER TALK ABOUT THE AMOUNT OF PARAGRAPHS OR LINES. JUST WRITE THE SCRIPT. """ prompt += f""" Subject: {video_subject} Number of paragraphs: {paragraph_number} Language: {voice} """ # Generate script response = generate_response(prompt, ai_model) log(response, "info") # Return the generated script if response: # Clean the script # Remove asterisks, hashes response = response.replace("*", "") response = response.replace("#", "") # Remove markdown syntax response = re.sub(r"\[.*\]", "", response) response = re.sub(r"\(.*\)", "", response) # Split the script into paragraphs paragraphs = response.split("\n\n") # Select the specified number of paragraphs selected_paragraphs = paragraphs[:paragraph_number] # Join the selected paragraphs into a single string final_script = "\n\n".join(selected_paragraphs) # Print to console the number of paragraphs used log(f"Number of paragraphs used: {len(selected_paragraphs)}", "success") return final_script else: log("[-] GPT returned an empty response.", "error") return None def get_search_terms( video_subject: str, amount: int, script: str, ai_model: str ) -> List[str]: """ Generate a JSON-Array of search terms for stock videos, depending on the subject of a video. Args: video_subject (str): The subject of the video. amount (int): The amount of search terms to generate. script (str): The script of the video. ai_model (str): The AI model to use for generation. Returns: List[str]: The search terms for the video subject. """ # Build prompt prompt = f""" Generate {amount} search terms for stock videos, depending on the subject of a video. Subject: {video_subject} The search terms are to be returned as a JSON-Array of strings. Each search term should consist of 1-3 words, always add the main subject of the video. YOU MUST ONLY RETURN THE JSON-ARRAY OF STRINGS. YOU MUST NOT RETURN ANYTHING ELSE. YOU MUST NOT RETURN THE SCRIPT. The search terms must be related to the subject of the video. Here is an example of a JSON-Array of strings: ["search term 1", "search term 2", "search term 3"] For context, here is the full text: {script} """ # Generate search terms response = generate_response(prompt, ai_model) log(response, "info") # Parse response into a list of search terms search_terms = [] try: search_terms = json.loads(response) if not isinstance(search_terms, list) or not all( isinstance(term, str) for term in search_terms ): raise ValueError("Response is not a list of strings.") except (json.JSONDecodeError, ValueError): log("[*] GPT returned an unformatted response. Attempting to clean...", "warning") # Attempt to extract JSON array first match = re.search(r"\[[\s\S]*\]", response) if match: try: search_terms = json.loads(match.group()) except json.JSONDecodeError: search_terms = [] # Last-resort fallback: collect quoted strings if not search_terms: search_terms = re.findall(r'"([^"\\]*(?:\\.[^"\\]*)*)"', response) search_terms = [term.strip() for term in search_terms if term.strip()] # Let user know log(f"\nGenerated {len(search_terms)} search terms: {', '.join(search_terms)}", "info") # Return search terms return search_terms def generate_metadata( video_subject: str, script: str, ai_model: str ) -> Tuple[str, str, List[str]]: """ Generate metadata for a YouTube video, including the title, description, and keywords. Args: video_subject (str): The subject of the video. script (str): The script of the video. ai_model (str): The AI model to use for generation. Returns: Tuple[str, str, List[str]]: The title, description, and keywords for the video. """ # Build prompt for title title_prompt = f""" Generate a catchy and SEO-friendly title for a YouTube shorts video about {video_subject}. """ # Generate title title = generate_response(title_prompt, ai_model).strip() # Build prompt for description description_prompt = f""" Write a brief and engaging description for a YouTube shorts video about {video_subject}. The video is based on the following script: {script} """ # Generate description description = generate_response(description_prompt, ai_model).strip() # Generate keywords keywords = get_search_terms(video_subject, 6, script, ai_model) return title, description, keywords ================================================ FILE: Backend/logstream.py ================================================ import json import queue import re import time class LogStream: """Thread-safe log queue that doubles as an SSE generator.""" def __init__(self, maxsize: int = 500): self._queue: queue.Queue = queue.Queue(maxsize=maxsize) def clear(self) -> None: """Drain all pending items from the queue.""" while True: try: self._queue.get_nowait() except queue.Empty: break def push(self, message: str, level: str = "info") -> None: """Add a log entry. Drops the oldest entry if the queue is full.""" entry = { "type": "log", "message": message, "level": level, "timestamp": time.time(), } try: self._queue.put_nowait(entry) except queue.Full: try: self._queue.get_nowait() except queue.Empty: pass try: self._queue.put_nowait(entry) except queue.Full: pass def push_event(self, event_type: str, data: dict | None = None) -> None: """Send a control event (complete, error, cancelled).""" entry = { "type": event_type, "timestamp": time.time(), **(data or {}), } try: self._queue.put_nowait(entry) except queue.Full: try: self._queue.get_nowait() except queue.Empty: pass try: self._queue.put_nowait(entry) except queue.Full: pass def stream(self, timeout: float = 30.0): """Generator yielding SSE-formatted lines. Terminates on control events.""" while True: try: entry = self._queue.get(timeout=timeout) yield f"data: {json.dumps(entry)}\n\n" if entry.get("type") in ("complete", "error", "cancelled"): return except queue.Empty: # Send keepalive comment to prevent connection timeout yield ": keepalive\n\n" # Strip ANSI escape codes from terminal-colored strings _ANSI_RE = re.compile(r"\x1b\[[0-9;]*m") # Singleton shared by all modules log_stream = LogStream() # Color-to-level mapping for termcolor colors _COLOR_LEVEL = { "green": "success", "red": "error", "yellow": "warning", "blue": "info", "cyan": "info", "magenta": "info", } def log(message: str, level: str = "info") -> None: """Drop-in replacement for ``print(colored(msg, color))``. Prints to the terminal **and** pushes an ANSI-stripped copy to the SSE queue. """ print(message) clean = _ANSI_RE.sub("", str(message)) log_stream.push(clean, level) ================================================ FILE: Backend/main.py ================================================ import os from dotenv import load_dotenv from flask import Flask, jsonify, request from flask_cors import CORS from sqlalchemy import and_, case, select from db import SessionLocal, init_db from gpt import list_ollama_models from logstream import log from repository import create_job, get_job, list_job_events, request_cancel from utils import ENV_FILE, SONGS_DIR, check_env_vars, clean_dir load_dotenv(ENV_FILE) check_env_vars() init_db() app = Flask(__name__) CORS(app) HOST = "0.0.0.0" PORT = 8080 @app.route("/api/models", methods=["GET"]) def models(): try: available_models, default_model = list_ollama_models() return jsonify( { "status": "success", "models": available_models, "default": default_model, } ) except Exception as err: log(f"[-] Error fetching Ollama models: {str(err)}", "error") return jsonify( { "status": "error", "message": "Could not fetch Ollama models. Is Ollama running?", "models": [os.getenv("OLLAMA_MODEL", "llama3.1:8b")], "default": os.getenv("OLLAMA_MODEL", "llama3.1:8b"), } ) @app.route("/api/generate", methods=["POST"]) def generate(): data = request.get_json() or {} if not data.get("videoSubject"): return jsonify({"status": "error", "message": "videoSubject is required."}), 400 with SessionLocal() as session: job = create_job(session, payload=data) return jsonify( { "status": "success", "message": "Video generation queued.", "jobId": job.id, } ) @app.route("/api/jobs/", methods=["GET"]) def get_job_status(job_id: str): with SessionLocal() as session: job = get_job(session, job_id) if not job: return jsonify({"status": "error", "message": "Job not found."}), 404 return jsonify( { "status": "success", "job": { "id": job.id, "state": job.status, "cancelRequested": job.cancel_requested, "resultPath": job.result_path, "errorMessage": job.error_message, "createdAt": job.created_at.isoformat() if job.created_at else None, "startedAt": job.started_at.isoformat() if job.started_at else None, "completedAt": job.completed_at.isoformat() if job.completed_at else None, }, } ) @app.route("/api/jobs//events", methods=["GET"]) def get_events(job_id: str): after_id = request.args.get("after", default=0, type=int) with SessionLocal() as session: job = get_job(session, job_id) if not job: return jsonify({"status": "error", "message": "Job not found."}), 404 events = list_job_events(session, job_id, after_id=after_id) return jsonify( { "status": "success", "events": [ { "id": event.id, "type": event.event_type, "level": event.level, "message": event.message, "payload": event.payload, "timestamp": event.created_at.timestamp() if event.created_at else None, } for event in events ], } ) @app.route("/api/jobs//cancel", methods=["POST"]) def cancel_job(job_id: str): with SessionLocal() as session: cancelled = request_cancel(session, job_id) if not cancelled: return jsonify({"status": "error", "message": "Job not found."}), 404 return jsonify({"status": "success", "message": "Cancellation requested."}) @app.route("/api/upload-songs", methods=["POST"]) def upload_songs(): try: files = request.files.getlist("songs") if not files: return jsonify({"status": "error", "message": "No files uploaded."}), 400 clean_dir(str(SONGS_DIR)) saved = 0 for file_item in files: if file_item.filename and file_item.filename.lower().endswith(".mp3"): safe_name = os.path.basename(file_item.filename) file_item.save(str(SONGS_DIR / safe_name)) saved += 1 if saved == 0: return jsonify({"status": "error", "message": "No MP3 files found."}), 400 log(f"[+] Uploaded {saved} song(s) to {SONGS_DIR}", "success") return jsonify({"status": "success", "message": f"Uploaded {saved} song(s)."}) except Exception as err: log(f"[-] Error uploading songs: {str(err)}", "error") return jsonify({"status": "error", "message": str(err)}), 500 @app.route("/api/cancel", methods=["POST"]) def cancel_latest_running_job(): with SessionLocal() as session: from models import GenerationJob stmt = ( select(GenerationJob) .where(and_(GenerationJob.status.in_(["queued", "running"]))) .order_by( case((GenerationJob.status == "running", 0), else_=1), GenerationJob.created_at.desc(), ) .limit(1) ) latest_job = session.scalars(stmt).first() if not latest_job: return jsonify({"status": "error", "message": "No active job found."}), 404 request_cancel(session, latest_job.id) return jsonify( { "status": "success", "message": "Cancellation requested.", "jobId": latest_job.id, } ) if __name__ == "__main__": app.run(debug=True, host=HOST, port=PORT, threaded=True) ================================================ FILE: Backend/models.py ================================================ from datetime import datetime from typing import Optional from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, JSON, String, Text, func from sqlalchemy.orm import Mapped, mapped_column, relationship from db import Base class Project(Base): __tablename__ = "projects" id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) name: Mapped[str] = mapped_column(String(255), nullable=False, unique=True) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), nullable=False ) class GenerationJob(Base): __tablename__ = "generation_jobs" id: Mapped[str] = mapped_column(String(36), primary_key=True) project_id: Mapped[Optional[int]] = mapped_column( Integer, ForeignKey("projects.id", ondelete="SET NULL"), nullable=True, index=True, ) status: Mapped[str] = mapped_column(String(20), nullable=False, index=True) payload: Mapped[dict] = mapped_column(JSON, nullable=False) cancel_requested: Mapped[bool] = mapped_column( Boolean, nullable=False, default=False ) attempt_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0) max_attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=1) result_path: Mapped[Optional[str]] = mapped_column(String(512), nullable=True) error_message: Mapped[Optional[str]] = mapped_column(Text, nullable=True) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), nullable=False, index=True ) started_at: Mapped[Optional[datetime]] = mapped_column( DateTime(timezone=True), nullable=True ) completed_at: Mapped[Optional[datetime]] = mapped_column( DateTime(timezone=True), nullable=True ) updated_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False, ) project: Mapped[Optional[Project]] = relationship("Project") events: Mapped[list["GenerationEvent"]] = relationship( "GenerationEvent", back_populates="job", cascade="all, delete-orphan" ) class GenerationEvent(Base): __tablename__ = "generation_events" id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) job_id: Mapped[str] = mapped_column( String(36), ForeignKey("generation_jobs.id", ondelete="CASCADE"), nullable=False, index=True, ) event_type: Mapped[str] = mapped_column(String(20), nullable=False, default="log") level: Mapped[str] = mapped_column(String(20), nullable=False, default="info") message: Mapped[str] = mapped_column(Text, nullable=False) payload: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), nullable=False, index=True ) job: Mapped[GenerationJob] = relationship("GenerationJob", back_populates="events") class Script(Base): __tablename__ = "scripts" id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) job_id: Mapped[str] = mapped_column( String(36), ForeignKey("generation_jobs.id", ondelete="CASCADE"), nullable=False, index=True, ) model_name: Mapped[Optional[str]] = mapped_column(String(255), nullable=True) content: Mapped[str] = mapped_column(Text, nullable=False) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), nullable=False ) class Artifact(Base): __tablename__ = "artifacts" id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) job_id: Mapped[str] = mapped_column( String(36), ForeignKey("generation_jobs.id", ondelete="CASCADE"), nullable=False, index=True, ) artifact_type: Mapped[str] = mapped_column(String(64), nullable=False) path: Mapped[str] = mapped_column(String(512), nullable=False) metadata_json: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True) created_at: Mapped[datetime] = mapped_column( DateTime(timezone=True), server_default=func.now(), nullable=False ) ================================================ FILE: Backend/pipeline.py ================================================ import os import shutil import subprocess from apiclient.errors import HttpError from moviepy import ( AudioFileClip, CompositeAudioClip, VideoFileClip, afx, concatenate_audioclips, ) from uuid import uuid4 from gpt import generate_metadata, generate_script, get_search_terms from logstream import log from search import search_for_stock_videos from tiktokvoice import tts from utils import ( BASE_DIR, PROJECT_ROOT, SONGS_DIR, SUBTITLES_DIR, TEMP_DIR, choose_random_song, ) from video import combine_videos, generate_subtitles, generate_video, save_video from youtube import upload_video class PipelineCancelled(Exception): pass def run_generation_pipeline( data: dict, is_cancelled, on_log, amount_of_stock_videos: int = 5, ) -> str: def emit(message: str, level: str = "info") -> None: log(message, level) if on_log: on_log(message, level) def guard_cancelled() -> None: if is_cancelled and is_cancelled(): raise PipelineCancelled("Video generation was cancelled.") paragraph_number = int(data.get("paragraphNumber", 1)) ai_model = data.get("aiModel") n_threads = data.get("threads") subtitles_position = data.get("subtitlesPosition") text_color = data.get("color") use_music = data.get("useMusic", False) automate_youtube_upload = data.get("automateYoutubeUpload", False) emit("[Video to be generated]", "info") emit(" Subject: " + data["videoSubject"], "info") emit(" AI Model: " + str(ai_model), "info") emit(" Custom Prompt: " + data["customPrompt"], "info") guard_cancelled() voice = data.get("voice", "") voice_prefix = voice[:2] if not voice: emit('[!] No voice was selected. Defaulting to "en_us_001"', "warning") voice = "en_us_001" voice_prefix = voice[:2] script = generate_script( data["videoSubject"], paragraph_number, ai_model, voice, data["customPrompt"], ) if not script: raise RuntimeError( "Could not generate a script. Try a different model or prompt." ) search_terms = get_search_terms( data["videoSubject"], amount_of_stock_videos, script, ai_model ) video_urls = [] it = 15 min_dur = 10 for search_term in search_terms: guard_cancelled() found_urls = search_for_stock_videos( search_term, os.getenv("PEXELS_API_KEY"), it, min_dur ) for url in found_urls: if url not in video_urls: video_urls.append(url) break if not video_urls: raise RuntimeError("No videos found to download.") video_paths = [] emit(f"[+] Downloading {len(video_urls)} videos...", "info") for video_url in video_urls: guard_cancelled() try: saved_video_path = save_video(video_url) video_paths.append(saved_video_path) except Exception: emit(f"[-] Could not download video: {video_url}", "error") emit("[+] Videos downloaded!", "success") emit("[+] Script generated!", "success") guard_cancelled() sentences = script.split(". ") sentences = list(filter(lambda x: x != "", sentences)) paths = [] for sentence in sentences: guard_cancelled() current_tts_path = str(TEMP_DIR / f"{uuid4()}.mp3") tts(sentence, voice, filename=current_tts_path) audio_clip = AudioFileClip(current_tts_path) paths.append(audio_clip) final_audio = concatenate_audioclips(paths) tts_path = str(TEMP_DIR / f"{uuid4()}.mp3") try: final_audio.write_audiofile(tts_path) finally: final_audio.close() for audio_clip in paths: audio_clip.close() try: subtitles_path = generate_subtitles( audio_path=tts_path, sentences=sentences, audio_clips=paths, voice=voice_prefix, ) except Exception as err: emit(f"[-] Error generating subtitles: {err}", "error") subtitles_path = None if not subtitles_path: raise RuntimeError( "Could not generate subtitles. Check AssemblyAI key or local subtitle settings." ) temp_audio = AudioFileClip(tts_path) try: combined_video_path = combine_videos( video_paths, temp_audio.duration, 5, n_threads or 2 ) finally: temp_audio.close() try: final_video_path = generate_video( combined_video_path, tts_path, subtitles_path, n_threads or 2, subtitles_position, text_color or "#FFFF00", ) except Exception as err: raise RuntimeError( f"Could not render final video. Check subtitle/font/ImageMagick setup. ({err})" ) from err title, description, keywords = generate_metadata( data["videoSubject"], script, ai_model ) emit("[-] Metadata for YouTube upload:", "info") emit(" Title:", "info") emit(f" {title}", "info") emit(" Description:", "info") emit(f" {description}", "info") emit(" Keywords:", "info") emit(f" {', '.join(keywords)}", "info") if automate_youtube_upload: client_secrets_file = str((BASE_DIR / "client_secret.json").resolve()) skip_yt_upload = False if not os.path.exists(client_secrets_file): skip_yt_upload = True emit( "[-] Client secrets file missing. YouTube upload will be skipped.", "warning", ) emit( "[-] Please download the client_secret.json from Google Cloud Platform and store this inside the /Backend directory.", "error", ) if not skip_yt_upload: video_category_id = "28" privacy_status = "private" video_metadata = { "video_path": str((TEMP_DIR / final_video_path).resolve()), "title": title, "description": description, "category": video_category_id, "keywords": ",".join(keywords), "privacyStatus": privacy_status, } try: video_response = upload_video( video_path=video_metadata["video_path"], title=video_metadata["title"], description=video_metadata["description"], category=video_metadata["category"], keywords=video_metadata["keywords"], privacy_status=video_metadata["privacyStatus"], ) emit(f"Uploaded video ID: {video_response.get('id')}", "success") except HttpError as err: emit( f"An HTTP error {err.resp.status} occurred:\n{err.content}", "error" ) final_output_path = str(PROJECT_ROOT / final_video_path) rendered_video_path = str(TEMP_DIR / final_video_path) render_threads = n_threads or (os.cpu_count() or 2) guard_cancelled() if use_music: song_path = choose_random_song() if not song_path: emit( "[-] Could not find songs in Songs/. Continuing without background music.", "warning", ) use_music = False if use_music: video_clip = VideoFileClip(rendered_video_path) song_clip = None mixed_audio = None mixed_audio_path = str(TEMP_DIR / f"{uuid4()}_mixed_audio.m4a") try: original_duration = video_clip.duration original_audio = video_clip.audio song_clip = AudioFileClip(song_path).with_fps(44100) song_clip = song_clip.with_effects( [afx.AudioLoop(duration=original_duration)] ) song_clip = song_clip.with_volume_scaled(0.1).with_fps(44100) mixed_audio = CompositeAudioClip( [original_audio, song_clip] ).with_duration(original_duration) mixed_audio.write_audiofile( mixed_audio_path, fps=44100, codec="aac", bitrate="192k", ) finally: video_clip.close() if mixed_audio is not None: mixed_audio.close() if song_clip is not None: song_clip.close() try: subprocess.run( [ "ffmpeg", "-y", "-i", rendered_video_path, "-i", mixed_audio_path, "-map", "0:v:0", "-map", "1:a:0", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", "-shortest", final_output_path, ], check=True, capture_output=True, text=True, ) except Exception: emit( "[!] ffmpeg remux failed. Falling back to MoviePy render for music mix.", "warning", ) video_clip = VideoFileClip(rendered_video_path) song_clip = None try: original_duration = video_clip.duration original_audio = video_clip.audio song_clip = AudioFileClip(song_path).with_fps(44100) song_clip = song_clip.with_effects( [afx.AudioLoop(duration=original_duration)] ) song_clip = song_clip.with_volume_scaled(0.1).with_fps(44100) comp_audio = CompositeAudioClip( [original_audio, song_clip] ).with_duration(original_duration) video_clip = ( video_clip.with_audio(comp_audio) .with_fps(30) .with_duration(original_duration) ) video_clip.write_videofile( final_output_path, threads=render_threads, fps=30, codec="libx264", audio_codec="aac", preset="medium", ) finally: video_clip.close() if song_clip is not None: song_clip.close() finally: if os.path.exists(mixed_audio_path): os.remove(mixed_audio_path) if not use_music: shutil.copy2(rendered_video_path, final_output_path) emit(f"[+] Video generated: {final_video_path}!", "success") if os.name == "nt": subprocess.run( ["taskkill", "/f", "/im", "ffmpeg.exe"], check=False, capture_output=True, text=True, ) elif shutil.which("pkill"): subprocess.run( ["pkill", "-f", "ffmpeg"], check=False, capture_output=True, text=True, ) return final_video_path ================================================ FILE: Backend/repository.py ================================================ from datetime import datetime, timezone from typing import Optional from uuid import uuid4 from sqlalchemy import and_, select, text from sqlalchemy.orm import Session from models import GenerationEvent, GenerationJob def utcnow() -> datetime: return datetime.now(timezone.utc) def create_job(session: Session, payload: dict, max_attempts: int = 1) -> GenerationJob: job = GenerationJob( id=str(uuid4()), status="queued", payload=payload, max_attempts=max_attempts, cancel_requested=False, ) session.add(job) session.flush() append_event(session, job.id, "queued", "info", "Job queued.") session.commit() session.refresh(job) return job def append_event( session: Session, job_id: str, event_type: str, level: str, message: str, payload: Optional[dict] = None, ) -> GenerationEvent: event = GenerationEvent( job_id=job_id, event_type=event_type, level=level, message=message, payload=payload, ) session.add(event) session.flush() return event def get_job(session: Session, job_id: str) -> Optional[GenerationJob]: return session.get(GenerationJob, job_id) def list_job_events( session: Session, job_id: str, after_id: int = 0, limit: int = 200 ) -> list[GenerationEvent]: stmt = ( select(GenerationEvent) .where( and_( GenerationEvent.job_id == job_id, GenerationEvent.id > after_id, ) ) .order_by(GenerationEvent.id.asc()) .limit(limit) ) return list(session.scalars(stmt).all()) def request_cancel(session: Session, job_id: str) -> bool: job = get_job(session, job_id) if not job: return False if job.status in ("completed", "failed", "cancelled"): return True job.cancel_requested = True job.updated_at = utcnow() append_event( session, job.id, "cancel_requested", "warning", "Cancellation requested." ) if job.status == "queued": job.status = "cancelled" job.completed_at = utcnow() append_event( session, job.id, "cancelled", "warning", "Job cancelled before execution." ) session.commit() return True def claim_next_queued_job(session: Session) -> Optional[GenerationJob]: dialect = session.bind.dialect.name if session.bind else "" if dialect == "postgresql": row = session.execute( text( """ SELECT id FROM generation_jobs WHERE status = 'queued' AND cancel_requested = false ORDER BY created_at ASC FOR UPDATE SKIP LOCKED LIMIT 1 """ ) ).first() if not row: return None job = get_job(session, row[0]) else: stmt = ( select(GenerationJob) .where( and_( GenerationJob.status == "queued", GenerationJob.cancel_requested.is_(False), ) ) .order_by(GenerationJob.created_at.asc()) .limit(1) ) job = session.scalars(stmt).first() if not job: return None job.status = "running" job.attempt_count = (job.attempt_count or 0) + 1 job.started_at = utcnow() job.updated_at = utcnow() append_event(session, job.id, "running", "info", "Job started.") session.commit() session.refresh(job) return job def mark_completed(session: Session, job_id: str, result_path: str) -> None: job = get_job(session, job_id) if not job: return job.status = "completed" job.result_path = result_path job.error_message = None job.completed_at = utcnow() job.updated_at = utcnow() append_event( session, job.id, "complete", "success", "Video generated successfully.", {"path": result_path}, ) session.commit() def mark_cancelled( session: Session, job_id: str, reason: str = "Job cancelled." ) -> None: job = get_job(session, job_id) if not job: return job.status = "cancelled" job.completed_at = utcnow() job.updated_at = utcnow() append_event(session, job.id, "cancelled", "warning", reason) session.commit() def mark_failed(session: Session, job_id: str, error_message: str) -> None: job = get_job(session, job_id) if not job: return job.status = "failed" job.error_message = error_message job.completed_at = utcnow() job.updated_at = utcnow() append_event(session, job.id, "error", "error", error_message) session.commit() ================================================ FILE: Backend/search.py ================================================ import requests from typing import List from logstream import log def search_for_stock_videos(query: str, api_key: str, it: int, min_dur: int) -> List[str]: """ Searches for stock videos based on a query. Args: query (str): The query to search for. api_key (str): The API key to use. Returns: List[str]: A list of stock videos. """ # Build headers headers = { "Authorization": api_key } # Build URL qurl = f"https://api.pexels.com/videos/search?query={query}&per_page={it}" # Send the request r = requests.get(qurl, headers=headers) # Parse the response response = r.json() # Parse each video raw_urls = [] video_url = [] video_res = 0 try: # loop through each video in the result for i in range(it): #check if video has desired minimum duration if response["videos"][i]["duration"] < min_dur: continue raw_urls = response["videos"][i]["video_files"] temp_video_url = "" # loop through each url to determine the best quality for video in raw_urls: # Check if video has a valid download link if ".com/video-files" in video["link"]: # Only save the URL with the largest resolution if (video["width"]*video["height"]) > video_res: temp_video_url = video["link"] video_res = video["width"]*video["height"] # add the url to the return list if it's not empty if temp_video_url != "": video_url.append(temp_video_url) except Exception as e: log("[-] No Videos found.", "error") log(str(e), "error") # Let user know log(f"\t=> \"{query}\" found {len(video_url)} Videos", "info") # Return the video url return video_url ================================================ FILE: Backend/tiktokvoice.py ================================================ # author: GiorDior aka Giorgio # date: 12.06.2023 # topic: TikTok-Voice-TTS # version: 1.0 # credits: https://github.com/oscie57/tiktok-voice # --- MODIFIED VERSION --- # import base64 import requests import threading from typing import List from logstream import log from playsound import playsound VOICES = [ # DISNEY VOICES "en_us_ghostface", # Ghost Face "en_us_chewbacca", # Chewbacca "en_us_c3po", # C3PO "en_us_stitch", # Stitch "en_us_stormtrooper", # Stormtrooper "en_us_rocket", # Rocket # ENGLISH VOICES "en_au_001", # English AU - Female "en_au_002", # English AU - Male "en_uk_001", # English UK - Male 1 "en_uk_003", # English UK - Male 2 "en_us_001", # English US - Female (Int. 1) "en_us_002", # English US - Female (Int. 2) "en_us_006", # English US - Male 1 "en_us_007", # English US - Male 2 "en_us_009", # English US - Male 3 "en_us_010", # English US - Male 4 # EUROPE VOICES "fr_001", # French - Male 1 "fr_002", # French - Male 2 "de_001", # German - Female "de_002", # German - Male "es_002", # Spanish - Male # AMERICA VOICES "es_mx_002", # Spanish MX - Male "br_001", # Portuguese BR - Female 1 "br_003", # Portuguese BR - Female 2 "br_004", # Portuguese BR - Female 3 "br_005", # Portuguese BR - Male # ASIA VOICES "id_001", # Indonesian - Female "jp_001", # Japanese - Female 1 "jp_003", # Japanese - Female 2 "jp_005", # Japanese - Female 3 "jp_006", # Japanese - Male "kr_002", # Korean - Male 1 "kr_003", # Korean - Female "kr_004", # Korean - Male 2 # SINGING VOICES "en_female_f08_salut_damour", # Alto "en_male_m03_lobby", # Tenor "en_female_f08_warmy_breeze", # Warmy Breeze "en_male_m03_sunshine_soon", # Sunshine Soon # OTHER "en_male_narration", # narrator "en_male_funny", # wacky "en_female_emotional", # peaceful ] ENDPOINTS = [ "https://tiktok-tts.weilnet.workers.dev/api/generation", "https://tiktoktts.com/api/tiktok-tts", ] current_endpoint = 0 # in one conversion, the text can have a maximum length of 300 characters TEXT_BYTE_LIMIT = 300 # create a list by splitting a string, every element has n chars def split_string(string: str, chunk_size: int) -> List[str]: words = string.split() result = [] current_chunk = "" for word in words: if ( len(current_chunk) + len(word) + 1 <= chunk_size ): # Check if adding the word exceeds the chunk size current_chunk += f" {word}" else: if current_chunk: # Append the current chunk if not empty result.append(current_chunk.strip()) current_chunk = word if current_chunk: # Append the last chunk if not empty result.append(current_chunk.strip()) return result # checking if the website that provides the service is available def get_api_response() -> requests.Response: url = f'{ENDPOINTS[current_endpoint].split("/a")[0]}' response = requests.get(url) return response # saving the audio file def save_audio_file(base64_data: str, filename: str = "output.mp3") -> None: audio_bytes = base64.b64decode(base64_data) with open(filename, "wb") as file: file.write(audio_bytes) # send POST request to get the audio data def generate_audio(text: str, voice: str) -> bytes: url = f"{ENDPOINTS[current_endpoint]}" headers = {"Content-Type": "application/json"} data = {"text": text, "voice": voice} response = requests.post(url, headers=headers, json=data) return response.content # creates an text to speech audio file def tts( text: str, voice: str = "none", filename: str = "output.mp3", play_sound: bool = False, ) -> None: # checking if the website is available global current_endpoint if get_api_response().status_code == 200: log("[+] TikTok TTS Service available!", "success") else: current_endpoint = (current_endpoint + 1) % 2 if get_api_response().status_code == 200: log("[+] TTS Service available!", "success") else: log("[-] TTS Service not available and probably temporarily rate limited, try again later...", "error") return # checking if arguments are valid if voice == "none": log("[-] Please specify a voice", "error") return if voice not in VOICES: log("[-] Voice not available", "error") return if not text: log("[-] Please specify a text", "error") return # creating the audio file try: if len(text) < TEXT_BYTE_LIMIT: audio = generate_audio((text), voice) if current_endpoint == 0: audio_base64_data = str(audio).split('"')[5] else: audio_base64_data = str(audio).split('"')[3].split(",")[1] if audio_base64_data == "error": log("[-] This voice is unavailable right now", "error") return else: # Split longer text into smaller parts text_parts = split_string(text, 299) audio_base64_data = [None] * len(text_parts) # Define a thread function to generate audio for each text part def generate_audio_thread(text_part, index): audio = generate_audio(text_part, voice) if current_endpoint == 0: base64_data = str(audio).split('"')[5] else: base64_data = str(audio).split('"')[3].split(",")[1] if audio_base64_data == "error": log("[-] This voice is unavailable right now", "error") return "error" audio_base64_data[index] = base64_data threads = [] for index, text_part in enumerate(text_parts): # Create and start a new thread for each text part thread = threading.Thread( target=generate_audio_thread, args=(text_part, index) ) thread.start() threads.append(thread) # Wait for all threads to complete for thread in threads: thread.join() # Concatenate the base64 data in the correct order audio_base64_data = "".join(audio_base64_data) save_audio_file(audio_base64_data, filename) log(f"[+] Audio file saved successfully as '{filename}'", "success") if play_sound: playsound(filename) except Exception as e: log(f"[-] An error occurred during TTS: {e}", "error") ================================================ FILE: Backend/utils.py ================================================ import os import sys import random import logging import shutil from pathlib import Path from typing import Optional from termcolor import colored BASE_DIR = Path(__file__).resolve().parent PROJECT_ROOT = BASE_DIR.parent TEMP_DIR = PROJECT_ROOT / "temp" SUBTITLES_DIR = PROJECT_ROOT / "subtitles" SONGS_DIR = PROJECT_ROOT / "Songs" FONTS_DIR = PROJECT_ROOT / "fonts" ENV_FILE = PROJECT_ROOT / ".env" # Configure logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) def clean_dir(path: str) -> None: """ Removes every file in a directory. Args: path (str): Path to directory. Returns: None """ try: directory = Path(path).expanduser().resolve() directory.mkdir(parents=True, exist_ok=True) logger.info(f"Ensured directory exists: {directory}") for entry in directory.iterdir(): if entry.is_dir(): shutil.rmtree(entry) logger.info(f"Removed directory: {entry}") else: entry.unlink(missing_ok=True) logger.info(f"Removed file: {entry}") logger.info(colored(f"Cleaned {directory} directory", "green")) except Exception as e: logger.error(f"Error occurred while cleaning directory {path}: {str(e)}") def choose_random_song() -> Optional[str]: """ Chooses a random MP3 from the Songs/ directory. Returns: str: The path to the chosen song, or None if no MP3 files found. """ try: if not SONGS_DIR.exists(): return None songs = [ song for song in SONGS_DIR.iterdir() if song.is_file() and song.suffix.lower() == ".mp3" ] if not songs: return None song = random.choice(songs) logger.info(colored(f"Chose song: {song}", "green")) return str(song) except Exception as e: logger.error( colored(f"Error occurred while choosing random song: {str(e)}", "red") ) def resolve_imagemagick_binary() -> Optional[str]: """ Resolves an ImageMagick executable path across Linux, macOS, and Windows. Returns: Optional[str]: Absolute executable path if found. """ configured_binary = os.getenv("IMAGEMAGICK_BINARY", "").strip().strip('"') if configured_binary: expanded = Path(configured_binary).expanduser() if expanded.exists(): return str(expanded.resolve()) logger.warning( colored("Configured IMAGEMAGICK_BINARY was not found on disk.", "yellow") ) candidate_names = [ "magick", "magick.exe", "convert", "convert.exe", ] for candidate in candidate_names: found = shutil.which(candidate) if found: return found return None def check_env_vars() -> None: """ Checks if the necessary environment variables are set. Returns: None Raises: SystemExit: If any required environment variables are missing. """ try: required_vars = ["PEXELS_API_KEY", "TIKTOK_SESSION_ID"] missing_vars = [] for var in required_vars: value = os.getenv(var) if value is None or len(value) == 0: missing_vars.append(var) if missing_vars: missing_vars_str = ", ".join(missing_vars) logger.error( colored( f"The following environment variables are missing: {missing_vars_str}", "red", ) ) logger.error( colored( "Please consult 'docs/configuration.md' for setup instructions.", "yellow", ) ) sys.exit(1) # Aborts the program imagemagick_binary = resolve_imagemagick_binary() if not imagemagick_binary: logger.error( colored( "IMAGEMAGICK_BINARY is not set and no ImageMagick executable was detected in PATH.", "red", ) ) logger.error( colored( "Set IMAGEMAGICK_BINARY in .env or install ImageMagick and add it to PATH.", "yellow", ) ) sys.exit(1) os.environ["IMAGEMAGICK_BINARY"] = imagemagick_binary except Exception as e: logger.error(f"Error occurred while checking environment variables: {str(e)}") sys.exit(1) # Aborts the program if an unexpected error occurs ================================================ FILE: Backend/video.py ================================================ import os import uuid import requests import srt_equalizer import assemblyai as aai from typing import List from pathlib import Path from moviepy import ( AudioFileClip, CompositeVideoClip, TextClip, VideoFileClip, concatenate_videoclips, ) from dotenv import load_dotenv from logstream import log from moviepy.video.tools.subtitles import SubtitlesClip from utils import ENV_FILE, TEMP_DIR, SUBTITLES_DIR, FONTS_DIR load_dotenv(ENV_FILE) ASSEMBLY_AI_API_KEY = os.getenv("ASSEMBLY_AI_API_KEY") FRAME_EPSILON = 1 / 120 def save_video(video_url: str, directory: str = str(TEMP_DIR)) -> str: """ Saves a video from a given URL and returns the path to the video. Args: video_url (str): The URL of the video to save. directory (str): The path of the temporary directory to save the video to Returns: str: The path to the saved video. """ destination = Path(directory).expanduser().resolve() destination.mkdir(parents=True, exist_ok=True) video_id = uuid.uuid4() video_path = destination / f"{video_id}.mp4" with open(video_path, "wb") as f: f.write(requests.get(video_url).content) return str(video_path) def __generate_subtitles_assemblyai(audio_path: str, voice: str) -> str: """ Generates subtitles from a given audio file and returns the path to the subtitles. Args: audio_path (str): The path to the audio file to generate subtitles from. Returns: str: The generated subtitles """ language_mapping = { "br": "pt", "id": "en", # AssemblyAI doesn't have Indonesian "jp": "ja", "kr": "ko", } if voice in language_mapping: lang_code = language_mapping[voice] else: lang_code = voice aai.settings.api_key = ASSEMBLY_AI_API_KEY config = aai.TranscriptionConfig(language_code=lang_code) transcriber = aai.Transcriber(config=config) transcript = transcriber.transcribe(audio_path) subtitles = transcript.export_subtitles_srt() return subtitles def __generate_subtitles_locally( sentences: List[str], audio_clips: List[AudioFileClip] ) -> str: """ Generates subtitles from a given audio file and returns the path to the subtitles. Args: sentences (List[str]): all the sentences said out loud in the audio clips audio_clips (List[AudioFileClip]): all the individual audio clips which will make up the final audio track Returns: str: The generated subtitles """ def convert_to_srt_time_format(total_seconds: float) -> str: # Convert total seconds to the SRT time format: HH:MM:SS,mmm milliseconds_total = int(round(total_seconds * 1000)) hours, remainder = divmod(milliseconds_total, 3_600_000) minutes, remainder = divmod(remainder, 60_000) seconds, milliseconds = divmod(remainder, 1000) return f"{hours:02d}:{minutes:02d}:{seconds:02d},{milliseconds:03d}" start_time = 0 subtitles = [] for i, (sentence, audio_clip) in enumerate(zip(sentences, audio_clips), start=1): duration = audio_clip.duration end_time = start_time + duration # Format: subtitle index, start time --> end time, sentence subtitle_entry = f"{i}\n{convert_to_srt_time_format(start_time)} --> {convert_to_srt_time_format(end_time)}\n{sentence}\n" subtitles.append(subtitle_entry) start_time += duration # Update start time for the next subtitle return "\n".join(subtitles) def generate_subtitles( audio_path: str, sentences: List[str], audio_clips: List[AudioFileClip], voice: str ) -> str: """ Generates subtitles from a given audio file and returns the path to the subtitles. Args: audio_path (str): The path to the audio file to generate subtitles from. sentences (List[str]): all the sentences said out loud in the audio clips audio_clips (List[AudioFileClip]): all the individual audio clips which will make up the final audio track Returns: str: The path to the generated subtitles. """ def equalize_subtitles(srt_path: str, max_chars: int = 10) -> None: # Equalize subtitles srt_equalizer.equalize_srt_file(srt_path, srt_path, max_chars) # Save subtitles SUBTITLES_DIR.mkdir(parents=True, exist_ok=True) subtitles_path = SUBTITLES_DIR / f"{uuid.uuid4()}.srt" if ASSEMBLY_AI_API_KEY is not None and ASSEMBLY_AI_API_KEY != "": log("[+] Creating subtitles using AssemblyAI", "info") subtitles = __generate_subtitles_assemblyai(audio_path, voice) else: log("[+] Creating subtitles locally", "info") subtitles = __generate_subtitles_locally(sentences, audio_clips) # print(colored("[-] Local subtitle generation has been disabled for the time being.", "red")) # print(colored("[-] Exiting.", "red")) # sys.exit(1) with open(subtitles_path, "w", encoding="utf-8") as file: file.write(subtitles) # Equalize subtitles equalize_subtitles(str(subtitles_path)) log("[+] Subtitles generated.", "success") return str(subtitles_path) def combine_videos( video_paths: List[str], max_duration: int, max_clip_duration: int, threads: int ) -> str: """ Combines a list of videos into one video and returns the path to the combined video. Args: video_paths (List): A list of paths to the videos to combine. max_duration (int): The maximum duration of the combined video. max_clip_duration (int): The maximum duration of each clip. threads (int): The number of threads to use for the video processing. Returns: str: The path to the combined video. """ video_id = uuid.uuid4() TEMP_DIR.mkdir(parents=True, exist_ok=True) combined_video_path = TEMP_DIR / f"{video_id}.mp4" if not video_paths: raise ValueError("No source videos were provided for concatenation.") max_duration = float(max_duration) max_clip_duration = float(max_clip_duration) # Required duration of each clip req_dur = max_duration / len(video_paths) log("[+] Combining videos...", "info") log(f"[+] Each clip will be maximum {req_dur} seconds long.", "info") clips = [] tot_dur = 0 # Add downloaded clips over and over until the duration of the audio (max_duration) has been reached while tot_dur < (max_duration - FRAME_EPSILON): progressed = False for video_path in video_paths: remaining = max_duration - tot_dur if remaining <= FRAME_EPSILON: break clip = VideoFileClip(video_path) clip = clip.without_audio() max_safe_source_duration = clip.duration - FRAME_EPSILON if max_safe_source_duration <= 0: clip.close() continue target_duration = min(req_dur, max_clip_duration, remaining) target_duration = min(target_duration, max_safe_source_duration) if target_duration <= 0: clip.close() continue if target_duration < clip.duration: clip = clip.subclipped(0, target_duration) clip = clip.with_fps(30) # Not all videos are same size, # so we need to resize them if round((clip.w / clip.h), 4) < 0.5625: clip = clip.cropped( width=clip.w, height=round(clip.w / 0.5625), x_center=clip.w / 2, y_center=clip.h / 2, ) else: clip = clip.cropped( width=round(0.5625 * clip.h), height=clip.h, x_center=clip.w / 2, y_center=clip.h / 2, ) clip = clip.resized(new_size=(1080, 1920)) clips.append(clip) tot_dur += clip.duration progressed = True if not progressed: raise RuntimeError("Could not reach target duration from source videos.") if not clips: raise RuntimeError("No valid clips were produced for concatenation.") final_clip = concatenate_videoclips(clips, method="compose") final_clip = final_clip.with_fps(30).with_duration(max_duration) try: final_clip.write_videofile( str(combined_video_path), threads=threads, fps=30, codec="libx264", preset="medium", audio=False, ) finally: final_clip.close() for clip in clips: clip.close() return str(combined_video_path) def generate_video( combined_video_path: str, tts_path: str, subtitles_path: str, threads: int, subtitles_position: str, text_color: str, ) -> str: """ This function creates the final video, with subtitles and audio. Args: combined_video_path (str): The path to the combined video. tts_path (str): The path to the text-to-speech audio. subtitles_path (str): The path to the subtitles. threads (int): The number of threads to use for the video processing. subtitles_position (str): The position of the subtitles. Returns: str: The path to the final video. """ # Make a generator that returns a TextClip when called with consecutive font_path = str((FONTS_DIR / "bold_font.ttf").resolve()) generator = lambda txt: TextClip( font=font_path, text=txt, font_size=100, color=text_color, stroke_color="black", stroke_width=5, ) # Split the subtitles position into horizontal and vertical horizontal_subtitles_position, vertical_subtitles_position = ( subtitles_position.split(",") ) # Burn the subtitles into the video subtitles = SubtitlesClip(subtitles_path, make_textclip=generator) subtitle_vertical_position = vertical_subtitles_position if vertical_subtitles_position == "top": subtitle_vertical_position = 80 base_video = VideoFileClip(str(combined_video_path)) audio = AudioFileClip(tts_path) target_duration = min(base_video.duration, audio.duration) result = CompositeVideoClip( [ base_video.subclipped(0, target_duration), subtitles.with_position( (horizontal_subtitles_position, subtitle_vertical_position) ).with_duration(target_duration), ] ) # Clamp audio/video to exactly the same duration to avoid end-frame overreads. result = result.with_audio(audio.subclipped(0, target_duration)).with_duration( target_duration ) output_path = TEMP_DIR / "output.mp4" try: result.write_videofile( str(output_path), threads=threads or 2, fps=30, codec="libx264", audio_codec="aac", preset="medium", ) finally: result.close() subtitles.close() audio.close() base_video.close() return "output.mp4" ================================================ FILE: Backend/worker.py ================================================ import time from dotenv import load_dotenv from db import SessionLocal, init_db from pipeline import PipelineCancelled, run_generation_pipeline from repository import ( append_event, claim_next_queued_job, get_job, mark_cancelled, mark_completed, mark_failed, ) from utils import ENV_FILE, SUBTITLES_DIR, TEMP_DIR, check_env_vars, clean_dir POLL_SECONDS = 1.0 def _job_cancelled(job_id: str) -> bool: with SessionLocal() as session: job = get_job(session, job_id) if not job: return True return bool(job.cancel_requested or job.status == "cancelled") def _log_event(job_id: str, message: str, level: str) -> None: with SessionLocal() as session: append_event(session, job_id, "log", level, str(message)) session.commit() def process_next_job() -> bool: with SessionLocal() as session: job = claim_next_queued_job(session) if not job: return False job_id = job.id clean_dir(str(TEMP_DIR)) clean_dir(str(SUBTITLES_DIR)) try: result_path = run_generation_pipeline( data=job.payload, is_cancelled=lambda: _job_cancelled(job_id), on_log=lambda message, level: _log_event(job_id, message, level), ) with SessionLocal() as session: mark_completed(session, job_id, result_path) except PipelineCancelled as err: with SessionLocal() as session: mark_cancelled(session, job_id, str(err)) except Exception as err: with SessionLocal() as session: mark_failed(session, job_id, str(err)) return True def main() -> None: load_dotenv(ENV_FILE) check_env_vars() init_db() while True: processed = process_next_job() if not processed: time.sleep(POLL_SECONDS) if __name__ == "__main__": main() ================================================ FILE: Backend/youtube.py ================================================ import os import sys import time import random import httplib2 from pathlib import Path from logstream import log from oauth2client.file import Storage from apiclient.discovery import build from apiclient.errors import HttpError from apiclient.http import MediaFileUpload from oauth2client.tools import argparser, run_flow from oauth2client.client import flow_from_clientsecrets # Explicitly tell the underlying HTTP transport library not to retry, since # we are handling retry logic ourselves. httplib2.RETRIES = 1 # Maximum number of times to retry before giving up. MAX_RETRIES = 10 # Always retry when these exceptions are raised. RETRIABLE_EXCEPTIONS = (httplib2.HttpLib2Error, IOError, httplib2.ServerNotFoundError) # Always retry when an apiclient.errors.HttpError with one of these status # codes is raised. RETRIABLE_STATUS_CODES = [500, 502, 503, 504] # The CLIENT_SECRETS_FILE variable specifies the name of a file that contains # the OAuth 2.0 information for this application, including its client_id and # client_secret. BASE_DIR = Path(__file__).resolve().parent CLIENT_SECRETS_FILE = str((BASE_DIR / "client_secret.json").resolve()) # This OAuth 2.0 access scope allows an application to upload files to the # authenticated user's YouTube channel, but doesn't allow other types of access. # YOUTUBE_UPLOAD_SCOPE = "https://www.googleapis.com/auth/youtube.upload" SCOPES = [ "https://www.googleapis.com/auth/youtube.upload", "https://www.googleapis.com/auth/youtube", "https://www.googleapis.com/auth/youtubepartner", ] YOUTUBE_API_SERVICE_NAME = "youtube" YOUTUBE_API_VERSION = "v3" # This variable defines a message to display if the CLIENT_SECRETS_FILE is # missing. MISSING_CLIENT_SECRETS_MESSAGE = f""" WARNING: Please configure OAuth 2.0 To make this sample run you will need to populate the client_secrets.json file found at: {os.path.abspath(os.path.join(os.path.dirname(__file__), CLIENT_SECRETS_FILE))} with information from the API Console https://console.cloud.google.com/ For more information about the client_secrets.json file format, please visit: https://developers.google.com/api-client-library/python/guide/aaa_client_secrets """ VALID_PRIVACY_STATUSES = ("public", "private", "unlisted") def get_authenticated_service(): """ This method retrieves the YouTube service. Returns: any: The authenticated YouTube service. """ flow = flow_from_clientsecrets( CLIENT_SECRETS_FILE, scope=SCOPES, message=MISSING_CLIENT_SECRETS_MESSAGE ) oauth_store = BASE_DIR / f"{Path(sys.argv[0]).name}-oauth2.json" storage = Storage(str(oauth_store)) credentials = storage.get() if credentials is None or credentials.invalid: flags = argparser.parse_args() credentials = run_flow(flow, storage, flags) return build( YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, http=credentials.authorize(httplib2.Http()), ) def initialize_upload(youtube: any, options: dict): """ This method uploads a video to YouTube. Args: youtube (any): The authenticated YouTube service. options (dict): The options to upload the video with. Returns: response: The response from the upload process. """ tags = None if options["keywords"]: tags = options["keywords"].split(",") body = { "snippet": { "title": options["title"], "description": options["description"], "tags": tags, "categoryId": options["category"], }, "status": { "privacyStatus": options["privacyStatus"], "madeForKids": False, # Video is not made for kids "selfDeclaredMadeForKids": False, # You declare that the video is not made for kids }, } # Call the API's videos.insert method to create and upload the video. insert_request = youtube.videos().insert( part=",".join(body.keys()), body=body, media_body=MediaFileUpload(options["file"], chunksize=-1, resumable=True), ) return resumable_upload(insert_request) def resumable_upload(insert_request: MediaFileUpload): """ This method implements an exponential backoff strategy to resume a failed upload. Args: insert_request (MediaFileUpload): The request to insert the video. Returns: response: The response from the upload process. """ response = None error = None retry = 0 while response is None: try: log(" => Uploading file...", "info") status, response = insert_request.next_chunk() if "id" in response: log(f"Video id '{response['id']}' was successfully uploaded.", "success") return response except HttpError as e: if e.resp.status in RETRIABLE_STATUS_CODES: error = f"A retriable HTTP error {e.resp.status} occurred:\n{e.content}" else: raise except RETRIABLE_EXCEPTIONS as e: error = f"A retriable error occurred: {e}" if error is not None: log(error, "error") retry += 1 if retry > MAX_RETRIES: raise Exception("No longer attempting to retry.") max_sleep = 2**retry sleep_seconds = random.random() * max_sleep log(f" => Sleeping {sleep_seconds} seconds and then retrying...", "info") time.sleep(sleep_seconds) def upload_video(video_path, title, description, category, keywords, privacy_status): try: # Get the authenticated YouTube service youtube = get_authenticated_service() # Retrieve and print the channel ID for the authenticated user channels_response = youtube.channels().list(mine=True, part="id").execute() for channel in channels_response["items"]: log(f" => Channel ID: {channel['id']}", "info") # Initialize the upload process video_response = initialize_upload( youtube, { "file": video_path, # The path to the video file "title": title, "description": description, "category": category, "keywords": keywords, "privacyStatus": privacy_status, }, ) return video_response # Return the response from the upload process except HttpError as e: log(f"[-] An HTTP error {e.resp.status} occurred:\n{e.content}", "error") if e.resp.status in [401, 403]: # Here you could refresh the credentials and retry the upload youtube = ( get_authenticated_service() ) # This will prompt for re-authentication if necessary video_response = initialize_upload( youtube, { "file": video_path, "title": title, "description": description, "category": category, "keywords": keywords, "privacyStatus": privacy_status, }, ) return video_response else: raise e ================================================ FILE: CLAUDE.md ================================================ # CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview MoneyPrinter automates YouTube Shorts creation from text topics. It uses Ollama for script generation, TikTok TTS for voiceover, Pexels for stock footage, and moviepy/ImageMagick for video composition. Output: a 9:16 vertical video (`output.mp4`). ## Commands ### Setup ```bash cp .env.example .env # then fill in API keys uv sync # install dependencies ollama serve # start Ollama (separate terminal) ollama pull llama3.1:8b # pull default model ``` ### Run (local) ```bash uv run python Backend/main.py # API on :8080 uv run python Backend/worker.py # queue worker python3 -m http.server 3000 --directory Frontend # frontend on :3000 ``` ### Run (Docker) ```bash docker compose up --build # frontend :8001, backend :8080, postgres :5432 ``` ### Verify ```bash uv run python -m compileall Backend # syntax check curl http://localhost:8080/api/models # API smoke test ``` ### Tests No test suite exists yet. If added, use pytest: ```bash uv run pytest -q # all tests uv run pytest tests/test_file.py::test_name -q # single test ``` ## Architecture ### Video Generation Pipeline (end-to-end flow) ``` User input (Frontend) → POST /api/generate → generation_jobs (Postgres queue) → worker.py claims queued job → gpt.py: generate_script() via Ollama → gpt.py: get_search_terms() → JSON keywords → search.py: Pexels API → download stock clips to temp/ → tiktokvoice.py: TTS per sentence → MP3 chunks (threaded) → video.py: generate_subtitles() → .srt (AssemblyAI or local timestamps) → video.py: combine_videos() → concatenate/crop to 9:16 → video.py: generate_video() → burn subtitles via ImageMagick, merge audio → (optional) mix background music from Songs/ at 10% volume → (optional) youtube.py: OAuth2 upload → output.mp4 ``` ### Frontend ↔ Backend Communication - **REST**: JSON payloads to Flask endpoints (`/api/generate`, `/api/jobs/:id`, `/api/jobs/:id/events`, `/api/jobs/:id/cancel`, `/api/models`, `/api/upload-songs`) - **Polling**: frontend polls job status and persisted generation events. ### Key Backend Modules | File | Responsibility | |------|---------------| | `main.py` | Flask app and queue/job endpoints | | `worker.py` | Job consumer that executes generation pipeline | | `db.py`/`models.py`/`repository.py` | DB engine, schema, queue/event persistence | | `gpt.py` | Ollama client: script generation, search terms, YouTube metadata | | `video.py` | Video processing: combine clips, burn subtitles, merge audio | | `search.py` | Pexels stock video search and download | | `tiktokvoice.py` | TikTok TTS API (60+ voices, 300-char chunking, threaded) | | `youtube.py` | YouTube upload via Google API with OAuth2 | | `utils.py` | Path constants, env validation, ImageMagick detection | | `pipeline.py` | Reusable generation pipeline used by worker | ### Frontend - `index.html`: UI with inline CSS, form fields, live log viewer - `app.js`: API client (`apiRequest()`), job polling UI, localStorage persistence ### Runtime Directories - `temp/`: intermediate video/audio files (cleared each generation) - `subtitles/`: generated .srt files (cleared each generation) - `Songs/`: user-uploaded background music MP3s - `fonts/`: subtitle font (`bold_font.ttf`) ## Required Environment Variables - `TIKTOK_SESSION_ID` — TikTok cookie for TTS - `PEXELS_API_KEY` — stock video API - `IMAGEMAGICK_BINARY` — leave empty to auto-detect from PATH Optional: `OLLAMA_BASE_URL`, `OLLAMA_MODEL`, `ASSEMBLY_AI_API_KEY`, `DATABASE_URL` ## Conventions - **Python**: 4-space indent, `snake_case`, type hints on all new/modified signatures, `pathlib.Path` for filesystem ops - **JS**: `camelCase`, centralized API calls via `apiRequest()` - **API responses**: `{"status": "success|error", ...}` with proper HTTP codes - **Long-running work**: database-backed queue and separate worker process - **Concurrency**: multiple jobs can be queued; worker processes them safely via DB locking - Update `docs/` when setup, env vars, or runtime behavior changes ================================================ FILE: Dockerfile ================================================ FROM python:3.11-slim-buster COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ RUN apt-get update && apt-get install --no-install-recommends -y \ build-essential autoconf pkg-config wget ghostscript curl libpng-dev RUN wget https://github.com/ImageMagick/ImageMagick/archive/refs/tags/7.1.0-31.tar.gz && \ tar xzf 7.1.0-31.tar.gz && \ rm 7.1.0-31.tar.gz && \ apt-get clean && \ apt-get autoremove RUN sh ./ImageMagick-7.1.0-31/configure --prefix=/usr/local --with-bzlib=yes --with-fontconfig=yes --with-freetype=yes --with-gslib=yes --with-gvc=yes --with-jpeg=yes --with-jp2=yes --with-png=yes --with-tiff=yes --with-xml=yes --with-gs-font-dir=yes && \ make -j && make install && ldconfig /usr/local/lib/ WORKDIR /app COPY pyproject.toml . RUN uv pip install --system -r pyproject.toml ================================================ FILE: Frontend/app.js ================================================ // ===== DOM REFS ===== const videoSubject = document.getElementById("videoSubject"); const aiModel = document.getElementById("aiModel"); const voice = document.getElementById("voice"); const songFiles = document.getElementById("songFiles"); const paragraphNumber = document.getElementById("paragraphNumber"); const youtubeToggle = document.getElementById("youtubeUploadToggle"); const useMusicToggle = document.getElementById("useMusicToggle"); const customPrompt = document.getElementById("customPrompt"); const generateButton = document.getElementById("generateButton"); const cancelButton = document.getElementById("cancelButton"); const advancedOptionsToggle = document.getElementById("advancedOptionsToggle"); const advancedPanel = document.getElementById("advancedOptions"); const statusArea = document.getElementById("statusArea"); const colorDot = document.getElementById("colorDot"); const subtitlesColor = document.getElementById("subtitlesColor"); const logViewer = document.getElementById("logViewer"); const logViewerBody = document.getElementById("logViewerBody"); const logClearBtn = document.getElementById("logClearBtn"); const backendHost = window.location.hostname || "localhost"; const backendProtocol = window.location.protocol || "http:"; const API_BASE_URL = `${backendProtocol}//${backendHost}:8080`; const API_FALLBACK_URL = `http://${backendHost}:8080`; let activeJobId = null; let pollHandle = null; let lastEventId = 0; // ===== API HELPERS ===== async function apiRequest(path, options = {}) { const endpoint = path.startsWith("/") ? path : `/${path}`; async function request(baseUrl) { const response = await fetch(`${baseUrl}${endpoint}`, options); const data = await response.json(); if (!response.ok) { throw new Error(data.message || `Request failed with status ${response.status}`); } return data; } try { return await request(API_BASE_URL); } catch (firstError) { if (API_BASE_URL !== API_FALLBACK_URL) { return request(API_FALLBACK_URL); } throw firstError; } } function setModelOptions(models, preferredModel) { aiModel.innerHTML = ""; models.forEach((modelName) => { const option = document.createElement("option"); option.value = modelName; option.textContent = modelName; aiModel.appendChild(option); }); if (preferredModel && models.includes(preferredModel)) { aiModel.value = preferredModel; } else if (models.length > 0) { aiModel.value = models[0]; } } async function loadOllamaModels(reuseEnabled) { const fallbackModel = localStorage.getItem("aiModelValue") || "llama3.1:8b"; try { const data = await apiRequest("/api/models", { method: "GET", headers: { Accept: "application/json", }, }); const models = Array.isArray(data.models) ? data.models.filter((item) => typeof item === "string" && item.trim()) : []; const defaultModel = typeof data.default === "string" && data.default.trim() ? data.default.trim() : fallbackModel; const preferredModel = reuseEnabled && localStorage.getItem("aiModelValue") ? localStorage.getItem("aiModelValue") : defaultModel; if (data.status && data.status !== "success" && data.message) { showToast(data.message, "error"); } if (models.length === 0) { setModelOptions([defaultModel], preferredModel); showToast("No Ollama models found. Pull one with: ollama pull llama3.1:8b", "error"); return; } setModelOptions(models, preferredModel); } catch { setModelOptions([fallbackModel], fallbackModel); showToast("Could not load Ollama models. Is backend/Ollama running?", "error"); } } // ===== TOAST NOTIFICATIONS ===== function showToast(message, type = "info") { const container = document.getElementById("toastContainer"); const toast = document.createElement("div"); toast.className = `toast toast-${type}`; toast.innerHTML = ` ${message} `; toast.querySelector(".toast-close").addEventListener("click", () => { dismissToast(toast); }); container.appendChild(toast); requestAnimationFrame(() => { requestAnimationFrame(() => toast.classList.add("show")); }); setTimeout(() => dismissToast(toast), 5000); } function dismissToast(toast) { toast.classList.remove("show"); toast.addEventListener("transitionend", () => toast.remove(), { once: true }); } // ===== COLOR DOT ===== function updateColorDot() { if (colorDot && subtitlesColor) { colorDot.style.backgroundColor = subtitlesColor.value; } } updateColorDot(); subtitlesColor.addEventListener("change", updateColorDot); // ===== ADVANCED OPTIONS TOGGLE ===== advancedOptionsToggle.addEventListener("click", () => { advancedOptionsToggle.classList.toggle("open"); advancedPanel.classList.toggle("open"); }); // ===== LOG STREAM (SSE) ===== function formatTimestamp(ts) { const d = new Date((ts || Date.now() / 1000) * 1000); return d.toLocaleTimeString("en-GB", { hour12: false }); } function appendLogEntry(entry) { const row = document.createElement("div"); row.className = "log-entry"; const time = document.createElement("span"); time.className = "log-time"; time.textContent = formatTimestamp(entry.timestamp); const msg = document.createElement("span"); msg.className = `log-msg log-${entry.level || "info"}`; msg.textContent = entry.message; row.appendChild(time); row.appendChild(msg); logViewerBody.appendChild(row); // Auto-scroll to bottom logViewerBody.scrollTop = logViewerBody.scrollHeight; } async function pollJob() { if (!activeJobId) return; try { const eventsResult = await apiRequest(`/api/jobs/${activeJobId}/events?after=${lastEventId}`, { method: "GET", headers: { Accept: "application/json", }, }); const events = Array.isArray(eventsResult.events) ? eventsResult.events : []; events.forEach((event) => { appendLogEntry({ timestamp: event.timestamp, message: event.message, level: event.level || "info", }); lastEventId = Math.max(lastEventId, event.id || 0); }); const jobResult = await apiRequest(`/api/jobs/${activeJobId}`, { method: "GET", headers: { Accept: "application/json", }, }); const state = jobResult?.job?.state; if (state === "completed") { showToast("Video generated successfully.", "success"); stopJobPolling(); setGeneratingState(false); } else if (state === "failed") { showToast(jobResult?.job?.errorMessage || "Generation failed.", "error"); stopJobPolling(); setGeneratingState(false); } else if (state === "cancelled") { showToast("Generation cancelled.", "warning"); stopJobPolling(); setGeneratingState(false); } } catch { // Ignore transient polling failures. } } function startJobPolling(jobId) { stopJobPolling(); activeJobId = jobId; lastEventId = 0; logViewer.classList.add("active"); pollHandle = setInterval(pollJob, 1200); pollJob(); } function stopJobPolling() { if (pollHandle) { clearInterval(pollHandle); pollHandle = null; } } // ===== GENERATE / CANCEL ===== function setGeneratingState(active) { if (active) { generateButton.classList.add("hidden"); cancelButton.classList.remove("hidden"); statusArea.classList.add("active"); } else { stopJobPolling(); activeJobId = null; generateButton.classList.remove("hidden"); cancelButton.classList.add("hidden"); statusArea.classList.remove("active"); generateButton.disabled = false; logViewer.classList.remove("active"); } } function cancelGeneration() { const targetPath = activeJobId ? `/api/jobs/${activeJobId}/cancel` : "/api/cancel"; apiRequest(targetPath, { method: "POST", headers: { "Content-Type": "application/json", Accept: "application/json", }, }) .then((data) => showToast(data.message, "success")) .catch(() => showToast("Failed to cancel. Is the server running?", "error")); } async function uploadSongs() { const files = songFiles.files; if (!files || files.length === 0) return true; const mp3s = Array.from(files).filter((f) => f.name.toLowerCase().endsWith(".mp3")); if (mp3s.length === 0) { showToast("No MP3 files found in the selected folder.", "error"); return false; } const formData = new FormData(); mp3s.forEach((file) => formData.append("songs", file)); try { await apiRequest("/api/upload-songs", { method: "POST", body: formData, }); return true; } catch { showToast("Failed to upload songs.", "error"); return false; } } async function generateVideo() { const subject = videoSubject.value.trim(); if (!subject) { showToast("Please enter a video subject.", "error"); videoSubject.focus(); return; } generateButton.disabled = true; setGeneratingState(true); // Clear previous log entries logViewerBody.innerHTML = ""; // Upload songs first if a folder was selected if (useMusicToggle.checked && songFiles.files.length > 0) { const uploaded = await uploadSongs(); if (!uploaded) { setGeneratingState(false); return; } } const data = { videoSubject: subject, aiModel: aiModel.value || "llama3.1:8b", voice: voice.value, paragraphNumber: paragraphNumber.value, automateYoutubeUpload: youtubeToggle.checked, useMusic: useMusicToggle.checked, threads: document.getElementById("threads").value, subtitlesPosition: document.getElementById("subtitlesPosition").value, customPrompt: customPrompt.value, color: subtitlesColor.value, }; try { const result = await apiRequest("/api/generate", { method: "POST", body: JSON.stringify(data), headers: { "Content-Type": "application/json", Accept: "application/json", }, }); if (result.status === "success") { if (!result.jobId) { showToast("Generation queued, but no job ID was returned.", "error"); setGeneratingState(false); return; } startJobPolling(result.jobId); } else { showToast(result.message, "error"); setGeneratingState(false); } } catch { showToast("Connection error. Is the backend server running?", "error"); setGeneratingState(false); } } generateButton.addEventListener("click", generateVideo); cancelButton.addEventListener("click", cancelGeneration); videoSubject.addEventListener("keydown", (event) => { if (event.key === "Enter" && !event.shiftKey) { event.preventDefault(); generateVideo(); } }); // ===== LOG CLEAR BUTTON ===== logClearBtn.addEventListener("click", () => { logViewerBody.innerHTML = ""; }); // ===== LOCAL STORAGE PERSISTENCE ===== const toggleIds = [ "youtubeUploadToggle", "useMusicToggle", "reuseChoicesToggle", ]; const fieldIds = [ "voice", "paragraphNumber", "videoSubject", "customPrompt", "threads", "subtitlesPosition", "subtitlesColor", ]; document.addEventListener("DOMContentLoaded", async () => { const reuseEnabled = localStorage.getItem("reuseChoicesToggleValue") === "true"; await loadOllamaModels(reuseEnabled); aiModel.addEventListener("change", (e) => { localStorage.setItem("aiModelValue", e.target.value); }); // Restore toggles toggleIds.forEach((id) => { const el = document.getElementById(id); if (!el) return; const stored = localStorage.getItem(`${id}Value`); if (stored !== null && reuseEnabled) { el.checked = stored === "true"; } el.addEventListener("change", (e) => { localStorage.setItem(`${id}Value`, e.target.checked); }); }); // Restore fields fieldIds.forEach((id) => { const el = document.getElementById(id); if (!el) return; const stored = localStorage.getItem(`${id}Value`); if (stored && reuseEnabled) { el.value = stored; } el.addEventListener("change", (e) => { localStorage.setItem(`${id}Value`, e.target.value); }); }); // Update color dot after restoring values updateColorDot(); }); ================================================ FILE: Frontend/index.html ================================================ MoneyPrinter

MoneyPrinter 💸

Automate YouTube Shorts with AI

Generating your video…
Live Output
================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2024 FujiwaraChoki Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ # MoneyPrinter 💸 > ♥︎ Sponsor: The Best AI Chat App: [shiori.ai](https://www.shiori.ai) --- > 𝕏 Also, follow me on X: [@DevBySami](https://x.com/DevBySami). Automate the creation of YouTube Shorts by providing a video topic. MoneyPrinter is Ollama-first: script generation and metadata are fully powered by local Ollama models. MoneyPrinter now uses a DB-backed generation queue (API + worker + Postgres in Docker) for reliable, restart-safe processing. FujiwaraChoki%2FMoneyPrinter | Trendshift > **Important** Please make sure you look through existing/closed issues before opening your own. If it's just a question, please join our [discord](https://dsc.gg/fuji-community) and ask there. > **🎥** Watch the video on [YouTube](https://youtu.be/mkZsaDA2JnA?si=pNne3MnluRVkWQbE). ## Documentation Docs are centralized in [`docs/`](docs/README.md): - [Interactive Setup Script](setup.sh) - [Quickstart](docs/quickstart.md) - [Configuration](docs/configuration.md) - [Architecture](docs/architecture.md) - [Docker](docs/docker.md) - [Testing](docs/testing.md) - [Troubleshooting](docs/troubleshooting.md) ## FAQ 🤔 ### Which AI provider does MoneyPrinter use? MoneyPrinter is fully Ollama-based. Start Ollama, pull a model, and select the model in the UI. ```bash ollama serve ollama pull llama3.1:8b ``` ### How do I get the TikTok session ID? You can obtain your TikTok session ID by logging into TikTok in your browser and copying the value of the `sessionid` cookie. ### My ImageMagick binary is not being detected MoneyPrinter auto-detects ImageMagick from your `PATH` on Linux, macOS, and Windows. If auto-detection fails, set the executable path manually in `.env`, for example: ```env IMAGEMAGICK_BINARY="C:\\Program Files\\ImageMagick-7.1.0-Q16\\magick.exe" ``` Don't forget to use double backslashes (`\\`) in the path, instead of one. ### I can't install `playsound`: Wheel failed to build If you're having trouble installing `playsound`, you can try installing it using the following command: ```bash uv pip install -U wheel uv pip install -U playsound ``` If you were not able to find your solution, check [Troubleshooting](docs/troubleshooting.md), ask in Discord, or create an issue. ## Donate 🎁 If you like and enjoy `MoneyPrinter`, and would like to donate, you can do that by clicking on the button on the right hand side of the repository. ❤️ You will have your name (and/or logo) added to this repository as a supporter as a sign of appreciation. ## Contributing 🤝 Pull Requests will not be accepted for the time-being. ## Star History 🌟 [![Star History Chart](https://api.star-history.com/svg?repos=FujiwaraChoki/MoneyPrinter&type=Date)](https://star-history.com/#FujiwaraChoki/MoneyPrinter&Date) ## License 📝 See [`LICENSE`](LICENSE) file for more information. ================================================ FILE: docker-compose.yml ================================================ version: "3" services: postgres: image: postgres:16-alpine container_name: "postgres" ports: - "5432:5432" environment: - POSTGRES_DB=${POSTGRES_DB:-moneyprinter} - POSTGRES_USER=${POSTGRES_USER:-moneyprinter} - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-moneyprinter} volumes: - postgres_data:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-moneyprinter} -d ${POSTGRES_DB:-moneyprinter}"] interval: 5s timeout: 5s retries: 10 restart: always frontend: build: context: . dockerfile: Dockerfile container_name: "frontend" ports: - "8001:8001" command: ["python3", "-m", "http.server", "8001", "--directory", "frontend"] volumes: - ./Frontend:/app/frontend restart: always backend: build: context: . dockerfile: Dockerfile container_name: "backend" ports: - "8080:8080" command: ["python3", "backend/main.py"] volumes: - ./files:/temp - ./Backend:/app/backend - ./fonts:/app/fonts environment: - ASSEMBLY_AI_API_KEY=${ASSEMBLY_AI_API_KEY} - TIKTOK_SESSION_ID=${TIKTOK_SESSION_ID} - IMAGEMAGICK_BINARY=/usr/local/bin/magick - PEXELS_API_KEY=${PEXELS_API_KEY} - OLLAMA_BASE_URL=${OLLAMA_BASE_URL:-http://host.docker.internal:11434} - OLLAMA_MODEL=${OLLAMA_MODEL:-llama3.1:8b} - DATABASE_URL=${DATABASE_URL:-postgresql+psycopg://moneyprinter:moneyprinter@postgres:5432/moneyprinter} extra_hosts: - "host.docker.internal:host-gateway" depends_on: - frontend - postgres restart: always worker: build: context: . dockerfile: Dockerfile container_name: "worker" command: ["python3", "backend/worker.py"] volumes: - ./files:/temp - ./Backend:/app/backend - ./fonts:/app/fonts environment: - ASSEMBLY_AI_API_KEY=${ASSEMBLY_AI_API_KEY} - TIKTOK_SESSION_ID=${TIKTOK_SESSION_ID} - IMAGEMAGICK_BINARY=/usr/local/bin/magick - PEXELS_API_KEY=${PEXELS_API_KEY} - OLLAMA_BASE_URL=${OLLAMA_BASE_URL:-http://host.docker.internal:11434} - OLLAMA_MODEL=${OLLAMA_MODEL:-llama3.1:8b} - DATABASE_URL=${DATABASE_URL:-postgresql+psycopg://moneyprinter:moneyprinter@postgres:5432/moneyprinter} extra_hosts: - "host.docker.internal:host-gateway" depends_on: - postgres - backend - frontend restart: always volumes: files: postgres_data: ================================================ FILE: docs/README.md ================================================ # MoneyPrinter Docs This folder is the single source of truth for setup, configuration, and troubleshooting. ## Start Here - [Interactive Setup Script](../setup.sh) - [Quickstart](quickstart.md) - [Configuration](configuration.md) - [Architecture](architecture.md) - [Docker](docker.md) - [Testing](testing.md) - [Troubleshooting](troubleshooting.md) ## Recommended Reading Order 1. Quickstart 2. Configuration 3. Docker (if you use containers) 4. Testing 5. Troubleshooting (when something breaks) ================================================ FILE: docs/architecture.md ================================================ # Architecture MoneyPrinter now uses a database-backed queue architecture designed for reliability, restart safety, and future scaling. ## Overview - `Frontend` submits generation requests and polls job status/events. - `API (Flask)` validates input and enqueues jobs in Postgres. - `Worker` claims queued jobs and runs the generation pipeline. - `Postgres` is the source of truth for job state, progress events, and artifacts. ```mermaid flowchart LR U[User] --> F[Frontend\nindex.html + app.js] F -->|POST /api/generate| A[API\nBackend/main.py] F -->|GET /api/jobs/:id| A F -->|GET /api/jobs/:id/events| A F -->|POST /api/jobs/:id/cancel| A A -->|insert job| DB[(Postgres)] A -->|read status/events| DB W[Worker\nBackend/worker.py] -->|claim queued job| DB W -->|write logs/events| DB W -->|update final state| DB W --> P[Pipeline\nBackend/pipeline.py] P --> FS[(temp/subtitles/output files)] ``` ## Runtime Services (Docker) ```mermaid flowchart TB subgraph Compose FE[frontend] API[backend] WK[worker] PG[(postgres)] end FE --> API API --> PG WK --> PG WK --> API ``` ## Generation Lifecycle ```mermaid stateDiagram-v2 [*] --> queued queued --> running: worker claims job queued --> cancelled: cancel before claim running --> completed: success running --> failed: unrecoverable error running --> cancelled: cancellation requested completed --> [*] failed --> [*] cancelled --> [*] ``` ## API + Worker Sequence ```mermaid sequenceDiagram participant UI as Frontend participant API as Flask API participant DB as Postgres participant WK as Worker participant PL as Pipeline UI->>API: POST /api/generate API->>DB: INSERT generation_jobs(status=queued) API-->>UI: { status: success, jobId } loop Polling UI->>API: GET /api/jobs/:id API->>DB: SELECT job API-->>UI: job state UI->>API: GET /api/jobs/:id/events?after=n API->>DB: SELECT events > n API-->>UI: event list end WK->>DB: claim queued job WK->>DB: UPDATE status=running + INSERT event WK->>PL: run generation pipeline PL-->>WK: result path OR error WK->>DB: UPDATE status + INSERT terminal event UI->>API: POST /api/jobs/:id/cancel API->>DB: set cancel_requested=true WK->>DB: observes cancel and marks cancelled ``` ## Data Model (Current Core) ```mermaid erDiagram projects ||--o{ generation_jobs : contains generation_jobs ||--o{ generation_events : has generation_jobs ||--o{ scripts : produces generation_jobs ||--o{ artifacts : produces projects { int id PK string name datetime created_at } generation_jobs { string id PK int project_id FK string status json payload boolean cancel_requested int attempt_count int max_attempts string result_path text error_message datetime created_at datetime started_at datetime completed_at datetime updated_at } generation_events { int id PK string job_id FK string event_type string level text message json payload datetime created_at } scripts { int id PK string job_id FK string model_name text content datetime created_at } artifacts { int id PK string job_id FK string artifact_type string path json metadata_json datetime created_at } ``` ## Current Guarantees - API is fast and non-blocking for generation requests. - Job state and logs survive API/worker restarts. - Cancellation is job-scoped (`cancel_requested`) and checked during processing. - Frontend can recover progress after refresh by polling persisted events. ## Planned Next Hardening - Add migration tool (Alembic) for schema versioning. - Add retries/backoff with `next_retry_at` and dead-letter semantics. - Add artifact metadata population and checksum tracking. - Add worker concurrency controls and queue metrics endpoints. ================================================ FILE: docs/configuration.md ================================================ # Configuration MoneyPrinter reads configuration from `.env` (project root). Use `.env.example` as your template. ## Required | Variable | Description | |---|---| | `TIKTOK_SESSION_ID` | TikTok session cookie (`sessionid`) used for TTS voice endpoint calls. | | `PEXELS_API_KEY` | API key used to fetch stock video clips. | ## Optional | Variable | Description | Default | |---|---|---| | `IMAGEMAGICK_BINARY` | Absolute path to ImageMagick executable. If empty, auto-detected from `PATH`. | auto-detect | | `OLLAMA_BASE_URL` | Ollama server base URL used for model listing and chat generation. | `http://localhost:11434` | | `OLLAMA_MODEL` | Fallback model if frontend does not send a model value. | `llama3.1:8b` | | `ASSEMBLY_AI_API_KEY` | If set, subtitles are generated with AssemblyAI; otherwise local subtitle generation is used. | empty | | `POSTGRES_DB` | Database name for Docker Postgres service. | `moneyprinter` | | `POSTGRES_USER` | Database user for Docker Postgres service. | `moneyprinter` | | `POSTGRES_PASSWORD` | Database password for Docker Postgres service. | `moneyprinter` | | `DATABASE_URL` | SQLAlchemy DSN used by API and worker (`postgresql+psycopg://...` or `sqlite:///...`). | `sqlite:///moneyprinter.db` | ## Notes - Ollama models shown in the frontend are fetched from backend endpoint `/api/models`, which queries `OLLAMA_BASE_URL/api/tags`. - Pull models before use, for example: ```bash ollama pull llama3.1:8b ``` - If ImageMagick is not discovered automatically, set `IMAGEMAGICK_BINARY` explicitly. - New architecture uses a database-backed job queue. In Docker, use Postgres via `DATABASE_URL`. ================================================ FILE: docs/docker.md ================================================ # Docker Run MoneyPrinter frontend, API, worker, and Postgres with Docker Compose. ## 1) Prepare environment ```bash cp .env.example .env ``` Set required keys in `.env`: - `TIKTOK_SESSION_ID` - `PEXELS_API_KEY` Database defaults (already in `.env.example`): - `POSTGRES_DB=moneyprinter` - `POSTGRES_USER=moneyprinter` - `POSTGRES_PASSWORD=moneyprinter` - `DATABASE_URL=postgresql+psycopg://moneyprinter:moneyprinter@postgres:5432/moneyprinter` ## 2) Ollama connectivity By default, Docker backend expects Ollama on host machine: - `OLLAMA_BASE_URL=http://host.docker.internal:11434` Linux support is included via compose `extra_hosts` host-gateway mapping. If Ollama runs in another container or machine, set `OLLAMA_BASE_URL` accordingly. ## 3) Start services ```bash docker compose up --build ``` ## 4) Access apps - Frontend: `http://localhost:8001` - Backend API: `http://localhost:8080` - Postgres: `localhost:5432` ## 5) Verify model listing ```bash curl http://localhost:8080/api/models ``` You should receive a JSON payload with `models` and `default`. ## 6) Queue a generation job ```bash curl -X POST http://localhost:8080/api/generate \ -H "Content-Type: application/json" \ -d '{"videoSubject":"AI business ideas","aiModel":"llama3.1:8b","voice":"en_us_001","paragraphNumber":1,"customPrompt":""}' ``` Response includes `jobId`. Query status and events: ```bash curl http://localhost:8080/api/jobs/ curl "http://localhost:8080/api/jobs//events?after=0" ``` ================================================ FILE: docs/quickstart.md ================================================ # Quickstart Run MoneyPrinter locally with an Ollama model. ## 1) Clone repository ```bash git clone https://github.com/FujiwaraChoki/MoneyPrinter.git cd MoneyPrinter ``` ## 2) Quick setup (recommended) Run the interactive setup script: ```bash ./setup.sh ``` This script checks dependencies, sets up `.env`, installs Python packages with `uv`, and can optionally pull an Ollama model. ## 3) Manual setup Use this path if you prefer to run each step yourself. ### Prerequisites - Python 3.11+ - [uv](https://docs.astral.sh/uv/getting-started/installation/) - FFmpeg - ImageMagick - Ollama ### Install and create env file ```bash uv sync cp .env.example .env ``` Windows PowerShell for `.env` copy: ```powershell Copy-Item .env.example .env ``` ## 4) Configure environment Set required values in `.env`: - `TIKTOK_SESSION_ID` - `PEXELS_API_KEY` See [Configuration](configuration.md) for all variables. ## Optional: Run tests ```bash uv sync --group dev uv run pytest ``` ## 5) Start Ollama and pull a model ```bash ollama serve ollama pull llama3.1:8b ``` If Ollama runs on another machine/port, set `OLLAMA_BASE_URL` in `.env`. ## 6) Run backend ```bash uv run python Backend/main.py ``` ## 7) Run worker In a new terminal: ```bash uv run python Backend/worker.py ``` ## 8) Run frontend In a new terminal: ```bash cd Frontend python3 -m http.server 3000 ``` Open `http://localhost:3000`. ## 9) Generate video 1. Enter a video subject. 2. Expand advanced options. 3. Choose an Ollama model from the dropdown (loaded dynamically from Ollama). 4. Click Generate. Output file: `output.mp4` at project root. ================================================ FILE: docs/testing.md ================================================ # Testing MoneyPrinter uses `pytest` for backend tests. ## Install test dependencies Install dev dependencies (includes `pytest`): ```bash uv sync --group dev ``` ## Run tests Run all tests: ```bash uv run pytest ``` Run one test file: ```bash uv run pytest tests/test_repository.py ``` Run one test: ```bash uv run pytest tests/test_repository.py::test_create_job_persists_payload_and_queued_event ``` ## Current test scope - `tests/test_api_jobs.py`: API queue, job status/events, and cancellation endpoints. - `tests/test_api_misc.py`: API model listing fallback and song upload endpoint behavior. - `tests/test_repository.py`: queue/repository behavior for create, claim, cancel, and completion events. - `tests/test_worker.py`: worker loop behavior for success, cancellation, failure, and empty queue. - `tests/test_utils.py`: filesystem cleanup, song selection, and ImageMagick binary resolution. - `tests/conftest.py`: isolated SQLite session fixture per test. ================================================ FILE: docs/troubleshooting.md ================================================ # Troubleshooting ## No Ollama models in dropdown - Ensure Ollama is running: `ollama serve` - Ensure at least one model exists: `ollama list` - Pull a model if needed: `ollama pull llama3.1:8b` - Verify backend can reach Ollama base URL in `.env` (`OLLAMA_BASE_URL`) ## Frontend cannot connect to backend - Confirm backend is running on port `8080` - Confirm frontend is opened from local server (for example `python3 -m http.server`) - Check browser console/network for `/api/generate` or `/api/models` failures ## ImageMagick not detected - Install ImageMagick and ensure executable is on `PATH` - Or set explicit path in `.env`, for example: ```env IMAGEMAGICK_BINARY="/usr/local/bin/magick" ``` Windows example: ```env IMAGEMAGICK_BINARY="C:\\Program Files\\ImageMagick-7.1.1-Q16-HDRI\\magick.exe" ``` ## No stock videos found - Verify `PEXELS_API_KEY` is valid - Try a broader video subject - Retry generation; stock results vary by query ## Subtitles fail - If using AssemblyAI, verify `ASSEMBLY_AI_API_KEY` - If not using AssemblyAI, local subtitle generation should still work ## YouTube upload skipped - Place `client_secret.json` inside `Backend/` - Enable required YouTube scopes and OAuth consent in Google Cloud ================================================ FILE: pyproject.toml ================================================ [project] name = "moneyprinter" version = "1.0.0" description = "Automate the creation of YouTube Shorts by providing a video topic." readme = "README.md" requires-python = ">=3.11" dependencies = [ "requests==2.31.0", "ollama==0.5.1", "moviepy==2.2.1", "termcolor==2.4.0", "flask==3.0.0", "curl-cffi", "flask-cors==4.0.0", "playsound==1.2.2", "pillow==9.5.0", "python-dotenv==1.0.0", "srt-equalizer==0.1.8", "platformdirs==4.1.0", "undetected-chromedriver", "assemblyai", "brotli", "google-api-python-client", "oauth2client", "sqlalchemy==2.0.36", "psycopg[binary]==3.2.3", ] [dependency-groups] dev = [ "pytest==8.4.1", ] [tool.pytest.ini_options] testpaths = ["tests"] addopts = "-q" ================================================ FILE: setup.sh ================================================ #!/usr/bin/env bash set -u SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" PROJECT_ROOT="$SCRIPT_DIR" if ! cd "$PROJECT_ROOT"; then printf 'Failed to enter project directory: %s\n' "$PROJECT_ROOT" exit 1 fi if [ -t 1 ] && command -v tput >/dev/null 2>&1; then COLOR_COUNT="$(tput colors 2>/dev/null || printf '0')" else COLOR_COUNT="0" fi if [ "$COLOR_COUNT" -ge 8 ]; then BOLD="$(tput bold)" RESET="$(tput sgr0)" RED="$(tput setaf 1)" GREEN="$(tput setaf 2)" YELLOW="$(tput setaf 3)" BLUE="$(tput setaf 4)" MAGENTA="$(tput setaf 5)" CYAN="$(tput setaf 6)" else BOLD='' RESET='' RED='' GREEN='' YELLOW='' BLUE='' MAGENTA='' CYAN='' fi print_banner() { printf '\n' printf '%s%sMoneyPrinter Interactive Setup%s\n' "$BOLD" "$MAGENTA" "$RESET" printf '%s--------------------------------%s\n' "$MAGENTA" "$RESET" printf '%sThis script helps you prepare your local environment.%s\n\n' "$CYAN" "$RESET" } info() { printf '%s[INFO]%s %s\n' "$BLUE" "$RESET" "$1" } ok() { printf '%s[OK]%s %s\n' "$GREEN" "$RESET" "$1" } warn() { printf '%s[WARN]%s %s\n' "$YELLOW" "$RESET" "$1" } error() { printf '%s[ERR]%s %s\n' "$RED" "$RESET" "$1" } command_exists() { command -v "$1" >/dev/null 2>&1 } ask_yes_no() { prompt="$1" default="$2" while true; do if [ "$default" = "y" ]; then printf '%s [Y/n]: ' "$prompt" else printf '%s [y/N]: ' "$prompt" fi read -r reply case "$reply" in [Yy]|[Yy][Ee][Ss]) return 0 ;; [Nn]|[Nn][Oo]) return 1 ;; '') if [ "$default" = "y" ]; then return 0 fi return 1 ;; *) warn 'Please answer y or n.' ;; esac done } check_python_version() { if ! command_exists python3; then error 'python3 not found (required: 3.11+).' return 1 fi PYTHON_VERSION="$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:3])))' 2>/dev/null || printf '0.0.0')" MAJOR="$(printf '%s' "$PYTHON_VERSION" | cut -d. -f1)" MINOR="$(printf '%s' "$PYTHON_VERSION" | cut -d. -f2)" if [ "$MAJOR" -gt 3 ] || { [ "$MAJOR" -eq 3 ] && [ "$MINOR" -ge 11 ]; }; then ok "python3 found ($PYTHON_VERSION)" return 0 fi error "python3 version is $PYTHON_VERSION (need 3.11+)" return 1 } check_prereqs() { info 'Checking prerequisites...' missing_critical=0 if ! check_python_version; then missing_critical=1 fi if command_exists uv; then ok 'uv found' else error 'uv not found (install: https://docs.astral.sh/uv/getting-started/installation/)' missing_critical=1 fi if command_exists ffmpeg; then ok 'ffmpeg found' else warn 'ffmpeg not found (required for video generation).' fi if command_exists magick || command_exists convert; then ok 'ImageMagick found' else warn 'ImageMagick not found (some text rendering features may fail).' fi if command_exists ollama; then ok 'ollama found' else warn 'ollama not found (required for script generation).' fi if [ "$missing_critical" -eq 1 ]; then error 'Missing critical dependencies. Please install them, then rerun setup.' return 1 fi return 0 } configure_local_database_url() { if [ ! -f .env ]; then return 0 fi db_result="$(python3 - <<'PY' from pathlib import Path env_path = Path('.env') text = env_path.read_text(encoding='utf-8') has_trailing_newline = text.endswith('\n') lines = text.splitlines() target = 'DATABASE_URL="sqlite:///moneyprinter.db"' for index, line in enumerate(lines): if not line.startswith('DATABASE_URL='): continue value = line.split('=', 1)[1].strip().strip('"').strip("'") if value == '' or value.startswith('postgresql+psycopg://'): lines[index] = target env_path.write_text( '\n'.join(lines) + ('\n' if has_trailing_newline else ''), encoding='utf-8', ) print('updated') else: print('kept') break else: lines.append(target) env_path.write_text( '\n'.join(lines) + ('\n' if has_trailing_newline or lines else ''), encoding='utf-8', ) print('added') PY )" case "$db_result" in updated) info 'Set DATABASE_URL to local SQLite default in .env' ;; added) info 'Added DATABASE_URL local SQLite default to .env' ;; *) info 'Keeping existing DATABASE_URL in .env' ;; esac } setup_env_file() { if [ ! -f .env.example ]; then warn '.env.example is missing; skipping env setup.' return 0 fi if [ -f .env ]; then if ask_yes_no '.env already exists. Overwrite it from .env.example?' 'n'; then cp .env.example .env ok '.env overwritten from .env.example' else info 'Keeping existing .env' fi else cp .env.example .env ok 'Created .env from .env.example' fi configure_local_database_url if ask_yes_no 'Open .env now to edit required keys?' 'y'; then if [ -n "${EDITOR:-}" ] && command_exists "$EDITOR"; then "$EDITOR" .env elif command_exists nano; then nano .env elif command_exists vi; then vi .env else warn "No terminal editor detected. Please edit $PROJECT_ROOT/.env manually." fi fi } install_dependencies() { if ask_yes_no 'Install Python dependencies with uv sync?' 'y'; then info 'Running uv sync...' if uv sync; then ok 'Dependencies installed' else error 'uv sync failed' return 1 fi else warn 'Skipped dependency installation.' fi return 0 } check_ollama_models() { if ! command_exists ollama; then return 0 fi if ! ask_yes_no 'Check local Ollama models now?' 'y'; then return 0 fi info 'Querying Ollama model list...' if ollama list; then ok 'Ollama is reachable.' else warn 'Could not query Ollama. If needed, run: ollama serve' return 0 fi if ask_yes_no 'Pull default model llama3.1:8b now?' 'n'; then printf 'Model name [llama3.1:8b]: ' read -r model_name model_name="${model_name:-llama3.1:8b}" info "Pulling model $model_name ..." if ollama pull "$model_name"; then ok "Model $model_name is ready" else warn "Failed to pull model $model_name" fi fi } print_next_steps() { printf '\n%sNext steps%s\n' "$BOLD" "$RESET" printf '%s1.%s Start backend: %suv run python Backend/main.py%s\n' "$CYAN" "$RESET" "$BOLD" "$RESET" printf '%s2.%s Start worker (new terminal): %suv run python Backend/worker.py%s\n' "$CYAN" "$RESET" "$BOLD" "$RESET" printf '%s3.%s Start frontend (new terminal): %spython3 -m http.server 3000 --directory Frontend%s\n' "$CYAN" "$RESET" "$BOLD" "$RESET" printf '%s4.%s Open: %shttp://localhost:3000%s\n\n' "$CYAN" "$RESET" "$BOLD" "$RESET" } main() { print_banner if ! check_prereqs; then exit 1 fi setup_env_file if ! install_dependencies; then exit 1 fi check_ollama_models print_next_steps ok 'Setup complete. Happy building!' } main "$@" ================================================ FILE: tests/conftest.py ================================================ import os import sys from pathlib import Path import pytest from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker PROJECT_ROOT = Path(__file__).resolve().parents[1] BACKEND_DIR = PROJECT_ROOT / "Backend" if str(BACKEND_DIR) not in sys.path: sys.path.insert(0, str(BACKEND_DIR)) os.environ["DATABASE_URL"] = "sqlite:///:memory:" from db import Base # noqa: E402 import models # noqa: F401,E402 @pytest.fixture def session_factory(tmp_path: Path): database_file = tmp_path / "test.db" engine = create_engine( f"sqlite:///{database_file}", connect_args={"check_same_thread": False}, ) session_factory = sessionmaker( bind=engine, autoflush=False, autocommit=False, expire_on_commit=False, ) Base.metadata.create_all(bind=engine) yield session_factory Base.metadata.drop_all(bind=engine) engine.dispose() @pytest.fixture def session(session_factory): with session_factory() as db_session: yield db_session ================================================ FILE: tests/test_api_jobs.py ================================================ import os import pytest from repository import append_event, create_job, get_job, list_job_events os.environ.setdefault("PEXELS_API_KEY", "test-key") os.environ.setdefault("TIKTOK_SESSION_ID", "test-session") os.environ.setdefault("IMAGEMAGICK_BINARY", "/bin/echo") os.environ.setdefault("DATABASE_URL", "sqlite:///moneyprinter_api_bootstrap.db") import main @pytest.fixture def client(monkeypatch, session_factory): monkeypatch.setattr(main, "SessionLocal", session_factory) return main.app.test_client() def test_generate_requires_video_subject(client): response = client.post("/api/generate", json={}) assert response.status_code == 400 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "videoSubject is required." def test_generate_creates_job_and_job_status_is_fetchable(client): response = client.post( "/api/generate", json={ "videoSubject": "api queue test", "paragraphNumber": 1, "customPrompt": "", }, ) assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert payload["message"] == "Video generation queued." job_id = payload["jobId"] job_response = client.get(f"/api/jobs/{job_id}") job_payload = job_response.get_json() assert job_response.status_code == 200 assert job_payload["status"] == "success" assert job_payload["job"]["id"] == job_id assert job_payload["job"]["state"] == "queued" def test_get_events_respects_after_query_parameter(client, session_factory): with session_factory() as session: job = create_job(session, payload={"videoSubject": "events"}) first_event = list_job_events(session, job.id)[0] append_event(session, job.id, "log", "info", "step 2") append_event(session, job.id, "log", "info", "step 3") session.commit() response = client.get(f"/api/jobs/{job.id}/events?after={first_event.id}") assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert len(payload["events"]) == 2 assert payload["events"][0]["message"] == "step 2" assert payload["events"][1]["message"] == "step 3" def test_cancel_job_endpoint_cancels_existing_job(client, session_factory): with session_factory() as session: job = create_job(session, payload={"videoSubject": "cancel endpoint"}) response = client.post(f"/api/jobs/{job.id}/cancel") assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert payload["message"] == "Cancellation requested." with session_factory() as session: updated_job = get_job(session, job.id) assert updated_job is not None assert updated_job.status == "cancelled" assert updated_job.cancel_requested is True def test_cancel_job_endpoint_returns_404_for_unknown_job(client): response = client.post("/api/jobs/missing-id/cancel") assert response.status_code == 404 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "Job not found." def test_cancel_latest_running_job_returns_404_when_no_active_job(client): response = client.post("/api/cancel") assert response.status_code == 404 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "No active job found." def test_cancel_latest_running_job_cancels_active_job(client, session_factory): with session_factory() as session: older_job = create_job(session, payload={"videoSubject": "older"}) newer_job = create_job(session, payload={"videoSubject": "newer"}) response = client.post("/api/cancel") assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert payload["message"] == "Cancellation requested." assert payload["jobId"] in {older_job.id, newer_job.id} with session_factory() as session: older = get_job(session, older_job.id) newer = get_job(session, newer_job.id) assert older is not None assert newer is not None cancelled_count = int(older.cancel_requested) + int(newer.cancel_requested) assert cancelled_count == 1 ================================================ FILE: tests/test_api_misc.py ================================================ import io import os import pytest os.environ.setdefault("PEXELS_API_KEY", "test-key") os.environ.setdefault("TIKTOK_SESSION_ID", "test-session") os.environ.setdefault("IMAGEMAGICK_BINARY", "/bin/echo") os.environ.setdefault("DATABASE_URL", "sqlite:///moneyprinter_api_misc_bootstrap.db") import main @pytest.fixture def client(): return main.app.test_client() def test_models_endpoint_success_response(client, monkeypatch): def fake_list_models(): return ["llama3.1:8b", "qwen3:8b"], "qwen3:8b" monkeypatch.setattr(main, "list_ollama_models", fake_list_models) response = client.get("/api/models") assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert payload["models"] == ["llama3.1:8b", "qwen3:8b"] assert payload["default"] == "qwen3:8b" def test_models_endpoint_fallback_on_error(client, monkeypatch): monkeypatch.setenv("OLLAMA_MODEL", "custom:model") def fake_list_models(): raise RuntimeError("ollama unavailable") monkeypatch.setattr(main, "list_ollama_models", fake_list_models) response = client.get("/api/models") assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "Could not fetch Ollama models. Is Ollama running?" assert payload["models"] == ["custom:model"] assert payload["default"] == "custom:model" def test_upload_songs_requires_files(client): response = client.post( "/api/upload-songs", data={}, content_type="multipart/form-data" ) assert response.status_code == 400 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "No files uploaded." def test_upload_songs_rejects_non_mp3_files(client, monkeypatch, tmp_path): songs_dir = tmp_path / "Songs" songs_dir.mkdir() monkeypatch.setattr(main, "SONGS_DIR", songs_dir) monkeypatch.setattr(main, "clean_dir", lambda path: None) data = { "songs": (io.BytesIO(b"not-mp3"), "track.wav"), } response = client.post( "/api/upload-songs", data=data, content_type="multipart/form-data", ) assert response.status_code == 400 payload = response.get_json() assert payload["status"] == "error" assert payload["message"] == "No MP3 files found." assert list(songs_dir.iterdir()) == [] def test_upload_songs_saves_mp3_and_sanitizes_filename(client, monkeypatch, tmp_path): songs_dir = tmp_path / "Songs" songs_dir.mkdir() stale_file = songs_dir / "stale.mp3" stale_file.write_bytes(b"old") def fake_clean_dir(path: str): assert path == str(songs_dir) for item in songs_dir.iterdir(): if item.is_file(): item.unlink() monkeypatch.setattr(main, "SONGS_DIR", songs_dir) monkeypatch.setattr(main, "clean_dir", fake_clean_dir) data = { "songs": [ (io.BytesIO(b"song-a"), "../danger.mp3"), (io.BytesIO(b"song-b"), "safe.mp3"), (io.BytesIO(b"ignore"), "note.txt"), ] } response = client.post( "/api/upload-songs", data=data, content_type="multipart/form-data", ) assert response.status_code == 200 payload = response.get_json() assert payload["status"] == "success" assert payload["message"] == "Uploaded 2 song(s)." saved_names = sorted(path.name for path in songs_dir.iterdir()) assert saved_names == ["danger.mp3", "safe.mp3"] ================================================ FILE: tests/test_repository.py ================================================ from repository import ( claim_next_queued_job, create_job, list_job_events, mark_failed, mark_completed, mark_cancelled, request_cancel, ) def test_create_job_persists_payload_and_queued_event(session): payload = {"videoSubject": "money basics", "paragraphNumber": 1} job = create_job(session, payload=payload) assert job.id assert job.status == "queued" assert job.payload == payload events = list_job_events(session, job.id) assert len(events) == 1 assert events[0].event_type == "queued" assert events[0].message == "Job queued." def test_request_cancel_cancels_queued_job_and_tracks_events(session): job = create_job(session, payload={"videoSubject": "cancel me"}) cancelled = request_cancel(session, job.id) assert cancelled is True events = list_job_events(session, job.id) event_types = [event.event_type for event in events] assert "cancel_requested" in event_types assert "cancelled" in event_types def test_claim_next_queued_job_marks_running_and_skips_cancelled(session): first_job = create_job(session, payload={"videoSubject": "first"}) second_job = create_job(session, payload={"videoSubject": "second"}) request_cancel(session, first_job.id) claimed_job = claim_next_queued_job(session) assert claimed_job is not None assert claimed_job.id == second_job.id assert claimed_job.status == "running" assert claimed_job.attempt_count == 1 def test_mark_completed_updates_status_and_emits_complete_event(session): job = create_job(session, payload={"videoSubject": "done"}) running_job = claim_next_queued_job(session) assert running_job is not None mark_completed(session, job.id, result_path="/tmp/output.mp4") events = list_job_events(session, job.id) complete_events = [event for event in events if event.event_type == "complete"] assert len(complete_events) == 1 assert complete_events[0].payload == {"path": "/tmp/output.mp4"} def test_mark_failed_updates_error_message_and_event(session): job = create_job(session, payload={"videoSubject": "bad run"}) mark_failed(session, job.id, error_message="render crash") events = list_job_events(session, job.id) assert events[-1].event_type == "error" assert events[-1].message == "render crash" def test_mark_cancelled_sets_status_and_writes_cancelled_event(session): job = create_job(session, payload={"videoSubject": "stop"}) mark_cancelled(session, job.id, reason="cancelled in worker") events = list_job_events(session, job.id) assert events[-1].event_type == "cancelled" assert events[-1].message == "cancelled in worker" ================================================ FILE: tests/test_utils.py ================================================ from pathlib import Path import utils def test_clean_dir_removes_existing_files_and_directories(tmp_path: Path): target_dir = tmp_path / "cleanup" nested_dir = target_dir / "nested" nested_dir.mkdir(parents=True) (target_dir / "root.txt").write_text("root") (nested_dir / "nested.txt").write_text("nested") utils.clean_dir(str(target_dir)) assert target_dir.exists() assert list(target_dir.iterdir()) == [] def test_choose_random_song_returns_none_if_songs_dir_missing( monkeypatch, tmp_path: Path ): songs_dir = tmp_path / "Songs" monkeypatch.setattr(utils, "SONGS_DIR", songs_dir) assert utils.choose_random_song() is None def test_choose_random_song_returns_selected_mp3(monkeypatch, tmp_path: Path): songs_dir = tmp_path / "Songs" songs_dir.mkdir() first_song = songs_dir / "a.mp3" second_song = songs_dir / "b.mp3" ignored_file = songs_dir / "notes.txt" first_song.write_text("a") second_song.write_text("b") ignored_file.write_text("ignore") monkeypatch.setattr(utils, "SONGS_DIR", songs_dir) monkeypatch.setattr(utils.random, "choice", lambda songs: songs[0]) selected = utils.choose_random_song() assert selected == str(first_song) def test_resolve_imagemagick_binary_prefers_configured_existing_path( monkeypatch, tmp_path: Path ): fake_binary = tmp_path / "magick" fake_binary.write_text("binary") monkeypatch.setenv("IMAGEMAGICK_BINARY", str(fake_binary)) resolved = utils.resolve_imagemagick_binary() assert resolved == str(fake_binary.resolve()) def test_resolve_imagemagick_binary_falls_back_to_path_lookup(monkeypatch): monkeypatch.setenv("IMAGEMAGICK_BINARY", "") def fake_which(candidate: str): if candidate == "magick": return "/usr/local/bin/magick" return None monkeypatch.setattr(utils.shutil, "which", fake_which) resolved = utils.resolve_imagemagick_binary() assert resolved == "/usr/local/bin/magick" ================================================ FILE: tests/test_worker.py ================================================ from repository import create_job, get_job, list_job_events import worker def _disable_cleanup(monkeypatch): monkeypatch.setattr(worker, "clean_dir", lambda _: None) def test_process_next_job_returns_false_when_queue_is_empty( monkeypatch, session_factory ): monkeypatch.setattr(worker, "SessionLocal", session_factory) _disable_cleanup(monkeypatch) assert worker.process_next_job() is False def test_process_next_job_marks_completed_on_pipeline_success( monkeypatch, session_factory ): with session_factory() as session: job = create_job(session, payload={"videoSubject": "worker success"}) monkeypatch.setattr(worker, "SessionLocal", session_factory) _disable_cleanup(monkeypatch) def fake_pipeline(data, is_cancelled, on_log): assert data["videoSubject"] == "worker success" assert is_cancelled() is False on_log("pipeline started", "info") return "rendered.mp4" monkeypatch.setattr(worker, "run_generation_pipeline", fake_pipeline) assert worker.process_next_job() is True with session_factory() as session: updated_job = get_job(session, job.id) assert updated_job is not None assert updated_job.status == "completed" assert updated_job.result_path == "rendered.mp4" event_types = [event.event_type for event in list_job_events(session, job.id)] assert "running" in event_types assert "log" in event_types assert "complete" in event_types def test_process_next_job_marks_cancelled_on_pipeline_cancelled( monkeypatch, session_factory ): with session_factory() as session: job = create_job(session, payload={"videoSubject": "worker cancelled"}) monkeypatch.setattr(worker, "SessionLocal", session_factory) _disable_cleanup(monkeypatch) def fake_pipeline(*_args, **_kwargs): raise worker.PipelineCancelled("cancelled by user") monkeypatch.setattr(worker, "run_generation_pipeline", fake_pipeline) assert worker.process_next_job() is True with session_factory() as session: updated_job = get_job(session, job.id) assert updated_job is not None assert updated_job.status == "cancelled" events = list_job_events(session, job.id) assert events[-1].event_type == "cancelled" assert events[-1].message == "cancelled by user" def test_process_next_job_marks_failed_on_pipeline_error(monkeypatch, session_factory): with session_factory() as session: job = create_job(session, payload={"videoSubject": "worker failure"}) monkeypatch.setattr(worker, "SessionLocal", session_factory) _disable_cleanup(monkeypatch) def fake_pipeline(*_args, **_kwargs): raise RuntimeError("pipeline exploded") monkeypatch.setattr(worker, "run_generation_pipeline", fake_pipeline) assert worker.process_next_job() is True with session_factory() as session: updated_job = get_job(session, job.id) assert updated_job is not None assert updated_job.status == "failed" assert updated_job.error_message == "pipeline exploded" events = list_job_events(session, job.id) assert events[-1].event_type == "error" assert events[-1].message == "pipeline exploded" def test_job_cancelled_helper_returns_true_for_missing_job( monkeypatch, session_factory ): monkeypatch.setattr(worker, "SessionLocal", session_factory) assert worker._job_cancelled("missing-job-id") is True def test_job_cancelled_helper_reflects_cancel_flag(monkeypatch, session_factory): with session_factory() as session: job = create_job(session, payload={"videoSubject": "cancel-check"}) monkeypatch.setattr(worker, "SessionLocal", session_factory) assert worker._job_cancelled(job.id) is False with session_factory() as session: job_to_update = get_job(session, job.id) assert job_to_update is not None job_to_update.cancel_requested = True session.commit() assert worker._job_cancelled(job.id) is True def test_log_event_helper_persists_log_event(monkeypatch, session_factory): with session_factory() as session: job = create_job(session, payload={"videoSubject": "log-check"}) monkeypatch.setattr(worker, "SessionLocal", session_factory) worker._log_event(job.id, "hello event", "warning") with session_factory() as session: events = list_job_events(session, job.id) assert events[-1].event_type == "log" assert events[-1].level == "warning" assert events[-1].message == "hello event"