Showing preview only (215K chars total). Download the full file or copy to clipboard to get everything.
Repository: MichaelYochpaz/iSubRip
Branch: main
Commit: 6bfc3663afea
Files: 38
Total size: 203.2 KB
Directory structure:
gitextract_mbnetjyr/
├── .github/
│ └── ISSUE_TEMPLATE/
│ ├── 1_issue_report.yml
│ ├── 2_feature_request.yml
│ ├── 3_question.yml
│ └── config.yml
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── README.md
├── example-config.toml
├── isubrip/
│ ├── __init__.py
│ ├── __main__.py
│ ├── cli.py
│ ├── commands/
│ │ ├── __init__.py
│ │ └── download.py
│ ├── config.py
│ ├── constants.py
│ ├── data_structures.py
│ ├── logger.py
│ ├── scrapers/
│ │ ├── __init__.py
│ │ ├── appletv_scraper.py
│ │ ├── itunes_scraper.py
│ │ └── scraper.py
│ ├── subtitle_formats/
│ │ ├── __init__.py
│ │ ├── subrip.py
│ │ ├── subtitles.py
│ │ └── webvtt.py
│ ├── ui.py
│ └── utils.py
├── pyproject.toml
└── tests/
├── .gitignore
├── __init__.py
├── benchmarks/
│ ├── __init__.py
│ ├── download_benchmark.py
│ └── download_benchmark_module.py
├── test_utils.py
└── tools/
├── __init__.py
├── generate_mock_data.py
└── mock_loader.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/1_issue_report.yml
================================================
name: Bug / Issue Report
description: Report a bug or an issue.
title: "[Issue]: "
labels: [bug]
body:
- type: markdown
attributes:
value: >
**Before opening an issue, please make sure you are running the latest version of iSubRip,
and that there isn't an already-existing open issue for your issue under the [issues tab](https://github.com/MichaelYochpaz/iSubRip/labels/bug).**
- type: checkboxes
id: check-confirmation
attributes:
label: Confirmations
options:
- label: "I have checked the issues tab, and couldn't find an existing open issue for the issue I want to report."
required: true
- type: dropdown
id: os-type
attributes:
label: OS Type
description: The operation system that's being used to run iSubRip.
options:
- Windows
- MacOS
- Linux
validations:
required: true
- type: input
id: python-version
attributes:
label: Python Version
description: |
The Python version that's being used to run iSubRip.
Can be checked by running `python --version`.
placeholder: |
Example: "3.10.6"
validations:
required: true
- type: input
id: version
attributes:
label: Package Version
description: |
iSubRip's version that's being used.
Can be checked by running `python -m pip show isubrip`.
placeholder: |
Example: "2.3.2"
validations:
required: true
- type: textarea
id: description
attributes:
label: Description
description: |
A summary of the issue.
Include as much information as possible, and steps to reproduce (if they're known).
Log files (see README for more information) can be attached by clicking the area to highlight it, and then dragging & dropping files in.
validations:
required: true
- type: textarea
id: output-log
attributes:
label: Output Log
description: |
iSubRip's output when the issue occurred.
Please include the command that was used to run iSubRip.
render: Text
placeholder: |
Example:
isubrip https://itunes.apple.com/us/movie/can-you-hear-us-now/id1617191490
Scraping https://itunes.apple.com/us/movie/can-you-hear-us-now/id1617191490...
Found movie: Can You Hear Us Now?
Traceback (most recent call last):
File "%appdata%\local\programs\python\python38-32\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "%appdata%\local\programs\python\python38-32\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "%appdata%\local\programs\python\python38-32\scripts\isubrip.exe\__main__.py", line 7, in <module>
File "%appdata%\local\programs\python\python38-32\lib\site-packages\isubrip\__main__.py", line 91, in main
os.makedirs(current_download_path, exist_ok=True)
File "%appdata%\local\programs\python\python38-32\lib\os.py", line 221, in makedirs
mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\%appdata%\\Local\\Temp\\iSubRip\\Can.You.Hear.Us.Now?.iT.WEB'
validations:
required: true
- type: textarea
id: config
attributes:
label: Config
description: |
The iSubRip config file you are using.
**Leave empty only if there is no config file in use.**
render: TOML
placeholder: |
Example:
[downloads]
folder = "C:\\Subtitles\\iTunes"
languages = ["en-US", "fr-FR", "he"]
zip = false
[subtitles]
convert-to-srt = true
fix-rtl = true
validations:
required: false
================================================
FILE: .github/ISSUE_TEMPLATE/2_feature_request.yml
================================================
name: Feature Request
description: Request a new feature or improvement to an existing one.
title: "[Feature Request]: "
labels: [feature-request]
body:
- type: markdown
attributes:
value: >
**Before opening an issue, please make sure there isn't an already-existing issue, open or closed,
for this feature request under the [issues tab](https://github.com/MichaelYochpaz/iSubRip/issues?q=label%3Afeature-request).**
- type: checkboxes
id: check-confirmation
attributes:
label: Confirmations
options:
- label: "I have checked the issues tab, and couldn't find an existing issue with my feature request."
required: true
- type: textarea
id: description
attributes:
label: Description
description: A summary of the feature you want to request.
validations:
required: true
================================================
FILE: .github/ISSUE_TEMPLATE/3_question.yml
================================================
name: Ask a question
description: Ask a question regarding iSubRip.
title: "[Question]: "
labels: [question]
body:
- type: markdown
attributes:
value: |
**Please use this template only for questions.
For issue / bug reports and feature requests, use one of the other templates.**
- type: textarea
id: description
attributes:
label: Question
description: The question you want to ask.
validations:
required: true
================================================
FILE: .github/ISSUE_TEMPLATE/config.yml
================================================
blank_issues_enabled: false
================================================
FILE: .gitignore
================================================
# From: https://github.com/github/gitignore/blob/main/Python.gitignore
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintainted in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
================================================
FILE: CHANGELOG.md
================================================
# Changelog
## 2.6.8 [2025-10-14]
### Changes:
* Removed Python 3.9 support, added Python 3.14 support.
### Bug Fixes:
* Fixed a regression issue where the source type (e.g. "WEB") was missing from zip file names following the changes in version 2.6.7. ([Issue #100](https://github.com/MichaelYochpaz/iSubRip/issues/100))
---
## 2.6.7 [2025-10-10]
### Added:
* Updated M3U8 playlists loading to use alternative playlist URLs as fallback in case of an empty response.
* Added playlist URLs to debug-level logs. ([Issue #94](https://github.com/MichaelYochpaz/iSubRip/issues/94))
---
## 2.6.6 [2025-08-13]
### Changes:
* Updated the date format in logs for unreleased content that has a release date. ([Issue #91](https://github.com/MichaelYochpaz/iSubRip/issues/91))
### Bug Fixes:
* Fixed an issue where in some cases, when converting from WebVTT to SubRip with the `subrip-alignment-conversion` setting enabled, top-aligned captions were not detected correctly. ([Issue #90](https://github.com/MichaelYochpaz/iSubRip/issues/90))
---
## 2.6.5 [2025-06-20]
### Changes:
* Added missing languages to the list of RTL languages (relevant if the `languages.fix-rtl` config setting is enabled).
* Minor improvements to the download progress bar.
### Bug Fixes:
* Fixed an issue where in some cases, the "Downloaded subtitles (X/Y)" progress log line would print repeatedly. ([Issue #87](https://github.com/MichaelYochpaz/iSubRip/issues/87))
---
## 2.6.4 [2025-06-13]
### Changes:
* Minor improvements to the download progress bar.
### Bug Fixes:
* Fixed an issue where if there is no config file, an error is raised instead of using default settings. ([Issue #86](https://github.com/MichaelYochpaz/iSubRip/issues/86))
---
## 2.6.3 [2025-03-17]
### Bug Fixes:
* Fixed an issue where logs containing the percentage character (`%`) would raise an error. ([Issue #82](https://github.com/MichaelYochpaz/iSubRip/issues/82))
---
## 2.6.2 [2025-02-04]
### Bug Fixes:
* Fixed an issue where AppleTV API calls would fail due to changes on AppleTV requiring a missing `utsk` parameter. ([Issue #80](https://github.com/MichaelYochpaz/iSubRip/issues/80))
* Fixed an issue where iTunes URLs would not work due to iTunes no longer redirecting to AppleTV. A different method will be used now to find corresponding AppleTV URLs. Also added a retry mechanism as it appears to be a bit unreliable at times. (thanks @yonatand1230 for suggesting this method!). ([Issue #78](https://github.com/MichaelYochpaz/iSubRip/issues/78))
* Removed progress bar when where there are no matching subtitles to download (previously, it would just show 0/0 with 0% progress).
---
## 2.6.1 [2025-01-31]
### Bug Fixes:
* Fixed a backwards compatibility issue in code, which would cause errors when running on Python versions lower than 3.12. ([Issue #78](https://github.com/MichaelYochpaz/iSubRip/issues/78))
---
## 2.6.0 [2025-01-28]
**The following update contains breaking changes to the config file.
If you are using one, please update your config file accordingly.**
### Added:
* Added a new `general.log-level` config setting, the log level of stdout (console) output. Set to `info` by default. Can be changed to `debug`, `warning`, or `error`. See the updated [example config](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml) for an example.
### Changes:
* Console output has been overhauled and improved, with colorful interactive output.
* Config file is now parsed and validated in a more reliable and efficient manner. Configuration errors will now be more readable and descriptive.
* **Breaking config changes** - the `scrapers` config category has been updated. Settings that should apply for all scrapers are now under the `scrapers.default` category instead of straight under `scrapers`. See the updated [example config](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml) for examples.
* Updated AppleTV scraper request parameters.
* Minor improvements to logs.
* Python 3.8 is no longer supported. Minimum supported version has been updated to 3.9.
### Bug Fixes:
* Fixed an issue where if `verify-ssl` is set to `false`, and the `urllib3` package (which isn't a dependency of iSubRip) is not installed, an error could be thrown.
---
## 2.5.6 [2024-07-07]
### Bug Fixes:
* Fixed an issue where the update message from version `2.5.4` to `2.5.5` would still appear after updating. ([Issue #73](https://github.com/MichaelYochpaz/iSubRip/issues/73))
---
## 2.5.5 [2024-07-06]
### Added:
* Added new `timeout` setting to the config file, for the option to change the timeout for all / specific scrapers. See the updated [example config](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml) for usage examples. ([Issue #71](https://github.com/MichaelYochpaz/iSubRip/issues/71))
### Changes:
* Default timeout for requests has been updated from 5 seconds to 10 seconds. ([Issue #71](https://github.com/MichaelYochpaz/iSubRip/issues/71))
---
## 2.5.4 [2024-04-28]
### Bug Fixes:
* Fixed an issue where if the `logs` directory does not exist, the folder isn't created, causing an error. ([Issue #67](https://github.com/MichaelYochpaz/iSubRip/issues/67))
* Fixed an issue where the summary log of successful and failed download would not account for failed downloads. ([Issue #68](https://github.com/MichaelYochpaz/iSubRip/issues/68))
---
## 2.5.3 [2024-04-09]
### Added:
* Added new `proxy` and `verify-ssl` settings to the config file, for allowing the usage of a proxy when making requests, and disabling SSL verification. See the updated [example config](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml) for usage examples.
### Changes:
* `subtitles.rtl-languages` config setting is no longer supported, and its values are now hardcoded and can't be modified.
### Bug Fixes:
* Fixed an issue where in some cases, `STYLE` blocks would repeat throughout the subtitles file, and cause inaccurate cue count. ([Issue #63](https://github.com/MichaelYochpaz/iSubRip/issues/63))
* Fixed an issue where the WebVTT Style blocks would have their `STYLE` tag replaced with a `REGION` tag in downloaded subtitles.
* Fixed an issue where an empty playlist (with a size of 0 bytes) would be reported as a valid playlist with no matching subtitles. ([Issue #65](https://github.com/MichaelYochpaz/iSubRip/issues/65))
---
## 2.5.2 [2024-01-06]
### Bug Fixes:
* Fixed an issue where errors would not be handled gracefully, and cause an unexpected crash. ([Issue #55](https://github.com/MichaelYochpaz/iSubRip/issues/55))
---
## 2.5.1 [2023-12-23]
### Bug Fixes:
* Fixed an issue where source abbreviation was missing from file names of downloaded subtitles files. ([Issue #53](https://github.com/MichaelYochpaz/iSubRip/issues/53))
---
## 2.5.0 [2023-12-16]
### Added:
* Added logs. See the new [Logs section in the README](https://github.com/MichaelYochpaz/iSubRip#logs) for more information.
* Added a new `subtitles.webvtt.subrip-alignment-conversion` config setting (which is off by default), which if set to true, will add the `{\an8}` tag at the start of lines that are annotated at the top (with the `line:0.00%` WebVTT setting) when converting to SubRip. ([Issue #35](https://github.com/MichaelYochpaz/iSubRip/issues/35))
* Implemented caching for AppleTV's storefront configuration data, which should reduce the amount of requests used when scraping multiple AppleTV URLs from the same storefront.
### Changes:
* Big backend changes to the structure of the code, mostly to improve modularity and allow for easier development in the future, and improve performance.
* Updated the CLI output to utilize logs and print with colors according to log-level.
* Improved error handling in some cases where an invalid URL is used.
### Bug Fixes:
* Fixed an issue where if a movie is a pre-order with a set release date, a message with availability date wouldn't be printed in some cases.
---
## 2.4.3 [2023-06-18]
### Bug Fixes:
* Fixed an issue where some AppleTV URLs (or iTunes links that refer to such URLs) would not be matched in some cases, resulting in a "No matching scraper was found..." error. ([Issue #46](https://github.com/MichaelYochpaz/iSubRip/issues/46))
---
## 2.4.2 [2023-06-02]
### Changes:
* Improved error handling for subtitles downloads. ([Issue #44](https://github.com/MichaelYochpaz/iSubRip/issues/44))
### Bug Fixes:
* Fixed an issue where using a ZIP file, and saving to a different drive than the OS drive would fail. ([Issue #43](https://github.com/MichaelYochpaz/iSubRip/issues/43))
---
## 2.4.1 [2023-05-25]
### Bug Fixes:
* Fixed an issue where saving subtitles to a different drive than the OS drive would fail. ([Issue #41](https://github.com/MichaelYochpaz/iSubRip/issues/41))
* Fixed AppleTV URLs with multiple iTunes playlists causing an error. ([Issue #42](https://github.com/MichaelYochpaz/iSubRip/issues/42))
---
## 2.4.0 [2023-05-23]
### Added:
- iTunes links will now redirect to AppleTV and scrape metadata from there, as AppleTV has additional and more accurate metadata.
- Improved error messages to be more informative and case-specific:
- If a movie is a pre-order and has no available playlist, a proper error message will be printed with its release date (if available).
- If trying to scrape AppleTV+ content or series (which aren't currently supported), a proper error will be printed.
### Changes:
- A major refactor to the code, to make it more modular and allow for easier development of new features in the future.
- Multiple changes (with some breaking changes) to the config file:
- The `downloads.format` setting is deprecated, and replaced by the `subtitles.convert-to-srt` setting.
- The `downloads.merge-playlists` setting is deprecated, with no replacement.
If an AppleTV link has multiple playlists, they will be downloaded separately.
- The `downloads.user-agent` setting is deprecated, with no replacement.
The user-agent used by the scraper, will be used for downloads as well.
- The `scraping` config category no longer exists, and is replaced by a `scrapers` category, which has a sub-category with settings for each scraper (for example, a `scrapers.itunes` sub-category).
- Old config paths that were previously deprecated are no longer supported and will no longer work.
The updated config settings can be found in the [example config](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml).
### Notes:
* This release includes a major rewrite of the code, which may have introduced new bugs to some core features. If you encountered one, [please report it](https://github.com/MichaelYochpaz/iSubRip/issues/new/choose).
* Minimum supported Python version bumped to 3.8.
* `beautifulsoup4` and `lxml` packages are no longer required or used.
---
## 2.3.3 [2022-10-09]
### Changes:
* Added release year to zip file names. ([Issue #31](https://github.com/MichaelYochpaz/iSubRip/issues/31))
* If the generated path for a zip file is already taken, a number will be appended at the end of the file's name to avoid overwriting. ([Issue #34](https://github.com/MichaelYochpaz/iSubRip/issues/34))
### Bug Fixes:
* Fixed an exception being thrown if the path to downloads folder on the config is invalid.
* Fixed AppleTV URLs without a movie title not working. ([Issue #29](https://github.com/MichaelYochpaz/iSubRip/issues/29))
* Fixed issues for movies with specific characters (`/`, `:`), and Windows reserved names in their title. ([Issue #30](https://github.com/MichaelYochpaz/iSubRip/issues/30))
---
## 2.3.2 [2022-08-06]
### Changes:
* Changed config paths to the following locations:
Windows: `%USERPROFILE%\.isubrip\config.json`
Linux / macOS: `$HOME/.isubrip/config.json`
More info under Notes (and examples on the [README](https://github.com/MichaelYochpaz/iSubRip#configuration) file).
### Bug Fixes:
* Fixed an error with AppleTV links for movies released before 1970 (Epoch time). ([Issue #21](https://github.com/MichaelYochpaz/iSubRip/issues/21))
* Fixed config file not being loaded on macOS. ([Issue #22](https://github.com/MichaelYochpaz/iSubRip/issues/22))
* Fixed AppleTV scraping from the same storefront. ([Issue #24](https://github.com/MichaelYochpaz/iSubRip/issues/24))
### Notes:
* Running iSubRip with a config file in the previous locations will still work, but support for them will be dropped in the future.
* `xdg` package is no longer required or used.
---
## 2.3.1 [2022-07-15]
### Changes:
* Improved AppleTV scraping to utilize AppleTV's API instead of scraping HTML.
### Bug Fixes:
* Fixed HTML escaped (for non-English) characters not matching AppleTV's URL RegEx. ([Issue #15](https://github.com/MichaelYochpaz/iSubRip/issues/15))
---
## 2.3.0 [2022-06-23]
### Added:
* AppleTV movie URLs are now supported.
* Added a `merge-playlists` config option to treat multiple playlists that can be found on AppleTV pages as one (more info on the example config).
### Changes:
* Improved subtitles parser to perserve additional WebVTT data.
* The config value `user-agent` under `scraping` is now separated to 2 different values: `itunes-user-agent`, and `appletv-user-agent`.
### Bug Fixes:
* Fixed movie titles with invalid Windows file-name characters (example: '?') causing a crash. ([Issue #14](https://github.com/MichaelYochpaz/iSubRip/issues/14))
* Fixed iTunes store URLs without a movie title not working. ([Issue #13](https://github.com/MichaelYochpaz/iSubRip/issues/13))
---
## 2.2.0 [2022-04-25]
### Added:
* Replaced FFmpeg usage for parsing with a native subtitles parser (downloads are much faster now).
* Added a `remove-duplicates` configuration remove duplicate paragraphs. (Was previously automatically fixed by FFmpeg.)
* Added `fix-rtl` and `rtl-languages` configuration to fix RTL in RTL-languaged subtitles (has to be enabled in the config).
### Changes:
* FFmpeg is no longer required or used, and all FFmpeg-related settings are deprecated.
### Notes:
* `fix-rtl` is off by default and has to be enabled on the config. Check the `config.toml` example file for more info.
* Minimum supported Python version bumped to 3.7.
---
## 2.1.2 [2022-04-03]
### Bug Fixes:
* Fixed subtitles being downloaded twice, which causes long (doubled) download times.
---
## 2.1.1 [2022-03-28]
### Bug Fixes:
* Fixed a compatibility issue with Python versions that are lower than 3.10.
* Fixed downloading subtitles to an archive file not working properly.
* Fixed a bug where the code continues to run if subtitles download failed, as if the download was successful.
---
## 2.1.0 [2022-03-19]
### Added:
* A note will be printed if a newer version is available on PyPI (can be disabled on the config).
* Config will now be checked for errors before running.
### Changes:
* Big improvements to scraping, which is now far more reliable.
* Added release year to subtitles file names.
* Config structure slightly changed.
### Notes:
* If you use a user-config, it might need to be updated to match the new config structure.
Example of an updated valid structure can be found [here](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml).
---
## 2.0.0 [2022-01-30]
The script is now a Python package that can be installed using pip.
### Added:
* Added a config file for changing configurations. (Example can be found [here](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml))
* Added an option to choose subtitles format (vtt / srt).
* Added an option to choose whether to zip subtitles files or not.
* Multiple links can be passed for downloading subtitles for multiple movies one after another.
* Temporary files are automatically removed if the script stops unexpectedly.
### Changes:
* A complete code overhaul from a single python script file to a package, while utilizing OOP and classes.
* Improved scraping algorithm for faster playlist scraping.
* FFmpeg will now automatically overwrite existing subtitles with the same file name.
### Bug Fixes:
* Fixed a bug where in some cases, no subtitles were found since the title has HTML escaped characters, which causes bad matching when checking if a valid playlist was found.
---
## 1.0.6 [2021-07-23]
### Bug Fixes:
* Fixed an issue where in some cases subtitles won't download when using `DOWNLOAD_FILTER` because of letter casing not matching.
* Fixed and improved error handling, and added more descriptive error messages. ([Issue #9](https://github.com/MichaelYochpaz/iSubRip/issues/9))
---
## 1.0.5 [2021-05-27]
### Bug Fixes:
* Fixed subtitles for some movies not being found after previous release. ([Issue #8](https://github.com/MichaelYochpaz/iSubRip/issues/8))
---
## 1.0.4 [2021-05-25]
### Bug Fixes:
* Fixed the script not working after iTunes webpage data orientation slightly changed. ([Issue #6](https://github.com/MichaelYochpaz/iSubRip/issues/6) , [Issue #7](https://github.com/MichaelYochpaz/iSubRip/issues/7))
---
## 1.0.3 [2021-04-30]
### Bug Fixes:
* Fixed a bug where subtitles for suggested movies are being downloaded if movie's main playlist is not found. ([Issue #2](https://github.com/MichaelYochpaz/iSubRip/issues/2))
* Added a "cc" tag to closed-caption subtitles' filename to avoid a collision with non-cc subtitles. ([Issue #3](https://github.com/MichaelYochpaz/iSubRip/issues/3))
---
## 1.0.2 [2021-04-15]
### Added:
* Added a User-Agent for sessions to avoid being blocked.
### Changes:
* `DOWNLOAD_FILTER` is no longer case-sensitive.
* Added `lxml` to `requirements.txt`. ([Issue #1](https://github.com/MichaelYochpaz/iSubRip/issues/1))
### Bug Fixes:
* Fixed the script not working after iTunes webpage data orientation slightly changed. ([Issue #1](https://github.com/MichaelYochpaz/iSubRip/issues/1))
---
## 1.0.1 [2020-12-13]
### Changes:
* Improved error handling.
### Bug Fixes:
* Fixed file name formatting.
---
## 1.0.0 [2020-11-02]
* Initial release.
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2022 Michael Yochpaz
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
> [!CAUTION]
> iSubRip is currently not working due to changes on Apple's backend.
> The future of this project is currently unknown. See [#103](https://github.com/MichaelYochpaz/iSubRip/issues/103) for details.
# iSubRip
**iSubRip** is a Python command-line tool for scraping and downloading subtitles from AppleTV and iTunes movie pages.
<div align="center">
<a href="https://python.org/pypi/isubrip"><img alt="Python Version" src="https://img.shields.io/pypi/pyversions/isubrip"></a>
<a href="https://python.org/pypi/isubrip"><img alt="PyPI Version" src="https://img.shields.io/pypi/v/isubrip"></a>
<a href="https://github.com/MichaelYochpaz/iSubRip/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/github/license/MichaelYochpaz/iSubRip"></a>
<a href="https://python.org/pypi/isubrip"><img alt="Monthly Downloads" src="https://pepy.tech/badge/isubrip/month"></a>
<a href="https://python.org/pypi/isubrip"><img alt="Total Downloads" src="https://pepy.tech/badge/isubrip"></a>
<a href="https://github.com/MichaelYochpaz/iSubRip"><img alt="Repo Stars" src="https://img.shields.io/github/stars/MichaelYochpaz/iSubRip?style=flat&color=gold"></a>
<a href="https://github.com/MichaelYochpaz/iSubRip/issues"><img alt="Issues" src="https://img.shields.io/github/issues/MichaelYochpaz/iSubRip?color=red"></a>
</div>
<br/>
<div align="center">
<img src="https://github.com/user-attachments/assets/ffdbb366-8ad0-427d-af00-9b70cc0d6b01" width="800">
</div>
---
## ✨ Features
- Scrape subtitles from AppleTV and iTunes movies without needing a purchase or account.
- Retrieve the expected streaming release date (if available) for unreleased movies.
- Utilize asynchronous downloading to speed up the download of chunked subtitles.
- Automatically convert subtitles to SubRip (SRT) format.
- Fix right-to-left (RTL) alignment in RTL language subtitles automatically.
- Configure settings such as download folder, preferred languages, and toggling features.
## 🚀 Quick Start
### Installation
```shell
pip install isubrip
```
### Usage
```shell
isubrip <URL> [URL...]
```
<sub>(URL can be either an AppleTV or iTunes movie URL)</sub>
<br/>
> [!WARNING]
> iSubRip is not recommended for use as a library in other projects.
> The API frequently changes, and breaking changes to the API are common, even in minor versions.
>
> Support will not be provided for issues arising from using this package as a library.
## 🛠 Configuration
A [TOML](https://toml.io) configuration file can be created to customize various options and features.
The configuration file will be searched for in one of the following paths based on your operating system:
- **Windows**: `%USERPROFILE%\.isubrip\config.toml`
- **Linux / macOS**: `$HOME/.isubrip/config.toml`
### Path Examples
- **Windows**: `C:\Users\Michael\.isubrip\config.toml`
- **Linux**: `/home/Michael/.isubrip/config.toml`
- **macOS**: `/Users/Michael/.isubrip/config.toml`
### Example Configuration
```toml
[downloads]
folder = "C:\\Subtitles\\iTunes"
languages = ["en-US", "fr-FR", "he"]
zip = false
[subtitles]
convert-to-srt = true
fix-rtl = true
[subtitles.webvtt]
subrip-alignment-conversion = true
```
An example config with details and explanations for all available settings can be found [here](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml).
## 📜 Logs
Log files are created for each run in the following paths, depending on your operating system:
**Windows**: `%USERPROFILE%\.isubrip\logs`
**Linux / macOS**: `$HOME/.isubrip/logs`
Log rotation (deletion of old files once a certain number of files is reached) can be configured in the configuration file using the `general.log-rotation-size` setting. The default value is `15`.
For more details, see the [example configuration](https://github.com/MichaelYochpaz/iSubRip/blob/main/example-config.toml).
## 📓 Changelog
The changelog for the latest, and all previous versions, can be found [here](https://github.com/MichaelYochpaz/iSubRip/blob/main/CHANGELOG.md).
## 👨🏽💻 Contributing
This project is open-source but currently lacks the infrastructure to fully support external contributions.
If you wish to contribute, please open an issue first to discuss your proposed changes to avoid working on something that might not be accepted.
## 🙏🏽 Support
If you find this project helpful, please consider supporting it by:
- 🌟 Starring the repository
- 💖 [Sponsoring the project](https://github.com/sponsors/MichaelYochpaz)
## 📝 End User License Agreement
By using iSubRip, you agree to the following terms:
1. **Disclaimer of Affiliation**: iSubRip is an independent, open-source project. It is not affiliated with, endorsed by, or in any way officially connected to Apple Inc., iTunes, or AppleTV.
2. **Educational Purpose**: This tool is developed and provided for educational and research purposes only. It demonstrates techniques for accessing and processing publicly available, unencrypted subtitle data from HLS playlists.
3. **User Responsibility and Compliance**: Any use of iSubRip is solely at the user's own risk and discretion. Users are responsible for ensuring that their use of the tool complies with all applicable laws, regulations, and terms of service of the content providers. This includes adhering to local, state, national, and international laws and regulations.
4. **Limitation of Liability**: The developers of iSubRip shall not be held responsible for any legal consequences arising from the use of this tool. This includes, but is not limited to, claims of copyright infringement, intellectual property violations, or breaches of terms of service of content providers. Users assume all risks associated with acquiring and using subtitle data through this tool.
By using iSubRip, you acknowledge that you have read, understood, and agree to be bound by this agreement's terms and conditions.
## ⚖️ License
This project is licensed under the MIT License. For more details, see the [LICENSE file](https://github.com/MichaelYochpaz/iSubRip/blob/main/LICENSE).
================================================
FILE: example-config.toml
================================================
# ---------------- ⚠️ IMPORTANT - READ BEFORE USING ⚠️ ----------------
# This is an example config file with all available settings and their default values (if they have one).
# All settings are optional, and setting them in the config file will override their default values.
#
# In your config file, set only settings you wish to change from their default values.
# Do NOT copy this file and use it as your config, as it will override ALL settings with the values specified here.
# Use this file only as a reference to understand what different settings do,
# and to decide which settings you should use in your config.
#
# Your config file should be saved in the following path (according to OS):
# - Windows: %USERPROFILE%\.isubrip\config.toml
# - Linux / macOS: $HOME/.isubrip/config.toml
# ---------------------------------------------------------------------
[general]
# Check for updates before running, and show a note if a new version exists.
# Value can be either 'true' or 'false'.
check-for-updates = true
# Maximum number of log files to keep in the logs folder.
# Once the maximum number is reached, the oldest logs files will be deleted in rotation
# until the number of files equals the maximum.
log-rotation-size = 15
# Log level to use for stdout (console) output.
# Value can be one of: "debug", "info", "error", "warning", "critical".
log-level = "info"
[downloads]
# Folder to downloads files to.
# The default "." value means it will download to the same folder the script ran from.
# Use double backslashes in path to avoid escaping characters. Example: "C:\\Users\\<username>\\Downloads\\"
folder = "."
# A list of iTunes language codes to download.
# An empty array (like the one currently being used) will result in downloading all of the available subtitles.
# Example: ["en-US", "fr-FR", "he"]
languages = []
# Whether to overwrite existing subtitles files.
# If set to false, names of existing subtitles will have a number appended to them to avoid overwriting.
# Value can be either 'true' or 'false'.
overwrite-existing = false
# Save files into a zip archive if there is more than one matching subtitles.
# Value can be either 'true' or 'false'.
zip = false
[subtitles]
# Fix RTL for RTL languages (Arabic & Hebrew).
# Value can be either 'true' or 'false'.
#
# NOTE: This is off by default as some subtitles use other methods to fix RTL (like writing punctuations backwards).
# Using this option on these type of subtitles can break the already-fixed RTL issues.
fix-rtl = false
# Remove duplicate paragraphs (same text and timestamps).
# Value can be either 'true' or 'false'.
remove-duplicates = true
# Whether to convert subtitles to SRT format.
# NOTE: This can cause loss of subtitles metadata that is not supported by SRT format.
convert-to-srt = false
[subtitles.webvtt]
# Whether to add a '{\an8}' tag to lines that are aligned at the top when converting format from WebVTT to SubRip.
# Relevant only if 'subtitles.convert-to-srt' is set to 'true'.
# Value can be either 'true' or 'false'.
subrip-alignment-conversion = false
[scrapers.default]
# A subcategory to set default values for all scrapers.
# These settings will be overridden by scraper-specific configuration, if set,
# These settings will not apply if the scraper has a different specific default value.
# Timeout in seconds for requests sent by all scrapers.
timeout = 10
# User-Agent to use by default for requests sent by all scrapers.
user-agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"
# Proxy to use by default for requests sent by all scrapers.
proxy = "http://127.0.0.1:8080"
# Whether to verify SSL certificates when making requests for all scrapers.
# Value can be either 'true' or 'false'.
verify-ssl = true
[scrapers.scraper-name]
# Scraper-specific settings (set for each scraper separately).
# Will override any default values previously set.
# Replace 'scraper-name' with the name of the scraper to configure.
# Available scrapers: itunes, appletv
# Timeout in seconds for requests sent by the scraper.
timeout = 10
# User-Agent to use for requests sent by the scraper.
user-agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"
# Proxy to use for requests sent by the scraper.
proxy = "http://127.0.0.1:8080"
# Whether to verify SSL certificates when making requests for the scraper.
# Value can be either 'true' or 'false'.
verify-ssl = true
================================================
FILE: isubrip/__init__.py
================================================
================================================
FILE: isubrip/__main__.py
================================================
from __future__ import annotations
import asyncio
import logging
import sys
from typing import TYPE_CHECKING
import httpx
from pydantic import ValidationError
from isubrip.cli import console
from isubrip.commands.download import download
from isubrip.config import Config
from isubrip.constants import (
PACKAGE_NAME,
PACKAGE_VERSION,
data_folder_path,
log_files_path,
user_config_file_path,
)
from isubrip.logger import logger, setup_loggers
from isubrip.scrapers.scraper import Scraper, ScraperFactory
from isubrip.subtitle_formats.webvtt import WebVTTCaptionBlock
from isubrip.utils import (
convert_log_level,
format_config_validation_error,
get_model_field,
raise_for_status,
single_string_to_list,
)
if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib
if TYPE_CHECKING:
from pathlib import Path
log_rotation_size: int = 15 # Default size, before being updated by the config file.
def main() -> None:
"""A wrapper for the actual main function that handles exceptions and cleanup."""
try:
asyncio.run(_main())
except Exception as ex:
logger.error(f"Error: {ex}")
logger.debug("Debug information:", exc_info=True)
exit(1)
except KeyboardInterrupt:
logger.debug("Keyboard interrupt detected, exiting...")
exit(0)
finally:
if log_rotation_size > 0:
handle_log_rotation(rotation_size=log_rotation_size)
for scraper in ScraperFactory.get_initialized_scrapers():
logger.debug(f"Requests count for '{scraper.name}' scraper: {scraper.requests_count}")
async def _main() -> None:
# Assure at least one argument was passed
if len(sys.argv) < 2:
logger.info(f"Usage: {PACKAGE_NAME} <iTunes movie URL> [iTunes movie URL...]")
exit(0)
# Generate the data folder if it doesn't previously exist
if not data_folder_path().is_dir():
data_folder_path().mkdir(parents=True)
# If config file exists, parse it. Otherwise, create a config with default values
if user_config_file_path().is_file():
config = parse_config(config_file_location=user_config_file_path())
else:
config = Config()
setup_loggers(
stdout_loglevel=convert_log_level(log_level=config.general.log_level),
stdout_console=console,
logfile_output=True,
logfile_output_path=log_files_path(),
logfile_loglevel=logging.DEBUG,
)
cli_args = " ".join(sys.argv[1:])
logger.debug(f"CLI Command: {PACKAGE_NAME} {cli_args}")
logger.debug(f"Python version: {sys.version}")
logger.debug(f"Package version: {PACKAGE_VERSION}")
logger.debug(f"OS: {sys.platform}")
update_settings(config=config)
if config.general.check_for_updates:
check_for_updates(current_package_version=PACKAGE_VERSION)
try:
await download(
*single_string_to_list(item=sys.argv[1:]),
download_path=config.downloads.folder,
language_filter=config.downloads.languages,
convert_to_srt=config.subtitles.convert_to_srt,
overwrite_existing=config.downloads.overwrite_existing,
zip=config.downloads.zip,
)
finally:
async_cleanup_coroutines = []
for scraper in ScraperFactory.get_initialized_scrapers():
async_cleanup_coroutines.append(scraper.async_close())
if async_cleanup_coroutines:
try:
await asyncio.gather(*async_cleanup_coroutines)
except Exception as e:
logger.warning(f"Error during async cleanup: {e}")
logger.debug("Cleanup debug info:", exc_info=True)
def check_for_updates(current_package_version: str) -> None:
"""
Check and print if a newer version of the package is available, and log accordingly.
Args:
current_package_version (str): The current version of the package.
"""
api_url = f"https://pypi.org/pypi/{PACKAGE_NAME}/json"
logger.debug("Checking for package updates on PyPI...")
try:
response = httpx.get(
url=api_url,
headers={"Accept": "application/json"},
timeout=5,
)
raise_for_status(response)
response_data = response.json()
pypi_latest_version = response_data["info"]["version"]
if pypi_latest_version != current_package_version:
logger.warning(f"You are currently using version '{current_package_version}' of '{PACKAGE_NAME}', "
f"however version '{pypi_latest_version}' is available."
f'\nConsider upgrading by running "pip install --upgrade {PACKAGE_NAME}"')
else:
logger.debug(f"Latest version of '{PACKAGE_NAME}' ({current_package_version}) is currently installed.")
except Exception as e:
logger.warning(f"Update check failed: {e}")
logger.debug("Debug information:", exc_info=True)
return
def handle_log_rotation(rotation_size: int) -> None:
"""
Handle log rotation and remove old log files if needed.
Args:
rotation_size (int): Maximum amount of log files to keep.
"""
sorted_log_files = sorted(log_files_path().glob("*.log"), key=lambda file: file.stat().st_mtime, reverse=True)
if len(sorted_log_files) > rotation_size:
for log_file in sorted_log_files[rotation_size:]:
log_file.unlink()
def parse_config(config_file_location: Path) -> Config:
"""
Parse the configuration file and return a Config instance.
Exit the program (with code 1) if an error occurs while parsing the configuration file.
Args:
config_file_location (Path): The location of the configuration file.
Returns:
Config: An instance of the Config.
"""
try:
with config_file_location.open('rb') as file:
config_data = tomllib.load(file)
return Config.model_validate(config_data)
except ValidationError as e:
logger.error("Invalid configuration - the following errors were found in the configuration file:\n" +
format_config_validation_error(exc=e) +
"\nPlease update your configuration to resolve this issue.")
logger.debug("Debug information:", exc_info=True)
exit(1)
except tomllib.TOMLDecodeError as e:
logger.error(f"Error parsing config file: {e}")
logger.debug("Debug information:", exc_info=True)
exit(1)
except Exception as e:
logger.error(f"Error loading configuration: {e}")
logger.debug("Debug information:", exc_info=True)
exit(1)
def update_settings(config: Config) -> None:
"""
Update settings according to config.
Args:
config (Config): An instance of a config to set settings according to.
"""
if config.general.log_level.casefold() == "debug":
console.is_interactive = False
Scraper.subtitles_fix_rtl = config.subtitles.fix_rtl
Scraper.subtitles_remove_duplicates = config.subtitles.remove_duplicates
Scraper.default_timeout = config.scrapers.default.timeout
Scraper.default_user_agent = config.scrapers.default.user_agent
Scraper.default_proxy = config.scrapers.default.proxy
Scraper.default_verify_ssl = config.scrapers.default.verify_ssl
for scraper in ScraperFactory.get_scraper_classes():
if scraper_config := get_model_field(model=config.scrapers, field=scraper.id):
scraper.config = scraper_config
WebVTTCaptionBlock.subrip_alignment_conversion = (
config.subtitles.webvtt.subrip_alignment_conversion
)
if config.general.log_rotation_size:
global log_rotation_size
log_rotation_size = config.general.log_rotation_size
if __name__ == "__main__":
main()
================================================
FILE: isubrip/cli.py
================================================
from collections.abc import Iterator
from contextlib import contextmanager
from typing import Any
from rich.console import Console
from rich.live import Live
console = Console(
highlight=False,
)
@contextmanager
def conditional_live(renderable: Any) -> Iterator[Live | None]:
"""
A context manager that conditionally enables Rich's Live display based on console interactivity.
When console.is_interactive is True, this behaves like Rich's Live display.
When console.is_interactive is False, live updates are disabled.
Args:
renderable: The Rich renderable object to display in live mode.
Yields:
Optional[Live]: The Live display object if console is interactive, None otherwise.
Example:
```python
with conditional_live(progress) as live:
# Your code here
if live: # Optional: Check if live display is active
live.update(...)
```
"""
if console.is_interactive:
with Live(renderable, console=console) as live:
yield live
else:
yield None
================================================
FILE: isubrip/commands/__init__.py
================================================
================================================
FILE: isubrip/commands/download.py
================================================
from __future__ import annotations
from pathlib import Path
import shutil
from rich.console import Group
from rich.progress import BarColumn, Progress, SpinnerColumn, TextColumn
from rich.text import Text
from isubrip.cli import conditional_live, console
from isubrip.data_structures import (
Episode,
MediaData,
Movie,
ScrapedMediaResponse,
Season,
Series,
SubtitlesData,
SubtitlesDownloadResults,
)
from isubrip.logger import logger
from isubrip.scrapers.scraper import PlaylistLoadError, Scraper, ScraperError, ScraperFactory, SubtitlesDownloadError
from isubrip.ui import MinsAndSecsTimeElapsedColumn
from isubrip.utils import (
TemporaryDirectory,
download_subtitles_to_file,
format_list,
format_media_description,
format_release_name,
generate_non_conflicting_path,
)
async def download(*urls: str,
download_path: Path,
language_filter: list[str] | None = None,
convert_to_srt: bool = False,
overwrite_existing: bool = True,
zip: bool = False) -> None:
"""
Download subtitles from given URLs.
Args:
urls (list[str]): A list of URLs to download subtitles from.
download_path (Path): Path to a folder where the subtitles will be downloaded to.
language_filter (list[str] | None): List of specific languages to download. None for all languages (no filter).
Defaults to None.
convert_to_srt (bool, optional): Whether to convert the subtitles to SRT format. Defaults to False.
overwrite_existing (bool, optional): Whether to overwrite existing subtitles. Defaults to True.
zip (bool, optional): Whether to zip multiple subtitles. Defaults to False.
"""
for url in urls:
try:
logger.info(f"Scraping [blue]{url}[/blue]")
scraper = ScraperFactory.get_scraper_instance(url=url)
try:
logger.debug(f"Fetching {url}")
scraper_response: ScrapedMediaResponse = await scraper.get_data(url=url)
except ScraperError as e:
logger.error(f"Error: {e}")
logger.debug("Debug information:", exc_info=True)
continue
media_data = scraper_response.media_data
playlist_scraper = ScraperFactory.get_scraper_instance(scraper_id=scraper_response.playlist_scraper)
if not media_data:
logger.error(f"Error: No supported media was found for {url}.")
continue
for media_item in media_data:
try:
logger.info(f"Found {media_item.media_type}: "
f"[cyan]{format_media_description(media_data=media_item)}[/cyan]")
await download_media(scraper=playlist_scraper,
media_item=media_item,
download_path=download_path,
language_filter=language_filter,
convert_to_srt=convert_to_srt,
overwrite_existing=overwrite_existing,
zip=zip)
except Exception as e:
if len(media_data) > 1:
logger.warning(f"Error scraping media item "
f"'{format_media_description(media_data=media_item)}': {e}\n"
f"Skipping to next media item...")
logger.debug("Debug information:", exc_info=True)
continue
raise
except Exception as e:
logger.error(f"Error while scraping '{url}': {e}")
logger.debug("Debug information:", exc_info=True)
continue
async def download_media(scraper: Scraper, media_item: MediaData, download_path: Path,
language_filter: list[str] | None = None, convert_to_srt: bool = False,
overwrite_existing: bool = True, zip: bool = False) -> None:
"""
Download a media item.
Args:
scraper (Scraper): A Scraper object to use for downloading subtitles.
media_item (MediaData): A media data item to download subtitles for.
download_path (Path): Path to a folder where the subtitles will be downloaded to.
language_filter (list[str] | None): List of specific languages to download. None for all languages (no filter).
Defaults to None.
convert_to_srt (bool, optional): Whether to convert the subtitles to SRT format. Defaults to False.
overwrite_existing (bool, optional): Whether to overwrite existing subtitles. Defaults to True.
zip (bool, optional): Whether to zip multiple subtitles. Defaults to False.
"""
if isinstance(media_item, Series):
for season in media_item.seasons:
await download_media(media_item=season, scraper=scraper, download_path=download_path,
language_filter=language_filter, convert_to_srt=convert_to_srt,
overwrite_existing=overwrite_existing, zip=zip)
elif isinstance(media_item, Season):
for episode in media_item.episodes:
logger.info(f"{format_media_description(media_data=episode, shortened=True)}:")
await download_media_item(media_item=episode, scraper=scraper, download_path=download_path,
language_filter=language_filter, convert_to_srt=convert_to_srt,
overwrite_existing=overwrite_existing, zip=zip)
elif isinstance(media_item, (Movie | Episode)):
await download_media_item(media_item=media_item, scraper=scraper, download_path=download_path,
language_filter=language_filter, convert_to_srt=convert_to_srt,
overwrite_existing=overwrite_existing, zip=zip)
async def download_media_item(scraper: Scraper, media_item: Movie | Episode, download_path: Path,
language_filter: list[str] | None = None, convert_to_srt: bool = False,
overwrite_existing: bool = True, zip: bool = False) -> None:
"""
Download subtitles for a single media item.
Args:
scraper (Scraper): A Scraper object to use for downloading subtitles.
media_item (Movie | Episode): A movie or episode data object.
download_path (Path): Path to a folder where the subtitles will be downloaded to.
language_filter (list[str] | None): List of specific languages to download. None for all languages (no filter).
Defaults to None.
convert_to_srt (bool, optional): Whether to convert the subtitles to SRT format. Defaults to False.
overwrite_existing (bool, optional): Whether to overwrite existing subtitles. Defaults to True.
zip (bool, optional): Whether to zip multiple subtitles. Defaults to False.
"""
ex: Exception | None = None
if media_item.playlist:
try:
results = await download_subtitles(
scraper=scraper,
media_data=media_item,
download_path=download_path,
language_filter=language_filter,
convert_to_srt=convert_to_srt,
overwrite_existing=overwrite_existing,
zip=zip,
)
success_count = len(results.successful_subtitles)
failed_count = len(results.failed_subtitles)
if success_count or failed_count:
logger.info(f"{success_count}/{success_count + failed_count} subtitles were successfully downloaded.")
else:
logger.info("No matching subtitles were found.")
return # noqa: TRY300
except PlaylistLoadError as e:
ex = e
# We get here if there is no playlist, or there is one, but it failed to load
if isinstance(media_item, Movie) and media_item.preorder_availability_date:
logger.info(f"[gold1]'{media_item.name}' is currently unavailable on {scraper.name}, "
f"and will be available on {media_item.preorder_availability_date.strftime(r'%B %e, %Y')}.[/gold1]")
else:
if ex:
logger.error(f"Error: {ex}")
else:
logger.error("Error: No valid playlist was found.")
async def download_subtitles(scraper: Scraper, media_data: Movie | Episode, download_path: Path,
language_filter: list[str] | None = None, convert_to_srt: bool = False,
overwrite_existing: bool = True, zip: bool = False) -> SubtitlesDownloadResults:
"""
Download subtitles for the given media data.
Args:
scraper (Scraper): A Scraper object to use for downloading subtitles.
media_data (Movie | Episode): A movie or episode data object.
download_path (Path): Path to a folder where the subtitles will be downloaded to.
language_filter (list[str] | None): List of specific languages to download. None for all languages (no filter).
Defaults to None.
convert_to_srt (bool, optional): Whether to convert the subtitles to SRT format. Defaults to False.
overwrite_existing (bool, optional): Whether to overwrite existing subtitles. Defaults to True.
zip (bool, optional): Whether to zip multiple subtitles. Defaults to False.
Returns:
SubtitlesDownloadResults: A SubtitlesDownloadResults object containing the results of the download.
"""
temp_dir_name = format_release_name(
title=media_data.name if isinstance(media_data, Movie) else media_data.series_name,
release_date=media_data.release_date,
season_number=None if isinstance(media_data, Movie) else media_data.season_number,
episode_number=None if isinstance(media_data, Movie) else media_data.episode_number,
episode_name=None if isinstance(media_data, Movie) else media_data.episode_name,
media_source=scraper.abbreviation,
)
successful_downloads: list[SubtitlesData] = []
failed_downloads: list[SubtitlesDownloadError] = []
with TemporaryDirectory(directory_name=temp_dir_name) as temp_download_path:
temp_downloads: list[Path] = []
if not media_data.playlist:
raise PlaylistLoadError("No playlist was found for provided media data.")
main_playlist = await scraper.load_playlist(url=media_data.playlist) # type: ignore[func-returns-value]
if not main_playlist:
raise PlaylistLoadError("Failed to load the main playlist.")
matching_subtitles = scraper.find_matching_subtitles(main_playlist=main_playlist, # type: ignore[var-annotated]
language_filter=language_filter)
# If no matching subtitles were found, there's no need to continue
if not matching_subtitles:
return SubtitlesDownloadResults(
media_data=media_data,
successful_subtitles=successful_downloads,
failed_subtitles=failed_downloads,
is_zip=zip,
)
logger.info(f"{len(matching_subtitles)} matching subtitles were found.", extra={"hide_when_interactive": True})
downloaded_subtitles: list[str] = []
progress_log = Text(f"Downloaded subtitles ({len(downloaded_subtitles)}/{len(matching_subtitles)}):")
downloads_list = Text()
progress_bar = Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
TextColumn("[progress.percentage][yellow]{task.percentage:>3.0f}%[/yellow]"),
TextColumn("[yellow]{task.completed}/{task.total}[/yellow]"),
MinsAndSecsTimeElapsedColumn(),
console=console,
)
task = progress_bar.add_task("Starting download", total=len(matching_subtitles))
with conditional_live(
Group(progress_log, downloads_list, Text(), progress_bar), # Empty 'Text' for line spacing
) as live:
for matching_subtitles_item in matching_subtitles:
language_info = scraper.format_subtitles_description(
subtitles_media=matching_subtitles_item,
)
if live:
progress_bar.update(task, advance=1, description=f"Processing [magenta]{language_info}[/magenta]")
try:
subtitles_data = await scraper.download_subtitles(media_data=matching_subtitles_item,
subrip_conversion=convert_to_srt)
except Exception as e:
if isinstance(e, SubtitlesDownloadError):
failed_downloads.append(e)
original_error = e.original_exc
else:
original_error = e
logger.warning(f"Failed to download '{language_info}' subtitles: {original_error}")
logger.debug("Debug information:", exc_info=original_error)
continue
try:
temp_downloads.append(download_subtitles_to_file(
media_data=media_data,
subtitles_data=subtitles_data,
output_path=temp_download_path,
source_abbreviation=scraper.abbreviation,
overwrite=overwrite_existing,
))
downloaded_subtitles.append(f"• {language_info}")
if live:
progress_log.plain = (f"Downloaded subtitles "
f"({len(downloaded_subtitles)}/{len(matching_subtitles)}):")
downloads_list.plain = f"{format_list(downloaded_subtitles, width=live.console.width)}"
logger.info(f"{language_info} subtitles were successfully downloaded.",
extra={"hide_when_interactive": True})
successful_downloads.append(subtitles_data)
except Exception as e:
logger.warning(f"Failed to save '{language_info}' subtitles: {e}")
logger.debug("Debug information:", exc_info=True)
failed_downloads.append(
SubtitlesDownloadError(
language_code=subtitles_data.language_code,
language_name=subtitles_data.language_name,
special_type=subtitles_data.special_type,
original_exc=e,
),
)
if live:
progress_bar.update(task, visible=False)
if not zip or len(temp_downloads) == 1:
for file_path in temp_downloads:
if overwrite_existing:
new_path = download_path / file_path.name
else:
new_path = generate_non_conflicting_path(file_path=download_path / file_path.name)
shutil.move(src=file_path, dst=new_path)
elif len(temp_downloads) > 0:
zip_path = Path(shutil.make_archive(
base_name=str(temp_download_path.parent / temp_download_path.name),
format="zip",
root_dir=temp_download_path,
))
file_name = format_release_name(
title=media_data.name if isinstance(media_data, Movie) else media_data.series_name,
release_date=media_data.release_date,
season_number=None if isinstance(media_data, Movie) else media_data.season_number,
episode_number=None if isinstance(media_data, Movie) else media_data.episode_number,
episode_name=None if isinstance(media_data, Movie) else media_data.episode_name,
media_source=scraper.abbreviation,
file_format="zip",
)
if overwrite_existing:
destination_path = download_path / file_name
else:
destination_path = generate_non_conflicting_path(file_path=download_path / file_name)
shutil.move(src=zip_path, dst=destination_path)
return SubtitlesDownloadResults(
media_data=media_data,
successful_subtitles=successful_downloads,
failed_subtitles=failed_downloads,
is_zip=zip,
)
================================================
FILE: isubrip/config.py
================================================
from __future__ import annotations
from abc import ABC
from pathlib import Path
from typing import TYPE_CHECKING, Literal
from pydantic import AliasGenerator, BaseModel, ConfigDict, Field, create_model, field_validator
from pydantic_core import PydanticCustomError
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict, TomlConfigSettingsSource
from isubrip.scrapers.scraper import DefaultScraperConfig, ScraperFactory
class ConfigCategory(BaseModel, ABC):
"""A base class for settings categories."""
model_config = ConfigDict(
extra='allow',
alias_generator=AliasGenerator(
validation_alias=lambda field_name: field_name.replace('_', '-'),
),
)
class GeneralCategory(ConfigCategory):
check_for_updates: bool = Field(default=True)
verbose: bool = Field(default=False)
log_level: Literal["debug", "info", "warning", "error", "critical"] = Field(default="info")
log_rotation_size: int = Field(default=15)
class DownloadsCategory(ConfigCategory):
folder: Path = Field(default=Path.cwd().resolve())
languages: list[str] = Field(default=[])
overwrite_existing: bool = Field(default=False)
zip: bool = Field(default=False)
@field_validator('folder')
@classmethod
def assure_path_exists(cls, value: Path) -> Path:
if value.exists():
if not value.is_dir():
raise PydanticCustomError(
"invalid_path",
"Path is not a directory.",
)
else:
raise PydanticCustomError(
"invalid_path",
"Path does not exist.")
return value
class WebVTTSubcategory(ConfigCategory):
subrip_alignment_conversion: bool = Field(default=False)
class SubtitlesCategory(ConfigCategory):
fix_rtl: bool = Field(default=False)
remove_duplicates: bool = Field(default=True)
convert_to_srt: bool = Field(default=False)
webvtt: WebVTTSubcategory = WebVTTSubcategory()
class ScrapersCategory(ConfigCategory):
default: DefaultScraperConfig = Field(default_factory=DefaultScraperConfig)
# Resolve mypy errors as mypy doesn't support dynamic models.
if TYPE_CHECKING:
DynamicScrapersCategory = ScrapersCategory
else:
# A config model that's dynamically created based on the available scrapers and their configurations.
DynamicScrapersCategory = create_model(
'DynamicScrapersCategory',
__base__=ScrapersCategory,
**{
scraper.id: (scraper.ScraperConfig, Field(default_factory=scraper.ScraperConfig))
for scraper in ScraperFactory.get_scraper_classes()
}, # type: ignore[call-overload]
)
class Config(BaseSettings):
model_config = SettingsConfigDict(
extra='forbid',
)
general: GeneralCategory = Field(default_factory=GeneralCategory)
downloads: DownloadsCategory = Field(default_factory=DownloadsCategory)
subtitles: SubtitlesCategory = Field(default_factory=SubtitlesCategory)
scrapers: DynamicScrapersCategory = Field(default_factory=DynamicScrapersCategory)
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
TomlConfigSettingsSource(settings_cls),
env_settings,
dotenv_settings,
file_secret_settings,
)
================================================
FILE: isubrip/constants.py
================================================
from __future__ import annotations
from functools import lru_cache
from pathlib import Path
from tempfile import gettempdir
# General
PACKAGE_NAME = "isubrip"
PACKAGE_VERSION = "2.6.8"
SCRAPER_MODULES_SUFFIX = "_scraper"
USER_CONFIG_FILE_NAME = "config.toml"
@lru_cache(maxsize=1)
def data_folder_path() -> Path:
return Path.home() / f".{PACKAGE_NAME}"
@lru_cache(maxsize=1)
def temp_folder_path() -> Path:
return Path(gettempdir()) / PACKAGE_NAME
@lru_cache(maxsize=1)
def user_config_file_path() -> Path:
return data_folder_path() / USER_CONFIG_FILE_NAME
# Logging Paths
@lru_cache(maxsize=1)
def log_files_path() -> Path:
return data_folder_path() / "logs"
# Other
WINDOWS_RESERVED_FILE_NAMES = frozenset(
["CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1",
"LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9"],
)
RTL_LANGUAGES = frozenset(["ar", "arc", "az", "dv", "he", "ks", "ku", "fa", "ur", "yi"])
================================================
FILE: isubrip/data_structures.py
================================================
from __future__ import annotations
from abc import ABC
import datetime as dt # noqa: TC003
from enum import Enum
from typing import TYPE_CHECKING, Generic, Literal, NamedTuple, TypeVar
import m3u8
from pydantic import BaseModel
if TYPE_CHECKING:
from isubrip.scrapers.scraper import SubtitlesDownloadError
T = TypeVar("T")
MainPlaylist = TypeVar("MainPlaylist", bound=m3u8.M3U8)
PlaylistMediaItem = TypeVar("PlaylistMediaItem", bound=m3u8.Media)
MediaData = TypeVar("MediaData", bound="MediaBase")
class SubtitlesDownloadResults(NamedTuple):
"""
A named tuple containing download results.
Attributes:
media_data (Movie | Episode): An object containing metadata about the media the subtitles were downloaded for.
successful_subtitles (list[SubtitlesData]): List of subtitles that were successfully downloaded.
failed_subtitles (list[SubtitlesData]): List of subtitles that failed to download.
is_zip (bool): Whether the subtitles were saved in a zip file.
"""
media_data: Movie | Episode
successful_subtitles: list[SubtitlesData]
failed_subtitles: list[SubtitlesDownloadError]
is_zip: bool
class SubtitlesFormat(BaseModel):
"""
An object containing subtitles format data.
Attributes:
name (str): Name of the format.
file_extension (str): File extension of the format.
"""
name: str
file_extension: str
class SubtitlesFormatType(Enum):
"""
An Enum representing subtitles formats.
Attributes:
SUBRIP (SubtitlesFormat): SubRip format.
WEBVTT (SubtitlesFormat): WebVTT format.
"""
SUBRIP = SubtitlesFormat(name="SubRip", file_extension="srt")
WEBVTT = SubtitlesFormat(name="WebVTT", file_extension="vtt")
class SubtitlesType(Enum):
"""
Subtitles special type.
Attributes:
CC (SubtitlesType): Closed captions.
FORCED (SubtitlesType): Forced subtitles.
"""
CC = "CC"
FORCED = "Forced"
class SubtitlesData(BaseModel):
"""
An object containing subtitles data and metadata.
Attributes:
language_code (str): Language code of the language the subtitles are in.
language_name (str | None, optional): Name of the language the subtitles are in.
subtitles_format (SubtitlesFormatType): Format of the subtitles.
content (bytes): Content of the subtitles in binary format.
content_encoding (str): Encoding of subtitles content (ex. "utf-8").
special_type (SubtitlesType | None, optional): Type of the subtitles, if they're not regular. Defaults to None.
"""
language_code: str
subtitles_format: SubtitlesFormatType
content: bytes
content_encoding: str
language_name: str | None = None
special_type: SubtitlesType | None = None
class ConfigDict:
str_strip_whitespace = True
class MediaBase(BaseModel, ABC):
"""A base class for media objects."""
class Movie(MediaBase):
"""
An object containing movie metadata.
Attributes:
id (str | None, optional): ID of the movie on the service it was scraped from. Defaults to None.
referrer_id (str | None, optional): ID of the movie on the original referring service. Defaults to None.
name (str): Title of the movie.
release_date (datetime | int | None, optional): Release date (datetime), or year (int) of the movie.
Defaults to None.
duration (timedelta | None, optional): Duration of the movie. Defaults to None.
preorder_availability_date (datetime | None, optional):
Date when the movie will be available for pre-order on the service it was scraped from.
None if not a pre-order. Defaults to None.
playlist (str | None, optional): Main playlist URL(s).
"""
media_type: Literal["movie"] = "movie"
name: str
release_date: dt.datetime | int
id: str | None = None
referrer_id: str | None = None
duration: dt.timedelta | None = None
preorder_availability_date: dt.datetime | None = None
playlist: str | list[str] | None = None
class Episode(MediaBase):
"""
An object containing episode metadata.
Attributes:
id (str | None, optional): ID of the episode on the service it was scraped from. Defaults to None.
referrer_id (str | None, optional): ID of the episode on the original referring service. Defaults to None.
series_name (str): Name of the series the episode is from.
series_release_date (datetime | int | None, optional): Release date (datetime), or year (int) of the series.
Defaults to None.
season_number (int): Season number.
season_name (str | None, optional): Season name. Defaults to None.
episode_number (int): Episode number.
episode_name (str | None, optional): Episode name. Defaults to None.
episode_release_date (datetime | None): Release date of the episode. Defaults to None.
episode_duration (timedelta | None, optional): Duration of the episode. Defaults to None.
playlist (str | None, optional): Main playlist URL(s).
"""
media_type: Literal["episode"] = "episode"
series_name: str
season_number: int
episode_number: int
id: str | None = None
referrer_id: str | None = None
series_release_date: dt.datetime | int | None = None
season_name: str | None = None
release_date: dt.datetime | None = None
duration: dt.timedelta | None = None
episode_name: str | None = None
episode_release_date: dt.datetime | None = None
episode_duration: dt.timedelta | None = None
playlist: str | list[str] | None = None
class Season(MediaBase):
"""
An object containing season metadata.
Attributes:
id (str | None, optional): ID of the season on the service it was scraped from. Defaults to None.
referrer_id (str | None, optional): ID of the season on the original referring service. Defaults to None.
series_name (str): Name of the series the season is from.
season_number (int): Season number.
series_release_date (datetime | int | None, optional): Release date (datetime), or year (int) of the series.
Defaults to None.
season_name (str | None, optional): Season name. Defaults to None.
season_release_date (datetime | None, optional): Release date of the season, or release year. Defaults to None.
episodes (list[Episode]): A list of episode objects containing metadata about episodes of the season.
"""
media_type: Literal["season"] = "season"
series_name: str
season_number: int
id: str | None = None
referrer_id: str | None = None
series_release_date: dt.datetime | int | None = None
season_name: str | None = None
season_release_date: dt.datetime | int | None = None
episodes: list[Episode] = []
class Series(MediaBase):
"""
An object containing series metadata.
Attributes:
id (str | None, optional): ID of the series on the service it was scraped from. Defaults to None.
series_name (str): Series name.
referrer_id (str | None, optional): ID of the series on the original referring service. Defaults to None.
series_release_date (datetime | int | None, optional): Release date (datetime), or year (int) of the series.
Defaults to None.
seasons (list[Season]): A list of season objects containing metadata about seasons of the series.
"""
media_type: Literal["series"] = "series"
series_name: str
seasons: list[Season] = []
id: str | None = None
referrer_id: str | None = None
series_release_date: dt.datetime | int | None = None
class ScrapedMediaResponse(BaseModel, Generic[MediaData]):
"""
An object containing scraped media data and metadata.
Attributes:
media_data (list[Movie] | list[Episode] | list[Season] | list[Series]):
An object containing the scraped media data.
metadata_scraper (str): ID of the scraper that was used to scrape metadata.
playlist_scraper (str): ID of the scraper that should be used to parse and scrape the playlist.
original_data (dict): Original raw data from the API that was used to extract media's data.
"""
media_data: list[MediaData]
metadata_scraper: str
playlist_scraper: str
original_data: dict
================================================
FILE: isubrip/logger.py
================================================
from __future__ import annotations
import datetime as dt
from functools import lru_cache
import logging
import re
from typing import TYPE_CHECKING, ClassVar
from rich.highlighter import NullHighlighter
from rich.logging import RichHandler
from isubrip.cli import console
from isubrip.constants import (
PACKAGE_NAME,
)
if TYPE_CHECKING:
from pathlib import Path
from rich.console import Console
BBCOE_REGEX = re.compile(
r"(?i)(?P<opening_tag>\[(?P<tag_name>[a-z#@][^[]*?)])(?P<content>.*)(?P<closing_tag>\[/(?P=tag_name)])")
LOG_FILE_METADATA = "[%(asctime)s | %(levelname)s | %(threadName)s | %(filename)s::%(funcName)s::%(lineno)d] "
def set_logger(_logger: logging.Logger) -> None:
"""
Set an external logger to be used by the package.
Args:
_logger (logging.Logger): A logger instance to be used by the package.
"""
global logger
logger = _logger
class CustomStdoutFormatter(RichHandler):
"""
Custom formatter for stdout logging with Rich integration.
This formatter adds color to log messages based on their level and
supports hiding messages in interactive mode.
"""
LEVEL_COLORS: ClassVar[dict[int, str]] = {
logging.ERROR: "red",
logging.WARNING: "dark_orange",
logging.DEBUG: "grey54",
}
def __init__(self, console: Console | None = None, debug_mode: bool = False) -> None:
"""
Initialize the stdout formatter.
Args:
console (Console | None, optional): Rich console instance to use for output. Defaults to None.
debug_mode (bool, optional): Whether to show additional debug information. Defaults to False.
"""
super().__init__(
console=console,
show_time=debug_mode,
show_level=debug_mode,
show_path=debug_mode,
highlighter=NullHighlighter(),
markup=True,
log_time_format="%H:%M:%S",
rich_tracebacks=debug_mode,
tracebacks_extra_lines=0,
)
self._console = console
def emit(self, record: logging.LogRecord) -> None:
"""
Emit a log record, respecting the 'hide_when_interactive' flag.
Args:
record (LogRecord): The log record to emit.
"""
# Skip emission if record is marked to be hidden in interactive mode
if getattr(record, 'hide_when_interactive', False) and self._console and self._console.is_interactive:
return
super().emit(record)
def format(self, record: logging.LogRecord) -> str:
"""
Format the log record with appropriate color based on level.
Args:
record (LogRecord): The log record to format.
Returns:
str: Formatted log message with Rich markup.
"""
# Get the message once
message = record.getMessage()
# Apply color based on log level using the class variable mapping
if color := self.LEVEL_COLORS.get(record.levelno):
record.msg = f"[{color}]{message}[/{color}]"
return super().format(record)
class CustomLogFileFormatter(logging.Formatter):
"""
Custom formatter for log files that removes Rich markup tags.
"""
def __init__(self) -> None:
"""
Initialize the formatter with metadata format but without message part.
We'll append the message manually to avoid issues with special characters.
"""
super().__init__(
fmt=LOG_FILE_METADATA,
datefmt=r"%Y-%m-%d %H:%M:%S",
)
@staticmethod
@lru_cache(maxsize=64)
def _remove_rich_markup(text: str) -> str:
"""
Remove Rich markup tags from text efficiently with caching.
Args:
text: Text containing Rich markup tags
Returns:
Text with Rich markup tags removed
"""
while match := BBCOE_REGEX.search(text):
text = text[:match.start()] + match.group('content') + text[match.end():]
return text
def format(self, record: logging.LogRecord) -> str:
"""
Format the log record for file output, removing Rich markup.
This implementation uses the standard formatter for the metadata part
and then appends the message without formatting to avoid issues with
special characters within the log message.
Args:
record: The log record to format
Returns:
Formatted log message suitable for file output
"""
message = record.getMessage()
clean_message = self._remove_rich_markup(message)
# Store the original message
original_msg = record.msg
original_args = record.args
# Temporarily set an empty message to format just the metadata
record.msg = ""
record.args = None
# Format the metadata part using the standard formatter
metadata = super().format(record)
# Restore the original message and args
record.msg = original_msg
record.args = original_args
# Combine metadata and message without formatting the message
return metadata + clean_message
def setup_loggers(stdout_output: bool = True, stdout_console: Console | None = None,
stdout_loglevel: int = logging.INFO, logfile_output: bool = False,
logfile_output_path: Path | None = None, logfile_loglevel: int = logging.DEBUG) -> None:
"""
Configure loggers for both stdout and file output.
Args:
stdout_output (bool, optional): Whether to output logs to STDOUT. Defaults to True.
stdout_console (Console | None, optional): A Rich console instance to be used for STDOUT logging.
Relevant only if `stdout_output` is True. Defaults to None.
stdout_loglevel (int, optional): Log level for STDOUT logger. Relevant only if `stdout_output` is True.
Defaults to logging.INFO.
logfile_output (bool, optional): Whether to output logs to a logfile. Defaults to True.
logfile_output_path (Path | None, optional): Path to the directory where log files will be saved.
Required only if `logfile_output` is True. Defaults to None.
logfile_loglevel (int, optional): Log level for logfile logger. Relevant only if `logfile_output` is True.
Defaults to logging.DEBUG.
"""
logger.handlers.clear() # Remove and reset existing handlers
logger.setLevel(logging.DEBUG)
if stdout_output:
debug_mode = (stdout_loglevel == logging.DEBUG)
stdout_handler = CustomStdoutFormatter(
debug_mode=debug_mode,
console=stdout_console,
)
stdout_handler.setLevel(stdout_loglevel)
logger.addHandler(stdout_handler)
if logfile_output:
if not logfile_output_path:
raise ValueError("Missing required 'logfile_output_path' argument (required when 'logfile_output' is True.")
if not logfile_output_path.is_dir():
logger.debug("Logs directory could not be found and will be created.")
logfile_output_path.mkdir()
logfile_path = logfile_output_path / f"{PACKAGE_NAME}_{dt.datetime.now().strftime(r'%Y-%m-%d_%H-%M-%S')}.log"
logfile_handler = logging.FileHandler(filename=logfile_path, encoding="utf-8")
logfile_handler.setLevel(logfile_loglevel)
logfile_handler.setFormatter(CustomLogFileFormatter())
logger.debug(f"Log file location: '{logfile_path}'")
logger.addHandler(logfile_handler)
logger = logging.getLogger(PACKAGE_NAME)
# Temporarily set the logger to INFO level until the config is loaded and the logger is properly set up
logger.setLevel(logging.INFO)
logger.addHandler(CustomStdoutFormatter(console=console))
================================================
FILE: isubrip/scrapers/__init__.py
================================================
================================================
FILE: isubrip/scrapers/appletv_scraper.py
================================================
from __future__ import annotations
import datetime as dt
from enum import Enum
import fnmatch
import re
from typing import Any
from httpx import HTTPError
from isubrip.data_structures import Episode, Movie, ScrapedMediaResponse, Season, Series
from isubrip.logger import logger
from isubrip.scrapers.scraper import HLSScraper, ScraperError
from isubrip.subtitle_formats.webvtt import WebVTTSubtitles
from isubrip.utils import convert_epoch_to_datetime, parse_url_params, raise_for_status
class AppleTVScraper(HLSScraper):
"""An Apple TV scraper."""
id = "appletv"
name = "Apple TV"
abbreviation = "ATV"
url_regex = re.compile(r"(?i)(?P<base_url>https?://tv\.apple\.com/(?:(?P<country_code>[a-z]{2})/)?(?P<media_type>movie|episode|season|show)/(?:(?P<media_name>[\w\-%]+)/)?(?P<media_id>umc\.cmc\.[a-z\d]{23,25}))(?:\?(?P<url_params>.*))?")
subtitles_class = WebVTTSubtitles
is_movie_scraper = True
is_series_scraper = True
uses_scrapers = ["itunes"]
default_storefront = "US"
storefronts_mapping = {
"AE": "143481", "AG": "143540", "AI": "143538", "AM": "143524", "AR": "143505", "AT": "143445", "AU": "143460",
"AZ": "143568", "BE": "143446", "BG": "143526", "BH": "143559", "BM": "143542", "BN": "143560", "BO": "143556",
"BR": "143503", "BS": "143539", "BW": "143525", "BY": "143565", "BZ": "143555", "CA": "143455", "CH": "143459",
"CL": "143483", "CO": "143501", "CR": "143495", "CV": "143580", "CY": "143557", "CZ": "143489", "DE": "143443",
"DK": "143458", "DM": "143545", "DO": "143508", "EC": "143509", "EE": "143518", "EG": "143516", "ES": "143454",
"FI": "143447", "FJ": "143583", "FM": "143591", "FR": "143442", "GB": "143444", "GD": "143546", "GH": "143573",
"GM": "143584", "GR": "143448", "GT": "143504", "GW": "143585", "HK": "143463", "HN": "143510", "HU": "143482",
"ID": "143476", "IE": "143449", "IL": "143491", "IN": "143467", "IT": "143450", "JO": "143528", "JP": "143462",
"KH": "143579", "KN": "143548", "KR": "143466", "KY": "143544", "LA": "143587", "LB": "143497", "LK": "143486",
"LT": "143520", "LU": "143451", "LV": "143519", "MD": "143523", "MN": "143592", "MO": "143515", "MT": "143521",
"MU": "143533", "MX": "143468", "MY": "143473", "MZ": "143593", "NA": "143594", "NE": "143534", "NI": "143512",
"NL": "143452", "NO": "143457", "NZ": "143461", "OM": "143562", "PA": "143485", "PE": "143507", "PH": "143474",
"PL": "143478", "PT": "143453", "PY": "143513", "QA": "143498", "RU": "143469", "SA": "143479", "SE": "143456",
"SG": "143464", "SI": "143499", "SK": "143496", "SV": "143506", "SZ": "143602", "TH": "143475", "TJ": "143603",
"TM": "143604", "TR": "143480", "TT": "143551", "TW": "143470", "UA": "143492", "UG": "143537", "US": "143441",
"VE": "143502", "VG": "143543", "VN": "143471", "ZA": "143472", "ZW": "143605",
}
_api_base_url = "https://tv.apple.com/api/uts/v3"
_api_base_params = {
"utscf": "OjAAAAAAAAA~",
"caller": "web",
"v": "84",
"pfm": "web",
}
class Channel(Enum):
"""
An Enum representing AppleTV channels.
Value represents the channel ID as used by the API.
"""
APPLE_TV_PLUS = "tvs.sbd.4000"
DISNEY_PLUS = "tvs.sbd.1000216"
ITUNES = "tvs.sbd.9001"
HULU = "tvs.sbd.10000"
MAX = "tvs.sbd.9050"
NETFLIX = "tvs.sbd.9000"
PRIME_VIDEO = "tvs.sbd.12962"
STARZ = "tvs.sbd.1000308"
def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self._storefronts_request_params_cache: dict[str, dict[str, str]] = {}
def _decide_locale(self, preferred_locales: str | list[str], default_locale: str, locales: list[str]) -> str:
"""
Decide which locale to use.
Args:
preferred_locales (str | list[str]): The preferred locales to use.
default_locale (str): The default locale to use if there is no match.
locales (list[str]): The locales to search in.
Returns:
str: The locale to use.
"""
if isinstance(preferred_locales, str):
preferred_locales = [preferred_locales]
for locale in preferred_locales:
if locale in locales:
return locale.replace("_", "-")
if result := fnmatch.filter(locales, "en_*"):
return result[0].replace("_", "-")
return default_locale
async def _fetch_api_data(self, storefront_id: str, endpoint: str, additional_params: dict | None = None) -> dict:
"""
Send a request to AppleTV's API and return the JSON response.
Args:
endpoint (str): The endpoint to send the request to.
additional_params (dict[str, str]): Additional parameters to send with the request.
Returns:
dict: The JSON response.
Raises:
HttpError: If an HTTP error response is received.
"""
request_params = await self._fetch_request_params(storefront_id=storefront_id)
if additional_params:
request_params.update(additional_params)
response = await self._client.get(url=f"{self._api_base_url}{endpoint}", params=request_params)
try:
raise_for_status(response)
except HTTPError as e:
if response.status_code == 404:
raise ScraperError(
"Media not found. This could indicate that the provided URL is invalid.",
) from e
raise
response_json: dict = response.json()
response_data: dict = response_json.get("data", {})
return response_data
async def _fetch_request_params(self, storefront_id: str) -> dict[str, str]:
"""
Fetch from the API request parameters for the given storefront ID.
Uses caching with `self._storefronts_request_params_cache` for efficiency.
Args:
storefront_id (str): The ID of the storefront to fetch the request parameters for.
Returns:
dict: The request parameters for the given storefront ID. If returned from cache, a copy is returned.
"""
if storefront_cached_params := self._storefronts_request_params_cache.get(storefront_id):
logger.debug(f"Using cached request parameters for storefront '{storefront_id}':"
f"'{storefront_cached_params}'.")
return storefront_cached_params.copy()
configuration_data = await self._get_configuration_data(storefront_id=storefront_id)
request_params: dict[str, str] = configuration_data["applicationProps"]["requiredParamsMap"]["Default"]
default_locale: str = configuration_data["applicationProps"]["storefront"]["defaultLocale"]
available_locales: list[str] = configuration_data["applicationProps"]["storefront"]["localesSupported"]
logger.debug(f"Available locales for storefront '{storefront_id}': {available_locales}'. "
f"Storefront's default locale: '{default_locale}'.")
locale = self._decide_locale(
preferred_locales=["en_US", "en_GB"],
default_locale=default_locale,
locales=available_locales,
)
request_params["sf"] = storefront_id
request_params["locale"] = locale
logger.debug(f"Using and caching request parameters for storefront '{storefront_id}': {request_params}")
self._storefronts_request_params_cache[storefront_id] = request_params.copy()
return request_params
async def _get_configuration_data(self, storefront_id: str) -> dict:
"""
Get configuration data for the given storefront ID.
Args:
storefront_id (str): The ID of the storefront to get the configuration data for.
Returns:
dict: Configuration data as returned by the API for the given storefront ID.
"""
logger.debug(f"Fetching configuration data for storefront '{storefront_id}'...")
url = f"{self._api_base_url}/configurations"
params = self._api_base_params.copy()
params["sf"] = storefront_id
response = await self._client.get(url=url, params=params)
raise_for_status(response)
logger.debug("Configuration data fetched successfully.")
response_data: dict = response.json()["data"]
return response_data
def _map_playables_by_channel(self, playables: list[dict]) -> dict[str, dict]:
"""
Map playables by channel name.
Args:
playables (list[dict]): Playables data to map.
Returns:
dict: The mapped playables (in a `channel_name (str): [playables]` format).
"""
mapped_playables: dict = {}
for playable in playables:
if channel_id := playable.get("channelId"):
mapped_playables.setdefault(channel_id, []).append(playable)
return mapped_playables
async def get_movie_data(self, storefront_id: str, movie_id: str) -> ScrapedMediaResponse[Movie]:
data = await self._fetch_api_data(
storefront_id=storefront_id,
endpoint=f"/movies/{movie_id}",
)
mapped_playables = self._map_playables_by_channel(playables=data["playables"].values())
logger.debug(f"Available channels for movie '{movie_id}': "
f"{' '.join(list(mapped_playables.keys()))}")
if self.Channel.ITUNES.value not in mapped_playables:
if self.Channel.APPLE_TV_PLUS.value in mapped_playables:
raise ScraperError("Scraping AppleTV+ content is not currently supported.")
raise ScraperError("No iTunes playables could be found.")
return_data = []
for playable_data in mapped_playables[self.Channel.ITUNES.value]:
return_data.append(self._extract_itunes_movie_data(playable_data))
if len(return_data) > 1:
logger.debug(f"{len(return_data)} iTunes playables were found for movie '{movie_id}'.")
return ScrapedMediaResponse(
media_data=return_data,
metadata_scraper=self.id,
playlist_scraper="itunes",
original_data=data,
)
def _extract_itunes_movie_data(self, playable_data: dict) -> Movie:
"""
Extract movie data from an AppleTV's API iTunes playable data.
Args:
playable_data (dict): The playable data from the AppleTV API.
Returns:
Movie: A Movie object.
"""
itunes_movie_id = playable_data["itunesMediaApiData"]["id"]
appletv_movie_id = playable_data["canonicalId"]
movie_title = playable_data["canonicalMetadata"]["movieTitle"]
movie_release_date = convert_epoch_to_datetime(playable_data["canonicalMetadata"]["releaseDate"] // 1000)
movie_playlists = []
movie_duration = None
if offers := playable_data["itunesMediaApiData"].get("offers"):
for offer in offers:
if (playlist := offer.get("hlsUrl")) and offer["hlsUrl"] not in movie_playlists:
movie_playlists.append(playlist)
if movie_duration_int := offers[0].get("durationInMilliseconds"):
movie_duration = dt.timedelta(milliseconds=movie_duration_int)
if movie_expected_release_date := playable_data["itunesMediaApiData"].get("futureRentalAvailabilityDate"):
movie_expected_release_date = dt.datetime.strptime(movie_expected_release_date, "%Y-%m-%d")
return Movie(
id=itunes_movie_id,
referrer_id=appletv_movie_id,
name=movie_title,
release_date=movie_release_date,
duration=movie_duration,
preorder_availability_date=movie_expected_release_date,
playlist=movie_playlists if movie_playlists else None,
)
async def get_episode_data(self, storefront_id: str, episode_id: str) -> ScrapedMediaResponse[Episode]:
raise NotImplementedError("Series scraping is not currently supported.")
async def get_season_data(self, storefront_id: str, season_id: str, show_id: str) -> ScrapedMediaResponse[Season]:
raise NotImplementedError("Series scraping is not currently supported.")
async def get_show_data(self, storefront_id: str, show_id: str) -> ScrapedMediaResponse[Series]:
raise NotImplementedError("Series scraping is not currently supported.")
async def get_data(self, url: str) -> ScrapedMediaResponse:
regex_match = self.match_url(url=url, raise_error=True)
url_data = regex_match.groupdict()
media_type = url_data["media_type"]
if storefront_code := url_data.get("country_code"):
storefront_code = storefront_code.upper()
else:
storefront_code = self.default_storefront
media_id = url_data["media_id"]
if storefront_code not in self.storefronts_mapping:
raise ScraperError(f"ID mapping for storefront '{storefront_code}' could not be found.")
storefront_id = self.storefronts_mapping[storefront_code]
if media_type == "movie":
return await self.get_movie_data(storefront_id=storefront_id, movie_id=media_id)
if media_type == "episode":
return await self.get_episode_data(storefront_id=storefront_id, episode_id=media_id)
if media_type == "season":
if (url_params := url_data.get("url_params")) and (show_id := parse_url_params(url_params).get("showId")):
return await self.get_season_data(storefront_id=storefront_id, season_id=media_id, show_id=show_id)
raise ScraperError("Invalid AppleTV URL: Missing 'showId' parameter.")
if media_type == "show":
return await self.get_show_data(storefront_id=storefront_id, show_id=media_id)
raise ScraperError(f"Invalid media type '{media_type}'.")
================================================
FILE: isubrip/scrapers/itunes_scraper.py
================================================
from __future__ import annotations
import asyncio
import re
from typing import TYPE_CHECKING, Any
from isubrip.logger import logger
from isubrip.scrapers.scraper import HLSScraper, ScraperError, ScraperFactory
from isubrip.subtitle_formats.webvtt import WebVTTSubtitles
if TYPE_CHECKING:
from m3u8.model import Media
from isubrip.data_structures import Movie, ScrapedMediaResponse
REDIRECT_MAX_RETRIES = 5
REDIRECT_SLEEP_TIME = 2
class ItunesScraper(HLSScraper):
"""An iTunes movie data scraper."""
id = "itunes"
name = "iTunes"
abbreviation = "iT"
url_regex = re.compile(r"(?i)(?P<base_url>https?://itunes\.apple\.com/(?:(?P<country_code>[a-z]{2})/)?(?P<media_type>movie|tv-show|tv-season|show)/(?:(?P<media_name>[\w\-%]+)/)?(?P<media_id>id\d{9,10}))(?:\?(?P<url_params>.*))?")
subtitles_class = WebVTTSubtitles
is_movie_scraper = True
uses_scrapers = ["appletv"]
_subtitles_filters = {
HLSScraper.M3U8Attribute.GROUP_ID.value: ["subtitles_ak", "subtitles_vod-ak-amt.tv.apple.com"],
**HLSScraper._subtitles_filters, # noqa: SLF001
}
def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self._appletv_scraper = ScraperFactory.get_scraper_instance(
scraper_id="appletv",
raise_error=True,
)
async def get_data(self, url: str) -> ScrapedMediaResponse[Movie]:
"""
Scrape iTunes to find info about a movie, and it's M3U8 main_playlist.
Args:
url (str): An iTunes store movie URL.
Raises:
InvalidURL: `itunes_url` is not a valid iTunes store movie URL.
PageLoadError: HTML page did not load properly.
HTTPError: HTTP request failed.
Returns:
Movie: A Movie (NamedTuple) object with movie's name, and an M3U8 object of the main_playlist
if the main_playlist is found. None otherwise.
"""
regex_match = self.match_url(url, raise_error=True)
url_data = regex_match.groupdict()
country_code: str = url_data["country_code"]
media_id: str = url_data["media_id"]
appletv_redirect_finding_url = f"https://tv.apple.com/{country_code}/movie/{media_id}"
logger.debug("Attempting to fetch redirect location from: " + appletv_redirect_finding_url)
retries = 0
while True:
response = await self._client.get(url=appletv_redirect_finding_url, follow_redirects=False)
if response.status_code != 301 and retries < REDIRECT_MAX_RETRIES:
retries += 1
logger.debug(f"AppleTV redirect URL not found (Response code: {response.status_code}),"
f" retrying... ({retries}/{REDIRECT_MAX_RETRIES})")
await asyncio.sleep(REDIRECT_SLEEP_TIME)
continue
break
redirect_location = response.headers.get("Location")
if response.status_code != 301 or not redirect_location:
raise ScraperError(f"AppleTV redirect URL not found (Response code: {response.status_code}).")
# Add 'https:' if redirect_location starts with '//'
if redirect_location.startswith('//'):
redirect_location = "https:" + redirect_location
logger.debug(f"Redirect URL: {redirect_location}")
if not self._appletv_scraper.match_url(redirect_location):
raise ScraperError("Redirect URL is not a valid AppleTV URL.")
return await self._appletv_scraper.get_data(url=redirect_location)
@staticmethod
def parse_language_name(media_data: Media) -> str | None:
name: str | None = media_data.name
if name:
return name.replace(' (forced)', '').strip()
return None
================================================
FILE: isubrip/scrapers/scraper.py
================================================
from __future__ import annotations
from abc import ABC, abstractmethod
import asyncio
from enum import Enum
import importlib
import inspect
from pathlib import Path
import re
import sys
from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypeVar, overload
import httpx
import m3u8
from pydantic import AliasGenerator, BaseModel, ConfigDict, Field, create_model
from isubrip.constants import PACKAGE_NAME, SCRAPER_MODULES_SUFFIX
from isubrip.data_structures import (
MainPlaylist,
PlaylistMediaItem,
ScrapedMediaResponse,
SubtitlesData,
SubtitlesFormatType,
SubtitlesType,
)
from isubrip.logger import logger
from isubrip.utils import (
SingletonMeta,
format_subtitles_description,
get_model_field,
merge_dict_values,
return_first_valid,
single_string_to_list,
)
if TYPE_CHECKING:
from isubrip.subtitle_formats.subtitles import Subtitles
ScraperT = TypeVar("ScraperT", bound="Scraper")
class ScraperConfigBase(BaseModel, ABC):
"""
A Pydantic BaseModel for base class for scraper's configuration classes.
Also serves for setting default configuration settings for all scrapers.
Attributes:
timeout (int | float): Timeout to use when making requests.
user_agent (st): User agent to use when making requests.
proxy (str | None): Proxy to use when making requests.
verify_ssl (bool): Whether to verify SSL certificates.
"""
model_config = ConfigDict(
extra='forbid',
alias_generator=AliasGenerator(
validation_alias=lambda field_name: field_name.replace('_', '-'),
),
)
timeout: int | float | None = Field(default=None)
user_agent: str | None = Field(default=None)
proxy: str | None = Field(default=None)
verify_ssl: bool | None = Field(default=None)
class DefaultScraperConfig(ScraperConfigBase):
"""
A Pydantic BaseModel for scraper's configuration classes.
Also serves as a default configuration for all scrapers.
Attributes:
timeout (int | float): Timeout to use when making requests.
user_agent (st): User agent to use when making requests.
proxy (str | None): Proxy to use when making requests.
verify_ssl (bool): Whether to verify SSL certificates.
"""
timeout: int | float = Field(default=10)
user_agent: str = Field(
default="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36", # noqa: E501
)
proxy: str | None = Field(default=None)
verify_ssl: bool = Field(default=True)
class ScraperConfigSubcategory(BaseModel, ABC):
"""A Pydantic BaseModel for a scraper's configuration subcategory (which can be set under 'ScraperConfig')."""
model_config = ConfigDict(
extra='forbid',
alias_generator=AliasGenerator(
validation_alias=lambda field_name: field_name.replace('_', '-'),
),
)
class Scraper(ABC, metaclass=SingletonMeta):
"""
A base class for scrapers.
Attributes:
default_user_agent (str): [Class Attribute]
Default user agent to use if no other user agent is specified when making requests.
default_proxy (str | None): [Class Attribute] Default proxy to use when making requests.
default_verify_ssl (bool): [Class Attribute] Whether to verify SSL certificates by default.
subtitles_fix_rtl (bool): [Class Attribute] Whether to fix RTL from downloaded subtitles.
subtitles_remove_duplicates (bool): [Class Attribute]
Whether to remove duplicate lines from downloaded subtitles.
id (str): [Class Attribute] ID of the scraper (must be unique).
name (str): [Class Attribute] Name of the scraper.
abbreviation (str): [Class Attribute] Abbreviation of the scraper.
url_regex (re.Pattern | list[re.Pattern]): [Class Attribute] A RegEx pattern to find URLs matching the service.
subtitles_class (type[Subtitles]): [Class Attribute] Class of the subtitles format returned by the scraper.
is_movie_scraper (bool): [Class Attribute] Whether the scraper is for movies.
is_series_scraper (bool): [Class Attribute] Whether the scraper is for series.
uses_scrapers (list[str]): [Class Attribute] A list of IDs for other scraper classes that this scraper uses.
This assures that the config data for the other scrapers is passed as well.
config (ScraperConfig | None): [Class Attribute] A ScraperConfig instance for the scraper,
containing configurations.
_session (httpx.Client): A synchronous HTTP client session.
_async_session (httpx.AsyncClient): An asynchronous HTTP client session.
Notes:
Each scraper implements its own `ScraperConfig` class (which can be overridden and updated),
inheriting from `ScraperConfigBase`, which sets configurable options for the scraper.
"""
class ScraperConfig(ScraperConfigBase):
"""A class representing scraper's configuration settings.
Can be overridden to create a custom configuration with overridden default values,
and additional settings."""
default_timeout: ClassVar[int | float] = 10
default_user_agent: ClassVar[str] = httpx._client.USER_AGENT # noqa: SLF001
default_proxy: ClassVar[str | None] = None
default_verify_ssl: ClassVar[bool] = True
subtitles_fix_rtl: ClassVar[bool] = False
subtitles_remove_duplicates: ClassVar[bool] = True
id: ClassVar[str]
name: ClassVar[str]
abbreviation: ClassVar[str]
url_regex: ClassVar[re.Pattern | list[re.Pattern]]
subtitles_class: ClassVar[type[Subtitles]]
is_movie_scraper: ClassVar[bool] = False
is_series_scraper: ClassVar[bool] = False
uses_scrapers: ClassVar[list[str]] = []
config: ClassVar[ScraperConfig | None] = None
def __init__(self, timeout: int | float | None = None, user_agent: str | None = None,
proxy: str | None = None, verify_ssl: bool | None = None):
"""
Initialize a Scraper object.
Args:
timeout (int | float | None, optional): A timeout to use when making requests. Defaults to None.
user_agent (str | None, optional): A user agent to use when making requests. Defaults to None.
proxy (str | None, optional): A proxy to use when making requests. Defaults to None.
verify_ssl (bool | None, optional): Whether to verify SSL certificates. Defaults to None.
"""
self._timeout = return_first_valid(timeout,
get_model_field(model=self.config, field='timeout'),
self.default_timeout,
raise_error=True)
self._user_agent = return_first_valid(user_agent,
get_model_field(model=self.config, field='user_agent'),
self.default_user_agent,
raise_error=True)
self._proxy = return_first_valid(proxy,
get_model_field(model=self.config, field='proxy'),
self.default_proxy)
self._verify_ssl = return_first_valid(verify_ssl,
get_model_field(model=self.config, field='verify_ssl'),
self.default_verify_ssl,
raise_error=True)
if self._timeout != self.default_timeout:
logger.debug(f"Initializing '{self.name}' scraper with custom timeout: '{self._timeout}'.")
if self._user_agent != self.default_user_agent:
logger.debug(f"Initializing '{self.name}' scraper with custom user-agent: '{self._user_agent}'.")
if self._proxy != self.default_proxy:
logger.debug(f"Initializing '{self.name}' scraper with proxy: '{self._proxy}'.")
if self._verify_ssl != self.default_verify_ssl:
logger.debug(f"Initializing '{self.name}' scraper with SSL verification set to: '{self._verify_ssl}'.")
self._requests_counter = 0
clients_params: dict[str, Any] = {
"headers": {"User-Agent": self._user_agent},
"verify": self._verify_ssl,
"proxy": self._proxy,
"timeout": float(self._timeout),
}
self._client = httpx.AsyncClient(
**clients_params,
event_hooks={
"request": [self._async_increment_requests_counter],
},
)
# Update session settings according to configurations
self._client.headers.update({"User-Agent": self._user_agent})
def _increment_requests_counter(self, request: httpx.Request) -> None: # noqa: ARG002
self._requests_counter += 1
async def _async_increment_requests_counter(self, request: httpx.Request) -> None: # noqa: ARG002
self._requests_counter += 1
@property
def requests_count(self) -> int:
return self._requests_counter
@classmethod
@overload
def match_url(cls, url: str, raise_error: Literal[True] = ...) -> re.Match:
...
@classmethod
@overload
def match_url(cls, url: str, raise_error: Literal[False] = ...) -> re.Match | None:
...
@classmethod
def match_url(cls, url: str, raise_error: bool = False) -> re.Match | None:
"""
Checks if a URL matches scraper's url regex.
Args:
url (str): A URL to check against the regex.
raise_error (bool, optional): Whether to raise an error instead of returning None if the URL doesn't match.
Returns:
re.Match | None: A Match object if the URL matches the regex, None otherwise (if raise_error is False).
Raises:
ValueError: If the URL doesn't match the regex and raise_error is True.
"""
if isinstance(cls.url_regex, re.Pattern) and (match_result := re.fullmatch(pattern=cls.url_regex, string=url)):
return match_result
if isinstance(cls.url_regex, list):
for url_regex_item in cls.url_regex:
if result := re.fullmatch(pattern=url_regex_item, string=url):
return result
if raise_error:
raise ValueError(f"URL '{url}' doesn't match the URL regex of {cls.name}.")
return None
async def async_close(self) -> None:
await self._client.aclose()
@abstractmethod
async def get_data(self, url: str) -> ScrapedMediaResponse:
"""
Scrape media information about the media on a URL.
Args:
url (str): A URL to get media information about.
Returns:
ScrapedMediaResponse: A ScrapedMediaResponse object containing scraped media information.
"""
@abstractmethod
async def download_subtitles(self, media_data: PlaylistMediaItem, subrip_conversion: bool = False) -> SubtitlesData:
"""
Download subtitles from a media object.
Args:
media_data (PlaylistMediaItem): A media object to download subtitles from.
subrip_conversion (bool, optional): Whether to convert the subtitles to SubRip format. Defaults to False.
Returns:
SubtitlesData: A SubtitlesData object containing downloaded subtitles.
Raises:
SubtitlesDownloadError: If the subtitles failed to download.
"""
@abstractmethod
def find_matching_media(self, main_playlist: MainPlaylist,
filters: dict[str, str | list[str]] | None = None) -> list:
"""
Find media items that match the given filters in the main playlist (or all media items if no filters are given).
Args:
main_playlist (MainPlaylist): Main playlist to search for media items in.
filters (dict[str, str | list[str]] | None, optional): A dictionary of filters to match media items against.
Defaults to None.
Returns:
list: A list of media items that match the given filters.
"""
@abstractmethod
def find_matching_subtitles(self, main_playlist: MainPlaylist,
language_filter: list[str] | None = None) -> list[PlaylistMediaItem]:
"""
Find subtitles that match the given language filter in the main playlist.
Args:
main_playlist (MainPlaylist): Main playlist to search for subtitles in.
language_filter (list[str] | None, optional): A list of language codes to filter subtitles by.
Defaults to None.
Returns:
list[PlaylistMediaItem]: A list of subtitles media objects that match the given language filter.
"""
@abstractmethod
async def load_playlist(self, url: str | list[str], headers: dict | None = None) -> MainPlaylist | None:
"""
Load a playlist from a URL to a representing object.
Multiple URLs can be given, in which case the first one that loads successfully will be returned.
Args:
url (str | list[str]): URL of the M3U8 playlist to load. Can also be a list of URLs (for redundancy).
headers (dict | None, optional): A dictionary of headers to use when making the request.
Defaults to None (results in using session's configured headers).
Returns:
MainPlaylist | None: A playlist object (matching the type), or None if the playlist couldn't be loaded.
"""
@staticmethod
@abstractmethod
def detect_subtitles_type(subtitles_media: PlaylistMediaItem) -> SubtitlesType | None:
"""
Detect the subtitles type (Closed Captions, Forced, etc.) from a media object.
Args:
subtitles_media (PlaylistMediaItem): Subtitles media object to detect the type of.
Returns:
SubtitlesType | None: The type of the subtitles, None for regular subtitles.
"""
@classmethod
@abstractmethod
def format_subtitles_description(cls, subtitles_media: PlaylistMediaItem) -> str:
"""
Format a description of the subtitles media object.
Args:
subtitles_media (PlaylistMediaItem): Subtitles media object to format the description of.
Returns:
str: A formatted description of the subtitles media object.
Raises:
ValueError: If minimal required data is missing from the media object.
"""
class HLSScraper(Scraper, ABC):
"""A base class for HLS (m3u8) scrapers."""
class M3U8Attribute(Enum):
"""
An enum representing all possible M3U8 attributes.
Names / Keys represent M3U8 Media object attributes (should be converted to lowercase),
and values represent the name of the key for config usage.
"""
ASSOC_LANGUAGE = "assoc-language"
AUTOSELECT = "autoselect"
CHARACTERISTICS = "characteristics"
CHANNELS = "channels"
DEFAULT = "default"
FORCED = "forced"
GROUP_ID = "group-id"
INSTREAM_ID = "instream-id"
LANGUAGE = "language"
NAME = "name"
STABLE_RENDITION_ID = "stable-rendition-id"
TYPE = "type"
default_playlist_filters: ClassVar[dict[str, str | list[str] | None] | None] = None
_subtitles_filters: dict[str, str | list[str]] = {
M3U8Attribute.TYPE.value: "SUBTITLES",
}
# Resolve mypy errors as mypy doesn't support dynamic models.
if TYPE_CHECKING:
PlaylistFiltersSubcategory = ScraperConfigSubcategory
else:
PlaylistFiltersSubcategory = create_model(
"PlaylistFiltersSubcategory",
__base__=ScraperConfigSubcategory,
**{
m3u8_attribute.value: (str | list[str] | None,
Field(default=None))
for m3u8_attribute in M3U8Attribute
}, # type: ignore[call-overload]
)
class ScraperConfig(Scraper.ScraperConfig):
playlist_filters: HLSScraper.PlaylistFiltersSubcategory = Field( # type: ignore[valid-type]
default_factory=lambda: HLSScraper.PlaylistFiltersSubcategory(),
)
def __init__(self, playlist_filters: dict[str, str | list[str] | None] | None = None,
*args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self._playlist_filters = return_first_valid(playlist_filters,
get_model_field(model=self.config,
field='playlist_filters',
convert_to_dict=True,
exclude_none=True),
self.default_playlist_filters)
if self._playlist_filters:
logger.debug(f"Scraper '{self.name}' initialized with playlist filters: {self._playlist_filters}.")
@staticmethod
def parse_language_name(media_data: m3u8.Media) -> str | None:
"""
Parse the language name from an M3U8 Media object.
Can be overridden in subclasses for normalization.
Args:
media_data (m3u8.Media): Media object to parse the language name from.
Returns:
str | None: The language name if found, None otherwise.
"""
name: str | None = media_data.name
return name
async def load_playlist(self, url: str | list[str], headers: dict[str, str] | None = None) -> m3u8.M3U8 | None:
_headers = headers or self._client.headers
result: m3u8.M3U8 | None = None
urls = single_string_to_list(item=url)
response: httpx.Response | None = None
for idx, url_item in enumerate(urls):
try:
logger.debug(f"Loading M3U8 playlist from {url_item} ({idx + 1} of {len(urls)})")
response = await self._client.get(url=url_item, headers=_headers, timeout=5)
if not response.text:
logger.debug("Received an empty response for the playlist.")
continue
except Exception as e:
logger.debug(f"Failed to load playlist: {e}")
continue
if not response:
raise PlaylistLoadError("Failed to load playlists from server.")
result = m3u8.loads(content=response.text, uri=url_item)
break
return result
@staticmethod
def detect_subtitles_type(subtitles_media: m3u8.Media) -> SubtitlesType | None:
"""
Detect the subtitles type (Closed Captions, Forced, etc.) from an M3U8 Media object.
Args:
subtitles_media (m3u8.Media): Subtitles Media object to detect the type of.
Returns:
SubtitlesType | None: The type of the subtitles, None for regular subtitles.
"""
if subtitles_media.forced == "YES":
return SubtitlesType.FORCED
if subtitles_media.characteristics is not None and "public.accessibility" in subtitles_media.characteristics:
return SubtitlesType.CC
return None
async def download_subtitles(self, media_data: m3u8.Media, subrip_conversion: bool = False) -> SubtitlesData:
try:
playlist_m3u8 = await self.load_playlist(url=media_data.absolute_uri)
if playlist_m3u8 is None:
raise PlaylistLoadError("Could not load subtitles M3U8 playlist.") # noqa: TRY301
if not media_data.language:
raise ValueError("Language code not found in media data.") # noqa: TRY301
downloaded_segments = await self.download_segments(playlist=playlist_m3u8)
subtitles = self.subtitles_class(data=downloaded_segments[0], language_code=media_data.language)
if len(downloaded_segments) > 1:
for segment_data in downloaded_segments[1:]:
segment_subtitles_obj = self.subtitles_class(data=segment_data, language_code=media_data.language)
subtitles.append_subtitles(segment_subtitles_obj)
subtitles.polish(
fix_rtl=self.subtitles_fix_rtl,
remove_duplicates=self.subtitles_remove_duplicates,
)
if subrip_conversion:
subtitles_format = SubtitlesFormatType.SUBRIP
content = subtitles.to_srt().dump()
else:
subtitles_format = SubtitlesFormatType.WEBVTT
content = subtitles.dump()
return SubtitlesData(
language_code=media_data.language,
language_name=self.parse_language_name(media_data=media_data),
subtitles_format=subtitles_format,
content=content,
content_encoding=subtitles.encoding,
special_type=self.detect_subtitles_type(subtitles_media=media_data),
)
except Exception as e:
raise SubtitlesDownloadError(
language_code=media_data.language,
language_name=self.parse_language_name(media_data=media_data),
special_type=self.detect_subtitles_type(subtitles_media=media_data),
original_exc=e,
) from e
async def download_segments(self, playlist: m3u8.M3U8) -> list[bytes]:
responses = await asyncio.gather(
*[
self._client.get(url=segment.absolute_uri)
for segment in playlist.segments
],
)
responses_data = []
for result in responses:
try:
result.raise_for_status()
responses_data.append(result.content)
except Exception as e:
raise DownloadError("One of the subtitles segments failed to download.") from e
return responses_data
def find_matching_media(self, main_playlist: m3u8.M3U8,
filters: dict[str, str | list[str]] | None = None) -> list[m3u8.Media]:
results: list[m3u8.Media] = []
playlist_filters: dict[str, str | list[str]] | None
if self._playlist_filters:
# Merge filtering dictionaries into a single dictionary
playlist_filters = merge_dict_values(
*[dict_item for dict_item in (filters, self._playlist_filters)
if dict_item is not None],
)
else:
playlist_filters = filters
for media in main_playlist.media:
if not playlist_filters:
results.append(media)
continue
is_valid = True
for filter_name, filter_value in playlist_filters.items():
# Skip filter if its value is None
if filter_value is None:
continue
try:
filter_name_enum = HLSScraper.M3U8Attribute(filter_name)
attribute_value = getattr(media, filter_name_enum.name.lower(), None)
if (attribute_value is None) or (
isinstance(filter_value, list) and
attribute_value.casefold() not in (x.casefold() for x in filter_value)
) or (
isinstance(filter_value, str) and filter_value.casefold() != attribute_value.casefold()
):
is_valid = False
break
except Exception:
is_valid = False
if is_valid:
results.append(media)
return results
def find_matching_subtitles(self, main_playlist: m3u8.M3U8,
language_filter: list[str] | None = None) -> list[m3u8.Media]:
_filters = self._subtitles_filters
if language_filter:
_filters[self.M3U8Attribute.LANGUAGE.value] = language_filter
return self.find_matching_media(main_playlist=main_playlist, filters=_filters)
@classmethod
def format_subtitles_description(cls, subtitles_media: m3u8.Media) -> str:
return format_subtitles_description(
language_code=subtitles_media.language,
language_name=cls.parse_language_name(media_data=subtitles_media),
special_type=cls.detect_subtitles_type(subtitles_media=subtitles_media),
)
class ScraperFactory:
_scraper_classes_cache: list[type[Scraper]] | None = None
_scraper_instances_cache: dict[type[Scraper], Scraper] = {}
_currently_initializing: list[type[Scraper]] = [] # Used to prevent infinite recursion
@classmethod
def get_initialized_scrapers(cls) -> list[Scraper]:
"""
Get a list of all previously initialized scrapers.
Returns:
list[Scraper]: A list of initialized scrapers.
"""
return list(cls._scraper_instances_cache.values())
@classmethod
def get_scraper_classes(cls) -> list[type[Scraper]]:
"""
Find all scraper classes in the scrapers directory.
Returns:
list[Scraper]: A Scraper subclass.
"""
if cls._scraper_classes_cache is not None:
return cls._scraper_classes_cache
cls._scraper_classes_cache = []
scraper_modules_paths = Path(__file__).parent.glob(f"*{SCRAPER_MODULES_SUFFIX}.py")
for scraper_module_path in scraper_modules_paths:
sys.path.append(str(scraper_module_path))
module = importlib.import_module(f"{PACKAGE_NAME}.scrapers.{scraper_module_path.stem}")
# Find all 'Scraper' subclasses
for _, obj in inspect.getmembers(module,
predicate=lambda x: inspect.isclass(x) and issubclass(x, Scraper)):
# Skip object if it's an abstract or imported from another module
if not inspect.isabstract(obj) and obj.__module__ == module.__name__:
cls._scraper_classes_cache.append(obj)
return cls._scraper_classes_cache
@classmethod
def _get_scraper_instance(cls, scraper_class: type[ScraperT], kwargs: dict | None = None) -> ScraperT:
"""
Initialize and return a scraper instance.
Args:
scraper_class (type[ScraperT]): A scraper class to initialize.
kwargs (dict | None, optional): A dictionary containing parameters to pass to the scraper's constructor.
Defaults to None.
Returns:
Scraper: An instance of the given scraper class.
"""
logger.debug(f"Initializing '{scraper_class.name}' scraper...")
kwargs = kwargs or {}
if scraper_class not in cls._scraper_instances_cache:
logger.debug(f"'{scraper_class.name}' scraper not found in cache, creating a new instance...")
if scraper_class in cls._currently_initializing:
raise ScraperError(f"'{scraper_class.name}' scraper is already being initialized.\n"
f"Make sure there are no circular dependencies between scrapers.")
cls._currently_initializing.append(scraper_class)
cls._scraper_instances_cache[scraper_class] = scraper_class(**kwargs)
cls._currently_initializing.remove(scraper_class)
else:
logger.debug(f"Cached '{scraper_class.name}' scraper instance found and will be used.")
return cls._scraper_instances_cache[scraper_class] # type: ignore[return-value]
@classmethod
@overload
def get_scraper_instance(cls, scraper_class: type[ScraperT], scraper_id: str | None = ...,
url: str | None = ..., kwargs: dict | None = ...,
raise_error: Literal[True] = ...) -> ScraperT:
...
@classmethod
@overload
def get_scraper_instance(cls, scraper_class: type[ScraperT], scraper_id: str | None = ...,
url: str | None = ..., kwargs: dict | None = ...,
raise_error: Literal[False] = ...) -> ScraperT | None:
...
@classmethod
@overload
def get_scraper_instance(cls, scraper_class: None = ..., scraper_id: str | None = ...,
url: str | None = ..., kwargs: dict | None = ...,
raise_error: Literal[True] = ...) -> Scraper:
...
@classmethod
@overload
def get_scraper_instance(cls, scraper_class: None = ..., scraper_id: str | None = ...,
url: str | None = ..., kwargs: dict | None = ...,
raise_error: Literal[False] = ...) -> Scraper | None:
...
@classmethod
def get_scraper_instance(cls, scraper_class: type[Scraper] | None = None, scraper_id: str | None = None,
url: str | None = None, kwargs: dict | None = None,
raise_error: bool = True) -> Scraper | None:
"""
Find, initialize and return a scraper that matches the given URL or ID.
Args:
scraper_class (type[ScraperT] | None, optional): A scraper class to initialize. Defaults to None.
scraper_id (str | None, optional): ID of a scraper to initialize. Defaults to None.
url (str | None, optional): A URL to match a scraper for to initialize. Defaults to None.
kwargs (dict | None, optional): A dictionary containing parameters to pass to the scraper's constructor.
Defaults to None.
raise_error (bool, optional): Whether to raise an error if no scraper was found. Defaults to False.
Returns:
ScraperT | Scraper | None: An instance of a scraper that matches the given URL or ID,
None otherwise (if raise_error is False).
Raises:
ValueError: If no scraper was found and 'raise_error' is True.
"""
if not any((scraper_class, scraper_id, url)):
raise ValueError("At least one of: 'scraper_class', 'scraper_id', or 'url' must be provided.")
if scraper_class:
return cls._get_scraper_instance(
scraper_class=scraper_class,
kwargs=kwargs,
)
if scraper_id:
logger.debug(f"Searching for a scraper object with ID '{scraper_id}'...")
for scraper in cls.get_scraper_classes():
if scraper.id == scraper_id:
return cls._get_scraper_instance(
scraper_class=scraper,
kwargs=kwargs,
)
elif url:
logger.debug(f"Searching for a scraper object that matches URL '{url}'...")
for scraper in cls.get_scraper_classes():
if scraper.match_url(url) is not None:
return cls._get_scraper_instance(
scraper_class=scraper,
kwargs=kwargs,
)
error_message = "No matching scraper was found."
if raise_error:
raise ValueError(error_message)
logger.debug(error_message)
return None
class ScraperError(Exception):
pass
class DownloadError(ScraperError):
pass
class PlaylistLoadError(ScraperError):
pass
class SubtitlesDownloadError(ScraperError):
def __init__(self, language_code: str | None, language_name: str | None = None,
special_type: SubtitlesType | None = None, original_exc: Exception | None = None,
*args: Any, **kwargs: dict[str, Any]):
"""
Initialize a SubtitlesDownloadError instance.
Args:
language_code (str | None, optional): Language code of the subtitles that failed to download.
language_name (str | None, optional): Language name of the subtitles that failed to download.
special_type (SubtitlesType | None, optional): Type of the subtitles that failed to download.
original_exc (Exception | None, optional): The original exception that caused the error.
"""
super().__init__(*args, **kwargs)
self.language_code = language_code
self.language_name = language_name
self.special_type = special_type
self.original_exc = original_exc
================================================
FILE: isubrip/subtitle_formats/__init__.py
================================================
================================================
FILE: isubrip/subtitle_formats/subrip.py
================================================
from __future__ import annotations
from typing import Any
from isubrip.data_structures import SubtitlesFormatType
from isubrip.subtitle_formats.subtitles import Subtitles, SubtitlesCaptionBlock
class SubRipCaptionBlock(SubtitlesCaptionBlock):
"""A subtitles caption block based on the SUBRIP format."""
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and \
self.start_time == other.start_time and self.end_time == other.end_time and self.payload == other.payload
def __str__(self) -> str:
result_str = ""
time_format = "%H:%M:%S,%f"
result_str += f"{self.start_time.strftime(time_format)[:-3]} --> {self.end_time.strftime(time_format)[:-3]}\n"
result_str += f"{self.payload}"
return result_str
def to_srt(self) -> SubRipCaptionBlock:
return self
class SubRipSubtitles(Subtitles[SubRipCaptionBlock]):
"""An object representing a SubRip subtitles file."""
format = SubtitlesFormatType.SUBRIP
def _dumps(self) -> str:
subtitles_str = ""
for i, block in enumerate(iterable=self.blocks, start=1):
subtitles_str += f"{i}\n{str(block)}\n\n"
return subtitles_str.rstrip('\n')
def _loads(self, data: str) -> None:
raise NotImplementedError("SubRip subtitles loading is not supported.")
================================================
FILE: isubrip/subtitle_formats/subtitles.py
================================================
from __future__ import annotations
from abc import ABC, abstractmethod
from copy import deepcopy
from datetime import time
from typing import TYPE_CHECKING, Any, ClassVar, Generic, TypeVar
from isubrip.constants import RTL_LANGUAGES
from isubrip.logger import logger
if TYPE_CHECKING:
from isubrip.data_structures import SubtitlesFormatType
from isubrip.subtitle_formats.subrip import SubRipCaptionBlock, SubRipSubtitles
RTL_CONTROL_CHARS = ('\u200e', '\u200f', '\u202a', '\u202b', '\u202c', '\u202d', '\u202e')
RTL_CHAR = '\u202b'
SubtitlesT = TypeVar('SubtitlesT', bound='Subtitles')
SubtitlesBlockT = TypeVar('SubtitlesBlockT', bound='SubtitlesBlock')
class SubtitlesBlock(ABC):
"""
Abstract base class for subtitles blocks.
Attributes:
modified (bool): Whether the block has been modified.
"""
def __init__(self) -> None:
self.modified: bool = False
@abstractmethod
def __copy__(self) -> SubtitlesBlock:
"""Create a copy of the block."""
@abstractmethod
def __eq__(self, other: Any) -> bool:
"""Check if two objects are equal."""
@abstractmethod
def __str__(self) -> str:
"""Return a string representation of the block."""
class SubtitlesCaptionBlock(SubtitlesBlock, ABC):
"""
A base class for subtitles caption blocks.
Attributes:
start_time (time): Start timestamp of the caption block.
end_time (time): End timestamp of the caption block.
payload (str): Caption block's payload.
"""
def __init__(self, start_time: time, end_time: time, payload: str):
"""
Initialize a new SubtitlesCaptionBlock object.
Args:
start_time: Start timestamp of the caption block.
end_time: End timestamp of the caption block.
payload: Caption block's payload.
"""
super().__init__()
self.start_time = start_time
self.end_time = end_time
self.payload = payload
def __copy__(self) -> SubtitlesCaptionBlock:
copy = self.__class__(self.start_time, self.end_time, self.payload)
copy.modified = self.modified
return copy
def fix_rtl(self) -> None:
"""Fix payload's text direction to RTL."""
previous_payload = self.payload
# Remove previous RTL-related formatting
for char in RTL_CONTROL_CHARS:
self.payload = self.payload.replace(char, '')
# Add RLM char at the start of every line
self.payload = RTL_CHAR + self.payload.replace("\n", f"\n{RTL_CHAR}")
if self.payload != previous_payload:
self.modified = True
@abstractmethod
def to_srt(self) -> SubRipCaptionBlock:
"""
Convert WebVTT caption block to SRT caption block.
Returns:
SubRipCaptionBlock: The caption block in SRT format.
"""
...
class Subtitles(Generic[SubtitlesBlockT], ABC):
"""
An object representing subtitles, made out of blocks.
Attributes:
_modified (bool): Whether the subtitles have been modified.
format (SubtitlesFormatType): [Class Attribute] Format of the subtitles (contains name and file extension).
language_code (str): Language code of the subtitles.
blocks (list[SubtitlesBlock]): A list of subtitles blocks that make up the subtitles.
encoding (str): Encoding of the subtitles.
raw_data (bytes | None): Raw data of the subtitles.
"""
format: ClassVar[SubtitlesFormatType]
def __init__(self, data: bytes | None, language_code: str, encoding: str = "utf-8"):
"""
Initialize a new Subtitles object.
Args:
data (bytes | None): Raw data of the subtitles.
language_code (str): Language code of the subtitles.
encoding (str, optional): Encoding of the subtitles. Defaults to "utf-8".
"""
self._modified = False
self.raw_data = None
self.blocks: list[SubtitlesBlockT] = []
self.language_code = language_code
self.encoding = encoding
if data:
self.raw_data = data
self._load(data=data)
def __add__(self: SubtitlesT, obj: SubtitlesBlockT | SubtitlesT) -> SubtitlesT:
"""
Add a new subtitles block, or append blocks from another subtitles object.
Args:
obj (SubtitlesBlock | Subtitles): A subtitles block or another subtitles object.
Returns:
Subtitles: The current subtitles object.
"""
if isinstance(obj, SubtitlesBlock):
self.add_blocks(obj)
elif isinstance(obj, self.__class__):
self.append_subtitles(obj)
else:
logger.warning(f"Cannot add object of type '{type(obj)}' to '{type(self)}' object. Skipping...")
return self
def __copy__(self: SubtitlesT) -> SubtitlesT:
"""Create a copy of the subtitles object."""
copy = self.__class__(data=None, language_code=self.language_code, encoding=self.encoding)
copy.raw_data = self.raw_data
copy.blocks = [block.__copy__() for block in self.blocks]
copy._modified = self.modified() # noqa: SLF001
return copy
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and self.blocks == other.blocks
def __str__(self) -> str:
return self.dumps()
def _dump(self) -> bytes:
"""
Dump subtitles object to bytes representing the subtitles.
Returns:
bytes: The subtitles in a bytes object.
"""
return self._dumps().encode(encoding=self.encoding)
@abstractmethod
def _dumps(self) -> str:
"""
Dump subtitles object to a string representing the subtitles.
Returns:
str: The subtitles in a string format.
"""
...
def _load(self, data: bytes) -> None:
"""
Load and parse subtitles data from bytes.
Args:
data (bytes): Subtitles data to load.
"""
parsed_data = data.decode(encoding=self.encoding)
self._loads(data=parsed_data)
@abstractmethod
def _loads(self, data: str) -> None:
"""
Load and parse subtitles data from a string.
Args:
data (bytes): Subtitles data to load.
"""
...
def dump(self) -> bytes:
"""
Dump subtitles to a bytes object representing the subtitles.
Returns the original raw subtitles data if they have not been modified, and raw data is available.
Returns:
bytes: The subtitles in a bytes object.
"""
if self.raw_data is not None and not self.modified():
logger.debug("Returning original raw data as subtitles have not been modified.")
return self.raw_data
return self._dump()
def dumps(self) -> str:
"""
Dump subtitles to a string representing the subtitles.
Returns the original raw subtitles data if they have not been modified, and raw data is available.
Returns:
"""
if self.raw_data is not None and not self.modified():
logger.debug("Returning original raw data (decoded) as subtitles have not been modified.")
return self.raw_data.decode(encoding=self.encoding)
return self._dumps()
def add_blocks(self: SubtitlesT,
blocks: SubtitlesBlockT | list[SubtitlesBlockT],
set_modified: bool = True) -> SubtitlesT:
"""
Add a new subtitles block to current subtitles.
Args:
blocks (SubtitlesBlock | list[SubtitlesBlock]):
A block object or a list of block objects to append.
set_modified (bool, optional): Whether to set the subtitles as modified. Defaults to True.
Returns:
Subtitles: The current subtitles object.
"""
if isinstance(blocks, list):
if not blocks:
return self
self.blocks.extend(blocks)
else:
self.blocks.append(blocks)
if set_modified:
self._modified = True
return self
def append_subtitles(self: SubtitlesT,
subtitles: SubtitlesT) -> SubtitlesT:
"""
Append subtitles to an existing subtitles object.
Args:
subtitles (Subtitles): Subtitles object to append to current subtitles.
Returns:
Subtitles: The current subtitles object.
"""
if subtitles.blocks:
self.add_blocks(deepcopy(subtitles.blocks))
if subtitles.modified():
self._modified = True
return self
def polish(self: SubtitlesT,
fix_rtl: bool = False,
remove_duplicates: bool = True,
) -> SubtitlesT:
"""
Apply various fixes to subtitles.
Args:
fix_rtl (bool, optional): Whether to fix text direction of RTL languages. Defaults to False.
remove_duplicates (bool, optional): Whether to remove duplicate captions. Defaults to True.
Returns:
Subtitles: The current subtitles object.
"""
fix_rtl = (fix_rtl and self.language_code.split('-')[0] in RTL_LANGUAGES)
if not any((
fix_rtl,
remove_duplicates,
)):
return self
previous_block: SubtitlesBlockT | None = None
for block in self.blocks:
if fix_rtl:
block.fix_rtl()
if remove_duplicates and previous_block is not None and block == previous_block:
self.blocks.remove(previous_block)
self._modified = True
previous_block = block
return self
def modified(self) -> bool:
"""
Check if the subtitles have been modified (by checking if any of its blocks have been modified).
Returns:
bool: True if the subtitles have been modified, False otherwise.
"""
return self._modified or any(block.modified for block in self.blocks)
def to_srt(self) -> SubRipSubtitles:
"""
Convert subtitles to SRT format.
Returns:
SubRipSubtitles: The subtitles in SRT format.
"""
from isubrip.subtitle_formats.subrip import SubRipSubtitles
subrip_subtitles = SubRipSubtitles(
data=None,
language_code=self.language_code,
encoding=self.encoding,
)
subrip_blocks = [block.to_srt() for block in self.blocks if isinstance(block, SubtitlesCaptionBlock)]
subrip_subtitles.add_blocks(subrip_blocks)
return subrip_subtitles
def split_timestamp(timestamp: str) -> tuple[time, time]:
"""
Split a subtitles timestamp into start and end.
Args:
timestamp (str): A subtitles timestamp. For example: "00:00:00.000 --> 00:00:00.000"
Returns:
tuple(time, time): A tuple containing start and end times as a datetime object.
"""
# Support ',' character in timestamp's milliseconds (used in SubRip format).
timestamp = timestamp.replace(',', '.')
start_time, end_time = timestamp.split(" --> ")
return time.fromisoformat(start_time), time.fromisoformat(end_time)
================================================
FILE: isubrip/subtitle_formats/webvtt.py
================================================
from __future__ import annotations
from abc import ABCMeta
from copy import deepcopy
import re
from typing import TYPE_CHECKING, Any, ClassVar
from isubrip.data_structures import SubtitlesFormatType
from isubrip.subtitle_formats.subrip import SubRipCaptionBlock
from isubrip.subtitle_formats.subtitles import RTL_CHAR, Subtitles, SubtitlesBlock, SubtitlesCaptionBlock
from isubrip.utils import split_subtitles_timestamp
if TYPE_CHECKING:
from datetime import time
# WebVTT Documentation:
# https://www.w3.org/TR/webvtt1/#cues
# https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API#webvtt_cues
bottom_line_alignment_regex = re.compile(r"line:0+(?:\.0+)?%")
class WebVTTBlock(SubtitlesBlock, metaclass=ABCMeta):
"""
Abstract base class for WEBVTT cue blocks.
"""
is_caption_block: bool = False
class WebVTTCaptionBlock(SubtitlesCaptionBlock, WebVTTBlock):
"""An object representing a WebVTT caption block."""
subrip_alignment_conversion: ClassVar[bool] = False
is_caption_block: bool = True
def __init__(self, start_time: time, end_time: time, payload: str, settings: str = "", identifier: str = ""):
"""
Initialize a new object representing a WebVTT caption block.
Args:
start_time (time): Cue start time.
end_time (time): Cue end time.
settings (str): Cue settings.
payload (str): Cue payload.
"""
super().__init__(start_time=start_time, end_time=end_time, payload=payload)
self.identifier = identifier
self.settings = settings
def __copy__(self) -> WebVTTCaptionBlock:
copy = self.__class__(start_time=self.start_time, end_time=self.end_time, payload=self.payload,
settings=self.settings, identifier=self.identifier)
copy.modified = self.modified
return copy
def to_srt(self) -> SubRipCaptionBlock:
# Add a {\an8} tag at the start of the payload if the `line` setting is set to 0%
if self.subrip_alignment_conversion and WEBVTT_BOTTOM_LINE_ALIGNMENT_REGEX.search(self.settings):
# If the payload starts with an RTL control char, add the tag after it
if self.payload.startswith(RTL_CHAR):
payload = RTL_CHAR + WEBVTT_ALIGN_TOP_TAG + self.payload[len(RTL_CHAR):]
else:
payload = WEBVTT_ALIGN_TOP_TAG + self.payload
else:
payload = self.payload
return SubRipCaptionBlock(start_time=self.start_time, end_time=self.end_time, payload=payload)
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and \
self.start_time == other.start_time and self.end_time == other.end_time and self.payload == other.payload
def __str__(self) -> str:
result_str = ""
time_format = "%H:%M:%S.%f"
# Add identifier (if it exists)
if self.identifier:
result_str += f"{self.identifier}\n"
result_str += f"{self.start_time.strftime(time_format)[:-3]} --> {self.end_time.strftime(time_format)[:-3]}"
if self.settings:
result_str += f" {self.settings}"
result_str += f"\n{self.payload}"
return result_str
class WebVTTCommentBlock(WebVTTBlock):
"""An object representing a WebVTT comment block."""
header = "NOTE"
def __init__(self, payload: str, inline: bool = False) -> None:
"""
Initialize a new object representing a WebVTT comment block.
Args:
payload (str): Comment payload.
"""
super().__init__()
self.payload = payload
self.inline = inline
def __copy__(self) -> WebVTTCommentBlock:
copy = self.__class__(payload=self.payload, inline=self.inline)
copy.modified = self.modified
return copy
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and self.inline == other.inline and self.payload == other.payload
def __str__(self) -> str:
if self.inline:
return f"{self.header} {self.payload}"
if self.payload:
return f"{self.header}\n{self.payload}"
return self.header
class WebVTTStyleBlock(WebVTTBlock):
"""An object representing a WebVTT style block."""
header = "STYLE"
def __init__(self, payload: str) -> None:
"""
Initialize a new object representing a WebVTT style block.
Args:
payload (str): Style payload.
"""
super().__init__()
self.payload = payload
def __copy__(self) -> WebVTTStyleBlock:
copy = self.__class__(payload=self.payload)
copy.modified = self.modified
return copy
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and self.payload == other.payload
def __str__(self) -> str:
return f"{self.header}\n{self.payload}"
class WebVTTRegionBlock(WebVTTBlock):
"""An object representing a WebVTT region block."""
header = "REGION"
def __init__(self, payload: str) -> None:
"""
Initialize a new object representing a WebVTT region block.
Args:
payload (str): Region payload.
"""
super().__init__()
self.payload = payload
def __copy__(self) -> WebVTTRegionBlock:
copy = self.__class__(payload=self.payload)
copy.modified = self.modified
return copy
def __eq__(self, other: Any) -> bool:
return isinstance(other, type(self)) and self.payload == other.payload
def __str__(self) -> str:
return f"{self.header} {self.payload}"
class WebVTTSubtitles(Subtitles[WebVTTBlock]):
"""An object representing a WebVTT subtitles file."""
format = SubtitlesFormatType.WEBVTT
def _dumps(self) -> str:
"""
Dump subtitles to a string representing the subtitles in a WebVTT format.
Returns:
str: The subtitles in a string using a WebVTT format.
"""
subtitles_str = "WEBVTT\n\n"
for block in self.blocks:
subtitles_str += str(block) + "\n\n"
return subtitles_str.rstrip('\n')
def _loads(self, data: str) -> None:
"""
Load and parse WebVTT subtitles data from a string.
Args:
data (bytes): Subtitles data to load.
"""
prev_line: str = ""
lines_iterator = iter(data.splitlines())
for line in lines_iterator:
# If the line is a timestamp
if caption_block_regex := re.match(WEBVTT_CAPTION_BLOCK_REGEX, line):
# If previous line wasn't empty, add it as an identifier
if prev_line:
caption_identifier = prev_line
else:
caption_identifier = ""
caption_timestamps = split_subtitles_timestamp(caption_block_regex.group(1))
caption_settings = caption_block_regex.group(2)
caption_payload = ""
for additional_line in lines_iterator:
if not additional_line:
line = additional_line
break
caption_payload += additional_line + "\n"
caption_payload = caption_payload.rstrip("\n")
self.blocks.append(WebVTTCaptionBlock(
identifier=caption_identifier,
start_time=caption_timestamps[0],
end_time=caption_timestamps[1],
settings=caption_settings,
payload=caption_payload))
elif comment_block_regex := re.match(WEBVTT_COMMENT_HEADER_REGEX, line):
comment_payload = ""
inline = False
if comment_block_regex.group(1) is not None:
comment_payload += comment_block_regex.group(1) + "\n"
inline = True
for additional_line in lines_iterator:
if not additional_line:
line = additional_line
break
comment_payload += additional_line + "\n"
self.blocks.append(WebVTTCommentBlock(comment_payload.rstrip("\n"), inline=inline))
elif line.rstrip(' \t') == WebVTTRegionBlock.header:
region_payload = ""
for additional_line in lines_iterator:
if not additional_line:
line = additional_line
break
region_payload += additional_line + "\n"
self.blocks.append(WebVTTRegionBlock(region_payload.rstrip("\n")))
elif line.rstrip(' \t') == WebVTTStyleBlock.header:
style_payload = ""
for additional_line in lines_iterator:
if not additional_line:
line = additional_line
break
style_payload += additional_line + "\n"
self.blocks.append(WebVTTStyleBlock(style_payload.rstrip("\n")))
prev_line = line
def append_subtitles(self: WebVTTSubtitles,
subtitles: WebVTTSubtitles) -> WebVTTSubtitles:
if subtitles.blocks:
subtitles_copy = deepcopy(subtitles)
# Remove head blocks from the subtitles that will be appended
subtitles_copy.remove_head_blocks()
self.add_blocks(subtitles_copy.blocks)
if subtitles_copy.modified():
self._modified = True
return self
def remove_head_blocks(self) -> None:
"""
Remove all head blocks (Style / Region) from the subtitles.
NOTE:
Comment blocks are removed as well if they are before the first caption block (since they're probably
related to the head blocks).
"""
for block in self.blocks:
if isinstance(block, WebVTTCaptionBlock):
break
if isinstance(block, WebVTTCommentBlock | WebVTTStyleBlock | WebVTTRegionBlock):
self.blocks.remove(block)
# --- Constants ---
WEBVTT_PERCENTAGE_REGEX = r"\d{1,3}(?:\.\d+)?%"
WEBVTT_CAPTION_TIMINGS_REGEX = \
r"(?:[0-5]\d:)?[0-5]\d:[0-5]\d[\.,]\d{3}[ \t]+-->[ \t]+(?:[0-5]\d:)?[0-5]\d:[0-5]\d[\.,]\d{3}"
WEBVTT_CAPTION_SETTING_ALIGNMENT_REGEX = r"align:(?:start|center|middle|end|left|right)"
WEBVTT_CAPTION_SETTING_LINE_REGEX = rf"line:(?:{WEBVTT_PERCENTAGE_REGEX}|-?\d+%)(?:,(?:start|center|middle|end))?"
WEBVTT_CAPTION_SETTING_POSITION_REGEX = rf"position:{WEBVTT_PERCENTAGE_REGEX}(?:,(?:start|center|middle|end))?"
WEBVTT_CAPTION_SETTING_REGION_REGEX = r"region:(?:(?!(?:-->)|\t)\S)+"
WEBVTT_CAPTION_SETTING_SIZE_REGEX = rf"size:{WEBVTT_PERCENTAGE_REGEX}"
WEBVTT_CAPTION_SETTING_VERTICAL_REGEX = r"vertical:(?:lr|rl)"
WEBVTT_CAPTION_SETTINGS_REGEX = ("(?:"
f"(?:{WEBVTT_CAPTION_SETTING_ALIGNMENT_REGEX})|"
f"(?:{WEBVTT_CAPTION_SETTING_LINE_REGEX})|"
f"(?:{WEBVTT_CAPTION_SETTING_POSITION_REGEX})|"
f"(?:{WEBVTT_CAPTION_SETTING_REGION_REGEX})|"
f"(?:{WEBVTT_CAPTION_SETTING_SIZE_REGEX})|"
f"(?:{WEBVTT_CAPTION_SETTING_VERTICAL_REGEX})|"
f"(?:[ \t]+)"
")*")
WEBVTT_CAPTION_BLOCK_REGEX = re.compile(rf"^({WEBVTT_CAPTION_TIMINGS_REGEX})[ \t]*({WEBVTT_CAPTION_SETTINGS_REGEX})?")
WEBVTT_COMMENT_HEADER_REGEX = re.compile(rf"^{WebVTTCommentBlock.header}(?:$|[ \t])(.+)?")
WEBVTT_BOTTOM_LINE_ALIGNMENT_REGEX = re.compile(r"line:0+(?:\.0+)?%")
WEBVTT_ALIGN_TOP_TAG = r"{\an8}"
================================================
FILE: isubrip/ui.py
================================================
from __future__ import annotations
import math
from typing import TYPE_CHECKING
from rich.progress import TimeElapsedColumn
from rich.text import Text
if TYPE_CHECKING:
from rich.progress import Task
class MinsAndSecsTimeElapsedColumn(TimeElapsedColumn):
"""Renders time elapsed in minutes and seconds."""
def render(self, task: Task) -> Text:
"""Show time elapsed."""
elapsed = task.finished_time if task.finished else task.elapsed
if elapsed is None:
return Text("-:--", style="progress.elapsed")
minutes, seconds = divmod(math.ceil(elapsed), 60)
return Text(f"{minutes:02d}:{seconds:02d}", style="progress.elapsed")
================================================
FILE: isubrip/utils.py
================================================
from __future__ import annotations
from abc import ABCMeta
import datetime as dt
from functools import lru_cache
import logging
from pathlib import Path
import re
import secrets
import shutil
import sys
from typing import TYPE_CHECKING, Any, Literal, cast, overload
from wcwidth import wcswidth
from isubrip.constants import WINDOWS_RESERVED_FILE_NAMES, temp_folder_path
from isubrip.data_structures import (
Episode,
MediaBase,
Movie,
Season,
Series,
SubtitlesData,
SubtitlesFormatType,
SubtitlesType,
T,
)
from isubrip.logger import logger
if TYPE_CHECKING:
from os import PathLike
from types import TracebackType
import httpx
from pydantic import BaseModel, ValidationError
class SingletonMeta(ABCMeta):
"""
A metaclass that implements the Singleton pattern.
When a class using this metaclass is initialized, it will return the same instance every time.
"""
_instances: dict[object, object] = {}
def __call__(cls, *args: Any, **kwargs: Any) -> object:
if cls._instances.get(cls) is None:
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]
class TemporaryDirectory:
"""
A context manager for creating and managing a temporary directory.
Args:
directory_name (str | None, optional): Name of the directory to generate.
If not specified, a random string will be generated. Defaults to None.
"""
def __init__(self, directory_name: str | None = None):
if directory_name:
self.directory_name = sanitize_path_segment(directory_name)
else:
self.directory_name = secrets.token_hex(5)
self.path = temp_folder_path() / self.directory_name
def __enter__(self) -> Path:
"""Create the temporary directory and return its path."""
if self.path.is_dir():
logger.debug(f"Temporary directory '{self.path}' already exists. "
f"Emptying directory from all contents...")
shutil.rmtree(self.path)
self.path.mkdir(parents=True)
logger.debug(f"Temporary directory has been generated: '{self.path}'")
return self.path
def __exit__(self, exc_type: type[BaseException] | None,
exc_val: BaseException | None, exc_tb: TracebackType | None) -> None:
"""Clean up the temporary directory."""
self.cleanup()
def cleanup(self) -> None:
"""Remove the temporary directory."""
if not self.path.exists():
return
logger.debug(f"Removing temporary directory: '{self.path}'")
try:
shutil.rmtree(self.path)
except Exception as e:
logger.warning(f"Failed to remove temporary directory '{self.path}': {e}")
def convert_epoch_to_datetime(epoch_timestamp: int) -> dt.datetime:
"""
Convert an epoch timestamp to a datetime object.
Args:
epoch_timestamp (int): Epoch timestamp.
Returns:
datetime: A datetime object representing the timestamp.
"""
if epoch_timestamp >= 0:
return dt.datetime.fromtimestamp(epoch_timestamp)
return dt.datetime(1970, 1, 1) + dt.timedelta(seconds=epoch_timestamp)
def convert_log_level(log_level: str) -> int:
"""
Convert a log level string to a logging level.
Args:
log_level (str): Log level string.
Returns:
int: Logging level.
Raises:
ValueError: If the log level is invalid.
"""
log_level_upper = log_level.upper()
if log_level_upper not in ('DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'):
raise ValueError(f"Invalid log level: {log_level}")
return cast("int", getattr(logging, log_level_upper))
def download_subtitles_to_file(media_data: Movie | Episode, subtitles_data: SubtitlesData, output_path: str | PathLike,
source_abbreviation: str | None = None, overwrite: bool = False) -> Path:
"""
Download subtitles to a file.
Args:
media_data (Movie | Episode): An object containing media data.
subtitles_data (SubtitlesData): A SubtitlesData object containing subtitles data.
output_path (str | PathLike): Path to the output folder.
source_abbreviation (str | None, optional): Abbreviation of the source the subtitles are downloaded from.
Defaults to None.
overwrite (bool, optional): Whether to overwrite files if they already exist. Defaults to True.
Returns:
Path: Path to the downloaded subtitles file.
Raises:
ValueError: If the path in `output_path` does not exist.
"""
output_path = Path(output_path)
if not output_path.is_dir():
raise ValueError(f"Invalid path: {output_path}")
if isinstance(media_data, Movie):
file_name = format_release_name(title=media_data.name,
release_date=media_data.release_date,
media_source=source_abbreviation,
language_code=subtitles_data.language_code,
subtitles_type=subtitles_data.special_type,
file_format=subtitles_data.subtitles_format)
else: # isinstance(media_data, Episode):
file_name = format_release_name(title=media_data.series_name,
release_date=media_data.release_date,
season_number=media_data.season_number,
episode_number=media_data.episode_number,
episode_name=media_data.episode_name,
media_source=source_abbreviation,
language_code=subtitles_data.language_code,
subtitles_type=subtitles_data.special_type,
file_format=subtitles_data.subtitles_format)
file_path = output_path / file_name
if file_path.exists() and not overwrite:
file_path = generate_non_conflicting_path(file_path=file_path)
with file_path.open('wb') as f:
f.write(subtitles_data.content)
return file_path
def format_config_validation_error(exc: ValidationError) -> str:
"""
Format a Pydantic ValidationError into a human-readable string.
Args:
exc (ValidationError): The ValidationError instance containing validation errors.
Returns:
str: A formatted string describing the validation errors, including the location,
type, value, and error messages for each invalid field.
"""
validation_errors = exc.errors()
error_str = ""
consolidated_errors: dict[str, dict[str, Any]] = {}
for validation_error in validation_errors:
value: Any = validation_error['input']
value_type: str = type(value).__name__
location: list[str] = [str(item) for item in validation_error['loc']]
error_msg: str = validation_error['msg']
# When the expected type is a union, Pydantic returns several errors for each type,
# with the type being the last item in the location list
if (
isinstance(location[-1], str) and
(location[-1].endswith(']') or location[-1] in ('str', 'int', 'float', 'bool'))
):
location.pop()
if len(location) > 1:
location_str = ".".join(location)
else:
location_str = location[0]
if location_str in consolidated_errors:
consolidated_errors[location_str]['errors'].append(error_msg)
else:
consolidated_errors[location_str] = {}
consolidated_errors[location_str]['info'] = {
"value": value,
"type": value_type,
}
consolidated_errors[location_str]['errors'] = [error_msg]
for error_loc, error_data in consolidated_errors.items():
error_type = error_data['info']['type']
error_value = error_data['info']['value']
error_str += f"'{error_loc}' (type: '{error_type}', value: '{error_value}'):\n"
for error in error_data['errors']:
error_str += f" {error}\n"
return error_str
def format_list(items: list[str], width: int = 80) -> str:
"""
Format a list of strings into a grid-like display with dynamic column widths.
The function automatically calculates the optimal number of columns based on the maximum item width
and the desired total width. It properly handles Unicode characters by using their display width.
Args:
items (list[str]): List of strings to format
width (int, optional): Maximum width of the output in characters. Defaults to 80.
Returns:
str: A formatted string with items arranged in columns
Example:
>>> items = ["Item 1", "Long Item 2", "Item 3", "Item 4"]
>>> print(format_list(items, width=40))
Item 1 Long Item 2
Item 3 Item 4
"""
if not items:
return ""
# Calculate true display width for each item and add spacing
item_widths = [(s, wcswidth(s)) for s in items]
column_width = max(width for _, width in item_widths) + 4 # Add spacing between columns
columns = max(1, width // column_width) # At least one column
# Build rows with proper spacing
rows = []
for i in range(0, len(item_widths), columns):
row_items = item_widths[i:i + columns]
cols = []
for text, text_width in row_items:
padding = " " * (column_width - text_width)
cols.append(f"{text}{padding}")
rows.append("".join(cols).rstrip())
return "\n".join(rows)
def format_media_description(media_data: MediaBase, shortened: bool = False) -> str:
"""
Generate a short description string of a media object.
Args:
media_data (MediaBase): An object containing media data.
shortened (bool, optional): Whether to generate a shortened description. Defaults to False.
Returns:
str: A short description string of the media object.
"""
if isinstance(media_data, Movie):
release_year = (
media_data.release_date.year
if isinstance(media_data.release_date, dt.datetime)
else media_data.release_date
)
description_str = f"{media_data.name.strip()} [{release_year}]"
if media_data.id:
description_str += f" (ID: {media_data.id})"
return description_str
if isinstance(media_data, Series):
description_str = f"{media_data.series_name.strip()}"
if media_data.series_release_date:
if isinstance(media_data.series_release_date, dt.datetime):
description_str += f" [{media_data.series_release_date.year}]"
else:
description_str += f" [{media_data.series_release_date}]"
if media_data.id:
description_str += f" (ID: {media_data.id})"
return description_str
if isinstance(media_data, Season):
description_str = ""
if not shortened:
description_str = f"{media_data.series_name.strip()} - "
description_str += f"Season {media_data.season_number}"
if media_data.season_name:
description_str += f" - {media_data.season_name.strip()}"
if media_data.id:
description_str += f" (ID: {media_data.id})"
return description_str
if isinstance(media_data, Episode):
description_str = ""
if not shortened:
description_str = f"{media_data.series_name.strip()} - "
description_str += f"S{media_data.season_number:02d}E{media_data.episode_number:02d}"
if media_data.episode_name:
description_str += f" - {media_data.episode_name.strip()}"
if media_data.id:
description_str += f" (ID: {media_data.id})"
return description_str
raise ValueError(f"Unsupported media type: '{type(media_data)}'")
def format_release_name(title: str,
release_date: dt.datetime | int | None = None,
season_number: int | None = None,
episode_number: int | None = None,
episode_name: str | None = None,
media_source: str | None = None,
source_type: str | None = "WEB",
additional_info: str | list[str] | None = None,
language_code: str | None = None,
subtitles_type: SubtitlesType | None = None,
file_format: str | SubtitlesFormatType | None = None) -> str:
"""
Format a release name.
Args:
title (str): Media title.
release_date (int | None, optional): Release date (datetime), or year (int) of the media. Defaults to None.
season_number (int | None, optional): Season number. Defaults to None.
episode_number (int | None, optional): Episode number. Defaults to None.
episode_name (str | None, optional): Episode name. Defaults to None.
media_source (str | None, optional): Media source name (full or abbreviation). Defaults to None.
source_type(str | None, optional): General source type (WEB, BluRay, etc.). Defaults to None.
additional_info (list[str] | str | None, optional): Additional info to add to the file name. Defaults to None.
language_code (str | None, optional): Language code. Defaults to None.
subtitles_type (SubtitlesType | None, optional): Subtitles type. Defaults to None.
file_format (SubtitlesFormat | str | None, optional): File format to use. Defaults to None.
Returns:
str: Generated file name.
"""
file_name = slugify_title(title=title, separator=".")
if release_date is not None:
if isinstance(release_date, dt.datetime):
release_year = release_date.year
else:
release_year = release_date
file_name += f".{release_year}"
if season_number is not None and episode_number is not None:
file_name += f".S{season_number:02}E{episode_number:02}"
if episode_name is not None:
file_name += f".{slugify_title(title=episode_name, separator='.')}"
if media_source is not None:
file_name += f".{media_source}"
if source_type is not None:
file_name += f".{source_type}"
if additional_info is not None:
if isinstance(additional_info, list | tuple):
additional_info = '.'.join(additional_info)
file_name += f".{additional_info}"
if language_code is not None:
file_name += f".{language_code}"
if subtitles_type is not None:
file_name += f".{subtitles_type.value.lower()}"
sanitized_basename = sanitize_path_segment(file_name)
if file_format is not None:
if isinstance(file_format, SubtitlesFormatType):
file_format_str = file_format.value.file_extension
else:
file_format_str = file_format.lstrip('.')
return f"{sanitized_basename}.{file_format_str}"
return sanitized_basename
@lru_cache
def format_subtitles_description(language_code: str | None = None, language_name: str | None = None,
special_type: SubtitlesType | None = None) -> str:
"""
Format a subtitles description using its attributes.
Args:
language_code (str | None, optional): Language code. Defaults to None.
language_name (str | None, optional): Language name. Defaults to None.
special_type (SubtitlesType | None, optional): Subtitles type. Defaults to None.
Returns:
str: Formatted subtitles description.
Raises:
ValueError: If neither `language_code` nor `language_name` is provided.
"""
if language_name and language_code:
language_str = f"{language_name} ({language_code})"
elif result := (language_name or language_code):
language_str = result
else:
raise ValueError("Either 'language_code' or 'language_name' must be provided.")
if special_type:
language_str += f" [{special_type.value}]"
return language_str
def get_model_field(model: BaseModel | None, field: str, convert_to_dict: bool = False, **kwargs: Any) -> Any:
"""
Get a field from a Pydantic model.
Args:
model (BaseModel | None): A Pydantic model.
field (str): Field name to retrieve.
convert_to_dict (bool, optional): Whether to convert the field value to a dictionary. Defaults to False.
**kwargs: Additional keyword arguments to pass to the serialization method (`model_dump`).
Relevant only if `convert_to_dict` is True, and `field_value` is a Pydantic model.
Returns:
Any: The field value.
"""
if model and hasattr(model, field):
field_value = getattr(model, field)
if convert_to_dict and hasattr(field_value, 'model_dump'):
return field_value.model_dump(**kwargs)
return field_value
return None
def generate_non_conflicting_path(file_path: Path, has_extension: bool = True) -> Path:
"""
Generate a non-conflicting path for a file.
If the file already exists, a number will be added to the end of the file name.
Args:
file_path (Path): Path to a file.
has_extension (bool, optional): Whether the name of the file includes file extension. Defaults to True.
Returns:
Path: A non-conflicting file path.
"""
if isinstance(file_path, str):
file_path = Path(file_path)
if not file_path.exists():
return file_path
i = 1
while True:
if has_extension:
new_file_path = file_path.parent / f"{file_path.stem}-{i}{file_path.suffix}"
else:
new_file_path = file_path.parent / f"{file_path}-{i}"
if not new_file_path.exists():
return new_file_path
i += 1
def merge_dict_values(*dictionaries: dict) -> dict:
"""
A function for merging the values of multiple dictionaries using the same keys.
If a key already exists, the value will be added to a list of values mapped to that key.
Examples:
merge_dict_values({'a': 1, 'b': 3}, {'a': 2, 'b': 4}) -> {'a': [1, 2], 'b': [3, 4]}
merge_dict_values({'a': 1, 'b': 2}, {'a': 1, 'b': [2, 3]}) -> {'a': 1, 'b': [2, 3]}
Note:
This function support only merging of lists or single items (no tuples or other iterables),
and without any nesting (lists within lists).
Args:
*dictionaries (dict): Dictionaries to merge.
Returns:
dict: A merged dictionary.
"""
_dictionaries: list[dict] = [d for d in dictionaries if d]
if len(_dictionaries) == 0:
return {}
if len(_dictionaries) == 1:
return _dictionaries[0]
result: dict = {}
for _dict in _dictionaries:
for key, value in _dict.items():
if key in result:
if isinstance(result[key], list):
if isinstance(value, list):
result[key].extend(value)
else:
result[key].append(value)
else:
if isinstance(value, list):
result[key] = [result[key], *value]
else:
result[key] = [result[key], value]
else:
result[key] = value
return result
def raise_for_status(response: httpx.Response) -> None:
"""
Raise an exception if the response status code is invalid.
Uses 'response.raise_for_status()' internally, with additional logging.
Args:
response (httpx.Response): A response object.
"""
truncation_threshold = 1500
if not response.is_error:
return
if len(response.text) > truncation_threshold:
# Truncate the response as in some cases there could be an unexpected long HTML response
response_text = response.text[:truncation_threshold].rstrip() + " <TRUNCATED...>"
else:
response_text = response.text
logger.debug(f"Response status code: {response.status_code}")
if response.headers.get('Content-Type'):
logger.debug(f"Response type: {response.headers['Content-Type']}")
logger.debug(f"Response text: {response_text}")
response.raise_for_status()
def parse_url_params(url_params: str) -> dict:
"""
Parse GET parameters from a URL to a dictionary.
Args:
url_params (str): URL parameters. (e.g. 'param1=value1¶m2=value2')
Returns:
dict: A dictionary containing the URL parameters.
"""
url_params = url_params.split('?')[-1].rstrip('&')
params_list = url_params.split('&')
if len(params_list) == 0 or \
(len(params_list) == 1 and '=' not in params_list[0]):
return {}
return {key: value for key, value in (param.split('=') for param in params_list)}
@overload
def return_first_valid(*values: T | None, raise_error: Literal[True] = ...) -> T:
...
@overload
def return_first_valid(*values: T | None, raise_error: Literal[False] = ...) -> T | None:
...
def return_first_valid(*values: T | None, raise_error: bool = False) -> T | None:
"""
Return the first non-None value from a list of values.
Args:
*values (T): Values to check.
raise_error (bool, optional): Whether to raise an error if all values are None. Defaults to False.
Returns:
T | None: The first non-None value, or None if all values are None and `raise_error` is False.
Raises:
ValueError: If all values are None and `raise_error` is True.
"""
for value in values:
if value is not None:
return value
if raise_error:
raise ValueError("No valid value found.")
return None
def single_string_to_list(item: str | list[str]) -> list[str]:
"""
Convert a single string to a list containing the string.
If None is passed, an empty list will be returned.
Args:
item (str | list[str]): A string or a list of strings.
Returns:
list[str]: A list containing the string, or an empty list if None was passed.
"""
if item is None:
return []
if isinstance(item, list):
return item
return [item]
def split_subtitles_timestamp(timestamp: str) -> tuple[dt.time, dt.time]:
"""
Split a subtitles timestamp into start and end.
Args:
timestamp (str): A subtitles timestamp. For example: "00:00:00.000 --> 00:00:00.000"
Returns:
tuple(time, time): A tuple containing start and end times as a datetime object.
"""
# Support ',' character in timestamp's milliseconds (used in SubRip format).
timestamp = timestamp.replace(',', '.')
start_time, end_time = timestamp.split(" --> ")
return dt.time.fromisoformat(start_time), dt.time.fromisoformat(end_time)
_REMOVED_CHARS_SLUG = str.maketrans("", "", '<>():"?*')
_REMOVED_CHARS_FS = str.maketrans("", "", '<>:"/\\|?*')
_REPLACE_WITH_SEPARATOR = [" - ", " & ", ",", ":", "|", "/", " "]
@lru_cache
def slugify_title(title: str, separator: str = " ") -> str:
"""
Normalize a title into a slug-like string for creating name components.
This function is for normalization, not for filesystem safety.
Args:
title (str): A media title.
separator (str, optional): A separator to use between words. Defaults to " ".
Returns:
str: A slugified title.
"""
title = title.strip()
title = title.replace("…", "...")
# Replace multi-character sequences first
for item in _REPLACE_WITH_SEPARATOR:
if separator == " " and item in (" & ", " - "):
continue
if item == " & ":
title = title.replace(item, f"{separator}&{separator}")
else:
title = title.replace(item, separator)
# Remove invalid characters for a slug
title = title.translate(_REMOVED_CHARS_SLUG)
# Replace multiple separators with a single one
if separator:
title = re.sub(f"[{re.escape(separator)}]+", separator, title)
return title
@lru_cache
def sanitize_path_segment(segment: str, platform: str | None = None) -> str:
"""
Sanitize a file or directory name (path segment) for a given OS.
Args:
segment (str): The path segment to sanitize (file or directory name).
platform (str | None, optional): Target platform ('win32', 'linux', 'darwin').
Defaults to 'sys.platform' (current platform).
Returns:
str: A sanitized path segment safe for use on the target filesystem.
"""
if platform is None:
platform = sys.platform
# Remove characters illegal on all filesystems
segment = segment.translate(_REMOVED_CHARS_FS)
if platform == "win32":
# On Windows, remove trailing dots and spaces
segment = segment.rstrip(". ")
# Handle reserved device names (match first segment before dot)
base_name, _, _ = segment.partition('.')
if base_name.upper() in WINDOWS_RESERVED_FILE_NAMES:
segment = f"_{segment}"
# Ensure the segment is not empty
if not segment:
return "_"
return segment
================================================
FILE: pyproject.toml
================================================
[project]
name = "isubrip"
version = "2.6.8"
description = "A Python package for scraping and downloading subtitles from AppleTV / iTunes movie pages."
authors = [
{name = "Michael Yochpaz"}
]
readme = "README.md"
keywords = [
"iTunes",
"AppleTV",
"movies",
"subtitles",
"scrape",
"scraper",
"download",
"m3u8"
]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: End Users/Desktop",
"Operating System :: Microsoft :: Windows",
"Operating System :: MacOS",
"Operating System :: POSIX :: Linux",
"Topic :: Utilities",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
requires-python = ">= 3.10"
dependencies = [
"httpx[http2]>=0.28.1",
"m3u8>=6.0.0",
"pydantic>=2.12.0",
"pydantic-settings>=2.11.0",
"pygments>=2.19.1", # Used by 'rich'. Specified here as version 2.18 appears to cause issues.
"rich>=14.2.0",
"tomli>=2.3.0",
"wcwidth>=0.2.14",
]
[project.urls]
Homepage = "https://github.com/MichaelYochpaz/iSubRip"
Repository = "https://github.com/MichaelYochpaz/iSubRip"
Issues = "https://github.com/MichaelYochpaz/iSubRip/issues"
Changelog = "https://github.com/MichaelYochpaz/iSubRip/blob/main/CHANGELOG.md"
[project.scripts]
isubrip = "isubrip.__main__:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.uv]
dev-dependencies = [
"mypy>=1.14.1",
"pyperf>=2.9.0",
"pytest>=8.4.2",
"ruff>=0.9.3",
]
[tool.mypy]
check_untyped_defs = true
disallow_untyped_defs = true
explicit_package_bases = true
ignore_missing_imports = true
python_version = "3.10"
warn_return_any = true
exclude = ["tests/mock_data"]
plugins = ["pydantic.mypy"]
[tool.ruff]
line-length = 120
target-version = "py310"
[tool.ruff.lint]
select = [
"ARG",
"ASYNC",
"B",
"C4",
"COM",
"E",
"F",
"FA",
"I",
"INP",
"ISC",
"N",
"PIE",
"PGH",
"PT",
"PTH",
"Q",
"RSE",
"RET",
"RUF",
"S",
"SIM",
"SLF",
"T20",
"TCH",
"TID",
"TRY",
"UP",
]
ignore = [
"C416",
"Q000",
"RUF010",
"RUF012",
"SIM108",
"TD002",
"TD003",
"TRY003",
]
unfixable = ["ARG"]
[tool.ruff.lint.per-file-ignores]
"tests/*.py" = ["S101"]
[tool.ruff.lint.flake8-tidy-imports]
ban-relative-imports = "all"
[tool.ruff.lint.flake8-quotes]
docstring-quotes = "double"
[tool.ruff.lint.isort]
force-sort-within-sections = true
[tool.ruff.lint.pyupgrade]
keep-runtime-typing = true
================================================
FILE: tests/.gitignore
================================================
mock_data/
================================================
FILE: tests/__init__.py
================================================
================================================
FILE: tests/benchmarks/__init__.py
================================================
================================================
FILE: tests/benchmarks/download_benchmark.py
================================================
import pyperf
from tests.benchmarks.download_benchmark_module import benchmark
# Note: Run this script with the `-o` flag to output results to a file.
def main() -> None:
runner = pyperf.Runner(
processes=5,
values=10,
warmups=1,
)
runner.bench_async_func(
name="Download Benchmark",
func=benchmark,
)
if __name__ == "__main__":
main()
================================================
FILE: tests/benchmarks/download_benchmark_module.py
================================================
import asyncio
import logging
from pathlib import Path
from unittest.mock import patch
from isubrip.cli import console
from isubrip.commands.download import download
from isubrip.logger import logger, setup_loggers
from isubrip.utils import TemporaryDirectory
from tests.tools.mock_loader import MockLoader
async def benchmark() -> None:
setup_loggers(
stdout_loglevel=logging.INFO,
stdout_console=console,
logfile_output=False,
)
url = "https://tv.apple.com/il/movie/interstellar/umc.cmc.1vrwat5k1ucm5k42q97ioqyq3"
mock_data_path = Path("tests/mock_data/appletv/il/umc.cmc.1vrwat5k1ucm5k42q97ioqyq3")
mock_loader = MockLoader(mock_data_path, logger=logger)
with TemporaryDirectory() as temp_dir:
logger.info(f"Temporary directory created at: {temp_dir}")
with patch("httpx.AsyncClient.send", side_effect=mock_loader.mock_send_handler):
await download(url, download_path=temp_dir)
if __name__ == "__main__":
asyncio.run(benchmark())
================================================
FILE: tests/test_utils.py
================================================
import datetime as dt
import pytest
from isubrip.data_structures import Episode, Movie, Season, Series, SubtitlesFormatType, SubtitlesType
from isubrip.utils import (
format_media_description,
format_release_name,
sanitize_path_segment,
slugify_title,
)
class TestSlugifyTitle:
@pytest.mark.parametrize(
("input_title", "separator", "expected_output"),
[
("The Lord of the Rings: The Fellowship of the Ring", ".",
"The.Lord.of.the.Rings.The.Fellowship.of.the.Ring"),
("The Lord of the Rings: The Fellowship of the Ring", " ",
"The Lord of the Rings The Fellowship of the Ring"),
("Once Upon a Time... in Hollywood", ".", "Once.Upon.a.Time.in.Hollywood"),
("Once Upon a Time... in Hollywood", " ", "Once Upon a Time... in Hollywood"),
("Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb?", ".",
"Dr.Strangelove.or.How.I.Learned.to.Stop.Worrying.and.Love.the.Bomb"),
("Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb?", " ",
"Dr. Strangelove or How I Learned to Stop Worrying and Love the Bomb"),
("Mission: Impossible - The Final Reckoning", ".", "Mission.Impossible.The.Final.Reckoning"),
("Mission: Impossible - The Final Reckoning", " ", "Mission Impossible - The Final Reckoning"),
("Deadpool & Wolverine", ".", "Deadpool.&.Wolverine"),
("Deadpool & Wolverine", " ", "Deadpool & Wolverine"),
("50/50", ".", "50.50"),
("50/50", " ", "50 50"),
("50 / 50", ".", "50.50"),
("50 / 50", " ", "50 50"),
("V/H/S", ".", "V.H.S"),
("V/H/S", " ", "V H S"),
("What If...?", ".", "What.If."),
("What If...?", " ", "What If..."),
],
)
def test_slugify_title(self, input_title: str, separator: str, expected_output: str) -> None:
assert slugify_title(title=input_title, separator=separator) == expected_output
class TestSanitizePath:
@pytest.mark.parametrize(
("name", "expected_name"),
[
("A/B/C", "ABC"),
("A<B>C", "ABC"),
('A"B"C', "ABC"),
("A:B:C", "ABC"),
("A|B|C", "ABC"),
("A?B?C", "ABC"),
("A*B*C", "ABC"),
],
)
def test_sanitize_common_illegal_chars(self, name: str, expected_name: str) -> None:
assert sanitize_path_segment(name) == expected_name
@pytest.mark.parametrize(
("name", "expected_unix", "expected_windows"),
[
("name.", "name.", "name"),
("name..", "name..", "name"),
("name ", "name ", "name"),
(" name", " name", " name"),
("name. ", "name. ", "name"),
("COM1", "COM1", "_COM1"),
("Con.Air", "Con.Air", "_Con.Air"),
("aux.txt", "aux.txt", "_aux.txt"),
("", "_", "_"),
],
)
def test_sanitize_platform_specific(self, name: str, expected_unix: str, expected_windows: str) -> None:
# Test Unix-like behavior
assert sanitize_path_segment(name, platform='linux') == expected_unix
# Test Windows behavior
assert sanitize_path_segment(name, platform='win32') == expected_windows
class TestFormatMediaDescription:
def test_movie_with_datetime_and_id(self) -> None:
movie = Movie(name="Inception", release_date=dt.datetime(2010, 7, 16), id="ID123")
assert format_media_description(media_data=movie) == "Inception [2010] (ID: ID123)"
def test_series_with_year_no_id(self) -> None:
series = Series(series_name="The Office", series_release_date=2005)
assert format_media_description(media_data=series) == "The Office [2005]"
def test_season_full_with_name_and_id(self) -> None:
season = Season(series_name="True Detective", series_release_date=2024,
season_number=4, season_name="Night Country", id="ID321")
assert format_media_description(media_data=season) == "True Detective - Season 4 - Night Country (ID: ID321)"
def test_season_full_with_name_and_id_and_extra_spaces(self) -> None:
season = Season(series_name=" True Detective ", series_release_date=2024,
season_number=4, season_name=" Night Country ", id="ID321")
assert format_media_description(media_data=season) == "True Detective - Season 4 - Night Country (ID: ID321)"
def test_season_shortened_no_id(self) -> None:
season = Season(series_name="Stranger Things", season_number=3)
assert format_media_description(media_data=season, shortened=True) == "Season 3"
def test_episode_full_with_name_and_id(self) -> None:
ep = Episode(series_name="Breaking Bad",
season_number=5, episode_number=14, episode_name="Ozymandias", id="ID111")
assert format_media_description(media_data=ep) == "Breaking Bad - S05E14 - Ozymandias (ID: ID111)"
def test_episode_shortened_no_id(self) -> None:
ep = Episode(series_name="Breaking Bad",
season_number=5, episode_number=14, episode_name="O
gitextract_mbnetjyr/
├── .github/
│ └── ISSUE_TEMPLATE/
│ ├── 1_issue_report.yml
│ ├── 2_feature_request.yml
│ ├── 3_question.yml
│ └── config.yml
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── README.md
├── example-config.toml
├── isubrip/
│ ├── __init__.py
│ ├── __main__.py
│ ├── cli.py
│ ├── commands/
│ │ ├── __init__.py
│ │ └── download.py
│ ├── config.py
│ ├── constants.py
│ ├── data_structures.py
│ ├── logger.py
│ ├── scrapers/
│ │ ├── __init__.py
│ │ ├── appletv_scraper.py
│ │ ├── itunes_scraper.py
│ │ └── scraper.py
│ ├── subtitle_formats/
│ │ ├── __init__.py
│ │ ├── subrip.py
│ │ ├── subtitles.py
│ │ └── webvtt.py
│ ├── ui.py
│ └── utils.py
├── pyproject.toml
└── tests/
├── .gitignore
├── __init__.py
├── benchmarks/
│ ├── __init__.py
│ ├── download_benchmark.py
│ └── download_benchmark_module.py
├── test_utils.py
└── tools/
├── __init__.py
├── generate_mock_data.py
└── mock_loader.py
SYMBOL INDEX (234 symbols across 20 files)
FILE: isubrip/__main__.py
function main (line 44) | def main() -> None:
function _main (line 66) | async def _main() -> None:
function check_for_updates (line 126) | def check_for_updates(current_package_version: str) -> None:
function handle_log_rotation (line 160) | def handle_log_rotation(rotation_size: int) -> None:
function parse_config (line 174) | def parse_config(config_file_location: Path) -> Config:
function update_settings (line 211) | def update_settings(config: Config) -> None:
FILE: isubrip/cli.py
function conditional_live (line 13) | def conditional_live(renderable: Any) -> Iterator[Live | None]:
FILE: isubrip/commands/download.py
function download (line 34) | async def download(*urls: str,
function download_media (line 102) | async def download_media(scraper: Scraper, media_item: MediaData, downlo...
function download_media_item (line 137) | async def download_media_item(scraper: Scraper, media_item: Movie | Epis...
function download_subtitles (line 194) | async def download_subtitles(scraper: Scraper, media_data: Movie | Episo...
FILE: isubrip/config.py
class ConfigCategory (line 14) | class ConfigCategory(BaseModel, ABC):
class GeneralCategory (line 24) | class GeneralCategory(ConfigCategory):
class DownloadsCategory (line 31) | class DownloadsCategory(ConfigCategory):
method assure_path_exists (line 39) | def assure_path_exists(cls, value: Path) -> Path:
class WebVTTSubcategory (line 55) | class WebVTTSubcategory(ConfigCategory):
class SubtitlesCategory (line 59) | class SubtitlesCategory(ConfigCategory):
class ScrapersCategory (line 66) | class ScrapersCategory(ConfigCategory):
class Config (line 86) | class Config(BaseSettings):
method settings_customise_sources (line 97) | def settings_customise_sources(
FILE: isubrip/constants.py
function data_folder_path (line 15) | def data_folder_path() -> Path:
function temp_folder_path (line 19) | def temp_folder_path() -> Path:
function user_config_file_path (line 23) | def user_config_file_path() -> Path:
function log_files_path (line 28) | def log_files_path() -> Path:
FILE: isubrip/data_structures.py
class SubtitlesDownloadResults (line 21) | class SubtitlesDownloadResults(NamedTuple):
class SubtitlesFormat (line 37) | class SubtitlesFormat(BaseModel):
class SubtitlesFormatType (line 49) | class SubtitlesFormatType(Enum):
class SubtitlesType (line 61) | class SubtitlesType(Enum):
class SubtitlesData (line 73) | class SubtitlesData(BaseModel):
class ConfigDict (line 92) | class ConfigDict:
class MediaBase (line 96) | class MediaBase(BaseModel, ABC):
class Movie (line 100) | class Movie(MediaBase):
class Episode (line 126) | class Episode(MediaBase):
class Season (line 160) | class Season(MediaBase):
class Series (line 186) | class Series(MediaBase):
class ScrapedMediaResponse (line 206) | class ScrapedMediaResponse(BaseModel, Generic[MediaData]):
FILE: isubrip/logger.py
function set_logger (line 28) | def set_logger(_logger: logging.Logger) -> None:
class CustomStdoutFormatter (line 39) | class CustomStdoutFormatter(RichHandler):
method __init__ (line 52) | def __init__(self, console: Console | None = None, debug_mode: bool = ...
method emit (line 73) | def emit(self, record: logging.LogRecord) -> None:
method format (line 85) | def format(self, record: logging.LogRecord) -> str:
class CustomLogFileFormatter (line 105) | class CustomLogFileFormatter(logging.Formatter):
method __init__ (line 109) | def __init__(self) -> None:
method _remove_rich_markup (line 121) | def _remove_rich_markup(text: str) -> str:
method format (line 135) | def format(self, record: logging.LogRecord) -> str:
function setup_loggers (line 170) | def setup_loggers(stdout_output: bool = True, stdout_console: Console | ...
FILE: isubrip/scrapers/appletv_scraper.py
class AppleTVScraper (line 18) | class AppleTVScraper(HLSScraper):
class Channel (line 56) | class Channel(Enum):
method __init__ (line 70) | def __init__(self, *args: Any, **kwargs: Any) -> None:
method _decide_locale (line 74) | def _decide_locale(self, preferred_locales: str | list[str], default_l...
method _fetch_api_data (line 98) | async def _fetch_api_data(self, storefront_id: str, endpoint: str, add...
method _fetch_request_params (line 135) | async def _fetch_request_params(self, storefront_id: str) -> dict[str,...
method _get_configuration_data (line 173) | async def _get_configuration_data(self, storefront_id: str) -> dict:
method _map_playables_by_channel (line 196) | def _map_playables_by_channel(self, playables: list[dict]) -> dict[str...
method get_movie_data (line 214) | async def get_movie_data(self, storefront_id: str, movie_id: str) -> S...
method _extract_itunes_movie_data (line 245) | def _extract_itunes_movie_data(self, playable_data: dict) -> Movie:
method get_episode_data (line 284) | async def get_episode_data(self, storefront_id: str, episode_id: str) ...
method get_season_data (line 287) | async def get_season_data(self, storefront_id: str, season_id: str, sh...
method get_show_data (line 290) | async def get_show_data(self, storefront_id: str, show_id: str) -> Scr...
method get_data (line 293) | async def get_data(self, url: str) -> ScrapedMediaResponse:
FILE: isubrip/scrapers/itunes_scraper.py
class ItunesScraper (line 20) | class ItunesScraper(HLSScraper):
method __init__ (line 35) | def __init__(self, *args: Any, **kwargs: Any) -> None:
method get_data (line 42) | async def get_data(self, url: str) -> ScrapedMediaResponse[Movie]:
method parse_language_name (line 94) | def parse_language_name(media_data: Media) -> str | None:
FILE: isubrip/scrapers/scraper.py
class ScraperConfigBase (line 44) | class ScraperConfigBase(BaseModel, ABC):
class DefaultScraperConfig (line 68) | class DefaultScraperConfig(ScraperConfigBase):
class ScraperConfigSubcategory (line 87) | class ScraperConfigSubcategory(BaseModel, ABC):
class Scraper (line 96) | class Scraper(ABC, metaclass=SingletonMeta):
class ScraperConfig (line 128) | class ScraperConfig(ScraperConfigBase):
method __init__ (line 150) | def __init__(self, timeout: int | float | None = None, user_agent: str...
method _increment_requests_counter (line 206) | def _increment_requests_counter(self, request: httpx.Request) -> None:...
method _async_increment_requests_counter (line 209) | async def _async_increment_requests_counter(self, request: httpx.Reque...
method requests_count (line 213) | def requests_count(self) -> int:
method match_url (line 218) | def match_url(cls, url: str, raise_error: Literal[True] = ...) -> re.M...
method match_url (line 223) | def match_url(cls, url: str, raise_error: Literal[False] = ...) -> re....
method match_url (line 227) | def match_url(cls, url: str, raise_error: bool = False) -> re.Match | ...
method async_close (line 254) | async def async_close(self) -> None:
method get_data (line 258) | async def get_data(self, url: str) -> ScrapedMediaResponse:
method download_subtitles (line 270) | async def download_subtitles(self, media_data: PlaylistMediaItem, subr...
method find_matching_media (line 286) | def find_matching_media(self, main_playlist: MainPlaylist,
method find_matching_subtitles (line 301) | def find_matching_subtitles(self, main_playlist: MainPlaylist,
method load_playlist (line 316) | async def load_playlist(self, url: str | list[str], headers: dict | No...
method detect_subtitles_type (line 333) | def detect_subtitles_type(subtitles_media: PlaylistMediaItem) -> Subti...
method format_subtitles_description (line 347) | def format_subtitles_description(cls, subtitles_media: PlaylistMediaIt...
class HLSScraper (line 362) | class HLSScraper(Scraper, ABC):
class M3U8Attribute (line 364) | class M3U8Attribute(Enum):
class ScraperConfig (line 405) | class ScraperConfig(Scraper.ScraperConfig):
method __init__ (line 411) | def __init__(self, playlist_filters: dict[str, str | list[str] | None]...
method parse_language_name (line 425) | def parse_language_name(media_data: m3u8.Media) -> str | None:
method load_playlist (line 439) | async def load_playlist(self, url: str | list[str], headers: dict[str,...
method detect_subtitles_type (line 468) | def detect_subtitles_type(subtitles_media: m3u8.Media) -> SubtitlesTyp...
method download_subtitles (line 486) | async def download_subtitles(self, media_data: m3u8.Media, subrip_conv...
method download_segments (line 534) | async def download_segments(self, playlist: m3u8.M3U8) -> list[bytes]:
method find_matching_media (line 554) | def find_matching_media(self, main_playlist: m3u8.M3U8,
method find_matching_subtitles (line 602) | def find_matching_subtitles(self, main_playlist: m3u8.M3U8,
method format_subtitles_description (line 612) | def format_subtitles_description(cls, subtitles_media: m3u8.Media) -> ...
class ScraperFactory (line 620) | class ScraperFactory:
method get_initialized_scrapers (line 626) | def get_initialized_scrapers(cls) -> list[Scraper]:
method get_scraper_classes (line 636) | def get_scraper_classes(cls) -> list[type[Scraper]]:
method _get_scraper_instance (line 664) | def _get_scraper_instance(cls, scraper_class: type[ScraperT], kwargs: ...
method get_scraper_instance (line 698) | def get_scraper_instance(cls, scraper_class: type[ScraperT], scraper_i...
method get_scraper_instance (line 705) | def get_scraper_instance(cls, scraper_class: type[ScraperT], scraper_i...
method get_scraper_instance (line 712) | def get_scraper_instance(cls, scraper_class: None = ..., scraper_id: s...
method get_scraper_instance (line 719) | def get_scraper_instance(cls, scraper_class: None = ..., scraper_id: s...
method get_scraper_instance (line 725) | def get_scraper_instance(cls, scraper_class: type[Scraper] | None = No...
class ScraperError (line 782) | class ScraperError(Exception):
class DownloadError (line 786) | class DownloadError(ScraperError):
class PlaylistLoadError (line 790) | class PlaylistLoadError(ScraperError):
class SubtitlesDownloadError (line 794) | class SubtitlesDownloadError(ScraperError):
method __init__ (line 795) | def __init__(self, language_code: str | None, language_name: str | Non...
FILE: isubrip/subtitle_formats/subrip.py
class SubRipCaptionBlock (line 9) | class SubRipCaptionBlock(SubtitlesCaptionBlock):
method __eq__ (line 11) | def __eq__(self, other: Any) -> bool:
method __str__ (line 15) | def __str__(self) -> str:
method to_srt (line 24) | def to_srt(self) -> SubRipCaptionBlock:
class SubRipSubtitles (line 28) | class SubRipSubtitles(Subtitles[SubRipCaptionBlock]):
method _dumps (line 32) | def _dumps(self) -> str:
method _loads (line 40) | def _loads(self, data: str) -> None:
FILE: isubrip/subtitle_formats/subtitles.py
class SubtitlesBlock (line 23) | class SubtitlesBlock(ABC):
method __init__ (line 31) | def __init__(self) -> None:
method __copy__ (line 35) | def __copy__(self) -> SubtitlesBlock:
method __eq__ (line 39) | def __eq__(self, other: Any) -> bool:
method __str__ (line 43) | def __str__(self) -> str:
class SubtitlesCaptionBlock (line 47) | class SubtitlesCaptionBlock(SubtitlesBlock, ABC):
method __init__ (line 57) | def __init__(self, start_time: time, end_time: time, payload: str):
method __copy__ (line 71) | def __copy__(self) -> SubtitlesCaptionBlock:
method fix_rtl (line 76) | def fix_rtl(self) -> None:
method to_srt (line 91) | def to_srt(self) -> SubRipCaptionBlock:
class Subtitles (line 101) | class Subtitles(Generic[SubtitlesBlockT], ABC):
method __init__ (line 115) | def __init__(self, data: bytes | None, language_code: str, encoding: s...
method __add__ (line 136) | def __add__(self: SubtitlesT, obj: SubtitlesBlockT | SubtitlesT) -> Su...
method __copy__ (line 157) | def __copy__(self: SubtitlesT) -> SubtitlesT:
method __eq__ (line 165) | def __eq__(self, other: Any) -> bool:
method __str__ (line 168) | def __str__(self) -> str:
method _dump (line 171) | def _dump(self) -> bytes:
method _dumps (line 181) | def _dumps(self) -> str:
method _load (line 190) | def _load(self, data: bytes) -> None:
method _loads (line 201) | def _loads(self, data: str) -> None:
method dump (line 210) | def dump(self) -> bytes:
method dumps (line 224) | def dumps(self) -> str:
method add_blocks (line 238) | def add_blocks(self: SubtitlesT,
method append_subtitles (line 266) | def append_subtitles(self: SubtitlesT,
method polish (line 285) | def polish(self: SubtitlesT,
method modified (line 321) | def modified(self) -> bool:
method to_srt (line 330) | def to_srt(self) -> SubRipSubtitles:
function split_timestamp (line 350) | def split_timestamp(timestamp: str) -> tuple[time, time]:
FILE: isubrip/subtitle_formats/webvtt.py
class WebVTTBlock (line 24) | class WebVTTBlock(SubtitlesBlock, metaclass=ABCMeta):
class WebVTTCaptionBlock (line 31) | class WebVTTCaptionBlock(SubtitlesCaptionBlock, WebVTTBlock):
method __init__ (line 37) | def __init__(self, start_time: time, end_time: time, payload: str, set...
method __copy__ (line 51) | def __copy__(self) -> WebVTTCaptionBlock:
method to_srt (line 57) | def to_srt(self) -> SubRipCaptionBlock:
method __eq__ (line 72) | def __eq__(self, other: Any) -> bool:
method __str__ (line 76) | def __str__(self) -> str:
class WebVTTCommentBlock (line 94) | class WebVTTCommentBlock(WebVTTBlock):
method __init__ (line 98) | def __init__(self, payload: str, inline: bool = False) -> None:
method __copy__ (line 109) | def __copy__(self) -> WebVTTCommentBlock:
method __eq__ (line 114) | def __eq__(self, other: Any) -> bool:
method __str__ (line 117) | def __str__(self) -> str:
class WebVTTStyleBlock (line 127) | class WebVTTStyleBlock(WebVTTBlock):
method __init__ (line 131) | def __init__(self, payload: str) -> None:
method __copy__ (line 141) | def __copy__(self) -> WebVTTStyleBlock:
method __eq__ (line 146) | def __eq__(self, other: Any) -> bool:
method __str__ (line 149) | def __str__(self) -> str:
class WebVTTRegionBlock (line 153) | class WebVTTRegionBlock(WebVTTBlock):
method __init__ (line 157) | def __init__(self, payload: str) -> None:
method __copy__ (line 167) | def __copy__(self) -> WebVTTRegionBlock:
method __eq__ (line 172) | def __eq__(self, other: Any) -> bool:
method __str__ (line 175) | def __str__(self) -> str:
class WebVTTSubtitles (line 179) | class WebVTTSubtitles(Subtitles[WebVTTBlock]):
method _dumps (line 183) | def _dumps(self) -> str:
method _loads (line 197) | def _loads(self, data: str) -> None:
method append_subtitles (line 279) | def append_subtitles(self: WebVTTSubtitles,
method remove_head_blocks (line 294) | def remove_head_blocks(self) -> None:
FILE: isubrip/ui.py
class MinsAndSecsTimeElapsedColumn (line 13) | class MinsAndSecsTimeElapsedColumn(TimeElapsedColumn):
method render (line 16) | def render(self, task: Task) -> Text:
FILE: isubrip/utils.py
class SingletonMeta (line 38) | class SingletonMeta(ABCMeta):
method __call__ (line 45) | def __call__(cls, *args: Any, **kwargs: Any) -> object:
class TemporaryDirectory (line 52) | class TemporaryDirectory:
method __init__ (line 60) | def __init__(self, directory_name: str | None = None):
method __enter__ (line 68) | def __enter__(self) -> Path:
method __exit__ (line 79) | def __exit__(self, exc_type: type[BaseException] | None,
method cleanup (line 84) | def cleanup(self) -> None:
function convert_epoch_to_datetime (line 96) | def convert_epoch_to_datetime(epoch_timestamp: int) -> dt.datetime:
function convert_log_level (line 112) | def convert_log_level(log_level: str) -> int:
function download_subtitles_to_file (line 132) | def download_subtitles_to_file(media_data: Movie | Episode, subtitles_da...
function format_config_validation_error (line 184) | def format_config_validation_error(exc: ValidationError) -> str:
function format_list (line 242) | def format_list(items: list[str], width: int = 80) -> str:
function format_media_description (line 283) | def format_media_description(media_data: MediaBase, shortened: bool = Fa...
function format_release_name (line 357) | def format_release_name(title: str,
function format_subtitles_description (line 433) | def format_subtitles_description(language_code: str | None = None, langu...
function get_model_field (line 464) | def get_model_field(model: BaseModel | None, field: str, convert_to_dict...
function generate_non_conflicting_path (line 489) | def generate_non_conflicting_path(file_path: Path, has_extension: bool =...
function merge_dict_values (line 523) | def merge_dict_values(*dictionaries: dict) -> dict:
function raise_for_status (line 571) | def raise_for_status(response: httpx.Response) -> None:
function parse_url_params (line 601) | def parse_url_params(url_params: str) -> dict:
function return_first_valid (line 622) | def return_first_valid(*values: T | None, raise_error: Literal[True] = ....
function return_first_valid (line 627) | def return_first_valid(*values: T | None, raise_error: Literal[False] = ...
function return_first_valid (line 631) | def return_first_valid(*values: T | None, raise_error: bool = False) -> ...
function single_string_to_list (line 654) | def single_string_to_list(item: str | list[str]) -> list[str]:
function split_subtitles_timestamp (line 674) | def split_subtitles_timestamp(timestamp: str) -> tuple[dt.time, dt.time]:
function slugify_title (line 697) | def slugify_title(title: str, separator: str = " ") -> str:
function sanitize_path_segment (line 732) | def sanitize_path_segment(segment: str, platform: str | None = None) -> ...
FILE: tests/benchmarks/download_benchmark.py
function main (line 7) | def main() -> None:
FILE: tests/benchmarks/download_benchmark_module.py
function benchmark (line 13) | async def benchmark() -> None:
FILE: tests/test_utils.py
class TestSlugifyTitle (line 14) | class TestSlugifyTitle:
method test_slugify_title (line 49) | def test_slugify_title(self, input_title: str, separator: str, expecte...
class TestSanitizePath (line 53) | class TestSanitizePath:
method test_sanitize_common_illegal_chars (line 66) | def test_sanitize_common_illegal_chars(self, name: str, expected_name:...
method test_sanitize_platform_specific (line 83) | def test_sanitize_platform_specific(self, name: str, expected_unix: st...
class TestFormatMediaDescription (line 91) | class TestFormatMediaDescription:
method test_movie_with_datetime_and_id (line 92) | def test_movie_with_datetime_and_id(self) -> None:
method test_series_with_year_no_id (line 96) | def test_series_with_year_no_id(self) -> None:
method test_season_full_with_name_and_id (line 100) | def test_season_full_with_name_and_id(self) -> None:
method test_season_full_with_name_and_id_and_extra_spaces (line 105) | def test_season_full_with_name_and_id_and_extra_spaces(self) -> None:
method test_season_shortened_no_id (line 110) | def test_season_shortened_no_id(self) -> None:
method test_episode_full_with_name_and_id (line 114) | def test_episode_full_with_name_and_id(self) -> None:
method test_episode_shortened_no_id (line 119) | def test_episode_shortened_no_id(self) -> None:
class TestFormatReleaseName (line 125) | class TestFormatReleaseName:
method test_movie_with_source_and_web_default (line 126) | def test_movie_with_source_and_web_default(self) -> None:
method test_movie_with_source_type_none (line 133) | def test_movie_with_source_type_none(self) -> None:
method test_episode_with_source_web (line 141) | def test_episode_with_source_web(self) -> None:
method test_episode_with_name_included (line 149) | def test_episode_with_name_included(self) -> None:
method test_additional_info_language_and_subtitles_type_and_format_enum (line 158) | def test_additional_info_language_and_subtitles_type_and_format_enum(s...
method test_movie_zip_with_source_and_web (line 169) | def test_movie_zip_with_source_and_web(self) -> None:
FILE: tests/tools/generate_mock_data.py
class MockDataGenerator (line 35) | class MockDataGenerator(ABC):
method __init_subclass__ (line 39) | def __init_subclass__(cls, **kwargs: Any) -> None:
method __init__ (line 54) | def __init__(self, scraper: Scraper):
method output_path (line 58) | def output_path(self, url: str) -> Path:
method generate (line 69) | async def generate(self, url: str, languages: list[str] | None = None,...
class AppleTVMockDataGenerator (line 168) | class AppleTVMockDataGenerator(MockDataGenerator):
method output_path (line 172) | def output_path(self, url: str) -> Path:
function main (line 179) | async def main() -> None:
FILE: tests/tools/mock_loader.py
class MockLoader (line 11) | class MockLoader:
method __init__ (line 15) | def __init__(self, mock_data_dir: Path | str, logger: logging.Logger |...
method mock_send_handler (line 48) | async def mock_send_handler(self, request: httpx.Request, *args: Any, ...
Condensed preview — 38 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (219K chars).
[
{
"path": ".github/ISSUE_TEMPLATE/1_issue_report.yml",
"chars": 3855,
"preview": "name: Bug / Issue Report\ndescription: Report a bug or an issue.\ntitle: \"[Issue]: \"\nlabels: [bug]\nbody:\n - type: markdow"
},
{
"path": ".github/ISSUE_TEMPLATE/2_feature_request.yml",
"chars": 870,
"preview": "name: Feature Request\ndescription: Request a new feature or improvement to an existing one.\ntitle: \"[Feature Request]: \""
},
{
"path": ".github/ISSUE_TEMPLATE/3_question.yml",
"chars": 471,
"preview": "name: Ask a question\ndescription: Ask a question regarding iSubRip.\ntitle: \"[Question]: \"\nlabels: [question]\nbody:\n - t"
},
{
"path": ".github/ISSUE_TEMPLATE/config.yml",
"chars": 27,
"preview": "blank_issues_enabled: false"
},
{
"path": ".gitignore",
"chars": 2487,
"preview": "# From: https://github.com/github/gitignore/blob/main/Python.gitignore\n\n# Byte-compiled / optimized / DLL files\n__pycach"
},
{
"path": "CHANGELOG.md",
"chars": 17903,
"preview": "# Changelog\n## 2.6.8 [2025-10-14]\n### Changes:\n* Removed Python 3.9 support, added Python 3.14 support.\n\n### Bug Fixes:\n"
},
{
"path": "LICENSE",
"chars": 1072,
"preview": "MIT License\n\nCopyright (c) 2022 Michael Yochpaz\n\nPermission is hereby granted, free of charge, to any person obtaining a"
},
{
"path": "README.md",
"chars": 6065,
"preview": "> [!CAUTION]\n> iSubRip is currently not working due to changes on Apple's backend. \n> The future of this project is cur"
},
{
"path": "example-config.toml",
"chars": 4554,
"preview": "# ---------------- ⚠️ IMPORTANT - READ BEFORE USING ⚠️ ----------------\n# This is an example config file with all availa"
},
{
"path": "isubrip/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "isubrip/__main__.py",
"chars": 7912,
"preview": "from __future__ import annotations\n\nimport asyncio\nimport logging\nimport sys\nfrom typing import TYPE_CHECKING\n\nimport ht"
},
{
"path": "isubrip/cli.py",
"chars": 1102,
"preview": "from collections.abc import Iterator\nfrom contextlib import contextmanager\nfrom typing import Any\n\nfrom rich.console imp"
},
{
"path": "isubrip/commands/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "isubrip/commands/download.py",
"chars": 16822,
"preview": "from __future__ import annotations\n\nfrom pathlib import Path\nimport shutil\n\nfrom rich.console import Group\nfrom rich.pro"
},
{
"path": "isubrip/config.py",
"chars": 3709,
"preview": "from __future__ import annotations\n\nfrom abc import ABC\nfrom pathlib import Path\nfrom typing import TYPE_CHECKING, Liter"
},
{
"path": "isubrip/constants.py",
"chars": 1013,
"preview": "from __future__ import annotations\n\nfrom functools import lru_cache\nfrom pathlib import Path\nfrom tempfile import gettem"
},
{
"path": "isubrip/data_structures.py",
"chars": 8409,
"preview": "from __future__ import annotations\n\nfrom abc import ABC\nimport datetime as dt # noqa: TC003\nfrom enum import Enum\nfrom "
},
{
"path": "isubrip/logger.py",
"chars": 7999,
"preview": "from __future__ import annotations\n\nimport datetime as dt\nfrom functools import lru_cache\nimport logging\nimport re\nfrom "
},
{
"path": "isubrip/scrapers/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "isubrip/scrapers/appletv_scraper.py",
"chars": 14117,
"preview": "from __future__ import annotations\n\nimport datetime as dt\nfrom enum import Enum\nimport fnmatch\nimport re\nfrom typing imp"
},
{
"path": "isubrip/scrapers/itunes_scraper.py",
"chars": 3813,
"preview": "from __future__ import annotations\n\nimport asyncio\nimport re\nfrom typing import TYPE_CHECKING, Any\n\nfrom isubrip.logger "
},
{
"path": "isubrip/scrapers/scraper.py",
"chars": 32803,
"preview": "from __future__ import annotations\n\nfrom abc import ABC, abstractmethod\nimport asyncio\nfrom enum import Enum\nimport impo"
},
{
"path": "isubrip/subtitle_formats/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "isubrip/subtitle_formats/subrip.py",
"chars": 1365,
"preview": "from __future__ import annotations\n\nfrom typing import Any\n\nfrom isubrip.data_structures import SubtitlesFormatType\nfrom"
},
{
"path": "isubrip/subtitle_formats/subtitles.py",
"chars": 11417,
"preview": "from __future__ import annotations\n\nfrom abc import ABC, abstractmethod\nfrom copy import deepcopy\nfrom datetime import t"
},
{
"path": "isubrip/subtitle_formats/webvtt.py",
"chars": 11874,
"preview": "from __future__ import annotations\n\nfrom abc import ABCMeta\nfrom copy import deepcopy\nimport re\nfrom typing import TYPE_"
},
{
"path": "isubrip/ui.py",
"chars": 692,
"preview": "from __future__ import annotations\n\nimport math\nfrom typing import TYPE_CHECKING\n\nfrom rich.progress import TimeElapsedC"
},
{
"path": "isubrip/utils.py",
"chars": 25588,
"preview": "from __future__ import annotations\n\nfrom abc import ABCMeta\nimport datetime as dt\nfrom functools import lru_cache\nimport"
},
{
"path": "pyproject.toml",
"chars": 2762,
"preview": "[project]\nname = \"isubrip\"\nversion = \"2.6.8\"\ndescription = \"A Python package for scraping and downloading subtitles from"
},
{
"path": "tests/.gitignore",
"chars": 10,
"preview": "mock_data/"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/benchmarks/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/benchmarks/download_benchmark.py",
"chars": 405,
"preview": "import pyperf\n\nfrom tests.benchmarks.download_benchmark_module import benchmark\n\n# Note: Run this script with the `-o` f"
},
{
"path": "tests/benchmarks/download_benchmark_module.py",
"chars": 1020,
"preview": "import asyncio\nimport logging\nfrom pathlib import Path\nfrom unittest.mock import patch\n\nfrom isubrip.cli import console\n"
},
{
"path": "tests/test_utils.py",
"chars": 7180,
"preview": "import datetime as dt\n\nimport pytest\n\nfrom isubrip.data_structures import Episode, Movie, Season, Series, SubtitlesForma"
},
{
"path": "tests/tools/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/tools/generate_mock_data.py",
"chars": 8425,
"preview": "from __future__ import annotations\n\nfrom abc import ABC, abstractmethod\nimport argparse\nimport asyncio\nimport hashlib\nim"
},
{
"path": "tests/tools/mock_loader.py",
"chars": 2355,
"preview": "from __future__ import annotations\n\nimport json\nimport logging\nfrom pathlib import Path\nfrom typing import Any\n\nimport h"
}
]
About this extraction
This page contains the full source code of the MichaelYochpaz/iSubRip GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 38 files (203.2 KB), approximately 47.7k tokens, and a symbol index with 234 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.