![\"deemon\"](\"deemon/assets/images/deemon.png\")
\n\n[About](#about) | [Installation](#installation) | [Docker](#docker) | [Documentation](https://digitalec.github.io/deemon) | [Support](#support)\n\n\n\n\n\n\n\n\n\n\n\n### About\ndeemon is a command line tool written in Python that monitors artists for new releases, provides email notifications and can also integrate with the deemix library to automatically download new releases.\n\n### Support\n[Open an Issue](https://github.com/digitalec/deemon/issues/new) | [Discord](https://discord.gg/KzNCG2tkvn)\n\n### Installation\n\n#### Using pip\n\n```bash\n$ pip install deemon\n```\n\n#### From source\n```bash\n$ pip install -r requirements.txt\n$ python3 -m deemon\n```\n\n#### Docker\n\nDocker support has been added for `amd64`, `arm64` and `armv7` architectures. It is recommended to save your `docker run` command as a script to execute via cron/Task Scheduler.\n\n**Note:** Inside deemon's `config.json`, download_location **must** be set to `/downloads` until I can integrate this myself.\n\n**Example: Refreshing an existing database**\n```\ndocker run --name deemon \\\n --rm \\\n -v /path/to/deemon/config:/config \\\n -v /path/to/music:/downloads \\\n -v /path/to/deemix/config:/deemix \\\n ghcr.io/digitalec/deemon:latest \\\n python3 -m deemon refresh\n```\n#### Unraid\n\nInstall Python/PIP using either Nerd-tools Plugin (Unraid 6), Python 3 for UNRAID Plugin (Unraid 6 or 7), or manually via command line.\n\nSee the installation instructions [here](https://digitalec.github.io/deemon/docs/installation.html) or install as root (**NOT** recommended!):\n\n```bash\npip install deemon\n```\nThen:\n```bash\ndeemon --init\n```\n\nIf deemon is not found in your path, you can also call it as a python module:\n```bash\npython3 -m deemon --init\n``` \nIf installed using the **root** account, the config.json will be located at: **/root/.config/deemon/config.json**. Edit your configuration using the documentation located [here](https://digitalec.github.io/deemon/docs/configuration.html).\n\nUse `deemon monitor -h` for help on adding artists, playlists, or albums to monitor for new releases.\n\n#### Installation in a Python Virtual Environment (venv)\n\nIf you wish to install deemon and it's dependencies in a sandbox-style environment, I would recommend using venv.\n\nCreate a venv and install deemon (you may need to use `python3` and `pip3` depending on your system):\n```commandline\n$ python -m venv venv\n$ source ./venv/bin/activate\n$ pip install deemon\n```\n\nWhen you are finished, close the terminal or exit our venv:\n```commandline\n$ deactivate\n```\n\nNext time you want to run deemon, activate the venv first:\n```commandline\n$ source ./venv/bin/activate\n$ deemon refresh\n```\n\nIf you are moving to venv from the Docker container, be sure to update your cron/Task Scheduler scripts.\n\n### Getting Started\nYou have to manually add artists, playlists, albums, etc.. deemon does not automatically pull artists unless they're being monitored. Refer to the documentation [here](https://digitalec.github.io/deemon/docs/commands/monitor.html).\n"
},
{
"path": "deemon/__init__.py",
"content": "#!/usr/bin/env python3\nfrom deemon.utils import startup\n\n__version__ = '2.22'\n__dbversion__ = '3.7'\n\nappdata = startup.get_appdata_dir()\nstartup.init_appdata_dir(appdata)\n"
},
{
"path": "deemon/__main__.py",
"content": "from deemon import cli\n\n\ndef main():\n cli.run()\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "deemon/assets/index.html",
"content": "\n\n \n \n \n \n \n\t\t
\n \n"
},
{
"path": "deemon/cli.py",
"content": "import logging\nimport platform\nimport sys\nimport time\nfrom pathlib import Path\n\nimport click\nfrom packaging.version import parse as parse_version\n\nfrom deemon import __version__\nfrom deemon.cmd import download, rollback, backup, extra, tests, upgradelib\nfrom deemon.cmd.artistconfig import artist_lookup\nfrom deemon.cmd.monitor import Monitor\nfrom deemon.cmd.profile import ProfileConfig\nfrom deemon.cmd.refresh import Refresh\nfrom deemon.cmd.search import Search\nfrom deemon.cmd.show import Show\nfrom deemon.core import notifier\nfrom deemon.core.config import Config, LoadProfile\nfrom deemon.core.db import Database\nfrom deemon.core.logger import setup_logger\nfrom deemon.utils import startup, dataprocessor, validate\n\nlogger = None\nconfig = None\ndb = None\n\nCONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])\n\n@click.group(context_settings=CONTEXT_SETTINGS, invoke_without_command=True,\n no_args_is_help=True)\n@click.option('--whats-new', is_flag=True, help=\"Show release notes from this version\")\n@click.option('--init', is_flag=True, help=\"\"\"Initialize deemon application data\n directory. Warning: if directory exists, this will delete existing config and database.\"\"\")\n@click.option('--arl', help=\"Update ARL\")\n@click.option('-P', '--profile', help=\"Specify profile to run deemon as\")\n@click.version_option(__version__, '-V', '--version', message='deemon %(version)s')\n@click.option('-v', '--verbose', is_flag=True, help=\"Show debug output\")\ndef run(whats_new, init, arl, verbose, profile):\n \"\"\"Monitoring and alerting tool for new music releases using the Deezer API.\n\n deemon is a free and open source tool. To report issues or to contribute,\n please visit https://github.com/digitalec/deemon\n \"\"\"\n global logger\n global config\n global db\n\n setup_logger(log_level='DEBUG' if verbose else 'INFO', log_file=startup.get_log_file())\n logger = logging.getLogger(__name__)\n logger.debug(f\"deemon {__version__}\")\n logger.debug(f\"command: \\\"{' '.join([x for x in sys.argv[1:]])}\\\"\")\n logger.debug(\"Python \" + platform.python_version())\n logger.debug(platform.platform())\n logger.debug(f\"deemon appdata is located at {startup.get_appdata_dir()}\")\n \n if whats_new:\n return startup.get_changelog(__version__)\n\n if init:\n app_data_path = startup.get_appdata_dir()\n startup.reinit_appdata_dir(app_data_path)\n\n config = Config()\n db = Database()\n\n db.do_upgrade()\n tid = db.get_next_transaction_id()\n config.set('tid', tid, validate=False)\n \n if arl:\n if config.set(\"arl\", arl):\n config._Config__write_modified_config()\n reload_config = Config()\n return print(f\"ARL has been successfully updated to: {reload_config.arl()}\")\n else:\n return print(\"Error when updating ARL.\")\n\n if profile:\n profile_config = db.get_profile(profile)\n if profile_config:\n LoadProfile(profile_config)\n else:\n logger.error(f\"Profile {profile} does not exist.\")\n sys.exit(1)\n else:\n profile_config = db.get_profile_by_id(1)\n if profile_config:\n LoadProfile(profile_config)\n\n if not any(x in sys.argv[1:] for x in ['-h', '--help']):\n last_checked: int = int(db.last_update_check())\n next_check: int = last_checked + (config.check_update() * 86400)\n if config.release_channel() != db.get_release_channel()['value']:\n # If release_channel has changed, check for latest release\n logger.debug(f\"Release channel changed to '{config.release_channel()}'\")\n db.set_release_channel()\n last_checked = 0\n if time.time() >= next_check or last_checked == 0:\n logger.info(f\"Checking for updates ({config.release_channel()})...\")\n config.set('update_available', 0, False)\n latest_ver = str(startup.get_latest_version(config.release_channel()))\n if latest_ver:\n db.set_latest_version(latest_ver)\n db.set_last_update_check()\n new_version = db.get_latest_ver()\n if parse_version(new_version) > parse_version(__version__):\n if parse_version(new_version).major > parse_version(__version__).major:\n config.set('update_available', new_version, False)\n print(\"*\" * 80)\n logger.info(f\"deemon {parse_version(new_version).major} is available. \"\n f\"Please see the release notes before upgrading.\")\n logger.info(\"Release notes available at: https://github.com/digitalec/deemon/releases\")\n print(\"*\" * 80)\n else:\n config.set('update_available', new_version, False)\n print(\"*\" * 50)\n logger.info(f\"* New version is available: v{__version__} -> v{new_version}\")\n if config.release_channel() == \"beta\":\n logger.info(\"* To upgrade, run `pip install --upgrade --pre deemon`\")\n else:\n logger.info(\"* To upgrade, run `pip install --upgrade deemon`\")\n print(\"*\" * 50)\n print(\"\")\n\n config.set(\"start_time\", int(time.time()), False)\n\n\n@run.command(name='test')\n@click.option('-e', '--email', is_flag=True, help=\"Send test notification to configured email\")\n@click.option('-E', '--exclusions', metavar=\"URL\", type=str, help=\"Test exclude regex pattern against URL\")\ndef test(email, exclusions):\n \"\"\"Run tests on email configuration, exclusion filters, etc.\"\"\"\n if email:\n notification = notifier.Notify()\n notification.test()\n elif exclusions:\n if config.exclusion_patterns() or config.exclusion_keywords:\n tests.exclusion_test(exclusions)\n else:\n logger.info(\"You don't have any exclusions configured or they're disabled\")\n\n@run.command(name='download', no_args_is_help=True)\n@click.argument('artist', nargs=-1, required=False)\n@click.option('-m', '--monitored', is_flag=True, help='Download all currently monitored artists')\n@click.option('-i', '--artist-id', multiple=True, metavar='ID', type=int, help='Download by artist ID')\n@click.option('-A', '--album-id', multiple=True, metavar='ID', type=int, help='Download by album ID')\n@click.option('-T', '--track-id', multiple=True, metavar='ID', type=int, help='Download by track ID')\n@click.option('-u', '--url', metavar='URL', multiple=True, help='Download by URL of artist/album/track/playlist')\n@click.option('-f', '--file', metavar='FILE', help='Download batch of artists or artist IDs from file', hidden=True)\n@click.option('--artist-file', metavar='FILE', help='Download batch of artists or artist IDs from file')\n@click.option('--album-file', metavar='FILE', help='Download batch of album IDs from file')\n@click.option('--track-file', metavar='FILE', help='Download batch of track IDs from file')\n@click.option('-a', '--after', 'from_date', metavar=\"YYYY-MM-DD\", type=str, help='Grab releases released after this date')\n@click.option('-B', '--before', 'to_date', metavar=\"YYYY-MM-DD\", type=str, help='Grab releases released before this date')\n@click.option('-b', '--bitrate', metavar=\"BITRATE\", help='Set custom bitrate for this operation')\n@click.option('-o', '--download-path', metavar=\"PATH\", type=str, help='Specify custom download directory')\n@click.option('-t', '--record-type', metavar=\"TYPE\", type=str, help='Specify record types to download')\ndef download_command(artist, artist_id, album_id, url, file, bitrate,\n record_type, download_path, from_date, to_date,\n monitored, track_id, track_file, artist_file,\n album_file):\n \"\"\"\n Download specific artist, album ID or by URL\n\n \\b\n Examples:\n download Mozart\n download -i 100 -t album -b 9\n \"\"\"\n\n if bitrate:\n config.set('bitrate', bitrate)\n if download_path:\n config.set('download_path', download_path)\n if record_type:\n config.set('record_type', record_type)\n\n artists = dataprocessor.csv_to_list(artist) if artist else None\n artist_ids = [x for x in artist_id] if artist_id else None\n album_ids = [x for x in album_id] if album_id else None\n track_ids = [x for x in track_id] if track_id else None\n urls = [x for x in url] if url else None\n\n if file:\n logger.info(\"WARNING: -f/--file has been replaced with --artist-file and will be removed in future versions.\")\n artist_file = file\n\n if download_path and download_path != \"\":\n if Path(download_path).exists:\n config.set('download_path', download_path)\n logger.debug(f\"Download path has changed: {config.download_path()}\")\n else:\n return logger.error(f\"Invalid download path: {download_path}\")\n\n dl = download.Download()\n dl.set_dates(from_date, to_date)\n dl.download(artists, artist_ids, album_ids, urls, artist_file, track_file, album_file, track_ids, monitored=monitored)\n\n\n@run.command(name='monitor', context_settings={\"ignore_unknown_options\": False}, no_args_is_help=True)\n@click.argument('artist', nargs=-1)\n@click.option('-a', '--alerts', is_flag=True, help=\"Enable or disable alerts\")\n@click.option('-b', '--bitrate', metavar=\"BITRATE\", help=\"Specify bitrate\")\n@click.option('-D', '--download', 'dl', is_flag=True, help='Download all releases matching record type')\n@click.option('-d', '--download-path', type=str, metavar=\"PATH\", help='Specify custom download directory')\n@click.option('-I', '--import', 'im', metavar=\"PATH\", help=\"Monitor artists/IDs from file or directory\")\n@click.option('-i', '--artist-id', is_flag=True, help=\"Monitor artist by ID\")\n@click.option('-p', '--playlist', is_flag=True, help='Monitor Deezer playlist by URL')\n@click.option('--include-artists', is_flag=True, help='Also monitor artists from playlist')\n@click.option('-u', '--url', is_flag=True, help='Monitor artist by URL')\n@click.option('-R', '--remove', is_flag=True, help='Stop monitoring an artist')\n@click.option('-s', '--search', 'search_flag', is_flag=True, help='Show similar artist results to choose from')\n@click.option('-T', '--time-machine', type=str, metavar=\"YYYY-MM-DD\", help=\"Refresh newly added artists on this date\")\n@click.option('-t', '--record-type', metavar=\"TYPE\", type=str, help='Specify record types to download')\ndef monitor_command(artist, im, playlist, include_artists, bitrate, record_type, alerts, artist_id,\n dl, remove, url, download_path, search_flag, time_machine):\n \"\"\"\n Monitor artist for new releases by ID, URL or name.\n\n \\b\n Examples:\n monitor Mozart\n monitor --artist-id 100\n monitor --url https://www.deezer.com/us/artist/000\n \"\"\"\n monitor = Monitor()\n if download_path:\n if not Path(download_path).exists():\n return logger.error(\"Invalid download path provided\")\n\n if time_machine:\n validated = validate.validate_date(time_machine)\n if validated:\n monitor.time_machine = validated\n else:\n return logger.error(\"Date for time machine is invalid\")\n \n if not alerts:\n alerts = None\n\n monitor.set_options(remove, dl, search_flag)\n monitor.set_config(bitrate, alerts, record_type, download_path)\n\n if url:\n artist_id = True\n urls = [x.replace(\",\", \"\") for x in artist]\n artist = []\n for u in urls:\n id_from_url = u.split('/artist/')\n try:\n aid = int(id_from_url[1])\n except (IndexError, ValueError):\n logger.error(f\"Invalid artist URL -- {u}\")\n return\n artist.append(aid)\n\n if playlist:\n urls = [x.replace(\",\", \"\") for x in artist]\n playlist_id = []\n for u in urls:\n id_from_url = u.split('/playlist/')\n try:\n aid = int(id_from_url[1])\n except (IndexError, ValueError):\n logger.error(f\"Invalid playlist URL -- {u}\")\n return\n playlist_id.append(aid)\n\n if im:\n monitor.importer(im)\n elif playlist:\n monitor.playlists(playlist_id, include_artists)\n elif artist_id:\n monitor.artist_ids(dataprocessor.csv_to_list(artist))\n elif artist:\n monitor.artists(dataprocessor.csv_to_list(artist))\n\n\n@run.command(name='refresh')\n@click.argument('NAME', nargs=-1, type=str, required=False)\n@click.option('-p', '--playlist', is_flag=True, help=\"Refresh a specific playlist by name\")\n@click.option('-s', '--skip-download', is_flag=True, help=\"Skips downloading of new releases\")\n@click.option('-T', '--time-machine', metavar='DATE', type=str, help='Refresh as if it were this date (YYYY-MM-DD)')\ndef refresh_command(name, playlist, skip_download, time_machine):\n \"\"\"Check artists for new releases\"\"\"\n\n if time_machine:\n time_machine = validate.validate_date(time_machine)\n if not time_machine:\n return logger.error(\"Date for time machine is invalid\")\n\n logger.info(\":: Starting database refresh\")\n refresh = Refresh(time_machine, skip_download)\n if playlist:\n if not len(name):\n return logger.warning(\"You must provide the name of a playlist\")\n refresh.run(playlists=dataprocessor.csv_to_list(name))\n elif name:\n refresh.run(artists=dataprocessor.csv_to_list(name))\n else:\n refresh.run()\n\n\n@click.group(name=\"show\")\ndef show_command():\n \"\"\"\n Show monitored artists and latest releases\n \"\"\"\n\n\n@show_command.command(name=\"artists\")\n@click.argument('artist', nargs=-1, required=False)\n@click.option('-c', '--csv', is_flag=True, help='Output artists as CSV')\n@click.option('-e', '--export', type=Path, help='Export CSV data to file; same as -ce')\n@click.option('-f', '--filter', type=str, help='Specify filter for CSV output')\n@click.option('-H', '--hide-header', is_flag=True, help='Hide header on CSV output')\n@click.option('-b', '--backup', type=Path, help='Backup artist IDs to CSV, same as -cHf id -e ...')\ndef show_artists(artist, csv, export, filter, hide_header, backup):\n \"\"\"Show artist info monitored by profile\"\"\"\n if artist:\n artist = ' '.join([x for x in artist])\n\n show = Show()\n show.monitoring(artist=True, query=artist, export_csv=csv, save_path=export, filter=filter, hide_header=hide_header,\n backup=backup)\n\n\n@show_command.command(name=\"playlists\")\n@click.argument('title', nargs=-1, required=False)\n@click.option('-c', '--csv', is_flag=True, help='Output artists as CSV')\n@click.option('-f', '--filter', type=str, help='Specify filter for CSV output')\n@click.option('-H', '--hide-header', is_flag=True, help='Hide header on CSV output')\n@click.option('-i', '--playlist-id', is_flag=True, help='Show playlist info by playlist ID')\ndef show_artists(title, playlist_id, csv, filter, hide_header):\n \"\"\"Show playlist info monitored by profile\"\"\"\n if title:\n title = ' '.join([x for x in title])\n\n show = Show()\n show.monitoring(artist=False, query=title, export_csv=csv, filter=filter, hide_header=hide_header, is_id=playlist_id)\n\n\n@show_command.command(name=\"releases\")\n@click.argument('N', default=7)\n@click.option('-f', '--future', is_flag=True, help='Display future releases')\ndef show_releases(n, future):\n \"\"\"\n Show list of new or future releases\n \"\"\"\n show = Show()\n show.releases(n, future)\n\n\nrun.add_command(show_command)\n\n\n@run.command(name=\"backup\")\n@click.option('-i', '--include-logs', is_flag=True, help='include log files in backup')\n@click.option('-r', '--restore', is_flag=True, help='Restore from existing backup')\ndef backup_command(restore, include_logs):\n \"\"\"Backup configuration and database to a tar file\"\"\"\n\n if restore:\n backup.restore()\n else:\n backup.run(include_logs)\n\n\n# TODO @click.option does not support nargs=-1; unable to use spaces without quotations\n@run.command(name=\"api\", help=\"View raw API data for artist, artist ID or playlist ID\", hidden=True)\n@click.option('-A', '--album-id', type=int, help='Get album ID result via API')\n@click.option('-a', '--artist', type=str, help='Get artist result via API')\n@click.option('-i', '--artist-id', type=int, help='Get artist ID result via API')\n@click.option('-l', '--limit', type=int, help='Set max number of artist results; default=1', default=1)\n@click.option('-p', '--playlist-id', type=int, help='Get playlist ID result via API')\n@click.option('-r', '--raw', is_flag=True, help='Dump as raw data returned from API')\ndef api_test(artist, artist_id, album_id, playlist_id, limit, raw):\n \"\"\"View API result - for testing purposes\"\"\"\n import deezer\n dz = deezer.Deezer()\n if artist or artist_id:\n if artist:\n result = dz.api.search_artist(artist, limit=limit)['data']\n else:\n result = dz.api.get_artist(artist_id)\n\n if raw:\n if isinstance(result, list):\n for row in result:\n for key, value in row.items():\n print(f\"{key}: {value}\")\n print(\"\\n\")\n else:\n for key, value in result.items():\n print(f\"{key}: {value}\")\n else:\n if isinstance(result, list):\n for row in result:\n print(f\"Artist ID: {row['id']}\\nArtist Name: {row['name']}\\n\")\n else:\n print(f\"Artist ID: {result['id']}\\nArtist Name: {result['name']}\")\n\n if album_id:\n result = dz.api.get_album(album_id)\n\n if raw:\n for key, value in result.items():\n print(f\"{key}: {value}\")\n else:\n print(f\"Album ID: {result['id']}\\nAlbum Title: {result['title']}\")\n\n if playlist_id:\n result = dz.api.get_playlist(playlist_id)\n\n if raw:\n for key, value in result.items():\n print(f\"{key}: {value}\")\n else:\n print(f\"Playlist ID: {result['id']}\\nPlaylist Title: {result['title']}\")\n\n\n@run.command(name=\"reset\")\ndef reset_db():\n \"\"\"Reset monitoring database\"\"\"\n logger.warning(\"** WARNING: All artists and playlists will be removed regardless of profile! **\")\n confirm = input(\":: Type 'reset' to confirm: \")\n if confirm.lower() == \"reset\":\n print(\"\")\n db.reset_database()\n else:\n logger.info(\"Reset aborted. Database has NOT been modified.\")\n return\n\n\n@run.command(name='profile')\n@click.argument('profile', required=False)\n@click.option('-a', '--add', is_flag=True, help=\"Add new profile\")\n@click.option('-c', '--clear', is_flag=True, help=\"Clear config for existing profile\")\n@click.option('-d', '--delete', is_flag=True, help=\"Delete an existing profile\")\n@click.option('-e', '--edit', is_flag=True, help=\"Edit an existing profile\")\ndef profile_command(profile, add, clear, delete, edit):\n \"\"\"Add, modify and delete configuration profiles\"\"\"\n\n pc = ProfileConfig(profile)\n if profile:\n if add:\n pc.add()\n elif clear:\n pc.clear()\n elif delete:\n pc.delete()\n elif edit:\n pc.edit()\n else:\n pc.show()\n else:\n pc.show()\n\n\n@run.command(name=\"extra\")\ndef extra_command():\n \"\"\"Fetch extra release info\"\"\"\n extra.main()\n\n\n@run.command(name=\"search\")\n@click.argument('query', nargs=-1, required=False)\ndef search(query):\n \"\"\"Interactively search and download/monitor artists\"\"\"\n if query:\n query = ' '.join(query)\n\n client = Search()\n client.search_menu(query)\n\n\n@run.command(name=\"config\")\n@click.argument('artist', nargs=-1, required=True)\ndef config_command(artist):\n \"\"\"Configure per-artist settings by name or ID\"\"\"\n artist = ' '.join([x for x in artist])\n artist_lookup(artist)\n\n\n@run.command(name=\"rollback\", no_args_is_help=True)\n@click.argument('num', type=int, required=False)\n@click.option('-v', '--view', is_flag=True, help=\"View recent refresh transactions\")\ndef rollback_command(num, view):\n \"\"\"Rollback a previous monitor or refresh transaction\"\"\"\n if view:\n rollback.view_transactions()\n elif num:\n rollback.rollback_last(num)\n\n\n@click.group(name=\"library\")\ndef library_command():\n \"\"\"\n Library options such as upgrading from MP3 to FLAC\n \"\"\"\n\n\n@library_command.command(name=\"upgrade\")\n@click.argument('library', metavar='PATH')\n@click.option('-A', '--album-only', is_flag=True, help=\"Get album IDs instead of track IDs (Fastest)\")\n@click.option('-E', '--allow-exclusions', is_flag=True, help=\"Allow exclusions to be applied\")\n@click.option('-O', '--output', metavar='PATH', help=\"Output file to save IDs (default: current directory)\")\ndef library_upgrade_command(library, output, album_only, allow_exclusions):\n \"\"\" (BETA) Scans MP3 files in PATH and generates a text file containing album/track IDs \"\"\"\n if not output:\n output = Path.cwd()\n upgradelib.upgrade(library, output, album_only, allow_exclusions)\n\n\nrun.add_command(library_command)\n"
},
{
"path": "deemon/cmd/__init__.py",
"content": ""
},
{
"path": "deemon/cmd/artistconfig.py",
"content": "import logging\n\nfrom deemon.core.config import Config as config\nfrom deemon.core.db import Database\n\nlogger = logging.getLogger(__name__)\ndb = Database()\n\n\ndef print_header(message: str = None):\n from os import system, name\n if name == 'nt':\n _ = system('cls')\n else:\n _ = system('clear')\n print(\"deemon Artist Configurator\")\n if message:\n print(\":: \" + message + \"\\n\")\n else:\n print(\"\")\n\n\ndef get_artist(query: str):\n artist_as_id = False\n artist_fromdb = None\n\n try:\n artist_as_id = int(query)\n except ValueError:\n pass\n\n by_name = db.get_monitored_artist_by_name(query)\n if by_name:\n logger.debug(\"Artist found by name\")\n artist_fromdb = by_name\n\n if artist_as_id:\n by_id = db.get_monitored_artist_by_id(artist_as_id)\n if by_id:\n logger.debug(f\"Artist found by ID\")\n if not artist_fromdb:\n artist_fromdb = by_id\n else:\n logger.debug(\"Artist found by both ID and name, prompting user\")\n\n while True:\n prompt = input(\":: Multiple artists found. Was that a name or ID? \")\n if prompt.lower() == \"name\":\n logger.debug(\"Artist confirmed by user to be a name\")\n return artist_fromdb\n elif prompt.lower() == \"id\":\n logger.debug(\"Artist confirmed by user to be an ID\")\n return by_id\n else:\n return logger.error(f\"Artist/Artist ID not found for '{query}'\")\n\n if artist_fromdb:\n return artist_fromdb\n else:\n return logger.error(f\"Artist/Artist ID not found for '{query}'\")\n\n\ndef artist_lookup(query):\n result = get_artist(query)\n if not result:\n return\n print_header(f\"Configuring '{result['artist_name']}' (Artist ID: {result['artist_id']})\")\n modified = 0\n for property in result:\n if property not in ['alerts', 'bitrate', 'record_type', 'download_path']:\n continue\n allowed_opts = config.allowed_values(property)\n if isinstance(allowed_opts, dict):\n allowed_opts = [str(x.lower()) for x in allowed_opts.values()]\n\n while True:\n friendly_text = property.replace(\"_\", \" \").title()\n user_input = input(f\"{friendly_text} [{result[property]}]: \")\n\n if property != \"download_path\":\n user_input = user_input.lower()\n\n if user_input == \"\":\n break\n elif user_input == \"false\" or user_input == \"0\":\n user_input = False\n elif user_input == \"true\" or user_input == \"1\":\n user_input = True\n if user_input == \"none\":\n user_input = None\n elif allowed_opts:\n if user_input not in allowed_opts:\n print(f\"Allowed options: \" + ', '.join(str(x) for x in allowed_opts))\n continue\n logger.debug(f\"User set {property} to {user_input}\")\n result[property] = user_input\n modified += 1\n break\n if modified > 0:\n i = input(\"\\n:: Save these settings? [y|N] \")\n if i.lower() != \"y\":\n logger.info(\"No changes made, exiting...\")\n else:\n db.update_artist(result)\n print(f\"\\nArtist '{result['artist_name']}' has been updated!\")\n else:\n print(\"No changes made, exiting...\")\n"
},
{
"path": "deemon/cmd/backup.py",
"content": "import logging\nimport os\nimport tarfile\nfrom datetime import datetime\nfrom pathlib import Path\n\nfrom packaging.version import parse as parse_version\nfrom tqdm import tqdm\n\nfrom deemon import __version__\nfrom deemon.utils import startup, dates\n\nlogger = logging.getLogger(__name__)\n\n\ndef run(include_logs: bool = False):\n def filter_func(item):\n includes = ['deemon', 'deemon/config.json', 'deemon/deemon.db']\n if include_logs:\n if 'deemon/logs' in item.name:\n includes.append(item.name)\n if item.name in includes:\n return item\n\n backup_tar = dates.generate_date_filename(\"backup-\" + __version__ + \"-\") + \".tar\"\n backup_path = startup.get_backup_dir()\n\n with tarfile.open(backup_path / backup_tar, \"w\") as tar:\n tar.add(startup.get_appdata_dir(), arcname='deemon', filter=filter_func)\n logger.info(f\"Backed up to {backup_path / backup_tar}\")\n\n\ndef restore():\n restore_file_list = ['deemon/config.json', 'deemon/deemon.db']\n\n def inspect_tar(fn: Path) -> dict:\n fn_name = fn.name\n fn_name = fn_name.replace('.tar', '').split('-')\n if fn_name[0] == \"backup\" and len(fn_name) > 3:\n if check_tar_contents(fn):\n backup_appversion = '-'.join(fn_name[1:-2])\n if is_newer_backup(backup_appversion):\n logger.debug(f\"Backup found for newer version {backup_appversion} is not compatible!\")\n return\n backup_time = datetime.strptime(fn_name[-1], \"%H%M%S\")\n backup_date = datetime.strptime(fn_name[-2], \"%Y%m%d\")\n try:\n friendly_time = datetime.strftime(backup_time, \"%-I:%M:%S %p\")\n except ValueError:\n # Gotta keep Windows happy...\n friendly_time = datetime.strftime(backup_time, \"%#I:%M:%S %p\")\n try:\n friendly_date = datetime.strftime(backup_date, \"%b %-d, %Y\")\n except ValueError:\n # Gotta keep Windows happy...\n friendly_date = datetime.strftime(backup_date, \"%b %#d, %Y\")\n backup_info = {\n 'version': backup_appversion,\n 'date': friendly_date,\n 'time': friendly_time,\n 'age': fn_name[-2] + fn_name[-1],\n 'filename': fn\n }\n return backup_info\n else:\n return\n\n def check_tar_contents(archive: Path):\n tar = tarfile.open(archive)\n file_list = tar.getmembers()\n files = [x.name for x in file_list]\n if all(item in files for item in restore_file_list):\n return True\n logger.debug(\"Archive is invalid or corrupt: \" + str(archive))\n\n def restore_tarfile(archive: dict):\n logger.debug(\"Restoring backup from `\" + str(archive['filename'].name + \"`\"))\n extract_dir = startup.get_appdata_dir()\n tar = tarfile.open(archive['filename'])\n progress = tqdm(tar.getmembers(), ascii=\" #\",\n bar_format='{desc} [{bar}] {percentage:3.0f}%')\n for member in progress:\n if member.isreg():\n if member.name in restore_file_list:\n member.name = os.path.basename(member.name)\n logger.info(f\"Restoring {member.name}...\")\n progress.set_description_str(f\"Restoring {member.name}\")\n tar.extract(member, extract_dir)\n logger.debug(f\"Restored {member.name} to {extract_dir}\")\n if member == tar.getmembers()[-1]:\n progress.set_description_str(\"Restore complete\")\n\n def is_newer_backup(version: str):\n if parse_version(version) > parse_version(__version__):\n return True\n\n def display_backup_list(available_backups: list):\n print(\"deemon Backup Manager\\n\")\n for index, backup in enumerate(available_backups, start=1):\n print(f\"{index}. {backup['date']} @ {backup['time']} (ver {backup['version']})\")\n\n selected_backup = int\n while selected_backup not in range(len(available_backups)):\n selected_backup = input(\"\\nSelect a backup to restore (or press Enter to exit): \")\n if selected_backup == \"\":\n return\n try:\n selected_backup = int(selected_backup)\n selected_backup -= 1\n except ValueError:\n logger.warning(\"Invalid entry. Enter a number corresponding to the backup you wish to restore.\")\n print(\"\")\n restore_tarfile(available_backups[selected_backup])\n\n backups = []\n backup_path = startup.get_backup_dir()\n file_list = [x for x in sorted(Path(backup_path).glob('*.tar'))]\n for backup in file_list:\n tar_files = inspect_tar(backup)\n if tar_files:\n backups.append(tar_files)\n if backups:\n backups = sorted(backups, key=lambda x: x['age'], reverse=True)\n display_backup_list(backups)\n else:\n logger.info(\"No backups available to restore\")\n"
},
{
"path": "deemon/cmd/download.py",
"content": "import logging\nimport os\nimport sys\n\nimport requests\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\nfrom tqdm import tqdm\n\nimport deemix.errors\nimport deezer\nfrom deezer import errors\nimport plexapi.exceptions\nfrom plexapi.server import PlexServer\n\nfrom deemon import utils\nfrom deemon.core import dmi, db, api, common\nfrom deemon.core.config import Config as config\nfrom deemon.utils import ui, dataprocessor, startup, dates\n\nlogger = logging.getLogger(__name__)\n\n\nclass QueueItem:\n # TODO - Accept new playlist tracks for output/alerts\n def __init__(self, artist=None, album=None, track=None, playlist=None,\n bitrate: str = None, download_path: str = None,\n release_full: dict = None):\n self.artist_name = None\n self.album_id = None\n self.album_title = None\n self.track_id = None\n self.track_title = None\n self.url = None\n self.playlist_title = None\n self.bitrate = bitrate or config.bitrate()\n self.download_path = download_path or config.download_path()\n self.release_type = None\n \n if release_full:\n self.artist_name = release_full['artist_name']\n self.album_id = release_full['id']\n self.album_title = release_full['title']\n self.url = f\"https://www.deezer.com/album/{self.album_id}\"\n self.release_type = release_full['record_type']\n self.bitrate = release_full['bitrate']\n self.download_path = release_full['download_path']\n\n if artist:\n try:\n self.artist_name = artist[\"artist_name\"]\n except KeyError:\n self.artist_name = artist[\"name\"]\n if not album and not track:\n self.url = artist[\"link\"]\n\n if album:\n if not artist:\n self.artist_name = album[\"artist\"][\"name\"]\n self.album_id = album[\"id\"]\n self.album_title = album[\"title\"]\n try:\n self.url = album[\"link\"]\n except KeyError:\n self.url = f\"https://www.deezer.com/album/{album['id']}\"\n\n if track:\n self.artist_name = track[\"artist\"][\"name\"]\n self.track_id = track[\"id\"]\n self.track_title = track[\"title\"]\n self.url = f\"https://deezer.com/track/{self.track_id}\"\n\n if playlist:\n try:\n self.url = playlist[\"link\"]\n except KeyError:\n logger.debug(\"DEPRECATED dict key: playlist['url'] should not be used in favor of playlist['link']\")\n self.url = playlist.get(\"url\", None)\n self.playlist_title = playlist[\"title\"]\n\n\ndef get_deemix_bitrate(bitrate: str):\n for bitrate_id, bitrate_name in config.allowed_values('bitrate').items():\n if bitrate_name.lower() == bitrate.lower():\n logger.debug(f\"Setting deemix bitrate to {str(bitrate_id)}\")\n return bitrate_id\n\n\ndef get_plex_server():\n if (config.plex_baseurl() != \"\") and (config.plex_token() != \"\"):\n session = None\n if not config.plex_ssl_verify():\n requests.packages.urllib3.disable_warnings()\n session = requests.Session()\n session.verify = False\n try:\n print(\"Plex settings found, trying to connect (10s)... \", end=\"\")\n plex_server = PlexServer(config.plex_baseurl(), config.plex_token(), timeout=10, session=session)\n print(\" OK\")\n return plex_server\n except Exception as e:\n print(\" FAILED\")\n logger.error(\"Error: Unable to reach Plex server, please refresh manually.\")\n logger.debug(e)\n return False\n\n\ndef refresh_plex(plexobj):\n try:\n plexobj.library.section(config.plex_library()).update()\n logger.debug(\"Plex library refreshed successfully\")\n except plexapi.exceptions.BadRequest as e:\n logger.error(\"Error occurred while refreshing your library. See logs for additional info.\")\n logger.debug(f\"Error during Plex refresh: {e}\")\n except plexapi.exceptions.NotFound as e:\n logger.error(\"Error: Plex library not found. See logs for additional info.\")\n logger.debug(f\"Error during Plex refresh: {e}\")\n\n\nclass Download:\n\n def __init__(self, active_api=None):\n super().__init__()\n self.api = active_api or api.PlatformAPI()\n self.dz = deezer.Deezer()\n self.di = dmi.DeemixInterface()\n self.queue_list = []\n self.db = db.Database()\n self.bitrate = None\n self.release_from = None\n self.release_to = None\n self.verbose = os.environ.get(\"VERBOSE\")\n self.duplicate_id_count = 0\n\n def set_dates(self, from_date: str = None, to_date: str = None) -> None:\n \"\"\"Set to/from dates to get while downloading\"\"\"\n if from_date:\n try:\n self.release_from = dates.str_to_datetime_obj(from_date)\n except ValueError as e:\n raise ValueError(f\"Invalid date provided - {from_date}: {e}\")\n if to_date:\n try:\n self.release_to = dates.str_to_datetime_obj(to_date)\n except ValueError as e:\n raise ValueError(f\"Invalid date provided - {to_date}: {e}\")\n\n # @performance.timeit\n def download_queue(self, queue_list: list = None):\n if queue_list:\n self.queue_list = queue_list\n\n if not self.di.login():\n logger.error(\"Failed to login, aborting download...\")\n return False\n\n if self.queue_list:\n plex = get_plex_server()\n print(\"\")\n logger.info(\":: Sending \" + str(len(self.queue_list)) + \" release(s) to deemix for download:\")\n\n with open(startup.get_appdata_dir() / \"queue.csv\", \"w\", encoding=\"utf-8\") as f:\n f.writelines(','.join([str(x) for x in vars(self.queue_list[0]).keys()]) + \"\\n\")\n logger.debug(f\"Writing queue to CSV file - {len(self.queue_list)} items in queue\")\n for q in self.queue_list:\n raw_values = [str(x) for x in vars(q).values()]\n # TODO move this to shared function\n for i, v in enumerate(raw_values):\n if '\"' in v:\n raw_values[i] = v.replace('\"', \"'\")\n if ',' in v:\n raw_values[i] = f'\"{v}\"'\n f.writelines(','.join(raw_values) + \"\\n\")\n logger.debug(f\"Queue exported to {startup.get_appdata_dir()}/queue.csv\")\n\n failed_count = []\n download_progress = tqdm(\n self.queue_list,\n total=len(self.queue_list),\n desc=\"Downloading releases...\",\n ascii=\" #\",\n bar_format=ui.TQDM_FORMAT\n )\n for index, item in enumerate(download_progress):\n i = str(index + 1)\n t = str(len(download_progress))\n download_progress.set_description_str(f\"Downloading release {i} of {t}...\")\n dx_bitrate = get_deemix_bitrate(item.bitrate)\n if self.verbose == \"true\":\n logger.debug(f\"Processing queue item {vars(item)}\")\n try:\n if item.download_path:\n download_path = item.download_path\n else:\n download_path = None\n\n if item.artist_name:\n if item.album_title:\n logger.info(f\" > {item.artist_name} - {item.album_title}... \")\n self.di.download_url([item.url], dx_bitrate, download_path)\n else:\n logger.info(f\" > {item.artist_name} - {item.track_title}... \")\n self.di.download_url([item.url], dx_bitrate, download_path)\n else:\n logger.info(f\" > {item.playlist_title} (playlist)...\")\n self.di.download_url([item.url], dx_bitrate, download_path, override_deemix=True)\n except (deemix.errors.GenerationError, errors.WrongGeolocation) as e:\n logger.debug(e)\n failed_count.append([(item, \"No tracks listed or unavailable in your country\")])\n except Exception as e:\n if item.artist_name and item.album_title:\n logger.info(f\"The following error occured while downloading {item.artist_name} - {item.album_title}: {e}\")\n elif item.artist_name and item.track_title:\n logger.info(f\"The following error occured while downloading {item.artist_name} - {item.track_title}: {e}\")\n else:\n logger.info(f\"The following error occured while downloading {item.playlist_title}: {e}\")\n pass\n\n\n failed_count = [x for x in failed_count if x]\n\n print(\"\")\n if len(failed_count):\n logger.info(f\" [!] Downloads completed with {len(failed_count)} error(s):\")\n with open(startup.get_appdata_dir() / \"failed.csv\", \"w\", encoding=\"utf-8\") as f:\n f.writelines(','.join([str(x) for x in vars(self.queue_list[0]).keys()]) + \"\\n\")\n for failed in failed_count:\n try:\n raw_values = [str(x) for x in vars(failed[0]).values()]\n except TypeError as e:\n print(f\"Error reading from failed.csv. Entry that failed was either invalid or empty: {failed}\")\n logger.error(e)\n else:\n # TODO move this to shared function\n for i, v in enumerate(raw_values):\n if '\"' in v:\n raw_values[i] = v.replace('\"', \"'\")\n if ',' in v:\n raw_values[i] = f'\"{v}\"'\n f.writelines(','.join(raw_values) + \"\\n\")\n print(f\"+ {failed[0].artist_name} - {failed[0].album_title} --- Reason: {failed[1]}\")\n print(\"\")\n logger.info(f\":: Failed downloads exported to: {startup.get_appdata_dir()}/failed.csv\")\n else:\n logger.info(\" Downloads complete!\")\n if plex and (config.plex_library() != \"\"):\n refresh_plex(plex)\n return True\n\n def download(self, artist, artist_id, album_id, url,\n artist_file, track_file, album_file, track_id, auto=True, monitored=False):\n\n def filter_artist_by_record_type(artist):\n album_api = self.api.get_artist_albums(query={'artist_name': '', 'artist_id': artist['id']})\n filtered_albums = []\n for album in album_api['releases']:\n if (album['record_type'] == config.record_type()) or config.record_type() == \"all\":\n album_date = dates.str_to_datetime_obj(album['release_date'])\n if self.release_from and self.release_to:\n if album_date > self.release_from and album_date < self.release_to:\n filtered_albums.append(album)\n elif self.release_from:\n if album_date > self.release_from:\n filtered_albums.append(album)\n elif self.release_to:\n if album_date < self.release_to:\n filtered_albums.append(album)\n else:\n filtered_albums.append(album)\n return filtered_albums\n\n def get_api_result(artist=None, artist_id=None, album_id=None, track_id=None):\n if artist:\n try:\n return self.api.search_artist(artist, limit=1)['results'][0]\n except (deezer.api.DataException, IndexError):\n logger.error(f\"Artist {artist} not found.\")\n if artist_id:\n try:\n return self.api.get_artist_by_id(artist_id)\n except (deezer.api.DataException, IndexError):\n logger.error(f\"Artist ID {artist_id} not found.\")\n if album_id:\n try:\n return self.api.get_album(album_id)\n except (deezer.api.DataException, IndexError):\n logger.error(f\"Album ID {album_id} not found.\")\n if track_id:\n try:\n return self.api.get_track(track_id)\n except (deezer.api.DataException, IndexError):\n logger.error(f\"Track ID {track_id} not found.\")\n\n def queue_filtered_releases(api_object):\n filtered = filter_artist_by_record_type(api_object)\n filtered = common.exclude_filtered_versions(filtered)\n\n for album in filtered:\n if not queue_item_exists(album['id']):\n self.queue_list.append(QueueItem(artist=api_object, album=album))\n\n def queue_item_exists(i):\n for q in self.queue_list:\n if q.album_id == i:\n logger.debug(f\"Album ID {i} is already in queue\")\n self.duplicate_id_count += 1\n return True\n return False\n\n def process_artist_by_name(name):\n artist_result = get_api_result(artist=name)\n if not artist_result:\n return\n logger.debug(f\"Requested Artist: '{name}', Found: '{artist_result['name']}'\")\n if artist_result:\n queue_filtered_releases(artist_result)\n\n def process_artist_by_id(i):\n artist_id_result = get_api_result(artist_id=i)\n if not artist_id_result:\n return\n logger.debug(f\"Requested Artist ID: {i}, Found: {artist_id_result['name']}\")\n if artist_id_result:\n queue_filtered_releases(artist_id_result)\n\n def process_album_by_id(i):\n logger.debug(\"Processing album by ID\")\n album_id_result = get_api_result(album_id=i)\n if not album_id_result:\n logger.debug(f\"Album ID {i} was not found\")\n return\n logger.debug(f\"Requested album: {i}, \"\n f\"Found: {album_id_result['artist']['name']} - {album_id_result['title']}\")\n if album_id_result and not queue_item_exists(album_id_result['id']):\n self.queue_list.append(QueueItem(album=album_id_result))\n\n def process_track_by_id(id):\n logger.debug(\"Processing track by ID\")\n track_id_result = get_api_result(track_id=id)\n if not track_id_result:\n return\n logger.debug(f\"Requested track: {id}, \"\n f\"Found: {track_id_result['artist']['name']} - {track_id_result['title']}\")\n if track_id_result and not queue_item_exists(id):\n self.queue_list.append(QueueItem(track=track_id_result))\n\n def process_track_file(id):\n if not queue_item_exists(id):\n track_data = {\n \"artist\": {\n \"name\": \"TRACK ID\"\n },\n \"id\": id,\n \"title\": id\n }\n self.queue_list.append(QueueItem(track=track_data))\n\n def process_playlist_by_id(id):\n playlist_api = self.api.get_playlist(id)\n self.queue_list.append(QueueItem(playlist=playlist_api))\n\n def extract_id_from_url(url):\n id_group = ['artist', 'album', 'track', 'playlist']\n for group in id_group:\n id_type = group\n try:\n # Strip ID from URL\n id_from_url = url.split(f'/{group}/')[1]\n\n # Support for share links: http://deezer.com/us/track/12345?utm_campaign...\n id_from_url_extra = id_from_url.split('?')[0]\n\n id = int(id_from_url_extra)\n logger.debug(f\"Extracted group={id_type}, id={id}\")\n return id_type, id\n except (IndexError, ValueError) as e:\n continue\n return False, False\n\n logger.info(\"[!] Queueing releases, this might take awhile...\")\n\n if self.release_from or self.release_to:\n if self.release_from and self.release_to:\n logger.info(\":: Getting releases that were released between \"\n f\"{dates.ui_date(self.release_from)} and \"\n f\"{dates.ui_date(self.release_to)}\")\n elif self.release_from:\n logger.info(\":: Getting releases that were released after \"\n f\"{dates.ui_date(self.release_from)}\")\n elif self.release_to:\n logger.info(\":: Getting releases that were released before \"\n f\"{dates.ui_date(self.release_to)}\")\n\n if monitored:\n artist_id = self.db.get_all_monitored_artist_ids()\n\n if artist:\n [process_artist_by_name(a) for a in artist]\n\n if artist_id:\n [process_artist_by_id(i) for i in artist_id]\n\n if album_id:\n [process_album_by_id(i) for i in album_id]\n\n if track_id:\n [process_track_by_id(i) for i in track_id]\n\n if album_file:\n logger.info(f\":: Reading from file {album_file}\")\n if Path(album_file).exists():\n album_list = utils.dataprocessor.read_file_as_csv(album_file, split_new_line=False)\n album_list = utils.dataprocessor.process_input_file(album_list)\n if album_list:\n if isinstance(album_list[0], int):\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n _api_results = list(tqdm(ex.map(process_album_by_id, album_list),\n total=len(album_list),\n desc=f\"Fetching album data for {len(album_list)} \"\n f\"album(s), please wait...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT))\n else:\n logger.debug(f\"Invalid album ID: \\\"{album_list[0]}\\\"\")\n logger.error(f\"Invalid album ID file detected.\")\n else:\n logger.error(f\"The file {album_file} could not be found\")\n sys.exit()\n\n if artist_file:\n # TODO artist_file is in different format than album_file and track_file\n # TODO is one continuous CSV line better than separate lines?\n logger.info(f\":: Reading from file {artist_file}\")\n if Path(artist_file).exists():\n artist_list = utils.dataprocessor.read_file_as_csv(artist_file)\n if artist_list:\n if isinstance(artist_list[0], int):\n logger.debug(f\"{artist_file} contains artist IDs\")\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n _api_results = list(tqdm(ex.map(process_artist_by_id, artist_list),\n total=len(artist_list),\n desc=f\"Fetching artist release data for {len(artist_list)} \"\n f\"artist(s), please wait...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT))\n elif isinstance(artist_list[0], str):\n logger.debug(f\"{artist_file} contains artist names\")\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n _api_results = list(tqdm(ex.map(process_artist_by_name, artist_list),\n total=len(artist_list),\n desc=f\"Fetching artist release data for {len(artist_list)} \"\n f\"artist(s), please wait...\",\n ascii=\" #\",\n bar_format=ui.TQDM_FORMAT))\n else:\n logger.error(f\"The file {artist_file} could not be found\")\n sys.exit()\n\n if track_file:\n logger.info(f\":: Reading from file {track_file}\")\n if Path(track_file).exists():\n track_list = utils.dataprocessor.read_file_as_csv(track_file, split_new_line=False)\n try:\n track_list = [int(x) for x in track_list]\n except TypeError:\n logger.info(\"Track file must only contain track IDs\")\n return\n\n if track_list:\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n _api_results = list(tqdm(ex.map(process_track_file, track_list),\n total=len(track_list),\n desc=f\"Fetching track release data for {len(track_list)} \"\n f\"track(s), please wait...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT))\n else:\n logger.error(f\"The file {track_file} could not be found\")\n sys.exit()\n\n if url:\n logger.debug(\"Processing URLs\")\n for u in url:\n egroup, eid = extract_id_from_url(u)\n if not egroup or not eid:\n logger.error(f\"Invalid URL -- {u}\")\n continue\n\n if egroup == \"artist\":\n process_artist_by_id(eid)\n elif egroup == \"album\":\n process_album_by_id(eid)\n elif egroup == \"playlist\":\n process_playlist_by_id(eid)\n elif egroup == \"track\":\n process_track_by_id(eid)\n\n if self.duplicate_id_count > 0:\n logger.info(f\"Cleaned up {self.duplicate_id_count} duplicate release(s). See log for additional info.\")\n\n if auto:\n if len(self.queue_list):\n self.download_queue()\n else:\n print(\"\")\n logger.info(\"No releases found matching applied filters.\")\n"
},
{
"path": "deemon/cmd/extra.py",
"content": "from concurrent.futures import ThreadPoolExecutor\nimport logging\nfrom tqdm import tqdm\nfrom deemon.core import db as dbase\nfrom deemon.core.api import PlatformAPI\nfrom deemon.core.config import Config as config\nfrom deemon.utils import ui, performance\n\nlogger = logging.getLogger(__name__)\n\n\n\ndef debugger(message: str, payload = None):\n if config.debug_mode():\n if not payload:\n payload = \"\"\n logger.debug(f\"DEBUG_MODE: {message} {str(payload)}\")\n \n\ndef main():\n db = dbase.Database()\n api = PlatformAPI()\n releases = db.get_artist_releases()\n if not len(releases):\n return logger.warning(\"No releases found in local database\")\n logger.debug(\"Fetching extra release data...\")\n debugger(\"SpawningThreads\", api.max_threads)\n with ThreadPoolExecutor(max_workers=api.max_threads) as ex:\n api_result = list(\n tqdm(ex.map(api.get_extra_release_info, releases),\n total=len(releases),\n desc=f\"Fetching extra release data for {len(releases):,} \"\n \"releases, please wait...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n )\n\n if len(api_result):\n logger.info(\":: Saving changes to database, this can take several minutes...\")\n db.add_extra_release_info(api_result)\n db.commit()\n print(\"\")\n performance.operation_time(config.get('start_time'))\n logger.info(\"Extra release info has been updated\")\n"
},
{
"path": "deemon/cmd/generate.py",
"content": "from pathlib import Path\n\nimport tqdm as tqdm\nfrom deezer import Deezer\n\n\ndef read_album_ids_from_file(filename):\n if not Path(filename).exists():\n raise Exception(\"File does not exist\")\n ids = []\n with open(filename, 'r') as f:\n f.readline()\n for l in f:\n ids.append(l.encode(\"ascii\", \"ignore\"))\n print(\"Total lines read from text file: \" + str(len(ids)))\n return ids\n\n\ndef clean_absolute_paths(album_list):\n stripped = []\n for line in album_list:\n stripped.append(line.split(\"\\\\\"))\n return stripped\n\n\ndef clean_year_from_album(album_list, level: int = 5):\n artist_album = []\n for line in album_list:\n if len(line) == level:\n strip_year = line[3][-6:].strip(\"()\")\n try:\n int(strip_year)\n except Exception:\n artist_album.append([line[2], line[3]])\n continue\n artist_album.append([line[2], line[3][:-6]])\n return artist_album\n\n\ndef clean_artist_album_text(album_list: list):\n stripped = []\n for line in album_list:\n line = line.decode()\n line = line.replace('\\n', '')\n line = line.replace('�', '')\n line = line.replace('¡', '')\n line = line.replace('É', 'E')\n line = line.replace('.', '')\n line = line.replace('+', ' ')\n line = line.replace('/', ' ')\n stripped.append(line.split(\" - \"))\n return stripped\n\n\ndef get_artist_album(filename: str, absolute_path: bool = False):\n list_from_file = read_album_ids_from_file(filename)\n if absolute_path:\n stripped_paths = clean_absolute_paths(list_from_file)\n artist_album = clean_year_from_album(stripped_paths, level=5)\n else:\n artist_album = clean_artist_album_text(list_from_file)\n print(\"Total albums to lookup: \" + str(len(artist_album)))\n return sorted(x for x in artist_album)\n\n\ndef get_api_results(album_list, artist_name: str = None):\n dz = Deezer()\n\n for x in album_list:\n album_list.set_description_str(f\"Pass: {str(len(id_list))} | Fail: {str(len(fail_list))}\")\n # For testing, only process this artist\n if artist_name and artist_name != x[0]:\n continue\n # Remove things like \"(Bonus Tracks)\"\n artist_from_file = x[0]\n album_from_file = x[1]\n stripped_album_from_file = x[1].split(\" (\")[0]\n\n api_artist = dz.api.search_artist(x[0], limit=10)['data']\n found_artist = True\n for artist_result in api_artist:\n # TODO name decode - replace unknown with ? - use as wildcard\n encoded_name = artist_result['name'].encode(\"ascii\", \"replace\")\n decoded_name = encoded_name.decode()\n if artist_from_file == decoded_name:\n api_artist = artist_result\n found_artist = True\n break\n else:\n found_artist = False\n\n # TODO make this add albums to id and break out\n if found_artist is False:\n for artist in api_artist:\n get_albums = dz.api.get_artist_albums(artist['id'])['data']\n if album_from_file in [x['title'] for x in get_albums]:\n api_artist = artist\n break\n #\n # print(\"Searched all albums, nothing matches!\")\n # exit()\n try:\n all_albums = dz.api.get_artist_albums(api_artist['id'])['data']\n except:\n fail_list.append(f\"{x[0]} - {x[1]}\")\n continue\n api_albums = [[x['title'].split(\" (\")[0], x['id']] for x in all_albums]\n for [title, id] in api_albums:\n clean_title = title\n clean_title = clean_title.replace('¡', '')\n clean_title = clean_title.replace('É', 'E')\n clean_title = clean_title.replace('.', '')\n clean_title = clean_title.replace(' + ', ' ')\n clean_title = clean_title.replace(' / ', ' ')\n if album_from_file.lower() == clean_title.lower() or stripped_album_from_file.lower() == clean_title.lower():\n if id not in id_list:\n id_list.append(id)\n break\n if album_from_file.lower() in clean_title.lower() or stripped_album_from_file.lower() in clean_title.lower():\n if id not in id_list:\n id_list.append(id)\n break\n else:\n fail_list.append(f\"{x[0]} - {x[1]}\")\n\n\nid_list = []\nfail_list = []\ninput_file_or_directory: str = None\noutput_file_passing: str = None\noutput_file_failing: str = None\n\nalbum_list = get_artist_album(input_file_or_directory)\n\nprogress = tqdm.tqdm(album_list, ascii=\" #\",\n bar_format='{desc}... {n_fmt}/{total_fmt} [{bar:40}] {percentage:3.0f}%')\nprogress.set_description_str(\"Getting IDs\")\n\nget_api_results(progress)\n\nprint(\"PASS: \" + str(len(id_list)))\nwith open(output_file_passing, \"w\") as f:\n for id in id_list:\n f.write(str(id) + \"\\n\")\n\nprint(\"FAIL: \" + str(len(fail_list)))\nwith open(output_file_failing, \"w\") as f:\n for id in fail_list:\n f.write(str(id) + \"\\n\")\n"
},
{
"path": "deemon/cmd/monitor.py",
"content": "import logging\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nfrom tqdm import tqdm\n\nfrom deemon.cmd import search\nfrom deemon.cmd.refresh import Refresh\nfrom deemon.core.api import PlatformAPI\nfrom deemon.core.config import Config as config\nfrom deemon.core.db import Database\nfrom deemon.utils import dataprocessor, ui\n\nlogger = logging.getLogger(__name__)\n\n\nclass Monitor:\n\n def __init__(self, active_api=None):\n self.bitrate = None\n self.alerts = False\n self.record_type = None\n self.download_path = None\n self.remove = False\n self.refresh = True\n self.is_search = False\n self.duplicates = 0\n self.time_machine = None\n self.dl = None\n self.db = Database()\n self.api = active_api or PlatformAPI()\n\n def set_config(self, bitrate: str, alerts: bool, record_type: str, download_path: Path):\n self.bitrate = bitrate\n self.alerts = alerts\n self.record_type = record_type\n self.download_path = download_path\n self.debugger(\"SetConfig\", {'bitrate': bitrate, 'alerts': alerts, 'type': record_type, 'path': download_path})\n\n def set_options(self, remove, dl_all, search):\n self.remove = True if remove else False\n self.dl = True if dl_all else False\n self.is_search = True if search else False\n self.debugger(\"SetOptions\", {'remove': remove, 'dl': dl_all, 'search': search})\n\n def debugger(self, message: str, payload=None):\n if config.debug_mode():\n if not payload:\n payload = \"\"\n logger.debug(f\"DEBUG_MODE: {message} {str(payload)}\")\n\n def get_best_result(self, api_result) -> list:\n name = api_result['query']\n\n if self.is_search:\n logger.debug(\"Waiting for user input...\")\n prompt = self.prompt_search(name, api_result['results'])\n if prompt:\n logger.debug(f\"User selected {prompt}\")\n return [prompt]\n\n matches = [r for r in api_result['results'] if r['name'].lower() == name.lower()]\n self.debugger(\"Matches\", matches)\n\n if len(matches) == 1:\n return [matches[0]]\n elif len(matches) > 1:\n logger.debug(f\"Multiple matches were found for artist \\\"{api_result['query']}\\\"\")\n if config.prompt_duplicates():\n logger.debug(\"Waiting for user input...\")\n prompt = self.prompt_search(name, matches)\n if prompt:\n logger.debug(f\"User selected {prompt}\")\n return [prompt]\n else:\n logger.info(f\"No selection made, skipping {name}...\")\n return []\n else:\n self.duplicates += 1\n return [matches[0]]\n elif not len(matches):\n logger.debug(f\" [!] No matches were found for artist \\\"{api_result['query']}\\\"\")\n if config.prompt_no_matches() and len(api_result['results']):\n logger.debug(\"Waiting for user input...\")\n prompt = self.prompt_search(name, api_result['results'])\n if prompt:\n logger.debug(f\"User selected {prompt}\")\n return [prompt]\n else:\n logger.info(f\"No selection made, skipping {name}...\")\n return []\n else:\n logger.info(f\" [!] Artist {name} not found\")\n return []\n\n def prompt_search(self, value, api_result):\n menu = search.Search(active_api=self.api)\n ask_user = menu.artist_menu(value, api_result, True)\n if ask_user:\n return {'id': ask_user['id'], 'name': ask_user['name']}\n return logger.debug(\"No artist selected, skipping...\")\n\n # @performance.timeit\n def build_artist_query(self, api_result: list):\n existing = self.db.get_all_monitored_artist_ids()\n artists_to_add = []\n pbar = tqdm(api_result, total=len(api_result), desc=\"Setting up artists for monitoring...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n for artist in pbar:\n if artist is None:\n continue\n if artist['id'] in existing:\n logger.info(f\" - Already monitoring {artist['name']}, skipping...\")\n else:\n artist.update({'bitrate': self.bitrate, 'alerts': self.alerts, 'record_type': self.record_type,\n 'download_path': self.download_path, 'profile_id': config.profile_id(),\n 'trans_id': config.transaction_id()})\n artists_to_add.append(artist)\n if len(artists_to_add):\n logger.debug(\"New artists have been monitored. Saving changes to the database...\")\n self.db.new_transaction()\n self.db.fast_monitor(artists_to_add)\n self.db.commit()\n return True\n\n def build_playlist_query(self, api_result: list, include_artists: bool):\n\n if include_artists:\n include_artists = '1'\n\n existing = self.db.get_all_monitored_playlist_ids() or []\n playlists_to_add = []\n pbar = tqdm(api_result, total=len(api_result), desc=\"Setting up playlists for monitoring...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n for i, playlist in enumerate(pbar):\n if not playlist:\n continue\n if playlist['id'] in existing:\n logger.info(f\" Already monitoring {playlist['title']}, skipping...\")\n else:\n playlist.update(\n {\n 'bitrate': self.bitrate,\n 'alerts': self.alerts,\n 'download_path': self.download_path,\n 'profile_id': config.profile_id(),\n 'trans_id': config.transaction_id(),\n 'monitor_artists': include_artists\n }\n )\n playlists_to_add.append(playlist)\n if len(playlists_to_add):\n logger.debug(\"New playlists have been monitored. Saving changes to the database...\")\n self.db.new_transaction()\n self.db.fast_monitor_playlist(playlists_to_add)\n self.db.commit()\n return True\n\n def call_refresh(self):\n refresh = Refresh(self.time_machine, ignore_filters=self.dl, active_api=self.api)\n refresh.run()\n\n # @performance.timeit\n def artists(self, names: list) -> None:\n \"\"\"\n Return list of dictionaries containing each artist\n \"\"\"\n if self.remove:\n return self.purge_artists(names=names)\n self.debugger(\"SpawningThreads\", self.api.max_threads)\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n api_result = list(\n tqdm(ex.map(self.api.search_artist, names), total=len(names),\n desc=f\"Fetching artist data for {len(names):,} artist(s), please wait...\",\n ascii=\" #\", bar_format=ui.TQDM_FORMAT))\n\n select_artist = tqdm(api_result, total=len(api_result), desc=\"Examining results for best match...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n\n to_monitor = []\n for artist in select_artist:\n best_result = self.get_best_result(artist)\n if best_result:\n to_monitor.append(best_result)\n\n to_process = [item for elem in to_monitor for item in elem if len(item)]\n if self.build_artist_query(to_process):\n self.call_refresh()\n else:\n print(\"\")\n logger.info(\"No new artists have been added, skipping refresh.\")\n\n # @performance.timeit\n def artist_ids(self, ids: list):\n ids = [int(x) for x in ids]\n if self.remove:\n return self.purge_artists(ids=ids)\n self.debugger(\"SpawningThreads\", self.api.max_threads)\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n api_result = list(\n tqdm(ex.map(self.api.get_artist_by_id, ids), total=len(ids),\n desc=f\"Fetching artist data for {len(ids):,} artist(s), please wait...\",\n ascii=\" #\", bar_format=ui.TQDM_FORMAT))\n\n if self.build_artist_query(api_result):\n self.call_refresh()\n else:\n print(\"\")\n logger.info(\"No new artists have been added, skipping refresh.\")\n\n # @performance.timeit\n def importer(self, import_path: str):\n if Path(import_path).is_file():\n imported_file = dataprocessor.read_file_as_csv(import_path)\n artist_list = dataprocessor.process_input_file(imported_file)\n if isinstance(artist_list[0], int):\n self.artist_ids(artist_list)\n else:\n self.artists(artist_list)\n elif Path(import_path).is_dir():\n import_list = [x.relative_to(import_path).name for x in sorted(Path(import_path).iterdir()) if x.is_dir()]\n if import_list:\n self.artists(import_list)\n else:\n logger.error(f\"File or directory not found: {import_path}\")\n return\n\n # @performance.timeit\n def playlists(self, playlists: list, include_artists: bool):\n if self.remove:\n return self.purge_playlists(ids=playlists)\n ids = [int(x) for x in playlists]\n self.debugger(\"SpawningThreads\", self.api.max_threads)\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n api_result = list(\n tqdm(ex.map(self.api.get_playlist, ids), total=len(ids),\n desc=f\"Fetching playlist data for {len(ids):,} playlist(s), please wait...\",\n ascii=\" #\", bar_format=ui.TQDM_FORMAT))\n\n if self.build_playlist_query(api_result, include_artists):\n self.call_refresh()\n else:\n print(\"\")\n logger.info(\"No new playlists have been added, skipping refresh.\")\n\n # @performance.timeit\n def purge_artists(self, names: list = None, ids: list = None):\n if names:\n for n in names:\n monitored = self.db.get_monitored_artist_by_name(n)\n if monitored:\n self.db.remove_monitored_artist(monitored['artist_id'])\n logger.info(f\"No longer monitoring {monitored['artist_name']}\")\n else:\n logger.info(f\"{n} is not being monitored yet\")\n if ids:\n for i in ids:\n monitored = self.db.get_monitored_artist_by_id(i)\n if monitored:\n self.db.remove_monitored_artist(monitored['artist_id'])\n logger.info(f\"\\nNo longer monitoring {monitored['artist_name']}\")\n else:\n logger.info(f\"{i} is not being monitored yet\")\n\n def purge_playlists(self, titles: list = None, ids: list = None):\n if ids:\n for i in ids:\n monitored = self.db.get_monitored_playlist_by_id(i)\n if monitored:\n self.db.remove_monitored_playlists(monitored['id'])\n logger.info(f\"\\nNo longer monitoring {monitored['title']}\")\n else:\n logger.info(f\"{i} is not being monitored yet\")\n"
},
{
"path": "deemon/cmd/profile.py",
"content": "import logging\n\nfrom deemon.core.config import Config as config\nfrom deemon.core.db import Database\n\nlogger = logging.getLogger(__name__)\n\n\nclass ProfileConfig:\n def __init__(self, profile_name):\n self.db = Database()\n self.profile_name = profile_name\n self.profile = None\n\n # TODO move this to utils\n @staticmethod\n def print_header(message: str = None):\n print(\"deemon Profile Editor\")\n if message:\n print(\":: \" + message + \"\\n\")\n else:\n print(\"\")\n\n def edit(self):\n profile = self.db.get_profile(self.profile_name)\n self.print_header(f\"Configuring '{profile['name']}' (Profile ID: {profile['id']})\")\n modified = 0\n for property in profile:\n if property == \"id\":\n continue\n allowed_opts = config.allowed_values(property)\n if isinstance(allowed_opts, dict):\n allowed_opts = [str(x.lower()) for x in allowed_opts.values()]\n\n while True:\n friendly_text = property.replace(\"_\", \" \").title()\n user_input = input(f\"{friendly_text} [{profile[property]}]: \").lower()\n if user_input == \"\":\n break\n # TODO move to function to share with Config.set()?\n elif user_input == \"false\" or user_input == \"0\":\n user_input = False\n elif user_input == \"true\" or user_input == \"1\":\n user_input = True\n elif property == \"name\" and self.profile_name != user_input:\n if self.db.get_profile(user_input):\n print(\"Name already in use\")\n continue\n if user_input == \"none\" and property != \"name\":\n user_input = None\n elif allowed_opts:\n if user_input not in allowed_opts:\n print(f\"Allowed options: \" + ', '.join(str(x) for x in allowed_opts))\n continue\n logger.debug(f\"User set {property} to {user_input}\")\n profile[property] = user_input\n modified += 1\n break\n\n if modified > 0:\n user_input = input(\"\\n:: Save these settings? [y|N] \")\n if user_input.lower() != \"y\":\n logger.info(\"No changes made, exiting...\")\n else:\n self.db.update_profile(profile)\n print(f\"\\nProfile '{profile['name']}' has been updated!\")\n else:\n print(\"No changes made, exiting...\")\n\n def add(self):\n new_profile = {}\n profile_config = self.db.get_profile(self.profile_name)\n if profile_config:\n return logger.error(f\"Profile {self.profile_name} already exists\")\n else:\n logger.info(\"Adding new profile: \" + self.profile_name)\n print(\"** Any option left blank will fallback to global config **\\n\")\n new_profile['name'] = self.profile_name\n\n menu = [\n {'setting': 'email', 'type': str, 'text': 'Email address', 'allowed': []},\n {'setting': 'alerts', 'type': bool, 'text': 'Alerts', 'allowed': config.allowed_values('alerts')},\n {'setting': 'bitrate', 'type': str, 'text': 'Bitrate',\n 'allowed': config.allowed_values('bitrate').values()},\n {'setting': 'record_type', 'type': str, 'text': 'Record Type',\n 'allowed': config.allowed_values('record_type')},\n {'setting': 'plex_baseurl', 'type': str, 'text': 'Plex Base URL', 'allowed': []},\n {'setting': 'plex_token', 'type': str, 'text': 'Plex Token', 'allowed': []},\n {'setting': 'plex_library', 'type': str, 'text': 'Plex Library', 'allowed': []},\n {'setting': 'download_path', 'type': str, 'text': 'Download Path', 'allowed': []},\n ]\n\n for m in menu:\n repeat = True\n while repeat:\n i = input(m['text'] + \": \")\n if i == \"\":\n new_profile[m['setting']] = None\n break\n if not isinstance(i, m['type']):\n try:\n i = int(i)\n except ValueError:\n print(\" - Allowed options: \" + ', '.join(str(x) for x in m['allowed']))\n continue\n if len(m['allowed']) > 0:\n if i not in m['allowed']:\n print(\" - Allowed options: \" + ', '.join(str(x) for x in m['allowed']))\n continue\n new_profile[m['setting']] = i\n break\n\n print(\"\\n\")\n i = input(\":: Save these settings? [y|N] \")\n if i.lower() != \"y\":\n return logger.info(\"Operation cancelled. No changes saved.\")\n else:\n self.db.create_profile(new_profile)\n logger.debug(f\"New profile created with the following configuration: {new_profile}\")\n\n def delete(self):\n profile_config = self.db.get_profile(self.profile_name)\n if not profile_config:\n return logger.error(f\"Profile {self.profile_name} not found\")\n\n if profile_config['id'] == 1:\n return logger.info(\"You cannot delete the default profile.\")\n\n i = input(f\":: Remove the profile '{self.profile_name}'? [y|N] \")\n if i.lower() == \"y\":\n self.db.delete_profile(self.profile_name)\n return logger.info(\"Profile \" + self.profile_name + \" deleted.\")\n else:\n return logger.info(\"Operation cancelled\")\n\n def show(self):\n if not self.profile_name:\n profile = self.db.get_all_profiles()\n self.print_header(f\"Showing all profiles\")\n else:\n profile = [self.db.get_profile(self.profile_name)]\n self.print_header(f\"Showing profile '{profile[0]['name']}' (Profile ID: {profile[0]['id']})\")\n if len(profile) == 0:\n return logger.error(f\"Profile {self.profile_name} not found\")\n\n print(\"{:<10} {:<40} {:<8} {:<8} {:<8} {:<25} \"\n \"{:<20} {:<20} {:<20}\".format('Name', 'Email', 'Alerts', 'Bitrate', 'Type',\n 'Plex Base URL', 'Plex Token', 'Plex Library', 'Download Path'))\n for u in profile:\n id, name, email, alerts, bitrate, rtype, url, token, \\\n lib, dl_path = [x if x is not None else '' for x in u.values()]\n print(\"{:<10} {:<40} {:<8} {:<8} {:<8} {:<25} \"\n \"{:<20} {:<20} {:<20}\".format(name, email, alerts, bitrate, rtype, url, token, lib, dl_path))\n print(\"\")\n\n def clear(self):\n profile = self.db.get_profile(self.profile_name)\n self.print_header(f\"Configuring '{profile['name']}' (Profile ID: {profile['id']})\")\n if not profile:\n return logger.error(f\"Profile {self.profile_name} not found\")\n\n for value in profile:\n if value in [\"id\", \"name\"]:\n continue\n profile[value] = None\n self.db.update_profile(profile)\n logger.info(\"All values have been cleared.\")\n"
},
{
"path": "deemon/cmd/refresh.py",
"content": "import logging\nimport re\nimport time\nfrom concurrent.futures import ThreadPoolExecutor\nfrom datetime import datetime, timedelta\n\nfrom tqdm import tqdm\n\nfrom deemon.cmd.download import QueueItem, Download\nfrom deemon.core import db, api, notifier, common\nfrom deemon.core.config import Config as config\nfrom deemon.utils import dates, ui, performance\n\nlogger = logging.getLogger(__name__)\n\n\nclass Refresh:\n def __init__(self, time_machine: datetime = None, skip_download: bool = False, ignore_filters: bool = False, active_api=None):\n self.db = db.Database()\n self.refresh_date = datetime.now()\n self.max_refresh_date = None\n self.api = active_api or api.PlatformAPI()\n self.new_releases = []\n self.new_releases_alert = []\n self.new_playlist_releases = []\n self.time_machine = time_machine\n self.total_new_releases = 0\n self.queue_list = []\n self.skip_download = skip_download\n self.download_all = ignore_filters\n self.seen = None\n\n if self.time_machine:\n logger.info(f\":: Time Machine active: {datetime.strftime(self.time_machine, '%b %d, %Y')}!\")\n config._CONFIG['new_releases']['release_max_age'] = 0\n if not self.waiting_for_refresh():\n self.db.remove_specific_releases({'tm_date': str(self.time_machine)})\n self.db.commit()\n\n @staticmethod\n def debugger(message: str, payload = None):\n if config.debug_mode():\n if not payload:\n payload = \"\"\n logger.debug(f\"DEBUG_MODE: {message} {str(payload)}\")\n\n def remove_existing_releases(self, payload: dict, seen: dict) -> list:\n \"\"\"\n Return list of releases that have not been stored in the database\n \"\"\"\n new_releases = []\n\n if payload.get('artist_id'):\n seen_releases = seen\n if seen_releases:\n seen_releases = [v for x in seen_releases for k, v in x.items() if not x.get('future_release', 0)]\n new_releases = [x for x in payload['releases'] if type(x) == dict for k, v in x.items() if\n k == \"id\" and v not in seen_releases]\n return new_releases\n return [x for x in payload['releases']]\n\n if payload.get('tracks'):\n playlist_id = payload['id']\n seen_releases = self.db.get_playlist_tracks(playlist_id)\n if seen_releases:\n seen_releases = [v for x in seen_releases for k, v in x.items()]\n new_releases = [x for x in payload['tracks']\n if type(x) == dict for k, v in x.items()\n if k == \"id\" and v not in seen_releases]\n return new_releases\n return [x for x in payload['tracks']]\n\n return new_releases\n\n def filter_artist_releases(self, payload: dict):\n \"\"\" Inspect artist releases and decide what to do with each release \"\"\"\n self.debugger(f\"{payload['artist_name']} has {len(payload['releases'])} new releases\")\n\n for release in payload['releases']:\n release['artist_id'] = payload['artist_id']\n release['artist_name'] = payload['artist_name']\n release['bitrate'] = payload['bitrate'] or config.bitrate()\n release['download_path'] = payload['download_path'] or config.download_path()\n release['future'] = self.is_future_release(release['release_date'])\n release['alerts'] = payload['alerts']\n \n if release['explicit_lyrics'] != 1:\n release['explicit_lyrics'] = 0\n \n self.append_database_release(release)\n \n if release['future']:\n continue\n\n if not common.exclude_filtered_versions([{'title': release['title']}]):\n # exclude_filtered_versions returns empty list if excluded\n continue\n\n explicit_album_id = self.explicit_id(release['title'], payload['releases'])\n if explicit_album_id:\n if explicit_album_id == release['id']:\n logger.debug(f\"An explicit release was found for {release['title']}\")\n else:\n continue\n\n if self.download_all:\n self.queue_release(release)\n continue\n\n if not self.allowed_record_type(payload['record_type'], release['record_type']):\n logger.debug(f\"Record type \\\"{release['record_type']}\\\" has been filtered out, skipping release \"\n f\"{release['id']}\")\n continue\n\n if self.release_too_old(release['release_date']):\n logger.debug(f\"Release {release['id']} is too old, skipping it.\")\n continue\n\n if not payload['refreshed'] and not self.time_machine:\n continue\n\n self.queue_release(release)\n\n def append_database_release(self, new_release: dict):\n self.new_releases.append(new_release)\n \n @staticmethod\n def explicit_id(release_title: str, payload: list):\n for release in payload:\n if release['title'] == release_title:\n if release['explicit_lyrics'] == 1:\n return release['id']\n\n def release_too_old(self, release_date: str):\n release_date_dt = dates.str_to_datetime_obj(release_date)\n if self.time_machine:\n if release_date_dt <= self.time_machine:\n self.debugger(f\"Release date \\\"{release_date}\\\" is older than TIME_MACHINE ({str(dates.ui_date(self.time_machine))})\")\n return True\n if config.release_max_age():\n if release_date_dt < (self.refresh_date - timedelta(config.release_max_age())):\n self.debugger(f\"Release date \\\"{release_date}\\\" is older than RELEASE_MAX_AGE ({config.release_max_age()} day(s))\")\n return True\n \n\n @staticmethod\n def is_future_release(release_date: str):\n \"\"\" Return 1 if release date is in future, otherwise return 0 \"\"\"\n release_date_dt = dates.str_to_datetime_obj(release_date)\n if release_date_dt > datetime.now():\n return 1\n else:\n return 0\n\n @staticmethod\n def allowed_record_type(artist_rec_type, release_rec_type: str):\n \"\"\" Compare actual record_type against allowable \"\"\"\n \n if artist_rec_type:\n if artist_rec_type == release_rec_type or artist_rec_type == \"all\":\n return True\n else:\n return\n elif config.record_type() == release_rec_type:\n return True\n elif config.record_type() == \"all\":\n return True\n\n def queue_release(self, release: dict):\n \"\"\" Add release to download queue and create alert notification \"\"\"\n\n # Create notification of release if per-artist is set to True\n if release['alerts'] is not False and config.alerts():\n self.create_notification(release)\n self.queue_list.append(QueueItem(release_full=release))\n\n def filter_playlist_releases(self, payload: dict):\n self.debugger(f\"Filtering {len(payload['tracks'])} tracks for playlist {payload['title']}\")\n\n if len(payload['tracks']):\n for track in payload['tracks']:\n new_track = track.copy()\n new_track['playlist_id'] = payload['id']\n self.new_playlist_releases.append(new_track)\n\n if payload['refreshed'] == 0:\n return\n\n queue_obj = QueueItem(playlist=payload, bitrate=payload['bitrate'], download_path=payload['download_path'])\n self.debugger(\"QueuePlaylistItem\", queue_obj)\n self.queue_list.append(queue_obj)\n\n def waiting_for_refresh(self):\n playlists = self.db.get_unrefreshed_playlists()\n artists = self.db.get_unrefreshed_artists()\n if len(playlists) or len(artists):\n return {'artists': artists, 'playlists': playlists}\n\n def prep_payload(self, p):\n if len(p):\n p['releases'] = self.remove_existing_releases(p, self.seen)\n self.filter_artist_releases(p)\n else:\n logger.debug(\"No payload provided\")\n\n def run(self, artists: list = None, playlists: list = None):\n\n if config.check_account_status():\n if self.api.account_type == \"free\" and config.bitrate() != \"128\":\n notification = notifier.Notify()\n notification.expired_arl()\n return logger.error(\" [X] ARL expired? Deezer account only allows low\"\n \" quality. If you wish to download \"\n \"anyway, set `check_account_status` \"\n \"to False in the config.\")\n\n if artists:\n self.debugger(\"ManualRefresh\", artists)\n monitored_artists = [x for x in (self.db.get_monitored_artist_by_name(a) for a in artists) if x]\n if not len(monitored_artists):\n return logger.warning(\"Specified artist(s) were not found\")\n api_result = self.get_release_data({'artists': monitored_artists})\n elif playlists:\n self.debugger(\"ManualRefresh\", playlists)\n monitored_playlists = [x for x in (self.db.get_monitored_playlist_by_name(p) for p in playlists) if x]\n if not len(monitored_playlists):\n return logger.warning(\"Specified playlist(s) were not found\")\n api_result = self.get_release_data({'playlists': monitored_playlists})\n else:\n waiting = self.waiting_for_refresh()\n if waiting:\n logger.debug(f\"There are {len(waiting['playlists'])} playlist(s) and \"\n f\"{len(waiting['artists'])} artist(s) waiting to be refreshed.\")\n api_result = self.get_release_data(waiting)\n else:\n self.debugger(\"FullRefresh\")\n monitored_playlists = self.db.get_all_monitored_playlists()\n monitored_artists = self.db.get_all_monitored_artists()\n if not len(monitored_playlists) and not len(monitored_artists):\n return logger.warning(\"No artists found to refresh\")\n api_result = self.get_release_data({'artists': monitored_artists, 'playlists': monitored_playlists})\n\n if len(api_result):\n self.seen = self.db.get_artist_releases()\n payload_container = tqdm(api_result['artists'], total=len(api_result['artists']),\n desc=f\"Scanning release data for new releases...\",\n ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n for payload in payload_container:\n self.prep_payload(payload)\n\n playlist_monitor_artists = []\n for payload in api_result['playlists']:\n if payload and len(payload):\n self.seen = self.db.get_playlist_tracks(payload['id'])\n payload['tracks'] = self.remove_existing_releases(payload, self.seen)\n self.filter_playlist_releases(payload)\n\n if payload['monitor_artists']:\n logger.debug(f\"Artists from this playlist ({payload['id']}) are to be monitored!\")\n for track in payload['tracks']:\n playlist_monitor_artists.append(track['artist_id'])\n playlist_monitor_artists = list(set(playlist_monitor_artists))\n\n if self.skip_download:\n logger.info(f\" [!] You have opted to skip downloads, clearing {len(self.queue_list):,} item(s) from queue...\")\n self.queue_list.clear()\n self.new_releases_alert.clear()\n\n if len(self.queue_list):\n dl = Download(active_api=self.api)\n dl.download_queue(self.queue_list)\n\n if len(self.new_playlist_releases) or len(self.new_releases):\n if len(self.new_playlist_releases):\n logger.debug(\"Updating playlist releases in database...\")\n self.db.add_new_playlist_releases(self.new_playlist_releases)\n if len(self.new_releases):\n logger.debug(\"Updating artist releases in database...\")\n self.db.add_new_releases(self.new_releases)\n self.db.commit()\n self.db_stats()\n performance.operation_time(config.get('start_time'))\n logger.info(\"Database is up-to-date.\")\n else:\n self.db_stats()\n performance.operation_time(config.get('start_time'))\n logger.info(\"Database is up-to-date. No new releases were found.\")\n\n if len(self.new_releases_alert) > 0:\n notification = notifier.Notify(self.new_releases_alert)\n notification.send()\n\n if playlist_monitor_artists:\n print(\"\")\n logger.info(\":: New artists to monitor, stand by...\")\n time.sleep(2)\n from deemon.cmd.monitor import Monitor\n monitor = Monitor(active_api=self.api)\n monitor.artist_ids(playlist_monitor_artists)\n\n def db_stats(self):\n artists = len(self.db.get_all_monitored_artist_ids())\n playlists = len(self.db.get_all_monitored_playlist_ids())\n releases = len(self.db.get_artist_releases())\n future = len(self.db.get_future_releases())\n\n print(\"\")\n print(f\"+ Artists monitored: {artists:,}\")\n print(f\"+ Playlists monitored: {playlists:,}\")\n print(f\"+ Releases seen: {releases:,}\")\n print(f\"+ Pending future releases: {future:,}\")\n print(\"\")\n\n def get_release_data(self, to_refresh: dict) -> dict:\n \"\"\"\n Generate a list of dictionaries containing artist (DB) and release (API)\n information.\n \"\"\"\n\n api_result = {'artists': [], 'playlists': []}\n\n logger.debug(f\"Standby, starting refresh...\")\n\n if to_refresh.get('playlists') and len(to_refresh.get('playlists')):\n logger.debug(\"Fetching playlist track data...\")\n self.debugger(\"SpawningThreads\", self.api.max_threads)\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n api_result['playlists'] = list(\n tqdm(ex.map(self.api.get_playlist_tracks,\n to_refresh['playlists']),\n total=len(to_refresh['playlists']),\n desc=f\"Fetching playlist track data for \"\n f\"{len(to_refresh['playlists'])} playlist(s), \"\n \"please wait...\",\n ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n )\n\n if to_refresh.get('artists') and len(to_refresh['artists']):\n logger.debug(\"Fetching artist release data...\")\n self.debugger(\"SpawningThreads\", self.api.max_threads)\n with ThreadPoolExecutor(max_workers=self.api.max_threads) as ex:\n api_result['artists'] = list(\n tqdm(ex.map(self.api.get_artist_albums, to_refresh['artists']),\n total=len(to_refresh['artists']), desc=f\"Fetching artist release data for {len(to_refresh['artists']):,} artist(s), please wait...\", ascii=\" #\",\n bar_format=ui.TQDM_FORMAT)\n )\n return api_result\n\n def create_notification(self, release: dict):\n for days in self.new_releases_alert:\n for key in days:\n if key == \"release_date\":\n if release['release_date'] in days[key]:\n days[\"releases\"].append(\n {\n 'artist': release['artist_name'],\n 'album': release['title'],\n 'cover': release['cover_big'],\n 'url': release['link'],\n 'track_num': release.get('nb_tracks', None),\n 'record_type': release['record_type'],\n }\n )\n return\n\n self.new_releases_alert.append(\n {\n 'release_date': release['release_date'], \n 'releases': [\n {\n 'artist': release['artist_name'],\n 'album': release['title'],\n 'cover': release['cover_big'],\n 'url': release['link'],\n 'track_num': release.get('nb_tracks', None),\n 'record_type': release['record_type'],\n }\n ]\n }\n )\n"
},
{
"path": "deemon/cmd/rollback.py",
"content": "import logging\n\nfrom deemon.core.db import Database\nfrom deemon.utils import dates\n\nlogger = logging.getLogger(__name__)\ndb = Database()\n\n\ndef view_transactions():\n transactions = db.get_transactions()\n if not transactions:\n return logger.info(\"No transactions are available to be rolled back.\")\n\n for i, transaction in enumerate(transactions, start=1):\n release_id = []\n artist_names = []\n playlist_titles = []\n\n for k, v in transaction.items():\n if (k == \"releases\" or k == \"playlist_tracks\") and transaction[k]:\n for item in transaction[k]:\n for key, val in item.items():\n if key == \"album_id\" or key == \"track_id\":\n if item[key] not in release_id:\n release_id.append(item[key])\n if k == \"monitor\" and transaction[k]:\n if transaction[k] not in artist_names:\n artist_names = [x['artist_name'] for x in transaction[k]]\n if k == \"playlists\" and transaction[k]:\n if transaction[k] not in playlist_titles:\n playlist_titles.append(transaction[k][0]['title'])\n\n release_id_len = len(release_id)\n\n if release_id_len == 1:\n release_text = f\" and {release_id_len} release\"\n elif release_id_len > 1:\n release_text = f\" and {release_id_len} releases\"\n else:\n release_text = \"\"\n\n if len(artist_names) > 1:\n if len(playlist_titles) > 1:\n playlist_text = f\", {len(playlist_titles)} playlists\"\n elif len(playlist_titles) == 1:\n playlist_text = f\", {len(playlist_titles)} playlist\"\n else:\n playlist_text = \"\"\n output_text = f\"Added {artist_names[0]} + {len(artist_names) - 1} artist(s){playlist_text}{release_text}\"\n elif len(artist_names) == 1:\n if len(playlist_titles) > 1:\n playlist_text = f\", {len(playlist_titles)} playlists\"\n elif len(playlist_titles) == 1:\n playlist_text = f\", {len(playlist_titles)} playlist\"\n else:\n playlist_text = \"\"\n output_text = f\"Added {artist_names[0]}{playlist_text}{release_text}\"\n else:\n if len(playlist_titles) > 1:\n output_text = f\"Added {playlist_titles[0]} + {len(playlist_titles) - 1}{release_text}\"\n elif len(playlist_titles) == 1:\n output_text = f\"Added {playlist_titles[0]}{release_text}\"\n else:\n output_text = f\"Added {release_text[4:]}\"\n\n print(f\"{i}. {dates.get_friendly_date(transaction['timestamp'])} - {output_text}\")\n\n rollback = None\n while rollback not in range(len(transactions)):\n rollback = input(\"\\nSelect specific refresh to rollback (or press Enter to exit): \")\n if rollback == \"\":\n return\n try:\n rollback = int(rollback) - 1\n except ValueError:\n logger.error(\"Invalid input\")\n\n rollback = transactions[rollback]['id']\n logger.debug(f\"Rolling back transaction {rollback}\")\n db.rollback_refresh(rollback)\n\n\ndef rollback_last(i: int):\n db.rollback_last_refresh(i)\n logger.info(f\"Rolled back the last {i} transaction(s).\")\n"
},
{
"path": "deemon/cmd/search.py",
"content": "import logging\nimport sys\n\nfrom deezer import Deezer\n\nfrom deemon.cmd import download\nfrom deemon.cmd import monitor as mon\nfrom deemon.core import db, api\nfrom deemon.core.config import Config as config\nfrom deemon.utils import dates\n\nlogger = logging.getLogger(__name__)\n\n\nclass Search:\n def __init__(self, active_api=None):\n self.api = active_api or api.PlatformAPI()\n self.artist_id: int = None\n self.artist: str = None\n self.choices: list = []\n self.status_message: str = None\n self.queue_list = []\n self.select_mode = False\n self.explicit_only = False\n self.search_results: str = None\n\n self.sort: str = \"release_date\"\n self.filter: str = None\n self.desc: bool = True\n\n self.gte_year = None\n self.lte_year = None\n self.eq_year = None\n\n self.db = db.Database()\n self.dz = Deezer()\n\n @staticmethod\n def truncate_artist(name: str):\n if len(name) > 45:\n return name[0:40] + \"...\"\n return name\n\n def get_latest_release(self, artist_id: int):\n try:\n all_releases = self.dz.api.get_artist_albums(artist_id)['data']\n sorted_releases = sorted(all_releases, key=lambda x: x['release_date'], reverse=True)\n latest_release = sorted_releases[0]\n except IndexError:\n return \" - No releases found\"\n return f\" - Latest release: {latest_release['title']} ({dates.get_year(latest_release['release_date'])})\"\n\n def display_monitored_status(self, artist_id: int):\n if self.db.get_monitored_artist_by_id(artist_id):\n return \"[M] \"\n return \" \"\n\n @staticmethod\n def has_duplicate_artists(name: str, artist_dicts: dict):\n names = [x['name'] for x in artist_dicts if x['name'] == name]\n if len(names) > 1:\n return True\n\n def show_mini_queue(self):\n num_queued = len(self.queue_list)\n if num_queued > 0:\n return f\" ({str(num_queued)} Queued)\"\n return \"\"\n\n def search_menu(self, query: str = None):\n exit_search: bool = False\n quick_search: bool = False\n\n while exit_search is False:\n self.clear()\n print(\"deemon Interactive Search Client\\n\")\n if len(self.queue_list) > 0:\n self.display_options(options=\"(d) Download Queue (Q) Show Queue\")\n if query:\n search_query = query\n query = None\n else:\n search_query = input(f\":: Enter an artist to search for{self.show_mini_queue()}: \")\n if search_query == \"exit\":\n if self.exit_search():\n sys.exit()\n continue\n elif search_query == \"d\":\n if len(self.queue_list) > 0:\n self.start_queue()\n continue\n elif search_query == \"Q\":\n if len(self.queue_list) > 0:\n self.queue_menu()\n else:\n self.status_message = \"Queue is empty\"\n continue\n elif search_query == \"\":\n continue\n \n self.clear()\n print(\"deemon Interactive Search Client\\n\")\n self.search_results = self.api.search_artist(search_query, config.query_limit())\n if not self.search_results['results']:\n self.status_message = \"No results found for: \" + search_query\n continue\n\n smart_search = None\n if config.smart_search():\n for result in self.search_results['results']:\n if result['name'].lower() == search_query.lower():\n if not smart_search:\n smart_search = result\n else:\n smart_search = None\n break\n \n if smart_search:\n self.artist = smart_search['name']\n album_selected = self.album_menu(smart_search)\n if album_selected:\n return [album_selected]\n\n artist_selected = self.artist_menu(self.search_results['query'], self.search_results['results'], quick_search)\n if artist_selected:\n return [artist_selected]\n\n def queue_menu_options(self):\n ui_options = (\"(d) Download Queue (c) Clear Queue (b) Back\")\n self.display_options(options=ui_options)\n\n def artist_menu(self, query: str, results: dict, artist_only: bool = False):\n exit_artist: bool = False\n while exit_artist is False:\n self.clear()\n print(\"Search results for artist: \" + query)\n for idx, option in enumerate(results, start=1):\n print(f\"{self.display_monitored_status(option['id'])}{idx}. {self.truncate_artist(option['name'])}\")\n if self.has_duplicate_artists(option['name'], results):\n print(self.get_latest_release(option['id']))\n print(\" - Artist ID: \" + str(option['id']))\n if not option.get('nb_album'):\n option['nb_album'] = self.dz.api.get_artist(option['id'])['nb_album']\n print(\" - Total releases: \" + str(option['nb_album']))\n self.status_message = \"Duplicate artists found\"\n # TODO make options smarter/modular\n if len(self.queue_list) > 0:\n self.display_options(options=\"(b) Back (d) Download Queue (Q) Show Queue\")\n else:\n self.display_options(options=\"(b) Back\")\n response = input(f\":: Please choose an option or type 'exit'{self.show_mini_queue()}: \")\n if response == \"d\":\n if len(self.queue_list) > 0:\n self.start_queue()\n continue\n elif response == \"Q\":\n if len(self.queue_list) > 0:\n self.queue_menu()\n else:\n self.status_message = \"Queue is empty\"\n continue\n elif response == \"b\":\n break\n elif response == \"exit\":\n if self.exit_search() and not artist_only:\n sys.exit()\n else:\n return\n elif response == \"\":\n continue\n\n try:\n response = int(response)\n except ValueError:\n self.status_message = f\"Invalid selection: {response}\"\n else:\n response = response - 1\n if response in range(len(results)):\n self.artist = results[response]['name']\n if artist_only:\n self.clear()\n return results[response]\n self.album_menu(results[response])\n else:\n self.status_message = f\"Invalid selection: {response}\"\n continue\n\n def get_filtered_year(self):\n if self.gte_year and self.lte_year:\n return f\"{self.gte_year} - {self.lte_year}\"\n elif self.gte_year:\n return f\">={self.gte_year}\"\n elif self.lte_year:\n return f\"<={self.lte_year}\"\n elif self.eq_year:\n return f\"{self.eq_year}\"\n else:\n return \"All\"\n\n def album_menu_header(self, artist: str):\n filter_text = \"All\" if not self.filter else self.filter.title()\n filter_year = self.get_filtered_year()\n if self.explicit_only:\n filter_text = filter_text + \" (Explicit Only)\"\n desc_text = \"desc\" if self.desc else \"asc\"\n sort_text = self.sort.replace(\"_\", \" \").title() + \" (\" + desc_text + \")\"\n print(\"Discography for artist: \" + artist)\n print(\"Filter: \" + filter_text + \" | Sort: \" + sort_text + \" | Year: \" + filter_year + \"\\n\")\n\n def album_menu_options(self, monitored):\n print(\"\")\n if not monitored:\n monitor_opt = \"(m) Monitor\"\n else:\n monitor_opt = \"(m) Stop Monitoring\"\n ui_filter = \"Filters: (*) All (a) Albums (e) EP (s) Singles - (E) Explicit (r) Reset\"\n ui_sort = \" Sort: (y) Release Date (desc) (Y) Release Date (asc) (t) Title (desc) (T) Title (asc)\"\n ui_mode = \" Mode: (S) Toggle Select\"\n ui_options = (\"(b) Back (d) Download Queue (Q) Show Queue (f) Queue Filtered \"\n f\"{monitor_opt}\")\n self.display_options(ui_filter, ui_sort, ui_mode, ui_options)\n\n @staticmethod\n def explicit_lyrics(is_explicit):\n if is_explicit > 0:\n return f\"[E]\"\n else:\n return f\" \"\n\n def item_selected(self, id):\n if self.select_mode:\n if [x for x in self.queue_list if x.album_id == id or x.track_id == id]:\n return \"[*] \"\n else:\n return \"[ ] \"\n else:\n return \" \"\n\n def show_mode(self):\n if self.select_mode:\n return \"[SELECT] \"\n return \"\"\n\n def album_menu(self, artist: dict):\n exit_album_menu: bool = False\n # Rewrite DICT to follow old format used by get_artist_albums\n artist_tmp = {'artist_id': artist['id'], 'artist_name': artist['name']}\n\n artist_albums = self.api.get_artist_albums(artist_tmp)['releases']\n while exit_album_menu is False:\n self.clear()\n self.album_menu_header(artist['name'])\n filtered_choices = self.filter_choices(artist_albums)\n for idx, album in enumerate(filtered_choices, start=1):\n print(f\"{self.explicit_lyrics(album['explicit_lyrics'])} {self.item_selected(album['id'])}{idx}. ({dates.get_year(album['release_date'])}) \"\n f\"{album['title']}\")\n monitored = self.db.get_monitored_artist_by_id(artist['id'])\n self.album_menu_options(monitored)\n\n prompt = input(f\":: {self.show_mode()}Please choose an option or type 'exit'{self.show_mini_queue()}: \")\n if prompt == \"a\":\n self.filter = \"album\"\n elif prompt == \"e\":\n self.filter = \"ep\"\n elif prompt == \"s\":\n self.filter = \"single\"\n elif prompt == \"*\":\n self.filter = None\n elif prompt == \"E\":\n self.explicit_only ^= True\n elif prompt == \"r\":\n self.filter = None\n self.explicit_only = False\n self.sort = \"release_date\"\n self.desc = True\n self.gte_year = None\n self.lte_year = None\n self.eq_year = None\n elif prompt.startswith(\">=\"):\n self.eq_year = None\n self.gte_year = int(prompt[2:])\n elif prompt.startswith(\"<=\"):\n self.eq_year = None\n self.lte_year = int(prompt[2:])\n elif prompt.startswith(\"=\"):\n self.lte_year = None\n self.gte_year = None\n self.eq_year = int(prompt[1:])\n elif prompt == \"y\":\n self.sort = \"release_date\"\n self.desc = True\n elif prompt == \"Y\":\n self.sort = \"release_date\"\n self.desc = False\n elif prompt == \"t\":\n self.sort = \"title\"\n self.desc = True\n elif prompt == \"T\":\n self.sort = \"title\"\n self.desc = False\n elif prompt == \"S\":\n self.select_mode ^= True\n elif prompt == \"m\":\n if monitored:\n stop = True\n else:\n stop = False\n record_type = self.filter or config.record_type()\n self.clear()\n monitor = mon.Monitor()\n monitor.set_config(None, None, record_type, None)\n monitor.set_options(stop, False, False)\n monitor.artist_ids([artist['id']])\n elif prompt == \"f\":\n if len(filtered_choices) > 0:\n for item in filtered_choices:\n self.send_to_queue(item)\n else:\n self.status_message = \"No items to add\"\n elif prompt == \"d\":\n if len(self.queue_list) > 0:\n self.start_queue()\n elif prompt == \"Q\":\n if len(self.queue_list) > 0:\n self.queue_menu()\n else:\n self.status_message = \"Queue is empty\"\n elif prompt == \"b\":\n break\n elif prompt == \"\":\n self.status_message = \"Hint: to exit, type 'exit'!\"\n continue\n elif prompt == \"exit\":\n if self.exit_search():\n sys.exit()\n else:\n try:\n selected_index = (int(prompt) - 1)\n except ValueError:\n self.status_message = \"Invalid filter, sort or option provided\"\n continue\n except IndexError:\n self.status_message = \"Invalid selection, please choose from above\"\n continue\n\n if selected_index in range(len(filtered_choices)):\n if self.select_mode:\n selected_item = filtered_choices[selected_index]\n self.send_to_queue(selected_item)\n continue\n else:\n self.track_menu(filtered_choices[selected_index])\n else:\n self.status_message = \"Invalid selection, please choose from above\"\n continue\n\n def track_menu_options(self):\n ui_options = (\"(b) Back (d) Download Queue (Q) Show Queue\")\n self.display_options(options=ui_options)\n\n def track_menu_header(self, album):\n print(\"deemon Interactive Search Client\")\n print(f\"Artist: {self.artist} | Album: {album['title']}\\n\")\n\n def track_menu(self, album):\n exit_track_menu: bool = False\n track_list = self.dz.api.get_album_tracks(album['id'])['data']\n self.select_mode = True\n while not exit_track_menu:\n self.clear()\n self.track_menu_header(album)\n\n for idx, track in enumerate(track_list, start=1):\n print(f\"{self.item_selected(track['id'])}{idx}. {track['title']}\")\n self.track_menu_options()\n\n prompt = input(f\":: {self.show_mode()}Please choose an option or type 'exit'{self.show_mini_queue()}: \")\n if prompt == \"d\":\n if len(self.queue_list) > 0:\n self.start_queue()\n else:\n self.status_message = \"Queue is empty\"\n elif prompt == \"Q\":\n if len(self.queue_list) > 0:\n self.queue_menu()\n else:\n self.status_message = \"Queue is empty\"\n elif prompt == \"b\":\n self.select_mode = False\n break\n elif prompt == \"\":\n self.status_message = \"Hint: to exit, type 'exit'!\"\n continue\n elif prompt == \"exit\":\n if self.exit_search():\n sys.exit()\n else:\n try:\n selected_index = (int(prompt) - 1)\n except ValueError:\n self.status_message = \"Invalid filter, sort or option provided\"\n continue\n except IndexError:\n self.status_message = \"Invalid selection, please choose from above\"\n continue\n\n if selected_index in range(len(track_list)):\n selected_item = track_list[selected_index]\n selected_item['record_type'] = 'track'\n self.send_to_queue(selected_item)\n continue\n else:\n self.status_message = \"Invalid selection, please choose from above\"\n continue\n\n def search_header(self):\n pass\n\n def queue_menu(self):\n exit_queue_list = False\n while exit_queue_list is False:\n self.clear()\n for idx, q in enumerate(self.queue_list, start=1):\n if q.album_title:\n print(f\"{idx}. {q.artist_name} - {q.album_title}\")\n else:\n print(f\"{idx}. {q.artist_name} - {q.track_title}\")\n print(\"\")\n self.queue_menu_options()\n response = input(f\":: Please choose an option or type exit {self.show_mini_queue()}: \")\n if response == \"d\":\n if len(self.queue_list) > 0:\n self.start_queue()\n else:\n self.status_message = \"Queue is empty\"\n if response == \"c\":\n self.queue_list = []\n break\n if response == \"b\":\n break\n if response == \"exit\":\n if self.exit_search():\n sys.exit()\n try:\n response = int(response) - 1\n except ValueError:\n continue\n if response in range(len(self.queue_list)):\n self.queue_list.pop(response)\n if len(self.queue_list) == 0:\n break\n\n def exit_search(self):\n if len(self.queue_list) > 0:\n exit_all = input(\":: Quit before downloading queue? [y|N] \")\n if exit_all.lower() != 'y':\n return False\n return True\n\n def display_options(self, filter=None, sort=None, mode=None, options=None):\n if filter:\n print(filter)\n if sort:\n print(sort)\n if mode:\n print(mode)\n if options:\n print(\"\")\n print(options)\n if self.status_message:\n print(\"** \" + self.status_message + \" **\")\n self.status_message = None\n\n @staticmethod\n def clear():\n from os import system, name\n if name == 'nt':\n _ = system('cls')\n else:\n _ = system('clear')\n\n def filter_choices(self, choices):\n apply_filter = [x for x in choices if x['record_type'] == self.filter or self.filter is None]\n if self.explicit_only:\n apply_filter = [x for x in apply_filter if x['explicit_lyrics'] > 0]\n\n if any([self.gte_year, self.lte_year, self.eq_year]):\n if self.eq_year:\n apply_filter = [x for x in apply_filter if dates.get_year(x['release_date']) == self.eq_year]\n elif self.gte_year and self.lte_year:\n apply_filter = [x for x in apply_filter if dates.get_year(x['release_date']) >= self.gte_year and dates.get_year(x['release_date']) <= self.lte_year]\n elif self.gte_year:\n apply_filter = [x for x in apply_filter if dates.get_year(x['release_date']) >= self.gte_year]\n elif self.lte_year:\n apply_filter = [x for x in apply_filter if dates.get_year(x['release_date']) <= self.lte_year]\n\n return sorted(apply_filter, key=lambda x: x[self.sort], reverse=self.desc)\n\n def start_queue(self):\n self.clear()\n dl = download.Download(active_api=self.api)\n dl.queue_list = self.queue_list\n download_result = dl.download_queue()\n self.queue_list.clear()\n if download_result:\n self.status_message = \"Downloads complete\"\n else:\n self.status_message = \"Downloads failed, please check logs\"\n\n def send_to_queue(self, item):\n if item['record_type'] in ['album', 'ep', 'single']:\n album = {\n 'id': item['id'],\n 'title': item['title'],\n 'link': item['link'],\n 'artist': {\n 'name': self.artist\n }\n }\n for i, q in enumerate(self.queue_list):\n if q.album_id == album['id']:\n del self.queue_list[i]\n return\n self.queue_list.append(download.QueueItem(album=album))\n elif item['record_type'] == 'track':\n track = {\n 'id': item['id'],\n 'title': item['title'],\n 'link': item['link'],\n 'artist': {\n 'name': self.artist\n }\n }\n for i, q in enumerate(self.queue_list):\n if q.track_id == track['id']:\n del self.queue_list[i]\n return\n\n self.queue_list.append(download.QueueItem(track=track))\n\n else:\n logger.error(\"Unknown record type. Please report this to add support:\")\n logger.error(item)\n"
},
{
"path": "deemon/cmd/show.py",
"content": "import csv\nimport logging\nimport os\nimport time\nfrom pathlib import Path\nfrom typing import Union\n\nfrom deemon.core.db import Database\nfrom deemon.utils.dates import generate_date_filename\n\nlogger = logging.getLogger(__name__)\n\n\nclass Show:\n\n def __init__(self):\n self.db = Database()\n\n def monitoring(self, artist: bool = True, query: str = None, export_csv: bool = False,\n save_path: Union[str, Path] = None, filter: str = None, hide_header: bool = False,\n is_id: bool = False, backup: Union[str, Path] = None):\n\n def csv_output(line: str):\n if save_path:\n output_to_file.append(line)\n else:\n print(line)\n\n output_to_file = []\n\n if backup:\n export_csv = True\n filter = \"id\"\n hide_header = True\n save_path = backup\n\n if artist:\n if query:\n db_result = self.db.get_monitored_artist_by_name(query)\n else:\n db_result = self.db.get_all_monitored_artists()\n\n if not db_result:\n if query:\n return logger.error(\"Artist not found: \" + str(query))\n else:\n return logger.error(\"No artists are being monitored\")\n else:\n if query:\n if is_id:\n try:\n query = int(query)\n except ValueError:\n return logger.error(f\"Invalid Playlist ID - {query}\")\n db_result = self.db.get_monitored_playlist_by_id(query)\n else:\n db_result = self.db.get_monitored_playlist_by_name(query)\n else:\n db_result = self.db.get_all_monitored_playlists()\n\n if not db_result:\n if query:\n return logger.error(\"Playlist/ID not found: \" + str(query))\n else:\n return logger.error(\"No playlists are being monitored\")\n\n if artist and query:\n for key, val in db_result.items():\n if val == None:\n db_result[key] = \"-\"\n\n print(\"{:<10} {:<35} {:<10} {:<10} {:<10} {:<25}\".format('ID', 'Artist', 'Alerts',\n 'Bitrate', 'Type', 'Download Path'))\n\n print(\"{!s:<10} {!s:<35} {!s:<10} {!s:<10} {!s:<10} {!s:<25}\".format(db_result['artist_id'],\n db_result['artist_name'],\n db_result['alerts'],\n db_result['bitrate'],\n db_result['record_type'],\n db_result['download_path']))\n print(\"\")\n elif not artist and query:\n for key, val in db_result.items():\n if val == None:\n db_result[key] = \"-\"\n\n print(\"{:<15} {:<30} {:<50} {:<10} {:<10} {:<25}\".format('ID', 'Title', 'URL', 'Alerts',\n 'Bitrate', 'Download Path'))\n\n print(\"{!s:<15} {!s:<30} {!s:<50} {!s:<10} {!s:<10} {!s:<25}\".format(db_result['id'], db_result['title'],\n db_result['url'], db_result['alerts'],\n db_result['bitrate'],\n db_result['download_path']))\n print(\"\")\n else:\n if export_csv or save_path:\n if artist:\n if not filter:\n filter = \"name,id,bitrate,alerts,type,path\"\n filter = filter.split(',')\n logger.debug(f\"Generating CSV data using filters: {', '.join(filter)}\")\n column_names = ['artist_id' if x == 'id' else x for x in filter]\n column_names = ['artist_name' if x == 'name' else x for x in column_names]\n column_names = ['record_type' if x == 'type' else x for x in column_names]\n column_names = ['download_path' if x == 'path' else x for x in column_names]\n else:\n if not filter:\n filter = \"id,title,url,bitrate,alerts,path\"\n filter = filter.split(',')\n logger.debug(f\"Generating CSV data using filters: {', '.join(filter)}\")\n column_names = ['download_path' if x == 'path' else x for x in filter]\n\n for column in column_names:\n if not len([x for x in db_result if column in x.keys()]):\n logger.warning(f\"Unknown filter specified: {column}\")\n column_names.remove(column)\n\n if not hide_header:\n csv_output(','.join(filter))\n for artist in db_result:\n filtered_artists = []\n for key, value in artist.items():\n if value is None:\n artist[key] = \"\"\n for column in column_names:\n filtered_artists.append(str(artist[column]))\n if len(filtered_artists):\n for i, a in enumerate(filtered_artists):\n if '\"' in a:\n a = a.replace('\"', \"'\")\n if ',' in a:\n filtered_artists[i] = f'\"{a}\"'\n csv_output(\",\".join(filtered_artists))\n\n if output_to_file:\n if Path(save_path).is_dir():\n output_filename = Path(save_path / f\"{generate_date_filename('deemon_')}.csv\")\n else:\n output_filename = Path(save_path)\n\n with open(output_filename, 'w', encoding=\"utf-8\") as f:\n for line in output_to_file:\n if line == output_to_file[-1]:\n f.writelines(line)\n break\n f.writelines(line + \"\\n\")\n\n return logger.info(\"CSV data has been saved to: \" + str(output_filename))\n\n return\n\n if artist:\n db_result = [x['artist_name'] for x in db_result]\n else:\n db_result = [x['title'] for x in db_result]\n if len(db_result) < 10:\n for artist in db_result:\n print(artist)\n else:\n db_result = self.truncate_long_artists(db_result)\n\n try:\n size = os.get_terminal_size()\n max_cols = (int(size.columns / 30))\n except:\n max_cols = 5\n \n if max_cols > 5:\n max_cols = 5\n\n while len(db_result) % max_cols != 0:\n db_result.append(\" \")\n\n if max_cols >= 5:\n for a, b, c, d, e in zip(db_result[0::5], db_result[1::5], db_result[2::5], db_result[3::5],\n db_result[4::5]):\n print('{:<30}{:<30}{:<30}{:<30}{:<30}'.format(a, b, c, d, e))\n elif max_cols >= 4:\n for a, b, c, d in zip(db_result[0::4], db_result[1::4], db_result[2::4], db_result[3::4]):\n print('{:<30}{:<30}{:<30}{:<30}'.format(a, b, c, d))\n elif max_cols >= 3:\n for a, b, c in zip(db_result[0::3], db_result[1::3], db_result[2::3]):\n print('{:<30}{:<30}{:<30}'.format(a, b, c))\n elif max_cols >= 2:\n for a, b in zip(db_result[0::2], db_result[1::2]):\n print('{:<30}{:<30}'.format(a, b))\n else:\n for a in db_result:\n print(a)\n\n def playlists(self, csv=False):\n monitored_playlists = self.db.get_all_monitored_playlists()\n for p in monitored_playlists:\n print(f\"{p[1]} ({p[2]})\")\n\n @staticmethod\n def truncate_long_artists(all_artists):\n for idx, artist in enumerate(all_artists):\n if len(artist) > 25:\n all_artists[idx] = artist[:22] + \"...\"\n all_artists[idx] = artist\n return all_artists\n\n def releases(self, days, future):\n if future:\n future_releases = self.db.get_future_releases()\n future_release_list = [x for x in future_releases]\n if len(future_release_list) > 0:\n logger.info(f\"Future releases:\")\n print(\"\")\n future_release_list.sort(key=lambda x: x['album_release'], reverse=True)\n for release in future_release_list:\n print('+ [%-10s] %s - %s' % (release['album_release'], release['artist_name'], release['album_name']))\n else:\n logger.info(\"No future releases have been detected\")\n else:\n seconds_per_day = 86400\n days_in_seconds = (days * seconds_per_day)\n now = int(time.time())\n back_date = (now - days_in_seconds)\n releases = self.db.show_new_releases(back_date, now)\n release_list = [x for x in releases]\n if len(release_list) > 0:\n logger.info(f\"New releases found within last {days} day(s):\")\n print(\"\")\n release_list.sort(key=lambda x: x['album_release'], reverse=True)\n for release in release_list:\n print('+ [%-10s] %s - %s' % (release['album_release'], release['artist_name'], release['album_name']))\n else:\n logger.info(f\"No releases found in the last {days} day(s)\")\n"
},
{
"path": "deemon/cmd/upgradelib.py",
"content": "import sys\nimport time\nimport logging\nfrom datetime import timedelta\nfrom pathlib import Path\nfrom mutagen.easyid3 import EasyID3\nfrom itertools import groupby\nfrom operator import itemgetter\nfrom deezer import Deezer\nfrom concurrent.futures import ThreadPoolExecutor\nfrom unidecode import unidecode\nfrom tqdm import tqdm\nfrom deemon.core.common import exclude_filtered_versions\nfrom deemon.core.config import Config as config\n\nlogger = logging.getLogger(__name__)\n\ndz = Deezer()\n\nLIBRARY_ROOT = None\nALBUM_ONLY = None\nALLOW_EXCLUSIONS = None\n\n# TODO - Add an 'exclusions' key to albums/tracks for count\n# TODO - to improve album title matching, extract all a-zA-Z0-9 and compare (remove special chars)\n\nlibrary_metadata = []\nperformance = {\n 'startID3': 0,\n 'endID3': 0,\n 'completeID3': 0,\n 'startAPI': 0,\n 'endAPI': 0,\n 'completeAPI': 0\n}\n\n\nclass Performance:\n def __init__(self):\n self.startID3 = 0\n self.endID3 = 0\n self.completeID3 = 0\n self.startAPI = 0\n self.endAPI = 0\n self.completeAPI = 0\n\n def start(self, module: str):\n if module == 'ID3':\n self.startID3 = time.time()\n elif module == 'API':\n self.startAPI = time.time()\n\n def end(self, module: str):\n if module == 'ID3':\n self.endID3 = time.time()\n self.completeID3 = self.endID3 - self.startID3\n elif module == 'API':\n self.endAPI = time.time()\n self.completeAPI = self.endAPI - self.startAPI\n\n\ndef read_metadata(file):\n metadata = {\n 'abs_path': file,\n 'rel_path': str(file).replace(LIBRARY_ROOT, \"..\"),\n 'error': None\n }\n\n try:\n _audio = EasyID3(file)\n\n # Remove featured artists from artist tag\n metadata['artist'] = _audio['artist'][0].split(\"/\")[0].strip()\n\n # Remove special character replacement for search query\n metadata['album'] = _audio['album'][0].replace(\"_\", \" \").strip()\n metadata['title'] = _audio['title'][0].strip()\n except Exception as e:\n metadata['error'] = e\n\n return metadata\n\n\ndef get_time_from_secs(secs):\n td_str = str(timedelta(seconds=secs))\n x = td_str.split(\":\")\n x[2] = x[2].split(\".\")[0]\n\n if x[0] != \"0\":\n friendly_time = f\"{x[0]} Hours {x[1]} Minutes {x[2]} Seconds\"\n elif x[1] != \"00\":\n friendly_time = f\"{x[1]} Minutes {x[2]} Seconds\"\n elif x[2] == \"00\":\n friendly_time = f\"Less than 1 second\"\n else:\n friendly_time = f\"{x[2]} Seconds\"\n\n return friendly_time\n\n\ndef invalid_metadata(track: dict) -> bool:\n if not all([track['artist'], track['album'], track['title']]):\n return True\n else:\n return False\n\n\ndef get_artist_api(name: str) -> list:\n \"\"\" Get list of artists with exact name matches from API \"\"\"\n artist_api = dz.gw.search(name)['ARTIST']['data']\n artist_matches = []\n\n for artist in artist_api:\n if artist['ART_NAME'].lower() == name.lower():\n artist_matches.append(artist)\n\n return artist_matches\n\n\ndef get_artist_discography_api(artist_name, artist_id) -> list:\n \"\"\" Get list of albums with exact name matches from API \"\"\"\n album_search = dz.gw.search(artist_name)['ALBUM']['data']\n album_gw = dz.gw.get_artist_discography(artist_id)['data']\n album_api = dz.api.get_artist_albums(artist_id)['data']\n\n albums = []\n\n for album in album_api:\n if album['record_type'] == 'single':\n album['record_type'] = '0'\n elif album['record_type'] == 'album':\n album['record_type'] = '1'\n elif album['record_type'] == 'compilation':\n album['record_type'] = '2'\n elif album['record_type'] == 'ep':\n album['record_type'] = '3'\n\n if album['explicit_lyrics']:\n album['explicit_lyrics'] = '1'\n else:\n album['explicit_lyrics'] = '0'\n\n alb = {\n 'ALB_ID': str(album['id']),\n 'ALB_TITLE': album['title'],\n 'EXPLICIT_LYRICS': album['explicit_lyrics'],\n 'TYPE': album['record_type']\n }\n albums.append(alb)\n\n for album in album_gw:\n if album['ALB_ID'] not in [x['ALB_ID'] for x in albums]:\n albums.append(album)\n\n for album in album_search:\n if album['ART_ID'] == artist_id:\n if album['ALB_ID'] not in [x['ALB_ID'] for x in albums]:\n # Album returned via Search is missing EXPLICIT_LYRICS key\n if not album.get('EXPLICIT_LYRICS'):\n album['EXPLICIT_LYRICS'] = '0'\n albums.append(album)\n\n return albums\n\n\ndef get_album_tracklist_api(album_id: str) -> list:\n \"\"\" Get tracklist for album based on album_id \"\"\"\n tracklist_api = dz.gw.get_album_tracks(album_id)\n return tracklist_api\n\n\ndef retrieve_track_ids_per_artist(discography: tuple):\n artist = discography[0]\n albums = discography[1]\n\n # TODO - Need to implement this\n duplicate_artists = False\n found_artist = False\n\n track_ids = []\n album_ids = []\n\n api_artists = get_artist_api(artist)\n\n tqdm.write(f\"Getting track IDs for tracks by \\\"{artist}\\\"\")\n\n if len(api_artists):\n if len(api_artists) > 1:\n tqdm.write(f\"Duplicate artists detected for \\\"{artist}\\\"\")\n duplicate_artists = True\n\n for api_artist in api_artists:\n\n if found_artist:\n tqdm.write(\"Found correct artist, skipping duplicates\")\n break\n\n if duplicate_artists:\n tqdm.write(f\"Searching with: {api_artist['ART_NAME']} ({api_artist['ART_ID']})\")\n\n discog = get_artist_discography_api(api_artist['ART_NAME'], api_artist['ART_ID'])\n\n for album, tracks in groupby(albums, key=itemgetter('album')):\n\n # Convert itertools.groupby to list so we can use it more than once\n tracks = [track for track in tracks]\n\n api_album_matches = [alb for alb in discog if alb['ALB_TITLE'].lower() == album.lower()]\n\n if ALLOW_EXCLUSIONS:\n filtered_album_matches = exclude_filtered_versions(api_album_matches)\n api_album = get_preferred_album(filtered_album_matches, len(tracks))\n else:\n api_album = get_preferred_album(api_album_matches, len(tracks))\n\n if ALBUM_ONLY:\n if api_album:\n album_ids.append(api_album)\n continue\n else:\n album_ids.append(\n {\n 'artist': artist,\n 'title': album,\n 'info': \"Album not found\"\n }\n )\n continue\n\n if api_album:\n tracklist = get_album_tracklist_api(api_album['ALB_ID'])\n for track in tracks:\n track_variations = [track['title'].lower(), unidecode(track['title']).lower()]\n\n for i, track_api in enumerate(tracklist, start=1):\n if track_api['SNG_TITLE'].lower() in track_variations:\n found_artist = True\n track['id'] = track_api['SNG_ID']\n track_ids.append(track)\n break\n\n elif f\"{track_api['SNG_TITLE']} {track_api.get('VERSION', '')}\".lower() in track_variations:\n found_artist = True\n track['id'] = track_api['SNG_ID']\n track_ids.append(track)\n break\n\n if i == len(tracklist):\n track['info'] = \"Track not found\"\n tqdm.write(f\"{track['info']}: {track['title']}\")\n track_ids.append(track)\n break\n else:\n if duplicate_artists:\n info = f\"Album not found under artist ID {api_artist['ART_ID']}\"\n else:\n info = f\"Album not found\"\n tqdm.write(f\"{info}: {album}\")\n for track in tracks:\n track['info'] = info\n track_ids.append(track)\n else:\n tqdm.write(f\"Artist not found: {artist}\")\n for album, tracks in groupby(albums, key=itemgetter('album')):\n for track in tracks:\n track['info'] = \"Artist not found\"\n track_ids.append(track)\n\n if ALBUM_ONLY:\n return album_ids\n else:\n return track_ids\n\n\ndef get_preferred_album(api_albums: list, num_tracks: int):\n \"\"\" Return preferred album order based on config.prefer_explicit() \"\"\"\n preferred_album = None\n\n if num_tracks < 4:\n preferred_album = [album for album in api_albums if album['EXPLICIT_LYRICS'] == '1' and album['TYPE'] == '0']\n if not preferred_album:\n preferred_album = [album for album in api_albums if album['TYPE'] == '0']\n\n if num_tracks >= 4 or not preferred_album:\n preferred_album = [album for album in api_albums if album['EXPLICIT_LYRICS'] == '1' and album['TYPE'] in ['1', '2', '3']]\n if not preferred_album:\n preferred_album = [album for album in api_albums if album['TYPE'] in ['1', '2', '3']]\n\n if preferred_album:\n return preferred_album[0]\n\n\ndef get_preferred_track_id(title: str, tracklist: list):\n \"\"\" Return preferred track ID by comparing against title of local track \"\"\"\n track_id = None\n for track in tracklist:\n if track.get('VERSION'):\n api_title = f\"{track['SNG_TITLE']} {track['VERSION']}\".lower()\n if title.lower() == api_title:\n return track['SNG_ID']\n else:\n if track['SNG_TITLE'].lower() == title.lower():\n track_id = track['SNG_ID']\n return track_id\n\n\ndef upgrade(library, output, albums=False, exclusions=False):\n\n global ALBUM_ONLY\n global ALLOW_EXCLUSIONS\n global LIBRARY_ROOT\n\n ALBUM_ONLY = albums\n ALLOW_EXCLUSIONS = exclusions\n LIBRARY_ROOT = library\n\n output_ids = Path(output) / \"library_upgrade_ids.txt\"\n output_log = Path(output) / \"library_upgrade.log\"\n\n perf = Performance()\n logger.info(\"Scanning library, standby...\")\n logger.debug(f\"Library path: {LIBRARY_ROOT}\")\n library_files = Path(LIBRARY_ROOT).glob(\"**/*.mp3\")\n files = [file for file in library_files if not file.name.startswith(\".\")]\n files.sort()\n\n if files:\n print(f\"Found {len(files)} MP3 files\")\n else:\n print(\"No MP3 files found\")\n sys.exit()\n\n perf.start('ID3')\n with ThreadPoolExecutor(10) as executor:\n library_metadata = list(\n tqdm(\n executor.map(read_metadata, files),\n total=(len(files)),\n desc=\"Reading metadata\",\n )\n )\n perf.end('ID3')\n\n library_metadata_errors = [file for file in library_metadata if file.get('error')]\n library_metadata = [file for file in library_metadata if not file.get('error')]\n\n artists = sorted(library_metadata, key=itemgetter('artist'))\n artist_list = [(artist, list(albums)) for artist, albums in groupby(artists, key=itemgetter('artist'))]\n\n perf.start('API')\n with ThreadPoolExecutor(20) as executor:\n result = list(\n tqdm(\n executor.map(retrieve_track_ids_per_artist, artist_list),\n total=len(artist_list),\n desc=\"Processing tracks by artist\"\n )\n )\n if ALBUM_ONLY:\n album_result = result\n track_result = []\n else:\n track_result = result\n album_result = []\n perf.end('API')\n\n albums = [album for artist in album_result for album in artist]\n album_ids = [album['ALB_ID'] for album in albums if album.get('ALB_ID')]\n album_not_found = [album for album in albums if not album.get('ALB_ID')]\n\n tracks = [track for artist in track_result for track in artist]\n track_ids = [track['id'] for track in tracks if track.get('id') and not track.get('error')]\n track_not_found = [track for track in tracks if not track.get('id') and not track.get('error')]\n\n # TODO move this to function for reuse in f.write() below\n print(f\"Found: {len(track_ids)} | Not Found: {len(track_not_found)} | \"\n f\"Errors: {len(library_metadata_errors)} | Total: {len(files)}\\n\\n\")\n print(f\"Time to read metadata: {get_time_from_secs(perf.completeID3)}\")\n print(f\"Time to retrieve API data: {get_time_from_secs(perf.completeAPI)}\\n\\n\")\n\n if (ALBUM_ONLY and album_ids) or track_ids:\n with open(output_ids, \"w\") as f:\n if ALBUM_ONLY and album_ids:\n f.write(', '.join(album_ids))\n elif track_ids:\n f.write(', '.join(track_ids))\n\n with open(output_log, \"w\") as f:\n if ALBUM_ONLY:\n f.write(f\"Albums Found: {len(album_ids)} | Albums Not Found: {len(album_not_found)} | \"\n f\"ID3 Errors: {len(library_metadata_errors)} | Total Files: {len(files)}\\n\\n\")\n f.write(f\"Time to read metadata: {get_time_from_secs(perf.completeID3)}\\n\")\n f.write(f\"Time to retrieve API data: {get_time_from_secs(perf.completeAPI)}\\n\\n\")\n\n if library_metadata_errors:\n f.write(\"The following files had missing/invalid ID3 tag data:\\n\\n\")\n for track in library_metadata_errors:\n if track.get('error'):\n f.write(f\"\\tFile: {track['rel_path']}\\n\")\n f.write(f\"\\t\\tError: {track['error']}\\n\\n\")\n f.write(\"\\n\")\n\n if album_not_found:\n\n artists_not_found = [track['artist'] for track in tracks if track['info'] == 'Artist not found']\n if artists_not_found:\n f.write(\"The following artists were not found:\\n\")\n for artist in artists_not_found:\n f.write(\"\\n\\t{track['artist']\\n\")\n\n f.write(\"The following albums were not found:\\n\")\n for album in album_not_found:\n f.write(f\"\\n\\tArtist: {album['artist']}\\n\")\n f.write(f\"\\tAlbum: {album['title']}\\n\")\n if album.get('info'):\n f.write(f\"\\tInfo: {album['info']}\\n\")\n else:\n f.write(f\"Tracks Found: {len(track_ids)} | Tracks Not Found: {len(track_not_found)} | \"\n f\"ID3 Errors: {len(library_metadata_errors)} | Total Files: {len(files)}\\n\\n\")\n f.write(f\"Time to read metadata: {get_time_from_secs(perf.completeID3)}\\n\")\n f.write(f\"Time to retrieve API data: {get_time_from_secs(perf.completeAPI)}\\n\\n\")\n\n if library_metadata_errors:\n f.write(\"The following files had missing/invalid ID3 tag data:\\n\\n\")\n for track in library_metadata_errors:\n if track.get('error'):\n f.write(f\"\\tFile: {track['rel_path']}\\n\")\n f.write(f\"\\t\\tError: {track['error']}\\n\\n\")\n f.write(\"\\n\")\n\n if track_not_found:\n\n artists_not_found = [track['artist'] for track in tracks if track.get('info', '') == 'Artist not found']\n if artists_not_found:\n f.write(\"The following artists were not found:\\n\")\n for artist in artists_not_found:\n f.write(\"\\n\\t{track['artist']\\n\")\n\n f.write(\"The following tracks were not found:\\n\")\n for track in track_not_found:\n f.write(f\"\\n\\tArtist: {track['artist']}\\n\")\n f.write(f\"\\tAlbum: {track['album']}\\n\")\n f.write(f\"\\tTrack: {track['title']}\\n\")\n f.write(f\"\\tFile: {track['rel_path']}\\n\")\n if track.get('info'):\n f.write(f\"\\tInfo: {track['info']}\\n\")"
},
{
"path": "deemon/core/__init__.py",
"content": ""
},
{
"path": "deemon/core/api.py",
"content": "import json\nimport logging\nfrom datetime import datetime\n\nimport deezer.errors\nfrom deezer import Deezer\n\nfrom deemon.core.config import Config as config\n\nlogger = logging.getLogger(__name__)\n\n\nclass PlatformAPI:\n\n def __init__(self):\n self.max_threads = 2\n self.dz = Deezer()\n self.platform = self.get_platform()\n self.account_type = None\n self.api = self.set_platform()\n \n if config.check_account_status():\n self.account_type = self.get_account_type()\n\n def debugger(self, message: str, payload = None):\n if config.debug_mode():\n if not payload:\n payload = \"\"\n logger.debug(f\"DEBUG_MODE: {message} {str(payload)}\")\n \n def get_platform(self):\n if config.fast_api():\n return \"deezer-gw\"\n return \"deezer-api\"\n\n def set_platform(self):\n if self.platform == \"deezer-gw\":\n self.max_threads = config.fast_api_threads()\n if self.max_threads > 50:\n self.max_threads = 50\n if self.max_threads < 1:\n self.max_threads = 1\n logger.debug(\"Using GW API, max_threads set \"\n f\"to {self.max_threads}\")\n return self.dz.gw\n else:\n return self.dz.api\n \n def get_account_type(self):\n logger.info(\"Verifying ARL, please wait...\")\n temp_dz = Deezer()\n temp_dz.login_via_arl(config.arl())\n if temp_dz.get_session()['current_user'].get('can_stream_lossless'):\n logger.debug(f\"Deezer account type is \\\"Hi-Fi\\\"\")\n return \"hifi\"\n elif temp_dz.get_session()['current_user'].get('can_stream_hq'):\n logger.debug(f\"Deezer account type is \\\"Premium\\\"\")\n return \"premium\"\n else:\n logger.debug(f\"Deezer account type is \\\"Free\\\"\")\n return \"free\"\n\n #TODO GW API appears to ignore limit; must implement afterwards\n def search_artist(self, query: str, limit: int = 5):\n \"\"\"\n Return a list of dictionaries from API containing {'id': int, 'name': str}\n \"\"\"\n if self.platform == \"deezer-gw\":\n api_result = []\n try:\n logger.info(f\"Searching for {query}, please wait...\")\n result = self.api.search(query=query)['ARTIST']['data'][:limit]\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while searching for artist {query}, retrying...\")\n try:\n result = self.api.search(query=query)['ARTIST']['data'][:limit]\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response while searching for artist {query}\")\n return []\n for r in result:\n api_result.append({'id': int(r['ART_ID']), 'name': r['ART_NAME']})\n else:\n api_result = self.api.search_artist(query=query, limit=limit)['data']\n\n return {'query': query, 'results': api_result}\n\n def get_artist_by_id(self, query: int, limit: int = 1):\n \"\"\"\n Return a dictionary from API containing {'id': int, 'name': str}\n \"\"\"\n if self.platform == \"deezer-gw\":\n try:\n result = self.api.get_artist(query)\n except deezer.errors.GWAPIError as e:\n logger.debug(f\"API error on artist ID {query}: {e}\")\n return\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for artist ID {query}, retrying...\")\n try:\n result = self.api.get_artist(query)\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response for artist ID {query}\")\n return\n return {'id': int(result['ART_ID']), 'name': result['ART_NAME']}\n else:\n try:\n result = self.api.get_artist(query)\n except deezer.errors.DataException as e:\n logger.debug(f\"API error: {e}\")\n return\n return {'id': result['id'], 'name': result['name']}\n \n def get_album(self, query: int) -> dict:\n \"\"\"Return a dictionary from API containing album info\"\"\"\n if self.platform == \"deezer-gw\":\n try:\n result = self.api.get_album(query)\n except deezer.errors.GWAPIError as e:\n logger.debug(f\"API error: {e}\")\n return {}\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for album ID {query}, retrying...\")\n try:\n result = self.api.get_album(query)\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response for album ID {query}\")\n return {}\n return {'id': int(result['ALB_ID']), 'title': result['ALB_TITLE'], 'artist': {'name': result['ART_NAME']}}\n else:\n try:\n result = self.api.get_album(query)\n except deezer.errors.DataException as e:\n logger.debug(f\"API error: {e}\")\n return\n else:\n return result\n\n def get_track(self, query: int) -> dict:\n \"\"\"Return a dictionary from API containing album info\"\"\"\n if self.platform == \"deezer-gw\":\n try:\n result = self.api.get_track(query)\n except deezer.errors.GWAPIError as e:\n logger.debug(f\"API error: {e}\")\n return {}\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for album ID {query}, retrying...\")\n try:\n result = self.api.get_album(query)\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response for album ID {query}\")\n return {}\n return {'id': int(result['SNG_ID']), 'title': result['SNG_TITLE'], 'artist': {'name': result['ART_NAME']}}\n else:\n try:\n result = self.api.get_track(query)\n except deezer.errors.DataException as e:\n logger.debug(f\"API error: {e}\")\n else:\n return result\n\n def get_extra_release_info(self, query: dict):\n album = {'id': query['album_id'], 'label': None}\n if self.platform == \"deezer-gw\":\n album_details = self.api.get_album(query['album_id'])\n if album_details.get('LABEL_NAME'):\n album['label'] = album_details['LABEL_NAME']\n else:\n album_details = self.api.get_album(query['album_id'])\n if album_details.get('label'):\n album['label'] = album_details['label']\n \n return album\n\n def get_artist_albums(self, query: dict, limit: int = -1):\n \"\"\"\n Return a list of dictionaries from API containing\n \"\"\"\n self.debugger(f\"Refreshing artist releases for {query['artist_name']} ({query['artist_id']})\")\n if self.platform == \"deezer-gw\":\n try:\n result = self.api.get_artist_discography(art_id=query['artist_id'], limit=limit)['data']\n except deezer.errors.GWAPIError as e:\n if \"UNKNOWN\" in str(e):\n logger.debug(e)\n logger.warning(f\" [!] Artist discography is not available for \"\n f\"{query['artist_name']} ({query['artist_id']})\")\n else:\n logger.debug(e)\n logger.error(f\"An error occured while attempting to get the discography for \"\n f\"{query['artist_name']} ({query['artist_id']})\")\n query['releases'] = []\n return query\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for discography for {query['artist_name']}, retrying...\")\n try:\n result = self.api.get_artist_discography(art_id=query['artist_id'], limit=limit)['data']\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response for discography for {query['artist_name']}\")\n query['releases'] = []\n return query\n api_result = []\n for r in result:\n # Remove ID check to get compilations\n if (r['ART_ID'] == str(query['artist_id']) and r['ARTISTS_ALBUMS_IS_OFFICIAL']) or (r['ART_ID'] == str(query['artist_id']) and config.allow_unofficial()) or config.allow_compilations():\n # TYPE 0 - single, TYPE 1 - album, TYPE 2 - compilation, TYPE 3 - ep\n if r['TYPE'] == '0':\n r['TYPE'] = \"single\"\n elif r['TYPE'] == '1' and r['ART_ID'] != str(query['artist_id']):\n if not config.allow_featured_in():\n logger.debug(f\"Featured In for {query['artist_name']} detected but are disabled in config\")\n continue\n else:\n logger.debug(f\"Featured In detected for artist {query['artist_name']}: {r['ALB_TITLE']}\")\n r['TYPE'] = \"album\"\n # TODO set unique r['TYPE'] for FEATURED IN\n elif r['TYPE'] == '2':\n if not config.allow_compilations():\n logger.debug(f\"Compilation for {query['artist_name']} detected but are disabled in config\")\n continue\n else:\n logger.debug(f\"Compilation detected for artist {query['artist_name']}: {r['ALB_TITLE']}\")\n r['TYPE'] = \"album\"\n # TODO set unique r['TYPE'] for COMPILATIONS\n elif r['TYPE'] == '3':\n r['TYPE'] = \"ep\"\n else:\n r['TYPE'] = \"album\"\n\n if r['ORIGINAL_RELEASE_DATE'] != \"0000-00-00\":\n release_date = r['ORIGINAL_RELEASE_DATE']\n elif r['PHYSICAL_RELEASE_DATE'] != \"0000-00-00\":\n release_date = r['PHYSICAL_RELEASE_DATE']\n elif r['DIGITAL_RELEASE_DATE'] != \"0000-00-00\":\n release_date = r['DIGITAL_RELEASE_DATE']\n else:\n # In the event of an unknown release date, set it to today's date\n # See album ID: 417403\n logger.warning(f\" [!] Found release without release date, assuming today: \"\n f\"{query['artist_name']} - {r['ALB_TITLE']}\")\n release_date = datetime.strftime(datetime.today(), \"%Y-%m-%d\")\n \n cover_art = f\"https://e-cdns-images.dzcdn.net/images/cover/{r['ALB_PICTURE']}/500x500-00000-80-0-0.jpg\"\n album_url = f\"https://www.deezer.com/album/{r['ALB_ID']}\"\n \n api_result.append(\n {\n 'id': int(r['ALB_ID']),\n 'title': r['ALB_TITLE'],\n 'release_date': release_date,\n 'explicit_lyrics': r['EXPLICIT_ALBUM_CONTENT']['EXPLICIT_LYRICS_STATUS'],\n 'record_type': r['TYPE'],\n 'cover_big': cover_art,\n 'link': album_url,\n 'nb_tracks': r['NUMBER_TRACK'],\n }\n )\n else:\n api_result = self.api.get_artist_albums(artist_id=query['artist_id'], limit=limit)['data']\n\n query['releases'] = api_result\n return query\n\n @staticmethod\n def get_playlist(query: int):\n try:\n api_result = Deezer().api.get_playlist(query)\n except deezer.errors.PermissionException:\n logger.warning(f\" [!] Permission Denied: Playlist {query} is private\")\n return\n except deezer.errors.DataException:\n logger.warning(f\" [!] Playlist ID {query} was not found\")\n return\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for playlist ID {query}, retrying...\")\n try:\n api_result = Deezer().api.get_playlist(query)\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response while getting data for playlist ID {query}\")\n return\n return {'id': query, 'title': api_result['title'],\n 'link': f\"https://deezer.com/playlist/{str(api_result['id'])}\"}\n\n @staticmethod\n def get_playlist_tracks(query: dict):\n track_list = []\n try:\n api_result = Deezer().api.get_playlist_tracks(query['id'])\n except deezer.errors.PermissionException:\n logger.warning(f\" [!] Permission Denied: Playlist {query['title']} ({query['id']}) is private\")\n return\n except deezer.errors.DataException:\n logger.warning(f\" [!] Playlist ID {query} was not found\")\n return\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] Empty response from API while getting data for playlist ID {query['id']}\")\n try:\n api_result = Deezer().api.get_playlist_tracks(query['id'])\n except json.decoder.JSONDecodeError:\n logger.error(f\" [!] API still sending empty response while getting data for playlist ID {query['id']}\")\n return\n for track in api_result['data']:\n track_list.append({'id': track['id'], 'title': track['title'],\n 'artist_id': track['artist']['id'],\n 'artist_name': track['artist']['name']})\n query['tracks'] = track_list\n return query\n"
},
{
"path": "deemon/core/common.py",
"content": "import re\nimport logging\nfrom deemon.core.config import Config as config\n\nlogger = logging.getLogger(__name__)\n\n\ndef exclude_filtered_versions(albums: list) -> list:\n \"\"\" Remove album versions containing specified text \"\"\"\n exclusion_patterns = config.exclusion_patterns()\n exclusion_keywords = config.exclusion_keywords()\n allowed = []\n\n if exclusion_keywords or exclusion_patterns:\n for album in albums:\n album_title = album['title']\n exclusion_pattern_match = [p for p in exclusion_patterns if re.search(p, album_title)]\n keyword_search = re.search(r'\\(([^)]+)\\)|\\[([^)]+)]', album_title.lower())\n exclusion_keyword_match = [e for e in exclusion_keywords if keyword_search and e in keyword_search.group()]\n result = exclusion_pattern_match + exclusion_keyword_match\n result = '\", \"'.join(result)\n\n if exclusion_keyword_match or exclusion_pattern_match:\n logger.info(f\" Album \\\"{album_title}\\\" excluded by filter: \\\"{result}\\\"\")\n continue\n else:\n allowed.append(album)\n return allowed\n else:\n return albums\n"
},
{
"path": "deemon/core/config.py",
"content": "import json\nimport logging\nfrom copy import deepcopy\nfrom pathlib import Path\nfrom typing import Optional\n\nfrom deemon.core.exceptions import ValueNotAllowed, UnknownValue, PropertyTypeMismatch\nfrom deemon.utils import startup\n\nlogger = logging.getLogger(__name__)\n\nALLOWED_VALUES = {\n 'bitrate': {1: \"128\", 3: \"320\", 9: \"FLAC\"},\n 'alerts': [True, False],\n 'record_type': ['all', 'album', 'ep', 'single'],\n 'release_channel': ['stable', 'beta']\n}\n\nDEFAULT_CONFIG = {\n \"check_update\": 1,\n \"debug_mode\": False,\n \"release_channel\": \"stable\",\n \"query_limit\": 5,\n \"smart_search\": True,\n \"rollback_view_limit\": 10,\n \"prompt_duplicates\": False,\n \"prompt_no_matches\": True,\n \"fast_api\": True,\n \"fast_api_threads\": 25,\n \"exclusions\": {\n \"enable_exclusions\": True,\n \"patterns\": [],\n \"keywords\": [],\n },\n \"new_releases\": {\n \"release_max_age\": 90,\n \"include_unofficial\": False,\n \"include_compilations\": False,\n \"include_featured_in\": False,\n },\n \"global\": {\n \"bitrate\": \"320\",\n \"alerts\": False,\n \"record_type\": \"all\",\n \"download_path\": \"\",\n \"email\": \"\"\n },\n \"deemix\": {\n \"path\": \"\",\n \"arl\": \"\",\n \"check_account_status\": True,\n \"halt_download_on_error\": False,\n },\n \"smtp_settings\": {\n \"server\": \"\",\n \"port\": 465,\n \"starttls\": False,\n \"username\": \"\",\n \"password\": \"\",\n \"from_addr\": \"\"\n },\n \"plex\": {\n \"base_url\": \"\",\n \"ssl_verify\": True,\n \"token\": \"\",\n \"library\": \"\"\n }\n}\n\n\nclass Config(object):\n _CONFIG_FILE: Optional[Path] = startup.get_config()\n _CONFIG: Optional[dict] = None\n\n def __init__(self):\n if not Config._CONFIG_FILE.exists():\n self.__create_default_config()\n\n if Config._CONFIG is None:\n with open(Config._CONFIG_FILE, 'r', encoding='utf8') as f:\n try:\n Config._CONFIG = json.load(f)\n except json.decoder.JSONDecodeError as e:\n logger.exception(f\"An error occured while reading from config: {e}\")\n raise\n\n if self.validate() > 0:\n self.__write_modified_config()\n\n if not self.arl():\n logger.debug(\"Attempting to locate deemix's .arl file\")\n arl_file = startup.get_appdata_root() / 'deemix' / '.arl'\n if Path(arl_file).is_file():\n with open(arl_file) as f:\n arl_from_file = f.readline().replace(\"\\n\", \"\")\n self.set('arl', arl_from_file)\n logger.debug(\"Successfully loaded ARL\")\n\n if len(self.arl()) > 0 and len(self.arl()) != 192:\n logger.warning(f\" [!] Possible invalid ARL detected (length: {len(self.arl())}). ARL should \"\n \"be 192 characters\")\n\n # Set default for runtime settings\n self.set('profile_id', 1, validate=False)\n\n @staticmethod\n def __create_default_config():\n with open(Config._CONFIG_FILE, 'w', encoding='utf8') as f:\n json.dump(DEFAULT_CONFIG, f, indent=4)\n\n @staticmethod\n def __write_modified_config():\n with open(Config._CONFIG_FILE, 'w', encoding='utf8') as f:\n json.dump(Config._CONFIG, f, indent=4)\n\n @staticmethod\n def validate():\n modified = 0\n\n def process_config(dict1, dict2):\n \"\"\"\n Process user configuration, applying values to a default config\n \"\"\"\n for key, value in dict1.items():\n if key in dict2.keys():\n if isinstance(dict1[key], dict):\n process_config(dict1[key], dict2[key])\n else:\n dict2[key] = dict1[key]\n return dict2\n\n def find_position(d, key):\n for k, v in d.items():\n if isinstance(v, dict):\n next = find_position(v, key)\n if next:\n return [k] + next\n elif k == key:\n return [k]\n\n def update_config_layout(user_config, reference_config):\n \"\"\" Used to move existing values to new property names/locations \"\"\"\n nonlocal modified\n\n if user_config.get('exclude'):\n user_config['exclusions'] = {\n 'enable_exclusions': True,\n 'patterns': user_config['exclude'],\n 'keywords': []\n }\n modified += 1\n\n if user_config.get('experimental'):\n if user_config['experimental'].get('allow_unofficial_releases'):\n user_config['new_releases']['include_unofficial'] = True\n modified += 1\n if user_config['experimental'].get('allow_compilations'):\n user_config['new_releases']['include_compilations'] = True\n modified += 1\n if user_config['experimental'].get('allow_featured_in'):\n user_config['new_releases']['include_featured_in'] = True\n modified += 1\n\n if user_config.get('new_releases'):\n if not user_config['new_releases'].get('by_release_date', True):\n user_config['new_releases']['release_max_age'] = 0\n modified += 1\n\n migration_map = [\n {'check_update': 'check_update'},\n {'plex_baseurl': 'base_url'},\n {'plex_token': 'token'},\n {'plex_library': 'library'},\n {'deemix_path': 'path'},\n {'arl': 'arl'},\n {'smtp_recipient': 'email'},\n {'smtp_server': 'server'},\n {'smtp_user': 'username'},\n {'smtp_pass': 'password'},\n {'smtp_port': 'port'},\n {'smtp_sender': 'from_addr'},\n {'bitrate': 'bitrate'},\n {'alerts': 'alerts'},\n {'record_type': 'record_type'},\n {'download_path': 'download_path'},\n {'release_max_days': 'release_max_age'},\n {'ranked_duplicates': 'prompt_duplicates'}\n ]\n for mlist in migration_map:\n\n for old, new in mlist.items():\n user_config_tmp = deepcopy(user_config)\n user_config_copy = user_config\n if not user_config.get(old):\n continue\n\n old_position = find_position(user_config, old) or [old]\n new_position = find_position(reference_config, new) or [new]\n\n for i in old_position[:-1]:\n user_config_tmp = user_config_tmp.setdefault(i, {})\n\n for i in new_position[:-1]:\n user_config_copy = user_config_copy.setdefault(i, {})\n\n if user_config_tmp != user_config_copy:\n logger.debug(\"Migrating \" + ':'.join([str(x) for x in old_position]) + \" -> \" + ':'.join(\n [str(x) for x in new_position]))\n user_config_copy[new_position[-1]] = user_config_tmp[old_position[-1]]\n modified += 1\n\n return user_config\n\n def test_values(dict1, dict2):\n nonlocal modified\n for key, value in dict1.items():\n if key in dict2.keys():\n if isinstance(dict1[key], dict):\n test_values(dict1[key], dict2[key])\n else:\n if key == \"starttls\" and value is True:\n if dict1['port'] == 465:\n logger.debug(\"STARTTLS enabled in config but port set to 465 (SSL); disabling STARTTLS\")\n dict1['starttls'] = False\n modified += 1\n if key in ALLOWED_VALUES:\n if isinstance(ALLOWED_VALUES[key], dict):\n if key == \"bitrate\" and value in [\"1\", \"3\", \"9\"]:\n if value == \"1\":\n dict1['bitrate'] = \"128\"\n if value == \"3\":\n dict1['bitrate'] = \"320\"\n if value == \"9\":\n dict1['bitrate'] = \"FLAC\"\n modified += 1\n elif value in ALLOWED_VALUES[key].keys():\n dict1_tmp = dict1\n pos = find_position(dict1_tmp, key)\n for i in pos[:-1]:\n dict1_tmp = dict1.setdefault(i, {})\n dict1_tmp[key] = ALLOWED_VALUES[key][value]\n modified += 1\n elif (\n value in ALLOWED_VALUES[key].values() or\n value.upper() in ALLOWED_VALUES[key].values()\n ):\n continue\n else:\n raise UnknownValue(\n f\"Unknown value in config - '{key}': {value} (type: {type(value).__name__})\")\n elif not isinstance(dict1[key], type(dict2[key])):\n if isinstance(dict2[key], bool):\n if dict1[key] == 1:\n dict1[key] = True\n modified += 1\n if dict1[key] == 0:\n dict1[key] = False\n modified += 1\n else:\n raise UnknownValue(\n f\"Unknown value in config - '{key}': {value} (type: {type(value).__name__})\")\n else:\n if value in ALLOWED_VALUES[key]:\n continue\n else:\n raise UnknownValue(\n f\"Unknown value in config - '{key}': {value} (type: {type(value).__name__})\")\n elif not isinstance(dict1[key], type(dict2[key])):\n if isinstance(dict2[key], bool):\n if dict1[key] == 1:\n dict1[key] = True\n modified += 1\n if dict1[key] == 0:\n dict1[key] = False\n modified += 1\n else:\n raise PropertyTypeMismatch(\n f\"Invalid type in config - '{str(key)}' incorrectly set as {type(value).__name__}\")\n else:\n pass\n\n def add_new_options(dict1, dict2):\n nonlocal modified\n for key, value in dict1.items():\n if key not in dict2.keys():\n logger.debug(f\"New option added to config: {key}\")\n dict2[key] = value\n modified += 1\n elif isinstance(value, dict):\n for k, v in value.items():\n if k not in dict2[key].keys():\n logger.debug(\"New option added to config: \"\n f\"{key}/{k}\")\n dict2[key][k] = v\n modified += 1\n\n logger.debug(\"Loading configuration, please wait...\")\n add_new_options(DEFAULT_CONFIG, Config._CONFIG)\n migrated_config = update_config_layout(Config._CONFIG, DEFAULT_CONFIG)\n Config._CONFIG = process_config(migrated_config, DEFAULT_CONFIG)\n test_values(Config._CONFIG, DEFAULT_CONFIG)\n return modified\n\n @staticmethod\n def get_config_file() -> Path:\n return Config._CONFIG_FILE\n\n @staticmethod\n def get_config() -> dict:\n return Config._CONFIG\n\n @staticmethod\n def plex_baseurl() -> str:\n return Config._CONFIG.get('plex').get('base_url')\n\n @staticmethod\n def plex_token() -> str:\n return Config._CONFIG.get('plex').get('token')\n\n @staticmethod\n def plex_library() -> str:\n return Config._CONFIG.get('plex').get('library')\n\n @staticmethod\n def download_path() -> str:\n return Config._CONFIG.get('global').get('download_path')\n\n @staticmethod\n def deemix_path() -> str:\n return Config._CONFIG.get('deemix').get('path')\n\n @staticmethod\n def arl() -> str:\n return Config._CONFIG.get('deemix').get('arl')\n\n @staticmethod\n def release_max_age() -> int:\n return Config._CONFIG.get('new_releases').get('release_max_age')\n\n @staticmethod\n def bitrate() -> str:\n return Config._CONFIG.get('global').get('bitrate')\n\n @staticmethod\n def alerts() -> bool:\n return Config._CONFIG.get('global').get('alerts')\n\n @staticmethod\n def record_type() -> str:\n return Config._CONFIG.get('global').get('record_type')\n\n @staticmethod\n def smtp_server() -> str:\n return Config._CONFIG.get('smtp_settings').get('server')\n\n @staticmethod\n def smtp_port() -> int:\n return Config._CONFIG.get('smtp_settings').get('port')\n\n @staticmethod\n def smtp_user() -> str:\n return Config._CONFIG.get('smtp_settings').get('username')\n\n @staticmethod\n def smtp_pass() -> str:\n return Config._CONFIG.get('smtp_settings').get('password')\n\n @staticmethod\n def smtp_sender() -> str:\n return Config._CONFIG.get('smtp_settings').get('from_addr')\n\n @staticmethod\n def smtp_recipient() -> list:\n return Config._CONFIG.get('global').get('email')\n\n @staticmethod\n def smtp_starttls() -> bool:\n return Config._CONFIG.get('smtp_settings').get('starttls')\n\n @staticmethod\n def check_update() -> int:\n return Config._CONFIG.get('check_update')\n\n @staticmethod\n def debug_mode() -> bool:\n return Config._CONFIG.get('debug_mode')\n\n @staticmethod\n def profile_id() -> int:\n return Config._CONFIG.get('profile_id')\n\n @staticmethod\n def update_available() -> int:\n return Config._CONFIG.get('update_available')\n\n @staticmethod\n def query_limit() -> int:\n return Config._CONFIG.get('query_limit')\n\n @staticmethod\n def prompt_duplicates() -> int:\n return Config._CONFIG.get('prompt_duplicates')\n\n @staticmethod\n def prompt_no_matches() -> bool:\n return Config._CONFIG.get('prompt_no_matches')\n\n @staticmethod\n def allowed_values(prop):\n return ALLOWED_VALUES.get(prop)\n\n @staticmethod\n def release_channel() -> str:\n return Config._CONFIG.get('release_channel')\n\n @staticmethod\n def rollback_view_limit() -> int:\n return Config._CONFIG.get('rollback_view_limit')\n\n @staticmethod\n def transaction_id() -> int:\n return Config._CONFIG.get('tid')\n \n @staticmethod\n def check_account_status() -> bool:\n return Config._CONFIG.get('deemix').get('check_account_status')\n \n @staticmethod\n def fast_api() -> bool:\n return Config._CONFIG['fast_api']\n\n @staticmethod\n def fast_api_threads() -> int:\n return Config._CONFIG['fast_api_threads']\n\n @staticmethod\n def allow_compilations() -> bool:\n return Config._CONFIG['new_releases']['include_compilations']\n\n @staticmethod\n def allow_featured_in() -> bool:\n return Config._CONFIG['new_releases']['include_featured_in']\n\n @staticmethod\n def allow_unofficial() -> bool:\n return Config._CONFIG['new_releases']['include_unofficial']\n\n @staticmethod\n def enable_exclusions() -> bool:\n return Config._CONFIG['exclusions']['enable_exclusions']\n\n @staticmethod\n def exclusion_keywords() -> list:\n if Config._CONFIG['exclusions']['enable_exclusions']:\n return [x.lower() for x in Config._CONFIG['exclusions']['keywords']]\n else:\n return []\n\n @staticmethod\n def exclusion_patterns() -> list:\n if Config._CONFIG['exclusions']['enable_exclusions']:\n return Config._CONFIG['exclusions']['patterns']\n else:\n return []\n\n @staticmethod\n def plex_ssl_verify() -> bool:\n return Config._CONFIG.get('plex').get('ssl_verify')\n\n @staticmethod\n def halt_download_on_error() -> bool:\n return Config._CONFIG.get('deemix').get('halt_download_on_error')\n\n @staticmethod\n def smart_search() -> bool:\n return Config._CONFIG.get('smart_search')\n\n\n @staticmethod\n def find_position(d, property):\n for k, v in d.items():\n if isinstance(v, dict):\n next = Config.find_position(v, property)\n if next:\n return [k] + next\n elif k == property:\n return [k]\n\n @staticmethod\n def get(property):\n return Config._CONFIG.get(property)\n\n @staticmethod\n def set(property, value, validate=True):\n if not validate:\n Config._CONFIG[property] = value\n if Config._CONFIG.get(property):\n if property in ALLOWED_VALUES:\n if value.lower() == \"true\" or value == \"1\":\n value = True\n elif value.lower() == \"false\" or value == \"0\":\n value = \"\"\n if value in ALLOWED_VALUES[property]:\n Config._CONFIG[property] = value\n return\n raise ValueNotAllowed(f\"Property {property} requires one of \"\n f\"{', '.join(ALLOWED_VALUES[property])}, not {value}.\")\n\n if isinstance(value, type(Config._CONFIG[property])):\n Config._CONFIG[property] = value\n return\n else:\n raise PropertyTypeMismatch(f\"Type mismatch while setting {property} \"\n f\"to {value} (type: {type(value).__name__})\")\n\n else:\n property_path = Config.find_position(Config._CONFIG, property)\n tmpConfig = Config._CONFIG\n for k in property_path[:-1]:\n tmpConfig = tmpConfig.setdefault(k, {})\n if property in ALLOWED_VALUES:\n if isinstance(value, str):\n if value.lower() == \"true\" or value == \"1\":\n value = True\n elif value.lower() == \"false\" or value == \"0\":\n value = False\n if isinstance(ALLOWED_VALUES[property], dict):\n if value in [str(x.lower()) for x in ALLOWED_VALUES[property].values()]:\n tmpConfig[property_path[-1]] = value\n return True\n if value in ALLOWED_VALUES[property]:\n tmpConfig[property_path[-1]] = value\n return True\n raise ValueNotAllowed(f\"Value for {property} is invalid: {value} (type: {type(value).__name__})\")\n\n if isinstance(value, type(tmpConfig[property])):\n tmpConfig[property] = value\n return True\n else:\n raise PropertyTypeMismatch(f\"Type mismatch while setting {property} \"\n f\"to {value} (type: {type(value).__name__})\")\n\n\nclass LoadProfile(object):\n def __init__(self, profile: dict):\n logger.debug(f\"Loaded config for profile {str(profile['id'])} ({str(profile['name'])})\")\n # Rename keys to match config\n profile[\"profile_id\"] = profile.pop(\"id\")\n profile[\"base_url\"] = profile.pop(\"plex_baseurl\")\n profile[\"token\"] = profile.pop(\"plex_token\")\n profile[\"library\"] = profile.pop(\"plex_library\")\n\n # Append to config for debug output; Remove profile name from dict\n Config.set(\"profile_name\", profile.pop(\"name\"), validate=False)\n\n for key, value in profile.items():\n if value is None:\n continue\n Config.set(key, value)\n\n for key, value in Config.get_config().items():\n if isinstance(value, dict):\n for k, v in value.items():\n if key in ['smtp_settings'] or k in ['arl', 'email', 'token']:\n if v:\n v = \"********\"\n logger.debug(f\"> {key}/{k}: {v}\")\n else:\n logger.debug(f\"> {key}: {value}\")\n"
},
{
"path": "deemon/core/db.py",
"content": "import logging\nimport sqlite3\nimport time\nfrom datetime import datetime\nfrom pathlib import Path\n\nfrom packaging.version import parse as parse_version\n\nfrom deemon import __dbversion__, __version__\nfrom deemon.core.config import Config as config\nfrom deemon.utils import startup, performance, dates\n\nlogger = logging.getLogger(__name__)\n\n\nclass Database(object):\n\n def __init__(self):\n self.conn = None\n self.cursor = None\n self.db = startup.get_database()\n\n if not Path(self.db).exists():\n self.connect()\n self.create_new_database()\n else:\n self.connect()\n\n def __enter__(self):\n return self\n\n def __exit__(self):\n self.close()\n\n @staticmethod\n def dict_factory(cursor, row):\n d = {}\n for idx, col in enumerate(cursor.description):\n d[col[0]] = row[idx]\n return d\n\n def connect(self):\n try:\n self.conn = sqlite3.connect(self.db)\n self.conn.row_factory = self.dict_factory\n self.cursor = self.conn.cursor()\n except sqlite3.OperationalError as e:\n logger.error(f\"Error opening database: {e}\")\n\n def close(self):\n if self.conn:\n self.conn.commit()\n self.conn.close()\n\n def commit(self):\n self.conn.commit()\n\n def commit_and_close(self):\n self.commit()\n self.close()\n\n def create_new_database(self):\n logger.debug(\"DATABASE CREATION IN PROGRESS!\")\n\n self.query(\"CREATE TABLE monitor (\"\n \"'artist_id' INTEGER,\"\n \"'artist_name' TEXT,\"\n \"'bitrate' TEXT,\"\n \"'record_type' TEXT,\"\n \"'alerts' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'download_path' TEXT,\"\n \"'refreshed' INTEGER DEFAULT 0,\"\n \"'trans_id' INTEGER)\")\n\n self.query(\"CREATE TABLE playlists (\"\n \"'id' INTEGER UNIQUE,\"\n \"'title' TEXT,\"\n \"'url' TEXT,\"\n \"'bitrate' TEXT,\"\n \"'alerts' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'download_path' TEXT,\"\n \"'refreshed' INTEGER DEFAULT 0,\"\n \"'trans_id' INTEGER,\"\n \"'monitor_artists' INTEGER DEFAULT 0)\")\n\n self.query(\"CREATE TABLE playlist_tracks (\"\n \"'track_id' INTEGER,\"\n \"'playlist_id' INTEGER,\"\n \"'artist_id' INTEGER,\"\n \"'artist_name' TEXT,\"\n \"'track_name' TEXT,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'track_added' TEXT,\"\n \"'trans_id' INTEGER)\")\n\n self.query(\"CREATE TABLE releases (\"\n \"'artist_id' INTEGER,\"\n \"'artist_name' TEXT,\"\n \"'album_id' INTEGER,\"\n \"'album_name' TEXT,\"\n \"'album_release' TEXT,\"\n \"'album_added' INTEGER,\"\n \"'explicit' INTEGER,\"\n \"'label' TEXT,\"\n \"'record_type' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'future_release' INTEGER DEFAULT 0,\"\n \"'trans_id' INTEGER,\"\n \"unique(album_id, profile_id))\")\n\n self.query(\"CREATE TABLE 'deemon' (\"\n \"'property' TEXT,\"\n \"'value' TEXT)\")\n\n self.query(\"CREATE TABLE 'profiles' (\"\n \"'id' INTEGER,\"\n \"'name' TEXT,\"\n \"'email' TEXT,\"\n \"'alerts' INTEGER,\"\n \"'bitrate' TEXT,\"\n \"'record_type' TEXT,\"\n \"'plex_baseurl' TEXT,\"\n \"'plex_token' TEXT,\"\n \"'plex_library' TEXT,\"\n \"'download_path' TEXT,\"\n \"PRIMARY KEY('id' AUTOINCREMENT))\")\n\n self.query(\"CREATE TABLE transactions (\"\n \"'id' INTEGER,\"\n \"'timestamp' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"PRIMARY KEY('id' AUTOINCREMENT))\")\n\n self.query(\"CREATE UNIQUE INDEX 'idx_property' ON 'deemon' ('property')\")\n self.query(\"CREATE INDEX 'artist' ON 'releases' ('artist_id', 'profile_id')\")\n self.query(f\"INSERT INTO 'deemon' ('property', 'value') VALUES ('version', '{__dbversion__}')\")\n self.query(\"INSERT OR REPLACE INTO 'deemon' ('property', 'value') VALUES ('latest_ver', '')\")\n self.query(\"INSERT INTO 'deemon' ('property', 'value') VALUES ('last_update_check', 0)\")\n self.query(\"INSERT INTO 'deemon' ('property', 'value') VALUES ('release_channel', 'stable')\")\n self.query(\"INSERT INTO 'profiles' ('name') VALUES ('default')\")\n self.commit()\n\n def get_latest_ver(self):\n return self.query(\"SELECT value FROM deemon WHERE property = 'latest_ver'\").fetchone()['value']\n\n def get_db_version(self):\n try:\n version = self.query(f\"SELECT value FROM deemon WHERE property = 'version'\").fetchone()['value']\n except sqlite3.OperationalError:\n version = '0.0.0'\n\n logger.debug(f\"Database version {version}\")\n return version\n\n def do_upgrade(self):\n current_ver = parse_version(self.get_db_version())\n app_db_version = parse_version(__dbversion__)\n\n if current_ver == app_db_version:\n return\n\n logger.debug(\"DATABASE UPGRADE IN PROGRESS!\")\n \n if current_ver < parse_version(\"3.5\"):\n logger.error(\"Due to database changes, you must be on at least \"\n f\"v2.5 before upgrading to v{__version__}.\")\n exit()\n \n if current_ver < parse_version(\"3.5.2\"):\n self.query(\"CREATE TABLE releases_tmp (\"\n \"'artist_id' INTEGER,\"\n \"'artist_name' TEXT,\"\n \"'album_id' INTEGER,\"\n \"'album_name' TEXT,\"\n \"'album_release' TEXT,\"\n \"'album_added' INTEGER,\"\n \"'explicit' INTEGER,\"\n \"'label' TEXT,\"\n \"'record_type' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'future_release' INTEGER DEFAULT 0,\"\n \"'trans_id' INTEGER,\"\n \"unique(album_id, profile_id))\")\n self.query(\"INSERT OR REPLACE INTO releases_tmp(artist_id, artist_name, \"\n \"album_id, album_name, album_release, album_added, \"\n \"explicit, label, record_type, profile_id, \"\n \"future_release, trans_id) SELECT artist_id, \"\n \"artist_name, album_id, album_name, album_release, \"\n \"album_added, explicit, label, record_type, \"\n \"profile_id, future_release, trans_id FROM releases\")\n self.query(\"DROP TABLE releases\")\n self.query(\"ALTER TABLE releases_tmp RENAME TO releases\")\n self.query(\"INSERT OR REPLACE INTO 'deemon' ('property', 'value') VALUES ('version', '3.5.2')\")\n self.commit()\n \n if current_ver < parse_version(\"3.6\"):\n # album_release_ts REMOVED\n pass\n\n if current_ver < parse_version(\"3.7\"):\n self.query(\"CREATE TABLE playlists_tmp (\"\n \"'id' INTEGER UNIQUE,\"\n \"'title' TEXT,\"\n \"'url' TEXT,\"\n \"'bitrate' TEXT,\"\n \"'alerts' INTEGER,\"\n \"'profile_id' INTEGER DEFAULT 1,\"\n \"'download_path' TEXT,\"\n \"'refreshed' INTEGER DEFAULT 0,\"\n \"'trans_id' INTEGER,\"\n \"'monitor_artists' INTEGER DEFAULT 0)\")\n self.query(\"INSERT OR REPLACE INTO playlists_tmp (id, title, url, bitrate, \"\n \"alerts, profile_id, download_path, refreshed, trans_id) SELECT \"\n \"id, title, url, bitrate, alerts, profile_id, download_path, \"\n \"refreshed, trans_id FROM playlists\")\n self.query(\"DROP TABLE playlists\")\n self.query(\"ALTER TABLE playlists_tmp RENAME TO playlists\")\n self.query(\"INSERT OR REPLACE INTO 'deemon' ('property', 'value') VALUES ('version', '3.7')\")\n self.commit()\n logger.debug(f\"Database upgraded to version 3.7\")\n\n def query(self, query, values=None):\n if values is None:\n values = {}\n return self.cursor.execute(query, values)\n\n def reset_future(self, album_id):\n logger.debug(\"Clearing future_release flag from \" + str(album_id))\n values = {'album_id': album_id, 'profile_id': config.profile_id()}\n sql = \"UPDATE 'releases' SET future_release = 0 WHERE album_id = :album_id AND profile_id = :profile_id\"\n self.query(sql, values)\n\n def get_all_monitored_artists(self):\n vals = {'profile_id': config.profile_id()}\n return self.query(f\"SELECT * FROM monitor WHERE profile_id = :profile_id \"\n f\"ORDER BY artist_name COLLATE NOCASE ASC\", vals).fetchall()\n\n def get_monitored_artist_by_id(self, artist_id: int):\n values = {'id': artist_id, 'profile_id': config.profile_id()}\n return self.query(f\"SELECT * FROM monitor WHERE artist_id = :id AND profile_id = :profile_id\",\n values).fetchone()\n\n def get_monitored_artist_by_name(self, name: str):\n values = {'name': name, 'profile_id': config.profile_id()}\n return self.query(f\"SELECT * FROM monitor WHERE artist_name = :name COLLATE NOCASE \"\n f\"AND profile_id = :profile_id\", values).fetchone()\n\n def get_all_monitored_playlist_ids(self):\n vals = {'profile_id': config.profile_id()}\n query = self.query(\"SELECT id FROM playlists WHERE profile_id = :profile_id\", vals).fetchall()\n return [v for x in query for v in x.values()]\n\n def get_all_monitored_playlists(self):\n vals = {'profile_id': config.profile_id()}\n return self.query(\"SELECT * FROM playlists WHERE profile_id = :profile_id \"\n \"ORDER BY title COLLATE NOCASE ASC\", vals).fetchall()\n\n def get_monitored_playlist_by_id(self, playlist_id):\n values = {'id': playlist_id, 'profile_id': config.profile_id()}\n return self.query(\"SELECT * FROM playlists WHERE id = :id AND profile_id = :profile_id\", values).fetchone()\n\n def get_monitored_playlist_by_name(self, title):\n values = {'title': title, 'profile_id': config.profile_id()}\n return self.query(\"SELECT * FROM playlists WHERE title = :title COLLATE NOCASE \"\n \"AND profile_id = :profile_id\", values).fetchone()\n\n def monitor_artist(self, artist: dict, artist_config: dict):\n self.new_transaction()\n vals = {\n 'artist_id': artist['id'], 'artist_name': artist['name'], 'bitrate': artist_config['bitrate'],\n 'record_type': artist_config['record_type'], 'alerts': artist_config['alerts'],\n 'download_path': artist_config['download_path'], 'profile_id': config.profile_id(),\n 'trans_id': config.transaction_id()\n }\n query = (\"INSERT INTO monitor \"\n \"(artist_id, artist_name, bitrate, record_type, alerts, download_path, profile_id, trans_id) \"\n \"VALUES \"\n \"(:artist_id, :artist_name, :bitrate, :record_type, :alerts, :download_path, :profile_id, :trans_id)\")\n self.query(query, vals)\n self.commit()\n\n def get_artist_releases(self, artist_id=None):\n sql_values = {'artist_id': artist_id, 'profile_id': config.profile_id()}\n if artist_id:\n query = \"SELECT album_id, future_release FROM 'releases' WHERE artist_id = :artist_id AND profile_id = :profile_id\"\n else:\n query = \"SELECT album_id, future_release FROM 'releases' WHERE profile_id = :profile_id\"\n return self.query(query, sql_values).fetchall()\n\n def get_future_releases(self):\n vals = {'profile_id': config.profile_id()}\n return self.query(\"SELECT * FROM releases \"\n \"WHERE future_release = 1 AND profile_id = :profile_id\", vals).fetchall()\n\n def get_playlist_tracks(self, playlist_id):\n sql_values = {'playlist_id': playlist_id, 'profile_id': config.profile_id()}\n query = \"SELECT * FROM 'playlist_tracks' WHERE playlist_id = :playlist_id AND profile_id = :profile_id\"\n return self.query(query, sql_values).fetchall()\n\n def get_track_from_playlist(self, playlist_id, track_id):\n values = {'pid': playlist_id, 'tid': track_id, 'profile_id': config.profile_id()}\n query = \"SELECT * FROM 'playlist_tracks' WHERE track_id = :tid AND playlist_id = :pid AND profile_id = :profile_id\"\n result = self.query(query, values).fetchone()\n if result:\n return True\n\n def monitor_playlist(self, api_result):\n self.new_transaction()\n values = {'id': api_result['id'], 'title': api_result['title'], 'url': api_result['link'],\n 'bitrate': api_result['bitrate'], 'alerts': api_result['alerts'],\n 'download_path': api_result['download_path'], 'profile_id': config.profile_id(),\n 'trans_id': config.transaction_id()}\n query = (\"INSERT INTO playlists ('id', 'title', 'url', 'bitrate', 'alerts', 'download_path',\"\n \"'profile_id', 'trans_id') \"\n \"VALUES (:id, :title, :url, :bitrate, :alerts, :download_path, :profile_id, :trans_id)\")\n self.query(query, values)\n self.commit()\n\n def remove_monitored_artist(self, id: int = None):\n values = {'id': id, 'profile_id': config.profile_id()}\n self.query(\"DELETE FROM monitor WHERE artist_id = :id AND profile_id = :profile_id\", values)\n self.query(\"DELETE FROM releases WHERE artist_id = :id AND profile_id = :profile_id\", values)\n self.commit()\n\n def remove_monitored_playlists(self, id: int = None):\n values = {'id': id, 'profile_id': config.profile_id()}\n self.query(\"DELETE FROM playlists WHERE id = :id AND profile_id = :profile_id\", values)\n self.query(\"DELETE FROM playlist_tracks WHERE playlist_id = :id AND profile_id = :profile_id\", values)\n self.commit()\n\n def get_specified_artist(self, artist):\n values = {'artist': artist, 'profile_id': config.profile_id()}\n if type(artist) is int:\n return self.query(\"SELECT * FROM monitor WHERE artist_id = :artist \"\n \"AND profile_id = :profile_id\", values).fetchone()\n else:\n return self.query(\"SELECT * FROM monitor WHERE artist_name = ':artist' \"\n \"AND profile_id = :profile_id COLLATE NOCASE\", values).fetchone()\n\n def set_all_artists_refreshed(self):\n self.query(\"UPDATE monitor SET refreshed = 1 WHERE refreshed = 0\")\n \n def set_all_playlists_refreshed(self):\n self.query(\"UPDATE playlists SET refreshed = 1 WHERE refreshed = 0\")\n\n def add_new_releases(self, values):\n self.new_transaction()\n sql = (f\"INSERT OR REPLACE INTO releases ('artist_id', 'artist_name', 'album_id', 'album_name', 'album_release', \"\n f\"'album_added', 'future_release', 'explicit', 'record_type', 'profile_id', 'trans_id') \"\n f\"VALUES (:artist_id, :artist_name, :id, :title, :release_date, {int(time.time())}, :future, \"\n f\":explicit_lyrics, :record_type, {config.profile_id()}, {config.transaction_id()})\")\n self.cursor.executemany(sql, values)\n self.set_all_artists_refreshed()\n\n def add_new_playlist_releases(self, values):\n self.new_transaction()\n sql = (f\"INSERT INTO playlist_tracks ('artist_id', 'artist_name', 'track_id', 'track_name', 'playlist_id', \"\n f\"'track_added', 'profile_id', 'trans_id') VALUES (:artist_id, :artist_name, :id, :title, :playlist_id, \"\n f\"{int(time.time())}, {config.profile_id()}, {config.transaction_id()})\")\n self.cursor.executemany(sql, values)\n self.set_all_playlists_refreshed()\n\n def show_new_releases(self, from_date_ts, now_ts):\n today_date = datetime.utcfromtimestamp(now_ts).strftime('%Y-%m-%d')\n from_date = datetime.utcfromtimestamp(from_date_ts).strftime('%Y-%m-%d')\n values = {'from': from_date, 'today': today_date, 'profile_id': config.profile_id()}\n sql = \"SELECT * FROM 'releases' WHERE album_release >= :from AND album_release <= :today AND profile_id = :profile_id\"\n return self.query(sql, values).fetchall()\n\n def get_album_by_id(self, album_id):\n values = {'id': album_id, 'profile_id': config.profile_id()}\n sql = \"SELECT * FROM 'releases' WHERE album_id = :id AND profile_id = :profile_id\"\n return self.query(sql, values).fetchone()\n\n def reset_database(self):\n self.query(\"DELETE FROM monitor\")\n self.query(\"DELETE FROM releases\")\n self.query(\"DELETE FROM playlists\")\n self.query(\"DELETE FROM playlist_tracks\")\n self.query(\"DELETE FROM transactions\")\n self.commit()\n logger.info(\"Database has been reset\")\n\n def update_artist(self, settings: dict):\n self.query(\"UPDATE monitor SET bitrate = :bitrate, alerts = :alerts, record_type = :record_type,\"\n \"download_path = :download_path WHERE artist_id = :artist_id AND profile_id = :profile_id\", settings)\n self.commit()\n\n def add_playlist_track(self, playlist, track):\n self.new_transaction()\n values = {'pid': playlist['id'], 'tid': track['id'], 'tname': track['title'], 'aid': track['artist']['id'],\n 'aname': track['artist']['name'], 'time': int(time.time()), 'profile_id': config.profile_id(),\n 'trans_id': config.transaction_id()}\n query = (\"INSERT INTO 'playlist_tracks' \"\n \"('track_id', 'playlist_id', 'artist_id', 'artist_name', 'track_name', 'track_added', 'profile_id',\"\n \"'trans_id') VALUES (:tid, :pid, :aid, :aname, :tname, :time, :profile_id, :trans_id)\")\n return self.query(query, values)\n\n def create_profile(self, settings: dict):\n self.query(\"INSERT INTO profiles (name, email, alerts, bitrate, record_type, plex_baseurl, plex_token,\"\n \"plex_library, download_path) VALUES (:name, :email, :alerts, :bitrate, :record_type,\"\n \":plex_baseurl, :plex_token, :plex_library, :download_path)\", settings)\n self.commit()\n\n def delete_profile(self, profile_name: str):\n profile = self.get_profile(profile_name)\n vals = {'profile_id': profile['id']}\n self.query(\"DELETE FROM monitor WHERE profile_id = :profile_id\", vals)\n self.query(\"DELETE FROM releases WHERE profile_id = :profile_id\", vals)\n self.query(\"DELETE FROM playlists WHERE profile_id = :profile_id\", vals)\n self.query(\"DELETE FROM playlist_tracks WHERE profile_id = :profile_id\", vals)\n self.query(\"DELETE FROM profiles WHERE id = :profile_id\", vals)\n self.commit()\n\n def get_all_profiles(self):\n return self.query(\"SELECT * FROM profiles\").fetchall()\n\n def get_profile(self, profile_name: str):\n vals = {'profile': profile_name}\n return self.query(\"SELECT * FROM profiles WHERE name = :profile COLLATE NOCASE\", vals).fetchone()\n\n def get_profile_by_id(self, profile_id: int):\n vals = {'profile_id': profile_id}\n return self.query(\"SELECT * FROM profiles WHERE id = :profile_id\", vals).fetchone()\n\n def update_profile(self, settings: dict):\n self.query(\"UPDATE profiles SET name = :name, email = :email, alerts = :alerts, bitrate = :bitrate,\"\n \"record_type = :record_type, plex_baseurl = :plex_baseurl, plex_token = :plex_token,\"\n \"plex_library = :plex_library, download_path = :download_path \"\n \"WHERE id = :id\", settings)\n self.commit()\n\n def last_update_check(self):\n return self.query(\"SELECT value FROM 'deemon' WHERE property = 'last_update_check'\").fetchone()['value']\n\n def set_last_update_check(self):\n now = int(time.time())\n self.query(f\"UPDATE deemon SET value = {now} WHERE property = 'last_update_check'\")\n self.commit()\n\n def get_next_transaction_id(self):\n tid = self.query(f\"SELECT seq FROM sqlite_sequence WHERE name = 'transactions'\").fetchone()\n if not tid:\n return 0\n return tid['seq'] + 1\n\n def new_transaction(self):\n check_exists = self.query(f\"SELECT * FROM transactions WHERE id = {config.transaction_id()}\").fetchone()\n if not check_exists:\n current_time = int(time.time())\n vals = {'timestamp': current_time, 'profile_id': config.profile_id()}\n self.query(f\"INSERT INTO transactions ('timestamp', 'profile_id') \"\n f\"VALUES (:timestamp, :profile_id)\", vals)\n self.commit()\n\n def rollback_last_refresh(self, rollback: int):\n vals = {'rollback': rollback, 'profile_id': config.profile_id()}\n transactions = self.query(\"SELECT id FROM transactions WHERE profile_id = :profile_id \"\n f\"ORDER BY id DESC LIMIT {rollback}\", vals).fetchall()\n for t in transactions:\n vals = {'id': t['id'], 'profile_id': config.profile_id()}\n self.query(f\"DELETE FROM monitor WHERE trans_id = :id AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM releases WHERE trans_id = :id AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM playlist_tracks WHERE trans_id = :id AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM transactions WHERE id = :id AND profile_id = :profile_id\", vals)\n self.commit()\n\n def rollback_refresh(self, rollback: int):\n vals = {'rollback': rollback, 'profile_id': config.profile_id()}\n self.query(f\"DELETE FROM monitor WHERE trans_id = {rollback} AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM releases WHERE trans_id = {rollback} AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM playlist_tracks WHERE trans_id = {rollback} AND profile_id = :profile_id\", vals)\n self.query(f\"DELETE FROM transactions WHERE id = {rollback} AND profile_id = :profile_id\", vals)\n self.commit()\n\n def set_artist_refreshed(self, id):\n vals = {'id': id, 'profile_id': config.profile_id()}\n return self.query(\"UPDATE monitor SET refreshed = 1 WHERE artist_id = :id AND profile_id = :profile_id\", vals)\n\n def set_playlist_refreshed(self, id):\n vals = {'id': id, 'profile_id': config.profile_id()}\n return self.query(\"UPDATE playlists SET refreshed = 1 WHERE id = :id AND profile_id = :profile_id\", vals)\n\n def set_latest_version(self, version):\n vals = {'version': version}\n self.query(\"INSERT OR REPLACE INTO deemon (property, value) VALUES ('latest_ver', :version)\", vals)\n return self.commit()\n\n def get_release_channel(self):\n return self.query(\"SELECT value FROM deemon WHERE property = 'release_channel'\").fetchone()\n\n def set_release_channel(self):\n self.query(f\"INSERT OR REPLACE INTO deemon (property, value) \"\n f\"VALUES ('release_channel', '{config.release_channel()}')\")\n return self.commit()\n\n def get_transactions(self):\n vals = {'profile_id': config.profile_id(), 'trans_limit': config.rollback_view_limit()}\n transaction_list = self.query(\"SELECT id, timestamp FROM transactions WHERE profile_id = :profile_id \"\n \"ORDER BY id DESC LIMIT :trans_limit\", vals).fetchall()\n results = []\n for tid in transaction_list:\n vals = {'tid': tid['id'], 'profile_id': config.profile_id()}\n transaction = {}\n transaction['id'] = tid['id']\n transaction['timestamp'] = tid['timestamp']\n transaction['releases'] = self.query(\"SELECT album_id \"\n \"FROM releases \"\n \"WHERE trans_id = :tid \"\n \"AND profile_id = :profile_id\", vals).fetchall()\n transaction['playlist_tracks'] = self.query(\"SELECT track_id \"\n \"FROM playlist_tracks \"\n \"WHERE trans_id = :tid \"\n \"AND profile_id = :profile_id\", vals).fetchall()\n transaction['playlists'] = self.query(\"SELECT title \"\n \"FROM playlists \"\n \"WHERE trans_id = :tid \"\n \"AND profile_id = :profile_id\", vals).fetchall()\n transaction['monitor'] = self.query(\"SELECT artist_name \"\n \"FROM monitor \"\n \"WHERE trans_id = :tid \"\n \"AND profile_id = :profile_id\", vals).fetchall()\n results.append(transaction)\n return results\n\n def get_all_monitored_artist_ids(self):\n values = {\"profile_id\": config.profile_id()}\n query = self.query(\"SELECT artist_id FROM monitor WHERE profile_id = :profile_id\", values).fetchall()\n return [v for x in query for v in x.values()]\n\n @performance.timeit\n def get_monitored(self):\n values = {\"profile_id\": config.profile_id()}\n query = self.query(\"SELECT artist_id, artist_name FROM monitor WHERE profile_id = :profile_id\",\n values).fetchall()\n return query\n\n def get_unrefreshed_artists(self):\n values = {\"profile_id\": config.profile_id()}\n return self.query(\"SELECT * FROM monitor WHERE profile_id = :profile_id AND refreshed = 0\", values).fetchall()\n\n def get_unrefreshed_playlists(self):\n values = {\"profile_id\": config.profile_id()}\n return self.query(\"SELECT * FROM playlists WHERE profile_id = :profile_id AND refreshed = 0\", values).fetchall()\n\n def fast_monitor(self, values):\n self.cursor.executemany(\n \"INSERT OR REPLACE INTO monitor (artist_id, artist_name, bitrate, record_type, alerts, profile_id, download_path, trans_id) VALUES (:id, :name, :bitrate, :record_type, :alerts, :profile_id, :download_path, :trans_id)\",\n values)\n\n def fast_monitor_playlist(self, values):\n self.cursor.executemany(\n \"INSERT OR REPLACE INTO playlists (id, title, url, bitrate, alerts, profile_id, download_path, trans_id, monitor_artists) VALUES (:id, :title, :link, :bitrate, :alerts, :profile_id, :download_path, :trans_id, :monitor_artists)\",\n values)\n\n def insert_multiple(self, table, values):\n self.cursor.executemany(\n f\"INSERT INTO {table} (artist_id, artist_name, album_id, album_name, album_release, album_added, profile_id, future_release, trans_id) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)\",\n values)\n\n def remove_by_name(self, values):\n self.cursor.executemany(f\"DELETE FROM monitor WHERE profile_id = {config.profile_id()} AND artist_name = ?\",\n values)\n self.cursor.executemany(f\"DELETE FROM releases WHERE profile_id = {config.profile_id()} AND artist_name = ?\",\n values)\n self.commit()\n\n def remove_by_id(self, values):\n self.cursor.executemany(f\"DELETE FROM monitor WHERE profile_id = {config.profile_id()} AND artist_id = ?\",\n values)\n self.cursor.executemany(f\"DELETE FROM releases WHERE profile_id = {config.profile_id()} AND artist_id = ?\",\n values)\n self.commit()\n\n # @performance.timeit\n def remove_specific_releases(self, values):\n self.query(f\"DELETE FROM releases WHERE album_release > :tm_date AND profile_id = {config.profile_id()}\", values)\n\n def add_extra_release_info(self, values):\n self.new_transaction()\n sql = (\"UPDATE releases SET label = :label WHERE album_id = :id AND \"\n f\"profile_id = {config.profile_id()}\")\n self.cursor.executemany(sql, values)\n"
},
{
"path": "deemon/core/dmi.py",
"content": "import logging\nimport sys\nfrom pathlib import Path\n\nimport deemix\nimport deemix.utils.localpaths as localpaths\nfrom deemix import generateDownloadObject\nfrom deemix.downloader import Downloader\nfrom deemix.settings import load as LoadSettings\nfrom deemix.types.DownloadObjects import Collection\nfrom deemix.utils import formatListener, pathtemplates\nfrom deezer import Deezer\nfrom deezer.api import APIError\nfrom deezer.gw import GWAPIError\nfrom deezer.utils import map_user_playlist, LyricsStatus, map_track\n\nfrom deemon.core import notifier\nfrom deemon.core.config import Config as config\nfrom deemon.core.db import Database\n\nlogger = logging.getLogger(__name__)\n\n\nclass DeemixLogListener:\n @classmethod\n def send(cls, key, value=None):\n if isinstance(value, dict):\n if value.get('failed') and value['failed'] == True:\n\n if value.get('stack') and \"WrongGeolocation\" in value['stack']:\n logger.error(f\" [!] Not available in your country: {value['data']['title']} by {value['data']['artist']}\")\n else:\n logger.error(f\" [!] Error while downloading {value['data']['title']} by {value['data']['artist']}: {value['error']}\")\n if config.halt_download_on_error():\n logger.info(\"[X] Exiting due to halt_download_on_error being set to True in config.\")\n sys.exit()\n \n log_string = formatListener(key, value)\n if config.debug_mode():\n if log_string: logger.debug(f\"[DEEMIX] {log_string}\")\n\n\nclass DeemixInterface:\n def __init__(self):\n logger.debug(\"Initializing deemix library\")\n self.db = Database()\n self.dz = Deezer()\n\n if config.deemix_path() == \"\":\n self.config_dir = localpaths.getConfigFolder()\n else:\n self.config_dir = Path(config.deemix_path())\n\n self.dx_settings = LoadSettings(self.config_dir)\n\n logger.debug(\"deemix \" + deemix.__version__)\n logger.debug(f\"deemix config path: {self.config_dir}\")\n\n def download_url(self, url, bitrate, download_path, override_deemix=True):\n listener = DeemixLogListener()\n\n if override_deemix:\n deemix.generatePlaylistItem = self.generatePlaylistItem\n\n if download_path:\n self.dx_settings['downloadLocation'] = download_path\n logger.debug(f\"deemix download path set to: {self.dx_settings['downloadLocation']}\")\n\n links = []\n for link in url:\n if ';' in link:\n for l in link.split(\";\"):\n links.append(l)\n else:\n links.append(link)\n for link in links:\n download_object = generateDownloadObject(self.dz, link, bitrate, listener=listener)\n\n if isinstance(download_object, list):\n for obj in download_object:\n Downloader(self.dz, obj, self.dx_settings, listener=listener).start()\n else:\n Downloader(self.dz, download_object, self.dx_settings, listener=listener).start()\n\n def deezer_acct_type(self):\n user_session = self.dz.get_session()['current_user']\n \n if user_session.get('can_stream_lossless'):\n logger.debug(\"Deezer account connected and supports lossless\")\n config.set('deezer_quality', 'lossless', validate=False)\n elif user_session.get('can_stream_hq'):\n logger.debug(\"Deezer account connected and supports high quality\")\n config.set('deezer_quality', 'hq', validate=False)\n else:\n logger.warning(\"Deezer account connected but only supports 128k\")\n config.set('deezer_quality', 'lq', validate=False)\n\n def verify_arl(self, arl):\n if not self.dz.login_via_arl(arl):\n print(\"FAILED\")\n logger.debug(f\"ARL Failed: {arl}\")\n return False\n self.deezer_acct_type()\n \n print(\"OK\")\n logger.debug(\"ARL is valid\")\n return True\n\n def login(self):\n failed_logins = 0\n logger.debug(\"Looking for ARL...\")\n if config.arl():\n logger.debug(\"ARL found in deemon config\")\n print(\":: Found ARL in deemon config, checking... \", end=\"\")\n if self.verify_arl(config.arl()):\n return True\n else:\n logger.error(\"Unable to login using ARL found in deemon config\")\n failed_logins += 1\n else:\n logger.debug(\"ARL was not found in deemon config, checking if deemix has it...\")\n\n if self.config_dir.is_dir():\n if Path(self.config_dir / '.arl').is_file():\n with open(self.config_dir / '.arl', 'r') as f:\n arl_from_file = f.readline().rstrip(\"\\n\")\n logger.debug(\"ARL found in deemix config\")\n print(\":: Found ARL in deemix .arl file, checking... \", end=\"\")\n if self.verify_arl(arl_from_file):\n return True\n else:\n logger.error(\"Unable to login using ARL found in deemix config directory\")\n failed_logins += 1\n else:\n logger.debug(f\"ARL not found in {self.config_dir}\")\n else:\n logger.error(f\"ARL directory {self.config_dir} was not found\")\n\n if failed_logins > 1:\n notification = notifier.Notify()\n notification.expired_arl()\n else:\n logger.error(\"No ARL was found, aborting...\")\n return False\n\n def generatePlaylistItem(self, dz, link_id, bitrate, playlistAPI=None, playlistTracksAPI=None):\n if not playlistAPI:\n if not str(link_id).isdecimal(): raise InvalidID(f\"https://deezer.com/playlist/{link_id}\")\n # Get essential playlist info\n try:\n playlistAPI = dz.api.get_playlist(link_id)\n except APIError:\n playlistAPI = None\n # Fallback to gw api if the playlist is private\n if not playlistAPI:\n try:\n userPlaylist = dz.gw.get_playlist_page(link_id)\n playlistAPI = map_user_playlist(userPlaylist['DATA'])\n except GWAPIError as e:\n raise GenerationError(f\"https://deezer.com/playlist/{link_id}\", str(e)) from e\n\n # Check if private playlist and owner\n if not playlistAPI.get('public', False) and playlistAPI['creator']['id'] != str(self.dz.current_user['id']):\n logger.warning(\"You can't download others private playlists.\")\n raise NotYourPrivatePlaylist(f\"https://deezer.com/playlist/{link_id}\")\n\n if not playlistTracksAPI:\n playlistTracksAPI = dz.gw.get_playlist_tracks(link_id)\n playlistAPI['various_artist'] = dz.api.get_artist(5080) # Useful for save as compilation\n\n totalSize = len(playlistTracksAPI)\n playlistAPI['nb_tracks'] = totalSize\n collection = []\n for pos, trackAPI in enumerate(playlistTracksAPI, start=1):\n #\n # BEGIN DEEMON PATCH\n #\n vals = {'track_id': trackAPI['SNG_ID'], 'playlist_id': playlistAPI['id']}\n sql = \"SELECT * FROM 'playlist_tracks' WHERE track_id = :track_id AND playlist_id = :playlist_id\"\n result = self.db.query(sql, vals).fetchone()\n if result:\n continue\n #\n # END DEEMON PATCH\n #\n trackAPI = map_track(trackAPI)\n if trackAPI['explicit_lyrics']:\n playlistAPI['explicit'] = True\n if 'track_token' in trackAPI: del trackAPI['track_token']\n trackAPI['position'] = pos\n collection.append(trackAPI)\n\n if 'explicit' not in playlistAPI: playlistAPI['explicit'] = False\n\n return Collection({\n 'type': 'playlist',\n 'id': link_id,\n 'bitrate': bitrate,\n 'title': playlistAPI['title'],\n 'artist': playlistAPI['creator']['name'],\n 'cover': playlistAPI['picture_small'][:-24] + '/75x75-000000-80-0-0.jpg',\n 'explicit': playlistAPI['explicit'],\n 'size': totalSize,\n 'collection': {\n 'tracks': collection,\n 'playlistAPI': playlistAPI\n }\n })\n\n\nclass GenerationError(Exception):\n def __init__(self, link, message, errid=None):\n super().__init__()\n self.link = link\n self.message = message\n self.errid = errid\n\n def toDict(self):\n return {\n 'link': self.link,\n 'error': self.message,\n 'errid': self.errid\n }\n\n\nclass InvalidID(GenerationError):\n def __init__(self, link):\n super().__init__(link, \"Link ID is invalid!\", \"invalidID\")\n\n\nclass NotYourPrivatePlaylist(GenerationError):\n def __init__(self, link):\n super().__init__(link, \"You can't download others private playlists.\", \"notYourPrivatePlaylist\")\n"
},
{
"path": "deemon/core/exceptions.py",
"content": "class ValueNotAllowed(Exception):\n pass\n\n\nclass PropertyTypeMismatch(Exception):\n pass\n\n\nclass UnknownValue(Exception):\n pass\n"
},
{
"path": "deemon/core/logger.py",
"content": "import logging\nfrom logging.handlers import RotatingFileHandler\n\nimport tqdm\n\nLOG_FORMATS = {\n 'DEBUG': '%(asctime)s [%(levelname)s] %(name)s: %(message)s',\n 'INFO': '%(message)s',\n}\n\nLOG_DATE = '%Y-%m-%d %H:%M:%S'\n\n\nclass tqdmStream(object):\n\n @classmethod\n def write(cls, msg):\n tqdm.tqdm.write(msg, end='')\n\n\ndef setup_logger(log_level='DEBUG', log_file=None):\n \"\"\"\n Configure logging for the deemon application\n \"\"\"\n\n logger = logging.getLogger()\n logger.setLevel(logging.DEBUG)\n\n deemon_logger = logging.getLogger(\"deemon\")\n deemon_logger.setLevel(logging.DEBUG)\n\n urllib3_logger = logging.getLogger(\"urllib3\")\n urllib3_logger.setLevel(logging.ERROR)\n\n spotipy_logger = logging.getLogger(\"spotipy\")\n spotipy_logger.setLevel(logging.INFO)\n\n del logger.handlers[:]\n\n if log_file is not None:\n rotate = RotatingFileHandler(log_file, maxBytes=10485760, backupCount=0, encoding=\"utf-8\")\n rotate.setLevel(logging.DEBUG)\n rotate.setFormatter(logging.Formatter(LOG_FORMATS['DEBUG'], datefmt=LOG_DATE))\n logger.addHandler(rotate)\n\n stream = logging.StreamHandler(stream=tqdmStream)\n stream.setLevel(log_level)\n stream.setFormatter(logging.Formatter(LOG_FORMATS[log_level], datefmt=LOG_DATE))\n deemon_logger.addHandler(stream)\n"
},
{
"path": "deemon/core/notifier.py",
"content": "import logging\nimport platform\nimport smtplib\nimport ssl\nfrom datetime import datetime\nfrom email.message import EmailMessage\nfrom email.mime.multipart import MIMEMultipart\nfrom email.mime.text import MIMEText\nfrom email.utils import formataddr, formatdate\n\nimport pkgutil\n\nfrom deemon import __version__\nfrom deemon.core.config import Config as config\n\nlogger = logging.getLogger(__name__)\n\n\nclass Notify:\n\n def __init__(self, new_releases: list = None):\n logger.debug(f\"Sending notification for {new_releases} release(s)\")\n self.subject = \"deemon Notification\"\n self.releases = new_releases\n\n def send(self, body=None, test=False):\n \"\"\"\n Send email notification message\n \"\"\"\n if not all([config.smtp_server(), config.smtp_port(), config.smtp_user(),\n config.smtp_pass(), config.smtp_sender(), config.smtp_recipient()]):\n if test:\n logger.info(\" [!] Unable to send test notification. Please configure \"\n \"email server settings and provide recipient address.\")\n logger.debug(\"Email not configured, no notifications will be sent\")\n return False\n\n if not body:\n body = self.html_message()\n\n context = ssl.create_default_context()\n\n logger.debug(\"Sending notification email\")\n logger.debug(f\"Using server: {config.smtp_server()}:{config.smtp_port()}\")\n\n if config.smtp_starttls():\n with smtplib.SMTP(config.smtp_server(), config.smtp_port()) as server:\n server.starttls()\n server.login(config.smtp_user(), config.smtp_pass())\n server.sendmail(config.smtp_sender(), config.smtp_recipient(), body.as_string())\n logger.debug(\"Email notification has been sent\")\n else:\n with smtplib.SMTP_SSL(config.smtp_server(), config.smtp_port(), context=context) as server:\n server.login(config.smtp_user(), config.smtp_pass())\n server.sendmail(config.smtp_sender(), config.smtp_recipient(), body.as_string())\n logger.debug(\"Email notification has been sent\")\n\n def construct_header(self, is_plain_text=True, subject=None):\n subject = subject or self.subject\n\n if is_plain_text:\n message = EmailMessage()\n else:\n message = MIMEMultipart('mixed')\n\n message['To'] = config.smtp_recipient()\n message['From'] = formataddr(('deemon', config.smtp_sender()))\n message['Subject'] = subject\n message['Date'] = formatdate(localtime=True)\n\n return message\n\n def html_message(self):\n \"\"\"\n Builds HTML messages\n \"\"\"\n html_body = MIMEText(self.html_new_releases(), 'html')\n msg = self.construct_header(is_plain_text=False)\n msg.attach(html_body)\n return msg\n\n def test(self):\n \"\"\"\n Verify SMTP settings by sending test email\n \"\"\"\n msg = self.construct_header(subject=\"deemon Test Notification\")\n msg.set_content(\"Congrats! You'll now receive new release notifications.\")\n self.send(msg, test=True)\n\n def expired_arl(self):\n \"\"\"\n Notify user of expired ARL\n \"\"\"\n msg = self.construct_header(subject=\"deemon - ARL expired\")\n msg.set_content(\"Your ARL has expired. Please update your ARL to receive new releases.\")\n self.send(msg)\n\n def expired_sub(self):\n \"\"\"\n Notify user of expired subscription\n \"\"\"\n msg = self.construct_header(subject=\"deemon - Subscription expired\")\n msg.set_content(\"Your Deezer subscription appears to have expired.\")\n self.send(msg)\n\n def plaintext_message(self) -> str:\n \"\"\"\n Plaintext version of email to send\n \"\"\"\n message = \"The following new releases were detected:\\n\\n\"\n for release in self.releases:\n release_date_ts = datetime.strptime(release[\"release_date\"], \"%Y-%m-%d\")\n release_date_str = datetime.strftime(release_date_ts, \"%A, %B %-d\")\n message += f\"\\n{release_date_str}\\n\"\n for album in release[\"releases\"]:\n message += f\"+ {album['artist']} - {album['album']}\\n\"\n return message\n\n def html_new_releases(self):\n\n app_version = f\"deemon {__version__}\"\n py_version = f\"python {platform.python_version()}\"\n sys_version = f\"{platform.system()} {platform.release()}\"\n\n new_release_list_spacer = f\"\"\"\n \n \n\t\t\tdeemon\n\t\t
\n\t\t\n\t\t\t{UPDATE_MESSAGE}\n\t\t
\n\t\t\n\t\t\t{NEW_RELEASE_COUNT} new release(s) were found!\n\t\t
\n\t\t\n\t\t\n\t\t\t{NEW_RELEASE_LIST}\n\t\t
\n\t\t\n\t\t\n\t\t\n\t\t\t{DEEMON_VER} | {PY_VER} | {SYS_VER}\n\t\t
\n\t\n \"\"\"\n\n all_new_releases = \"\"\n\n self.releases = sorted(self.releases, key=lambda x: x['release_date'], reverse=True)\n\n new_release_count = 0\n \n for release in self.releases:\n\n if all_new_releases != \"\":\n all_new_releases += new_release_list_spacer\n\n release_date_ts = datetime.strptime(release[\"release_date\"], \"%Y-%m-%d\")\n release_date_str = datetime.strftime(release_date_ts, \"%A, %B %d\").replace(\" 0\", \" \")\n\n new_release_list_header = f\"\"\"\n\t\t\t
\n\t\t\t\t\n\t\t\t\t\t{release_date_str}\n\t\t\t\t\n\t\t\t
\n \"\"\"\n\n new_release_list_item = \"\"\n\n for album in release[\"releases\"]:\n new_release_count += 1\n if album['record_type'].lower() == \"ep\":\n record_type = \"EP\"\n else:\n record_type = album['record_type'].title()\n \n if not album['track_num']:\n album_info = record_type\n else:\n album_info = f\"{record_type} | {album['track_num']} track(s)\"\n \n new_release_list_item += f\"\"\"\n \n\t\t\t\t
\n \"\"\"\n\n all_new_releases += new_release_list_header + new_release_list_item\n\n if new_release_count > 1:\n self.subject = f\"{str(new_release_count)} new releases found!\"\n else:\n self.subject = f\"1 new release found!\"\n \n html_output = pkgutil.get_data('deemon', 'assets/index.html').decode('ascii')\n \n if config.update_available():\n html_output = html_output.replace(\"{UPDATE_MESSAGE}\", f\"Update to v{config.update_available()} is now available!\")\n else:\n html_output = html_output.replace(\"{UPDATE_MESSAGE}\", \"\")\n html_output = html_output.replace(\"{VIEW_UPDATE_MESSAGE}\", \"display:none;\")\n\n html_output = html_output.replace(\"{NEW_RELEASE_COUNT}\", str(new_release_count))\n html_output = html_output.replace(\"{NEW_RELEASE_LIST}\", all_new_releases)\n html_output = html_output.replace(\"{DEEMON_VER}\", app_version)\n html_output = html_output.replace(\"{PY_VER}\", py_version)\n html_output = html_output.replace(\"{SYS_VER}\", sys_version)\n\n return html_output\n"
},
{
"path": "deemon/utils/__init__.py",
"content": ""
},
{
"path": "deemon/utils/dataprocessor.py",
"content": "import logging\nfrom csv import reader\n\nlogger = logging.getLogger(__name__)\n\n\ndef read_file_as_csv(file, split_new_line=True):\n with open(file, 'r', encoding=\"utf-8-sig\", errors=\"replace\") as f:\n make_csv = f.read()\n if split_new_line:\n csv_to_list = make_csv.split('\\n')\n else:\n csv_to_list = make_csv.split(', ')\n sorted_list = sorted(list(filter(None, csv_to_list)))\n sorted_list = list(set(sorted_list))\n \n try:\n sorted_list = [int(s) for s in sorted_list]\n except ValueError:\n pass\n\n return sorted_list\n\n\ndef process_input_file(artist_list):\n logger.debug(\"Processing file contents\")\n try:\n artists = [int(x) for x in artist_list]\n total_artist_count = len(artists)\n logger.debug(f\"File detected as containing {total_artist_count} artist IDs\")\n logger.debug(\"Checking for duplicates\")\n artists_removed_duplicates = list(set(artists))\n new_artists_count = len(artists_removed_duplicates)\n duplicates = total_artist_count - new_artists_count\n if duplicates:\n logger.debug(f\"Removed {duplicates} duplicate(s)\")\n else:\n logger.debug(\"No duplicates found, continuing...\")\n\n except ValueError:\n artists = [x for x in artist_list]\n total_artist_count = len(artists)\n logger.debug(f\"File detected as containing {len(artists)} artist names\")\n\n artists_removed_duplicates = list(set(artists))\n new_artists_count = len(artists_removed_duplicates)\n duplicates = total_artist_count - new_artists_count\n if duplicates:\n logger.debug(f\"Removed {duplicates} duplicate(s)\")\n\n logger.info(f\"Detected {new_artists_count} unique artists\")\n return artists_removed_duplicates\n\n\ndef csv_to_list(all_artists) -> list:\n \"\"\"\n Separate artists and replace delimiter to find artists containing commas in their name\n \"\"\"\n all_artists = [str(x) for x in all_artists]\n processed_artists = []\n for artist in all_artists:\n if artist[-1] == ',':\n processed_artists.append(artist[:-1] + \"|\")\n else:\n processed_artists.append(artist)\n processed_artists = ' '.join(processed_artists)\n processed_artists = processed_artists.split('|')\n\n result = []\n csv_artists = reader(processed_artists, delimiter=\"|\")\n for line in csv_artists:\n combined_line = ([x.lstrip() for x in line])\n result.append(','.join(combined_line))\n return (result)\n"
},
{
"path": "deemon/utils/dates.py",
"content": "import logging\nimport time\nfrom datetime import datetime, date\n\nlogger = logging.getLogger(__name__)\n\n\ndef get_todays_date():\n now_ts = int(time.time())\n today_date = datetime.utcfromtimestamp(now_ts).strftime('%Y-%m-%d')\n return today_date\n\n\ndef generate_date_filename(prefix: str):\n return prefix + datetime.today().strftime('%Y%m%d-%H%M%S')\n\n\ndef get_max_release_date(days):\n day_in_secs = 86400\n input_days_in_secs = days * day_in_secs\n max_date_ts = int(time.time()) - input_days_in_secs\n max_date = datetime.utcfromtimestamp(max_date_ts).strftime('%Y-%m-%d')\n return max_date\n\n\ndef get_year(release_date: str):\n return datetime.strptime(release_date, '%Y-%m-%d').year\n\n\ndef format_date_string(d: str):\n date_string = datetime.strptime(d, \"%Y-%m-%d\")\n return datetime.strftime(date_string, \"%Y-%m-%d\")\n\ndef ui_date(d: datetime):\n return datetime.strftime(d, '%b %d, %Y')\n\ndef str_to_datetime_obj(d: str) -> datetime:\n if d == \"0000-00-00\":\n d = \"1980-01-01\"\n return datetime.strptime(d, \"%Y-%m-%d\")\n\n\ndef get_friendly_date(d: int):\n input_date = datetime.fromtimestamp(d).date()\n input_time = datetime.fromtimestamp(d).time()\n today = date.today()\n delta = today - input_date\n if delta.days == 0:\n try:\n return f\"{input_time.strftime('%-I:%M %p')}\"\n except ValueError:\n # Gotta keep Windows happy...\n return f\"{input_time.strftime('%#I:%M %p')}\"\n elif delta.days == 1:\n try:\n return f\"Yesterday, {input_time.strftime('%-I:%M %p')}\"\n except ValueError:\n # Gotta keep Windows happy...\n return f\"Yesterday, {input_time.strftime('%#I:%M %p')}\"\n\n elif 1 < delta.days < 7:\n try:\n return input_date.strftime(\"%A, \") + input_time.strftime(\"%-I:%M %p\")\n except ValueError:\n # Gotta keep Windows happy...\n return input_date.strftime(\"%A, \") + input_time.strftime(\"%#I:%M %p\")\n else:\n try:\n return input_date.strftime(\"%Y-%m-%d - \") + input_time.strftime(\"%-I:%M %p\")\n except ValueError:\n # Gotta keep Windows happy...\n return input_date.strftime(\"%Y-%m-%d - \") + input_time.strftime(\"%#I:%M %p\")\n"
},
{
"path": "deemon/utils/performance.py",
"content": "import logging\nimport time\n\nlogger = logging.getLogger(__name__)\n\n\ndef timeit(method):\n def timed(*args, **kwargs):\n ts = time.time()\n result = method(*args, **kwargs)\n te = time.time()\n\n logger.debug(f\"{method.__name__} finished in ({str((te - ts))})\")\n return result\n\n return timed\n\n\ndef operation_time(start_time):\n end_time = int(time.time())\n duration = end_time - start_time\n output = time.strftime(\"%H:%M:%S\", time.gmtime(duration))\n logger.info(f\"Operation completed in {output}\")"
},
{
"path": "deemon/utils/startup.py",
"content": "import logging\nimport os\nimport sys\nfrom pathlib import Path\n\nimport requests\nfrom packaging.version import parse as parse_version\n\nlogger = logging.getLogger(__name__)\n\n\ndef get_appdata_root():\n\n home_dir = Path.home()\n\n if os.getenv(\"XDG_CONFIG_HOME\"):\n appdata_dir = Path(os.getenv(\"XDG_CONFIG_HOME\"))\n elif os.getenv(\"APPDATA\"):\n appdata_dir = Path(os.getenv(\"APPDATA\"))\n elif sys.platform.startswith('darwin'):\n appdata_dir = home_dir / 'Library' / 'Application Support'\n else:\n appdata_dir = home_dir / '.config'\n\n return appdata_dir\n\n\ndef get_appdata_dir():\n \"\"\"\n Get appdata directory where configuration and data is stored\n \"\"\"\n return get_appdata_root() / 'deemon'\n\n\ndef get_backup_dir():\n return Path(get_appdata_dir() / \"backups\")\n\n\ndef init_appdata_dir(appdata):\n Path(appdata / 'logs').mkdir(parents=True, exist_ok=True)\n Path(appdata / 'backups').mkdir(exist_ok=True)\n\n\ndef delete_appdata(appdata):\n import shutil\n try:\n shutil.rmtree(appdata)\n except OSError as e:\n logger.info(f\"Error while deleting path: {e}\")\n\ndef reinit_appdata_dir(appdata):\n if appdata.exists():\n logger.info(\"Deleting existing application data directory (config, database, etc.)\")\n delete_appdata(appdata)\n logger.info(\"Initializing new application data directory...\")\n init_appdata_dir(appdata)\n\n\ndef get_config():\n return get_appdata_dir() / 'config.json'\n\n\ndef get_database():\n return get_appdata_dir() / 'deemon.db'\n\n\ndef get_log_file():\n \"\"\"\n Get path to log file\n \"\"\"\n return Path(get_appdata_dir() / 'logs' / 'deemon.log')\n\n\ndef get_latest_version(release_type):\n latest_ver = \"https://pypi.org/pypi/deemon/json\"\n\n try:\n response = requests.get(latest_ver)\n except requests.exceptions.ConnectionError:\n return\n\n latest_stable = parse_version(response.json()['info']['version'])\n\n if release_type == \"beta\":\n all_releases = [parse_version(x) for x in response.json()['releases']]\n sorted_releases = sorted(all_releases, reverse=True)\n for release in sorted_releases:\n if \"b\" in str(release) or \"rc\" in str(release):\n if release > latest_stable:\n return release\n else:\n return latest_stable\n else:\n return latest_stable\n\ndef get_changelog(ver: str):\n try:\n response = requests.get(\"https://api.github.com/repos/digitalec/\"\n \"deemon/releases\")\n except requests.exceptions.ConnectionError:\n return print(\"Unable to reach GitHub API\")\n \n for release in response.json():\n if release['name'] == ver:\n return print(release['body'])\n return print(f\"Changelog for v{ver} was not found.\")"
},
{
"path": "deemon/utils/ui.py",
"content": "import os\n\n\nTQDM_FORMAT = \":: {desc} {percentage:3.0f}%\"\n\n\ndef get_progress_bar_size() -> int:\n try:\n screen_size = int(os.get_terminal_size().columns)\n except OSError:\n screen_size = 80\n\n dynamic_size = int(screen_size / 4)\n if dynamic_size > 30:\n return 30\n elif dynamic_size < 16:\n return 16\n else:\n return dynamic_size\n\n\ndef set_progress_bar_text(msg: str, max_length: int) -> str:\n max_cols = get_progress_bar_size()\n max_length += 11\n\n if max_length < max_cols:\n max_cols = max_length\n\n while len(msg) < max_cols:\n msg += \" \"\n while len(msg) > max_cols:\n msg = msg[:-1]\n msg += \"...\"\n return msg\n"
},
{
"path": "deemon/utils/validate.py",
"content": "import logging\nfrom datetime import datetime\n\nlogger = logging.getLogger(__name__)\n\n\ndef validate_date(d):\n try:\n return datetime.strptime(d, '%Y-%m-%d')\n except ValueError:\n return False\n"
},
{
"path": "docs/_config.yml",
"content": "remote_theme: pmarsceill/just-the-docs\n"
},
{
"path": "docs/_sass/custom/custom.scss",
"content": "@import url('https://fonts.googleapis.com/css2?family=Baloo+Tammudu+2:wght@500&display=swap');\n\n.site-title {\n\tfont-family: 'Baloo Tammudu 2', cursive;\n\tcolor: #6200EE;\n}\n"
},
{
"path": "docs/docs/automations/automations.md",
"content": "---\nlayout: default\ntitle: Automations\nnav_order: 5\nhas_children: true\npermalink: /docs/automations\n---\n"
},
{
"path": "docs/docs/automations/cron.md",
"content": "---\nlayout: default\ntitle: cron (Linux/macOS)\nparent: Automations\nnav_order: 1\n---\n\n# cron\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nA cron job is the ideal way to run deemon in regular intervals to check for new releases:\n\n## Check for New Releases\n\nThis example checks for new releases every day at 06:00:\n\n```bash\n$ crontab -l\n0 6 * * * /home/user/.local/bin/deemon refresh\n```\n"
},
{
"path": "docs/docs/automations/task-scheduler.md",
"content": "---\nlayout: default\ntitle: Task Scheduler (Windows)\nparent: Automations\n---\n\n# Task Scheduler\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Create a New Task\n\n\nInside Task Scheduler, select the **Action** menu button and click _Create Task..._\n\n\n## Name the Task\n\n\nGive the task a name so you'll know what it's for. In this case, this task will be setup to run\n`deemon refresh` so that's what I've named it.\n\n\n## Setup the Schedule\n\n\nThis is where Task Scheduler becomes a little cumbersome. If you want to run a task every few minutes/hours, \nyou have to select _Daily_ (1) and under _Advanced settings_ check the box to repeat the task (2). The drop down \nbox for _Repeat task every:_ doesn't give many options, but thankfully you can type in your preference. I went \nahead and put in _12 hours_ with a duration of _indefinitely_.\n\nWith these settings, our task will run every 12 hours until we disable it.\n\n\n## Set the Application and Arguments\n\n\nUnder _Program/script_ (1) we need to set the **absolute path** to deemon:\n\n`_%localappdata%\\Programs\\Python\\Python39\\Scripts\\deemon.exe`\n\nUnder _Add arguments (optional)_ (2) is where you put the deemon command you wish to run. Since this task is being \nsetup for doing a refresh, we only need to put _refresh_ in this box. When Task Scheduler runs this task, it would be \nthe equivalent of typing `deemon refresh` at a command prompt.\n\n\n## Configure Task Settings\n\n\nHere you can configure your various preferences with how to handle various situations/conditions. The most important \nones are:\n\n(1) _Allow task to be run on demand_\n\n(2) _Run task as soon as possible after a scheduled start is missed_ \n\nThis will allow us to manually run the task to test it and will also run the task if your computer was off or sleeping \nthe last scheduled run.\n"
},
{
"path": "docs/docs/commands/backup.md",
"content": "---\nlayout: default\ntitle: backup\nparent: Commands\n---\n\n# backup\n{: .no_toc }\n\n---\n\nIntroduced in version 1.0, you can now make backups of your deemon installation including your _config.json_, \n_deemon.db_ and (optionally) the _logs_ directory.\n\n## Creating a Backup\nA backup can be created by using the `backup` command:\n\n```bash\nuser@localhost:~$ deemon backup --include-logs\nBacked up to /home/user/.config/deemon/backups/20210603-233151.tar\n```\n\n**Warning: ** Do **NOT** send your backup to anyone unless you have personally removed all sensitive data from your configuration such as email addresses, SMTP server settings and your ARL.\n\n## Restore a Backup\nTo restore from an existing backup, use `backup --restore` to be presented with valid backups to restore from:\n\n```bash\nuser@localhost:~$ deemon backup --restore\ndeemon Backup Manager\n\n1. Sep 23, 2021 @ 1:32:05 AM (ver 2.0)\n\nSelect a backup to restore: \n```\n\n**Note: ** You cannot restore from a backup that is newer than the version of deemon you are presently running. For example, if you create a backup while using deemon 2.9 you cannot restore it on version 2.8. However, you can restore manually by extracting the tar file and replacing the files as necessary. This is to prevent users from using an incompatible configuration or database versions."
},
{
"path": "docs/docs/commands/commands.md",
"content": "---\nlayout: default\ntitle: Commands\nnav_order: 4\nhas_children: true\npermalink: /docs/commands\n---\n\n# arguments\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nThe following arguments may be run directly after `deemon`:\n\n`--whats-new` - prints current version release notes from GitHub\n`--init` - Initializes deemon's application data directory. (*If directory exists, it will be deleted!*)\n`--arl` - Update your ARL quickly without having to edit your config.json\n`-P ID`, `--profile ID` - Uses specified profile ID\n`-V` - Prints current version and exits\n`-v`, `--verbose` - Show all verbose log messages\n"
},
{
"path": "docs/docs/commands/config.md",
"content": "---\nlayout: default\ntitle: config\nparent: Commands\n---\n\n# config\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nThe `config` command allows you to specify a per-artist configuration that overrides _global_ and _profile_ \nconfigurations for one specific artist.\n\nIf you wish to clear a particular setting for an artist, type 'none'. Providing no input leaves the setting unchanged.\n\n```bash\nuser@localhost:~$ deemon config ARTIST\ndeemon Artist Configurator\n:: Configuring 'ARTIST' (Artist ID: ...)\n\nBitrate [None]: 320\nRecord Type [None]: album\nAlerts [None]: true\nDownload Path [None]:\n\n:: Save these settings? [y|N] y\n\nArtist 'ARTIST' has been updated!\n```"
},
{
"path": "docs/docs/commands/download.md",
"content": "---\nlayout: default\ntitle: download\nparent: Commands\n---\n\n# download\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\ndeemon includes a command line interface to the deemix library allowing you to download directly by artist name, \nartist ID, album ID or URL.\n\nThe `download` command inherits all global settings configured in `config.json` such as bitrate and record type. These settings can be overriden using options available with the `download` command.\n\nThe `download` command is fairly straightforward and usage information including options can be found by running `deemon download --help`. Below are a few common usages of the `download` command.\n\n### Download by artist name\nTo download by artist name, simply run the `download` command followed by the artist's name:\n\n```bash\nuser@localhost:~$ deemon download John Doe\n```\n\n### Download by artist ID\nTo download by the artist's ID:\n\n```bash\nuser@localhost:~$ deemon download --artist-id 100\n```\n\n### Download by URL\nIn the below example, you can download a specific URL (artist, album, track or playlist):\n\n```bash\nuser@localhost:~$ deemon download --url https://...\n```\n\n### Download all monitored artists\nIf you'd like to download all releases by all artists currently being monitored, you can use the `--monitored` option to do so:\n\n```bash\nuser@localhost:~$ deemon download --monitored\n```\n\n### Download a date range\nIntroduced in version 2.5, you can now specify a date range when downloading releases.\n\nTo download releases by all monitored artists between January 1, 2022 and January 31, 2022:\n\n```bash\nuser@localhost:~$ deemon download --monitored --after 2021-12-13 --before 2022-02-01\n```"
},
{
"path": "docs/docs/commands/library.md",
"content": "---\nlayout: default\ntitle: library\nparent: Commands\n---\n\n# library\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n**Warning: This feature is a working prototype and is provided as-is. It should work but it requires accurate local metadata when querying for track/album IDs.**\n\nStarting in v2.18, deemon includes a library upgrade script to upgrade your existing collection from MP3 to FLAC by generating a file containing track/album IDs to be used with the `download` command.\n\n## Generate Track IDs\nTo generate a file containing track IDs:\n\n```bash\nuser@localhost:~$ deemon library upgrade /path/to/music/library\n```\n\nThis will generate a file in the current working directory called `library_upgrade_ids.txt`.\n\n## Generate Album IDs\nTo generate a file containing album IDs:\n\n```bash\nuser@localhost:~$ deemon library upgrade -A /path/to/music/library\n```\n\n## Specify output path of ID file\n\n```bash\nuser@localhost:~$ deemon library upgrade -O /path/to/save /path/to/music/library\n```\n\n## Obey exclusions set in config.json\nIf you'd like to obey the exclusions defined in your config.json file, add `-E` or `--allow-exclusions`.\n\n```bash\nuser@localhost:~$ deemon library upgrade -E /path/to/music/library\n```\n\n## Using library_upgrade_ids.txt\nTo process this file for downloading of the tracks/albums, use one of the following commands depending on which type of file you have generated:\n\nTrack IDs:\n```bash\nuser@localhost:~$ deemon download --track-file library_upgrade_ids.txt\n```\n\nAlbum IDs:\n```bash\nuser@localhost:~$ deemon download --album-file library_upgrade_ids.txt\n```"
},
{
"path": "docs/docs/commands/monitor.md",
"content": "---\nlayout: default\ntitle: monitor\nparent: Commands\n---\n\n# monitor\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nMonitoring artists is the core feature of deemon. Using the `monitor` command, you can monitor artists by name, their Deezer ID or Deezer URL. Starting with version 1.1, you can now provide multiple values in CSV format.\n\n## Monitor by Artist Name\nThis is the easiest way to monitor an artist but has some limitations. When using an artist name, deemon searches Deezer via an API call which returns the most likely result. In some situations you may find yourself monitoring the wrong artist. In this case, it would be best to [monitor the artist by ID](#monitor-by-artist-id).\n\nYou can monitor multiple artists at once by comma separating the artist names.\n\n```bash\nuser@localhost:~$ deemon monitor ArtistA, ArtistB, ArtistC\n```\n\n## Monitor by Artist ID\nThe Artist ID is the number located in the URL after `/artist/` and can be used to monitor an artist directly. This is the most accurate way to monitor an artist because this number is unique.\n\nIf monitoring by artist name doesn't give you the correct artist or an artist has more than one artist profile, this method is guaranteed to give you this exact artist.\n\n```bash\n$ deemon monitor --artist-id 1234\n```\n\n## Monitor by URL\n\nMonitoring by URL was implemented with the intention of using it for integration with automation tools like Siri Shortcuts.\n\n```bash\n$ deemon monitor --url https://www.deezer.com/us/artist/1234\n```\n## Monitor by Playlist\n\nDeemon will monitor the playlist URL, and will download any new additions to the playlist.\n\n```bash\n$ deemon monitor --playlist https://www.deezer.com/en/playlist/1234\n```\n\n## Monitor Playlist including Playlist Artists\nIf you'd also like to setup monitoring for artists included in your playlist, you can add `--include-artists`:\n\n```bash\n$ deemon monitor --playlist https://www.deezer.com/en/playlist/1234 --include-artists\n```\n\n## Import artists from file or directory\n\nThis method imports artist names or IDs from a file (CSV or Text) or a directory and stores them in the database.\n\n>**Note**: As of version 1.3, this does not actively monitor a file or directory for changes. This strictly imports the artists.\n\n**File Method:**\n```bash\n$ deemon monitor --import file.csv\n```\n\n**Directory Method:**\n```bash\n$ deemon monitor --import /home/user/Music\n```\n\n## Specify custom bitrate, record type and alerts\nBy default, deemon uses the settings configured in the `config.json` configuration file for all operations. This can be overridden at any time by using the available options such as `--bitrate`, `--record-type` and `--alerts`.\n\n```bash\n$ deemon monitor ArtistA --bitrate FLAC\n```\n\n## Monitor and download existing releases\nWhen setting up an artist (or artists) for monitoring, you can use the `-D` or `--download` flag to also download all releases matching the configured `record_type`.\n\n## Stop Monitoring an Artist\n\nIf you no longer wish to monitor an artist, include the `--remove` flag with one of the above methods and they will be removed from the database.\n\n```bash\n$ deemon monitor --remove ArtistA\n$ deemon monitor --remove --artist-id 1234\n```\n\n## Stop Monitoring a Playlist\n\nIf you no longer wish to monitor an playlist, include the `--remove --playlist` flags along with the playlist URL.\n\n```bash\n$ deemon monitor --remove --playlist https://www.deezer.com/en/playlist/1234\n```\n\n## Reset database\n\nTo reset the database and remove all artists/playlists from monitoring:\n```bash\n$ deemon monitor --reset\n** ALL ARTISTS AND PLAYLISTS WILL BE REMOVED! **\nType 'reset' to confirm: reset\nDatabase has been reset\n\n```\n\n## Skip Refresh\n\nIf you'd like to monitor an artist and wish to bypass refreshing the database afterwards, simply use `--no-refresh`.\n"
},
{
"path": "docs/docs/commands/profile.md",
"content": "---\nlayout: default\ntitle: profile\nparent: Commands\n---\n\n# profile\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nProfiles are a new feature of deemon and give you the ability to apply a set of configuration options to a completely \nseparate set of monitored artists. Profiles were developed with the intention of keeping up-to-date on multiple, \nseparate music libraries.\n\nThe default profile cannot be deleted but can be renamed and modified. To unset a setting, type \"none\" when prompted.\n\n## Viewing an Existing Profile\nYou can view an existing profile and it's configuration by running the below command:\n\n```bash\nuser@localhost:~$ deemon profile default\ndeemon Profile Editor\n:: Showing profile 'default' (Profile ID: 1)\n\nName Email Alerts Bitrate Type Plex Base URL Plex Token Plex Library Download Path\ndefault \n```\n\nYou can also view all existing profiles:\n```bash\nuser@localhost:~$ deemon profile ✔ \ndeemon Profile Editor\n:: Showing all profiles\n\nName Email Alerts Bitrate Type Plex Base URL Plex Token Plex Library Download Path\ndefault \n\nSoundtracks 0 320 album /music/soundtracks \n\nFavorites 1 FLAC all /music/favorites \n```\n\n## Managing Profiles\nYou can add, edit and delete profiles using the following syntax:\n```bash\ndeemon profile --add NAME\n\ndeemon profile --edit NAME\n\ndeemon profile --delete NAME\n```\n"
},
{
"path": "docs/docs/commands/refresh.md",
"content": "---\nlayout: default\ntitle: refresh\nparent: Commands\n---\n\n# refresh\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nThe `refresh` command is used to check for new releases, update the database and queue new releases with deemix for download. By default, running `refresh` will refresh the both artists and playlists for the active profile.\n\n## Refresh\nBy executing the `refresh` command by itself, deemon will refresh the releases for all artists and playlists contained in your database.\n\n> **Note:** For large databases, this can take several minutes to complete.\n\n```bash\nuser@localhost:~$ deemon refresh\n```\n\n## Refreshing a single artist\nThe `refresh` command has the ability to refresh a single artist or your entire database. To refresh an artist, simply specify that artists name after the `refresh` command:\n\n```bash\nuser@localhost:~$ deemon refresh Artist Name\n```\n\n## Refreshing a single playlist\nThe `refresh` command also has the ability to refresh a single playlist. To refresh a playlist, specify that playlists name after the `refresh` command:\n\n```bash\nuser@localhost:~$ deemon refresh My Awesome Playlist\n```\n\n## Refreshing with downloads disabled\nIf you wish to run a refresh without downloading any releases automatically, you can specify `--skip-download`.\n\n## Refresh to a specific date\nThe `refresh` command has a feature developed for resetting the database to a certain point in time called _time machine_. This makes rebuilding a music collection simple and can also ensure you have all releases released after a certain date.\n\nLet's say for example you want to download all releases released on or after January 1, 2022 for your entire database. All you have to do is run _time machine_ with the date of the day prior:\n\n```bash\nuser@localhost:~$ deemon refresh --time-machine 2021-12-31\n```\n\nThis tells deemon to first clear any release from the database that is newer than _December 31, 2021_ and then will do a full refresh. Any releases found between _January 1, 2022_ and today's date will be queued for download.\n\nIn the event a release is found with a release date in the future, deemon will save this to the database and flag it is a _future release_. Once the release date of the _future release_ has come, that release will then be queued for download."
},
{
"path": "docs/docs/commands/reset.md",
"content": "---\nlayout: default\ntitle: reset\nparent: Commands\n---\n\n# reset\n{: .no_toc }\n\n---\n\nThe `reset` command allows you to remove **ALL** monitored artists and releases regardless of profile. The `reset` \ncommand does not remove any profiles. To remove a profile, see the [profile]({{ site.baseurl }}{% link docs/commands/profile.md %}) docs.\n\n```bash\nuser@localhost:~$ deemon reset\n** ALL ARTISTS AND PLAYLISTS WILL BE REMOVED! **\n:: Type 'reset' to confirm: reset\nDatabase has been reset\n```"
},
{
"path": "docs/docs/commands/rollback.md",
"content": "---\nlayout: default\ntitle: rollback\nparent: Commands\n---\n\n# reset\n{: .no_toc }\n\n---\n\nThe `rollback` command allows you to rollback the last N transactions or a specific transaction. A _transaction_ is \ncreated anytime an artist is monitored or a refresh finds new releases. This does not delete any downloaded files but \ncan be useful in the event a download failed and you want to quickly re-download those releases.\n\n## Rollback by last _N_ transactions\n```bash\nuser@localhost:~$ deemon rollback 2 ✔ 2m 22s \nRolled back the last 2 transaction(s).\n```\n\n## Rollback a specific transaction\nBy default, deemon shows only the last 10 transactions. To change this, edit _rollback_view_limit_ in your config.json \nfile to increase or lower this amount.\n\n```bash\nuser@localhost:~$ deemon rollback --view\n1. 10:00 AM - Added Ludwig van Beethoven and 389 releases\n2. Yesterday, 8:22 PM - Added Mozart and 31 releases\n\nSelect specific refresh to rollback (or Enter to exit):\n```"
},
{
"path": "docs/docs/commands/search.md",
"content": "---\nlayout: default\ntitle: search\nparent: Commands\n---\n\n# deemon Interactive Search Client (dISC)\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nThe `search` command is an interactive client to search for artists and download albums or setup monitoring.\n\nTo open the search client:\n\n```bash\n$ deemon search\n```\n\nYou will then be prompted for an artist to search for:\n```bash\ndeemon Interactive Search Client\n\n:: Enter an artist to search for:\n```\n\nYou can also save yourself a step and include your query as an argument to the search command:\n\n```bash\n$ deemon search ArtistA \n```\n\n## Search Results\nWhen searching for an artist, you'll be presented with a list of results. If you have `smart_search` enabled, you will be taken to the releases menu provided that there was only one exact match.\n\nIn the event more than one artist is found with the exact same name, you'll be presented with some data regarding that artist to help you determine the one you're looking for:\n\n```bash\nSearch results for artist: John Doe\n 1. John Doe\n - Latest release: One Hit Wonder (1996)\n - Artist ID: 1234\n - Total releases: 1\n 2. John Doe\n - Latest release: Broken (2020)\n - Artist ID: 3210329\n - Total releases: 14\n\n(b) Back\n** Duplicate artists found **\n:: Please choose an option or type 'exit': \n```\n\n\n\t\t\t\t\t![](\"{album['cover']}\")
\n\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t{album['album']}\n\t\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t\t
\n\t\t\t\t{album['artist']}
\n\t\t\t\t\t\t{album_info}\n\t\t\t\t\t\n\nJudging by the results, it's possible that both artists are the same artist but for some reason have two separate artist profiles. It's also possible that they are unrelated and both artists happen to have the same name. You can then (hopefully) tell them apart based on their _Latest release_ and also by their _Total releases_. The _Artist ID_ is provided for reference so you can visit that specific artist in a browser if you need to.\n\n## Navigation\n\n```bash\nDiscography for artist: John Doe\nFilter by: All | Sort by: Year (desc)\n\n 1. (2020) Broken\n 2. (2018) Greatest Hits\n\nFilters: (*) All (a) Albums (e) EP (s) Singles - (E) Explicit (r) Reset\n Sort: (y) Year Desc (Y) Year Asc (t) Title Desc (T) Title Asc\n Mode: (S) Toggle Select\n\n(b) Back (d) Download Queue (Q) Show Queue (f) Queue Filtered (m) Stop Monitoring\n:: Please choose an option or type 'exit':\n```\n\n
\n\nAt the top of the screen is information related to the menu you are currently on. In the example above you can see that you are looking at the _Discography_ for the artist _John Doe_.\n\nAt the bottom of the screen you see four rows: Filters, Sort, Mode and Actions.\n\n### Filters\nHere you can filter albums by type (albums, EP and singles), show only explicit releases and reset filters back to their default view.\n\nAs of v2.22, you can now filter by date using `>=`, `<=` or `=` followed by the four digit year. For example, to find all releases between (and including) 2000 and 2003:\n\nReleased in year 2000 or later:\n```bash\n:: [SELECT] Please choose an option or type 'exit': >=2000\n```\n\nReleased in year 2003 or prior:\n```bash\n:: [SELECT] Please choose an option or type 'exit': <=2003\n```\n\nYou'll notice the header has updated to reflect the filter:\n```bash\nFilter: All | Sort: Release Date (desc) | Year: 2000 - 2003\n```\n\n### Sorting\nYou can sort the results by release year and title, both ascending and descending.\n\n### Modes\nThere are two different _Modes_ available when it comes to selecting releases.\n\n_Regular_ mode is the default mode which displays a number to the left of each menu item.\n\n_Select_ mode allows you to select an album or track to add to the queue and is identified by two brackets: `[ ]` for unselected and `[*]` for selected. When selecting items, the prompt will change to let you know you are in _Select_ mode and how many items are in the queue:\n\n```bash\n:: [SELECT] Please choose an option or type 'exit' (1 Queued):\n```\n\n### Actions\n\nThe _Actions_ bar is a group of actions you can perform while in the current view. Below is a list of all available actions throughout the dISC interface. You can only use an option if it is available in the _Actions_ bar.\n\n|Key|Name|Description|\n|-|--|---|\n|b|Back|Go back to the previous screen|\n|d|Download Queue|Download items currently in queue|\n|Q|Show Queue|Show all items presently in the queue|\n|c|Clear Queue|Clear all items from the queue|\n|f|Queue Filtered|Queue all items currently visible on the screen|\n|m|Start/Stop Monitoring|Toggle monitoring status of an artist|\n" }, { "path": "docs/docs/commands/show.md", "content": "---\nlayout: default\ntitle: show\nparent: Commands\n---\n\n# show\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nUsing the `show` command, you can currently view information pertaining to artists, playlists and new releases. You can also view artist and playlist data in CSV format and export that data to a CSV file.\n\n## Viewing artists or playlists\nTo show a list of monitored artists:\n\n```bash\n$ deemon show artists\nArtistA\nArtistB\n```\n\nTo show a list of monitored playlists:\n\n```bash\n$ deemon show playlists\nSummer Hits Playlist\n2022 Indie Folk\n```\n\n## CSV, filters and exporting data\nThe `show` command allows you to view and export more data pertaining to each artist other than just their names.\n\n### Viewing the data in CSV format\nFirst, we'll use the `-c` or `--csv` option to see all the data available:\n```bash\n$ deemon show artists --csv\nname,id,bitrate,alerts,type,path\nJohn Doe,1234,,,,\n```\n\nYou can also toggle the header of the CSV output by passing `--hide-header` or `-H`:\n\n```bash\n$ deemon show artists -cH\nJohn Doe,1234,,,,\n```\n\n### Using filters to customize the CSV output\nNext, we can apply filters to view only certain pieces of this data. If you would like to generate a CSV file containing only the ID and artists' name, you can do so by using the `-f` or `--filter` option:\n```bash\n$ deemon show artists --csv --filter id,name\nid,name\n1234,John Doe\n```\n\nNotice how the data is presented in the order in which you specify the filters. Filters can also be used more than once if you find the need to do so:\n\n```bash\n$ deemon show artists -cf id,name,id\nid,name,id\n1234,John Doe,1234\n```\n\n### Exporting monitored artists\nAnother option for the `show artists` command is `-e` or `--export`. This allows you to export your artists to a CSV file. You can also combine this with `--hide-headers` and `--filter` to customize the data output to meet your needs.\n\n```bash\n$ deemon show artists --export artists.csv\nCSV data has been saved to: artists.csv\n\n$ cat artists.csv\nname,id,bitrate,alerts,type,path\nJohn Doe,1234,,,,\n```\n\nA common use case is backing up a list of all monitored artist IDs to a CSV file which you can then import into deemon if you ever want to rebuild your database:\n\n```bash\n$ deemon show artists -Hf id -e artists.csv\nCSV data has been saved to: artists.csv\n\n$ cat artists.csv\n1234\n```\n\nIntroduced in version 2.9 is a new alias option `-b` or `--backup`. This option does the exact same thing as the previous example but does so in a much more user-friendly and simple way:\n\n```bash\n$ deemon show artists --backup artists.csv\nCSV data has been saved to: artists.csv\n\n$ cat artists.csv\n1234\n```\n\n## New releases\nYou can view a list of releases that have been detected in chronological order (newest to oldest). By default, `show releases` will display the last 7 days worth of releases. You can override this by specify a number like so:\n\n```bash\n$ deemon show releases 30\n```\n\n## Future releases\nYou may have seen `Pending future releases` displayed in the output of the `refresh` command. When deemon detects a release with a release date in the future, it is flagged and is stored in the database until the release date approaches.\n\nIf you'd like to view these future releases, you can use the `show` command:\n\n```bash\n$ deemon show releases --future\n```" }, { "path": "docs/docs/commands/test.md", "content": "---\nlayout: default\ntitle: test\nparent: Commands\n---\n\n# test\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n## SMTP settings test\nTo verify that your SMTP settings are correctly configured, you may use `test --email` to send a test email to yourself. If you don't receive the email, confirm your SMTP settings with your mail provider and check the logs for additional information.\n\n---\n## Exclusions test\nIf you have opted to use exclusion patterns or keywords to filter out releases, you may test those exclusions against any release URL to identify if that URL will be appropriately filtered out:\n\n```\nArtist: Various Artists\nAlbum: Broken Bow (Remix)\n\nChecking for the following patterns:\n 1. (?i)\\bremix\\b >> ** MATCH **\n\nChecking for the following keywords:\n 1. remix >> ** MATCH **\n 2. deluxe >> NO MATCH\n 3. bonus >> NO MATCH\n 4. special >> NO MATCH\n 5. live >> NO MATCH\n\nResult: This release would be excluded\n```\n" }, { "path": "docs/docs/configuration.md", "content": "---\nlayout: default\ntitle: Configuration\nnav_order: 3\n---\n\n# Configuration\n{: .no_toc }\n\n\ndeemon has some specific configuration parameters that can be defined in your \nconfig.json file.\n{: .fs-6 .fw-300 }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Configuration Location\nDepending on your operating system, your config.json file will be located in \none of the locations below. When deemon is run with a command, it will \nautomatically generate a default config if an existing configuration file is \nnot present. For example: to generate this configuration file, run \n`deemon refresh`.\n\n- **Linux**: /home/user/.config/deemon\n\n- **macOS**: /User/user/Library/Application Support/deemon\n\n- **Windows**: %appdata%\\deemon\n\n### Default Configuration (Version 2.19.2)\n```json\n{\n \"check_update\": 1,\n \"debug_mode\": false,\n \"release_channel\": \"stable\",\n \"query_limit\": 5,\n \"smart_search\": true,\n \"rollback_view_limit\": 10,\n \"prompt_duplicates\": false,\n \"prompt_no_matches\": true,\n \"fast_api\": true,\n \"fast_api_threads\": 25,\n \"exclusions\": {\n \"enable_exclusions\": true,\n \"patterns\": [],\n \"keywords\": [],\n },\n \"new_releases\": {\n \"release_max_age\": 90,\n \"include_unofficial\": false,\n \"include_compilations\": false,\n \"include_featured_in\": false,\n },\n \"global\": {\n \"bitrate\": \"320\",\n \"alerts\": false,\n \"record_type\": \"all\",\n \"download_path\": \"\",\n \"email\": \"\"\n },\n \"deemix\": {\n \"path\": \"\",\n \"arl\": \"\",\n \"check_account_status\": true,\n \"halt_download_on_error\": false,\n },\n \"smtp_settings\": {\n \"server\": \"\",\n \"port\": 465,\n \"starttls\": false,\n \"username\": \"\",\n \"password\": \"\",\n \"from_addr\": \"\"\n },\n \"plex\": {\n \"base_url\": \"\",\n \"ssl_verify\": true,\n \"token\": \"\",\n \"library\": \"\"\n }\n}\n```\n\n## Configuration Types\n\nThere are technically three different \"levels\" of configuration in deemon. The \nfirst level (and default) is the \"global\" configuration, the second level is \nthe \"profile\" configuration and the third level is the \"per-artist\" \nconfiguration.\n\n### Global Configuration\n\nThe global configuration is defined inside the `config.json` configuration \nfile. This configuration is used by default when running deemon and the values \ndefined in this configuration may be temporarily superseded by a \n[Profile](#profile-configuration-optional) or an \n[Artist](#per-artist-configuration-optional) configuration.\n\n### Profile Configuration *(optional)*\n\nWithin deemon, you can configure multiple profiles (using the `profile` \ncommand) which monitors their own sets of artists and can use settings that \noverride the _Global Configuration_ while inheriting the settings that are not \ndefined. You can also think of these profiles as separate users. For example, \nif you create a new profile and only specify a download path, that profile will \ninherit the other settings from the _Global Configuration_.\n\n### Per-Artist Configuration *(optional)*\n\nWithin deemon, you can configure different settings for each individual artist \n(using the `config` command). This gives you the flexibility to disable alerts \nfor certain artists or specify a certain release type, bitrate or download path.\n\n
\n\n## Global Configuration\nThis section outlines each setting available in the configuration file and their respective options.\n\n---\n\n### Global settings\nThese settings affect the general operation of deemon.\n\n|Setting|Description|\n|-|---|\n|**check_update**
|This option allows you to specify how frequently (in days) to check for new updates to deemon. To disable checking for updates, change this to `0`.
|\n|**debug_mode**
options: _true, false_
|This option will allow you to print extra debug messages in the logs or on screen if used with `--verbose`.
|\n|**release_channel**
options: _stable, beta_
|When checking for updates (if enabled), this option allows you to choose what updates you are notified about. Most users will want to use _stable_. If you are interested in testing pre-release versions of deemon, you can set this to _beta_.
|\n|**query_limit**
options: _number_
|This option allows you to specify how many results are displayed when using the `search` command or when prompted using the `monitor` command (see _prompt_duplicates_ and _prompt_no_matches_ below).
|\n|**smart_search**
options: _true, false_
|This option allows you to skip the list of artist search results and proceed directly to the list of artist albums, provided there is only one exact match of the artists name (case insensitive).
|\n|**rollback_view_limit**
|This option allows you to specify the maximum number of transactions to display using the `rollback` command
|\n|**prompt_duplicates**
options: _true, false_
|When adding a new artist using the `monitor` command, deemon will choose the highest ranked artist in situations where two artists have identical names. Instead, you can set this option to `true` which will prompt you with choices including the latest release from each artist to help you better decide which is the artist you're looking for.
|\n|**prompt_no_matches**
options: _true, false_
|When adding a new artist using the `monitor` command, if deemon does not find an **exact** match for the artist you're searching for, it will prompt you with a list of results returned from the Deezer API.
|\n|**fast_api**
options: _true, false_
|In previous versions of deemon, this was referred to as the _experimental_api_ and has been the default API since version 2.1.
|\n|**fast_api_threads**
options: _number_
|This sets the number of threads to spawn when accessing the API. The higher the number, the faster artist data is retrieved. However, setting this number too high may result in a temporary ban of your IP address. **It is recommended to keep this number below 50.**
|\n\n---\n\n### Exclusion settings\nExclusions can be setup to ignore releases matching a specific regular expression (\"pattern\") or by matching against specific keywords. You can test your exclusion settings by using the `test` command.\n\n|Setting|Description|\n|-|---|\n|**enable_exclusions**
options: _true, false_
|This setting enables exclusion patterns and keywords (below) to filter out releases.
|\n|**patterns**
options: _regex_
|Provide a list of regular expressions to filter out releases.
|\n|**keywords**
options: _true, false_
|Provide a list of keywords to filter out releases (_remix, deluxe, bonus_).
|\n\n---\n\n### New Release settings\nThese settings affect what releases are filtered out by deemon. As of version 2.9, these settings are global only and cannot be configured in a profile nor per-artist configuration.\n\n|Setting|Description|\n|-|---|\n|**release_max_age**
|This lets you control how far back to consider a new release 'new' from it's actual release date. This setting is helpful if you only run a `refresh` monthly or if there is a delay in Deezer adding a release to their catalogue. If you wish to get all releases regardless of when they were released, set this to _0_.
**Note:** Version 2.8.x and earlier relied on a separate toggle _by_release_date_ which has been deprecated in favor of setting _release_max_age_ to _0_.
|\n|**include_unofficial**
options: _true, false_
|Each release result returned from the API includes a boolean value designating the release as either an official artist release or not. In many cases, this flag is incorrectly set resulting in some official releases not being detected.
**Warning:** If you are enabling this option on an existing database, you could potentially have hundreds or thousands of new releases detected.
|\n|**include_compilations**
options: _true, false_
|If you want to include all compilation albums that your artists are featured on, you can enable this.
**Warning:** If you are enabling this option on an existing database, you could potentially have hundreds or thousands of new releases detected.
|\n|**include_featured_in**
options: _true, false_
|Enabling this option will include every single release/track an artist is associated with. It is highly recommend to avoid using this but is included for testing purposes.
**Warning:** If you are enabling this option on an existing database, you could potentially have **_tens of thousands_** of new releases detected. Most users will NOT want this option enabled.
|\n\n---\n\n### Global settings\nThese settings can be overriden within deemon using _profiles_ or by specifying a _per-artist configuration_.\n\n|Setting|Description|\n|-|---|\n|**bitrate**
options: _128, 320, FLAC_
|This option allows you to specify the bitrate used for downloads.
|\n|**alerts**
options: _true, false_
|Enable or disable email notification alerts when new releases are present. You must also have your email settings configured (see _[SMTP Settings](#smtp-settings)_).
|\n|**record_type**
options: all, album, ep, single
|This option allows you to specify what release types you wish to download upon release. Keep in mind, most EPs are labelled as albums in the Deezer API.
**Limitation:** Currently only one option at a time is allowed.
|\n|**download_path**
|This option allows you to specify where downloads are saved. If no path is provided, downloads will be saved in the default directory specified by deemix.
**Windows users:** When providing a path, you **must** use double backslashes: `C:\\\\Music` or forward slashes: `C:/Music`.
|\n|**email**
|This option allows you to specify the default email address to use when alerts are enabled and SMTP settings are defined.
|\n\n---\n\n### deemix settings\nThese settings are needed for deemon to interface with deemix which is a third party library that does the actual downloading.\n\n|Setting|Description|\n|-|---|\n|**path**
|Specify the path to where your deemix configuration is stored. For most users, leave this blank. If you have moved the deemix configuration, you must specify that here.
|\n|**arl**
|This is your authorization token required by `deemix` to authenticate your Deezer account. This is stored in a cookie named `arl` in your browser after logging in to Deezer.
|\n|**check_account_status**
options: _true, false_
|This option allows you to force account verification before doing a refresh. If you have _bitrate_ set to FLAC and your account type is not HiFi, deemon will exit until you correct the issue (expired ARL or subscription). This option is useful for preventing low quality downloads due to an expired subscription.
|\n|**halt_download_on_error**
options: _true, false_
|If enabled, deemon will exit if deemix reports any errors when downloading. This prevents releases from being logged in the database so that you can try again later.
|\n\n---\n\n### SMTP settings\nThese settings are needed for deemon to interface with deemix which is a third party library that does the actual downloading.\n\n|Setting|Description|\n|-|---|\n|**server**
|Server address for your mail server
|\n|**port**
|Port used to connect to your mail server (typically 587 or 465)
|\n|**starttls**
options: _true, false_
|If your mail server requires STARTTLS, you can enable that here.
|\n|**username**
|Username required to login to your mail server
|\n|**password**
|Password used to authenticate your account with your mail server
|\n|**from_addr**
|This is the email address your email is to be sent _from_ and typically must be a real address associated with your account on your mail server.
|\n\n---\n\n### Plex integration settings\ndeemon features support for refreshing your Plex library after a download operation is complete. Plex has the ability to automatically detect changes and rescan but if you're having issues with that, this should help.\n\n|Setting|Description|\n|-|---|\n|**base_url**
|This is the URL to reach your Plex server including the port and protocol.
_Example_: http://192.168.0.2:32400
**Note:** You may need to use _https_ if your Plex setup is configured for secure connections.
|\n|**ssl_verify**
options: _true, false_
|If you have Plex configured to require secure connections, but have not provided a custom certificate, keep this set to _false_ to avoid SSL certificate errors.
|\n|**token**
|Authentication token required to connect to your Plex server
For instructions on how to find your token, [click here](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/).
|\n|**library**
|The name of your Plex library to be refreshed.
|" }, { "path": "docs/docs/installation.md", "content": "---\nlayout: default\ntitle: Installation\nnav_order: 2\n---\n\n# Installation\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\n\n## Step 1 - Required Dependencies\n\nIn order to install and run deemon, you'll need to have at least Python 3.6 or higher installed along with the `pip` package manager.\n\nPlease refer to [python.org](https://www.python.org/downloads/) for more information.\n\n### Step 2 - Installing deemon\nOnce you have at least Python 3.6 installed, go ahead and install deemon using \n`pip`. On some distributions, the `pip` command is for Python2. In this case, \nsubstitute `pip` for `pip3` in the commands below.\n\n**Windows users**: These commands should be typed in a Command Prompt, Windows Terminal or Powershell window.\n\n```bash\n# Latest stable release\n$ pip install deemon\n\n# Latest release (including pre-release/beta)\n$ pip install --pre deemon\n```\n\nAt this point, pip will download deemon and any other modules required to allow \ndeemon to function. Once it's complete, use the following command to make sure \ndeemon is installed:\n\n```bash\n$ deemon -V\ndeemon 2.19.2\n```\n\n## Configuration & First Use\n\nCongrats! If you've made it this far, you have successfully installed deemon. \nThere are a few things you should configure before using deemon. Head on over \nto the [configuration](configuration.md) page to learn more." }, { "path": "docs/docs/troubleshooting/logs.md", "content": "---\nlayout: default\ntitle: Logs\nparent: Troubleshooting\n---\n\n# Logs\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nIf you're having issues or would like to see how things are running, you can view the log files which are located inside the [deemon config directory](/docs/configuration#configuration-file).\n\nAs of version 1.0, the logs from deemix are now included (as well as other third-party modules used) in the deemon log file." }, { "path": "docs/docs/troubleshooting/queue.md", "content": "---\nlayout: default\ntitle: Queue\nparent: Troubleshooting\n---\n\n# Queue\n{: .no_toc }\n\n## Table of contents\n{: .no_toc .text-delta }\n\n1. TOC\n{:toc}\n\n---\nEach time deemon finds new releases, they are dumped to `queue.csv` inside the deemon application directory. This provides a way to recover a failed job.\n\nTo redownload a queue, you can manually extract the `album_id` or `track_id` column and save it into a new file to pass to deemon:\n\nAlbum IDs: `deemon download --album-file file.csv`\n\nTrack IDs: `deemon download --track-file file.csv`" }, { "path": "docs/docs/troubleshooting/troubleshooting.md", "content": "---\nlayout: default\ntitle: Troubleshooting\nnav_order: 6\nhas_children: true\npermalink: /docs/troubleshooting\n---\n" }, { "path": "docs/index.md", "content": "---\nlayout: default\ntitle: Home\nnav_order: 1\ndescription: \"deemon is a monitoring utility for new artist releases that can provide email alerts and automate downloading via the deemix library\"\npermalink: /\n---\n\n# deemon 2.22 Documentation\n{: .fs-9 }\n\ndeemon is a monitoring utility for new artist releases that can provide email alerts and automate downloading via the deemix library\n{: .fs-6 .fw-300 }\n\n[Get started now](#getting-started){: .btn .btn-primary .fs-5 .mb-4 .mb-md-0 .mr-2 } [View it on GitHub](https://github.com/digitalec/deemon){: .btn .fs-5 .mb-4 .mb-md-0 }\n\nVersion 2.22\n---\n\n## Disclaimer\n\ndeemon does not download anything by itself. It requires a third party library \ncalled *deemix* in order to do this and will be installed automatically when \ninstalled via pip.\n\n---\n\n## Getting started\n\n### Dependencies\n\ndeemon depends on various python modules and libraries to perform all of its functions. Please refer to the `requirements.txt` file to see what those dependencies are.\n\n### Installation & Configuration\n\nWhen you're ready to install deemon, head on over to the \n[installation]({{ site.baseurl }}{% link docs/installation.md %}) page. Once \nyou've installed deemon, it is important to \n[configure]({{ site.baseurl }}{% link docs/configuration.md %}) it properly to \nget the best experience.\n\n\n---\n\n## About the project\n\ndeemon (**dee**zer **mon**itor) is an open source project that came from the need to stay on top of new \nreleases by some of my favorite artists.\n\n### License\n\ndeemon is distributed by a [GPL-3.0 license](https://github.com/digitalec/deemon/blob/main/LICENSE).\n" }, { "path": "requirements.txt", "content": "deemix~=3.6\npackaging~=23.0\nrequests~=2.28.0\nclick~=8.1.0\nsetuptools~=66.1.0\nwheel~=0.43.0\nPlexAPI~=4.5.2\ntqdm~=4.61.0\nmutagen~=1.46\nUnidecode~=1.3.6\n" }, { "path": "setup.py", "content": "from pathlib import Path\nfrom setuptools import setup, find_packages\nfrom deemon import __version__\n\nwith open('requirements.txt') as f:\n required = f.read().splitlines()\n\nHERE = Path(__file__).parent\nREADME = (HERE / \"README.md\").read_text()\nDESCRIPTION = \"Monitor new releases by a specified list of artists and auto download using the deemix library\"\n\nsetup(\n name=\"deemon\",\n version=__version__,\n author=\"digitalec\",\n description=DESCRIPTION,\n long_description=README,\n long_description_content_type=\"text/markdown\",\n license=\"GPL3\",\n classifiers=[\n \"License :: OSI Approved :: GNU General Public License v3 (GPLv3)\",\n \"Programming Language :: Python :: 3.8\",\n \"Operating System :: OS Independent\",\n ],\n packages=find_packages(),\n include_package_data=True,\n python_requires=\">=3.8\",\n install_requires=required,\n url=\"https://github.com/digitalec/deemon\",\n entry_points = {\n \"console_scripts\": [\"deemon=deemon.__main__:main\"],\n }\n)\n" } ]